* bug#42028: [Feature Request] 27.0.91; Provide the ability dynamic modules to post events in emacs event loop
@ 2020-06-24 7:14 Ivan Yonchovski
2020-06-24 14:44 ` Eli Zaretskii
0 siblings, 1 reply; 9+ messages in thread
From: Ivan Yonchovski @ 2020-06-24 7:14 UTC (permalink / raw
To: 42028
This is needed for the cases when the module is listening for extenal
events and wants to call back emacs to process them. ATM this is kind of
possible by using signals on linux and using WM_INPUTLANGCHANGE as
described in https://nullprogram.com/blog/2017/02/14/ but this looks
more a hack for a missing feature.
^ permalink raw reply [flat|nested] 9+ messages in thread
* bug#42028: [Feature Request] 27.0.91; Provide the ability dynamic modules to post events in emacs event loop
2020-06-24 7:14 bug#42028: [Feature Request] 27.0.91; Provide the ability dynamic modules to post events in emacs event loop Ivan Yonchovski
@ 2020-06-24 14:44 ` Eli Zaretskii
2020-06-24 16:17 ` yyoncho
0 siblings, 1 reply; 9+ messages in thread
From: Eli Zaretskii @ 2020-06-24 14:44 UTC (permalink / raw
To: Ivan Yonchovski; +Cc: 42028
> From: Ivan Yonchovski <yyoncho@gmail.com>
> Date: Wed, 24 Jun 2020 10:14:44 +0300
>
> This is needed for the cases when the module is listening for extenal
> events and wants to call back emacs to process them. ATM this is kind of
> possible by using signals on linux and using WM_INPUTLANGCHANGE as
> described in https://nullprogram.com/blog/2017/02/14/ but this looks
> more a hack for a missing feature.
It is almost trivial to let modules insert events into the event
queue. The problem is that I expect a module that inserts such events
to want to be called to process those events as well. Is that
expectation correct? If it is, then we need to think about extending
the mechanism that calls event handlers, not just about exposing the
likes of kbd_buffer_store_event to modules. We should also consider
how a module could define its own events.
IOW, this calls for a slightly more detailed specification, before
someone could sit down and code the stuff. I suggest to take a look
at how existing events are handled, and propose such a detailed
specification.
Thanks.
^ permalink raw reply [flat|nested] 9+ messages in thread
* bug#42028: [Feature Request] 27.0.91; Provide the ability dynamic modules to post events in emacs event loop
2020-06-24 14:44 ` Eli Zaretskii
@ 2020-06-24 16:17 ` yyoncho
2020-06-24 16:28 ` Eli Zaretskii
0 siblings, 1 reply; 9+ messages in thread
From: yyoncho @ 2020-06-24 16:17 UTC (permalink / raw
To: Eli Zaretskii; +Cc: 42028
[-- Attachment #1: Type: text/plain, Size: 322 bytes --]
> The problem is that I expect a module that inserts such events
> to want to be called to process those events as well. Is that
> expectation correct?
>
Yes. As a side note, this feature will be useful even for the core C part
for anything
that wants to perform async processing not including LispObject.
Thanks,
Ivan
[-- Attachment #2: Type: text/html, Size: 615 bytes --]
^ permalink raw reply [flat|nested] 9+ messages in thread
* bug#42028: [Feature Request] 27.0.91; Provide the ability dynamic modules to post events in emacs event loop
2020-06-24 16:17 ` yyoncho
@ 2020-06-24 16:28 ` Eli Zaretskii
2020-06-24 16:46 ` yyoncho
0 siblings, 1 reply; 9+ messages in thread
From: Eli Zaretskii @ 2020-06-24 16:28 UTC (permalink / raw
To: yyoncho; +Cc: 42028
> From: yyoncho <yyoncho@gmail.com>
> Date: Wed, 24 Jun 2020 19:17:30 +0300
> Cc: 42028@debbugs.gnu.org
>
> As a side note, this feature will be useful even for the core C part for anything
> that wants to perform async processing not including LispObject.
How will it help that? The event queue is processed synchronously.
^ permalink raw reply [flat|nested] 9+ messages in thread
* bug#42028: [Feature Request] 27.0.91; Provide the ability dynamic modules to post events in emacs event loop
2020-06-24 16:28 ` Eli Zaretskii
@ 2020-06-24 16:46 ` yyoncho
2020-06-24 17:07 ` Eli Zaretskii
0 siblings, 1 reply; 9+ messages in thread
From: yyoncho @ 2020-06-24 16:46 UTC (permalink / raw
To: Eli Zaretskii; +Cc: 42028
[-- Attachment #1: Type: text/plain, Size: 1017 bytes --]
Here it is an example: If we oversimplify json parsing consist of 2 parts.
1. JSON parsing to Jansson data structures
2. Converting Jasson data structures to lisp data structures.
If we assume that 1 is taking most of the time then we could write the
following code:
start(string, callback) -> convert string to char* -> post it to worker
thread -> worker thread does the parsing
-> worker thread post to main loop -> main loop converts Jansson ds to lisp
ds -> main loop
calls the callback passed on start
The same strategy probably could be applied to Font locking with
TreeSitter.
Thanks,
Ivan
On Wed, Jun 24, 2020 at 7:29 PM Eli Zaretskii <eliz@gnu.org> wrote:
> > From: yyoncho <yyoncho@gmail.com>
> > Date: Wed, 24 Jun 2020 19:17:30 +0300
> > Cc: 42028@debbugs.gnu.org
> >
> > As a side note, this feature will be useful even for the core C part for
> anything
> > that wants to perform async processing not including LispObject.
>
> How will it help that? The event queue is processed synchronously.
>
[-- Attachment #2: Type: text/html, Size: 1633 bytes --]
^ permalink raw reply [flat|nested] 9+ messages in thread
* bug#42028: [Feature Request] 27.0.91; Provide the ability dynamic modules to post events in emacs event loop
2020-06-24 16:46 ` yyoncho
@ 2020-06-24 17:07 ` Eli Zaretskii
2020-06-24 17:33 ` Ivan Yonchovski
0 siblings, 1 reply; 9+ messages in thread
From: Eli Zaretskii @ 2020-06-24 17:07 UTC (permalink / raw
To: yyoncho; +Cc: 42028
> From: yyoncho <yyoncho@gmail.com>
> Date: Wed, 24 Jun 2020 19:46:56 +0300
> Cc: 42028@debbugs.gnu.org
>
> 1. JSON parsing to Jansson data structures
> 2. Converting Jasson data structures to lisp data structures.
>
> If we assume that 1 is taking most of the time
Does it? Did someone time these separately?
> start(string, callback) -> convert string to char* -> post it to worker thread -> worker thread does the parsing
> -> worker thread post to main loop -> main loop converts Jansson ds to lisp ds -> main loop
> calls the callback passed on start
You cannot easily post to the queue from a worker thread, because the
queue cannot be posted to asynchronously.
And I lack the larger picture: how would this work within a framework
of some Emacs feature? E.g., how will this processed be triggered,
and by what kind of trigger?
> The same strategy probably could be applied to Font locking with TreeSitter.
Again, I don't see the wider picture. Are you familiar with how the
current JIT font-lock works? If so, can you explain which parts of
that will be replaced/modified, and how?
^ permalink raw reply [flat|nested] 9+ messages in thread
* bug#42028: [Feature Request] 27.0.91; Provide the ability dynamic modules to post events in emacs event loop
2020-06-24 17:07 ` Eli Zaretskii
@ 2020-06-24 17:33 ` Ivan Yonchovski
2020-06-24 18:04 ` Eli Zaretskii
0 siblings, 1 reply; 9+ messages in thread
From: Ivan Yonchovski @ 2020-06-24 17:33 UTC (permalink / raw
To: Eli Zaretskii; +Cc: 42028
Eli Zaretskii writes:
> Does it? Did someone time these separately?
IDK, this was just an example.
>> start(string, callback) -> convert string to char* -> post it to worker thread -> worker thread does the parsing
>> -> worker thread post to main loop -> main loop converts Jansson ds to lisp ds -> main loop
>> calls the callback passed on start
>
> You cannot easily post to the queue from a worker thread, because the
> queue cannot be posted to asynchronously.
AFAIK this is not possible at all - thus this feature request.
>
> And I lack the larger picture: how would this work within a framework
> of some Emacs feature? E.g., how will this processed be triggered,
> and by what kind of trigger?
The goal users to be able to handle more usecases. ATM the only way to
achieve that is hackish as described here
https://nullprogram.com/blog/2017/02/14/ + several practical usecases.
> Again, I don't see the wider picture. Are you familiar with how the
> current JIT font-lock works? If so, can you explain which parts of
> that will be replaced/modified, and how?
AFAIK the initial tree-sitter parsing (which does not involve anything
emacs related) may take 200-300ms. In other editors (e.g. vscode) this
part is not happening on the UI thread but in some extenal background
thread which then passes the AST to the main UI thread to do the actual
fontlock. IMO this model might be replicated in emacs in the event of TS
integration.
Thanks,
Ivan
^ permalink raw reply [flat|nested] 9+ messages in thread
* bug#42028: [Feature Request] 27.0.91; Provide the ability dynamic modules to post events in emacs event loop
2020-06-24 17:33 ` Ivan Yonchovski
@ 2020-06-24 18:04 ` Eli Zaretskii
2020-06-24 18:38 ` Ivan Yonchovski
0 siblings, 1 reply; 9+ messages in thread
From: Eli Zaretskii @ 2020-06-24 18:04 UTC (permalink / raw
To: Ivan Yonchovski; +Cc: 42028
> From: Ivan Yonchovski <yyoncho@gmail.com>
> Cc: 42028@debbugs.gnu.org
> Date: Wed, 24 Jun 2020 20:33:29 +0300
>
> > You cannot easily post to the queue from a worker thread, because the
> > queue cannot be posted to asynchronously.
>
> AFAIK this is not possible at all - thus this feature request.
Your feature just asked for a way to queue events, it didn't say
anything about doing that asynchronously, not from a function invoked
by the main thread. If you mean the latter, then it AFAIU would need
a serious redesign of the Emacs event queue, to make it accessible
from several threads running in parallel at once.
> > Again, I don't see the wider picture. Are you familiar with how the
> > current JIT font-lock works? If so, can you explain which parts of
> > that will be replaced/modified, and how?
>
> AFAIK the initial tree-sitter parsing (which does not involve anything
> emacs related) may take 200-300ms. In other editors (e.g. vscode) this
> part is not happening on the UI thread but in some extenal background
> thread which then passes the AST to the main UI thread to do the actual
> fontlock. IMO this model might be replicated in emacs in the event of TS
> integration.
What you describe is very different from how JIT font-lock works now.
Which is why I asked the question: I know (more or less) how it works
in other editors, I just don't yet understand well enough how
something like that would fit into the existing fontification
framework. I hope there is a way of fitting it, because otherwise it
would mean a serious surgery of the display engine as well, which will
make the job significantly larger and harder.
^ permalink raw reply [flat|nested] 9+ messages in thread
* bug#42028: [Feature Request] 27.0.91; Provide the ability dynamic modules to post events in emacs event loop
2020-06-24 18:04 ` Eli Zaretskii
@ 2020-06-24 18:38 ` Ivan Yonchovski
0 siblings, 0 replies; 9+ messages in thread
From: Ivan Yonchovski @ 2020-06-24 18:38 UTC (permalink / raw
To: Eli Zaretskii; +Cc: 42028
Eli Zaretskii writes:
>
> Your feature just asked for a way to queue events, it didn't say
> anything about doing that asynchronously, not from a function invoked
> by the main thread. If you mean the latter, then it AFAIU would need
> a serious redesign of the Emacs event queue, to make it accessible
> from several threads running in parallel at once.
Yes, sorry about that. I didn't mention it explicitly in the bug report
it was described in more details in the linked article.
> What you describe is very different from how JIT font-lock works now.
> Which is why I asked the question: I know (more or less) how it works
> in other editors, I just don't yet understand well enough how
> something like that would fit into the existing fontification
> framework. I hope there is a way of fitting it, because otherwise it
> would mean a serious surgery of the display engine as well, which will
> make the job significantly larger and harder.
IMO we are good here. At least on lsp-mode side we were able to
implement delayed semantic fontification as a result of async source.
But in lsp-mode case we are reseiving the messages from the server on
the UI thread.
Thanks,
Ivan
^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2020-06-24 18:38 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2020-06-24 7:14 bug#42028: [Feature Request] 27.0.91; Provide the ability dynamic modules to post events in emacs event loop Ivan Yonchovski
2020-06-24 14:44 ` Eli Zaretskii
2020-06-24 16:17 ` yyoncho
2020-06-24 16:28 ` Eli Zaretskii
2020-06-24 16:46 ` yyoncho
2020-06-24 17:07 ` Eli Zaretskii
2020-06-24 17:33 ` Ivan Yonchovski
2020-06-24 18:04 ` Eli Zaretskii
2020-06-24 18:38 ` Ivan Yonchovski
Code repositories for project(s) associated with this external index
https://git.savannah.gnu.org/cgit/emacs.git
https://git.savannah.gnu.org/cgit/emacs/org-mode.git
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.