Running process filters in another thread

Running process filters in another thread

[yyoncho]

[-- Attachment #1: Type: text/plain, Size: 916 bytes --]

Hi all,

I want to raise this topic regarding the rise of Language servers and the
performance
problems that are related to parsing process output on UI thread.

I am not familiar with emacs internals and I am not sure whether this is
doable
but I wonder whether providing the option to do the parsing(and probably
more?)
in a separate thread and then call the *filter* function on emacs side in UI
thread with elisp data structures like lists, hashmaps etc. instead of raw
string is feasible which would be similar to what is happening in
Javascript world.

I want to mention that the alternative approach is to create JSON-RPC
specific process e.
g. provide make-json-rpc-process which does what I have described above.

Sorry if this topic has been discussed in the past, I was unable to find
something on the topic and please correct me if there is a way to move the
parsing
outside out UI thread.

Thanks,
Ivan

[-- Attachment #2: Type: text/html, Size: 1227 bytes --]

Re: Running process filters in another thread

[Eli Zaretskii]

> From: yyoncho <yyoncho@gmail.com>
> Date: Fri, 28 Sep 2018 21:21:23 +0300
> 
> I want to raise this topic regarding the rise of Language servers and the performance
> problems that are related to parsing process output on UI thread.
> 
> I am not familiar with emacs internals and I am not sure whether this is doable
> but I wonder whether providing the option to do the parsing(and probably more?)
> in a separate thread and then call the *filter* function on emacs side in UI
> thread with elisp data structures like lists, hashmaps etc. instead of raw
> string is feasible which would be similar to what is happening in Javascript world.

Emacs threads are cooperative, and only one thread can run at any
given time.  So it i's unclear to me how running process filters in a
separate thread will help improve the performance in this case,
because Emacs will still be locked up when that other thread runs the
filter.

Or am I missing something?

Re: Running process filters in another thread

[Stefan Monnier]

> I want to raise this topic regarding the rise of Language servers and
> the performance problems that are related to parsing process output on
> UI thread.

I'm not necessarily surprised that there are performance problems there,
but I'm not actually familiar with them.  Could you point to concrete
examples, bug reports, and things like that?

Ideally, has someone investigated to see exactly where/when the
performance problems show up?

> I am not familiar with Emacs internals and I am not sure whether this
> is doable but I wonder whether providing the option to do the
> parsing(and probably more?)  in a separate thread and then call the
> *filter* function on emacs side in UI thread with elisp data
> structures like lists, hashmaps etc. instead of raw string is feasible
> which would be similar to what is happening in Javascript world.

It shouldn't be too hard to use a separate (OS-level) Emacs *process*
if you want to do the parsing without blocking the normal UI thread.
But it comes with other performance tradeoffs.


        Stefan

Fwd: Running process filters in another thread

[yyoncho]

[-- Attachment #1: Type: text/plain, Size: 2126 bytes --]

+emacs-devel

---------- Forwarded message ---------
From: yyoncho <yyoncho@gmail.com>
Date: Sat, Sep 29, 2018 at 12:39 AM
Subject: Re: Running process filters in another thread
To: <eliz@gnu.org>


Hi Eli,

Thanks for the quick response!

Excuse me if I am talking nonsense - my understanding is that there are
Emacs threads(the threads that are available using elisp code) which run
cooperatively and there are OS level threads which for example read the
output of a process and then pass it to the filter function as a string and
they might run in parallel and I was talking about running the pre-filter
function into such thread.

If this is not the case, I guess it won't be easy to spawn new os thread
and run eventually *pure* function to process the output?

Alternatively, I was thinking about creating some faster emacs specific
binary serialization/deserialization of data structures so elisp
programmers could move the parsing into separate emacs instance(ala
https://github.com/jwiegley/emacs-async) and then consume the output.

Thanks,
Ivan

On Sat, Sep 29, 2018 at 12:14 AM Eli Zaretskii <eliz@gnu.org> wrote:

> > From: yyoncho <yyoncho@gmail.com>
> > Date: Fri, 28 Sep 2018 21:21:23 +0300
> >
> > I want to raise this topic regarding the rise of Language servers and
> the performance
> > problems that are related to parsing process output on UI thread.
> >
> > I am not familiar with emacs internals and I am not sure whether this is
> doable
> > but I wonder whether providing the option to do the parsing(and probably
> more?)
> > in a separate thread and then call the *filter* function on emacs side
> in UI
> > thread with elisp data structures like lists, hashmaps etc. instead of
> raw
> > string is feasible which would be similar to what is happening in
> Javascript world.
>
> Emacs threads are cooperative, and only one thread can run at any
> given time.  So it i's unclear to me how running process filters in a
> separate thread will help improve the performance in this case,
> because Emacs will still be locked up when that other thread runs the
> filter.
>
> Or am I missing something?
>

[-- Attachment #2: Type: text/html, Size: 3045 bytes --]

Re: Running process filters in another thread

[yyoncho]

[-- Attachment #1: Type: text/plain, Size: 2648 bytes --]

Hi Stefan,

> Could you point to concrete
> examples, bug reports, and things like that?

There are several discussions in reddit/github related to lsp-mode being
slow
due to json parsing and the usual answer is that things are going to be
better
after switching to native json parsing available in Emacs 27 and the
following
issue is referenced: https://github.com/emacs-lsp/lsp-mode/issues/210 but I
believe that this is not going to happen to have the following points in
mind:

1. You could have multiple running LSP servers and each of them could post
big(1m+)
updates at any time.

2. A similar problem exists for Debug Adapter Protocol client dap-mode
(https://github.com/yyoncho/dap-mode) where you could have as much debug
sessions as you want and each of them could send an update at any time too.

My point is that Emacs should be able to process a lot of json
requests/responses
and still being responsive and if this requires parsing some text format
messages(e. g. Json/XML/emacs built-in) on UI thread this is not going to
happen.

> It shouldn't be too hard to use a separate (OS-level) Emacs *process*
> if you want to do the parsing without blocking the normal UI thread.
> But it comes with other performance tradeoffs.

Please correct me if I am wrong but I believe that I will still have to
parse
the input on UI thread but from a different format(e. g. using built-in
print1/read) which in the end will have the same result.

Thanks,
Ivan

On Sat, Sep 29, 2018 at 2:04 AM Stefan Monnier <monnier@iro.umontreal.ca>
wrote:

> > I want to raise this topic regarding the rise of Language servers and
> > the performance problems that are related to parsing process output on
> > UI thread.
>
> I'm not necessarily surprised that there are performance problems there,
> but I'm not actually familiar with them.  Could you point to concrete
> examples, bug reports, and things like that?
>
> Ideally, has someone investigated to see exactly where/when the
> performance problems show up?
>
> > I am not familiar with Emacs internals and I am not sure whether this
> > is doable but I wonder whether providing the option to do the
> > parsing(and probably more?)  in a separate thread and then call the
> > *filter* function on emacs side in UI thread with elisp data
> > structures like lists, hashmaps etc. instead of raw string is feasible
> > which would be similar to what is happening in Javascript world.
>
> It shouldn't be too hard to use a separate (OS-level) Emacs *process*
> if you want to do the parsing without blocking the normal UI thread.
> But it comes with other performance tradeoffs.
>
>
>         Stefan
>
>
>

[-- Attachment #2: Type: text/html, Size: 3606 bytes --]

Re: Fwd: Running process filters in another thread

[Eli Zaretskii]

> From: yyoncho <yyoncho@gmail.com>
> Date: Sat, 29 Sep 2018 09:54:30 +0300
> 
> Excuse me if I am talking nonsense - my understanding is that there are Emacs threads(the threads that are
> available using elisp code) which run cooperatively and there are OS level threads which for example read the
> output of a process and then pass it to the filter function as a string and they might run in parallel and I was
> talking about running the pre-filter function into such thread. 

Emacs reads output from subprocesses in its "normal" Lisp thread(s).
No special threads are involved.

> If this is not the case, I guess it won't be easy to spawn new os thread and run eventually *pure* function to
> process the output?

This should be possible, but I think we wouldn't like doing that, as
running async threads that need to feed the main thread is tricky, and
doesn't really go well with the architecture of Emacs.

> Alternatively, I was thinking about creating some faster emacs specific binary serialization/deserialization of
> data structures so elisp programmers could move the parsing into separate emacs instance(ala
> https://github.com/jwiegley/emacs-async) and then consume the output. 

Something like that, yes.  But I think we should first understand well
what part(s) of the LSP processing takes most CPU cycles, and then
discuss how best to integrate that into Emacs.  It could be, for
example, that implementing some portions of that in C will provide new
primitives that would allow to slash the processing time without
complicating the architecture.

Re: Running process filters in another thread

[Eli Zaretskii]

> From: yyoncho <yyoncho@gmail.com>
> Date: Sat, 29 Sep 2018 10:35:19 +0300
> Cc: emacs-devel@gnu.org
> 
> 1. You could have multiple running LSP servers and each of them could post big(1m+)
> updates at any time.

That's okay, since Emacs processes input from subprocesses only when
it's idle.  So as long as the user types commands, neither the user
nor the Lisp code that those commands run will be interrupted.

> My point is that Emacs should be able to process a lot of json requests/responses
> and still being responsive and if this requires parsing some text format
> messages(e. g. Json/XML/emacs built-in) on UI thread this is not going to
> happen.

I'm not sure I understand why this couldn't happen.

> > It shouldn't be too hard to use a separate (OS-level) Emacs *process*
> > if you want to do the parsing without blocking the normal UI thread.
> > But it comes with other performance tradeoffs.
> 
> Please correct me if I am wrong but I believe that I will still have to parse
> the input on UI thread but from a different format(e. g. using built-in
> print1/read) which in the end will have the same result.

No, you could have the subprocess send a parsed JSON object, I think.

But all of this is pure theory.  I suggest to create a benchmark for
parsing a large input of the kind that's expected from LSPs, and then
time it with the native JSON support in Emacs 27.  Then we will have a
better idea of what issues are involved.

Re: Running process filters in another thread

[Stefan Monnier]

>> It shouldn't be too hard to use a separate (OS-level) Emacs *process*
>> if you want to do the parsing without blocking the normal UI thread.
>> But it comes with other performance tradeoffs.
>
> Please correct me if I am wrong but I believe that I will still have
> to parse the input on UI thread but from a different
> format (e. g. using built-in print1/read) which in the end will have
> the same result.

The idea is that the other process would do the bulk of the "digestion"
so the amount of data it would send to the main Emacs (the "UI thread")
would hopefully be much smaller.

But I must admit not being sufficiently familiar with the details to be
able to discuss it seriously.  E.g. why would the LSP servers send us
megabytes of data?  Is it because it sends us back the whole file with
lots of detailed annotations everywhere (e.g. the kind of info we'd use
for font-lock-like purposes rather than for flymake-like purposes)?


        Stefan

Re: Running process filters in another thread

[yyoncho]

[-- Attachment #1: Type: text/plain, Size: 1880 bytes --]

Hi Stefan,

> E.g. why would the LSP servers send us
> megabytes of data?

Some of the responses/notifications might be big e. g.
1. Diagnostics
2. Semantic highlight data(proportional to file size)

and these combined with several concurrent LSP/DAP servers sending async
notifications. But as Eli mentioned I will have to prepare some test
results and
also with emacs 27 native json support to see how it behaves. I have
started the discussion assuming that the reading of the process output is
not
performed in the emacs UI thread but in OS thread and I was thinking that
adding
a filter to that pipeline won't be a problem. One we (lsp-mode
contributors) move
it to native json I will resume the discussion if with some concrete
testing data.

Thank you for your time.

Thanks,
Ivan





On Sat, Sep 29, 2018 at 9:11 PM Stefan Monnier <monnier@iro.umontreal.ca>
wrote:

> >> It shouldn't be too hard to use a separate (OS-level) Emacs *process*
> >> if you want to do the parsing without blocking the normal UI thread.
> >> But it comes with other performance tradeoffs.
> >
> > Please correct me if I am wrong but I believe that I will still have
> > to parse the input on UI thread but from a different
> > format (e. g. using built-in print1/read) which in the end will have
> > the same result.
>
> The idea is that the other process would do the bulk of the "digestion"
> so the amount of data it would send to the main Emacs (the "UI thread")
> would hopefully be much smaller.
>
> But I must admit not being sufficiently familiar with the details to be
> able to discuss it seriously.  E.g. why would the LSP servers send us
> megabytes of data?  Is it because it sends us back the whole file with
> lots of detailed annotations everywhere (e.g. the kind of info we'd use
> for font-lock-like purposes rather than for flymake-like purposes)?
>
>
>         Stefan
>
>
>

[-- Attachment #2: Type: text/html, Size: 2664 bytes --]

Re: Running process filters in another thread

[Stefan Monnier]

> Some of the responses/notifications might be big e. g.
> 1. Diagnostics
> 2. Semantic highlight data(proportional to file size)

So, IIUC the scenario is that the LSP sends us large JSON data
structures, but we only ever use a fairly small portion of it, so
presumably a significant part of the total processing time is spent
gobbling data and parsing it into an internal data structure.

Beside limiting the total amount of time spent doing this (e.g. by
processing the data more efficiently (e.g. in C rather in Elisp) or in
parallel), the other issue is the fact that it shouldn't cause pauses in
the user interaction.  This can be obtained via true concurrency, or by
slicing the processing so it can be interleaved finely with user
interaction without any noticeable pauses.

Maybe an "easy" way to solve this particular problem is to define the
json parser as a "yield point".  This would even allow the
implementation of the json parser to spawn a separate thread and do the
parsing in a truly concurrent thread (tho the GC would need to be
extended to be able to deal with such truly concurrent non-Elisp
threads allocating Elisp data structures, maybe this can be done by
making those json threads use their own Elisp heap which gets joined
into the main heap at the end).

> a filter to that pipeline won't be a problem. One we (lsp-mode
> contributors) move it to native json I will resume the discussion if
> with some concrete testing data.

IIUC eglot does use the native json parser.  BTW, I'd be happy to hear
about the use of jsonrpc.el in lsp-mode (either failures or successes)
and more generally consolidation of effort between eglot and lsp-mode.


        Stefan

Re: Running process filters in another thread

[yyoncho]

[-- Attachment #1: Type: text/plain, Size: 3703 bytes --]

Hi Stefan,

> So, IIUC the scenario is that the LSP sends us large JSON data
> structures, but we only ever use a fairly small portion of it, so
> presumably a significant part of the total processing time is spent
> gobbling data and parsing it into an internal data structure.

AFAIK this is not the case, LSP is pretty minimal. This could be true for
some
limited cases but in general we need all of the data.

> Beside limiting the total amount of time spent doing this (e.g. by
> processing the data more efficiently (e.g. in C rather in Elisp) or in
> parallel), the other issue is the fact that it shouldn't cause pauses in
> the user interaction.  This can be obtained via true concurrency, or by
> slicing the processing so it can be interleaved finely with user
> interaction without any noticeable pauses.

> Maybe an "easy" way to solve this particular problem is to define the
> json parser as a "yield point".  This would even allow the
> implementation of the json parser to spawn a separate thread and do the
> parsing in a truly concurrent thread (tho the GC would need to be
> extended to be able to deal with such truly concurrent non-Elisp
> threads allocating Elisp data structures, maybe this can be done by
> making those json threads use their own Elisp heap which gets joined
> into the main heap at the end).

Yeah, that is what I was looking for. I believe it will be a big
improvement from
UX standpoint of view. Also, it seems like if we have that kind of
infrastructure inplace this will open many other options.

> IIUC eglot does use the native json parser.  BTW, I'd be happy to hear
> about the use of jsonrpc.el in lsp-mode (either failures or successes)
> and more generally consolidation of effort between eglot and lsp-mode.

I would like to see consolidation too. I will file a task for investigating
jsonrpc.el.

Thanks,
Ivan


On Sat, Sep 29, 2018 at 11:30 PM Stefan Monnier <monnier@iro.umontreal.ca>
wrote:

> > Some of the responses/notifications might be big e. g.
> > 1. Diagnostics
> > 2. Semantic highlight data(proportional to file size)
>
> So, IIUC the scenario is that the LSP sends us large JSON data
> structures, but we only ever use a fairly small portion of it, so
> presumably a significant part of the total processing time is spent
> gobbling data and parsing it into an internal data structure.
>
> Beside limiting the total amount of time spent doing this (e.g. by
> processing the data more efficiently (e.g. in C rather in Elisp) or in
> parallel), the other issue is the fact that it shouldn't cause pauses in
> the user interaction.  This can be obtained via true concurrency, or by
> slicing the processing so it can be interleaved finely with user
> interaction without any noticeable pauses.
>
> Maybe an "easy" way to solve this particular problem is to define the
> json parser as a "yield point".  This would even allow the
> implementation of the json parser to spawn a separate thread and do the
> parsing in a truly concurrent thread (tho the GC would need to be
> extended to be able to deal with such truly concurrent non-Elisp
> threads allocating Elisp data structures, maybe this can be done by
> making those json threads use their own Elisp heap which gets joined
> into the main heap at the end).
>
> > a filter to that pipeline won't be a problem. One we (lsp-mode
> > contributors) move it to native json I will resume the discussion if
> > with some concrete testing data.
>
> IIUC eglot does use the native json parser.  BTW, I'd be happy to hear
> about the use of jsonrpc.el in lsp-mode (either failures or successes)
> and more generally consolidation of effort between eglot and lsp-mode.
>
>
>         Stefan
>

[-- Attachment #2: Type: text/html, Size: 4736 bytes --]

Re: Running process filters in another thread

[Stefan Monnier]

>> So, IIUC the scenario is that the LSP sends us large JSON data
>> structures, but we only ever use a fairly small portion of it, so
>> presumably a significant part of the total processing time is spent
>> gobbling data and parsing it into an internal data structure.
>
> AFAIK this is not the case, LSP is pretty minimal. This could be true
> for some limited cases but in general we need all of the data.

If we really use all of the data, then I can't see how processing the
bytes and parsing them can be the dominating factor, so
a `start-json-process` wouldn't make any significant difference.

But at least in your example of semantic highlight we definitely don't
have to use all of the data, but only the portion relevant for the
currently displayed parts of the buffer.


        Stefan

Re: Running process filters in another thread

[yyoncho]

[-- Attachment #1: Type: text/plain, Size: 1627 bytes --]

Hi Stefan,

> If we really use all of the data, then I can't see how processing the
> bytes and parsing them can be the dominating factor,

There are a lot of requests that are handled by updating some map(e. g.
received diagnostics
for buffer X) and the data is not used right away and the parsing is the
dominating factor.
This makes me think that we can span subprocess and use it as a store. I
will have to do some
benchmarking of communication between emacs and subprocess.

> No, you could have the subprocess send a parsed JSON object, I think.
@eliz@gnu.org <eliz@gnu.org> can you elaborate on that are you referring to
what emacs-async
(https://github.com/jwiegley/emacs-async) is doing?

Thanks,
Ivan

On Sun, Sep 30, 2018 at 4:01 PM Stefan Monnier <monnier@iro.umontreal.ca>
wrote:

> >> So, IIUC the scenario is that the LSP sends us large JSON data
> >> structures, but we only ever use a fairly small portion of it, so
> >> presumably a significant part of the total processing time is spent
> >> gobbling data and parsing it into an internal data structure.
> >
> > AFAIK this is not the case, LSP is pretty minimal. This could be true
> > for some limited cases but in general we need all of the data.
>
> If we really use all of the data, then I can't see how processing the
> bytes and parsing them can be the dominating factor, so
> a `start-json-process` wouldn't make any significant difference.
>
> But at least in your example of semantic highlight we definitely don't
> have to use all of the data, but only the portion relevant for the
> currently displayed parts of the buffer.
>
>
>         Stefan
>

[-- Attachment #2: Type: text/html, Size: 2356 bytes --]

Re: Running process filters in another thread

[Tom Tromey]

>>>>> ">" == yyoncho  <yyoncho@gmail.com> writes:

>> I am not familiar with emacs internals and I am not sure whether this is doable
>> but I wonder whether providing the option to do the parsing(and probably more?)
>> in a separate thread and then call the *filter* function on emacs side in UI
>> thread with elisp data structures like lists, hashmaps etc. instead of raw
>> string is feasible which would be similar to what is happening in Javascript world.

As Eli and Stefan have pointed out, this isn't likely to gain
performance due to the global lock.

I wanted to point out, though, that it is possible to lock a process to
a thread.  (info "(elisp) Processes and Threads")  So part of what you
want is there.

I think it would be good for Emacs to experiment with less cooperative
thread scheduling.  This came up before and was rejected; I should have
spoken up then, since my view is that this really isn't all that
dangerous, because in practice there just aren't that many threaded
programs.  Also, this is a necessary step toward true multi threading.

Tom