bug#38807: [Feature request]: Support lisp workers like web workers.

[Eli Zaretskii <eliz@gnu.org>]

> Date: Thu, 26 Mar 2020 21:37:39 +0800
> From: HaiJun Zhang <netjune@outlook.com>
> Cc: michael.albinus@gmx.de, 38807@debbugs.gnu.org, yyoncho@gmail.com
> 
>  I'm not sure I understand the response. Let me repeat the question:
>  while the module thread parses JSON, will the main thread wait for it,
>  or will it do something else? Can you describe how this would work
>  using some use case where lsp-mode is used, like for completing on
>  program symbols?
> 
> 1 After user inputs a char, lsp-mode call the module api to setup a completion task. The task has a task id.
>  The lsp-mode saves the task id as the current task and bind a callback function to it. It then returns.
> 2 The module create a task and adds it to its task queue.
> 3 The worker thread in the module fetches the task from task queue and executes it. It builds the json-rpc
>  request and sends it to the lsp server. And then it waits for the reply from the lsp server.
> 4 The worker receives the reply from lsp server. It parses the json message and builds the completion
>  result to a list(in lisp).
> 5 The worker thread posts a message to emacs. The message includes the task id and the completion
>  result.
> 6 Emacs receives the message and dispatches it to lsp-mode (by calling a function in lsp-mode). The
>  function checks that it is the result of the completion ask and call the task callback function which will
>  popups a menu to display the completion items.

If the above describes what happens when the user requests completion,
then I deduce that the user waits for the entire process you described
to finish, because the user cannot continue without seeing the
completion candidates, and those are only available after item 6 above
is done.

Since the user waits for this job to finish anyway, why does it help
to run some of this processing in a separate thread?

>  It will be good if module thread can post message to lisp thread. It will be
>  better if module thread can send lisp data within the message to lisp thread.
> 
>  Posting messages is possible by writing to a pipe. But I don't think
>  I understand what you mean by "send Lisp data" -- how (in what form)
>  can Lisp data be sent?
> 
> It  can be the above completion list, a point to a lisp object which can be passed to emacs to for lsp-mode to
> use. 

A Lisp object that is not stored in the data structures maintained by
alloc.c is not really a Lisp object that has a meaning for Emacs, I
think.

>  My understanding is that pdumper can serialize and deserialize lisp data. Maybe we can prepare
>  data with its
>  format and let it deserialize them.
> 
>  But we already do that: the libjansson library "serializes" the data,
>  and we then deserialize it in Emacs as we get the data from the
>  library. That deserialization is what takes the time you are trying
>  to make shorter. 
> 
> Before lsp-mode, I used completion tools like irony-mode for c/c++ and gocode for golang. They work very
> smoothly.
> 
> Now I have used lsp-mode with emacs-27 or emacs master for several weeks. It doesn’t work as smoothly
> as the above tools. It lags.
> 
> I added some debug messages to lsp-mode and see that there are too many(about 10~30) json messages
> arrived after I input every char. Emacs has to parse and process all of them on every key pressing.
> 
> For the old completion tools, there are only one or two json messages arrived on every key pressing.
> 
> Maybe it is the problem of the lsp server. But it is hard to modify them.

I don't think I understand how this is related to the serialization
issue and the pdumper.