bug#38807: [Feature request]: Support lisp workers like web workers.

[Eli Zaretskii <eliz@gnu.org>]

> Date: Tue, 31 Dec 2019 08:40:06 +0800
> From: HaiJun Zhang <netjune@outlook.com>
> Cc: 38807@debbugs.gnu.org
> 
>  We do have that on MS-Windows. Except that you'll be surprised how
>  much of "UI" in Emacs cannot be done in a separate thread, mainly
>  because the Lisp machine is under such complete control of what the UI
>  does, and you cannot run several instances of the Lisp machine
>  simultaneously and asynchronously. 
> 
> What about the following idea:
> 1. Make the current lisp machine be customized which has two profiles:
>     + full featured: as the current running lisp machine in emacs
>     + subset one: without all UI functions
> 2. Run the full featured one as emacs does now. It acts as the master lisp machine(for UI only), which behave
> like the UI thread(process) in the web browser.
> 3. Run some subset ones for workers. Workers are started by the master lisp machine. Workers can send
> messages to the master machine by calling some APIs. The messages are copied to the master lisp
> machine, so GCs don’t need to work across machines.
> 4. Provide some APIs for them to communicate with each other.

The Lisp interpreter already sort-of "queues" display changes, because
it only changes Lisp data structures, and then the display engine
references those data structures at display time.  So those data
structures serve as a kind-of "queue", since redisplay runs when Emacs
is idle (i.e., the last Lisp command completed its job).

But the problem is in the other direction: it's not that Lisp somehow
"drives" the UI, it's that the UI frequently calls into Lisp as part
of its job.  The simplest example is mode-line format: it includes
references to variables, and can also include :eval forms that call
the interpreter.

Your idea in fact means to have several isolated Lisp machines in the
same process.  But how can we do something like that without a very
radical redesign of Emacs, when so many things in Emacs are implicitly
part of the global state?  Buffers, global variables, windows,
frames--all those are global resources, and every thread will want to
access them.  Emacs was not designed to allow that.

Your idea can be implemented with two processes, though.  And there is
already a package called emacs-async which does that:

  https://github.com/jwiegley/emacs-async

The disadvantage is that it is cumbersome to share data between the
two instances of Emacs, and large amounts of data will make that
inefficient.