Re: Concurrency via isolated process/thread

[Dmitry Gutov <dmitry@gutov.dev>]

On 09/07/2023 11:41, Eli Zaretskii wrote:
>> From: Ihor Radchenko<yantar92@posteo.net>
>> Cc: Po Lu<luangruo@yahoo.com>,emacs-devel@gnu.org
>> Date: Sun, 09 Jul 2023 07:57:47 +0000
>>
>> Eli Zaretskii<eliz@gnu.org>  writes:
>>
>>>> When programmers write such code for other interactive programs, they
>>>> are comfortable with the limitations of running code outside of the UI
>>>> thread.  Why should writing new, thread-safe Lisp for Emacs be any more
>>>> difficult?
>>> Because we'd need to throw away 40 years of Lisp programming, and
>>> rewrite almost every bit of what was written since then.  It's a huge
>>> setback for writing Emacs applications.
>> May you please elaborate why exactly do we need to rewrite everything?
> We already did, please read the previous messages.  In a nutshell:
> because most of the Lisp code we have cannot be run from an async
> thread.

IME most of the code we have works decently already, and what we've been 
missing is a way to speed up certain crunchy bits without spawning an 
additional Emacs process (with all the coding pain and overhead that 
that entails). Workloads, for example, like creating a buffer (pinned to 
the thread), calling a process (asynchronously or not), getting JSON 
from it, parsing said JSON, processing the result, and returning it in 
some shape to the parent (probably main) thread. Most of the work that 
Gnus does, I think, also fits in that rough category.

Reporting on progress can be done by sending messages to the main 
thread, either via a dedicated mechanism ("messages" like in JavaScript 
Workers), or by changing some global variable as decided by the code 
author. The variable access would have to be synchronized, of course.

The worker thread will need an efficient way to return the computation 
result too (which still might be large, memory-wise), though. Maybe 
it'll use the same communication mechanism as progress report, maybe 
not. But it's something to keep in mind.