On Wed, Jul 5, 2023, 8:41 AM Ihor Radchenko <yantar92@posteo.net> wrote:
Eli Zaretskii <eliz@gnu.org> writes:

>> It may be dumb (I have no experience with processes in C), but I have
>> something like the following in mind:
>>
>> 1. Main Emacs process has a normal Elisp thread that watches for async
>>    Emacs process requests.
>> 2. Once a request arrives, asking to get/modify main Emacs process data,
>>    the request is fulfilled synchronously and signaled back by writing
>>    to memory accessible by the async process.
>
> That solves part of the problem, maybe (assuming we'd want to allow
> shared memory in Emacs).

My idea is basically similar to the current schema of interacting
between process input/output and Emacs. But using data stream rather
than text stream.

Shared memory is one way. Or it may be something like sockets.
It's just that shared memory will be faster, AFAIU.

> ... The other parts -- how to implement async
> process requests so that they don't suffer from the same problem, and
> how to reference objects outside of the shared memory -- are still
> there.

I imagine that there will be a special "remote Lisp object" type.

1. Imagine that child Emacs process asks for a value of variable `foo',
   which is a list (1 2 3 4).
2. The child process requests parent Emacs to put the variable value
   into shared memory.
3. The parent process creates a new variable storing a link to (1 2 3
   4), to prevent (1 . (2 3 4)) cons cell from GC in the parent process
   - `foo#'. Then, it informs the child process about this variable.
4. The child process creates a new remote Lisp object #<remote cons foo#>.

5. Now consider that child process tries (setcar #<remote cons foo#> value).
   The `setcar' and other primitives will be modified to query parent
   process to perform the actual modification to
   (#<remote value> . (2 3 4))

6. Before exiting the child thread, or every time we need to copy remote
   object, #<remote ...> will be replaced by an actual newly created
   traditional object.

The best idea I've had for a general solution would be to make "concurrent" versions of the fundamental lisp objects that act like immutable git repositories, with the traditional versions of the objects acting as working copies but only recording changes.  Then each checked out copy could push charges back, and if the merge fails an exception would be thrown in the thread of that working copy which the elisp code could decide how to handle.  That would work for inter-process shared memory or plain in-process memory between threads.  Then locks are only needed for updating the main reference to the concurrent object.

Lynn