Re: Concurrency via isolated process/thread

[Ihor Radchenko <yantar92@posteo.net>]

Eli Zaretskii <eliz@gnu.org> writes:

>> My idea with isolated thread is similar to having a bunch of state
>> variables coped to the thread before executing it. Interlocking will
>> still be necessary if the isolated thread wants to do anything with the
>> actual global state (like buffer modification, for example).
>
> How would we know which part(s) of the global state to copy, and how
> will the Lisp program running in the thread know which variables it
> can safely access?

This is a difficult question.
On one hand, it does not make sense to copy everything - it will imply
multiplying Emacs memory footprint.
On the other hand, we cannot know in advance which variables will
actually be used by a thread.

So, it can be something like copy-on-read - the child thread will copy
the necessary values from parent thread as running Elisp code needs
them.

By default, such copy will not be synchronized with the parent Emacs
thread, so that we do not need to worry about parent thread changing the
values asynchronously. 

Manually, it may also be allowed to create "remote" values that will
query the other thread: thread 1 will hold (foo = '(1 2 3)) and thread 2
will hold (foo = #<remote foo>). Then, thread 2 requesting to read/write
value will query thread 1.
The range of variables that can be made remote should be minimized and
possibly also restricted to a safe subset of variables.

> ... If I am the Lisp programmer writing code for such
> a thread, how can I know what is and what isn't allowed?

Everything is allowed, but global state changes will not necessarily
propagate to the parent Emacs thread. They will be confined to that
child async thread.

> ...And what
> happens if I do something that is not allowed?  And finally, does it
> mean we cannot run existing Lisp programs in such threads, but must
> program for them from scratch?

Non-trivial programs that need to modify the global Emacs state will
have to be written specially. But programs that only return a value will
work as usual, without a need to modify them.

The idea is similar to https://github.com/jwiegley/emacs-async (or to
`org-export-async-start'), but with more tight communication between
Emacs processes. The original emacs-async requires a new Emacs instance
that will also need to re-load all the necessary packages manually from
startup.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>