Re: Concurrency via isolated process/thread

[Ihor Radchenko <yantar92@posteo.net>]

Ihor Radchenko <yantar92@posteo.net> writes:

> 4. Buffer-local variables, defined in C have C variable equivalents that
>    are updated as Emacs changes current_buffer.
>    
>    AFAIU, their purpose is to make buffer-local variables and normal
>    Elisp variables uniformly accessible from C code - C code does not
>    need to worry about Vfoo being buffer-local or not, and just set it.
>
>    This is not compatible with async threads that work with several buffers.
>
>    I currently do not fully understand how defining C variables works in
>    DEFVAR_LISP.

Currently, Emacs has a huge `globals' struct, where each record is a
variable defined in Lisp.

On C side, we have macros like
#define Vdebug_on_error globals.f_Vdebug_on_error
So, Vdebug_on_error can be used pretending that it is an ordinary
variable.
The location of stored Elisp value is thus fixed in memory.

On Elisp side, the value cells of symbols point to special internal
type, called "object forwarder". It arranges the actual symbol value
to be a pointer to where the value is actually stored.

For global variables the above scheme is easy.
Tricky things start to happen when we make variable that has C
reference buffer-local - the Vfoo C variable will always point to
the same address in ~globals~, but that address can only represent a
single value.

So, Emacs has to keep updating ~globals~ every time we switch buffer:

1. An old value stored in ~globals~ should be recorded in
to-be-switched-away buffer object.
2. ~globals~ is reassigned to load the value from new buffer.

(the situation is a bit more complex, because updating is done
lazily, only when the new buffer actually has a separate
buffer-local binding for a given variable; but the basic scheme is
as described).

The same update is happening when Emacs enters/exits let-binding or
switches between current cooperative threads.

---

The easiest way to not break the current Emacs logic would be making
~globals~ thread-local. Then, the current code may remain intact, if
we are content with global variables synchronized (or not) between
threads manually (say, when a thread exits or when it calls something
like `thread-synchronize-globals').

~globals~ currently has 457 variables, which is \approx{} =sizeof (int) * 457=
memory per thread.

If memory is an issue, we may want to store only a list of actually
changed variables in thread and change the current approach with
Vfoo macro and have something like VSETfoo + VGETfoo. VSETfoo will
always assign thread-local binding, adding it as necessary. VGETfoo
will first check thread-local binding and reach out to global struct
as fallback.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>