Re: Concurrency via isolated process/thread

[Ihor Radchenko <yantar92@posteo.net>]

Eli Zaretskii <eliz@gnu.org> writes:

>> I was talking about Elisp heaps.
>
> I don't understand what you mean by "Elisp heaps".  Emacs allocates
> memory by using the system's memory-allocation routines.  We don't
> have our own heaps.
>
>> AFAIU, Elisp memory management is conceptually single-threaded.
>
> Again, I don't understand why you say that and what you mean by that.
> We just call malloc, and malloc on modern systems is thread-safe.

Does it mean that we can safely call, for example, Fcons asynchronously?

(I am saying the above because my understanding is limited, hoping that
you can give some pointers when I happen to be wrong.)

>> >   buffer-alist
>> 
>> I do not see how this is a problem to lock/unlock this variable.
>
> So your solution to each such problem is to lock variables?  If so,
> you will end up locking a lot of them, and how is this different from
> using the global lock we do today with Lisp threads?

The idea is to prevent simultaneous write, which will only lock for a
small fraction of time.

It is not always sufficient, of course. When the code expects the value
to be unchanged, the lock will take much longer, and may cause global
locking eventually.

>> >   buffer's base-buffer
>> 
>> I do not see it. May you point me to where this is changed?
>
> See set_buffer_internal_2.
>
> How do you investigate this stuff?  I type M-. on every macro and
> function call I see, recursively, and look what they do.  If you do
> the same, how come you overlook all these details?  And if you do not
> use M-., how do you expect to learn what the code does internally?

Yes, I use M-. and C-x p g. And I do follow the code. But it does not
mean that I fully understand it, sorry.

And I still fail to see where base-buffer is _changed_. Is base buffer
ever supposed to be changed?

>> >   buffer's undo-list
>> 
>> That's just a synchronization between old_buffer and
>> old_buffer->base_buffer.
>> I am not 100% sure why it is necessary to be done this way and manually
>> instead of making undo-list values in indirect buffers point to base
>> buffer.
>
> So you are now saying that code which worked in Emacs for decades does
> unnecessary stuff, and should be removed or ignored?

No, I am saying that the current logic of updating the undo-list will not work
when multiple async threads are involved. It will no longer be safe to
assume that we can safely update undo-list right before/after switching
current_buffer.

So, I asked if an alternative approach could be used instead.

>> I admit that I do not understand what the following comment is talking
>> about:
>> 
>>   /* Look down buffer's list of local Lisp variables
>>      to find and update any that forward into C variables.  */
>
> The C code accesses some buffer-local variables via Vfoo_bar C
> variables.  Those need to be updated when the current buffer changes.

Now, when you explained this, it is also a big problem. Such C variables
are a global state that needs to be kept up to date. Async will break
the existing logic of these updates.

>> >   buffer's point and begv/zv markers
>> 
>> AFAIU, these store the last point position and narrowing state.
>> I do not see much problem here, except a need to lock these variables
>> while writing them. They will not affect PT, BEGZ, and ZV in other
>> threads, even if those operate on the same buffer now.
>
> Oh, yes, they will: see fetch_buffer_markers, called by
> set_buffer_internal_2.

Do you mean that in the existing cooperative Elisp threads, if one
thread moves the point and yields to other thread, the other thread will
be left with point in the same position (arbitrary, from the point of
view of this other thread)?

>> >   buffer's marker list
>> 
>> May you point me where it is?
>
> In fetch_buffer_markers.  Again, I don't understand how you missed
> that.

Is it buffer's marker list? I thought that you are referring to
BUF_MARKERS, not to PT, BEGV, and ZV.

>> This sounds like a problem that is already solved by any program that
>> uses async threads. Maybe Po Lu can provide good insights.
>
> Programs that use async threads avoid global variables like the
> plague.  Emacs is full of them.

Fair. I still hope that Po Lu can comment on this.

(Do note that my original proposal is not about "true" async thread and
it is Po Lu who were pushing for "proper interlocking". But the current
discussion is still helpful either way.).

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>