Re: Concurrency via isolated process/thread

[Ihor Radchenko <yantar92@posteo.net>]

Eli Zaretskii <eliz@gnu.org> writes:

>> AFAIK, reading buffer does not require moving the gap.
>
> We've been through that: at least xml.c moves the gap to allow
> external libraries access to buffer text as a single contiguous C
> string.  This is a capability I wouldn't want to lose, because it
> might come in handy in future developments.

move_gap_both should lock the buffer as it is buffer modification (of
the buffer_text object). htmlReadMemory should also lock it because it
expects constant string segment.

And if we really want xml.c:parse_region to be asynchronous (not
blocking other threads reading the same buffer), we can still do it at
the cost of extra memcpy into newly allocated contiguous memory segment.

>> The point adjustment in the base buffer is done simply by storing point
>> as a marker. We can do the same for thread-local point positions.
>
> I still don't quite see how this will work.  Indirect buffers don't
> introduce parallelism, and programs that modify indirect buffers
> _know_ that the text of the base buffer will also be modified.  By
> contrast, a thread that has been preempted won't know and won't expect
> that.  It could, for example, keep buffer positions in simple
> variables, not in markers; almost all Lisp programs do that, and use
> markers only in very special situations.

Any async thread should expect that current buffer might be modified. Or
lock the buffer for text modifications explicitly (the feature we should
probably provide - something more enforcing compared to read-only-mode
we have now).

> In addition, on the C level, some code computes pointers to buffer
> text via BYTE_POS_ADDR, and then uses the pointer as any C program
> would.  If such a thread is suspended, and some other thread modifies
> buffer text in the meantime, all those pointers will be invalid, and
> we have bugs.  So it looks like, if we want to allow concurrent access
> to buffers from several threads, we will have a lot of code rewriting
> on our hands, and the rewritten code will be less efficient, because
> it will have to always access buffer text via buffer positions and
> macros like FETCH_BYTE and fetch_char_advance; access through char *
> pointers will be lost forever.

Not necessarily lost. We should provide facilities to prevent buffer
from being modified ("write mutex").

This problem is not limited to buffers - any low-level function that
modifies C object struct must enforce the condition when other threads
cannot modify the same object. For example SETCAR will have to mark the
modified object non-writable first, set its car, and release the lock.

So, any time we need a guarantee that an object remains unchanged, we
should acquire object-specific write-preventing mutex.

Of course, such write locks should happen for short periods of time to
be efficient.

Some of the existing uses of BYTE_POS_ADDRS may be converted into
explicit dynamic calls to
 (n < GPT_BYTE ? 0 : GAP_SIZE) + n + BEG_ADDR - BEG_BYTE;
If necessary.

> So maybe we should take a step back and consider a restriction that
> only one thread can access a buffer at any given time?  WDYT?

Buffers are so central in Emacs that I do not want to give up before we
try our best.

Alternatively, I can try to look into other global states first and
leave async buffer access to later. If we can get rid of the truly
global states (which buffer point is not; given that each thread has an
exclusive lock on its buffer), we can later come back to per-thread
point and restriction.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>