Re: Concurrency via isolated process/thread

[Eli Zaretskii <eliz@gnu.org>]

> From: Po Lu <luangruo@yahoo.com>
> Cc: yantar92@posteo.net,  emacs-devel@gnu.org
> Date: Sat, 08 Jul 2023 15:48:02 +0800
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > That is already a problem, as long as we are talking about leaving
> > most of Emacs application code intact.  How do you ensure only the
> > main thread can process input and display?  A non-main thread can
> > easily call some function which prompts the user, e.g., with
> > yes-or-no-p, or force redisplay with sit-for, and what do you do when
> > that happens?
> 
> To signal an error.

Great! that means in practice no existing Lisp program could ever run
in a non-main thread.  It isn't a very practical solution.

Besides, non-main threads do sometimes legitimately need to prompt
the user.  It is not a programmer's error when they do.

> Threads other than the main thread should not call
> such functions, which is the approach taken by most toolkits and window
> systems.  (X being a notable exception, where every Xlib display is
> surrounded by one massive lock.)

I don't think such a simplistic solution suits a program such as
Emacs.

> This will mean that most user facing Lisp won't be able to run in
> parallel with the main thread, but that can be fixed, given enough time.

Fixed how?

> > That's not enough!  Interlocking will prevent disastrous changes to
> > the buffer object which risk leaving the buffer in inconsistent state,
> > but it cannot prevent one thread from changing point under the feet of
> > another thread.  Consider this sequence:
> >
> >   . thread A moves point to position P1
> >   . thread A yields
> >   . thread B moves point of the same buffer to position P2
> >   . thread B yields
> >   . thread A resumes and performs some processing assuming point is at P1
> 
> Lisp code that is interested in making edits this way will need to
> utilize synchronization mechanisms of their own, yes.

The above doesn't do any editing, it just accesses buffer text without
changing it.

> > Without some kind of critical section, invoked on the Lisp level,
> > whereby moving point and the subsequent processing cannot be
> > interrupted, how will asynchronous processing by several threads that
> > use the same buffer ever work?  And please note that the above is
> > problematic even if none of the threads change buffer text, i.e. they
> > all are just _reading_ buffer text.
> >
> > It follows that such asynchronous processing will have to be
> > explicitly accounted for on the level of Lisp programs, which means
> > thorough rewrite of most of Lisp code (and also a lot of C code).
> 
> Insofar as that Lisp code is actually interested in making use of
> multiple threads.

Why are we talking about multiple threads at all? don't we want to
allow some Lisp code run from non-main threads?

> > IOW, we are no longer talking about some change to Emacs, we are
> > talking about rewriting most or all of Emacs!
> 
> I think that as long as we make the C text editing primitives robust
> against such problems, authors of Lisp code that needs to edit buffer
> text from multiple threads will devise appropriate synchronization
> procedures for their specific use cases.  For example, to parse a buffer
> in the background, Semantic may chose to create a new thread with a
> temporary copy of the buffer that is being read.  Or, Gnus might do the
> same to fontify a Diff attachment in an article.  Lisp changes can all
> be accomplished gradually, of course, whereas the C-level changes will
> have to be completed all in one go.

Using a snapshot of some global resource, such as buffer text, works
only up to a point, and basically prohibits many potentially
interesting uses of threads.  That's because such snapshotting assumes
no significant changes happen in the original objects while processing
the snapshot, and that is only sometimes true.