Re: Blocking calls and threads

[Eli Zaretskii <eliz@gnu.org>]

> From: Lynn Winebarger <owinebar@gmail.com>
> Date: Thu, 20 Apr 2023 09:06:16 -0400
> Cc: emacs-devel@gnu.org
> 
> > What do you mean by "blocking system calls", exactly?
> >
> > If you mean the likes of 'read' and 'write' (i.e. "blocking system
> > calls" on the OS level),
> 
> Almost - I mean the subrs that make those operations available to the
> lisp machine, e.g. insert-file-contents.

insert-file-contents does a lot of simple bookkeeping stuff, then
calls 'read' in a loop to read the file and decode the stuff it reads.
We could perhaps yield between reading and processing chunks (we don't
in the current Emacs), but would it help?  Modern systems are very
fast in reading local files, so you'd make insert-file-contents and
its callers much slower for no good reason.  Likewise in write-region.

> > then no, a thread which makes these calls
> > will not yield.  How can it? the implementation of those calls is not
> > in Emacs, so how can Emacs change the way these syscalls work?
> 
> Presumably whatever mechanism is used for the calls you identified
> below could be generalized.  In practical terms, it would mean
> assigning locks to every system resource that isn't inherently part of
> the lisp machine, in this case at least file descriptors.  Then, for
> example, the read call in emacs_intr_read (in sysdep.c) could be
> surrounded by a release of the global lock (which yields the thread of
> the lisp machine) and the re-acquisition of the global lock.  The file
> descriptor lock might be acquired after yielding the lisp thread, or
> it might be owned exclusively by the thread that opened it.

If you yield before issuing the system call, the system call will wait
until you re-acquire the lock.  So how will this help?

To be effective, this needs to yield _after_ issuing the system call,
so that the system call proceeds in parallel with Emacs doing
something else.  But since the system call is not implemented by
Emacs, I don't see how this could be done?  In a new non-Lisp thread
that we would start to issue the system call from it? is that what you
have in mind?  But then we'd have the "usual" problem with Emacs: the
huge global state that we have.  That non-Lisp thread cannot use any
machinery that changes the global state, nor call any Lisp or Lisp
primitives, so it will only be able to do very simple processing, thus
making the whole business much less beneficial, from the user's POV.

> > The "blocking system calls" which do yield are calls emitted from
> > Lisp: accept-process-output, sit-for, read-key-sequence, etc.
> 
> Are these identified as a group anywhere for reference?  Otherwise, I
> don't know what is included in the "etc".

Basically, anything that ends up calling thread_select, usually via
wait_reading_process_output.