Re: Blocking calls and threads

[Lynn Winebarger <owinebar@gmail.com>]

On Thu, Apr 20, 2023 at 9:37 AM Eli Zaretskii <eliz@gnu.org> wrote:
> > From: Lynn Winebarger <owinebar@gmail.com>
> > Date: Thu, 20 Apr 2023 09:06:16 -0400
> > Cc: emacs-devel@gnu.org
> >
> > > What do you mean by "blocking system calls", exactly?
> > >
> > > If you mean the likes of 'read' and 'write' (i.e. "blocking system
> > > calls" on the OS level),
> >
> > Almost - I mean the subrs that make those operations available to the
> > lisp machine, e.g. insert-file-contents.
>
> insert-file-contents does a lot of simple bookkeeping stuff, then
> calls 'read' in a loop to read the file and decode the stuff it reads.
> We could perhaps yield between reading and processing chunks (we don't
> in the current Emacs), but would it help?  Modern systems are very
> fast in reading local files, so you'd make insert-file-contents and
> its callers much slower for no good reason.  Likewise in write-region.

I don't know whether it's a good idea.  I'm not sure how anyone can
work out how to provide explicit but generic asynchronous programming
constructs, as Stefan is, when some operations are implemented in ways
that don't allow it.

>
> If you yield before issuing the system call, the system call will wait
> until you re-acquire the lock.  So how will this help?

You're talking about yielding the system thread, I'm talking about
yielding the Lisp machine thread.  Even though Lisp machine threads
are implemented by mapping them to the underlying system thread,  the
Lisp machine execution state is kept coherent by the global
(interpreter) lock.  Releasing the lock is effectively yielding the
Lisp machine thread.  The system thread will yield implicitly if the
read blocks.

>
> To be effective, this needs to yield _after_ issuing the system call,
> so that the system call proceeds in parallel with Emacs doing
> something else.  But since the system call is not implemented by
> Emacs, I don't see how this could be done?  In a new non-Lisp thread
> that we would start to issue the system call from it? is that what you
> have in mind?  But then we'd have the "usual" problem with Emacs: the
> huge global state that we have.  That non-Lisp thread cannot use any
> machinery that changes the global state, nor call any Lisp or Lisp
> primitives, so it will only be able to do very simple processing, thus
> making the whole business much less beneficial, from the user's POV.

So making "insert-file-contents" yield would introduce race conditions
with respect to the buffer being read into that the lisp programmer
choosing to use threads would have to be responsible for (which would
be what Stefan is trying to simplify, I think).  I don't mean
lisp-machine violating race conditions - the read operation should
either use some temporary buffer and copy into the lisp buffer, or
wait for data to be ready, then reacquire the GIL before invoking the
read syscall with a pointer into the Lisp buffer object.

Lynn