Blocking calls and threads

Blocking calls and threads

[Lynn Winebarger]

The thread on how to make asynchronous behavior explicit made me
curious whether making a blocking system call would cause the lisp
thread to yield to other lisp threads (release the global lock).  Do
blocking system calls yield the lisp thread, or is there any way in
lisp code to call blocking functions so that the lisp thread will
yield while the system thread blocks?

Thanks,
Lynn

Re: Blocking calls and threads

[Po Lu]

Lynn Winebarger <owinebar@gmail.com> writes:

> The thread on how to make asynchronous behavior explicit made me
> curious whether making a blocking system call would cause the lisp
> thread to yield to other lisp threads (release the global lock).  Do
> blocking system calls yield the lisp thread, or is there any way in
> lisp code to call blocking functions so that the lisp thread will
> yield while the system thread blocks?

System calls don't currently yield the Lisp thread, apart from pselect.

Re: Blocking calls and threads

[Eli Zaretskii]

> From: Lynn Winebarger <owinebar@gmail.com>
> Date: Thu, 20 Apr 2023 01:31:14 -0400
> 
> The thread on how to make asynchronous behavior explicit made me
> curious whether making a blocking system call would cause the lisp
> thread to yield to other lisp threads (release the global lock).  Do
> blocking system calls yield the lisp thread, or is there any way in
> lisp code to call blocking functions so that the lisp thread will
> yield while the system thread blocks?

What do you mean by "blocking system calls", exactly?

If you mean the likes of 'read' and 'write' (i.e. "blocking system
calls" on the OS level), then no, a thread which makes these calls
will not yield.  How can it? the implementation of those calls is not
in Emacs, so how can Emacs change the way these syscalls work?

The "blocking system calls" which do yield are calls emitted from
Lisp: accept-process-output, sit-for, read-key-sequence, etc.

Re: Blocking calls and threads

[Lynn Winebarger]

On Thu, Apr 20, 2023 at 3:23 AM Eli Zaretskii <eliz@gnu.org> wrote:
> > From: Lynn Winebarger <owinebar@gmail.com>
> > Date: Thu, 20 Apr 2023 01:31:14 -0400
> >
> > The thread on how to make asynchronous behavior explicit made me
> > curious whether making a blocking system call would cause the lisp
> > thread to yield to other lisp threads (release the global lock).  Do
> > blocking system calls yield the lisp thread, or is there any way in
> > lisp code to call blocking functions so that the lisp thread will
> > yield while the system thread blocks?
>
> What do you mean by "blocking system calls", exactly?
>
> If you mean the likes of 'read' and 'write' (i.e. "blocking system
> calls" on the OS level),

Almost - I mean the subrs that make those operations available to the
lisp machine, e.g. insert-file-contents.

> then no, a thread which makes these calls
> will not yield.  How can it? the implementation of those calls is not
> in Emacs, so how can Emacs change the way these syscalls work?

Presumably whatever mechanism is used for the calls you identified
below could be generalized.  In practical terms, it would mean
assigning locks to every system resource that isn't inherently part of
the lisp machine, in this case at least file descriptors.  Then, for
example, the read call in emacs_intr_read (in sysdep.c) could be
surrounded by a release of the global lock (which yields the thread of
the lisp machine) and the re-acquisition of the global lock.  The file
descriptor lock might be acquired after yielding the lisp thread, or
it might be owned exclusively by the thread that opened it.

>
> The "blocking system calls" which do yield are calls emitted from
> Lisp: accept-process-output, sit-for, read-key-sequence, etc.

Are these identified as a group anywhere for reference?  Otherwise, I
don't know what is included in the "etc".

Thanks,
Lynn

Re: Blocking calls and threads

[Po Lu]

Lynn Winebarger <owinebar@gmail.com> writes:

> Presumably whatever mechanism is used for the calls you identified
> below could be generalized.  In practical terms, it would mean
> assigning locks to every system resource that isn't inherently part of
> the lisp machine, in this case at least file descriptors.  Then, for
> example, the read call in emacs_intr_read (in sysdep.c) could be
> surrounded by a release of the global lock (which yields the thread of
> the lisp machine) and the re-acquisition of the global lock.  The file
> descriptor lock might be acquired after yielding the lisp thread, or
> it might be owned exclusively by the thread that opened it.

I'm afraid that's not so easy.  Parts of Emacs have certainly been
designed with the assumption that context switching cannot happen inside
read and friends; at least the Android and NS ports have this problem.

> Are these identified as a group anywhere for reference?  Otherwise, I
> don't know what is included in the "etc".

Basically anything that calls `thread_select'.  This includes almost
everything that waits for input.  Please grep around in the C sources.

Re: Blocking calls and threads

[Eli Zaretskii]

> From: Lynn Winebarger <owinebar@gmail.com>
> Date: Thu, 20 Apr 2023 09:06:16 -0400
> Cc: emacs-devel@gnu.org
> 
> > What do you mean by "blocking system calls", exactly?
> >
> > If you mean the likes of 'read' and 'write' (i.e. "blocking system
> > calls" on the OS level),
> 
> Almost - I mean the subrs that make those operations available to the
> lisp machine, e.g. insert-file-contents.

insert-file-contents does a lot of simple bookkeeping stuff, then
calls 'read' in a loop to read the file and decode the stuff it reads.
We could perhaps yield between reading and processing chunks (we don't
in the current Emacs), but would it help?  Modern systems are very
fast in reading local files, so you'd make insert-file-contents and
its callers much slower for no good reason.  Likewise in write-region.

> > then no, a thread which makes these calls
> > will not yield.  How can it? the implementation of those calls is not
> > in Emacs, so how can Emacs change the way these syscalls work?
> 
> Presumably whatever mechanism is used for the calls you identified
> below could be generalized.  In practical terms, it would mean
> assigning locks to every system resource that isn't inherently part of
> the lisp machine, in this case at least file descriptors.  Then, for
> example, the read call in emacs_intr_read (in sysdep.c) could be
> surrounded by a release of the global lock (which yields the thread of
> the lisp machine) and the re-acquisition of the global lock.  The file
> descriptor lock might be acquired after yielding the lisp thread, or
> it might be owned exclusively by the thread that opened it.

If you yield before issuing the system call, the system call will wait
until you re-acquire the lock.  So how will this help?

To be effective, this needs to yield _after_ issuing the system call,
so that the system call proceeds in parallel with Emacs doing
something else.  But since the system call is not implemented by
Emacs, I don't see how this could be done?  In a new non-Lisp thread
that we would start to issue the system call from it? is that what you
have in mind?  But then we'd have the "usual" problem with Emacs: the
huge global state that we have.  That non-Lisp thread cannot use any
machinery that changes the global state, nor call any Lisp or Lisp
primitives, so it will only be able to do very simple processing, thus
making the whole business much less beneficial, from the user's POV.

> > The "blocking system calls" which do yield are calls emitted from
> > Lisp: accept-process-output, sit-for, read-key-sequence, etc.
> 
> Are these identified as a group anywhere for reference?  Otherwise, I
> don't know what is included in the "etc".

Basically, anything that ends up calling thread_select, usually via
wait_reading_process_output.

Re: Blocking calls and threads

[Lynn Winebarger]

On Thu, Apr 20, 2023 at 9:37 AM Eli Zaretskii <eliz@gnu.org> wrote:
> > From: Lynn Winebarger <owinebar@gmail.com>
> > Date: Thu, 20 Apr 2023 09:06:16 -0400
> > Cc: emacs-devel@gnu.org
> >
> > > What do you mean by "blocking system calls", exactly?
> > >
> > > If you mean the likes of 'read' and 'write' (i.e. "blocking system
> > > calls" on the OS level),
> >
> > Almost - I mean the subrs that make those operations available to the
> > lisp machine, e.g. insert-file-contents.
>
> insert-file-contents does a lot of simple bookkeeping stuff, then
> calls 'read' in a loop to read the file and decode the stuff it reads.
> We could perhaps yield between reading and processing chunks (we don't
> in the current Emacs), but would it help?  Modern systems are very
> fast in reading local files, so you'd make insert-file-contents and
> its callers much slower for no good reason.  Likewise in write-region.

I don't know whether it's a good idea.  I'm not sure how anyone can
work out how to provide explicit but generic asynchronous programming
constructs, as Stefan is, when some operations are implemented in ways
that don't allow it.

>
> If you yield before issuing the system call, the system call will wait
> until you re-acquire the lock.  So how will this help?

You're talking about yielding the system thread, I'm talking about
yielding the Lisp machine thread.  Even though Lisp machine threads
are implemented by mapping them to the underlying system thread,  the
Lisp machine execution state is kept coherent by the global
(interpreter) lock.  Releasing the lock is effectively yielding the
Lisp machine thread.  The system thread will yield implicitly if the
read blocks.

>
> To be effective, this needs to yield _after_ issuing the system call,
> so that the system call proceeds in parallel with Emacs doing
> something else.  But since the system call is not implemented by
> Emacs, I don't see how this could be done?  In a new non-Lisp thread
> that we would start to issue the system call from it? is that what you
> have in mind?  But then we'd have the "usual" problem with Emacs: the
> huge global state that we have.  That non-Lisp thread cannot use any
> machinery that changes the global state, nor call any Lisp or Lisp
> primitives, so it will only be able to do very simple processing, thus
> making the whole business much less beneficial, from the user's POV.

So making "insert-file-contents" yield would introduce race conditions
with respect to the buffer being read into that the lisp programmer
choosing to use threads would have to be responsible for (which would
be what Stefan is trying to simplify, I think).  I don't mean
lisp-machine violating race conditions - the read operation should
either use some temporary buffer and copy into the lisp buffer, or
wait for data to be ready, then reacquire the GIL before invoking the
read syscall with a pointer into the Lisp buffer object.

Lynn

Re: Blocking calls and threads

[Lynn Winebarger]

On Thu, Apr 20, 2023 at 9:28 AM Po Lu <luangruo@yahoo.com> wrote:
> Lynn Winebarger <owinebar@gmail.com> writes:
> > Presumably whatever mechanism is used for the calls you identified
> > below could be generalized.  In practical terms, it would mean
> > assigning locks to every system resource that isn't inherently part of
> > the lisp machine, in this case at least file descriptors.  Then, for
> > example, the read call in emacs_intr_read (in sysdep.c) could be
> > surrounded by a release of the global lock (which yields the thread of
> > the lisp machine) and the re-acquisition of the global lock.  The file
> > descriptor lock might be acquired after yielding the lisp thread, or
> > it might be owned exclusively by the thread that opened it.
>
> I'm afraid that's not so easy.  Parts of Emacs have certainly been
> designed with the assumption that context switching cannot happen inside
> read and friends; at least the Android and NS ports have this problem.

I didn't mean to imply that was *all* that was required - although I
would distinguish concerns around maintaining consistency of the Lisp
machine state (heap/globals) from the correctness of Lisp code that
has race conditions on buffer objects, etc.  I'm not saying the latter
don't have to be addressed, just that they pose distinct engineering
problems from the former.  It's not unlike the move from dynamic to
lexical scope.  Perhaps some day there will be a `thread-safe' local
variable.

> > Are these identified as a group anywhere for reference?  Otherwise, I
> > don't know what is included in the "etc".
>
> Basically anything that calls `thread_select'.  This includes almost
> everything that waits for input.  Please grep around in the C sources.

Thanks,
Lynn

Re: Blocking calls and threads

[Eli Zaretskii]

> From: Lynn Winebarger <owinebar@gmail.com>
> Date: Thu, 20 Apr 2023 10:19:11 -0400
> Cc: emacs-devel@gnu.org
> 
> > If you yield before issuing the system call, the system call will wait
> > until you re-acquire the lock.  So how will this help?
> 
> You're talking about yielding the system thread, I'm talking about
> yielding the Lisp machine thread.

No, I'm also talking about the Lisp machine thread.  The thread which
calls insert-file-contents and runs the C code of insert-file-contents
and of the subroutines called by insert-file-contents.

> Even though Lisp machine threads are implemented by mapping them to
> the underlying system thread, the Lisp machine execution state is
> kept coherent by the global (interpreter) lock.  Releasing the lock
> is effectively yielding the Lisp machine thread.  The system thread
> will yield implicitly if the read blocks.

What you say here is not relevant to the issue at hand.

> the read operation should either use some temporary buffer and copy
> into the lisp buffer, or wait for data to be ready, then reacquire
> the GIL before invoking the read syscall with a pointer into the
> Lisp buffer object.

Yes, and that's exactly where we will lose: most of the heavy
processing cannot be done in a separate temporary buffer, because it
calls functions from the Lisp machine, and those are written assuming
nothing else is running in the Lisp machine concurrently.  For
example, take the code which decodes the file's contents we have just
read.  I encourage you to take a good look at that code (most of it is
in coding.c) to appreciate the magnitude of the problem.

So the code which can run in parallel with another Lisp thread will be
able to do only very simple jobs, and will also add overhead due to
the need of copying stuff from temporary buffers to Lisp objects.

Of course, we could redesign and reimplement this stuff, but that's a
lot of non-trivial work.  My assumption was that you are considering
relatively lightweight changes on top of the existing code, not a
complete redesign of how these primitives work.

Re: Blocking calls and threads

[Lynn Winebarger]

I'm not sure what the etiquette is here - I keep referring to Stefan's
effort on futur.el, so I've added him explicitly.

On Thu, Apr 20, 2023 at 10:36 AM Eli Zaretskii <eliz@gnu.org> wrote:
>
> > From: Lynn Winebarger <owinebar@gmail.com>
> > Date: Thu, 20 Apr 2023 10:19:11 -0400
> > Cc: emacs-devel@gnu.org
> >
> > > If you yield before issuing the system call, the system call will wait
> > > until you re-acquire the lock.  So how will this help?
> >
> > You're talking about yielding the system thread, I'm talking about
> > yielding the Lisp machine thread.
>
> No, I'm also talking about the Lisp machine thread.  The thread which
> calls insert-file-contents and runs the C code of insert-file-contents
> and of the subroutines called by insert-file-contents.

A lisp thread is the context (lisp machine state) observable
(intentionally) by lisp programs.  That is conceptually distinct from
the context tracked by the OS thread.  To paraphrase a great mind, the
identification of the two "is an implementation detail".  Meaning, it
is not normative with respect to the intended semantics of lisp
threads.

> > Even though Lisp machine threads are implemented by mapping them to
> > the underlying system thread, the Lisp machine execution state is
> > kept coherent by the global (interpreter) lock.  Releasing the lock
> > is effectively yielding the Lisp machine thread.  The system thread
> > will yield implicitly if the read blocks.
>
> What you say here is not relevant to the issue at hand.

I don't know what you think is the issue at hand.  My question was
about when (or if) the lisp thread could yield during a blocking I/O
operation.  You appear to have interpreted that question differently
than I intended, so I attempted to be more explicit about what I meant
by "the lisp thread" and "yielding".

> > the read operation should either use some temporary buffer and copy
> > into the lisp buffer, or wait for data to be ready, then reacquire
> > the GIL before invoking the read syscall with a pointer into the
> > Lisp buffer object.
>
> Yes, and that's exactly where we will lose: most of the heavy
> processing cannot be done in a separate temporary buffer, because it
> calls functions from the Lisp machine, and those are written assuming
> nothing else is running in the Lisp machine concurrently.  For
> example, take the code which decodes the file's contents we have just
> read.  I encourage you to take a good look at that code (most of it is
> in coding.c) to appreciate the magnitude of the problem.

I believe you. I did mention that the global lock could be (evidently
*must be*) reacquired before the actual call to "read".  There's also
Tromey's comment on
https://www.reddit.com/r/emacs/comments/utzxir/a_vision_of_a_multithreaded_emacs/.

OTOH, I asked the question in order to understand what it means to
give programmers control over asynchronous execution in controlled
ways.  The most basic kind of control I can think of is whether
functions called in that code are expected to behave synchronously or
asynchronously.  And what is a more basic operation, that could be
done synchronously or asynchronously, than reading text from a file.
If something like insert-file-contents can't be performed
asynchronously (not in parallel, just with other code running while
waiting for IO), the scope of what Stefan's effort is going to be very
limited.  Limited to the point of not being very interesting.

It's also possible that something could be built on the existing data
structures that doesn't rely on more fine-grained locking.  I just
happened to be reading
https://www.gnu.org/software/emacs/manual/html_node/emacs/VCS-Merging.html
today, after seeing this message, and it seemed to me highly relevant
to the problem of concurrent work on a text buffer.  Particularly the
comment "Experience has shown that merging is superior to locking".
Maybe the thing to do to enable asynchronous/concurrent/parallel work
would be to add a new type of buffer, built on the existing one, that
behaves something like a git repo/working copy of the text buffer,
that eventually merges edits, say by windows displaying the buffer
each pulling updates from all the "repos" with checked-out copies.
Then the model for support of asynchronous programming could
encapsulate specifying how to merge and/or handle merge failure.

Maybe that would be too expensive, but at least at first, these
distributed buffers would only be used by programs using explicit
asynchronous programming.  Maybe that approach would even be helpful
in dealing with extremely large files or long lines.  We could call it
"merge-oriented programming". :-)

> So the code which can run in parallel with another Lisp thread will be
> able to do only very simple jobs, and will also add overhead due to
> the need of copying stuff from temporary buffers to Lisp objects.

I'm not talking about running in parallel - there is still just one
lisp machine in this hypothetical.

> Of course, we could redesign and reimplement this stuff, but that's a
> lot of non-trivial work.  My assumption was that you are considering
> relatively lightweight changes on top of the existing code, not a
> complete redesign of how these primitives work.

I wasn't considering anything.  I asked a very limited question for
the purpose of giving some hard thought to the language problem Stefan
requested assistance on.  I didn't offer these elaborations because I
have any plans, only to respond to your question "How could it?".
That is a purely hypothetical question.

Lynn

Re: Blocking calls and threads

[Lynn Winebarger]

[-- Attachment #1: Type: text/plain, Size: 6487 bytes --]

On Fri, Apr 21, 2023, 12:39 AM Lynn Winebarger <owinebar@gmail.com> wrote:

> I'm not sure what the etiquette is here - I keep referring to Stefan's
> effort on futur.el, so I've added him explicitly.
>
> On Thu, Apr 20, 2023 at 10:36 AM Eli Zaretskii <eliz@gnu.org> wrote:
> >
> > > From: Lynn Winebarger <owinebar@gmail.com>
> > > Date: Thu, 20 Apr 2023 10:19:11 -0400
> > > Cc: emacs-devel@gnu.org
> > >
> > > > If you yield before issuing the system call, the system call will
> wait
> > > > until you re-acquire the lock.  So how will this help?
> > >
> > > You're talking about yielding the system thread, I'm talking about
> > > yielding the Lisp machine thread.
> >
> > No, I'm also talking about the Lisp machine thread.  The thread which
> > calls insert-file-contents and runs the C code of insert-file-contents
> > and of the subroutines called by insert-file-contents.
>
> A lisp thread is the context (lisp machine state) observable
> (intentionally) by lisp programs.  That is conceptually distinct from
> the context tracked by the OS thread.  To paraphrase a great mind, the
> identification of the two "is an implementation detail".  Meaning, it
> is not normative with respect to the intended semantics of lisp
> threads.
>
> > > Even though Lisp machine threads are implemented by mapping them to
> > > the underlying system thread, the Lisp machine execution state is
> > > kept coherent by the global (interpreter) lock.  Releasing the lock
> > > is effectively yielding the Lisp machine thread.  The system thread
> > > will yield implicitly if the read blocks.
> >
> > What you say here is not relevant to the issue at hand.
>
> I don't know what you think is the issue at hand.  My question was
> about when (or if) the lisp thread could yield during a blocking I/O
> operation.  You appear to have interpreted that question differently
> than I intended, so I attempted to be more explicit about what I meant
> by "the lisp thread" and "yielding".
>
> > > the read operation should either use some temporary buffer and copy
> > > into the lisp buffer, or wait for data to be ready, then reacquire
> > > the GIL before invoking the read syscall with a pointer into the
> > > Lisp buffer object.
> >
> > Yes, and that's exactly where we will lose: most of the heavy
> > processing cannot be done in a separate temporary buffer, because it
> > calls functions from the Lisp machine, and those are written assuming
> > nothing else is running in the Lisp machine concurrently.  For
> > example, take the code which decodes the file's contents we have just
> > read.  I encourage you to take a good look at that code (most of it is
> > in coding.c) to appreciate the magnitude of the problem.
>
> I believe you. I did mention that the global lock could be (evidently
> *must be*) reacquired before the actual call to "read".  There's also
> Tromey's comment on
>
> https://www.reddit.com/r/emacs/comments/utzxir/a_vision_of_a_multithreaded_emacs/
> .
>
> OTOH, I asked the question in order to understand what it means to
> give programmers control over asynchronous execution in controlled
> ways.  The most basic kind of control I can think of is whether
> functions called in that code are expected to behave synchronously or
> asynchronously.  And what is a more basic operation, that could be
> done synchronously or asynchronously, than reading text from a file.
> If something like insert-file-contents can't be performed
> asynchronously (not in parallel, just with other code running while
> waiting for IO), the scope of what Stefan's effort is going to be very
> limited.  Limited to the point of not being very interesting.
>
> It's also possible that something could be built on the existing data
> structures that doesn't rely on more fine-grained locking.  I just
> happened to be reading
> https://www.gnu.org/software/emacs/manual/html_node/emacs/VCS-Merging.html
> today, after seeing this message, and it seemed to me highly relevant
> to the problem of concurrent work on a text buffer.  Particularly the
> comment "Experience has shown that merging is superior to locking".
> Maybe the thing to do to enable asynchronous/concurrent/parallel work
> would be to add a new type of buffer, built on the existing one, that
> behaves something like a git repo/working copy of the text buffer,
> that eventually merges edits, say by windows displaying the buffer
> each pulling updates from all the "repos" with checked-out copies.
> Then the model for support of asynchronous programming could
> encapsulate specifying how to merge and/or handle merge failure.
>
> Maybe that would be too expensive, but at least at first, these
> distributed buffers would only be used by programs using explicit
> asynchronous programming.  Maybe that approach would even be helpful
> in dealing with extremely large files or long lines.  We could call it
> "merge-oriented programming". :-)
>

I forgot to mention the concurrent version of the buffer would need a
functional representation to avoid copying during the merge.  Something
along the lines of Okasaki's purely functional strings, except including
all the other components of buffers - overlays, local variables, and
whatever else would be implicated.  I don't know if this would require a
complete reimplementation of buffers, or if the current implementation
could be tweaked to serve as an underlying component of a zippered buffer.




> > So the code which can run in parallel with another Lisp thread will be
> > able to do only very simple jobs, and will also add overhead due to
> > the need of copying stuff from temporary buffers to Lisp objects.
>
> I'm not talking about running in parallel - there is still just one
> lisp machine in this hypothetical.
>
> > Of course, we could redesign and reimplement this stuff, but that's a
> > lot of non-trivial work.  My assumption was that you are considering
> > relatively lightweight changes on top of the existing code, not a
> > complete redesign of how these primitives work.
>
> I wasn't considering anything.  I asked a very limited question for
> the purpose of giving some hard thought to the language problem Stefan
> requested assistance on.  I didn't offer these elaborations because I
> have any plans, only to respond to your question "How could it?".
> That is a purely hypothetical question.
>
> Lynn
>

[-- Attachment #2: Type: text/html, Size: 8205 bytes --]