Re: "Asynchronous Requests from Emacs Dynamic Modules"

[Eli Zaretskii <eliz@gnu.org>]

> From: Akira Kyle <ak@akirakyle.com>
> Cc: yyoncho <yyoncho@gmail.com>, all_but_last@163.com, emacs-devel@gnu.org,
>  Stefan Monnier <monnier@iro.umontreal.ca>, Philipp Stephani
>  <p.stephani2@gmail.com>
> Date: Sat, 31 Oct 2020 13:25:58 -0600
> 
> 1) Chris Wellon's solution in the article I linked using the 
> SIGUSR1 signal which Emacs handles in the event queue notify the 
> running Lisp Thread that it should call back into module 
> functions. This idea could be expanded to allow modules to define 
> their own events and lisp handlers that will respond to them.

A terrible design, if you ask me.  Reminds me how timers worked in
Emacs long ago: we had an external program which would periodically
deliver a signal to Emacs.  We switched off of that to what we have
now, for good reasons.

And please keep in mind that doing anything non-trivial from a signal
handler is inherently unsafe.

> 2) Stefan's and Philipp's proposal of using the process interface 
> with file descriptors to pipes and perhaps `process-filter` to 
> notify the Lisp Thread that it should call back into the module 
> functions.

This is the way to go, IMO.  This is how Emacs itself handles async
events, so the infrastructure for doing this, both in C and in Lisp,
already exists, and is mature.  All you need is use it.

> 3) yyoncho's proposal to use the lisp threading interface, 
> specifically condition variables to allow dynamic modules to 
> `condition-notify`.

This is unworkable within the current design of Lisp threads.
condition-notify releases the global lock, something that under the
current design cannot be done at an arbitrary time, because it could
cause two threads run in parallel.

> I think the most natural solution would be a dynamic module
> interface that allows grabbing and releasing Lisp's global thread
> lock.

I encourage you to study how threading works in Emacs (the code is in
thread.c).  I'm sure you will see right away why this cannot be done
without a complete redesign of how switching threads works.  There's
no way of interrupting a running Lisp thread in an arbitrary place and
switching to another thread.