"Asynchronous Requests from Emacs Dynamic Modules"

[Akira Kyle <ak@akirakyle.com>]

I'm trying to use Emacs' Dynamic Modules in an asynchronous 
setting where C module functions run in a separate thread from 
Emacs' main thread which is responsible for calling into the 
module functions. The natural problem then is how to notify Emacs 
when one wants to retrieve the result of the asynchronous 
operation from some lisp. Since the module docs explicitly say 
that module functions "may only interact with Emacs from Lisp 
interpreter threads (including the main thread) created by Emacs", 
it is not possible then for a separate module thread to interact 
with Emacs in an unprompted way through the module interface [1].

This blog post [2] has a solution to this problem by sending 
SIGUSR1 to Emacs, but that feels a bit hackish and forces one to 
do more work to manage all the different possible module threads 
and events that one may want to signal to Emacs. I suppose one 
could also conceivably also use Emacs' dbus support but that still 
feels clunky. Maybe its possible it's possible to use Emacs lisp's 
threads [2] to solve this but it's not clear to me how to go about 
doing so and I haven't found anyone that has done so.

Does anyone know of the best way to handle this? Perhaps the 
module interface should be expanded to handle this situation?

[1] 
https://www.gnu.org/software/emacs/manual/html_node/elisp/Module-Functions.html#Module-Functions
[2] 
https://www.gnu.org/software/emacs/manual/html_node/elisp/Threads.html#Threads
[3] https://nullprogram.com/blog/2017/02/14/