Re: "Asynchronous Requests from Emacs Dynamic Modules"

[Akira Kyle <ak@akirakyle.com>]

On Sun, Nov 01, 2020 at 11:28 AM, Eli Zaretskii <eliz@gnu.org> 
wrote:

>> It seems the lisp threading functionality is still pretty beta 
>> quality. Have the data races that Chris Wellons mentions here 
>> [1] 
>> ever been addressed?
>
> It's hard to tell, because that blog has no specifics, just a 
> broad
> accusation.  It is also quite old.
>
>> It looks like it's mostly just exposing the pthread interface.
>
> That's simply not true.  It _uses_ pthreads to start and manage
> threads, and to implement mutexes, condvars, etc.  But if you 
> look
> closely at the level of thread.c (_not_ systhread.c), you will 
> see
> something very different from a typical pthreads application.

Sorry I meant not that it directly exposes pthreads but that its 
interface is modeled off of pthreads (i.e. mutexes, condition 
variables), but lisp should really have some higher level 
interface to parallel and concurrent execution (like Python's 
multiprocessing or asyncio interface). Perhaps it can be built on 
top of the current lisp threading interface, but it seems there 
may be some limitations that make that difficult.

>> What is the use case for the current lisp threads given the 
>> severe
>> limitation that only will ever run at a time and the 
>> constraints the
>> cooperative nature puts on when threads can switch?
>
> The main use case is to allow you to write one or more 
> background Lisp
> programs that go about their job while a foreground command lets 
> the
> user interact normally with Emacs.

Right, but due to the global lock this only allows for concurrent 
execution, and due to the cooperative nature needing one to 
explicitly yield execution for threads to switch, it seems like in 
reality it would be very cumbersome to work with. Do you know of 
anyone who uses it? I'd be very interested in seeing some examples 
how to successfully use it.

I think libraries like emacs-aio [1] accomplish the same thing but 
with a much easier to use interface. By using (run-at-time 0 nil 
callback args) they accomplish a sort of "green threading" that 
doesn't actually need any sort of pthread support or the 
associated issues with ensuring atomicity with the interpreter. It 
would be great to have something like this built in.

[1] https://github.com/skeeto/emacs-aio