On Sat, Mar 02, 2024 at 01:53:05AM +0200, Dmitry Gutov wrote:
> On 01/03/2024 21:30, tomas@tuxteam.de wrote:
> > > - Unrelated Lisp thread B is able to take the global lock and run Lisp code
> > >    in parallel with module_work on thread A.
> > > - On thread A, module_work finishes and returns to Lisp.
> > Why has thread A wait up to here? This is what's keeping your thread B
> > from playing, no?
> 
> I imagine thread A will want to continue its execution when the results of
> the "Emacs-independent work" arrive.

In that case, I think your only choice would be to "pass the continuation":
in A, stash away whatever you need to continue, let A die, and when things
"come back", start a thread A' to pick up where A left.

> Said work might look like making a network request (as Spencer outlined),
> getting a response, parsing the received JSON structure (not into Lisp
> objects yet, just into the native data structures provided by the library),
> and potentially filtering the parsed structure as well.
> 
> Then the lock is re-acquired (which will naturally involve some spinning
> waiting for it), and the parsed data is converted into Lisp structures.

I'm always wary when people call for "threads" when they mean concurrency.
The javascript/browser folks do it all the time with explicit continuations
and don't seem to care. It's messier in one way, but when you start locking
and potentially deadlocking, threads have their own ways of messiness, as
the Java folks know all too well (sometimes I have the hunch that those
two messineesses are complementary in some weird way ;-)

As sketched above, you can even combine things (I'd be scared to do, but
there you go).

Real threads, IMO, only really help you in some cases:

 - when you really want more than one CPU core working on your stuff
   (and distributing to several processes is not an option)
 - when you have to use some lower level lib which blocks (and you
   can't afford to take it apart and re-implement it non-blocking)

My experience (this was mostly plain old C), once you've wrapped your
head around continuation style, it is far easier to debug.

Cheers
-- 
t