Re: Releasing the thread global_lock from the module API

[Dmitry Gutov <dmitry@gutov.dev>]

On 02/03/2024 07:57, tomas@tuxteam.de wrote:
> On Sat, Mar 02, 2024 at 01:53:05AM +0200, Dmitry Gutov wrote:
>> On 01/03/2024 21:30, tomas@tuxteam.de wrote:
>>>> - Unrelated Lisp thread B is able to take the global lock and run Lisp code
>>>>     in parallel with module_work on thread A.
>>>> - On thread A, module_work finishes and returns to Lisp.
>>> Why has thread A wait up to here? This is what's keeping your thread B
>>> from playing, no?
>>
>> I imagine thread A will want to continue its execution when the results of
>> the "Emacs-independent work" arrive.
> 
> In that case, I think your only choice would be to "pass the continuation":
> in A, stash away whatever you need to continue, let A die, and when things
> "come back", start a thread A' to pick up where A left.

Almost, except "suspend/yield" instead of "let A die".

And if the Lisp threads are backed by OS threads, such thread is waiting 
without purpose while "suspended" - Spencer's suggestion could put it to 
work in the meantime as well. Though the difference in performance might 
not be very significant, given our overhead in other places.

>> Said work might look like making a network request (as Spencer outlined),
>> getting a response, parsing the received JSON structure (not into Lisp
>> objects yet, just into the native data structures provided by the library),
>> and potentially filtering the parsed structure as well.
>>
>> Then the lock is re-acquired (which will naturally involve some spinning
>> waiting for it), and the parsed data is converted into Lisp structures.
> 
> I'm always wary when people call for "threads" when they mean concurrency.

Yes, our Lisp threads are a bit unusual, terminology-wise.