Releasing the thread global_lock from the module API

Releasing the thread global_lock from the module API

[Spencer Baugh]

Lisp threads all take global_lock while they are running Lisp code.
This is good and correct.

Lisp threads can also call module code.  This is also good and correct.

In other languages with an FFI and a global interpreter lock taken by
threads, such as Python, it's possible for a native function called from
the interpreter to release the global lock, and then re-acquire it
before calling any interpreter functions (or automatically re-acquire it
upon returning to the interpreter).

This allows for a limited form of actual parallelism: if a native
function is doing work that doesn't involve the Python interpreter,
e.g. doing numerical computations in native code, it can release the
lock so that it can run in parallel with other threads.  If the native
function needs to call a function which does involve the Python
interpreter, it can re-acquire the global lock around that call.

Could the same functionality be added to the Emacs module API?  Then
module code could release the global lock when doing Emacs-independent
computation, and re-acquire the lock when calling into Emacs.  This
would allow a limited form of parallelism: if a Lisp thread is calling
module code which releases the lock, then the module code could run in
parallel with the rest of Emacs on that thread.

This should be safe, because this is in effect already possibly through
thread-yield: that can be called from anywhere in a Lisp thread, and
releases the global lock, calls the operating system thread_yield
function, and then re-acquires the global lock.  If instead of doing
thread_yield, it did some computation, e.g.

int x = 0
for (int i = 0; i<999999; i++)
  x += i;

that would have the same effect as yielding, but the computation would
run in parallel with the main Emacs thread.  This is what would happen
if module code released the lock, did some Emacs-independent work, and
then re-acquired the lock.

As an additional bonus, this would allow the module API to extend Lisp
threads: if a C library provides some blocking function which does some
complicated form of IO, a module providing bindings for that C library
can release global_lock before calling that function, and then
re-acquire the lock after the function returns.  Then Lisp threads
calling this module function will not block the main Emacs thread.

Re: Releasing the thread global_lock from the module API

[Eli Zaretskii]

> From: Spencer Baugh <sbaugh@janestreet.com>
> Cc: Philipp Stephani <phst@google.com>
> Date: Fri, 01 Mar 2024 09:53:41 -0500
> 
> In other languages with an FFI and a global interpreter lock taken by
> threads, such as Python, it's possible for a native function called from
> the interpreter to release the global lock, and then re-acquire it
> before calling any interpreter functions (or automatically re-acquire it
> upon returning to the interpreter).
> 
> This allows for a limited form of actual parallelism: if a native
> function is doing work that doesn't involve the Python interpreter,
> e.g. doing numerical computations in native code, it can release the
> lock so that it can run in parallel with other threads.  If the native
> function needs to call a function which does involve the Python
> interpreter, it can re-acquire the global lock around that call.
> 
> Could the same functionality be added to the Emacs module API?  Then
> module code could release the global lock when doing Emacs-independent
> computation, and re-acquire the lock when calling into Emacs.  This
> would allow a limited form of parallelism: if a Lisp thread is calling
> module code which releases the lock, then the module code could run in
> parallel with the rest of Emacs on that thread.

It isn't clear what exactly do you want added, since calls to
thread-yield and similar APIs (like sleep-for and
accept-process-output), which release and re-acquire the global lock,
are already possible from Lisp threads.  What exactly is missing?

> This should be safe, because this is in effect already possibly through
> thread-yield: that can be called from anywhere in a Lisp thread, and
> releases the global lock, calls the operating system thread_yield
> function, and then re-acquires the global lock.  If instead of doing
> thread_yield, it did some computation, e.g.
> 
> int x = 0
> for (int i = 0; i<999999; i++)
>   x += i;
> 
> that would have the same effect as yielding, but the computation would
> run in parallel with the main Emacs thread.  This is what would happen
> if module code released the lock, did some Emacs-independent work, and
> then re-acquired the lock.

If a module wants to do some Emacs-independent work, it can start a
native thread and do it there, no?  So again, what is missing?

> As an additional bonus, this would allow the module API to extend Lisp
> threads: if a C library provides some blocking function which does some
> complicated form of IO, a module providing bindings for that C library
> can release global_lock before calling that function, and then
> re-acquire the lock after the function returns.  Then Lisp threads
> calling this module function will not block the main Emacs thread.

I think this is already possible, see above.  I guess I'm missing
something.

Re: Releasing the thread global_lock from the module API

[Spencer Baugh]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Spencer Baugh <sbaugh@janestreet.com>
>> Cc: Philipp Stephani <phst@google.com>
>> Date: Fri, 01 Mar 2024 09:53:41 -0500
>> 
>> In other languages with an FFI and a global interpreter lock taken by
>> threads, such as Python, it's possible for a native function called from
>> the interpreter to release the global lock, and then re-acquire it
>> before calling any interpreter functions (or automatically re-acquire it
>> upon returning to the interpreter).
>> 
>> This allows for a limited form of actual parallelism: if a native
>> function is doing work that doesn't involve the Python interpreter,
>> e.g. doing numerical computations in native code, it can release the
>> lock so that it can run in parallel with other threads.  If the native
>> function needs to call a function which does involve the Python
>> interpreter, it can re-acquire the global lock around that call.
>> 
>> Could the same functionality be added to the Emacs module API?  Then
>> module code could release the global lock when doing Emacs-independent
>> computation, and re-acquire the lock when calling into Emacs.  This
>> would allow a limited form of parallelism: if a Lisp thread is calling
>> module code which releases the lock, then the module code could run in
>> parallel with the rest of Emacs on that thread.
>
> It isn't clear what exactly do you want added, since calls to
> thread-yield and similar APIs (like sleep-for and
> accept-process-output), which release and re-acquire the global lock,
> are already possible from Lisp threads.  What exactly is missing?

Those release and re-acquire the global lock, yes, but they don't allow
module code to run on the thread while the global lock is released.

>> This should be safe, because this is in effect already possibly through
>> thread-yield: that can be called from anywhere in a Lisp thread, and
>> releases the global lock, calls the operating system thread_yield
>> function, and then re-acquires the global lock.  If instead of doing
>> thread_yield, it did some computation, e.g.
>> 
>> int x = 0
>> for (int i = 0; i<999999; i++)
>>   x += i;
>> 
>> that would have the same effect as yielding, but the computation would
>> run in parallel with the main Emacs thread.  This is what would happen
>> if module code released the lock, did some Emacs-independent work, and
>> then re-acquired the lock.
>
> If a module wants to do some Emacs-independent work, it can start a
> native thread and do it there, no?  So again, what is missing?

Yes, it's true that a module can provide parallelism in that way.  The
issue is whether Emacs can run in parallel with the module.

Today, we can do:
- Lisp thread A calls module_work
- On thread A, module_work starts native threads X and Y,
  and does Emacs-independent work across all of these threads in parallel.
- On thread A, module_work finishes and returns to Lisp.

All Lisp threads will be blocked while the Lisp thread A is running the
module.

We can't do:
- Lisp thread A calls module_work
- On thread A, module_work releases the global lock.
- On thread A, module_work starts native threads X and Y,
  and does Emacs-independent work across all of these threads in parallel.
- Unrelated Lisp thread B is able to take the global lock and run Lisp code
  in parallel with module_work on thread A.
- On thread A, module_work finishes and returns to Lisp.

Releasing the global lock allows Lisp thread B to run while Lisp thread
A is running the module.

>> As an additional bonus, this would allow the module API to extend Lisp
>> threads: if a C library provides some blocking function which does some
>> complicated form of IO, a module providing bindings for that C library
>> can release global_lock before calling that function, and then
>> re-acquire the lock after the function returns.  Then Lisp threads
>> calling this module function will not block the main Emacs thread.
>
> I think this is already possible, see above.  I guess I'm missing
> something.

I don't think it is.  How would a module release global_lock, call a C
library function on the current thread, and then re-acquire the global
lock?  Can you say how you would do that with the APIs we have today?

Re: Releasing the thread global_lock from the module API

[Eli Zaretskii]

> From: Spencer Baugh <sbaugh@janestreet.com>
> Date: Fri, 01 Mar 2024 12:34:36 -0500
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > It isn't clear what exactly do you want added, since calls to
> > thread-yield and similar APIs (like sleep-for and
> > accept-process-output), which release and re-acquire the global lock,
> > are already possible from Lisp threads.  What exactly is missing?
> 
> Those release and re-acquire the global lock, yes, but they don't allow
> module code to run on the thread while the global lock is released.

That's because the Emacs Lisp thread machinery cannot allow that.  A
Lisp thread has full access to the Emacs global state, so we cannot
allow more than one thread to have that at the same time.

> We can't do:
> - Lisp thread A calls module_work
> - On thread A, module_work releases the global lock.
> - On thread A, module_work starts native threads X and Y,

You can do that if the native threads are started before releasing the
global lock.

>   and does Emacs-independent work across all of these threads in parallel.

What do you mean by "Emacs-independent work across all of these
threads in parallel"?  If you want to break out of the restriction of
having only one Lisp thread running at any given time, then this is
IMO impossible without redesigning and reimplementing the current
threads machinery.

> Releasing the global lock allows Lisp thread B to run while Lisp thread
> A is running the module.

As explained above, you can do that already.

> > I think this is already possible, see above.  I guess I'm missing
> > something.
> 
> I don't think it is.  How would a module release global_lock, call a C
> library function on the current thread, and then re-acquire the global
> lock?  Can you say how you would do that with the APIs we have today?

If a thread releases a global lock, it cannot run the Lisp machine
until it re-acquire the global lock.  So what you want is impossible
within the current threads framework, unless you start native
(non-Lisp) threads which cannot safely access the Emacs global state.

Re: Releasing the thread global_lock from the module API

[Spencer Baugh]

Eli Zaretskii <eliz@gnu.org> writes:
>> > I think this is already possible, see above.  I guess I'm missing
>> > something.
>> 
>> I don't think it is.  How would a module release global_lock, call a C
>> library function on the current thread, and then re-acquire the global
>> lock?  Can you say how you would do that with the APIs we have today?
>
> If a thread releases a global lock, it cannot run the Lisp machine
> until it re-acquire the global lock.

Yes.  But calling a C library function (by some other library not
related to Emacs but used by the module) does not involve calling into
the Lisp machine or any Emacs machinery.  So, because it does not
involve calling into Emacs, we could allow the thread to make that call
without the Emacs global lock which protects Emacs.

Re: Releasing the thread global_lock from the module API

[Eli Zaretskii]

> From: Spencer Baugh <sbaugh@janestreet.com>
> Cc: emacs-devel@gnu.org
> Date: Fri, 01 Mar 2024 14:02:24 -0500
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> >> > I think this is already possible, see above.  I guess I'm missing
> >> > something.
> >> 
> >> I don't think it is.  How would a module release global_lock, call a C
> >> library function on the current thread, and then re-acquire the global
> >> lock?  Can you say how you would do that with the APIs we have today?
> >
> > If a thread releases a global lock, it cannot run the Lisp machine
> > until it re-acquire the global lock.
> 
> Yes.  But calling a C library function (by some other library not
> related to Emacs but used by the module) does not involve calling into
> the Lisp machine or any Emacs machinery.

There's very little you can do usefully this way without touching some
aspect of the Lisp machine.  Assuming that what you do has some
relevance to the Emacs session, of course.

> So, because it does not involve calling into Emacs, we could allow
> the thread to make that call without the Emacs global lock which
> protects Emacs.

Once again: you can do that if you start a native thread before
releasing the global lock.  Why is that not a solution to what you
have in mind?

Re: Releasing the thread global_lock from the module API

[tomas]

[-- Attachment #1: Type: text/plain, Size: 842 bytes --]

On Fri, Mar 01, 2024 at 12:34:36PM -0500, Spencer Baugh wrote:

[...]

> We can't do:
> - Lisp thread A calls module_work
> - On thread A, module_work releases the global lock.
> - On thread A, module_work starts native threads X and Y,
>   and does Emacs-independent work across all of these threads in parallel.

If X and Y are truly independent of the Lisp machinery, I don't
understand what keeps you from sending both their ways and
terminating thread A, thus setting Lisp free for whatever other
thread wants to take over.

> - Unrelated Lisp thread B is able to take the global lock and run Lisp code
>   in parallel with module_work on thread A.
> - On thread A, module_work finishes and returns to Lisp.

Why has thread A wait up to here? This is what's keeping your thread B
from playing, no?

Cheers
-- 
t

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: Releasing the thread global_lock from the module API

[Spencer Baugh]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Spencer Baugh <sbaugh@janestreet.com>
>> Cc: emacs-devel@gnu.org
>> Date: Fri, 01 Mar 2024 14:02:24 -0500
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> >> > I think this is already possible, see above.  I guess I'm missing
>> >> > something.
>> >> 
>> >> I don't think it is.  How would a module release global_lock, call a C
>> >> library function on the current thread, and then re-acquire the global
>> >> lock?  Can you say how you would do that with the APIs we have today?
>> >
>> > If a thread releases a global lock, it cannot run the Lisp machine
>> > until it re-acquire the global lock.
>> 
>> Yes.  But calling a C library function (by some other library not
>> related to Emacs but used by the module) does not involve calling into
>> the Lisp machine or any Emacs machinery.
>
> There's very little you can do usefully this way without touching some
> aspect of the Lisp machine.  Assuming that what you do has some
> relevance to the Emacs session, of course.

Actually, there is lots of useful stuff that can be done this way.

For example, if a C library internally opens a network socket and sends
a request to a network service, that does not touch the Lisp machine and
so can be done this way.

Is that not useful?  Or do these touch the Lisp machine in some way I
don't know about?

Since it's useful for me, I'd like to write a patch which allow modules
to do this; would it be considered?

Re: Releasing the thread global_lock from the module API

[Eli Zaretskii]

> From: Spencer Baugh <sbaugh@janestreet.com>
> Date: Fri, 01 Mar 2024 14:51:01 -0500
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > There's very little you can do usefully this way without touching some
> > aspect of the Lisp machine.  Assuming that what you do has some
> > relevance to the Emacs session, of course.
> 
> Actually, there is lots of useful stuff that can be done this way.
> 
> For example, if a C library internally opens a network socket and sends
> a request to a network service, that does not touch the Lisp machine and
> so can be done this way.
> 
> Is that not useful?  Or do these touch the Lisp machine in some way I
> don't know about?

Where would you take the data for opening the socket? doesn't that
come from some Lisp program or from a Lisp variable?  And how would
you know what kind of request to send? doesn't that come from Lisp as
well?  And what would you do with the stuff you get in response? don't
you want to put this in some Lisp variable or provide as input for
some Lisp program?  Etc. etc.

> Since it's useful for me, I'd like to write a patch which allow modules
> to do this; would it be considered?

Once again: what cannot you do from a native thread started by the
module?  IOW, why would you need access to the global lock machinery
in the first place, if all you want to do is something that is
unrelated to Emacs and its global state?

The code which manipulates the global lock is extremely fragile, so
exposing this to external agents without a very good reason and
without being able to fore some discipline is a bad idea.  And I don't
yet see such a good reason, let alone how to do that in a way that
will preserve the discipline of taking and releasing the lock.  So
please do try to explain better why native threads that are unrelated
to the lock are not the solution for what you have in mind.

Re: Releasing the thread global_lock from the module API

[Spencer Baugh]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Spencer Baugh <sbaugh@janestreet.com>
>> Date: Fri, 01 Mar 2024 14:51:01 -0500
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> > There's very little you can do usefully this way without touching some
>> > aspect of the Lisp machine.  Assuming that what you do has some
>> > relevance to the Emacs session, of course.
>> 
>> Actually, there is lots of useful stuff that can be done this way.
>> 
>> For example, if a C library internally opens a network socket and sends
>> a request to a network service, that does not touch the Lisp machine and
>> so can be done this way.
>> 
>> Is that not useful?  Or do these touch the Lisp machine in some way I
>> don't know about?
>
> Where would you take the data for opening the socket? doesn't that
> come from some Lisp program or from a Lisp variable?  And how would
> you know what kind of request to send? doesn't that come from Lisp as
> well?

Yes: I get those things as arguments from Lisp and then convert them
into the native datastructures of my library, which can be used without
further interacting with the Lisp machine.

Then I would release the lock and call into my library, which does some
useful work which takes a while.

> And what would you do with the stuff you get in response? don't
> you want to put this in some Lisp variable or provide as input for
> some Lisp program?  Etc. etc.

Yes: After I finish the call into my library, I would take the lock
again and call further Lisp functions to put the results back into the
Lisp machine.

>> Since it's useful for me, I'd like to write a patch which allow modules
>> to do this; would it be considered?
>
> Once again: what cannot you do from a native thread started by the
> module?  IOW, why would you need access to the global lock machinery
> in the first place, if all you want to do is something that is
> unrelated to Emacs and its global state?

See above: The call into my library, which takes a while, and is useful,
does not touch the Lisp machine.  But other code around that call does
touch the Lisp machine, and so needs to run with the lock.  The ability
to release and re-acquire the lock means my module doesn't hold the lock
when it doesn't need to.

Re: Releasing the thread global_lock from the module API

[Eli Zaretskii]

> From: Spencer Baugh <sbaugh@janestreet.com>
> Date: Fri, 01 Mar 2024 16:21:03 -0500
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Where would you take the data for opening the socket? doesn't that
> > come from some Lisp program or from a Lisp variable?  And how would
> > you know what kind of request to send? doesn't that come from Lisp as
> > well?
> 
> Yes: I get those things as arguments from Lisp and then convert them
> into the native datastructures of my library, which can be used without
> further interacting with the Lisp machine.
> 
> Then I would release the lock and call into my library, which does some
> useful work which takes a while.

How is this different from starting your own native thread, then
releasing the lock?

> > And what would you do with the stuff you get in response? don't
> > you want to put this in some Lisp variable or provide as input for
> > some Lisp program?  Etc. etc.
> 
> Yes: After I finish the call into my library, I would take the lock
> again and call further Lisp functions to put the results back into the
> Lisp machine.

How is this different from doing the same when your native thread
finishes?

> >> Since it's useful for me, I'd like to write a patch which allow modules
> >> to do this; would it be considered?
> >
> > Once again: what cannot you do from a native thread started by the
> > module?  IOW, why would you need access to the global lock machinery
> > in the first place, if all you want to do is something that is
> > unrelated to Emacs and its global state?
> 
> See above: The call into my library, which takes a while, and is useful,
> does not touch the Lisp machine.  But other code around that call does
> touch the Lisp machine, and so needs to run with the lock.  The ability
> to release and re-acquire the lock means my module doesn't hold the lock
> when it doesn't need to.

You didn't answer my question about doing this from a native thread.

As for re-acquiring the lock: you cannot do that with how the Lisp
threads are currently implemented.  You can only wait for the lock to
be released, and try re-acquiring it; whether you succeed is anyone's
guess, if there are other threads competing for the lock.

Re: Releasing the thread global_lock from the module API

[Spencer Baugh]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Spencer Baugh <sbaugh@janestreet.com>
>> Date: Fri, 01 Mar 2024 16:21:03 -0500
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> > Where would you take the data for opening the socket? doesn't that
>> > come from some Lisp program or from a Lisp variable?  And how would
>> > you know what kind of request to send? doesn't that come from Lisp as
>> > well?
>> 
>> Yes: I get those things as arguments from Lisp and then convert them
>> into the native datastructures of my library, which can be used without
>> further interacting with the Lisp machine.
>> 
>> Then I would release the lock and call into my library, which does some
>> useful work which takes a while.
>
> How is this different from starting your own native thread, then
> releasing the lock?

They are completely different things.

Let me say again what this sub-thread is about:

> If a C library provides some blocking function which does some
> complicated form of IO, a module providing bindings for that C library
> can release global_lock before calling that function, and then
> re-acquire the lock after the function returns.  Then Lisp threads
> calling this module function will not block the main Emacs thread.

So, if I am running the following program in a Lisp thread:

(while (do-emacs-things)
  (let ((input (do-emacs-things)))
    (let ((result (call-into-native-module input)))
      (do-emacs-things result))))

and call-into-native-module unlocks the global lock around the calls
into my library, then during the part of call-into-native-module which
calls into my library, the main Emacs thread will not be blocked and can
run in parallel with call-into-native-module.

>> > And what would you do with the stuff you get in response? don't
>> > you want to put this in some Lisp variable or provide as input for
>> > some Lisp program?  Etc. etc.
>> 
>> Yes: After I finish the call into my library, I would take the lock
>> again and call further Lisp functions to put the results back into the
>> Lisp machine.
>
> How is this different from doing the same when your native thread
> finishes?
>
>> >> Since it's useful for me, I'd like to write a patch which allow modules
>> >> to do this; would it be considered?
>> >
>> > Once again: what cannot you do from a native thread started by the
>> > module?  IOW, why would you need access to the global lock machinery
>> > in the first place, if all you want to do is something that is
>> > unrelated to Emacs and its global state?
>> 
>> See above: The call into my library, which takes a while, and is useful,
>> does not touch the Lisp machine.  But other code around that call does
>> touch the Lisp machine, and so needs to run with the lock.  The ability
>> to release and re-acquire the lock means my module doesn't hold the lock
>> when it doesn't need to.
>
> You didn't answer my question about doing this from a native thread.

I hope my answer above clarifies it.

> As for re-acquiring the lock: you cannot do that with how the Lisp
> threads are currently implemented.  You can only wait for the lock to
> be released, and try re-acquiring it; whether you succeed is anyone's
> guess, if there are other threads competing for the lock.

Yes, that is the normal behavior of locks.  I am well aware of it.  By
"re-acquire", I mean (in your words) "wait for the lock to be released,
and try re-acquiring it".

It is okay if my module has to wait for the main Emacs Lisp thread to
stop running before it gets the global lock again.

Re: Releasing the thread global_lock from the module API

[Dmitry Gutov]

On 01/03/2024 21:30, tomas@tuxteam.de wrote:
>> - Unrelated Lisp thread B is able to take the global lock and run Lisp code
>>    in parallel with module_work on thread A.
>> - On thread A, module_work finishes and returns to Lisp.
> Why has thread A wait up to here? This is what's keeping your thread B
> from playing, no?

I imagine thread A will want to continue its execution when the results 
of the "Emacs-independent work" arrive.

Said work might look like making a network request (as Spencer 
outlined), getting a response, parsing the received JSON structure (not 
into Lisp objects yet, just into the native data structures provided by 
the library), and potentially filtering the parsed structure as well.

Then the lock is re-acquired (which will naturally involve some spinning 
waiting for it), and the parsed data is converted into Lisp structures.

Re: Releasing the thread global_lock from the module API

[tomas]

[-- Attachment #1: Type: text/plain, Size: 2197 bytes --]

On Sat, Mar 02, 2024 at 01:53:05AM +0200, Dmitry Gutov wrote:
> On 01/03/2024 21:30, tomas@tuxteam.de wrote:
> > > - Unrelated Lisp thread B is able to take the global lock and run Lisp code
> > >    in parallel with module_work on thread A.
> > > - On thread A, module_work finishes and returns to Lisp.
> > Why has thread A wait up to here? This is what's keeping your thread B
> > from playing, no?
> 
> I imagine thread A will want to continue its execution when the results of
> the "Emacs-independent work" arrive.

In that case, I think your only choice would be to "pass the continuation":
in A, stash away whatever you need to continue, let A die, and when things
"come back", start a thread A' to pick up where A left.

> Said work might look like making a network request (as Spencer outlined),
> getting a response, parsing the received JSON structure (not into Lisp
> objects yet, just into the native data structures provided by the library),
> and potentially filtering the parsed structure as well.
> 
> Then the lock is re-acquired (which will naturally involve some spinning
> waiting for it), and the parsed data is converted into Lisp structures.

I'm always wary when people call for "threads" when they mean concurrency.
The javascript/browser folks do it all the time with explicit continuations
and don't seem to care. It's messier in one way, but when you start locking
and potentially deadlocking, threads have their own ways of messiness, as
the Java folks know all too well (sometimes I have the hunch that those
two messineesses are complementary in some weird way ;-)

As sketched above, you can even combine things (I'd be scared to do, but
there you go).

Real threads, IMO, only really help you in some cases:

 - when you really want more than one CPU core working on your stuff
   (and distributing to several processes is not an option)
 - when you have to use some lower level lib which blocks (and you
   can't afford to take it apart and re-implement it non-blocking)

My experience (this was mostly plain old C), once you've wrapped your
head around continuation style, it is far easier to debug.

Cheers
-- 
t

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: Releasing the thread global_lock from the module API

[Eli Zaretskii]

> From: Spencer Baugh <sbaugh@janestreet.com>
> Cc: emacs-devel@gnu.org
> Date: Fri, 01 Mar 2024 16:56:55 -0500
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> Then I would release the lock and call into my library, which does some
> >> useful work which takes a while.
> >
> > How is this different from starting your own native thread, then
> > releasing the lock?
> 
> They are completely different things.

I think there's a misunderstanding here, because I cannot see how they
could be "completely different things."  See below.

> > If a C library provides some blocking function which does some
> > complicated form of IO, a module providing bindings for that C library
> > can release global_lock before calling that function, and then
> > re-acquire the lock after the function returns.  Then Lisp threads
> > calling this module function will not block the main Emacs thread.
> 
> So, if I am running the following program in a Lisp thread:
> 
> (while (do-emacs-things)
>   (let ((input (do-emacs-things)))
>     (let ((result (call-into-native-module input)))
>       (do-emacs-things result))))
> 
> and call-into-native-module unlocks the global lock around the calls
> into my library, then during the part of call-into-native-module which
> calls into my library, the main Emacs thread will not be blocked and can
> run in parallel with call-into-native-module.

I'm saying that your call-into-native-module method could do the
following:

  . start a native thread running the call into your library
  . release the global lock by calling thread-yield or sleep-for or
    any other API which yields to other Lisp threads
  . wait for the native thread to finish
  . return when it succeeds to acquire the global lock following the
    completion of the native thread

The _only_ difference between the above and what you described is that
portions of call-into-native-module are run in a separate native
thread, and the Lisp thread which runs the above snippet keeps
yielding until the native thread finishes its job.  How is this
"completely different"?

> > You didn't answer my question about doing this from a native thread.
> 
> I hope my answer above clarifies it.

I think it demonstrates some significant misunderstanding, which I
how I have now clarified.

Re: Releasing the thread global_lock from the module API

[Dmitry Gutov]

On 02/03/2024 07:57, tomas@tuxteam.de wrote:
> On Sat, Mar 02, 2024 at 01:53:05AM +0200, Dmitry Gutov wrote:
>> On 01/03/2024 21:30, tomas@tuxteam.de wrote:
>>>> - Unrelated Lisp thread B is able to take the global lock and run Lisp code
>>>>     in parallel with module_work on thread A.
>>>> - On thread A, module_work finishes and returns to Lisp.
>>> Why has thread A wait up to here? This is what's keeping your thread B
>>> from playing, no?
>>
>> I imagine thread A will want to continue its execution when the results of
>> the "Emacs-independent work" arrive.
> 
> In that case, I think your only choice would be to "pass the continuation":
> in A, stash away whatever you need to continue, let A die, and when things
> "come back", start a thread A' to pick up where A left.

Almost, except "suspend/yield" instead of "let A die".

And if the Lisp threads are backed by OS threads, such thread is waiting 
without purpose while "suspended" - Spencer's suggestion could put it to 
work in the meantime as well. Though the difference in performance might 
not be very significant, given our overhead in other places.

>> Said work might look like making a network request (as Spencer outlined),
>> getting a response, parsing the received JSON structure (not into Lisp
>> objects yet, just into the native data structures provided by the library),
>> and potentially filtering the parsed structure as well.
>>
>> Then the lock is re-acquired (which will naturally involve some spinning
>> waiting for it), and the parsed data is converted into Lisp structures.
> 
> I'm always wary when people call for "threads" when they mean concurrency.

Yes, our Lisp threads are a bit unusual, terminology-wise.

Re: Releasing the thread global_lock from the module API

[tomas]

[-- Attachment #1: Type: text/plain, Size: 1305 bytes --]

On Sat, Mar 02, 2024 at 05:35:26PM +0200, Dmitry Gutov wrote:
> On 02/03/2024 07:57, tomas@tuxteam.de wrote:
> > On Sat, Mar 02, 2024 at 01:53:05AM +0200, Dmitry Gutov wrote:
> > > On 01/03/2024 21:30, tomas@tuxteam.de wrote:
> > > > > - Unrelated Lisp thread B is able to take the global lock and run Lisp code
> > > > >     in parallel with module_work on thread A.
> > > > > - On thread A, module_work finishes and returns to Lisp.
> > > > Why has thread A wait up to here? This is what's keeping your thread B
> > > > from playing, no?
> > > 
> > > I imagine thread A will want to continue its execution when the results of
> > > the "Emacs-independent work" arrive.
> > 
> > In that case, I think your only choice would be to "pass the continuation":
> > in A, stash away whatever you need to continue, let A die, and when things
> > "come back", start a thread A' to pick up where A left.
> 
> Almost, except "suspend/yield" instead of "let A die".

This only if you can let Lisp suspend/yield safely -- i.e. in a way nothing
bad happens if someone else gets a turn at playing the "interpreter" [1].

I don't think this is currently possible.

Cheers

[1] Well, with a compiler and that, you'd call it a "runtime", but the
   intention should be clear.

Cheers
-- 
t

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: Releasing the thread global_lock from the module API

[sbaugh]

Eli Zaretskii <eliz@gnu.org> writes:
>> From: Spencer Baugh <sbaugh@janestreet.com>
>> Cc: emacs-devel@gnu.org
>> Date: Fri, 01 Mar 2024 16:56:55 -0500
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> >> Then I would release the lock and call into my library, which does some
>> >> useful work which takes a while.
>> >
>> > How is this different from starting your own native thread, then
>> > releasing the lock?
>> 
>> They are completely different things.
>
> I think there's a misunderstanding here, because I cannot see how they
> could be "completely different things."  See below.

Oh, yes, you are correct, I didn't understand what you were proposing.
Thank you for your patience, your longer explanation has made your
proposal clear to me now.

>> > If a C library provides some blocking function which does some
>> > complicated form of IO, a module providing bindings for that C library
>> > can release global_lock before calling that function, and then
>> > re-acquire the lock after the function returns.  Then Lisp threads
>> > calling this module function will not block the main Emacs thread.
>> 
>> So, if I am running the following program in a Lisp thread:
>> 
>> (while (do-emacs-things)
>>   (let ((input (do-emacs-things)))
>>     (let ((result (call-into-native-module input)))
>>       (do-emacs-things result))))
>> 
>> and call-into-native-module unlocks the global lock around the calls
>> into my library, then during the part of call-into-native-module which
>> calls into my library, the main Emacs thread will not be blocked and can
>> run in parallel with call-into-native-module.
>
> I'm saying that your call-into-native-module method could do the
> following:
>
>   . start a native thread running the call into your library
>   . release the global lock by calling thread-yield or sleep-for or
>     any other API which yields to other Lisp threads
>   . wait for the native thread to finish
>   . return when it succeeds to acquire the global lock following the
>     completion of the native thread
>
> The _only_ difference between the above and what you described is that
> portions of call-into-native-module are run in a separate native
> thread, and the Lisp thread which runs the above snippet keeps
> yielding until the native thread finishes its job.  How is this
> "completely different"?

Oh, yes, that is similar to what I'm proposing.

Just to summarize, if call_into_native_module looks like:

emacs_value call_into_native_module(emacs_env *env, emacs_value input) {
  native_value native_input = convert_to_native(env, input);
  native_value native_output;

  [...some code...]

  return convert_to_emacs(env, native_output);
}

Then the current state of the world ("hold the lock" model):

  native_output = some_native_function(native_input);

If I understand correctly, you are proposing (the "new thread" model):

  native_thread_handle handle = native_thread_create(some_native_function, native_input);
  while (!native_thread_done(handle)) {
    emacs_thread_yield(env);
  }
  native_output = native_thread_result(handle);

And I am proposing (the "release lock" model):

  release_global_lock(env);
  native_output = some_native_function(native_input);
  acquire_global_lock(env);
  
All three of these are used in the same way from Lisp programs.  But the
"new thread" and "release lock" models have the advantage over the "hold
the lock" model that if called from a Lisp thread, that Lisp thread will
run some_native_function in parallel with Lisp execution on other Lisp
threads, including the main Emacs thread.

To check my understanding: does this all seem correct so far, and match
your proposal?

So, the main difference between the "new thread" model and the "release
lock" model is that creating a native thread takes a nontrivial amount
of time; maybe around 0.1 milliseconds.  If some_native_function would
takes less time than that, the thread creation cost will slow down
Emacs, especially because the native module creates the native thread
while holding the Lisp global_lock.

Reducing the thread creation cost is quite hard; a thread pool can help,
but that is complex and still has substantial cost in communication and
context switching.

So, to avoid the thread creation overhead, a native module should only
create a native thread to run some_native_function if the native module
knows that some_native_function will take a long time.

Unfortunately, most of the time a native module can't know how long
some_native_function will take before calling it.  For example, native
functions which make network calls might return immediately when there's
already data available from the network, but will have to send new
network requests sometimes.  Or, native functions which provide an
in-memory database might be fast or slow depending on the database
contents.

Since the native module doesn't know if some_native_function will take a
long time, the native module needs a way to allow some_native_function
to run in parallel which is cheap, so the native module can do it for
all calls to some_native_function.

The "release lock" model fits this need.  Releasing the lock is
essentially free, and allows some_native_function to run in parallel
with other Lisp threads.  Re-acquiring the lock afterwards will take
time if the operating system switched to other Lisp threads which
acquired the lock, but that's OK: the other Lisp threads are doing
useful work in Lisp in parallel with call_into_native_module waiting for
them to release the lock.

Re: Releasing the thread global_lock from the module API

[Eli Zaretskii]

> From: sbaugh@catern.com
> Date: Sat, 02 Mar 2024 16:39:26 +0000 (UTC)
> Cc: Spencer Baugh <sbaugh@janestreet.com>, emacs-devel@gnu.org
> 
> Oh, yes, that is similar to what I'm proposing.
> 
> Just to summarize, if call_into_native_module looks like:
> 
> emacs_value call_into_native_module(emacs_env *env, emacs_value input) {
>   native_value native_input = convert_to_native(env, input);
>   native_value native_output;
> 
>   [...some code...]
> 
>   return convert_to_emacs(env, native_output);
> }
> 
> Then the current state of the world ("hold the lock" model):
> 
>   native_output = some_native_function(native_input);
> 
> If I understand correctly, you are proposing (the "new thread" model):
> 
>   native_thread_handle handle = native_thread_create(some_native_function, native_input);
>   while (!native_thread_done(handle)) {
>     emacs_thread_yield(env);
>   }
>   native_output = native_thread_result(handle);
> 
> And I am proposing (the "release lock" model):
> 
>   release_global_lock(env);
>   native_output = some_native_function(native_input);
>   acquire_global_lock(env);
>   
> All three of these are used in the same way from Lisp programs.  But the
> "new thread" and "release lock" models have the advantage over the "hold
> the lock" model that if called from a Lisp thread, that Lisp thread will
> run some_native_function in parallel with Lisp execution on other Lisp
> threads, including the main Emacs thread.
> 
> To check my understanding: does this all seem correct so far, and match
> your proposal?

Yes.

> So, the main difference between the "new thread" model and the "release
> lock" model is that creating a native thread takes a nontrivial amount
> of time; maybe around 0.1 milliseconds.  If some_native_function would
> takes less time than that, the thread creation cost will slow down
> Emacs, especially because the native module creates the native thread
> while holding the Lisp global_lock.

Why are you worried by 0.1 msec slowdown (if it indeed takes that
long; I think it should be around 10 to 20 usec at most)?  If this
kind of slowdown is important for you, you are using the wrong
software package (and probably the wrong OS as well).

> The "release lock" model fits this need.

But it exposes the sensitive internals and runs the risk of more than
one Lisp thread running at the same time, and thus is not acceptable.
I'm afraid we will have to live with the 0.1 msec overhead.

> Releasing the lock is essentially free

Nothing is ever free in the world of synchronization primitives.  But
that's a btw, not the most important aspect of this.

Re: Releasing the thread global_lock from the module API

[Spencer Baugh]

Eli Zaretskii <eliz@gnu.org> writes:
>> From: sbaugh@catern.com
>> Date: Sat, 02 Mar 2024 16:39:26 +0000 (UTC)
>> Cc: Spencer Baugh <sbaugh@janestreet.com>, emacs-devel@gnu.org
>> 
>> Oh, yes, that is similar to what I'm proposing.
>> 
>> Just to summarize, if call_into_native_module looks like:
>> 
>> emacs_value call_into_native_module(emacs_env *env, emacs_value input) {
>>   native_value native_input = convert_to_native(env, input);
>>   native_value native_output;
>> 
>>   [...some code...]
>> 
>>   return convert_to_emacs(env, native_output);
>> }
>> 
>> Then the current state of the world ("hold the lock" model):
>> 
>>   native_output = some_native_function(native_input);
>> 
>> If I understand correctly, you are proposing (the "new thread" model):
>> 
>>   native_thread_handle handle = native_thread_create(some_native_function, native_input);
>>   while (!native_thread_done(handle)) {
>>     emacs_thread_yield(env);
>>   }
>>   native_output = native_thread_result(handle);
>> 
>> And I am proposing (the "release lock" model):
>> 
>>   release_global_lock(env);
>>   native_output = some_native_function(native_input);
>>   acquire_global_lock(env);
>>   
>> All three of these are used in the same way from Lisp programs.  But the
>> "new thread" and "release lock" models have the advantage over the "hold
>> the lock" model that if called from a Lisp thread, that Lisp thread will
>> run some_native_function in parallel with Lisp execution on other Lisp
>> threads, including the main Emacs thread.
>> 
>> To check my understanding: does this all seem correct so far, and match
>> your proposal?
>
> Yes.
>
>> So, the main difference between the "new thread" model and the "release
>> lock" model is that creating a native thread takes a nontrivial amount
>> of time; maybe around 0.1 milliseconds.  If some_native_function would
>> takes less time than that, the thread creation cost will slow down
>> Emacs, especially because the native module creates the native thread
>> while holding the Lisp global_lock.
>
> Why are you worried by 0.1 msec slowdown (if it indeed takes that
> long; I think it should be around 10 to 20 usec at most)?  If this
> kind of slowdown is important for you, you are using the wrong
> software package (and probably the wrong OS as well).

Because a Lisp program that uses a native module might make thousands of
module calls.  This is fine when each call takes a microsecond.  If we
add, for example, 500 microseconds of overhead to each module call, then
1000 module calls will take half a second.

For example: I have a project.el backend which uses a native module.
Looking up the project for the current directory and then getting the
name of the project makes around 5 module calls.  I have around 200
projects.  That works out to 1000 module calls to get the names of all
my projects.  Currently with the native module backend this takes around
a millisecond.  With 500 extra microseconds per call, it will take half
a second.

>> The "release lock" model fits this need.
>
> But it exposes the sensitive internals and runs the risk of more than
> one Lisp thread running at the same time, and thus is not acceptable.

Yes.  But of course in practice we would find a design which allows
releasing the lock but is hard to misuse.

How about this:

   native_output = env->call_without_lock(some_native_function, native_input);

call_without_lock would be a function which releases the global lock,
calls a specified function, then acquires the global lock again.

That seems hard to misuse.  It is about equivalent to:

   native_output = run_in_native_thread(some_native_function, native_input);

which is possible today for module programmers.  Just, call_without_lock
would be much faster.

Re: Releasing the thread global_lock from the module API

[sbaugh]

tomas@tuxteam.de writes:
> On Sat, Mar 02, 2024 at 05:35:26PM +0200, Dmitry Gutov wrote:
>> On 02/03/2024 07:57, tomas@tuxteam.de wrote:
>> > On Sat, Mar 02, 2024 at 01:53:05AM +0200, Dmitry Gutov wrote:
>> > > On 01/03/2024 21:30, tomas@tuxteam.de wrote:
>> > > > > - Unrelated Lisp thread B is able to take the global lock and run Lisp code
>> > > > >     in parallel with module_work on thread A.
>> > > > > - On thread A, module_work finishes and returns to Lisp.
>> > > > Why has thread A wait up to here? This is what's keeping your thread B
>> > > > from playing, no?
>> > > 
>> > > I imagine thread A will want to continue its execution when the results of
>> > > the "Emacs-independent work" arrive.
>> > 
>> > In that case, I think your only choice would be to "pass the continuation":
>> > in A, stash away whatever you need to continue, let A die, and when things
>> > "come back", start a thread A' to pick up where A left.
>> 
>> Almost, except "suspend/yield" instead of "let A die".
>
> This only if you can let Lisp suspend/yield safely -- i.e. in a way nothing
> bad happens if someone else gets a turn at playing the "interpreter" [1].
>
> I don't think this is currently possible.

It is currently possible: this is what thread-yield, sleep-for, and
accept-process-output do when run from a Lisp thread.  The thread pauses
execution and other threads execute.  This has worked since the
introduction of thread support.

Re: Releasing the thread global_lock from the module API

[Eli Zaretskii]

> From: Spencer Baugh <sbaugh@catern.com>
> Date: Sat, 02 Mar 2024 20:33:05 +0000 (UTC)
> Cc: sbaugh@janestreet.com, emacs-devel@gnu.org
> 
> >> So, the main difference between the "new thread" model and the "release
> >> lock" model is that creating a native thread takes a nontrivial amount
> >> of time; maybe around 0.1 milliseconds.  If some_native_function would
> >> takes less time than that, the thread creation cost will slow down
> >> Emacs, especially because the native module creates the native thread
> >> while holding the Lisp global_lock.
> >
> > Why are you worried by 0.1 msec slowdown (if it indeed takes that
> > long; I think it should be around 10 to 20 usec at most)?  If this
> > kind of slowdown is important for you, you are using the wrong
> > software package (and probably the wrong OS as well).
> 
> Because a Lisp program that uses a native module might make thousands of
> module calls.  This is fine when each call takes a microsecond.  If we
> add, for example, 500 microseconds of overhead to each module call, then
> 1000 module calls will take half a second.
> 
> For example: I have a project.el backend which uses a native module.
> Looking up the project for the current directory and then getting the
> name of the project makes around 5 module calls.  I have around 200
> projects.  That works out to 1000 module calls to get the names of all
> my projects.  Currently with the native module backend this takes around
> a millisecond.  With 500 extra microseconds per call, it will take half
> a second.

A module whose call takes this little time to complete and whose calls
from Lisp are so massive should not, and need not, use this technique,
because releasing the global lock for such a short time will not
benefit anything.  Releasing a global lock is only beneficial if a
module call does a lot of work, during which other Lisp threads could
do something useful.  but if a module call takes a significant time
(say, a few seconds), then adding a 20 usecs to that time is
insignificant.

> >> The "release lock" model fits this need.
> >
> > But it exposes the sensitive internals and runs the risk of more than
> > one Lisp thread running at the same time, and thus is not acceptable.
> 
> Yes.  But of course in practice we would find a design which allows
> releasing the lock but is hard to misuse.

Sorry, I don't believe this is reliably possible, or safe enough to
justify exposing the lock to Lisp.  Certainly not to modules, which is
an open-ended gateway into the bowels of the Emacs Lisp machine.

> How about this:
> 
>    native_output = env->call_without_lock(some_native_function, native_input);
> 
> call_without_lock would be a function which releases the global lock,
> calls a specified function, then acquires the global lock again.

Sorry, I don't agree to adding such interfaces to Emacs.  And if you
are still not convinced, let's agree to disagree.

Re: Releasing the thread global_lock from the module API

[tomas]

[-- Attachment #1: Type: text/plain, Size: 481 bytes --]

On Sat, Mar 02, 2024 at 09:41:38PM +0000, sbaugh@catern.com wrote:
> tomas@tuxteam.de writes:

[safe suspend wrt. concurrency]

> > I don't think this is currently possible.
> 
> It is currently possible: this is what thread-yield, sleep-for, and
> accept-process-output do when run from a Lisp thread.  The thread pauses
> execution and other threads execute.  This has worked since the
> introduction of thread support.

Thanks for the correction.

Cheers
-- 
t

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: Releasing the thread global_lock from the module API

[sbaugh]

Eli Zaretskii <eliz@gnu.org> writes:
>> From: Spencer Baugh <sbaugh@catern.com>
>> Date: Sat, 02 Mar 2024 20:33:05 +0000 (UTC)
>> Cc: sbaugh@janestreet.com, emacs-devel@gnu.org
>> 
>> >> So, the main difference between the "new thread" model and the "release
>> >> lock" model is that creating a native thread takes a nontrivial amount
>> >> of time; maybe around 0.1 milliseconds.  If some_native_function would
>> >> takes less time than that, the thread creation cost will slow down
>> >> Emacs, especially because the native module creates the native thread
>> >> while holding the Lisp global_lock.
>> >
>> > Why are you worried by 0.1 msec slowdown (if it indeed takes that
>> > long; I think it should be around 10 to 20 usec at most)?  If this
>> > kind of slowdown is important for you, you are using the wrong
>> > software package (and probably the wrong OS as well).
>> 
>> Because a Lisp program that uses a native module might make thousands of
>> module calls.  This is fine when each call takes a microsecond.  If we
>> add, for example, 500 microseconds of overhead to each module call, then
>> 1000 module calls will take half a second.
>> 
>> For example: I have a project.el backend which uses a native module.
>> Looking up the project for the current directory and then getting the
>> name of the project makes around 5 module calls.  I have around 200
>> projects.  That works out to 1000 module calls to get the names of all
>> my projects.  Currently with the native module backend this takes around
>> a millisecond.  With 500 extra microseconds per call, it will take half
>> a second.
>
> A module whose call takes this little time to complete and whose calls
> from Lisp are so massive should not, and need not, use this technique,
> because releasing the global lock for such a short time will not
> benefit anything.  Releasing a global lock is only beneficial if a
> module call does a lot of work, during which other Lisp threads could
> do something useful.  but if a module call takes a significant time
> (say, a few seconds), then adding a 20 usecs to that time is
> insignificant.

Yes, to avoid the thread creation overhead, a native module should only
create a native thread to run some_native_function if the native module
knows that some_native_function will take a long time.

Unfortunately, most of the time a native module can't know how long
some_native_function will take before calling it.  For example, native
functions which make network calls might return immediately when there's
already data available from the network, but will have to send new
network requests sometimes.  Or, native functions which provide an
in-memory database might be fast or slow depending on the database
contents.

Since the native module doesn't know if some_native_function will take a
long time, the native module needs a way to allow some_native_function
to run in parallel which is cheap, so the native module can do it for
all calls to some_native_function.

>> >> The "release lock" model fits this need.
>> >
>> > But it exposes the sensitive internals and runs the risk of more than
>> > one Lisp thread running at the same time, and thus is not acceptable.
>> 
>> Yes.  But of course in practice we would find a design which allows
>> releasing the lock but is hard to misuse.
>
> Sorry, I don't believe this is reliably possible, or safe enough to
> justify exposing the lock to Lisp.  Certainly not to modules, which is
> an open-ended gateway into the bowels of the Emacs Lisp machine.
>
>> How about this:
>> 
>>    native_output = env->call_without_lock(some_native_function, native_input);
>> 
>> call_without_lock would be a function which releases the global lock,
>> calls a specified function, then acquires the global lock again.
>
> Sorry, I don't agree to adding such interfaces to Emacs.  And if you
> are still not convinced, let's agree to disagree.

That is understandable, but I think you are not yet appreciating that
this can be a very useful way to introduce parallelism with high
performance.  And that env->call_without_lock is no less safe than the
method you suggested using.

Is there anything that would convince you of such things?

Re: Releasing the thread global_lock from the module API

[Dmitry Gutov]

On 03/03/2024 15:19, sbaugh@catern.com wrote:
>>> How about this:
>>>
>>>     native_output = env->call_without_lock(some_native_function, native_input);
>>>
>>> call_without_lock would be a function which releases the global lock,
>>> calls a specified function, then acquires the global lock again.
>> Sorry, I don't agree to adding such interfaces to Emacs.  And if you
>> are still not convinced, let's agree to disagree.
> That is understandable, but I think you are not yet appreciating that
> this can be a very useful way to introduce parallelism with high
> performance.  And that env->call_without_lock is no less safe than the
> method you suggested using.
> 
> Is there anything that would convince you of such things?

Perhaps some data from benchmarking real-life code?

For example, you could take the route proposed by Eli, implement your 
intended workload, make some measurements, and then try to calculate 
whether the native thread creation overhead makes a noticeable difference.

Re: Releasing the thread global_lock from the module API

[Eli Zaretskii]

> From: sbaugh@catern.com
> Date: Sun, 03 Mar 2024 13:19:04 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> >
> > A module whose call takes this little time to complete and whose calls
> > from Lisp are so massive should not, and need not, use this technique,
> > because releasing the global lock for such a short time will not
> > benefit anything.  Releasing a global lock is only beneficial if a
> > module call does a lot of work, during which other Lisp threads could
> > do something useful.  but if a module call takes a significant time
> > (say, a few seconds), then adding a 20 usecs to that time is
> > insignificant.
> 
> Yes, to avoid the thread creation overhead, a native module should only
> create a native thread to run some_native_function if the native module
> knows that some_native_function will take a long time.
> 
> Unfortunately, most of the time a native module can't know how long
> some_native_function will take before calling it.  For example, native
> functions which make network calls might return immediately when there's
> already data available from the network, but will have to send new
> network requests sometimes.  Or, native functions which provide an
> in-memory database might be fast or slow depending on the database
> contents.

I believe the cases where it is not known in advance whether a module
call can take only a few tens or hundreds of microseconds or longer
are very rare.  And if the timing depends on some internally
maintained data, such as the size of a DB, the decision can even be
made dynamically, at least in some cases (although I'd consider that a
premature optimization).

> Since the native module doesn't know if some_native_function will take a
> long time

I think in most cases it can and does know.  In general, only rare
module calls will take such a long time that releasing the lock will
be justified.  Module calls that are fast enough shouldn't bother.

> > Sorry, I don't agree to adding such interfaces to Emacs.  And if you
> > are still not convinced, let's agree to disagree.
> 
> That is understandable, but I think you are not yet appreciating that
> this can be a very useful way to introduce parallelism with high
> performance.

I think I do appreciate the odds.  More importantly, I have a lot of
gray hair from fixing various problems with our threads.  I also don't
believe Emacs is a platform suitable for high-performance parallelism.

> Is there anything that would convince you of such things?

Unlikely.  Maybe a complete redesign of how threads are switched in
Emacs.  I don't think you have any idea how fragile the machinery we
have now really is.