Re: call-process blocks the active thread

[Tom Tromey <tom@tromey.com>]

Eli> Of course, it's possible I don't see some clever way to make
Eli> call-process thread-aware, which is why I said that if someone has
Eli> concrete ideas for how to do that, they are encouraged to speak up.

I looked at it briefly.  I have two ideas.

Maybe the simplest idea is to rewrite call-process in Lisp, using
make-process and accept-process-output.  This would automatically allow
other threads to run.  I didn't really investigate this in depth (so for
instance I don't know if this can be done without adding extensions to
make-process), but it seems to me that it would be nice to migrate code
out of C when possible.


Otherwise, I think call_process can be made to yield.  A bit of
background (I know Eli doesn't need this but maybe others do):

Only one thread can run Lisp at a time.  A global lock enforces this.
However, threads can run non-Lisp code while not holding the global lock
-- this is why Emacs can wait for input while Lisp runs in another
thread.

The way this is done is that a thread calls flush_stack_call_func to
save registers on the stack (so that GC will do the right thing); this
calls a provided callback, and the callback handles releasing the global
lock, doing whatever non-Lisp thing it wants to do, and then
re-acquiring the lock before returning.

A good example of this is thread.c:thread_select, which winds up in
thread.c:really_call_select.

So, the idea for call-process would be to modify the inner loop of
call_process to do this.  I think the best spot would be to call
flush_stack_call_func around the call to emacs_read_quit.

However, the tricky thing here is that call-process relies on some
global state.  For example, it assumes that current-buffer will not
change while it is working.  So, after emacs_read_quit returns, this
code would have to be careful to restore the state it needs.  The hard
part of the project is figuring out which bits of state must be
preserved.

It might also be reasonable to wrap the wait_for_termination call in a
lock-releasing function.

Tom