Re: User interaction from multiple threads

[John Shahid <jvshahid@gmail.com>]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: John Shahid <jvshahid@gmail.com>
>> Cc: psainty@orcon.net.nz, gazally@runbox.com, emacs-devel@gnu.org
>> Date: Thu, 30 Aug 2018 12:08:32 -0400
>>
>> > I'm afraid there's too much to encapsulate.  E.g., every buffer-local
>> > variable in every buffer to be used by the thread will need to be
>> > encapsulated; and how will the main thread know in advance all that?
>>
>> Why do we need buffer local variables for all the buffers ?
>
> Not all the buffers, only those which the thread function will make
> current at some point.  Surely, you've seen and written code that
> switches buffers as part of doing whatever it needs.

Yeah, but I'm not sure why the portion that is prompting the user should
be switching buffers.  Even in that case the code will look like the
following where all the necessary variables are captured by the closure:

          (on-main-thread
            (read-file-name ...)
            (with-current-buffer other-buffer ;; captured by the closure
              (do-something)
              (read-file-name ..)))


>> Isn't the current buffer during the prompt the only relevant buffer.
>
> No, it is not.  There are also various other parts of the global state
> that should be specific to a thread, like the list of condition-case
> handlers; see the members of 'struct thread_state' as it is defined
> now, for more examples.

I am not sure I totally understand the impact of every field in that
struct has on my proposal.  Speaking of signals, I think those should be
re-emitted on the child thread stack.  For example, the response of the
main thread can indicate the error symbol and data, which can be used in
the child thread to raise/signal again.

>> My idea, is that each thread is started with a pipe which is used to
>> interact with the main thread when it needs to prompt the user.
>
> If you mean literally a pipe, then does this mean you suggest that
> Emacs forks itself and runs each thread in a separate process, talking
> via pipes with other processes?  That's a completely different
> architecture from what we have now, and it wasn't my intent to discuss
> such a complete rewrite of the concurrency code.

No, I am not proposing changing the architecture.  I was proposing to
literally use pipes to communicate between the different threads.  In
fact that doesn't have to be a pipe it could any implementation of a
queue, as was proposed earlier.

> If this is not what you mean, please clarify what you mean by "pipe"
> in this context.
>
>> The child thread write the lambda that needs to run on the main
>> thread then wait for main thread response using
>> `accept-process-output'.  The main thread handles the request in the
>> process filter displaying the prompt and writing the result back to
>> the pipe. Since the child and main thread are sharing the same
>> memory, the communication can simply be the symbol name that holds
>> the closure and the response, respectively.
>
> We already have the async package written by John Wiegley; it sounds
> like you are describing the idea which he already implemented.
>
> The limitation of this is that you need to communicate to much of a
> state to the subordinate Emacs, and some stuff is very hard, sometimes
> impossible, to share this way.

I am proposing something similar but taking advantage of the fact that
threads are sharing the same memory space.  Similar in the sense of
preferring using the existing communication primitives
(e.g. `make-pipe-process' and `accept-process-output').

> In any case, let's stay in the framework of the current basic design
> of how threads work in Emacs.

I sincerely apologize if what I'm being off topic or not helping this
conversation.  This is either due to my inability to explain my thoughts
clearly or my flawed understanding of Emacs internals.  May be trying to
hack together the solution I have in mind will educate me and prove test
the feasibility of this solution.

Thanks,

-js