Threading IO-bound functions

Threading IO-bound functions

[Elias Mårtenson]

[-- Attachment #1: Type: text/plain, Size: 1335 bytes --]

The number one function that I call that sometimes hang for a significant
amount of time is ‘gnus’. I decided to try running it in a thread, and it
worked surprisingly well. Initial loading of the messages can now be done
in the background.

To prevent myself from running this function more than once at the same
time, I created a wrapper function for this, and I have extracted it into a
macro.

I'd like to have people's opinions on this strategy, and if it might be
reasonable to default ‘gnus’ to do this when run on Emacs versions with
concurrency support.

(defmacro define-background-function-wrapper (bg-function fn)
  (let ((is-loading-sym (intern (concat "*" (symbol-name bg-function)
"-is-loading*"))))
    `(progn
       (defvar ,is-loading-sym nil)
       (defun ,bg-function ()
         (interactive)
         (when ,is-loading-sym
           (message ,(concat (symbol-name fn) " is already loading")))
         (setq ,is-loading-sym t)
         (make-thread (lambda ()
                        (unwind-protect
                            (,fn)
                          (setq ,is-loading-sym nil))))))))

It's invoked like ‘(define-background-function-wrapper bg-gnus gnus)’. This
will then define the function ‘bg-gnus’ that runs ‘gnus‘ in a thread.

Regards,
Elias

[-- Attachment #2: Type: text/html, Size: 2277 bytes --]

Re: Threading IO-bound functions

[Eli Zaretskii]

> From: Elias Mårtenson <lokedhs@gmail.com>
> Date: Thu, 15 Dec 2016 12:01:51 +0800
> 
> The number one function that I call that sometimes hang for a significant amount of time is ‘gnus’. I decided to
> try running it in a thread, and it worked surprisingly well. Initial loading of the messages can now be done in the
> background.

Beginner's luck ;-)

Did you try this in "emacs -nw"?  And without your Gnus
customizations, which allow some shortcuts?  Half way through its
initialization, Gnus asks a question, and then you get bug#25214.

> To prevent myself from running this function more than once at the same time, I created a wrapper function
> for this, and I have extracted it into a macro.

Thanks.

> I'd like to have people's opinions on this strategy, and if it might be reasonable to default ‘gnus’ to do this when
> run on Emacs versions with concurrency support.

I think we need first to solve the above problem in some way.  Or
change Gnus to not ask any questions from a background thread.

Re: Threading IO-bound functions

[Elias Mårtenson]

[-- Attachment #1: Type: text/plain, Size: 1915 bytes --]

On 16 December 2016 at 23:26, Eli Zaretskii <eliz@gnu.org> wrote:

> > From: Elias Mårtenson <lokedhs@gmail.com>
> > Date: Thu, 15 Dec 2016 12:01:51 +0800
> >
> > The number one function that I call that sometimes hang for a
> significant amount of time is ‘gnus’. I decided to
> > try running it in a thread, and it worked surprisingly well. Initial
> loading of the messages can now be done in the
> > background.
>
> Beginner's luck ;-)
>

Oh for sure. I was also careful not to do anything that may upset it (like
running other gnus-functions at the same time).

The fact that my macro implements one lock per function only makes it
usable for single-operation functions. Gnus should probably a global lock
that ensures that only one Gnus function at a time is called. This issue
can't be limited to just Gnus, so perhaps Emacs should provide some
higher-level facility to implement this stuff.


> Did you try this in "emacs -nw"?  And without your Gnus
> customizations, which allow some shortcuts?  Half way through its
> initialization, Gnus asks a question, and then you get bug#25214.
>

I was using the GTK Emacs, and yes, if it asks a question, the entire Emacs
crashes is the most bizarre way.

My workaround is simply making sure that it doesn't interact with the user.


> > I'd like to have people's opinions on this strategy, and if it might be
> reasonable to default ‘gnus’ to do this when
> > run on Emacs versions with concurrency support.
>
> I think we need first to solve the above problem in some way.  Or
> change Gnus to not ask any questions from a background thread.
>

I think most questions are asked using ‘read-string’ or some variant like
‘completing-read’, right? At the very least, I think that concurrent Emacs
needs to make sure that the behaviour of these functions are well-defined
in the context of threading.

[-- Attachment #2: Type: text/html, Size: 2779 bytes --]

Re: Threading IO-bound functions

[Eli Zaretskii]

> From: Elias Mårtenson <lokedhs@gmail.com>
> Date: Mon, 19 Dec 2016 11:03:32 +0800
> Cc: emacs-devel <emacs-devel@gnu.org>
> 
>  I think we need first to solve the above problem in some way. Or
>  change Gnus to not ask any questions from a background thread.
> 
> I think most questions are asked using ‘read-string’ or some variant like ‘completing-read’, right? At the very
> least, I think that concurrent Emacs needs to make sure that the behaviour of these functions are well-defined
> in the context of threading.

I asked for thoughts and ideas about this in bug#25214.  So far no
replies, which I find unfortunate and, frankly, surprising.

Thanks.

Re: Threading IO-bound functions

[Elias Mårtenson]

[-- Attachment #1: Type: text/plain, Size: 1696 bytes --]

On 19 December 2016 at 11:39, Eli Zaretskii <eliz@gnu.org> wrote:

> > From: Elias Mårtenson <lokedhs@gmail.com>
> > Date: Mon, 19 Dec 2016 11:03:32 +0800
> > Cc: emacs-devel <emacs-devel@gnu.org>
> >
> >  I think we need first to solve the above problem in some way. Or
> >  change Gnus to not ask any questions from a background thread.
> >
> > I think most questions are asked using ‘read-string’ or some variant
> like ‘completing-read’, right? At the very
> > least, I think that concurrent Emacs needs to make sure that the
> behaviour of these functions are well-defined
> > in the context of threading.
>
> I asked for thoughts and ideas about this in bug#25214.  So far no
> replies, which I find unfortunate and, frankly, surprising.
>

I thought that I had been reading all the concurrency-related threads, but
I don't recall seeing this. If it was only mentioned in a bug report it
doesn't surprise me that not so many people see it.

Now, my thoughts on this is that keyboard entry is an inherently
single-threaded operation *from the user's point of view* and that the
Emacs platform should enforce this. Thus, keyboard input should only be
allowed from the main thread.

Naturally, a threaded function may want to invoke ‘or-or-n-p’ or
‘read-string’ or other similar things. This should be implemented using a
queue where the thread places an input request on the queue and waits for
the main thread to reply with the result. The main thread would see the
request on the queue and invoke the appropriate
keyboard-interaction-function at the appropriate time.

At least that's how I would design it. Thoughts?

Regards,
Elias

[-- Attachment #2: Type: text/html, Size: 2269 bytes --]

Re: Threading IO-bound functions

[Eli Zaretskii]

> From: Elias Mårtenson <lokedhs@gmail.com>
> Date: Mon, 19 Dec 2016 15:06:32 +0800
> Cc: emacs-devel <emacs-devel@gnu.org>
> 
>  I asked for thoughts and ideas about this in bug#25214. So far no
>  replies, which I find unfortunate and, frankly, surprising.
> 
> I thought that I had been reading all the concurrency-related threads, but I don't recall seeing this. If it was only
> mentioned in a bug report it doesn't surprise me that not so many people see it.

All the people whom I expected to respond are either subscribed to the
bug list or were explicitly CC'ed  ;-)

> Now, my thoughts on this is that keyboard entry is an inherently single-threaded operation from the user's
> point of view and that the Emacs platform should enforce this. Thus, keyboard input should only be allowed
> from the main thread.

That is not the only possibility: we could also serialize minibuffer
input.  (Note that keyboard input per se is not an issue: the
implementation of threads already lets just one thread at a time wait
on keyboard input.)

> Naturally, a threaded function may want to invoke ‘or-or-n-p’ or ‘read-string’ or other similar things. This should
> be implemented using a queue where the thread places an input request on the queue and waits for the main
> thread to reply with the result. The main thread would see the request on the queue and invoke the appropriate
> keyboard-interaction-function at the appropriate time.

The general idea is almost a no-brainer, but the devil is in the
details.  I'm afraid asking another thread to do minibuffer input
might only work on a very low level, because of the thread-specific
bindings and other thread-specific stuff.  So any Lisp code or code on
similar levels (e.g., C code that is "just below" Lisp) cannot be run
"on behalf of another thread", as it will not generally DTRT.  And
that contradicts your idea, I think, because it presumes putting some
high-level Lisp objects on those queues.

> From: Clément Pit--Claudel <clement.pit@gmail.com>
> Date: Sun, 18 Dec 2016 23:31:28 -0500
> 
> Alternatively, could we simply disallow this type of interaction until we have a good story about it? That would bring us closer to the way web workers work in Javascript (they need to post a message to interact with the UI/DOM).

How exactly do we "disallow" that?  Let's keep in mind that Emacs
generally does UI very casually, and many times out of control of the
application-level code.  For example, modifying a buffer might pop up
the "supersession threat" prompt without any prior notice.  What would
"disallow" mean in that case? always answer a standard answer
automatically? signal an error?  How can an application protect itself
against something like that, when it's hard to know in advance it will
happen?  We don't want to require that code that can be run from
non-primary threads be written using some special very restrictive
techniques, at least not without trying to find better solutions.

The event handling thread (was: Threading IO-bound functions)

[John Wiegley]

>>>>> "EM" == Elias Mårtenson <lokedhs@gmail.com> writes:

EM> Now, my thoughts on this is that keyboard entry is an inherently
EM> single-threaded operation *from the user's point of view* and that the
EM> Emacs platform should enforce this. Thus, keyboard input should only be
EM> allowed from the main thread.

I too had thought keyboard events would be managed by an IO queueing thread,
and dispatched to other threads when relevant (i.e., when the receipt of IO
preempted another thread that was "awaiting input").

That is, I never thought event handling would be entirely cooperative, since
the user invoking a command should suspend what Emacs is doing, unless what
Emacs was doing was awaiting input.

Since there's some confusion on this point -- and maybe what I've stated above
doesn't make sense, since it could impose concurrency in places we've not
thought about yet -- I'd like us to discuss this a bit more.

-- 
John Wiegley                  GPG fingerprint = 4710 CF98 AF9B 327B B80F
http://newartisans.com                          60E1 46C4 BD1A 7AC1 4BA2

Re: The event handling thread (was: Threading IO-bound functions)

[Eli Zaretskii]

> From: John Wiegley <jwiegley@gmail.com>
> Date: Mon, 19 Dec 2016 09:43:30 -0800
> Cc: Eli Zaretskii <eliz@gnu.org>, emacs-devel <emacs-devel@gnu.org>
> 
> EM> Now, my thoughts on this is that keyboard entry is an inherently
> EM> single-threaded operation *from the user's point of view* and that the
> EM> Emacs platform should enforce this. Thus, keyboard input should only be
> EM> allowed from the main thread.
> 
> I too had thought keyboard events would be managed by an IO queueing thread,
> and dispatched to other threads when relevant (i.e., when the receipt of IO
> preempted another thread that was "awaiting input").

As I explained here, the de-facto "IO queueing thread" is the "primary
thread", the one started when Emacs starts.  This is by virtue of the
current design, whereby only one thread is ever allowed to wait on
each handle through which Emacs communicates with the external world;
that includes the keyboard handle.

This works by marking each handle with the thread that waits for it.
When a thread calls one of the APIs that need input from external
sources, such as accept-process-output, or when it is about to become
idle, the code in wait_reading_process_output generates the descriptor
sets to be passed to 'pselect', and excludes from the set any
descriptors that are already marked with some other thread waiting on
them.

Since the primary thread is the only one at session genesis, it finds
the descriptor for the keyboard input free and "up for grabs", so it
marks the keyboard descriptor with its thread ID when it calls
'pselect'.  Other threads can only run when the primary thread calls
'pselect', so they all find the keyboard descriptor already "owned" by
the primary thread.

That "ownership" will only be lifted when the primary thread exits
'pselect', acquires the global lock, and starts running again, at
which time all the other threads are either in their 'pselect' call or
wait for the global lock to become unlocked.  The primary thread will
run until it again enters wait_reading_process_output, at which time
it will find the keyboard descriptor either owned by itself or
unowned, and will mark it as owned by itself.  So the other threads
will again see it as owned by the primary thread.

This pattern will repeat itself forever, so I see no way for any
non-primary thread to grab the keyboard descriptor and wait on it,
unless we add something that would provide a means for them to do so,
or until the primary thread runs some Lisp that calls thread-yield or
some other synchronization primitive.

To see the above machinery in action, try this:

  (defun infloop ()
    (with-temp-buffer (while t (insert "foo"))))

  (make-thread #'infloop "thread-loop")

As soon as you start the looping thread, any keyboard input, like M-x
or cursor motion commands, doesn't have any effect, until you type C-g
(which causes the looping thread to exit), because the looping thread
never yields.

> That is, I never thought event handling would be entirely cooperative, since
> the user invoking a command should suspend what Emacs is doing, unless what
> Emacs was doing was awaiting input.

I'm not sure I understand what you are saying here.  Remember: only
one thread runs Lisp at any given time, so all the other threads are
already "suspended" when the user input arrives at whatever thread is
waiting on the keyboard descriptor.  If the current thread that runs
when the user types is not waiting on the keyboard descriptor, it will
never know the user typed something, and will continue running until
it yields or calls one of the "yielding" APIs, which end up in
wait_reading_process_output.  The thread that did wait on the keyboard
descriptor will exit its 'pselect' call, try to acquire the global
lock, and wait there for the running thread to yield.

> Since there's some confusion on this point -- and maybe what I've stated above
> doesn't make sense, since it could impose concurrency in places we've not
> thought about yet -- I'd like us to discuss this a bit more.

Well, let's discuss ;-)

Re: The event handling thread

[John Wiegley]

>>>>> Eli Zaretskii <eliz@gnu.org> writes:

> To see the above machinery in action, try this:

>   (defun infloop ()
>     (with-temp-buffer (while t (insert "foo"))))

>   (make-thread #'infloop "thread-loop")

> As soon as you start the looping thread, any keyboard input, like M-x or
> cursor motion commands, doesn't have any effect, until you type C-g (which
> causes the looping thread to exit), because the looping thread never yields.

Ah, this is the bit I was missing. It's clear to me now, thanks.

-- 
John Wiegley                  GPG fingerprint = 4710 CF98 AF9B 327B B80F
http://newartisans.com                          60E1 46C4 BD1A 7AC1 4BA2

Re: The event handling thread

[Elias Mårtenson]

[-- Attachment #1: Type: text/plain, Size: 796 bytes --]

On 21 December 2016 at 09:04, John Wiegley <jwiegley@gmail.com> wrote:

> >>>>> Eli Zaretskii <eliz@gnu.org> writes:
>
> > To see the above machinery in action, try this:
>
> >   (defun infloop ()
> >     (with-temp-buffer (while t (insert "foo"))))
>
> >   (make-thread #'infloop "thread-loop")
>
> > As soon as you start the looping thread, any keyboard input, like M-x or
> > cursor motion commands, doesn't have any effect, until you type C-g
> (which
> > causes the looping thread to exit), because the looping thread never
> yields.
>
> Ah, this is the bit I was missing. It's clear to me now, thanks.


But that doesn't change the fact that it makes more sense to never allow
keyboard input to be processed by anything other than the main thread? C-g
isn't "normal" keyboard input, is it?

[-- Attachment #2: Type: text/html, Size: 1278 bytes --]

Re: The event handling thread

[Eli Zaretskii]

> From: Elias Mårtenson <lokedhs@gmail.com>
> Date: Wed, 21 Dec 2016 19:05:39 +0800
> 
> On 21 December 2016 at 09:04, John Wiegley <jwiegley@gmail.com> wrote:
> 
>  >>>>> Eli Zaretskii <eliz@gnu.org> writes:
> 
>  > To see the above machinery in action, try this:
> 
>  > (defun infloop ()
>  > (with-temp-buffer (while t (insert "foo"))))
> 
>  > (make-thread #'infloop "thread-loop")
> 
>  > As soon as you start the looping thread, any keyboard input, like M-x or
>  > cursor motion commands, doesn't have any effect, until you type C-g (which
>  > causes the looping thread to exit), because the looping thread never yields.
> 
>  Ah, this is the bit I was missing. It's clear to me now, thanks.
> 
> But that doesn't change the fact that it makes more sense to never allow keyboard input to be processed by
> anything other than the main thread?

Why does it make more sense than, say, serialize access to the
minibuffer such that only one thread can access it at a time?

> C-g isn't "normal" keyboard input, is it? 

It depends.  It's a bit of both, at least on GUI frames.

Re: The event handling thread

[Elias Mårtenson]

[-- Attachment #1: Type: text/plain, Size: 1079 bytes --]

On 22 December 2016 at 01:42, Eli Zaretskii <eliz@gnu.org> wrote:

> But that doesn't change the fact that it makes more sense to never allow
> keyboard input to be processed by
> > anything other than the main thread?
>
> Why does it make more sense than, say, serialize access to the
> minibuffer such that only one thread can access it at a time?
>

It would, if the minibuffer was the only place where the keyboard was read.
However, isn't this problem wider than just the minibuffer?

In my tests with ‘sit-for’ and threads, I noticed that if multiple threads
do ‘sit-for’ at the same time, and a press a key, then only one of the
threads gets released. As you keep generating input, more and more of the
threads exit the call to ‘sit-for’. A central input processing thread would
fix this issue too, right?


> > C-g isn't "normal" keyboard input, is it?
>
> It depends.  It's a bit of both, at least on GUI frames.
>

Is this related to the messy event loop in GTK Emacs that people were
commenting on in some other thread recently?

[-- Attachment #2: Type: text/html, Size: 1671 bytes --]

Re: The event handling thread

[Eli Zaretskii]

> From: Elias Mårtenson <lokedhs@gmail.com>
> Date: Thu, 22 Dec 2016 11:12:08 +0800
> Cc: emacs-devel <emacs-devel@gnu.org>
> 
>  > But that doesn't change the fact that it makes more sense to never allow keyboard input to be
>  processed by
>  > anything other than the main thread?
> 
>  Why does it make more sense than, say, serialize access to the
>  minibuffer such that only one thread can access it at a time?
> 
> It would, if the minibuffer was the only place where the keyboard was read. However, isn't this problem wider
> than just the minibuffer?

It is.  But minibuffer input is IMO the most urgent issue to solve,
because Emacs interacts with the user like that quite a lot.

> In my tests with ‘sit-for’ and threads, I noticed that if multiple threads do ‘sit-for’ at the same time, and a press
> a key, then only one of the threads gets released. As you keep generating input, more and more of the threads
> exit the call to ‘sit-for’. A central input processing thread would fix this issue too, right?

Could you show some code where this can be seen?  I'm not sure I
understand how exactly this happens.  What I saw in my testing is that
only the primary thread waits on the keyboard descriptor, but maybe
I'm missing something.  Perhaps your tests made the primary thread
yield, or wait on some synchronization object?

>  > C-g isn't "normal" keyboard input, is it?
> 
>  It depends. It's a bit of both, at least on GUI frames.
> 
> Is this related to the messy event loop in GTK Emacs that people were commenting on in some other thread
> recently?

It is related to that, and also to how C-g and keyboard input are
treated in other configurations we support.  AFAIR, TTY, X11, GTK, and
w32 -- each one of these has its own slightly different model of
handling keyboard input and C-g.  Probably NS has yet another one.

Re: Threading IO-bound functions

[Ken Raeburn]

On Dec 14, 2016, at 23:01, Elias Mårtenson <lokedhs@gmail.com> wrote:

> The number one function that I call that sometimes hang for a significant amount of time is ‘gnus’. I decided to try running it in a thread, and it worked surprisingly well. Initial loading of the messages can now be done in the background.

I’m a bit surprised… pleased, but surprised… :-)

> 
> To prevent myself from running this function more than once at the same time, I created a wrapper function for this, and I have extracted it into a macro.
> 
> I'd like to have people's opinions on this strategy, and if it might be reasonable to default ‘gnus’ to do this when run on Emacs versions with concurrency support.

For a few things I think it might be reasonable.  But with something like Gnus, there are many things one might want to do in background (fetch the initial list of newsgroups and article counts, update that list, fetch article lists when entering a newsgroup), and — just generally speaking — each of them has to lock out all the others, lest they stomp on each others’ data structures.

The one advantage the “gnus” command has, specifically, compared to the whole rest of Gnus, is that it can do the initial setting up of those data structures.  If the group buffer isn’t ready for command input (what key bindings are applicable at that point?), it’s harder to trigger other Gnus commands to run.

Though, what about sending email with message mode?  If I use a “Gcc:” header to save sent emails in an IMAP folder, what happens if Gnus is only part-way set up?

When do gnus-demon timers get set up?  Can they fire before gnus is ready?

Etc….

Ken