list-threads

list-threads

[Gemini Lasswell]

Hello All,

I've just pushed a commit to scratch/list-threads which might be useful
to those of you working with threads in Emacs. There is a new command
'list-threads' which produces a tabulated list buffer showing the
currently live threads with their status, sets a timer to keep it
refreshed, and provides two commands, 's q' and 's e' for sending quit
and error signals, respectively, to the thread at point.

Next up I'm going to see if my rusty C skills are up to adding a
'thread' argument to the backtrace primitives, to make it possible to
generate a backtrace for any thread from the thread list buffer.

Re: list-threads

[Michael Albinus]

Gemini Lasswell <gazally@runbox.com> writes:

> Hello All,

Hi Gemini,

> I've just pushed a commit to scratch/list-threads which might be useful
> to those of you working with threads in Emacs. There is a new command
> 'list-threads' which produces a tabulated list buffer showing the
> currently live threads with their status, sets a timer to keep it
> refreshed, and provides two commands, 's q' and 's e' for sending quit
> and error signals, respectively, to the thread at point.

Nice. I've cherry-picked it into the feature/tramp-thread-safe
branch. Works as expected.

Just some first thoughts:

If I'm visiting many files (say .../emacs/admin/*), there are as many
threads as files in that directory, 30+ in this case. Refreshing does
not work as fast as hoped - sure, the main thread is just one thread
amongst the other threads. Maybe we need some priorization, that a given
thread (the main thread here) gets mor hits in thread-yield.

What happens if you call list-threads from another thread but the main
one? (Haven't tested myself).

> Next up I'm going to see if my rusty C skills are up to adding a
> 'thread' argument to the backtrace primitives, to make it possible to
> generate a backtrace for any thread from the thread list buffer.

Sounds very useful. On my wishlist would also be edebug support for
threads, this doesn't work at all.

Thanks for your work, and best regards, Micghael.

Re: list-threads

[Michael Albinus]

Michael Albinus <michael.albinus@gmx.de> writes:

Hi Gemini,

> If I'm visiting many files (say .../emacs/admin/*), there are as many
> threads as files in that directory, 30+ in this case. Refreshing does
> not work as fast as hoped - sure, the main thread is just one thread
> amongst the other threads. Maybe we need some priorization, that a given
> thread (the main thread here) gets mor hits in thread-yield.

FTR, hitting 'g' in the *Threads* buffer works sufficient. Don't worry
too much about my comment.

Best regards, Micghael.

Re: list-threads

[Gemini Lasswell]

Michael Albinus <michael.albinus@gmx.de> writes:

> If I'm visiting many files (say .../emacs/admin/*), there are as many
> threads as files in that directory, 30+ in this case. Refreshing does
> not work as fast as hoped - sure, the main thread is just one thread
> amongst the other threads. Maybe we need some priorization, that a given
> thread (the main thread here) gets mor hits in thread-yield.

There's a defcustom called thread-list-refresh-seconds which controls
how often the auto refresh happens.  Try setting it to 0.05 instead of
0.5.  I welcome your input on what the default ought to be.

> What happens if you call list-threads from another thread but the main
> one? (Haven't tested myself).

Seems to work fine.  It uses a timer, and those run in the main thread.

> Sounds very useful. On my wishlist would also be edebug support for
> threads, this doesn't work at all.

It's on my wishlist too, and I'm thinking of doing something about it
once I've gained a bit more experience working with threads.

Re: list-threads

[Eli Zaretskii]

> From: Gemini Lasswell <gazally@runbox.com>
> Date: Sun, 29 Jul 2018 07:31:02 -0700
> Cc: Emacs Development <emacs-devel@gnu.org>
> 
> > What happens if you call list-threads from another thread but the main
> > one? (Haven't tested myself).
> 
> Seems to work fine.  It uses a timer, and those run in the main thread.

Are you sure timers always run in the main thread?  I think they will
run by whatever thread calls wait_reading_process_output.

Re: list-threads

[Gemini Lasswell]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Gemini Lasswell <gazally@runbox.com>
>> Date: Sun, 29 Jul 2018 07:31:02 -0700
>> Cc: Emacs Development <emacs-devel@gnu.org>
>> 
>> > What happens if you call list-threads from another thread but the main
>> > one? (Haven't tested myself).
>> 
>> Seems to work fine.  It uses a timer, and those run in the main thread.
>
> Are you sure timers always run in the main thread?  I think they will
> run by whatever thread calls wait_reading_process_output.

Yes, that is correct.

Re: list-threads

[Eli Zaretskii]

> From: Gemini Lasswell <gazally@runbox.com>
> Date: Sat, 28 Jul 2018 18:11:51 -0700
> 
> I've just pushed a commit to scratch/list-threads which might be useful
> to those of you working with threads in Emacs. There is a new command
> 'list-threads' which produces a tabulated list buffer showing the
> currently live threads with their status, sets a timer to keep it
> refreshed, and provides two commands, 's q' and 's e' for sending quit
> and error signals, respectively, to the thread at point.

Thanks, please see some comments below.

> Next up I'm going to see if my rusty C skills are up to adding a
> 'thread' argument to the backtrace primitives, to make it possible to
> generate a backtrace for any thread from the thread list buffer.

You mean, take a backtrace while the target thread is waiting to run?

Here are the comments to your branch:

> +++ b/lisp/emacs-lisp/thread-list.el

I'm not sure this file should be called thread-list.el, nor that it
should be in emacs-lisp/.  I think it should be under lisp/, and
should be the home for all thread-related features implemented in
Lisp.

> +(defun thread-list--timer-func (buf)
> +  "Revert BUF and set a timer to do it again."

Suggest to rename the argument BUFFER, then the doc string will be
more clear, IMO.

> +(defun thread-list--get-entries ()
> +  "Return tabulated list entries for the threads currently active."

"for the currently active threads".

Actually, I'd use "live", not "active", because they really aren't (or
else the definition of "active" in this case could confuse users).

> +Return a list of two strings, the first describing THREAD's
> +status and the second describing what it is blocked on if it is
> +blocked."

I'd rephrase:

  Return a list of two strings: one describing THREAD's status,
  the other describing THREAD's blocker, if any.

> +(defun thread-list--send-signal (sgnl)
> +  "Send the signal SGNL to the thread at point.

I'd use

  Send the specified SIGNAL to the thread at point.

> +Confirm with the user first."

"Ask for user confirmation before signaling the thread."

> +  (let ((thread (tabulated-list-get-id)))
> +    (when (and (threadp thread) (thread-alive-p thread))
> +      (when (y-or-n-p (format "Send %s signal to %s? " sgnl thread))
> +        (when (and (threadp thread) (thread-alive-p thread))
> +          (thread-signal thread sgnl nil))))))

This silently does nothing if the thread is no longer alive.  I'd
suggest some echo-area message to that effect.

Finally, this needs NEWS entry and a short description for the user
manual.

Re: list-threads

[Stefan Monnier]

>> Sounds very useful. On my wishlist would also be edebug support for
>> threads, this doesn't work at all.
> It's on my wishlist too, and I'm thinking of doing something about it
> once I've gained a bit more experience working with threads.

Not sure how Edebug should work, but for plain `debug`, I think it would
be great if it could work by suspending the erroring thread (i.e. the
debug session runs in its own thread, so it's not affected by the
dynamic-bindings currently active in the debugged thread).


        Stefan

Re: list-threads

[Gemini Lasswell]

Eli Zaretskii <eliz@gnu.org> writes:

> Thanks, please see some comments below.

Thanks, I'll use all your suggestions.

> You mean, take a backtrace while the target thread is waiting to run?

Yes.  My thought is that this might be helpful for figuring out where a
stuck thread is blocked.  If a thread is actively working the backtrace
will just be a snapshot of where it was when you asked for the
backtrace.  I'll explain that in the documentation.

> I'm not sure this file should be called thread-list.el, nor that it
> should be in emacs-lisp/.  I think it should be under lisp/, and
> should be the home for all thread-related features implemented in
> Lisp.

I'll move it to lisp/thread.el.

Re: list-threads

[Eli Zaretskii]

> From: Gemini Lasswell <gazally@runbox.com>
> Date: Wed, 01 Aug 2018 09:03:10 -0700
> Cc: emacs-devel@gnu.org
> 
> > You mean, take a backtrace while the target thread is waiting to run?
> 
> Yes.  My thought is that this might be helpful for figuring out where a
> stuck thread is blocked.  If a thread is actively working the backtrace
> will just be a snapshot of where it was when you asked for the
> backtrace.

If a thread is running, no other thread will be able to run, so you
will be unable to take the backtrace of that running thread.  Right?

> I'll move it to lisp/thread.el.

Thanks.

Re: list-threads

[Gemini Lasswell]

Eli Zaretskii <eliz@gnu.org> writes:

> If a thread is running, no other thread will be able to run, so you
> will be unable to take the backtrace of that running thread.  Right?

I don't understand the question, but I can try to explain what I'm
thinking in more detail.

In the command which I haven't written yet called
thread-list-pop-to-backtrace, I'll get the thread at point and compare
it to the current thread. If they are the same then I can make a
backtrace buffer in the usual way, by using mapbacktrace to collect all
the frames (or call the new backtrace-get-frames which does that), and
the backtrace will include command-execute through
thread-list-pop-to-backtrace, and it will not be very interesting unless
there's a recursive edit or something going on.

If the thread at point is not the current thread, then I'll call the new
primitive that I'm working on, maybe to be called
backtrace--get-frames-from-thread, pass it the thread as an argument,
and it will cons up a list of backtrace frames using that thread's
specpdl stack, which I can use to fill my backtrace buffer.

As an experiment I added a thread argument to mapbacktrace, and changed
it and its subroutines to use thread->m_specpdl instead of specpdl, etc.
That works and lets me get backtraces from threads other than the
current one whether blocked or just yielded, but the problem with
mapbacktrace is that it calls a Lisp function which could yield and
allow the thread to run that I'm trying to get the backtrace of.  Hence
the new primitive.

I'd also like to collect local variable names and values for the
backtrace frames of the other thread, but I don't know how easy that
will be because I don't yet understand the backtrace_eval_unrewind hack
in backtrace--locals well enough to know if I can make it work safely on
threads other than the current one.

Re: list-threads

[Eli Zaretskii]

> From: Gemini Lasswell <gazally@runbox.com>
> Cc: emacs-devel@gnu.org
> Date: Wed, 01 Aug 2018 13:04:18 -0700
> 
> If the thread at point is not the current thread, then I'll call the new
> primitive that I'm working on, maybe to be called
> backtrace--get-frames-from-thread, pass it the thread as an argument,
> and it will cons up a list of backtrace frames using that thread's
> specpdl stack, which I can use to fill my backtrace buffer.

I'm saying that this will always show a thread that is blocked, either
on some mutex/condvar, or waiting for the global lock to be released.
You will never be able to see the backtrace of another running thread,
because the only thread that is running is the one which takes the
backtrace snapshot.

That question was response to what you said here:

> If a thread is actively working the backtrace will just be a
> snapshot of where it was when you asked for the backtrace.

I'm saying that this is impossible: any thread but the current one is
not "actively working".

Re: list-threads

[Gemini Lasswell]

Eli Zaretskii <eliz@gnu.org> writes:

>> If a thread is actively working the backtrace will just be a
>> snapshot of where it was when you asked for the backtrace.
>
> I'm saying that this is impossible: any thread but the current one is
> not "actively working".

Yes.  I used the wrong words there.  Sorry for making it confusing.

Re: list-threads

[Gemini Lasswell]

I've just pushed an update to the scratch/list-threads branch which
includes:

- lisp/thread.el, the new home for thread-related Lisp functions.

- A 'b' command in the *Threads* buffer which shows a backtrace of where
  the thread at point had yielded or was blocked at the moment you
  pressed 'b'.

- Documentation and a NEWS entry.

- Tests.

There are no local variables in the thread backtraces.  I experimented a
bit with making Fbacktrace__locals work on non-current threads, with
mixed results (success in extracting variable values at the cost of
corrupting something making Emacs prone to crashing afterwards), and
decided that feature is more difficult (for me, at the present time)
than it is important.

If you have any feedback, let me know, otherwise I'll merge this in a
few days.

Re: list-threads

[Eli Zaretskii]

> From: Gemini Lasswell <gazally@runbox.com>
> Date: Mon, 27 Aug 2018 10:38:06 -0700
> 
> I've just pushed an update to the scratch/list-threads branch which
> includes:
> 
> - lisp/thread.el, the new home for thread-related Lisp functions.
> 
> - A 'b' command in the *Threads* buffer which shows a backtrace of where
>   the thread at point had yielded or was blocked at the moment you
>   pressed 'b'.
> 
> - Documentation and a NEWS entry.
> 
> - Tests.

Thanks.

A few of comments:

 . I'm not sure the section bout list-threads should be in the ELisp
   manual.  It's a user command.  OTOH, list-processes is also
   described in the ELisp manual, hmm...

 . But if we decide to leave this section where it is, then it should
   behave like any other section in that manual, which means variables
   are described with @defvar.

 . Please add a cross-reference to the section where make-thread is
   described where you tell that the thread's name comes from that
   call.  Info manuals should always cross-link to the description of
   each major topic when they refer to it, because people frequently
   read the manual out of order.

 . "Send a signal to the thread" is slightly misleading, because it
   can be interpreted as talking about OS signals.  Suggest to use
   "signal the thread" instead.  I'd also qualify that what each
   thread does for each of the 2 signals is entirely up to the thread
   (default is to exit on any signal).

 . This text is slightly inaccurate:

    Be aware that by the time you see the backtrace, the thread
    may have resumed execution, and be in a different section of
    code, or be completed.

   I would rephrase like this:

    Be aware that the backtrace is a snapshot; the thread could have
    meanwhile resumed execution, and could be in a different state, or
    could have exited.

 . Will the new function backtrace--frames-from-thread (and its
   subroutines) compile and work in an Emacs configured without
   threads?  If not, they should be conditioned by THREADS_ENABLED.

 . Finally, I think the NEWS entry should be a bit longer, and should
   say this is a new command.

Re: list-threads

[Gemini Lasswell]

Eli Zaretskii <eliz@gnu.org> writes:

> A few of comments:

Thanks for the feedback.

>  . I'm not sure the section bout list-threads should be in the ELisp
>    manual.  It's a user command.  OTOH, list-processes is also
>    described in the ELisp manual, hmm...

list-timers is also described in the ELisp manual.  All three commands
are more useful to Lisp programmers than to Emacs users who don't know
Lisp.  So I've left it where it is.

>  . But if we decide to leave this section where it is, then it should
>    behave like any other section in that manual, which means variables
>    are described with @defvar.
>
>  . Please add a cross-reference to the section where make-thread is
>    described where you tell that the thread's name comes from that
>    call.  Info manuals should always cross-link to the description of
>    each major topic when they refer to it, because people frequently
>    read the manual out of order.
>
>  . "Send a signal to the thread" is slightly misleading, because it
>    can be interpreted as talking about OS signals.  Suggest to use
>    "signal the thread" instead.  I'd also qualify that what each
>    thread does for each of the 2 signals is entirely up to the thread
>    (default is to exit on any signal).
>
>  . This text is slightly inaccurate:
>
>     Be aware that by the time you see the backtrace, the thread
>     may have resumed execution, and be in a different section of
>     code, or be completed.
>
>    I would rephrase like this:
>
>     Be aware that the backtrace is a snapshot; the thread could have
>     meanwhile resumed execution, and could be in a different state, or
>     could have exited.

>  . Finally, I think the NEWS entry should be a bit longer, and should
>    say this is a new command.

I've made all these changes, in 8adc0e518f.

>  . Will the new function backtrace--frames-from-thread (and its
>    subroutines) compile and work in an Emacs configured without
>    threads?  If not, they should be conditioned by THREADS_ENABLED.

Yes, they compile.  In an Emacs configured without threads,
backtrace--frames-from-thread is present but will always give a
wrong-type-argument error, because you can't make a thread object to
pass to it without make-thread.

Re: list-threads

[Eli Zaretskii]

> From: Gemini Lasswell <gazally@runbox.com>
> Date: Sun, 09 Sep 2018 08:44:41 -0700
> Cc: emacs-devel@gnu.org
> 
> >  . Will the new function backtrace--frames-from-thread (and its
> >    subroutines) compile and work in an Emacs configured without
> >    threads?  If not, they should be conditioned by THREADS_ENABLED.
> 
> Yes, they compile.  In an Emacs configured without threads,
> backtrace--frames-from-thread is present but will always give a
> wrong-type-argument error, because you can't make a thread object to
> pass to it without make-thread.

It is better to signal an error with a more friendly description that
wrong-type-argument, I think.

Thanks.