Why does this thread code crash my emacs?

Why does this thread code crash my emacs?

[Derek Davies]

Hi,


Working with the thread-signal example from:  https://emacs-berlin.org/thread-safe-tramp-2018-09.html

it works for me as given, but when I run hanoi instead of sleeping in
thread2 my emacs consistently crashes (back to command line from which
it was invoked).

I've been poking at it, but without much insight -- does anyone know
what I or hanoi or threading is doing wrong?

Thanks!
derek

GNU Emacs 26.3 (build 1, x86_64-pc-linux-gnu, X toolkit, Xaw scroll bars) of 2020-02-04



;; 2.5 A thread could be signaled from another thread by thread-signal.

(defun my-thread1 ()
  (sleep-for 10)
  (thread-signal thread2 'error '(my-data)))

(defun my-thread2 ()
  (condition-case err
      (hanoi 11)
    (error (message "Thread %s signaled by `%s'" (current-thread) err))))

(setq thread1 (make-thread #'my-thread1 "my thread 1"))
(message "my thread1 %S" thread1)
  (setq thread2 (make-thread #'my-thread2 "my thread 2")))
(message "my thread2 %S" thread2)

Re: Why does this thread code crash my emacs?

[dick]

Yes, I've for a while noticed "s q" (thread-list-send-quit-signal) from
`list-threads` produces crashes.

I've entered a stopgap patch in bug#51229.  The more correct fix is to get rid
of a 1991 boolean called "waiting_for_input" but that bill would be much
tougher to pass politically.

Re: Why does this thread code crash my emacs?

[Eli Zaretskii]

> From: dick <dick.r.chiang@gmail.com>
> Date: Fri, 15 Oct 2021 14:45:57 -0400
> Cc: emacs-devel@gnu.org
> 
> Yes, I've for a while noticed "s q" (thread-list-send-quit-signal) from
> `list-threads` produces crashes.
> 
> I've entered a stopgap patch in bug#51229.  The more correct fix is to get rid
> of a 1991 boolean called "waiting_for_input" but that bill would be much
> tougher to pass politically.

There's nothing political about it.  That variable is part of a
complex state machine used for Emacs input, and to remove it, we'd
need to understand very well when and why it was needed when it was
introduced.  Don't forget that on TTYs we actually handle SIGINT as
part of keyboard input, and waiting_for_input has a role in that case.

IOW, instead of blaming the Emacs development to have some political
issues, why not present a serious analysis of this issue which led you
to the conclusion that the variable can be removed?  Who knows? we
could even (gasp!) accept your analysis.

Re: Why does this thread code crash my emacs?

[dick]

> There's nothing political about it.

You make it sound like politics are a bad thing.

Everything is politics.  Denying there is politics is among the most political things
one can say.

That `waiting_for_input` is a per-thread flag is reason enough to consider it
bogus.  No thread but the main thread could wait for input.

I've already extricated it from Commercial Emacs.

Re: Why does this thread code crash my emacs?

[tomas]

[-- Attachment #1: Type: text/plain, Size: 699 bytes --]

On Fri, Oct 15, 2021 at 03:56:30PM -0400, dick wrote:
> > There's nothing political about it.
> 
> You make it sound like politics are a bad thing.

[...]

To be fair, "political" is a loaded term, and pretty ambiguous.
So if you are interested on a productive discussion, it's perhaps
wise to use it sparingly.

Feel free to continue this discussion off-list or on emacs-tangents
(in the latter case make sure to CC me).

> Everything is politics.  Denying there is politics is among the most political things
> one can say.

[...]

In some way I do agree with you, in some other, not: corresponding
to the extreme semantical bandwidth of the word "political",

Cheers
 - t

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Why does this thread code crash my emacs?

[Eli Zaretskii]

> From: dick <dick.r.chiang@gmail.com>
> Cc: ddavies@ddavies.net,  emacs-devel@gnu.org
> Date: Fri, 15 Oct 2021 15:56:30 -0400
> 
> > There's nothing political about it.
> 
> You make it sound like politics are a bad thing.
> 
> Everything is politics.  Denying there is politics is among the most political things
> one can say.

When discussing solutions to technical problems, politics gets in the
way and usually doesn't help to find a solution.  So yes, it's a bad
thing in this context.

> That `waiting_for_input` is a per-thread flag is reason enough to consider it
> bogus.  No thread but the main thread could wait for input.

But the facts are the OP's recipe proves that a non-main thread _can_
wait for input.  (I already asked what's the difference between the
main and the other threads for this purpose, and I don't think you
answered that.)  As long as no other thread signals that thread, the
OP's program will run without problems.  So, unless by the above you
mean something very specific that you didn't explain in enough detail,
the facts seem to directly contradict what you say.

> I've already extricated it from Commercial Emacs.

We are talking about Emacs, not about your fork, where you can do
whatever you like.  If you don't care about Emacs, why do you insist
on speaking about what we do, time and again?  If you do care, why
does it matter what you did in your private fork, and what does it
prove?

Re: Why does this thread code crash my emacs?

[dick]

> But the facts are the OP's recipe proves that a non-main thread _can_ wait
> for input.

It does not.  Try typing into the *Hanoi* buffer.  It does not stop the Hanoi
as it would if it were running in main thread.  While noble, it's folly to
insist treating main and non-main the same as in bug#36609, which incidentally
still causes deadlocks even after your fix with atomics (I've reverted your
fix in favor of mine in my fork).

> If you do care, why does it matter what you did in your private fork

Narcissism of course.  Why does anyone do anything for no pay?  The more
serious answer is I am using emacs-devel as a recruiting mechanism to show
what's possible under a maintainer who:

1. much prefers a PR-based workflow with automated CI, superior changeset
management, conversation visibility, commit and line citation, but equally
poor search

2. is unafraid to address problems at their root, quite distinct from the
current committee's timorous "a stopgap patch here and a stopgap there"
approach.

Re: Why does this thread code crash my emacs?

[Eli Zaretskii]

> From: dick <dick.r.chiang@gmail.com>
> Cc: emacs-devel@gnu.org
> Date: Sat, 16 Oct 2021 06:53:17 -0400
> 
> > But the facts are the OP's recipe proves that a non-main thread _can_ wait
> > for input.
> 
> It does not.

Of course, it does: it calls read_char.

> Try typing into the *Hanoi* buffer.  It does not stop the Hanoi as
> it would if it were running in main thread.

And why is that?

> > If you do care, why does it matter what you did in your private fork
> 
> Narcissism of course.  Why does anyone do anything for no pay?  The more
> serious answer is I am using emacs-devel as a recruiting mechanism to show
> what's possible under a maintainer who:
> 
> 1. much prefers a PR-based workflow with automated CI, superior changeset
> management, conversation visibility, commit and line citation, but equally
> poor search
> 
> 2. is unafraid to address problems at their root, quite distinct from the
> current committee's timorous "a stopgap patch here and a stopgap there"
> approach.

Let's talk in a year or 5 or 10 and see where did your fork get under
such a maintainer.

Re: Why does this thread code crash my emacs?

[Eli Zaretskii]

> From: Derek Davies <ddavies@ddavies.net>
> Date: Fri, 15 Oct 2021 12:04:35 -0400
> 
> Working with the thread-signal example from:  https://emacs-berlin.org/thread-safe-tramp-2018-09.html
> 
> it works for me as given, but when I run hanoi instead of sleeping in
> thread2 my emacs consistently crashes (back to command line from which
> it was invoked).
> 
> I've been poking at it, but without much insight -- does anyone know
> what I or hanoi or threading is doing wrong?

You currently cannot signal a thread that is waiting for input (hanoi
waits for input inside sit-for), because Emacs doesn't (yet) know how
to recover from signals in that state.

IOW, this is a limitation of the current implementation that is more
visible when signaling threads than when there's only one (main)
thread.

Re: Why does this thread code crash my emacs?

[Derek Davies]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Derek Davies <ddavies@ddavies.net>
>> Date: Fri, 15 Oct 2021 12:04:35 -0400
>> 
>> Working with the thread-signal example from:  https://emacs-berlin.org/thread-safe-tramp-2018-09.html
>> 
>> it works for me as given, but when I run hanoi instead of sleeping in
>> thread2 my emacs consistently crashes (back to command line from which
>> it was invoked).
>> 
>> I've been poking at it, but without much insight -- does anyone know
>> what I or hanoi or threading is doing wrong?
>
> You currently cannot signal a thread that is waiting for input (hanoi
> waits for input inside sit-for), because Emacs doesn't (yet) know how
> to recover from signals in that state.
>
> IOW, this is a limitation of the current implementation that is more
> visible when signaling threads than when there's only one (main)
> thread.

Just for fun, and since the wise monks of hanoi wrapped sit-for in
hanoi-sit-for, I seem to be able to get away with the following hack.


(defun my-thread1 ()
  (sleep-for 10)
  (thread-signal thread2 'error '()))

(defun my-thread2 ()
  (condition-case err
      (let ((ex-fx (symbol-function 'hanoi-sit-for)))
        (fset 'hanoi-sit-for #'(lambda (secs &optional nodisp)
                                 (sleep-for secs)
                                 (or nodisp (redisplay))
                                 t))
        (unwind-protect
            (hanoi 11)
          (progn (fset 'hanoi-sit-for ex-fx)
                 (message "Done"))))
    (error (message "Thread %s signaled by `%s'" (current-thread) err))))

(setq thread1 (make-thread #'my-thread1 "my thread 1"))
(message "my thread1 %S" thread1)

  (setq thread2 (make-thread #'my-thread2 "my thread 2")))
(message "my thread2 %S" thread2)


Thanks for the attention to my question!

Derek

Re: Why does this thread code crash my emacs?

[dick]

While the case in question is pedagogical (and as such, any workaround
comes to nothing), I cannot but indulge my urge to de-obfuscate
(cl-letf is a friend and patriot):

emacs-26.3 -Q -l cl --eval                                        \
"(lexical-let* ((thread2                                          \
                 (make-thread                                     \
                  (lambda ()                                      \
                    (cl-letf (((symbol-function (quote sit-for))  \
                               (lambda (secs &optional nodisp)    \
                                 (sleep-for secs)                 \
                                 (or nodisp (redisplay))          \
                                 t)))                             \
                      (hanoi 11)))))                              \
                (thread1                                          \
                 (make-thread                                     \
                  (lambda ()                                      \
                    (sleep-for 10)                                \
                    (thread-signal thread2 (quote error) nil))))))"

Re: Why does this thread code crash my emacs?

[Eli Zaretskii]

> Date: Fri, 15 Oct 2021 22:01:04 +0300
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: emacs-devel@gnu.org
> 
> You currently cannot signal a thread that is waiting for input (hanoi
> waits for input inside sit-for), because Emacs doesn't (yet) know how
> to recover from signals in that state.
> 
> IOW, this is a limitation of the current implementation that is more
> visible when signaling threads than when there's only one (main)
> thread.

I've now attempted to fix this on the emacs-28 branch.  Hopefully,
this won't have any adverse effects elsewhere, otherwise we'd have to
move this to master, and continue debugging that.