unofficial mirror of help-gnu-emacs@gnu.org
 help / color / mirror / Atom feed
* Async process sentinel running exclusively in main thread?
@ 2022-06-20  8:43 Félix Baylac Jacqué
  2022-06-20 11:23 ` Eli Zaretskii
  0 siblings, 1 reply; 4+ messages in thread
From: Félix Baylac Jacqué @ 2022-06-20  8:43 UTC (permalink / raw)
  To: help-gnu-emacs

Hi folks,

Let's start with a bit of context:

I'd like to test a function spinning up an asynchronous process call
using ERT. Sadly, at the moment (Emacs 28.1), there's no way for ERT to
cope with asynchronous calls out of the box.

You can find a clever trick allowing you to do that online though:
Rejeep's ert-async.el library. This trick is based on abusing the
accept-process-output function. This library has been written in 2014:
at the time, condition-variables were not a thing and
accept-process-output was the best option you had to block the main
thread and wait for another thread to be done.

Things changed since then, we now have better tools to manage
concurrency in Emacs! I came to think we could probably re-write this
library using condition variables.

The overall idea is to wait for the subprocess to be done by blocking
the main thread (the one running the ERT BODY test case) using a
condition variable. The subprocess would then unblock the main thread
using its sentinel.

Overall, it transforms the async subprocess call into a synchronous for
the test case.

Which bring us to the issue: this approach results in deadlocking the
main thread.

/!\ WARNING: RUNNING THIS SNIPPET WILL DEADLOCK YOUR EMACS INSTANCE!!!!

--8<---------------cut here---------------start------------->8---
(defun test-sentinel (process event)
  (with-mutex mutex
    (progn
      (message "Sentinel: running")
      (setq exit-code 1)
      (condition-notify cond-var)
      (message "Sentinel: notified"))))

(defun test-make-process ()
  (let* ((run-async-process
          (lambda ()
            (progn
              (make-process
               :name "dummy-async-subprocess"
               :buffer "ls-buf"
               :command '("ls")
               :sentinel 'test-sentinel))))
         (run-wait-process
          (lambda ()
            (progn
              (with-mutex mutex
                (while (not exit-code)
                  (message "Wait: Waiting for exit-code")
                  (condition-wait cond-var)
                  (message "Wait: notified")))))))
    (progn
      (setq exit-code nil)
      (setq mutex (make-mutex "mutex-test"))
      (setq cond-var (make-condition-variable mutex "cond-test"))
      (make-thread run-async-process)
      (funcall run-wait-process))))

(test-make-process)
--8<---------------cut here---------------end--------------->8---

This deadlock is quite surprising for an Emacs newcomer like me: the
sentinel end up never being executed. According to the make-process
documentation, the process is supposed to be attached to the thread it
originates from. In this case the run-async-process thread. The
condition-wait originating from the main thread shouldn't lock the
subprocess sentinel.

This issue confused me for quite a while. Until I noticed something
quite surprising with the following snippet:

Note: this one is safe to run :)

--8<---------------cut here---------------start------------->8---
(defun test-sentinel (process event)
  (with-mutex mutex
    (progn
      (message "Sentinel: running")
      (message (format "Sentinel: run in main thread? %s" (equal (current-thread) main-thread)))
      (setq exit-code 1)
      (condition-notify cond-var)
      (message "Sentinel: notified"))))

(defun test-make-process ()
  (let* ((run-async-process
          (lambda ()
            (progn
              (message (format "run-async-process: run in main thread? %s" (equal (current-thread) main-thread)))
              (make-process
               :name "dummy-async-subprocess"
               :buffer "ls-buf"
               :command '("ls")
               :sentinel 'test-sentinel))))
         (run-wait-process
          (lambda ()
            (progn
              (with-mutex mutex
                (while (not exit-code)
                  (message (format "Wait: run in main thread? %s" (equal (current-thread) main-thread)))
                  (message "Wait: Waiting for exit-code")
                  (condition-wait cond-var)
                  (message "Wait: notified")))))))
    (progn
      (setq exit-code nil)
      (setq mutex (make-mutex "mutex-test"))
      (setq cond-var (make-condition-variable mutex "cond-test"))
      (make-thread run-async-process)
      (make-thread run-wait-process))))

(test-make-process)
--8<---------------cut here---------------end--------------->8---

Message output on Emacs 28.1:

--8<---------------cut here---------------start------------->8---
Wait: run in main thread? nil
Wait: Waiting for exit-code
run-async-process: run in main thread? nil
Sentinel: running
Sentinel: run in main thread? t
Sentinel: notified
Wait: notified
--8<---------------cut here---------------end--------------->8---

As you can see, while the run-async-process and the run-wait-process
functions are each living in their own threads, but the sentinel ends up
running in the main thread instead of the run-wait-process one!?

Surprising. That explains the deadlock origin though.

I couldn't find any mention of the fact that the sentinel function will
exclusively run in the main thread in the documentation. Regardless of
the thread in which the process is actually executed.

Which makes me wonder:

Is this a bug or is this an expected behavior?

In case it is an expected behavior, is there any way to wait for a
subprocess to be done in the main thread using a condition variable
without deadlocking Emacs?

Thanks a lot for taking the time to read this long message :)
Félix



^ permalink raw reply	[flat|nested] 4+ messages in thread

end of thread, other threads:[~2022-06-21  6:29 UTC | newest]

Thread overview: 4+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2022-06-20  8:43 Async process sentinel running exclusively in main thread? Félix Baylac Jacqué
2022-06-20 11:23 ` Eli Zaretskii
2022-06-21  6:03   ` Félix Baylac Jacqué
2022-06-21  6:29     ` Emanuel Berg

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).