No thread yield when using synchronous processes?

No thread yield when using synchronous processes?

[Philipp Stephani]

Hi,

it seems like Emacs doesn't yield when using synchronous processes:

(make-thread (lambda () (call-process "sleep" nil nil nil "5s")))

⇒ Emacs freezes for 5 seconds.

Is there a reason for this behavior? The manual claims that Emacs
yields when waiting for process output, but that only seems to be the
case for asynchronous processes. It's of course possible to simulate
synchronous processes with asynchronous processes, but the
asynchronous process API is more difficult to use.

Thanks,
Philipp

Re: No thread yield when using synchronous processes?

[Michael Albinus]

Philipp Stephani <p.stephani2@gmail.com> writes:

> Hi,

Hi Philipp,

> it seems like Emacs doesn't yield when using synchronous processes:
>
> (make-thread (lambda () (call-process "sleep" nil nil nil "5s")))
>
> ⇒ Emacs freezes for 5 seconds.
>
> Is there a reason for this behavior? The manual claims that Emacs
> yields when waiting for process output, but that only seems to be the
> case for asynchronous processes.

The manual is clear about this: "when waiting for ... process output
(e.g., during ‘accept-process-output’)". `accept-process-output' is used
only for asynchronous processes.

> Thanks,
> Philipp

Best regards, Michael.

Re: No thread yield when using synchronous processes?

[Eli Zaretskii]

> From: Michael Albinus <michael.albinus@gmx.de>
> Date: Thu, 13 Dec 2018 10:47:32 +0100
> Cc: help-gnu-emacs <help-gnu-emacs@gnu.org>
> 
> > it seems like Emacs doesn't yield when using synchronous processes:
> >
> > (make-thread (lambda () (call-process "sleep" nil nil nil "5s")))
> >
> > ⇒ Emacs freezes for 5 seconds.
> >
> > Is there a reason for this behavior? The manual claims that Emacs
> > yields when waiting for process output, but that only seems to be the
> > case for asynchronous processes.
> 
> The manual is clear about this: "when waiting for ... process output
> (e.g., during ‘accept-process-output’)". `accept-process-output' is used
> only for asynchronous processes.

Right.  Though perhaps we should say that more explicitly, as "e.g."
could be interpreted like just one example, not necessarily a
representative of a class of examples.

Technically, thread switches happen when Emacs calls 'pselect', which
it does as part of its idle loop.  call-process doesn't enter the idle
loop, it busy-waits until the process exits.  One could think about
calling the idle loop while we wait, but this will have significant
Emacs-wide effects, which need to be considered carefully before we
decide to do it.

(Btw, pedantically, the example above doesn't really _wait_ for
process output, it explicitly discards that output.)

Re: No thread yield when using synchronous processes?

[Philipp Stephani]

Am Do., 13. Dez. 2018 um 10:47 Uhr schrieb Michael Albinus
<michael.albinus@gmx.de>:
>
> Philipp Stephani <p.stephani2@gmail.com> writes:
>
> > Hi,
>
> Hi Philipp,
>
> > it seems like Emacs doesn't yield when using synchronous processes:
> >
> > (make-thread (lambda () (call-process "sleep" nil nil nil "5s")))
> >
> > ⇒ Emacs freezes for 5 seconds.
> >
> > Is there a reason for this behavior? The manual claims that Emacs
> > yields when waiting for process output, but that only seems to be the
> > case for asynchronous processes.
>
> The manual is clear about this: "when waiting for ... process output
> (e.g., during ‘accept-process-output’)". `accept-process-output' is used
> only for asynchronous processes.

That parenthetical says "e.g." and not "i.e.", implying that there are
other situations where Emacs waits for process output. Clearly the
"read" call made by call-process is such a situation, so I'd expect
this sentence to apply to them.

Re: No thread yield when using synchronous processes?

[Philipp Stephani]

Am Do., 13. Dez. 2018 um 14:46 Uhr schrieb Eli Zaretskii <eliz@gnu.org>:
>
> > From: Michael Albinus <michael.albinus@gmx.de>
> > Date: Thu, 13 Dec 2018 10:47:32 +0100
> > Cc: help-gnu-emacs <help-gnu-emacs@gnu.org>
> >
> > > it seems like Emacs doesn't yield when using synchronous processes:
> > >
> > > (make-thread (lambda () (call-process "sleep" nil nil nil "5s")))
> > >
> > > ⇒ Emacs freezes for 5 seconds.
> > >
> > > Is there a reason for this behavior? The manual claims that Emacs
> > > yields when waiting for process output, but that only seems to be the
> > > case for asynchronous processes.
> >
> > The manual is clear about this: "when waiting for ... process output
> > (e.g., during ‘accept-process-output’)". `accept-process-output' is used
> > only for asynchronous processes.
>
> Right.  Though perhaps we should say that more explicitly, as "e.g."
> could be interpreted like just one example, not necessarily a
> representative of a class of examples.

Yes, I think this should be much more explicit. Given that thread
yields are highly observable (buffer contents, point, etc. can
change), it would be best to explicitly document each function that
yields, and state that other functions don't yield.

>
> Technically, thread switches happen when Emacs calls 'pselect', which
> it does as part of its idle loop.  call-process doesn't enter the idle
> loop, it busy-waits until the process exits.  One could think about
> calling the idle loop while we wait, but this will have significant
> Emacs-wide effects, which need to be considered carefully before we
> decide to do it.
>
> (Btw, pedantically, the example above doesn't really _wait_ for
> process output, it explicitly discards that output.)
>

Re: No thread yield when using synchronous processes?

[Michael Albinus]

Philipp Stephani <p.stephani2@gmail.com> writes:

 Hi Philipp,

>> The manual is clear about this: "when waiting for ... process output
>> (e.g., during ‘accept-process-output’)". `accept-process-output' is used
>> only for asynchronous processes.
>
> That parenthetical says "e.g." and not "i.e.", implying that there are
> other situations where Emacs waits for process output. Clearly the
> "read" call made by call-process is such a situation, so I'd expect
> this sentence to apply to them.

You are right, the documentation shall say it explicitly. That's what
Eli has recommended as well.

I've pushed a fix to the emacs-26 branch. If thread switching will be
implemented also for synchronous processes, this might be adapted.

Best regards, Michael-

Re: No thread yield when using synchronous processes?

[Philipp Stephani]

Am So., 23. Dez. 2018 um 09:07 Uhr schrieb Michael Albinus
<michael.albinus@gmx.de>:
>
> Philipp Stephani <p.stephani2@gmail.com> writes:
>
>  Hi Philipp,
>
> >> The manual is clear about this: "when waiting for ... process output
> >> (e.g., during ‘accept-process-output’)". `accept-process-output' is used
> >> only for asynchronous processes.
> >
> > That parenthetical says "e.g." and not "i.e.", implying that there are
> > other situations where Emacs waits for process output. Clearly the
> > "read" call made by call-process is such a situation, so I'd expect
> > this sentence to apply to them.
>
> You are right, the documentation shall say it explicitly. That's what
> Eli has recommended as well.
>
> I've pushed a fix to the emacs-26 branch. If thread switching will be
> implemented also for synchronous processes, this might be adapted.

Thanks!