bug#31214: hang in Faccept_process_output

bug#31214: hang in Faccept_process_output

[Glenn Morris]

Package: emacs
Version: 27.0.50

Current master on RHEL7.4. If I do:

n=0
while : ; do
 let n++; echo $n
 rm -f test/src/process-tests.log
 make -C test src/process-tests.log \
   SELECTOR='\"make-process/mix-stderr$$\"' >& /dev/null
done

then eventually (after a few hundred loops), the spawned Emacs hangs in
Faccept_process_output. The associated external process is dead and
gone. gdb says:

#0  0x00007fe00d135879 in pselect () at /lib64/libc.so.6
#1  0x000000000068087e in really_call_select (arg=0x7ffda4c66e60)
    at thread.c:572
#2  0x00000000005d6f13 in flush_stack_call_func (func=0x6807ed <really_call_select>, arg=0x7ffda4c66e60) at alloc.c:5213
#3  0x0000000000680933 in thread_select (func=0x413330 <pselect@plt>, max_fds=10, rfds=0x7ffda4c67390, wfds=0x7ffda4c67310, efds=0x0, timeout=0x7ffda4c67680, sigmask=0x0) at thread.c:602
#4  0x00000000006aa642 in xg_select (fds_lim=10, rfds=0x7ffda4c67720, wfds=0x7ffda4c676a0, efds=0x0, timeout=0x7ffda4c67680, sigmask=0x0) at xgselect.c:117
#5  0x0000000000656410 in wait_reading_process_output (time_limit=0, nsecs=0, read_kbd=0, do_display=false, wait_for_cell=..., wait_proc=0x140b6a0, just_wait_proc=0) at process.c:5384
#6  0x00000000006547cf in Faccept_process_output (process=..., seconds=..., millisec=..., just_this_one=...) at process.c:4672

Maybe resembles https://debbugs.gnu.org/24201 ?
But Emacs is sleeping using no CPU.

bug#31214: hang in Faccept_process_output

[Glenn Morris]

Glenn Morris wrote:

> Current master on RHEL7.4.

Also reproduced on current Debian testing.

bug#31214: hang in Faccept_process_output

[Eli Zaretskii]

> From: Glenn Morris <rgm@gnu.org>
> Date: Thu, 19 Apr 2018 13:28:33 -0400
> 
> Glenn Morris wrote:
> 
> > Current master on RHEL7.4.
> 
> Also reproduced on current Debian testing.

If the emacs-26 branch doesn't have this problem, it could be due to
the changes in commit 4ba32858.  Can you try backing out that
changeset?

Thanks.

bug#31214: hang in Faccept_process_output

[Glenn Morris]

Eli Zaretskii wrote:

> If the emacs-26 branch doesn't have this problem

It does.

bug#31214: hang in Faccept_process_output

[Eli Zaretskii]

> From: Glenn Morris <rgm@gnu.org>
> Cc: 31214@debbugs.gnu.org
> Date: Thu, 19 Apr 2018 13:39:45 -0400
> 
> Eli Zaretskii wrote:
> 
> > If the emacs-26 branch doesn't have this problem
> 
> It does.

Then maybe we should pass a non-nil timeout to accept-process-output,
and/or a non-nil last argument.

IOW, I'm not sure the bug is not in the test itself.

bug#31214: hang in Faccept_process_output

[Glenn Morris]

Eli Zaretskii wrote:

> Then maybe we should pass a non-nil timeout to accept-process-output,
> and/or a non-nil last argument.
>
> IOW, I'm not sure the bug is not in the test itself.

Is it not a standard usage (start a process, and while it's live, accept
output from it)?

(Could the process by exiting part-way through accept-process-output?)

bug#31214: hang in Faccept_process_output

[Glenn Morris]

Glenn Morris wrote:

>> Then maybe we should pass a non-nil timeout to accept-process-output,
>> and/or a non-nil last argument.
>>
>> IOW, I'm not sure the bug is not in the test itself.
>
> Is it not a standard usage (start a process, and while it's live, accept
> output from it)?

In fact, doesn't this imply that any use of the form
(accept-process-output process) may hang Emacs forever, which seems bad?

bug#31214: hang in Faccept_process_output

[Eli Zaretskii]

> From: Glenn Morris <rgm@gnu.org>
> Cc: 31214@debbugs.gnu.org
> Date: Thu, 19 Apr 2018 15:48:33 -0400
> 
> Eli Zaretskii wrote:
> 
> > Then maybe we should pass a non-nil timeout to accept-process-output,
> > and/or a non-nil last argument.
> >
> > IOW, I'm not sure the bug is not in the test itself.
> 
> Is it not a standard usage (start a process, and while it's live, accept
> output from it)?

When there are many processes lying around, I wouldn't trust the
defaults to produce the desired effect.  Those arguments are there for
a reason, are they not?

> (Could the process by exiting part-way through accept-process-output?)

Which process?

bug#31214: hang in Faccept_process_output

[Eli Zaretskii]

> From: Glenn Morris <rgm@gnu.org>
> Cc: 31214@debbugs.gnu.org
> Date: Thu, 19 Apr 2018 15:53:52 -0400
> 
> In fact, doesn't this imply that any use of the form
> (accept-process-output process) may hang Emacs forever, which seems bad?

Under very rare conditions, yes.  That's what that commit on master
was about, and we have some evidence that it didn't solve all of the
problem, just a big chunk of it.

bug#31214: hang in Faccept_process_output

[Glenn Morris]

Eli Zaretskii wrote:

>> (Could the process by exiting part-way through accept-process-output?)
>
> Which process?

The 'process' argument in '(accept-process-output process)'.

BTW, this test not infrequently fails on hydra.nixos.org, with the
process output buffer being empty. I have no idea how that is possible.

Eg https://hydra.nixos.org/build/73055559

(equal "" "stdout
stderr
")

bug#31214: hang in Faccept_process_output

[Paul Eggert]

I ran Glenn's test case (Bug#31214#3) and could not reproduce the problem in 
either Ubuntu 18.04 or Fedora 29 (both x86-64), using either the current 
emacs-26 or the current master branch. I tried the test 10,000 times on all 
platforms; the output always consisted of the numbers 1 through 10000, one per 
line, and I typed ^C to exit the test case.

I don't know whether the bug is fixed or my platforms don't tickle the bug. I 
don't see any change to src/process.c that would have obviously fixed the bug.

bug#31214: hang in Faccept_process_output

[Glenn Morris]

I can't reproduce it either any more (now on RHEL 7.6).
Oh well.