gnuplot vs. emacs' compile command

gnuplot vs. emacs' compile command

[Hans-Bernhard Broeker]

Hello to bug-emacs and the gnuplot developers,

as reportedly requested by RMS, here's some more background
information about the problem found by a gnuplot user, Dan Jacobson,
in interacting with emacs.

In the essence, M-x compile apparently has problems coping with the
way gnuplot generates X11 graphs that survive the termination of the
main program (gnuplot -persist).  I'm not quite sure that's even
something the user should have been trying to do, but let's set that
aside for the moment.

The simplest way found so far to reproduce the problem is to have a
shell script file:

--- gpaction ---
gnuplot -persist <<EOS
 plot x
EOS
---- end ---

This creates a minimalistic plot in an X11 output window.  Now,
gnuplot's X11 output is managed by a separate driver program,
gnuplot_x11, which gives us the opportunity of the "-persist" option.
If set, the gnuplot main process will terminate itself, but the child
process executing gnuplot_x11 will keep running and maintain the
window content.

Different ways of invoking this script yield rather different results:

sh gpaction                                 # does the plot and returns
emacs -eval '(compile "sh gpaction")'       # doesn't plot, but returns
emacs -eval '(shell-command "sh gpaction")' # plots, but then hangs

'emacs' was GNU emacs-20.4.1 --- but the problem seems to happen with
other versions, too.  The middle case, using "compile", is what
originally triggered this investigation.  "Returns" here means that
the compilation buffer reports the compile as "finished with exit[0]".

In the latter case, "hangs" means that emacs is unresponsive --- no
reaction to mouse klicks to the menu, nor to key presses into the
*scratch* buffer that opens.  You can't show the buffer menu or
anything.  It's apparently waiting for something to happen, which
doesn't.  Only after you terminate the gnuplot graph window (type 'q'
into it), or press Ctrl-G in emacs to forcibly break out of the wait
loop, it will continue.

During that pause, a look at the output from 'ps jx' shows that
gnuplot_x11 is an orphan --- i.e. it's the only surviving process in
its process group.  In "shell-command", emacs apparently doesn't kill
this orphan until you explicitly tell it to.  In "compile", it seems
to manage killing it quite fine --- so successfully in fact, that it
kills gnuplot_x11 before it ever gets a chance to display its window.

Now, part of this may be pilot error, but at least the completely
unresponsive emacs session in the case of shell-command is something
I'm worried about, too.  Whether it qualifies as a bug, I'm not fully
sure --- you decide.

-- 
Hans-Bernhard Broeker (broeker@physik.rwth-aachen.de)
Even if all the snow were burnt, ashes would remain.

Re: gnuplot vs. emacs' compile command

[David Kastrup]

Hans-Bernhard Broeker <broeker@physik.rwth-aachen.de> writes:

> Hello to bug-emacs and the gnuplot developers,
> 
> as reportedly requested by RMS, here's some more background
> information about the problem found by a gnuplot user, Dan Jacobson,
> in interacting with emacs.
> 
> In the essence, M-x compile apparently has problems coping with the
> way gnuplot generates X11 graphs that survive the termination of the
> main program (gnuplot -persist).  I'm not quite sure that's even
> something the user should have been trying to do, but let's set that
> aside for the moment.
> 
> The simplest way found so far to reproduce the problem is to have a
> shell script file:
> 
> --- gpaction ---
> gnuplot -persist <<EOS
>  plot x
> EOS
> ---- end ---

Another possibility is something like putting

  xdvi test &

into such a file.  Now the problem is that if we kill an Emacs
process, we don't want anybody writing uncontrolledly around in it,
but closing our end of the I/O should be sufficient.  If there are
too many processes detaching themselves improperly, we might run out
of ptys at some time.

The only asynchronous processes that Emacs at the moment can leave
alone safely are those started with call-process and a buffer
argument of 0.

Definitely a nuisance.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: gnuplot vs. emacs' compile command

[Richard Stallman]

   emacs -eval '(compile "sh gpaction")'       # doesn't plot, but returns

M-x compile kills the whole process group when the compiler exits.
This is because it needs to deallocate the pty.  In situations like
this, the result is undesirable, but I don't see what change we could
reasonably make.,

   emacs -eval '(shell-command "sh gpaction")' # plots, but then hangs

This uses call-process.  I think that waits until the output pipe
returns EOF, which I think requires it to get closed by all writers.
This might be a bug, but I am not sure it is a bug.

If gnuplot were to give the subprocess a different standard output
descriptor, Emacs would not wait for it to terminate.

Re: gnuplot vs. emacs' compile command

[Hans-Bernhard Broeker]

On Mon, 19 May 2003, Richard Stallman wrote:

>    emacs -eval '(shell-command "sh gpaction")' # plots, but then hangs
>
> This uses call-process.  I think that waits until the output pipe
> returns EOF, which I think requires it to get closed by all writers.
> This might be a bug, but I am not sure it is a bug.
>
> If gnuplot were to give the subprocess a different standard output
> descriptor, Emacs would not wait for it to terminate.

Yep, gnuplot_x11 is keeping its stderr open (stdin and stdout are
redirected by gnuplot), which remains to be the stderr of gnuplot
itself, and thus the one emacs passed to the shell, or directly to
gnuplot.

I'll see if it helps to have gnuplot_x11 fclose(stderr) as it
enters the -persist state.

-- 
Hans-Bernhard Broeker (broeker@physik.rwth-aachen.de)
Even if all the snow were burnt, ashes would remain.

Re: gnuplot vs. emacs' compile command

[Andreas Schwab]

Hans-Bernhard Broeker <broeker@physik.rwth-aachen.de> writes:

|> I'll see if it helps to have gnuplot_x11 fclose(stderr) as it
|> enters the -persist state.

freopen to /dev/null might be preferrable, so that stderr remains valid.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux AG, Deutschherrnstr. 15-19, D-90429 Nürnberg
Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: gnuplot vs. emacs' compile command

[Hans-Bernhard Broeker]

On Tue, 20 May 2003, Andreas Schwab wrote:

> Hans-Bernhard Broeker <broeker@physik.rwth-aachen.de> writes:
>
> |> I'll see if it helps to have gnuplot_x11 fclose(stderr) as it
> |> enters the -persist state.
>
> freopen to /dev/null might be preferrable, so that stderr remains valid.

Yep, that seems like a better plan --- gnuplot_x11 can be built to
optionally output execution trace information to stderr, which had better
not break completely.

I tried both, and both apparently do the trick.  Unless someone on the
gnuplot development list has an objection, this change to gnuplot will
be checked in.


-- 
Hans-Bernhard Broeker (broeker@physik.rwth-aachen.de)
Even if all the snow were burnt, ashes would remain.

Re: gnuplot vs. emacs' compile command

[Hans-Bernhard Broeker]

Hans-Bernhard Broeker wrote:
> On Tue, 20 May 2003, Andreas Schwab wrote:

> > freopen to /dev/null might be preferrable, so that stderr remains valid.

> Yep, that seems like a better plan --- gnuplot_x11 can be built to
> optionally output execution trace information to stderr, which had better
> not break completely.

That change has been checked into gnuplot, so for the case of
shell-command, the
gnuplot end of the problem should be fixed.  It remains somewhat
worrying, though, that Emacs would be so irresponsive while waiting for
the subprocess to end --- not that many newbies would have known to try
Ctrl-G to break it out of its busy-looping state.

-- 
Hans-Bernhard Broeker (broeker@physik.rwth-aachen.de)
Even if all the snow were burnt, ashes would remain.