bug#6202: Async shell command, switching mode, and sleep-for

bug#6202: Async shell command, switching mode, and sleep-for

[Arni Magnusson]

I maintain a major mode, where some functions have broken between Emacs 
23.1 and 23.2, and I believe the current Emacs behavior could be described 
as a bug.

What I want to do is to run a shell command asynchronously (so I can watch 
the output scroll by in a secondary window while continuing my work), but 
the output should not be syntax-highlighted. By default, the output is 
shown in `shell-mode', where negative numbers are highlighted in 
`font-lock-comment-face',

   123 -456 789

but I want these three numbers to be shown in the `default' face.

My approach is to switch the secondary window to `fundamental-mode' to 
ensure that nothing is syntax-highlighted. The following example uses the 
shell command "set", because it produces some output Linux and Windows, 
although it does not demonstrate negative numbers:

(defun foo ()
   (interactive)
   (shell-command "set &")
   (select-window (get-buffer-window "*Async Shell Command*"))
   ;; (sleep-for 3)
   (fundamental-mode)
   (other-window 1))

My window and buffer control is slightly more sophisticated, but the 
function above focuses on the problem. This function works in Emacs 23.1, 
but in Emacs 23.2 the secondary window is empty, and the shell command 
output is not found in any buffer. The same problem occurs when switching 
to some other mode than `fundamental-mode'. The same problem also occurs 
when replacing the ampersand `shell-command' with the new 
`async-shell-command'.

While debugging the problem, I found that the function does work in Emacs 
23.2 if `sleep-for' is called before switching to `fundamental-mode', so 
the foo function above can be "fixed" by uncommenting that line. Sleeping 
for 0.001 seconds is not enough, though.

I also stumbled upon a separate issue, where the above function (after 
uncommenting the `sleep-for' line) makes Emacs dutifully freeze for 3 
seconds in Linux, but not in Windows. This bug/limitation is not mentioned 
on the help page for `sleep-for'.

Thanks in advance for looking at these issues,

Arni

P.S. It would of course be nice if the `*-shell-command' functions would 
have an optional argument to specify a mode other than the currently 
hardwired `shell-mode'.

bug#6202: Async shell command, switching mode, and sleep-for

[Stefan Monnier]

> My approach is to switch the secondary window to `fundamental-mode' to
> ensure that nothing is syntax-highlighted. The following example uses the
> shell command "set", because it produces some output Linux and Windows,
> although it does not demonstrate negative numbers:

> (defun foo ()
>   (interactive)
>   (shell-command "set &")
>   (select-window (get-buffer-window "*Async Shell Command*"))
>   ;; (sleep-for 3)
>   (fundamental-mode)
>   (other-window 1))

> My window and buffer control is slightly more sophisticated, but the
> function above focuses on the problem. This function works in Emacs 23.1,
> but in Emacs 23.2 the secondary window is empty, and the shell command
> output is not found in any buffer. The same problem occurs when switching to
> some other mode than `fundamental-mode'. The same problem also occurs when
> replacing the ampersand `shell-command' with the new `async-shell-command'.

> While debugging the problem, I found that the function does work in Emacs
> 23.2 if `sleep-for' is called before switching to `fundamental-mode', so the
> foo function above can be "fixed" by uncommenting that line. Sleeping for
> 0.001 seconds is not enough, though.

This does look like a bug, indeed, and I can't begin to imagine what the
problem might be (and don't have time to investigate now).

> P.S. It would of course be nice if the `*-shell-command' functions would
> have an optional argument to specify a mode other than the currently
> hardwired `shell-mode'.

I strongly recommend to stay away from shell-command and friends in Lisp
code (they're mostly used and meant for interactive use), and use
start-process instead.


        Stefan

bug#6202: Async shell command, switching mode, and sleep-for

[Štěpán Němec]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> I strongly recommend to stay away from shell-command and friends in Lisp
> code (they're mostly used and meant for interactive use), and use
> start-process instead.

Wouldn't it make sense to have such a strong recommendation mentioned
somewhere user-visible, such as in the relevant commands' docstrings?

Štěpán

bug#6202: Async shell command, switching mode, and sleep-for

[Stefan Monnier]

>> I strongly recommend to stay away from shell-command and friends in Lisp
>> code (they're mostly used and meant for interactive use), and use
>> start-process instead.
> Wouldn't it make sense to have such a strong recommendation mentioned
> somewhere user-visible, such as in the relevant commands' docstrings?

Yes, maybe that would make sense.  We do that for some other
commands already.  But note that for shell-command it's not nearly as
much of a problem, for the reasons you mentioned: it has a lot of stuff
built-in that annoys Elisp uses so programmers tend to find
start-process on their own out of necessity.

This is in contrast to things like next-line, which appear to do the
right thing for a naive Lisp programmer and will only produce unexpected
results in "unusual" cases.


        Stefan

bug#6202: Async shell command, switching mode, and sleep-for

[npostavs]

tags 6202 notabug
quit

Still happens with 25.0.95

Got this error:

Debugger entered--Lisp error: (wrong-type-argument markerp nil)
  comint-output-filter(#<process Shell> "!;=;\\\nACLOCAL_PATH=C[...]")

Edebbugging comint-output-filter I found problem happens at

        (set-marker comint-last-output-start (point))

comint-last-output-start is set (buffer-locally) in comint-mode,
switching to fundamental-mode kills this variable (and all other
buffer-local variables).  Adding the sleep-for allows the subprocess to
finish before this happens.

I think this is not a bug, the foo command can be fixed by doing:

(defun foo ()
  (interactive)
  (shell-command "set &")
  (with-current-buffer (get-buffer "*Async Shell Command*")
    (comint-mode)))

Arni Magnusson <arnima@hafro.is> writes:
>
> I also stumbled upon a separate issue, where the above function (after
> uncommenting the `sleep-for' line) makes Emacs dutifully freeze for 3
> seconds in Linux, but not in Windows. This bug/limitation is not
> mentioned on the help page for `sleep-for'.

I noticed (sleep-for 3) did not pause in Windows with Emacs 24.5, but
did pause with 25.0.95, so it seems this problem has been fixed.

And regarding use of call-process instead (mentioned elsewhere in this
bug thread), current docstring of shell-command does have

    In Elisp, you will often be better served by calling `call-process'
    or `start-process' directly, since it offers more control and does
    not impose the use of a shell (with its need to quote arguments).

bug#6202: Async shell command, switching mode, and sleep-for

[Eli Zaretskii]

> From: npostavs@users.sourceforge.net
> Date: Wed, 06 Jul 2016 20:01:10 -0400
> Cc: 6202@debbugs.gnu.org
> 
> I think this is not a bug, the foo command can be fixed by doing:
> 
> (defun foo ()
>   (interactive)
>   (shell-command "set &")
>   (with-current-buffer (get-buffer "*Async Shell Command*")
>     (comint-mode)))

I agree.  In general, programs that assume some specific timing
immediately after an async subprocess is launched are fragile by
definition, and prone to breakage due to changes in Emacs internals.

> Arni Magnusson <address@hidden> writes:
> >
> > I also stumbled upon a separate issue, where the above function (after
> > uncommenting the `sleep-for' line) makes Emacs dutifully freeze for 3
> > seconds in Linux, but not in Windows. This bug/limitation is not
> > mentioned on the help page for `sleep-for'.
> 
> I noticed (sleep-for 3) did not pause in Windows with Emacs 24.5, but
> did pause with 25.0.95, so it seems this problem has been fixed.

Yes, see bug #20935.

Thanks.