bug#45951: 27.1; comint-redirect-results-list-from-process sometimes fails to skip over command if echoed

bug#45951: 27.1; comint-redirect-results-list-from-process sometimes fails to skip over command if echoed

[Richard Sharman]

[-- Attachment #1: Type: text/plain, Size: 6223 bytes --]

When function comint-redirect-results-list-from-process is examining the
output buffer of the command it attempts to skip past the command if it
was echoed.

However, this test uses looking-at and so will fail if the command
contains characters with special meansing to regexps. It can also fail
if the command contains a trailing newline character but the
buffer may have a control-M character.

The latter problem can be fixed by calling comint-carriage-motion before
the test is done; the former by looking-at (regexp-quote command)
instead of just at command.

Note that bug 42662 (27.1; `comint-redirect-results-list-from-process’ 
treats COMMAND as a regexp) is another symptom of the former problem.


Example code to reproduce the problem:

(defun example ()
  (interactive)
  ;; create shell if not already created
  (save-window-excursion
    (shell))
  (let ((p (get-process "shell"))
	command
	reply)
    (process-send-string p "irb\n")
    (setq command "puts File.atime(\"example.el\") if File.exist?(\"example.el\")\n")
    (setq reply (comint-redirect-results-list-from-process p command "^.*$" 0))
    (process-send-string p "quit\n")	;; cleanup!
    (message (car reply))	;; this should be the a time of the file
    reply
    ))

File example.el creates a shell process then runs “irb”, the ruby
interactive shell, then sends it a command with question-marks in it.
The first line returned should be the atime of the file.


Patch:

--- comint.el.orig	2021-01-17 14:22:17.000000000 -0800
+++ comint.el	2021-01-17 16:49:47.000000000 -0800
@@ -3833,9 +3833,11 @@
 		  (accept-process-output process)))
       ;; Collect the output
       (set-buffer output-buffer)
+      (unless comint-inhibit-carriage-motion
+	(comint-carriage-motion (point-min)(point-max)))
       (goto-char (point-min))
       ;; Skip past the command, if it was echoed
-      (and (looking-at command)
+      (and (looking-at (regexp-quote command))
 	   (forward-line))
       (while (and (not (eobp))
 		  (re-search-forward regexp nil t))



Example output in *scratch* buffer before the patch:

(example)
("irb
" "puts File.atime(\"example.el\") if File.exist?(\"example.el\")
" "
" "")

and after patch applied:

(example)
("2021-01-17 18:33:45 -0800" "nil" "")


In GNU Emacs 27.1 (build 2, x86_64-apple-darwin17.7.0, NS appkit-1561.61 Version 10.13.6 (Build 17G14033))
of 2020-12-13 built on henry.local
Windowing system distributor 'Apple', version 10.3.1561
System Description:  Mac OS X 10.13.6

Recent messages:
2021-01-17 18:33:45 -0800
Result: "2021-01-17 18:33:45 -0800"
("2021-01-17 18:33:45 -0800" "nil" "")

Result: ("2021-01-17 18:33:45 -0800" "nil" "")

Result: ("2021-01-17 18:33:45 -0800" "nil" "")

Quit
Type C-x 4 C-o RET to restore the other window.  
Quit
Configured features:
NOTIFY KQUEUE ACL GNUTLS LIBXML2 ZLIB TOOLKIT_SCROLL_BARS NS MODULES
THREADS JSON PDUMPER GMP

Important settings:
  value of $LANG: en_CA.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Shell

Minor modes in effect:
  shell-dirtrack-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug sendmail org-timer org-colview org-clock
org-attach org-id org-archive org-agenda ol-eww ol-rmail ol-mhe ol-irc
ol-info ol-gnus nnir gnus-sum url url-proxy url-privacy url-expand
url-methods url-history mailcap shr url-cookie url-domsuf url-util
url-parse auth-source cl-seq eieio eieio-core cl-macs eieio-loaddefs
json map url-vars svg xml dom browse-url gnus-group gnus-undo gnus-start
gnus-cloud nnimap nnmail mail-source utf7 netrc nnoo parse-time iso8601
gnus-spec gnus-int gnus-range message rmc puny rfc822 mml mml-sec
password-cache epa epg epg-config mm-decode mm-bodies mm-encode
mail-parse rfc2231 mailabbrev gmm-utils mailheader gnus-win gnus
nnheader gnus-util rmail rmail-loaddefs rfc2047 rfc2045 ietf-drums
text-property-search seq byte-opt bytecomp byte-compile cconv mail-utils
mm-util mail-prsvr wid-edit ol-docview doc-view jka-compr image-mode
exif dired dired-loaddefs ol-bibtex bibtex ol-bbdb ol-w3m reporter org
ob ob-tangle ob-ref ob-lob ob-table ob-exp org-macro org-footnote
org-src ob-comint org-pcomplete org-list org-faces org-entities noutline
outline easy-mmode org-version ob-emacs-lisp ob-core ob-eval org-table
ol org-keys org-compat advice org-macs org-loaddefs format-spec cal-menu
calendar cal-loaddefs apropos edebug gv derived time-date subr-x
help-fns radix-tree cl-print debug backtrace help-mode easymenu
find-func cl-loaddefs cl-lib shell pcomplete comint ansi-color ring
tooltip eldoc electric uniquify ediff-hook vc-hooks lisp-float-type
mwheel term/ns-win ns-win ucs-normalize mule-util term/common-win
tool-bar dnd fontset image regexp-opt fringe tabulated-list replace
newcomment text-mode elisp-mode lisp-mode prog-mode register page
tab-bar menu-bar rfn-eshadow isearch timer select scroll-bar mouse
jit-lock font-lock syntax facemenu font-core term/tty-colors frame
minibuffer cl-generic cham georgian utf-8-lang misc-lang vietnamese
tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek
romanian slovak czech european ethiopic indian cyrillic chinese
composite charscript charprop case-table epa-hook jka-cmpr-hook help
simple abbrev obarray cl-preloaded nadvice loaddefs button faces
cus-face macroexp files text-properties overlay sha1 md5 base64 format
env code-pages mule custom widget hashtable-print-readable backquote
threads kqueue cocoa ns multi-tty make-network-process emacs)

Memory information:
((conses 16 179913 11829)
(symbols 48 19738 1)
(strings 32 63869 1510)
(string-bytes 1 2209108)
(vectors 16 29528)
(vector-slots 8 338974 19272)
(floats 8 213 69)
(intervals 56 717 0)
(buffers 1000 19))

[-- Attachment #2: Type: text/html, Size: 11188 bytes --]

bug#45951: 27.1; comint-redirect-results-list-from-process sometimes fails to skip over command if echoed

[Lars Ingebrigtsen]

Richard Sharman <rsharman@pobox.com> writes:

> The latter problem can be fixed by calling comint-carriage-motion before
> the test is done; the former by looking-at (regexp-quote command)
> instead of just at command.

This bit seems to have been fixed in Emacs 28.

> Example code to reproduce the problem:

When testing your example code in Emacs 27, I just get the following
error:

comint-redirect-send-command-to-process: No prompt found or ‘comint-prompt-regexp’ not set properly

Do you have a recipe, starting from "emacs -Q"?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#45951: 27.1; comint-redirect-results-list-from-process sometimes fails to skip over command if echoed

[Lars Ingebrigtsen]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> comint-redirect-send-command-to-process: No prompt found or
> ‘comint-prompt-regexp’ not set properly
>
> Do you have a recipe, starting from "emacs -Q"?

Oh, we have to `M-x shell' manually first, and then we don't get that
error.

But the `M-x example' just infloops for me -- this bit never completes:

      (while (and (null comint-redirect-completed)
		  (accept-process-output process)))


-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#45951: 27.1; comint-redirect-results-list-from-process sometimes fails to skip over command if echoed

[Lars Ingebrigtsen]

(Please keep the debbugs address in the CC header -- otherwise the bug
tracker won't receive the message.  Resent to put it into the tracker.)

Richard Sharman <rsharman@pobox.com> writes:

>> On Jan 18, 2021, at 8:03 PM, Lars Ingebrigtsen <larsi@gnus.org> wrote:
>> 
>> Richard Sharman <rsharman@pobox.com> writes:
>> 
>>> The latter problem can be fixed by calling comint-carriage-motion before
>>> the test is done; the former by looking-at (regexp-quote command)
>>> instead of just at command.
>> 
>> This bit seems to have been fixed in Emacs 28.
>
> Ok, thanks.
>
>> On Jan 18, 2021, at 8:24 PM, Lars Ingebrigtsen <larsi@gnus.org> wrote:
>> 
>> Lars Ingebrigtsen <larsi@gnus.org> writes:
>> 
>>> comint-redirect-send-command-to-process: No prompt found or
>>> ‘comint-prompt-regexp’ not set properly
>>> 
>>> Do you have a recipe, starting from "emacs -Q"?
>> 
>> Oh, we have to `M-x shell' manually first, and then we don't get that
>> error.
>> 
>
> Yes, I think that’s another bug I was going to look into but didn’t.
> After the M-x shell it works.
>
>> But the `M-x example' just infloops for me -- this bit never completes:
>> 
>>      (while (and (null comint-redirect-completed)
>> 		  (accept-process-output process)))
>> 
>
> I saw that once but couldn’t reproduce it. I *think* it happened when
> there was a comint process that had terminated (abnormally perhaps?) I
> tried to see why accept-process-output didn’t return but it was in
> C-code and I didn’t feel up to chasing it.  I think I manually deleted
> the process but I may have restarted emacs; after that I didn’t see it
> happen again.
> …
> I tried running example.el just now from an emacs -Q and apart from
> the “no prompt found” it works fine; I haven’t been able to reproduce
> the problem of accept-process-output not returning.
>
> Richard

bug#45951: 27.1; comint-redirect-results-list-from-process sometimes fails to skip over command if echoed

[Lars Ingebrigtsen]

Lars Ingebrigtsen <larsi@gnus.org> writes:

>> I tried running example.el just now from an emacs -Q and apart from
>> the “no prompt found” it works fine; I haven’t been able to reproduce
>> the problem of accept-process-output not returning.

This was half a year ago, so I'm guessing this now works as it should (I
haven't re-tried the test case), so I'm closing this bug report.  If
this is still a problem in Emacs 28, please respond to the debbugs
address and we'll reopen.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no