Re: Shell-mode: File completion breaks editing previous commands

Re: Shell-mode: File completion breaks editing previous commands

[Chong Yidong]

richard@rswheeldon.com writes:

>>> Emacs shell mode allows editing of previous commands. These can then
>>> be executed by pressing return. Shell mode also allows tab completion
>>> to expand filenames. These do not work together. If we try to modify
>>> a previously executed command (to re-execute it with different params)
>>> and attempt to use filename completion, the beginning of the command
>>> is lost.
>>>
>>> touch foo bar
>>> echo bar
>>> Edit the echo command to "touch foo bar" and press return.
>>>
>>> produces the following output:
>>>
>>> % touch foo bar
>>> % echo bar foo 
>>> foo
>>> % foo 
>>> bash: ./foo: Permission denied
>>> % 
>>
>> I can't reproduce this on either Emacs 22 or Emacs 23.  Please provide a
>> more precise recipe, include all keystrokes beginning from emacs -Q.
>
> Recent input:
> <escape> x s h e l l <return> c d SPC / t m p <return> 
> t o u c h SPC f o o <return> e <backspace> t o u c 
> h SPC b a r <return> e <backspace> l s SPC f o <tab> 
> <return> <up> <up> <right> <right> <right> b <tab> 
> a <tab> <return> <escape> x r e p <tab> r <backspace> 
> o r <tab> <return>
>
> Sorry, I can't see how I can make this much simpler.

I still can't reproduce this problem.  Does anyone on emacs-devel see
the bug?

Re: Shell-mode: File completion breaks editing previous commands

[Andreas Röhler]

Am Montag, 7. April 2008 07:10 schrieb Chong Yidong:
> richard@rswheeldon.com writes:
> 
> >>> Emacs shell mode allows editing of previous commands. These can then
> >>> be executed by pressing return. Shell mode also allows tab completion
> >>> to expand filenames. These do not work together. If we try to modify
> >>> a previously executed command (to re-execute it with different params)
> >>> and attempt to use filename completion, the beginning of the command
> >>> is lost.
> >>>
> >>> touch foo bar
> >>> echo bar
> >>> Edit the echo command to "touch foo bar" and press return.
> >>>
> >>> produces the following output:
> >>>
> >>> % touch foo bar
> >>> % echo bar foo 
> >>> foo
> >>> % foo 
> >>> bash: ./foo: Permission denied
> >>> % 
> >>
> >> I can't reproduce this on either Emacs 22 or Emacs 23.  Please provide a
> >> more precise recipe, include all keystrokes beginning from emacs -Q.
> >
> > Recent input:
> > <escape> x s h e l l <return> c d SPC / t m p <return> 
> > t o u c h SPC f o o <return> e <backspace> t o u c 
> > h SPC b a r <return> e <backspace> l s SPC f o <tab> 
> > <return> <up> <up> <right> <right> <right> b <tab> 
> > a <tab> <return> <escape> x r e p <tab> r <backspace> 
> > o r <tab> <return>
> >
> > Sorry, I can't see how I can make this much simpler.
> 
> I still can't reproduce this problem.  Does anyone on emacs-devel see
> the bug?
> 
> 
> 

AFAIU that has nothing to do with emacs, but with
file-permissions: `foo' isn't executable, "x" it wasn't set.

Andreas Röhler

Re: Shell-mode: File completion breaks editing previous commands

[Bob Rogers]

   From: Chong Yidong <cyd@stupidchicken.com>
   Date: Mon, 07 Apr 2008 01:10:04 -0400

   richard@rswheeldon.com writes:

   >>> Emacs shell mode allows editing of previous commands. These can then
   >>> be executed by pressing return. Shell mode also allows tab completion
   >>> to expand filenames. These do not work together. If we try to modify
   >>> a previously executed command (to re-execute it with different params)
   >>> and attempt to use filename completion, the beginning of the command
   >>> is lost.
   >>>
   >>> touch foo bar
   >>> echo bar
   >>> Edit the echo command to "touch foo bar" and press return.
   >>>
   >>> produces the following output:
   >>>
   >>> % touch foo bar
   >>> % echo bar foo 
   >>> foo
   >>> % foo 
   >>> bash: ./foo: Permission denied
   >>> % 
   >>
   >> I can't reproduce this on either Emacs 22 or Emacs 23.  Please provide a
   >> more precise recipe, include all keystrokes beginning from emacs -Q.
   >
   > Recent input:
   > <escape> x s h e l l <return> c d SPC / t m p <return> 
   > t o u c h SPC f o o <return> e <backspace> t o u c 
   > h SPC b a r <return> e <backspace> l s SPC f o <tab> 
   > <return> <up> <up> <right> <right> <right> b <tab> 
   > a <tab> <return> <escape> x r e p <tab> r <backspace> 
   > o r <tab> <return>
   >
   > Sorry, I can't see how I can make this much simpler.

   I still can't reproduce this problem.  Does anyone on emacs-devel see
   the bug?

I have seen this intermittently for a long time now, but still haven't
been able to reproduce it reliably.

					-- Bob Rogers
					   http://rgrjr.dyndns.org/

bug#114: Shell-mode: File completion breaks editing previous commands

[Lars Magne Ingebrigtsen]

Bob Rogers <rogers-emacs@rgrjr.dyndns.org> writes:

>    >>> Emacs shell mode allows editing of previous commands. These can then
>    >>> be executed by pressing return. Shell mode also allows tab completion
>    >>> to expand filenames. These do not work together. If we try to modify
>    >>> a previously executed command (to re-execute it with different params)
>    >>> and attempt to use filename completion, the beginning of the command
>    >>> is lost.

[...]

> I have seen this intermittently for a long time now, but still haven't
> been able to reproduce it reliably.

Is this still a problem?

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

bug#114: Shell-mode: File completion breaks editing previous commands

[Lars Magne Ingebrigtsen]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

>> I have seen this intermittently for a long time now, but still haven't
>> been able to reproduce it reliably.
>
> Is this still a problem?

More information, but was apparently not given, so I'm closing this bug
report.  If this is still a problem, please reopen the bug report.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

bug#114: Shell-mode: File completion breaks editing previous commands

[richard]

Lars Magne Ingebrigtsen writes:
 > Lars Magne Ingebrigtsen <larsi@gnus.org> writes:
 > >> I have seen this intermittently for a long time now, but still haven't
 > >> been able to reproduce it reliably.
 > >
 > > Is this still a problem?
 > 
 > More information, but was apparently not given, so I'm closing this bug
 > report.  If this is still a problem, please reopen the bug report.

This is still broken in Emacs 23 at least. I'm happy to provide any extra
info you need,

Richard

GNU Emacs 23.2.1 (x86_64-slackware-linux-gnu, GTK+ Version 2.18.9) of 2010-05-08 on midas64

bug#114: Shell-mode: File completion breaks editing previous commands

[Lars Magne Ingebrigtsen]

richard@rswheeldon.com writes:

> This is still broken in Emacs 23 at least.

Ok; I'll reopen the report.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

bug#114: Shell-mode: File completion breaks editing previous commands

[Glenn Morris]

For anyone like me who had trouble following the recipe:

emacs -Q -f shell
touch foo bar RET
echo foo RET
C-u 2 C-p
C-u 5 C-f
 ; point now on "f" of "echo foo"
baTAB
 ; completes so that line reads "echo bar foo", with cursor on "f".
RET
 ; -> tries to run the command "foo"


If you type "bar " by hand rather than using tab completion, there is no
problem.

bug#114: Shell-mode: File completion breaks editing previous commands

[Stefan Monnier]

> For anyone like me who had trouble following the recipe:
> emacs -Q -f shell
> touch foo bar RET
> echo foo RET
> C-u 2 C-p
> C-u 5 C-f
>  ; point now on "f" of "echo foo"
> baTAB
>  ; completes so that line reads "echo bar foo", with cursor on "f".
> RET
>  ; -> tries to run the command "foo"
> If you type "bar " by hand rather than using tab completion, there is no
> problem.

Ah, I guess there's some funny `field' text property going on.


        Stefan

bug#114: Shell-mode: File completion breaks editing previous commands

[Glenn Morris]

Stefan Monnier wrote:

> Ah, I guess there's some funny `field' text property going on.

I think you guess correctly. Looks like the completion does not put an
`input' property on the text that it inserts.

bug#114: Shell-mode: File completion breaks editing previous commands

[Stefan Monnier]

[ Hi, Miles, I think you're the "fields in comint" guru; coud you give
  us a hand? ]

>> For anyone like me who had trouble following the recipe:
>> emacs -Q -f shell
>> touch foo bar RET
>> echo foo RET
>> C-u 2 C-p
>> C-u 5 C-f
>> ; point now on "f" of "echo foo"
>> baTAB
>> ; completes so that line reads "echo bar foo", with cursor on "f".
>> RET
>> ; -> tries to run the command "foo"
>> If you type "bar " by hand rather than using tab completion, there is no
>> problem.

> Ah, I guess there's some funny `field' text property going on.

Indeed, the problem is that previous commands are given a `field' text
property of value `input', but since this is a text property, insertion
of text within this line won't have that property unless it's done via
insert-and-inherit.
This can be seen as above with completion, but it also happens with C-y.

Maybe the best fix is to get rid of this `input' field value (the rest
of the text already gets an `output' field value, so a nil value would
still work as a field).

E.g. the patch below seemed to fix the problem, but I'm not sure what
other consequences it might have.


        Stefan


=== modified file 'lisp/comint.el'
--- lisp/comint.el	2011-10-03 16:49:56 +0000
+++ lisp/comint.el	2011-10-04 02:05:59 +0000
@@ -849,8 +849,7 @@
       (and (< pos (field-end pos))
            (setq field (field-at-pos pos))
 	   (setq input (field-string-no-properties pos))))
-    (if (or (null comint-accum-marker)
-	    (not (eq field 'input)))
+    (if (or (null comint-accum-marker) field)
 	;; Fall back to the global definition if (i) the selected
 	;; buffer is not a comint buffer (which can happen if a
 	;; non-comint window was selected and we clicked in a comint
@@ -1803,8 +1802,7 @@
               (add-text-properties
                beg end
                '(mouse-face highlight
-                 help-echo "mouse-2: insert after prompt as new input"
-                 field input))))
+                 help-echo "mouse-2: insert after prompt as new input"))))
           (unless (or no-newline comint-use-prompt-regexp)
             ;; Cover the terminating newline
             (add-text-properties end (1+ end)
@@ -2153,7 +2151,7 @@
 the current line with any initial string matching the regexp
 `comint-prompt-regexp' removed."
   (let ((bof (field-beginning)))
-    (if (eq (get-char-property bof 'field) 'input)
+    (if (null (get-char-property bof 'field)) ;Not `output'.
 	(field-string-no-properties bof)
       (comint-bol)
       (buffer-substring-no-properties (point) (line-end-position)))))
@@ -2473,7 +2471,7 @@
 	      (while (/= n 0)
 		(unless (re-search-backward regexp nil t dir)
 		  (error "Not found"))
-		(when (eq (get-char-property (point) 'field) 'input)
+		(unless (get-char-property (point) 'field)
 		  (setq n (- n dir))))
 	      (field-beginning))))
       (goto-char pos))))
@@ -2520,7 +2518,7 @@
 		 (setq input-pos (point-max)))
 	       ;; stop iterating
 	       (setq n 0))
-	      ((eq (get-char-property pos 'field) 'input)
+	      ((null (get-char-property pos 'field))
 	       (setq n (if (< n 0) (1+ n) (1- n)))
 	       (setq input-pos pos))))
       (when input-pos

bug#114: Shell-mode: File completion breaks editing previous commands

[Glenn Morris]

Stefan Monnier wrote:

> Maybe the best fix is to get rid of this `input' field value (the rest
> of the text already gets an `output' field value, so a nil value would
> still work as a field).
>
> E.g. the patch below seemed to fix the problem, but I'm not sure what
> other consequences it might have.

For one, it leaves the help echo ("mouse-2: insert after prompt as new
input") and mouse highlight properties broken in two on either side of
the portion inserted by completion.

bug#114: Shell-mode: File completion breaks editing previous commands

[Stefan Monnier]

>> Maybe the best fix is to get rid of this `input' field value (the rest
>> of the text already gets an `output' field value, so a nil value would
>> still work as a field).
>> E.g. the patch below seemed to fix the problem, but I'm not sure what
>> other consequences it might have.

> For one, it leaves the help echo ("mouse-2: insert after prompt as new
> input") and mouse highlight properties broken in two on either side of
> the portion inserted by completion.

But that's also the case before my patch, right?  So it's not
a consequence of my patch.


        Stefan

Shell-mode: File completion breaks editing previous commands

[richard]

Emacs shell mode allows editing of previous commands. These can then
be executed by pressing return. Shell mode also allows tab completion
to expand filenames. These do not work together. If we try to modify
a previously executed command (to re-execute it with different params)
and attempt to use filename completion, the beginning of the command
is lost.

For example, the following sequence:

touch foo bar
echo bar
Edit the echo command to "touch foo bar" and press return.

produces the following output:

% touch foo bar
% echo bar foo 
foo
% foo 
bash: ./foo: Permission denied
% 

This behaviour worked under Emacs 21.

Richard


In GNU Emacs 22.2.1 (i686-pc-linux-gnu, X toolkit, Xaw3d scroll bars)
 of 2008-03-31 on belgrano
Windowing system distributor `The X.Org Foundation', version 11.0.10300000
configured using `configure  '--with-x' '--without-sound''

Important settings:
  value of $LC_ALL: nil
  value of $LC_COLLATE: C
  value of $LC_CTYPE: en_GB.UTF-8
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: nil
  value of $LANG: en_US
  locale-coding-system: utf-8
  default-enable-multibyte-characters: t

Major mode: Shell

Minor modes in effect:
  shell-dirtrack-mode: t
  delete-selection-mode: t
  pc-selection-mode: t
  tooltip-mode: t
  mouse-wheel-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  unify-8859-on-encoding-mode: t
  utf-translate-cjk-mode: t
  auto-compression-mode: t
  column-number-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent input:

SPC SPC SPC SPC SPC SPC SPC <escape> x s h e l l <return> 
t o u c h SPC f o o b a <backspace> <backspace> SPC 
b a r <return> e c h o SPC f o <tab> <return> <up> 
<up> <right> <right> <right> <right> <right> b a <tab> 
<return> <escape> x <up> <up> <return>

Recent messages:
Completed
Completing file name...
Completed
Making completion list... [2 times]
Loading emacsbug...done
Mark set [2 times]
Completing file name...
Completed
Completing file name...
Completed

bug#114: Shell-mode: File completion breaks editing previous commands

[Glenn Morris]

Stefan Monnier wrote:

> But that's also the case before my patch, right?  So it's not
> a consequence of my patch.

OK then, pretend I said:

I'm also not sure what consequences simply removing this field property
will have.

However, I noticed that your patch leaves the help-echo and mouse-face
properties missing on the completed text, which is a little unaesthetic.
An alternative solution that avoids this issue would be if callers of
completion could optionally request that it insert-and-inherit the
completed text rather than just insert it. I could imagine that other
callers of completion might need such a feature. But I have no idea if
this is feasible.

bug#114: Shell-mode: File completion breaks editing previous commands

[Stefan Monnier]

>> But that's also the case before my patch, right?  So it's not
>> a consequence of my patch.

> OK then, pretend I said:

> I'm also not sure what consequences simply removing this field property
> will have.

> However, I noticed that your patch leaves the help-echo and mouse-face
> properties missing on the completed text, which is a little unaesthetic.
> An alternative solution that avoids this issue would be if callers of
> completion could optionally request that it insert-and-inherit the
> completed text rather than just insert it. I could imagine that other
> callers of completion might need such a feature. But I have no idea if
> this is feasible.

I think there are two issues:
1- handling the `insert' case as well as possible, because a lot of code
   uses just `insert' (tho in practice very few cases seem to show up:
   yank, completion, any other?).
2- changing (some) users of `insert' such as completion to use
   insert-and-inherit.

(2) might be good, indeed, tho I'm not completely sure what are the consequences.


        Stefan

bug#114: Shell-mode: File completion breaks editing previous commands

[Stefan Monnier]

> I think there are two issues:
> 1- handling the `insert' case as well as possible, because a lot of code
>    uses just `insert' (tho in practice very few cases seem to show up:
>    yank, completion, any other?).

I've installed the patch that makes comint use nil rather than `input'
for the field property.

> 2- changing (some) users of `insert' such as completion to use
>    insert-and-inherit.

And I've also installed a patch that makes completion use
insert-and-inherit for good measure.


        Stefan

Re: Shell-mode: File completion breaks editing previous commands

[Devon Sean McCullough]

Text properties break shell mode by making motion commands erratic
and unpredictable, so all comint modes are now worse than useless.

BOLD rendered as upper case for illustration ...

$ LS BAr FOO
foo
$ FOO
bash: ./foo: Permission denied
$ HISTORY
...  ls foo
...  foo
...  history
$

... your command got left-truncated, a variation on the usual lossage
where return sends only command characters up to the next font change,
then sends a newline, the remaining characters up to end of line are
not sent, a potentially catastrophic failure.

		Peace
			--Devon
	 /~\
	 \ /	Health Care
	  X 	not warfare
	 / \

	Dubya slime won the digital vote
	Kerry slime won the popular vote

PS: Easier to show than tell so here's a movie,
use EXTREME CAUTION and read before you run,
it types commands at your shell!

(require 'cl)

(defun unread-string (string)
  (dolist (c (reverse (string-to-list string)))
    (push c unread-command-events)))

(defun demo (&rest args)
  (let ((delay 0))
    (dolist (keys args)
      (run-at-time delay nil 'unread-string keys)
      (incf delay 2))))

(demo "\exshell\r"
      "\e>\^C\^C"
      "\^Q\^C\r"
      "PS1='\\$ '\r"
      "mkdir /tmp/junk\r"
      "cd /tmp/junk\r"
      "touch foo bar\r"
      "echo bar\r"
      "\^P\^P" "\^F\^F" "\^F\^F"
      " f\t"
      "\r"
      "\^Roo  "
      "\r"
      "\r"
      "history 9\r"
      "# That's all folks!")