bug#15385: [Eshell] Directory completion overwrites preceding characters

bug#15385: [Eshell] Directory completion overwrites preceding characters

[Joost Kremers]

This bug report will be sent to the Bug-GNU-Emacs mailing list
and the GNU bug tracker at debbugs.gnu.org.  Please check that
the From: line contains a valid email address.  After a delay of up
to one day, you should receive an acknowledgment at that address.

Please write in English if possible, as the Emacs maintainers
usually do not have translators for other languages.

Please describe exactly what actions triggered the bug, and
the precise symptoms of the bug.  If you can, give a recipe
starting from `emacs -Q':

==============================

Starting from `emacs -Q', if I do

   (setq eshell-cmpl-ignore-case t)

and then start up a shell, TAB completion behaves incorrectly under some
conditions.

Specifically, I type a command such as:

    $ cp ~/Dropbox/Work/Reading/Wilbur_Martinez2002.pdf ~/Dro

At this point, I hit TAB to complete `~/Dro' and get:

    $ cp ~/Dropbox/Work/Reading/Wilbur_Martine/home/joost/Dropbox/

Note that the problem does *not* occur with the first file path. If I
hit TAB to complete

    $ cp ~/Dro

then completion produces the correct result:

    $ cp ~/Dropbox/

It seems the problem occurs when the file path is the second argument on
the command line, because if I type:

    $ ls ~/Dropbox/Work/Literature/k/kim2002.pdf ~/dro

and hit TAB to complete, the result is also:

    $ ls ~/Dropbox/Work/Literature/k/ki/home/joost/Dropbox/

It does not occur when there is a switch preceding the file path:

    $ ls -l ~/dro

completes fine.

Thanks,

Joost Kremers

==============================



If Emacs crashed, and you have the Emacs process in the gdb debugger,
please include the output from the following gdb commands:
    `bt full' and `xbacktrace'.
For information about debugging Emacs, please read the file
/usr/share/emacs/24.3/etc/DEBUG.


In GNU Emacs 24.3.1 (i686-pc-linux-gnu, GTK+ Version 3.4.2)
 of 2013-08-22 on chindi02, modified by Debian
Windowing system distributor `The X.Org Foundation', version 11.0.11103000
System Description:	elementary OS Luna

Configured using:
 `configure '--build' 'i686-linux-gnu' '--build' 'i686-linux-gnu'
 '--prefix=/usr' '--sharedstatedir=/var/lib' '--libexecdir=/usr/lib'
 '--localstatedir=/var/lib' '--infodir=/usr/share/info'
 '--mandir=/usr/share/man' '--with-pop=yes'
 '--enable-locallisppath=/etc/emacs24:/etc/emacs:/usr/local/share/emacs/24.3/site-lisp:/usr/local/share/emacs/site-lisp:/usr/share/emacs/24.3/site-lisp:/usr/share/emacs/site-lisp'
 '--with-crt-dir=/usr/lib/i386-linux-gnu' '--with-x=yes'
 '--with-x-toolkit=gtk3' '--with-toolkit-scroll-bars'
 'build_alias=i686-linux-gnu' 'CFLAGS=-g -O2 -fstack-protector
 --param=ssp-buffer-size=4 -Wformat -Wformat-security
 -Werror=format-security -Wall' 'LDFLAGS=-Wl,-Bsymbolic-functions
 -Wl,-z,relro' 'CPPFLAGS=-D_FORTIFY_SOURCE=2''

Important settings:
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix
  default enable-multibyte-characters: t

Major mode: EShell

Minor modes in effect:
  tooltip-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent input:
( s e t q SPC c m <backspace> <backspace> e s h e l 
l - c m p l - i g n o r e - c a s e SPC t ) M-x e s 
h e l l <return> c d SPC d r o <tab> <backspace> <backspace> 
<backspace> D r o <tab> W o r <tab> R e <tab> <return> 
c d SPC . . / R e a <tab> <return> c p SPC W i l <tab> 
<tab> SPC ~ / D r o <tab> <M-backspace> D r o <C-tab> 
C-x b <return> C-x C-e C-x b <return> <tab> <M-backspace> 
d r o <tab> <M-backspace> <M-backspace> <M-backspace> 
<M-backspace> <M-backspace> e x i t <return> M-x e 
s h e l l <return> c d SPC d r o <tab> r e a <tab> 
<backspace> <backspace> <backspace> w o r <tab> r e 
a <tab> <return> c p SPC w i l b <tab> ~ / r <backspace> 
d r o <tab> <M-backspace> <M-backspace> <M-backspace> 
<M-backspace> <M-backspace> <M-backspace> c d <return> 
c d SPC ~ / d r o <tab> w o <tab> M-x r e p o r <tab> 
<return>

Recent messages:
Loading em-hist...done
Loading em-ls...done
Loading em-prompt...done
Loading em-script...done
Loading em-term...done
Loading em-unix...done
No completions of dro
t
No completions of /home/joost/dro
No completions of Dropbox/rea

Load-path shadows:
None found.

Features:
(shadow sort gnus-util mail-extr emacsbug message format-spec rfc822 mml
mml-sec mm-decode mm-bodies mm-encode mail-parse rfc2231 mailabbrev
gmm-utils mailheader sendmail rfc2047 rfc2045 ietf-drums mm-util
mail-prsvr mail-utils pcmpl-unix em-unix pcomplete comint ansi-color
em-term term disp-table easymenu ehelp electric em-script em-prompt
em-ls em-hist em-pred em-glob em-dirs em-cmpl em-basic em-banner
em-alias ring esh-var esh-io esh-cmd esh-opt esh-ext esh-proc esh-arg
eldoc help-fns esh-groups eshell esh-module esh-mode esh-util time-date
tooltip ediff-hook vc-hooks lisp-float-type mwheel x-win x-dnd tool-bar
dnd fontset image regexp-opt fringe tabulated-list newcomment lisp-mode
register page menu-bar rfn-eshadow timer select scroll-bar mouse
jit-lock font-lock syntax facemenu font-core frame cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese case-table epa-hook jka-cmpr-hook help simple abbrev
minibuffer loaddefs button faces cus-face macroexp files text-properties
overlay sha1 md5 base64 format env code-pages mule custom widget
hashtable-print-readable backquote make-network-process dbusbind
dynamic-setting system-font-setting font-render-setting move-toolbar gtk
x-toolkit x multi-tty emacs)

-- 
Joost Kremers
Life has its moments

bug#15385: [Eshell] Directory completion overwrites preceding characters

[Aidan Gauland]

The bug seems to be in pcomplete, line 1140 of pcomplete.el, to be
exact:

  (delete-char (- (length (comint-quote-filename stub))))

`comint-quote-filename' is given a relative path, "~/Dro", and returning
an absolute path, "/home/joost/Dropbox/".  The length of this is then
passed to `delete-char' to clear the argument being completed, but the
length is wrong when it's a relative path, because
`comint-quote-filename' seems to always return an absolute path.

I'm not sure how to fix this without breaking something else or leaving
out other special cases, such as ${FOO}tm[TAB], because I still don't
really understand how pcomplete works.  I still want to sit down with
pcomplete for a while and document it (it was left undocumented along
with Eshell, of which it was originally a component), so unless someone
else can come up with a good fix, this bug will have to remain
outstanding for a while, unfortunately.

--Aidan

Joost Kremers <joostkremers@fastmail.fm> writes:

> Starting from `emacs -Q', if I do
>
>    (setq eshell-cmpl-ignore-case t)
>
> and then start up a shell, TAB completion behaves incorrectly under some
> conditions.
>
> Specifically, I type a command such as:
>
>     $ cp ~/Dropbox/Work/Reading/Wilbur_Martinez2002.pdf ~/Dro
>
> At this point, I hit TAB to complete `~/Dro' and get:
>
>     $ cp ~/Dropbox/Work/Reading/Wilbur_Martine/home/joost/Dropbox/
>
> Note that the problem does *not* occur with the first file path. If I
> hit TAB to complete
>
>     $ cp ~/Dro
>
> then completion produces the correct result:
>
>     $ cp ~/Dropbox/
>
> It seems the problem occurs when the file path is the second argument on
> the command line, because if I type:
>
>     $ ls ~/Dropbox/Work/Literature/k/kim2002.pdf ~/dro
>
> and hit TAB to complete, the result is also:
>
>     $ ls ~/Dropbox/Work/Literature/k/ki/home/joost/Dropbox/
>
> It does not occur when there is a switch preceding the file path:
>
>     $ ls -l ~/dro
>
> completes fine.
>
> Thanks,
>
> Joost Kremers

bug#15385: [Eshell] Directory completion overwrites preceding characters

[Thierry Volpiatto]

Aidan Gauland <aidalgol@no8wireless.co.nz> writes:

> The bug seems to be in pcomplete, line 1140 of pcomplete.el, to be
> exact:
>
>   (delete-char (- (length (comint-quote-filename stub))))

With helm completion enabled, this bug is not reproductible, if that
help you can look at the code here:
https://github.com/emacs-helm/helm/blob/master/helm-eshell.el

How to enable helm completion in eshell:
https://github.com/emacs-helm/helm/wiki#wiki-helmeshellcompletion

-- 
Thierry
Get my Gnupg key:
gpg --keyserver pgp.mit.edu --recv-keys 59F29997 

bug#15385: [Eshell] Directory completion overwrites preceding characters

[Joost Kremers]

But that still does not explain why the problem only occurs with the
second argument on the command line. The first argument is completed
without any problem, and *without* expanding ~ to /home/joost.

Joost


On Sat, Sep 21 2013, aidalgol@no8wireless.co.nz wrote:
> The bug seems to be in pcomplete, line 1140 of pcomplete.el, to be
> exact:
>
>   (delete-char (- (length (comint-quote-filename stub))))
>
> `comint-quote-filename' is given a relative path, "~/Dro", and returning
> an absolute path, "/home/joost/Dropbox/".  The length of this is then
> passed to `delete-char' to clear the argument being completed, but the
> length is wrong when it's a relative path, because
> `comint-quote-filename' seems to always return an absolute path.
>
> I'm not sure how to fix this without breaking something else or leaving
> out other special cases, such as ${FOO}tm[TAB], because I still don't
> really understand how pcomplete works.  I still want to sit down with
> pcomplete for a while and document it (it was left undocumented along
> with Eshell, of which it was originally a component), so unless someone
> else can come up with a good fix, this bug will have to remain
> outstanding for a while, unfortunately.
>
> --Aidan
>
> Joost Kremers <joostkremers@fastmail.fm> writes:
>
>> Starting from `emacs -Q', if I do
>>
>>    (setq eshell-cmpl-ignore-case t)
>>
>> and then start up a shell, TAB completion behaves incorrectly under some
>> conditions.
>>
>> Specifically, I type a command such as:
>>
>>     $ cp ~/Dropbox/Work/Reading/Wilbur_Martinez2002.pdf ~/Dro
>>
>> At this point, I hit TAB to complete `~/Dro' and get:
>>
>>     $ cp ~/Dropbox/Work/Reading/Wilbur_Martine/home/joost/Dropbox/
>>
>> Note that the problem does *not* occur with the first file path. If I
>> hit TAB to complete
>>
>>     $ cp ~/Dro
>>
>> then completion produces the correct result:
>>
>>     $ cp ~/Dropbox/
>>
>> It seems the problem occurs when the file path is the second argument on
>> the command line, because if I type:
>>
>>     $ ls ~/Dropbox/Work/Literature/k/kim2002.pdf ~/dro
>>
>> and hit TAB to complete, the result is also:
>>
>>     $ ls ~/Dropbox/Work/Literature/k/ki/home/joost/Dropbox/
>>
>> It does not occur when there is a switch preceding the file path:
>>
>>     $ ls -l ~/dro
>>
>> completes fine.
>>
>> Thanks,
>>
>> Joost Kremers


-- 
Joost Kremers
Life has its moments

bug#15385: [Eshell] Directory completion overwrites preceding characters

[Aidan Gauland]

Thierry Volpiatto <thierry.volpiatto@gmail.com> writes:

> Aidan Gauland <aidalgol@no8wireless.co.nz> writes:
>
>> The bug seems to be in pcomplete, line 1140 of pcomplete.el, to be
>> exact:
>>
>>   (delete-char (- (length (comint-quote-filename stub))))
>
> With helm completion enabled, this bug is not reproductible, if that
> help you can look at the code here:
> https://github.com/emacs-helm/helm/blob/master/helm-eshell.el
>
> How to enable helm completion in eshell:
> https://github.com/emacs-helm/helm/wiki#wiki-helmeshellcompletion

Oh great, this gets even weirder.  Thanks for the info!

bug#15385: [Eshell] Directory completion overwrites preceding characters

[Aidan Gauland]

Joost Kremers <joostkremers@fastmail.fm> writes:

> But that still does not explain why the problem only occurs with the
> second argument on the command line. The first argument is completed
> without any problem, and *without* expanding ~ to /home/joost.

You're right, it doesn't.  I forgot to mention this in my last post.
I'm not even sure whether this is pcomplete or the way eshell uses
pcomplete.

bug#15385: [Eshell] Directory completion overwrites preceding characters

[Lars Ingebrigtsen]

Joost Kremers <joostkremers@fastmail.fm> writes:

> Starting from `emacs -Q', if I do
>
>    (setq eshell-cmpl-ignore-case t)
>
> and then start up a shell, TAB completion behaves incorrectly under some
> conditions.
>
> Specifically, I type a command such as:
>
>     $ cp ~/Dropbox/Work/Reading/Wilbur_Martinez2002.pdf ~/Dro
>
> At this point, I hit TAB to complete `~/Dro' and get:
>
>     $ cp ~/Dropbox/Work/Reading/Wilbur_Martine/home/joost/Dropbox/

(I'm going through old bug reports that unfortunately weren't resolved
at the time.)

I tried to reproduce this (now with that exact path, but something
similar) in Emacs 29, and I was unable to -- it completed to the
expected path.

Are you still seeing this issue in recent Emacs versions?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#15385: [Eshell] Directory completion overwrites preceding characters

[Joost Kremers]

On Sat, Dec 04 2021, Lars Ingebrigtsen wrote:
> Joost Kremers <joostkremers@fastmail.fm> writes:
>
>> Starting from `emacs -Q', if I do
>>
>>    (setq eshell-cmpl-ignore-case t)
>>
>> and then start up a shell, TAB completion behaves incorrectly under some
>> conditions.
>>
>> Specifically, I type a command such as:
>>
>>     $ cp ~/Dropbox/Work/Reading/Wilbur_Martinez2002.pdf ~/Dro
>>
>> At this point, I hit TAB to complete `~/Dro' and get:
>>
>>     $ cp ~/Dropbox/Work/Reading/Wilbur_Martine/home/joost/Dropbox/
>
> (I'm going through old bug reports that unfortunately weren't resolved
> at the time.)
>
> I tried to reproduce this (now with that exact path, but something
> similar) in Emacs 29, and I was unable to -- it completed to the
> expected path.
>
> Are you still seeing this issue in recent Emacs versions?

I vaguely remember being disappointed that it hadn't been fixed in the first
release after my bug report, but that would have been Emacs 25. I haven't seen
this issue for a long time, probably since Emacs 26. I just installed the Emacs
28 pretest and I can't reproduce the issue either.

So I think it's safe to close this bug.


-- 
Joost Kremers
Life has its moments

bug#15385: [Eshell] Directory completion overwrites preceding characters

[Lars Ingebrigtsen]

Joost Kremers <joostkremers@fastmail.fm> writes:

> I vaguely remember being disappointed that it hadn't been fixed in the
> first release after my bug report, but that would have been Emacs
> 25. I haven't seen this issue for a long time, probably since Emacs
> 26. I just installed the Emacs 28 pretest and I can't reproduce the
> issue either.
>
> So I think it's safe to close this bug.

Thanks for checking; closing this bug report, then.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no