bug#38458: 27.0.50; case-insensitive substring completion

bug#38458: 27.0.50; case-insensitive substring completion

[Stephen Berman]

Typing TAB for case-insensitive substring completion of file and buffer
names can put the cursor in the wrong position in the minibuffer.  To
reproduce:

0. $ mkdir /tmp/test; touch /tmp/test/{testing,Testing}
1. $ emacs-master -Q --eval "(setq read-buffer-completion-ignore-case t
     read-file-name-completion-ignore-case t completion-category-overrides
     '((buffer (styles substring)) (file (styles substring))))"
2. Type `C-x C-f /tmp/test/tes TAB'
=> The minibuffer displays this, with point (^) after `s' in `testing':
Find file: /tmp/test/testing
                        ^
3. Visit /tmp/test/testing and /tmp/test/Testing, then from a buffer
other than these, e.g. /temp/test, type `C-x b tes TAB'

=> The minibuffer displays this, with point (^) after `s' in `testing':
Switch to buffer (default *scratch*): testing
                                         ^

I think these are unintended side effects of these two changes:

commit 0b21ecdb5ea9db8cf7a4a4ee59b29bf6273f2915
Author: Eli Zaretskii <eliz@gnu.org>
Date:   Sun Nov 3 18:08:45 2019 +0200

  Fix case-insensitive completion of non-ASCII file names

  * src/dired.c (scmp): Comment on (non)applicability to
  comparisons of  non-ASCII strings case-insensitively.
  (file_name_completion): Decode early the file names read from
  the directory, and use the decoded names for comparison when
  letter-case should be ignored.  (Bug#11339)

commit f2019fc676c2206bbdc53855e3bc4f1086676d3d
Author: Eli Zaretskii <eliz@gnu.org>
Date:   Sat Nov 9 21:56:30 2019 +0200

  Fix case-insensitive completion of buffer names

  * test/src/minibuf-tests.el (test-try-completion-ignore-case):
  New test, suggested by Stefan Monnier <monnier@iro.umontreal.ca>.

  * src/minibuf.c (Ftry_completion): Don't treat strings that
  are identical but for the case as if they were identical for
  the purposes of not counting the same string twice.  This
  fixes case-insensitive completion when all the candidates are
  identical but for the letter-case.  (Bug#11339)

In builds prior to the first change, after step 2 the minibuffer
displays this, with point (^) after `g' in `testing':

Find file: /tmp/test/testing
                            ^

This is also the result without file name substring completion, i.e.,
omitting the setting of completion-category-overrides.  For reasons I
don't understand, according to my testing, after the second change it is
necessary to make the buffer substring setting of
completion-category-overrides to get the effect in step 3, although
substring completion of buffer names is included in
completion-category-defaults.  (In builds prior to the second change,
after step 3 the minibuffer shows `testing' as the only completion,
because case-insensitive completion of buffer names didn't work prior to
this change.)


In GNU Emacs 27.0.50 (build 27, x86_64-pc-linux-gnu, GTK+ Version 3.24.5, cairo version 1.16.0)
 of 2019-12-01 built on strobe-lfs84
Repository revision: 9f2145f42daab13aed5cf89fdb6a7c5579819ec0
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.12003000
System Description: Linux From Scratch

bug#38458: 27.0.50; case-insensitive substring completion

[Eli Zaretskii]

> From: Stephen Berman <stephen.berman@gmx.net>
> Date: Mon, 02 Dec 2019 19:56:26 +0100
> 
> Typing TAB for case-insensitive substring completion of file and buffer
> names can put the cursor in the wrong position in the minibuffer.  To
> reproduce:
> 
> 0. $ mkdir /tmp/test; touch /tmp/test/{testing,Testing}
> 1. $ emacs-master -Q --eval "(setq read-buffer-completion-ignore-case t
>      read-file-name-completion-ignore-case t completion-category-overrides
>      '((buffer (styles substring)) (file (styles substring))))"
> 2. Type `C-x C-f /tmp/test/tes TAB'
> => The minibuffer displays this, with point (^) after `s' in `testing':
> Find file: /tmp/test/testing
>                         ^
> 3. Visit /tmp/test/testing and /tmp/test/Testing, then from a buffer
> other than these, e.g. /temp/test, type `C-x b tes TAB'
> 
> => The minibuffer displays this, with point (^) after `s' in `testing':
> Switch to buffer (default *scratch*): testing
>                                          ^
> 
> I think these are unintended side effects of these two changes:

If those changes caused this issue, then I think there was already a
bug in the substring completion style, which was just exposed by those
changes.

CC'ing Stefan in the hope that he could show us the light.

bug#38458: 27.0.50; case-insensitive substring completion

[Stephen Berman]

On Mon, 02 Dec 2019 22:45:11 +0200 Eli Zaretskii <eliz@gnu.org> wrote:

>> From: Stephen Berman <stephen.berman@gmx.net>
>> Date: Mon, 02 Dec 2019 19:56:26 +0100
>>
>> Typing TAB for case-insensitive substring completion of file and buffer
>> names can put the cursor in the wrong position in the minibuffer.  To
>> reproduce:
>>
>> 0. $ mkdir /tmp/test; touch /tmp/test/{testing,Testing}
>> 1. $ emacs-master -Q --eval "(setq read-buffer-completion-ignore-case t
>>      read-file-name-completion-ignore-case t completion-category-overrides
>>      '((buffer (styles substring)) (file (styles substring))))"
>> 2. Type `C-x C-f /tmp/test/tes TAB'
>> => The minibuffer displays this, with point (^) after `s' in `testing':
>> Find file: /tmp/test/testing
>>                         ^
>> 3. Visit /tmp/test/testing and /tmp/test/Testing, then from a buffer
>> other than these, e.g. /temp/test, type `C-x b tes TAB'
>>
>> => The minibuffer displays this, with point (^) after `s' in `testing':
>> Switch to buffer (default *scratch*): testing
>>                                          ^
>>
>> I think these are unintended side effects of these two changes:
>
> If those changes caused this issue, then I think there was already a
> bug in the substring completion style, which was just exposed by those
> changes.
>
> CC'ing Stefan in the hope that he could show us the light.

FWIW, when I step through completion--do-completion at step 2 in the
above recipe, the return value of completion-try-completion is
("/tmp/test/testing" . 17) in builds before commit 0b21ecdb5e and
("/tmp/test/testing" . 13) in builds after that commit.  Consequently,
on executing (forward-char (- comp-pos (length completion))) later in
completion--do-completion, point moves back 4 characters in later builds
and stays put in earlier builds.

Steve Berman

bug#38458: 27.0.50; case-insensitive substring completion

[Stefan Monnier]

> If those changes caused this issue, then I think there was already a
> bug in the substring completion style, which was just exposed by those
> changes.

Indeed.  I just pushed a fix to `master`.  Thank you.


        Stefan

bug#38458: 27.0.50; case-insensitive substring completion

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: Stephen Berman <stephen.berman@gmx.net>,  38458-done@debbugs.gnu.org
> Date: Tue, 03 Dec 2019 09:47:18 -0500
> 
> > If those changes caused this issue, then I think there was already a
> > bug in the substring completion style, which was just exposed by those
> > changes.
> 
> Indeed.  I just pushed a fix to `master`.  Thank you.

Thanks.

bug#38458: 27.0.50; case-insensitive substring completion

[Stephen Berman]

On Tue, 03 Dec 2019 09:47:18 -0500 Stefan Monnier <monnier@iro.umontreal.ca> wrote:

>> If those changes caused this issue, then I think there was already a
>> bug in the substring completion style, which was just exposed by those
>> changes.
>
> Indeed.  I just pushed a fix to `master`.  Thank you.

FTR, I confirm that fixes the issue I reported, thanks.

While I was testing cases for the bug report, I noticed that,
independent of case, the placement of point in substring completion is
sometimes suboptimal.  E.g. if there two files (or buffers) file1test
and file2test and you enter `fi TAB' then it completes to `file' with
point after the `e', which is fine for further input; but if you enter
`te TAB' then it completes to `test' with point after the second `t', so
you have to move point to complete the name.  It would be nice if in
such cases point were placed at the start rather than the end of the
partial match.  Should I make a feature request?

Steve Berman

bug#38458: 27.0.50; case-insensitive substring completion

[Stefan Monnier]

> point after the `e', which is fine for further input; but if you enter
> `te TAB' then it completes to `test' with point after the second `t', so
> you have to move point to complete the name.  It would be nice if in
> such cases point were placed at the start rather than the end of the
> partial match.

The completion code already tries to do that in a few other
circumstances, but indeed not this one.

> Should I make a feature request?

Yes (and please put me in the `X-Debbugs-Cc:`).


        Stefan