bug#38458: 27.0.50; case-insensitive substring completion

[Stephen Berman <stephen.berman@gmx.net>]

Typing TAB for case-insensitive substring completion of file and buffer
names can put the cursor in the wrong position in the minibuffer.  To
reproduce:

0. $ mkdir /tmp/test; touch /tmp/test/{testing,Testing}
1. $ emacs-master -Q --eval "(setq read-buffer-completion-ignore-case t
     read-file-name-completion-ignore-case t completion-category-overrides
     '((buffer (styles substring)) (file (styles substring))))"
2. Type `C-x C-f /tmp/test/tes TAB'
=> The minibuffer displays this, with point (^) after `s' in `testing':
Find file: /tmp/test/testing
                        ^
3. Visit /tmp/test/testing and /tmp/test/Testing, then from a buffer
other than these, e.g. /temp/test, type `C-x b tes TAB'

=> The minibuffer displays this, with point (^) after `s' in `testing':
Switch to buffer (default *scratch*): testing
                                         ^

I think these are unintended side effects of these two changes:

commit 0b21ecdb5ea9db8cf7a4a4ee59b29bf6273f2915
Author: Eli Zaretskii <eliz@gnu.org>
Date:   Sun Nov 3 18:08:45 2019 +0200

  Fix case-insensitive completion of non-ASCII file names

  * src/dired.c (scmp): Comment on (non)applicability to
  comparisons of  non-ASCII strings case-insensitively.
  (file_name_completion): Decode early the file names read from
  the directory, and use the decoded names for comparison when
  letter-case should be ignored.  (Bug#11339)

commit f2019fc676c2206bbdc53855e3bc4f1086676d3d
Author: Eli Zaretskii <eliz@gnu.org>
Date:   Sat Nov 9 21:56:30 2019 +0200

  Fix case-insensitive completion of buffer names

  * test/src/minibuf-tests.el (test-try-completion-ignore-case):
  New test, suggested by Stefan Monnier <monnier@iro.umontreal.ca>.

  * src/minibuf.c (Ftry_completion): Don't treat strings that
  are identical but for the case as if they were identical for
  the purposes of not counting the same string twice.  This
  fixes case-insensitive completion when all the candidates are
  identical but for the letter-case.  (Bug#11339)

In builds prior to the first change, after step 2 the minibuffer
displays this, with point (^) after `g' in `testing':

Find file: /tmp/test/testing
                            ^

This is also the result without file name substring completion, i.e.,
omitting the setting of completion-category-overrides.  For reasons I
don't understand, according to my testing, after the second change it is
necessary to make the buffer substring setting of
completion-category-overrides to get the effect in step 3, although
substring completion of buffer names is included in
completion-category-defaults.  (In builds prior to the second change,
after step 3 the minibuffer shows `testing' as the only completion,
because case-insensitive completion of buffer names didn't work prior to
this change.)


In GNU Emacs 27.0.50 (build 27, x86_64-pc-linux-gnu, GTK+ Version 3.24.5, cairo version 1.16.0)
 of 2019-12-01 built on strobe-lfs84
Repository revision: 9f2145f42daab13aed5cf89fdb6a7c5579819ec0
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.12003000
System Description: Linux From Scratch