bug#72176: 30.0.60; icomplete-vertical-mode failed to work with Error

bug#72176: 30.0.60; icomplete-vertical-mode failed to work with Error

[Eason Huang]

Hello Emacs devel,

I tried to use icomplete-vertical-mode, but it doesn't works with
Emacs 30.0.60 and Emacs 31.0.50.

when trying `C-x C-f ~/` and input the first character of the names of
folders under home directory, will get bellow error:
```
Error in post-command-hook (icomplete-post-command-hook): (wrong-type-argument number-or-marker-p nil)
```

steps to reproduce:

1. start Emacs with `emacs -Q`
2. `M-x icomplete-vertical-mode` enable icomplete-vertical-mode
3. try to open some file under the directory under Home directory
   such as: `C-x C-f ~/` and then input the name a directory,
   here I will input `t` charactor, because I have the tmp folder.

4. Now you will get the error:

```
Error in post-command-hook (icomplete-post-command-hook): (wrong-type-argument number-or-marker-p nil)
```

 

In GNU Emacs 30.0.60 (build 1, x86_64-apple-darwin23.5.0, NS
 appkit-2487.60 Version 14.5 (Build 23F79), git sha1 5916b172bdc)
 of 2024-07-18 built on macbook
Windowing system distributor 'Apple', version 10.3.2487
System Description:  macOS 14.5

Configured using:
 'configure --without-native-compilation --without-dbus
 'CPPFLAGS=-I/opt/local/include
 -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk'
 'LDFLAGS=-L/opt/local/lib -Wl,-headerpad_max_install_names -Wl,-rpath
 /opt/local/lib/gcc13
 -Wl,-syslibroot,/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
 -arch x86_64''

Configured features:
ACL GIF GLIB GMP GNUTLS JPEG LCMS2 LIBXML2 MODULES NOTIFY KQUEUE NS
PDUMPER PNG RSVG SQLITE3 THREADS TIFF TOOLKIT_SCROLL_BARS TREE_SITTER
WEBP XIM ZLIB

Important settings:
  value of $LANG: zh_CN.UTF-8
  locale-coding-system: utf-8-unix

Major mode: ELisp/l

Minor modes in effect:
  icomplete-vertical-mode: t
  icomplete-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  minibuffer-regexp-mode: t
  line-number-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message mailcap yank-media puny rfc822
mml mml-sec password-cache epa derived epg rfc6068 epg-config gnus-util
text-property-search mm-decode mm-bodies mm-encode mail-parse rfc2231
mailabbrev gmm-utils mailheader sendmail rfc2047 rfc2045 ietf-drums
mm-util mail-prsvr mail-utils vc-git diff-mode track-changes easy-mmode
vc-dispatcher time-date subr-x dired-aux dired dired-loaddefs cus-start
cus-load icomplete cl-loaddefs cl-lib china-util rmc iso-transl tooltip
cconv eldoc paren electric uniquify ediff-hook vc-hooks lisp-float-type
elisp-mode mwheel term/ns-win ns-win ucs-normalize mule-util
term/common-win tool-bar dnd fontset image regexp-opt fringe
tabulated-list replace newcomment text-mode lisp-mode prog-mode register
page tab-bar menu-bar rfn-eshadow isearch easymenu timer select
scroll-bar mouse jit-lock font-lock syntax font-core term/tty-colors
frame minibuffer nadvice seq simple cl-generic indonesian philippine
cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech
european ethiopic indian cyrillic chinese composite emoji-zwj charscript
charprop case-table epa-hook jka-cmpr-hook help abbrev obarray oclosure
cl-preloaded button loaddefs theme-loaddefs faces cus-face macroexp
files window text-properties overlay sha1 md5 base64 format env
code-pages mule custom widget keymap hashtable-print-readable backquote
threads kqueue cocoa ns lcms2 multi-tty make-network-process emacs)

Memory information:
((conses 16 61505 15630) (symbols 48 7204 0) (strings 32 17696 2392)
 (string-bytes 1 421152) (vectors 16 11559)
 (vector-slots 8 187120 13794) (floats 8 26 263)
 (intervals 56 1455 20) (buffers 992 14))

-- 
Eason Huang

bug#72176: 30.0.60; icomplete-vertical-mode failed to work with Error

[Eli Zaretskii]

> From: Eason Huang <aqua0210@foxmail.com>
> Date: Thu, 18 Jul 2024 20:13:21 +0800
> 
> I tried to use icomplete-vertical-mode, but it doesn't works with
> Emacs 30.0.60 and Emacs 31.0.50.
> 
> when trying `C-x C-f ~/` and input the first character of the names of
> folders under home directory, will get bellow error:
> ```
> Error in post-command-hook (icomplete-post-command-hook): (wrong-type-argument number-or-marker-p nil)
> ```
> 
> steps to reproduce:
> 
> 1. start Emacs with `emacs -Q`
> 2. `M-x icomplete-vertical-mode` enable icomplete-vertical-mode
> 3. try to open some file under the directory under Home directory
>    such as: `C-x C-f ~/` and then input the name a directory,
>    here I will input `t` charactor, because I have the tmp folder.
> 
> 4. Now you will get the error:
> 
> ```
> Error in post-command-hook (icomplete-post-command-hook): (wrong-type-argument number-or-marker-p nil)
> ```

I cannot reproduce this, but I'm not on macOS.

bug#72176: 30.0.60; icomplete-vertical-mode failed to work with Error

[Visuwesh]

[வியாழன் ஜூலை 18, 2024] Eli Zaretskii wrote:

> I cannot reproduce this, but I'm not on macOS.

I cannot reproduce this on Linux either.

bug#72176: 30.0.60; icomplete-vertical-mode failed to work with Error

[Arash Esbati]

Eli Zaretskii <eliz@gnu.org> writes:

> I cannot reproduce this, but I'm not on macOS.

I can't reproduce this either, and I'm on macOS.

In GNU Emacs 31.0.50 (build 1, aarch64-apple-darwin23.5.0, NS
 appkit-2487.60 Version 14.5 (Build 23F79)) of 2024-07-15 built on
 MacMutant.fritz.box
Repository revision: eae1104f97ef944127eb5c977129b55f137e0830
Repository branch: master
Windowing system distributor 'Apple', version 10.3.2487
System Description:  macOS 14.5

Configured using:
 'configure --with-ns --without-pop --without-mailutils --with-threads
 --with-modules --with-native-compilation --without-compress-install
 'CFLAGS=-O2 -g0 -pipe'
 'CPPFLAGS=-I/opt/homebrew/Cellar/gcc/14.1.0_2/include
 -I/opt/homebrew/Cellar/libgccjit/14.1.0/include
 -I/opt/homebrew/Cellar/gmp/6.3.0/include'
 'LDFLAGS=-L/opt/homebrew/Cellar/gcc/14.1.0_2/lib/gcc/current
 -L/opt/homebrew/Cellar/gmp/6.3.0/lib''

Best, Arash

bug#72176: 30.0.60; icomplete-vertical-mode failed to work with Error

[Robert Pluim]

>>>>> On Thu, 18 Jul 2024 18:19:32 +0530, Visuwesh <visuweshm@gmail.com> said:

    Visuwesh> [வியாழன் ஜூலை 18, 2024] Eli Zaretskii wrote:
    >> I cannot reproduce this, but I'm not on macOS.

    Visuwesh> I cannot reproduce this on Linux either.

I can, on Linux:

135	  xsignal2 (Qwrong_type_argument, predicate, value);
(gdb) bt
#0  wrong_type_argument (predicate=predicate@entry=XIL(0xe490), value=XIL(0)) at data.c:135
#1  0x00005555555a4fc0 in CHECK_TYPE (x=<optimized out>, predicate=XIL(0xe490), ok=0) at /home/rpluim/repos/emacs-30/src/lisp.h:780
#2  check_number_coerce_marker (x=<optimized out>) at data.c:2682
#3  0x0000555555746ae9 in arithcompare (num1=make_fixnum(25), num2=XIL(0), comparison=comparison@entry=ARITH_GRTR) at data.c:2695
#4  0x00005555557485f5 in arithcompare_driver (comparison=<optimized out>, args=<optimized out>, nargs=<optimized out>) at data.c:2819
#5  Fgtr (nargs=2, args=0x7fffffffb940) at data.c:2851
#6  0x000055555575dac8 in eval_sub (form=<optimized out>) at eval.c:2570
#7  0x000055555575e8d2 in Fif (args=XIL(0x555556a5fad3)) at eval.c:391
#8  0x000055555575dbd7 in eval_sub (form=<optimized out>) at eval.c:2549
#9  0x000055555575fba9 in Fprogn (body=<optimized out>) at eval.c:439
#10 FletX (args=<optimized out>) at eval.c:1042
#11 0x000055555575dbd7 in eval_sub (form=<optimized out>) at eval.c:2549
#12 0x000055555575ebe1 in Fprogn (body=<optimized out>) at eval.c:439
#13 funcall_lambda (fun=fun@entry=XIL(0x5555563a4ad5), nargs=nargs@entry=3, arg_vector=arg_vector@entry=0x7fffffffbc90) at eval.c:3350
#14 0x000055555575f2fc in apply_lambda (fun=XIL(0x5555563a4ad5), args=<optimized out>, count=count@entry=...) at eval.c:3215
#15 0x000055555575d8cf in eval_sub (form=<optimized out>) at eval.c:2687
#16 0x000055555575ebe1 in Fprogn (body=<optimized out>) at eval.c:439

Lisp Backtrace:
">" (0xffffb940)
"if" (0xffffba50)
"let*" (0xffffbb80)
"completion--hilit-from-re" (0xffffbc90)
0x563e4930 PVEC_CLOSURE
"funcall" (0xffffbe70)
"if" (0xffffbf60)
"completion-lazy-hilit" (0xf0dff400)
"icomplete--render-vertical" (0xf0dff280)
"icomplete-completions" (0xf0dff1d0)
"icomplete-exhibit" (0xf0dff188)
"icomplete-post-command-hook" (0xffffc180)
"read-from-minibuffer" (0xffffc5e0)

which means `pos' is nil:

(defun completion--hilit-from-re (string regexp &optional point-idx)
  "Fontify STRING using REGEXP POINT-IDX.
`completions-common-part' and `completions-first-difference' are
used.  POINT-IDX is the position of point in the presumed \"PCM\"
pattern that was used to generate derive REGEXP from."
(let* ((md (and regexp (string-match regexp string) (cddr (match-data t))))
       (pos (if point-idx (match-beginning point-idx) (match-end 0)))
       (me (and md (match-end 0)))
       (from 0))
  (while md
    (add-face-text-property from (pop md) 'completions-common-part nil string)
    (setq from (pop md)))
  (if (> (length string) pos) <== right here
      (add-face-text-property
       pos (1+ pos)
       'completions-first-difference
       nil string))
  (unless (or (not me) (= from me))
    (add-face-text-property from me 'completions-common-part nil string))
  string))


Robert
-- 

bug#72176: 30.0.60; icomplete-vertical-mode failed to work with Error

[Eli Zaretskii]

> From: Robert Pluim <rpluim@gmail.com>
> Cc: Eli Zaretskii <eliz@gnu.org>,  72176@debbugs.gnu.org,  Eason Huang
>  <aqua0210@foxmail.com>
> Date: Thu, 18 Jul 2024 16:34:00 +0200
> 
> >>>>> On Thu, 18 Jul 2024 18:19:32 +0530, Visuwesh <visuweshm@gmail.com> said:
> 
>     Visuwesh> [வியாழன் ஜூலை 18, 2024] Eli Zaretskii wrote:
>     >> I cannot reproduce this, but I'm not on macOS.
> 
>     Visuwesh> I cannot reproduce this on Linux either.
> 
> I can, on Linux:

(I tested on GNU/Linux as well, and couldn't reproduce.)  Why does it
happen to you and not to others?  What conditions should happen and
how to modify the recipe to reproduce this?

bug#72176: 30.0.60; icomplete-vertical-mode failed to work with Error

[Robert Pluim]

>>>>> On Thu, 18 Jul 2024 18:34:35 +0300, Eli Zaretskii <eliz@gnu.org> said:

    Eli> (I tested on GNU/Linux as well, and couldn't reproduce.)  Why does it
    Eli> happen to you and not to others?  What conditions should happen and
    Eli> how to modify the recipe to reproduce this?

If I start emacs from "~", it doesnʼt happen, if I start it from eg
"~/repos/emacs-30" via "src/emacs -Q" then it does happen.

Robert
-- 

bug#72176: 30.0.60; icomplete-vertical-mode failed to work with Error

[Eli Zaretskii]

> From: Robert Pluim <rpluim@gmail.com>
> Cc: 72176@debbugs.gnu.org,  aqua0210@foxmail.com,  visuweshm@gmail.com
> Date: Fri, 19 Jul 2024 09:15:49 +0200
> 
> >>>>> On Thu, 18 Jul 2024 18:34:35 +0300, Eli Zaretskii <eliz@gnu.org> said:
> 
>     Eli> (I tested on GNU/Linux as well, and couldn't reproduce.)  Why does it
>     Eli> happen to you and not to others?  What conditions should happen and
>     Eli> how to modify the recipe to reproduce this?
> 
> If I start emacs from "~", it doesnʼt happen, if I start it from eg
> "~/repos/emacs-30" via "src/emacs -Q" then it does happen.

Thanks.  I've applied the trivial band-aid on emacs-30 to avoid the
error in post-command-hook, but the real problem is not in the
function that signals the error, it's in icomplete.el where it calls
the lazy-highlight code.  The evidence is that after the fix, the
display shown in those cases that caused the error is not the expected
one, unless you set icomplete-tidy-shadowed-file-names non-nil.  It
sounds like the problem happens specifically when the minibuffer text
begins with a "~/" and there's another "~/" later in the minibuffer,
the one the user types: the code probably doesn't expect that.

Adding João, who I think wrote this code.

bug#72176: 30.0.60; icomplete-vertical-mode failed to work with Error

[João Távora]

[-- Attachment #1: Type: text/plain, Size: 579 bytes --]

On Sat, Jul 20, 2024 at 7:05 AM Eli Zaretskii <eliz@gnu.org> wrote:

> Adding João, who I think wrote this code.

Don't understand root cause, but it's in the whereabouts of
the current analysis, yes. I recall regexp is supposed to always
match, because it has (presumably) been used for a previous
filtering. This breakage would seem to be a consequence of
3b07d33059150 by Eshel Yaron which in turn seems to be fixing
something stemming from some feature Dmitry added in
6b7ff60a5e71c in June. I went one commit before that one and
the recipe didn't break.

J

[-- Attachment #2: Type: text/html, Size: 793 bytes --]

bug#72176: 30.0.60; icomplete-vertical-mode failed to work with Error

[Eason Huang]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Robert Pluim <rpluim@gmail.com>
>> Cc: 72176@debbugs.gnu.org,  aqua0210@foxmail.com,  visuweshm@gmail.com
>> Date: Fri, 19 Jul 2024 09:15:49 +0200
>> 
>> >>>>> On Thu, 18 Jul 2024 18:34:35 +0300, Eli Zaretskii <eliz@gnu.org> said:
>> 
>>     Eli> (I tested on GNU/Linux as well, and couldn't reproduce.)  Why does it
>>     Eli> happen to you and not to others?  What conditions should happen and
>>     Eli> how to modify the recipe to reproduce this?
>> 
>> If I start emacs from "~", it doesnʼt happen, if I start it from eg
>> "~/repos/emacs-30" via "src/emacs -Q" then it does happen.
>
> Thanks.  I've applied the trivial band-aid on emacs-30 to avoid the
> error in post-command-hook, but the real problem is not in the
> function that signals the error, it's in icomplete.el where it calls
> the lazy-highlight code.  The evidence is that after the fix, the
> display shown in those cases that caused the error is not the expected
> one, unless you set icomplete-tidy-shadowed-file-names non-nil.  It
> sounds like the problem happens specifically when the minibuffer text
> begins with a "~/" and there's another "~/" later in the minibuffer,
> the one the user types: the code probably doesn't expect that.
>
> Adding João, who I think wrote this code.
>
>


Hi Eli,

I tried with the latest commit of Emacs-30(git sha1 ea30ffc52b3),
Now the issue is fixed.

Thanks

-- 
Eason Huang

bug#72176: 30.0.60; icomplete-vertical-mode failed to work with Error

[Eli Zaretskii]

> From: João Távora <joaotavora@gmail.com>
> Date: Sat, 20 Jul 2024 10:09:38 +0100
> Cc: Robert Pluim <rpluim@gmail.com>, 72176@debbugs.gnu.org, aqua0210@foxmail.com, 
> 	visuweshm@gmail.com
> 
> On Sat, Jul 20, 2024 at 7:05 AM Eli Zaretskii <eliz@gnu.org> wrote:
> 
> > Adding João, who I think wrote this code.
> 
> Don't understand root cause, but it's in the whereabouts of
> the current analysis, yes. I recall regexp is supposed to always 
> match, because it has (presumably) been used for a previous
> filtering. This breakage would seem to be a consequence of 
> 3b07d33059150 by Eshel Yaron which in turn seems to be fixing 
> something stemming from some feature Dmitry added in 
> 6b7ff60a5e71c in June. I went one commit before that one and 
> the recipe didn't break.

Thanks, I'm adding the guilty parties to this discussion.  Eshel and
Dmitry, could you please look into this?

bug#72176: 30.0.60; icomplete-vertical-mode failed to work with Error

[Eli Zaretskii]

> From: Eason Huang <aqua0210@foxmail.com>
> Cc: Robert Pluim <rpluim@gmail.com>,  João Távora
>  <joaotavora@gmail.com>,
>   72176@debbugs.gnu.org,  visuweshm@gmail.com
> Date: Sat, 20 Jul 2024 18:07:19 +0800
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Thanks.  I've applied the trivial band-aid on emacs-30 to avoid the
> > error in post-command-hook, but the real problem is not in the
> > function that signals the error, it's in icomplete.el where it calls
> > the lazy-highlight code.  The evidence is that after the fix, the
> > display shown in those cases that caused the error is not the expected
> > one, unless you set icomplete-tidy-shadowed-file-names non-nil.  It
> > sounds like the problem happens specifically when the minibuffer text
> > begins with a "~/" and there's another "~/" later in the minibuffer,
> > the one the user types: the code probably doesn't expect that.
> >
> > Adding João, who I think wrote this code.
> >
> >
> 
> 
> Hi Eli,
> 
> I tried with the latest commit of Emacs-30(git sha1 ea30ffc52b3),
> Now the issue is fixed.

Thanks for testing.

bug#72176: 30.0.60; icomplete-vertical-mode failed to work with Error

[Eshel Yaron via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: João Távora <joaotavora@gmail.com>
>> Date: Sat, 20 Jul 2024 10:09:38 +0100
>> Cc: Robert Pluim <rpluim@gmail.com>, 72176@debbugs.gnu.org, aqua0210@foxmail.com, 
>> 	visuweshm@gmail.com
>> 
>> On Sat, Jul 20, 2024 at 7:05 AM Eli Zaretskii <eliz@gnu.org> wrote:
>> 
>> > Adding João, who I think wrote this code.
>> 
>> Don't understand root cause, but it's in the whereabouts of
>> the current analysis, yes. I recall regexp is supposed to always 
>> match, because it has (presumably) been used for a previous
>> filtering. This breakage would seem to be a consequence of 
>> 3b07d33059150 by Eshel Yaron which in turn seems to be fixing 
>> something stemming from some feature Dmitry added in 
>> 6b7ff60a5e71c in June. I went one commit before that one and 
>> the recipe didn't break.
>
> Thanks, I'm adding the guilty parties to this discussion.

I'm afraid you missed your mark, because AFAICT it is Stefan Monnier
that's behind the root cause of this issue, which goes back way before
Dmitry's change or mine :)

It is a bug in completion--sifn-requote, that leads to incorrect
completion strings to be passed to completion-lazy-hilit.  It is not
related to icomplete per se.  To see the issue, try the following:

1. emacs -Q
2. say C-x C-f, clear the minibuffer, and insert ~/.config/~/f
3. hit ? to pop the completions list
4. see something like:

--8<---------------cut here---------------start------------->8---
Click or type M-RET on a completion to select it.
Type M-<down> or M-<up> to move point between completions.

6 possible completions:
.config/~/foo.json 	.config/~/foo.org 	.config/~/foo.php
.config/~/foo.py 	.config/~/foo.py~ 	.config/~/foo.rs
--8<---------------cut here---------------end--------------->8---

Note the incorrect prefix ".config/~/", completions should appear as
"foo.json", "foo.org", etc.

The following diff fixes this for me (including the icomplete symptom),
although I can't claim to fully understand completion--sifn-requote yet:

diff --git a/lisp/minibuffer.el b/lisp/minibuffer.el
index 31c365bf850..d0eb6b43c80 100644
--- a/lisp/minibuffer.el
+++ b/lisp/minibuffer.el
@@ -3855,13 +3855,13 @@ completion--sifn-requote
       ;; Second assumptions: If qpos is far from the end this can be a bit slow,
       ;; so we speed it up by doing a first loop that skips a word at a time.
       ;; This word-sized loop is careful not to cut in the middle of env-vars.
-      (while (let ((boundary (string-match "\\(\\$+{?\\)?\\w+\\W*\\'" qstr)))
-               (and boundary
-                    (progn
-                      (setq qprefix (substring qstr 0 boundary))
-                      (string-prefix-p uprefix
-                                       (substitute-in-file-name qprefix)))))
-        (setq qstr qprefix))
+      ;; (while (let ((boundary (string-match "\\(\\$+{?\\)?\\w+\\W*\\'" qstr)))
+      ;;          (and boundary
+      ;;               (progn
+      ;;                 (setq qprefix (substring qstr 0 boundary))
+      ;;                 (string-prefix-p uprefix
+      ;;                                  (substitute-in-file-name qprefix)))))
+      ;;   (setq qstr qprefix))
       (let ((qpos (length qstr)))
         (while (and (> qpos 0)
                     (string-prefix-p uprefix

bug#72176: 30.0.60; icomplete-vertical-mode failed to work with Error

[Dmitry Gutov]

Hi Eshel,

On 20/07/2024 19:48, Eshel Yaron wrote:
> Eli Zaretskii <eliz@gnu.org> writes:
> 
>>> From: João Távora <joaotavora@gmail.com>
>>> Date: Sat, 20 Jul 2024 10:09:38 +0100
>>> Cc: Robert Pluim <rpluim@gmail.com>, 72176@debbugs.gnu.org, aqua0210@foxmail.com,
>>> 	visuweshm@gmail.com
>>>
>>> On Sat, Jul 20, 2024 at 7:05 AM Eli Zaretskii <eliz@gnu.org> wrote:
>>>
>>>> Adding João, who I think wrote this code.
>>>
>>> Don't understand root cause, but it's in the whereabouts of
>>> the current analysis, yes. I recall regexp is supposed to always
>>> match, because it has (presumably) been used for a previous
>>> filtering. This breakage would seem to be a consequence of
>>> 3b07d33059150 by Eshel Yaron which in turn seems to be fixing
>>> something stemming from some feature Dmitry added in
>>> 6b7ff60a5e71c in June. I went one commit before that one and
>>> the recipe didn't break.
>>
>> Thanks, I'm adding the guilty parties to this discussion.
> 
> I'm afraid you missed your mark, because AFAICT it is Stefan Monnier
> that's behind the root cause of this issue, which goes back way before
> Dmitry's change or mine :)

Thanks for investigating! It might or might not be considered a bug (the 
behavior had been working out okay before our changes), but your 
proposal does make it more consistent, so I hope we can use it.

vc-annotate for lines in question points to commit 86957a0cd which 
references bug#11714. I _haven't_ been able to reproduce it with them 
commented out now - my guess it's because of the file-name-quote 
operation that had been added much later (the last comment in the bug 
mentions that functionality as a hypothetical).

So maybe they really are unnecessary now. Though the 
quoting-unquoting-requoting stuff is pretty gnarly, so it'd be great to 
get a go-ahead from Stefan as well.

> It is a bug in completion--sifn-requote, that leads to incorrect
> completion strings to be passed to completion-lazy-hilit.  It is not
> related to icomplete per se.  To see the issue, try the following:
> 
> 1. emacs -Q
> 2. say C-x C-f, clear the minibuffer, and insert ~/.config/~/f
> 3. hit ? to pop the completions list
> 4. see something like:
> 
> --8<---------------cut here---------------start------------->8---
> Click or type M-RET on a completion to select it.
> Type M-<down> or M-<up> to move point between completions.
> 
> 6 possible completions:
> .config/~/foo.json 	.config/~/foo.org 	.config/~/foo.php
> .config/~/foo.py 	.config/~/foo.py~ 	.config/~/foo.rs
> --8<---------------cut here---------------end--------------->8---
> 
> Note the incorrect prefix ".config/~/", completions should appear as
> "foo.json", "foo.org", etc.
> 
> The following diff fixes this for me (including the icomplete symptom),
> although I can't claim to fully understand completion--sifn-requote yet:
> 
> diff --git a/lisp/minibuffer.el b/lisp/minibuffer.el
> index 31c365bf850..d0eb6b43c80 100644
> --- a/lisp/minibuffer.el
> +++ b/lisp/minibuffer.el
> @@ -3855,13 +3855,13 @@ completion--sifn-requote
>         ;; Second assumptions: If qpos is far from the end this can be a bit slow,
>         ;; so we speed it up by doing a first loop that skips a word at a time.
>         ;; This word-sized loop is careful not to cut in the middle of env-vars.
> -      (while (let ((boundary (string-match "\\(\\$+{?\\)?\\w+\\W*\\'" qstr)))
> -               (and boundary
> -                    (progn
> -                      (setq qprefix (substring qstr 0 boundary))
> -                      (string-prefix-p uprefix
> -                                       (substitute-in-file-name qprefix)))))
> -        (setq qstr qprefix))
> +      ;; (while (let ((boundary (string-match "\\(\\$+{?\\)?\\w+\\W*\\'" qstr)))
> +      ;;          (and boundary
> +      ;;               (progn
> +      ;;                 (setq qprefix (substring qstr 0 boundary))
> +      ;;                 (string-prefix-p uprefix
> +      ;;                                  (substitute-in-file-name qprefix)))))
> +      ;;   (setq qstr qprefix))
>         (let ((qpos (length qstr)))
>           (while (and (> qpos 0)
>                       (string-prefix-p uprefix

bug#72176: 30.0.60; icomplete-vertical-mode failed to work with Error

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> I'm afraid you missed your mark, because AFAICT it is Stefan Monnier
> that's behind the root cause of this issue, which goes back way before
> Dmitry's change or mine :)

How do you dare to put in doubt the perfection of my code!

> The following diff fixes this for me (including the icomplete symptom),
> although I can't claim to fully understand completion--sifn-requote yet:
>
> diff --git a/lisp/minibuffer.el b/lisp/minibuffer.el
> index 31c365bf850..d0eb6b43c80 100644
> --- a/lisp/minibuffer.el
> +++ b/lisp/minibuffer.el
> @@ -3855,13 +3855,13 @@ completion--sifn-requote
>        ;; Second assumptions: If qpos is far from the end this can be a bit slow,
>        ;; so we speed it up by doing a first loop that skips a word at a time.
>        ;; This word-sized loop is careful not to cut in the middle of env-vars.
> -      (while (let ((boundary (string-match "\\(\\$+{?\\)?\\w+\\W*\\'" qstr)))
> -               (and boundary
> -                    (progn
> -                      (setq qprefix (substring qstr 0 boundary))
> -                      (string-prefix-p uprefix
> -                                       (substitute-in-file-name qprefix)))))
> -        (setq qstr qprefix))
> +      ;; (while (let ((boundary (string-match "\\(\\$+{?\\)?\\w+\\W*\\'" qstr)))
> +      ;;          (and boundary
> +      ;;               (progn
> +      ;;                 (setq qprefix (substring qstr 0 boundary))
> +      ;;                 (string-prefix-p uprefix
> +      ;;                                  (substitute-in-file-name qprefix)))))
> +      ;;   (setq qstr qprefix))
>        (let ((qpos (length qstr)))
>          (while (and (> qpos 0)
>                      (string-prefix-p uprefix

That's helpful.  So the "main assumption" is valid.

Indeed in your recipe the problem is that the while loop you comment out
(which implements the shortcut based on the "second assumption") ends up
throwing away the chunk we need.  This is by nature very hackish,
and there will still be cases where it back-fires.  Maybe indeed we
should just get rid of this shortcut, and maybe the resulting
performance is good enough.

Otherwise, the patch below should fix this particular occurrence.

BTW, while there's no doubt that the behavior displayed in your recipe
is undesirable, I consider it technically as "suboptimal" rather
than incorrect: the displayed completions are too verbose and confusing
but using them still ends up selecting the right file.
[ Which is why I changed the comment to clarify we want "the largest"
  QPOS: in your recipe we end up returning a valid QPOS but not the
  largest one.  ]


        Stefan


diff --git a/lisp/minibuffer.el b/lisp/minibuffer.el
index 9ad072daaf5..6fa04e9a062 100644
--- a/lisp/minibuffer.el
+++ b/lisp/minibuffer.el
@@ -3556,7 +3556,7 @@ completion-file-name-table
     (file-error nil)))               ;PCM often calls with invalid directories.
 
 (defun completion--sifn-requote (upos qstr)
-  ;; We're looking for `qpos' such that:
+  ;; We're looking for (the largest) `qpos' such that:
   ;; (equal (substring (substitute-in-file-name qstr) 0 upos)
   ;;        (substitute-in-file-name (substring qstr 0 qpos)))
   ;; Big problem here: we have to reverse engineer substitute-in-file-name to
@@ -3586,11 +3586,12 @@ completion--sifn-requote
       ;; Main assumption: nothing after qpos should affect the text before upos,
       ;; so we can work our way backward from the end of qstr, one character
       ;; at a time.
-      ;; Second assumptions: If qpos is far from the end this can be a bit slow,
+      ;; Second assumption: If qpos is far from the end this can be a bit slow,
       ;; so we speed it up by doing a first loop that skips a word at a time.
       ;; This word-sized loop is careful not to cut in the middle of env-vars.
       (while (let ((boundary (string-match "\\(\\$+{?\\)?\\w+\\W*\\'" qstr)))
                (and boundary
+                    (not (string-match-p "/[/~]" qstr boundary))
                     (progn
                       (setq qprefix (substring qstr 0 boundary))
                       (string-prefix-p uprefix

bug#72176: 30.0.60; icomplete-vertical-mode failed to work with Error

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: Eli Zaretskii <eliz@gnu.org>,  rpluim@gmail.com,  Dmitry Gutov
>  <dmitry@gutov.dev>,  João Távora
>  <joaotavora@gmail.com>,
>   visuweshm@gmail.com,  72176@debbugs.gnu.org,  aqua0210@foxmail.com
> Date: Sun, 21 Jul 2024 17:42:24 -0400
> 
> That's helpful.  So the "main assumption" is valid.
> 
> Indeed in your recipe the problem is that the while loop you comment out
> (which implements the shortcut based on the "second assumption") ends up
> throwing away the chunk we need.  This is by nature very hackish,
> and there will still be cases where it back-fires.  Maybe indeed we
> should just get rid of this shortcut, and maybe the resulting
> performance is good enough.
> 
> Otherwise, the patch below should fix this particular occurrence.

Thanks, please install on the emacs-30 branch.

bug#72176: 30.0.60; icomplete-vertical-mode failed to work with Error

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> Thanks, please install on the emacs-30 branch.

Done,


        Stefan