bug#72425: 29.2.50; substring (and other PCM styles) fails with candidates containing newlines

bug#72425: 29.2.50; substring (and other PCM styles) fails with candidates containing newlines

[Spencer Baugh]

1. emacs -Q
2. (let ((completion-styles '(substring)))
     (completing-read ":" '("foo1bar" "foo2bar") nil nil "bar"))
3. Press TAB
4. foo1bar and foo2bar are suggested as completion candidates, because
   they contain "bar" as a substring.
5. (let ((completion-styles '(substring)))
     (completing-read ":" '("foo1\nbar" "foo2\nbar") nil nil "bar"))
6. Press TAB
7. Observe "[No match]" is printed.

This is due to a regex using "." when it should use a pattern which
actually matches anything, such as "[^z-a]".  A patch to fix will
follow.



In GNU Emacs 29.2.50 (build 9, x86_64-pc-linux-gnu, X toolkit, cairo
 version 1.15.12, Xaw scroll bars) of 2024-07-30 built on
 igm-qws-u22796a
Repository revision: cd9604db959c439c5695cf79f6533b5cbd340851
Repository branch: emacs-29
Windowing system distributor 'The X.Org Foundation', version 11.0.12011000
System Description: Rocky Linux 8.10 (Green Obsidian)

Configured using:
 'configure --with-x-toolkit=lucid --without-gpm --without-gconf
 --without-selinux --without-imagemagick --with-modules --with-gif=no
 --with-cairo --with-rsvg --without-compress-install
 --with-native-compilation=aot --with-tree-sitter
 PKG_CONFIG_PATH=/usr/local/home/garnish/libtree-sitter/0.22.6-1/lib/pkgconfig/'

Configured features:
CAIRO DBUS FREETYPE GLIB GMP GNUTLS GSETTINGS HARFBUZZ JPEG JSON
LIBSYSTEMD LIBXML2 MODULES NATIVE_COMP NOTIFY INOTIFY PDUMPER PNG RSVG
SECCOMP SOUND SQLITE3 THREADS TIFF TOOLKIT_SCROLL_BARS TREE_SITTER X11
XDBE XIM XINPUT2 XPM LUCID ZLIB

Important settings:
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: ELisp/l

bug#72425: 29.2.50; substring (and other PCM styles) fails with candidates containing newlines

[Spencer Baugh]

[-- Attachment #1: Type: text/plain, Size: 16 bytes --]


Patch to fix:


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Fix-partial-completion-for-completion-candidates-con.patch --]
[-- Type: text/x-patch, Size: 1417 bytes --]

From dfac112fa23d503b671fa5f9901411a4556bf054 Mon Sep 17 00:00:00 2001
From: Spencer Baugh <sbaugh@janestreet.com>
Date: Fri, 2 Aug 2024 12:15:58 -0400
Subject: [PATCH] Fix partial-completion for completion candidates containing
 newlines

partial-completion tries to match a pattern containing wildcards (such
as `any' or `prefix') against completion candidates.  Wildcards are
supposed to match any sequence of characters, but
completion-pcm--pattern->regex transformed the wildcards into ".*",
which won't match sequences containing newlines.  Fix this to properly
match anything by using "[^z-a]*" instead.  (That's (rx (* anything)))

* lisp/minibuffer.el (completion-pcm--pattern->regex): Fix
regex. (bug#72425)
---
 lisp/minibuffer.el | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lisp/minibuffer.el b/lisp/minibuffer.el
index 5860c4238c2..cefd4247370 100644
--- a/lisp/minibuffer.el
+++ b/lisp/minibuffer.el
@@ -3896,7 +3896,7 @@ completion-pcm--pattern->regex
                      (t
                       (let ((re (if (eq x 'any-delim)
                                     (concat completion-pcm--delim-wild-regex "*?")
-                                  ".*?")))
+                                  "[^z-a]*?")))
                         (if (if (consp group) (memq x group) group)
                             (concat "\\(" re "\\)")
                           re)))))
-- 
2.39.3

bug#72425: 29.2.50; substring (and other PCM styles) fails with candidates containing newlines

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> Patch to fix:
>
> From dfac112fa23d503b671fa5f9901411a4556bf054 Mon Sep 17 00:00:00 2001
> From: Spencer Baugh <sbaugh@janestreet.com>
> Date: Fri, 2 Aug 2024 12:15:58 -0400
> Subject: [PATCH] Fix partial-completion for completion candidates containing
>  newlines
>
> partial-completion tries to match a pattern containing wildcards (such
> as `any' or `prefix') against completion candidates.  Wildcards are
> supposed to match any sequence of characters, but
> completion-pcm--pattern->regex transformed the wildcards into ".*",
> which won't match sequences containing newlines.  Fix this to properly
> match anything by using "[^z-a]*" instead.  (That's (rx (* anything)))

Looks fine to me.  I find [^z-a] a bit too magical, so maybe an `rx`
pattern would be more clear.

My guess is that completion of thingies with newlines is probably going
to bump into other problems.  🙂


        Stefan

bug#72425: 29.2.50; substring (and other PCM styles) fails with candidates containing newlines

[Eli Zaretskii]

> Cc: 72425@debbugs.gnu.org, Dmitry Gutov <dmitry@gutov.dev>
> Date: Tue, 13 Aug 2024 20:51:28 -0400
> From:  Stefan Monnier via "Bug reports for GNU Emacs,
>  the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>
> 
> > Patch to fix:
> >
> > From dfac112fa23d503b671fa5f9901411a4556bf054 Mon Sep 17 00:00:00 2001
> > From: Spencer Baugh <sbaugh@janestreet.com>
> > Date: Fri, 2 Aug 2024 12:15:58 -0400
> > Subject: [PATCH] Fix partial-completion for completion candidates containing
> >  newlines
> >
> > partial-completion tries to match a pattern containing wildcards (such
> > as `any' or `prefix') against completion candidates.  Wildcards are
> > supposed to match any sequence of characters, but
> > completion-pcm--pattern->regex transformed the wildcards into ".*",
> > which won't match sequences containing newlines.  Fix this to properly
> > match anything by using "[^z-a]*" instead.  (That's (rx (* anything)))
> 
> Looks fine to me.  I find [^z-a] a bit too magical, so maybe an `rx`
> pattern would be more clear.

Thanks, installed on master, and closing the bug.

> My guess is that completion of thingies with newlines is probably going
> to bump into other problems.  🙂

Exciting new bugs!

bug#72425: 29.2.50; substring (and other PCM styles) fails with candidates containing newlines

[Drew Adams via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> This is due to a regex using "." when it should use a pattern which
> actually matches anything, such as "[^z-a]".  A patch to fix will
> follow.

"[^z-a]" doesn't match any char.

"\\(.\\|[\n]\\)" matches any char.

bug#72425: 29.2.50; substring (and other PCM styles) fails with candidates containing newlines

[Spencer Baugh]

Drew Adams <drew.adams@oracle.com> writes:

>> This is due to a regex using "." when it should use a pattern which
>> actually matches anything, such as "[^z-a]".  A patch to fix will
>> follow.
>
> "[^z-a]" doesn't match any char.
>
> "\\(.\\|[\n]\\)" matches any char.

(rx anychar) expands to [^z-a], I'm just following that.  File a
different bug if you think rx is wrong, and if that gets changed, I'll
change this patch as well.

bug#72425: 29.2.50; substring (and other PCM styles) fails with candidates containing newlines

[Drew Adams via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> >> This is due to a regex using "." when it should use a pattern which
> >> actually matches anything, such as "[^z-a]".  A patch to fix will
> >> follow.
> >
> > "[^z-a]" doesn't match any char.
> >
> > "\\(.\\|[\n]\\)" matches any char.
> 
> (rx anychar) expands to [^z-a], I'm just following that.  File a
> different bug if you think rx is wrong, and if that gets changed, I'll
> change this patch as well.

I guess [^z-a] does match any char.
Looks odd, but makes sense no char is in that range,
so every char is not in that range.