bug#20307: 25.0.50; (regexp-opt nil ...) returns ""

bug#20307: 25.0.50; (regexp-opt nil ...) returns ""

[David Kastrup]

Both

M-: (regexp-opt nil) RET

and

M-: (regexp-opt nil t) RET

return "".  However, they should return a regexp matching _nothing_
rather than everything, and the second invocation should also count as
one \(\) pairing.

So something like "[b-a]" and "\([b-a]\)" or
"[^[:unibyte:][:multibyte:]]" or something similarly contorted.



In GNU Emacs 25.0.50.1 (i686-pc-linux-gnu, GTK+ Version 3.12.2)
 of 2015-03-04 on lola
Repository revision: ca2b0e220ee6b2cab538e84703559696ce477e71
Windowing system distributor `The X.Org Foundation', version 11.0.11600000
System Description:	Ubuntu 14.10

Configured using:
 `configure --without-toolkit-scroll-bars'

Configured features:
XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND GPM DBUS GCONF GSETTINGS
NOTIFY LIBSELINUX GNUTLS LIBXML2 FREETYPE M17N_FLT LIBOTF XFT ZLIB

Important settings:
  value of $LC_MONETARY: en_US.UTF-8
  value of $LC_NUMERIC: en_US.UTF-8
  value of $LC_TIME: en_US.UTF-8
  value of $LANG: en_US.UTF-8
  value of $XMODIFIERS: @im=ibus
  locale-coding-system: utf-8-unix

Major mode: Group

Minor modes in effect:
  gnus-undo-mode: t
  diff-auto-refine-mode: t
  TeX-PDF-mode: t
  desktop-save-mode: t
  minibuffer-electric-default-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  buffer-read-only: t
  line-number-mode: t

Recent messages:
nnml: Reading incoming mail (no new mail)...done
Reading active file from private via nnml...done
Reading active file from archive via nnfolder...done
Reading active file via nndraft...done
Reading active file from /home/dak3/Mail/family via nndir...done
Checking new news...done
Opening nnml server on private...done
Expiring articles...done
"" [2 times]
Auto-saving...
Quit

Load-path shadows:
None found.

Features:
(shadow nnir emacsbug sendmail mule-util sort smiley gnus-cite flow-fill
mm-archive mail-extr gnus-bcklg gnus-kill gnus-async qp gnus-ml
disp-table pop3 nndir nndraft nnmh gnutls network-stream nsm auth-source
cl-macs eieio gv eieio-core cl-generic pcase starttls nnml nnfolder
nnnil gnus-agent gnus-srvr gnus-score score-mode nnvirtual gnus-msg
gnus-art mm-uu mml2015 mm-view mml-smime smime password-cache dig
mailcap nntp gnus-cache gnus-sum gnus-group gnus-undo gnus-start
gnus-cloud nnimap nnmail mail-source tls utf7 netrc nnoo parse-time
gnus-spec gnus-int gnus-range gnus-win warnings help-mode debug tar-mode
message format-spec rfc822 mml mml-sec mm-decode mm-bodies mm-encode
mail-parse rfc2231 rfc2047 rfc2045 ietf-drums mailabbrev gmm-utils
mailheader sh-script smie executable make-mode smerge-mode nxml-uchnm
rng-xsd xsd-regexp rng-cmpct rng-nxml rng-valid rng-loc rng-uri
rng-parse nxml-parse rng-match rng-dt rng-util rng-pttrn nxml-ns
nxml-mode nxml-outln nxml-rap nxml-util nxml-glyph nxml-enc xmltok
python json autorevert filenotify add-log tex-info texinfo latexenc
jka-compr preview prv-emacs tex-bar toolbar-x noutline outline latex
edmacro kmacro tex-style reftex-dcr reftex-auc reftex reftex-vars
dired-x dired scheme vc vc-dispatcher vc-git diff-mode easy-mmode
lilypond-mode compile comint ansi-color ring font-latex byte-opt
bytecomp byte-compile cl-extra seq cconv plain-tex tex-buf tex dbus xml
crm cc-mode cc-fonts cc-guess cc-menus cc-cmds cc-styles cc-align
cc-engine cc-vars cc-defs info easymenu package epg-config advice
desktop frameset minibuf-eldef gnus gnus-ems nnheader gnus-util
mail-utils mm-util help-fns mail-prsvr wid-edit cl-loaddefs cl-lib
cus-start cus-load preview-latex tex-site auto-loads server time-date
tooltip eldoc electric uniquify ediff-hook vc-hooks lisp-float-type
mwheel x-win x-dnd tool-bar dnd fontset image regexp-opt fringe
tabulated-list newcomment elisp-mode lisp-mode prog-mode register page
menu-bar rfn-eshadow timer select scroll-bar mouse jit-lock font-lock
syntax facemenu font-core frame cham georgian utf-8-lang misc-lang
vietnamese tibetan thai tai-viet lao korean japanese hebrew greek
romanian slovak czech european ethiopic indian cyrillic chinese
case-table epa-hook jka-cmpr-hook help simple abbrev minibuffer
cl-preloaded nadvice loaddefs button faces cus-face macroexp files
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget hashtable-print-readable backquote make-network-process
dbusbind gfilenotify dynamic-setting system-font-setting
font-render-setting move-toolbar gtk x-toolkit x multi-tty emacs)

Memory information:
((conses 8 624077 47421)
 (symbols 24 53351 145)
 (miscs 20 1251 1014)
 (strings 16 106692 17355)
 (string-bytes 1 3514198)
 (vectors 8 38616)
 (vector-slots 4 967485 13436)
 (floats 8 355 575)
 (intervals 28 23406 1055)
 (buffers 520 305)
 (heap 1024 69082 6867))

-- 
David Kastrup

bug#20307: 25.0.50; (regexp-opt nil ...) returns ""

[Stefan Monnier]

> So something like "[b-a]" and "\([b-a]\)" or
> "[^[:unibyte:][:multibyte:]]" or something similarly contorted.

I can't remember which regexp I used last time I needed one like that,
but something like "\\`\\'a" has the advantage of being efficient (the
\` anchor at the start is detected by Emacs and prevents looking for
a match over the whole searched area).


        Stefan

bug#20307: 25.0.50; (regexp-opt nil ...) returns ""

[Artur Malabarba]

2015-04-12 10:50 GMT+01:00 David Kastrup
> Both
>
> M-: (regexp-opt nil) RET
>
> and
>
> M-: (regexp-opt nil t) RET
>
> return "".  However, they should return a regexp matching nothing
> rather than everything, and the second invocation should also count as
> one () pairing.

I agree there should be () on the second one, but I strongly disagree
they should match nothing.

regexp-opt is NOT meant to match only the given strings. It is meant
to match anything containing the given strings.

There is a very fundamental difference in that. The less strings you
pass to regexp-opt, the MORE things the regexp will match. Why would
we suddently flip that on its head when going from 1 to 0 strings?

bug#20307: 25.0.50; (regexp-opt nil ...) returns ""

[Stefan Monnier]

> The less strings you pass to regexp-opt, the MORE things the regexp
> will match.

Hmm... I don't think so.


        Stefan

bug#20307: 25.0.50; (regexp-opt nil ...) returns ""

[David Kastrup]

Artur Malabarba <bruce.connor.am@gmail.com> writes:

> 2015-04-12 10:50 GMT+01:00 David Kastrup
>> Both
>>
>> M-: (regexp-opt nil) RET
>>
>> and
>>
>> M-: (regexp-opt nil t) RET
>>
>> return "".  However, they should return a regexp matching nothing
>> rather than everything, and the second invocation should also count as
>> one () pairing.
>
> I agree there should be () on the second one, but I strongly disagree
> they should match nothing.
>
> regexp-opt is NOT meant to match only the given strings. It is meant
> to match anything containing the given strings.

Well, and no string to match has been given.  This is not
(regexp-opt '(""))
but rather
(regexp-opt '())

> There is a very fundamental difference in that. The less strings you
> pass to regexp-opt, the MORE things the regexp will match.

Come again?

> Why would we suddently flip that on its head when going from 1 to 0
> strings?

(regexp-opt '("a" "b" "c")) -> "[abc]"
(regexp-opt '("a" "b")) -> "[ab]"

Quite literally (execute C-x C-e after the expressions above if you
don't believe me).  So how does "[ab]" match more than "[abc]" ?

-- 
David Kastrup

bug#20307: 25.0.50; (regexp-opt nil ...) returns ""

[Artur Malabarba]

[-- Attachment #1: Type: text/plain, Size: 1362 bytes --]

I believe you. Please ignore this babbling baboon.
That's what I get for typing on the bus.
On Apr 13, 2015 4:34 PM, "David Kastrup" <dak@gnu.org> wrote:

> Artur Malabarba <bruce.connor.am@gmail.com> writes:
>
> > 2015-04-12 10:50 GMT+01:00 David Kastrup
> >> Both
> >>
> >> M-: (regexp-opt nil) RET
> >>
> >> and
> >>
> >> M-: (regexp-opt nil t) RET
> >>
> >> return "".  However, they should return a regexp matching nothing
> >> rather than everything, and the second invocation should also count as
> >> one () pairing.
> >
> > I agree there should be () on the second one, but I strongly disagree
> > they should match nothing.
> >
> > regexp-opt is NOT meant to match only the given strings. It is meant
> > to match anything containing the given strings.
>
> Well, and no string to match has been given.  This is not
> (regexp-opt '(""))
> but rather
> (regexp-opt '())
>
> > There is a very fundamental difference in that. The less strings you
> > pass to regexp-opt, the MORE things the regexp will match.
>
> Come again?
>
> > Why would we suddently flip that on its head when going from 1 to 0
> > strings?
>
> (regexp-opt '("a" "b" "c")) -> "[abc]"
> (regexp-opt '("a" "b")) -> "[ab]"
>
> Quite literally (execute C-x C-e after the expressions above if you
> don't believe me).  So how does "[ab]" match more than "[abc]" ?
>
> --
> David Kastrup
>

[-- Attachment #2: Type: text/html, Size: 2030 bytes --]

bug#20307: [PATCH] make regexp-opt return a no-match return value with empty input

[Mattias Engdegård]

[-- Attachment #1: Type: text/plain, Size: 76 bytes --]

Here is a patch (moved from Bug#34641 where it was independently reported).

[-- Attachment #2: 0001-Correct-regexp-opt-return-value-for-empty-string-lis.patch --]
[-- Type: application/octet-stream, Size: 3955 bytes --]

From 8efe696f635c948496e3bcfe1e69e902264b957c Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Mattias=20Engdeg=C3=A5rd?= <mattiase@acm.org>
Date: Mon, 25 Feb 2019 15:22:02 +0100
Subject: [PATCH] Correct regexp-opt return value for empty string list

When regexp-opt is called with an empty list of strings, return a regexp
that doesn't match anything instead of the empty string (Bug#20307).

* doc/lispref/searching.texi (Regular Expression Functions):
* etc/NEWS:
Document the new behaviour.
* lisp/emacs-lisp/regexp-opt.el (regexp-opt):
Return a never-match regexp for empty inputs.
---
 doc/lispref/searching.texi    |  3 +++
 etc/NEWS                      |  6 ++++++
 lisp/emacs-lisp/regexp-opt.el | 23 +++++++++++++++--------
 3 files changed, 24 insertions(+), 8 deletions(-)

diff --git a/doc/lispref/searching.texi b/doc/lispref/searching.texi
index 73a7304a3b..0b944a2711 100644
--- a/doc/lispref/searching.texi
+++ b/doc/lispref/searching.texi
@@ -960,6 +960,9 @@ possible.  A hand-tuned regular expression can sometimes be slightly
 more efficient, but is almost never worth the effort.}.
 @c E.g., see https://debbugs.gnu.org/2816
 
+If @var{strings} is empty, the return value is a regexp that never
+matches anything.
+
 The optional argument @var{paren} can be any of the following:
 
 @table @asis
diff --git a/etc/NEWS b/etc/NEWS
index 5f7616429b..6506a1c6b5 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -1624,6 +1624,12 @@ in any order.  If the new third argument is non-nil, the match is
 guaranteed to be performed in the order given, as if the strings were
 made into a regexp by joining them with '\|'.
 
++++
+** The function 'regexp-opt', when given an empty list of strings, now
+returns a regexp that never matches anything, which is an identity for
+this operation.  Previously, the empty string was returned in this
+case.
+
 \f
 * Changes in Emacs 27.1 on Non-Free Operating Systems
 
diff --git a/lisp/emacs-lisp/regexp-opt.el b/lisp/emacs-lisp/regexp-opt.el
index 33a5b770a0..107b453637 100644
--- a/lisp/emacs-lisp/regexp-opt.el
+++ b/lisp/emacs-lisp/regexp-opt.el
@@ -90,6 +90,9 @@ Each string should be unique in STRINGS and should not contain
 any regexps, quoted or not.  Optional PAREN specifies how the
 returned regexp is surrounded by grouping constructs.
 
+If STRINGS is empty, the return value is a regexp that never
+matches anything.
+
 The optional argument PAREN can be any of the following:
 
 a string
@@ -139,14 +142,18 @@ usually more efficient than that of a simplified version:
 	   (sorted-strings (delete-dups
 			    (sort (copy-sequence strings) 'string-lessp)))
 	   (re
-            ;; If NOREORDER is non-nil and the list contains a prefix
-            ;; of another string, we give up all attempts at optimisation.
-            ;; There is plenty of room for improvement (Bug#34641).
-            (if (and noreorder (regexp-opt--contains-prefix sorted-strings))
-                (concat (or open "\\(?:")
-                        (mapconcat #'regexp-quote strings "\\|")
-                        "\\)")
-              (regexp-opt-group sorted-strings (or open t) (not open)))))
+            (cond
+             ;; No strings: return a\` which cannot match anything.
+             ((null strings)
+              (concat (or open "\\(?:") "a\\`\\)"))
+             ;; If we cannot reorder, give up all attempts at
+             ;; optimisation.  There is room for improvement (Bug#34641).
+             ((and noreorder (regexp-opt--contains-prefix sorted-strings))
+              (concat (or open "\\(?:")
+                      (mapconcat #'regexp-quote strings "\\|")
+                      "\\)"))
+             (t
+              (regexp-opt-group sorted-strings (or open t) (not open))))))
       (cond ((eq paren 'words)
 	     (concat "\\<" re "\\>"))
 	    ((eq paren 'symbols)
-- 
2.17.2 (Apple Git-113)

bug#20307: [PATCH] make regexp-opt return a no-match return value with empty input

[Eli Zaretskii]

> From: Mattias Engdegård <mattiase@acm.org>
> Date: Mon, 25 Feb 2019 15:57:44 +0100
> 
> +If STRINGS is empty, the return value is a regexp that never
> +matches anything.

This says "empty", but the actual test will catch nil as well as an
empty list.  Should we perhaps mention that?

Thanks.

bug#20307: [PATCH] make regexp-opt return a no-match return value with empty input

[Mattias Engdegård]

2 mars 2019 kl. 13.37 skrev Eli Zaretskii <eliz@gnu.org>:
> 
>> From: Mattias Engdegård <mattiase@acm.org>
>> Date: Mon, 25 Feb 2019 15:57:44 +0100
>> 
>> +If STRINGS is empty, the return value is a regexp that never
>> +matches anything.
> 
> This says "empty", but the actual test will catch nil as well as an
> empty list.  Should we perhaps mention that?

Sorry, I don't understand. Do we in general distinguish nil from the empty list in documentation?
Or did you mean that the phrase should be "If STRINGS is the empty list..."?

bug#20307: [PATCH] make regexp-opt return a no-match return value with empty input

[Eli Zaretskii]

> From: Mattias Engdegård <mattiase@acm.org>
> Date: Sat, 2 Mar 2019 15:21:52 +0100
> Cc: 20307@debbugs.gnu.org
> 
> > This says "empty", but the actual test will catch nil as well as an
> > empty list.  Should we perhaps mention that?
> 
> Sorry, I don't understand. Do we in general distinguish nil from the empty list in documentation?

I don't know.  I thought the reader might not be aware of their
equivalence, so being explicit would be better.

> Or did you mean that the phrase should be "If STRINGS is the empty list..."?

Yes, that would take care of the issue.

bug#20307: [PATCH] make regexp-opt return a no-match return value with empty input

[Mattias Engdegård]

[-- Attachment #1: Type: text/plain, Size: 436 bytes --]


>> Sorry, I don't understand. Do we in general distinguish nil from the empty list in documentation?
> 
> I don't know.  I thought the reader might not be aware of their
> equivalence, so being explicit would be better.
> 
>> Or did you mean that the phrase should be "If STRINGS is the empty list..."?
> 
> Yes, that would take care of the issue.

Now done in the doc string and in searching.texi.

New patch attached.


[-- Attachment #2: 0001-Correct-regexp-opt-return-value-for-empty-string-lis.patch --]
[-- Type: application/octet-stream, Size: 3973 bytes --]

From 944a6100d5d2563fdd355f9f91a6190117a79efc Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Mattias=20Engdeg=C3=A5rd?= <mattiase@acm.org>
Date: Mon, 25 Feb 2019 15:22:02 +0100
Subject: [PATCH] Correct regexp-opt return value for empty string list

When regexp-opt is called with an empty list of strings, return a regexp
that doesn't match anything instead of the empty string (Bug#20307).

* doc/lispref/searching.texi (Regular Expression Functions):
* etc/NEWS:
Document the new behaviour.
* lisp/emacs-lisp/regexp-opt.el (regexp-opt):
Return a never-match regexp for empty inputs.
---
 doc/lispref/searching.texi    |  3 +++
 etc/NEWS                      |  6 ++++++
 lisp/emacs-lisp/regexp-opt.el | 23 +++++++++++++++--------
 3 files changed, 24 insertions(+), 8 deletions(-)

diff --git a/doc/lispref/searching.texi b/doc/lispref/searching.texi
index fb7f48474d..38e6204055 100644
--- a/doc/lispref/searching.texi
+++ b/doc/lispref/searching.texi
@@ -960,6 +960,9 @@ possible.  A hand-tuned regular expression can sometimes be slightly
 more efficient, but is almost never worth the effort.}.
 @c E.g., see https://debbugs.gnu.org/2816
 
+If @var{strings} is the empty list, the return value is a regexp that
+never matches anything.
+
 The optional argument @var{paren} can be any of the following:
 
 @table @asis
diff --git a/etc/NEWS b/etc/NEWS
index 7c95988ff5..65eb9ba1af 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -1649,6 +1649,12 @@ in any order.  If the new third argument is non-nil, the match is
 guaranteed to be performed in the order given, as if the strings were
 made into a regexp by joining them with '\|'.
 
++++
+** The function 'regexp-opt', when given an empty list of strings, now
+returns a regexp that never matches anything, which is an identity for
+this operation.  Previously, the empty string was returned in this
+case.
+
 \f
 * Changes in Emacs 27.1 on Non-Free Operating Systems
 
diff --git a/lisp/emacs-lisp/regexp-opt.el b/lisp/emacs-lisp/regexp-opt.el
index d0c5f2d3fc..4404b905a6 100644
--- a/lisp/emacs-lisp/regexp-opt.el
+++ b/lisp/emacs-lisp/regexp-opt.el
@@ -90,6 +90,9 @@ Each string should be unique in STRINGS and should not contain
 any regexps, quoted or not.  Optional PAREN specifies how the
 returned regexp is surrounded by grouping constructs.
 
+If STRINGS is the empty list, the return value is a regexp that
+never matches anything.
+
 The optional argument PAREN can be any of the following:
 
 a string
@@ -140,14 +143,18 @@ usually more efficient than that of a simplified version:
 	   (sorted-strings (delete-dups
 			    (sort (copy-sequence strings) 'string-lessp)))
 	   (re
-            ;; If NOREORDER is non-nil and the list contains a prefix
-            ;; of another string, we give up all attempts at optimisation.
-            ;; There is plenty of room for improvement (Bug#34641).
-            (if (and noreorder (regexp-opt--contains-prefix sorted-strings))
-                (concat (or open "\\(?:")
-                        (mapconcat #'regexp-quote strings "\\|")
-                        "\\)")
-              (regexp-opt-group sorted-strings (or open t) (not open)))))
+            (cond
+             ;; No strings: return a\` which cannot match anything.
+             ((null strings)
+              (concat (or open "\\(?:") "a\\`\\)"))
+             ;; If we cannot reorder, give up all attempts at
+             ;; optimisation.  There is room for improvement (Bug#34641).
+             ((and noreorder (regexp-opt--contains-prefix sorted-strings))
+              (concat (or open "\\(?:")
+                      (mapconcat #'regexp-quote strings "\\|")
+                      "\\)"))
+             (t
+              (regexp-opt-group sorted-strings (or open t) (not open))))))
       (cond ((eq paren 'words)
 	     (concat "\\<" re "\\>"))
 	    ((eq paren 'symbols)
-- 
2.17.2 (Apple Git-113)

bug#20307: [PATCH] make regexp-opt return a no-match return value with empty input

[Eli Zaretskii]

> From: Mattias Engdegård <mattiase@acm.org>
> Date: Sat, 2 Mar 2019 15:49:07 +0100
> Cc: 20307@debbugs.gnu.org
> 
> >> Or did you mean that the phrase should be "If STRINGS is the empty list..."?
> > 
> > Yes, that would take care of the issue.
> 
> Now done in the doc string and in searching.texi.
> 
> New patch attached.

Fine with me, thanks.

bug#20307: [PATCH] make regexp-opt return a no-match return value with empty input

[Mattias Engdegård]

2 mars 2019 kl. 15.57 skrev Eli Zaretskii <eliz@gnu.org>:
>> 
>> New patch attached.
> 
> Fine with me, thanks.

Thank you, pushed.