bug#66406: Broken `re-search-forward' (?)

bug#66406: Broken `re-search-forward' (?)

[Paul Pogonyshev]

[-- Attachment #1: Type: text/plain, Size: 325 bytes --]

In a recent Emacs master (as provided in Silex's Docker image):

$ emacs --batch --eval "(with-temp-buffer (insert
\"\n;;;###autoload\n(defun ...)\n\") (goto-char 1) (print
(re-search-forward \"^[[:space:]]*;;;###autoload[[:space:]]*$\" nil t)))"

Prints nil.

With every stable version of Emacs I have this prints 16.

Paul

[-- Attachment #2: Type: text/html, Size: 540 bytes --]

bug#66406: Broken `re-search-forward' (?)

[Eli Zaretskii]

> From: Paul Pogonyshev <pogonyshev@gmail.com>
> Date: Sun, 8 Oct 2023 12:35:47 +0200
> 
> In a recent Emacs master (as provided in Silex's Docker image):
> 
> $ emacs --batch --eval "(with-temp-buffer (insert \"\n;;;###autoload\n(defun ...)\n\") (goto-char 1) (print
> (re-search-forward \"^[[:space:]]*;;;###autoload[[:space:]]*$\" nil t)))"
> 
> Prints nil.
> 
> With every stable version of Emacs I have this prints 16.

Adding Stefan, who made some changes there recently.

bug#66406: Broken `re-search-forward' (?)

[Basil L. Contovounesios via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Paul Pogonyshev [2023-10-08 12:35 +0200] wrote:

> In a recent Emacs master (as provided in Silex's Docker image):

Any chance of knowing which commit precisely?

> $ emacs --batch --eval "(with-temp-buffer (insert \"\n;;;###autoload\n(defun ...)\n\") (goto-char 1) (print
> (re-search-forward \"^[[:space:]]*;;;###autoload[[:space:]]*$\" nil t)))"
>
> Prints nil.
>
> With every stable version of Emacs I have this prints 16.

I get 16 with today's master as well
(M-x report-emacs-bug info follows below).

HTH,
-- 
Basil

In GNU Emacs 30.0.50 (build 1, x86_64-pc-linux-gnu, X toolkit, cairo
 version 1.18.0, Xaw3d scroll bars) of 2023-10-08 built on tia
Repository revision: 940df63d2daf50b899a66cd1542bec0c0f3eaebf
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.12101008
System Description: Debian GNU/Linux trixie/sid

Configured using:
 'configure 'CFLAGS=-O2 -ggdb3' -C --prefix=/home/blc/.local
 --enable-checking=structs --with-file-notification=yes
 --with-x-toolkit=lucid --with-x'

Configured features:
ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM GSETTINGS HARFBUZZ JPEG
JSON LCMS2 LIBOTF LIBSELINUX LIBSYSTEMD LIBXML2 M17N_FLT MODULES NOTIFY
INOTIFY PDUMPER PNG RSVG SECCOMP SOUND SQLITE3 THREADS TIFF
TOOLKIT_SCROLL_BARS TREE_SITTER WEBP X11 XAW3D XDBE XIM XINPUT2 XPM
LUCID ZLIB

Important settings:
  value of $LANG: en_IE.UTF-8
  value of $XMODIFIERS: @im=ibus
  locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  minibuffer-regexp-mode: t
  line-number-mode: t
  indent-tabs-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message mailcap yank-media puny dired
dired-loaddefs rfc822 mml mml-sec password-cache epa derived epg rfc6068
epg-config gnus-util text-property-search time-date subr-x mm-decode
mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader
cl-loaddefs cl-lib sendmail rfc2047 rfc2045 ietf-drums mm-util
mail-prsvr mail-utils rmc iso-transl tooltip cconv eldoc paren electric
uniquify ediff-hook vc-hooks lisp-float-type elisp-mode mwheel
term/x-win x-win term/common-win x-dnd touch-screen tool-bar dnd fontset
image regexp-opt fringe tabulated-list replace newcomment text-mode
lisp-mode prog-mode register page tab-bar menu-bar rfn-eshadow isearch
easymenu timer select scroll-bar mouse jit-lock font-lock syntax
font-core term/tty-colors frame minibuffer nadvice seq simple cl-generic
indonesian philippine cham georgian utf-8-lang misc-lang vietnamese
tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek
romanian slovak czech european ethiopic indian cyrillic chinese
composite emoji-zwj charscript charprop case-table epa-hook
jka-cmpr-hook help abbrev obarray oclosure cl-preloaded button loaddefs
theme-loaddefs faces cus-face macroexp files window text-properties
overlay sha1 md5 base64 format env code-pages mule custom widget keymap
hashtable-print-readable backquote threads dbusbind inotify lcms2
dynamic-setting system-font-setting font-render-setting cairo x-toolkit
xinput2 x multi-tty move-toolbar make-network-process emacs)

Memory information:
((conses 16 38056 9275) (symbols 48 5209 0) (strings 32 14029 1781)
 (string-bytes 1 386914) (vectors 16 10297)
 (vector-slots 8 156208 17511) (floats 8 23 23) (intervals 56 259 0)
 (buffers 992 10))

bug#66406: Broken `re-search-forward' (?)

[Paul Pogonyshev]

[-- Attachment #1: Type: text/plain, Size: 4784 bytes --]

> > In a recent Emacs master (as provided in Silex's Docker image):
>
> Any chance of knowing which commit precisely?

Not really, if you evaluate `emacs-repository-version' on
Silex-image-packaged Emacs, result is nil. As far as I know, they are just
built from Emacs VCS snapshot tarballs, but exact commit hash is lost in
the build process.  This was even discussed before:
https://github.com/Silex/docker-emacs/issues/86#issuecomment-1234266607 I
don't know the details, but apparently GNU provides the snapshots ("archive
tarballs are downloaded and unpacked"), so you could just inject revision
number in the proper place and then get better bug reports. I'd provide
commit number if I knew it somehow.

> I get 16 with today's master as well

Then it is apparently fixed already.

Paul

On Sun, 8 Oct 2023 at 21:39, Basil L. Contovounesios <contovob@tcd.ie>
wrote:

> Paul Pogonyshev [2023-10-08 12:35 +0200] wrote:
>
> > In a recent Emacs master (as provided in Silex's Docker image):
>
> Any chance of knowing which commit precisely?
>
> > $ emacs --batch --eval "(with-temp-buffer (insert
> \"\n;;;###autoload\n(defun ...)\n\") (goto-char 1) (print
> > (re-search-forward \"^[[:space:]]*;;;###autoload[[:space:]]*$\" nil t)))"
> >
> > Prints nil.
> >
> > With every stable version of Emacs I have this prints 16.
>
> I get 16 with today's master as well
> (M-x report-emacs-bug info follows below).
>
> HTH,
> --
> Basil
>
> In GNU Emacs 30.0.50 (build 1, x86_64-pc-linux-gnu, X toolkit, cairo
>  version 1.18.0, Xaw3d scroll bars) of 2023-10-08 built on tia
> Repository revision: 940df63d2daf50b899a66cd1542bec0c0f3eaebf
> Repository branch: master
> Windowing system distributor 'The X.Org Foundation', version 11.0.12101008
> System Description: Debian GNU/Linux trixie/sid
>
> Configured using:
>  'configure 'CFLAGS=-O2 -ggdb3' -C --prefix=/home/blc/.local
>  --enable-checking=structs --with-file-notification=yes
>  --with-x-toolkit=lucid --with-x'
>
> Configured features:
> ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM GSETTINGS HARFBUZZ JPEG
> JSON LCMS2 LIBOTF LIBSELINUX LIBSYSTEMD LIBXML2 M17N_FLT MODULES NOTIFY
> INOTIFY PDUMPER PNG RSVG SECCOMP SOUND SQLITE3 THREADS TIFF
> TOOLKIT_SCROLL_BARS TREE_SITTER WEBP X11 XAW3D XDBE XIM XINPUT2 XPM
> LUCID ZLIB
>
> Important settings:
>   value of $LANG: en_IE.UTF-8
>   value of $XMODIFIERS: @im=ibus
>   locale-coding-system: utf-8-unix
>
> Major mode: Lisp Interaction
>
> Minor modes in effect:
>   tooltip-mode: t
>   global-eldoc-mode: t
>   eldoc-mode: t
>   show-paren-mode: t
>   electric-indent-mode: t
>   mouse-wheel-mode: t
>   tool-bar-mode: t
>   menu-bar-mode: t
>   file-name-shadow-mode: t
>   global-font-lock-mode: t
>   font-lock-mode: t
>   blink-cursor-mode: t
>   minibuffer-regexp-mode: t
>   line-number-mode: t
>   indent-tabs-mode: t
>   transient-mark-mode: t
>   auto-composition-mode: t
>   auto-encryption-mode: t
>   auto-compression-mode: t
>
> Load-path shadows:
> None found.
>
> Features:
> (shadow sort mail-extr emacsbug message mailcap yank-media puny dired
> dired-loaddefs rfc822 mml mml-sec password-cache epa derived epg rfc6068
> epg-config gnus-util text-property-search time-date subr-x mm-decode
> mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader
> cl-loaddefs cl-lib sendmail rfc2047 rfc2045 ietf-drums mm-util
> mail-prsvr mail-utils rmc iso-transl tooltip cconv eldoc paren electric
> uniquify ediff-hook vc-hooks lisp-float-type elisp-mode mwheel
> term/x-win x-win term/common-win x-dnd touch-screen tool-bar dnd fontset
> image regexp-opt fringe tabulated-list replace newcomment text-mode
> lisp-mode prog-mode register page tab-bar menu-bar rfn-eshadow isearch
> easymenu timer select scroll-bar mouse jit-lock font-lock syntax
> font-core term/tty-colors frame minibuffer nadvice seq simple cl-generic
> indonesian philippine cham georgian utf-8-lang misc-lang vietnamese
> tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek
> romanian slovak czech european ethiopic indian cyrillic chinese
> composite emoji-zwj charscript charprop case-table epa-hook
> jka-cmpr-hook help abbrev obarray oclosure cl-preloaded button loaddefs
> theme-loaddefs faces cus-face macroexp files window text-properties
> overlay sha1 md5 base64 format env code-pages mule custom widget keymap
> hashtable-print-readable backquote threads dbusbind inotify lcms2
> dynamic-setting system-font-setting font-render-setting cairo x-toolkit
> xinput2 x multi-tty move-toolbar make-network-process emacs)
>
> Memory information:
> ((conses 16 38056 9275) (symbols 48 5209 0) (strings 32 14029 1781)
>  (string-bytes 1 386914) (vectors 16 10297)
>  (vector-slots 8 156208 17511) (floats 8 23 23) (intervals 56 259 0)
>  (buffers 992 10))
>

[-- Attachment #2: Type: text/html, Size: 5804 bytes --]

bug#66406: Broken `re-search-forward' (?)

[Eli Zaretskii]

> Cc: 66406@debbugs.gnu.org
> From: Paul Pogonyshev <pogonyshev@gmail.com>
> Date: Sun, 8 Oct 2023 22:06:23 +0200
> 
> > > In a recent Emacs master (as provided in Silex's Docker image):
> >
> > Any chance of knowing which commit precisely?
> 
> Not really, if you evaluate `emacs-repository-version' on Silex-image-packaged Emacs, result is nil.
> As far as I know, they are just built from Emacs VCS snapshot tarballs, but exact commit hash is lost
> in the build process.  This was even discussed before:
> https://github.com/Silex/docker-emacs/issues/86#issuecomment-1234266607 I don't know the
> details, but apparently GNU provides the snapshots ("archive tarballs are downloaded and
> unpacked"), so you could just inject revision number in the proper place and then get better bug
> reports. I'd provide commit number if I knew it somehow.
> 
> > I get 16 with today's master as well
> 
> Then it is apparently fixed already.

I also get 16, so I presume this bug was already fixed, and I'm
closing it.

bug#66406: Broken `re-search-forward' (?)

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Basil L. Contovounesios [2023-10-08 21:39:44] wrote:

> Paul Pogonyshev [2023-10-08 12:35 +0200] wrote:
>
>> In a recent Emacs master (as provided in Silex's Docker image):
>
> Any chance of knowing which commit precisely?
>
>> $ emacs --batch --eval "(with-temp-buffer (insert
>> \"\n;;;###autoload\n(defun ...)\n\") (goto-char 1) (print
>> (re-search-forward \"^[[:space:]]*;;;###autoload[[:space:]]*$\" nil t)))"
>>
>> Prints nil.
>>
>> With every stable version of Emacs I have this prints 16.
>
> I get 16 with today's master as well
> (M-x report-emacs-bug info follows below).

According, to my test, this was fixed by

    commit 37130fd500fbf78ff0d0037aa6275f0f70a415dd
    Author: Stefan Monnier <monnier@iro.umontreal.ca>
    Date:   Tue Oct 3 10:10:57 2023 -0400
    
        regex.c: Fix recent regression with mutually_exclusive_p
        
        The new analysis code ended up increasing the scope of an optimization
        a bit too far.  Reign it in.
        
        * src/regex-emacs.c (struct mutexcl_data): Add `unconstrained` field.
        (mutually_exclusive_one): Use and set it.
        (mutually_exclusive_p): Initialize it.
        
        * test/src/regex-emacs-tests.el (regexp-tests-backtrack-optimization):
        Add test.


-- Stefan