bug#72296: 29.4; cperl-mode parsing error

bug#72296: 29.4; cperl-mode parsing error

[brian]

I started emacs using 'emacs -Q /tmp/some-empty-file.pl' and activated
cperl mode by 'M-x cperl-mode'. I typed 'for (2..$n/2) {}' into the
empty buffer and as soon as I pressed '/' there was an error message in
the minibuffer

End of ‘/ ... /’ string/RE not found: (scan-error Unbalanced parentheses 
11 38)

and syntax hilighting was broken. Inserting a single space before '$'
cleared the error and fixed the hilighting.



In GNU Emacs 29.4 (build 2, x86_64-pc-linux-gnu, GTK+ Version 3.24.38,
cairo version 1.18.0) of 2024-06-24 built on dev
Windowing system distributor 'The X.Org Foundation', version 11.0.12302000
System Description: Ubuntu 23.10

Configured using:
'configure --prefix=/opt/emacs --with-cairo --with-libsystemd
--with-mailutils --with-native-compilation --with-pop=yes
--with-sound=alsa --with-toolkit-scroll-bars --with-x-toolkit=gtk3
--with-x=yes --without-gconf 'CFLAGS=-g -O2''

Configured features:
CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GSETTINGS HARFBUZZ JPEG JSON
LIBSELINUX LIBXML2 MODULES NATIVE_COMP NOTIFY INOTIFY PDUMPER PNG
SECCOMP SOUND SQLITE3 THREADS TIFF TOOLKIT_SCROLL_BARS TREE_SITTER WEBP
X11 XDBE XIM XINPUT2 XPM GTK3 ZLIB

Important settings:
value of $LANG: en_GB.UTF-8
value of $XMODIFIERS: @im=ibus
locale-coding-system: utf-8-unix

Major mode: CPerl

Minor modes in effect:
rainbow-delimiters-mode: t
global-hl-line-mode: t
savehist-mode: t
which-key-mode: t
marginalia-mode: t
vertico-mode: t
yas-global-mode: t
yas-minor-mode: t
global-flycheck-mode: t
flycheck-mode: t
global-display-fill-column-indicator-mode: t
display-fill-column-indicator-mode: t
global-display-line-numbers-mode: t
display-line-numbers-mode: t
override-global-mode: t
tooltip-mode: t
global-eldoc-mode: t
show-paren-mode: t
electric-indent-mode: t
mouse-wheel-mode: t
file-name-shadow-mode: t
global-font-lock-mode: t
font-lock-mode: t
blink-cursor-mode: t
column-number-mode: t
line-number-mode: t
transient-mark-mode: t
auto-composition-mode: t
auto-encryption-mode: t
auto-compression-mode: t

Load-path shadows:
/home/brian/.emacs.d/elpa/transient-20240603.2200/transient hides 
/opt/emacs/share/emacs/29.4/lisp/transient

Features:
(shadow sort mail-extr emacsbug message yank-media puny dired
dired-loaddefs rfc822 mml mml-sec epa derived epg rfc6068 epg-config
gnus-util text-property-search mailabbrev gmm-utils mailheader sendmail
mail-utils mule-util orderless comp comp-cstr warnings cperl-mode
facemenu rainbow-delimiters hl-line savehist which-key move-text
org-tempo org ob ob-tangle ob-ref ob-lob ob-table ob-exp org-macro
org-src ob-comint org-pcomplete pcomplete comint ansi-osc ring org-list
org-footnote org-faces org-entities time-date noutline outline
ob-emacs-lisp ob-core ob-eval org-cycle org-table ol org-fold
org-fold-core org-keys oc org-loaddefs cal-menu calendar cal-loaddefs
org-compat tempo org-version org-macs format-spec marginalia vertico
compat yasnippet-snippets yasnippet diminish flycheck ansi-color
find-func rx display-fill-column-indicator display-line-numbers
ef-dark-theme ef-themes cus-edit pp cus-load icons wid-edit edmacro
kmacro quelpa-use-package cl-extra quelpa mm-decode mm-bodies mm-encode
mail-parse rfc2231 rfc2047 rfc2045 mm-util ietf-drums mail-prsvr
lisp-mnt help-fns radix-tree help-mode use-package use-package-ensure
use-package-delight use-package-diminish use-package-bind-key bind-key
easy-mmode use-package-core finder-inf all-the-icons-dired-autoloads
all-the-icons-autoloads auctex-autoloads tex-site avy-autoloads
cdlatex-autoloads corfu-autoloads diminish-autoloads ef-themes-autoloads
embark-consult-autoloads consult-autoloads embark-autoloads
flycheck-autoloads helpful-autoloads elisp-refs-autoloads jinx-autoloads
json-mode-autoloads keycast-autoloads logos-autoloads
magit-todos-autoloads magit-autoloads pcase git-commit-autoloads
hl-todo-autoloads f-autoloads async-autoloads marginalia-autoloads
markdown-mode-autoloads move-text-autoloads orderless-autoloads
org-modern-autoloads osm-autoloads pcre2el-autoloads pdf-tools-autoloads
perltidy-autoloads quelpa-use-package-autoloads quelpa-autoloads
rainbow-delimiters-autoloads realgud-autoloads
realgud-recursive-autoloads loc-changes-autoloads
load-relative-autoloads s-autoloads spacious-padding-autoloads
svg-lib-autoloads tablist-autoloads taxy-magit-section-autoloads
taxy-autoloads magit-section-autoloads dash-autoloads
test-simple-autoloads transient-autoloads treepy-autoloads
vertico-autoloads which-key-autoloads with-editor-autoloads info
compat-autoloads ws-butler-autoloads yaml-autoloads yaml-mode-autoloads
yasnippet-snippets-autoloads yasnippet-autoloads package browse-url url
url-proxy url-privacy url-expand url-methods url-history url-cookie
generate-lisp-file url-domsuf url-util mailcap url-handlers url-parse
auth-source cl-seq eieio eieio-core cl-macs password-cache json subr-x
map byte-opt gv bytecomp byte-compile url-vars cl-loaddefs cl-lib rmc
iso-transl tooltip cconv eldoc paren electric uniquify ediff-hook
vc-hooks lisp-float-type elisp-mode mwheel term/x-win x-win
term/common-win x-dnd tool-bar dnd fontset image regexp-opt fringe
tabulated-list replace newcomment text-mode lisp-mode prog-mode register
page tab-bar menu-bar rfn-eshadow isearch easymenu timer select
scroll-bar mouse jit-lock font-lock syntax font-core term/tty-colors
frame minibuffer nadvice seq simple cl-generic indonesian philippine
cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech
european ethiopic indian cyrillic chinese composite emoji-zwj charscript
charprop case-table epa-hook jka-cmpr-hook help abbrev obarray oclosure
cl-preloaded button loaddefs theme-loaddefs faces cus-face macroexp
files window text-properties overlay sha1 md5 base64 format env
code-pages mule custom widget keymap hashtable-print-readable backquote
threads dbusbind inotify dynamic-setting system-font-setting
font-render-setting cairo move-toolbar gtk x-toolkit xinput2 x multi-tty
make-network-process native-compile emacs)

Memory information:
((conses 16 507989 330210)
(symbols 48 28255 420)
(strings 32 141117 47270)
(string-bytes 1 4103290)
(vectors 16 44098)
(vector-slots 8 844183 452452)
(floats 8 157 942)
(intervals 56 373 368)
(buffers 984 12))

bug#72296: 29.4; cperl-mode parsing error

[Andrea Corallo]

brian <me@briang.org> writes:

> I started emacs using 'emacs -Q /tmp/some-empty-file.pl' and activated
> cperl mode by 'M-x cperl-mode'. I typed 'for (2..$n/2) {}' into the
> empty buffer and as soon as I pressed '/' there was an error message in
> the minibuffer
>
> End of ‘/ ... /’ string/RE not found: (scan-error Unbalanced
> parentheses 11 38)
>
> and syntax hilighting was broken. Inserting a single space before '$'
> cleared the error and fixed the hilighting.
>
>
>
> In GNU Emacs 29.4 (build 2, x86_64-pc-linux-gnu, GTK+ Version 3.24.38,
> cairo version 1.18.0) of 2024-06-24 built on dev
> Windowing system distributor 'The X.Org Foundation', version 11.0.12302000
> System Description: Ubuntu 23.10

I might be doing something wrong but I cannot reproduce on emacs-30,
could you confirm?  29 is not under development anymore.

  Andrea

bug#72296: 29.4; cperl-mode parsing error

[Eli Zaretskii]

> Cc: 72296@debbugs.gnu.org
> From: Andrea Corallo <acorallo@gnu.org>
> Date: Thu, 25 Jul 2024 13:01:30 -0400
> 
> brian <me@briang.org> writes:
> 
> > I started emacs using 'emacs -Q /tmp/some-empty-file.pl' and activated
> > cperl mode by 'M-x cperl-mode'. I typed 'for (2..$n/2) {}' into the
> > empty buffer and as soon as I pressed '/' there was an error message in
> > the minibuffer
> >
> > End of ‘/ ... /’ string/RE not found: (scan-error Unbalanced
> > parentheses 11 38)
> >
> > and syntax hilighting was broken. Inserting a single space before '$'
> > cleared the error and fixed the hilighting.
> >
> >
> >
> > In GNU Emacs 29.4 (build 2, x86_64-pc-linux-gnu, GTK+ Version 3.24.38,
> > cairo version 1.18.0) of 2024-06-24 built on dev
> > Windowing system distributor 'The X.Org Foundation', version 11.0.12302000
> > System Description: Ubuntu 23.10
> 
> I might be doing something wrong but I cannot reproduce on emacs-30,
> could you confirm?  29 is not under development anymore.

I can reproduce on master.  Be sure to invoke cperl-mode, and type
exactly the text the OP shows.

bug#72296: 29.4; cperl-mode parsing error

[Harald Jörg]

tags 72296 confirmed
severity 72296 normal
owner 72296 Harald Jörg <haj@posteo.de>
thanks

brian <me@briang.org> writes:

> I started emacs using 'emacs -Q /tmp/some-empty-file.pl' and activated
> cperl mode by 'M-x cperl-mode'. I typed 'for (2..$n/2) {}' into the
> empty buffer and as soon as I pressed '/' there was an error message in
> the minibuffer
>
> End of ‘/ ... /’ string/RE not found: (scan-error Unbalanced
> parentheses 11 38)
>
> and syntax hilighting was broken. Inserting a single space before '$'
> cleared the error and fixed the hilighting.

I can reproduce this and am about to prepare a fix.

That code sits here since 1997, I find it rather surprising that it has
not surfaced until now.  I'll run some tests and add a test case.  If I
am not mistaken, then replacing a `progn` with `save-excursion` should
do the trick:

index b85db699e72..34481925d3e 100644
--- a/lisp/progmodes/cperl-mode.el
+++ b/lisp/progmodes/cperl-mode.el
@@ -4108,7 +4108,7 @@ cperl-find-pods-heres
                                    (and (eq (preceding-char) ?\})
                                         (cperl-after-block-p (point-min)))
                                    (and (eq (char-syntax (preceding-char)) ?w)
-                                        (progn
+                                        (save-excursion
                                           (forward-sexp -1)
 ;; After these keywords `/' starts a RE.  One should add all the
 ;; functions/builtins which expect an argument, but ...

In newer versions the line numbers are different (~4650) but the issue
is the same.  The unprotected (forward-sexp -1) skips over $n and the
two characters before that are '..' which, in the following clause,
makes the construct look like a flip-flop operator where '/' does start
a regular expression.  Using 'save-excursion' undoes the skip before
running the next checks.

-- 
Cheers,
haj

bug#72296: 29.4; cperl-mode parsing error

[Andrea Corallo]

Eli Zaretskii <eliz@gnu.org> writes:

>> Cc: 72296@debbugs.gnu.org
>> From: Andrea Corallo <acorallo@gnu.org>
>> Date: Thu, 25 Jul 2024 13:01:30 -0400
>> 
>> brian <me@briang.org> writes:
>> 
>> > I started emacs using 'emacs -Q /tmp/some-empty-file.pl' and activated
>> > cperl mode by 'M-x cperl-mode'. I typed 'for (2..$n/2) {}' into the
>> > empty buffer and as soon as I pressed '/' there was an error message in
>> > the minibuffer
>> >
>> > End of ‘/ ... /’ string/RE not found: (scan-error Unbalanced
>> > parentheses 11 38)
>> >
>> > and syntax hilighting was broken. Inserting a single space before '$'
>> > cleared the error and fixed the hilighting.
>> >
>> >
>> >
>> > In GNU Emacs 29.4 (build 2, x86_64-pc-linux-gnu, GTK+ Version 3.24.38,
>> > cairo version 1.18.0) of 2024-06-24 built on dev
>> > Windowing system distributor 'The X.Org Foundation', version 11.0.12302000
>> > System Description: Ubuntu 23.10
>> 
>> I might be doing something wrong but I cannot reproduce on emacs-30,
>> could you confirm?  29 is not under development anymore.
>
> I can reproduce on master.  Be sure to invoke cperl-mode, and type
> exactly the text the OP shows.

So I do but can't reproduce, I must be doing something wrong sorry 🤷

bug#72296: 29.4; cperl-mode parsing error

[Andrea Corallo]

brian greenfield <dev@briang.org> writes:

> On Thu, Jul 25, 2024 at 01:01:30PM -0400, Andrea Corallo wrote:
>> I might be doing something wrong but I cannot reproduce on emacs-30,
>> could you confirm?  29 is not under development anymore.
>
> No, it was me in the wrong. My emacs-30 was too old, retrying on a newer
> emacs-30 has the bug fixed.
>
> Sorry for the noise.

[re-adding the list, please keep it Cced in the replies]

Actually the others can still reproduce on more recent versions (I
can't), so I'm all but sure this is solved (and I'm a little puzzled on
the reproducibility of this).

  Andrea

bug#72296: 29.4; cperl-mode parsing error

[Harald Jörg]

Andrea Corallo <acorallo@gnu.org> writes:

> brian greenfield <dev@briang.org> writes:
>
>> On Thu, Jul 25, 2024 at 01:01:30PM -0400, Andrea Corallo wrote:
>>> I might be doing something wrong but I cannot reproduce on emacs-30,
>>> could you confirm?  29 is not under development anymore.
>>
>> No, it was me in the wrong. My emacs-30 was too old, retrying on a newer
>> emacs-30 has the bug fixed.
>>
>> Sorry for the noise.
>
> [re-adding the list, please keep it Cced in the replies]
>
> Actually the others can still reproduce on more recent versions (I
> can't), so I'm all but sure this is solved (and I'm a little puzzled on
> the reproducibility of this).

If either of you built your emacs-30 with the commit c27055a9 in place,
then it is supposed to be not reproducible: I fixed it yesterday.

Until the regular rebasing happens, it can still be reproduced on
master.  I may be stating the obvious, but the code is

for (2..$n/2) {}

without any enclosing quotes.  If you apply cperl-mode for this snippet,
you should see the message "End of ‘/ ... /’ string/RE not found:
(scan-error Unbalanced parentheses 11 17)" in the echo area, the slash
should be formatted with font-lock-constant-face, the 2 following it
with font-lock-string-face and the closing paren with
font-lock-warning-face.

If you can not reproduce the error on master, then I'd like to examine
this further before closing the bug.

-- 
Cheers,
haj

bug#72296: 29.4; cperl-mode parsing error

[Eli Zaretskii]

> Cc: 72296@debbugs.gnu.org
> From: Andrea Corallo <acorallo@gnu.org>
> Date: Fri, 26 Jul 2024 04:03:35 -0400
> 
> brian greenfield <dev@briang.org> writes:
> 
> > On Thu, Jul 25, 2024 at 01:01:30PM -0400, Andrea Corallo wrote:
> >> I might be doing something wrong but I cannot reproduce on emacs-30,
> >> could you confirm?  29 is not under development anymore.
> >
> > No, it was me in the wrong. My emacs-30 was too old, retrying on a newer
> > emacs-30 has the bug fixed.
> >
> > Sorry for the noise.
> 
> [re-adding the list, please keep it Cced in the replies]
> 
> Actually the others can still reproduce on more recent versions (I
> can't), so I'm all but sure this is solved (and I'm a little puzzled on
> the reproducibility of this).

Wasn't this fixed today on the emacs-30 branch?

bug#72296: Acknowledgement (29.4; cperl-mode parsing error)

[brian greenfield]

I compiled emacs-30 (87389f9ff90) and master (52cae67e1e7) in the last couple of
hours, and tested with 'for (2..$n/2) {}'.

emacs-30 is fixed: the spurious error has gone, and hilighting is working as
expected. master is still broken.

bug#72296: Acknowledgement (29.4; cperl-mode parsing error)

[Eli Zaretskii]

> Date: Fri, 26 Jul 2024 14:21:28 +0100
> From: brian greenfield <dev@briang.org>
> Cc: Harald Jörg <haj@posteo.de>,
> 	Andrea Corallo <acorallo@gnu.org>, Eli Zaretskii <eliz@gnu.org>
> 
> I compiled emacs-30 (87389f9ff90) and master (52cae67e1e7) in the last couple of
> hours, and tested with 'for (2..$n/2) {}'.
> 
> emacs-30 is fixed: the spurious error has gone, and hilighting is working as
> expected. master is still broken.

The fix was installed on the emacs-30 branch, it will be merged to
master soon.

bug#72296: 29.4; cperl-mode parsing error

[Eli Zaretskii]

> Cc: brian <me@briang.org>
> From: Harald Jörg <haj@posteo.de>
> Date: Thu, 25 Jul 2024 19:06:35 +0000
> 
> brian <me@briang.org> writes:
> 
> > I started emacs using 'emacs -Q /tmp/some-empty-file.pl' and activated
> > cperl mode by 'M-x cperl-mode'. I typed 'for (2..$n/2) {}' into the
> > empty buffer and as soon as I pressed '/' there was an error message in
> > the minibuffer
> >
> > End of ‘/ ... /’ string/RE not found: (scan-error Unbalanced
> > parentheses 11 38)
> >
> > and syntax hilighting was broken. Inserting a single space before '$'
> > cleared the error and fixed the hilighting.
> 
> I can reproduce this and am about to prepare a fix.
> 
> That code sits here since 1997, I find it rather surprising that it has
> not surfaced until now.  I'll run some tests and add a test case.  If I
> am not mistaken, then replacing a `progn` with `save-excursion` should
> do the trick:
> 
> index b85db699e72..34481925d3e 100644
> --- a/lisp/progmodes/cperl-mode.el
> +++ b/lisp/progmodes/cperl-mode.el
> @@ -4108,7 +4108,7 @@ cperl-find-pods-heres
>                                     (and (eq (preceding-char) ?\})
>                                          (cperl-after-block-p (point-min)))
>                                     (and (eq (char-syntax (preceding-char)) ?w)
> -                                        (progn
> +                                        (save-excursion
>                                            (forward-sexp -1)
>  ;; After these keywords `/' starts a RE.  One should add all the
>  ;; functions/builtins which expect an argument, but ...
> 
> In newer versions the line numbers are different (~4650) but the issue
> is the same.  The unprotected (forward-sexp -1) skips over $n and the
> two characters before that are '..' which, in the following clause,
> makes the construct look like a flip-flop operator where '/' does start
> a regular expression.  Using 'save-excursion' undoes the skip before
> running the next checks.

Thanks, since this appears to be fixed on the emacs-30 branch, I'm
closing this bug.