bug#70948: 29.3; cperl mode hangs on comments like "#___"

bug#70948: 29.3; cperl mode hangs on comments like "#___"

[Patrick Goldmann]

[-- Attachment #1: Type: text/plain, Size: 3219 bytes --]

While trying to open files (with cperl-mode, perl-mode works) containing
comments of this sort, emacs hangs.

This also occurs when editing a new file in cperl-mode and trying to input
a comment like that.

Verified it's not my ~/.emacs.d/ by running emacs -Q

I guess cperl-mode is mistaking this for something like __PACKAGE__ etc


In GNU Emacs 29.3 (build 2, x86_64-w64-mingw32) of 2024-03-24 built on
 AVALON
Windowing system distributor 'Microsoft Corp.', version 10.0.22631
System Description: Microsoft Windows 10 Pro (v10.0.2009.22631.3447)

Configured using:
 'configure --with-modules --without-dbus --with-native-compilation=aot
 --without-compress-install --with-sqlite3 --with-tree-sitter
 CFLAGS=-O2'

Configured features:
ACL GIF GMP GNUTLS HARFBUZZ JPEG JSON LCMS2 LIBXML2 MODULES NATIVE_COMP
NOTIFY W32NOTIFY PDUMPER PNG RSVG SOUND SQLITE3 THREADS TIFF
TOOLKIT_SCROLL_BARS TREE_SITTER WEBP XPM ZLIB

(NATIVE_COMP present but libgccjit not available)

Important settings:
  value of $LANG: DEU
  locale-coding-system: cp1252

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  line-number-mode: t
  indent-tabs-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message mailcap yank-media puny dired
dired-loaddefs rfc822 mml mml-sec password-cache epa derived epg rfc6068
epg-config gnus-util text-property-search time-date subr-x mm-decode
mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader
cl-loaddefs cl-lib sendmail rfc2047 rfc2045 ietf-drums mm-util
mail-prsvr mail-utils rmc iso-transl tooltip cconv eldoc paren electric
uniquify ediff-hook vc-hooks lisp-float-type elisp-mode mwheel dos-w32
ls-lisp disp-table term/w32-win w32-win w32-vars term/common-win
tool-bar dnd fontset image regexp-opt fringe tabulated-list replace
newcomment text-mode lisp-mode prog-mode register page tab-bar menu-bar
rfn-eshadow isearch easymenu timer select scroll-bar mouse jit-lock
font-lock syntax font-core term/tty-colors frame minibuffer nadvice seq
simple cl-generic indonesian philippine cham georgian utf-8-lang
misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms
cp51932 hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese composite emoji-zwj charscript charprop case-table
epa-hook jka-cmpr-hook help abbrev obarray oclosure cl-preloaded button
loaddefs theme-loaddefs faces cus-face macroexp files window
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget keymap hashtable-print-readable backquote threads
w32notify w32 lcms2 multi-tty make-network-process native-compile emacs)

Memory information:
((conses 16 50309 9400)
 (symbols 48 5188 0)
 (strings 32 15119 1435)
 (string-bytes 1 415182)
 (vectors 16 11033)
 (vector-slots 8 262420 20014)
 (floats 8 24 38)
 (intervals 56 237 0)
 (buffers 984 10))

[-- Attachment #2: Type: text/html, Size: 3645 bytes --]

bug#70948: 29.3; cperl mode hangs on comments like "#___"

[Eli Zaretskii]

> From: Patrick Goldmann <patrick.goldmann@gmail.com>
> Date: Tue, 14 May 2024 18:35:48 +0200
> 
> While trying to open files (with cperl-mode, perl-mode works) containing comments of this sort, emacs hangs.
> 
> This also occurs when editing a new file in cperl-mode and trying to input a comment like that.
> 
> Verified it's not my ~/.emacs.d/ by running emacs -Q
> 
> I guess cperl-mode is mistaking this for something like __PACKAGE__ etc

Can you please show an example of such a file?  I tried to reproduce,
perhaps too naïvely, and failed to see Emacs hang.

Thanks.

bug#70948: 29.3; cperl mode hangs on comments like "#___"

[Harald Jörg]

[-- Attachment #1: Type: text/plain, Size: 399 bytes --]

Patrick Goldmann <patrick.goldmann@gmail.com> writes:

> While trying to open files (with cperl-mode, perl-mode works)
> containing comments of this sort, emacs hangs.

Unfortunately I can not reproduce this.  I tried on Windows with exactly
the same Emacs version with the attached file, and it works as expected.

Can you pleas try this file, and provide a file where Emacs hanvs?
--
Cheers,
haj


[-- Attachment #2: Test with #___ comments --]
[-- Type: text/plain, Size: 89 bytes --]

#___ A comment with underscores

package Strange;

#___ Another comment with underscores

bug#70948: 29.3; cperl mode hangs on comments like "#___"

[Patrick Goldmann]

[-- Attachment #1: Type: text/plain, Size: 1152 bytes --]

Just got home and tried to reproduce this on my private machine, it worked
just fine.

So I guess this is something on my work machine, maybe version control
related or something.
Still strange this happened while running emacs -Q.

I can try your file tomorrow, but my guess is if I drop this into one of my
project directories it won't work.
I'll try it outside of version controlled directories, too, and provide an
example file.

But if it turns out this is somehow related to version control or something
else in my project tree, this will be hard to debug and might not even be
directly related to cperl-mode at all :(

On Tue, May 14, 2024 at 8:20 PM Harald Jörg <haj@posteo.de> wrote:

> Patrick Goldmann <patrick.goldmann@gmail.com> writes:
>
> > While trying to open files (with cperl-mode, perl-mode works)
> > containing comments of this sort, emacs hangs.
>
> Unfortunately I can not reproduce this.  I tried on Windows with exactly
> the same Emacs version with the attached file, and it works as expected.
>
> Can you pleas try this file, and provide a file where Emacs hanvs?
> --
> Cheers,
> haj
>
>

[-- Attachment #2: Type: text/html, Size: 1634 bytes --]

bug#70948: 29.3; cperl mode hangs on comments like "#___"

[Patrick Goldmann]

[-- Attachment #1: Type: text/plain, Size: 1731 bytes --]

This was bugging me, so I played around with it a bit more.

It seems this has nothing to do at all with version control etc, but with
the number of underscores.

I could reproduce it on my private machine with a clean emacs 29.3 install
by:
- C-x C-f foo.pl
- M-x cperl-mode
- just keeping underscore pressed

Emacs will slow down eventually and then hang. Haven't waited long enough
to see if it recovers.

Hope this helps.

On Tue, May 14, 2024 at 8:46 PM Patrick Goldmann <patrick.goldmann@gmail.com>
wrote:

> Just got home and tried to reproduce this on my private machine, it worked
> just fine.
>
> So I guess this is something on my work machine, maybe version control
> related or something.
> Still strange this happened while running emacs -Q.
>
> I can try your file tomorrow, but my guess is if I drop this into one of
> my project directories it won't work.
> I'll try it outside of version controlled directories, too, and provide an
> example file.
>
> But if it turns out this is somehow related to version control or
> something else in my project tree, this will be hard to debug and might not
> even be directly related to cperl-mode at all :(
>
> On Tue, May 14, 2024 at 8:20 PM Harald Jörg <haj@posteo.de> wrote:
>
>> Patrick Goldmann <patrick.goldmann@gmail.com> writes:
>>
>> > While trying to open files (with cperl-mode, perl-mode works)
>> > containing comments of this sort, emacs hangs.
>>
>> Unfortunately I can not reproduce this.  I tried on Windows with exactly
>> the same Emacs version with the attached file, and it works as expected.
>>
>> Can you pleas try this file, and provide a file where Emacs hanvs?
>> --
>> Cheers,
>> haj
>>
>>

[-- Attachment #2: Type: text/html, Size: 2624 bytes --]

bug#70948: 29.3; cperl mode hangs on comments like "#___"

[Harald Jörg]

tags 70948 confirmed
owner 70948 !
thanks

Patrick Goldmann <patrick.goldmann@gmail.com> writes:

> This was bugging me, so I played around with it a bit more.
>
> It seems this has nothing to do at all with version control etc, but
> with the number of underscores.
>
> I could reproduce it on my private machine with a clean emacs 29.3 install by:
> - C-x C-f foo.pl
> - M-x cperl-mode
> - just keeping underscore pressed

Ok, I can reproduce it:  I see a delay starting after ~20 underscores
and getting quickly worse as more underscores are added.  My first bet
would be some exponential regexp backtracking.

The slowdown does not happen with Emacs built from the repository under
Linux, nor with Emacs 28.2 under Windows.  I need to run more variations
to narrow it down.
--
Cheers,
haj

bug#70948: 29.3; cperl mode hangs on comments like "#___"

[Harald Jörg]

Harald Jörg <haj@posteo.de> writes:

> [...]
> The slowdown does not happen with Emacs built from the repository under
> Linux, nor with Emacs 28.2 under Windows.  I need to run more variations
> to narrow it down.

A bisection reveals that this problem does no longer occur in the
repository due to commit de6c1c4d by Mattias Engdegård (Aug 12, 2023):
"rx: Better translation of char-matching patterns" with many changes in
rx.el.

This is plausible because many rx expressions have been introduced to
cperl-mode for Emacs 29 to catch up with current Perl syntax.  I guess
that it is possible to rewrite the offending rx expression(s) so that
they do not misbehave under Emacs 29, but need to identify the culprit
now.
-- 
Cheers,
haj

bug#70948: 29.3; cperl mode hangs on comments like "#___"

[Eli Zaretskii]

> From: Harald Jörg <haj@posteo.de>
> Date: Wed, 15 May 2024 21:54:38 +0000
> 
> A bisection reveals that this problem does no longer occur in the
> repository due to commit de6c1c4d by Mattias Engdegård (Aug 12, 2023):
> "rx: Better translation of char-matching patterns" with many changes in
> rx.el.
> 
> This is plausible because many rx expressions have been introduced to
> cperl-mode for Emacs 29 to catch up with current Perl syntax.  I guess
> that it is possible to rewrite the offending rx expression(s) so that
> they do not misbehave under Emacs 29, but need to identify the culprit
> now.

Please note that, barring any unforeseen emergencies, there will be no
more Emacs 29.x releases.  So I'm not sure the effort of making the
emacs-29 branch behave better in this matter is justified.

Thanks.

bug#70948: 29.3; cperl mode hangs on comments like "#___"

[Harald Jörg]

Eli Zaretskii writes:

> Please note that, barring any unforeseen emergencies, there will be no
> more Emacs 29.x releases.  So I'm not sure the effort of making the
> emacs-29 branch behave better in this matter is justified.

I understand!

My suggested course of action would be like this:

1) Make cperl-mode.el from the master branch work with older Emacsen
   (again).  This needs a one-line fix.

2) Work around the inefficient rx expansion in the master branch.  This
   seems to be another one-line fix in which I replace a rx sequence by
   the expansion it gets in master (explanatory comment omitted here):

     (defconst cperl--basic-identifier-rx
   -    '(sequence (or alpha "_") (* (or word "_")))
   +    '(regex "[_[:alpha:]][_[:word:]]*"))

3) Close the bug report, noting it has already been fixed in master.
   Explain that users can use cperl-mode.el from the repository as a
   drop-in replacement in Emacs 27 or newer if they are affected by the
   bug.

Current "stable" Linux distributions ship with Emacs 28 (Debian
bookworm) or Emacs 29 (Ubuntu Noble Numbat). It won't happen too often,
nevertheless I find it unsettling that Emacs would freeze beyond repair
when opening files with a line of underscores.

On a related note, Perl 5.40 will be released this month.  The new
syntax will be added to cperl-mode.el in the master branch, and Perlers
who want current syntax to be recognized seen to have developed the
habit to pick cperl-mode.el from git.  Such things will run very smooth
once we find a way to publish cperl-mode in ELPA (but keep it in the
Emacs repository, of course).
-- 
Cheers,
haj

bug#70948: 29.3; cperl mode hangs on comments like "#___"

[Eli Zaretskii]

> From: Harald Jörg <haj@posteo.de>
> Cc: 70948@debbugs.gnu.org
> Date: Thu, 16 May 2024 16:43:24 +0000
> 
> Eli Zaretskii writes:
> 
> > Please note that, barring any unforeseen emergencies, there will be no
> > more Emacs 29.x releases.  So I'm not sure the effort of making the
> > emacs-29 branch behave better in this matter is justified.
> 
> I understand!
> 
> My suggested course of action would be like this:
> 
> 1) Make cperl-mode.el from the master branch work with older Emacsen
>    (again).  This needs a one-line fix.
> 
> 2) Work around the inefficient rx expansion in the master branch.  This
>    seems to be another one-line fix in which I replace a rx sequence by
>    the expansion it gets in master (explanatory comment omitted here):
> 
>      (defconst cperl--basic-identifier-rx
>    -    '(sequence (or alpha "_") (* (or word "_")))
>    +    '(regex "[_[:alpha:]][_[:word:]]*"))
> 
> 3) Close the bug report, noting it has already been fixed in master.
>    Explain that users can use cperl-mode.el from the repository as a
>    drop-in replacement in Emacs 27 or newer if they are affected by the
>    bug.

SGTM, thanks.

bug#70948: 29.3; cperl mode hangs on comments like "#___"

[Harald Jörg]

Patrick Goldmann writes:

> I could reproduce it on my private machine with a clean emacs 29.3 install by:
> - C-x C-f foo.pl
> - M-x cperl-mode
> - just keeping underscore pressed
>
> Emacs will slow down eventually and then hang. Haven't waited long enough to see if it
> recovers.
>
> Hope this helps.

The bug has already been fixed in the Emacs source code.  For Emacs 29,
we have a workaround: You can pull cperl-mode.el from the Emacs source
code repository:
https://git.savannah.gnu.org/cgit/emacs.git/plain/lisp/progmodes/cperl-mode.el
and put it on your load path before the cperl-mode.el from the
distribution.

I just followed that procedure, it works for Emacs 29 (and Emacs 27, 28) on
Windows (and Linux).

Thanks for reporting!
-- 
Cheers,
haj