bug#49426: 28.0.50; Byte-compilation can fail with large fill-column

bug#49426: 28.0.50; Byte-compilation can fail with large fill-column

[Basil L. Contovounesios]

[-- Attachment #1: Type: text/plain, Size: 1578 bytes --]

Tags: patch

As seen on the Dash issue tracker:

https://github.com/magnars/dash.el/issues/383

0. cd "$(mktemp -d)"
1. HOME="$PWD" XDG_CONFIG_HOME="$PWD/.config" emacs -Q
2. (setq-default
    package-archives '(("melpa" . "https://melpa.org/packages/"))
    fill-column #xffff)
3. C-x C-e
4. M-x package-install RET dash RET

Installation fails with the following in *Compile-Log*:

  Compiling file
  /tmp/tmp.IEFN9nE4Fs/.emacs.d/elpa/dash-20210704.1302/dash.el
  at Tue Jul 6 00:44:46 2021
  Entering directory
  ‘/tmp/tmp.IEFN9nE4Fs/.emacs.d/elpa/dash-20210704.1302/’
  dash.el:551:1: Error: Invalid regexp: "Invalid content of \\{\\}"

The same error occurs post-installation with 'M-x byte-compile-file' on
dash.el.  Either way, the error confusingly only shows up when Dash is
installed from MELPA:

  package-archives = '(("melpa" . "https://melpa.org/packages/"))

and not from GNU-devel ELPA:

  package-archives = '(("gnu" . "https://elpa.gnu.org/devel/"))

even though both contain identical versions of the file dash.el.  So I
guess it's some other part of the package .tar that gives rise to the
discrepancy.

Turning on debug-on-error doesn't give a backtrace, presumably because
displaying-byte-compile-warnings or similar demotes errors.

But a breakpoint in signal_or_quit reveals that
byte-compile--wide-docstring-p constructs an invalid regexp whose
repetition exceeds the new (since Emacs 27) RE_DUP_MAX limit of #xffff.

Maybe byte-compile--wide-docstring-p should defend itself against this
as follows?


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Avoid-invalid-regexp-in-wide-docstring-check.patch --]
[-- Type: text/x-diff, Size: 1099 bytes --]

From 42de0117547a462ce1f161a3a9cae92fd322e9ec Mon Sep 17 00:00:00 2001
From: "Basil L. Contovounesios" <contovob@tcd.ie>
Date: Tue, 6 Jul 2021 01:05:41 +0100
Subject: [PATCH] Avoid invalid regexp in wide docstring check

* lisp/emacs-lisp/bytecomp.el (byte-compile--wide-docstring-p):
Avoid constructing an invalid regexp during byte-compilation by
limiting the number of columns to the current RE_DUP_MAX of 65535.
This protects against pathological values of fill-column, for
example.
---
 lisp/emacs-lisp/bytecomp.el | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lisp/emacs-lisp/bytecomp.el b/lisp/emacs-lisp/bytecomp.el
index 70999648d4..6970c8a505 100644
--- a/lisp/emacs-lisp/bytecomp.el
+++ b/lisp/emacs-lisp/bytecomp.el
@@ -1627,7 +1627,7 @@ byte-compile--wide-docstring-p
 `byte-compile--wide-docstring-substitution-len'.  Also ignore
 URLs."
   (string-match
-   (format "^.\\{%s,\\}$" (int-to-string (1+ col)))
+   (format "^.\\{%d,\\}$" (min (1+ col) #xffff)) ; Heed RE_DUP_MAX.
    (replace-regexp-in-string
     (rx (or
          ;; Ignore some URLs.
-- 
2.30.2


[-- Attachment #3: Type: text/plain, Size: 3549 bytes --]


Thanks,

-- 
Basil

In GNU Emacs 28.0.50 (build 1, x86_64-pc-linux-gnu, X toolkit, cairo version 1.16.0, Xaw3d scroll bars)
 of 2021-07-05 built on tia
Repository revision: 68276f6d30bbdc09cc26fb49d7f0c3aa4bce35f2
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.12011000
System Description: Debian GNU/Linux 11 (bullseye)

Configured using:
 'configure 'CC=ccache gcc' 'CFLAGS=-Og -ggdb' --config-cache
 --prefix=/home/blc/.local --enable-checking=structs
 --with-x-toolkit=lucid --with-file-notification=yes --with-x'

Configured features:
ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM GSETTINGS HARFBUZZ JPEG
JSON LCMS2 LIBOTF LIBSELINUX LIBSYSTEMD LIBXML2 M17N_FLT MODULES NOTIFY
INOTIFY PDUMPER PNG RSVG SECCOMP SOUND THREADS TIFF TOOLKIT_SCROLL_BARS
X11 XAW3D XDBE XIM XPM LUCID ZLIB

Important settings:
  value of $LANG: en_IE.UTF-8
  value of $XMODIFIERS: @im=ibus
  locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug sendmail warnings compile comint
ansi-color ring info find-func easy-mmode autoload radix-tree lisp-mnt
tar-mode arc-mode archive-mode cus-edit pp cus-start cus-load wid-edit
mm-archive message dired dired-loaddefs rfc822 mml mml-sec epa derived
gnus-util rmail rmail-loaddefs text-property-search time-date mailabbrev
gmm-utils mailheader mm-decode mm-bodies mm-encode mail-utils gnutls
network-stream url-http mail-parse rfc2231 rfc2047 rfc2045 mm-util
ietf-drums mail-prsvr url-gw nsm rmc puny url-cache url-auth epg
epg-config finder-inf package browse-url url url-proxy url-privacy
url-expand url-methods url-history url-cookie url-domsuf url-util
mailcap url-handlers url-parse auth-source cl-seq eieio eieio-core
cl-macs eieio-loaddefs password-cache json subr-x map url-vars seq
byte-opt gv bytecomp byte-compile cconv cl-loaddefs cl-lib iso-transl
tooltip eldoc electric uniquify ediff-hook vc-hooks lisp-float-type
mwheel term/x-win x-win term/common-win x-dnd tool-bar dnd fontset image
regexp-opt fringe tabulated-list replace newcomment text-mode elisp-mode
lisp-mode prog-mode register page tab-bar menu-bar rfn-eshadow isearch
easymenu timer select scroll-bar mouse jit-lock font-lock syntax
font-core term/tty-colors frame minibuffer cl-generic cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese eucjp-ms cp51932 hebrew greek romanian slovak czech european
ethiopic indian cyrillic chinese composite charscript charprop
case-table epa-hook jka-cmpr-hook help simple abbrev obarray
cl-preloaded nadvice button loaddefs faces cus-face macroexp files
window text-properties overlay sha1 md5 base64 format env code-pages
mule custom widget hashtable-print-readable backquote threads dbusbind
inotify lcms2 dynamic-setting system-font-setting font-render-setting
cairo x-toolkit x multi-tty make-network-process emacs)

Memory information:
((conses 16 243317 18548)
 (symbols 48 15849 1)
 (strings 32 83582 1296)
 (string-bytes 1 2065467)
 (vectors 16 24463)
 (vector-slots 8 318902 13721)
 (floats 8 39 454)
 (intervals 56 285 0)
 (buffers 992 15))

bug#49426: 28.0.50; Byte-compilation can fail with large fill-column

[Basil L. Contovounesios]

"Basil L. Contovounesios" <contovob@tcd.ie> writes:

> The same error occurs post-installation with 'M-x byte-compile-file' on
> dash.el.  Either way, the error confusingly only shows up when Dash is
> installed from MELPA:
>
>   package-archives = '(("melpa" . "https://melpa.org/packages/"))
>
> and not from GNU-devel ELPA:
>
>   package-archives = '(("gnu" . "https://elpa.gnu.org/devel/"))
>
> even though both contain identical versions of the file dash.el.  So I
> guess it's some other part of the package .tar that gives rise to the
> discrepancy.

Duh, the GNU ELPA .tar includes a .dir-locals.el with a custom
fill-column, whereas the MELPA package installs only dash.el and its
Texinfo manual, so there is nothing to override the pathological default
value of fill-column.

In which case the recipe becomes much simpler:

0. emacs -Q
1. M-: (setq-default fill-column 9999999) RET
2. (defalias 'foo #'ignore "Foo.")
3. C-x C-s foo.el RET
4. M-x byte-compile-file RET foo.el RET

This gives:

Compiling file /home/blc/foo.el at Tue Jul  6 09:05:44 2021
foo.el:4:1: Error: Invalid regexp: "Invalid content of \\{\\}"

-- 
Basil

bug#49426: 28.0.50; Byte-compilation can fail with large fill-column

[Lars Ingebrigtsen]

"Basil L. Contovounesios" <contovob@tcd.ie> writes:

> Maybe byte-compile--wide-docstring-p should defend itself against this
> as follows?

Makes sense to me.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#49426: 28.0.50; Byte-compilation can fail with large fill-column

[Basil L. Contovounesios]

close 49426 28.1
quit

Lars Ingebrigtsen <larsi@gnus.org> writes:

> "Basil L. Contovounesios" <contovob@tcd.ie> writes:
>
>> Maybe byte-compile--wide-docstring-p should defend itself against this
>> as follows?
>
> Makes sense to me.

Thanks, pushed and closing.

Avoid invalid regexp in wide docstring check
044742bfe8 2021-07-06 18:56:15 +0100
https://git.sv.gnu.org/cgit/emacs.git/commit/?id=044742bfe8c7c22e303242c40e16fbe9e564727a

-- 
Basil