bug#73872: 30.0.91; emacs-lisp-mode-syntax-table active when calling `describe-variable' on variable with textually large value.

bug#73872: 30.0.91; emacs-lisp-mode-syntax-table active when calling `describe-variable' on variable with textually large value.

[Sigurd Dam Sonniks]

[-- Attachment #1: Type: text/plain, Size: 7869 bytes --]

Hello Emacs folks!

When you call `describle-variable' on a variable with a value that's
large enough that it's moved to the end of the help buffer, the syntax
table is changed to `emacs-lisp-mode-syntax-table' but never changed
back. There is a very easy fix where you just set it back to
`help-mode-syntax-table' at the end of the enclosing
`with-current-buffer', but I guess you could also switch to a
temporary buffer instead. Below is the relevant code snippet with the
first suggested fix applied.

;; If the value is large, move it to the end.
(with-current-buffer standard-output
      (when (> (count-lines (point-min) (point-max)) 10)
            ;; Note that setting the syntax table like below
            ;; makes forward-sexp move over a `'s' at the end
            ;; of a symbol.
            (set-syntax-table emacs-lisp-mode-syntax-table)
            (goto-char val-start-pos)
            (when (looking-at "value is") (replace-match ""))
            (save-excursion
              (insert "\n\nValue:")
      (setq-local help-button-cache (point-marker)))
            (insert "value is shown ")
            (insert-button "below"
                               'action help-button-cache
                               'follow-link t
                               'help-echo "mouse-2, RET: show value")
            (insert ".\n")
    (set-syntax-table help-mode-syntax-table)))


Thank you very much for all your work. I absolutely adore Emacs.

Kind regards,
Sigurd Dam Sonniks

In GNU Emacs 30.0.91 (build 1, x86_64-pc-linux-gnu, X toolkit, cairo
version 1.18.2, Xaw3d scroll bars)
Windowing system distributor 'The X.Org Foundation', version 11.0.12101013
System Description: NixOS 24.11 (Vicuna)

Configured using:
 'configure
 --prefix=/nix/store/6qc388fy8gl95smi8q0az92d558nkp4k-emacs-30.0.91
 --disable-build-details --with-modules --with-x-toolkit=lucid
 --with-cairo --with-xft --with-compress-install
 --with-toolkit-scroll-bars --with-native-compilation
 --without-imagemagick --with-mailutils --without-small-ja-dic
 --with-tree-sitter --with-xinput2 --without-xwidgets --with-dbus
 --with-selinux'

Configured features:
CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM GSETTINGS HARFBUZZ JPEG
LIBOTF LIBSELINUX LIBSYSTEMD LIBXML2 M17N_FLT MODULES NATIVE_COMP NOTIFY
INOTIFY PDUMPER PNG RSVG SECCOMP SOUND SQLITE3 THREADS TIFF
TOOLKIT_SCROLL_BARS TREE_SITTER WEBP X11 XAW3D XDBE XIM XINPUT2 XPM
LUCID ZLIB

Important settings:
  value of $EMACSLOADPATH: /nix/store/ml4x8pxvz40klw1yabgbjrrpjxlgbbb0-emacs-packages-deps/share/emacs/site-lisp:
  value of $EMACSNATIVELOADPATH: /nix/store/ml4x8pxvz40klw1yabgbjrrpjxlgbbb0-emacs-packages-deps/share/emacs/native-lisp:
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  minibuffer-regexp-mode: t
  line-number-mode: t
  indent-tabs-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t

Load-path shadows:
/nix/store/ml4x8pxvz40klw1yabgbjrrpjxlgbbb0-emacs-packages-deps/share/emacs/site-lisp/elpa/external-completion-0.1/external-completion hides /nix/store/6qc388fy8gl95smi8q0az92d558nkp4k-emacs-30.0.91/share/emacs/30.0.91/lisp/external-completion
/nix/store/ml4x8pxvz40klw1yabgbjrrpjxlgbbb0-emacs-packages-deps/share/emacs/site-lisp/elpa/jsonrpc-1.0.25/jsonrpc hides /nix/store/6qc388fy8gl95smi8q0az92d558nkp4k-emacs-30.0.91/share/emacs/30.0.91/lisp/jsonrpc
/nix/store/ml4x8pxvz40klw1yabgbjrrpjxlgbbb0-emacs-packages-deps/share/emacs/site-lisp/elpa/transient-20240819.1250/transient hides /nix/store/6qc388fy8gl95smi8q0az92d558nkp4k-emacs-30.0.91/share/emacs/30.0.91/lisp/transient
/nix/store/ml4x8pxvz40klw1yabgbjrrpjxlgbbb0-emacs-packages-deps/share/emacs/site-lisp/elpa/eglot-1.17/eglot hides /nix/store/6qc388fy8gl95smi8q0az92d558nkp4k-emacs-30.0.91/share/emacs/30.0.91/lisp/progmodes/eglot
/nix/store/ml4x8pxvz40klw1yabgbjrrpjxlgbbb0-emacs-packages-deps/share/emacs/site-lisp/elpa/project-0.11.1/project hides /nix/store/6qc388fy8gl95smi8q0az92d558nkp4k-emacs-30.0.91/share/emacs/30.0.91/lisp/progmodes/project
/nix/store/ml4x8pxvz40klw1yabgbjrrpjxlgbbb0-emacs-packages-deps/share/emacs/site-lisp/elpa/xref-1.7.0/xref hides /nix/store/6qc388fy8gl95smi8q0az92d558nkp4k-emacs-30.0.91/share/emacs/30.0.91/lisp/progmodes/xref
/nix/store/ml4x8pxvz40klw1yabgbjrrpjxlgbbb0-emacs-packages-deps/share/emacs/site-lisp/elpa/flymake-1.3.7/flymake hides /nix/store/6qc388fy8gl95smi8q0az92d558nkp4k-emacs-30.0.91/share/emacs/30.0.91/lisp/progmodes/flymake
/nix/store/ml4x8pxvz40klw1yabgbjrrpjxlgbbb0-emacs-packages-deps/share/emacs/site-lisp/elpa/compat-30.0.0.0/compat hides /nix/store/6qc388fy8gl95smi8q0az92d558nkp4k-emacs-30.0.91/share/emacs/30.0.91/lisp/emacs-lisp/compat
/nix/store/ml4x8pxvz40klw1yabgbjrrpjxlgbbb0-emacs-packages-deps/share/emacs/site-lisp/elpa/seq-2.24/seq hides /nix/store/6qc388fy8gl95smi8q0az92d558nkp4k-emacs-30.0.91/share/emacs/30.0.91/lisp/emacs-lisp/seq
/nix/store/ml4x8pxvz40klw1yabgbjrrpjxlgbbb0-emacs-packages-deps/share/emacs/site-lisp/elpa/eldoc-1.15.0/eldoc hides /nix/store/6qc388fy8gl95smi8q0az92d558nkp4k-emacs-30.0.91/share/emacs/30.0.91/lisp/emacs-lisp/eldoc

Features:
(mailalias rmail auth-source cl-seq eieio eieio-core cl-macs icons json
map byte-opt gv bytecomp byte-compile thingatpt help-fns radix-tree
help-mode pp shadow sort mail-extr emacsbug message mailcap yank-media
puny dired dired-loaddefs rfc822 mml mml-sec password-cache epa derived
epg rfc6068 epg-config gnus-util text-property-search time-date subr-x
mm-decode mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils
mailheader cl-loaddefs cl-lib sendmail rfc2047 rfc2045 ietf-drums
mm-util mail-prsvr mail-utils rmc iso-transl tooltip cconv eldoc paren
electric uniquify ediff-hook vc-hooks lisp-float-type elisp-mode mwheel
term/x-win x-win term/common-win x-dnd touch-screen tool-bar dnd fontset
image regexp-opt fringe tabulated-list replace newcomment text-mode
lisp-mode prog-mode register page tab-bar menu-bar rfn-eshadow isearch
easymenu timer select scroll-bar mouse jit-lock font-lock syntax
font-core term/tty-colors frame minibuffer nadvice seq simple cl-generic
indonesian philippine cham georgian utf-8-lang misc-lang vietnamese
tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek
romanian slovak czech european ethiopic indian cyrillic chinese
composite emoji-zwj charscript charprop case-table epa-hook
jka-cmpr-hook help abbrev obarray oclosure cl-preloaded button loaddefs
theme-loaddefs faces cus-face macroexp files window text-properties
overlay sha1 md5 base64 format env code-pages mule custom widget keymap
hashtable-print-readable backquote threads dbusbind inotify
dynamic-setting system-font-setting font-render-setting cairo x-toolkit
xinput2 x multi-tty move-toolbar make-network-process native-compile
emacs)

Memory information:
((conses 16 83734 22304) (symbols 48 7341 9) (strings 32 24215 2414)
 (string-bytes 1 761000) (vectors 16 14459)
 (vector-slots 8 178912 15298) (floats 8 32 11) (intervals 56 330 0)
 (buffers 992 12))

[-- Attachment #2: Type: text/html, Size: 22333 bytes --]

bug#73872: 30.0.91; emacs-lisp-mode-syntax-table active when calling `describe-variable' on variable with textually large value.

[Eli Zaretskii]

> From: Sigurd Dam Sonniks <sigurddam@hotmail.com>
> Date: Fri, 18 Oct 2024 22:14:21 +0000
> 
> When you call `describle-variable' on a variable with a value that's
> large enough that it's moved to the end of the help buffer, the syntax
> table is changed to `emacs-lisp-mode-syntax-table' but never changed
> back. There is a very easy fix where you just set it back to
> `help-mode-syntax-table' at the end of the enclosing
> `with-current-buffer', but I guess you could also switch to a
> temporary buffer instead. Below is the relevant code snippet with the
> first suggested fix applied.

Shouldn't the syntax table be left at that value until the buffer is
reused for another variable/function?  Otherwise, why do we set the
syntax table to this value?  Stefan, do you know the reason?

bug#73872: 30.0.91; emacs-lisp-mode-syntax-table active when calling `describe-variable' on variable with textually large value.

[Eli Zaretskii]

Ping!  Sigurd, could you please answer Stefan's questions?

> Cc: Sigurd Dam Sonniks <sigurddam@hotmail.com>, 73872@debbugs.gnu.org
> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Date: Thu, 14 Nov 2024 09:40:48 -0500
> 
> >> When you call `describle-variable' on a variable with a value that's
> >> large enough that it's moved to the end of the help buffer, the syntax
> >> table is changed to `emacs-lisp-mode-syntax-table' but never changed
> >> back. There is a very easy fix where you just set it back to
> >> `help-mode-syntax-table' at the end of the enclosing
> >> `with-current-buffer', but I guess you could also switch to a
> >> temporary buffer instead. Below is the relevant code snippet with the
> >> first suggested fix applied.
> 
> I'm curious: in which context did you notice this?
> Does it lead to an undesirable behavior in a specific case?
> 
> > Shouldn't the syntax table be left at that value until the buffer is
> > reused for another variable/function?  Otherwise, why do we set the
> > syntax table to this value?  Stefan, do you know the reason?
> 
> FWIW, according to my reading of the Git history, this behavior dates
> back at least to 2001 when Miles split `help.el` into several files
> (I didn't bother to check the previous history of that code on
> `help.el`), so it's far from new.
> 
> AFAICT the syntax-table of the *Help* buffer shouldn't matter very much
> once the buffer is filled: it affects only navigation by the user, where
> the prose part would presumably prefer a syntax-table somewhat like that
> of `text-mode` whereas the var's value part would prefer something like
> that of `elisp-mode`.
> 
> But, AFAICT the `help-mode-syntax-table` happens to be a derivative of
> `emacs-lisp-mode-syntax-table` just with extra matching of ‘...’, so
> reverting to it would be safe enough in most cases.  This said, I think
> The Right Thing™ would be to add a `syntax-table` property on the
> printed value (regardless if it stays at the beginning or is moved to
> the end of the buffer) so it's not affected by the syntax-table used in
> the rest of the buffer.
> 
> 
>         Stefan
> 
> 
> 
> 
> 

bug#73872: 30.0.91; emacs-lisp-mode-syntax-table active when calling `describe-variable' on variable with textually large value.

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: Eli Zaretskii <eliz@gnu.org>
> Date: Sun, 01 Dec 2024 18:41:23 -0500
> 
> >> Ping!  Sigurd, could you please answer Stefan's questions?
> > Oh, I am so sorry, Stefan's mail had ended up in my junk folder. I
> > thought we were still waiting for his reply. Thanks for pinging me!
> >> > I'm curious: in which context did you notice this?
> >> > Does it lead to an undesirable behavior in a specific case?
> > I have rainbow-delimiters-mode from the rainbow-delimiters package
> > enabled everywhere, which means that font lock is enabled everywhere.
> > The syntax table of help mode has ?; as a comment starter, so wherever
> > it is used in docstrings, the remainder of that line will be
> > highlighted as a comment in the help buffer. I changed the syntax
> > class of ?; in help-mode-syntax-table, but I noticed that sometimes it
> > would still be interpreted as a comment starter, which led me to
> > finding the bug.
> 
> Aahhhh!
> So the underlying problem already bite in the usual case but you worked
> around it by changing `help-mode-syntax-table`.
> I think your change to `help-mode-syntax-table` is about right, so we
> should include it in any fix to this bug.

I was going to install that change, but then I noticed that it removed
the call to terpri.  Is that intentional?

(And why did we start discussing this in private email, without CC'ing
the bug tracker?  I've now added it; please use Reply All to reply.)

bug#73872: 30.0.91; emacs-lisp-mode-syntax-table active when calling `describe-variable' on variable with textually large value.

[Eli Zaretskii]

Ping! Can we please make progress with this bug report?

> Cc: sigurddam@hotmail.com, 73872@debbugs.gnu.org
> Date: Sat, 07 Dec 2024 15:10:53 +0200
> From: Eli Zaretskii <eliz@gnu.org>
> 
> > From: Stefan Monnier <monnier@iro.umontreal.ca>
> > Cc: Eli Zaretskii <eliz@gnu.org>
> > Date: Sun, 01 Dec 2024 18:41:23 -0500
> > 
> > >> Ping!  Sigurd, could you please answer Stefan's questions?
> > > Oh, I am so sorry, Stefan's mail had ended up in my junk folder. I
> > > thought we were still waiting for his reply. Thanks for pinging me!
> > >> > I'm curious: in which context did you notice this?
> > >> > Does it lead to an undesirable behavior in a specific case?
> > > I have rainbow-delimiters-mode from the rainbow-delimiters package
> > > enabled everywhere, which means that font lock is enabled everywhere.
> > > The syntax table of help mode has ?; as a comment starter, so wherever
> > > it is used in docstrings, the remainder of that line will be
> > > highlighted as a comment in the help buffer. I changed the syntax
> > > class of ?; in help-mode-syntax-table, but I noticed that sometimes it
> > > would still be interpreted as a comment starter, which led me to
> > > finding the bug.
> > 
> > Aahhhh!
> > So the underlying problem already bite in the usual case but you worked
> > around it by changing `help-mode-syntax-table`.
> > I think your change to `help-mode-syntax-table` is about right, so we
> > should include it in any fix to this bug.
> 
> I was going to install that change, but then I noticed that it removed
> the call to terpri.  Is that intentional?
> 
> (And why did we start discussing this in private email, without CC'ing
> the bug tracker?  I've now added it; please use Reply All to reply.)
> 
> 
> 
> 

bug#73872: 30.0.91; emacs-lisp-mode-syntax-table active when calling `describe-variable' on variable with textually large value.

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

>> Aahhhh!
>> So the underlying problem already bite in the usual case but you worked
>> around it by changing `help-mode-syntax-table`.
>> I think your change to `help-mode-syntax-table` is about right, so we
>> should include it in any fix to this bug.
> I was going to install that change, but then I noticed that it removed
> the call to terpri.  Is that intentional?

[ I don't think the `terpri` was meant to be removed, no.  ]

Here's the patch I suggest we install (obviously not on `emacs-30`).


        Stefan


2024-12-21  Stefan Monnier  <monnier@iro.umontreal.ca>

    * lisp/help-mode.el (help-mode-syntax-table): Mark `;` as punctuation.
    (help-make-xrefs): Use `with-syntax-table`.

    * lisp/help-fns.el (describe-variable): Don't change the buffer's
    syntax-table when moving the var's value to the end (bug#73872).
    Put a `syntax-table` property on the var's value so sexp navigation
    does not depend on the help-mode-syntax-table.


diff --git a/lisp/help-fns.el b/lisp/help-fns.el
index c87c86bae84..ae6b77e7849 100644
--- a/lisp/help-fns.el
+++ b/lisp/help-fns.el
@@ -1437,21 +1437,29 @@ describe-variable
 			     (format-message "`%s'" rep)
 			   rep)))
                       (start (point)))
-		  (if (< (+ (length print-rep) (point) (- line-beg)) 68)
-		      (insert " " print-rep)
-		    (terpri)
-                    (let ((buf (current-buffer)))
-                      (with-temp-buffer
-                        (lisp-data-mode)
-                        (set-syntax-table emacs-lisp-mode-syntax-table)
-                        (insert print-rep)
-                        (pp-buffer)
-                        (font-lock-ensure)
-                        (let ((pp-buffer (current-buffer)))
-                          (with-current-buffer buf
-                            (insert-buffer-substring pp-buffer)))))
-                    ;; Remove trailing newline.
-                    (and (= (char-before) ?\n) (delete-char -1)))
+		 (let (beg)
+		   (if (< (+ (length print-rep) (point) (- line-beg)) 68)
+		       (progn
+		         (setq beg (1+ (point)))
+		         (insert " " print-rep))
+		     (terpri)
+                     (setq beg (point))
+                     (let ((buf (current-buffer)))
+                       (with-temp-buffer
+                         (lisp-data-mode)
+                         (insert print-rep)
+                         (pp-buffer)
+                         (font-lock-ensure)
+                         (let ((pp-buffer (current-buffer)))
+                           (with-current-buffer buf
+                             (insert-buffer-substring pp-buffer))))))
+                   ;; Remove trailing newline.
+                   (and (= (char-before) ?\n) (delete-char -1))
+                   ;; Put a `syntax-table' property on the data, as
+                   ;; a kind of poor man's multi-major-mode support here.
+                   (put-text-property beg (point)
+		                      'syntax-table
+		                      lisp-data-mode-syntax-table))
                   (help-fns--editable-variable start (point)
                                                variable val buffer)
 		  (let* ((sv (get variable 'standard-value))
@@ -1515,10 +1523,6 @@ describe-variable
 	    ;; If the value is large, move it to the end.
 	    (with-current-buffer standard-output
 	      (when (> (count-lines (point-min) (point-max)) 10)
-		;; Note that setting the syntax table like below
-		;; makes forward-sexp move over a `'s' at the end
-		;; of a symbol.
-		(set-syntax-table emacs-lisp-mode-syntax-table)
 		(goto-char val-start-pos)
 		(when (looking-at "value is") (replace-match ""))
 		(save-excursion
diff --git a/lisp/help-mode.el b/lisp/help-mode.el
index 33b8eccab2c..aa705bf56d1 100644
--- a/lisp/help-mode.el
+++ b/lisp/help-mode.el
@@ -143,6 +143,8 @@ help-mode-syntax-table
     ;; break when a quoted string contains punctuation.
     (modify-syntax-entry ?‘ "(’  " table)
     (modify-syntax-entry ?’ ")‘  " table)
+    ;; `;' doesn't start a comment (bug#73872).
+    (modify-syntax-entry ?\; "." table)
     table)
   "Syntax table used in `help-mode'.")
 
@@ -587,13 +587,10 @@
       ;; Skip the first bit, which has already been buttonized.
       (forward-paragraph)
       (let ((old-modified (buffer-modified-p)))
-        (let ((stab (syntax-table))
-              (case-fold-search t)
+        (let ((case-fold-search t)
               (inhibit-read-only t))
-          (set-syntax-table help-mode-syntax-table)
+          (with-syntax-table help-mode-syntax-table
           ;; The following should probably be abstracted out.
-          (unwind-protect
-              (progn
                 ;; Info references
                 (save-excursion
                   (while (re-search-forward help-xref-info-regexp nil t)
@@ -679,7 +676,6 @@
                     (let ((sym (intern-soft (match-string 1))))
                       (if (fboundp sym)
                           (help-xref-button 1 'help-function sym))))))
-            (set-syntax-table stab))
           ;; Delete extraneous newlines at the end of the docstring
           (goto-char (point-max))
           (while (and (not (bobp)) (bolp))