bug#62300: 29.0.60; No hyperlinks for some symbols in *Help* buffers

bug#62300: 29.0.60; No hyperlinks for some symbols in *Help* buffers

[Eli Zaretskii]

To reproduce:

  emacs -Q
  C-h f global-text-scale-adjust RET

Observe that in the *Help* buffer the variable
global-text-scale-adjust-resizes-frames does not have the link
appearance.  This is because:

  (boundp 'global-text-scale-adjust-resizes-frames) => nil

By contrast, if you try

  C-h f text-scale-adjust RET

then the variable text-scale-mode-step in the *Help* buffer does get
the link appearance, and boundp returns non-nil for that variable.

The reason seems to be that in the latter case typing the "C-h f"
command causes face-remap.el to be loaded, whereas in the former case
face-remap.el is not loaded.  I traced that to a problem which can be
demonstrated by the following recipe:

  emacs -Q
  M-x load-library RET help-fns RET

Now evaluate:

 (radix-tree-prefixes (help-definition-prefixes) "global-text-scale-adjust")

This returns nil, whereas if you do the same with "text-scale-adjust",
you get:

  (("text-scale-" "face-remap") ("tex" "flyspell"))

Interestingly, just appending a dash to global-text-scale-adjust, i.e.

 (radix-tree-prefixes (help-definition-prefixes) "global-text-scale-adjust-")

does yield non-nil result:

  (("global-text-scale-adjust-" "face-remap"))

The non-nil result causes help-fns.el:help--symbol-completion-table to
load the file:

    (when help-enable-completion-autoload
      (let ((prefixes (radix-tree-prefixes (help-definition-prefixes) string)))
        (help--load-prefixes prefixes)))

The load happens inside help--load-prefixes.  As result face-remap.el
is loaded for text-scale-adjust, and the variables defined in
face-remap.el become defined.  With global-text-scale-adjust the
loading doesn't happen, and the variable stays unbound.

This sounds like some snafu with some heuristic somewhere, perhaps in
radix-tree-prefixes, or in the code which registers the prefixes to
begin with?

Can this be fixed, please, so that variables referenced in doc strings
would reliably be displayed as links?

In GNU Emacs 29.0.60 (build 400, i686-pc-mingw32) of 2023-03-20 built on
 HOME-C4E4A596F7
Repository revision: 786de66ec3c4cff90cafd0f8a68f9bce027e9947
Repository branch: emacs-29
Windowing system distributor 'Microsoft Corp.', version 5.1.2600
System Description: Microsoft Windows XP Service Pack 3 (v5.1.0.2600)

Configured using:
 'configure -C --prefix=/d/usr --with-wide-int
 --enable-checking=yes,glyphs 'CFLAGS=-O0 -gdwarf-4 -g3''

Configured features:
ACL GIF GMP GNUTLS HARFBUZZ JPEG JSON LCMS2 LIBXML2 MODULES NOTIFY
W32NOTIFY PDUMPER PNG RSVG SOUND SQLITE3 THREADS TIFF
TOOLKIT_SCROLL_BARS TREE_SITTER WEBP XPM ZLIB

Important settings:
  value of $LANG: ENU
  locale-coding-system: cp1255

Major mode: Fundamental

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  line-number-mode: t
  indent-tabs-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message mailcap yank-media puny dired
dired-loaddefs rfc822 mml mml-sec password-cache epa epg rfc6068
epg-config gnus-util text-property-search time-date mm-decode mm-bodies
mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail
rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils dabbrev
misearch multi-isearch pp cl-macs derived pcase subr-x help-fns
radix-tree cl-print byte-opt gv bytecomp byte-compile debug backtrace
help-mode find-func cl-loaddefs cl-lib rmc iso-transl tooltip cconv
eldoc paren electric uniquify ediff-hook vc-hooks lisp-float-type
elisp-mode mwheel dos-w32 ls-lisp disp-table term/w32-win w32-win
w32-vars term/common-win tool-bar dnd fontset image regexp-opt fringe
tabulated-list replace newcomment text-mode lisp-mode prog-mode register
page tab-bar menu-bar rfn-eshadow isearch easymenu timer select
scroll-bar mouse jit-lock font-lock syntax font-core term/tty-colors
frame minibuffer nadvice seq simple cl-generic indonesian philippine
cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech
european ethiopic indian cyrillic chinese composite emoji-zwj charscript
charprop case-table epa-hook jka-cmpr-hook help abbrev obarray oclosure
cl-preloaded button loaddefs theme-loaddefs faces cus-face macroexp
files window text-properties overlay sha1 md5 base64 format env
code-pages mule custom widget keymap hashtable-print-readable backquote
threads w32notify w32 lcms2 multi-tty make-network-process emacs)

Memory information:
((conses 16 170820 17000)
 (symbols 48 7828 0)
 (strings 16 25056 2790)
 (string-bytes 1 712251)
 (vectors 16 12758)
 (vector-slots 8 191144 13690)
 (floats 8 36 30)
 (intervals 40 10017 117)
 (buffers 888 13))

bug#62300: 29.0.60; No hyperlinks for some symbols in *Help* buffers

[Eli Zaretskii]

> Date: Mon, 20 Mar 2023 20:34:04 +0200
> From: Eli Zaretskii <eliz@gnu.org>
> 
> To reproduce:
> 
>   emacs -Q
>   C-h f global-text-scale-adjust RET
> 
> Observe that in the *Help* buffer the variable
> global-text-scale-adjust-resizes-frames does not have the link
> appearance.  This is because:
> 
>   (boundp 'global-text-scale-adjust-resizes-frames) => nil
> 
> By contrast, if you try
> 
>   C-h f text-scale-adjust RET
> 
> then the variable text-scale-mode-step in the *Help* buffer does get
> the link appearance, and boundp returns non-nil for that variable.
> 
> The reason seems to be that in the latter case typing the "C-h f"
> command causes face-remap.el to be loaded, whereas in the former case
> face-remap.el is not loaded.  I traced that to a problem which can be
> demonstrated by the following recipe:
> 
>   emacs -Q
>   M-x load-library RET help-fns RET
> 
> Now evaluate:
> 
>  (radix-tree-prefixes (help-definition-prefixes) "global-text-scale-adjust")
> 
> This returns nil, whereas if you do the same with "text-scale-adjust",
> you get:
> 
>   (("text-scale-" "face-remap") ("tex" "flyspell"))
> 
> Interestingly, just appending a dash to global-text-scale-adjust, i.e.
> 
>  (radix-tree-prefixes (help-definition-prefixes) "global-text-scale-adjust-")
> 
> does yield non-nil result:
> 
>   (("global-text-scale-adjust-" "face-remap"))
> 
> The non-nil result causes help-fns.el:help--symbol-completion-table to
> load the file:
> 
>     (when help-enable-completion-autoload
>       (let ((prefixes (radix-tree-prefixes (help-definition-prefixes) string)))
>         (help--load-prefixes prefixes)))
> 
> The load happens inside help--load-prefixes.  As result face-remap.el
> is loaded for text-scale-adjust, and the variables defined in
> face-remap.el become defined.  With global-text-scale-adjust the
> loading doesn't happen, and the variable stays unbound.
> 
> This sounds like some snafu with some heuristic somewhere, perhaps in
> radix-tree-prefixes, or in the code which registers the prefixes to
> begin with?
> 
> Can this be fixed, please, so that variables referenced in doc strings
> would reliably be displayed as links?

Adding Stefan.

Stefan, any suggestions or ideas?

bug#62300: 29.0.60; No hyperlinks for some symbols in *Help* buffers

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

>>   emacs -Q
>>   C-h f global-text-scale-adjust RET
>> 
>> Observe that in the *Help* buffer the variable
>> global-text-scale-adjust-resizes-frames does not have the link
>> appearance.  This is because:
>> 
>>   (boundp 'global-text-scale-adjust-resizes-frames) => nil

`help-definition-prefixes` and friends
(`help-enable-completion-autoload`, ...) are the result of a tradeoff:
we offer to autoload files "on demand" but the "demand" is often
vague/implicit, so we have to judge when the demand is clear enough to
justify loading a file and when it's not.

If we're too trigger happy, we can end up auto-loading all the .el files
in sight, making Emacs unnecessarily bigger&slower (and increasing the
risk that we bump into a file that breaks the convention, such as
`c-ts-mode`).

In this case, `global-text-scale-adjust` has an explicit autoload in
`loaddefs.el` so we already have the docstring needed to display
`C-h f global-text-scale-adjust RET` without having to load
`face-remap.el`, so `help-enable-completion-autoload` doesn't load
`face-remap.el`.

>> By contrast, if you try
>> 
>>   C-h f text-scale-adjust RET
>> 
>> then the variable text-scale-mode-step in the *Help* buffer does get
>> the link appearance, and boundp returns non-nil for that variable.

Among the prefixes registered for `face-remap.el` there is `text-scale-`
because there are various other functions beside `text-scale-adjust`
which start with `text-scale-`.  I'm not completely sure why we end up
loading `face-remap.el` here (it doesn't seem necessary since the
function is also autoloaded), but this difference in the
definition-prefixes is probably the reason for the difference
in behavior (apparently completion gets involved somewhere even tho
it's not needed).

>>  (radix-tree-prefixes (help-definition-prefixes) "global-text-scale-adjust")
>> 
>> This returns nil, whereas if you do the same with "text-scale-adjust",
>> you get:
>> 
>>   (("text-scale-" "face-remap") ("tex" "flyspell"))
>> 
>> Interestingly, just appending a dash to global-text-scale-adjust, i.e.
>> 
>>  (radix-tree-prefixes (help-definition-prefixes) "global-text-scale-adjust-")

Yup: in `face-remap.el`, all the definitions that aren't known before
the file is loaded (i.e. not autoloaded) and whose name starts with
"global-text-" also start  with "global-text-scale-adjust-", so the
definition-prefixes data registers this longer prefix, which is more
precise.

BTW, this is also related to the part of `C-h f` which loads autoloaded
functions when it sees something like \\< or \\[ (this is controlled by
`help-enable-autoload`).  We could change it so it does it also when it
sees `...' in the docstring.


        Stefan

bug#62300: 29.0.60; No hyperlinks for some symbols in *Help* buffers

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: 62300@debbugs.gnu.org
> Date: Mon, 20 Mar 2023 17:55:40 -0400
> 
> >>   emacs -Q
> >>   C-h f global-text-scale-adjust RET
> >> 
> >> Observe that in the *Help* buffer the variable
> >> global-text-scale-adjust-resizes-frames does not have the link
> >> appearance.  This is because:
> >> 
> >>   (boundp 'global-text-scale-adjust-resizes-frames) => nil
> 
> `help-definition-prefixes` and friends
> (`help-enable-completion-autoload`, ...) are the result of a tradeoff:
> we offer to autoload files "on demand" but the "demand" is often
> vague/implicit, so we have to judge when the demand is clear enough to
> justify loading a file and when it's not.
> 
> If we're too trigger happy, we can end up auto-loading all the .el files
> in sight, making Emacs unnecessarily bigger&slower (and increasing the
> risk that we bump into a file that breaks the convention, such as
> `c-ts-mode`).
> 
> In this case, `global-text-scale-adjust` has an explicit autoload in
> `loaddefs.el` so we already have the docstring needed to display
> `C-h f global-text-scale-adjust RET` without having to load
> `face-remap.el`, so `help-enable-completion-autoload` doesn't load
> `face-remap.el`.

So you are saying that the only way of having
global-text-scale-adjust-resizes-frames decorated as a link is to
autoload it?  Even though autoloading defcustoms is frowned upon?  Or
is there a better way?

bug#62300: 29.0.60; No hyperlinks for some symbols in *Help* buffers

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> So you are saying that the only way of having
> global-text-scale-adjust-resizes-frames decorated as a link is to
> autoload it?

Not at all, no, I must have been unclear somewhere (maybe because
there's "autoload" as in using a `;;;###autoload` cookie somewhere, and
"autoload" as in having help-fns.el `load`ing a file in response to
something the user did in order to try and get more information about
the contents in that file).

I was saying that we can avoid the discrepancy either by:
- Change `help-enable-autoload` to also cause `help-fns.el` to (auto)load
  `face-remap.el` when you do `C-h f global-text-scale-adjust RET`
- To try and figure out why `C-h f text-scale-adjust RET`
  causes `help-fns.el` to (auto)load `face-remap.el` (presumably
  conditioned on `help-enable-completion-autoload`) and change it so
  it behaves like `C-h f global-text-scale-adjust RET` instead.


        Stefan

bug#62300: 29.0.60; No hyperlinks for some symbols in *Help* buffers

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: 62300@debbugs.gnu.org
> Date: Tue, 21 Mar 2023 10:19:55 -0400
> 
> > So you are saying that the only way of having
> > global-text-scale-adjust-resizes-frames decorated as a link is to
> > autoload it?
> 
> Not at all, no, I must have been unclear somewhere (maybe because
> there's "autoload" as in using a `;;;###autoload` cookie somewhere, and
> "autoload" as in having help-fns.el `load`ing a file in response to
> something the user did in order to try and get more information about
> the contents in that file).
> 
> I was saying that we can avoid the discrepancy either by:
> - Change `help-enable-autoload` to also cause `help-fns.el` to (auto)load
>   `face-remap.el` when you do `C-h f global-text-scale-adjust RET`
> - To try and figure out why `C-h f text-scale-adjust RET`
>   causes `help-fns.el` to (auto)load `face-remap.el` (presumably
>   conditioned on `help-enable-completion-autoload`) and change it so
>   it behaves like `C-h f global-text-scale-adjust RET` instead.

The latter means a regression, since symbols mentioned in the doc
string of text-scale-adjust will not be buttonized, like symbols in
the doc string of global-text-scale-adjust aren't now.  I'd like to
find a solution that would cause symbols in both doc strings to be
buttonized (autoloading the defcustom does).

bug#62300: 29.0.60; No hyperlinks for some symbols in *Help* buffers

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> The latter means a regression, since symbols mentioned in the doc
> string of text-scale-adjust will not be buttonized, like symbols in
> the doc string of global-text-scale-adjust aren't now.  I'd like to
> find a solution that would cause symbols in both doc strings to be
> buttonized (autoloading the defcustom does).

The difference between the two use cases is largely accidental (due to
details of how the definition-prefixes functionality works).  I don't
think tweaking the definition-prefixes would be wise way to solve
this problem.

(describe-function 'text-scale-adjust) suffers from the same "problem".
We have not treated this as a problem until now, but if we want to treat
it as a problem, then I suggest we do it by refining the way
`help-enable-autoload` works.

E.g. with the patch below.


        Stefan


diff --git a/lisp/help-fns.el b/lisp/help-fns.el
index a81051cee03..083acc5c98c 100644
--- a/lisp/help-fns.el
+++ b/lisp/help-fns.el
@@ -1138,7 +1138,7 @@ describe-function-1
     ;; key substitution constructs, load the library.
     (and (autoloadp real-def) doc-raw
          help-enable-autoload
-         (string-match "\\([^\\]=\\|[^=]\\|\\`\\)\\\\[[{<]" doc-raw)
+         (string-match "\\([^\\]=\\|[^=]\\|\\`\\)\\\\[[{<]\\|`.*'" doc-raw)
          (autoload-do-load real-def))
 
     (help-fns--key-bindings function)

bug#62300: 29.0.60; No hyperlinks for some symbols in *Help* buffers

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: 62300@debbugs.gnu.org
> Date: Tue, 21 Mar 2023 13:59:50 -0400
> 
> > The latter means a regression, since symbols mentioned in the doc
> > string of text-scale-adjust will not be buttonized, like symbols in
> > the doc string of global-text-scale-adjust aren't now.  I'd like to
> > find a solution that would cause symbols in both doc strings to be
> > buttonized (autoloading the defcustom does).
> 
> The difference between the two use cases is largely accidental (due to
> details of how the definition-prefixes functionality works).  I don't
> think tweaking the definition-prefixes would be wise way to solve
> this problem.
> 
> (describe-function 'text-scale-adjust) suffers from the same "problem".
> We have not treated this as a problem until now, but if we want to treat
> it as a problem, then I suggest we do it by refining the way
> `help-enable-autoload` works.
> 
> E.g. with the patch below.

Thanks.  Do you consider this safe enough for the emacs-29 branch?

bug#62300: 29.0.60; No hyperlinks for some symbols in *Help* buffers

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

>> E.g. with the patch below.
> Thanks.  Do you consider this safe enough for the emacs-29 branch?

The risk is very small, but the gain is also very small (the problem
is definitely not a regression: AFAICT this is unchanged since Emacs-26
and before that the result was more consistent but with fewer links).

I'd keep it on `master`, personally.


        Stefan

bug#62300: 29.0.60; No hyperlinks for some symbols in *Help* buffers

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: 62300@debbugs.gnu.org
> Date: Tue, 21 Mar 2023 15:09:00 -0400
> 
> >> E.g. with the patch below.
> > Thanks.  Do you consider this safe enough for the emacs-29 branch?
> 
> The risk is very small, but the gain is also very small (the problem
> is definitely not a regression: AFAICT this is unchanged since Emacs-26
> and before that the result was more consistent but with fewer links).
> 
> I'd keep it on `master`, personally.

OK, master it is, then.

bug#62300: 29.0.60; No hyperlinks for some symbols in *Help* buffers

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> OK, master it is, then.

Pushed,


        Stefan

bug#62300: 29.0.60; No hyperlinks for some symbols in *Help* buffers

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: 62300@debbugs.gnu.org
> Date: Tue, 21 Mar 2023 21:48:42 -0400
> 
> > OK, master it is, then.
> 
> Pushed,

Thanks, closing.