bug#64143: 29.0.60; find-function-search-for-symbol truncates library path

bug#64143: 29.0.60; find-function-search-for-symbol truncates library path

[Troy Brown]

When user-emacs-directory contains "emacs.el" in it's path (e.g.,
"~/.emacs.elpa"), which is much easier these days with the availability
of --init-directory, the find-function-search-for-symbol truncates the
library path after "emacs".

I was experimenting with a different user directory (i.e.,
"--init-directory=~/.emacs.elpa") and was surprised to find out that the
Help system was breaking when using this path.  When looking up a symbol
in the Help system (e.g., "C-h f company-mode RET") and then attempting
to visit the source file from the help window (i.e., help-view-source),
if the symbol belongs to an installed package (where the source is
under the "elpa" directory beneath user-emacs-directory), it will
generate the following error message:

  Can't find library: /home/troy/.emacs

Toggling "debug-on-error" and repeating the above, I can generate a
backtrace for this as follows:

Debugger entered--Lisp error: (file-error "Can't find library"
"/home/troy/.emacs")
  find-library-name("/home/troy/.emacs")
  #<subr find-function-search-for-symbol>(company-mode nil
"/home/troy/.emacs.elpa/elpa/company-20230416.2034/company.el")
  ad-Advice-find-function-search-for-symbol(#<subr
find-function-search-for-symbol> company-mode nil
"/home/troy/.emacs.elpa/elpa/company-20230416.2034/company.el")
  apply(ad-Advice-find-function-search-for-symbol #<subr
find-function-search-for-symbol> (company-mode nil
"/home/troy/.emacs.elpa/elpa/company-20230416.2034/company.el"))
  find-function-search-for-symbol(company-mode nil
"/home/troy/.emacs.elpa/elpa/company-20230416.2034/company.el")
  help-function-def--button-function(company-mode
"/home/troy/.emacs.elpa/elpa/company-20230416.2034/company.el" nil)
  help-view-source()
  funcall-interactively(help-view-source)
  call-interactively(help-view-source nil nil)
  command-execute(help-view-source)

I believe the following snippet of code in
find-function-search-for-symbol is to blame for this:

    (when (string-match "\\.emacs\\(.el\\)" library)
      (setq library (substring library 0 (match-beginning 1))))

The regex likely should be improved to only match .emacs or .emacs.el at
the end of the path, not in the middle of it.

bug#64143: 29.0.60; find-function-search-for-symbol truncates library path

[Eli Zaretskii]

> From: Troy Brown <brownts@troybrown.dev>
> Date: Sun, 18 Jun 2023 00:45:12 -0400
> 
> When user-emacs-directory contains "emacs.el" in it's path (e.g.,
> "~/.emacs.elpa"), which is much easier these days with the availability
> of --init-directory, the find-function-search-for-symbol truncates the
> library path after "emacs".

What do you mean by "truncates the library path"?  Do you mean
truncation of the file name 'emacs.el' to yield just 'emacs'? or do
you mean something else?  (To avoid confusion, please don't use "path"
to refer to file names, only to refer to lists of directories in the
style of PATH or load-path.)

> I was experimenting with a different user directory (i.e.,
> "--init-directory=~/.emacs.elpa") and was surprised to find out that the
> Help system was breaking when using this path.  When looking up a symbol
> in the Help system (e.g., "C-h f company-mode RET") and then attempting
> to visit the source file from the help window (i.e., help-view-source),
> if the symbol belongs to an installed package (where the source is
> under the "elpa" directory beneath user-emacs-directory), it will
> generate the following error message:
> 
>   Can't find library: /home/troy/.emacs

This says it cannot find ".emacs", not "emacs".  So is your init file
called ".emacs.el" or (as you mentioned above) "emacs.el"?

> Toggling "debug-on-error" and repeating the above, I can generate a
> backtrace for this as follows:
> 
> Debugger entered--Lisp error: (file-error "Can't find library"
> "/home/troy/.emacs")
>   find-library-name("/home/troy/.emacs")
>   #<subr find-function-search-for-symbol>(company-mode nil
> "/home/troy/.emacs.elpa/elpa/company-20230416.2034/company.el")
>   ad-Advice-find-function-search-for-symbol(#<subr
> find-function-search-for-symbol> company-mode nil
> "/home/troy/.emacs.elpa/elpa/company-20230416.2034/company.el")
>   apply(ad-Advice-find-function-search-for-symbol #<subr
> find-function-search-for-symbol> (company-mode nil
> "/home/troy/.emacs.elpa/elpa/company-20230416.2034/company.el"))
>   find-function-search-for-symbol(company-mode nil
> "/home/troy/.emacs.elpa/elpa/company-20230416.2034/company.el")
>   help-function-def--button-function(company-mode
> "/home/troy/.emacs.elpa/elpa/company-20230416.2034/company.el" nil)
>   help-view-source()
>   funcall-interactively(help-view-source)
>   call-interactively(help-view-source nil nil)
>   command-execute(help-view-source)
> 
> I believe the following snippet of code in
> find-function-search-for-symbol is to blame for this:
> 
>     (when (string-match "\\.emacs\\(.el\\)" library)
>       (setq library (substring library 0 (match-beginning 1))))
> 
> The regex likely should be improved to only match .emacs or .emacs.el at
> the end of the path, not in the middle of it.

Does the patch below fix this problem?

Also, I see that you have some advice installed for
find-function-search-for-symbol.  Would you please verify that
removing that advice still produces the problem in Emacs 29?

diff --git a/lisp/emacs-lisp/find-func.el b/lisp/emacs-lisp/find-func.el
index f9f919a..bf890fc 100644
--- a/lisp/emacs-lisp/find-func.el
+++ b/lisp/emacs-lisp/find-func.el
@@ -407,7 +407,7 @@ find-function-search-for-symbol
       (setq library (substring library 0 (match-beginning 1))))
     ;; Strip extension from .emacs.el to make sure symbol is searched in
     ;; .emacs too.
-    (when (string-match "\\.emacs\\(.el\\)" library)
+    (when (string-match "\\.emacs\\(.el\\)\\'" library)
       (setq library (substring library 0 (match-beginning 1))))
     (let* ((filename (find-library-name library))
 	   (regexp-symbol (cdr (assq type find-function-regexp-alist))))

bug#64143: 29.0.60; find-function-search-for-symbol truncates library path

[Troy Brown]

Eli Zaretskii <eliz@gnu.org> writes:

> What do you mean by "truncates the library path"?  Do you mean
> truncation of the file name 'emacs.el' to yield just 'emacs'? or do
> you mean something else?  (To avoid confusion, please don't use "path"
> to refer to file names, only to refer to lists of directories in the
> style of PATH or load-path.)

I was referring specifically to the LIBRARY parameter passed into
find-function-search-for-symbol.  In this scenario, I'd expect it to be
passed to find-library-name unaltered.  In other words, the value of the
LIBRARY parameter passed in is:
  "/home/troy/.emacs.elpa/elpa/company-20230416.2034/company.el"

However, the value passed on to find-library-name is:
  "/home/troy/.emacs"

This is being altered due to the fact that it contains a ".emacs.el"
substring within it (i.e., "/home/troy/<.emacs.el>pa").

> This says it cannot find ".emacs", not "emacs".  So is your init file
> called ".emacs.el" or (as you mentioned above) "emacs.el"?

The problem has nothing to do with my init file (and manifests
regardless of init file name), only whether the path supplied by
--init-directory contains an ".emacs.el" substring in it (which
"~/.emacs.elpa" clearly does).

Here is a more precise way to reproduce this using "emacs -Q".  I was
previously trying to provide a real-world example of how I was seeing
this behavior, but maybe it added confusion.

If we toggle "debug-on-error" and then evaluate each expression below in
turn, we can examine the value that find-function-search-for-symbol
passes to find-library-name (as this assumes a bogus username "someuser"
and thus an invalid path to trigger the error).  In the first case
(simulating using --init-directory=/home/someuser/.emacs.d), the value
is as expected (the value passed as the LIBRARY parameter to
find-function-search-for-symbol).  In the second case (simulating using
--init-directory=/home/someuser/.emacs.elpa), we see that the value
passed to find-library-name is no longer the same as the value passed as
the LIBRARY parameter to find-function-search-for-symbol, but was
truncated instead.

(find-function-search-for-symbol 'company-mode nil
"/home/someuser/.emacs.d/elpa/company-20230416.2034/company.el")
(find-function-search-for-symbol 'company-mode nil
"/home/someuser/.emacs.elpa/elpa/company-20230416.2034/company.el")


Debugger entered--Lisp error: (file-error "Can't find library"
"/home/someuser/.emacs.d/elpa/company-20230416.2034/company.el")
  find-library-name("/home/someuser/.emacs.d/elpa/company-20230416.2034/company.el")
  find-function-search-for-symbol(company-mode nil
"/home/someuser/.emacs.d/elpa/company-20230416.2034/company.el")
  (progn (find-function-search-for-symbol 'company-mode nil
"/home/someuser/.emacs.d/elpa/company-20230416.2034/company.el"))
  eval((progn (find-function-search-for-symbol 'company-mode nil
"/home/someuser/.emacs.d/elpa/company-20230416.2034/company.el")) t)
  elisp--eval-last-sexp(nil)
  eval-last-sexp(nil)
  funcall-interactively(eval-last-sexp nil)
  call-interactively(eval-last-sexp nil nil)
  command-execute(eval-last-sexp)

Debugger entered--Lisp error: (file-error "Can't find library"
"/home/someuser/.emacs")
  find-library-name("/home/someuser/.emacs")
  find-function-search-for-symbol(company-mode nil
"/home/someuser/.emacs.elpa/elpa/company-20230416.2034/company.el")
  (progn (find-function-search-for-symbol 'company-mode nil
"/home/someuser/.emacs.elpa/elpa/company-20230416.2034/company.el"))
  eval((progn (find-function-search-for-symbol 'company-mode nil
"/home/someuser/.emacs.elpa/elpa/company-20230416.2034/company.el"))
t)
  elisp--eval-last-sexp(nil)
  eval-last-sexp(nil)
  funcall-interactively(eval-last-sexp nil)
  call-interactively(eval-last-sexp nil nil)
  command-execute(eval-last-sexp)

> Does the patch below fix this problem?
>

Indeed, it does.

> Also, I see that you have some advice installed for
> find-function-search-for-symbol.  Would you please verify that
> removing that advice still produces the problem in Emacs 29?
>

See above where I provided a way to reproduce this using just "emacs -Q".

bug#64143: 29.0.60; find-function-search-for-symbol truncates library path

[Eli Zaretskii]

> From: Troy Brown <brownts@troybrown.dev>
> Date: Sun, 18 Jun 2023 13:39:05 -0400
> Cc: 64143@debbugs.gnu.org
> 
> > Does the patch below fix this problem?
> >
> 
> Indeed, it does.

Thanks, installed on the emacs-29 branch, and closing the bug.