bug#44494: etags.el xref-location-marker does not handle TAGS references to .el.gz files

[Pierre Rouleau <prouleau001@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 5743 bytes --]

On Sat, Nov 7, 2020 at 3:00 AM Eli Zaretskii <eliz@gnu.org> wrote:

> > From: Pierre Rouleau <prouleau001@gmail.com>
> > Date: Fri, 6 Nov 2020 22:31:02 -0500
> > Cc: 44494@debbugs.gnu.org
> >
> >  Do you see the same problem with 'M-x find-tag'?
> >
> > Short answer:  yes
> >
> > Longer answer:  you can try it on Emacs lib files.
> >
> > For example, I created a TAGS file that contains the following:
> >
> > define-globalized-minor-mode global-prettify-symbols-mode^?247,10355
> > (define-derived-mode prog-mode ^?251,10485
> > ^L
> >
> /usr/local/Cellar/emacs/26.3/share/emacs/26.3/lisp/progmodes/cc-bytecomp.el,1014
> > (defvar cc-bytecomp-unbound-variables ^?76,2968
> > (defvar cc-bytecomp-original-functions ^?77,3011
> > (defvar cc-bytecomp-original-properties ^?78,3055
> > (defvar cc-bytecomp-loaded-files ^?79,3100
> > (defvar cc-bytecomp-environment-set ^?86,3302
> > (defmacro cc-bytecomp-debug-msg ^?88,3344
> > (defun cc-bytecomp-compiling-or-loading ^?93,3432
> > (defsubst cc-bytecomp-is-compiling ^?134,4714
> > (defsubst cc-bytecomp-is-loading ^?138,4857
> > (defun cc-bytecomp-setup-environment ^?143,5065
> > (defun cc-bytecomp-restore-environment ^?191,6703
> > (defun cc-bytecomp-load ^?256,8749
> > (defmacro cc-require ^?293,10222
> > (defmacro cc-conditional-require ^?305,10617
> > (defmacro cc-conditional-require-after-load ^?318,11068
> > (defmacro cc-provide ^?333,11627
> > (defmacro cc-load ^?340,11887
> > (defmacro cc-require-when-compile ^?351,12266
> > (defmacro cc-external-require ^?362,12703
> > (defmacro cc-bytecomp-defvar ^?371,13055
> > (defmacro cc-bytecomp-defun ^?392,13857
> > (defmacro cc-bytecomp-put ^?419,14990
> > (defmacro cc-bytecomp-boundp ^?437,15739
> > (defmacro cc-bytecomp-fboundp ^?447,16140
> > ^L
> >
> /usr/local/Cellar/emacs/26.3/share/emacs/26.3/lisp/progmodes/make-mode.el,6494
> > (defgroup makefile ^?95,3661
> > (defface makefile-space^?101,3839
> > (defface makefile-targets^?107,4026
> > (defface makefile-shell^?114,4302
> >
> > Then with the file
> /usr/local/Cellar/emacs/26.3/share/emacs/26.3/lisp/progmodes/cc-cmds.el.gz
> in a buffer
> > and cc-bytecomp not loaded trying both
> >
> > M-x xref-find-definitions cc-require
> >
> > and
> >
> > M-x find-tag cc-require
> >
> > I get:
> >
> > Rerun etags: ‘^(defmacro cc-require ’ not found in
> >
> /usr/local/Cellar/emacs/26.3/share/emacs/26.3/lisp/progmodes/cc-bytecomp.el
>
> I cannot reproduce this: find-tag works in this situation, at least in
> the current emacs-27 branch and in stock Emacs 27.1.  Which doesn't
> surprise me, since etags.el already has code that handles compressed
> files.
>
> Moreover, M-. (which uses xref) works as well.  So I'm no longer sure
> I understand what is the problem you are seeing.  If you see this in
> Emacs 26, please retry in Emacs 27, and let's take this from there.
>
> FTR, the steps I used for reproducing were slightly different:
>
>   . "make TAGS" in the top-level directory of the Emacs source tree
>   . gzip lisp/abbrevs.el
>   . emacs -Q
>   . C-x C-f lisp/simple.el
>   . C-u M-. kill-all-abbrevs RET
>
> And for find-tag, replace the last 2 commands with
>
>   . M-x visit-tags-table RET RET
>   . M-x find-tag RET kill-all-abbrevs RET
>
> Both of these work and show abbrevs.el.gz at the correct line.
>

Hi,  I did try your test, both on emacs 26.3 and 27.1,
and for the exact scenario you described, it works if you search for
kill-all-abbrevs.

I should have better describe the failure scenario.
- Problem  occurs in emacs 26.3 and 27.1.
- I search for cc-require ,  a defmacro, not a defun, and not something
  that is already loaded.
- on my system all files emacs/lisp directory are .el.gz file, however,
  I do not think that the fact they are all compressed is significant.

With both emacs 26.3 and 27.1, iI do:
. emacs -Q
. C-x C-f lisp/simple.el
. C-u M-x cc-require
==> message: No definitions found for cc-require

At this point:
- the xref backend is the default for .el files; etags.
- xref-backend-functions is set to (elisp--xref-backend t), a local setting.
- the abbrev feature is loaded, as M-: (featurep 'abbrev) returns t.
- So looking for kill-all-abbrevs using the elisp--xref-backend works, my
  understanding is that it works because the file that holds the definition
  is loaded.

I wanted to be able to access definitions from code that might not have
already been loaded so I thought I would use the etags--xref-backend
instead.

With emacs 26.3 and 27.1 with all lisp/simple files compressed,
 I do (and get approximately same results, shown below):
. emacs -Q
. C-x C-f lisp/simple.el.gz
. M-x xref-etags-mode
. C-u M-x cc-require
emacs==> prompts Visit tags table (default TAGS): ....
me ====> I select the TAGS file where all definitions are stored and hit RET
- emacs 26.3 ==> Rerun etags: ‘^(defmacro cc-require ’ not found in
/usr/local/Cellar/emacs/26.3/share/emacs/26.3/lisp/progmod\
es/cc-bytecomp.el
- emacs 27.1 ==> Rerun etags: ‘^(defmacro cc-require ’ not found in
/usr/local/Cellar/emacs/27.1/share/emacs/27.1/lisp/progmodes/cc-bytecomp.el

If I look inside the TAGS file and search for cc-require,
I see it listed under the cc-bytecomp.el file.

If at this point, I load the 2 functions I wrote and try again,
it succeeds in finding cc-require.

To recap you need to try searching for something that is not already
loaded and use the etags xref backend with a file that contains the
definition
of what one is searching and that is located inside a compressed file.

Let me know if you want me to try other scenarios.

Thanks!

/Pierre Rouleau

[-- Attachment #2: Type: text/html, Size: 7173 bytes --]