* bug#44494: etags.el xref-location-marker does not handle TAGS references to .el.gz files @ 2020-11-06 23:22 Pierre Rouleau 2020-11-07 2:22 ` Dmitry Gutov 2020-11-07 7:18 ` Eli Zaretskii 0 siblings, 2 replies; 12+ messages in thread From: Pierre Rouleau @ 2020-11-06 23:22 UTC (permalink / raw) To: 44494 [-- Attachment #1: Type: text/plain, Size: 5588 bytes --] This problem was detected in emacs 26.3, but is also present in emacs 27.1, according to the code posted inside https://github.com/emacs-mirror/emacs/blob/master/test/manual/etags/el-src/emacs/lisp/progmodes/etags.el#L2139 Problem description follows: ----------------------------BUG DESCRIPTION [ ------------------------------ The current version of etags distributed with emacs 27.0 is capable of parsing compressed files like something.el.gz but unfortunately the reference inside the generated TAGS file uses the name "something.el" instead of the complete file name "something.el.gz". The same is true for the other compressed files (.bz2, .xz and .lzma). When trying to use the etags xref back-end to move point to a definition inside such a file, the xref-find-definitions command will fail because the TAGS file identifies the .el file instead of the .el.gz file. xref-find-definitions fails because it only checks if the uncompressed file exists, it does not try to see if the compressed file exists. The issue is inside etags.el xref-location-marker method. The current code is: (cl-defmethod xref-location-marker ((l xref-etags-location)) (with-slots (tag-info file) l (let ((buffer (find-file-noselect file))) (with-current-buffer buffer (save-excursion (etags-goto-tag-location tag-info) (point-marker)))))) One could consider that the issue is inside the etags utility. An alternative would be to provide Emacs with the ability to check for the presence of the .el file and if it is not present look for the equivalent compressed files. Here's a proposal for a solution: (defun etags-file-or-compressed-file-for (fname) "Return the valid file name for FNAME. Check if FNAME is an existing file name, if not try FNAME appended with the following compression extensions: - \".gz\", the extension of compressed files created by gzip - \".bz2\", the extension for compressed files created by bzip2 - \".xz\", the extension for compressed files created by xz - \".lzma\", the extension for compressed files created by xz. Return the file that exists or nil if nothing found." (let ((fpath nil)) (cl-dolist (ext '("" ".gz" ".bz2" ".xz" ".lzma")) (setq fpath (concat fname ext)) (when (file-exists-p fpath) (cl-return fpath))))) (cl-defmethod xref-location-marker ((l xref-etags-location)) (with-slots (tag-info file) l (let (buffer (fname (pel-file-or-compressed-file-for file))) (if fname (setq buffer (find-file-noselect fname)) (user-error "file %s (or .gz, .bz2, .xz, .lzma) does not exist" file)) (with-current-buffer buffer (save-excursion (etags-goto-tag-location tag-info) (point-marker)))))) ------------------------------ END OF DESCRIPTION ] -------------------------------- In GNU Emacs 26.3 (build 1, x86_64-apple-darwin18.6.0) of 2019-08-30 built on Mojave.local Recent messages: For information about GNU Emacs and the GNU system, type C-h C-a. Configured using: 'configure --disable-dependency-tracking --disable-silent-rules --enable-locallisppath=/usr/local/share/emacs/site-lisp --infodir=/usr/local/Cellar/emacs/26.3/share/info/emacs --prefix=/usr/local/Cellar/emacs/26.3 --with-gnutls --without-x --with-xml2 --without-dbus --with-modules --without-ns --without-imagemagick' Configured features: NOTIFY ACL GNUTLS LIBXML2 ZLIB MODULES THREADS Important settings: value of $LANG: en_CA.UTF-8 locale-coding-system: utf-8-unix Major mode: Lisp Interaction Minor modes in effect: tooltip-mode: t global-eldoc-mode: t eldoc-mode: t electric-indent-mode: t menu-bar-mode: t file-name-shadow-mode: t global-font-lock-mode: t font-lock-mode: t auto-composition-mode: t auto-encryption-mode: t auto-compression-mode: t line-number-mode: t transient-mark-mode: t Load-path shadows: None found. Features: (shadow sort mail-extr emacsbug message rmc puny seq byte-opt gv bytecomp byte-compile cconv cl-loaddefs cl-lib dired dired-loaddefs format-spec rfc822 mml easymenu mml-sec password-cache epa derived epg epg-config gnus-util rmail tool-bar rmail-loaddefs mm-decode mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail regexp-opt rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils term/xterm xterm time-date elec-pair mule-util tooltip eldoc electric uniquify ediff-hook vc-hooks lisp-float-type tabulated-list replace newcomment text-mode elisp-mode lisp-mode prog-mode register page menu-bar rfn-eshadow isearch timer select mouse jit-lock font-lock syntax facemenu font-core term/tty-colors frame cl-generic cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech european ethiopic indian cyrillic chinese composite charscript charprop case-table epa-hook jka-cmpr-hook help simple abbrev obarray minibuffer cl-preloaded nadvice loaddefs button faces cus-face macroexp files text-properties overlay sha1 md5 base64 format env code-pages mule custom widget hashtable-print-readable backquote threads kqueue multi-tty make-network-process emacs) Memory information: ((conses 16 96177 5831) (symbols 48 19813 1) (miscs 40 33 96) (strings 32 28253 1011) (string-bytes 1 748198) (vectors 16 11977) (vector-slots 8 455846 6474) (floats 8 48 566) (intervals 56 197 0) (buffers 992 11)) ;; --------------------------- Thank you, /Pierre Rouleau [-- Attachment #2: Type: text/html, Size: 6816 bytes --] ^ permalink raw reply [flat|nested] 12+ messages in thread
* bug#44494: etags.el xref-location-marker does not handle TAGS references to .el.gz files 2020-11-06 23:22 bug#44494: etags.el xref-location-marker does not handle TAGS references to .el.gz files Pierre Rouleau @ 2020-11-07 2:22 ` Dmitry Gutov 2020-11-07 3:31 ` Pierre Rouleau 2020-11-07 7:18 ` Eli Zaretskii 1 sibling, 1 reply; 12+ messages in thread From: Dmitry Gutov @ 2020-11-07 2:22 UTC (permalink / raw) To: Pierre Rouleau, 44494 On 07.11.2020 01:22, Pierre Rouleau wrote: > This problem was detected in emacs 26.3, but is also present in emacs > 27.1, according to the code posted inside > https://github.com/emacs-mirror/emacs/blob/master/test/manual/etags/el-src/emacs/lisp/progmodes/etags.el#L2139 Hi! Do you see the same problem with 'M-x find-tag'? ^ permalink raw reply [flat|nested] 12+ messages in thread
* bug#44494: etags.el xref-location-marker does not handle TAGS references to .el.gz files 2020-11-07 2:22 ` Dmitry Gutov @ 2020-11-07 3:31 ` Pierre Rouleau 2020-11-07 8:00 ` Eli Zaretskii 0 siblings, 1 reply; 12+ messages in thread From: Pierre Rouleau @ 2020-11-07 3:31 UTC (permalink / raw) To: Dmitry Gutov; +Cc: 44494 [-- Attachment #1: Type: text/plain, Size: 2966 bytes --] On Fri, Nov 6, 2020 at 9:22 PM Dmitry Gutov <dgutov@yandex.ru> wrote: > On 07.11.2020 01:22, Pierre Rouleau wrote: > > This problem was detected in emacs 26.3, but is also present in emacs > > 27.1, according to the code posted inside > > > https://github.com/emacs-mirror/emacs/blob/master/test/manual/etags/el-src/emacs/lisp/progmodes/etags.el#L2139 > > Hi! > > Do you see the same problem with 'M-x find-tag'? > Short answer: yes Longer answer: you can try it on Emacs lib files. For example, I created a TAGS file that contains the following: define-globalized-minor-mode global-prettify-symbols-mode^?247,10355 (define-derived-mode prog-mode ^?251,10485 ^L /usr/local/Cellar/emacs/26.3/share/emacs/26.3/lisp/progmodes/cc-bytecomp.el,1014 (defvar cc-bytecomp-unbound-variables ^?76,2968 (defvar cc-bytecomp-original-functions ^?77,3011 (defvar cc-bytecomp-original-properties ^?78,3055 (defvar cc-bytecomp-loaded-files ^?79,3100 (defvar cc-bytecomp-environment-set ^?86,3302 (defmacro cc-bytecomp-debug-msg ^?88,3344 (defun cc-bytecomp-compiling-or-loading ^?93,3432 (defsubst cc-bytecomp-is-compiling ^?134,4714 (defsubst cc-bytecomp-is-loading ^?138,4857 (defun cc-bytecomp-setup-environment ^?143,5065 (defun cc-bytecomp-restore-environment ^?191,6703 (defun cc-bytecomp-load ^?256,8749 (defmacro cc-require ^?293,10222 (defmacro cc-conditional-require ^?305,10617 (defmacro cc-conditional-require-after-load ^?318,11068 (defmacro cc-provide ^?333,11627 (defmacro cc-load ^?340,11887 (defmacro cc-require-when-compile ^?351,12266 (defmacro cc-external-require ^?362,12703 (defmacro cc-bytecomp-defvar ^?371,13055 (defmacro cc-bytecomp-defun ^?392,13857 (defmacro cc-bytecomp-put ^?419,14990 (defmacro cc-bytecomp-boundp ^?437,15739 (defmacro cc-bytecomp-fboundp ^?447,16140 ^L /usr/local/Cellar/emacs/26.3/share/emacs/26.3/lisp/progmodes/make-mode.el,6494 (defgroup makefile ^?95,3661 (defface makefile-space^?101,3839 (defface makefile-targets^?107,4026 (defface makefile-shell^?114,4302 Then with the file /usr/local/Cellar/emacs/26.3/share/emacs/26.3/lisp/progmodes/cc-cmds.el.gz in a buffer and cc-bytecomp not loaded trying both M-x xref-find-definitions cc-require and M-x find-tag cc-require I get: Rerun etags: ‘^(defmacro cc-require ’ not found in /usr/local/Cellar/emacs/26.3/share/emacs/26.3/lisp/progmodes/cc-bytecomp.el If I update the etags.el with my suggested code, M-x xref-find-definition works but find-tag still does not. Since find-tag is obsolete since emacs 25.1 I did not look into it. I did not look into the etags utility itself yet. This looks like a consistency problem. I would think that the etags utility should generate a reference for el.gz files listing the real file name. On the other hand having the fix in both places both in etags utility and in emacs etags.el would reduce probability of errors. -- /Pierre [-- Attachment #2: Type: text/html, Size: 3989 bytes --] ^ permalink raw reply [flat|nested] 12+ messages in thread
* bug#44494: etags.el xref-location-marker does not handle TAGS references to .el.gz files 2020-11-07 3:31 ` Pierre Rouleau @ 2020-11-07 8:00 ` Eli Zaretskii 2020-11-07 14:15 ` Pierre Rouleau 0 siblings, 1 reply; 12+ messages in thread From: Eli Zaretskii @ 2020-11-07 8:00 UTC (permalink / raw) To: Pierre Rouleau; +Cc: 44494, dgutov > From: Pierre Rouleau <prouleau001@gmail.com> > Date: Fri, 6 Nov 2020 22:31:02 -0500 > Cc: 44494@debbugs.gnu.org > > Do you see the same problem with 'M-x find-tag'? > > Short answer: yes > > Longer answer: you can try it on Emacs lib files. > > For example, I created a TAGS file that contains the following: > > define-globalized-minor-mode global-prettify-symbols-mode^?247,10355 > (define-derived-mode prog-mode ^?251,10485 > ^L > /usr/local/Cellar/emacs/26.3/share/emacs/26.3/lisp/progmodes/cc-bytecomp.el,1014 > (defvar cc-bytecomp-unbound-variables ^?76,2968 > (defvar cc-bytecomp-original-functions ^?77,3011 > (defvar cc-bytecomp-original-properties ^?78,3055 > (defvar cc-bytecomp-loaded-files ^?79,3100 > (defvar cc-bytecomp-environment-set ^?86,3302 > (defmacro cc-bytecomp-debug-msg ^?88,3344 > (defun cc-bytecomp-compiling-or-loading ^?93,3432 > (defsubst cc-bytecomp-is-compiling ^?134,4714 > (defsubst cc-bytecomp-is-loading ^?138,4857 > (defun cc-bytecomp-setup-environment ^?143,5065 > (defun cc-bytecomp-restore-environment ^?191,6703 > (defun cc-bytecomp-load ^?256,8749 > (defmacro cc-require ^?293,10222 > (defmacro cc-conditional-require ^?305,10617 > (defmacro cc-conditional-require-after-load ^?318,11068 > (defmacro cc-provide ^?333,11627 > (defmacro cc-load ^?340,11887 > (defmacro cc-require-when-compile ^?351,12266 > (defmacro cc-external-require ^?362,12703 > (defmacro cc-bytecomp-defvar ^?371,13055 > (defmacro cc-bytecomp-defun ^?392,13857 > (defmacro cc-bytecomp-put ^?419,14990 > (defmacro cc-bytecomp-boundp ^?437,15739 > (defmacro cc-bytecomp-fboundp ^?447,16140 > ^L > /usr/local/Cellar/emacs/26.3/share/emacs/26.3/lisp/progmodes/make-mode.el,6494 > (defgroup makefile ^?95,3661 > (defface makefile-space^?101,3839 > (defface makefile-targets^?107,4026 > (defface makefile-shell^?114,4302 > > Then with the file /usr/local/Cellar/emacs/26.3/share/emacs/26.3/lisp/progmodes/cc-cmds.el.gz in a buffer > and cc-bytecomp not loaded trying both > > M-x xref-find-definitions cc-require > > and > > M-x find-tag cc-require > > I get: > > Rerun etags: ‘^(defmacro cc-require ’ not found in > /usr/local/Cellar/emacs/26.3/share/emacs/26.3/lisp/progmodes/cc-bytecomp.el I cannot reproduce this: find-tag works in this situation, at least in the current emacs-27 branch and in stock Emacs 27.1. Which doesn't surprise me, since etags.el already has code that handles compressed files. Moreover, M-. (which uses xref) works as well. So I'm no longer sure I understand what is the problem you are seeing. If you see this in Emacs 26, please retry in Emacs 27, and let's take this from there. FTR, the steps I used for reproducing were slightly different: . "make TAGS" in the top-level directory of the Emacs source tree . gzip lisp/abbrevs.el . emacs -Q . C-x C-f lisp/simple.el . C-u M-. kill-all-abbrevs RET And for find-tag, replace the last 2 commands with . M-x visit-tags-table RET RET . M-x find-tag RET kill-all-abbrevs RET Both of these work and show abbrevs.el.gz at the correct line. ^ permalink raw reply [flat|nested] 12+ messages in thread
* bug#44494: etags.el xref-location-marker does not handle TAGS references to .el.gz files 2020-11-07 8:00 ` Eli Zaretskii @ 2020-11-07 14:15 ` Pierre Rouleau 2020-11-07 14:22 ` Eli Zaretskii 2020-11-07 14:37 ` Eli Zaretskii 0 siblings, 2 replies; 12+ messages in thread From: Pierre Rouleau @ 2020-11-07 14:15 UTC (permalink / raw) To: Eli Zaretskii; +Cc: 44494, dgutov [-- Attachment #1: Type: text/plain, Size: 5743 bytes --] On Sat, Nov 7, 2020 at 3:00 AM Eli Zaretskii <eliz@gnu.org> wrote: > > From: Pierre Rouleau <prouleau001@gmail.com> > > Date: Fri, 6 Nov 2020 22:31:02 -0500 > > Cc: 44494@debbugs.gnu.org > > > > Do you see the same problem with 'M-x find-tag'? > > > > Short answer: yes > > > > Longer answer: you can try it on Emacs lib files. > > > > For example, I created a TAGS file that contains the following: > > > > define-globalized-minor-mode global-prettify-symbols-mode^?247,10355 > > (define-derived-mode prog-mode ^?251,10485 > > ^L > > > /usr/local/Cellar/emacs/26.3/share/emacs/26.3/lisp/progmodes/cc-bytecomp.el,1014 > > (defvar cc-bytecomp-unbound-variables ^?76,2968 > > (defvar cc-bytecomp-original-functions ^?77,3011 > > (defvar cc-bytecomp-original-properties ^?78,3055 > > (defvar cc-bytecomp-loaded-files ^?79,3100 > > (defvar cc-bytecomp-environment-set ^?86,3302 > > (defmacro cc-bytecomp-debug-msg ^?88,3344 > > (defun cc-bytecomp-compiling-or-loading ^?93,3432 > > (defsubst cc-bytecomp-is-compiling ^?134,4714 > > (defsubst cc-bytecomp-is-loading ^?138,4857 > > (defun cc-bytecomp-setup-environment ^?143,5065 > > (defun cc-bytecomp-restore-environment ^?191,6703 > > (defun cc-bytecomp-load ^?256,8749 > > (defmacro cc-require ^?293,10222 > > (defmacro cc-conditional-require ^?305,10617 > > (defmacro cc-conditional-require-after-load ^?318,11068 > > (defmacro cc-provide ^?333,11627 > > (defmacro cc-load ^?340,11887 > > (defmacro cc-require-when-compile ^?351,12266 > > (defmacro cc-external-require ^?362,12703 > > (defmacro cc-bytecomp-defvar ^?371,13055 > > (defmacro cc-bytecomp-defun ^?392,13857 > > (defmacro cc-bytecomp-put ^?419,14990 > > (defmacro cc-bytecomp-boundp ^?437,15739 > > (defmacro cc-bytecomp-fboundp ^?447,16140 > > ^L > > > /usr/local/Cellar/emacs/26.3/share/emacs/26.3/lisp/progmodes/make-mode.el,6494 > > (defgroup makefile ^?95,3661 > > (defface makefile-space^?101,3839 > > (defface makefile-targets^?107,4026 > > (defface makefile-shell^?114,4302 > > > > Then with the file > /usr/local/Cellar/emacs/26.3/share/emacs/26.3/lisp/progmodes/cc-cmds.el.gz > in a buffer > > and cc-bytecomp not loaded trying both > > > > M-x xref-find-definitions cc-require > > > > and > > > > M-x find-tag cc-require > > > > I get: > > > > Rerun etags: ‘^(defmacro cc-require ’ not found in > > > /usr/local/Cellar/emacs/26.3/share/emacs/26.3/lisp/progmodes/cc-bytecomp.el > > I cannot reproduce this: find-tag works in this situation, at least in > the current emacs-27 branch and in stock Emacs 27.1. Which doesn't > surprise me, since etags.el already has code that handles compressed > files. > > Moreover, M-. (which uses xref) works as well. So I'm no longer sure > I understand what is the problem you are seeing. If you see this in > Emacs 26, please retry in Emacs 27, and let's take this from there. > > FTR, the steps I used for reproducing were slightly different: > > . "make TAGS" in the top-level directory of the Emacs source tree > . gzip lisp/abbrevs.el > . emacs -Q > . C-x C-f lisp/simple.el > . C-u M-. kill-all-abbrevs RET > > And for find-tag, replace the last 2 commands with > > . M-x visit-tags-table RET RET > . M-x find-tag RET kill-all-abbrevs RET > > Both of these work and show abbrevs.el.gz at the correct line. > Hi, I did try your test, both on emacs 26.3 and 27.1, and for the exact scenario you described, it works if you search for kill-all-abbrevs. I should have better describe the failure scenario. - Problem occurs in emacs 26.3 and 27.1. - I search for cc-require , a defmacro, not a defun, and not something that is already loaded. - on my system all files emacs/lisp directory are .el.gz file, however, I do not think that the fact they are all compressed is significant. With both emacs 26.3 and 27.1, iI do: . emacs -Q . C-x C-f lisp/simple.el . C-u M-x cc-require ==> message: No definitions found for cc-require At this point: - the xref backend is the default for .el files; etags. - xref-backend-functions is set to (elisp--xref-backend t), a local setting. - the abbrev feature is loaded, as M-: (featurep 'abbrev) returns t. - So looking for kill-all-abbrevs using the elisp--xref-backend works, my understanding is that it works because the file that holds the definition is loaded. I wanted to be able to access definitions from code that might not have already been loaded so I thought I would use the etags--xref-backend instead. With emacs 26.3 and 27.1 with all lisp/simple files compressed, I do (and get approximately same results, shown below): . emacs -Q . C-x C-f lisp/simple.el.gz . M-x xref-etags-mode . C-u M-x cc-require emacs==> prompts Visit tags table (default TAGS): .... me ====> I select the TAGS file where all definitions are stored and hit RET - emacs 26.3 ==> Rerun etags: ‘^(defmacro cc-require ’ not found in /usr/local/Cellar/emacs/26.3/share/emacs/26.3/lisp/progmod\ es/cc-bytecomp.el - emacs 27.1 ==> Rerun etags: ‘^(defmacro cc-require ’ not found in /usr/local/Cellar/emacs/27.1/share/emacs/27.1/lisp/progmodes/cc-bytecomp.el If I look inside the TAGS file and search for cc-require, I see it listed under the cc-bytecomp.el file. If at this point, I load the 2 functions I wrote and try again, it succeeds in finding cc-require. To recap you need to try searching for something that is not already loaded and use the etags xref backend with a file that contains the definition of what one is searching and that is located inside a compressed file. Let me know if you want me to try other scenarios. Thanks! /Pierre Rouleau [-- Attachment #2: Type: text/html, Size: 7173 bytes --] ^ permalink raw reply [flat|nested] 12+ messages in thread
* bug#44494: etags.el xref-location-marker does not handle TAGS references to .el.gz files 2020-11-07 14:15 ` Pierre Rouleau @ 2020-11-07 14:22 ` Eli Zaretskii 2020-11-07 14:37 ` Eli Zaretskii 1 sibling, 0 replies; 12+ messages in thread From: Eli Zaretskii @ 2020-11-07 14:22 UTC (permalink / raw) To: Pierre Rouleau; +Cc: 44494, dgutov > From: Pierre Rouleau <prouleau001@gmail.com> > Date: Sat, 7 Nov 2020 09:15:12 -0500 > Cc: dgutov@yandex.ru, 44494@debbugs.gnu.org > > With both emacs 26.3 and 27.1, iI do: > . emacs -Q > . C-x C-f lisp/simple.el > . C-u M-x cc-require > ==> message: No definitions found for cc-require > > At this point: > - the xref backend is the default for .el files; etags. > - xref-backend-functions is set to (elisp--xref-backend t), a local setting. > - the abbrev feature is loaded, as M-: (featurep 'abbrev) returns t. > - So looking for kill-all-abbrevs using the elisp--xref-backend works, my > understanding is that it works because the file that holds the definition > is loaded. How does this explain the fact that find-tag also worked for me? Does it work for you? ^ permalink raw reply [flat|nested] 12+ messages in thread
* bug#44494: etags.el xref-location-marker does not handle TAGS references to .el.gz files 2020-11-07 14:15 ` Pierre Rouleau 2020-11-07 14:22 ` Eli Zaretskii @ 2020-11-07 14:37 ` Eli Zaretskii 2020-11-07 14:48 ` Pierre Rouleau 1 sibling, 1 reply; 12+ messages in thread From: Eli Zaretskii @ 2020-11-07 14:37 UTC (permalink / raw) To: Pierre Rouleau; +Cc: 44494, dgutov > From: Pierre Rouleau <prouleau001@gmail.com> > Date: Sat, 7 Nov 2020 09:15:12 -0500 > Cc: dgutov@yandex.ru, 44494@debbugs.gnu.org > > To recap you need to try searching for something that is not already > loaded and use the etags xref backend with a file that contains the definition > of what one is searching and that is located inside a compressed file. OK, I've now tried this with paren.el, which is not loaded in "emacs -Q". I can confirm that M-. fails to find functions in paren.el (I tried show-paren-function, FTR), even if I use xref-etags-mode, but "M-x find-tag" succeeds. So I think we should try to understand why find-tag does work in this case, and see how to make xref-find-definitions do the same. Could you perhaps do that? > . emacs -Q > . C-x C-f lisp/simple.el.gz > . M-x xref-etags-mode > . C-u M-x cc-require > emacs==> prompts Visit tags table (default TAGS): .... > me ====> I select the TAGS file where all definitions are stored and hit RET > - emacs 26.3 ==> Rerun etags: ‘^(defmacro cc-require ’ not found in > /usr/local/Cellar/emacs/26.3/share/emacs/26.3/lisp/progmod\ > es/cc-bytecomp.el > - emacs 27.1 ==> Rerun etags: ‘^(defmacro cc-require ’ not found in > /usr/local/Cellar/emacs/27.1/share/emacs/27.1/lisp/progmodes/cc-bytecomp.el Note that at this point, you have an empty cc-bytecomp.el buffer. Which I think gives a clue as to where the problem lies. ^ permalink raw reply [flat|nested] 12+ messages in thread
* bug#44494: etags.el xref-location-marker does not handle TAGS references to .el.gz files 2020-11-07 14:37 ` Eli Zaretskii @ 2020-11-07 14:48 ` Pierre Rouleau 2020-11-07 15:39 ` Pierre Rouleau 0 siblings, 1 reply; 12+ messages in thread From: Pierre Rouleau @ 2020-11-07 14:48 UTC (permalink / raw) To: Eli Zaretskii; +Cc: 44494, dgutov [-- Attachment #1: Type: text/plain, Size: 1916 bytes --] On Sat, Nov 7, 2020 at 9:37 AM Eli Zaretskii <eliz@gnu.org> wrote: > > From: Pierre Rouleau <prouleau001@gmail.com> > > Date: Sat, 7 Nov 2020 09:15:12 -0500 > > Cc: dgutov@yandex.ru, 44494@debbugs.gnu.org > > > > To recap you need to try searching for something that is not already > > loaded and use the etags xref backend with a file that contains the > definition > > of what one is searching and that is located inside a compressed file. > > OK, I've now tried this with paren.el, which is not loaded in "emacs > -Q". I can confirm that M-. fails to find functions in paren.el (I > tried show-paren-function, FTR), even if I use xref-etags-mode, but > "M-x find-tag" succeeds. > > So I think we should try to understand why find-tag does work in this > case, and see how to make xref-find-definitions do the same. Could > you perhaps do that? > > > . emacs -Q > > . C-x C-f lisp/simple.el.gz > > . M-x xref-etags-mode > > . C-u M-x cc-require > > emacs==> prompts Visit tags table (default TAGS): .... > > me ====> I select the TAGS file where all definitions are stored and hit > RET > > - emacs 26.3 ==> Rerun etags: ‘^(defmacro cc-require ’ not found in > > /usr/local/Cellar/emacs/26.3/share/emacs/26.3/lisp/progmod\ > > es/cc-bytecomp.el > > - emacs 27.1 ==> Rerun etags: ‘^(defmacro cc-require ’ not found in > > > /usr/local/Cellar/emacs/27.1/share/emacs/27.1/lisp/progmodes/cc-bytecomp.el > > Note that at this point, you have an empty cc-bytecomp.el buffer. > Which I think gives a clue as to where the problem lies. > You are correct, I tried it with find-tag in emacs 26.3 and 27.1 and find-tag cc-require does find it, even with the xref-backend-functions set to its default of (elisp--xref-backend t). It fails with xref-find-definitions but works with find-tag. I agree there's a need to see what differs there. Thanks -- /Pierre [-- Attachment #2: Type: text/html, Size: 2720 bytes --] ^ permalink raw reply [flat|nested] 12+ messages in thread
* bug#44494: etags.el xref-location-marker does not handle TAGS references to .el.gz files 2020-11-07 14:48 ` Pierre Rouleau @ 2020-11-07 15:39 ` Pierre Rouleau 2020-11-07 15:52 ` Eli Zaretskii 0 siblings, 1 reply; 12+ messages in thread From: Pierre Rouleau @ 2020-11-07 15:39 UTC (permalink / raw) To: Eli Zaretskii; +Cc: 44494, dgutov [-- Attachment #1: Type: text/plain, Size: 2804 bytes --] On Sat, Nov 7, 2020 at 9:48 AM Pierre Rouleau <prouleau001@gmail.com> wrote: > On Sat, Nov 7, 2020 at 9:37 AM Eli Zaretskii <eliz@gnu.org> wrote: > >> > From: Pierre Rouleau <prouleau001@gmail.com> >> > Date: Sat, 7 Nov 2020 09:15:12 -0500 >> > Cc: dgutov@yandex.ru, 44494@debbugs.gnu.org >> > >> > To recap you need to try searching for something that is not already >> > loaded and use the etags xref backend with a file that contains the >> definition >> > of what one is searching and that is located inside a compressed file. >> >> OK, I've now tried this with paren.el, which is not loaded in "emacs >> -Q". I can confirm that M-. fails to find functions in paren.el (I >> tried show-paren-function, FTR), even if I use xref-etags-mode, but >> "M-x find-tag" succeeds. >> >> So I think we should try to understand why find-tag does work in this >> case, and see how to make xref-find-definitions do the same. Could >> you perhaps do that? >> >> > . emacs -Q >> > . C-x C-f lisp/simple.el.gz >> > . M-x xref-etags-mode >> > . C-u M-x cc-require >> > emacs==> prompts Visit tags table (default TAGS): .... >> > me ====> I select the TAGS file where all definitions are stored and >> hit RET >> > - emacs 26.3 ==> Rerun etags: ‘^(defmacro cc-require ’ not found in >> > /usr/local/Cellar/emacs/26.3/share/emacs/26.3/lisp/progmod\ >> > es/cc-bytecomp.el >> > - emacs 27.1 ==> Rerun etags: ‘^(defmacro cc-require ’ not found in >> > >> /usr/local/Cellar/emacs/27.1/share/emacs/27.1/lisp/progmodes/cc-bytecomp.el >> >> Note that at this point, you have an empty cc-bytecomp.el buffer. >> Which I think gives a clue as to where the problem lies. >> > > You are correct, I tried it with find-tag in emacs 26.3 and 27.1 and > find-tag cc-require > does find it, even with the xref-backend-functions set to its default of > (elisp--xref-backend t). > > It fails with xref-find-definitions but works with find-tag. > > I agree there's a need to see what differs there. > > Thanks > > -- > /Pierre > One difference is that when using find-tag is using code from etags.el exclusively: - find-tag-noselect . - find-tag-in-order , which tries different predicates and the one that succeeds is tag-implicit-name-match-p . - it identifies the cc-bytecomp.el.gz .- calls etags-goto-tag-location When using xref-find-definition the xref backend is used. It's not the same code. The xref backend code for elisp does not find it. The backend code for etags does not find it either. It tries to open cc-bytecomp.el as its the file name it gets from the TAGS file. It detects the file not being present and reports it as missing, assuming the file have been removed. To me the 2 sets of code look very different. -- /Pierre [-- Attachment #2: Type: text/html, Size: 4019 bytes --] ^ permalink raw reply [flat|nested] 12+ messages in thread
* bug#44494: etags.el xref-location-marker does not handle TAGS references to .el.gz files 2020-11-07 15:39 ` Pierre Rouleau @ 2020-11-07 15:52 ` Eli Zaretskii 2020-11-07 16:09 ` Pierre Rouleau 0 siblings, 1 reply; 12+ messages in thread From: Eli Zaretskii @ 2020-11-07 15:52 UTC (permalink / raw) To: Pierre Rouleau; +Cc: 44494, dgutov > From: Pierre Rouleau <prouleau001@gmail.com> > Date: Sat, 7 Nov 2020 10:39:41 -0500 > Cc: dgutov@yandex.ru, 44494@debbugs.gnu.org > > One difference is that when using find-tag is using code from etags.el exclusively: > - find-tag-noselect > . - find-tag-in-order , which tries different predicates and the one that succeeds is > tag-implicit-name-match-p > . - it identifies the cc-bytecomp.el.gz > .- calls etags-goto-tag-location The Xref etags backend also uses tag-implicit-name-match-p: (defvar etags-xref-find-definitions-tag-order '(tag-exact-match-p tag-implicit-name-match-p) "Tag order used in `xref-backend-definitions' to look for definitions. If you want `xref-find-definitions' to find the tagged files by their file name, add `tag-partial-file-name-match-p' to the list value.") > When using xref-find-definition the xref backend is used. It's not the same code. > The xref backend code for elisp does not find it. The backend code for etags does not find it either. > It tries to open cc-bytecomp.el as its the file name it gets from the TAGS file. > It detects the file not being present and reports it as missing, assuming the file have been removed. > > To me the 2 sets of code look very different. They share some of the code, at least when xref-etags-mode is used. So it sounds like some information found by tag-implicit-name-match-p doesn't get handed back to Xref? For the Xref's own ELisp backend, we will probably need to code something in xref.el. ^ permalink raw reply [flat|nested] 12+ messages in thread
* bug#44494: etags.el xref-location-marker does not handle TAGS references to .el.gz files 2020-11-07 15:52 ` Eli Zaretskii @ 2020-11-07 16:09 ` Pierre Rouleau 0 siblings, 0 replies; 12+ messages in thread From: Pierre Rouleau @ 2020-11-07 16:09 UTC (permalink / raw) To: Eli Zaretskii; +Cc: 44494, dgutov [-- Attachment #1: Type: text/plain, Size: 2032 bytes --] On Sat, Nov 7, 2020 at 10:52 AM Eli Zaretskii <eliz@gnu.org> wrote: > > From: Pierre Rouleau <prouleau001@gmail.com> > > Date: Sat, 7 Nov 2020 10:39:41 -0500 > > Cc: dgutov@yandex.ru, 44494@debbugs.gnu.org > > > > One difference is that when using find-tag is using code from etags.el > exclusively: > > - find-tag-noselect > > . - find-tag-in-order , which tries different predicates and the one > that succeeds is > > tag-implicit-name-match-p > > . - it identifies the cc-bytecomp.el.gz > > .- calls etags-goto-tag-location > > The Xref etags backend also uses tag-implicit-name-match-p: > > (defvar etags-xref-find-definitions-tag-order '(tag-exact-match-p > > tag-implicit-name-match-p) > "Tag order used in `xref-backend-definitions' to look for definitions. > > If you want `xref-find-definitions' to find the tagged files by their > file name, add `tag-partial-file-name-match-p' to the list value.") > > > When using xref-find-definition the xref backend is used. It's not the > same code. > > The xref backend code for elisp does not find it. The backend code for > etags does not find it either. > > It tries to open cc-bytecomp.el as its the file name it gets from the > TAGS file. > > It detects the file not being present and reports it as missing, > assuming the file have been removed. > > > > To me the 2 sets of code look very different. > > They share some of the code, at least when xref-etags-mode is used. > So it sounds like some information found by tag-implicit-name-match-p > doesn't get handed back to Xref? > > For the Xref's own ELisp backend, we will probably need to code > something in xref.el. > Sorry, I meant that find-tag-in-order is able to find the reference in the TAGS file when it tries the tag-implicit-name-match-p predicate. find-tag-in-order returns a marker that identifies the .el.gz file. I agree that for the ELisp backend something probably needs to be done to support it. But I also think that something must also be done for the etags xref backend. -- /Pierre [-- Attachment #2: Type: text/html, Size: 2927 bytes --] ^ permalink raw reply [flat|nested] 12+ messages in thread
* bug#44494: etags.el xref-location-marker does not handle TAGS references to .el.gz files 2020-11-06 23:22 bug#44494: etags.el xref-location-marker does not handle TAGS references to .el.gz files Pierre Rouleau 2020-11-07 2:22 ` Dmitry Gutov @ 2020-11-07 7:18 ` Eli Zaretskii 1 sibling, 0 replies; 12+ messages in thread From: Eli Zaretskii @ 2020-11-07 7:18 UTC (permalink / raw) To: Pierre Rouleau; +Cc: 44494 > From: Pierre Rouleau <prouleau001@gmail.com> > Date: Fri, 6 Nov 2020 18:22:46 -0500 > > One could consider that the issue is inside the etags utility. I don't think this is the best alternative. TAGS tables are supposed to provide information in a way that doesn't require re-running etags each time some change in the sources is made. The way things are now, compressing or decompressing a file doesn't require re-running etags, which is a Good Thing. Handling this in the code which accesses and interprets TAGS sounds like a better alternative. > Here's a proposal for a solution: > > (defun etags-file-or-compressed-file-for (fname) > "Return the valid file name for FNAME. > Check if FNAME is an existing file name, if not > try FNAME appended with the following compression extensions: > - \".gz\", the extension of compressed files created by gzip > - \".bz2\", the extension for compressed files created by bzip2 > - \".xz\", the extension for compressed files created by xz > - \".lzma\", the extension for compressed files created by xz. > > Return the file that exists or nil if nothing found." > (let ((fpath nil)) > (cl-dolist (ext '("" > ".gz" > ".bz2" > ".xz" > ".lzma")) > (setq fpath (concat fname ext)) > (when (file-exists-p fpath) > (cl-return fpath))))) We already have in etags.el support for finding compressed files, see tag-find-file-of-tag-noselect. Can't that be reused? If not, let's at least reuse tags-compression-info-list, okay? ^ permalink raw reply [flat|nested] 12+ messages in thread
end of thread, other threads:[~2020-11-07 16:09 UTC | newest] Thread overview: 12+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2020-11-06 23:22 bug#44494: etags.el xref-location-marker does not handle TAGS references to .el.gz files Pierre Rouleau 2020-11-07 2:22 ` Dmitry Gutov 2020-11-07 3:31 ` Pierre Rouleau 2020-11-07 8:00 ` Eli Zaretskii 2020-11-07 14:15 ` Pierre Rouleau 2020-11-07 14:22 ` Eli Zaretskii 2020-11-07 14:37 ` Eli Zaretskii 2020-11-07 14:48 ` Pierre Rouleau 2020-11-07 15:39 ` Pierre Rouleau 2020-11-07 15:52 ` Eli Zaretskii 2020-11-07 16:09 ` Pierre Rouleau 2020-11-07 7:18 ` Eli Zaretskii
Code repositories for project(s) associated with this external index https://git.savannah.gnu.org/cgit/emacs.git https://git.savannah.gnu.org/cgit/emacs/org-mode.git This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.