* bug#23223: 25.0.92; Can xref-find-references be sped up? @ 2016-04-05 15:16 Eli Zaretskii 2016-04-06 0:37 ` Dmitry Gutov 0 siblings, 1 reply; 22+ messages in thread From: Eli Zaretskii @ 2016-04-05 15:16 UTC (permalink / raw) To: 23223 With an ID database present in the Emacs top-level directory, do this: emacs -Q C-x C-f src/buffer.c RET C-u 1316 M-x goto-char RET You should now see point on "current_buffer". Now: M-x xref-find-references RET RET When prompted for the project directory, remove the "src/" part, to specify the root of the Emacs tree, and type RET. This takes around 13 sec here, although the corresponding 'lid' command: lid --result=grep -l -w current_buffer takes only 0.07 sec. The important part of the Lisp-level profile appears below. It looks like we are visiting each match of each file in the list of matches, and that takes most of the time. Can this be avoided somehow? The 'lid' command already includes all the matches in Grep format, so why are we visiting each match, when the information should be already available? - command-execute 833 92% - call-interactively 833 92% - funcall-interactively 830 92% - execute-extended-command 830 92% - command-execute 826 91% - call-interactively 815 90% - funcall-interactively 813 90% - xref-find-references 813 90% - xref--find-xrefs 813 90% - xref-backend-references 800 88% - cl-mapcan 788 87% - apply 788 87% - cl-mapcar 788 87% - mapcar 788 87% - #<compiled 0x200000000842dea0> 788 87% - xref-collect-references 788 87% - cl-mapcan 770 85% - apply 770 85% - cl-mapcar 770 85% - mapcar 770 85% - #<compiled 0x20000000084ea738> 770 85% - xref--collect-matches 728 80% - semantic-find-file-noselect 556 61% - find-file-noselect 556 61% - find-file-noselect-1 519 57% - after-find-file 503 55% - normal-mode 499 55% - set-auto-mode 493 54% - set-auto-mode-0 488 54% - c-mode 488 54% - c-common-init 480 53% - mapc 473 52% - #<compiled 0x2000000001c06b28> 472 52% - c-neutralize-syntax-in-and-mark-CPP 466 51% - c-extend-font-lock-region-for-macros 205 22% - c-state-literal-at 205 22% - c-state-safe-place 199 22% - c-beginning-of-macro 3 0% back-to-indentation 3 0% c-syntactic-end-of-macro 1 0% c-syntactic-end-of-macro 9 1% c-neutralize-CPP-line 7 0% - c-beginning-of-macro 4 0% back-to-indentation 2 0% #<compiled 0x2000000001c06b00> 1 0% - c-basic-common-init 7 0% - c-set-style 4 0% - mapc 4 0% - #<compiled 0x2000000001ae61c0> 4 0% - c-set-style-1 2 0% - mapcar 2 0% #<compiled 0x2000000001ae53a0> 1 0% - run-mode-hooks 3 0% In GNU Emacs 25.0.92.75 (i686-pc-mingw32) of 2016-04-04 built on HOME-C4E4A596F7 Repository revision: f501116ea896b20f195f5c841e8770d7fe0418b9 Windowing system distributor 'Microsoft Corp.', version 5.1.2600 Configured using: 'configure --prefix=/d/usr --enable-checking=yes,glyphs --with-wide-int --with-modules 'CFLAGS=-O0 -gdwarf-4 -g3'' Configured features: XPM JPEG TIFF GIF PNG RSVG SOUND NOTIFY ACL GNUTLS LIBXML2 ZLIB TOOLKIT_SCROLL_BARS MODULES Important settings: value of $LANG: ENU locale-coding-system: cp1255 Major mode: Lisp Interaction Minor modes in effect: tooltip-mode: t global-eldoc-mode: t electric-indent-mode: t mouse-wheel-mode: t tool-bar-mode: t menu-bar-mode: t file-name-shadow-mode: t global-font-lock-mode: t font-lock-mode: t blink-cursor-mode: t auto-composition-mode: t auto-encryption-mode: t auto-compression-mode: t line-number-mode: t transient-mark-mode: t Recent messages: For information about GNU Emacs and the GNU system, type C-h C-a. Load-path shadows: None found. Features: (shadow sort mail-extr emacsbug message dired format-spec rfc822 mml mml-sec password-cache epg epg-config gnus-util mm-decode mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail rfc2047 rfc2045 ietf-drums mm-util help-fns help-mode easymenu cl-loaddefs pcase cl-lib mail-prsvr mail-utils time-date mule-util tooltip eldoc electric uniquify ediff-hook vc-hooks lisp-float-type mwheel dos-w32 ls-lisp disp-table w32-win w32-vars term/common-win tool-bar dnd fontset image regexp-opt fringe tabulated-list newcomment elisp-mode lisp-mode prog-mode register page menu-bar rfn-eshadow timer select scroll-bar mouse jit-lock font-lock syntax facemenu font-core frame cl-generic cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech european ethiopic indian cyrillic chinese charscript case-table epa-hook jka-cmpr-hook help simple abbrev minibuffer cl-preloaded nadvice loaddefs button faces cus-face macroexp files text-properties overlay sha1 md5 base64 format env code-pages mule custom widget hashtable-print-readable backquote w32notify w32 multi-tty make-network-process emacs) Memory information: ((conses 16 93253 6020) (symbols 56 20721 0) (miscs 48 36 107) (strings 16 17749 6443) (string-bytes 1 439871) (vectors 16 12404) (vector-slots 8 429720 3894) (floats 8 162 70) (intervals 40 261 118) (buffers 856 11)) ^ permalink raw reply [flat|nested] 22+ messages in thread
* bug#23223: 25.0.92; Can xref-find-references be sped up? 2016-04-05 15:16 bug#23223: 25.0.92; Can xref-find-references be sped up? Eli Zaretskii @ 2016-04-06 0:37 ` Dmitry Gutov 2016-04-06 15:09 ` Dmitry Gutov 2016-04-06 17:12 ` Eli Zaretskii 0 siblings, 2 replies; 22+ messages in thread From: Dmitry Gutov @ 2016-04-06 0:37 UTC (permalink / raw) To: Eli Zaretskii, 23223 On 04/05/2016 06:16 PM, Eli Zaretskii wrote: > The important part of the Lisp-level profile appears below. It looks > like we are visiting each match of each file in the list of matches, > and that takes most of the time. Can this be avoided somehow? The > 'lid' command already includes all the matches in Grep format, so why > are we visiting each match, when the information should be already > available? Huh, yes, that's the area of optimization I've forgotten about (mostly because of using plain Grep myself anyway). If id-utils (and some other tools we delegate to) know symbol boundaries, maybe we don't need to double-check. However, Grep output, which 'lid --result=grep' also uses, only outputs the line number, but not the starting column. Take this example: I'm searching for 'buffer'. The last line of the output looks like this: xmenu.c:832: set_buffer_internal_1 (XBUFFER (buffer)); How do I find the appropriate 'buffer' match in this line? Or can we ask 'lid' (and, ideally, Grep too) to include the column of a match in the output? And when there are several matches on the same line, output that line multiple times? Grep has colorized output, so I should be able to work with that. But not 'lid', it seems. ^ permalink raw reply [flat|nested] 22+ messages in thread
* bug#23223: 25.0.92; Can xref-find-references be sped up? 2016-04-06 0:37 ` Dmitry Gutov @ 2016-04-06 15:09 ` Dmitry Gutov 2016-04-06 17:20 ` Eli Zaretskii 2016-04-06 17:12 ` Eli Zaretskii 1 sibling, 1 reply; 22+ messages in thread From: Dmitry Gutov @ 2016-04-06 15:09 UTC (permalink / raw) To: Eli Zaretskii, 23223 On 04/06/2016 03:37 AM, Dmitry Gutov wrote: > If id-utils (and some other > tools we delegate to) know symbol boundaries Is that actually the case, though? At most, it's using some wonky heuristics. For instance, if I run 'mkid' in the top directory of the Emacs sources, I can search for 'defmacro', and get a few lines with 'cl-defmacro' instead among the results. Or search for 'eval-when', and get some lines with 'eval-when-compile'. But not, curiously, 'cl-eval-when'. id-utils version 4.6 here. ^ permalink raw reply [flat|nested] 22+ messages in thread
* bug#23223: 25.0.92; Can xref-find-references be sped up? 2016-04-06 15:09 ` Dmitry Gutov @ 2016-04-06 17:20 ` Eli Zaretskii 2016-04-06 23:11 ` Dmitry Gutov 0 siblings, 1 reply; 22+ messages in thread From: Eli Zaretskii @ 2016-04-06 17:20 UTC (permalink / raw) To: Dmitry Gutov; +Cc: 23223 > From: Dmitry Gutov <dgutov@yandex.ru> > Date: Wed, 6 Apr 2016 18:09:55 +0300 > > On 04/06/2016 03:37 AM, Dmitry Gutov wrote: > > > If id-utils (and some other > > tools we delegate to) know symbol boundaries > > Is that actually the case, though? At most, it's using some wonky > heuristics. > > For instance, if I run 'mkid' in the top directory of the Emacs sources, > I can search for 'defmacro', and get a few lines with 'cl-defmacro' > instead among the results. Or search for 'eval-when', and get some lines > with 'eval-when-compile'. But not, curiously, 'cl-eval-when'. > > id-utils version 4.6 here. Sounds like a bug in reporting matches. We invoke lid with the -w switch, which should have reported only complete symbols, AFAIU (although the documentation confusingly talks about "words"). Perhaps report this to the developers. Anyway, if you search in the hits instead of in the file's buffer, this problem should not affect us, I think. ^ permalink raw reply [flat|nested] 22+ messages in thread
* bug#23223: 25.0.92; Can xref-find-references be sped up? 2016-04-06 17:20 ` Eli Zaretskii @ 2016-04-06 23:11 ` Dmitry Gutov 2016-04-07 2:49 ` Eli Zaretskii 0 siblings, 1 reply; 22+ messages in thread From: Dmitry Gutov @ 2016-04-06 23:11 UTC (permalink / raw) To: Eli Zaretskii; +Cc: 23223 On 04/06/2016 08:20 PM, Eli Zaretskii wrote: > Sounds like a bug in reporting matches. A bug in the matcher, maybe? > We invoke lid with the -w > switch, which should have reported only complete symbols, AFAIU > (although the documentation confusingly talks about "words"). Perhaps > report this to the developers. I suspected the wording would mean that it just invokes Grep and uses the "boundary" anchors (\b), but no, doing a Grep search for "\beval-when\b" yields a bigger (and more consistent) set of matches. So it's most likely a bug. ^ permalink raw reply [flat|nested] 22+ messages in thread
* bug#23223: 25.0.92; Can xref-find-references be sped up? 2016-04-06 23:11 ` Dmitry Gutov @ 2016-04-07 2:49 ` Eli Zaretskii 0 siblings, 0 replies; 22+ messages in thread From: Eli Zaretskii @ 2016-04-07 2:49 UTC (permalink / raw) To: Dmitry Gutov; +Cc: 23223 > Cc: 23223@debbugs.gnu.org > From: Dmitry Gutov <dgutov@yandex.ru> > Date: Thu, 7 Apr 2016 02:11:39 +0300 > > On 04/06/2016 08:20 PM, Eli Zaretskii wrote: > > > Sounds like a bug in reporting matches. > > A bug in the matcher, maybe? Could be, but less likely. I looked at the Lisp scanner code and didn't say anything that could explain this. Moreover, the fid command correctly shows both defmacro and cl-defmacro being present in, say, lisp/ibuf-macs.el. > > We invoke lid with the -w > > switch, which should have reported only complete symbols, AFAIU > > (although the documentation confusingly talks about "words"). Perhaps > > report this to the developers. > > I suspected the wording would mean that it just invokes Grep and uses > the "boundary" anchors (\b), but no, doing a Grep search for > "\beval-when\b" yields a bigger (and more consistent) set of matches. No, lid doesn't invoke Grep, AFAIR. ^ permalink raw reply [flat|nested] 22+ messages in thread
* bug#23223: 25.0.92; Can xref-find-references be sped up? 2016-04-06 0:37 ` Dmitry Gutov 2016-04-06 15:09 ` Dmitry Gutov @ 2016-04-06 17:12 ` Eli Zaretskii 2016-04-07 0:11 ` Dmitry Gutov 1 sibling, 1 reply; 22+ messages in thread From: Eli Zaretskii @ 2016-04-06 17:12 UTC (permalink / raw) To: Dmitry Gutov; +Cc: 23223 > From: Dmitry Gutov <dgutov@yandex.ru> > Date: Wed, 6 Apr 2016 03:37:39 +0300 > > On 04/05/2016 06:16 PM, Eli Zaretskii wrote: > > > The important part of the Lisp-level profile appears below. It looks > > like we are visiting each match of each file in the list of matches, > > and that takes most of the time. Can this be avoided somehow? The > > 'lid' command already includes all the matches in Grep format, so why > > are we visiting each match, when the information should be already > > available? > > Huh, yes, that's the area of optimization I've forgotten about (mostly > because of using plain Grep myself anyway). But using plain Grep has the same problem: Grep reports the hits, and we then go ahead and visit each hit, right? > If id-utils (and some other tools we delegate to) know symbol > boundaries, maybe we don't need to double-check. It should know. > However, Grep output, which 'lid --result=grep' also uses, only outputs > the line number, but not the starting column. > > Take this example: > > I'm searching for 'buffer'. The last line of the output looks like this: > > xmenu.c:832: set_buffer_internal_1 (XBUFFER (buffer)); > > How do I find the appropriate 'buffer' match in this line? You already have the code that looks for the match in each line. What I had in mind is looking in the output of Grep or lid, instead of visiting the file and looking in that file's buffer. Does this make sense? > Or can we ask 'lid' (and, ideally, Grep too) to include the column of a > match in the output? No, I don't think such an option exists. It could be a great enhancement request, though ;-) > Grep has colorized output, so I should be able to work with that. But > not 'lid', it seems. Another enhancement to request, I guess. ^ permalink raw reply [flat|nested] 22+ messages in thread
* bug#23223: 25.0.92; Can xref-find-references be sped up? 2016-04-06 17:12 ` Eli Zaretskii @ 2016-04-07 0:11 ` Dmitry Gutov 2016-04-07 15:03 ` Eli Zaretskii 0 siblings, 1 reply; 22+ messages in thread From: Dmitry Gutov @ 2016-04-07 0:11 UTC (permalink / raw) To: Eli Zaretskii; +Cc: 23223 On 04/06/2016 08:12 PM, Eli Zaretskii wrote: > But using plain Grep has the same problem: Grep reports the hits, and > we then go ahead and visit each hit, right? Using Grep has the slowness problem, but it doesn't have this opportunity for optimization, since Grep definitely doesn't know what characters constitute a symbol in different languages. But your proposal may speed up Grep searches too, so that's a plus. >> How do I find the appropriate 'buffer' match in this line? > > You already have the code that looks for the match in each line. What > I had in mind is looking in the output of Grep or lid, instead of > visiting the file and looking in that file's buffer. Does this make > sense? Yes, but the search for \_<eval-when\_> needs a properly set syntax table, syntax-propertize-function, and possibly other variables that the latter might depend on. Basically, it needs a buffer in the right major mode. And switching major modes is actually not cheap: (benchmark 230 '(with-temp-buffer (c-mode))) => 1.20s in my working Emacs instance (much faster in a pristine one, admittedly). But I'll have a try at doing this while reusing the temporary buffer. Some major modes could also trip over incomplete source code, but we'll get to that when we see that. Syntax highlighting seems out of the question, though, with high likelihood of getting it wrong. We also have the possibility of syntax-propertize-function not applying the correct syntax classes to the code fragment because it doesn't see the whole file, but I guess it's a small-ish price to pay for the performance improvement, and could be fixed on a case-by-case basis. >> Or can we ask 'lid' (and, ideally, Grep too) to include the column of a >> match in the output? > > No, I don't think such an option exists. It could be a great > enhancement request, though ;-) I was hoping you could serve as a liaison in that, being the sole user of id-utils that I know of, so far. But anyway, if there is no such feature now, we have to choose another solution in Emacs 25.1. Even if id-utils were to add this today, it would take time to trickle to the majority of our users. The bar is actually higher: to make use of the enhancement, we'd need Global and CScope to support it, too, or at least to be able to detect the feature at runtime. ^ permalink raw reply [flat|nested] 22+ messages in thread
* bug#23223: 25.0.92; Can xref-find-references be sped up? 2016-04-07 0:11 ` Dmitry Gutov @ 2016-04-07 15:03 ` Eli Zaretskii 2016-04-09 3:12 ` Dmitry Gutov 0 siblings, 1 reply; 22+ messages in thread From: Eli Zaretskii @ 2016-04-07 15:03 UTC (permalink / raw) To: Dmitry Gutov; +Cc: 23223 > Cc: 23223@debbugs.gnu.org > From: Dmitry Gutov <dgutov@yandex.ru> > Date: Thu, 7 Apr 2016 03:11:25 +0300 > > We invoke lid with the -w > switch, which should have reported only complete symbols, AFAIU > (although the documentation confusingly talks about "words"). Perhaps > report this to the developers. > > I suspected the wording would mean that it just invokes Grep and uses the "boundary" anchors (\b), but no, doing a Grep search for "\beval-when\b" yields a bigger (and more consistent) set of matches. > > So it's most likely a bug. Heh, I completely forgot how IDutils work. The ID database doesn't store the locations where the identifiers were found, all it stores is the list of identifiers found in each file. When lid is invoked, it queries the database about the files that reference the identifier; then it opens each file in the query results, and searches for the identifier on each line of that file. When lid is invoked to produce a Grep-like output and with the -w option (which is what we do), it searches each line for the moral equivalent of [^[:alnum:]_]IDENT[^[:alnum:]_] where IDENT is the identifier we asked about. So therefore, your observation: > For instance, if I run 'mkid' in the top directory of the Emacs sources, I can search for 'defmacro', and get a few lines with 'cl-defmacro' instead among the results. Or search for 'eval-when', and get some lines with 'eval-when-compile'. But not, curiously, 'cl-eval-when'. can now be perfectly explained, because none of the files we have in Emacs that reference cl-eval-when also reference eval-when, but some files that reference defmacro have matches for cl-defmacro, and some files that reference eval-when have matches for eval-when-compile. IOW, the ID database is used to minimize the number of files to search, i.e. avoid searching files that cannot possibly result in a match, that's all. > Or can we ask 'lid' (and, ideally, Grep too) to include the column of a > match in the output? > > No, I don't think such an option exists. It could be a great > enhancement request, though ;-) > > I was hoping you could serve as a liaison in that, being the sole user of id-utils that I know of, so far. I could do that, but given the explanation above of what -w means, it will hardly help us, right? > But anyway, if there is no such feature now, we have to choose another solution in Emacs 25.1. Even if id-utils were to add this today, it would take time to trickle to the majority of our users. > > The bar is actually higher: to make use of the enhancement, we'd need Global and CScope to support it, too, or at least to be able to detect the feature at runtime. Right. So I think we need to assume that the program we invoke to bring the matches is not smart enough anyway, and Emacs will have to search in the matches for those we want to present, filtering out the false positives. This assumption seems to fit well all the programs that can support the queries required by xref-find-references. I'm not sure I understand what you have in mind for that, though. Are you going to switch the major mode of the temporary buffer as determined by the file name of each match? Maybe we should take a hint from the major mode of the buffer in which the command was invoked? Also, how about just using the syntax table of the major mode, instead of actually turning the mode in the buffer -- could that be significantly cheaper? ^ permalink raw reply [flat|nested] 22+ messages in thread
* bug#23223: 25.0.92; Can xref-find-references be sped up? 2016-04-07 15:03 ` Eli Zaretskii @ 2016-04-09 3:12 ` Dmitry Gutov 2016-04-09 7:25 ` Eli Zaretskii 0 siblings, 1 reply; 22+ messages in thread From: Dmitry Gutov @ 2016-04-09 3:12 UTC (permalink / raw) To: Eli Zaretskii; +Cc: 23223 On 04/07/2016 06:03 PM, Eli Zaretskii wrote: > Heh, I completely forgot how IDutils work. The ID database doesn't > store the locations where the identifiers were found, all it stores is > the list of identifiers found in each file. When lid is invoked, it > queries the database about the files that reference the identifier; > then it opens each file in the query results, and searches for the > identifier on each line of that file. When lid is invoked to produce > a Grep-like output and with the -w option (which is what we do), it > searches each line for the moral equivalent of > > [^[:alnum:]_]IDENT[^[:alnum:]_] > > where IDENT is the identifier we asked about. That explains everything, thanks. >> I was hoping you could serve as a liaison in that, being the sole user of id-utils that I know of, so far. > > I could do that, but given the explanation above of what -w means, it > will hardly help us, right? It might if -w were to be improved, though. We can request for it to be smarter and do the post-filtering itself, using a language-aware scanner. Even if no other feature is added, the accuracy of results will improve for all users of id-utils, and Emacs will automatically end up having to do less work for this feature. That's not pressing, though. > Right. So I think we need to assume that the program we invoke to > bring the matches is not smart enough anyway, and Emacs will have to > search in the matches for those we want to present, filtering out the > false positives. This assumption seems to fit well all the programs > that can support the queries required by xref-find-references. Probably. > I'm not sure I understand what you have in mind for that, though. Are > you going to switch the major mode of the temporary buffer as > determined by the file name of each match? Yes. We can avoid switching if the major mode of the previous match was the same as the current one. Having too many major modes involved in one search is highly unlikely, so the switching overhead shouldn't be too much of a problem, actually. > Maybe we should take a > hint from the major mode of the buffer in which the command was > invoked? That could only work for the output of xref-find-references, but not project-find-regexp, which is also in need of speeding up. Anyway, whether we "take a hint" or not, we're going to end up with imperfect results: if we don't visit the target buffers, we're going to have to ignore all the ways Emacs allows specifying the major mode, aside from auto-mode-alist ("mode: " specification, interpreter-mode-alist, magic-mode-alist, magic-fallback-mode-alist). Hmm. Maybe we can still support most of those using (insert-file-contents "file-name" nil 0 200), at the cost of some extra overhead. ^ permalink raw reply [flat|nested] 22+ messages in thread
* bug#23223: 25.0.92; Can xref-find-references be sped up? 2016-04-09 3:12 ` Dmitry Gutov @ 2016-04-09 7:25 ` Eli Zaretskii 2016-04-10 23:27 ` Dmitry Gutov 0 siblings, 1 reply; 22+ messages in thread From: Eli Zaretskii @ 2016-04-09 7:25 UTC (permalink / raw) To: Dmitry Gutov; +Cc: 23223 > Cc: 23223@debbugs.gnu.org > From: Dmitry Gutov <dgutov@yandex.ru> > Date: Sat, 9 Apr 2016 06:12:29 +0300 > > >> I was hoping you could serve as a liaison in that, being the sole user of id-utils that I know of, so far. > > > > I could do that, but given the explanation above of what -w means, it > > will hardly help us, right? > > It might if -w were to be improved, though. We can request for it to be > smarter and do the post-filtering itself, using a language-aware > scanner. Even if no other feature is added, the accuracy of results will > improve for all users of id-utils, and Emacs will automatically end up > having to do less work for this feature. > > That's not pressing, though. I agree. > > I'm not sure I understand what you have in mind for that, though. Are > > you going to switch the major mode of the temporary buffer as > > determined by the file name of each match? > > Yes. We can avoid switching if the major mode of the previous match was > the same as the current one. Having too many major modes involved in one > search is highly unlikely, so the switching overhead shouldn't be too > much of a problem, actually. Would it help to only use the mode's syntax table, and avoid switching on the major mode as a whole? > Anyway, whether we "take a hint" or not, we're going to end up with > imperfect results: if we don't visit the target buffers, we're going to > have to ignore all the ways Emacs allows specifying the major mode, > aside from auto-mode-alist ("mode: " specification, > interpreter-mode-alist, magic-mode-alist, magic-fallback-mode-alist). That problem is relevant for IDutils as well (the scanner is determined by the file's extension only), so we already have a certain (hopefully, small) number of misses and false positives. I think this cannot be entirely avoided. So maybe we shouldn't try so hard avoiding false positives. E.g., the "M-x gid" command, which comes with IDutils and is a trivial wrapper around lid invocation, simply shows the output in a Grep-like buffer, through which you can step with next-error. It is lightning-fast: what takes 13 sec with xref-find-references, takes less than 2 sec with "M-x gid". > Hmm. Maybe we can still support most of those using > (insert-file-contents "file-name" nil 0 200), at the cost of some extra > overhead. Perhaps use insert-file-contents-literally, as decoding could slow down things considerably. You are only looking for ASCII text. Also note that a mode can be specified in file-local variables at the file's end, though. ^ permalink raw reply [flat|nested] 22+ messages in thread
* bug#23223: 25.0.92; Can xref-find-references be sped up? 2016-04-09 7:25 ` Eli Zaretskii @ 2016-04-10 23:27 ` Dmitry Gutov 2016-04-11 15:56 ` Eli Zaretskii 0 siblings, 1 reply; 22+ messages in thread From: Dmitry Gutov @ 2016-04-10 23:27 UTC (permalink / raw) To: Eli Zaretskii; +Cc: 23223 [-- Attachment #1: Type: text/plain, Size: 2745 bytes --] Please try the attached patch. It cuts the time to search for 'current_buffer' from 4.5s to 0.75s here. You can comment out different parts of xref--collect-matches, to test if there are any easy bottlenecks left. The only one I see is find-buffer-visiting; replacing it with get-file-buffer yields some extra boost, but it would come with corresponding downsides in those exact cases for which find-buffer-visiting was designed (although I can add another local variable instead that would cache the previous file name and buffer used for it; but that's scraping the bottom). To measure, open src/xdisp.c and evaluate (benchmark 1 '(xref-collect-references "current_buffer" default-directory)). On 04/09/2016 10:25 AM, Eli Zaretskii wrote: > Would it help to only use the mode's syntax table, and avoid switching > on the major mode as a whole? Not for performance anymore (see above). You can't always determine the syntax table from the major mode name (there is a convention, but it's not iron-clad), it might be in an autoloaded file, etc... And like I mentioned before, we also need syntax-propertize-function, and potentially any buffer-local variables that it could use. > That problem is relevant for IDutils as well (the scanner is > determined by the file's extension only), so we already have a certain > (hopefully, small) number of misses and false positives. That affects xref-find-references (as long as you use id-utils, which is not mandatory) but not project-find-regexp. The currently discussed tuneup should improve both. > I think this > cannot be entirely avoided. So maybe we shouldn't try so hard > avoiding false positives. It seems we can't avoid ignoring the mode specification at the bottom, but that's about it. And nothing's stopping id-utils (or other tools) from using a better language detection scheme in the future. > E.g., the "M-x gid" command, which comes > with IDutils and is a trivial wrapper around lid invocation, simply > shows the output in a Grep-like buffer, through which you can step > with next-error. Do you actually want xref-find-regexp, and not xref-find-references? > It is lightning-fast: what takes 13 sec with > xref-find-references, takes less than 2 sec with "M-x gid". What's the new time you get from the former? > Perhaps use insert-file-contents-literally, as decoding could slow > down things considerably. No significant difference here, on the given example. By the way, the "insert-file-contents + set-auto-mode" dance comes with a new minor downside: extra chatter from the major modes. E.g. try project-file-regexp with "should have received a copy". We can avoid saving it to the message log, but it appears in the echo area either way. [-- Attachment #2: xref-with-temp-buffer.diff --] [-- Type: text/x-patch, Size: 11988 bytes --] diff --git a/lisp/cedet/semantic/symref/cscope.el b/lisp/cedet/semantic/symref/cscope.el index 4890b5b..3abd8b3 100644 --- a/lisp/cedet/semantic/symref/cscope.el +++ b/lisp/cedet/semantic/symref/cscope.el @@ -60,6 +60,9 @@ semantic-symref-tool-cscope (semantic-symref-parse-tool-output tool b) )) +(defconst semantic-symref-cscope--line-re + "^\\([^ ]+\\) [^ ]+ \\([0-9]+\\) ") + (cl-defmethod semantic-symref-parse-tool-output-one-line ((tool semantic-symref-tool-cscope)) "Parse one line of grep output, and return it as a match list. Moves cursor to end of the match." @@ -78,8 +81,13 @@ semantic-symref-tool-cscope ;; We have to return something at this point. subtxt))) ) - (t - (when (re-search-forward "^\\([^ ]+\\) [^ ]+ \\([0-9]+\\) " nil t) + ((eq (oref tool :resulttype) 'line-and-text) + (when (re-search-forward semantic-symref-cscope--line-re nil t) + (list (string-to-number (match-string 2)) + (expand-file-name (match-string 1)) + (buffer-substring-no-properties (point) (line-end-position))))) + (t ; :resulttype is 'line + (when (re-search-forward semantic-symref-cscope--line-re nil t) (cons (string-to-number (match-string 2)) (expand-file-name (match-string 1))) )))) diff --git a/lisp/cedet/semantic/symref/global.el b/lisp/cedet/semantic/symref/global.el index e4c114e..a33427e 100644 --- a/lisp/cedet/semantic/symref/global.el +++ b/lisp/cedet/semantic/symref/global.el @@ -49,6 +49,9 @@ semantic-symref-tool-global (semantic-symref-parse-tool-output tool b) )) +(defconst semantic-symref-global--line-re + "^\\([^ ]+\\) +\\([0-9]+\\) \\([^ ]+\\) ") + (cl-defmethod semantic-symref-parse-tool-output-one-line ((tool semantic-symref-tool-global)) "Parse one line of grep output, and return it as a match list. Moves cursor to end of the match." @@ -57,8 +60,13 @@ semantic-symref-tool-global ;; Search for files (when (re-search-forward "^\\([^\n]+\\)$" nil t) (match-string 1))) + ((eq (oref tool :resulttype) 'line-and-text) + (when (re-search-forward semantic-symref-global--line-re nil t) + (list (string-to-number (match-string 2)) + (match-string 3) + (buffer-substring-no-properties (point) (line-end-position))))) (t - (when (re-search-forward "^\\([^ ]+\\) +\\([0-9]+\\) \\([^ ]+\\) " nil t) + (when (re-search-forward semantic-symref-global--line-re nil t) (cons (string-to-number (match-string 2)) (match-string 3)) )))) diff --git a/lisp/cedet/semantic/symref/grep.el b/lisp/cedet/semantic/symref/grep.el index 5d1fea8..868e6c3 100644 --- a/lisp/cedet/semantic/symref/grep.el +++ b/lisp/cedet/semantic/symref/grep.el @@ -188,6 +188,9 @@ semantic-symref-grep-shell ;; Return the answer ans)) +(defconst semantic-symref-grep--line-re + "^\\(\\(?:[a-zA-Z]:\\)?[^:\n]+\\):\\([0-9]+\\):") + (cl-defmethod semantic-symref-parse-tool-output-one-line ((tool semantic-symref-tool-grep)) "Parse one line of grep output, and return it as a match list. Moves cursor to end of the match." @@ -195,8 +198,13 @@ semantic-symref-grep-shell ;; Search for files (when (re-search-forward "^\\([^\n]+\\)$" nil t) (match-string 1))) + ((eq (oref tool :resulttype) 'line-and-text) + (when (re-search-forward semantic-symref-grep--line-re nil t) + (list (string-to-number (match-string 2)) + (match-string 1) + (buffer-substring-no-properties (point) (line-end-position))))) (t - (when (re-search-forward "^\\(\\(?:[a-zA-Z]:\\)?[^:\n]+\\):\\([0-9]+\\):" nil t) + (when (re-search-forward semantic-symref-grep--line-re nil t) (cons (string-to-number (match-string 2)) (match-string 1)) )))) diff --git a/lisp/cedet/semantic/symref/idutils.el b/lisp/cedet/semantic/symref/idutils.el index 4127d7a..db3e9a0 100644 --- a/lisp/cedet/semantic/symref/idutils.el +++ b/lisp/cedet/semantic/symref/idutils.el @@ -49,6 +49,9 @@ semantic-symref-tool-idutils (semantic-symref-parse-tool-output tool b) )) +(defconst semantic-symref-idutils--line-re + "^\\(\\(?:[a-zA-Z]:\\)?[^:\n]+\\):\\([0-9]+\\):") + (cl-defmethod semantic-symref-parse-tool-output-one-line ((tool semantic-symref-tool-idutils)) "Parse one line of grep output, and return it as a match list. Moves cursor to end of the match." @@ -59,8 +62,13 @@ semantic-symref-tool-idutils ((eq (oref tool :searchtype) 'tagcompletions) (when (re-search-forward "^\\([^ ]+\\) " nil t) (match-string 1))) - (t - (when (re-search-forward "^\\(\\(?:[a-zA-Z]:\\)?[^:\n]+\\):\\([0-9]+\\):" nil t) + ((eq (oref tool :resulttype) 'line-and-text) + (when (re-search-forward semantic-symref-idutils--line-re nil t) + (list (string-to-number (match-string 2)) + (expand-file-name (match-string 1) default-directory) + (buffer-substring-no-properties (point) (line-end-position))))) + (t ; resulttype is line + (when (re-search-forward semantic-symref-idutils--line-re nil t) (cons (string-to-number (match-string 2)) (expand-file-name (match-string 1) default-directory)) )))) diff --git a/lisp/progmodes/xref.el b/lisp/progmodes/xref.el index feed0fb..7354f8b 100644 --- a/lisp/progmodes/xref.el +++ b/lisp/progmodes/xref.el @@ -859,19 +859,25 @@ xref-collect-references ;; to force the backend to use `default-directory'. (let* ((ede-minor-mode nil) (default-directory dir) + ;; FIXME: Remove CScope and Global from the recognized tools? + ;; The current implementations interpret the symbol search as + ;; "find all calls to the given function", but not function + ;; definition. And they return nothing when passed a variable + ;; name, even a global one. (semantic-symref-tool 'detect) (case-fold-search nil) - (res (semantic-symref-find-references-by-name symbol 'subdirs)) - (hits (and res (oref res hit-lines))) - (orig-buffers (buffer-list))) + (inst (semantic-symref-instantiate :searchfor symbol + :searchtype 'symbol + :searchscope 'subdirs + :resulttype 'line-and-text)) + (hits (semantic-symref-perform-search inst)) + (tmp-buffer (generate-new-buffer " *xref-temp*"))) (unwind-protect (cl-mapcan (lambda (hit) (xref--collect-matches - hit (format "\\_<%s\\_>" (regexp-quote symbol)))) + hit (format "\\_<%s\\_>" (regexp-quote symbol)) + tmp-buffer)) hits) - ;; TODO: Implement "lightweight" buffer visiting, so that we - ;; don't have to kill them. - (mapc #'kill-buffer - (cl-set-difference (buffer-list) orig-buffers))))) + (kill-buffer tmp-buffer)))) ;;;###autoload (defun xref-collect-matches (regexp files dir ignores) @@ -890,34 +896,24 @@ xref-collect-matches files (expand-file-name dir) ignores)) - (orig-buffers (buffer-list)) (buf (get-buffer-create " *xref-grep*")) (grep-re (caar grep-regexp-alist)) - (counter 0) - reporter - hits) + hits + (tmp-buffer (generate-new-buffer " *xref-temp*"))) (with-current-buffer buf (erase-buffer) (call-process-shell-command command nil t) (goto-char (point-min)) (while (re-search-forward grep-re nil t) - (push (cons (string-to-number (match-string 2)) - (match-string 1)) + (push (list (string-to-number (match-string 2)) + (match-string 1) + (buffer-substring-no-properties (point) (line-end-position))) hits))) - (setq reporter (make-progress-reporter - (format "Collecting search results...") - 0 (length hits))) (unwind-protect (cl-mapcan (lambda (hit) - (prog1 - (progress-reporter-update reporter counter) - (cl-incf counter)) - (xref--collect-matches hit regexp)) + (xref--collect-matches hit regexp tmp-buffer)) (nreverse hits)) - (progress-reporter-done reporter) - ;; TODO: Same as above. - (mapc #'kill-buffer - (cl-set-difference (buffer-list) orig-buffers))))) + (kill-buffer tmp-buffer)))) (defun xref--rgrep-command (regexp files dir ignores) (require 'find-dired) ; for `find-name-arg' @@ -980,30 +976,49 @@ xref--regexp-to-extended (match-string 1 str))))) str t t)) -(defun xref--collect-matches (hit regexp) - (pcase-let* ((`(,line . ,file) hit) - (buf (or (find-buffer-visiting file) - (semantic-find-file-noselect file)))) - (with-current-buffer buf - (save-excursion +(defvar xref--temp-buffer-file-name nil) + +(defun xref--collect-matches (hit regexp tmp-buffer) + (pcase-let* ((`(,line ,file ,text) hit) + (buf (find-buffer-visiting file))) + (if buf + (with-current-buffer buf + (save-excursion + (goto-char (point-min)) + (forward-line (1- line)) + (xref--collect-matches-1 regexp file line + (line-beginning-position) + (line-end-position)))) + (with-current-buffer tmp-buffer + (erase-buffer) + (unless (equal file xref--temp-buffer-file-name) + (insert-file-contents file nil 0 200) + (setq-local delay-mode-hooks t) + (set-auto-mode t) + (setq-local xref--temp-buffer-file-name file) + (setq-local inhibit-read-only t) + (erase-buffer)) + (insert text) (goto-char (point-min)) - (forward-line (1- line)) - (let ((line-end (line-end-position)) - (line-beg (line-beginning-position)) - matches) - (syntax-propertize line-end) - ;; FIXME: This results in several lines with the same - ;; summary. Solve with composite pattern? - (while (re-search-forward regexp line-end t) - (let* ((beg-column (- (match-beginning 0) line-beg)) - (end-column (- (match-end 0) line-beg)) - (loc (xref-make-file-location file line beg-column)) - (summary (buffer-substring line-beg line-end))) - (add-face-text-property beg-column end-column 'highlight - t summary) - (push (xref-make-match summary loc (- end-column beg-column)) - matches))) - (nreverse matches)))))) + (xref--collect-matches-1 regexp file line + (point) + (point-max)))))) + +(defun xref--collect-matches-1 (regexp file line line-beg line-end) + (let (matches) + (syntax-propertize line-end) + ;; FIXME: This results in several lines with the same + ;; summary. Solve with composite pattern? + (while (re-search-forward regexp line-end t) + (let* ((beg-column (- (match-beginning 0) line-beg)) + (end-column (- (match-end 0) line-beg)) + (loc (xref-make-file-location file line beg-column)) + (summary (buffer-substring line-beg line-end))) + (add-face-text-property beg-column end-column 'highlight + t summary) + (push (xref-make-match summary loc (- end-column beg-column)) + matches))) + (nreverse matches))) (provide 'xref) ^ permalink raw reply related [flat|nested] 22+ messages in thread
* bug#23223: 25.0.92; Can xref-find-references be sped up? 2016-04-10 23:27 ` Dmitry Gutov @ 2016-04-11 15:56 ` Eli Zaretskii 2016-04-11 23:25 ` Dmitry Gutov 0 siblings, 1 reply; 22+ messages in thread From: Eli Zaretskii @ 2016-04-11 15:56 UTC (permalink / raw) To: Dmitry Gutov; +Cc: 23223 > Cc: 23223@debbugs.gnu.org > From: Dmitry Gutov <dgutov@yandex.ru> > Date: Mon, 11 Apr 2016 02:27:34 +0300 > > Please try the attached patch. It cuts the time to search for > 'current_buffer' from 4.5s to 0.75s here. Thanks, I see a similar speedup (from 13 sec to just 3). Very impressive. Unfortunately, it seems to miss matches: out of 1127 matches of current_buffer with the original version, the new one only shows 963. It sounds like some conditions on what exactly is a symbol need adjustment, because the new code finds this: 122: (buf == current_buffer ? BEGV \ but not this: 42: #define BEGV (current_buffer->begv) IOW, if the symbol is followed by a "->", it is not recognized. As an aside, if I invoke xref-find-references without an ID file, which AFAIU means Emacs will invoke find+grep, I get this error: semantic-symref-derive-find-filepatterns: Customize ‘semantic-symref-filepattern-alist’ for lisp-interaction-mode unless I invoke xref-find-references from a buffer in C mode. Curiously, this doesn't happen when there's an ID file and IDutils is invoked. Is this expected? > > E.g., the "M-x gid" command, which comes > > with IDutils and is a trivial wrapper around lid invocation, simply > > shows the output in a Grep-like buffer, through which you can step > > with next-error. > > Do you actually want xref-find-regexp, and not xref-find-references? No, of course not. "gid" is just a short for "lid -R grep", so the contents I get is the same as xref-find-references does, it's just formatted differently. And that is what I want: symbol hits, not just regular expressions that ignore the source language syntax.1 > > It is lightning-fast: what takes 13 sec with > > xref-find-references, takes less than 2 sec with "M-x gid". > > What's the new time you get from the former? 3 sec (in an unoptimized build, I'd expect this to become 1 sec in an optimized build). So we are OK speed-wise, we just need to fix the misses mentioned above. > By the way, the "insert-file-contents + set-auto-mode" dance comes with > a new minor downside: extra chatter from the major modes. E.g. try > project-file-regexp with "should have received a copy". I don't see this in xref-find-references. Should I? > We can avoid saving it to the message log, but it appears in the > echo area either way. Can't you bind inhibit-message to a non-nil value? Thanks again for working on this. ^ permalink raw reply [flat|nested] 22+ messages in thread
* bug#23223: 25.0.92; Can xref-find-references be sped up? 2016-04-11 15:56 ` Eli Zaretskii @ 2016-04-11 23:25 ` Dmitry Gutov 2016-04-12 15:50 ` Eli Zaretskii 0 siblings, 1 reply; 22+ messages in thread From: Dmitry Gutov @ 2016-04-11 23:25 UTC (permalink / raw) To: Eli Zaretskii; +Cc: 23223 [-- Attachment #1: Type: text/plain, Size: 2537 bytes --] On 04/11/2016 06:56 PM, Eli Zaretskii wrote: > Unfortunately, it seems to miss matches: out of 1127 matches of > current_buffer with the original version, the new one only shows 963. > It sounds like some conditions on what exactly is a symbol need > adjustment, Ooh, that's a great catch, thanks! Turns out, auto-mode-alist wasn't getting applied because buffer-file-name wasn't set. Fixing that added ~25% performance hit, so I also added xref--find-buffer-visiting to cache the expensive lookup mentioned previously. See the new patch attached. That seems to about exhaust the optimization opportunity here. > As an aside, if I invoke xref-find-references without an ID file, > which AFAIU means Emacs will invoke find+grep, I get this error: > > semantic-symref-derive-find-filepatterns: Customize ‘semantic-symref-filepattern-alist’ for lisp-interaction-mode > > unless I invoke xref-find-references from a buffer in C mode. ...or the current major mode is one of the currently supported ones, via the above variable. > Curiously, this doesn't happen when there's an ID file and IDutils is > invoked. Is this expected? Yes. semantic-symref-filepattern-alist is defined in and used by semantic/symref/grep.el. We can add (lisp-interaction-mode "*.el") to it. > "gid" is just a short for "lid -R grep", so the > contents I get is the same as xref-find-references does, it's just > formatted differently. Not exactly: you get more false positives because it doesn't apply the language-aware filtering. >> What's the new time you get from the former? > > 3 sec (in an unoptimized build, I'd expect this to become 1 sec in an > optimized build). So we are OK speed-wise, we just need to fix the > misses mentioned above. Cool. >> By the way, the "insert-file-contents + set-auto-mode" dance comes with >> a new minor downside: extra chatter from the major modes. E.g. try >> project-file-regexp with "should have received a copy". > > I don't see this in xref-find-references. Should I? Not at all. xref-find-references only searches in the files related to the current one by type (or in a predefined database like id-utils's one). The chatter I got was from e.g. sh-mode (about the indentation variable being guessed), nxml-mode, and so on, with matches in ~4000 files. >> We can avoid saving it to the message log, but it appears in the >> echo area either way. > > Can't you bind inhibit-message to a non-nil value? That works! > Thanks again for working on this. Thanks for the thoughtful bug report. [-- Attachment #2: xref-with-temp-buffer.diff --] [-- Type: text/x-patch, Size: 13517 bytes --] diff --git a/lisp/cedet/semantic/symref/cscope.el b/lisp/cedet/semantic/symref/cscope.el index 4890b5b..3abd8b3 100644 --- a/lisp/cedet/semantic/symref/cscope.el +++ b/lisp/cedet/semantic/symref/cscope.el @@ -60,6 +60,9 @@ semantic-symref-tool-cscope (semantic-symref-parse-tool-output tool b) )) +(defconst semantic-symref-cscope--line-re + "^\\([^ ]+\\) [^ ]+ \\([0-9]+\\) ") + (cl-defmethod semantic-symref-parse-tool-output-one-line ((tool semantic-symref-tool-cscope)) "Parse one line of grep output, and return it as a match list. Moves cursor to end of the match." @@ -78,8 +81,13 @@ semantic-symref-tool-cscope ;; We have to return something at this point. subtxt))) ) - (t - (when (re-search-forward "^\\([^ ]+\\) [^ ]+ \\([0-9]+\\) " nil t) + ((eq (oref tool :resulttype) 'line-and-text) + (when (re-search-forward semantic-symref-cscope--line-re nil t) + (list (string-to-number (match-string 2)) + (expand-file-name (match-string 1)) + (buffer-substring-no-properties (point) (line-end-position))))) + (t ; :resulttype is 'line + (when (re-search-forward semantic-symref-cscope--line-re nil t) (cons (string-to-number (match-string 2)) (expand-file-name (match-string 1))) )))) diff --git a/lisp/cedet/semantic/symref/global.el b/lisp/cedet/semantic/symref/global.el index e4c114e..a33427e 100644 --- a/lisp/cedet/semantic/symref/global.el +++ b/lisp/cedet/semantic/symref/global.el @@ -49,6 +49,9 @@ semantic-symref-tool-global (semantic-symref-parse-tool-output tool b) )) +(defconst semantic-symref-global--line-re + "^\\([^ ]+\\) +\\([0-9]+\\) \\([^ ]+\\) ") + (cl-defmethod semantic-symref-parse-tool-output-one-line ((tool semantic-symref-tool-global)) "Parse one line of grep output, and return it as a match list. Moves cursor to end of the match." @@ -57,8 +60,13 @@ semantic-symref-tool-global ;; Search for files (when (re-search-forward "^\\([^\n]+\\)$" nil t) (match-string 1))) + ((eq (oref tool :resulttype) 'line-and-text) + (when (re-search-forward semantic-symref-global--line-re nil t) + (list (string-to-number (match-string 2)) + (match-string 3) + (buffer-substring-no-properties (point) (line-end-position))))) (t - (when (re-search-forward "^\\([^ ]+\\) +\\([0-9]+\\) \\([^ ]+\\) " nil t) + (when (re-search-forward semantic-symref-global--line-re nil t) (cons (string-to-number (match-string 2)) (match-string 3)) )))) diff --git a/lisp/cedet/semantic/symref/grep.el b/lisp/cedet/semantic/symref/grep.el index 5d1fea8..868e6c3 100644 --- a/lisp/cedet/semantic/symref/grep.el +++ b/lisp/cedet/semantic/symref/grep.el @@ -188,6 +188,9 @@ semantic-symref-grep-shell ;; Return the answer ans)) +(defconst semantic-symref-grep--line-re + "^\\(\\(?:[a-zA-Z]:\\)?[^:\n]+\\):\\([0-9]+\\):") + (cl-defmethod semantic-symref-parse-tool-output-one-line ((tool semantic-symref-tool-grep)) "Parse one line of grep output, and return it as a match list. Moves cursor to end of the match." @@ -195,8 +198,13 @@ semantic-symref-grep-shell ;; Search for files (when (re-search-forward "^\\([^\n]+\\)$" nil t) (match-string 1))) + ((eq (oref tool :resulttype) 'line-and-text) + (when (re-search-forward semantic-symref-grep--line-re nil t) + (list (string-to-number (match-string 2)) + (match-string 1) + (buffer-substring-no-properties (point) (line-end-position))))) (t - (when (re-search-forward "^\\(\\(?:[a-zA-Z]:\\)?[^:\n]+\\):\\([0-9]+\\):" nil t) + (when (re-search-forward semantic-symref-grep--line-re nil t) (cons (string-to-number (match-string 2)) (match-string 1)) )))) diff --git a/lisp/cedet/semantic/symref/idutils.el b/lisp/cedet/semantic/symref/idutils.el index 4127d7a..db3e9a0 100644 --- a/lisp/cedet/semantic/symref/idutils.el +++ b/lisp/cedet/semantic/symref/idutils.el @@ -49,6 +49,9 @@ semantic-symref-tool-idutils (semantic-symref-parse-tool-output tool b) )) +(defconst semantic-symref-idutils--line-re + "^\\(\\(?:[a-zA-Z]:\\)?[^:\n]+\\):\\([0-9]+\\):") + (cl-defmethod semantic-symref-parse-tool-output-one-line ((tool semantic-symref-tool-idutils)) "Parse one line of grep output, and return it as a match list. Moves cursor to end of the match." @@ -59,8 +62,13 @@ semantic-symref-tool-idutils ((eq (oref tool :searchtype) 'tagcompletions) (when (re-search-forward "^\\([^ ]+\\) " nil t) (match-string 1))) - (t - (when (re-search-forward "^\\(\\(?:[a-zA-Z]:\\)?[^:\n]+\\):\\([0-9]+\\):" nil t) + ((eq (oref tool :resulttype) 'line-and-text) + (when (re-search-forward semantic-symref-idutils--line-re nil t) + (list (string-to-number (match-string 2)) + (expand-file-name (match-string 1) default-directory) + (buffer-substring-no-properties (point) (line-end-position))))) + (t ; resulttype is line + (when (re-search-forward semantic-symref-idutils--line-re nil t) (cons (string-to-number (match-string 2)) (expand-file-name (match-string 1) default-directory)) )))) diff --git a/lisp/progmodes/xref.el b/lisp/progmodes/xref.el index feed0fb..68021c5 100644 --- a/lisp/progmodes/xref.el +++ b/lisp/progmodes/xref.el @@ -839,16 +839,18 @@ xref-etags-mode (kill-local-variable 'xref-backend-functions)) (setq-local xref-backend-functions xref-etags-mode--saved))) -(declare-function semantic-symref-find-references-by-name "semantic/symref") -(declare-function semantic-find-file-noselect "semantic/fw") +(declare-function semantic-symref-instantiate "semantic/symref") +(declare-function semantic-symref-perform-search "semantic/symref") (declare-function grep-expand-template "grep") (defvar ede-minor-mode) ;; ede.el +(defvar xref--last-visiting-buffer nil) + (defun xref-collect-references (symbol dir) "Collect references to SYMBOL inside DIR. This function uses the Semantic Symbol Reference API, see -`semantic-symref-find-references-by-name' for details on which -tools are used, and when." +`semantic-symref-tool-alist' for details on which tools are used, +and when." (cl-assert (directory-name-p dir)) (require 'semantic/symref) (defvar semantic-symref-tool) @@ -859,19 +861,26 @@ xref-collect-references ;; to force the backend to use `default-directory'. (let* ((ede-minor-mode nil) (default-directory dir) + ;; FIXME: Remove CScope and Global from the recognized tools? + ;; The current implementations interpret the symbol search as + ;; "find all calls to the given function", but not function + ;; definition. And they return nothing when passed a variable + ;; name, even a global one. (semantic-symref-tool 'detect) (case-fold-search nil) - (res (semantic-symref-find-references-by-name symbol 'subdirs)) - (hits (and res (oref res hit-lines))) - (orig-buffers (buffer-list))) + (inst (semantic-symref-instantiate :searchfor symbol + :searchtype 'symbol + :searchscope 'subdirs + :resulttype 'line-and-text)) + (hits (semantic-symref-perform-search inst)) + xref--last-visiting-buffer + (tmp-buffer (generate-new-buffer " *xref-temp*"))) (unwind-protect (cl-mapcan (lambda (hit) (xref--collect-matches - hit (format "\\_<%s\\_>" (regexp-quote symbol)))) + hit (format "\\_<%s\\_>" (regexp-quote symbol)) + tmp-buffer)) hits) - ;; TODO: Implement "lightweight" buffer visiting, so that we - ;; don't have to kill them. - (mapc #'kill-buffer - (cl-set-difference (buffer-list) orig-buffers))))) + (kill-buffer tmp-buffer)))) ;;;###autoload (defun xref-collect-matches (regexp files dir ignores) @@ -890,34 +899,25 @@ xref-collect-matches files (expand-file-name dir) ignores)) - (orig-buffers (buffer-list)) (buf (get-buffer-create " *xref-grep*")) (grep-re (caar grep-regexp-alist)) - (counter 0) - reporter - hits) + hits + xref--last-visiting-buffer + (tmp-buffer (generate-new-buffer " *xref-temp*"))) (with-current-buffer buf (erase-buffer) (call-process-shell-command command nil t) (goto-char (point-min)) (while (re-search-forward grep-re nil t) - (push (cons (string-to-number (match-string 2)) - (match-string 1)) + (push (list (string-to-number (match-string 2)) + (match-string 1) + (buffer-substring-no-properties (point) (line-end-position))) hits))) - (setq reporter (make-progress-reporter - (format "Collecting search results...") - 0 (length hits))) (unwind-protect (cl-mapcan (lambda (hit) - (prog1 - (progress-reporter-update reporter counter) - (cl-incf counter)) - (xref--collect-matches hit regexp)) + (xref--collect-matches hit regexp tmp-buffer)) (nreverse hits)) - (progress-reporter-done reporter) - ;; TODO: Same as above. - (mapc #'kill-buffer - (cl-set-difference (buffer-list) orig-buffers))))) + (kill-buffer tmp-buffer)))) (defun xref--rgrep-command (regexp files dir ignores) (require 'find-dired) ; for `find-name-arg' @@ -980,30 +980,60 @@ xref--regexp-to-extended (match-string 1 str))))) str t t)) -(defun xref--collect-matches (hit regexp) - (pcase-let* ((`(,line . ,file) hit) - (buf (or (find-buffer-visiting file) - (semantic-find-file-noselect file)))) - (with-current-buffer buf - (save-excursion +(defvar xref--temp-buffer-file-name nil) + +(defun xref--collect-matches (hit regexp tmp-buffer) + (pcase-let* ((`(,line ,file ,text) hit) + (buf (xref--find-buffer-visiting file))) + (if buf + (with-current-buffer buf + (save-excursion + (goto-char (point-min)) + (forward-line (1- line)) + (xref--collect-matches-1 regexp file line + (line-beginning-position) + (line-end-position)))) + (with-current-buffer tmp-buffer + (erase-buffer) + (unless (equal file xref--temp-buffer-file-name) + (insert-file-contents file nil 0 200) + ;; Can't (setq-local delay-mode-hooks t) because of + ;; bug#23272, but the performance penalty seems minimal. + (let ((buffer-file-name file) + (inhibit-message t) + message-log-max) + (ignore-errors + (set-auto-mode t))) + (setq-local xref--temp-buffer-file-name file) + (setq-local inhibit-read-only t) + (erase-buffer)) + (insert text) (goto-char (point-min)) - (forward-line (1- line)) - (let ((line-end (line-end-position)) - (line-beg (line-beginning-position)) - matches) - (syntax-propertize line-end) - ;; FIXME: This results in several lines with the same - ;; summary. Solve with composite pattern? - (while (re-search-forward regexp line-end t) - (let* ((beg-column (- (match-beginning 0) line-beg)) - (end-column (- (match-end 0) line-beg)) - (loc (xref-make-file-location file line beg-column)) - (summary (buffer-substring line-beg line-end))) - (add-face-text-property beg-column end-column 'highlight - t summary) - (push (xref-make-match summary loc (- end-column beg-column)) - matches))) - (nreverse matches)))))) + (xref--collect-matches-1 regexp file line + (point) + (point-max)))))) + +(defun xref--collect-matches-1 (regexp file line line-beg line-end) + (let (matches) + (syntax-propertize line-end) + ;; FIXME: This results in several lines with the same + ;; summary. Solve with composite pattern? + (while (re-search-forward regexp line-end t) + (let* ((beg-column (- (match-beginning 0) line-beg)) + (end-column (- (match-end 0) line-beg)) + (loc (xref-make-file-location file line beg-column)) + (summary (buffer-substring line-beg line-end))) + (add-face-text-property beg-column end-column 'highlight + t summary) + (push (xref-make-match summary loc (- end-column beg-column)) + matches))) + (nreverse matches))) + +(defun xref--find-buffer-visiting (file) + (unless (equal (car xref--last-visiting-buffer) file) + (setq xref--last-visiting-buffer + (cons file (find-buffer-visiting file)))) + (cdr xref--last-visiting-buffer)) (provide 'xref) ^ permalink raw reply related [flat|nested] 22+ messages in thread
* bug#23223: 25.0.92; Can xref-find-references be sped up? 2016-04-11 23:25 ` Dmitry Gutov @ 2016-04-12 15:50 ` Eli Zaretskii 2016-04-12 18:49 ` Dmitry Gutov 0 siblings, 1 reply; 22+ messages in thread From: Eli Zaretskii @ 2016-04-12 15:50 UTC (permalink / raw) To: Dmitry Gutov; +Cc: 23223 > Cc: 23223@debbugs.gnu.org > From: Dmitry Gutov <dgutov@yandex.ru> > Date: Tue, 12 Apr 2016 02:25:54 +0300 > > > Unfortunately, it seems to miss matches: out of 1127 matches of > > current_buffer with the original version, the new one only shows 963. > > It sounds like some conditions on what exactly is a symbol need > > adjustment, > > Ooh, that's a great catch, thanks! Turns out, auto-mode-alist wasn't > getting applied because buffer-file-name wasn't set. Fixing that added > ~25% performance hit, so I also added xref--find-buffer-visiting to > cache the expensive lookup mentioned previously. See the new patch attached. > > That seems to about exhaust the optimization opportunity here. LGTM, please push. Btw, I noticed another strange thing, but I was only able to reproduce it with the current version, before your patch: the set of results returned by xref-find-references, when it uses IDUtils, is not entirely repeatable. Sometimes, a few matches, sometimes a few dozen of them, are missing. I have no idea why; the lid command consistently returns the same number of lines each time I invoke it. > > semantic-symref-derive-find-filepatterns: Customize ‘semantic-symref-filepattern-alist’ for lisp-interaction-mode > > > > unless I invoke xref-find-references from a buffer in C mode. > > ...or the current major mode is one of the currently supported ones, via > the above variable. > > > Curiously, this doesn't happen when there's an ID file and IDutils is > > invoked. Is this expected? > > Yes. semantic-symref-filepattern-alist is defined in and used by > semantic/symref/grep.el. We can add (lisp-interaction-mode "*.el") to it. I think we should do that, yes. Thanks. ^ permalink raw reply [flat|nested] 22+ messages in thread
* bug#23223: 25.0.92; Can xref-find-references be sped up? 2016-04-12 15:50 ` Eli Zaretskii @ 2016-04-12 18:49 ` Dmitry Gutov 2016-04-12 19:16 ` Eli Zaretskii 0 siblings, 1 reply; 22+ messages in thread From: Dmitry Gutov @ 2016-04-12 18:49 UTC (permalink / raw) To: Eli Zaretskii; +Cc: 23223-done Version: 25.1 On 04/12/2016 06:50 PM, Eli Zaretskii wrote: > LGTM, please push. Done, and closing. > Btw, I noticed another strange thing, but I was only able to reproduce > it with the current version, before your patch: the set of results > returned by xref-find-references, when it uses IDUtils, is not > entirely repeatable. Sometimes, a few matches, sometimes a few dozen > of them, are missing. I have no idea why; the lid command > consistently returns the same number of lines each time I invoke it. That sounds troubling. Please report if you encounter this again, together with: do you get different results when immediately repeating the same search? Or do you search from different files, in different directories? Getting different results between searches from c-mode and emacs-lisp-mode is to be expected, because the latter searches in all load-path as well (and it ends up using Grep in most of those directories anyway). >semantic-symref-filepattern-alist is defined in and used by >> semantic/symref/grep.el. We can add (lisp-interaction-mode "*.el") to it. > > I think we should do that, yes. Also done (with a slightly longer value). ^ permalink raw reply [flat|nested] 22+ messages in thread
* bug#23223: 25.0.92; Can xref-find-references be sped up? 2016-04-12 18:49 ` Dmitry Gutov @ 2016-04-12 19:16 ` Eli Zaretskii 2016-04-12 20:26 ` Dmitry Gutov 0 siblings, 1 reply; 22+ messages in thread From: Eli Zaretskii @ 2016-04-12 19:16 UTC (permalink / raw) To: Dmitry Gutov; +Cc: 23223 > Cc: 23223-done@debbugs.gnu.org > From: Dmitry Gutov <dgutov@yandex.ru> > Date: Tue, 12 Apr 2016 21:49:14 +0300 > > > Btw, I noticed another strange thing, but I was only able to reproduce > > it with the current version, before your patch: the set of results > > returned by xref-find-references, when it uses IDUtils, is not > > entirely repeatable. Sometimes, a few matches, sometimes a few dozen > > of them, are missing. I have no idea why; the lid command > > consistently returns the same number of lines each time I invoke it. > > That sounds troubling. Please report if you encounter this again, I just did, with the current emacs-25 branch. > together with: do you get different results when immediately repeating > the same search? Or do you search from different files, in different > directories? The same search, immediately repeated from the same directory. More accurately, the first time I invoke xref-find-references in the top-level directory of the Emacs repository, I get 980 matches; thereafter, all the subsequent searches from the same Emacs session yield 1128 matches. This is with the ID database present in the top-level directory. > Getting different results between searches from c-mode and > emacs-lisp-mode is to be expected, because the latter searches in all > load-path as well (and it ends up using Grep in most of those > directories anyway). All the searches are done from *scratch* in "emacs -Q". Let me know if I can provide more info. ^ permalink raw reply [flat|nested] 22+ messages in thread
* bug#23223: 25.0.92; Can xref-find-references be sped up? 2016-04-12 19:16 ` Eli Zaretskii @ 2016-04-12 20:26 ` Dmitry Gutov 2016-04-13 2:44 ` Eli Zaretskii 0 siblings, 1 reply; 22+ messages in thread From: Dmitry Gutov @ 2016-04-12 20:26 UTC (permalink / raw) To: Eli Zaretskii; +Cc: 23223 On 04/12/2016 10:16 PM, Eli Zaretskii wrote: > The same search, immediately repeated from the same directory. More > accurately, the first time I invoke xref-find-references in the > top-level directory of the Emacs repository, I get 980 matches; > thereafter, all the subsequent searches from the same Emacs session > yield 1128 matches. This is with the ID database present in the > top-level directory. In the same scenario, I consistently get 1076 matches (on a GNU/Linux system): https://asciinema.org/a/69fdwo7os38fxu01e4fcwftwv > All the searches are done from *scratch* in "emacs -Q". The screencast above starts from Dired, but it's the same if I start from *scratch*. > Let me know if I can provide more info. Could you compare the lists of matches you get? E.g. C-x C-s on the xref buffer in both cases, and then diff the results. ^ permalink raw reply [flat|nested] 22+ messages in thread
* bug#23223: 25.0.92; Can xref-find-references be sped up? 2016-04-12 20:26 ` Dmitry Gutov @ 2016-04-13 2:44 ` Eli Zaretskii 2016-04-13 10:18 ` Dmitry Gutov 0 siblings, 1 reply; 22+ messages in thread From: Eli Zaretskii @ 2016-04-13 2:44 UTC (permalink / raw) To: Dmitry Gutov; +Cc: 23223 > Cc: 23223@debbugs.gnu.org > From: Dmitry Gutov <dgutov@yandex.ru> > Date: Tue, 12 Apr 2016 23:26:13 +0300 > > Could you compare the lists of matches you get? E.g. C-x C-s on the xref > buffer in both cases, and then diff the results. Attached. --- x00 2016-04-13 05:42:17.702125000 +0300 +++ x01 2016-04-13 05:42:35.092750000 +0300 @@ -692,14 +692,162 @@ 2174: i = find_interval (buffer_intervals (current_buffer), pos); 2465: INTERVAL i = buffer_intervals (current_buffer); D:/gnu/git/emacs/branch/src/keyboard.c - 736: && current_buffer != XBUFFER (XWINDOW (selected_window)->contents)) -1410: prev_buffer = current_buffer; -1469: buffer_before_last_command_or_undo = current_buffer; -1516: if (!NILP (BVAR (current_buffer, mark_active)) -1538: && XMARKER (BVAR (current_buffer, mark))->buffer -1553: if (current_buffer != prev_buffer || MODIFF != prev_modiff) -1562: if (current_buffer == prev_buffer + 736: && current_buffer != XBUFFER (XWINDOW (selected_window)->contents)) + 1410: prev_buffer = current_buffer; + 1469: buffer_before_last_command_or_undo = current_buffer; + 1516: if (!NILP (BVAR (current_buffer, mark_active)) + 1538: && XMARKER (BVAR (current_buffer, mark))->buffer + 1553: if (current_buffer != prev_buffer || MODIFF != prev_modiff) + 1562: if (current_buffer == prev_buffer + 1563: && XBUFFER (XWINDOW (selected_window)->contents) == current_buffer + 1633: eassert (XBUFFER (XWINDOW (selected_window)->contents) == current_buffer); + 2833: struct buffer *prev_buffer = current_buffer; + 2844: if (current_buffer != prev_buffer) + 3905: XSETBUFFER (obj, current_buffer); + 7392: if (tem = get_local_map (PT, current_buffer, Qkeymap), !NILP (tem)) + 7399: maps[nmaps++] = get_local_map (PT, current_buffer, Qlocal_map); + 7945: if (tem = get_local_map (PT, current_buffer, Qkeymap), !NILP (tem)) + 7952: maps[nmaps++] = get_local_map (PT, current_buffer, Qlocal_map); + 8808: If FIX_CURRENT_BUFFER, we restore current_buffer + 8935: starting_buffer = current_buffer; + 9151: != current_buffer) + 9163: && current_buffer != starting_buffer) + 9201: != current_buffer)) + 9252: && XBUFFER (XWINDOW (window)->contents) != current_buffer) + 9642: < XCHAR_TABLE (BVAR (current_buffer, downcase_table))->header.size) +11759: * confused because it doesn't realize that the current_buffer was +D:/gnu/git/emacs/branch/src/keymap.c +1486: && XBUFFER (XWINDOW (window)->contents) != current_buffer) +1515: Lisp_Object local_map = get_local_map (pt, current_buffer, Qlocal_map); +1517: Lisp_Object keymap = get_local_map (pt, current_buffer, Qkeymap); +1536: current_buffer, Qlocal_map); +1539: current_buffer, Qkeymap); +1677: map = BVAR (current_buffer, keymap); +1777: bset_keymap (current_buffer, keymap); +1787: return BVAR (current_buffer, keymap); +D:/gnu/git/emacs/branch/src/lread.c +1470: filename = Fexpand_file_name (filename, BVAR (current_buffer, directory)); +D:/gnu/git/emacs/branch/src/marker.c +715: for (tail = BUF_MARKERS (current_buffer); tail; tail = tail->next) +750: BUF_INC_POS (current_buffer, below_byte); +D:/gnu/git/emacs/branch/src/minibuf.c +422: ambient_dir = BVAR (current_buffer, directory); +538: enable_multibyte = BVAR (current_buffer, enable_multibyte_characters); +548: bset_truncate_lines (current_buffer, Qnil); +552: bset_enable_multibyte_characters (current_buffer, enable_multibyte); +557: the current_buffer, so reset_buffer leaves buf's default +563: bset_directory (current_buffer, ambient_dir); +571: bset_directory (current_buffer, +619: if (!NILP (BVAR (current_buffer, enable_multibyte_characters)) +649: bset_keymap (current_buffer, map); +658: bset_undo_list (current_buffer, Qnil); +D:/gnu/git/emacs/branch/src/print.c + 91: struct buffer *old = current_buffer; \ + 97: = !NILP (BVAR (current_buffer, enable_multibyte_characters)); \ +102: if (XBUFFER (printcharfun) != current_buffer) \ +111: if (XMARKER (printcharfun)->buffer != current_buffer) \ +128: if (NILP (BVAR (current_buffer, enable_multibyte_characters)) \ +131: if (! NILP (BVAR (current_buffer, enable_multibyte_characters)) \ +158: && NILP (BVAR (current_buffer, enable_multibyte_characters)))\ +302: = !NILP (BVAR (current_buffer, enable_multibyte_characters)); +359: = !NILP (BVAR (current_buffer, enable_multibyte_characters)); +429: : ! NILP (BVAR (current_buffer, enable_multibyte_characters)))) +543: register struct buffer *old = current_buffer; +551: delete_all_overlays (current_buffer); +552: bset_directory (current_buffer, BVAR (old, directory)); +553: bset_read_only (current_buffer, Qnil); +554: bset_filename (current_buffer, Qnil); +555: bset_undo_list (current_buffer, Qt); +556: eassert (current_buffer->overlays_before == NULL); +557: eassert (current_buffer->overlays_after == NULL); +559: (current_buffer, BVAR (&buffer_defaults, enable_multibyte_characters)); +563: XSETBUFFER (buf, current_buffer); +672: struct buffer *previous = current_buffer; +856: struct buffer *old = current_buffer; +D:/gnu/git/emacs/branch/src/process.c +2194: else if (NILP (BVAR (current_buffer, enable_multibyte_characters))) +3721: else if (NILP (BVAR (current_buffer, enable_multibyte_characters))) +4604: struct buffer *old_buffer = current_buffer; +4612: && (old_buffer != current_buffer +4959: struct buffer *old_buffer = current_buffer; +4974: && (old_buffer != current_buffer +5377: XSETBUFFER (obuffer, current_buffer); +5378: okeymap = BVAR (current_buffer, keymap); +5462: || ! EQ (current_buffer->keymap, okeymap)) +5497: old_read_only = BVAR (current_buffer, read_only); +5503: bset_read_only (current_buffer, Qnil); +5520: if (NILP (BVAR (current_buffer, enable_multibyte_characters)) +5534: && (b = XBUFFER (p->buffer), b != current_buffer)) +5563: bset_read_only (current_buffer, old_read_only); +5723: struct buffer *cur = current_buffer; +6559: XSETBUFFER (obuffer, current_buffer); +6560: okeymap = BVAR (current_buffer, keymap); +6607: || ! EQ (current_buffer->keymap, okeymap)) +6720: struct buffer *old = current_buffer; +6730: if (NILP (BVAR (current_buffer, enable_multibyte_characters))) +6747: tem = BVAR (current_buffer, read_only); +6748: bset_read_only (current_buffer, Qnil); +6754: bset_read_only (current_buffer, tem); +D:/gnu/git/emacs/branch/src/search.c + 147: cp->syntax_table = cp->buf.used_syntax ? BVAR (current_buffer, syntax_table) : Qt; + 226: || EQ (cp->syntax_table, BVAR (current_buffer, syntax_table))) + 275: set_char_table_extras (BVAR (current_buffer, case_canon_table), 2, + 276: BVAR (current_buffer, case_eqv_table)); + 282: (!NILP (BVAR (current_buffer, case_fold_search)) + 283: ? BVAR (current_buffer, case_canon_table) : Qnil), + 285: !NILP (BVAR (current_buffer, enable_multibyte_characters))); + 333: XSETBUFFER (last_thing_searched, current_buffer); + 391: set_char_table_extras (BVAR (current_buffer, case_canon_table), 2, + 392: BVAR (current_buffer, case_eqv_table)); + 397: (!NILP (BVAR (current_buffer, case_fold_search)) + 398: ? BVAR (current_buffer, case_canon_table) : Qnil), + 559: multibyte = ! NILP (BVAR (current_buffer, enable_multibyte_characters)); + 670: newline_cache = newline_cache_on_off (current_buffer); + 671: if (current_buffer->base_buffer) + 672: cache_buffer = current_buffer->base_buffer; + 674: cache_buffer = current_buffer; +1055: set_char_table_extras (BVAR (current_buffer, case_canon_table), 2, +1056: BVAR (current_buffer, case_eqv_table)); +1059: (!NILP (BVAR (current_buffer, case_fold_search)) +1060: ? BVAR (current_buffer, case_canon_table) +1062: (!NILP (BVAR (current_buffer, case_fold_search)) +1063: ? BVAR (current_buffer, case_eqv_table) +1189: !NILP (BVAR (current_buffer, enable_multibyte_characters))); +1243: XSETBUFFER (last_thing_searched, current_buffer); +1287: XSETBUFFER (last_thing_searched, current_buffer); +1312: bool multibyte = !NILP (BVAR (current_buffer, enable_multibyte_characters)); +1528: bool multibyte = ! NILP (BVAR (current_buffer, enable_multibyte_characters)); +1748: bool multibyte = ! NILP (BVAR (current_buffer, enable_multibyte_characters)); +2159: XSETBUFFER (last_thing_searched, current_buffer); +2574: bool buf_multibyte = !NILP (BVAR (current_buffer, enable_multibyte_characters)); +3220: buf = current_buffer; +3225: old = current_buffer; +D:/gnu/git/emacs/branch/src/syntax.c + 423: gl_state.current_syntax_table = BVAR (current_buffer, syntax_table); + 563: if (NILP (BVAR (current_buffer, enable_multibyte_characters))) + 590: if (current_buffer == find_start_buffer + 640: find_start_buffer = current_buffer; + 981: return BVAR (current_buffer, syntax_table); +1026: bset_syntax_table (current_buffer, table); +1029: SET_PER_BUFFER_VALUE_P (current_buffer, idx, 1); +1233: syntax_table = BVAR (current_buffer, syntax_table); +1654: multibyte = (!NILP (BVAR (current_buffer, enable_multibyte_characters)) +2137: multibyte = (!NILP (BVAR (current_buffer, enable_multibyte_characters)) +2895: if (! NILP (BVAR (current_buffer, enable_multibyte_characters))) +D:/gnu/git/emacs/branch/src/term.c +3692: make_save_ptr_ptr (menu, current_buffer)); D:/gnu/git/emacs/branch/src/textprop.c + 83: struct buffer *buf = XBUFFER (buffer), *old = current_buffer; + 94: bset_point_before_scroll (current_buffer, Qnil); + 525: XSETBUFFER (object, current_buffer); + 566: XSETBUFFER (object, current_buffer); + 610: XSETBUFFER (object, current_buffer); + 622: struct buffer *obuf = current_buffer; + 793: if (BUFFERP (object) && current_buffer != XBUFFER (object)) + 877: if (BUFFERP (object) && current_buffer != XBUFFER (object)) + 948: XSETBUFFER (object, current_buffer); +1015: XSETBUFFER (object, current_buffer); +1062: XSETBUFFER (object, current_buffer); 1112: XSETBUFFER (object, current_buffer); 1160: XSETBUFFER (object, current_buffer); 1360: XSETBUFFER (object, current_buffer); ^ permalink raw reply [flat|nested] 22+ messages in thread
* bug#23223: 25.0.92; Can xref-find-references be sped up? 2016-04-13 2:44 ` Eli Zaretskii @ 2016-04-13 10:18 ` Dmitry Gutov 2016-04-13 15:16 ` Eli Zaretskii 0 siblings, 1 reply; 22+ messages in thread From: Dmitry Gutov @ 2016-04-13 10:18 UTC (permalink / raw) To: Eli Zaretskii; +Cc: 23223 On 04/13/2016 05:44 AM, Eli Zaretskii wrote: > Attached. Thank you. Unfortunately, nothing jumps out. Could you try instrumenting a couple of functions, to see where the matches get lost? First, semantic-symref-parse-tool-output, see whether the buffer contents are different between invocations (press `e' and evaluate (write-region (point-min) (point-max) "lid-output-x.txt")). Next, xref--convert-hits. See that its gets the expected number of hits as input (plus maybe a few that we legitimately need to filter out). If both are true, I guess there's nothing better to do than step through xref--convert-hits and see where it loses the matches. But I'm guessing the problem is somewhere before it. ^ permalink raw reply [flat|nested] 22+ messages in thread
* bug#23223: 25.0.92; Can xref-find-references be sped up? 2016-04-13 10:18 ` Dmitry Gutov @ 2016-04-13 15:16 ` Eli Zaretskii 2016-04-15 14:29 ` Eli Zaretskii 0 siblings, 1 reply; 22+ messages in thread From: Eli Zaretskii @ 2016-04-13 15:16 UTC (permalink / raw) To: Dmitry Gutov; +Cc: 23223 > Cc: 23223@debbugs.gnu.org > From: Dmitry Gutov <dgutov@yandex.ru> > Date: Wed, 13 Apr 2016 13:18:14 +0300 > > First, semantic-symref-parse-tool-output, see whether the buffer > contents are different between invocations They are different: the first one has a long run of binary nulls right in the middle of it. So this is unrelated to xref-find-references, it's some basic problem with buffer memory management. Could be the same problem that causes this: http://lists.gnu.org/archive/html/emacs-devel/2016-04/msg00308.html I will debug it, now that I have a reproducible recipe. Thanks. ^ permalink raw reply [flat|nested] 22+ messages in thread
* bug#23223: 25.0.92; Can xref-find-references be sped up? 2016-04-13 15:16 ` Eli Zaretskii @ 2016-04-15 14:29 ` Eli Zaretskii 0 siblings, 0 replies; 22+ messages in thread From: Eli Zaretskii @ 2016-04-15 14:29 UTC (permalink / raw) To: dgutov; +Cc: 23223 > Date: Wed, 13 Apr 2016 18:16:53 +0300 > From: Eli Zaretskii <eliz@gnu.org> > Cc: 23223@debbugs.gnu.org > > > Cc: 23223@debbugs.gnu.org > > From: Dmitry Gutov <dgutov@yandex.ru> > > Date: Wed, 13 Apr 2016 13:18:14 +0300 > > > > First, semantic-symref-parse-tool-output, see whether the buffer > > contents are different between invocations > > They are different: the first one has a long run of binary nulls right > in the middle of it. > > So this is unrelated to xref-find-references, it's some basic problem > with buffer memory management. Could be the same problem that causes > this: > > http://lists.gnu.org/archive/html/emacs-devel/2016-04/msg00308.html > > I will debug it, now that I have a reproducible recipe. Done, see commit ab849b7 on the emacs-25 branch. ^ permalink raw reply [flat|nested] 22+ messages in thread
end of thread, other threads:[~2016-04-15 14:29 UTC | newest] Thread overview: 22+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2016-04-05 15:16 bug#23223: 25.0.92; Can xref-find-references be sped up? Eli Zaretskii 2016-04-06 0:37 ` Dmitry Gutov 2016-04-06 15:09 ` Dmitry Gutov 2016-04-06 17:20 ` Eli Zaretskii 2016-04-06 23:11 ` Dmitry Gutov 2016-04-07 2:49 ` Eli Zaretskii 2016-04-06 17:12 ` Eli Zaretskii 2016-04-07 0:11 ` Dmitry Gutov 2016-04-07 15:03 ` Eli Zaretskii 2016-04-09 3:12 ` Dmitry Gutov 2016-04-09 7:25 ` Eli Zaretskii 2016-04-10 23:27 ` Dmitry Gutov 2016-04-11 15:56 ` Eli Zaretskii 2016-04-11 23:25 ` Dmitry Gutov 2016-04-12 15:50 ` Eli Zaretskii 2016-04-12 18:49 ` Dmitry Gutov 2016-04-12 19:16 ` Eli Zaretskii 2016-04-12 20:26 ` Dmitry Gutov 2016-04-13 2:44 ` Eli Zaretskii 2016-04-13 10:18 ` Dmitry Gutov 2016-04-13 15:16 ` Eli Zaretskii 2016-04-15 14:29 ` Eli Zaretskii
Code repositories for project(s) associated with this external index https://git.savannah.gnu.org/cgit/emacs.git https://git.savannah.gnu.org/cgit/emacs/org-mode.git This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.