Please try the attached patch. It cuts the time to search for 
'current_buffer' from 4.5s to 0.75s here.

You can comment out different parts of xref--collect-matches, to test if 
there are any easy bottlenecks left. The only one I see is 
find-buffer-visiting; replacing it with get-file-buffer yields some 
extra boost, but it would come with corresponding downsides in those 
exact cases for which find-buffer-visiting was designed (although I can 
add another local variable instead that would cache the previous file 
name and buffer used for it; but that's scraping the bottom).

To measure, open src/xdisp.c and evaluate (benchmark 1 
'(xref-collect-references "current_buffer" default-directory)).

On 04/09/2016 10:25 AM, Eli Zaretskii wrote:

> Would it help to only use the mode's syntax table, and avoid switching
> on the major mode as a whole?

Not for performance anymore (see above). You can't always determine the 
syntax table from the major mode name (there is a convention, but it's 
not iron-clad), it might be in an autoloaded file, etc... And like I 
mentioned before, we also need syntax-propertize-function, and 
potentially any buffer-local variables that it could use.

> That problem is relevant for IDutils as well (the scanner is
> determined by the file's extension only), so we already have a certain
> (hopefully, small) number of misses and false positives.

That affects xref-find-references (as long as you use id-utils, which is 
not mandatory) but not project-find-regexp. The currently discussed 
tuneup should improve both.

> I think this
> cannot be entirely avoided.  So maybe we shouldn't try so hard
> avoiding false positives.

It seems we can't avoid ignoring the mode specification at the bottom, 
but that's about it. And nothing's stopping id-utils (or other tools) 
from using a better language detection scheme in the future.

> E.g., the "M-x gid" command, which comes
> with IDutils and is a trivial wrapper around lid invocation, simply
> shows the output in a Grep-like buffer, through which you can step
> with next-error.

Do you actually want xref-find-regexp, and not xref-find-references?

> It is lightning-fast: what takes 13 sec with
> xref-find-references, takes less than 2 sec with "M-x gid".

What's the new time you get from the former?

> Perhaps use insert-file-contents-literally, as decoding could slow
> down things considerably.

No significant difference here, on the given example.

By the way, the "insert-file-contents + set-auto-mode" dance comes with 
a new minor downside: extra chatter from the major modes. E.g. try 
project-file-regexp with "should have received a copy". We can avoid 
saving it to the message log, but it appears in the echo area either way.