bug#26710: Fwd: 25.2; project-find-regexp makes emacs use 100% cpu

[Eli Zaretskii <eliz@gnu.org>]

> Cc: 26710@debbugs.gnu.org, control@debbugs.gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Sun, 30 Apr 2017 13:35:53 +0300
> 
> > When the search is in progress, I'm unable to use emacs.
> > Emacs doesn't respond to any action until the search is over.
> 
> OK, that's a reasonable complaint, especially when the search term is a 
> rarely occurring one (so most of the time is spent in the external process).
> 
> It is non-trivial to fix, however, while keeping the xref backend API 
> and associated code sane.
> 
> I'm hoping that the newly-added concurrency support can help us in that. 
> I have not looked into actually using it, though.
> 
> If someone (Eli?) would like to try their hand at it, or to even outline 
> the basic direction such solution would use, that would be welcome.

I'll try to look at this.  According to my profiling, the lion's share
of time is taken by xref--collect-matches, so that's the place to try
concurrency.

> - Being able to hook up in an asynchronous fashion to that sequence in a 
> (second?) background thread, to render the search results in the xref 
> buffer as soon as they appear.

For the other thread to be background, it will need to yield from time
to time, otherwise the user experience will be identical to what we
have today, because an un-yielding thread will hold the execution unit
until it does its job completely, and no other thread gets to run
until it does.