bug#26710: Fwd: 25.2; project-find-regexp makes emacs use 100% cpu

[Eli Zaretskii <eliz@gnu.org>]

> Cc: hariharanrangasamy@gmail.com, control@debbugs.gnu.org,
>  26710@debbugs.gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Mon, 1 May 2017 05:42:21 +0300
> 
> On 30.04.2017 21:47, Eli Zaretskii wrote:
> 
> > I'll try to look at this.  According to my profiling, the lion's share
> > of time is taken by xref--collect-matches, so that's the place to try
> > concurrency.
> 
> I think that's too late. By the time xref--collect-matches is called 
> (and it's called for each hit), we've already spent time synchronously 
> waiting for the find-grep invocation to finish.

In my testing, find-grep finishes almost instantaneously.  The
exception is when you have a cold cache, but even then it takes about
10% of the total run time, for the Emacs source tree (which yields
about 100,000 hits in the test case).

> When there are a lot of matches, xref--collect-matches can take some 
> significant time, with opening the buffers and everything. That can be 
> optimized, however, as a separate issue, and I don't think there's 
> anything to parallelize there, since it all happens in Elisp.

I thought the request was to allow the user do something in the
foreground, while this processing runs in the background.  If that's
not what was requested, then I guess I no longer understand the
request.

> What we _can_ manage to run in parallel, in the find-grep process in the 
> background, and the post-processing of the results in Elisp.

Yes, you can -- if you invoke find-grep asynchronously and move the
processing of the hits to the filter function.  But that doesn't need
to involve threads, and is being done in many packages/features out
there, so I'm not sure what did you ask me to do with this.  IOW, it
should be "trivial", at least in principle, to make this command work
in the background, just like, say, "M-x grep".

> Here's how I imagine it:

I'm not sure I understand the need for this complexity, given that
async subprocesses are available.  I'm probably missing something
because I know too little about the internals of the involved code.