bug#50733: 28.0.1; project-find-regexp can block Emacs for a long time

[Eli Zaretskii <eliz@gnu.org>]

> Cc: mardani29@yahoo.es, 50733@debbugs.gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Fri, 24 Sep 2021 14:40:51 +0300
> 
> > Bundling an external program is problematic, but it can be done, of
> > course, provided that the license is right.  The problem is, we don't
> > provide macOS binaries, and we cannot rely on the fact that the
> > Windows binaries distribution will have its maintainer forever: we
> > already had periods of time without such a person.
> 
> If we just bundled GNU find and GNU grep in both distributions (or, for 
> macOS, tell the distributors to do that), that would improve OOTB 
> behavior for many users. It's not like there's a real chance that the 
> user would prefer some different version of Grep, for example.

You don't need to convince me, I have all of those installed, and
couldn't do without them.  The difficulties are practical, not
principal.

> What is the concern about the presence of the current maintainer? Would 
> we do something different if he leaves? We'd probably need to find 
> another maintainer, no?

Yes, and the question is: will we succeed, and how quickly?  We
already had a period of time without one, which means having no one is
not a theoretical danger.

> > Btw, I don't understand why we focus on general-purpose text-searching
> > tools for these features.  Why not focus on packages like ID Utils
> > instead, they are so much faster.  Daniel, could you time the same
> > search in that large tree when xref-search-program is 'gid'?  (You'd
> > need to run 'mkid' first, to create the ID database, but that is
> > one-time, and is very fast.) 
> 
> There is no such option in xref-search-program.

Hmm... I'm sure this worked in the past, at least for
xref-find-references?  Or does that use a different variable?

> If you can suggest an 
> appropriate invocation for xref-search-program-alist, I can try and add it.

Just "gid -r <R>".  But if this could run in an arbitrary directory,
there should also be a "-f .../ID" argument, telling it where to find
the ID database (usually, in the project's root).

> But I thought id-tools are for scanning for identifiers, not for 
> arbitrary regexp searches?

They can scan for identifiers that match regular expressions, where
"identifier" is defined by each file's PL.

>  >  As I told many times, I think this is
>  > the future: program language sensitive tools that use a precomputed
>  > DB.
> 
> xref-find-references implies some language-awareness. 
> project-find-regexp does not.

Are you sure people indeed use project-find-regexp _only_ for
language-agnostic searches?

Anyway, maybe we need a separate command, or a different (but similar)
tool, one that indexes the tokens in the project files instead of
scanning all of the files each time anew.