bug#64735: 29.0.92; find invocations are ~15x slower because of ignores

[Eli Zaretskii <eliz@gnu.org>]

> From: Spencer Baugh <sbaugh@janestreet.com>
> Date: Wed, 19 Jul 2023 17:16:31 -0400
> 
> 
> Several important commands and functions invoke find; for example rgrep
> and project-find-regexp.
> 
> Most of these add some set of ignores to the find command, pulling from
> grep-find-ignored-files in the former case.  So the find command looks
> like:
> 
> find -H . \( -path \*/SCCS/\* -o -path \*/RCS/\* [...more ignores...] \)
> -prune -o -type f -print0
> 
> Alas, on my system, using GNU find, these ignores slow down find by
> about 15x on a large directory tree, taking it from around .5 seconds to
> 7.8 seconds.
> 
> This is very noticeable overhead; removing the ignores makes rgrep and
> other find-invoking commands substantially faster for me.

grep-find-ignored-files is a customizable user option, so if this
slowdown bothers you, just customize it to avoid that.  And if there
are patterns there that are no longer pertinent or rare, we could
remove them from the default value.

I'm not sure we should bother more than these two simple measures.

> The overhead is linear in the number of ignores - that is, each
> additional ignore adds a small fixed cost.  This suggests that find is
> linearly scanning the list of ignores and checking each one, rather than
> optimizing them to a single regexp and checking that regexp.

If it uses fnmatch, it cannot do it any other way, I think