Question about dired-do-find-regexp and xref-collect-matches

Question about dired-do-find-regexp and xref-collect-matches

[Eric Abrahamsen]

A while ago I switched to the fish shell, which I generally like better
than bash.

A bit after that I noticed that marking files in Dired and using "A"
(dired-do-find-regexp) to search them was extraordinarily slow. Going up
a level to the parent directory and only marking the child directory,
the search was nearly instantaneous. But that meant I had to search all
the files in the child directory.

I suspected it was something to do with fish, and indeed setting
shell-file-name to "/usr/bin/bash" sped it up a little -- but that's a
bug report for another day.

While poking around I also noticed that dired-do-find-regexp calls
xref-collect-matches once per marked file. That means a full find+grep
call for each file, when a single find+grep call (or even just a single
grep call!) would do the trick.

The search hits are collected as:

(mapcan
 (lambda (file)
   (xref-collect-matches regexp "*" file
                         (and (file-directory-p file)
                              ignores)))
 files)

But the second argument to `xref-collect-matches' can be a
space-separated string of file names -- wouldn't it be easier just to
call `xref-collect-matches' once?

Eric

RE: Question about dired-do-find-regexp and xref-collect-matches

[Drew Adams]

> I noticed that marking files in Dired and using "A"
> (dired-do-find-regexp) to search them was extraordinarily slow. Going up
> a level to the parent directory and only marking the child directory,
> the search was nearly instantaneous. But that meant I had to search all
> the files in the child directory.
> 
> I suspected it was something to do with fish, and indeed setting
> shell-file-name to "/usr/bin/bash" sped it up a little -- but that's a
> bug report for another day.
> 
> While poking around I also noticed that dired-do-find-regexp calls
> xref-collect-matches once per marked file. That means a full find+grep
> call for each file, when a single find+grep call (or even just a single
> grep call!) would do the trick.
> 
> The search hits are collected as:
> 
> (mapcan
>  (lambda (file)
>    (xref-collect-matches regexp "*" file
>                          (and (file-directory-p file)
>                               ignores)))
>  files)
> 
> But the second argument to `xref-collect-matches' can be a
> space-separated string of file names -- wouldn't it be easier just to
> call `xref-collect-matches' once?

Meanwhile (fortunately), you can still use
`dired-do-search' (which used to be bound to `A').

If that wasn't "extraordinarily slow" for you
before, it still might not be.  Maybe try it.

Re: Question about dired-do-find-regexp and xref-collect-matches

[Eric Abrahamsen]

Drew Adams <drew.adams@oracle.com> writes:


[...]

> Meanwhile (fortunately), you can still use
> `dired-do-search' (which used to be bound to `A').
>
> If that wasn't "extraordinarily slow" for you
> before, it still might not be. Maybe try it.

That returns instantly! At least, for the first result. But the xref
interface is so nice to use...

Re: Question about dired-do-find-regexp and xref-collect-matches

[Dmitry Gutov]

Hi Eric,

On 10.07.2019 2:18, Eric Abrahamsen wrote:
> A while ago I switched to the fish shell, which I generally like better
> than bash.

I also like fish, a lot. I don't use it inside Emacs, though.

> While poking around I also noticed that dired-do-find-regexp calls
> xref-collect-matches once per marked file. That means a full find+grep
> call for each file, when a single find+grep call (or even just a single
> grep call!) would do the trick.

That sounds like a bug report material. Up until now I figured a few 
extra shell process calls wouldn't matter, but maybe they can.

Just how many files are you marking in the given example?

> The search hits are collected as:
> 
> (mapcan
>   (lambda (file)
>     (xref-collect-matches regexp "*" file
>                           (and (file-directory-p file)
>                                ignores)))
>   files)
> 
> But the second argument to `xref-collect-matches' can be a
> space-separated string of file names -- wouldn't it be easier just to
> call `xref-collect-matches' once?

Not exactly. That FILES argument is something different (a list of globs).

Overall, this shouldn't be too hard to fix. Could you send your response 
to the bug tracker as a new report?

Re: Question about dired-do-find-regexp and xref-collect-matches

[Eric Abrahamsen]

Dmitry Gutov <dgutov@yandex.ru> writes:

> Hi Eric,
>
> On 10.07.2019 2:18, Eric Abrahamsen wrote:
>> A while ago I switched to the fish shell, which I generally like better
>> than bash.
>
> I also like fish, a lot. I don't use it inside Emacs, though.

Yeah, I just set Emacs' shell to bash, seems safer.

>> While poking around I also noticed that dired-do-find-regexp calls
>> xref-collect-matches once per marked file. That means a full find+grep
>> call for each file, when a single find+grep call (or even just a single
>> grep call!) would do the trick.
>
> That sounds like a bug report material. Up until now I figured a few
> extra shell process calls wouldn't matter, but maybe they can.
>
> Just how many files are you marking in the given example?

This was somewhere between 250-300 XML files.

>> The search hits are collected as:
>>
>> (mapcan
>>   (lambda (file)
>>     (xref-collect-matches regexp "*" file
>>                           (and (file-directory-p file)
>>                                ignores)))
>>   files)
>>
>> But the second argument to `xref-collect-matches' can be a
>> space-separated string of file names -- wouldn't it be easier just to
>> call `xref-collect-matches' once?
>
> Not exactly. That FILES argument is something different (a list of globs).
>
> Overall, this shouldn't be too hard to fix. Could you send your
> response to the bug tracker as a new report?

Done, as bug#36857. Thanks for looking at this.