vc-git-grep vs project-find-regexp

vc-git-grep vs project-find-regexp

[Nicolás Ojeda Bär]

Hello,

I have used M-x vc-git-grep for a long time, but decided to give M-x
project-find-regexp a try after switching to 29.1.

While it seems to work well, project-find-regexp seems much slower
than vc-git-grep. After you execute the command, it pauses for what it
feels a long time (~2s in a medium-size codebase) before showing the
results. vc-git-grep on the other hand, starts displaying results
immediately.

Is there a good reason for this difference? The only visible
difference between both commands is that project-find-regexp's results
are displayed in a XREF buffer while vc-git-grep uses Grep mode.

Insights appreciated!

Best wishes,
Nicolas

Re: vc-git-grep vs project-find-regexp

[Philip Kaludercic]

Nicolás Ojeda Bär <n.oje.bar@gmail.com> writes:

> Hello,
>
> I have used M-x vc-git-grep for a long time, but decided to give M-x
> project-find-regexp a try after switching to 29.1.
>
> While it seems to work well, project-find-regexp seems much slower
> than vc-git-grep. After you execute the command, it pauses for what it
> feels a long time (~2s in a medium-size codebase) before showing the
> results. vc-git-grep on the other hand, starts displaying results
> immediately.

I suspect the reason is that in the former case, you are actually using
the "git grep" sub-command, that is more efficient, while
`project-find-regexp' defers to `xref-matches-in-files' and uses
`xref-search-program', which is set to regular "grep" by default.  You
can try out the other available tools (ripgrep and ugrep) to see if it
might improve your performance?

> Is there a good reason for this difference? The only visible
> difference between both commands is that project-find-regexp's results
> are displayed in a XREF buffer while vc-git-grep uses Grep mode.

That shouldn't make much of a difference, the main bottle-neck here is
disk IO.

> Insights appreciated!
>
> Best wishes,
> Nicolas

Re: vc-git-grep vs project-find-regexp

[Eli Zaretskii]

> From: Philip Kaludercic <philipk@posteo.net>
> Cc: emacs-devel@gnu.org
> Date: Mon, 21 Aug 2023 18:51:59 +0000
> 
> I suspect the reason is that in the former case, you are actually using
> the "git grep" sub-command, that is more efficient, while
> `project-find-regexp' defers to `xref-matches-in-files' and uses
> `xref-search-program', which is set to regular "grep" by default.

Isn't this rather because vc-git-grep runs the shell command
asynchronously, whereas xref-matches-in-files runs synchronously?

Re: vc-git-grep vs project-find-regexp

[Rudolf Adamkovič]

Philip Kaludercic <philipk@posteo.net> writes:

> [...], while `project-find-regexp' defers to `xref-matches-in-files'
> and uses `xref-search-program', which is set to regular "grep" by
> default.

I have always wondered, why does Emacs not
test if a modern grep is available on PATH
and use it by default?

Rudy
-- 
"The whole science is nothing more than a refinement of everyday
thinking."
-- Albert Einstein, 1879-1955

Rudolf Adamkovič <salutis@me.com> [he/him]
Studenohorská 25
84103 Bratislava
Slovakia

Re: vc-git-grep vs project-find-regexp

[Daniel Martín]

Nicolás Ojeda Bär <n.oje.bar@gmail.com> writes:

> Hello,
>
> I have used M-x vc-git-grep for a long time, but decided to give M-x
> project-find-regexp a try after switching to 29.1.
>
> While it seems to work well, project-find-regexp seems much slower
> than vc-git-grep. After you execute the command, it pauses for what it
> feels a long time (~2s in a medium-size codebase) before showing the
> results. vc-git-grep on the other hand, starts displaying results
> immediately.
>
> Is there a good reason for this difference? The only visible
> difference between both commands is that project-find-regexp's results
> are displayed in a XREF buffer while vc-git-grep uses Grep mode.
>
> Insights appreciated!
>
> Best wishes,
> Nicolas

See bug#50733.  There were some performance improvements as a result of
that discussion, but I think that, without substantial architectural
changes to Xref, git grep/grep will always "feel" faster because the
grep process is executed asynchronously.

Re: vc-git-grep vs project-find-regexp

[Dmitry Gutov]

Hi!

On 21/08/2023 18:29, Nicolás Ojeda Bär wrote:
> While it seems to work well, project-find-regexp seems much slower
> than vc-git-grep. After you execute the command, it pauses for what it
> feels a long time (~2s in a medium-size codebase) before showing the
> results. vc-git-grep on the other hand, starts displaying results
> immediately.

Have you tried measuring the time until Grep finishes too (not just 
until the output buffer is shown)? It's likely to be faster by that 
measure as well (for the moment), but that's not a certainty. Just 
curious how those compare as well.

> Is there a good reason for this difference? The only visible
> difference between both commands is that project-find-regexp's results
> are displayed in a XREF buffer while vc-git-grep uses Grep mode.

Xref uses a couple of abstractions to be able to show information from 
different sources. That has a cost, however. To be able to show the 
results buffer right away (while the search is in progress), requires 
more effort, including:

- Listing files in a project asynchronously (so that intermediate 
reporting is possible) with performance comparable to the synchronous 
approach, simply is order not to regress on that speed (there is some 
progress, I think, in the latest messages in bug#64735).

- To build on that, using some more advanced data structures than just 
lists for project files and search matches. Lazy sequences of futures, 
maybe.

The ability to abort a search in progress will require some thought too: 
currently it comes for free with 'C-g' in Xref; in Grep it's one 'C-c 
C-c' away because the reference to the process is available; but if we 
do an opaque data structure, it could need an "abort" primitive as well.