Re: RFC: libgit2 is slow/inefficient; switch to git command?

[zimoun <zimon.toutoune@gmail.com>]

Hi,

On Mon, 21 Nov 2022 at 21:21, Maxim Cournoyer <maxim.cournoyer@gmail.com> wrote:

> Given that:
>
> * the git CLI doesn't suffer from such poor performance;
> * This kind of performance problem has been known for years in libgit2
>   [0] with no fix in sight;
> * other projects such as Cargo support using the git CLI and that
>   projects are using it for that reason [1];

And I would add the lack of «Support for shallow repositories» [1].

1: <https://github.com/libgit2/libgit2/issues/3058>


> Would it make sense to switch to use the git command directly instead of
> calling into this libgit2 C library that ends up being slower?  It would
> provide a hefty speed-up when using 'guix refresh' or building new
> packages fetched from git without substitutes, or using 'git-checkout',
> etc.

Well, the question is about the closure and the bootstrap.

For instance,

--8<---------------cut here---------------start------------->8---
$ guix size guix | grep 'total:'
total: 629.5 MiB

$ guix size guix git-minimal | grep 'total:'
total: 671.0 MiB
--8<---------------cut here---------------end--------------->8---

which is not nothing but not so worse neither.  However, it would
require a fine scrutinizing about what would be added as dependencies.

The proposal is to fully drop ’guile-git’ and instead run ’(invoke "git"
<thing>)’ as in the module ’(guix build git)’, right?

Cheers,
simon

PS: For the record, Software Heritage, which ingests *a lot* of Git
repositories, relies on Dulwhich [2] (pure Python implementation), IIUC.

2: <https://www.dulwich.io/>