unofficial mirror of guix-devel@gnu.org 
 help / color / mirror / code / Atom feed
From: Maxim Cournoyer <maxim.cournoyer@gmail.com>
To: guix-devel <guix-devel@gnu.org>
Subject: RFC: libgit2 is slow/inefficient; switch to git command?
Date: Mon, 21 Nov 2022 21:21:02 -0500	[thread overview]
Message-ID: <87cz9fpw4x.fsf@gmail.com> (raw)

Hi,

While attempting to bisect against the Linux kernel tree, the
performance of libgit2 quickly became problematic, to the point where
simply cloning the repo became a multiple hours affair, using upward to
3 GiB of RAM for the clone and indexing of the objects (!)

Given that:

* the git CLI doesn't suffer from such poor performance;
* This kind of performance problem has been known for years in libgit2
  [0] with no fix in sight;
* other projects such as Cargo support using the git CLI and that
  projects are using it for that reason [1];

Would it make sense to switch to use the git command directly instead of
calling into this libgit2 C library that ends up being slower?  It would
provide a hefty speed-up when using 'guix refresh' or building new
packages fetched from git without substitutes, or using 'git-checkout',
etc.

What do you think?

[0] https://github.com/libgit2/libgit2/issues/4674
[1] https://github.com/artichoke/playground/pull/700

-- 
Thanks,
Maxim


             reply	other threads:[~2022-11-22  2:21 UTC|newest]

Thread overview: 8+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-11-22  2:21 Maxim Cournoyer [this message]
2022-11-22 15:39 ` RFC: libgit2 is slow/inefficient; switch to git command? zimoun
2022-11-22 16:49   ` Philip McGrath
2022-11-22 17:51     ` Wojtek Kosior via Development of GNU Guix and the GNU System distribution.
2022-11-22 21:15       ` Phil
2022-11-23  9:57         ` zimoun
2022-11-23 22:04         ` Ludovic Courtès
2022-11-23 22:16 ` Ludovic Courtès

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://guix.gnu.org/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=87cz9fpw4x.fsf@gmail.com \
    --to=maxim.cournoyer@gmail.com \
    --cc=guix-devel@gnu.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/guix.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).