From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp11.migadu.com ([2001:41d0:306:2d92::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms9.migadu.com with LMTPS id wNqZJdEn/2SrUgEA9RJhRA:P1 (envelope-from ) for ; Mon, 11 Sep 2023 16:44:33 +0200 Received: from aspmx1.migadu.com ([2001:41d0:306:2d92::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp11.migadu.com with LMTPS id wNqZJdEn/2SrUgEA9RJhRA (envelope-from ) for ; Mon, 11 Sep 2023 16:44:33 +0200 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 39E1456924 for ; Mon, 11 Sep 2023 16:44:33 +0200 (CEST) Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=wolfsden.cz header.s=mail header.b=bTC939dB; dkim=fail ("headers rsa verify failed") header.d=wolfsden.cz header.s=mail header.b="qW4/Mjo4"; spf=pass (aspmx1.migadu.com: domain of "bug-guix-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="bug-guix-bounces+larch=yhetil.org@gnu.org"; dmarc=fail reason="SPF not aligned (relaxed)" header.from=wolfsden.cz (policy=none) ARC-Seal: i=1; s=key1; d=yhetil.org; t=1694443473; a=rsa-sha256; cv=none; b=d9Up5f/QGnlwnFPTEtJZFLKyg82TeU+Kp2YEACA4pPVvfChb3oJ+WsF7Fb40mqltjWOb1+ PBb3ugUUtB645syocazKO7iOjCUX9dupCgblbkessTqZ5QqsstgdapQWA9P7y6sLNCUz+Y 3aEz/p3Ym9YAU8ulMZ43l4YkITC75y7lcNZQPknC9XjLWrh4VQseRsoob7jkWOHyu3mudQ 3BKJ+LBVWe4oB4/Dc5I3rfHcCWUBNnzkZ1poGa4WLhYcnwEJF+BsWLb3sQaVJ5LAWRwv1X 5vcFkwxEUAMpEeP/q0vuUYaSv9pf8b9OYfAjRXf9H8gs5Lqu3ILZkNHJDRIDqw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1694443473; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:resent-cc:resent-from:resent-sender: resent-message-id:in-reply-to:in-reply-to:references:references: list-id:list-help:list-unsubscribe:list-subscribe:list-post: dkim-signature; bh=pDtd2EQrcKvk1fn8NI6VLHaOtQyTAqHrUJ2FK+4St7E=; b=MooAOGGipohAclRSRnH9CzwhzUfyZB4rBuvXClpOSy5GH7IRhxQMdRFHM6K0DVzUbss91A qiQ42OdEbzCd73dJWG9OZd3Eti1nPQBpMoktmkPLEmfUECBV5VYFhaXF+kSPwLKlR6pFZ/ //9yozfmvQ3OtCLOpIzQym7NG8Rs71Vq0BBiJt+5Axv2pLJJIbbvePta0TCDm8o/tQgqdd bdWeGv0FoO+ZrArsLWHkZkm0TMkeAhBV8f8yo8aQ8Ivo25N26GYJugjd+4Iq6KFYhmgAdZ EtJs62wFwvxWheBRik6KGDlDnXLs8VBjN9MspKqgF//7b0kBve5cKE+2BqlGVw== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=wolfsden.cz header.s=mail header.b=bTC939dB; dkim=fail ("headers rsa verify failed") header.d=wolfsden.cz header.s=mail header.b="qW4/Mjo4"; spf=pass (aspmx1.migadu.com: domain of "bug-guix-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="bug-guix-bounces+larch=yhetil.org@gnu.org"; dmarc=fail reason="SPF not aligned (relaxed)" header.from=wolfsden.cz (policy=none) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qfi9A-0006qq-Tm; Mon, 11 Sep 2023 10:44:17 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qfi97-0006aw-Ny for bug-guix@gnu.org; Mon, 11 Sep 2023 10:44:14 -0400 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qfi8s-0006P9-2j for bug-guix@gnu.org; Mon, 11 Sep 2023 10:44:13 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1qfi8w-0007A3-48 for bug-guix@gnu.org; Mon, 11 Sep 2023 10:44:02 -0400 X-Loop: help-debbugs@gnu.org Subject: bug#65720: Guile-Git-managed checkouts grow way too much Resent-From: wolf Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Mon, 11 Sep 2023 14:44:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 65720 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Cc: Josselin Poiret , 65720@debbugs.gnu.org Received: via spool by 65720-submit@debbugs.gnu.org id=B65720.169444339127461 (code B ref 65720); Mon, 11 Sep 2023 14:44:02 +0000 Received: (at 65720) by debbugs.gnu.org; 11 Sep 2023 14:43:11 +0000 Received: from localhost ([127.0.0.1]:54353 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qfi86-00078r-Rb for submit@debbugs.gnu.org; Mon, 11 Sep 2023 10:43:11 -0400 Received: from wolfsden.cz ([37.205.8.62]:49238) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qfi82-00078e-Kf for 65720@debbugs.gnu.org; Mon, 11 Sep 2023 10:43:08 -0400 Received: by wolfsden.cz (Postfix, from userid 104) id 55CFE25A352; Mon, 11 Sep 2023 14:43:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=wolfsden.cz; s=mail; t=1694443380; bh=9UzFheRlBpqwhqYLwxRvcCIW6Z1Q4wMYjKudYbhoL0k=; h=Date:From:To:Cc:Subject:References:In-Reply-To; b=bTC939dBzFFk6esxnKSOdRbxsN5LTCAxeP+eUlfD6WXcMK1fPf+c/OMD3TJE9JWA0 V8YqtIuX2v7qqB89+jlxymepApZVyeZ7Wmx8KIFblCz3mp0xyBQDZHKu1e8H00TIak OW/PStKUBtzR554J/Q9rLtq6H8yKWbiAovHXG/A4nhLajO9+HQMNcC/e+ylMIrKVdy /UmiT1xTV8gMEw7Exngp0L283yRGtMMtiEmwhWNVKDxFpEo87xKvQ4aLmyyG0QB/f+ BhuIBpNggW9jk28mNB6Kk5Asyuow9HNWHKb+HVVTL3xmcaYFadGIT/quzNvgC0lLZj JiWf/jNTmRn6VDaCRi6ajjahLrCPMQiz7mTsYBDTjPv5Wr8i/5oYrYs+QXBKIBsvqY MT8pkNdbw/lVnM4hvHLiCJE8sCE70/1/clKzTRvn6r0H2s+J+ZSi4iTfhZvWD+r+6R ccpQERSnXMD41vYkc2Y5WLpn//7B7YSkdtw/JzE7kHkr6SN+MThp+kx2iJ1zGfc4lB nuHT3phEsdDNEcneitKWh94rPJRts0e+4GgFEyHaXF5ftnL86Sh0lYNDtPAKbPE5K+ 1BDl/U/kivM0/KiFoK/vF+8MPCgORoiwvA/azIgShwwU9OvMqPdid84h3N5gMojtSp GVKm6eEfYHasfGQwNWdFF92s= Received: from localhost (unknown [193.32.127.154]) by wolfsden.cz (Postfix) with ESMTPSA id BB86725B039; Mon, 11 Sep 2023 14:42:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=wolfsden.cz; s=mail; t=1694443379; bh=9UzFheRlBpqwhqYLwxRvcCIW6Z1Q4wMYjKudYbhoL0k=; h=Date:From:To:Cc:Subject:References:In-Reply-To; b=qW4/Mjo471X6YMVfJ3cUHTAk8/ewQ6oCyMfjBcToj1Mv2H4UhilOAq1NCf1lSotaT y2uyBO/bSj67hec6z/vW4AKC/LgD2tNhMoF8f9JY279CfZvnk5fYPiyyCthi5/47rG XoamN6Xl8UofI4jdT3viwzZPvVEfMQmftv1RtzAu656kXEKOK9+SGrLOa/DSrV/s6w Eatt+StL0NiiO5ViFmo7gRhgivSan2kHyg5eJElLwigstsapE2ufuo/KclZFsMIwF3 CuRjyo/kBdHwntelqHWRtVF7fVT7EhiRjk5kRHPsOl71/WeuyVIH6w27GenRJ44Vpp F7L7on1u+T3oJX/LiBM4+ngE/FY4V8Vfv8P+P9HMBwTES5gjMso6V8DOEQeRfudaE1 Ty+skOfuJFR6VSOHARKjYgOjFUcdNeHHAr3KzEahVG+HWANkX2SMNLAhD1JFO/f5oI unnqL/o4rEPogGsXbMGQvU6SWRMBDSv4dFO6EesebJB9n/Vqmyg4WneRi8u9SZhZeR q//58iTfvXhyUgMIwOxztlSg7ZRTSvBEZyodhehLcVNDPGDtsJqTYZK9+WifadKXvs TtZz7d4GUFPSBg11ahInlTs5nZfaXzz/P2HvM5UZ+lu6eW7nddAdiH82I9fM0VujUR tknbqMsHpZgDuE9KxmKPTTpo= Received: from localhost (localhost [local]) by localhost (OpenSMTPD) with ESMTPA id 99c8a95f; Mon, 11 Sep 2023 14:42:59 +0000 (UTC) Date: Mon, 11 Sep 2023 16:42:59 +0200 From: wolf Message-ID: References: <87bkejc7go.fsf@inria.fr> <87fs3tobju.fsf@gnu.org> <87tts9uj6x.fsf@jpoiret.xyz> <87msy0ln4m.fsf@gnu.org> <87pm2vvibo.fsf@jpoiret.xyz> <87pm2s385m.fsf@gnu.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="38x2NUBZrcLd/9Sh" Content-Disposition: inline In-Reply-To: <87pm2s385m.fsf@gnu.org> X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-guix@gnu.org List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-guix-bounces+larch=yhetil.org@gnu.org Sender: bug-guix-bounces+larch=yhetil.org@gnu.org X-Migadu-Flow: FLOW_IN X-Migadu-Country: US X-Migadu-Spam-Score: -6.72 X-Spam-Score: -6.72 X-Migadu-Queue-Id: 39E1456924 X-Migadu-Scanner: mx2.migadu.com X-TUID: m+o5F1aNjTz4 --38x2NUBZrcLd/9Sh Content-Type: multipart/mixed; boundary="ZmJCXHxmcj67AztS" Content-Disposition: inline --ZmJCXHxmcj67AztS Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 2023-09-08 19:08:05 +0200, Ludovic Court=C3=A8s wrote: > Hello! >=20 > Josselin Poiret skribis: >=20 > > Right, although I wouldn't necessarily say that the former doesn't have > > a proper API, but rather that it has a Unix-oriented API. That leads to > > performance issues on e.g. Windows but on Linux I'm not sure there's > > much of a difference. >=20 > [...] >=20 > > We could consider replacing the guile-git dependency with another > > library built directly on top of git-minimal, and have this be a > > dependency of Guix. Not ideal though, and not really scalable either: > > we can't just add every VCS as direct dependencies. >=20 > I cannot imagine a viable implementation of things like =E2=80=98commit-c= losure=E2=80=99 > and =E2=80=98commit-relation=E2=80=99 from (guix git) done by shelling ou= t to =E2=80=98git=E2=80=99. I am sure I must be missing some part of the contract of the function, but = at least the commit-relation seems fairly straightforward: (define (shelling-commit-relation old new) (let ((h-old (oid->string (commit-id old))) (h-new (oid->string (commit-id new)))) (cond ((eq? old new) 'self) ((zero? (git-C %repo "merge-base" "--is-ancestor" h-old h-new= )) 'ancestor) ((zero? (git-C %repo "merge-base" "--is-ancestor" h-new h-old= )) 'descendant) (else 'unrelated)))) I would argue it is even somewhat more readable than the current implementa= tion. > I=E2=80=99m quite confident this would be slow My version is ~2000x faster compared to (guix git): Guix: 1048.620992ms Git: 0.532143ms Again, I am sure I must have miss something, either in the implementation o= r in the measurements, because it is pretty hard to believe there is so much roo= m for improvement. The full script I used is attached to this email. > and brittle. In general git plumbing command are design to have stable CLI interface in = order to be usable in scripting. So I am not sure where the brittleness would co= me =66rom. >=20 > It looks like there=E2=80=99s no option other than carrying the two > implementations. Assuming I made no mistake (hard to believe), it is probably worth explorin= g the feasibility of just shelling out to the git binary some more. >=20 > ~~~ >=20 > Years ago, Andy Wingo sketched a plan for GNU hackers to implement Git > in pure Scheme. That was on April 1st though, so people mistakenly > assumed it was a joke and the project was never carried out. >=20 > I digress, but I wonder: is there not even a viable Haskell or OCaml > implementation of Git? >=20 > Thanks, > Ludo=E2=80=99. > W. --=20 There are only two hard things in Computer Science: cache invalidation, naming things and off-by-one errors. --ZmJCXHxmcj67AztS Content-Type: text/plain; charset=utf-8 Content-Disposition: attachment; filename="test.scm" Content-Transfer-Encoding: quoted-printable #!/bin/sh # -*-scheme-*- exec guile -s "$0" "$@" !# (use-modules (git) (guix git)) (define %repo "/tmp/guix-fork") (define h1 "72745172d155e489936f694d6b9013cb76272370") (define h2 "6d60d7ccba5a8e06c17d55a1772fa7f4529b5eff") (define h3 "c3db650680f995f0556d3ddce567cdc1c33e4603") ;;; r has to still be defined when the commit-relation is called. There is= *no* ;;; error, but it always returns 'unrelated. Quite a footgun. (define r (repository-open %repo)) (define c1 (commit-lookup r (string->oid h1))) (define c2 (commit-lookup r (string->oid h2))) (define c3 (commit-lookup r (string->oid h3))) (define (git-C dir . args) (apply system* "git" "-C" dir args)) (define (shelling-commit-relation old new) (let ((h-old (oid->string (commit-id old))) (h-new (oid->string (commit-id new)))) (cond ((eq? old new) 'self) ;; In real code, git-C should probably return #t (for 0), #f (for= 1) ;; or raise (for anything else). ((zero? (git-C %repo "merge-base" "--is-ancestor" h-old h-new)) 'ancestor) ((zero? (git-C %repo "merge-base" "--is-ancestor" h-new h-old)) 'descendant) (else 'unrelated)))) ;;; Make sure it actually works. (let ((tests `((,c1 . ,c1) (,c1 . ,c2) (,c2 . ,c1) (,c1 . ,c3)))) (for-each (=CE=BB (c) (format #t "Guix: ~a\nGit: ~a\n\n" (commit-relation (car c) (cdr c)) (shelling-commit-relation (car c) (cdr c)))) tests)) (define (time proc) (let* ((start (get-internal-run-time)) (_ (proc)) (end (get-internal-run-time))) (exact->inexact (* 1000 (/ (- end start) internal-time-units-per-second= ))))) (format #t "Guix: ~ams\nGit: ~ams\n" (time (=CE=BB () (commit-relation c1 c2))) (time (=CE=BB () (shelling-commit-relation c1 c2)))) --ZmJCXHxmcj67AztS-- --38x2NUBZrcLd/9Sh Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCgAdFiEEt4NJs4wUfTYpiGikL7/ufbZ/wakFAmT/J20ACgkQL7/ufbZ/ walA7BAAioswpeyaAYJlo/HjxXOUviMoZ49RJ0vjoWkcdKDJyZCvY1bSaa6E+o38 4mjw+8qT2VH3Su+GKTWgYJ66O6PT2IZh7kybzqPCdIFFXAK3KHNP2cQlweDgl6jG YhktsUBWalhzk06rEy3JXNPqrIinGHmMqm/pIxMQXPcOLN5/d90TMB304YqbjAio J5sCeNNYNhVL0A1jY7rZMefUcHISKX8B3XvsNr2A0AvofGv6OQrftf3OMEX4OeE1 5KFeukwv9FRZ38Cc6+Ob3Jw+Atmz5WrOutTPMXAbp4fxxXHQguG9/fIP3JinAtd1 3ruwT7Q4V5n6pGcz81vMYTR+24Tfbcs4thDqKfIM2uoPOvCh1c6dQ3ap2hI4uvls DlCSviISQkjjCqR30jj2ZhHIHF3kPDl+DnaDCn/LIKBRwEbLDJJ+eW9Bv9JJLG2h 6TCouuRrJzCZ7OpkTg6psZI7mhzwYNdJO2wIkGib8eI2U+/GxFDgWTi8U9HHQFiR Z8/97ph5AdoIObDz0R/hezyvpWOJMYuhI0IhKvBksyx8UYOnpM0lIaSASQt2DqU8 xmRztjNazvoUbTASBg9l4MedSejPcDVn6FFFQ+QpkBORXTMYJ5E572BVxOOmnTbu s2K2nZZMczHKbOgWyJt4rafRzSZeJRY6fr062Cu+TrHs9TO0i70= =UIrN -----END PGP SIGNATURE----- --38x2NUBZrcLd/9Sh--