From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <bug-guix-bounces+larch=yhetil.org@gnu.org>
Received: from mp10.migadu.com ([2001:41d0:403:478a::])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits))
	by ms9.migadu.com with LMTPS
	id CO0YEJS9/mQJFwEAG6o9tA:P1
	(envelope-from <bug-guix-bounces+larch=yhetil.org@gnu.org>)
	for <larch@yhetil.org>; Mon, 11 Sep 2023 09:11:16 +0200
Received: from aspmx1.migadu.com ([2001:41d0:403:478a::])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits))
	by mp10.migadu.com with LMTPS
	id CO0YEJS9/mQJFwEAG6o9tA
	(envelope-from <bug-guix-bounces+larch=yhetil.org@gnu.org>)
	for <larch@yhetil.org>; Mon, 11 Sep 2023 09:11:16 +0200
Received: from lists.gnu.org (lists.gnu.org [209.51.188.17])
	(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
	(No client certificate requested)
	by aspmx1.migadu.com (Postfix) with ESMTPS id 1D98B510AB
	for <larch@yhetil.org>; Mon, 11 Sep 2023 09:11:15 +0200 (CEST)
Authentication-Results: aspmx1.migadu.com;
	dkim=fail ("headers rsa verify failed") header.d=riseup.net header.s=squak header.b=p8OnFoG5;
	spf=pass (aspmx1.migadu.com: domain of "bug-guix-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="bug-guix-bounces+larch=yhetil.org@gnu.org";
	dmarc=fail reason="SPF not aligned (relaxed)" header.from=riseup.net (policy=none)
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org;
	s=key1; t=1694416275;
	h=from:from:sender:sender:reply-to:subject:subject:date:date:
	 message-id:message-id:to:to:cc:cc:mime-version:mime-version:
	 content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:resent-cc:
	 resent-from:resent-sender:resent-message-id:in-reply-to:in-reply-to:
	 references:references:list-id:list-help:list-unsubscribe:
	 list-subscribe:list-post:dkim-signature;
	bh=rjpviJRQTENoE7MjykTpMTZNK5FRa3WeLdHj8oDvpUM=;
	b=QGj35INta0ZpLbQyoCEW4zk5DgrB747kZGHH2sELShs2e25GeN0IJ2WhtMtr1qXhiJx9rS
	YdtmnKP5nd9NLBOnE4RIDqYhJaz40GFcSIEVpHquQpV0enNQ/j8C6ZhNV9ZH0RRlLXihlj
	WaK7CFKLiwUGW6thP6hHNJlvb/zHyZIAJY5RYcWTDkdyCq+dXNlEvKtymEzlOaTEYv6q+T
	BuR4BgyRCJRdsVltjlWrWw9Eh8S/uM1PvAtM+Ijf/3R9m7sDRgd8fft21GiPvdFpMvD0PX
	LtNUodz7wdWFXdcx1TdizD3jPvS8G20faUCVZhdQ90gvMP8uT5/Hpi/kc8NAGQ==
ARC-Seal: i=1; s=key1; d=yhetil.org; t=1694416275; a=rsa-sha256; cv=none;
	b=rAZ7CZIfUtz+4vCHcA7duQj9lNTVjaxqTAXy1Z4dNwUZdWs5i5ntyCQbysMyEBh86w+sRE
	/DHGTTsCmZ6wfwiTcJvnRxrJYzOJm0r1SpUf+jiGNDXeh4UNTuwmXA4dX/2J/kdxB2qDyz
	UHmVtRyfvFKUTyHDAlDwCNhJDePvT5nKLqiSIkQlVnKithSuP3bivTFFZjzM+gvMH0YWhJ
	09AtdePfFrijWLX+/ViG3ebWGoDfe/DbkJLXMP1v41n2KiAygsGkIvHiAzV7TH87u3HJIt
	p42tWZxhiaeqz9FK5NW0W5GJwT++8eDNwzJ+XjfgBKKtgZ0pjOvCYRIV69VkjA==
ARC-Authentication-Results: i=1;
	aspmx1.migadu.com;
	dkim=fail ("headers rsa verify failed") header.d=riseup.net header.s=squak header.b=p8OnFoG5;
	spf=pass (aspmx1.migadu.com: domain of "bug-guix-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="bug-guix-bounces+larch=yhetil.org@gnu.org";
	dmarc=fail reason="SPF not aligned (relaxed)" header.from=riseup.net (policy=none)
Received: from localhost ([::1] helo=lists1p.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.90_1)
	(envelope-from <bug-guix-bounces@gnu.org>)
	id 1qfb4Y-0002aL-69; Mon, 11 Sep 2023 03:11:02 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10])
 by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <Debian-debbugs@debbugs.gnu.org>)
 id 1qfb4V-0002Zw-DV
 for bug-guix@gnu.org; Mon, 11 Sep 2023 03:10:59 -0400
Received: from debbugs.gnu.org ([2001:470:142:5::43])
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128)
 (Exim 4.90_1) (envelope-from <Debian-debbugs@debbugs.gnu.org>)
 id 1qfb4U-0000ww-Ti
 for bug-guix@gnu.org; Mon, 11 Sep 2023 03:10:59 -0400
Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2)
 (envelope-from <Debian-debbugs@debbugs.gnu.org>) id 1qfb4Y-0006nn-N0
 for bug-guix@gnu.org; Mon, 11 Sep 2023 03:11:02 -0400
X-Loop: help-debbugs@gnu.org
Subject: bug#65720: Guile-Git-managed checkouts grow way too much
Resent-From: Csepp <raingloom@riseup.net>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces@debbugs.gnu.org>
Resent-CC: bug-guix@gnu.org
Resent-Date: Mon, 11 Sep 2023 07:11:02 +0000
Resent-Message-ID: <handler.65720.B65720.169441620826063@debbugs.gnu.org>
Resent-Sender: help-debbugs@gnu.org
X-GNU-PR-Message: followup 65720
X-GNU-PR-Package: guix
X-GNU-PR-Keywords: 
To: Simon Tournier <zimon.toutoune@gmail.com>
Cc: ludo@gnu.org, 65720@debbugs.gnu.org, dev@jpoiret.xyz
X-Debbugs-Original-Cc: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@gnu.org>,
 65720@debbugs.gnu.org, bug-guix@gnu.org, Josselin Poiret <dev@jpoiret.xyz>
Received: via spool by 65720-submit@debbugs.gnu.org id=B65720.169441620826063
 (code B ref 65720); Mon, 11 Sep 2023 07:11:02 +0000
Received: (at 65720) by debbugs.gnu.org; 11 Sep 2023 07:10:08 +0000
Received: from localhost ([127.0.0.1]:51995 helo=debbugs.gnu.org)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <debbugs-submit-bounces@debbugs.gnu.org>)
 id 1qfb3f-0006mI-S6
 for submit@debbugs.gnu.org; Mon, 11 Sep 2023 03:10:08 -0400
Received: from mx1.riseup.net ([198.252.153.129]:55064)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <raingloom@riseup.net>) id 1qfb3e-0006lg-2o
 for 65720@debbugs.gnu.org; Mon, 11 Sep 2023 03:10:06 -0400
Received: from fews02-sea.riseup.net (fews02-sea-pn.riseup.net [10.0.1.112])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256)
 (No client certificate requested)
 by mx1.riseup.net (Postfix) with ESMTPS id 4RkdBr30skzDqpZ;
 Mon, 11 Sep 2023 07:09:56 +0000 (UTC)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=riseup.net; s=squak;
 t=1694416196; bh=jCBwtrOdu4IjIK1DNoGSobWo/knpWw03ag0O4gDJiso=;
 h=References:From:To:Cc:Subject:Date:In-reply-to:From;
 b=p8OnFoG5FSZ4qVyuqU7eoO1NHJb8FeMLpDQu9pEtagI3uq2ku2qWHNqtfOFkZMiSk
 P9inRFGTA0r8s30MFTbGMkkBqwFCiediO2TsptW5zYwKk0QKIZglisSWlbs+QkSsCG
 CDxEWNaOUY4BssSxgZaE29mtnumAa53HgUXUmdhU=
X-Riseup-User-ID: 558C9CB603439A5DAFC0FDC808E10CF6C3CA907716EB7D430B23A41F042139CA
Received: from [127.0.0.1] (localhost [127.0.0.1])
 by fews02-sea.riseup.net (Postfix) with ESMTPSA id 4RkdBb6bTLzFrwy;
 Mon, 11 Sep 2023 07:09:43 +0000 (UTC)
References: <87bkejc7go.fsf@inria.fr> <87fs3tobju.fsf@gnu.org>
 <87tts9uj6x.fsf@jpoiret.xyz> <87msy0ln4m.fsf@gnu.org>
 <86il8mn7al.fsf@gmail.com> <87jzt0382l.fsf@gnu.org>
 <86cyyrskmj.fsf@gmail.com>
From: Csepp <raingloom@riseup.net>
Date: Mon, 11 Sep 2023 09:06:58 +0200
In-reply-to: <86cyyrskmj.fsf@gmail.com>
Message-ID: <cuca5tt6va2.fsf@riseup.net>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-BeenThere: debbugs-submit@debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
X-BeenThere: bug-guix@gnu.org
List-Id: Bug reports for GNU Guix <bug-guix.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/bug-guix>,
 <mailto:bug-guix-request@gnu.org?subject=unsubscribe>
List-Archive: <https://lists.gnu.org/archive/html/bug-guix>
List-Post: <mailto:bug-guix@gnu.org>
List-Help: <mailto:bug-guix-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/bug-guix>,
 <mailto:bug-guix-request@gnu.org?subject=subscribe>
Errors-To: bug-guix-bounces+larch=yhetil.org@gnu.org
Sender: bug-guix-bounces+larch=yhetil.org@gnu.org
X-Migadu-Country: US
X-Migadu-Flow: FLOW_IN
X-Migadu-Scanner: mx0.migadu.com
X-Migadu-Spam-Score: 2.59
X-Spam-Score: 2.59
X-Migadu-Queue-Id: 1D98B510AB
X-TUID: hYSr9Llolv5N


Simon Tournier <zimon.toutoune@gmail.com> writes:

> Hi,
>
> On Fri, 08 Sep 2023 at 19:09, Ludovic Court=C3=A8s <ludo@gnu.org> wrote:
>
>>>> It would also be pretty bad for closure size:
>>>>
>>>> --8<---------------cut here---------------start------------->8---
>>>> $ guix size guile-git | tail -1
>>>> total: 106.6 MiB
>>>> $ guix size guile-git git-minimal | tail -1
>>>> total: 169.8 MiB
>>>> --8<---------------cut here---------------end--------------->8---
>>>>
>>>> It=E2=80=99s also not clear concretely how we=E2=80=99d add that depen=
dency.  Try
>>>> invoking =E2=80=98git=E2=80=99 from $PATH and print a warning if it do=
esn=E2=80=99t work?
>>>> But then, what about applications like Cuirass and hpcguix-web?
>>>
>>> I think we can rely on something like,
>>>
>>>     guix shell -C git-minimal -- git gc
>>
>> We=E2=80=99re talking about the implementation of a cache (meant to spee=
d up
>> operations), that would actually fill said cache plus do a whole bunch
>> of expensive operations?  Nah.  :-)
>
> I do not think.  If I understand correctly, we need to run =E2=80=9Cgit g=
c=E2=80=9D at
> some point, therefore git-minimal needs to me around.  The question is
> how and when.
>
> Well, maybe I am missing what the bug is about.  For me, it is about
> running =E2=80=98git gc=E2=80=99 for cleaning the Git checkout cache, no?
>
>
> Solution #1.  Add git-minimal as inputs.  It increases the closure and
> the extra load (on average) is about the ratio between the rate of =E2=80=
=9Cguix
> pull=E2=80=9D and the rate of the git-minimal changes.
>
> Assuming, that people are running =E2=80=9Cguix pull=E2=80=9D once per we=
ek and say =E2=80=9Cgit
> gc=E2=80=9D is run after 50 pulls.  (These both number are totally arbitr=
ary and
> based on my personal estimate).
>
> Data Service [1] tells:
>
>         2023-07-07 15:45:22 2023-09-08 21:22:08
>         2023-05-11 16:10:48 2023-07-07 14:21:45
>         2023-05-01 16:40:08 2023-05-11 14:36:16
>         2023-04-25 13:34:54 2023-05-01 15:19:55
>         2023-04-25 13:34:54 2023-09-08 21:22:08=20=20=20=20=20=20=20=20
>         2023-03-06 17:22:28 2023-04-25 12:27:33
>         2023-01-17 23:49:19 2023-03-06 16:48:43
>         2022-11-08 13:06:42 2023-01-17 15:11:47
>         2022-10-08 05:14:46 2022-11-08 09:56:31
>         2022-09-06 15:00:08 2022-10-08 04:15:43
>         2022-08-13 22:02:31 2022-09-06 12:58:52
>         =E2=80=A6
>
> It means that an user will download ~10 times git-minimal for nothing.
>
>
> Solution #2.  The one I am proposing. :-)  Download git-minimal only
> when Guix needs it for running =E2=80=9Cgit gc=E2=80=9D.  Yeah, there is =
probably a
> small overload with some operations.  But, I bet this overload is much
> smaller than the one of solution #1.
>
> Well, it depends on the number of times people are updating the cache vs
> the rate of change of git-minimal.
>
> For sure, if one updates 100 times per week the cache, having
> git-minimal as inputs is far better.  But I do not think that the
> regular usage on average. :-)
>
> That=E2=80=99s why I am proposing to have an option for turning off this =
=E2=80=9Cgit
> gc=E2=80=9C operation.
>
> Well, we have lived since years without running =E2=80=98git gc=E2=80=99 =
so running it
> once per year on average is probably enough to keep the cache size
> reasonable.  And git-minimal is changing every month.
>
>
> Maybe, there is some solution #3. ;-)
>
> Cheers,
> simon
>
>
> 1: https://data.guix.gnu.org/repository/1/branch/master/package/git-minim=
al/output-history

Please don't create another situation like with guix system roll-back,
where a crucial sysadmin operation doesn't work without network access.
Or at least make it configurable, so things that are likely to be needed
for future operations are pre-fetched.