From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <bug-guix-bounces+larch=yhetil.org@gnu.org>
Received: from mp11.migadu.com ([2001:41d0:403:478a::])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits))
	by ms9.migadu.com with LMTPS
	id kM9kHUdm/GQb+AAA9RJhRA:P1
	(envelope-from <bug-guix-bounces+larch=yhetil.org@gnu.org>)
	for <larch@yhetil.org>; Sat, 09 Sep 2023 14:34:15 +0200
Received: from aspmx1.migadu.com ([2001:41d0:403:478a::])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits))
	by mp11.migadu.com with LMTPS
	id kM9kHUdm/GQb+AAA9RJhRA
	(envelope-from <bug-guix-bounces+larch=yhetil.org@gnu.org>)
	for <larch@yhetil.org>; Sat, 09 Sep 2023 14:34:15 +0200
Received: from lists.gnu.org (lists.gnu.org [209.51.188.17])
	(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
	(No client certificate requested)
	by aspmx1.migadu.com (Postfix) with ESMTPS id DA8F846ABC
	for <larch@yhetil.org>; Sat,  9 Sep 2023 14:34:14 +0200 (CEST)
Authentication-Results: aspmx1.migadu.com;
	dkim=fail ("headers rsa verify failed") header.d=gmail.com header.s=20221208 header.b=XYIjXe3Z;
	spf=pass (aspmx1.migadu.com: domain of "bug-guix-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="bug-guix-bounces+larch=yhetil.org@gnu.org";
	dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none)
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org;
	s=key1; t=1694262855;
	h=from:from:sender:sender:reply-to:subject:subject:date:date:
	 message-id:message-id:to:to:cc:cc:mime-version:mime-version:
	 content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:resent-cc:
	 resent-from:resent-sender:resent-message-id:in-reply-to:in-reply-to:
	 references:references:list-id:list-help:list-unsubscribe:
	 list-subscribe:list-post:dkim-signature;
	bh=d6ewT6orRq96ryi4+ofr8KXNoRevq8PABtO1IA0+1mQ=;
	b=TKMIqsVyapE/JsxJhmoLEsH5EBkvV97tY2r60aybItPedzqFp/BojAdBfefX4zjWlVues1
	nLJd7fGUr8UaweCf5tObVA1LhuVdIR3u9PINBCK/m6JGUimH74DuC7NJoMPnqWKiI22Mci
	EP5iOPjIT2aQXQs7mtvXyXPQIku42BS0ivdSv2V+Y4B6/+qfRlC6epn/OfnqAaLHTI9Ek3
	vqR3zWYB7cDNrwsK8BULjbvQgQs1sKjcOWJWGRGIQF7Vn7+3IbTceW2p9Gy5fm5NpWkVb9
	qnATKdCCTondwpFAvulTJCk54Ejui78klhwjlz+gX3y8TstJFrIemzvq4ws5VA==
ARC-Seal: i=1; s=key1; d=yhetil.org; t=1694262855; a=rsa-sha256; cv=none;
	b=elNaFShid3BR3SijchExoWzb6mP3qUQBfdg9NtYsgfxeNRamlCJgtZRggsuOOOVlmdM3d4
	7s8TQd/y0Xe3/9k8LG/fG6QxkadG0S02Af56NU8nDNkq4+ikB7MhOCZJBvd1iQJReuTb01
	DrybeDH/6DSN2YS0UikJeSwndtGL5UN/1cydLXwK5o/sLMZe8JymuJw8rJKCaBDtNhR1zF
	+sH7ZQvwomRhBEKJoNCAsveLNGhw0rzy61jYSG7EHQbTZWTxI9Ad4FBkXSoF8Ly6qIoFBO
	qEzZv/OtJRGSdv0cU4x+PPI12Qd5VUVu/AcYJQfa9r50HJPkjxwWNtxAlX1wXg==
ARC-Authentication-Results: i=1;
	aspmx1.migadu.com;
	dkim=fail ("headers rsa verify failed") header.d=gmail.com header.s=20221208 header.b=XYIjXe3Z;
	spf=pass (aspmx1.migadu.com: domain of "bug-guix-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="bug-guix-bounces+larch=yhetil.org@gnu.org";
	dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none)
Received: from localhost ([::1] helo=lists1p.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.90_1)
	(envelope-from <bug-guix-bounces@gnu.org>)
	id 1qexA2-0003TS-3P; Sat, 09 Sep 2023 08:34:02 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10])
 by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <Debian-debbugs@debbugs.gnu.org>)
 id 1qexA0-0003PO-7Z
 for bug-guix@gnu.org; Sat, 09 Sep 2023 08:34:00 -0400
Received: from debbugs.gnu.org ([2001:470:142:5::43])
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128)
 (Exim 4.90_1) (envelope-from <Debian-debbugs@debbugs.gnu.org>)
 id 1qexA0-0004jw-05
 for bug-guix@gnu.org; Sat, 09 Sep 2023 08:34:00 -0400
Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2)
 (envelope-from <Debian-debbugs@debbugs.gnu.org>) id 1qexA2-0000Tz-Q2
 for bug-guix@gnu.org; Sat, 09 Sep 2023 08:34:02 -0400
X-Loop: help-debbugs@gnu.org
Subject: bug#65720: Guile-Git-managed checkouts grow way too much
Resent-From: Simon Tournier <zimon.toutoune@gmail.com>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces@debbugs.gnu.org>
Resent-CC: bug-guix@gnu.org
Resent-Date: Sat, 09 Sep 2023 12:34:02 +0000
Resent-Message-ID: <handler.65720.B65720.16942627931775@debbugs.gnu.org>
Resent-Sender: help-debbugs@gnu.org
X-GNU-PR-Message: followup 65720
X-GNU-PR-Package: guix
X-GNU-PR-Keywords: 
To: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@gnu.org>
Cc: Josselin Poiret <dev@jpoiret.xyz>, 65720@debbugs.gnu.org
Received: via spool by 65720-submit@debbugs.gnu.org id=B65720.16942627931775
 (code B ref 65720); Sat, 09 Sep 2023 12:34:02 +0000
Received: (at 65720) by debbugs.gnu.org; 9 Sep 2023 12:33:13 +0000
Received: from localhost ([127.0.0.1]:46359 helo=debbugs.gnu.org)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <debbugs-submit-bounces@debbugs.gnu.org>)
 id 1qex9F-0000SZ-9p
 for submit@debbugs.gnu.org; Sat, 09 Sep 2023 08:33:13 -0400
Received: from mail-wr1-x434.google.com ([2a00:1450:4864:20::434]:39422)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <zimon.toutoune@gmail.com>) id 1qex9A-0000S5-2e
 for 65720@debbugs.gnu.org; Sat, 09 Sep 2023 08:33:08 -0400
Received: by mail-wr1-x434.google.com with SMTP id
 ffacd0b85a97d-3175e1bb38cso481221f8f.1
 for <65720@debbugs.gnu.org>; Sat, 09 Sep 2023 05:33:05 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=gmail.com; s=20221208; t=1694262779; x=1694867579; darn=debbugs.gnu.org;
 h=content-transfer-encoding:mime-version:message-id:date:references
 :in-reply-to:subject:cc:to:from:from:to:cc:subject:date:message-id
 :reply-to; bh=d6ewT6orRq96ryi4+ofr8KXNoRevq8PABtO1IA0+1mQ=;
 b=XYIjXe3ZYWc1X3QQqNofjFpv+cs0GRZcYFG+vrEmoZI4S0ZejGdSI7rTQLDhfta8nd
 GuJlldccjO31QGuVVr2d3RcwrXCDXWBzk0vSDAC8udAnKbb1EqnWg3JzX8idhf8IMfeT
 AwWSr5h1X/srL8/EaAcPcn8vBROodeep8WlioHgiZjIUuLr4qydGfxakAC8x8AKZs38Y
 CTQMar/tY9JnRm36GlMz2mfPLcLjRulh+3iAlj3YuDKVU0iBRZWvjlMnYvrvpAGyJIDh
 judu0S7+3+hKWeRe1p09oiMtm31mDx513cyBqXr513KW1JfbNmOMU/HlOqXB4lESd+9h
 BLGA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20230601; t=1694262779; x=1694867579;
 h=content-transfer-encoding:mime-version:message-id:date:references
 :in-reply-to:subject:cc:to:from:x-gm-message-state:from:to:cc
 :subject:date:message-id:reply-to;
 bh=d6ewT6orRq96ryi4+ofr8KXNoRevq8PABtO1IA0+1mQ=;
 b=UvCXaSGn0+f27zOtL9JBqs93VJM6vBA0KfvEu8lNq8Uiaxo+ZQgiLEXYogPJrYHHdo
 ZvtxwfMmOa4Fwj1k0SCyQNNNyjGvzR6T7j7YbrS0emU6ikG8PV7MywWgsJnIuWnZywyM
 4FNNGQdzQxF5T50rlAITbviiZDG163JYwc81zNsCdOZw2B3xwVWRaasjX/DIHEr2nC4g
 WGbKs21O3Yvi6UZLBR9Klj/mqbCqBOkcyXSO4voWd1kXJ7Hw0pG2dqgKT998cPSH+HUb
 +yl+ZtvrLJqF5/Cl/CGri8rkajAwAaYsQg5TL0Eph2LRATFXq7uuv6dKwW/GA40o3LkB
 5DAQ==
X-Gm-Message-State: AOJu0Yy9Rv8GKkSCkD8BLhWNCJGs9WmeZiUZMlNKYFcbdcHewsjJ5c7s
 MC9MAl2mUP9KJ6GxowdF1OeJhmAUlhg=
X-Google-Smtp-Source: AGHT+IHapoioggDK+q/PoFsj4M5AULl2AXhOseWEAOC9iCQL+ZAdc32q2RdfLcHAQ5DvZ0ZWZcEDtQ==
X-Received: by 2002:a5d:620f:0:b0:319:7624:4ca2 with SMTP id
 y15-20020a5d620f000000b0031976244ca2mr4014016wru.0.1694262779211; 
 Sat, 09 Sep 2023 05:32:59 -0700 (PDT)
Received: from lili ([2a01:e0a:59b:9120:65d2:2476:f637:db1e])
 by smtp.gmail.com with ESMTPSA id
 x15-20020a5d60cf000000b00317a04131c5sm4704478wrt.57.2023.09.09.05.32.58
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Sat, 09 Sep 2023 05:32:58 -0700 (PDT)
From: Simon Tournier <zimon.toutoune@gmail.com>
In-Reply-To: <87jzt0382l.fsf@gnu.org>
References: <87bkejc7go.fsf@inria.fr> <87fs3tobju.fsf@gnu.org>
 <87tts9uj6x.fsf@jpoiret.xyz> <87msy0ln4m.fsf@gnu.org>
 <86il8mn7al.fsf@gmail.com> <87jzt0382l.fsf@gnu.org>
Date: Sat, 09 Sep 2023 12:31:48 +0200
Message-ID: <86cyyrskmj.fsf@gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-BeenThere: debbugs-submit@debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
X-BeenThere: bug-guix@gnu.org
List-Id: Bug reports for GNU Guix <bug-guix.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/bug-guix>,
 <mailto:bug-guix-request@gnu.org?subject=unsubscribe>
List-Archive: <https://lists.gnu.org/archive/html/bug-guix>
List-Post: <mailto:bug-guix@gnu.org>
List-Help: <mailto:bug-guix-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/bug-guix>,
 <mailto:bug-guix-request@gnu.org?subject=subscribe>
Errors-To: bug-guix-bounces+larch=yhetil.org@gnu.org
Sender: bug-guix-bounces+larch=yhetil.org@gnu.org
X-Migadu-Country: US
X-Migadu-Flow: FLOW_IN
X-Migadu-Scanner: mx0.migadu.com
X-Migadu-Spam-Score: -0.31
X-Spam-Score: -0.31
X-Migadu-Queue-Id: DA8F846ABC
X-TUID: JB9hDxEb8qZa

Hi,

On Fri, 08 Sep 2023 at 19:09, Ludovic Court=C3=A8s <ludo@gnu.org> wrote:

>>> It would also be pretty bad for closure size:
>>>
>>> --8<---------------cut here---------------start------------->8---
>>> $ guix size guile-git | tail -1
>>> total: 106.6 MiB
>>> $ guix size guile-git git-minimal | tail -1
>>> total: 169.8 MiB
>>> --8<---------------cut here---------------end--------------->8---
>>>
>>> It=E2=80=99s also not clear concretely how we=E2=80=99d add that depend=
ency.  Try
>>> invoking =E2=80=98git=E2=80=99 from $PATH and print a warning if it doe=
sn=E2=80=99t work?
>>> But then, what about applications like Cuirass and hpcguix-web?
>>
>> I think we can rely on something like,
>>
>>     guix shell -C git-minimal -- git gc
>
> We=E2=80=99re talking about the implementation of a cache (meant to speed=
 up
> operations), that would actually fill said cache plus do a whole bunch
> of expensive operations?  Nah.  :-)

I do not think.  If I understand correctly, we need to run =E2=80=9Cgit gc=
=E2=80=9D at
some point, therefore git-minimal needs to me around.  The question is
how and when.

Well, maybe I am missing what the bug is about.  For me, it is about
running =E2=80=98git gc=E2=80=99 for cleaning the Git checkout cache, no?


Solution #1.  Add git-minimal as inputs.  It increases the closure and
the extra load (on average) is about the ratio between the rate of =E2=80=
=9Cguix
pull=E2=80=9D and the rate of the git-minimal changes.

Assuming, that people are running =E2=80=9Cguix pull=E2=80=9D once per week=
 and say =E2=80=9Cgit
gc=E2=80=9D is run after 50 pulls.  (These both number are totally arbitrar=
y and
based on my personal estimate).

Data Service [1] tells:

        2023-07-07 15:45:22 2023-09-08 21:22:08
        2023-05-11 16:10:48 2023-07-07 14:21:45
        2023-05-01 16:40:08 2023-05-11 14:36:16
        2023-04-25 13:34:54 2023-05-01 15:19:55
        2023-04-25 13:34:54 2023-09-08 21:22:08=20=20=20=20=20=20=20=20
        2023-03-06 17:22:28 2023-04-25 12:27:33
        2023-01-17 23:49:19 2023-03-06 16:48:43
        2022-11-08 13:06:42 2023-01-17 15:11:47
        2022-10-08 05:14:46 2022-11-08 09:56:31
        2022-09-06 15:00:08 2022-10-08 04:15:43
        2022-08-13 22:02:31 2022-09-06 12:58:52
        =E2=80=A6

It means that an user will download ~10 times git-minimal for nothing.


Solution #2.  The one I am proposing. :-)  Download git-minimal only
when Guix needs it for running =E2=80=9Cgit gc=E2=80=9D.  Yeah, there is pr=
obably a
small overload with some operations.  But, I bet this overload is much
smaller than the one of solution #1.

Well, it depends on the number of times people are updating the cache vs
the rate of change of git-minimal.

For sure, if one updates 100 times per week the cache, having
git-minimal as inputs is far better.  But I do not think that the
regular usage on average. :-)

That=E2=80=99s why I am proposing to have an option for turning off this =
=E2=80=9Cgit
gc=E2=80=9C operation.

Well, we have lived since years without running =E2=80=98git gc=E2=80=99 so=
 running it
once per year on average is probably enough to keep the cache size
reasonable.  And git-minimal is changing every month.


Maybe, there is some solution #3. ;-)

Cheers,
simon


1: https://data.guix.gnu.org/repository/1/branch/master/package/git-minimal=
/output-history