From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp11.migadu.com ([2001:41d0:403:478a::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms9.migadu.com with LMTPS id kM9kHUdm/GQb+AAA9RJhRA:P1 (envelope-from ) for ; Sat, 09 Sep 2023 14:34:15 +0200 Received: from aspmx1.migadu.com ([2001:41d0:403:478a::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp11.migadu.com with LMTPS id kM9kHUdm/GQb+AAA9RJhRA (envelope-from ) for ; Sat, 09 Sep 2023 14:34:15 +0200 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id DA8F846ABC for ; Sat, 9 Sep 2023 14:34:14 +0200 (CEST) Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=gmail.com header.s=20221208 header.b=XYIjXe3Z; spf=pass (aspmx1.migadu.com: domain of "bug-guix-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="bug-guix-bounces+larch=yhetil.org@gnu.org"; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1694262855; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:resent-cc: resent-from:resent-sender:resent-message-id:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=d6ewT6orRq96ryi4+ofr8KXNoRevq8PABtO1IA0+1mQ=; b=TKMIqsVyapE/JsxJhmoLEsH5EBkvV97tY2r60aybItPedzqFp/BojAdBfefX4zjWlVues1 nLJd7fGUr8UaweCf5tObVA1LhuVdIR3u9PINBCK/m6JGUimH74DuC7NJoMPnqWKiI22Mci EP5iOPjIT2aQXQs7mtvXyXPQIku42BS0ivdSv2V+Y4B6/+qfRlC6epn/OfnqAaLHTI9Ek3 vqR3zWYB7cDNrwsK8BULjbvQgQs1sKjcOWJWGRGIQF7Vn7+3IbTceW2p9Gy5fm5NpWkVb9 qnATKdCCTondwpFAvulTJCk54Ejui78klhwjlz+gX3y8TstJFrIemzvq4ws5VA== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1694262855; a=rsa-sha256; cv=none; b=elNaFShid3BR3SijchExoWzb6mP3qUQBfdg9NtYsgfxeNRamlCJgtZRggsuOOOVlmdM3d4 7s8TQd/y0Xe3/9k8LG/fG6QxkadG0S02Af56NU8nDNkq4+ikB7MhOCZJBvd1iQJReuTb01 DrybeDH/6DSN2YS0UikJeSwndtGL5UN/1cydLXwK5o/sLMZe8JymuJw8rJKCaBDtNhR1zF +sH7ZQvwomRhBEKJoNCAsveLNGhw0rzy61jYSG7EHQbTZWTxI9Ad4FBkXSoF8Ly6qIoFBO qEzZv/OtJRGSdv0cU4x+PPI12Qd5VUVu/AcYJQfa9r50HJPkjxwWNtxAlX1wXg== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=gmail.com header.s=20221208 header.b=XYIjXe3Z; spf=pass (aspmx1.migadu.com: domain of "bug-guix-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="bug-guix-bounces+larch=yhetil.org@gnu.org"; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qexA2-0003TS-3P; Sat, 09 Sep 2023 08:34:02 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qexA0-0003PO-7Z for bug-guix@gnu.org; Sat, 09 Sep 2023 08:34:00 -0400 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qexA0-0004jw-05 for bug-guix@gnu.org; Sat, 09 Sep 2023 08:34:00 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1qexA2-0000Tz-Q2 for bug-guix@gnu.org; Sat, 09 Sep 2023 08:34:02 -0400 X-Loop: help-debbugs@gnu.org Subject: bug#65720: Guile-Git-managed checkouts grow way too much Resent-From: Simon Tournier Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Sat, 09 Sep 2023 12:34:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 65720 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Cc: Josselin Poiret , 65720@debbugs.gnu.org Received: via spool by 65720-submit@debbugs.gnu.org id=B65720.16942627931775 (code B ref 65720); Sat, 09 Sep 2023 12:34:02 +0000 Received: (at 65720) by debbugs.gnu.org; 9 Sep 2023 12:33:13 +0000 Received: from localhost ([127.0.0.1]:46359 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qex9F-0000SZ-9p for submit@debbugs.gnu.org; Sat, 09 Sep 2023 08:33:13 -0400 Received: from mail-wr1-x434.google.com ([2a00:1450:4864:20::434]:39422) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qex9A-0000S5-2e for 65720@debbugs.gnu.org; Sat, 09 Sep 2023 08:33:08 -0400 Received: by mail-wr1-x434.google.com with SMTP id ffacd0b85a97d-3175e1bb38cso481221f8f.1 for <65720@debbugs.gnu.org>; Sat, 09 Sep 2023 05:33:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1694262779; x=1694867579; darn=debbugs.gnu.org; h=content-transfer-encoding:mime-version:message-id:date:references :in-reply-to:subject:cc:to:from:from:to:cc:subject:date:message-id :reply-to; bh=d6ewT6orRq96ryi4+ofr8KXNoRevq8PABtO1IA0+1mQ=; b=XYIjXe3ZYWc1X3QQqNofjFpv+cs0GRZcYFG+vrEmoZI4S0ZejGdSI7rTQLDhfta8nd GuJlldccjO31QGuVVr2d3RcwrXCDXWBzk0vSDAC8udAnKbb1EqnWg3JzX8idhf8IMfeT AwWSr5h1X/srL8/EaAcPcn8vBROodeep8WlioHgiZjIUuLr4qydGfxakAC8x8AKZs38Y CTQMar/tY9JnRm36GlMz2mfPLcLjRulh+3iAlj3YuDKVU0iBRZWvjlMnYvrvpAGyJIDh judu0S7+3+hKWeRe1p09oiMtm31mDx513cyBqXr513KW1JfbNmOMU/HlOqXB4lESd+9h BLGA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694262779; x=1694867579; h=content-transfer-encoding:mime-version:message-id:date:references :in-reply-to:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=d6ewT6orRq96ryi4+ofr8KXNoRevq8PABtO1IA0+1mQ=; b=UvCXaSGn0+f27zOtL9JBqs93VJM6vBA0KfvEu8lNq8Uiaxo+ZQgiLEXYogPJrYHHdo ZvtxwfMmOa4Fwj1k0SCyQNNNyjGvzR6T7j7YbrS0emU6ikG8PV7MywWgsJnIuWnZywyM 4FNNGQdzQxF5T50rlAITbviiZDG163JYwc81zNsCdOZw2B3xwVWRaasjX/DIHEr2nC4g WGbKs21O3Yvi6UZLBR9Klj/mqbCqBOkcyXSO4voWd1kXJ7Hw0pG2dqgKT998cPSH+HUb +yl+ZtvrLJqF5/Cl/CGri8rkajAwAaYsQg5TL0Eph2LRATFXq7uuv6dKwW/GA40o3LkB 5DAQ== X-Gm-Message-State: AOJu0Yy9Rv8GKkSCkD8BLhWNCJGs9WmeZiUZMlNKYFcbdcHewsjJ5c7s MC9MAl2mUP9KJ6GxowdF1OeJhmAUlhg= X-Google-Smtp-Source: AGHT+IHapoioggDK+q/PoFsj4M5AULl2AXhOseWEAOC9iCQL+ZAdc32q2RdfLcHAQ5DvZ0ZWZcEDtQ== X-Received: by 2002:a5d:620f:0:b0:319:7624:4ca2 with SMTP id y15-20020a5d620f000000b0031976244ca2mr4014016wru.0.1694262779211; Sat, 09 Sep 2023 05:32:59 -0700 (PDT) Received: from lili ([2a01:e0a:59b:9120:65d2:2476:f637:db1e]) by smtp.gmail.com with ESMTPSA id x15-20020a5d60cf000000b00317a04131c5sm4704478wrt.57.2023.09.09.05.32.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 09 Sep 2023 05:32:58 -0700 (PDT) From: Simon Tournier In-Reply-To: <87jzt0382l.fsf@gnu.org> References: <87bkejc7go.fsf@inria.fr> <87fs3tobju.fsf@gnu.org> <87tts9uj6x.fsf@jpoiret.xyz> <87msy0ln4m.fsf@gnu.org> <86il8mn7al.fsf@gmail.com> <87jzt0382l.fsf@gnu.org> Date: Sat, 09 Sep 2023 12:31:48 +0200 Message-ID: <86cyyrskmj.fsf@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-guix@gnu.org List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-guix-bounces+larch=yhetil.org@gnu.org Sender: bug-guix-bounces+larch=yhetil.org@gnu.org X-Migadu-Country: US X-Migadu-Flow: FLOW_IN X-Migadu-Scanner: mx0.migadu.com X-Migadu-Spam-Score: -0.31 X-Spam-Score: -0.31 X-Migadu-Queue-Id: DA8F846ABC X-TUID: JB9hDxEb8qZa Hi, On Fri, 08 Sep 2023 at 19:09, Ludovic Court=C3=A8s wrote: >>> It would also be pretty bad for closure size: >>> >>> --8<---------------cut here---------------start------------->8--- >>> $ guix size guile-git | tail -1 >>> total: 106.6 MiB >>> $ guix size guile-git git-minimal | tail -1 >>> total: 169.8 MiB >>> --8<---------------cut here---------------end--------------->8--- >>> >>> It=E2=80=99s also not clear concretely how we=E2=80=99d add that depend= ency. Try >>> invoking =E2=80=98git=E2=80=99 from $PATH and print a warning if it doe= sn=E2=80=99t work? >>> But then, what about applications like Cuirass and hpcguix-web? >> >> I think we can rely on something like, >> >> guix shell -C git-minimal -- git gc > > We=E2=80=99re talking about the implementation of a cache (meant to speed= up > operations), that would actually fill said cache plus do a whole bunch > of expensive operations? Nah. :-) I do not think. If I understand correctly, we need to run =E2=80=9Cgit gc= =E2=80=9D at some point, therefore git-minimal needs to me around. The question is how and when. Well, maybe I am missing what the bug is about. For me, it is about running =E2=80=98git gc=E2=80=99 for cleaning the Git checkout cache, no? Solution #1. Add git-minimal as inputs. It increases the closure and the extra load (on average) is about the ratio between the rate of =E2=80= =9Cguix pull=E2=80=9D and the rate of the git-minimal changes. Assuming, that people are running =E2=80=9Cguix pull=E2=80=9D once per week= and say =E2=80=9Cgit gc=E2=80=9D is run after 50 pulls. (These both number are totally arbitrar= y and based on my personal estimate). Data Service [1] tells: 2023-07-07 15:45:22 2023-09-08 21:22:08 2023-05-11 16:10:48 2023-07-07 14:21:45 2023-05-01 16:40:08 2023-05-11 14:36:16 2023-04-25 13:34:54 2023-05-01 15:19:55 2023-04-25 13:34:54 2023-09-08 21:22:08=20=20=20=20=20=20=20=20 2023-03-06 17:22:28 2023-04-25 12:27:33 2023-01-17 23:49:19 2023-03-06 16:48:43 2022-11-08 13:06:42 2023-01-17 15:11:47 2022-10-08 05:14:46 2022-11-08 09:56:31 2022-09-06 15:00:08 2022-10-08 04:15:43 2022-08-13 22:02:31 2022-09-06 12:58:52 =E2=80=A6 It means that an user will download ~10 times git-minimal for nothing. Solution #2. The one I am proposing. :-) Download git-minimal only when Guix needs it for running =E2=80=9Cgit gc=E2=80=9D. Yeah, there is pr= obably a small overload with some operations. But, I bet this overload is much smaller than the one of solution #1. Well, it depends on the number of times people are updating the cache vs the rate of change of git-minimal. For sure, if one updates 100 times per week the cache, having git-minimal as inputs is far better. But I do not think that the regular usage on average. :-) That=E2=80=99s why I am proposing to have an option for turning off this = =E2=80=9Cgit gc=E2=80=9C operation. Well, we have lived since years without running =E2=80=98git gc=E2=80=99 so= running it once per year on average is probably enough to keep the cache size reasonable. And git-minimal is changing every month. Maybe, there is some solution #3. ;-) Cheers, simon 1: https://data.guix.gnu.org/repository/1/branch/master/package/git-minimal= /output-history