From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp11.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms9.migadu.com with LMTPS id mGtIBLRaEGRRKAAASxT56A (envelope-from ) for ; Tue, 14 Mar 2023 12:29:56 +0100 Received: from aspmx1.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp11.migadu.com with LMTPS id yHtEBLRaEGTqNgAA9RJhRA (envelope-from ) for ; Tue, 14 Mar 2023 12:29:56 +0100 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id ADA4438BE3 for ; Tue, 14 Mar 2023 12:29:55 +0100 (CET) Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=buEuQOuT; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1678793395; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=lTPD4vkyRImB/IoQQRSRGVrbEAATMXh+rPVeE3fZ0rI=; b=CjCqNzxdPXC2bfgfon67QQeMNW2nRDTgCihcq7CpF79hZ/694U82fZsr71w/O8IsSMlyPA omFKB9ENtKSaQ91/LeG5E7bPiwOVKFwaLpoR8R5nvneUdqfaN+P27h5BjoEVakLz2Tyhjp WY19tXuTtzCp5M9MNGJMME0yyTbb3ZrY0v2imjtS/EWrjffW0c1hU/Obop5GJgTByFLKk/ p3R8PNT8gUXRqh75+k6tYCCRbhCAV7w0+KwjDZ6toe+KrfFfEIwIOag+mS+WcHQojwate7 pUZxr8pZSSY1n6wux7aAfXWI+PYz2hsnJHGe9H/J/s0648GeGcOk0/GJmBkzRg== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=buEuQOuT; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" ARC-Seal: i=1; s=key1; d=yhetil.org; t=1678793395; a=rsa-sha256; cv=none; b=jAiqBa2fDL12yiBf/9zwOZ2nZCFPQysFrAycVDLu9Q3Ad7APLCCQ0tl/DvFDVUv0l8gmg8 OepT+HKu4A3iXWxXhaIRndqfKmmI5k2dKFhPUaHNLhFVMvnq8TxiUEaeQ1pEFXVgb705uR CUbJrK+6w0X55TrbIdCdtfSj+BtrDlZBVX0lvIeB2uroHTYCBeJUkSe37Maw92atju7TK0 fGd362CMH2h2dZx6Rq13LeeY3Q9fiTy31azG0etgWmeJl+ygUDJH1QPVfaUrvd/VT1mSpw jovXkO2FffE4OkTpT9jKK1GuIVgZa4h3Dgiw+UtCkTgeTR2imuwybK4phe2MXA== Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pc2qN-0005R6-Ng; Tue, 14 Mar 2023 07:29:28 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pc2qK-0005Qb-GM for guix-devel@gnu.org; Tue, 14 Mar 2023 07:29:24 -0400 Received: from mail-wm1-x32b.google.com ([2a00:1450:4864:20::32b]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pc2qI-0001QO-Gs for guix-devel@gnu.org; Tue, 14 Mar 2023 07:29:24 -0400 Received: by mail-wm1-x32b.google.com with SMTP id k25-20020a7bc419000000b003ed23114fa7so4210647wmi.4 for ; Tue, 14 Mar 2023 04:29:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1678793360; h=content-transfer-encoding:mime-version:message-id:date:in-reply-to :subject:to:from:from:to:cc:subject:date:message-id:reply-to; bh=lTPD4vkyRImB/IoQQRSRGVrbEAATMXh+rPVeE3fZ0rI=; b=buEuQOuTCy23a+WBgHzLtSTkVeBxW2qKZ91hZ1V3/t1OJ+/YU+bt4bwByhqSoJzJSe BF5CLTDEvwOg/gTRMgK2nnuhlTVSWa3cUsUDgil6MmeXLQ7uSmDzRZFaPxxUQ8kEnj5g dZQGk0c+Huf1NnlNatasxR6fEnO5/gzqV3QMi7MNYnUIn6l/YBqI9zvIRxekxH/Vb3tl chfNTaWWled3a8nj4oDFq1TwMBiwA9dMZ5oE3Tbm4sSaClZg446+lUzD+utt/liM3AHf 3jJCB1u4TCEcrgCQN0LiK5PLXz47iRgRHDbW+2E3BZSviVcC5cDD7qsHJ7g5rvexILFT l7Ew== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1678793360; h=content-transfer-encoding:mime-version:message-id:date:in-reply-to :subject:to:from:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=lTPD4vkyRImB/IoQQRSRGVrbEAATMXh+rPVeE3fZ0rI=; b=3qucfjkJRZ6RZXHSz4TOWFRJeQJzT4mqNEda28e1xsmPpzYDD6SNP9yZCHRDT2QaJM LIirTq/FAwdLDnu3C52vLH/0SWMK0I2Nr+ZMWKzk6WJsfiVJPVhcZu+GR0l6y2Pbhgh9 1sXOOOtO2y1CCcIEUGI3uRpNi0wyywzwo50fQT4IjlkjhcE+15ZIN4aRNriyY89dQLjh Hz0kxivQXxbOaeEwVbPcjDcuZNbqB5O27ALt8apBsObfovfTZlej1XR+pT9mPiHxSU2n l1RD5Qf+kEXj+1DCXnS+otwyXHI1G+MdBjqEdmg1lmhTPv3LD3Rr5JjtQKmIIXx4oAN3 J6Uw== X-Gm-Message-State: AO0yUKVy5vMzJvwuyVyz/fsEX6cNVMg1SyWUNDnr6nGzNATdu+Z5osTz wKNSvGXFkRlBwWplQa9meIGlnDV/oNo= X-Google-Smtp-Source: AK7set9rlSsGS/zh/KamJnU7PZtJH+EJxp9/IDHAmSStS2xMmV/bARaOzGCnog0LvN9B0WgmloLeeQ== X-Received: by 2002:a05:600c:1c89:b0:3dc:5b88:e706 with SMTP id k9-20020a05600c1c8900b003dc5b88e706mr13419374wms.1.1678793360503; Tue, 14 Mar 2023 04:29:20 -0700 (PDT) Received: from lili ([2a01:e0a:59b:9120:65d2:2476:f637:db1e]) by smtp.gmail.com with ESMTPSA id z16-20020a7bc7d0000000b003eb5ce1b734sm2550585wmk.7.2023.03.14.04.29.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 14 Mar 2023 04:29:20 -0700 (PDT) From: Simon Tournier To: Timothy Sample , guix-devel@gnu.org Subject: Re: Preservation of Guix (PoG) report 2023-03-13 In-Reply-To: <87r0tsm7u4.fsf@ngyro.com> Date: Tue, 14 Mar 2023 11:36:48 +0100 Message-ID: <86356739hb.fsf@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Received-SPF: pass client-ip=2a00:1450:4864:20::32b; envelope-from=zimon.toutoune@gmail.com; helo=mail-wm1-x32b.google.com X-Spam_score_int: -3 X-Spam_score: -0.4 X-Spam_bar: / X-Spam_report: (-0.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URI_DOTEDU=1.678 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: X-Migadu-Queue-Id: ADA4438BE3 X-Spam-Score: -11.96 X-Migadu-Spam-Score: -11.96 X-Migadu-Scanner: scn0.migadu.com List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: guix-devel-bounces+larch=yhetil.org@gnu.org X-Migadu-Country: US X-Migadu-Flow: FLOW_IN X-TUID: 8q/qmw0O062K Hi, On Mon, 13 Mar 2023 at 19:37, Timothy Sample wrote: > Note that you can link to the most recent version of the report using > . Awesome! \o/ Well, I do not remember if you consider also the =E2=80=99origin=E2=80=99 (fixed-outputs) as =E2=80=99inputs=E2=80=99 or =E2=80=99patches=E2=80=99. = Do you? Basically, =E2=80=99package-direct-sources=E2=80=99 from (guix packages). For instance, see the package =E2=80=99ntp=E2=80=99, --8<---------------cut here---------------start------------->8--- (source (origin (method url-fetch) (uri (list (string-append "https://www.eecis.udel.edu/~ntp/ntp_spool/ntp4/ntp-" (version-major+minor version) [...] (sha256 (base32 "06cwhimm71safmwvp6nhxp6hvxsg62whnbgbgiflsqb8mgg40n7n")) ;; Add an upstream patch to fix build with GCC 10. Taken from ;; . (patches (list (origin (method url-fetch) (uri "https://bugs.ntp.org/attachment.cgi?id=3D1760\ &action=3Ddiff&context=3Dpatch&collapsed=3D&headers=3D1&format=3Draw") (file-name "ntp-gcc-compat.patch") (sha256 (base32 "13d28sg45rflc7kqiv30asrhna8n69wlpwx16l65rravgpvp= 90h2"))) --8<---------------cut here---------------end--------------->8--- or see the package =E2=80=99tensorflow=E2=80=99, --8<---------------cut here---------------start------------->8--- (native-inputs `(("pkg-config" ,pkg-config) [...] ("boringssl-src" ,(let ((commit "ee7aa02") (revision "1")) (origin (method git-fetch) (uri (git-reference (url "https://boringssl.googlesource.com/boringssl") (commit commit))) (file-name (string-append "boringssl-0-" revision (string-take commit 7) "-checkout")) (sha256 (base32 "1jf693q0nw0adsic6cgmbdx6g7wr4rj4vxa8j1hpn792fqhd8wgw"))))) --8<---------------cut here---------------end--------------->8--- > Over the whole set, 77.1% are known to be safely tucked away in the > Software Heritage archive. But it=E2=80=99s actually much better than th= at. If > we only look at the most recent sampled commit (from Sunday the 5th), > that number becomes 87.4%, which is starting to look pretty good! Just to be point the new nixguix loader [1] is still in SWH staging and not yet deployed, IIRC. It will not change much the coverage on our side but it should be fix some corner-cases. 1: > This is kinda like an automated version of Simon=E2=80=99s recent > investigation. Neat! Note that I also wanted to check the SWH capacity for cooking, not only checking the end points. For instance, it allowed to discover mismatch due to uncovered CR/LF normalization; now fixed with: 58f20fa8181bdcd4269671e1d3cef1268947af3a. > Here=E2=80=99s a rough road map for that based on a glance at the script= =E2=80=99s > output: > > =E2=80=A2 Subversion support (for TeX-based documentation stuff, I gu= ess) For the interested reader, details for helping in the implementation: https://issues.guix.gnu.org/issue/43442#9 https://issues.guix.gnu.org/issue/43442#11 However, it would ease all the dance if SWH would consider to store and expose NAR hashes on their side. As discussed here: https://gitlab.softwareheritage.org/swh/meta/-/issues/4538 > However, 42% of them are old Bioconductor packages. They > seem to be lost. It looks like Bioconductor now stores multiple package > versions per Bioconductor version [2], but before version 3.15 that was > not the case. As an example, take =E2=80=9Cggcyto=E2=80=9D from Biocondu= ctor 3.10 [3]. > We packaged version 1.14.0, and then at some point Bioconductor 3.10 > switched to version 1.14.1. We packaged that, too, but now 1.14.0 is > gone. Well, I have not investigated much because it is between December 2019 and March 2020 thus =E2=80=9Cguix time-machine=E2=80=9D is not smooth for t= his old time. First question, does we have the source tarball in Berlin or Bordeaux or somewhere else? If yes, there is a hope. :-) Else, it is probably gone forever. The hope is: https://git.bioconductor.org/packages/ggcyto If we have the tarball with the correct checksum from commit f5f440312d848e12463f0c6f7510a86b623a9e27 --8<---------------cut here---------------start------------->8--- + (version "1.14.0") + (source + (origin + (method url-fetch) + (uri (bioconductor-uri "ggcyto" version)) + (sha256 + (base32 + "165qszvy5z176h1l3dnjb5dcm279b6bjl5n5gzz8wfn4xpn8anc8")))) --8<---------------cut here---------------end--------------->8--- then we can disassemble it and then using the Git repository, we can try to assemble the content from SWH and the meta from Disarchive DB. For sure, it is again another example why we should augment by intrinsic identifiers the Guix way for fetching. See: https://lists.gnu.org/archive/html/guix-devel/2023-03/msg00025.html > I know it=E2=80=99s been discussed before, but I can=E2=80=99t rem= ember what the > conclusion was. Are these just gone forever? Discussed here: https://issues.guix.gnu.org/issue/39885 https://issues.guix.gnu.org/issue/54787=20=20=20=20 Cheers, simon