From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1.migadu.com ([2001:41d0:1008:1e59::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms8.migadu.com with LMTPS id 4NMuEwaShWV59AAAkFu2QA (envelope-from ) for ; Fri, 22 Dec 2023 14:41:26 +0100 Received: from aspmx1.migadu.com ([2001:41d0:403:58f0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1.migadu.com with LMTPS id eL6PDAaShWU1aQAA62LTzQ (envelope-from ) for ; Fri, 22 Dec 2023 14:41:26 +0100 X-Envelope-To: larch@yhetil.org Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=inria.fr header.s=dc header.b=eQVivPIr; spf=pass (aspmx1.migadu.com: domain of "bug-guix-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="bug-guix-bounces+larch=yhetil.org@gnu.org"; dmarc=fail reason="SPF not aligned (relaxed)" header.from=inria.fr (policy=none) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1703252486; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:resent-cc: resent-from:resent-sender:resent-message-id:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=wBVVTaXuXtyE3Duavl+Ctp/LxX6fjSWdQ1yW0GtVhmM=; b=JsVvgHJ1eDNsg8Lt52nS0zC2iQfr7BKF9fK3JdNNhXUaumWVqa2CaT/UyzBQ4zZk8JVai/ bMr/IX/rWmUwl4OxjF2+gLDAR0RCFTk5WG6qU3JjdUKn7Yu5dD7Bx9gFe+/EvQUek4TSLs Gg/+Jram0LI+FD8Xr+WOw/sg0/H+8Ellhk5rGdrm09ZE75vbYH+GWEu81tzvewlkQlJuL0 DNUtYMtT0HrnDr0sDn8UbGw7X99wK4bPErsgYVUx3xuiJDmJ+0vQ8b7s8RhU1FEVuAOqbN TonspVkuTJDUF4yqPhsWB7AARZq/2PCWMnW5hJWgXWR1RZZvd9yuH+1bIT3e1w== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1703252486; a=rsa-sha256; cv=none; b=ZHly2ClZ/nSmpbsfP8HgO/P7HXh5m4wO1saQj5JrS9vdxc4kHdDdcpDMGJcTUoj+bOS6x+ ZNcmcoug/DWfcvII6ha0Ul1bbhUr8bTEedJ+B5O2eGfFG4dhaL6gFddNAEbihud/Mrk5bB qWS0UQgby2ozI/wnMIvoFkqTmw6DUzxV8sZGXF6uyxT80GozBcXOjw7ra613yapnObE5B/ BmLTHniuzuyFezRRrdXIu48KH9GdqmsiDE82CZ+KalCJC18Gx+DmRUU3fB8E6slBscL/Ds YBVDku7ytah01HrTFzgyA+Y8Pw0N/JLyf881hnM7MJdSIar430XOak7CseYHxA== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=inria.fr header.s=dc header.b=eQVivPIr; spf=pass (aspmx1.migadu.com: domain of "bug-guix-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="bug-guix-bounces+larch=yhetil.org@gnu.org"; dmarc=fail reason="SPF not aligned (relaxed)" header.from=inria.fr (policy=none) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id AC02D15FE5 for ; Fri, 22 Dec 2023 14:41:25 +0100 (CET) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rGfm0-00069o-Lr; Fri, 22 Dec 2023 08:41:08 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rGflx-00069H-0Y for bug-guix@gnu.org; Fri, 22 Dec 2023 08:41:05 -0500 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rGflp-0002fQ-Bs for bug-guix@gnu.org; Fri, 22 Dec 2023 08:40:57 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1rGflt-0005R9-MP for bug-guix@gnu.org; Fri, 22 Dec 2023 08:41:01 -0500 X-Loop: help-debbugs@gnu.org Subject: bug#39885: Bioconductor tarballs are not archived Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Fri, 22 Dec 2023 13:41:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 39885 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: zimoun Cc: rekado@elephly.net, Timothy Sample , 39885@debbugs.gnu.org, me@tobias.gr Received: via spool by 39885-submit@debbugs.gnu.org id=B39885.170325242020840 (code B ref 39885); Fri, 22 Dec 2023 13:41:01 +0000 Received: (at 39885) by debbugs.gnu.org; 22 Dec 2023 13:40:20 +0000 Received: from localhost ([127.0.0.1]:46218 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rGflD-0005Q4-5p for submit@debbugs.gnu.org; Fri, 22 Dec 2023 08:40:19 -0500 Received: from mail3-relais-sop.national.inria.fr ([192.134.164.104]:27289) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rGfl8-0005Pk-6t for 39885@debbugs.gnu.org; Fri, 22 Dec 2023 08:40:18 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=inria.fr; s=dc; h=from:to:cc:subject:in-reply-to:references:date: message-id:mime-version:content-transfer-encoding; bh=wBVVTaXuXtyE3Duavl+Ctp/LxX6fjSWdQ1yW0GtVhmM=; b=eQVivPIrK61+sL53tl7iV2tfjvxbVsvhfXCw2OIV5kq5w2lft6BdqU/D 78DI/FAnHCeP70wrAcjQNTHz9EcNEfycNk6vXbP6oEwed9fTPOTJPQuI+ +/jSW4BqAGvAOz7mjpD/sB8qcvQt+bIOCMrjX5iCkrAOpZU7v4cMua6uv Y=; X-IronPort-AV: E=Sophos;i="6.04,296,1695679200"; d="scan'208";a="75191123" Received: from unknown (HELO ribbon) ([193.50.110.52]) by mail3-relais-sop.national.inria.fr with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Dec 2023 14:40:02 +0100 From: Ludovic =?UTF-8?Q?Court=C3=A8s?= In-Reply-To: <87lesqmmrr.fsf@gmail.com> (zimoun's message of "Mon, 18 Jul 2022 18:03:04 +0200") References: <87lesqmmrr.fsf@gmail.com> Date: Fri, 22 Dec 2023 14:40:01 +0100 Message-ID: <874jgacq4u.fsf_-_@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-guix@gnu.org List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-guix-bounces+larch=yhetil.org@gnu.org Sender: bug-guix-bounces+larch=yhetil.org@gnu.org X-Migadu-Flow: FLOW_IN X-Migadu-Country: US X-Migadu-Spam-Score: -3.60 X-Spam-Score: -3.60 X-Migadu-Queue-Id: AC02D15FE5 X-Migadu-Scanner: mx11.migadu.com X-TUID: WMPeZ2+2sse7 Hello! zimoun skribis: > Since 2020, I provided several examples of breakage with bug#39885 [1]. > Here another one: > > $ guix time-machine --commit=3D77e2de365497bf4c8b81cbd78624f78293490485 \ > -- build r-biocneighbors -S [...] > Starting download of /gnu/store/zgf7x09kgiqbvj0dmhplxi1xzpljxd7k-BiocNeig= hbors_1.4.1.tar.gz >>>From https://web.archive.org/web/20220718175152/https://bioconductor.org/= packages/release/bioc/src/contrib/BiocNeighbors_1.4.1.tar.gz... > download failed "https://web.archive.org/web/20220718175152/https://bioco= nductor.org/packages/release/bioc/src/contrib/BiocNeighbors_1.4.1.tar.gz" 4= 04 "NOT FOUND" > Trying to use Disarchive to assemble /gnu/store/zgf7x09kgiqbvj0dmhplxi1xz= pljxd7k-BiocNeighbors_1.4.1.tar.gz... > could not find its Disarchive specification > failed to download "/gnu/store/zgf7x09kgiqbvj0dmhplxi1xzpljxd7k-BiocNeigh= bors_1.4.1.tar.gz" from ("https://bioconductor.org/packages/release/bioc/sr= c/contrib/BiocNeighbors_1.4.1.tar.gz" "https://bioconductor.org/packages/3.= 10/bioc/src/contrib/Archive/BiocNeighbors_1.4.1.tar.gz") > builder for `/gnu/store/q9ggmh5a9bzmnr49p10x1w9sv6pzjarv-BiocNeighbors_1.= 4.1.tar.gz.drv' failed to produce output path `/gnu/store/zgf7x09kgiqbvj0dm= hplxi1xzpljxd7k-BiocNeighbors_1.4.1.tar.gz' > build of /gnu/store/q9ggmh5a9bzmnr49p10x1w9sv6pzjarv-BiocNeighbors_1.4.1.= tar.gz.drv failed > View build log at '/var/log/guix/drvs/q9/ggmh5a9bzmnr49p10x1w9sv6pzjarv-B= iocNeighbors_1.4.1.tar.gz.drv.gz'. > guix build: error: build of `/gnu/store/q9ggmh5a9bzmnr49p10x1w9sv6pzjarv-= BiocNeighbors_1.4.1.tar.gz.drv' failed > > Well, several comments: > > 1. Berlin or Bordeaux do not have it as substitutes, > 2. Diasarchive does not have it, > 3. Many others neither. I was wondering whether we=E2=80=99re now doing better for Bioconductor tarballs. The answer, based on small sample, seems to be =E2=80=9Cnot quit= e=E2=80=9D: --8<---------------cut here---------------start------------->8--- $ guix lint -c archival $(guix package -A ^r-bioc | cut -f1) gnu/packages/bioconductor.scm:19708:12: r-biocbaseutils@1.4.0: Disarchive e= ntry refers to non-existent SWH directory '726af85395d163b5a21e52e4df1bf18a= a0072f6b' gnu/packages/bioconductor.scm:19752:12: r-bioccheck@1.38.0: Disarchive entr= y refers to non-existent SWH directory '12cfedcbc27005a3fb7e01c5c4b727e0116= f596f' gnu/packages/bioconductor.scm:16892:5: r-biocfilecache@2.10.1: Disarchive e= ntry refers to non-existent SWH directory '6a2d6d909a7cedd56e96f5a98770deea= aaa8d220' gnu/packages/bioconductor.scm:4540:12: r-biocgenerics@0.48.1: Disarchive en= try refers to non-existent SWH directory '6f19ea14f46dbc75909b77bc08e9023da= ae6fb9e' gnu/packages/bioconductor.scm:19785:5: r-biocgraph@1.64.0: Disarchive entry= refers to non-existent SWH directory '977ff052b4e6c948af7af0fc14ae61f71427= cb1a' gnu/packages/bioconductor.scm:21524:6: r-biocio@1.12.0: Disarchive entry re= fers to non-existent SWH directory '29d8fef9a5b386384f20513c612f1e34f611853= 2' gnu/packages/bioconductor.scm:13090:5: r-biocneighbors@1.20.0: Disarchive e= ntry refers to non-existent SWH directory '6d3728b2dee78cceecdeba0318f3e57b= 6013d96f' gnu/packages/bioconductor.scm:19957:5: r-bioconcotk@1.22.0: Disarchive entr= y refers to non-existent SWH directory '251081d4bc3f061ef8e16338eb042ad4c71= ed02d' gnu/packages/bioconductor.scm:20003:5: r-biocor@1.26.0: Disarchive entry re= fers to non-existent SWH directory '0cc9d3dcde06fb353cdd77f3b538845d16a7772= 0' gnu/packages/bioconductor.scm:6613:12: r-biocparallel@1.36.0: Disarchive en= try refers to non-existent SWH directory '41e09414898f61655bcc99fdd44d69b05= 31c0b2d' gnu/packages/bioconductor.scm:20030:5: r-biocpkgtools@1.20.0: Disarchive en= try refers to non-existent SWH directory '55de8618648ed16797a8effd5b508c652= a5d7cbe' gnu/packages/bioconductor.scm:20144:5: r-biocset@1.16.0: Disarchive entry r= efers to non-existent SWH directory '1cfa6cac0cb453f2882a35c8f5ae6ddfa713ad= 2d' gnu/packages/bioconductor.scm:13276:5: r-biocsingular@1.18.0: Disarchive en= try refers to non-existent SWH directory '992d3f9d48633fa5d46b9a7640a825054= e9538aa' gnu/packages/bioconductor.scm:19806:12: r-biocstyle@2.30.0: Disarchive entr= y refers to non-existent SWH directory 'bb17c3bd9ac7c373b24782fcfecdde5fa2f= 0a965' gnu/packages/bioconductor.scm:22965:5: r-biocthis@1.12.0: Disarchive entry = refers to non-existent SWH directory '3d08f77aae1e81ce9ca9bb9ae2adf4d4c7421= d11' gnu/packages/bioconductor.scm:4521:5: r-biocversion@3.18.1: source not arch= ived on Software Heritage and missing from the Disarchive database gnu/packages/bioconductor.scm:19830:12: r-biocviews@1.70.0: Disarchive entr= y refers to non-existent SWH directory '47e0877ab988469fc09a37505dd769f9626= cac2e' gnu/packages/bioconductor.scm:20182:5: r-biocworkflowtools@1.28.0: Disarchi= ve entry refers to non-existent SWH directory '393f3472cc27f632caea3488aef9= 3a7675b403ef' $ guix describe Generation 285 Dec 17 2023 23:31:56 (current) guix 6ab2426 repository URL: https://git.savannah.gnu.org/git/guix.git branch: master commit: 6ab242609daec00e8bd54f7bff54557c92695724 --8<---------------cut here---------------end--------------->8--- In all cases but one, we=E2=80=99re doing the right thing Disarchive-wise, = but our SWH did not archive them. has entries like: --8<---------------cut here---------------start------------->8--- { "type": "url", "urls": [ "https://bioconductor.org/packages/release/bioc/src/contrib/BiocNei= ghbors_1.20.0.tar.gz", "https://bioconductor.org/packages/3.18/bioc/src/contrib/BiocNeighb= ors_1.20.0.tar.gz", "https://bordeaux.guix.gnu.org/file/BiocNeighbors_1.20.0.tar.gz/sha= 256/0a5wg099fgwjbzd6r3mr4l02rcmjqlkdcz1w97qzwx1mir41fmas", "https://ci.guix.gnu.org/file/BiocNeighbors_1.20.0.tar.gz/sha256/0a= 5wg099fgwjbzd6r3mr4l02rcmjqlkdcz1w97qzwx1mir41fmas", "https://tarballs.nixos.org/sha256/0a5wg099fgwjbzd6r3mr4l02rcmjqlkd= cz1w97qzwx1mir41fmas" ], "integrity": "sha256-WlUXSI41dP7xSTx81ibFsrIsACW5jmzaX5I/lxJ4vCg=3D", "outputHashAlgo": "sha256", "outputHashMode": "flat" }, --8<---------------cut here---------------end--------------->8--- Note that we have at least one copy on our infra: --8<---------------cut here---------------start------------->8--- $ wget -qO- "https://bordeaux.guix.gnu.org/file/BiocNeighbors_1.20.0.tar.gz= /sha256/0a5wg099fgwjbzd6r3mr4l02rcmjqlkdcz1w97qzwx1mir41fmas"|guix hash - = -f base64 WlUXSI41dP7xSTx81ibFsrIsACW5jmzaX5I/lxJ4vCg=3D --8<---------------cut here---------------end--------------->8--- is 404 (but I can see why: for /file, =E2=80=98guix publish=E2=80=99 relies= on things being available in the store and we no longer keep them on ci.guix; we do have a substitute at though; we should fix this). What about hypothesis (2)? This is what we have: --8<---------------cut here---------------start------------->8--- $ wget -qO- https://disarchive.guix.gnu.org/sha256//5a5517488e3574fef1493c7= cd626c5b2b22c0025b98e6cda5f923f971278bc28 |grep swh (swhid "swh:1:dir:6d3728b2dee78cceecdeba0318f3e57b6= 013d96f")) --8<---------------cut here---------------end--------------->8--- I checked with folks on #swh-devel and it turns out that =E2=80=9Cthe legacy nixguix lister that is still used in production did not detect the fallback URL as a tarball URL=E2=80=9D (the bordeaux.guix.gnu.org URL), but= this is fixed in the new lister, which should be in production =E2=80=9Csoon=E2= =80=9D. As for past tarballs, #swh-devel comrades say we could send them a list of URLs and they=E2=80=99d create =E2=80=9CSave Code Now=E2=80=9D requests = on our behalf (we cannot do it ourselves since the site doesn=E2=80=99t accept plain tarballs= .) Any volunteer to write a script that=E2=80=99d generate a list of Bioconduc= tor content-addressed URLs (the bordeaux.guix.gnu.org/file ones) for say the past couple of years? Thanks! Ludo=E2=80=99.