From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id aGsNFTv0m2HrKQEAgWs5BA (envelope-from ) for ; Mon, 22 Nov 2021 20:49:15 +0100 Received: from aspmx1.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id 0ACwEDv0m2F7IwAAB5/wlQ (envelope-from ) for ; Mon, 22 Nov 2021 19:49:15 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 836DFB801 for ; Mon, 22 Nov 2021 20:49:14 +0100 (CET) Received: from localhost ([::1]:53760 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mpFJR-0006mg-Ln for larch@yhetil.org; Mon, 22 Nov 2021 14:49:13 -0500 Received: from eggs.gnu.org ([209.51.188.92]:41924) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mpFJG-0006mW-K1 for bug-guix@gnu.org; Mon, 22 Nov 2021 14:49:02 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:37630) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mpFJG-00010w-Bl for bug-guix@gnu.org; Mon, 22 Nov 2021 14:49:02 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1mpFJG-0000Qd-Ap for bug-guix@gnu.org; Mon, 22 Nov 2021 14:49:02 -0500 X-Loop: help-debbugs@gnu.org Subject: bug#39885: Bioconductor URI, fallback and time-machine Resent-From: zimoun Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Mon, 22 Nov 2021 19:49:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 39885 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: 39885@debbugs.gnu.org Received: via spool by 39885-submit@debbugs.gnu.org id=B39885.16376104961587 (code B ref 39885); Mon, 22 Nov 2021 19:49:02 +0000 Received: (at 39885) by debbugs.gnu.org; 22 Nov 2021 19:48:16 +0000 Received: from localhost ([127.0.0.1]:49176 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mpFIW-0000PX-Cj for submit@debbugs.gnu.org; Mon, 22 Nov 2021 14:48:16 -0500 Received: from mail-wm1-f51.google.com ([209.85.128.51]:44696) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mpFIU-0000PL-Rl for 39885@debbugs.gnu.org; Mon, 22 Nov 2021 14:48:15 -0500 Received: by mail-wm1-f51.google.com with SMTP id p27-20020a05600c1d9b00b0033bf8532855so147272wms.3 for <39885@debbugs.gnu.org>; Mon, 22 Nov 2021 11:48:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version:content-transfer-encoding; bh=PbO9DjGigRt7Ug5DpsnQOOvGiVoz4QPB+Hqma00FBTE=; b=mmSWROrr6Rw5X64tuqLCmLsi4cbLxcKljzwAZgHGHHLE3EpRXJLvy7Q90dJHUpXsrp AB41f8c//9Or+ncFqFX0YsCyKhz+DWi2YqZbQ9uIjIBDpsUbGFIcKEyg+mWl8uSe8zAX /jK+ARjSxVTlbz3l0XIPYnc4Ajvd8eH7foKaYYTLuR9j1ZZX1y2NpElWuZmMnBmdfRtx 3D0za6TZ2sCYac3hp+U3EmYz72YBjDHn7YMKrEoSsc0qEBAu4HBv/LRVro0PBnuQIuGi v2EGKwyTLS68GtdZK9J3aUgHJTlqA6ShLqZ+cHvFYIV1jDPyYPiYju/mXtSztaw/UjTF cudg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version:content-transfer-encoding; bh=PbO9DjGigRt7Ug5DpsnQOOvGiVoz4QPB+Hqma00FBTE=; b=a3qJf8nF611b8YW6i28KEsV57qk3NsqiVLEmjbhAb5ZIo9y4tUsq6iKJK5fzTfC4jp YSqVBsUSmBD7EJj1GGg1cQ9pgvGv/61OX4MpM42pmd6Np7aMeNJqXhzfgfcO/4zXtDct xR9JhOK0YdCbA5sGqohDm6CCbkmFmHCfIFDJLTEWqPln9JOjEoL9RWXk0ZTPocjx1FRN sPLBuEvsNG5L/70Vt63ifJvDvaaJFAIVJ4BlseJJP5UGgpqSw+Zh7+LSrdD/NtcJ7iB4 jh43zloPw7T6PZgFrWkjwuaS0FqGkuRg6kQCknnD5Y3rRckBmeeNfDt49XyoYnbofq9e RCAg== X-Gm-Message-State: AOAM532vdE4WKojUNZ2r+1UHQSW+rGrl6MORjrQUMg4bbtF+L+yWLqPU aoP6zuRMcSbqMg5ZwLiijRo= X-Google-Smtp-Source: ABdhPJxepj7pREqfThzqYFgK6k+ZjuCRRmlrZ+WI/5VGM+Aa3OBo1+Qak57SWpOqheGfDZRLXL7o7Q== X-Received: by 2002:a05:600c:1e8d:: with SMTP id be13mr32698767wmb.79.1637610488869; Mon, 22 Nov 2021 11:48:08 -0800 (PST) Received: from pfiuh02 ([193.48.40.241]) by smtp.gmail.com with ESMTPSA id h204sm10716320wmh.33.2021.11.22.11.48.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 22 Nov 2021 11:48:08 -0800 (PST) From: zimoun References: Date: Mon, 22 Nov 2021 20:48:05 +0100 In-Reply-To: (zimoun's message of "Tue, 3 Mar 2020 16:59:03 +0100") Message-ID: <87wnl0q8ei.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-guix@gnu.org List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: rekado@elephly.net Errors-To: bug-guix-bounces+larch=yhetil.org@gnu.org Sender: "bug-Guix" X-Migadu-Flow: FLOW_IN X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1637610554; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:resent-cc: resent-from:resent-sender:resent-message-id:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=PbO9DjGigRt7Ug5DpsnQOOvGiVoz4QPB+Hqma00FBTE=; b=CeIuFnAfMAg/sI0p9Kp/zfTpVhXTGgteKunIWVF5V2CnSQlmR2T80xwLRIo1qgSdkVu4ec Z2NIaoGJknI0icikiGIJ7Zc7E1DA/X1G/12QEOLKIUV5Rg0203LmIOIwnMS1WG+tiLxSAr RVAfmqazQ1hyyJJ/xIYBoui9TVJcUUGCvxLrZyiM6uavPx34bcYS7B4ItwU8WaJ383qujI tjAnAV98QAQDaShuPZQFMG4pLw+IGbNdq1iOepWlqRJeBPqJs/N5dT8MXoeeqYULAQLBin 5IavGtZMuBfKxMkGrmm9KtkWSZjfcy8plWiwJb7LyJQP8hiMBg8j0bTMu41Muw== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1637610554; a=rsa-sha256; cv=none; b=QJowDWCAdUFi5jxUy5tN/FQVY/ACVGjvNK7FmYyTOkeohSKVIdi4RgmnGVY5InCCmrP/Hc RvbO/kfsOZgZg0OFAVPoM2IJ8Kn0RCbQbQ7wTsGhuYx74ZMXJ6cmL4CU1vkaUz+U24Mn8U q9d4SMzgWdqL68rkObQYE//P+s/Eyu3O5NjY7VAunZ+x1zRLyWvvwoHCtrr+b/QqTyoZy6 kmglgzO4IY3MctDfQ/LIFE2vjqx2EH9QpvY3W6RrRJP2aQ8znF+vso46BPaJ+5vv10W25S BKXJ90lG0ivF/04l2iojMveSuZw3blNXs8qz8W+yb4sfKuXWqI7QKgri8C+Euw== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=gmail.com header.s=20210112 header.b=mmSWROrr; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none); spf=pass (aspmx1.migadu.com: domain of "bug-guix-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="bug-guix-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -1.78 Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=gmail.com header.s=20210112 header.b=mmSWROrr; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none); spf=pass (aspmx1.migadu.com: domain of "bug-guix-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="bug-guix-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: 836DFB801 X-Spam-Score: -1.78 X-Migadu-Scanner: scn0.migadu.com X-TUID: taZA+FwXONO5 Hi, On Tue, 03 Mar 2020 at 16:59, zimoun wrote: > Currently, the URI scheme (see 'bioconductor-uri' in > guix/build-system/r.scm) is: > > https://bioconductor.org/packages/release/data//src/contr= ib/-.tar.gz > > which leads to 2 issues: > > 1. when Bioconductor updates their release, some package versions are > updated too, and so, the upstream return 404. > 2. for this reason 1., the "guix time-machine" is broken for all the > Bioconductor packages, at least if Berlin or SWH does not have a > substitute; which is not expected for 'annotation' packages. > > However, the Bioconductor archive still serves the old release, i.e., > > https://bioconductor.org/packages/3.x/data//src/contrib/-.tar.gz It is still the case and for concrete breakage, see [1]. I will not detail but each time Guix lags behind Bioconductor new release, it is broken. For sure, Guix upgrades more or less quickly. Each time Bioconductor remove a package, it is broken. Well, because a lot of care about R packages, the forward breakages happen barely. :-) But backward breakages are not negligible, IMHO. Well, this URL choice is not The Right Thing and somehow broken by design. 1: > The ways to fix the both issues are: > > a) Add the Bioconductor release (known at packaging time) to all the > packages; provide as argument to 'bioconductor-uri'. > b) Add more URLs to fallback. > > As discussed on IRC, Tobias seems more inclined with the option a) and > I am more in favour of option b. > > Attached, a quick patch showing the option b). We are now 1.5 years after. And we did nothing; well we did other things instead. ;-). Now, I have an strong opinion that option a) is not doable: I speak using my janitor moves of Bioconductor packages. Instead, something along the proposed patch below half-fixes the issue now. We just have to append the releases and let the fallback mechanism takes care. It reduces the maintenance burden, IMHO. For sure, it is not perfect but it appears to me a pragmatical fix waiting something better. This better is unknown (at least from me :-)). On one hand Disarchive would improve the situation for tarballs=E2=80=A6 but some work remains (ch= eck that SWH ingestion and rebuild is bullet-proof). On the other hand, Bioconductor uses Git, for instance: git clone https://git.bioconductor.org/packages/CATALYST And Bioconductor uses =E2=80=99origin/RELEASE_3.14=E2=80=99 as Git tag. Ba= sed on this, it would avoid the eternal inplace-change fixes. For instance, the package tximeta [2], recently updated by Ricardo. Well, from their Bioconductor Git repo, git clone https://git.bioconductor.org/packages/tximeta it is not clear that the current version is at 1.12.3. And it is not clear either if they tagged origin/RELEASE_3_14 at 1.12.0 and did something ugly to then get 1.12.3. Anyway, switch from url-fetch to git-fetch is an option. However, it is as option a) and I am not convinced it is doable with the resource at hand. 2: What could a plan to have a bullet-proof =E2=80=9Cguix time-machine=E2=80= =9D for Bioconductor? Cheers, simon > From 87e73e02202fe5e342d68f1fb17efdd4425737cd Mon Sep 17 00:00:00 2001 > From: zimoun > Date: Tue, 3 Mar 2020 16:53:39 +0100 > Subject: [PATCH] build-system: r: Use Bioconductor old releases to fallba= ck. > > * guix/build-system/r.scm (bioconductor-uri): Extend the fallback list. > --- > guix/build-system/r.scm | 21 ++++++++++++--------- > 1 file changed, 12 insertions(+), 9 deletions(-) > > diff --git a/guix/build-system/r.scm b/guix/build-system/r.scm > index 2d328764b0..8638e1b888 100644 > --- a/guix/build-system/r.scm > +++ b/guix/build-system/r.scm > @@ -54,15 +54,18 @@ release corresponding to NAME and VERSION." > ('annotation "/data/annotation") > ('experiment "/data/experiment") > (_ "/bioc")))) > - (list (string-append "https://bioconductor.org/packages/release" > - type-url-part > - "/src/contrib/" > - name "_" version ".tar.gz") > - ;; TODO: use %bioconductor-version from (guix import cran) > - (string-append "https://bioconductor.org/packages/3.10" > - type-url-part > - "/src/contrib/Archive/" > - name "_" version ".tar.gz")))) > + (append (list (string-append "https://bioconductor.org/packages/rele= ase" > + type-url-part > + "/src/contrib/" > + name "_" version ".tar.gz")) > + (map (lambda (release) > + (string-append "https://bioconductor.org/packages/" > + release > + type-url-part > + "/src/contrib/" > + name "_" version ".tar.gz")) > + (list (@@ (guix import cran) %bioconductor-version) > + "3.9" "3.8" "3.7"))))) > > (define %r-build-system-modules > ;; Build-side modules imported by default.