From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id IF+8NwBWRmERTQEAgWs5BA (envelope-from ) for ; Sat, 18 Sep 2021 23:11:28 +0200 Received: from aspmx1.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id 0KkqMwBWRmFKeQAAB5/wlQ (envelope-from ) for ; Sat, 18 Sep 2021 21:11:28 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id E26A1F13D for ; Sat, 18 Sep 2021 23:11:27 +0200 (CEST) Received: from localhost ([::1]:36936 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mRhcL-00068k-FV for larch@yhetil.org; Sat, 18 Sep 2021 17:11:25 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:46368) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mRhby-000653-AF for bug-guix@gnu.org; Sat, 18 Sep 2021 17:11:02 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:53770) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mRhby-0005R2-2m for bug-guix@gnu.org; Sat, 18 Sep 2021 17:11:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1mRhbx-0003lw-Ul for bug-guix@gnu.org; Sat, 18 Sep 2021 17:11:01 -0400 Subject: bug#44187: Channel clones lack SWH fallback Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-To: bug-guix@gnu.org Resent-Date: Sat, 18 Sep 2021 21:11:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: cc-closed 44187 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: zimoun Mail-Followup-To: 44187@debbugs.gnu.org, ludo@gnu.org, zimon.toutoune@gmail.com Received: via spool by 44187-done@debbugs.gnu.org id=D44187.163199945114480 (code D ref 44187); Sat, 18 Sep 2021 21:11:01 +0000 Received: (at 44187-done) by debbugs.gnu.org; 18 Sep 2021 21:10:51 +0000 Received: from localhost ([127.0.0.1]:37082 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mRhbm-0003lT-NN for submit@debbugs.gnu.org; Sat, 18 Sep 2021 17:10:51 -0400 Received: from eggs.gnu.org ([209.51.188.92]:57516) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mRhbj-0003lF-U7 for 44187-done@debbugs.gnu.org; Sat, 18 Sep 2021 17:10:49 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:37136) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mRhbe-0005AK-Mn; Sat, 18 Sep 2021 17:10:42 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=35836 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mRhbC-0003R2-PU; Sat, 18 Sep 2021 17:10:40 -0400 From: Ludovic =?UTF-8?Q?Court=C3=A8s?= References: <87pn0dk61v.fsf@gnu.org> <20210910143415.14783-1-ludo@gnu.org> <86o88r1vfe.fsf@gmail.com> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: Jour du =?UTF-8?Q?G=C3=A9nie?= de =?UTF-8?Q?l'Ann=C3=A9e?= 229 de la =?UTF-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Sat, 18 Sep 2021 23:10:12 +0200 In-Reply-To: <86o88r1vfe.fsf@gmail.com> (zimoun's message of "Fri, 17 Sep 2021 10:02:45 +0200") Message-ID: <87pmt5si8b.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-guix@gnu.org List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: 44187-done@debbugs.gnu.org Errors-To: bug-guix-bounces+larch=yhetil.org@gnu.org Sender: "bug-Guix" X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1631999488; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:resent-to: resent-from:resent-sender:resent-message-id:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post; bh=ioVs1IT5UP7m6QEwYldnTPxKjV/5tgmKvEeNIJVc3yk=; b=WBWRz5ek1SZax7paGpTA7Ft50CvLNbKHy3sLYoQgPQDUTMXT/5ipLR9Lfrm5Uv2R3yTQKW i0joiV6SMKsatAsbM231+THbzUbd+YpyeufwJvMQRWimiou1rjy+3gk0tHtD5vRycaowhV jjaYC2KO8/IvTo3iOByPMRGpWf0+EVgmxnhijZqtON0w3V/HTIb1AU/VL6GP5/fUa/QwrC wMk/XCqxyGo2SARdjsWUdnN2hvnIaz64X8PxKl/XOOhLY2iCLJOnKJikTqjde71wS+/07p r1X1ektix4qzwlkzthimEwnPAsWJZ+b/0qqc7jBBnAX6RzkMCVGQairZV8XIYA== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1631999488; a=rsa-sha256; cv=none; b=efTteNVKq12d7/cK/09utt44wyeOnIqP1svamNOVunCu4rKi2HWMUpnxtyt8mAhQcsBl9C yD0EFPBEsQTerG7YZYBfw0yT0sULm+IFsqk/tK09w/JLgL8HmTMc/joU2XoiDfpPxmXLrp WXremBJnO0TwjTvSyYO3itelbBOdZ5LPxNPHRRQgqjItIGoDEmC1QTsaZq+mxrWrSFcVor 84fOw6Emz0Q0fPjH+UPFis647ZLdOvIZ0/Bv2D5SqWHq2w87YHZxLf7UZux4KWns2ce/C/ Iid2xkkCzIVU1W+3gWeFZIHNYRaWD2GNOvQU2aYiDPkfpGUCWoNaNoy3LIcqMQ== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=none; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of bug-guix-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=bug-guix-bounces@gnu.org X-Migadu-Spam-Score: -1.39 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of bug-guix-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=bug-guix-bounces@gnu.org X-Migadu-Queue-Id: E26A1F13D X-Spam-Score: -1.39 X-Migadu-Scanner: scn0.migadu.com X-TUID: N/booQYSeW16 Hello! zimoun skribis: > The original URL of the channel was: > . And this channel > defines a package where the upstream has also disappeared > . Note the URL in the > package definition is not bogus=E2=80=A6 but using one was already workin= g. :-) > > All is saved on SWH, so now all is transparent! From my point of view, > this is a killer feature for scientific folks. :-) Yay! Great that you came up with a nice example to test it on! >> First, fallback is implemented only for fresh clones, not for updates. >> Thus, if I rerun the first example, having now the clone in >> ~/.cache/guix/checkouts, with a different commit, I get: > > SWH is not a forge but an archive. :-) Therefore, this update case does > not make sense to me. I mean, > > $ git -C ~/.cache/guix/checkouts/6k7wvrcpbdsw3pje5b4squybw3jfn3viyrj7gcl7= fipa5yjflaza fetch > fatal: d=C3=A9p=C3=B4t 'http://example.org/sdf/' non trouv=C3=A9 Right, that=E2=80=99s a reasonable limitation. > Well, maybe this cache could be removed if the commit is not found > inside this cache and retry to fetch it from SWH. Obviously, the > downdate case works. It=E2=80=99s still useful to keep it cached around in case the user is goin= g to use it several times in a row. > Note that on fresh clone, the error message could be improved: > > $ ./pre-inst-env guix build guix --with-git-url=3Dguix=3Dhttps://example.= org --with-commit=3Dguix=3Dff613c2b68aac539262822490448e637d8f315ba -n > updating checkout of 'https://example.org'... > guix build: error: Git failure while fetching https://example.org: unexpe= cted http status code: 404 > > > where https://example.org is bogus and > ff613c2b68aac539262822490448e637d8f315ba is not yet archived on SWH. It > could be nice to warn in addition to the 404 that it is not found in > SWH. WDYT? Agreed; I=E2=80=99ve made this change (actually =E2=80=98swh-download=E2=80= =99 prints something upon failure since commit 60b42bec8413aa9844e625fb1903257f1bc1e55c, but it looks more like a debugging message.) > $ guix build guix --with-git-url=3Dguix=3Dhttps://example.org --with-comm= it=3Dguix=3Dc75b30d58f0becb0a5cd6a8bfe69d1063b0d1ada -n > updating checkout of 'https://example.org'... > SWH: found revision c75b30d58f0becb0a5cd6a8bfe69d1063b0d1ada with directo= ry at 'https://archive.softwareheritage.org/api/1/directory/ca2e8a7222b4850= c7bea935dff86b9c2a905efd6/' > SWH vault: requested bundle cooking, waiting for completion... > SWH vault: Processing... > [...] > > > then after several hours, I get this: > > SWH vault: failure: Internal Server Error. This incident will be reported. > SWH vault: retrying... > SWH vault: requested bundle cooking, waiting for completion... > SWH vault: Processing... > > and after more than 12h, the status is still: =C2=ABSWH vault: Processing= ...=C2=BB > and nothing is complete. Did it eventually succeed? We obviously have no guarantee as to how long it might take to cook a bundle. > About this =E2=80=99keyring=E2=80=99 branch, somehow it could be as a sep= arated repo, so > why not effectively do it. :-) I mean, get the branch as it is and > mirror this branch in another Git repo saved on SWH; fallback to it if > =E2=80=99keyring=E2=80=99 branch is not there. I do not know=E2=80=A6 O= r simply wait that SWH > improves their things. :-) Yeah, they=E2=80=99re planning to support it eventually. >> *Third, and this answers the asterisk above, we must keep in mind that >> this is content-addressibility *with SHA1*. Generating a chosen-prefix >> collision is becoming affordable=C2=B3, so users absolutely need an addi= tional >> mechanism to authenticate code they fetched. [...] > How a chosen-prefix attack could work here? I understand why the second > preimage attack is an issue. But I miss how the SHA-1 chosen-prefix atta= ck > could be exploited here to compromise the user, because this hash is prov= ided > by this very same user. I think you=E2=80=99re right, it=E2=80=99s rather second-preimage attacks t= hat would be a serious problem. My point is: as time passes, assuming that a SHA1 resolves to a single revision on SWH is becoming more and more questionable. >> swh: Support downloads of bare Git repositories. >> git: 'update-cached-checkout' can fall back to SWH when cloning. >> git: 'reference-available?' recognizes 'tag-or-commit'. I=E2=80=99ve pushed this after adding the warning as you suggested: dce2cf311b * git: 'reference-available?' recognizes 'tag-or-commit'. 05f44c2d85 * git: 'update-cached-checkout' can fall back to SWH when clon= ing. 6ec81c31c0 * swh: Support downloads of bare Git repositories. Thanks a lot for reviewing and testing on real-world examples! Ludo=E2=80=99.