From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0 ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id qPRHOQlMRGH/kgAAgWs5BA (envelope-from ) for ; Fri, 17 Sep 2021 10:04:25 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0 with LMTPS id uFsCNQlMRGE0GwAA1q6Kng (envelope-from ) for ; Fri, 17 Sep 2021 08:04:25 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id E7C779899 for ; Fri, 17 Sep 2021 10:04:24 +0200 (CEST) Received: from localhost ([::1]:44530 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mR8r7-0002dK-N0 for larch@yhetil.org; Fri, 17 Sep 2021 04:04:21 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:40786) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mR8qo-0002Zd-5N for bug-guix@gnu.org; Fri, 17 Sep 2021 04:04:02 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:46843) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mR8qn-0006yy-Sr for bug-guix@gnu.org; Fri, 17 Sep 2021 04:04:01 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1mR8qn-0001JE-Jh for bug-guix@gnu.org; Fri, 17 Sep 2021 04:04:01 -0400 X-Loop: help-debbugs@gnu.org Subject: bug#44187: Channel clones lack SWH fallback In-Reply-To: <86pn581t9s.fsf@gmail.com> Resent-From: zimoun Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Fri, 17 Sep 2021 08:04:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 44187 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Received: via spool by 44187-submit@debbugs.gnu.org id=B44187.16318657894968 (code B ref 44187); Fri, 17 Sep 2021 08:04:01 +0000 Received: (at 44187) by debbugs.gnu.org; 17 Sep 2021 08:03:09 +0000 Received: from localhost ([127.0.0.1]:58389 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mR8pw-0001I3-Hc for submit@debbugs.gnu.org; Fri, 17 Sep 2021 04:03:09 -0400 Received: from mail-wr1-f46.google.com ([209.85.221.46]:36845) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mR8pu-0001HX-8L for 44187@debbugs.gnu.org; Fri, 17 Sep 2021 04:03:08 -0400 Received: by mail-wr1-f46.google.com with SMTP id g16so13711792wrb.3 for <44187@debbugs.gnu.org>; Fri, 17 Sep 2021 01:03:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:references:date:message-id:mime-version :content-transfer-encoding; bh=YorKxrG4pOBn05e57Q5JU2zPkqwXuq137IxfnRXZzXY=; b=G/EK+l0IXrqFA3FqqcPbhqX0cxT4Eu6LarqD2t0CCkrGv2kwEegdduZuf+KerKs2oH TaHayitA6u2Xrxe9Z430mCiAVcHUuUAatd4PO4zfJpDuduVMCXtaoldPKget/UJEcHYT 8+AXVrbAvc/oXWKM53+YBlTxdcJmpBgDcRwLikNLhCpHZtc0qoCbaHtDEc5lP6sdVoCX AyiTL7ThmSYr5N5n/OkHEg3sFWRSs5ZpyqlYkDRTbEgprSiwi90a4qaV6ovbpFlPPpmv /O14F6V97aisstC6xBtvUd1J87WY011liQsaUBTkergqxvHd0ZJkYcu+Sgbami+CGYLZ fH1Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:references:date:message-id :mime-version:content-transfer-encoding; bh=YorKxrG4pOBn05e57Q5JU2zPkqwXuq137IxfnRXZzXY=; b=goOod5YbJM9df1TcdNkwRqXa8Cy0fxLrtl8pszT6iawRSBHyVEShnU198cmIdKb0Wc muVuc9Gm/8oB7KVibzSMwDOQrLSgnvyy7mJSYlJ0Y88bXgpf+GeUI7EQOMYKcM/rZgqU Cmx8ySrVEQ23Ci7mCVxbvg5rw34ndIzzn64BlH0Rhet0kfDMLsnUFLMXeAXj5aNxXZ1U fK4WsO0O2gGtHD6dTR3pzi6tAzQtPhJjvKGaTKia4k3g8w2XaGbbYX5vY+UXcwtpXVD5 hTlq5DOgqDB0OYTDu1cbywbzbpOZ9fxj0SrS9ruLhZcjT/3bpR5LOVzY754oU2cEZQVT vbow== X-Gm-Message-State: AOAM533ggabwNBoUKn7Al6JSEx/N6GoDizc+pQV7TylTmqzYtwwU+x/v A92o2QwvVRsTxplHiCuyiV06peaayPk= X-Google-Smtp-Source: ABdhPJxpRw+xUltswsxNlADft9hgS1r7WO5UVWWGZ83mfY+IoRcvgEjstQMbPWvV6bCxkFkei/XueQ== X-Received: by 2002:adf:f88d:: with SMTP id u13mr10698738wrp.297.1631865779070; Fri, 17 Sep 2021 01:02:59 -0700 (PDT) Received: from jack-Precision-7820-Tower ([193.48.40.117]) by smtp.gmail.com with ESMTPSA id y8sm5809836wrh.44.2021.09.17.01.02.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 17 Sep 2021 01:02:58 -0700 (PDT) From: zimoun References: <87pn0dk61v.fsf@gnu.org> <20210910143415.14783-1-ludo@gnu.org> Date: Fri, 17 Sep 2021 10:02:45 +0200 Message-ID: <86o88r1vfe.fsf@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-guix@gnu.org List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: 44187@debbugs.gnu.org Errors-To: bug-guix-bounces+larch=yhetil.org@gnu.org Sender: "bug-Guix" X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1631865865; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:resent-cc: resent-from:resent-sender:resent-message-id:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=YorKxrG4pOBn05e57Q5JU2zPkqwXuq137IxfnRXZzXY=; b=qVfUtkGds9M7+ieTd7jNSLxa6RsilVdVL3eI/e7FC4K1LAIMM4mLZG0ixXzmj8yOEK+Vt9 8fZq0H7AAVc6qSKbAInHOV8BnrKxW+kxUCHHChIaUt+n8dPR1B6QhoJcD6h6HbMEvk0Fsc M2IYvC/Ef99li6VT+HE57Bsu1emDDcuE0nirqRbmcz3L85Mluh9NymhheAyw61W4nML1/G c/M3aSvx+aeXJp1n5OH994zQCUwoBWnAKIFRZ5sgacwY3sMEbvwInQOmubWSZZ2uUqlSt9 L2B0rd2Uf3UqpAYTyUbbatat3N8NShlKt5WYt4LmdoLHikxXH/4nvt29O490OQ== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1631865865; a=rsa-sha256; cv=none; b=KKeBYf9phP3Q4DDo338YAzImc3inTTS0NHwqgDJ8Ne4PAs+dQOj595kYOt2BAPhSYshSOl rowEC/7WEiMFBDJMfFIF66WisAUGQKoadU08lDQnHo6eBbJJp4tN3ANvF1ZI0Ez4TuaH4x mVPTFC4LYyFXOg3utsCk0Jyr3j3puy/aJAgfVeV/PgLtEsxMs6EJwYAfwqSJ03KB8tUnWE MXetgikqVEB0Bdc2B5qIhjrULfwYSbcp369/tTjUegI056vw/tnzLgJMaEojIE58fL/pip L0Bo1T2NMLO0JtA0VxexaSgPYVFz9PmKD4s7gZm7K9o3oPekRmi+/oP0gszDDg== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=gmail.com header.s=20210112 header.b="G/EK+l0I"; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none); spf=pass (aspmx1.migadu.com: domain of bug-guix-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=bug-guix-bounces@gnu.org X-Migadu-Spam-Score: -1.30 Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=gmail.com header.s=20210112 header.b="G/EK+l0I"; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none); spf=pass (aspmx1.migadu.com: domain of bug-guix-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=bug-guix-bounces@gnu.org X-Migadu-Queue-Id: E7C779899 X-Spam-Score: -1.30 X-Migadu-Scanner: scn1.migadu.com X-TUID: Ym2+IOSaItUt Hi, On ven., 10 sept. 2021 at 16:34, Ludovic Court=C3=A8s wrote: > Finally we can enjoy content-addressability and brittle URLs > are becoming a thing of the past!* Yeah, it is awesome! The original URL of the channel was: . And this channel defines a package where the upstream has also disappeared . Note the URL in the package definition is not bogus=E2=80=A6 but using one was already working.= :-) All is saved on SWH, so now all is transparent! From my point of view, this is a killer feature for scientific folks. :-) --8<---------------cut here---------------start------------->8--- $ cat /tmp/channels.scm (list (channel (name 'guix) (url "/home/sitour/src/guix/guix") (branch "fix-44187") (commit "cdea76a2fdaf7705583a02081a6468d436b8df05")) (channel (name 'example) (url "https://example.org/foo.git") (commit "67c9f2143aa6f545419ae913b4ae02af4cd3effc"))) $ ./pre-inst-env guix time-machine -C /tmp/channels.scm --disable-authentic= ation -- build hi Updating channel 'guix' from Git repository at '/home/sitour/src/guix/guix'= ... guix time-machine: warning: channel authentication disabled Updating channel 'example' from Git repository at 'https://example.org/foo.= git'... SWH: found revision 67c9f2143aa6f545419ae913b4ae02af4cd3effc with directory= at 'https://archive.softwareheritage.org/api/1/directory/fe423e88ce277d3fc= 230c88d408e42b14a3a458c/' SWH vault: requested bundle cooking, waiting for completion... swh:1:rev:67c9f2143aa6f545419ae913b4ae02af4cd3effc.git/ swh:1:rev:67c9f2143aa6f545419ae913b4ae02af4cd3effc.git/HEAD swh:1:rev:67c9f2143aa6f545419ae913b4ae02af4cd3effc.git/branches/ swh:1:rev:67c9f2143aa6f545419ae913b4ae02af4cd3effc.git/config swh:1:rev:67c9f2143aa6f545419ae913b4ae02af4cd3effc.git/description swh:1:rev:67c9f2143aa6f545419ae913b4ae02af4cd3effc.git/hooks/ swh:1:rev:67c9f2143aa6f545419ae913b4ae02af4cd3effc.git/info/ swh:1:rev:67c9f2143aa6f545419ae913b4ae02af4cd3effc.git/info/exclude swh:1:rev:67c9f2143aa6f545419ae913b4ae02af4cd3effc.git/info/refs swh:1:rev:67c9f2143aa6f545419ae913b4ae02af4cd3effc.git/objects/ swh:1:rev:67c9f2143aa6f545419ae913b4ae02af4cd3effc.git/objects/info/ swh:1:rev:67c9f2143aa6f545419ae913b4ae02af4cd3effc.git/objects/info/packs swh:1:rev:67c9f2143aa6f545419ae913b4ae02af4cd3effc.git/objects/pack/ swh:1:rev:67c9f2143aa6f545419ae913b4ae02af4cd3effc.git/objects/pack/pack-4e= 9279a1b64e4dda7bd9d84bb6b50bb1f80def08.idx swh:1:rev:67c9f2143aa6f545419ae913b4ae02af4cd3effc.git/objects/pack/pack-4e= 9279a1b64e4dda7bd9d84bb6b50bb1f80def08.pack swh:1:rev:67c9f2143aa6f545419ae913b4ae02af4cd3effc.git/refs/ swh:1:rev:67c9f2143aa6f545419ae913b4ae02af4cd3effc.git/refs/heads/ swh:1:rev:67c9f2143aa6f545419ae913b4ae02af4cd3effc.git/refs/heads/master swh:1:rev:67c9f2143aa6f545419ae913b4ae02af4cd3effc.git/refs/tags/ guix time-machine: warning: channel authentication disabled [...] Computing Guix derivation for 'x86_64-linux'... - [...] construction de /gnu/store/6g9qlysbbk7p4609xrv82j0wzbib1y4r-git-checkout.dr= v... guile: warning: failed to install locale environment variable `PATH' set to `/gnu/store/378zjf2kgajcfd7mfr98jn5xyc5w= a3qv-gzip-1.10/bin:/gnu/store/sf3rbvb6iqcphgm1afbplcs72hsywg25-tar-1.32/bin' hint: Using 'master' as the name for the initial branch. This default branc= h name hint: is subject to change. To configure the initial branch name to use in = all hint: of your new repositories, which will suppress this warning, call: hint: hint: git config --global init.defaultBranch hint: hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and hint: 'development'. The just-created branch can be renamed via this comman= d: hint: hint: git branch -m Initialized empty Git repository in /gnu/store/884nsva9r8wkp40kbqyvpj1ad57j= c5dd-git-checkout/.git/ fatal: could not read Username for 'https://github.com': No such device or = address Failed to do a shallow fetch; retrying a full fetch... fatal: could not read Username for 'https://github.com': No such device or = address git-fetch: '/gnu/store/5vai7bfrfkzv22dx13bxpszjrqyi78x6-git-minimal-2.33.0/= bin/git fetch origin' failed with exit code 128 Trying content-addressed mirror at berlin.guix.gnu.org... Trying content-addressed mirror at berlin.guix.gnu.org... Trying to download from Software Heritage... SWH: found revision e1eefd033b8a2c4c81babc6fde08ebb116c6abb8 with directory= at 'https://archive.softwareheritage.org/api/1/directory/c3e538ed2de412d54= c567ed7c8cfc46cbbc35d07/' swh:1:dir:c3e538ed2de412d54c567ed7c8cfc46cbbc35d07/ swh:1:dir:c3e538ed2de412d54c567ed7c8cfc46cbbc35d07/ABOUT-NLS swh:1:dir:c3e538ed2de412d54c567ed7c8cfc46cbbc35d07/AUTHORS swh:1:dir:c3e538ed2de412d54c567ed7c8cfc46cbbc35d07/COPYING [...] swh:1:dir:c3e538ed2de412d54c567ed7c8cfc46cbbc35d07/tests/hello-1 swh:1:dir:c3e538ed2de412d54c567ed7c8cfc46cbbc35d07/tests/last-1 swh:1:dir:c3e538ed2de412d54c567ed7c8cfc46cbbc35d07/tests/traditional-1 construction de /gnu/store/6g9qlysbbk7p4609xrv82j0wzbib1y4r-git-checkout.dr= v r=C3=A9ussie construction de /gnu/store/jx1r7w8xaw768176pjl0j0q1l1529w75-hi-2.10.drv... starting phase `set-SOURCE-DATE-EPOCH' phase `set-SOURCE-DATE-EPOCH' succeeded after 0.0 seconds [...] construction de /gnu/store/jx1r7w8xaw768176pjl0j0q1l1529w75-hi-2.10.drv r= =C3=A9ussie /gnu/store/jn8d031zx4znxy7s5zhj4dbr6xjsfq9v-hi-2.10 --8<---------------cut here---------------end--------------->8--- Well, it still misses the tarball and non-Git fetch method fallback and the story will be more than awesome! :-) > Limitations > ~~~~~~~~~~~~ > > Yes, there=E2=80=99s a couple of them. Well, yes some limitations but not so much. ;-) > First, fallback is implemented only for fresh clones, not for updates. > Thus, if I rerun the first example, having now the clone in > ~/.cache/guix/checkouts, with a different commit, I get: SWH is not a forge but an archive. :-) Therefore, this update case does not make sense to me. I mean, --8<---------------cut here---------------start------------->8--- $ git -C ~/.cache/guix/checkouts/6k7wvrcpbdsw3pje5b4squybw3jfn3viyrj7gcl7fi= pa5yjflaza fetch fatal: d=C3=A9p=C3=B4t 'http://example.org/sdf/' non trouv=C3=A9 --8<---------------cut here---------------end--------------->8--- Well, maybe this cache could be removed if the commit is not found inside this cache and retry to fetch it from SWH. Obviously, the downdate case works. Note that on fresh clone, the error message could be improved: --8<---------------cut here---------------start------------->8--- $ ./pre-inst-env guix build guix --with-git-url=3Dguix=3Dhttps://example.or= g --with-commit=3Dguix=3Dff613c2b68aac539262822490448e637d8f315ba -n updating checkout of 'https://example.org'... guix build: error: Git failure while fetching https://example.org: unexpect= ed http status code: 404 --8<---------------cut here---------------end--------------->8--- where https://example.org is bogus and ff613c2b68aac539262822490448e637d8f315ba is not yet archived on SWH. It could be nice to warn in addition to the 404 that it is not found in SWH. WDYT? > Second, clones from SWH only contain the one branch that the revision > is on. For channels, that means that the =E2=80=98keyring=E2=80=99 branc= h is not fetched, > which is why I commented out =E2=80=98introduction=E2=80=99 in /tmp/chan.= scm above. To me, it is not an issue. Because you reach a commit from the past knowing the hash. Aside my opinion, I wanted to know which kind of metadata we get back from the Git repo, so I tried: --8<---------------cut here---------------start------------->8--- $ guix build guix --with-git-url=3Dguix=3Dhttps://example.org --with-commit= =3Dguix=3Dc75b30d58f0becb0a5cd6a8bfe69d1063b0d1ada -n updating checkout of 'https://example.org'... SWH: found revision c75b30d58f0becb0a5cd6a8bfe69d1063b0d1ada with directory= at 'https://archive.softwareheritage.org/api/1/directory/ca2e8a7222b4850c7= bea935dff86b9c2a905efd6/' SWH vault: requested bundle cooking, waiting for completion... SWH vault: Processing... [...] --8<---------------cut here---------------end--------------->8--- then after several hours, I get this: --8<---------------cut here---------------start------------->8--- SWH vault: failure: Internal Server Error. This incident will be reported. SWH vault: retrying... SWH vault: requested bundle cooking, waiting for completion... SWH vault: Processing... --8<---------------cut here---------------end--------------->8--- and after more than 12h, the status is still: =C2=ABSWH vault: Processing..= .=C2=BB and nothing is complete. About this =E2=80=99keyring=E2=80=99 branch, somehow it could be as a separ= ated repo, so why not effectively do it. :-) I mean, get the branch as it is and mirror this branch in another Git repo saved on SWH; fallback to it if =E2=80=99keyring=E2=80=99 branch is not there. I do not know=E2=80=A6 Or = simply wait that SWH improves their things. :-) > *Third, and this answers the asterisk above, we must keep in mind that > this is content-addressibility *with SHA1*. Generating a chosen-prefix > collision is becoming affordable=C2=B3, so users absolutely need an addit= ional > mechanism to authenticate code they fetched. > > For origins, we have the content SHA256, so we=E2=80=99re fine. For chan= nels, > we have Guix=E2=80=99s authentication mechanism=C2=B9, except it=E2=80=99= s not available yet > via SWH, as I wrote above. For the footswitch example above using > =E2=80=98--with-commit=E2=80=99, we don=E2=80=99t have any authentication= method, but in fact, > that=E2=80=99s the situation of Git repositories in general: they can rar= ely be > authenticated. How a chosen-prefix attack could work here? I understand why the second preimage attack is an issue. But I miss how the SHA-1 chosen-prefix attack could be exploited here to compromise the user, because this hash is provid= ed by this very same user. > Ludovic Court=C3=A8s (3): > swh: Support downloads of bare Git repositories. > git: 'update-cached-checkout' can fall back to SWH when cloning. > git: 'reference-available?' recognizes 'tag-or-commit'. LGTM! Cheers, simon