all messages for Guix-related lists mirrored at yhetil.org
 help / color / mirror / code / Atom feed
From: "Ludovic Courtès" <ludo@gnu.org>
To: zimoun <zimon.toutoune@gmail.com>
Cc: 44187-done@debbugs.gnu.org
Subject: bug#44187: Channel clones lack SWH fallback
Date: Sat, 18 Sep 2021 23:10:12 +0200	[thread overview]
Message-ID: <87pmt5si8b.fsf@gnu.org> (raw)
In-Reply-To: <86o88r1vfe.fsf@gmail.com> (zimoun's message of "Fri, 17 Sep 2021 10:02:45 +0200")

Hello!

zimoun <zimon.toutoune@gmail.com> skribis:

> The original URL of the channel was:
> <https://github.com/zimoun/channel-example.git>.  And this channel
> defines a package where the upstream has also disappeared
> <https://github.com/zimoun/hello-example.git>.  Note the URL in the
> package definition is not bogus… but using one was already working. :-)
>
> All is saved on SWH, so now all is transparent!  From my point of view,
> this is a killer feature for scientific folks. :-)

Yay!  Great that you came up with a nice example to test it on!

>> First, fallback is implemented only for fresh clones, not for updates.
>> Thus, if I rerun the first example, having now the clone in
>> ~/.cache/guix/checkouts, with a different commit, I get:
>
> SWH is not a forge but an archive. :-)  Therefore, this update case does
> not make sense to me.  I mean,
>
> $ git -C ~/.cache/guix/checkouts/6k7wvrcpbdsw3pje5b4squybw3jfn3viyrj7gcl7fipa5yjflaza fetch
> fatal: dépôt 'http://example.org/sdf/' non trouvé

Right, that’s a reasonable limitation.

> Well, maybe this cache could be removed if the commit is not found
> inside this cache and retry to fetch it from SWH.  Obviously, the
> downdate case works.

It’s still useful to keep it cached around in case the user is going to
use it several times in a row.

> Note that on fresh clone, the error message could be improved:
>
> $ ./pre-inst-env guix build guix --with-git-url=guix=https://example.org --with-commit=guix=ff613c2b68aac539262822490448e637d8f315ba -n
> updating checkout of 'https://example.org'...
> guix build: error: Git failure while fetching https://example.org: unexpected http status code: 404
>
>
> where https://example.org is bogus and
> ff613c2b68aac539262822490448e637d8f315ba is not yet archived on SWH.  It
> could be nice to warn in addition to the 404 that it is not found in
> SWH.  WDYT?

Agreed; I’ve made this change (actually ‘swh-download’ prints something
upon failure since commit 60b42bec8413aa9844e625fb1903257f1bc1e55c, but
it looks more like a debugging message.)

> $ guix build guix --with-git-url=guix=https://example.org --with-commit=guix=c75b30d58f0becb0a5cd6a8bfe69d1063b0d1ada -n
> updating checkout of 'https://example.org'...
> SWH: found revision c75b30d58f0becb0a5cd6a8bfe69d1063b0d1ada with directory at 'https://archive.softwareheritage.org/api/1/directory/ca2e8a7222b4850c7bea935dff86b9c2a905efd6/'
> SWH vault: requested bundle cooking, waiting for completion...
> SWH vault: Processing...
> [...]
>
>
> then after several hours, I get this:
>
> SWH vault: failure: Internal Server Error. This incident will be reported.
> SWH vault: retrying...
> SWH vault: requested bundle cooking, waiting for completion...
> SWH vault: Processing...
>
> and after more than 12h, the status is still: «SWH vault: Processing...»
> and nothing is complete.

Did it eventually succeed?  We obviously have no guarantee as to how
long it might take to cook a bundle.

> About this ’keyring’ branch, somehow it could be as a separated repo, so
> why not effectively do it. :-) I mean, get the branch as it is and
> mirror this branch in another Git repo saved on SWH; fallback to it if
> ’keyring’ branch is not there.  I do not know…  Or simply wait that SWH
> improves their things. :-)

Yeah, they’re planning to support it eventually.

>> *Third, and this answers the asterisk above, we must keep in mind that
>> this is content-addressibility *with SHA1*.  Generating a chosen-prefix
>> collision is becoming affordable³, so users absolutely need an additional
>> mechanism to authenticate code they fetched.

[...]

> How a chosen-prefix attack could work here?  I understand why the second
> preimage attack is an issue.  But I miss how the SHA-1 chosen-prefix attack
> could be exploited here to compromise the user, because this hash is provided
> by this very same user.

I think you’re right, it’s rather second-preimage attacks that would be
a serious problem.  My point is: as time passes, assuming that a SHA1
resolves to a single revision on SWH is becoming more and more
questionable.

>>   swh: Support downloads of bare Git repositories.
>>   git: 'update-cached-checkout' can fall back to SWH when cloning.
>>   git: 'reference-available?' recognizes 'tag-or-commit'.

I’ve pushed this after adding the warning as you suggested:

  dce2cf311b * git: 'reference-available?' recognizes 'tag-or-commit'.
  05f44c2d85 * git: 'update-cached-checkout' can fall back to SWH when cloning.
  6ec81c31c0 * swh: Support downloads of bare Git repositories.

Thanks a lot for reviewing and testing on real-world examples!

Ludo’.




  reply	other threads:[~2021-09-18 21:11 UTC|newest]

Thread overview: 15+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-10-23 22:17 bug#44187: whishlist: time-machine --channel falls back to SWH zimoun
2021-03-05 14:51 ` Ludovic Courtès
2021-09-10 14:34   ` bug#44187: [PATCH 0/3] Fall back to Software Heritage (SWH) for Git clones Ludovic Courtès
2021-09-10 14:34     ` bug#44187: [PATCH 1/3] swh: Support downloads of bare Git repositories Ludovic Courtès
2021-09-17 17:31       ` bug#44187: Channel clones lack SWH fallback zimoun
2021-09-18 10:05         ` Ludovic Courtès
2021-09-18 10:27           ` zimoun
2021-09-10 14:34     ` bug#44187: [PATCH 2/3] git: 'update-cached-checkout' can fall back to SWH when cloning Ludovic Courtès
2021-09-10 14:34     ` bug#44187: [PATCH 3/3] git: 'reference-available?' recognizes 'tag-or-commit' Ludovic Courtès
2021-09-13 16:07     ` bug#44187: [PATCH 0/3] Fall back to Software Heritage (SWH) for Git clones zimoun
2021-09-14 13:37       ` Ludovic Courtès
2021-09-17  8:02 ` bug#44187: Channel clones lack SWH fallback zimoun
2021-09-18 21:10   ` Ludovic Courtès [this message]
2021-09-20  9:27     ` zimoun
2021-09-22 10:03       ` Ludovic Courtès

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=87pmt5si8b.fsf@gnu.org \
    --to=ludo@gnu.org \
    --cc=44187-done@debbugs.gnu.org \
    --cc=zimon.toutoune@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this external index

	https://git.savannah.gnu.org/cgit/guix.git

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.