Re: [swh-devel] Call for public review - SWH Nix/GNU Guix stack

["Ludovic Courtès" <ludovic.courtes@inria.fr>]

Hi Benoit and all!

(Cc: guix-devel rather than gnu-system-discuss.)

Benoit Chauvet <benoit.chauvet@inria.fr> skribis:

> Regarding the Nix/GNU Guix stack, Software Heritage will soon be ready to
> support the
> ingestion of specific versioned files, tarballs, git, hg, svn source code
> listed in their respective manifests [1] (as origins). The new lister (and
> extra loaders, namely
> {Content|Directory|GitCheckout|SvnExport|HgCheckout}Loader) have been
> deployed in our staging infrastructure [2].

Excellent!  I believe this addresses a problem we recently reported
regarding tarballs published with our own content-addressed URLs, which
look like:

  https://bordeaux.guix.gnu.org/file/BiocNeighbors_1.20.0.tar.gz/sha256/0a5wg099fgwjbzd6r3mr4l02rcmjqlkdcz1w97qzwx1mir41fmas

My understanding is that so far these URLs were ignored by the
lister/loader because they didn’t end in *.tar.*.⁰

> The initial NixGuix loader (currently in production) lists and loads
> origins from a manifest, ignoring the specific origins mentioned above. The
> new stack will be able to ingest those origins. It will also optionally
> associate, if present, a NAR hash (specific intrinsic identifier to Nix and
> Guix) to what’s called an ExtID (SWH side).
> Regarding the SWH API reading side of the ExtID though is a work to be done.
>
> On staging, we have currently ingested origins that were listed from the
> GNU Guix manifest [3].
>
> We have already improved the implementations after discussing multiple
> limitations encountered along the way with the Guix community [4].

I’m sure Simon Tournier (Cc’d) already discussed with others at SWH how
crucial it is for us to be able to query content by nar hash.

Essentially, it would fill the gap that currently prevents us from
retrieving Subversion checkouts from SWH¹ and more generally complicates
retrieval of anything not referenced by a Git hash.  So obviously, we’re
looking forward to that ExtID interface for SWH.

Thanks for sharing this status update, these are all exciting news and
perspectives!

Ludo’.

⁰ https://issues.guix.gnu.org/39885#15-lineno60
¹ https://issues.guix.gnu.org/43442#13-lineno37