Re: Downloader for "wrapped" tarbar?

[zimoun <zimon.toutoune@gmail.com>]

Dear Hartmut,

On Sat, 6 Jun 2020 at 17:29, Hartmut Goebel
<h.goebel@crazy-compilers.com> wrote:

> 2. When implementing some "wrapped-fetch" (name tdb), modeled like
> "git-fetch", there is no easy way for the user to verify the hash, as
> this is taken from the "inner" tarball. How does this work with
> substitutes, download-nar and SWH?

Today, Guix feeds SWH with only one stream "guix lint" and only for
'git-fetch' packages; if I understand well.  The origin methods for
Guix packages look like:

      1 bzr-fetch
      3 cvs-fetch
      9 url-fetch/tarbomb
     24 url-fetch/zipbomb
     28 hg-fetch
     30 computed-origin-method
     67 no-origin
    115 svn-fetch
    135 svn-multi-fetch
   3574 git-fetch
   9690 url-fetch

where 'svn-multi-fetch' are mainly CTAN/TeX packages.  Well, as you
see, most of the packages are not yet archived in SWH. Since SWH
supports 'svn-fetch' and 'hg-fetch', it is doable to add them to "guix
lint" but it is low-priority -- at least on my TODO. :-)

The SWH-side WIP is about 'url-fetch'.  I have not followed all the
recent developments by lewo but roughly speaking they are implementing
another "lister" [2,3,4] for tarballs.  Well, the final aim is that
SWH automatically ingests https://guix.gnu.org/sources.json which is
automatically generated every X minutes.  Currently, the compliance of
this 'sources.json' is still a WIP; the format is changing and the
specification not yet fixed.

What SWH archives is the upstream source, i.e., *not* "guix build -S"
but what comes from 'origin'.  What happens after and what Guix does
not matter for SWH.

Therefore, if I understand correctly, SWH will archive the initial
tarball.  (Sorry, I am lost with the "inner/outer" terminology.)  Note
that only the package tarball you pointed [5] needs to be checksummed,
well if this initial package tarball matches then 'contents.tar.gz'
will match too, isn't it?

I hope to not have misread and missed something.


All the best,
simon

[1] https://archive.softwareheritage.org/save/
[2] https://docs.softwareheritage.org/devel/swh-lister/index.html
[3] https://forge.softwareheritage.org/D2025
[4] https://forge.softwareheritage.org/T1991
[5] https://github.com/hexpm/specifications/blob/master/package_tarball.md