Re: Preservation of Guix Report 2021-12-06

["Ludovic Courtès" <ludo@gnu.org>]

Hello,

Timothy Sample <samplet@ngyro.com> skribis:

>     <https://ngyro.com/pog-reports/2021-12-06/>
>
> Since the last report, I added many more reference categories and moved
> them to the database.  The new categories are 'hg', 'svn', 'cvs', 'bzr',
> 'tar-bz2', 'tar', 'zip', and 'text'.  Of these, only 'tar' and 'text'
> are being processed.  The rest are currently unsupported by my scripts.
> Moving the categories to the database allows me to make manual
> corrections when needed.  It also encouraged me to look through the
> references a bit more carefully to track down some of the weirder 'text'
> sources (like Bash patches) and fix up some other ones (in the style of
> “/tar_gz?download=yes”).

Good to see these additional details.

The SWH folks told me that plain files (like .el or .patch files) that
appear in ‘sources.json’ are currently not archived, but that this could
change.  So seeing 86% of them are archived is good news.

> I also made the fetching code more tenacious.  Now it uses the
> content-addressed mirrors from Guix and Nix to find regular files, and
> will recover “easy” Git references from SWH (“easy” means the commit is
> specified).

I suppose the scripts could use ‘url-fetch’, or even build the
fixed-output tarballs, to benefit from Guix’s fallback methods.
(Apologies if I’m stating the obvious.)

> The “disarchive” category is all the tarballs Disarchive failed to
> process.  An interesting thing here is that most of them are from Cargo.
> Long story short: older versions of Cargo used the “miniz”
> implementation of DEFLATE (rewritten in Rust) to compress tarballs.
> Disarchive doesn’t support this (yet...?).  There are 686
> old-Cargo-produced tarballs in the “disarchive” category.

Ah, I don’t want to hear about Rust!  ;-)

> One last thing to add is that the SWH folks were very quick to fix the
> loading error, so the increase in missing sources for recent commits is
> now gone.

Awesome.

Thanks for the update!

Ludo’.