From: Timothy Sample <samplet@ngyro.com>
To: "Ludovic Courtès" <ludovic.courtes@inria.fr>
Cc: guix-devel@gnu.org
Subject: Re: Preservation of Guix report for 2024-01-26
Date: Tue, 30 Jan 2024 12:01:22 -0600 [thread overview]
Message-ID: <87wmrqya31.fsf@ngyro.com> (raw)
In-Reply-To: <87zfwokqlg.fsf@gnu.org> ("Ludovic Courtès"'s message of "Mon, 29 Jan 2024 18:16:11 +0100")
Ludovic Courtès <ludovic.courtes@inria.fr> writes:
> Thumbs up on bzip2 support! We should update Disarchive in Guix but
> perhaps that’s already in your pipeline?
I sent https://issues.guix.gnu.org/68769. Now I see that I didn’t have
the newest Git hooks installed, so no change ID and no email to the
relevant team. Sorry! (I use worktrees so the Makefile didn’t fix this
for me automatically – I should have double checked.)
> We’ll also have to sync the disarchive.guix.gnu.org with ngyro.com.
Hopefully our old system will work again, but I will have to consolidate
my collection of Disarchive specifications, first.
> How did you implement the Subversion check?
The hard way: download, verify Guix hash, compute SWHID, check existence
in SWHID. (That’s how everything works to date with PoG, but hopefully
the hard way will become obsolete with all the recent support SWH has
been providing us.)
>> Some of these (I didn’t check them all) are in SWH as content rather
>> than directories.
>
> Back in the day, they told me that tarballs can sometimes be ingested,
> for instance if they are committed to a VCS repo (that’s why our
> fallback code tries that as well). Maybe that’s what happened?
Probably, but I don’t quite understand the mechanism. The “nixguix”
loader uses ExtIDs for deduplication. My assumption is that it will
only skip unpacking a tarball if there’s an existing ExtID. It doesn’t
look like there are ExtIDs for these tarballs, so I’m not sure. (I’ve
been fumbling a bit trying to use the ExtID API, so maybe it’s a mistake
on my end.)
>> The long-term road map is to make it work like an archive. It will run
>> continuously and store *all* Guix sources. To make this easy data-wise,
>> it will only store what’s not covered by SWH.
>
> By “it”, you mean the Disarchive DB?
I mean the PoG “project”. Instead of just testing and reporting, it
will preserve. For instance, right now if it encounters a tarball that
Disarchive can’t unpack, it shrugs and moves on. I want it to store
those so that we are guaranteed to be able to revisit it in the future.
Same thing for sources not (yet) in SWH. I want to store those so that
SWH can ingest them later. Because we are doing so well working with
SWH, the storage requirements for this will be manageable (10s of
gigabytes). With that in place, the PoG report will simply explain what
the archive needs to store and why. Our goal, then, will be for it to
store nothing.
-- Tim
next prev parent reply other threads:[~2024-01-30 18:02 UTC|newest]
Thread overview: 4+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-01-28 0:47 Preservation of Guix report for 2024-01-26 Timothy Sample
2024-01-29 17:16 ` Ludovic Courtès
2024-01-30 18:01 ` Timothy Sample [this message]
2024-02-06 14:44 ` Ludovic Courtès
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://guix.gnu.org/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87wmrqya31.fsf@ngyro.com \
--to=samplet@ngyro.com \
--cc=guix-devel@gnu.org \
--cc=ludovic.courtes@inria.fr \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/guix.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).