Re: extend ’guix archive’?

[Christopher Baines <mail@cbaines.net>]

[-- Attachment #1: Type: text/plain, Size: 2093 bytes --]


zimoun <zimon.toutoune@gmail.com> writes:

> On Mon, 20 Dec 2021 at 23:07, Ludovic Courtès <ludo@gnu.org> wrote:
>
>> Regarding nar-herder, I think it’d be nice to have a solution to
>> mirroring in Guix proper, developed similarly to other components,
>> because it could be a fairly central tool.
>>
>> ‘guix publish’ is probably not extensible enough to support that, but we
>> could make it a new ‘guix mirror’ or ‘guix sync’ or whatever command.
>
> Why not extend “guix archive”?
>
> However, without all the details so my remark is totally naive, I miss
> what “nar-herder” is doing that “guix archive”+rsync+“guix publish” is
> not doing – other said I miss why another SQL database is required to
> serve stuff from one place to another.  I have read README but I did not
> get the point.

Apologies, I missed replying to this at the time.

Using an SQL database (sqlite) isn't required in my opinion, but I do
like that aspect of the design.

You could for example keep the narinfo's as files on the disk, and then
use rsync say to copy them between machines to setup mirrors. I think
this would perform worse compared to storing the narinfos in a database
when initially setting up a mirror. With a database, you only have to
download one large file, whereas with individual narinfo files, you have
to download lots of individual small files. Storing all the narinfo
files individually also generally takes up more space than storing them
in a database, since there's an overhead associated with each file on
the filesystem.

Additionally, the database is used to do extra things on top of just
storing the narinfos. The references from the narinfos are stored
separately to facilitate doing GC like removal of the
nars. Additionally, there's a way to tag nars, something which I was
thinking of using to allow selectively removing some nars which only
need to be stored for a short time.

https://git.cbaines.net/guix/nar-herder/tree/nar-herder/database.scm#n74

I hope that makes some sense,

Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 987 bytes --]