Hi Pierre,

Pierre Neidhardt <mail@ambrevar.xyz> skribis:

> It's definitely the ideal route.  Something like guile-compress or
> guile-archive, with a high-level abstraction for a collection of
> bindings including zlib and lzlib for now.
>
> Sadly I don't have the time for it at the moment.  Unless you do (:p) I
> suggest we add a TODO item and keep it for later.

Sounds good!

Below are the Autoconf-related changes I made.  Committed!

We’ll take care of (guix self) when (guix lzlib) is actually used by
other parts of the code.

> Regarding guix publish and the farms, what shall we do?

I think we should arrange for the client part, ‘guix substitute’, to be
ready to lzip-decode as soon as it talks to an lzip-capable server.

Then we should add support in ‘guix publish’.  At some later point, we’d
deploy it on the build farms.

For this migration to be incremental, we need (1) clients to be able to
transparently switch to lzip when it’s available, and (2) servers to be
able to produce both lzip archives (for new clients) and gzip archives
(for old clients) during the transition period.

That’s a bit of work in ‘guix publish’.  It’ll be extra CPU and storage
usage on the build farm since during the transition period it’d have to
produce and store both gzip and lzip archives for each store item.  I
don’t really see any way around that, though.

A difficulty is that narinfos currently include a fixed compression
scheme:

--8<---------------cut here---------------start------------->8---
$ wget -q -O - https://ci.guix.info/nrkm1683p1cqnkcmhlmhiig9q9qd7xqh.narinfo | head -3
StorePath: /gnu/store/nrkm1683p1cqnkcmhlmhiig9q9qd7xqh-sed-4.5
URL: nar/gzip/nrkm1683p1cqnkcmhlmhiig9q9qd7xqh-sed-4.5
Compression: gzip
--8<---------------cut here---------------end--------------->8---

So, depending on the client, ‘guix publish’ should return either a
narinfo-for-gzip or a narinfo-for-lzip.  To make it possible, new
clients could send an extra HTTP header, say ‘X-Guix-Compression’, that
would specify their preferred compression method(s).  ‘guix publish’
would take that into account when replying.

How does that sound?

Thanks,
Ludo’.