Re: "guix pack -f docker" does too much work

[Ricardo Wurmus <rekado@elephly.net>]

Ludovic Courtès <ludo@gnu.org> writes:

>> I think it would be great if "guix pack -f docker" could avoid building
>> all these identical layers again and again.  Perhaps it would be
>> possible to have a single derivation for each layer?  This way we
>> wouldn't have to recreate the same layer archives every time.
>
> That sounds nice in terms of saving CPU time.  It’s less nice in terms
> of disk usage: a single ‘guix pack -f docker’ run would populate the
> store with roughly twice the size of the closure.

Arguably we don't actually care all that much for the Docker image that
ends up in the store.  It's really a temporary thing that we want to
load into Docker or upload somewhere else.  I've often wanted to stream
the eventual output of "guix pack" to a pipe, precisely because I don't
want to store the same thing twice: once in the store and once in the
Docker storage backend.

It's actually worse than that: I often end up having dozens of packs in
the store whose layers are almost all identical.

> I think each solution (single derivation vs. one derivation per layer)
> makes a different tradeoff.  I don’t have a strong feeling about which
> one is better.

Can we have both?  I realize that adding the option to stream build
output to a pipe is not a trivial change, but it would solve the
unnecessary storage requirement for packs.  "docker load" reads from
standard input, but other packs would also benefit from a streaming
output; an example is Docker-free deployment to a remote server: just
pipe "guix pack" to a remote tar process and you're all set.

-- 
Ricardo