Experiment in generating multi-layer Docker images with guix pack

[Christopher Baines <mail@cbaines.net>]

These patches are very rough, and not ready, but do at least work in some
limited capacity. I've been testing with the following commands:

  guix pack --format=docker guile@2.2.6
  guix pack --format=docker guile@2.2.7

With the previous Docker image generation implementation, two different ~130MB
images would be generated. These patches mean that each .tar.gz file generated
by guix pack contains a ~53MB layer which contains the profile and directly
referenced store items, and then a ~77MB layer with all the other store items
which is identical for both the 2.2.6 and 2.2.7 pack file.

I think it could be useful to support multiple different strategies for
generating layers for Docker images, with different trade-offs. This approach
using two layers should make the resulting images more efficient to use in the
case where like the guile example above, where the packages you run guix pack
with have exactly matching inputs.

This could often be the case if you're developing an application, packaging it
with Guix, then using guix pack to generate a Docker image which you
deploy. With the single layer approach, if you change the application code,
you'll get an entirely different image. I haven't tried this out, but my hope
is that by generating a common base layer, if you change the application code
only the top layer of the Docker image will change, meaning you'll only have
to deploy that, rather than having to deploy the entire image. If you're
deploying the images across a network, having less data to send around can
save time, and reduce the amount of space required to store the images.

As well as these behaviour changes, these patches also modify the
implementation. Rather than having some build side code that's used in the
pack and vm module gexpressions, these patches introduce two new record types:
<docker-image-layer> and <docker-image>. This at least structures the
derivations so that each layer is represented by a derivation, and then
there's a derivation for the image itself, which is a little more efficient in
terms of computation.

What do people think about generating multi-layer images, and using record
types to represent the layers and image?

Thanks,

Chris

[PATCH 1/3] Rename (guix docker) to (guix build docker)
[PATCH 2/3] Make guix pack work with the new docker image
[PATCH 3/3] Generate two layers for docker images in guix pack