Re: Guix Docker image inflation

[Chris Marusich <cmmarusich@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 3114 bytes --]

Stephen Scheck <singularsyntax@gmail.com> writes:

> Hello,
>
> As an exercise, I set up daily Guix System Docker image builds using GitLab
> and Docker Hub, here:
> https://hub.docker.com/repository/registry-1.docker.io/singularsyntax/guix/tags?page=1
>
> The build process works as follows: if an existing `latest` image does not
> exist for a given branch (master, 1.1.0, etc.), then bootstrap an image by
> running `guix system docker-image` inside an Alpine Linux Docker container
> with a fresh Guix installation. Using this image as a seed, `guix pull` is
> run for the desired branch, and the resulting image is committed to the
> Docker repository. If a "latest" image does exist, it is used instead as
> the base from which to run `guix pull`. Daily images are thus built
> incrementally from the previous day's build. For anybody curious about the
> process, the build script can be browsed here:
> https://gitlab.com/singularsyntax-docker-hub/guix/-/blob/master/.gitlab-ci.yml
>
> It works pretty well, except that I'm observing substantial image size
> inflation day-over-day, starting at ~197 MB from the seed image, now up to
> 1.71 GB eleven days later despite running `guix gc --delete-generations`,
> `guix gc --collect-garbage`, and `guix gc --optimize` after pulling prior
> to committing each new image.
>
> I'm wondering if there is some other Guix GC operation or option I'm
> missing, or any other suggestions which could stop this unsustainable image
> bloat from occurring. I really do doubt that the Guix System itself is
> growing this quickly.

Could it be that you are accumulating layers without bound?

https://developers.redhat.com/blog/2016/03/09/more-about-docker-images-size/

Since Docker images are built up of immutable layers, if you build your
image from an existing base image, I'm not sure that it's possible to
produce a new image that is smaller than the base image.  Basically,
even if you run "guix gc" to remove dead store items, they will still
exist on a prior layer, so the size of the new image won't decrease.
And since you're installing new things, the size will actually increase.
If you repeat this process by using the new image as an input for yet
another build, I think you will accumulate layers and storage space
without bound.

If this is what's happening, you might consider always building starting
from the same base image every time.  You could then update the base
image (e.g., by changing the FROM line of a Dockerfile, if that's what
you're using) periodically as new versions of it are released.  This
would probably allow you to avoid accumulating layers without bound.

FYI, Guix itself can build Docker images from scratch - no base image
required!  It can even build a Docker image of a full-blown Guix System
from scratch.  Sorry if you already knew that - I just wanted to point
it out in case you didn't!

See:

https://guix.gnu.org/manual/en/html_node/Invoking-guix-pack.html
https://guix.gnu.org/manual/en/html_node/Invoking-guix-system.html

Hope that helps,

-- 
Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]