From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id GLfjEE6b0V5cGAAA0tVLHw (envelope-from ) for ; Fri, 29 May 2020 23:31:26 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0 with LMTPS id 2HDDDE6b0V50JAAA1q6Kng (envelope-from ) for ; Fri, 29 May 2020 23:31:26 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 7EA15940997 for ; Fri, 29 May 2020 23:31:25 +0000 (UTC) Received: from localhost ([::1]:34482 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jeoTC-00008m-PR for larch@yhetil.org; Fri, 29 May 2020 19:31:22 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:42288) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jeoSx-00005T-6Z for help-guix@gnu.org; Fri, 29 May 2020 19:31:07 -0400 Received: from mail-pf1-x42a.google.com ([2607:f8b0:4864:20::42a]:45003) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jeoSv-00061d-7b for help-guix@gnu.org; Fri, 29 May 2020 19:31:06 -0400 Received: by mail-pf1-x42a.google.com with SMTP id f3so550915pfd.11 for ; Fri, 29 May 2020 16:31:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version; bh=fxvtbahEpMvGEm9DI/PbRY37sKXvPorwfFjYLXxtAeE=; b=gxh5ThKu2gHLiUBY4zBO4pchGTPCEOaRhD2+V/haUQmNTo6q11E5uyM5lnV32Iwn1G mmYnejVPZ+S4pQo1lqlN7VTD1q7H7/2YZr1fYSW0O3NYw6dDAuxlKLYVXbKhmVYvUk4+ 8CYqwjPEf9GVGumvglzkx9HloTjSUNkRrDCiYfZDKO0ySiv8nDQZ7nzrM6LTgyX9Fy7w oxRYzSMuT0KgFiH5gZQeMRJMMULrKWJep9HSoRwQUf0IDly2tEHb635NcMW9ns0p+ExI n5BZD8TgIwtfKJZcmjrNqSHTTyyGQk+O5FgWs+eD3mgdmblGZwnFFKAduCAO0NtJQBye oJYA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version; bh=fxvtbahEpMvGEm9DI/PbRY37sKXvPorwfFjYLXxtAeE=; b=qmfInnVIvtPZ1ipbapu9aGEmvRQBegMLOAhnMo9c6Kflzuagb6tf/oHo911m84BfO2 8+X6wJkblpIbfseJcSZh+DQwN1Jq4DZWJgyvozqyH81eG0QKNh0klY5+c9zTI9SjHk7S mYCScrzVHcuyClDWAWKAVl3W5FzK5bHAZF4Blun3ztHL3QmNC3xi4As3JRWfRE9GYQjU zn6iXX3zBylz040WaZVoWrSMTCi/Ow/dw/EfkGnr6a1YYebmeHuExpDz1UCiFUqcXWr3 9VtkllYbL78UMxoEfTTPrhTkoz9DzIgYBE1Ya/SIg0LOe8pQkxDYWY/AEvipc4Ymytl3 +gWA== X-Gm-Message-State: AOAM533S5uT8foZIuJghgfz4zroUrVLU8WfgH46g8l9cWJ+JrysjzuO0 AcnqI2a1hX74ok7FQba/SEbjvEfgY6M= X-Google-Smtp-Source: ABdhPJyK5IAhBjcgyHamqZp/V3gmVBfQd1maFDnHDqzLtPMTn5zvxE0+wneW8E6LCduvWTgJwcXbOw== X-Received: by 2002:a63:4b0c:: with SMTP id y12mr10099604pga.56.1590795062495; Fri, 29 May 2020 16:31:02 -0700 (PDT) Received: from garuda-lan (c-73-97-103-127.hsd1.wa.comcast.net. [73.97.103.127]) by smtp.gmail.com with ESMTPSA id nl8sm451548pjb.13.2020.05.29.16.31.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 29 May 2020 16:31:01 -0700 (PDT) From: Chris Marusich To: Stephen Scheck Subject: Re: Guix Docker image inflation References: Date: Fri, 29 May 2020 16:30:56 -0700 In-Reply-To: (Stephen Scheck's message of "Wed, 27 May 2020 15:41:49 -0400") Message-ID: <87h7vyxqrz.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" Received-SPF: pass client-ip=2607:f8b0:4864:20::42a; envelope-from=cmmarusich@gmail.com; helo=mail-pf1-x42a.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: help-guix@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: help-guix Errors-To: help-guix-bounces+larch=yhetil.org@gnu.org Sender: "Help-Guix" X-Scanner: scn0 Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20161025 header.b=gxh5ThKu; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (aspmx1.migadu.com: domain of help-guix-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=help-guix-bounces@gnu.org X-Spam-Score: -0.31 X-TUID: PPF/cHigsA8X --=-=-= Content-Type: text/plain Content-Transfer-Encoding: quoted-printable Stephen Scheck writes: > Hello, > > As an exercise, I set up daily Guix System Docker image builds using GitL= ab > and Docker Hub, here: > https://hub.docker.com/repository/registry-1.docker.io/singularsyntax/gui= x/tags?page=3D1 > > The build process works as follows: if an existing `latest` image does not > exist for a given branch (master, 1.1.0, etc.), then bootstrap an image by > running `guix system docker-image` inside an Alpine Linux Docker container > with a fresh Guix installation. Using this image as a seed, `guix pull` is > run for the desired branch, and the resulting image is committed to the > Docker repository. If a "latest" image does exist, it is used instead as > the base from which to run `guix pull`. Daily images are thus built > incrementally from the previous day's build. For anybody curious about the > process, the build script can be browsed here: > https://gitlab.com/singularsyntax-docker-hub/guix/-/blob/master/.gitlab-c= i.yml > > It works pretty well, except that I'm observing substantial image size > inflation day-over-day, starting at ~197 MB from the seed image, now up to > 1.71 GB eleven days later despite running `guix gc --delete-generations`, > `guix gc --collect-garbage`, and `guix gc --optimize` after pulling prior > to committing each new image. > > I'm wondering if there is some other Guix GC operation or option I'm > missing, or any other suggestions which could stop this unsustainable ima= ge > bloat from occurring. I really do doubt that the Guix System itself is > growing this quickly. Could it be that you are accumulating layers without bound? https://developers.redhat.com/blog/2016/03/09/more-about-docker-images-size/ Since Docker images are built up of immutable layers, if you build your image from an existing base image, I'm not sure that it's possible to produce a new image that is smaller than the base image. Basically, even if you run "guix gc" to remove dead store items, they will still exist on a prior layer, so the size of the new image won't decrease. And since you're installing new things, the size will actually increase. If you repeat this process by using the new image as an input for yet another build, I think you will accumulate layers and storage space without bound. If this is what's happening, you might consider always building starting from the same base image every time. You could then update the base image (e.g., by changing the FROM line of a Dockerfile, if that's what you're using) periodically as new versions of it are released. This would probably allow you to avoid accumulating layers without bound. FYI, Guix itself can build Docker images from scratch - no base image required! It can even build a Docker image of a full-blown Guix System from scratch. Sorry if you already knew that - I just wanted to point it out in case you didn't! See: https://guix.gnu.org/manual/en/html_node/Invoking-guix-pack.html https://guix.gnu.org/manual/en/html_node/Invoking-guix-system.html Hope that helps, =2D-=20 Chris --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEy/WXVcvn5+/vGD+x3UCaFdgiRp0FAl7RmzAACgkQ3UCaFdgi Rp0kjA//WvogOmnGVkQh6mP+N4g/5ToXP14rI+yCgAQesanPdXj1rVljfH99mvII wRbTFQG6cUhSU2ThxpcgJIhiqwXHprdK6F/ramREtqVkvvEotDjDYJGNPKvixV4q h1RaDSqgwU6VH0p7wRWlEzh00gDmYGsND0t7r8T5j1kgPTqqF1D8RW+wrLB3k3O8 A1yxivWRA5P8vwoRaFp8E3BlESvCkydDV9JQ+SigzX6A6p1xqlyl295rnnQfqBy6 C+mBtrXcqG6Mpmkmoo2cDyaIZkmAF2rRTWrP0cuTnYrtwJzrNpow522sNLK84FhH RpjL6u+j4FXlnXZ4bsJshzD8moS7wsc51DJFXLnnj9teoD2SBTppkWhBejbNfjae 8I/Y24rm/StofIlo4SuBYoa8KzwW1ARzJ5L1i8Oh3qQu3Zc/t4OYqvmryhy5JbC9 cXEmYqaDm7ijfJAYVmmSID6/bluhk5Ob+bXABVopinR8CBS3XKVdLnL8rODH4X52 eY5ahGCQ1okUUyI1FP3G9lzlNF4tKVcp6hdDmmiqDJ8aTHBeeZqk3xt6G1iqOtsI epTmgrECLpxRCYC9hJAI7oGFZO3xN2OHxGczvHyjIunicG7knatA/Ow+q6skjVBu Y+btcj9UeqyRtaLkuotcR2+bdoGx5p5qy9WKAkKf2VDXztzYAhg= =PVJ+ -----END PGP SIGNATURE----- --=-=-=--