From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id CP/UBSES0F6KbAAA0tVLHw (envelope-from ) for ; Thu, 28 May 2020 19:33:53 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id QBy2ASES0F6qUAAAB5/wlQ (envelope-from ) for ; Thu, 28 May 2020 19:33:53 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id E54CA9400B1 for ; Thu, 28 May 2020 19:33:51 +0000 (UTC) Received: from localhost ([::1]:52522 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jeOHm-0005Zx-RZ for larch@yhetil.org; Thu, 28 May 2020 15:33:50 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48510) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jeOHU-0005Ut-HR for help-guix@gnu.org; Thu, 28 May 2020 15:33:32 -0400 Received: from wout4-smtp.messagingengine.com ([64.147.123.20]:48193) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jeOHR-0004Lt-EC for help-guix@gnu.org; Thu, 28 May 2020 15:33:32 -0400 Received: from compute2.internal (compute2.nyi.internal [10.202.2.42]) by mailout.west.internal (Postfix) with ESMTP id 7FBE3B6C; Thu, 28 May 2020 15:33:26 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute2.internal (MEProxy); Thu, 28 May 2020 15:33:26 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=famulari.name; h=date:from:to:cc:subject:message-id:references:mime-version :content-type:in-reply-to; s=mesmtp; bh=NCf2ugxJAFuacKjuQC+nEkoL 7l7ko81JcZjggZpQbIs=; b=gYqDo6qDTiPhjpCMdQbHIWKewNwVlhyxoRSazmzo RTUo0yjDFWJotmF9l/EOaFtxr108m7FpqAFywVS2YN10w/IS0XfuFfrBLpzDW6Zo CXRGrH1m5LlT00U/5OsFfSppK1xs75ZCYANvKdqS+UuQEIu1yP4ECZcykdavujwY nvk= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; bh=NCf2ug xJAFuacKjuQC+nEkoL7l7ko81JcZjggZpQbIs=; b=of3CSEW/sVErGcFFjcHJQy a28wjGy/3RvsK03YoWfXs7TSXflc1yzVnMup/mbSgvt+cXRF2A9w2jF8dZhGUZ8T YgVu0joHVuCnnIOlc0eJYTEy+SML9Y31tud5I2EZyJSIR2FPTwjmYXiMFgK3N1vv vMV3ik6Ko21JTDkvOmVY220FxsrQcA1hONZZw3ZfQmNyKwOhs/5BI1Ygq3EwCCG4 4ljIxr/SARfqI8bX19946zQaeWHdgAwd0Cgnqx7mAAELQnN+5vr70Is80LYjwzVV pEc4lkhMXKsKYrDsBB0QK7U4MFtSgmoqdrd20ckH1uuSCyCu4BH73SkC7pc2AsBw == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduhedruddviedgkeegucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpeffhffvuffkfhggtggujgesthdtre dttddtvdenucfhrhhomhepnfgvohcuhfgrmhhulhgrrhhiuceolhgvohesfhgrmhhulhgr rhhirdhnrghmvgeqnecuggftrfgrthhtvghrnheptdeutdegjefgudelkefgkedtkeeive dtvdegkeekgffffeeuvdejtdeutdduieehnecuffhomhgrihhnpeguohgtkhgvrhdrtgho mhdpghhithhlrggsrdgtohhmnecukfhppeejiedruddvgedrudefkedrieefnecuvehluh hsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomheplhgvohesfhgrmhhu lhgrrhhirdhnrghmvg X-ME-Proxy: Received: from localhost (c-76-124-138-63.hsd1.pa.comcast.net [76.124.138.63]) by mail.messagingengine.com (Postfix) with ESMTPA id C1B293060FE7; Thu, 28 May 2020 15:33:25 -0400 (EDT) Date: Thu, 28 May 2020 14:10:43 -0400 From: Leo Famulari To: Stephen Scheck Subject: Re: Guix Docker image inflation Message-ID: <20200528181043.GC23745@jasmine.lan> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Received-SPF: pass client-ip=64.147.123.20; envelope-from=leo@famulari.name; helo=wout4-smtp.messagingengine.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/05/28 15:33:27 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: help-guix@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: help-guix Errors-To: help-guix-bounces+larch=yhetil.org@gnu.org Sender: "Help-Guix" X-Scanner: scn0 Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=famulari.name header.s=mesmtp header.b=gYqDo6qD; dkim=pass header.d=messagingengine.com header.s=fm2 header.b=of3CSEW/; dmarc=none; spf=pass (aspmx1.migadu.com: domain of help-guix-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=help-guix-bounces@gnu.org X-Spam-Score: -0.21 X-TUID: I68zxR/Fd1IG On Wed, May 27, 2020 at 03:41:49PM -0400, Stephen Scheck wrote: > As an exercise, I set up daily Guix System Docker image builds using GitLab > and Docker Hub, here: > https://hub.docker.com/repository/registry-1.docker.io/singularsyntax/guix/tags?page=1 Cool! > The build process works as follows: if an existing `latest` image does not > exist for a given branch (master, 1.1.0, etc.), then bootstrap an image by > running `guix system docker-image` inside an Alpine Linux Docker container > with a fresh Guix installation. Using this image as a seed, `guix pull` is > run for the desired branch, and the resulting image is committed to the > Docker repository. If a "latest" image does exist, it is used instead as > the base from which to run `guix pull`. Daily images are thus built > incrementally from the previous day's build. For anybody curious about the > process, the build script can be browsed here: > https://gitlab.com/singularsyntax-docker-hub/guix/-/blob/master/.gitlab-ci.yml I'm not familiar with Docker so I'm not sure exactly what you are doing. Specifically, I can't tell if you are creating new Docker images from scratch each day, or if you are continuing to use the same one from day to day. > It works pretty well, except that I'm observing substantial image size > inflation day-over-day, starting at ~197 MB from the seed image, now up to > 1.71 GB eleven days later despite running `guix gc --delete-generations`, > `guix gc --collect-garbage`, and `guix gc --optimize` after pulling prior > to committing each new image. I'm also not sure which image is growing each day... In general, the parameters --delete-generations and --collect-garbage are supposed to be passed values like a reference to a profile or an amount of data to delete, respectively. Are you doing that? Are you removing / invalidating old generations before attempting to garbage collect them? The store items they refer to cannot be deleted until the generations themselves are no longer registered. You can list existing generations with e.g. `guix package --list-generations`. You can invalidate them with `guix package --delete-generations=42` or with time-based patterns like `guix package --delete-generations=1m`, which removes everything older than one month. The same-named argument to `guix gc` should be shorthand for that. Similarly for the profile used by `guix pull`, which is accessed like this: `guix package --profile=$HOME/.config/guix/current --list-generations`. Usually, these old profiles are responsible for most of the disk usage in /gnu/store.