From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id cHLgKOdM0V7bcQAA0tVLHw (envelope-from ) for ; Fri, 29 May 2020 17:56:55 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0 with LMTPS id AEC+JOdM0V5PQQAA1q6Kng (envelope-from ) for ; Fri, 29 May 2020 17:56:55 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 22BF3940144 for ; Fri, 29 May 2020 17:56:55 +0000 (UTC) Received: from localhost ([::1]:38494 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jejFT-0007Pt-7l for larch@yhetil.org; Fri, 29 May 2020 13:56:51 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49114) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jejFL-0007Oo-Ie for help-guix@gnu.org; Fri, 29 May 2020 13:56:43 -0400 Received: from mail-lf1-x12d.google.com ([2a00:1450:4864:20::12d]:42944) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jejFK-0003WM-Fv for help-guix@gnu.org; Fri, 29 May 2020 13:56:43 -0400 Received: by mail-lf1-x12d.google.com with SMTP id x27so186115lfg.9 for ; Fri, 29 May 2020 10:56:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=rEqyxpNy50W8DYCHirskEwb55w5sfUjwWdDfRbQo6yY=; b=aJ7QcWjND5rAoq+mbM8jbwl4qX/Nsy7potfuYu1U1r12E3GDO6YOdPh+7MNOIlKfkN 5PwGhpPTDsIoMaESOdt96H9gSuiKkpfPecQSNEzQsNi2LmK+5I/pcpbOVXHPVSI6SOX8 4AZBY49W//5yW3B9CkqW6qVfYfu3bMGNnoUx3ongaEOvX8+PmnSed7Y46KpoAgBH2A45 5INv/gKBapjus3pSrFQWq56crhNLk1cfTcCcz+2Vi2aG6a80UciKRaRrtRc/v/VnIhjq xiMkDJUy/GjF1fFQSMTsJacEwBpXNSjXHtIqmPlsaPD3kHPfPSJFDPgJAoQ5ca+LDbv8 ujyQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=rEqyxpNy50W8DYCHirskEwb55w5sfUjwWdDfRbQo6yY=; b=Ju/KKmun/oHg05w2AiqBN8rrwS5fj09PxlxZfJHU/jNiJcNiy0HueuB4WQWPMHNBC4 RRgtFDA8pktYslPPXKnQnW8evrZM0UxZVUPXWw0qg4XFX+4wjuMP6xzn/TMdBpLFK3qP 7ITrw26l5CKkdqXor9HNbFL8TwCQf3biELvTXr7Ii1Ko0xSGCRWTY8L+QZX6ZqBjC7sl PnUWCepCNJZvKzZ3jWqdPY6r3GMlxeIFDRczUOxsIqF4OUAwpalV28kYnzxKmRgwpb0e 3ragD9uSqK0BbLpCPJOVlcAkeVNdJ+zCHLu0abvlY+UQuvYpigE+zBaC1Quy08DaRiEl T5Vw== X-Gm-Message-State: AOAM533dGfYOGQaia4POi16ZJ78x+bIhyfKw795/RVu+0JB7FbiC5H01 g9Dsgar4trmHjMY/4w1skEv96Cv+WzHffHSxq9U= X-Google-Smtp-Source: ABdhPJwFu2okmcubE2udeMgbcIRJMDot6oTdnfxox0ZAfx26mIeanV0EeXHciMySVuPYZSSSb8E7o8EpUgjr0QcSI14= X-Received: by 2002:a05:6512:203:: with SMTP id a3mr5107267lfo.28.1590775000280; Fri, 29 May 2020 10:56:40 -0700 (PDT) MIME-Version: 1.0 References: <20200528181043.GC23745@jasmine.lan> <20200529170820.GA30828@jasmine.lan> In-Reply-To: <20200529170820.GA30828@jasmine.lan> From: Stephen Scheck Date: Fri, 29 May 2020 13:56:28 -0400 Message-ID: Subject: Re: Guix Docker image inflation To: Leo Famulari Received-SPF: pass client-ip=2a00:1450:4864:20::12d; envelope-from=singularsyntax@gmail.com; helo=mail-lf1-x12d.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: help-guix@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: help-guix Errors-To: help-guix-bounces+larch=yhetil.org@gnu.org Sender: "Help-Guix" X-Scanner: scn0 Authentication-Results: aspmx1.migadu.com; dkim=fail (body hash did not verify) header.d=gmail.com header.s=20161025 header.b=aJ7QcWjN; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none); spf=pass (aspmx1.migadu.com: domain of help-guix-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=help-guix-bounces@gnu.org X-Spam-Score: 0.09 X-TUID: nvdKMqn9rfGK On Fri, May 29, 2020 at 1:08 PM Leo Famulari wrote: > I'm still not quite sure what you are doing (or what Docker does) so > please bear with me. > > > root@localhost /# du -h --max-depth=1 /gnu/store | egrep > > "guix-system$|guix-packages-base$|guix-[0-9a-f]*-modules$" > [...] > > 191M > /gnu/store/l3amdz5xyhflg5wdzlxr2685dq5glic2-guix-527ab3125-modules > > 201M > /gnu/store/5mhn1ynxvy7jihsknsnv3yspkkvc0r5s-guix-2e59ae238-modules > > If I understand correctly, you should not need both of these directories > in a Guix VM image. The latter hashes are truncated guix.git commit > hashes and a VM image would only be based on a single one. > Exactly, I agree (to the extent that I understand Guix). I recommend looking into why all these directories are being copied into > your images. > Whatever is in /gnu/store (as managed by Guix) goes into the image, nothing more and nothing less. > > I figure you'd want to create each image with *only* the things > corresponding to the Git commit it's based on, but it sounds like they > are being created by copying the entire host image, which doesn't seem > right. > > If the Docker images are being created by simply snapshotting the file > system of a non-ephemeral Guix system, that's probably not the right way > to do it. Is that what's going on? > Yes, as I said, the image is created from a file system snapshot, after Guix is brought up to date via `guix pull` and those various Guix garbage collection operations are run. However, it's not quite "non-ephmeral" as each Guix operation is run as an atomic command inside the Docker container, with nothing else running (except for guix-daemon, which has to always be running for Guix to operate to the best of my understanding, and a couple other Guix System daemons which anyway would be equivalent to the situation to any Guix installation running outside of a Docker container). How else would you suggest that it be done? It would be nice if `guix system docker-image` took `--branch` and `--commit` options to build a container from a well-defined Guix check-in state, but that doesn't seem to be the case. And in any case - too slow. The point here is to leverage daily incremental pulls to keep data transfer and build times down.