From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1.migadu.com ([2001:41d0:1008:1e59::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms1.migadu.com with LMTPS id uAadIaZxW2Z58QAAA41jLg (envelope-from ) for ; Sat, 01 Jun 2024 21:08:22 +0200 Received: from aspmx1.migadu.com ([2001:41d0:403:4876::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1.migadu.com with LMTPS id mCorHKZxW2YLKQAA62LTzQ (envelope-from ) for ; Sat, 01 Jun 2024 21:08:22 +0200 X-Envelope-To: larch@yhetil.org Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=elephly.net header.s=zoho header.b=U6T8c3Ii; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org"; dmarc=none; arc=pass ("zohomail.com:s=zohoarc:i=1") ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1717268902; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=xGvjm10BMfc2bKZAk9rhv1+Xq+/x9jAxv+FqmnkgEPY=; b=i81iKSDmtgoy7QHeu/iLWTWw3PrSYPlCEVCkaigrjjTHiJHg8P57PLDT9qNJ3o2mFVmWFM eIfyuhodF71m4+21Zsy96SEcUTHK7X6ud25r1sSfOEuobsnyaKIaBcS9rXgCZUS3jg7aC1 N8AkKdnvtQpi3zBw4i9f4yRxDTdQXOTDTBE+ORIomzM5kkURSNhWl7OatDFv+usM9A0Fmy 3whOp8RrUIqzFM+PlpNHZWlghaD7PD10XnqrBjM3z/V9eZHOUUUjNB5QwSrCJxIWhrwTLy jZFTht1ciicmeclr2tW+MyhipifHibydgBUigxvYDcP/9uvHos9k/uLdgpg7Uw== ARC-Authentication-Results: i=2; aspmx1.migadu.com; dkim=pass header.d=elephly.net header.s=zoho header.b=U6T8c3Ii; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org"; dmarc=none; arc=pass ("zohomail.com:s=zohoarc:i=1") ARC-Seal: i=2; s=key1; d=yhetil.org; t=1717268902; a=rsa-sha256; cv=pass; b=ddgHdc/MOGVq8MffrXq4hDj9VpMibamDrrwcpA9j3H7S6DwP7zQSvSwkC5ao8E20JnO0QJ iCVitNteZ9445le1FxLooHkuIuJcUeWTl2hLbDbV7wOEOzt30HvGSaUkGQnxPdbtrzfUnV snxkGiINnHoIfD1yC77tBYMqF0NargEhlRS9KllXMt/5r1/n4ue8vnKyFTlUeTuMEmeLxI ZWcpi69uW9YnZ5y0BxbqP8gNqKQyUAmGdSwCYNrZ7kr8QDu4dYD7UWZK/81wB099NhgFbd ZWT/odvCBxk1HzyhJewsJHh00gX3UQdbYEheXkKK+KVc9VuqOllecCGdXBH9LA== Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 6959026379 for ; Sat, 1 Jun 2024 21:08:21 +0200 (CEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sDU56-0007Xz-1f; Sat, 01 Jun 2024 15:07:56 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sDU54-0007XN-1v for guix-devel@gnu.org; Sat, 01 Jun 2024 15:07:54 -0400 Received: from sender4-of-o51.zoho.com ([136.143.188.51]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sDU50-00077B-NV; Sat, 01 Jun 2024 15:07:53 -0400 ARC-Seal: i=1; a=rsa-sha256; t=1717268867; cv=none; d=zohomail.com; s=zohoarc; b=kfNZS58sBs9glcau7RIx7Reww63eMGWEwIfaC21qRe9LLAtUdJrR23nugTLTKn9rCmzzm5IX8E33vpKN736OGv5eFwkqQFjc+28bS4hUC6VRlgRFPMTJMaYzVKmFp35Knaj/uWihcg5P/1iwAcvz7Famyk+wVgbwAY5X96DvXoY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1717268867; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:Subject:To:To:Message-Id:Reply-To; bh=xGvjm10BMfc2bKZAk9rhv1+Xq+/x9jAxv+FqmnkgEPY=; b=RMCLmLDaJPbi7mx3CEnuCuOHCh13jtaPtT2C++mFCfA/aYAuDvq5vluVFm0hSy2FY+TXUkKGMq+a04cR2/6Bbm+AmAV9DFRH5LSrjhygB8avmwCIVv4t6uLLfJSLA2S3DUYv3Rpzi9uXosZkOiAyDOKzKGKtj5fBbierkUA4Dbk= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=elephly.net; spf=pass smtp.mailfrom=rekado@elephly.net; dmarc=pass header.from= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1717268867; s=zoho; d=elephly.net; i=rekado@elephly.net; h=From:From:To:To:Cc:Cc:Subject:Subject:In-Reply-To:References:Date:Date:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding:Message-Id:Reply-To; bh=xGvjm10BMfc2bKZAk9rhv1+Xq+/x9jAxv+FqmnkgEPY=; b=U6T8c3Iil/I9v2pyIaJoRg8cnTftlroV5NDUZ4zULZsh4Qc9flgVc2lo+JIgpC5L eUEfwLOMvTSq0ORUv2Hi+E7cD9Fe+3RSwG3aiuO8jwaT+D1wPb5UzZ7Y2Sz+pIMmwrn 0/lhPkJl4Slaz86X6qztU/cwcJSj1ZKyOQei5/70= Received: by mx.zohomail.com with SMTPS id 1717268866536264.0470435829002; Sat, 1 Jun 2024 12:07:46 -0700 (PDT) From: Ricardo Wurmus To: Ludovic =?utf-8?Q?Court=C3=A8s?= Cc: guix-devel@gnu.org Subject: Re: "guix pack -f docker" does too much work In-Reply-To: <87cyp0ojc5.fsf@gnu.org> ("Ludovic =?utf-8?Q?Court=C3=A8s=22'?= =?utf-8?Q?s?= message of "Sat, 01 Jun 2024 15:58:50 +0200") References: <87sey0lqpn.fsf@elephly.net> <87cyp0ojc5.fsf@gnu.org> User-Agent: mu4e 1.12.4; emacs 29.3 Date: Sat, 01 Jun 2024 21:07:43 +0200 Message-ID: <87plt0jxc0.fsf@elephly.net> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-ZohoMailClient: External Received-SPF: pass client-ip=136.143.188.51; envelope-from=rekado@elephly.net; helo=sender4-of-o51.zoho.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: guix-devel-bounces+larch=yhetil.org@gnu.org X-Migadu-Country: US X-Migadu-Flow: FLOW_IN X-Spam-Score: -10.39 X-Migadu-Queue-Id: 6959026379 X-Migadu-Scanner: mx10.migadu.com X-Migadu-Spam-Score: -10.39 X-TUID: gc2etpT+LEj9 Ludovic Court=C3=A8s writes: >> I think it would be great if "guix pack -f docker" could avoid building >> all these identical layers again and again. Perhaps it would be >> possible to have a single derivation for each layer? This way we >> wouldn't have to recreate the same layer archives every time. > > That sounds nice in terms of saving CPU time. It=E2=80=99s less nice in = terms > of disk usage: a single =E2=80=98guix pack -f docker=E2=80=99 run would p= opulate the > store with roughly twice the size of the closure. Arguably we don't actually care all that much for the Docker image that ends up in the store. It's really a temporary thing that we want to load into Docker or upload somewhere else. I've often wanted to stream the eventual output of "guix pack" to a pipe, precisely because I don't want to store the same thing twice: once in the store and once in the Docker storage backend. It's actually worse than that: I often end up having dozens of packs in the store whose layers are almost all identical. > I think each solution (single derivation vs. one derivation per layer) > makes a different tradeoff. I don=E2=80=99t have a strong feeling about = which > one is better. Can we have both? I realize that adding the option to stream build output to a pipe is not a trivial change, but it would solve the unnecessary storage requirement for packs. "docker load" reads from standard input, but other packs would also benefit from a streaming output; an example is Docker-free deployment to a remote server: just pipe "guix pack" to a remote tar process and you're all set. --=20 Ricardo