From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp12.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms9.migadu.com with LMTPS id uET4Hqs0EGRN4wAASxT56A (envelope-from ) for ; Tue, 14 Mar 2023 09:47:39 +0100 Received: from aspmx1.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp12.migadu.com with LMTPS id ECzXHqs0EGQbbQAAauVa8A (envelope-from ) for ; Tue, 14 Mar 2023 09:47:39 +0100 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id F27332AE0E for ; Tue, 14 Mar 2023 09:47:38 +0100 (CET) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pc0JE-0004uS-Kk; Tue, 14 Mar 2023 04:47:04 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pc0JD-0004uJ-4A for guix-patches@gnu.org; Tue, 14 Mar 2023 04:47:03 -0400 Received: from debbugs.gnu.org ([209.51.188.43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pc0JC-0004M3-Om for guix-patches@gnu.org; Tue, 14 Mar 2023 04:47:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1pc0JC-0002BM-LW for guix-patches@gnu.org; Tue, 14 Mar 2023 04:47:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#62153] [PATCH 1/2] guix: docker: Build layered image. Resent-From: Simon Tournier Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Tue, 14 Mar 2023 08:47:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 62153 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Oleg Pykhalov Cc: Josselin Poiret , Tobias Geerinckx-Rice , Simon Tournier , Mathieu Othacehe , Ludovic =?UTF-8?Q?Court=C3=A8s?= , Christopher Baines , Ricardo Wurmus , 62153@debbugs.gnu.org Received: via spool by 62153-submit@debbugs.gnu.org id=B62153.16787835878334 (code B ref 62153); Tue, 14 Mar 2023 08:47:02 +0000 Received: (at 62153) by debbugs.gnu.org; 14 Mar 2023 08:46:27 +0000 Received: from localhost ([127.0.0.1]:36212 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pc0Id-0002AK-0d for submit@debbugs.gnu.org; Tue, 14 Mar 2023 04:46:27 -0400 Received: from mail-wm1-f49.google.com ([209.85.128.49]:51993) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pc0Ia-0002A1-PG for 62153@debbugs.gnu.org; Tue, 14 Mar 2023 04:46:25 -0400 Received: by mail-wm1-f49.google.com with SMTP id ay8so4407867wmb.1 for <62153@debbugs.gnu.org>; Tue, 14 Mar 2023 01:46:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1678783578; h=mime-version:message-id:date:references:in-reply-to:subject:cc:to :from:from:to:cc:subject:date:message-id:reply-to; bh=S486eBDbmvoSKhNJhX2qss9v2rMcF0NLI7lr4vI/t+k=; b=Hlc7mhiCWn7cms0N0/EP+OZXQPc18hMyZySRrZgWxx1C6Wou0qxa/okhGnnfFjJx1s 38OFZM6FfWJzuiSgOCO3liiL5bEvjSUwJ912MiqgnN+8kclwDba2jf4mGeYW8kP5dsxT UPy6kNhnvaoVeYPevOsTgmz2BDudEznneGgXcWgj6L3j9amf3s9VL3vyKXbAwYwy8ilI dm60dBx+BrMKA/qfVvMXa59OW+gDMUedZLDXgUuLWJq3ZdxfwWLeF9/zmiW+CDjOyv4S BqZnjWqFKa3a/HRF7+4nqKmHKtlYVZXWnxIreQYs9tqeMOiQKARKjDupxwJRvkInWmCP W5Ng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1678783578; h=mime-version:message-id:date:references:in-reply-to:subject:cc:to :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=S486eBDbmvoSKhNJhX2qss9v2rMcF0NLI7lr4vI/t+k=; b=AjZSlm+zYnNBQCyaCRCEoSOwCgtGJvJYolKW5lNm0uULtrj2lBw1/8VjWzy1YDYufy /ywgGylEob/6yg2SBdkJEh5m+M4HANSWE24NP8aGVsjk7QENapJM2j7gD6qJd7LdAnZV RhBwyrHb12sMNZ4kxMPtlNEtlxO2l/EuNEOWM/cfeiGbANlTKYs/6ciYIgaze9TQfoiC YeRZ0TGa/xqG0e7hYR4vsYKRJBkQ4WrBTuL6J9/ZCHcAQQRKuqRcoHigAUS0BT6CMULu BnndwbYNvyr8TZhzljroG0oP0Uuv6MTxOzkJIJGv3fA6KHEZyHPqj+3Nt02atgqJU0yQ +dmg== X-Gm-Message-State: AO0yUKX/0iCALcefmb5EX+mj9wlhV9BTvT8uAUA716RleO4+GbbSsg6R u9V6CDqZWThwGd+SN2p7CSw= X-Google-Smtp-Source: AK7set8RYZlUWNysqYMHyruhhv+fvdFYr2kWtzPk/OjLAIl0mTKgR15MO2RoauO2ucCOlFycUMOBTw== X-Received: by 2002:a05:600c:3b15:b0:3eb:3986:9c03 with SMTP id m21-20020a05600c3b1500b003eb39869c03mr12067971wms.4.1678783578514; Tue, 14 Mar 2023 01:46:18 -0700 (PDT) Received: from lili ([2a01:e0a:59b:9120:65d2:2476:f637:db1e]) by smtp.gmail.com with ESMTPSA id f24-20020a1cc918000000b003e203681b26sm2140678wmb.29.2023.03.14.01.46.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 14 Mar 2023 01:46:18 -0700 (PDT) From: Simon Tournier In-Reply-To: <87bkkw2w7z.fsf@gmail.com> References: <20230313003012.14325-1-go.wigust@gmail.com> <20230313003310.17129-1-go.wigust@gmail.com> <87r0tsk85r.fsf@gmail.com> <87bkkw2w7z.fsf@gmail.com> Date: Tue, 14 Mar 2023 09:19:51 +0100 Message-ID: <86cz5b923c.fsf@gmail.com> MIME-Version: 1.0 Content-Type: text/plain X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: guix-patches-bounces+larch=yhetil.org@gnu.org X-Migadu-Country: US X-Migadu-Flow: FLOW_IN ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=gmail.com header.s=20210112 header.b=Hlc7mhiC; spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org"; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none) ARC-Seal: i=1; s=key1; d=yhetil.org; t=1678783659; a=rsa-sha256; cv=none; b=DwTEUEdNYxo/ZqlNV5Hnw8cI3NK4qbCNCxoHX6IZyXkhuC8sKl4KyyIdwNX57shUHQ0lda c9OaISqVWYDHz05r39nV/S9oJRwvnDPXlbPV/lSpkxTmz7rA4E56cr1jaMcxgMxr6kxh0Q O7p1oBc/Xs+8rZ6hMQOBgLCjoQOh1T7Lp0KA9SbATsWYEAyXJz7roy+MgCQopj6mNuzoN7 7m/yMBnbrMrr9CeJ8/Z2brTUQYWaillxbaADWUDfT4eHdxx1FFDVwake7UIM2YoQqHjhCL C8g4gG4PYDlOI/69ewOGnqmVLhAMkYH5QkkPvjw48Bf9UTqUwG5JI1/IGMHI4g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1678783659; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:resent-cc:resent-from:resent-sender: resent-message-id:in-reply-to:in-reply-to:references:references: list-id:list-help:list-unsubscribe:list-subscribe:list-post: dkim-signature; bh=S486eBDbmvoSKhNJhX2qss9v2rMcF0NLI7lr4vI/t+k=; b=pLTIRSBs9Ak1CYLQj/QmsBa+nhQ8echsDs+3CJcvj7oujcZ4ooqULO278F4npi87ew7ahO f1vBzHo3qfhfN/r3J61qWz7xfp0wBOXMh3j1mgyBVGBvFXnPBNhr+FMqy/75CWygxS9DyV zUAPTLKGlVCbYd0F0/aeWdz0o13p9CxgWKVL+pT9OPKz/a6DmKMlfQqpasSJb1qLGvBzpq qNms3C7pxEO48d12Qy3d7kcuQllcH9E+0DRpS7ChfFRM0IoqgWYASQ41URTqWQjfbz0oHg qTy7CvJeCB8udm/bIBY2zMZ5fHhylMMCFMX2bEkO1K/6RHzQGG2ir310IdbRyQ== X-Migadu-Spam-Score: 0.88 X-Spam-Score: 0.88 X-Migadu-Queue-Id: F27332AE0E X-Migadu-Scanner: scn1.migadu.com Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=gmail.com header.s=20210112 header.b=Hlc7mhiC; spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org"; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none) X-TUID: +8fpmtwlxvzJ Hi Oleg, CC: core teams On Tue, 14 Mar 2023 at 00:10, Oleg Pykhalov wrote: >>> diff --git a/gnu/packages/aux-files/python/stream-layered-image.py b/gnu/packages/aux-files/python/stream-layered-image.py >>> new file mode 100644 >>> index 0000000000..9ad2168c2d >>> --- /dev/null >>> +++ b/gnu/packages/aux-files/python/stream-layered-image.py >>> @@ -0,0 +1,391 @@ >>> +""" >>> +This script generates a Docker image from a set of store paths. Uses >>> +Docker Image Specification v1.2 as reference [1]. >> >> Instead of Python, would it possible to implement in Guile? I mean, >> does Python have something that is missing in Guile? >> >> The facility for manipulating Tar? Something else? > > I think nothing else. As I understand Python implemented Tar inside the > language itself in 2500 lines of code by manipulating binary data. > > /gnu/store/...-python-3.9.9/lib/python3.9/tarfile.py > > Technically it's probably possible to use tar utility with --append flag > instead of opening a new file and streaming to it as the Python script > does. To be honest I would like not to write it in this way if the > Python script does not block current patch for merge. Ok, thanks for explaining. > Also I don't see myself writing Tar implementation in Guile, yet. ;-) Maybe not reimplementing Tar in Guile, maybe just enough for working. Or maybe some Guile bindings. Or maybe something is already around for the bootstrap story. The use of external tools as Python for producing built-in Guix feature will be the first time, no? For what it is worth, I would prefer to consider the options before emitting an opinion about dragging Python building/packing layered Docker. :-) > The Nix project uses this script heavily to build layered images, so it > should be robust in terms of up to date to current Tar and Python > implementations. Do you mean this script is coming from Nix. Well, in all cases, this script is not trivial and so it requires Copyright for authorship. [...] > The following example shows common layers between images, which will be > not tranfered if you load image inside Docker as well as pull and push: Thanks for explaining. Cheers, simon PS: I will be off-line these 2-3 next weeks. So a lack of an answer from me will not be a lack of interest. ;-)