all messages for Guix-related lists mirrored at yhetil.org
 help / color / mirror / code / Atom feed
From: Antoine Eiche <lewo@abesis.fr>
To: Simon Tournier <zimon.toutoune@gmail.com>,
	Ricardo Wurmus <rekado@elephly.net>
Cc: guix-devel@gnu.org
Subject: Re: Building container images with nix2container
Date: Sun, 17 Mar 2024 18:32:10 +0100	[thread overview]
Message-ID: <8734sokbmt.fsf@tilia> (raw)
In-Reply-To: <87cys5817b.fsf@gmail.com>

Simon Tournier <zimon.toutoune@gmail.com> writes:

>> Well, I have not followed on which strategy Guix relies.  What is the
>> one of nix2container?  The one described here:
>>
>>     https://grahamc.com/blog/nix-and-layered-docker-images/
>
> To answer to my question, the way to build the container image is
> different, hence it does not make much sense to speak about a
> “strategy“. :-)

nix2container actually uses this strategy. To build an image,
nix2container builds several derivations. Each of these derivation
contains a set of layers. The layers of each set are determined by using
the algorithm mentionned above. At runtime, all of these derivations are
packed to build the final image.

> However, the blog post says:
>
>         To address this issue, we could add a nonReproducible option in
>         the containerTools.buildLayer function. Instead of only storing
>         the digest, we would also store the tar. Note in practice, an
>         important part of nixpkgs is bit reproducible and this would
>         rarely be needed.
>
> And so the question is how do you know beforehand if the flag
> ’nonReproducible’ must be applied or not?

Sorry, i have missed your mails:/ So, here is the answer I provided on
Mastodon few days ago.

Generally, i don't think it's possible to know when this flag is needed
before encountering an issue at image push time: Skopeo would then say
layer hashes mismatch.

However, in practice, it seems this flag is not often used.
To hit this issue, I think you need to substitute the JSON file from a
binary cache while building non bit reproducible store paths locally. In
this case, the hash specified in the JSON file could be different from
the hash of the derivation locally built. But, the locally built store
path is a dependency of the JSON file, they should also be in the binary
cache and would generally also be substituted.
I admit this is not rock-solid science... but it seems to make the job
in practice. And if it occurs, a strategy could be to isolate this
storepath into a layer with the non-reproducible flag.

There is also another option which is currently used by
nixpkgs.dockerTools.streamLayeredImage. To avoid the reproducibility
issue while still avoiding writing layer tarballs, we could introduce
another option to let nix2container generating the layer hashes on the
fly, just before pushing the image. This would then require more IOs
since nix2container would need to read layer store paths on each image
push.

> Indeed, the approach of nix2container could be helpful in addition to
> ‘guix pack’.  Maybe an extension… :-)

From my point of view, another advantage of the nix2container
approach is to delegate a important part of the build image mecanism
to Skopeo, a tool well maintained and used by a lot of people.

lewo.


  reply	other threads:[~2024-03-17 17:32 UTC|newest]

Thread overview: 8+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-02-24  9:27 Building container images with nix2container Antoine Eiche
2024-02-25 10:07 ` Ricardo Wurmus
2024-02-26 10:09   ` Antoine Eiche
2024-02-26 10:24     ` Ricardo Wurmus
2024-02-26 17:33     ` Simon Tournier
2024-03-08 10:31       ` Simon Tournier
2024-03-17 17:32         ` Antoine Eiche [this message]
2024-03-17 17:09       ` Antoine Eiche

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=8734sokbmt.fsf@tilia \
    --to=lewo@abesis.fr \
    --cc=guix-devel@gnu.org \
    --cc=rekado@elephly.net \
    --cc=zimon.toutoune@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this external index

	https://git.savannah.gnu.org/cgit/guix.git

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.