Relationship between Docker and Guix

Relationship between Docker and Guix

[Laura Lazzati]

Hi Guix!

I am preparing a talk to introduce the community in a meetup, and one of
the topics the people want to know about is its relationship with Docker.
I've read the manual, and have Ludo's talk from FOSDEM 2019, but if someone
more experienced than me can help me (maybe there is a blog post, another
talk, or through here) I would really appreciate it, because I am pretty
new to Docker.

Regards :)
Laura

Re: Relationship between Docker and Guix

[Ricardo Wurmus]

Hi Laura,

> I am preparing a talk to introduce the community in a meetup, and one of
> the topics the people want to know about is its relationship with
> Docker.

There is no direct relationship between the two.  They serve different
purposes and there’s little overlap, but there’s confusion about what
Docker can do ad that has often lead to people assuming that it does way
more for reproducibility than it actually does.

Docker and systems like it are a combination of a “disk image” (i.e. an
archive containing files and directories) and a tool to bind together
Linux kernel features that can be used for virtualizing parts of the
system.

There are different levels of virtualization.  One can emulate a CPU and
all hardware in software and run a full operating system on that fully
virtualized computer.  Or one can use special hardware support to avoid
having to implement the CPU in software.  Similar features exist to
avoid having to implement other hardware in software
(“paravirtualization”).  But traditionally with Linux that’s as far as
things would go: you’d still virtualize the whole computer, the whole
disk, and run a full operating system on it.

With the Hurd, on the other hand, virtualization was always fine grain.
You can, for example, redefine the directory tree as seen by a single
process (i.e. file system virtualization).  Or you could virtualize the
network interface and share everything else.

With Linux full-machine virtualization turned out to be a little too
much for many applications.  So eventually Linux gained features to
virtualize the process namespace (a process can think it runs alone on
the machine), the user account namespace (a process can think it runs as
root), cgroups to virtualize e.g. memory (a process can use *all* the
available memory, but the system pretends only a fraction of the memory
is “all the available memory”), etc.  With bind mounts and chroot (and
others) Linux also has a way to virtualize the file system (and
e.g. pretend that a certain directory is the root directory).

Docker coordinates all of these features and thus enables fine grain
virtualization for individual applications: applications can be
delivered as big disk archives that are unpacked, mounted, and then
declared to be the root file system from the point of view of the
application.

This makes application deployment really easy: just take a big disk
image that contains the application and *all* its dependencies, and
share the host system’s kernel, CPU, memory, and other file systems.

Some people thought that this ease of deployment also translates to
increased reproducibility: given the same disk image one could run an
application off that image without having any of the host system’s
libraries affect its operation.  This is hardly different from
old-school full system virtualization – it’s just more convenient and a
tad lighter.

Docker does not care about reproducibility of the disk images it uses.
Generating these images at different points in time usually yields
different disk images.  The images can also contain way more stuff than
needed.  There is no direct correspondence between the contents of the
disk image and the commands in a Dockerfile that may be used to generate
the disk image.  This can even be a security problem as the disk archive
cannot be verified and hence cannot be trusted.

Guix is not a container system, but when used on Linux it does support
the same virtualization features (they are also exposed by “guix
environment”, for example).  Guix is concerned with providing a path
from the declaration of a full environment to a reproducible binary.
Guix can thus be used to generate Docker disk images in a reproducible
fashion.  Guix does without disk images (it just installs things in
finer grain resolution to individual directories under /gnu/store) but
still achieves isolation and separation of packages.  Together with
Linux container/virtualization features it can be used for pretty much
the same things that Docker is often used for.

--
Ricardo

Re: Relationship between Docker and Guix

[Laura Lazzati]

[-- Attachment #1: Type: text/plain, Size: 187 bytes --]

Hi Ricardo!


Thank you sooo much for taking your time for the explanation :)
 I will probably come back to this when I prepare that part of the
presentation about it.


Regards :)
Laura

[-- Attachment #2: Type: text/html, Size: 357 bytes --]

Re: Relationship between Docker and Guix

[zimoun]

Hi,

Thank you Ricardo for the detailed explanations.


I do not know if my analogy below is correct and/or useful.

The relationship between Docker and GNU Guix is container and the LXC
[1] technology. They use both but differently:

 - Docker is rooted in mutable/imperative and tries to go to more functional;
 - Guix is rooted in immutable/functional.


Everything starts with a configuration file: Dockerfile versus manifest.scm.

 - Dockerfile depends on the state of the distribution that one will
use -- say Debian -- and each time "RUN apt-get update" and/or "RUN
apt-get install" is called then no one can know in advance what the
resulting disk image will *exactly* contain;
 - the manifest.scm depends on the state of the channel trees, other
said, on commit hashes -- manifest.scm acts as a pure function: same
inputs, same outputs -- so one obtain exactly the same container.

We cannot guarantee that the manifest.scm file which one runs today
will generate the same bit-to-bit disk image in the future. Mainly
because it has not been tested yet on the long run. :-)

However, Guix provides the tools to detect that something is not as
expected. For example, we can imagine that today one says: this is the
manifest to build the disk image and the hashes of the store are that
(the same way one provides the MD5 of files when downloading); then in
the future, building again the disk image, we can compare. Currently,
it is impossible with Docker because all the distro are doomed
(mutable). ;-) So what people are currently doing is to store all the
Docker disk images.

Docker motto: build once and run anywhere.
Guix motto(*): build anytime and run everywhere.


To me the relationship(**) is:

   guix pack -f docker -m stuff.scm
   docker load < /gnu/store/<hash>-fancy-name.tar
   docker push



Hope that helps.

All the best,
simon



[1] https://en.wikipedia.org/wiki/LXC

(*) it is not official and my personal view ;-)
(**) again my personal view.

Re: Relationship between Docker and Guix

[Giovanni Biscuolo]

[-- Attachment #1: Type: text/plain, Size: 2551 bytes --]

Hellp zimoun,

my two cents on Docker as container images builder (not as "container
instantiation toolbox")

zimoun <zimon.toutoune@gmail.com> writes:

[...]

> The relationship between Docker and GNU Guix is container and the LXC
> [1] technology. They use both but differently:
>
>  - Docker is rooted in mutable/imperative and tries to go to more
>  functional;

Docker images are immutable, they are usually (but not exclusively)
generated via `docker build` using an imperative "dockerfile language"
(so immutable/imperative :-) ); images are used to instantiate Linux
containers and there are tools (bind mounts, environment) to separate
state (data/config) from "system" to obtain a stateless container: the
"stateless" status of the container however is strictly tied to the
stateless properties of the underlying ditribution used to build the
container

I'm not sure what you mean with "tries to go more funtional" since AFAIU
dockerfile "language" is not going to be funcional anytime soon,
probably it does not even need to.

The main issue with the use of dockerfiles is that unfortunately the
FROM layering option (usually *heavily* used by docker images packagers)
often makes keeping control of what is actually distributed with the
image [1], cryptominers included, simply an... illusion... ehrm hard
work :-).  I've done it for work and that is the main reason I *avoid*
any docker image not built by me as soon as I can.

[...]

> Everything starts with a configuration file: Dockerfile versus manifest.scm.
>
>  - Dockerfile depends on the state of the distribution that one will
> use -- say Debian -- and each time "RUN apt-get update" and/or "RUN
> apt-get install" is called then no one can know in advance what the
> resulting disk image will *exactly* contain;

The non reproducible (not stateful) nature of the resulting docker
images directly depends on the distribution used to build it: Debian
have no means to "pin" a specific version in time (tag or commit, ala
Guix) to use; indeed if I use Guix as initial system image (FROM
guix-base) and a combination of manifests and channels definitions I can
get a reproducible image (never tried this, just theory :-) )

Anyway, using `guix pack -f docker...` is much much better than the
above dockerfile example :-O

This is just to say that it is not `docker build` fault not creating
reproducible images :-)

[...]

Ciao. Gio'


[1] even from a legal POV

-- 
Giovanni Biscuolo

Xelera IT Infrastructures

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: Relationship between Docker and Guix

[zimoun]

Hi,

Thank you for your input. You express more or less what I tried to
word. The keypoint is: build the Docker image with Guix because we
have a fine grain on the state and then use all the Docker machinery
because the world is as it is. :-)


Note that "guix pack -f docker" is one solution to create Docker
images. I have just discovered another one, thanks Ludo ;-)

  guix system docker-image config.scm

with for example config.scm [1].

[1] https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/system/examples/bare-bones.tmpl



Hope that help.
All the best,
simon