Containers on Guix

Containers on Guix

[David Thompson]

Hello all,

As some of you know, I wrote 'guix environment' as the first step on my
journey to use Guix to improve my (and hopefully others) software
development workflow.  Now that the basics are working, I'm interested
in taking things further.  In particular, I want to extend 'guix
environment' to support spawning VMs and then, hopefully, containers
(those things that the Docker folks have been talking about so much
lately).

I did some reading about how Docker creates containers and discovered
that it uses systemd-nspawn[0] to do it.  Since Guix uses dmd, using
systemd-nspawn isn't an option.  Does anyone have thoughts on how we
might have similar functionality in the Guix distro?

Would an nspawn equivalent be appropriate for dmd?  Or a completely
separate program?  This seems to be a Linux only feature, so if we added
a container creation program, would it make it harder to support the
Hurd?

I'm lost and confused!  Send help! :)

-- 
David Thompson
Web Developer - Free Software Foundation - http://fsf.org
GPG Key: 0FF1D807
Support the FSF: https://fsf.org/donate

Re: Containers on Guix

[David Thompson]

The missing footnote:

[0] http://www.freedesktop.org/software/systemd/man/systemd-nspawn.html

-- 
David Thompson
Web Developer - Free Software Foundation - http://fsf.org
GPG Key: 0FF1D807
Support the FSF: https://fsf.org/donate

Re: Containers on Guix

[宋文武]

David Thompson <dthompson2@worcester.edu> writes:

> Hello all,
>
> As some of you know, I wrote 'guix environment' as the first step on my
> journey to use Guix to improve my (and hopefully others) software
> development workflow.  Now that the basics are working, I'm interested
> in taking things further.  In particular, I want to extend 'guix
> environment' to support spawning VMs and then, hopefully, containers
> (those things that the Docker folks have been talking about so much
> lately).
>
> I did some reading about how Docker creates containers and discovered
> that it uses systemd-nspawn[0] to do it.  Since Guix uses dmd, using
> systemd-nspawn isn't an option.  Does anyone have thoughts on how we
> might have similar functionality in the Guix distro?
I think what we need is pflask: https://github.com/ghedo/pflask

A container is programs or full os running in an isolated environment.
For a full container with rootfs, we can:
  build the rootfs:
    of Guix:    by using a form of `guix system init'
    of Debian:  by using debootstrap

  get it running:
    # pflask --chroot=rootfs /sbin/init
  
For a lightweight container without rootfs, we can:
  build the activate script by `guix build'
  get it running:
    $ pflask --user=$USER activate
    (could run by normal user by using User namespace)
>
> Would an nspawn equivalent be appropriate for dmd?  Or a completely
> separate program?  This seems to be a Linux only feature, so if we
> added
Isolation is archieved by using Linux namespaces, IMO dmd will work
well.
> a container creation program, would it make it harder to support the
> Hurd?
>
> I'm lost and confused!  Send help! :)
>
> -- 
> David Thompson
> Web Developer - Free Software Foundation - http://fsf.org
> GPG Key: 0FF1D807
> Support the FSF: https://fsf.org/donate

Re: Containers on Guix

[Thompson, David]

On Thu, Nov 20, 2014 at 8:30 AM, 宋文武 <iyzsong@gmail.com> wrote:
> David Thompson <dthompson2@worcester.edu> writes:
>
>> I did some reading about how Docker creates containers and discovered
>> that it uses systemd-nspawn[0] to do it.  Since Guix uses dmd, using
>> systemd-nspawn isn't an option.  Does anyone have thoughts on how we
>> might have similar functionality in the Guix distro?
> I think what we need is pflask: https://github.com/ghedo/pflask
>
> A container is programs or full os running in an isolated environment.
> For a full container with rootfs, we can:
>   build the rootfs:
>     of Guix:    by using a form of `guix system init'
>     of Debian:  by using debootstrap
>
>   get it running:
>     # pflask --chroot=rootfs /sbin/init
>
> For a lightweight container without rootfs, we can:
>   build the activate script by `guix build'
>   get it running:
>     $ pflask --user=$USER activate
>     (could run by normal user by using User namespace)

Wow, thanks!  I think pflask is exactly what I'm looking for.  I'll
write a guix package and do some experiments with it soon.

>> Would an nspawn equivalent be appropriate for dmd?  Or a completely
>> separate program?
> Isolation is archieved by using Linux namespaces, IMO dmd will work
> well.

It seems that with pflask, there's no need to add anything to dmd.  We
could extend guix system to use it to generate containers.

This is a really great starting point.  Thank you very much for
pointing me in the right direction.

- Dave

Re: Containers on Guix

[Ludovic Courtès]

Hello!

Disclaimer: I’ve never used systemd-nspawn, and I’m not knowledgeable in
this area.  :-)


AIUI, “containers” are basically what the daemon creates: an execution
environment that uses a separate file system name space, network name
space, etc. (see ‘DerivationGoal::startBuilder’ in libstore/build.cc.)

For what you have in mind, one may want to be able to select which parts
should be separate (apparently systemd-nspawn allows that), rather than
the completely-isolated policy of guix-daemon.

So, in terms of functionality, I think we want that subset of the
daemon, in a more modular fashion (that subset would also be useful for
Plash-like sandboxed execution, something I’d like to have eventually.)

It doesn’t have to be part of the init system IMO, because it doesn’t
have much to do with it.  However, there has to be a mediating process
with root privileges that can create these containers on behalf on
unprivileged users–much like guix-daemon.

In terms of code, I can think of several approaches.

  1. Fork guix-daemon, and modularize it to do what we want.  Perhaps it
     would be enough to add RPCs to create and configure a container
     (see worker-protocol.hh and (guix store).)

     Alternately, create a C library that provides just the
     container-handling logic (possibly with Guile bindings), and use it
     to write a separate daemon responsible for container handling.

  2. Translate/rewrite the container-handling logic in Scheme.  Use it
     to write a separate daemon, with the eventual goal of having a new
     build daemon that uses the same code base (all in Scheme.)

  3. Use LXC to implement containers (?).  liblxc seems to be perhaps
     too high-level from the examples on the web page; does anyone know?

#2 is forward-looking, but quite a lot of work.

#1 and #3 are more pragmatic.

I hope that makes some sense.

Ludo’.

Re: Containers on Guix

[Ludovic Courtès]

宋文武 <iyzsong@gmail.com> skribis:

> I think what we need is pflask: https://github.com/ghedo/pflask

Indeed, that seems like a good fit!

> A container is programs or full os running in an isolated environment.
> For a full container with rootfs, we can:
>   build the rootfs:
>     of Guix:    by using a form of `guix system init'

I think it’s enough and cheaper to build the system (as per ‘guix system
build’) and to bind-mount its closure in the container’s file system.

Thanks,
Ludo’.

Re: Containers on Guix

[Thompson, David]

On Thu, Nov 20, 2014 at 3:49 PM, Ludovic Courtès <ludo@gnu.org> wrote:
> 宋文武 <iyzsong@gmail.com> skribis:
>
>> I think what we need is pflask: https://github.com/ghedo/pflask
>
> Indeed, that seems like a good fit!

The concern I have is something the author brings up in the README:

"Compared to LXC, pflask is easier to use since it doesn't require any
pre-configuration (all the options can be passed via the
command-line). pflask is mostly intended for testing, building and
experimenting, whereas LXC is a more complete solution, better suited
for production environments."

I want something that can handle production environments.  But it's a
good starting point anyway.  Using an improved version of the
guix-daemon to do this might still be the best long term solution.

>> A container is programs or full os running in an isolated environment.
>> For a full container with rootfs, we can:
>>   build the rootfs:
>>     of Guix:    by using a form of `guix system init'
>
> I think it’s enough and cheaper to build the system (as per ‘guix system
> build’) and to bind-mount its closure in the container’s file system.

Didn't know about the 'build' subcommand.  I will try that out.

Thanks!

- Dave

Re: Containers on Guix

[David Thompson]

Ludovic Courtès <ludo@gnu.org> writes:

>> A container is programs or full os running in an isolated environment.
>> For a full container with rootfs, we can:
>>   build the rootfs:
>>     of Guix:    by using a form of `guix system init'
>
> I think it’s enough and cheaper to build the system (as per ‘guix system
> build’) and to bind-mount its closure in the container’s file system.

I'm having trouble with this approach.  pflask tries to create a /proc
directory but can't because that would be writing to the store.  Do I
need to bind-mount each sub-directory within the /gnu/store/...-system
directory so that the root of the container directory is still writable?

-- 
David Thompson
Web Developer - Free Software Foundation - http://fsf.org
GPG Key: 0FF1D807
Support the FSF: https://fsf.org/donate

Re: Containers on Guix

[Ludovic Courtès]

David Thompson <dthompson2@worcester.edu> skribis:

> Ludovic Courtès <ludo@gnu.org> writes:
>
>>> A container is programs or full os running in an isolated environment.
>>> For a full container with rootfs, we can:
>>>   build the rootfs:
>>>     of Guix:    by using a form of `guix system init'
>>
>> I think it’s enough and cheaper to build the system (as per ‘guix system
>> build’) and to bind-mount its closure in the container’s file system.
>
> I'm having trouble with this approach.  pflask tries to create a /proc
> directory but can't because that would be writing to the store.

Hmm, the store should be under /container/gnu/store, not /container, no?

> Do I need to bind-mount each sub-directory within the
> /gnu/store/...-system directory so that the root of the container
> directory is still writable?

Let’s say /container is the root of the container.  It must be a
regular, writable directory.

As a first step it’s OK to bind-mount all of /gnu/store to
/container/gnu/store.

What guix-daemon does is to bind-mount precisely each element of the
store that is needed, so there’s no “leak”.

HTH!

Ludo’.

Re: Containers on Guix

[Ian Denhardt]

[-- Attachment #1: Type: text/plain, Size: 1062 bytes --]

Quoting David Thompson (2014-11-19 21:34:49)
> I did some reading about how Docker creates containers and discovered
> that it uses systemd-nspawn[0] to do it.  Since Guix uses dmd, using
> systemd-nspawn isn't an option.  Does anyone have thoughts on how we
> might have similar functionality in the Guix distro?


I'm not so sure that docker actually needs systemd -- the docker
documentation has instructions for a number of systems that don't use
systemd. A quick scan suggests they're using lxc, at least on some
systems.
 
> Would an nspawn equivalent be appropriate for dmd?  Or a completely
> separate program?  This seems to be a Linux only feature, so if we added
> a container creation program, would it make it harder to support the
> Hurd?

I'm sure there are other things already packaged that don't really have
any hope of working on top of another kernel. This is going to be hard
to do portably, but I don't think it's worth worrying about too much
yet. Probably some work will have to happen upstream on the Hurd
project.

[-- Attachment #2: signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQIcBAABAgAGBQJUcL8OAAoJEPZUuMfUyjy48kQP/0eNkJ8x9qGKbsGSqATV0E5f
FtJBY38+CUpulsI1HpZxUiKGej1UtM+8O8if5ajQAibtpg0oMnalkhT4yaBjloaU
i74AIM+nhBEmSOW/5LfeJggK5RSrrISwaomOTcN8XHPA4xrl2EpE2Z4B7PtJiqlO
jGkCg8bkoI0nn8heV6JZejqhzjWUWRsp2HJZE/tb7KsMohK6TvzP2FtHysvsjrU0
CCNSijlGu9Fuai7cQ9SCvVRz4tS1ZNYit//3DTQATEjsqc2zSFy7GLhw4V/3uVVH
fMSq/JpCnbUdj4DR+00uQ7OMSL8CieX4CHmEC/4uVtv9wtmw5RA1lgF6EZSTsiGv
XBRBYG2XCzrMKUfW1FRfIICLd7U2ctBVSyOG3nkUZA6VK2rFWQwFFtOr17z4mctq
X4YW7fQwxd2BYve/8z0FVOGsg5e5CIZsAF68uVQrKPqh+VB8eMe24PFDTeIEtRCh
yvGRCgWQ3UHM+NW6POU0rXgNyq53TI1RV5sqls90SN7378mM+grk5m42TwTEB+t/
YqLq9xRLteZ2td5Nf2vsUd433ZtQxi//Fd49NiT1gni0Rabjf8Vc76v64a9PC5XI
tTn9AZEzpg2loUDxBWuSUzNo4WPt7+dHJ8QKDSm82I2Nnd4gwwj23uPiCLL8oCFB
Q9Y2pOLoGHq/L4UdSo+I
=a2a7
-----END PGP SIGNATURE-----

Re: Containers on Guix

[Ludovic Courtès]

Ian Denhardt <ian@zenhack.net> skribis:

> I'm sure there are other things already packaged that don't really have
> any hope of working on top of another kernel.

I think packages that are Linux-specific by design are all in (gnu
packages linux).

> This is going to be hard to do portably, but I don't think it's worth
> worrying about too much yet.

Yes, agreed.

> Probably some work will have to happen upstream on the Hurd project.

The Hurd supports fine-grain virtualization, so it wouldn’t be hard to
do (“sub-hurds” correspond to completely isolated containers, launching
a separate ‘proc’ server gives you a separate PID name space, etc.)

Thanks,
Ludo’.