bug#61690: Failure to mount /sys in nested ‘guix shell’ container

bug#61690: Failure to mount /sys in nested ‘guix shell’ container

[Ludovic Courtès]

Hi!

As reported by Konrad¹, nested ‘guix shell -C’ fails:

--8<---------------cut here---------------start------------->8---
$ guix shell -CN guix \
      --expose=/var/guix/daemon-socket/socket \
      --expose=/gnu/store \
      -- guix shell -C coreutils -- ls /
guix shell: error: mount: mount "none" on "/tmp/guix-directory.xO3FIx/sys": Operation not permitted
--8<---------------cut here---------------end--------------->8---

Strace shows this:

--8<---------------cut here---------------start------------->8---
17541 clone(child_stack=NULL, flags=CLONE_NEWNS|CLONE_NEWCGROUP|CLONE_NEWUTS|CLONE_NEWIPC|CLONE_NEWUSER|CLONE_NEWPID|CLONE_NEWNET|SIGCHLD) = 7
[…]
17551 mount("none", "/tmp/guix-directory.d6rKy1", "tmpfs", 0, NULL) = 0
17551 mkdir("/tmp", 0777)               = -1 EEXIST (File exists)
17551 mkdir("/tmp/guix-directory.d6rKy1", 0777) = -1 EEXIST (File exists)
17551 mkdir("/tmp/guix-directory.d6rKy1/proc", 0777) = 0
17551 mount("none", "/tmp/guix-directory.d6rKy1/proc", "proc", MS_NOSUID|MS_NODEV|MS_NOEXEC, NULL) = 0
17551 mkdir("/tmp", 0777)               = -1 EEXIST (File exists)
17551 mkdir("/tmp/guix-directory.d6rKy1", 0777) = -1 EEXIST (File exists)
17551 mkdir("/tmp/guix-directory.d6rKy1/sys", 0777) = 0
17551 mount("none", "/tmp/guix-directory.d6rKy1/sys", "sysfs", MS_RDONLY|MS_NOSUID|MS_NODEV|MS_NOEXEC, NULL) = -1 EPERM (Operation not permitted)
--8<---------------cut here---------------end--------------->8---

It does work if the nested ‘guix shell’ uses ‘-CN’ instead of ‘-C’,
thanks to this bit in (gnu build linux-container)

        (mount-file-systems root mounts
                            #:mount-/proc? (memq 'pid namespaces)
                            #:mount-/sys?  (memq 'net
                                                 namespaces))  ;<---

The reason for this bug seems to be given here:

  https://github.com/nestybox/sysbox/issues/67#issuecomment-726285026

It’s not clear whether there’s anything we can do, other than
recommending ‘-CN’ as well in the nested container.

Thoughts?

Ludo’.

¹ https://lists.gnu.org/archive/html/guix-devel/2023-02/msg00027.html

bug#61690: Failure to mount /sys in nested ‘guix shell’ container

[Josselin Poiret via Bug reports for GNU Guix]

[-- Attachment #1: Type: text/plain, Size: 637 bytes --]

Hi Ludo,

Ludovic Courtès <ludovic.courtes@inria.fr> writes:

> The reason for this bug seems to be given here:
>
>   https://github.com/nestybox/sysbox/issues/67#issuecomment-726285026
>
> It’s not clear whether there’s anything we can do, other than
> recommending ‘-CN’ as well in the nested container.

Couldn't we always create a new network namespace, but when -N is passed
it also has a veth interface?  The one problem I can think of is that
we'd need to either create one veth per interface in the parent
namespace or let the user specify which interface should be shared.

Best,
-- 
Josselin Poiret

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 682 bytes --]

bug#61690: Failure to mount /sys in nested ‘guix shell’ container

[Ludovic Courtès]

Hi Josselin,

Josselin Poiret <dev@jpoiret.xyz> skribis:

> Ludovic Courtès <ludovic.courtes@inria.fr> writes:
>
>> The reason for this bug seems to be given here:
>>
>>   https://github.com/nestybox/sysbox/issues/67#issuecomment-726285026
>>
>> It’s not clear whether there’s anything we can do, other than
>> recommending ‘-CN’ as well in the nested container.
>
> Couldn't we always create a new network namespace, but when -N is passed
> it also has a veth interface?  The one problem I can think of is that
> we'd need to either create one veth per interface in the parent
> namespace or let the user specify which interface should be shared.

Maybe we could, but I must confess I’m totally clueless on this veth
thing.  :-)

What would this entail?  Hopefully guile-netlink can help?

Thanks,
Ludo’.

bug#61690: Failure to mount /sys in nested ‘guix shell’ container

[Josselin Poiret via Bug reports for GNU Guix]

[-- Attachment #1: Type: text/plain, Size: 990 bytes --]

Hi Ludo,

Ludovic Courtès <ludovic.courtes@inria.fr> writes:

> Maybe we could, but I must confess I’m totally clueless on this veth
> thing.  :-)
>
> What would this entail?  Hopefully guile-netlink can help?

So, a veth (Virtual Ethernet) device is basically a pipe but for network
devices: they're created in pairs, and any packet going through one end
is instantly received on the other end.  You can then transmit packets
between network namespaces.

One problem that totally slipped by me is that you need to be root to
create a veth device in the original namespace... Rootless containers
use slirp4netns, which is basically a userspace TCP/IP stack
communicating with a special network device in the new namespace (over
which you have complete rights). The situation might thus be a bit more
complicated, since we'd need another library/program as a dependency to
achieve this. I guess there's no best solution for now then :/

Best,
-- 
Josselin Poiret

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 682 bytes --]