I guess capabilities aren't handled by container creation code yet: https://github.com/guix-mirror/guix/blob/master/gnu/build/linux-container.scm#L262 IIUC this would also effect the idea of isolating system services as described in https://guix.gnu.org/en/blog/2017/running-system-services-in-containers/. It may require deeper understanding of Linux caps, namescase and how it all handled by Guix code, I suppose. Not something I'll be able to pull off without hand-holding from core devs, sigh. On Wed, 22 Mar 2023 at 12:14, Vladilen Kozin wrote: > I now have a hypothesis as to what's happening. Could someone confirm or > disprove and maybe suggest a solution or point at existing workarounds. > > Host and container will share the exact same kernel that's unsurprisingly > already running, so the above has nothing to do with kernel modules or > settings. Fwiw I figured a way to find where kernel modules reside by doing: > > $ sudo dmesg | grep -i "kernel command line" > which shows where current system is inside the store and relevant > /lib/modules will be under it. We could then > --expose=/gnu/store/hash-system/lib/modules=/lib/modules if we wanted to. > > Real problem, IIUC, is with capabilities. Notion of "container" can be > misleading and evokes thoughts of "vm" when in practice its just a process > with some isolation applied to it. So, presently I'm guessing container > Shepherd maybe PID 1 inside its isolated environment, but from the host pow > it is just a process and one that unlike our host's shepherd may lack > certain capabilities and privileges to e.g. create new devices or load > kernel modules on request, etc. In the sense of > https://man7.org/linux/man-pages/man7/capabilities.7.html maybe? > > Am I on the right track? But then, how does one test services like that > that may require ability to modify devices etc? Have we "outgrown" > container and ought to `guix system vm` for such services? Or is there a > way to bless container shepherd with necessary capabilities? If not from > `guix system container` command line, then perhaps dropping down to the > underlying programmatic interface i.e. whatever `guix system container` > ends up calling to containerize a system? > > Thanks > > > On Wed, 22 Mar 2023 at 10:20, Vladilen Kozin > wrote: > >> Hello guix. >> >> I put together a tailscale system service that's meant to start a >> tailscale daemon managed by the system shepherd, that is to say that my >> `tailscaled-service-type` specifies `(service-extension >> shepherd-root-service-type tailscaled-shepherd-service)`, where >> `tailscaled-shepherd-service` creates a `shepherd-service` with (provision >> '(tailscaled)) and (requirement '(networking)). >> >> I tested it by lowering to store via `shepherd-service-file` and then >> loading the generated script via `sudo herd load root ...`. This works fine >> and the daemon starts without a problem. >> >> Next, I try to spawn tailscaled as part of my OS definition: >> (services (cons* (service tailscaled-service-type >> (tailscaled-configuration)) %base-services)) >> ;; tried %desktop-services too >> >> To test, we create a container: >> sudo guix system -K -L /home/vlad/Code/fullmeta-guix/channel container >> os.scm --network --expose=/dev/net=/dev >> >> Earlier runs had it complaining that /dev/net/tun was missing, so I >> exposed that. Dunno if that's how I'm supposed to handle this. Now, >> /var/log/messages show: >> >> Mar 22 09:38:48 twgter shepherd[1]: [tailscaled] 2023/03/22 09:38:48 >> Linux kernel version: 5.18.10 >> Mar 22 09:38:48 twgter shepherd[1]: [tailscaled] 2023/03/22 09:38:48 is >> CONFIG_TUN enabled in your kernel? `modprobe tun` failed with: >> Mar 22 09:38:48 twgter shepherd[1]: [tailscaled] 2023/03/22 09:38:48 >> wgengine.NewUserspaceEngine(tun "tailscale0") error: >> tstun.New("tailscale0"): operation not permitted >> >> I feel like maybe I'm missing some kernel modules, but I would've >> expected host and container to share the kernel, so I dunno. In fact, when >> I randomly attempted adding (kernel-arguments (cons* "CONFIG_TUN=m" >> %default-kernel-arguments)) to my os definition, resulting script hash came >> out the same, which tells me, containers don't even look at these kernel >> params when generating a script. >> >> Any guesses as to why this works under host but not inside container? >> >> Relatedly, does anyone have a nicer workflow they use to define and test >> shepherd services? Such containerization was the next step in testing the >> service and would've been ok were it not for the above failure, but the >> initial indirection with lowering to store, then `sudo herd load root ...` >> is a bit too involved and "indirect" for my liking as well - anyone has an >> improved way of developing shepherd services? >> >> Thanks! >> -- >> Best regards >> Vlad Kozin >> > > > -- > Best regards > Vlad Kozin > -- Best regards Vlad Kozin