Re: [GSoC] Porting GuixSD to GNU Hurd draft

[Justus Winter <4winter@informatik.uni-hamburg.de>]

Hi,

Quoting Ludovic Courtès (2016-03-23 14:40:38)
> > 2. The Project
> >
> > The project consists of four main stages
> >
> > 1. Modify Guix so it will be able to create and mount the file-system needed to boot into a system with Hurd at its core. 
> > 2. Modify Guix so it can produce a working image, while isolating any cases of Linux assumptions.
> > 3. Successfully boot into one such system using GNU Shepherd with pid 1.
> > 4. Modify the new Guix system to take advantage of Hurd specific mechanisms.

For me, 4. is the most important bit, so we can build packages in
isolation.

> > Currently the tools Guix uses to interact with the filesystem exist inside the (guix build syscalls) module. 
> > This module provides bindings to libc's syscall wrappers, which are only available on a GNU/Linux system. 
> > In order to offer the same functionality on a GNU/Hurd system we must first write Guile bindings for the 
> > relevant Hurd functions, like 'file_set_translator' in hurd/fs.defs
> > for example.
> 
> Note that technically the ‘file_set_translator’ function is in libc:
> 
> --8<---------------cut here---------------start------------->8---
> scheme@(guile-user)> (dynamic-func "file_set_translator" (dynamic-link))
> $1 = #<pointer 0x1675660>
> --8<---------------cut here---------------end--------------->8---
> 
> > This module will be called (guix build hurd). This allows us to
> > re-implement the functionality of the 'settrans' command, as described
> > here[1], in Scheme and use that in place of 'mount()', where
> > applicable.

Right.  In fact, what settrans (or mount) does isn't that hard to
reproduce, though I wouldn't mind moving the c implementation to
libhurdutil or the like and binding to that.

> I think it’s important to think about:
> 
>   1. How functionality equivalent to linux-{initrd,boot} will be
>      implemented on the Hurd.
> 
>      In my experience the equivalent thing is simpler on the Hurd,
>      esp. if relying on passive translators (see daemons/runsystem.sh in
>      the Hurd), though here we’ll probably explicitly activate
>      translators at boot time.

Currently, there is not really a reason to have an initrd-like
solution on the Hurd.  Yes, one has to start the default pager
explicitly, but that's about it.

> > Finally, once GuixSD is successfully ported, we can cater to the last stage of taking advantage of Hurd specific 
> > mechanisms. 
> > This includes but is not limited to:
> > 1)Replacing (gnu build linux-container) with (gnu build subhurd) when on a GNU/Hurd system. Subhurds can offer 
> > isolation similar to Linux containers as described here[3].
> 
> This is really super optional IMO.  This module is only used by ‘guix
> environment --container’, which is an optional feature.

Yes, subhurds are more on the experimental side imho.

> > 2)Modify the guix-daemon to run without root privileges by utilizing the Hurd architecture. The daemon needs root 
> > privileges in order to setup chrooted environments for the builds.  In the Hurd this can be done by setting up a 
> > translator as described here[4]. 
> 
> This is more important, and already non-trivial work.  If libc provided
> ‘mount’ with support for MS_BIND (which I think it should), it would
> help a bit, but there’s still the problem of the other Linux name spaces
> that are used (the CLONE_NEW* clone(2) flags.)
> 
> Thus it may make more sense to write this functionality in guix-daemon
> using directly the Hurd interfaces.  Separate PID name spaces, UID name
> spaces, mount name spaces, etc. are fairly natural on the Hurd: it’s
> “just” a matter of giving the child process ports to separate proc,
> auth, etc. translators.
> 
> In itself this is also a bit of work.  I wonder what the Hurd folks
> think about priorities here?

I'd go for specializing guix-daemon over trying too hard to implement
Linux-compatible interfaces in the libc.  I consider the
filesystem-isolation part easy, UID isolation relatively easy, PID
isolation is a bit tricky.  Currently one cannot simply start another
proc server and expect it to work as it needs privileged kernel
interfaces.  I created an RPC to allow nested proc servers for
unprivileged subhurds.  That should do the trick, it is merely a
matter of making it actually work I guess.

> > 3)Guix uses symlink trees for user profiles. Instead we can use stowfs[5]. Stowfs creates a traditional Unix directory 
> > structure from all the files in the individual package directories.
> 
> Fun but optional.  ;-)

Agreed.

> > May 2 - May 22
> > Create (gnu build hurd-boot) and (gnu build hurd-initrd).
> > Start working on describing the GNU/Hurd system in (gnu system).
> 
> I know Debian at some point added initrd support to GNU Mach for some
> reason, but fundamentally, the whole point of multiboot is to provide a
> solution more flexible than initrds.  So, hopefully, no initrds here.
> Since there’s no initrd, there’s also no ‘switch_root’ dance (on
> Linux-based system the initrd is the initial root file system, so you
> need to switch to the real root file system as soon as you’re able to
> mount it), which simplifies things.
> 
> Justus, Samuel, WDYT?

Debian has the initrd for the live cds, but that is a bit hacky.  I
don't believe we need an initrd at this point.

> > May 23 - 12 June
> > Modify (gnu system *) modules as needed.
> > All the modules (guix build *) and (gnu build *) will be working as expected by now.
> > Try building a GuixSD image. (Milestone 2)
> 
> This is the crux of the matter.  :-)
> 
> Before starting, it would be nice to have a rough idea of how GuixSD
> startup will work on the Hurd, and what changes this entails.
> 
> For debugging purposes, it would be very helpful to say the least to
> have a working ‘guix system vm’: it would allow you to test your changes
> very quickly, without rebooting and so on.
> 
> This in itself requires some thought: currently (guix system vm) relies
> on QEMU’s ‘-kernel’ option to boot the kernel Linux.  For GNU/Hurd, we
> instead need to boot a complete image with GRUB, like ‘guix system
> vm-image’ does.  You’ll have to investigate how to port this.

qemu can boot multiboot operating systems.

> > Deliver a working GuixSD system image with Hurd as the kernel.

Hurd is not a kernel.

> Victory!  :-)
> 
> I think this is super cool, and also super ambitious!  I’d expect that
> we’d be able to reach milestone #2 if everything goes well (and assuming
> we don’t try to address isolation in guix-daemon), but milestone #3 is
> most likely for later.
> 
> What do people think?
> 
> The main question is whether you should implement build isolation in
> guix-daemon, in which case that would leave little time for the GuixSD
> parts.  I think I would rather let you focus on the GuixSD stuff as you
> wrote, but I’d like to hear what the Hurd folks think.

I consider isolation more important.

> Justus, Samuel: could you add yourselves as official co-mentors on
> Google’s web site, if not already done?  :-)

I clicked on 'want to mentor', do I need to do more?

Justus