From mboxrd@z Thu Jan 1 00:00:00 1970 From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) Subject: Re: [GSoC] Porting GuixSD to GNU Hurd draft Date: Wed, 23 Mar 2016 14:40:38 +0100 Message-ID: <878u19wbvt.fsf@gnu.org> References: <20160206113802.GA17867@thebird.nl> <87mvre2eyz.fsf@gnu.org> <56BC7496.8020409@gmail.com> <8760x5gpif.fsf@inria.fr> <20160302101227.GF2815@var.bordeaux.inria.fr> <87a8mgefcy.fsf@gnu.org> <20160302220654.GC3037@var.home> <56EFE480.6020007@gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Return-path: In-Reply-To: <56EFE480.6020007@gmail.com> (Manolis Ragkousis's message of "Mon, 21 Mar 2016 14:09:36 +0200") List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-hurd-bounces+gnu-bug-hurd=m.gmane.org@gnu.org Sender: bug-hurd-bounces+gnu-bug-hurd=m.gmane.org@gnu.org To: Manolis Ragkousis Cc: guix-devel@gnu.org, bug-hurd@gnu.org, Justus Winter <4winter@informatik.uni-hamburg.de>, Samuel Thibault List-Id: guix-devel.gnu.org Hello! Manolis Ragkousis skribis: > Although I have uploaded and shared my draft to the GSoC website, I am > also resending it to the lists as per Ludovic's instruction. Yeah, the GSoC website makes it possible to provide a link to a text file, so let=E2=80=99s do that instead of using their SaaSS. > 2. The Project > > The project consists of four main stages > > 1. Modify Guix so it will be able to create and mount the file-system nee= ded to boot into a system with Hurd at its core.=20 > 2. Modify Guix so it can produce a working image, while isolating any cas= es of Linux assumptions. > 3. Successfully boot into one such system using GNU Shepherd with pid 1. > 4. Modify the new Guix system to take advantage of Hurd specific mechanis= ms. > > > Currently the tools Guix uses to interact with the filesystem exist insid= e the (guix build syscalls) module.=20 > This module provides bindings to libc's syscall wrappers, which are only = available on a GNU/Linux system.=20 > In order to offer the same functionality on a GNU/Hurd system we must fir= st write Guile bindings for the=20 > relevant Hurd functions, like 'file_set_translator' in hurd/fs.defs > for example. Note that technically the =E2=80=98file_set_translator=E2=80=99 function is= in libc: --8<---------------cut here---------------start------------->8--- scheme@(guile-user)> (dynamic-func "file_set_translator" (dynamic-link)) $1 =3D # --8<---------------cut here---------------end--------------->8--- > This module will be called (guix build hurd). This allows us to > re-implement the functionality of the 'settrans' command, as described > here[1], in Scheme and use that in place of 'mount()', where > applicable. Good! (We might as well call it (hurd =E2=80=A6) in fact: (hurd fs) would correspond to fs.defs, (hurd io) for io.defs, and so on.) > Additionally, we need to make sure that all modules in (guix build *) and= (gnu build *) can offer the same=20 > functionalities on a GNU/Hurd system. For example (gnu build vm) relies = on QEMU's '-kernel' command-line=20 > option, which is Linux-specific. On the other hand, in the case of modul= es like (gnu build linux-boot) or=20 > (gnu build linux-initrd), which by design are Linux-specific, we will nee= d to provide a separate module with=20 > equivalent functionality. (gnu system *) modules will require changes as = well, in order to be able to use=20 > modifications that will happen in the (guix build *) and (gnu build *) mo= dules.=20 I think it=E2=80=99s important to think about: 1. How functionality equivalent to linux-{initrd,boot} will be implemented on the Hurd. In my experience the equivalent thing is simpler on the Hurd, esp. if relying on passive translators (see daemons/runsystem.sh in the Hurd), though here we=E2=80=99ll probably explicitly activate translators at boot time. 2. How to structure GuixSD code in a way that abstracts the underlying OS kernel. For instance, we could have a (guix build os) module providing an API that works for both kernels, probably close to the Linux one, and that would dispatch to the right implementation, the Linux or Hurd one. Both of these are quite a bit of design and implementation work that we should not underestimate. > Finally, once GuixSD is successfully ported, we can cater to the last sta= ge of taking advantage of Hurd specific=20 > mechanisms.=20 > This includes but is not limited to: > 1)Replacing (gnu build linux-container) with (gnu build subhurd) when on = a GNU/Hurd system. Subhurds can offer=20 > isolation similar to Linux containers as described here[3]. This is really super optional IMO. This module is only used by =E2=80=98gu= ix environment --container=E2=80=99, which is an optional feature. > 2)Modify the guix-daemon to run without root privileges by utilizing the = Hurd architecture. The daemon needs root=20 > privileges in order to setup chrooted environments for the builds. In th= e Hurd this can be done by setting up a=20 > translator as described here[4].=20 This is more important, and already non-trivial work. If libc provided =E2=80=98mount=E2=80=99 with support for MS_BIND (which I think it should),= it would help a bit, but there=E2=80=99s still the problem of the other Linux name s= paces that are used (the CLONE_NEW* clone(2) flags.) Thus it may make more sense to write this functionality in guix-daemon using directly the Hurd interfaces. Separate PID name spaces, UID name spaces, mount name spaces, etc. are fairly natural on the Hurd: it=E2=80=99s =E2=80=9Cjust=E2=80=9D a matter of giving the child process ports to separa= te proc, auth, etc. translators. In itself this is also a bit of work. I wonder what the Hurd folks think about priorities here? > 3)Guix uses symlink trees for user profiles. Instead we can use stowfs[5]= . Stowfs creates a traditional Unix directory=20 > structure from all the files in the individual package directories. Fun but optional. ;-) > 3. Estimated Project Timeline > > Before March 31 > Finish merging the wip-hurd branch to upstream. Note that this task depends on, ahem, other people as well. ;-) > Write a (guix build hurd) module which will contain Guile bindings to the= RPC stubs like hurd/fs.defs or hurd/exec.defs . > > April 1 - April 15 > Package missing dependencies (Hurd libs). > Re-implement 'settrans' in scheme. Don=E2=80=99t just straight into low-level =E2=80=9Cdetails.=E2=80=9D I=E2= =80=99d recommend familiarizing yourself with GuixSD on GNU/Linux, fiddling with bits of code and so on (you can try safely in a VM with =E2=80=98guix system vm=E2= =80=99), and then, familiarizing yourself with the GNU/Hurd startup process (looking at daemons/runsystem.sh already gives a good idea.) > April 16 - May 1 > At this point we will have the tools needed to build a Hurd based file-sy= stem. (Milestone 1) =E2=80=9CHurd-based file system=E2=80=9D? > May 2 - May 22 > Create (gnu build hurd-boot) and (gnu build hurd-initrd). > Start working on describing the GNU/Hurd system in (gnu system). I know Debian at some point added initrd support to GNU=C2=A0Mach for some reason, but fundamentally, the whole point of multiboot is to provide a solution more flexible than initrds. So, hopefully, no initrds here. Since there=E2=80=99s no initrd, there=E2=80=99s also no =E2=80=98switch_ro= ot=E2=80=99 dance (on Linux-based system the initrd is the initial root file system, so you need to switch to the real root file system as soon as you=E2=80=99re able = to mount it), which simplifies things. Justus, Samuel, WDYT? > May 23 - 12 June > Modify (gnu system *) modules as needed. > All the modules (guix build *) and (gnu build *) will be working as expec= ted by now. > Try building a GuixSD image. (Milestone 2) This is the crux of the matter. :-) Before starting, it would be nice to have a rough idea of how GuixSD startup will work on the Hurd, and what changes this entails. For debugging purposes, it would be very helpful to say the least to have a working =E2=80=98guix system vm=E2=80=99: it would allow you to test= your changes very quickly, without rebooting and so on. This in itself requires some thought: currently (guix system vm) relies on QEMU=E2=80=99s =E2=80=98-kernel=E2=80=99 option to boot the kernel Linux= . For GNU/Hurd, we instead need to boot a complete image with GRUB, like =E2=80=98guix system vm-image=E2=80=99 does. You=E2=80=99ll have to investigate how to port thi= s. > 13 June - 23 June > Solve any problems with booting into the system and running GNU Shepherd. > > 24 June - 9 July > Have a fully working GNU/Hurd system. (Milestone 3) > Make sure all the services/packages run correctly on the new GuixSD/Hurd = system and solve any issues. > > 10 July - 8 August > Start working on getting Hurd specific mechanisms integrated to Guix. What do you mean? > 9 August - 23 August > By now all the objectives will have been achieved. > Merge patches, code refactoring, documentation. > Deliver a working GuixSD system image with Hurd as the kernel. Victory! :-) I think this is super cool, and also super ambitious! I=E2=80=99d expect t= hat we=E2=80=99d be able to reach milestone #2 if everything goes well (and ass= uming we don=E2=80=99t try to address isolation in guix-daemon), but milestone #3= is most likely for later. What do people think? The main question is whether you should implement build isolation in guix-daemon, in which case that would leave little time for the GuixSD parts. I think I would rather let you focus on the GuixSD stuff as you wrote, but I=E2=80=99d like to hear what the Hurd folks think. Justus, Samuel: could you add yourselves as official co-mentors on Google=E2=80=99s web site, if not already done? :-) Thanks, Manolis! Ludo=E2=80=99.