[GSoC] Porting GuixSD to GNU/Hurd Update

[GSoC] Porting GuixSD to GNU/Hurd Update

[Manolis Ragkousis]

Hello Guix, Hello Hurd,

As GSoC is coming to an end I think it's time to sum up my work till
now, report issues I had and then talk about what our next steps are.

First I would like to mention what were the two main objectives of the
project and also what was the status of the port when the project
started, so I can give you a better understanding of the progress.

The objectives were:
1) Achieve build isolation in the daemon on the Hurd.
2) Modify Guix so it can produce a working image, while isolating any
cases of Linux assumptions.
3) Boot to GuixSD/Hurd

When the project started the guix-daemon could work on GNU/Hurd but the
builds were being affected by the system state. Normally the daemon
creates a chrooted environment to make the builds, inside /tmp, and bind
mounts the parts of the system it needs. This way it achieves isolation.

On Hurd though, because of the lack of mount(), the above could not
work. But with the help of my libhurdutil library, which I wrote at the
beginning of this project, I created 2 wrappers inside
nix/libutil/call.(hh.cc) for mount() and umount2(), called nixMount()
and nixUmount2(), and depending on the system the implementation
changes. On Hurd I am using /hurd/firmlink to offer the same
functionality to bind-mounts.

It seems to work but I am testing it thoroughly so we don't have any
unpleasant surprises in the future. We will have fully isolated builds
on Hurd as soon as I am sure that my code works as expected and it's
merged in Guix upstream.

Now on the GuixSD side, I will start on a big issue I had. If you check
the daemon, and specifically nix/libstore/build.cc:2205-2228, you will
see that when, for example, guix tries to create a 32 bit vm/image from
a 64 bit machine, the daemon actually builds the 32 bit binaries on 32
bit personality mode.

As a result it is not possible to build GuixSD/Hurd vm/images from
Linux, for now. But this is not a big problem because we can do it from
Guix running on Hurd :-). The approach I used, was to add a second hard
drive on my Hurd vm, mount it, and try to directly deploy a GuixSD
system on that drive.  Currently Guix can build most of the binaries for
the system but it still needs work to actually boot into one. You can
see the result of this work on the currect core-updates (and on my
github repo for some hacky commits).

I have also created a new module called (guix build syscalls-tools)
which contains some of the code from (guix build syscalls) which will be
used by a (guix build hurd) module, which will contain call wrappers for
some Hurd libraries. This work is still in my github repo because it
needs some work.

There was one more problem that appeared after we started using
C_INCLUDE_PATH in our cross-builds. As it seems MiG needs glibc in order
to be built.  That's why for now I patch cross-mig with the
glibc-headers so I don't have to depend on the Linux ones. But I will
talk more on this in a different email.

I think that's enough for now. I avoid talking about things discussed in
other mails, but if you want please feel free to ask me :-)

To sum things up, we almost have build isolation working but GuixSD
still needs some work to become bootable. From here on I will
finish/test my guix-daemon code, and then I will continue on finishing
the low-level call wrappers for the Hurd libraries, in order to get the
bootable system. We are close :-).

The repos which you can check any code not yet in upstream Guix or Hurd are:
https://github.com/Phant0mas/Guix-on-Hurd
https://github.com/Phant0mas/Hurd

I think that's it for now. I want to thank all of you for your help and
support and for answering my questions, and thank my two mentors Justus
and Ludo for their invaluable help (you guys are awesome :-)).

I would also like to state that thanks to the latest work from the Hurd
guys on Gnumach and Hurd, debian/hurd is more stable than ever!!

I am sure I forgot some things so feel free to ask anything or correct
me. :-)

Thank you,
Manolis

Re: [GSoC] Porting GuixSD to GNU/Hurd Update

[Ludovic Courtès]

[-- Attachment #1: Type: text/plain, Size: 3493 bytes --]

Hello Manolis,

Manolis Ragkousis <manolis837@gmail.com> skribis:

> On Hurd though, because of the lack of mount(), the above could not
> work. But with the help of my libhurdutil library, which I wrote at the
> beginning of this project, I created 2 wrappers inside
> nix/libutil/call.(hh.cc) for mount() and umount2(), called nixMount()
> and nixUmount2(), and depending on the system the implementation
> changes. On Hurd I am using /hurd/firmlink to offer the same
> functionality to bind-mounts.
>
> It seems to work but I am testing it thoroughly so we don't have any
> unpleasant surprises in the future. We will have fully isolated builds
> on Hurd as soon as I am sure that my code works as expected and it's
> merged in Guix upstream.

Neat!  You did the right thing.

> Now on the GuixSD side, I will start on a big issue I had. If you check
> the daemon, and specifically nix/libstore/build.cc:2205-2228, you will
> see that when, for example, guix tries to create a 32 bit vm/image from
> a 64 bit machine, the daemon actually builds the 32 bit binaries on 32
> bit personality mode.

Indeed; hopefully we can address that.

> As a result it is not possible to build GuixSD/Hurd vm/images from
> Linux, for now. But this is not a big problem because we can do it from
> Guix running on Hurd :-). The approach I used, was to add a second hard
> drive on my Hurd vm, mount it, and try to directly deploy a GuixSD
> system on that drive.  Currently Guix can build most of the binaries for
> the system but it still needs work to actually boot into one. You can
> see the result of this work on the currect core-updates (and on my
> github repo for some hacky commits).

Woow!  We knew booting GuixSD on GNU/Hurd was an ambitious goal, so what
you describe is already impressive.

> I have also created a new module called (guix build syscalls-tools)
> which contains some of the code from (guix build syscalls) which will be
> used by a (guix build hurd) module, which will contain call wrappers for
> some Hurd libraries. This work is still in my github repo because it
> needs some work.

Nice stuff again.

> There was one more problem that appeared after we started using
> C_INCLUDE_PATH in our cross-builds. As it seems MiG needs glibc in order
> to be built.  That's why for now I patch cross-mig with the
> glibc-headers so I don't have to depend on the Linux ones. But I will
> talk more on this in a different email.
>
> I think that's enough for now. I avoid talking about things discussed in
> other mails, but if you want please feel free to ask me :-)
>
> To sum things up, we almost have build isolation working but GuixSD
> still needs some work to become bootable. From here on I will
> finish/test my guix-daemon code, and then I will continue on finishing
> the low-level call wrappers for the Hurd libraries, in order to get the
> bootable system. We are close :-).
>
> The repos which you can check any code not yet in upstream Guix or Hurd are:
> https://github.com/Phant0mas/Guix-on-Hurd
> https://github.com/Phant0mas/Hurd

Now we need to chew that, both on the Guix and Hurd sides.  Make sure to
ping us with sizable chunks of this code!  :-)

This is again an impressive piece of work, especially given its breadth:
From cross-compilation toolchain issues, to low-level Linux- and
Hurd-specific programming, and to high-level Scheme hacking.  Kudos!

Thank you Manolis!

Ludo’.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 818 bytes --]