Re: boot the Hurd with Guix

Re: boot the Hurd with Guix

[rennes]

Hola,

Finally I was able to start the Hurd with the binaries generated with the guix package manager.

At the moment the image of Hurd I have built it manually and does not yet have any functionality like GuixSD / Linux.

Next goal:
  * Generate the image file in the style of guix.
  ** For this I have noticed that it is required to compile qemu for the architecture i586-pc-gnu, at the moment I could not compile qemu in Hurd. I have tried to disable some quemu features that come by default; however, I have not yet achieved it.

Other pending things:
  * Some packages fail during the compilation and testing phases.
  * Other packages used by guix, such as lsof are not ported in Hurd yet.

At the moment it is everything, any doubt or help is welcome.


Rene

Re: boot the Hurd with Guix

[Samuel Thibault]

rennes@openmailbox.org, on sam. 11 nov. 2017 18:02:11 -0000, wrote:
> Finally I was able to start the Hurd with the binaries generated with the guix package manager.

Groovy :D

Samuel

Re: boot the Hurd with Guix

[Manolis Ragkousis]

This is awesome Rene!! Have you uploaded your changes anywhere?

On November 11, 2017 8:02:11 PM GMT+02:00, rennes@openmailbox.org wrote:
>Hola,
>
>Finally I was able to start the Hurd with the binaries generated with
>the guix package manager.
>
>At the moment the image of Hurd I have built it manually and does not
>yet have any functionality like GuixSD / Linux.
>
>Next goal:
>  * Generate the image file in the style of guix.
>** For this I have noticed that it is required to compile qemu for the
>architecture i586-pc-gnu, at the moment I could not compile qemu in
>Hurd. I have tried to disable some quemu features that come by default;
>however, I have not yet achieved it.
>
>Other pending things:
>  * Some packages fail during the compilation and testing phases.
>* Other packages used by guix, such as lsof are not ported in Hurd yet.
>
>At the moment it is everything, any doubt or help is welcome.
>
>
>Rene

Re: boot the Hurd with Guix

[rennes]

> This is awesome Rene!! Have you uploaded your changes anywhere?
> 

Hello Manolis,

I will prepare the documentation and a small image of qemu and upload it.

Re: boot the Hurd with Guix

[Ludovic Courtès]

Hi rennes!

rennes@openmailbox.org skribis:

> Finally I was able to start the Hurd with the binaries generated with the guix package manager.

Woohoo!  Does that mean you were able to run packages cross-compiled
with Guix, or packages built natively with Guix?

> At the moment the image of Hurd I have built it manually and does not yet have any functionality like GuixSD / Linux.

… or did you build a VM image of GNU/Hurd?

> Next goal:
>   * Generate the image file in the style of guix.
>   ** For this I have noticed that it is required to compile qemu for the architecture i586-pc-gnu, at the moment I could not compile qemu in Hurd. I have tried to disable some quemu features that come by default; however, I have not yet achieved it.

Things like ‘guix system vm’ use QEMU because we need to start a VM
because on GNU/Linux one needs to be root to create a file system, mount
it, and populate it.

However, on GNU/Hurd, file systems can be mounted without being root.
So you could in fact skip QEMU altogether, though that’d require
tweaking in the VM-generation code (in (gnu build vm), etc.).

> Other pending things:
>   * Some packages fail during the compilation and testing phases.
>   * Other packages used by guix, such as lsof are not ported in Hurd yet.

We could probably get rid the lsof dependency.  What matters here is a
Linuxish /proc.

Anyway, congrats on this milestone!  :-)

Ludo’.

Re: boot the Hurd with Guix

[Pjotr Prins]

Great work. The Guix team never ceases to amaze me.

Pj.

On Sat, Nov 11, 2017 at 08:47:38PM +0200, Manolis Ragkousis wrote:
> This is awesome Rene!! Have you uploaded your changes anywhere?
> 
> On November 11, 2017 8:02:11 PM GMT+02:00, rennes@openmailbox.org wrote:
> >Hola,
> >
> >Finally I was able to start the Hurd with the binaries generated with
> >the guix package manager.
> >
> >At the moment the image of Hurd I have built it manually and does not
> >yet have any functionality like GuixSD / Linux.
> >
> >Next goal:
> >  * Generate the image file in the style of guix.
> >** For this I have noticed that it is required to compile qemu for the
> >architecture i586-pc-gnu, at the moment I could not compile qemu in
> >Hurd. I have tried to disable some quemu features that come by default;
> >however, I have not yet achieved it.
> >
> >Other pending things:
> >  * Some packages fail during the compilation and testing phases.
> >* Other packages used by guix, such as lsof are not ported in Hurd yet.
> >
> >At the moment it is everything, any doubt or help is welcome.
> >
> >
> >Rene
> 

-- 

Re: boot the Hurd with Guix

[rennes]

Hello Ludovic,

>> Finally I was able to start the Hurd with the binaries generated with the guix package manager.
> 
> Woohoo!  Does that mean you were able to run packages cross-compiled
> with Guix, or packages built natively with Guix?
> 

the packages were built on top of Debian/Hurd. At this moment I am copying everything that is compiled into /gnu/store to the image, 
and then create symbolic links so that grub, Mach and Hurd can locate the binaries.

>> At the moment the image of Hurd I have built it manually and does not yet have any functionality like GuixSD / Linux.
> 
> … or did you build a VM image of GNU/Hurd?
> 

Yes, I have built the VM image according to your notes [1].

Thanks for the feedback.


[1] http://git.savannah.gnu.org/cgit/hydra-recipes.git/tree/hurd

Re: boot the Hurd with Guix

[Ludovic Courtès]

Hi rennes,

(Something went wrong with your message headers.)

rennes@openmailbox.org skribis:

>>> Finally I was able to start the Hurd with the binaries generated with the guix package manager.
>> 
>> Woohoo!  Does that mean you were able to run packages cross-compiled
>> with Guix, or packages built natively with Guix?
>> 
>
> the packages were built on top of Debian/Hurd. At this moment I am copying everything that is compiled into /gnu/store to the image, 
> and then create symbolic links so that grub, Mach and Hurd can locate the binaries.

Neat.

>>> At the moment the image of Hurd I have built it manually and does not yet have any functionality like GuixSD / Linux.
>> 
>> … or did you build a VM image of GNU/Hurd?
>> 
>
> Yes, I have built the VM image according to your notes [1].

Oh, awesome!  I can’t wait to fiddle with this image!

Please make sure to send us a recap of what needs to be done in Guix
proper to make this happen.  We’ll think about ways to incorporate those
changes.

Congrats!

Ludo’.

PS: guix-daemon no longer depends on ‘lsof’, but it still depends on /proc.

Re: boot the Hurd with Guix

[Samuel Thibault]

Hello,

Ludovic Courtès, on lun. 13 nov. 2017 11:42:01 +0100, wrote:
> PS: guix-daemon no longer depends on ‘lsof’, but it still depends on /proc.

Does our procfs have everything it needs already?

Samuel

Re: boot the Hurd with Guix

[Ludovic Courtès]

Hi Samuel,  :-)

Samuel Thibault <samuel.thibault@gnu.org> skribis:

> Ludovic Courtès, on lun. 13 nov. 2017 11:42:01 +0100, wrote:
>> PS: guix-daemon no longer depends on ‘lsof’, but it still depends on /proc.
>
> Does our procfs have everything it needs already?

It needs /proc/PID/{exe,cwd,fd,maps,environ}:

  https://git.savannah.gnu.org/cgit/guix.git/tree/nix/scripts/list-runtime-roots.in

That said, it’s quite optional: the worst that can happen if one of
these things is missing is that something can be reclaimed “prematurely”
from /gnu/store.  I say “prematurely” with quotes because users are
supposed to register “real” GC roots anyway, as with ‘guix package -i’
or ‘guix build --root’.

Thanks,
Ludo’.

Re: boot the Hurd with Guix

[Manolis Ragkousis]

Hello everyone,

On 11/16/2017 12:13 PM, Ludovic Courtès wrote:
> Hi Samuel,  :-)
> 
> Samuel Thibault <samuel.thibault@gnu.org> skribis:
> 
>> Ludovic Courtès, on lun. 13 nov. 2017 11:42:01 +0100, wrote:
>>> PS: guix-daemon no longer depends on ‘lsof’, but it still depends on /proc.
>>
>> Does our procfs have everything it needs already?
> 
> It needs /proc/PID/{exe,cwd,fd,maps,environ}:
> 
>   https://git.savannah.gnu.org/cgit/guix.git/tree/nix/scripts/list-runtime-roots.in
> 
> That said, it’s quite optional: the worst that can happen if one of
> these things is missing is that something can be reclaimed “prematurely”
> from /gnu/store.  I say “prematurely” with quotes because users are
> supposed to register “real” GC roots anyway, as with ‘guix package -i’
> or ‘guix build --root’.

That reminds me, Rene have you tried running 'guix gc -C' ? What is the
output? Because I have a problem with this on Guix on Hurd and I haven't
managed to fix it yet.

Re: boot the Hurd with Guix

[Maxim Cournoyer]

rennes@openmailbox.org writes:

> Finally I was able to start the Hurd with the binaries generated with the guix package manager.

Exciting news! Congrats :)

Maxim

Re: boot the Hurd with Guix

[rennes]

Hello,

This is the demo generated with Guix:

https://github.com/methalo/boot-hurd

The binary files were generated in Debian/Hurd and placed in an 'img' file.

The command used to generate the binaries is:

'./pre-inst-env guix system init ~/light.scm /guix'

To test Hurd, execute:

'sudo qemu-system-i386 -enable-kvm -m 1G -hda guixsdhurd.img -curses'

I hope to upload the patches soon to github, for now I need to understand Guix/Guile to create the patches properly.

Comments and help is welcome.

Re: boot the Hurd with Guix

[Manolis Ragkousis]

Hello Rene :D

On 11/27/17 21:05, rennes@openmailbox.org wrote:
> Hello,
> 
> This is the demo generated with Guix:
> 
> https://github.com/methalo/boot-hurd
> 
> The binary files were generated in Debian/Hurd and placed in an 'img' file.
> 
> The command used to generate the binaries is:
> 
> './pre-inst-env guix system init ~/light.scm /guix'
> 
> To test Hurd, execute:
> 
> 'sudo qemu-system-i386 -enable-kvm -m 1G -hda guixsdhurd.img -curses'
> 
> I hope to upload the patches soon to github, for now I need to understand Guix/Guile to create the patches properly.
> 
> Comments and help is welcome.
> 

The repo has only a Readme file. I think you forgot to add the rest of
the files :P

Keep up the awesome work!!

Manolis

Re: boot the Hurd with Guix

[rennes]

Hello,

I update the download link:

https://ombx.io/ipoWt9uK

Re: boot the Hurd with Guix

[Daniel Pimentel]

Cool! Thanks,

---
Daniel Pimentel (d4n1)

Em 2017-11-27 22:16, rennes@openmailbox.org escreveu:
> Hello,
> 
> I update the download link:
> 
> https://ombx.io/ipoWt9uK

Re: boot the Hurd with Guix

[ranvijay vijay]

[-- Attachment #1: Type: text/plain, Size: 310 bytes --]

Keep up the good work! 😊

On Tuesday, November 28, 2017, Daniel Pimentel <d4n1@d4n1.org> wrote:

> Cool! Thanks,
>
> ---
> Daniel Pimentel (d4n1)
>
> Em 2017-11-27 22:16, rennes@openmailbox.org escreveu:
>
>> Hello,
>>
>> I update the download link:
>>
>> https://ombx.io/ipoWt9uK
>>
>
>

[-- Attachment #2: Type: text/html, Size: 667 bytes --]

Re: boot the Hurd with Guix

[Ludovic Courtès]

Hi rennes,

rennes@openmailbox.org skribis:

> This is the demo generated with Guix:
>
> https://github.com/methalo/boot-hurd
>
> The binary files were generated in Debian/Hurd and placed in an 'img' file.
>
> The command used to generate the binaries is:
>
> './pre-inst-env guix system init ~/light.scm /guix'
>
> To test Hurd, execute:
>
> 'sudo qemu-system-i386 -enable-kvm -m 1G -hda guixsdhurd.img -curses'

[...]

> https://ombx.io/ipoWt9uK

I just gave it a try, and woow!  :-)

It’s really nice to see that in action.

It lacks a couple of things such as the console server and client and
the pipe server, but tweaking this will be the funny part.  ;-)

Also, in GRUB, you currently load ext2fs.static and exec explicitly.
There’s now a /hurd/startup server that takes care of launching
/hurd/proc, /hurd/auth, and then passes control to /libexec/runsystem.
I suppose this is the preferred method, but hurd.texi doesn’t give the
exact GRUB commands.  Can anyone shed some light?

BTW, the image you posted is in “raw” format.  You would get a smaller
file by using the qcow2 format, which you can create with “qemu-img
create -f qcow2”.

Anyway, kudos, and keep up the good work!

Thank you,
Ludo’.

Re: boot the Hurd with Guix

[Samuel Thibault]

Hello,

Congrats on the achievement :D

Ludovic Courtès, on ven. 01 déc. 2017 14:17:48 +0100, wrote:
> Also, in GRUB, you currently load ext2fs.static and exec explicitly.

That's the normal way, yes. exec does the rest (including running
startup).

> BTW, the image you posted is in “raw” format.  You would get a smaller
> file by using the qcow2 format, which you can create with “qemu-img
> create -f qcow2”.

Well, using sparse files can work as well: create the image with

dd if=/dev/zero of=file.img bs=1M count=1 seek=1000

and pass -S to tar so that on decompression it gets sparse too.

The advantage is that standard tools (fdisk, etc.) will work on it.

Samuel

Re: boot the Hurd with Guix

[Vincent Legoll]

On Fri, Dec 1, 2017 at 5:16 PM, Samuel Thibault <samuel.thibault@gnu.org> wrote:
> Hello,
>
> Congrats on the achievement :D
>
> Ludovic Courtès, on ven. 01 déc. 2017 14:17:48 +0100, wrote:
>> Also, in GRUB, you currently load ext2fs.static and exec explicitly.
>
> That's the normal way, yes. exec does the rest (including running
> startup).
>
>> BTW, the image you posted is in “raw” format.  You would get a smaller
>> file by using the qcow2 format, which you can create with “qemu-img
>> create -f qcow2”.
>
> Well, using sparse files can work as well: create the image with
>
> dd if=/dev/zero of=file.img bs=1M count=1 seek=1000

Or you can use truncate from gnu coreutils:

$ du -sh file.img
0    file.img
vince@dell:~$ ls -l file.img
-rw------- 1 vince vince 1073741824 déc.  1 19:43 file.img
vince@dell:~$ ls -lh file.img
-rw------- 1 vince vince 1,0G déc.  1 19:43 file.img


-- 
Vincent Legoll

Re: boot the Hurd with Guix

[Vincent Legoll]

Sorry, the  mail was sent too early

>> Well, using sparse files can work as well: create the image with
>>
>> dd if=/dev/zero of=file.img bs=1M count=1 seek=1000
>
> Or you can use truncate from gnu coreutils:

$ truncate -s 1G file.img
$ du -sh file.img
0    file.img
$ ls -lh file.img
-rw------- 1 vince vince 1,0G déc.  1 19:43 file.img

-- 
Vincent Legoll

Re: boot the Hurd with Guix

[rennes]

Hello,

This is the updated repository with the necessary patches to compile  
GNU Guix in Debian GNU/Hurd.
  https://github.com/methalo/guix

In the README-hurd file, describe the steps to generate the binaries  
required to boot GNU Hurd (Missing more patches).

  ** Current STATUS

Currently I have error in guile-static-stripped-2.2.2 package[1].

  Steps to reproduce the error:

  1) Manually run :
      
'/gnu/store/6fgz3s8fjva40hsdvs2hs0f5p4bw12jc-guile-static-stripped-2.2.2/bin/guile  
--version'

  Output:

guile: warning: failed to install locale
..
Uncaught exception:
Throw to key misc-error with args ("primitive-load-path" "Unable to  
find file ~S in load path" ("ice-9/boot-9") #f)Can\
not exit gracefully when init is in progress; aborting.
Aborted


[1] https://lists.gnu.org/archive/html/guix-devel/2018-01/msg00009.html

Re: boot the Hurd with Guix

[Ludovic Courtès]

Hi rennes,

rennes <rennes@openmailbox.org> skribis:

>   1) Manually run :
>       
> '/gnu/store/6fgz3s8fjva40hsdvs2hs0f5p4bw12jc-guile-static-stripped-2.2.2/bin/guile  
> --version'
>
>   Output:
>
> guile: warning: failed to install locale
> ..
> Uncaught exception:
> Throw to key misc-error with args ("primitive-load-path" "Unable to  
> find file ~S in load path" ("ice-9/boot-9") #f)Can\
> not exit gracefully when init is in progress; aborting.
> Aborted

This must come from the lack of /proc/self/exe.

Specifically, “guile-static-stripped” has this patch to make it
relocatable:

  https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/patches/guile-relocatable.patch

This is necessary for the bootstrap Guile (see make-bootstrap.scm), and
was necessary in the initrd on GNU/Linux (I think we no longer need it
for the initrd).

At any rate, you may be able to boot the Hurd with the “real”,
dynamically-linked Guile, no?  That would solve the problem entirely.

HTH,
Ludo’.

Re: boot the Hurd with Guix

[Svante Signell]

On Mon, 2018-01-08 at 11:13 +0100, Ludovic Courtès wrote:
> Hi rennes,
> 
> rennes <rennes@openmailbox.org> skribis:
> 
> >   1) Manually run :
> >       
> > '/gnu/store/6fgz3s8fjva40hsdvs2hs0f5p4bw12jc-guile-static-stripped-
> > 2.2.2/bin/guile  
> > --version'
> > 
> >   Output:
> > 
> > guile: warning: failed to install locale
> > ..
> > Uncaught exception:
> > Throw to key misc-error with args ("primitive-load-path" "Unable to  
> > find file ~S in load path" ("ice-9/boot-9") #f)Can\
> > not exit gracefully when init is in progress; aborting.
> > Aborted
> 
> This must come from the lack of /proc/self/exe.

/proc/self/exe is supported in glibc for Hurd since 2.24-18. Are you using an
earlier version?

Re: boot the Hurd with Guix

[Svante Signell]

On Mon, 2018-01-08 at 11:20 +0100, Svante Signell wrote:
> On Mon, 2018-01-08 at 11:13 +0100, Ludovic Courtès wrote:

> > This must come from the lack of /proc/self/exe.
> 
> /proc/self/exe is supported in glibc for Hurd since 2.24-18. Are you using an
> earlier version?

And of course Hurd must be newer than Debian version 0.9.git20170507-1

Re: boot the Hurd with Guix

[rennes]

Hello Ludovic,

> > Uncaught exception:
> > Throw to key misc-error with args ("primitive-load-path" "Unable to
> > find file ~S in load path" ("ice-9/boot-9") #f)Can\
> > not exit gracefully when init is in progress; aborting.
> > Aborted
> 
> This must come from the lack of /proc/self/exe.
> 

In Debian GNU/Hurd we've '/proc/self/exe -> /bin/ls'

> Specifically, “guile-static-stripped” has this patch to make it
> relocatable:
> 
> This is necessary for the bootstrap Guile (see make-bootstrap.scm),  
> and
> was necessary in the initrd on GNU/Linux (I think we no longer need it
> for the initrd).
> 
> At any rate, you may be able to boot the Hurd with the “real”,
> dynamically-linked Guile, no?  That would solve the problem entirely.
> 

I will try not to use initrd. Thanks

Re: boot the Hurd with Guix

[rennes]

Hello Svante,

> > /proc/self/exe is supported in glibc for Hurd since 2.24-18. Are  
> you using an
> > earlier version?
> 

Currently we use  glibc/hurd 2.23,
see http://alpha.gnu.org/gnu/hurd/glibc-2.23-hurd+libpthread-20161218.

> And of course Hurd must be newer than Debian version 0.9.git20170507-1
> 
> 

The version level is:
GNU Hurd 0.9 GNU-Mach 1.8+git20170609-486/Hurd-0.9 i686-AT386 GNU

I think the solution is to eliminate the dependency of initrd for the  
moment in Guix, as Ludovic commented.

Thanks