Re: GuixSD on servers [Fwd: [rtracker.1984.is #131647] A question about VServer system specific requirements]

[Chris Marusich <cmmarusich@gmail.com>]

[-- Attachment #1.1: Type: text/plain, Size: 3940 bytes --]

ng0 <contact.ng0@cryptolab.net> writes:

> Hi, a short update on the multiple providers I'm talking with at the moment.
> This just in after the holidays from 1984 ehf, iceland.
>
> All in all not problematic, and they'd even add GuixSD to the list of available isos
> once there is an iso.
>
> So, what's the status on the ISO, who's working on it?

I've been looking into this off and on over the last few weeks, but I
haven't made any breakthroughs.  The closest I got was this:

1) Create a disk image for testing:

  ./pre-inst-env guix system --root=/tmp/disk-image-gc-root --fallback disk-image ~/guix/gnu/system/install.scm
  cp $the_resulting_path /tmp/disk-image
  
2) Try to boot it (with an attached hard disk), and watch it fail:

  qemu-img create -f qcow2 /tmp/test 10G
  sudo qemu-system-x86_64 -machine type=pc-i440fx-2.5,accel=kvm -boot order=dc,menu=on -m size=4G -k en-us -name guixsd -cdrom "/tmp/disk-image" "/tmp/test"

3) Mount it as loopback device:

  sudo losetup -P /dev/loop0 /tmp/disk-image
  sudo mkdir /mnt/disk-image-partition-1
  sudo mount /dev/loop0p1 /mnt/disk-image-partition-1
  
4) Make a bootable CD-ROM image of it (see (grub) Invoking grub-mkrescue):

  sudo grub-mkrescue -o /tmp/test-img.iso /mnt/disk-image-partition-1

5) Try to boot (partial success):

  sudo qemu-system-x86_64 -machine type=pc-i440fx-2.5,accel=kvm -boot order=dc,menu=on -m size=4G -k en-us -name guixsd -cdrom "/tmp/disk-image" "/tmp/test"

There appear to be (at least) two problem that prevent this naive
solution from working, which might point us in the right direction:

First, the GRUB menu is trying to find a file system with label
"gnu-disk-image" (via "search --label --set gnu-disk-image"), which
won't work because there is no file system with that label in the
resulting image.  Possible fix: the manual for grub-mkrescue says "The
root device will be set up appropriately on entering your 'grub.cfg'
configuration file", so perhaps we can simply omit our --search.  FYI,
the boot process continues successfully past this point precisely
because GRUB has already set the root; the fact that our search command
failed generates an error message but does not change the fact that it
succeeds in booting to the initrd.
  
Second, the init process from the initrd (I think that's what it's
called?) is trying to look for a file system with label
"gnu-disk-image", which it never finds.  It just sits there waiting to
find it, and it never shows up, so it freaks out.  Possible solution:
modify the behavior of our initrd's init process.  I'm not sure how to
customize the init process here, but there must be a way.  We'll
probably also need the kernel module that enables reading of iso9660
file systems, if it wasn't present already.

If you don't like grub-mkrescue, you can "roll your own" ISO generation
program, like Nix does by customizing the xorriso invocation [1]...  But
honestly, it looks pretty complicated [2].  So if we can let
grub-mkrescue do that work for us, that would be swell.

I've attached my raw notes that I took while trying to figure this out.
Hopefully somebody will find it useful.  Like most things that have to
do with computers, it's surprisingly complicated.  I'm very impressed by
what the NixOS folks were able to accomplish!

[1]
https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/installer/cd-dvd/iso-image.nix
https://github.com/NixOS/nixpkgs/blob/master/nixos/lib/make-iso9660-image.nix
https://github.com/NixOS/nixpkgs/blob/master/nixos/lib/make-iso9660-image.sh

[2] The xorriso-fs manual page (info xorriso-fs) explains most of the
options that Nix uses pretty well.  The xorriso manual (info xorriso) is
a bit more dense.  To be honest, I had a hard time figuring out what we
would have to do while reading xorriso's manual; xorriso-fs's manual is
more approachable.

-- 
Chris

[-- Attachment #1.2: guix-iso.txt --]
[-- Type: text/plain, Size: 9585 bytes --]

Goal: generate a bootable CD image.  Ideally, augment disk-image to be hybrid!

Already know that Nix creates one like this:

https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/installer/cd-dvd/iso-image.nix
https://github.com/NixOS/nixpkgs/blob/master/nixos/lib/make-iso9660-image.nix
https://github.com/NixOS/nixpkgs/blob/master/nixos/lib/make-iso9660-image.sh

Plan: try to do what Nix has done, but in Guix!

Phase 1: Figure out how to verify that it works.

Do this by getting an image of Nix and setting it up in qemu.  We
should be able to do exactly the same sort of thing to verify that
Guix is working...once we've implemented all of this!

Get Nix ISO: https://nixos.org/nixos/download.html

Start up qemu:
Create a qemu disk (to hold the installed system):

$ qemu-img create -f qcow2 /tmp/test 10G
Formatting 'test', fmt=qcow2 size=10737418240 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16

This works for testing it:

$ sudo qemu-system-x86_64 -machine type=pc-i440fx-2.5,accel=kvm -boot order=dc,menu=on -m size=4G -k en-us -name nixos -cdrom "/tmp/nixos-minimal-16.09.1836.067e66a-x86_64-linux.iso" "/tmp/test"

Confirm that this does not work with our current image:

$ qemu-img create -f qcow2 /tmp/test 10G
$ sudo qemu-system-x86_64 -machine type=pc-i440fx-2.5,accel=kvm -boot order=dc,menu=on -m size=4G -k en-us -name guixsd -cdrom "/tmp/disk-image" "/tmp/test"

OK, it fails.  Great!  Now we have a way to test for success!

Phase 2: Manually invoke xorriso to create the image.

Now that we know how to verify success, let's see if we can manually
create the image.  We'll invoke xorriso, following Nix's example.

We need the following stuff:

* A bootloader image.  What to use?  

* All the files that will reside in the OS's file system.  Clearly,
  this will include all the /gnu/store files, the /var/guix stuff, the
  bootloader configuration files, the system activation scripts - all
  of it.

* The right options for use with xorriso.  What to use? Nix provides
  an example; we'll use whatever we need to to get it working
  minimally.

First, let's start by trying to figure out how to invoke xorriso
correctly.  Conveniently, we already have all the files we need
sitting in the disk image.  It isn't an ISO, but it does contain
exactly the files we want, so let's worry first about how to invoke
xorriso after correctly generating the files we need.  We can worry
about the best way to correctly generate those files later.

To get at the files contained in the disk image, let's make it into a
loopback device:

$ sudo losetup -P /dev/loop0 /tmp/disk-image

$ sudo blkid /dev/loop0*
/dev/loop0: PTUUID="00831a15" PTTYPE="dos"
/dev/loop0p1: LABEL="root" UUID="fe1943fc-4cb8-44c6-a8a5-343a87bed681" TYPE="ext4" PARTUUID="00831a15-01"

$ ls /dev/loop0*
/dev/loop0  /dev/loop0p1

$ sudo mkdir /mnt/disk-image-partition-1

$ sudo mount /dev/loop0p1 /mnt/disk-image-partition-1

$ ls /mnt/disk-image-partition-1/
bin/  boot/  etc/  gnu/  home/  lost+found/  mnt/  root/  run/  tmp/  var/

Make a list of all the files:

$ sudo find /mnt/disk-image-partition-1/ > /tmp/all_files

$ head /tmp/all_files 
/mnt/disk-image-partition-1/
/mnt/disk-image-partition-1/tmp
/mnt/disk-image-partition-1/bin
/mnt/disk-image-partition-1/gnu
/mnt/disk-image-partition-1/gnu/store
/mnt/disk-image-partition-1/gnu/store/ykdzlcdyjjfhivids91d1xs36hmzrrp6-gmp-6.1.1
/mnt/disk-image-partition-1/gnu/store/ykdzlcdyjjfhivids91d1xs36hmzrrp6-gmp-6.1.1/include
/mnt/disk-image-partition-1/gnu/store/ykdzlcdyjjfhivids91d1xs36hmzrrp6-gmp-6.1.1/include/gmpxx.h
/mnt/disk-image-partition-1/gnu/store/ykdzlcdyjjfhivids91d1xs36hmzrrp6-gmp-6.1.1/include/gmp.h
/mnt/disk-image-partition-1/gnu/store/ykdzlcdyjjfhivids91d1xs36hmzrrp6-gmp-6.1.1/lib


The Xorriso manual seems to say I need to run this:

     El Torito only for GRUB: -boot_image "grub" "patch"

But it also says:

El Torito boot images have to be added to the ISO image by normal
     means (image loading, -map, -add, ...)

So I guess we have to add the files "first".

     *system_area=*disk_path copies at most 32768 bytes from the given
     disk file to the very start of the ISO image.  This System Area is
     reserved for system dependent boot software, e.g.  an MBR which can
     be used to boot from USB stick or hard disk.

OK, sure.  We'll try that.

     *grub2_mbr=*disk_path works like "any" system_area= with additional
     patching for modern GRUB MBRs.  The content start address of the
     first boot image is converted to a count of 512 byte blocks, and an
     offset of 4 is added.  The result is written as 64 bit
     little-endian number to byte address 0x1b0.
     This feature can be revoked either by grub2_mbr= with empty disk
     path, or by submitting a disk_path via system_area=.

What?  OK, sure, maybe we'll try that...  This is getting absurd.

Turns out, xororisofs manual is the one that contains the options that
Nix is using.

from xorrisofs manual (sec. "Bootable")
xorriso composes the boot catalog according to the boot image files
given and structured by options -b, -e, -el-torito-alt-boot, and
--efi-boot.

'xorrisofs' supports the example options out of the ISOLINUX wiki, the
options used in GRUB script grub-mkrescue, and the example in the
FreeBSD AvgLiveCD wiki.

OK, that's neat, but what are those options and what do they do?

need for sure:

-eltorito-boot
-eltorito-catalog
-no-emul-boot
-boot-load-size 4
-boot-info-table


Don't worry about EFI.  Don't worry about making it a hybrid ISO.
Just get El Torito boot working with PC-BIOS first.

Where do the boot image and catalog files come from?  GRUB package,
probably.

Hold the phone. According to GRUB's manual, it has a program that
makes bootable CD ROMs.  It's called grub-mkrescue.  Let's try
grub-mkrescue to see if we can save ourselves some pain.

$ sudo grub-mkrescue -o /tmp/test-img.iso /mnt/disk-image-partition-1GNU xorriso 1.4.6 : RockRidge filesystem manipulator, libburnia project.

Drive current: -outdev 'stdio:/tmp/test-img.iso'
Media current: stdio file, overwriteable
Media status : is blank
Media summary: 0 sessions, 0 data blocks, 0 data, 66.9g free
Added to ISO image: directory '/'='/tmp/grub.Qwzvrl'
xorriso : UPDATE : 318 files added in 1 seconds
xorriso : UPDATE : 15300 files added in 1 seconds
xorriso : UPDATE : 32000 files added in 2 seconds
xorriso : UPDATE : 37500 files added in 3 seconds
xorriso : UPDATE : 46400 files added in 4 seconds
xorriso : UPDATE : 48700 files added in 5 seconds
xorriso : UPDATE : 50300 files added in 6 seconds
xorriso : UPDATE : 51600 files added in 7 seconds
xorriso : UPDATE : 52600 files added in 8 seconds
xorriso : UPDATE : 53600 files added in 9 seconds
xorriso : UPDATE : 54500 files added in 10 seconds
xorriso : UPDATE : 55400 files added in 11 seconds
xorriso : UPDATE : 56300 files added in 13 seconds
xorriso : UPDATE : 57100 files added in 14 seconds
xorriso : UPDATE : 57900 files added in 15 seconds
xorriso : UPDATE : 58700 files added in 16 seconds
xorriso : UPDATE : 59400 files added in 17 seconds
xorriso : UPDATE : 60100 files added in 18 seconds
xorriso : UPDATE : 60800 files added in 19 seconds
xorriso : UPDATE : 61400 files added in 20 seconds
xorriso : UPDATE : 62000 files added in 21 seconds
xorriso : UPDATE : 62600 files added in 22 seconds
xorriso : UPDATE : 63200 files added in 23 seconds
xorriso : UPDATE : 63800 files added in 24 seconds
Added to ISO image: directory '/'='/mnt/disk-image-partition-1'
xorriso : UPDATE : 66811 files added in 26 seconds
xorriso : NOTE : Copying to System Area: 512 bytes from file '/gnu/store/2hxz9cpipsbf2hkiz5aq70k73wjj0fw1-grub-2.02rc1/lib/grub/i386-pc/boot_hybrid.img'
xorriso : UPDATE :  0.92% done
xorriso : UPDATE :  11.00% done
xorriso : UPDATE :  16.79% done, estimate finish Mon Apr 17 20:52:18 2017
xorriso : UPDATE :  23.60% done, estimate finish Mon Apr 17 20:52:19 2017
xorriso : UPDATE :  28.89% done, estimate finish Mon Apr 17 20:52:20 2017
xorriso : UPDATE :  34.79% done, estimate finish Mon Apr 17 20:52:20 2017
xorriso : UPDATE :  41.42% done, estimate finish Mon Apr 17 20:52:20 2017
xorriso : UPDATE :  49.49% done, estimate finish Mon Apr 17 20:52:20 2017
xorriso : UPDATE :  58.22% done, estimate finish Mon Apr 17 20:52:19 2017
xorriso : UPDATE :  69.48% done, estimate finish Mon Apr 17 20:52:19 2017
xorriso : UPDATE :  83.75% done, estimate finish Mon Apr 17 20:52:17 2017
xorriso : UPDATE :  97.00% done
ISO image produced: 361772 sectors
Written to medium : 361772 sectors at LBA 0
Writing to 'stdio:/tmp/test-img.iso' completed successfully.


let's try it out!

  sudo qemu-system-x86_64 -machine type=pc-i440fx-2.5,accel=kvm -boot order=dc,menu=on -m size=4G -k en-us -name guixsd -cdrom "/tmp/test-img.iso" "/tmp/test"

didn't work, can't find gnu-disk-image.  but GRUB got to the menu, and
then it tried to boot.

two problems:

* GRUB menu is trying to find a fs with label "gnu-disk-image", which
  won't work.  we could instead just tell it to use (cd,msdos1)
  directly through a custom entry, maybe.  or do a file search.
  
* init process is trying to look for a fs with label "gnu-disk-image",
  which it never finds.  We could instead just arrange for it to just
  load up the right partition.



Phase 3: Use Guix to build the ISO.

haven't gotten that far yet! :-)

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]