"guix system vm" to libvirt XML domain

"guix system vm" to libvirt XML domain

[Giovanni Biscuolo]

[-- Attachment #1.1: Type: text/plain, Size: 5821 bytes --]

Hi!

I'm using guix on a foreign distro (Debian 9) and I'm already using
libvirtd to manage some KVM guests; I'd like to also manage "guix
system vm" generated ones using libvirt

it would be great to be able to automatically manage the (re)definition
of VMs (and containers?) on - possibly remote - libvirtd servers using a
distributedVM.scm config :-): WDYT?

anyway... as for now it must be "manually" managed and this is
what I'm doing: it fails at the end, what am I doing wrong?

is there a better way to do it?

1. I save the "command line" from the script generated by "guix system
vm" (except the last "@") calling this "container.minimal.args.orig

--8<---------------cut here---------------start------------->8---
/gnu/store/4pnx934k40wni57r5swj2kr29l1j549f-qemu-3.1.0/bin/qemu-system-x86_64 -kernel /gnu/store/y5aiijqiqna1y8ypn6c0jp367813xxnc-linux-libre-4.20.3/bzImage -initrd /gnu/store/9xgqs9zf51i9binqjl5hgl8gl7ql40hc-system/initrd -append "--root=/dev/vda1 --system=/gnu/store/9xgqs9zf51i9binqjl5hgl8gl7ql40hc-system --load=/gnu/store/9xgqs9zf51i9binqjl5hgl8gl7ql40hc-system/boot" -enable-kvm -no-reboot -net nic,model=virtio -object rng-random,filename=/dev/urandom,id=guixsd-vm-rng -device virtio-rng-pci,rng=guixsd-vm-rng -virtfs local,path="/gnu/store",security_model=none,mount_tag="TAGjoptajej2oynju6yvboauz7pl6uj" -vga std -drive file=/gnu/store/4mz1qqv7h5g5885q1aq0izlmy01knf1g-qemu-image,if=virtio,cache=writeback,werror=report,readonly -m  256
--8<---------------cut here---------------end--------------->8---

2. try to convert it to libvirt XML domain definition

--8<---------------cut here---------------start------------->8---
$ virsh --connect qemu:///system domxml-from-native qemu-argv container-minimal.args.orig 
error: internal error: malformed keyword arguments in 'file=/gnu/store/4mz1qqv7h5g5885q1aq0izlmy01knf1g-qemu-image,if=virtio,cache=writeback,werror=report,readonly'
--8<---------------cut here---------------end--------------->8---

3. remove the "readonly" keyword from the last "-drive" arg, calling this
"container.minimal.args" (that *must* be then manually added to the XML
definition)

--8<---------------cut here---------------start------------->8---
/gnu/store/4pnx934k40wni57r5swj2kr29l1j549f-qemu-3.1.0/bin/qemu-system-x86_64 -kernel /gnu/store/y5aiijqiqna1y8ypn6c0jp367813xxnc-linux-libre-4.20.3/bzImage -initrd /gnu/store/9xgqs9zf51i9binqjl5hgl8gl7ql40hc-system/initrd -append "--root=/dev/vda1 --system=/gnu/store/9xgqs9zf51i9binqjl5hgl8gl7ql40hc-system --load=/gnu/store/9xgqs9zf51i9binqjl5hgl8gl7ql40hc-system/boot" -enable-kvm -no-reboot -net nic,model=virtio -object rng-random,filename=/dev/urandom,id=guixsd-vm-rng -device virtio-rng-pci,rng=guixsd-vm-rng -virtfs local,path="/gnu/store",security_model=none,mount_tag="TAGjoptajej2oynju6yvboauz7pl6uj" -vga std -drive file=/gnu/store/4mz1qqv7h5g5885q1aq0izlmy01knf1g-qemu-image,if=virtio,cache=writeback,werror=report -m  256
--8<---------------cut here---------------end--------------->8---

4. converting the file above

--8<---------------cut here---------------start------------->8---
$ virsh --connect qemu:///system domxml-from-native qemu-argv container-minimal.args
<domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
  <name>unnamed</name>
  <uuid>894f668e-32a5-4331-9d30-8604b89ae359</uuid>
  <memory unit='KiB'>262144</memory>
  <currentMemory unit='KiB'>262144</currentMemory>
  <vcpu placement='static'>1</vcpu>
  <os>
    <type arch='x86_64' machine='pc-i440fx-2.8'>hvm</type>
    <kernel>/gnu/store/y5aiijqiqna1y8ypn6c0jp367813xxnc-linux-libre-4.20.3/bzImage</kernel>
    <initrd>/gnu/store/9xgqs9zf51i9binqjl5hgl8gl7ql40hc-system/initrd</initrd>
    <cmdline>--root=/dev/vda1 --system=/gnu/store/9xgqs9zf51i9binqjl5hgl8gl7ql40hc-system --load=/gnu/store/9xgqs9zf51i9binqjl5hgl8gl7ql40hc-system/boot</cmdline>
  </os>
  <features>
    <acpi/>
  </features>
  <clock offset='utc'/>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>destroy</on_reboot>
  <on_crash>destroy</on_crash>
  <devices>
    <emulator>/gnu/store/4pnx934k40wni57r5swj2kr29l1j549f-qemu-3.1.0/bin/qemu-system-x86_64</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw' cache='writeback' error_policy='report'/>
      <source file='/gnu/store/4mz1qqv7h5g5885q1aq0izlmy01knf1g-qemu-image'/>
      <target dev='vda' bus='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </disk>
    <controller type='usb' index='0' model='piix3-uhci'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>
    </controller>
    <controller type='pci' index='0' model='pci-root'/>
    <input type='mouse' bus='ps2'/>
    <input type='keyboard' bus='ps2'/>
    <graphics type='sdl'/>
    <video>
      <model type='vga' vram='16384' heads='1' primary='yes'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </video>
    <memballoon model='none'/>
  </devices>
  <qemu:commandline>
    <qemu:arg value='-object'/>
    <qemu:arg value='rng-random,filename=/dev/urandom,id=guixsd-vm-rng'/>
    <qemu:arg value='-device'/>
    <qemu:arg value='virtio-rng-pci,rng=guixsd-vm-rng'/>
    <qemu:arg value='-virtfs'/>
    <qemu:arg value='local,path=&quot;/gnu/store&quot;,security_model=none,mount_tag=&quot;TAGjoptajej2oynju6yvboauz7pl6uj&quot;'/>
  </qemu:commandline>
</domain>
--8<---------------cut here---------------end--------------->8---

5. unfortunately some command line args are not automatically converted,
so I have to "manually" convert it [1]... also adapting some parametrs
to my needs (e.g. name, memory, network, graphics); saving as
container-minimal-fixed.xml


[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

[-- Attachment #2: container-minimal-fixed.xml --]
[-- Type: application/xml, Size: 2363 bytes --]

<domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
  <name>container-minimal</name>
  <uuid>ecfe2538-5315-44aa-bba5-8fe4d86d5397</uuid>
  <memory unit='GB'>2</memory>
  <currentMemory unit='GB'>2</currentMemory>
  <vcpu placement='static'>2</vcpu>
  <os>
    <type arch='x86_64' machine='pc-i440fx-2.8'>hvm</type>
    <kernel>/gnu/store/y5aiijqiqna1y8ypn6c0jp367813xxnc-linux-libre-4.20.3/bzImage</kernel>
    <initrd>/gnu/store/9xgqs9zf51i9binqjl5hgl8gl7ql40hc-system/initrd</initrd>
    <cmdline>--root=/dev/vda1 --system=/gnu/store/9xgqs9zf51i9binqjl5hgl8gl7ql40hc-system --load=/gnu/store/9xgqs9zf51i9binqjl5hgl8gl7ql40hc-system/boot</cmdline>
  </os>
  <features>
    <acpi/>
  </features>
  <clock offset='utc'/>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>destroy</on_reboot>
  <on_crash>destroy</on_crash>
  <devices>
    <emulator>/gnu/store/4pnx934k40wni57r5swj2kr29l1j549f-qemu-3.1.0/bin/qemu-system-x86_64</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw' cache='writeback' error_policy='report'/>
      <source file='/gnu/store/4mz1qqv7h5g5885q1aq0izlmy01knf1g-qemu-image'/>
      <target dev='vda' bus='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
      <readonly/>
    </disk>
    <controller type='usb' index='0' model='piix3-uhci'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>
    </controller>
    <controller type='pci' index='0' model='pci-root'/>
    <input type='mouse' bus='ps2'/>
    <input type='keyboard' bus='ps2'/>
    <graphics type='spice' port='3003' autoport='no' listen='127.0.0.1'>
      <listen type='address' address='127.0.0.1'/>
    </graphics>
    <video>
      <model type='qxl' ram='65536' vram='16384' vgamem='16384' heads='1' primary='yes'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </video>
    <memballoon model='none'/>
    <rng model='virtio'>
     <backend model='random'>/dev/urandom</backend>
    </rng>
    <interface type='network'>
     <mac address='52:54:00:ae:72:b9'/>
     <source network='gblabs'/>
     <model type='rtl8139'/>
   </interface>
  </devices>
  <filesystem type='mount' accessmode='none'>
   <source dir='/gnu/store'/>
   <target dir='TAGjoptajej2oynju6yvboauz7pl6uj'/>
  </filesystem>
</domain>

[-- Attachment #3: Type: text/plain, Size: 1032 bytes --]


6. register the domain definition in libvirtd

--8<---------------cut here---------------start------------->8---
$ virsh --connect qemu:///system define container-minimal-fixed.xml
Domain container-minimal defined from container-minimal-fixed.xml
--8<---------------cut here---------------end--------------->8---

7. start the VM, connect via SPICE and see the VM does not boot due to
an error in procedure mount, falling back to the guile REPL; the backtrace
is:

--8<---------------cut here---------------start------------->8---
In gnu/build/linux-boot.scm:
   516:13  2 (_)
    367:8  1 (mount-root-file-system "/dev/vda1" "ext4" # _)
In unknown file:
           0 (mount "/dev/vda1" "/real-root" "ext4" 1 #<undefined>)
--8<---------------cut here---------------end--------------->8---

any hint please?

Thanks!
Giovanni

[1] using this docs as ref:
https://wiki.qemu.org/Documentation/9psetup#Starting_the_Guest_using_libvirt
https://libvirt.org/formatdomain.html#elementsRng

-- 
Giovanni Biscuolo

Xelera IT Infrastructures

Re: "guix system vm" to libvirt XML domain

[Giovanni Biscuolo]

[-- Attachment #1: Type: text/plain, Size: 1428 bytes --]

Hi,

reply to myself:

Giovanni Biscuolo <g@xelera.eu> writes:

[...]

> 7. start the VM, connect via SPICE and see the VM does not boot due to
> an error in procedure mount, falling back to the guile REPL; the backtrace
> is:
>
> --8<---------------cut here---------------start------------->8---
> In gnu/build/linux-boot.scm:
>    516:13  2 (_)
>     367:8  1 (mount-root-file-system "/dev/vda1" "ext4" # _)
> In unknown file:
>            0 (mount "/dev/vda1" "/real-root" "ext4" 1 #<undefined>)
> --8<---------------cut here---------------end--------------->8---

OK I solved this, it was "virsh domxml-from-native" that missed to detect
the correct disk type of the vda disk, so I changed this part of the XML
domain definition:

--8<---------------cut here---------------start------------->8---
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2' cache='writeback' error_policy='report'/>
      <source file='/gnu/store/4mz1qqv7h5g5885q1aq0izlmy01knf1g-qemu-image'/>
      <target dev='vda' bus='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </disk>
--8<---------------cut here---------------end--------------->8---

type='raw' changed to type='qcow2'

but now I've a problem with the /gnu/store mount in guest

"9pnet_virtio: no channels available for device TAGjoptajej2oynju6yvboauz7pl6uj"

...still missimg someting

any hint please?

Thanks!


[-- Attachment #2.1: Type: text/plain, Size: 53 bytes --]

-- 
Giovanni Biscuolo

Xelera IT Infrastructures

[-- Attachment #2.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: "guix system vm" to libvirt XML domain

[Gábor Boskovits]

Hello,

Giovanni Biscuolo <g@xelera.eu> ezt írta (időpont: 2019. jan. 24., Cs, 20:05):
>
> Hi,
>
> reply to myself:
>
> Giovanni Biscuolo <g@xelera.eu> writes:
>
> [...]
>
> > 7. start the VM, connect via SPICE and see the VM does not boot due to
> > an error in procedure mount, falling back to the guile REPL; the backtrace
> > is:
> >
> > --8<---------------cut here---------------start------------->8---
> > In gnu/build/linux-boot.scm:
> >    516:13  2 (_)
> >     367:8  1 (mount-root-file-system "/dev/vda1" "ext4" # _)
> > In unknown file:
> >            0 (mount "/dev/vda1" "/real-root" "ext4" 1 #<undefined>)
> > --8<---------------cut here---------------end--------------->8---
>
> OK I solved this, it was "virsh domxml-from-native" that missed to detect
> the correct disk type of the vda disk, so I changed this part of the XML
> domain definition:
>
> --8<---------------cut here---------------start------------->8---
>     <disk type='file' device='disk'>
>       <driver name='qemu' type='qcow2' cache='writeback' error_policy='report'/>
>       <source file='/gnu/store/4mz1qqv7h5g5885q1aq0izlmy01knf1g-qemu-image'/>
>       <target dev='vda' bus='virtio'/>
>       <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
>     </disk>
> --8<---------------cut here---------------end--------------->8---
>
> type='raw' changed to type='qcow2'
>
> but now I've a problem with the /gnu/store mount in guest
>
> "9pnet_virtio: no channels available for device TAGjoptajej2oynju6yvboauz7pl6uj"
>
> ...still missimg someting
>
> any hint please?
>

I have seen this once, and the virtio-pci kernel module was missing.
When you run it from guix system
vm, then what bus you have the drive on? It might be easier to
replicate that version, I don't know if it
is virtio, but I guess not.


> Thanks!
>
> --
> Giovanni Biscuolo
>
> Xelera IT Infrastructures

Best regards,
g_bor

p.s.: A nice interface to libvirt would be a very good idea.

Re: "guix system vm" to libvirt XML domain

[Giovanni Biscuolo]

[-- Attachment #1: Type: text/plain, Size: 1570 bytes --]

Hi Gabor,

thanks for your interest

Gábor Boskovits <boskovits@gmail.com> writes:

[...]

>> but now I've a problem with the /gnu/store mount in guest
>>
>> "9pnet_virtio: no channels available for device TAGjoptajej2oynju6yvboauz7pl6uj"
>>
>> ...still missimg someting
>>
>> any hint please?
>>
>
> I have seen this once, and the virtio-pci kernel module was missing.

I had excluded this since starting the VM using the "guix system vm"
generated script succeeds, I expect the libvirt defined one to run alike

long story short: my domain XML definition was *wrong*, I defined the
/gnu/store filesystem

--8<---------------cut here---------------start------------->8---
  <filesystem type='mount' accessmode='passthrough'>
    <driver type='path' wrpolicy='immediate'/>
    <source dir='/gnu/store'/>
    <target dir='TAGjoptajej2oynju6yvboauz7pl6uj'/>
  </filesystem>
--8<---------------cut here---------------end--------------->8---

in the wrong part of the XML tree, namely outside

--8<---------------cut here---------------start------------->8---
<devices>
[...]
</devices>
--8<---------------cut here---------------end--------------->8---

and virsh ignored it (no error, no warning) when I used that XML to
define the VM via "virsh ... define cointainer-minimal.xml" [1]

hint: always use virsh-xml-validate to check the file, since virsh does
not validate it before registering the definition (AFAIK "virsh define"
also hasn't a proper flag to do it)

this is the correct XML domain definition, now working:

[-- Attachment #2: container-minimal-fixed.xml --]
[-- Type: application/xml, Size: 2431 bytes --]

<domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
  <name>container-minimal</name>
  <uuid>ecfe2538-5315-44aa-bba5-8fe4d86d5397</uuid>
  <memory unit='GB'>2</memory>
  <currentMemory unit='GB'>2</currentMemory>
  <vcpu placement='static'>2</vcpu>
  <os>
    <type arch='x86_64' machine='pc-i440fx-2.8'>hvm</type>
    <kernel>/gnu/store/y5aiijqiqna1y8ypn6c0jp367813xxnc-linux-libre-4.20.3/bzImage</kernel>
    <initrd>/gnu/store/9xgqs9zf51i9binqjl5hgl8gl7ql40hc-system/initrd</initrd>
    <cmdline>--root=/dev/vda1 --system=/gnu/store/9xgqs9zf51i9binqjl5hgl8gl7ql40hc-system --load=/gnu/store/9xgqs9zf51i9binqjl5hgl8gl7ql40hc-system/boot</cmdline>
  </os>
  <features>
    <acpi/>
  </features>
  <clock offset='utc'/>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>destroy</on_reboot>
  <on_crash>destroy</on_crash>
  <devices>
    <emulator>/gnu/store/4pnx934k40wni57r5swj2kr29l1j549f-qemu-3.1.0/bin/qemu-system-x86_64</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2' cache='writeback' error_policy='report'/>
      <source file='/gnu/store/4mz1qqv7h5g5885q1aq0izlmy01knf1g-qemu-image'/>
      <target dev='vda' bus='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
      <readonly/>
    </disk>
    <filesystem type='mount' accessmode='passthrough'>
      <driver type='path' wrpolicy='immediate'/>
      <source dir='/gnu/store'/>
      <target dir='TAGjoptajej2oynju6yvboauz7pl6uj'/>
    </filesystem>
    <controller type='usb' index='0' model='piix3-uhci'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>
    </controller>
    <controller type='pci' index='0' model='pci-root'/>
    <input type='mouse' bus='ps2'/>
    <input type='keyboard' bus='ps2'/>
    <graphics type='spice' port='3003' autoport='no' listen='127.0.0.1'>
      <listen type='address' address='127.0.0.1'/>
    </graphics>
    <video>
      <model type='qxl' ram='65536' vram='16384' vgamem='16384' heads='1' primary='yes'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </video>
    <memballoon model='none'/>
    <rng model='virtio'>
     <backend model='random'>/dev/urandom</backend>
    </rng>
    <interface type='network'>
     <mac address='52:54:00:ae:72:b9'/>
     <source network='gblabs'/>
     <model type='rtl8139'/>
   </interface>
  </devices>
</domain>

[-- Attachment #3: Type: text/plain, Size: 175 bytes --]


hope this will help others!

happy hacking!
Giovanni

[...]

[1] /var/log/libvirt/qemu/container-minimal.log was helpful since the
command line used to start the VM lacked 


[-- Attachment #4.1: Type: text/plain, Size: 53 bytes --]

-- 
Giovanni Biscuolo

Xelera IT Infrastructures

[-- Attachment #4.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]