[bug#41961] [PATCH 0/1] services: childhurd: Support more than one instance.

[bug#41961] [PATCH 0/1] services: childhurd: Support more than one instance.

[Jan (janneke) Nieuwenhuizen]

From: Jan Nieuwenhuizen <janneke@gnu.org>

Hi!

The first thing "someone" asked me when we finally got this amazing service
running and were still pretty excited about it, was

    can we have more than one Childhurd per host?[0]

Sounds like appreciation combined with a valid request, especially since it
implies the intention to setup some serious Hurd build nodes ;-)

Running two Childhurd instances on my laptop right now, using this
configuration

--8<---------------cut here---------------start------------->8---
      (service hurd-vm-service-type
               (hurd-vm-configuration
                (disk-size (* 5000 (expt 2 20))) ;5G
                (memory-size 1024)))

      (service hurd-vm-service-type
               (hurd-vm-configuration
                (os (operating-system
                      (inherit %hurd-vm-operating-system)
                      (host-name "childhurd1")))
                (id 1)))
--8<---------------cut here---------------end--------------->8---

look:

--8<---------------cut here---------------start------------->8---
$ sudo herd status | grep hurd
 + hurd-vm
 + hurd-vm1
11:10:46 janneke@dundal:~/src/guix/master [env]
$ ssh childhurd 'hostname; df --si / 2>/dev/null | tail -1'
childhurd
-               5.3G  934M  4.1G  19% /
11:11:00 janneke@dundal:~/src/guix/master [env]
$ ssh childhurd1 'hostname; df --si / 2>/dev/null | tail -1'
childhurd1
-               1.3G  894M  272M  77% /
--8<---------------cut here---------------end--------------->8---

Greetings,
Janneke

[0] http://logs.guix.gnu.org/guix/2020-06-13.log#001939

Jan (janneke) Nieuwenhuizen (1):
  services: childhurd: Support more than one instance.

 doc/guix.texi                   | 35 ++++++++++++++++++++--------
 gnu/services/virtualization.scm | 41 +++++++++++++++++++++++++--------
 2 files changed, 58 insertions(+), 18 deletions(-)

-- 
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com

[bug#41961] [PATCH 1/1] services: childhurd: Support more than one instance.

[Jan (janneke) Nieuwenhuizen]

* gnu/services/virtualization.scm (<hurd-vm-configuration>)[id,net-options]:
New fields.
(hurd-vm-net-options): New prodecure.  Parameterize port forwarding with ID.
* gnu/services/virtualization.scm (hurd-vm-shepherd-service): Use them.
Parameterize provision with ID, if set.
* doc/guix.texi (Virtualization Services): Document new fields.
---
 doc/guix.texi                   | 35 ++++++++++++++++++++--------
 gnu/services/virtualization.scm | 41 +++++++++++++++++++++++++--------
 2 files changed, 58 insertions(+), 18 deletions(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index 2268e159a2..59f8a89387 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -24728,12 +24728,31 @@ The size of the disk image.
 @item @code{memory-size} (default: @code{512})
 The memory size of the Virtual Machine in mebibytes.
 
-@item @code{options} (default: @code{'("--device"} @code{"rtl8139,netdev=net0"} @
-      @code{"--netdev"} @
-      @code{"user,id=net0,hostfwd=tcp:127.0.0.1:20022-:2222,hostfwd=tcp:127.0.0.1:25900-:5900"} @
-      @code{"--snapshot"} @
-      @code{"--hda")})
+@item @code{options} (default: @code{'("--snapshot" "--hda")})
 The extra options for running QEMU.
+
+@item @code{id} (default: @code{#f})
+If set, a non-zero positive integer used to parameterize Childhurd
+instances.  It is appended to the service's name,
+e.g. @code{childhurd1}.
+
+@item @code{net-options} (default: @var{hurd-vm-net-options})
+The procedure used to produce the list of QEMU networking options.
+
+By default, it produces
+
+@lisp
+'("--device" "rtl8139,netdev=net0"
+  "--netdev" "user,id=net0\
+              ,hostfwd=tcp:127.0.0.1:<ssh-port>-:2222\
+              ,hostfwd=tcp:127.0.0.1:<vnc-port>-:5900")
+@end lisp
+with forwarded ports
+@example
+<ssh-port>: @code{(+ 10022 (* 1000 @var{ID}))}
+<vnc-port>: @code{(+ 15900 (* 1000 @var{ID}))}
+@end example
+
 @end table
 @end deftp
 
@@ -24745,10 +24764,8 @@ the @code{--snapshot} flag using something along these lines:
 @lisp
 (service hurd-vm-service-type
          (hurd-vm-configuration
-          (image    (const "/out/of/store/writable/hurd.img"))
-          (options '("--device" "rtl8139,netdev=net0"
-                     "--netdev"
-                     "user,id=net0,hostfwd=tcp:127.0.0.1:20022-:2222"))))
+          (image   (const "/out/of/store/writable/hurd.img"))
+          (options '("--hda"))))
 @end lisp
 
 @node Version Control Services
diff --git a/gnu/services/virtualization.scm b/gnu/services/virtualization.scm
index 4e96607680..e60d169791 100644
--- a/gnu/services/virtualization.scm
+++ b/gnu/services/virtualization.scm
@@ -51,6 +51,10 @@
 
   #:export (%hurd-vm-operating-system
             hurd-vm-configuration
+            hurd-vm-disk-image
+            hurd-vm-id
+            hurd-vm-net-options
+            hurd-vm-options
             hurd-vm-service-type
 
             libvirt-configuration
@@ -833,13 +837,13 @@ functionality of the kernel Linux.")))
                (default 512))
   (options     hurd-vm-configuration-options            ;list of string
                (default
-                 `("--device" "rtl8139,netdev=net0"
-                   "--netdev" ,(string-append
-                                "user,id=net0"
-                                ",hostfwd=tcp:127.0.0.1:20022-:2222"
-                                ",hostfwd=tcp:127.0.0.1:25900-:5900")
-                   "--snapshot"
-                   "--hda"))))
+                 `("--snapshot"
+                   "--hda")))
+  (id          hurd-vm-configuration-id                 ;#f or integer [1..]
+               (default #f))
+  (net-options hurd-vm-configuration-net-options        ;list of string
+               (thunked)
+               (default (hurd-vm-net-options this-record))))
 
 (define (hurd-vm-disk-image config)
   "Return a disk-image for the Hurd according to CONFIG."
@@ -851,26 +855,45 @@ functionality of the kernel Linux.")))
       (size disk-size)
       (operating-system os)))))
 
+(define (hurd-vm-net-options config)
+  (let ((id (or (hurd-vm-configuration-id config) 0)))
+    (define (qemu-vm-port base)
+      (number->string (+ base (* 1000 id))))
+    `("--device" "rtl8139,netdev=net0"
+      "--netdev" ,(string-append
+                   "user,id=net0"
+                   ",hostfwd=tcp:127.0.0.1:" (qemu-vm-port 10022) "-:2222"
+                   ",hostfwd=tcp:127.0.0.1:" (qemu-vm-port 15900) "-:5900"))))
+
 (define (hurd-vm-shepherd-service config)
   "Return a <shepherd-service> for a Hurd in a Virtual Machine with CONFIG."
 
   (let ((image       (hurd-vm-configuration-image config))
         (qemu        (hurd-vm-configuration-qemu config))
         (memory-size (hurd-vm-configuration-memory-size config))
-        (options     (hurd-vm-configuration-options config)))
+        (options     (hurd-vm-configuration-options config))
+        (id          (hurd-vm-configuration-id config))
+        (net-options (hurd-vm-configuration-net-options config))
+        (provisions  '(hurd-vm childhurd)))
 
     (define vm-command
       #~(list
          (string-append #$qemu "/bin/qemu-system-i386")
          #$@(if (file-exists? "/dev/kvm") '("--enable-kvm") '())
          "-m" (number->string #$memory-size)
+         #$@net-options
          #$@options
          #+image))
 
     (list
      (shepherd-service
       (documentation "Run the Hurd in a Virtual Machine: a Childhurd.")
-      (provision '(hurd-vm childhurd))
+      (provision (if id
+                     (map
+                      (cute symbol-append <>
+                            (string->symbol (number->string id)))
+                      provisions)
+                     provisions))
       (requirement '(networking))
       (start #~(make-forkexec-constructor #$vm-command))
       (stop  #~(make-kill-destructor))))))
-- 
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com

[bug#41961] [PATCH 1/1] services: childhurd: Support more than one instance.

[Mathieu Othacehe]

Hey janneke!

> * gnu/services/virtualization.scm (<hurd-vm-configuration>)[id,net-options]:
> New fields.
> (hurd-vm-net-options): New prodecure.  Parameterize port forwarding with ID.
                               ^
                               typo

> +@item @code{options} (default: @code{'("--snapshot" "--hda")})
>  The extra options for running QEMU.

Does it really make sense to have "--hda" standalone here, without a
specific image argument?

> +  (net-options hurd-vm-configuration-net-options        ;list of string
> +               (thunked)
> +               (default (hurd-vm-net-options this-record))))

Why does it need to be thunked?

Otherwise this looks nice!

Thanks,

Mathieu

[bug#41961] [PATCH 1/1] services: childhurd: Support more than one instance.

[Jan Nieuwenhuizen]

Mathieu Othacehe writes:

Hey Mathieu,

>> * gnu/services/virtualization.scm (<hurd-vm-configuration>)[id,net-options]:
>> New fields.
>> (hurd-vm-net-options): New prodecure.  Parameterize port forwarding with ID.
>                                ^
>                                typo

Oops, thanks.

>> +@item @code{options} (default: @code{'("--snapshot" "--hda")})
>>  The extra options for running QEMU.
>
> Does it really make sense to have "--hda" standalone here, without a
> specific image argument?

Hmm...yes, this looked a bit awkward to me too.  It's being used like
this:

o--8<---------------cut here---------------start------------->8---
  (let ((image       (hurd-vm-configuration-image config))
     ...)
    (define vm-command
      #~(list
         (string-append #$qemu "/bin/qemu-system-i386")
         #$@(if (file-exists? "/dev/kvm") '("--enable-kvm") '())
         "-m" (number->string #$memory-size)
         #$@net-options
         #$@options
         #+image))
--8<---------------cut here---------------end--------------->8---

so that you can play with options and image; have the
"hurd-vm-configuration-image" procedure to return anything, something
that may require something else than --hda <image>...but it's quite
implicit.  Ideas?

>> +  (net-options hurd-vm-configuration-net-options        ;list of string
>> +               (thunked)
>> +               (default (hurd-vm-net-options this-record))))
>
> Why does it need to be thunked?

It uses ID from the configuration like so

--8<---------------cut here---------------start------------->8---
(define (hurd-vm-net-options config)
  (let ((id (or (hurd-vm-configuration-id config) 0)))
    (define (qemu-vm-port base)
      (number->string (+ base (* 1000 id))))
    [...]
     ",hostfwd=tcp:127.0.0.1:" (qemu-vm-port 10022) "-:2222"
    [...]))
--8<---------------cut here---------------end--------------->8---

to fix parameterize the ports for QEMU.  Is there a better way to do
that?

> Otherwise this looks nice!

Great, thanks!

Greetigs,
Janneke

-- 
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com

[bug#41961] [PATCH 1/1] services: childhurd: Support more than one instance.

[Mathieu Othacehe]

> Hmm...yes, this looked a bit awkward to me too.  It's being used like
> this:
>
> o--8<---------------cut here---------------start------------->8---
>   (let ((image       (hurd-vm-configuration-image config))
>      ...)
>     (define vm-command
>       #~(list
>          (string-append #$qemu "/bin/qemu-system-i386")
>          #$@(if (file-exists? "/dev/kvm") '("--enable-kvm") '())
>          "-m" (number->string #$memory-size)
>          #$@net-options
>          #$@options
>          #+image))

What about having something like:

--8<---------------cut here---------------start------------->8---
(define vm-command
  #~(list
     (string-append #$qemu "/bin/qemu-system-i386")
     #$@(if (file-exists? "/dev/kvm") '("--enable-kvm") '())
     "-m" (number->string #$memory-size)
     #$@options
     #+@(list "--hda" image)))
--8<---------------cut here---------------end--------------->8---

instead?

> to fix parameterize the ports for QEMU.  Is there a better way to do
> that?

Oh I see, then it's fine I guess.

Thanks,

Mathieu

bug#41961: [PATCH 1/1] services: childhurd: Support more than one instance.

[Jan Nieuwenhuizen]

Mathieu Othacehe writes:

>> Hmm...yes, this looked a bit awkward to me too.  It's being used like
>> this:
>>
>> o--8<---------------cut here---------------start------------->8---
>>   (let ((image       (hurd-vm-configuration-image config))
>>      ...)
>>     (define vm-command
>>       #~(list
>>          (string-append #$qemu "/bin/qemu-system-i386")
>>          #$@(if (file-exists? "/dev/kvm") '("--enable-kvm") '())
>>          "-m" (number->string #$memory-size)
>>          #$@net-options
>>          #$@options
>>          #+image))
>
> What about having something like:
>
> (define vm-command
>   #~(list
>      (string-append #$qemu "/bin/qemu-system-i386")
>      #$@(if (file-exists? "/dev/kvm") '("--enable-kvm") '())
>      "-m" (number->string #$memory-size)
>      #$@options
>      #+@(list "--hda" image)))
>
> instead?

So we hardcode it.  Simply changed to

--8<---------------cut here---------------start------------->8---
    (define vm-command
      #~(list
         (string-append #$qemu "/bin/qemu-system-i386")
         #$@(if (file-exists? "/dev/kvm") '("--enable-kvm") '())
         "-m" (number->string #$memory-size)
         #$@net-options
         #$@options
         "--hda" #+image))
--8<---------------cut here---------------end--------------->8---

I guess that's better than the original fragile softcoding, because it's
more robust and we have no usecase for changing '--hda' yet.  Maybe I'm
a bit too inclined to always enable the user to override stuff, even if
I cannot imagine its use case yet :-)

Pushed to master as, b7249aa4726193653e05e694ec4bb311aa4ec6c2.

Thanks,
Janneke

-- 
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com