[bug#36404] [PATCH 0/6] Add 'guix deploy'.

[zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze)]

[-- Attachment #1: Type: text/plain, Size: 6956 bytes --]

Hi Ludovic,

zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) writes:

> Yep, I have an unsquashed commit history on my personal branch with
> all renditions of the test suite. I can pull it out tomorrow and write
> a detailed report on the issues I ran into.

So we begin as I did about a month ago with a very naïve test, ensuring
that we can create a 'machine' object. This isn't particularly hard to
pull off in the system test suite.

#+BEGIN_SRC scheme
(define (run-sshable-machine-test)
  (define os
    (marionette-operating-system
     (simple-operating-system
      (service dhcp-client-service-type)
      (service openssh-service-type
               (openssh-configuration
                (permit-root-login #t)
                (allow-empty-passwords? #t))))
     #:imported-modules '((gnu services herd)
                          (guix combinators))))

  (define vm
    (virtual-machine
     (operating-system os)
     (port-forwardings '((2222 . 22)))))

  (define test
    (with-extensions (list guile-bytestructures guile-gcrypt
                           guile-git guile-ssh guile-sqlite3 guix)
      (with-imported-modules '((gnu build marionette)
                               (gnu)
                               (gnu machine)
                               (gnu machine ssh)
                               (guix remote))
        #~(begin
            (use-modules (gnu build marionette)
                         (gnu)
                         (gnu machine)
                         (gnu machine ssh)
                         (srfi srfi-64))
            (use-service-modules networking ssh)

            (define %system
              (operating-system
                (host-name "gnu-deployed")
                (timezone "Etc/UTC")
                (bootloader (bootloader-configuration
                             (bootloader grub-bootloader)
                             (target "/dev/vda")
                             (terminal-outputs '(console))))
                (file-systems (cons (file-system
                                      (mount-point "/")
                                      (device "/dev/vda1")
                                      (type "ext4"))
                                    %base-file-systems))
                (services
                 (append (list (service dhcp-client-service-type)
                               (service openssh-service-type
                                        (openssh-configuration
                                         (permit-root-login #t)
                                         (allow-empty-passwords? #t))))
                         %base-services))))


            (define %machine
              (machine
               (system %system)
               (environment managed-host-environment-type)
               (configuration (machine-ssh-configuration
                               (host-name "localhost")
                               (port 2222)))))

            (define marionette
              (make-marionette (list #$vm)))

            (mkdir #$output)
            (chdir #$output)

            (test-begin "remote-eval")

            (test-assert "machine instance was created"
              %machine)

            (test-end)
            (exit (= (test-runner-fail-count (test-runner-current)) 0))))))

  (gexp->derivation "sshable-machine" test))

(define %test-sshable-machine
  (system-test
   (name "sshable-machine")
   (description "Create a machine object")
   (value (run-sshable-machine-test))))
#+END_SRC

For onlookers unfamiliar with the system test suite, this is mostly
boilerplate. The important code begins at 'define %system' and ends at
'test-end'.

Wonderful! We've ensured that we can import '(gnu machine)', and that we
can create instances of 'machine'. Where to now? How about testing
'remote-eval'? (This snippet requires more changes to the surrounding
code to work. If you need a reproducible version, let me know.)

#+BEGIN_SRC scheme
(test-assert "can invoke machine-remote-eval"
  (with-store store
    (run-with-store store
      (machine-remote-eval %machine #~#t))))
#+END_SRC

Alas, this doesn't work in the context of a derivation.

#+BEGIN_SRC scheme
(srfi-34 #<condition &store-connection-error [file: "/var/guix/daemon-socket/socket" errno: 2] 101e0c0>)
#+END_SRC

This is around when I began to pester you on IRC with questions that I
realize are kind of silly now. In general, system tests can't use the
store. The only workaround that I'm aware of is 'gnu/tests/install.scm',
which makes use of the store monad to perform store operations before
running the test. For example:

#+BEGIN_SRC scheme
(define %test-iso-image-installer
  (system-test
   (name "iso-image-installer")
   (description
    "")
   (value
    (mlet* %store-monad ((image   (run-install
                                   %minimal-os-on-vda
                                   %minimal-os-on-vda-source
                                   #:script
                                   %simple-installation-script-for-/dev/vda
                                   #:installation-disk-image-file-system-type
                                   "iso9660"))
                         (command (qemu-command/writable-image image)))
      (run-basic-test %minimal-os-on-vda command name)))))
#+END_SRC

This is a bit less complicated than system deployment, since the tests
only need the store to build the virtual machine image. Deployment to a
machine requires that the machine is /up/, but if you look at the
initial, naïve test, you can see that the virtual machine isn't started
until the test derivation runs -- which is after everything in the store
monad is run.

c6e01898[1] has a version that starts the virtual machine while the
store monad is running so it can deploy to it. This is an absolute mess,
as seen in 'call-with-marionette'. Also, the use of 'dynamic-wind' in
that rendition causes the SSH session to close during deployment, which
is why that test fails. (I didn't figure that out until around the time
I began reimplementing the tests in the normal test suite.) In theory,
_I could fix that issue and implement the tests this way_. Another
possibility would be to spawn two virtual machines and have one deploy
to the other. This is implemented in 358f1287[2], which I believe I
would also be able to adapt now that I know I need to create writable
disk images for the virtual machines.

Before I go ahead with either, though, I'd like to know if either is the
"right way". Or if there's something better than what I'm suggesting.

Regards,
Jakob

[1]: https://git.sr.ht/~jakob/guix/tree/c6e01898dc774eef318c042595d6490e50e19486/gnu/tests/machine.scm
[2]: https://git.sr.ht/~jakob/guix/tree/358f12871326085c3e108181887ea36a8577de73/gnu/tests/machine.scm

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]