From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:470:142:3::10]:44575) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hiR4M-0006Sq-3X for guix-patches@gnu.org; Tue, 02 Jul 2019 18:16:12 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hiR4G-0008EY-68 for guix-patches@gnu.org; Tue, 02 Jul 2019 18:16:06 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:39275) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1hiR4D-0008Ba-QB for guix-patches@gnu.org; Tue, 02 Jul 2019 18:16:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1hiR4D-0004JZ-Ix for guix-patches@gnu.org; Tue, 02 Jul 2019 18:16:01 -0400 Subject: [bug#36404] [PATCH 0/6] Add 'guix deploy'. Resent-Message-ID: From: zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) References: <87o92ianbj.fsf@sdf.lonestar.org> <87o92glap5.fsf@dustycloud.org> <878sthoqzi.fsf@gnu.org> <87r2799tzd.fsf@sdf.lonestar.org> Date: Tue, 02 Jul 2019 18:14:43 -0400 In-Reply-To: <87r2799tzd.fsf@sdf.lonestar.org> (Jakob L. Kreuze's message of "Mon, 01 Jul 2019 20:10:30 -0400") Message-ID: <87d0isrsmk.fsf@sdf.lonestar.org> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+kyle=kyleam.com@gnu.org Sender: "Guix-patches" To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Cc: 36404@debbugs.gnu.org --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi Ludovic, zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) writes: > Yep, I have an unsquashed commit history on my personal branch with > all renditions of the test suite. I can pull it out tomorrow and write > a detailed report on the issues I ran into. So we begin as I did about a month ago with a very na=C3=AFve test, ensuring that we can create a 'machine' object. This isn't particularly hard to pull off in the system test suite. #+BEGIN_SRC scheme (define (run-sshable-machine-test) (define os (marionette-operating-system (simple-operating-system (service dhcp-client-service-type) (service openssh-service-type (openssh-configuration (permit-root-login #t) (allow-empty-passwords? #t)))) #:imported-modules '((gnu services herd) (guix combinators)))) (define vm (virtual-machine (operating-system os) (port-forwardings '((2222 . 22))))) (define test (with-extensions (list guile-bytestructures guile-gcrypt guile-git guile-ssh guile-sqlite3 guix) (with-imported-modules '((gnu build marionette) (gnu) (gnu machine) (gnu machine ssh) (guix remote)) #~(begin (use-modules (gnu build marionette) (gnu) (gnu machine) (gnu machine ssh) (srfi srfi-64)) (use-service-modules networking ssh) (define %system (operating-system (host-name "gnu-deployed") (timezone "Etc/UTC") (bootloader (bootloader-configuration (bootloader grub-bootloader) (target "/dev/vda") (terminal-outputs '(console)))) (file-systems (cons (file-system (mount-point "/") (device "/dev/vda1") (type "ext4")) %base-file-systems)) (services (append (list (service dhcp-client-service-type) (service openssh-service-type (openssh-configuration (permit-root-login #t) (allow-empty-passwords? #t)))) %base-services)))) (define %machine (machine (system %system) (environment managed-host-environment-type) (configuration (machine-ssh-configuration (host-name "localhost") (port 2222))))) (define marionette (make-marionette (list #$vm))) (mkdir #$output) (chdir #$output) (test-begin "remote-eval") (test-assert "machine instance was created" %machine) (test-end) (exit (=3D (test-runner-fail-count (test-runner-current)) 0))))= )) (gexp->derivation "sshable-machine" test)) (define %test-sshable-machine (system-test (name "sshable-machine") (description "Create a machine object") (value (run-sshable-machine-test)))) #+END_SRC For onlookers unfamiliar with the system test suite, this is mostly boilerplate. The important code begins at 'define %system' and ends at 'test-end'. Wonderful! We've ensured that we can import '(gnu machine)', and that we can create instances of 'machine'. Where to now? How about testing 'remote-eval'? (This snippet requires more changes to the surrounding code to work. If you need a reproducible version, let me know.) #+BEGIN_SRC scheme (test-assert "can invoke machine-remote-eval" (with-store store (run-with-store store (machine-remote-eval %machine #~#t)))) #+END_SRC Alas, this doesn't work in the context of a derivation. #+BEGIN_SRC scheme (srfi-34 #) #+END_SRC This is around when I began to pester you on IRC with questions that I realize are kind of silly now. In general, system tests can't use the store. The only workaround that I'm aware of is 'gnu/tests/install.scm', which makes use of the store monad to perform store operations before running the test. For example: #+BEGIN_SRC scheme (define %test-iso-image-installer (system-test (name "iso-image-installer") (description "") (value (mlet* %store-monad ((image (run-install %minimal-os-on-vda %minimal-os-on-vda-source #:script %simple-installation-script-for-/dev/vda #:installation-disk-image-file-system-ty= pe "iso9660")) (command (qemu-command/writable-image image))) (run-basic-test %minimal-os-on-vda command name))))) #+END_SRC This is a bit less complicated than system deployment, since the tests only need the store to build the virtual machine image. Deployment to a machine requires that the machine is /up/, but if you look at the initial, na=C3=AFve test, you can see that the virtual machine isn't started until the test derivation runs -- which is after everything in the store monad is run. c6e01898[1] has a version that starts the virtual machine while the store monad is running so it can deploy to it. This is an absolute mess, as seen in 'call-with-marionette'. Also, the use of 'dynamic-wind' in that rendition causes the SSH session to close during deployment, which is why that test fails. (I didn't figure that out until around the time I began reimplementing the tests in the normal test suite.) In theory, _I could fix that issue and implement the tests this way_. Another possibility would be to spawn two virtual machines and have one deploy to the other. This is implemented in 358f1287[2], which I believe I would also be able to adapt now that I know I need to create writable disk images for the virtual machines. Before I go ahead with either, though, I'd like to know if either is the "right way". Or if there's something better than what I'm suggesting. Regards, Jakob [1]: https://git.sr.ht/~jakob/guix/tree/c6e01898dc774eef318c042595d6490e50e= 19486/gnu/tests/machine.scm [2]: https://git.sr.ht/~jakob/guix/tree/358f12871326085c3e108181887ea36a857= 7de73/gnu/tests/machine.scm --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEa1VJLOiXAjQ2BGSm9Qb9Fp2P2VoFAl0b11MACgkQ9Qb9Fp2P 2VqPqhAAixvg0Fo4O3GLrUUhk57YHGEutqPaUmCmfHoWUcXc6JoPG2exTg/fWhe6 OgvEXwdqFMMh6IOUBx66T6Wid0ZyZwwZOufXzeX6U9fXn1e0KutmnCNFKKc5+TT9 EStbkEY5au9zDRK61OY2OOe6HW3EJIe0DscvAkiEs+cQWW/rbDRyl69EB+tEiyzP CoPgeyhPWvI9rffIeAy3ztGxljUszS7enOt7glJOZlX1S9ogsCp2mhz2HvH8kCtI XpZMBicL3+T0vfvXfqgPiV2mSR+gYaBalE141mPaU4fPfay1ncSEHxFPHNmDTNax cPk5XURJUITK+Zn5oLdFABsnOzfJ/TImXGEQM0QycTDFWUCxELHEErlJw4KCKfuV WNyxC2hsPquA5IHQGgVkG0+upUSO5gIj933ShBzlQBuvtYiE/oDjZjwzxmxvEfTm m2d9QSn+yjwq5d1suKE6Chph9hYSvaXUbwTZIAXF6C3dzpPe2wj4jbASqWCMgAdZ YwXumfBghQ6n5/Ylwtavdq85C9RY5+ELQ0lz4BL2kX5oKWbOncS+xTG0m+AB3z6i BE7JgNg+FlzNa++vuVC+lq0KpfBXoRCJSHw3nfywYlhp1Y2cMyk2HocV4gruSklv 6LNOO7bmSMf9U3ZzXDdsmwKpr4/MmjRZ0SlPRIxDeTbhjWFsxtE= =Tr26 -----END PGP SIGNATURE----- --=-=-=--