From mboxrd@z Thu Jan 1 00:00:00 1970 From: zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) Subject: Progress on 'guix deploy' Date: Fri, 07 Jun 2019 20:42:00 -0400 Message-ID: <875zpgrjqf.fsf@sdf.lonestar.org> Mime-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" Return-path: Received: from eggs.gnu.org ([2001:470:142:3::10]:57839) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hZPUp-0002L3-Vp for guix-devel@gnu.org; Fri, 07 Jun 2019 20:46:13 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hZPUn-00088c-Hs for guix-devel@gnu.org; Fri, 07 Jun 2019 20:46:11 -0400 Received: from mx.sdf.org ([205.166.94.20]:65254) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hZPUn-00084Y-6E for guix-devel@gnu.org; Fri, 07 Jun 2019 20:46:09 -0400 Received: from Upsilon (mobile-107-107-63-50.mycingular.net [107.107.63.50]) (authenticated (0 bits)) by mx.sdf.org (8.15.2/8.14.5) with ESMTPSA id x580gANp009484 (using TLSv1.2 with cipher AES256-GCM-SHA384 (256 bits) verified NO) for ; Sat, 8 Jun 2019 00:42:16 GMT List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: guix-devel@gnu.org --=-=-= Content-Type: text/plain Hello, Guix! Apart from a few patches and my introductory email about a month ago, I've been pretty silent. I feel it's time to finally break that silence, let people know where progress has been made, and request some feedback on the code I've written so far. As a quick refresher, my GSoC project this summer is 'guix deploy', a deployment automation tool for GuixSD that's been discussed more thoroughly in [1] and [2]. Development has taken place on my personal branch of Guix, specifically the 'wip-deploy' branch [3], and is represented by three new Scheme source files: - 'gnu/machine.scm', which provides an abstraction for /something/ that can be deployed to in a heterogeneous deployment. Currently, the only concrete implementation of this is the simple case of in-place updates to machines running SSH whose names and IP addresses we know. - 'gnu/tests/machine.scm', which implements some tests for 'gnu/machine.scm'. This is where I'm most interested in receiving feedback. More on that later. - 'guix/scripts/deploy.scm', which implements the rudimentary command-line interface for 'guix deploy'. The command-line interface hasn't really been fleshed out yet, but if you'd like to play around with it, it takes a "deployment" file as a parameter, which is a Scheme file looking something like the following: #+BEGIN_SRC scheme (use-modules ...) (define %system (operating-system (host-name "gnu-deployed") (timezone "Etc/UTC") (bootloader (bootloader-configuration (bootloader grub-bootloader) (target "/dev/sda") (terminal-outputs '(console)))) (file-systems (cons (file-system (mount-point "/") (device "/dev/vda1") (type "ext4")) %base-file-systems)) (services (append (list (service dhcp-client-service-type) (service openssh-service-type (openssh-configuration (permit-root-login #t) (allow-empty-passwords? #t)))) %base-services)))) (list (make #:host-name "localhost" #:ssh-port 5556 #:system %system)) #+END_SRC Basically, you attach an 'operating-system' declaration to a 'machine'. In this case, 'sshable-machine' is the specific type of 'machine' that we're deploying to (one that's running an SSH daemon and has a known IP + port + hostname). I've found that the GuixSD QEMU images work well for playing around with this, provided that you add the SSH service to the system configuration and start it. In the case of this deployment file, I had a GuixSD guest with port 22 forwarded to my host's port 5556. You'll also need to set up some sort of public key auth in your SSH config. The current code isn't capable of handling other forms of SSH authentication. In terms of implementation, GOOPS feels like a bit of an unusual choice when juxtaposed with the rest of the Guix codebase, but I've come to really enjoy it. I'll roll with it for now, since I think it will make it easier to flesh out the vocabulary for specifying deployments. The implementation of '' is doing what 'switch-to-system' and 'install-bootloader' in 'guix/scripts/system.scm' do, but in terms of data that can be sent with 'remote-eval'. I imagine the code will make more sense if you read both simultaneously. Okay, on to the test suite. My understanding of the system test suite (tests run with 'check-system' as opposed to those run with 'check') is that the meat of the test code exists in a G-Expression and should _not_ be interacting with the store of the machine running the test suite (i.e. that's the reason we're using marionettes in the first place). 'gnu/tests/install.scm' seems to be somewhat of an exception, and because the code in '(gnu machine)' depends heavily on having access to a store, I've tried to extend what's done in 'gnu/tests/install.scm' so that my tests have access to store while instrumenting the marionettes. To be specific, the chicken and egg scenario I'm working around is that the SSH daemon on the marionette needs to be running in order for 'deploy-os' to work, but I can't call 'deploy-os' from the test G-Expression because the store wouldn't be accessible then. My gut is telling me that this is absolutely the wrong way to go about it, though. 'call-with-marionette' is one of a couple red flags making me feel that way -- I don't think marionettes were meant to be started outside the context of a derivation... If anyone has suggestions on how I might go about properly testing this code, I would appreciate it very much. Another point about the test suite: the 'ssh-deploy-os' test fails, but it's a reproducible version of the issue outlined in [2]. I'd like to conscript some help from those more familiar with guile-ssh before breaking out the ol' RFCs myself. ... So, if anyone's in the mood to peek at what's been written so far and give feedback, that'd make my day. Doesn't have to be feedback related to the test code, either. Any sort of comment, be it regarding the code style or perhaps even some suggestions on improving the interface, would be appreciated. I'm going to continue to spend time on the "internals" of 'guix deploy' this coming week, incorporating any feedback I receive and ensuring that I have a framework to build upon when I extend this to more complicated scenarios like deploying to a VPS provider. After that, I'll tackle fleshing out the way deployments are specified (the file part. I'm holding off on the command-line tool part for right now). Signing off for now. Huge thanks to everyone for the warm welcome I received following my introduction email. Regards, Jakob [1]: https://lists.gnu.org/archive/html/guix-devel/2019-02/msg00145.html [2]: https://lists.gnu.org/archive/html/guix-devel/2019-03/msg00114.html [3]: https://git.sr.ht/~jakob/guix/tree/wip-deploy --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEa1VJLOiXAjQ2BGSm9Qb9Fp2P2VoFAlz7BFkACgkQ9Qb9Fp2P 2Vqe6Q/8CAZr/WoVhyqKjT8JC6h9agmU9Um8fu7+gdelpSBEB3mkrBWg5fB0/MQJ 3BbwXaaESXFjCH2IO/CL5AlT86gh65wv6iNArvKCqweICh86o/tV/+Z6L+QEDPiM K1L9rO3r9UvlEmxIus3cTNP+LqZctHy/H77PyloCJvruymw/jjFHgbKs54lYeU84 H3WTOxGBQjQCg4gxGUr6CD1j1p9fjujOhRDPRB5dxPmum1UhK4jCC9eDz1zMi7ou ztFxnKWY/IZeG1pwTonZPKHutXEJGhsweALbGp1NeGDxrLvU6piKwuaElH1VwUB6 0bPcVso+8ny/LVPlol6uWpiUDIc11yx3ojRZWchCSfeVUxFSjXhbxb6o3zE6q43A G4EwWT3pxW1NJHyfQeZWGF7dvWxfQJuz3XjCeeQUKqzBj9aeClQd6h+fWSDG8bNB eS79KqfQMxcoKu+19KarbCRxd5zNHmbEOPncbiEqegbS1Lqy7lCORI/itWeaK05k fgtyjjaUxzqcpVgf3Bc2mbY7VZHGofBwQfaMcKTMW1oB4hDS3aYnE3M1Jh54QEwi xoDqCg7CXP+8Og6raR0PRgjlxNux5zQjlQdPSSuvL5pso2zJqINHdwi1RJwBVzUN PGVQBbEBn8gpQ/TaLNt76+exCTpxOkFDFpsyys2oBCVbvjDY3kU= =g1Nl -----END PGP SIGNATURE----- --=-=-=--