From mboxrd@z Thu Jan 1 00:00:00 1970 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= Subject: Re: Progress on 'guix deploy' Date: Mon, 10 Jun 2019 11:31:13 +0200 Message-ID: <87h88x7pni.fsf@gnu.org> References: <875zpgrjqf.fsf@sdf.lonestar.org> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Return-path: Received: from eggs.gnu.org ([2001:470:142:3::10]:44403) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1haGeI-0001Ps-Jm for guix-devel@gnu.org; Mon, 10 Jun 2019 05:31:42 -0400 In-Reply-To: <875zpgrjqf.fsf@sdf.lonestar.org> (Jakob L. Kreuze's message of "Fri, 07 Jun 2019 20:42:00 -0400") List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: "Jakob L. Kreuze" Cc: guix-devel@gnu.org Hello Jakob, zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) skribis: > As a quick refresher, my GSoC project this summer is 'guix deploy', a > deployment automation tool for GuixSD that's been discussed more > thoroughly in [1] and [2]. Development has taken place on my personal > branch of Guix, specifically the 'wip-deploy' branch [3], and is > represented by three new Scheme source files: > > - 'gnu/machine.scm', which provides an abstraction for /something/ that > can be deployed to in a heterogeneous deployment. Currently, the only > concrete implementation of this is the simple case of in-place updates > to machines running SSH whose names and IP addresses we know. > - 'gnu/tests/machine.scm', which implements some tests for > 'gnu/machine.scm'. This is where I'm most interested in receiving > feedback. More on that later. > - 'guix/scripts/deploy.scm', which implements the rudimentary > command-line interface for 'guix deploy'. Nice! > The command-line interface hasn't really been fleshed out yet, but if > you'd like to play around with it, it takes a "deployment" file as a > parameter, which is a Scheme file looking something like the following: > > #+BEGIN_SRC scheme > (use-modules ...) > > (define %system > (operating-system > (host-name "gnu-deployed") > (timezone "Etc/UTC") > (bootloader (bootloader-configuration > (bootloader grub-bootloader) > (target "/dev/sda") > (terminal-outputs '(console)))) > (file-systems (cons (file-system > (mount-point "/") > (device "/dev/vda1") > (type "ext4")) > %base-file-systems)) > (services > (append (list (service dhcp-client-service-type) > (service openssh-service-type > (openssh-configuration > (permit-root-login #t) > (allow-empty-passwords? #t)))) > %base-services)))) > > (list (make > #:host-name "localhost" > #:ssh-port 5556 > #:system %system)) > #+END_SRC > > Basically, you attach an 'operating-system' declaration to a 'machine'. > In this case, 'sshable-machine' is the specific type of 'machine' that > we're deploying to (one that's running an SSH daemon and has a known IP > + port + hostname). Looks great to me. > I've found that the GuixSD QEMU images work well for playing around with > this, provided that you add the SSH service to the system configuration > and start it. In the case of this deployment file, I had a GuixSD guest > with port 22 forwarded to my host's port 5556. You'll also need to set > up some sort of public key auth in your SSH config. The current code > isn't capable of handling other forms of SSH authentication. That=E2=80=99s reasonable. > In terms of implementation, GOOPS feels like a bit of an unusual choice > when juxtaposed with the rest of the Guix codebase, but I've come to > really enjoy it. I'll roll with it for now, since I think it will make > it easier to flesh out the vocabulary for specifying deployments. It=E2=80=99s more than unusual. :-) Perhaps it=E2=80=99s the right choice= here, we=E2=80=99ll have to discuss it at some point! > The implementation of '' is doing what > 'switch-to-system' and 'install-bootloader' in 'guix/scripts/system.scm' > do, but in terms of data that can be sent with 'remote-eval'. I imagine > the code will make more sense if you read both simultaneously. OK, sounds good. Some time ago, I proposed to have =E2=80=98remove-eval=E2=80=99 where you c= ould do: (remote-eval #~(begin =E2=80=A6) #:session =E2=80=A6) which would take care of building and copying everything the gexp refers to (see .) That would generalize a bit what you describe above. But anyway, that=E2= =80=99s something that can always come later. > Okay, on to the test suite. > > My understanding of the system test suite (tests run with 'check-system' > as opposed to those run with 'check') is that the meat of the test code > exists in a G-Expression and should _not_ be interacting with the store > of the machine running the test suite (i.e. that's the reason we're > using marionettes in the first place). 'gnu/tests/install.scm' seems to > be somewhat of an exception, and because the code in '(gnu machine)' > depends heavily on having access to a store, I've tried to extend what's > done in 'gnu/tests/install.scm' so that my tests have access to store > while instrumenting the marionettes. When you say =E2=80=9Caccess the store=E2=80=9D, you mean interact with the= build daemon, right? The (gnu tests install) tests are indeed the only ones that talk to the build daemon. > To be specific, the chicken and egg scenario I'm working around is that > the SSH daemon on the marionette needs to be running in order for > 'deploy-os' to work, but I can't call 'deploy-os' from the test > G-Expression because the store wouldn't be accessible then. > > My gut is telling me that this is absolutely the wrong way to go about > it, though. 'call-with-marionette' is one of a couple red flags making > me feel that way -- I don't think marionettes were meant to be started > outside the context of a derivation... Marionettes are just a way to instrument the system under test, which is running in a VM. I think what you need is to ensure the target VM (the one you=E2=80=99re deploying to) has a writable store. To do that, you could create the target VM with the equivalent of =E2=80=98guix system vm-image=E2=80=99, wh= ich is the =E2=80=98system-qemu-image=E2=80=99 procedure. (That will require a bit of= disk space, because that image needs to be large enough to hold the store of the machine.) How does that sound? > I'm going to continue to spend time on the "internals" of 'guix deploy' > this coming week, incorporating any feedback I receive and ensuring that > I have a framework to build upon when I extend this to more complicated > scenarios like deploying to a VPS provider. After that, I'll tackle > fleshing out the way deployments are specified (the file part. I'm > holding off on the command-line tool part for right now). Sounds good! Consider hanging out on #guix and sending email here=E2=80=94it=E2=80=99s u= sually easier to get feedback on specific topics. Anyway, glad to see so much progress already! Thanks for the update! Ludo=E2=80=99.