Re: "guix deploy" is in git master

Re: "guix deploy" is in git master

[Jakob L. Kreuze]

[-- Attachment #1: Type: text/plain, Size: 1057 bytes --]

Christopher Lemmer Webber <cwebber@dustycloud.org> writes:

> Huge congratulations to Jakob in that "guix deploy" has just hit git
> master!  There's a new section of the manual, "8.15 Invoking ‘guix
> deploy’"; see that for details.
>
> Please play with it and let us know what your experiences are!
>
> MANY THANKS and HUGE CONGRATS to Jakob for getting this key piece in of
> their GSoC project already this summer!  More cool stuff is ahead, I
> understand :)
>
> Now who's going to be the first brave soul to use this for server
> deployment? ;)
>  - Chris

Thank you for all the kind words!

As Chris wrote, we're hoping that people will have an opportunity to
play with the new tool and provide comments; if you're looking for
something low-stakes to deploy to and get your feet wet, I'd suggest
setting up an SSH daemon on the 1.0.1 QEMU Image[1] and forwarding that
port to your host. Hopefully the example in the manual can serve as a
starting point from there.

Regards,
Jakob

[1]: http://guix.gnu.org/download/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: "guix deploy" is in git master

[Jakob L. Kreuze]

[-- Attachment #1: Type: text/plain, Size: 1057 bytes --]

Christopher Lemmer Webber <cwebber@dustycloud.org> writes:

> Huge congratulations to Jakob in that "guix deploy" has just hit git
> master!  There's a new section of the manual, "8.15 Invoking ‘guix
> deploy’"; see that for details.
>
> Please play with it and let us know what your experiences are!
>
> MANY THANKS and HUGE CONGRATS to Jakob for getting this key piece in of
> their GSoC project already this summer!  More cool stuff is ahead, I
> understand :)
>
> Now who's going to be the first brave soul to use this for server
> deployment? ;)
>  - Chris

Thank you for all the kind words!

As Chris wrote, we're hoping that people will have an opportunity to
play with the new tool and provide comments; if you're looking for
something low-stakes to deploy to and get your feet wet, I'd suggest
setting up an SSH daemon on the 1.0.1 QEMU Image[1] and forwarding that
port to your host. Hopefully the example in the manual can serve as a
starting point from there.

Regards,
Jakob

[1]: http://guix.gnu.org/download/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

"guix deploy" is in git master

[Christopher Lemmer Webber]

Huge congratulations to Jakob in that "guix deploy" has just hit git
master!  There's a new section of the manual, "8.15 Invoking ‘guix
deploy’"; see that for details.

Please play with it and let us know what your experiences are!

MANY THANKS and HUGE CONGRATS to Jakob for getting this key piece in of
their GSoC project already this summer!  More cool stuff is ahead, I
understand :)

Now who's going to be the first brave soul to use this for server
deployment? ;)
 - Chris

Re: "guix deploy" is in git master

[Christopher Lemmer Webber]

Ricardo, can you see if 834b8a4110 fixes the problem?

Jakob L. Kreuze writes:

> Hi, Ricardo!
>
> Ricardo Wurmus <rekado@elephly.net> writes:
>
>> Me!  I tried to use it on ci.guix.gnu.org to upgrade a node, but it
>> failed like this:
>>
>> rekado@berlin /root/maintenance/hydra$ guix deploy deploy-berlin-node.scm
>> Backtrace:
>>            8 (primitive-load "/home/rekado/.config/guix/current/bin/��")
>> In guix/ui.scm:
>>   1655:12  7 (run-guix-command _ . _)
>> In guix/scripts/deploy.scm:
>>     78:33  6 (guix-deploy . _)
>>     69:16  5 (load-source-file "deploy-berlin-node.scm")
>> In guix/ui.scm:
>>     149:4  4 (make-user-module _)
>> In srfi/srfi-1.scm:
>>     640:9  3 (for-each #<procedure 3f2d260 at guix/ui.scm:149:14 (i��> ��)
>> In guix/ui.scm:
>>    150:36  2 (_ (gnu machine ssh))
>> In ice-9/boot-9.scm:
>>    2803:6  1 (resolve-interface _ #:select _ #:hide _ #:prefix _ # _ ��)
>> In unknown file:
>>            0 (scm-error misc-error #f "~A ~S" ("no code for modu��" ��) ��)
>>
>> ERROR: In procedure scm-error:
>> no code for module (gnu machine ssh)
>>
>>
>> Looks like (gnu machine ssh) is not installed.  I only see
>> “~/.config/guix/current/share/guile/site/2.2/gnu/machine.scm”, but not
>> “gnu/machine/ssh.scm”.
>>
>> “deploy-berlin-node.scm” is not special.  It just uses a procedure to
>> generate the system configuration:
>>
>> (use-service-modules networking ssh)
>> (use-package-modules bootloaders)
>>
>> (use-modules (sysadmin build-machines)
>>              (sysadmin people))
>>
>> (define %id 14)
>>
>> (define %system
>>   (berlin-build-machine-os %id))
>>
>> (define (id->ip id)
>>   (format #f "141.80.167.~d" (+ 131 id)))
>>
>> (list (machine
>>        (system %system)
>>        (environment managed-host-environment-type)
>>        (configuration (machine-ssh-configuration
>>                        (host-name (id->ip %id))
>>                        (identity "./id_rsa")
>>                        (port 22)))))
>>
>>
>> --
>> Ricardo
>
> Thanks for trying it out! I'm realizing that 'guix pull' didn't install
> it for me, either, yet 'gnu/machine/ssh.scm' is in the source tree and
> there's a line for it in 'gnu/local.mk'[1]. Does anyone know what's
> happening here?
>
> Regards,
> Jakob
>
> [1]: https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/local.mk#n569

Re: "guix deploy" is in git master

[Ricardo Wurmus]

Christopher Lemmer Webber <cwebber@dustycloud.org> writes:

> Huge congratulations to Jakob in that "guix deploy" has just hit git
> master!

I second this.  Thank you Jakob!

> Now who's going to be the first brave soul to use this for server
> deployment? ;)

Me!  I tried to use it on ci.guix.gnu.org to upgrade a node, but it
failed like this:

--8<---------------cut here---------------start------------->8---
rekado@berlin /root/maintenance/hydra$ guix deploy deploy-berlin-node.scm
Backtrace:
           8 (primitive-load "/home/rekado/.config/guix/current/bin/��")
In guix/ui.scm:
  1655:12  7 (run-guix-command _ . _)
In guix/scripts/deploy.scm:
    78:33  6 (guix-deploy . _)
    69:16  5 (load-source-file "deploy-berlin-node.scm")
In guix/ui.scm:
    149:4  4 (make-user-module _)
In srfi/srfi-1.scm:
    640:9  3 (for-each #<procedure 3f2d260 at guix/ui.scm:149:14 (i��> ��)
In guix/ui.scm:
   150:36  2 (_ (gnu machine ssh))
In ice-9/boot-9.scm:
   2803:6  1 (resolve-interface _ #:select _ #:hide _ #:prefix _ # _ ��)
In unknown file:
           0 (scm-error misc-error #f "~A ~S" ("no code for modu��" ��) ��)

ERROR: In procedure scm-error:
no code for module (gnu machine ssh)
--8<---------------cut here---------------end--------------->8---

Looks like (gnu machine ssh) is not installed.  I only see
“~/.config/guix/current/share/guile/site/2.2/gnu/machine.scm”, but not
“gnu/machine/ssh.scm”.

“deploy-berlin-node.scm” is not special.  It just uses a procedure to
generate the system configuration:

--8<---------------cut here---------------start------------->8---
(use-service-modules networking ssh)
(use-package-modules bootloaders)

(use-modules (sysadmin build-machines)
             (sysadmin people))

(define %id 14)

(define %system
  (berlin-build-machine-os %id))

(define (id->ip id)
  (format #f "141.80.167.~d" (+ 131 id)))

(list (machine
       (system %system)
       (environment managed-host-environment-type)
       (configuration (machine-ssh-configuration
                       (host-name (id->ip %id))
                       (identity "./id_rsa")
                       (port 22)))))
--8<---------------cut here---------------end--------------->8---


--
Ricardo

Re: "guix deploy" is in git master

[Jakob L. Kreuze]

[-- Attachment #1: Type: text/plain, Size: 2275 bytes --]

Hi, Ricardo!

Ricardo Wurmus <rekado@elephly.net> writes:

> Me!  I tried to use it on ci.guix.gnu.org to upgrade a node, but it
> failed like this:
>
> rekado@berlin /root/maintenance/hydra$ guix deploy deploy-berlin-node.scm
> Backtrace:
>            8 (primitive-load "/home/rekado/.config/guix/current/bin/��")
> In guix/ui.scm:
>   1655:12  7 (run-guix-command _ . _)
> In guix/scripts/deploy.scm:
>     78:33  6 (guix-deploy . _)
>     69:16  5 (load-source-file "deploy-berlin-node.scm")
> In guix/ui.scm:
>     149:4  4 (make-user-module _)
> In srfi/srfi-1.scm:
>     640:9  3 (for-each #<procedure 3f2d260 at guix/ui.scm:149:14 (i��> ��)
> In guix/ui.scm:
>    150:36  2 (_ (gnu machine ssh))
> In ice-9/boot-9.scm:
>    2803:6  1 (resolve-interface _ #:select _ #:hide _ #:prefix _ # _ ��)
> In unknown file:
>            0 (scm-error misc-error #f "~A ~S" ("no code for modu��" ��) ��)
>
> ERROR: In procedure scm-error:
> no code for module (gnu machine ssh)
>
>
> Looks like (gnu machine ssh) is not installed.  I only see
> “~/.config/guix/current/share/guile/site/2.2/gnu/machine.scm”, but not
> “gnu/machine/ssh.scm”.
>
> “deploy-berlin-node.scm” is not special.  It just uses a procedure to
> generate the system configuration:
>
> (use-service-modules networking ssh)
> (use-package-modules bootloaders)
>
> (use-modules (sysadmin build-machines)
>              (sysadmin people))
>
> (define %id 14)
>
> (define %system
>   (berlin-build-machine-os %id))
>
> (define (id->ip id)
>   (format #f "141.80.167.~d" (+ 131 id)))
>
> (list (machine
>        (system %system)
>        (environment managed-host-environment-type)
>        (configuration (machine-ssh-configuration
>                        (host-name (id->ip %id))
>                        (identity "./id_rsa")
>                        (port 22)))))
>
>
> --
> Ricardo

Thanks for trying it out! I'm realizing that 'guix pull' didn't install
it for me, either, yet 'gnu/machine/ssh.scm' is in the source tree and
there's a line for it in 'gnu/local.mk'[1]. Does anyone know what's
happening here?

Regards,
Jakob

[1]: https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/local.mk#n569

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: "guix deploy" is in git master

[Christopher Lemmer Webber]

Ricardo Wurmus writes:

> Christopher Lemmer Webber <cwebber@dustycloud.org> writes:
>
>> Ricardo, can you see if 834b8a4110 fixes the problem?
>
> It does not seem to fix this problem.
> “gnu/machine/ssh.scm” still isn’t included.

Unfortunately my ability to debug autotools problems is... almost
nonexistant.  Any autotools-savvy people willing to take a look?

Re: "guix deploy" is in git master

[Ricardo Wurmus]

Christopher Lemmer Webber <cwebber@dustycloud.org> writes:

> Ricardo, can you see if 834b8a4110 fixes the problem?

It does not seem to fix this problem.
“gnu/machine/ssh.scm” still isn’t included.

-- 
Ricardo

Re: "guix deploy" is in git master

[Mark H Weaver]

Hi Ricardo,

Ricardo Wurmus <rekado@elephly.net> writes:

> Christopher Lemmer Webber <cwebber@dustycloud.org> writes:
>
>> Ricardo, can you see if 834b8a4110 fixes the problem?
>
> It does not seem to fix this problem.
> “gnu/machine/ssh.scm” still isn’t included.

The problem is that (guix self) has its own logic, independent of the
*.am files, to determine the set of modules to be compiled and
installed.  That logic needs to be updated when adding a new module
directory.

I just pushed commit 079c93e1c1dd93639417095000d5e56d8db62d61, which
should hopefully fix the problem.

     Thanks,
       Mark


--8<---------------cut here---------------start------------->8---
commit 079c93e1c1dd93639417095000d5e56d8db62d61
Author: Mark H Weaver <mhw@netris.org>
Date:   Sat Jul 6 20:01:35 2019 -0400

  self: Ship the (gnu machine …) modules.
  
  This is a followup to commit fa9edf09e992db7510c7471486dffc93e1e707e5.
  
  * guix/self.scm (compiled-guix)[*system-modules*]: Explicitly add all of
  gnu/machine/*.

diff --git a/guix/self.scm b/guix/self.scm
index be90b60863..b8581d01d5 100644
--- a/guix/self.scm
+++ b/guix/self.scm
@@ -769,7 +769,8 @@ Info manual."
                    (gnu services)
                    ,@(scheme-modules* source "gnu/bootloader")
                    ,@(scheme-modules* source "gnu/system")
-                   ,@(scheme-modules* source "gnu/services"))
+                   ,@(scheme-modules* source "gnu/services")
+                   ,@(scheme-modules* source "gnu/machine"))
                  (list *core-package-modules* *package-modules*
                        *extra-modules* *core-modules*)
                  #:extensions dependencies
--8<---------------cut here---------------end--------------->8---

Re: "guix deploy" is in git master

[Christopher Lemmer Webber]

Ricardo Wurmus writes:

> Hi Mark,
>
>> Ricardo Wurmus <rekado@elephly.net> writes:
>>
>>> Christopher Lemmer Webber <cwebber@dustycloud.org> writes:
>>>
>>>> Ricardo, can you see if 834b8a4110 fixes the problem?
>>>
>>> It does not seem to fix this problem.
>>> “gnu/machine/ssh.scm” still isn’t included.
>>
>> The problem is that (guix self) has its own logic, independent of the
>> *.am files, to determine the set of modules to be compiled and
>> installed.  That logic needs to be updated when adding a new module
>> directory.
>>
>> I just pushed commit 079c93e1c1dd93639417095000d5e56d8db62d61, which
>> should hopefully fix the problem.
>
> Thank you, this fixes it.

Yes, thank you so much Mark! :)

Re: "guix deploy" is in git master

[Ricardo Wurmus]

Hi Mark,

> Ricardo Wurmus <rekado@elephly.net> writes:
>
>> Christopher Lemmer Webber <cwebber@dustycloud.org> writes:
>>
>>> Ricardo, can you see if 834b8a4110 fixes the problem?
>>
>> It does not seem to fix this problem.
>> “gnu/machine/ssh.scm” still isn’t included.
>
> The problem is that (guix self) has its own logic, independent of the
> *.am files, to determine the set of modules to be compiled and
> installed.  That logic needs to be updated when adding a new module
> directory.
>
> I just pushed commit 079c93e1c1dd93639417095000d5e56d8db62d61, which
> should hopefully fix the problem.

Thank you, this fixes it.

-- 
Ricardo

Re: "guix deploy" is in git master

[Ricardo Wurmus]

Hi Jakob,

Ricardo Wurmus <rekado@elephly.net> writes:

>> Now who's going to be the first brave soul to use this for server
>> deployment? ;)
>
> Me!  I tried to use it on ci.guix.gnu.org to upgrade a node, but it
> failed like this: […]

This is now fixed with the inclusion of gnu/machine/ssh.scm.

I tried again and it started building things but then aborted like this:


--8<---------------cut here---------------start------------->8---
…
offloading '/gnu/store/x1q848ra6lm3y0ma9n2i73k8ic1gfyz9-references.drv' to '141.80.167.145'...
@ build-remote /gnu/store/x1q848ra6lm3y0ma9n2i73k8ic1gfyz9-references.drv 141.80.167.145
retrieving 1 store item from '141.80.167.145'...
importing file or directory '/gnu/store/1bm51aa2knvpp4b7kls0m6v19mmhb910-references'...
found valid signature for '/gnu/store/1bm51aa2knvpp4b7kls0m6v19mmhb910-references'
done with offloaded '/gnu/store/x1q848ra6lm3y0ma9n2i73k8ic1gfyz9-references.drv'
sending 47 store items (240 MiB) to '141.80.167.145'...
[ 1/ 2] Loading './gnu/services/herd.scm'...
[ 2/ 2] Compiling './gnu/services/herd.scm'...
sending 3 store items (0 MiB) to '141.80.167.145'...
Backtrace:
           7 (primitive-load "/home/rekado/.config/guix/current/bin/��")
In guix/ui.scm:
  1655:12  6 (run-guix-command _ . _)
In guix/store.scm:
   623:10  5 (call-with-store _)
In srfi/srfi-1.scm:
    640:9  4 (for-each #<procedure 3a8e500 at guix/scripts/deploy.s��> ��)
In guix/store.scm:
  1803:24  3 (run-with-store #<store-connection 256.99 252e3c0> _ # _ ��)
In gnu/machine/ssh.scm:
    339:2  2 (_ _)
In guix/remote.scm:
   122:20  1 (_ _)
    66:17  0 (%remote-eval _ _)

guix/remote.scm:66:17: In procedure %remote-eval:
Throw to key `srfi-34' with args `(#<inferior-object #<condition &action-exception-error [service: user-homes action: start key: match-error args: ("match" "no matching pattern" "#<<user-account> name: \"root\" password: \"\" uid: 0 group: \"root\" supplementary-groups: () comment: \"System administrator\" home-directory: #<procedure home-directory (x)> create-home-directory?: #t shell: \"/gnu/store/qn1ax1fkj16x280m1rv7mcimfmn9l2pf-bash-4.4.23/bin/bash\" system?: #f>")] 1837f00>>)'.
--8<---------------cut here---------------end--------------->8---

Do you know what might be wrong here?

Another thing I noticed is that an SSH authentication error prints a
backtrace.  It would be nice if a failure to authenticate would be
communicated with a clearer error message.

-- 
Ricardo

Re: "guix deploy" is in git master

[Ludovic Courtès]

Hello,

Ricardo Wurmus <rekado@elephly.net> skribis:

> In guix/remote.scm:
>    122:20  1 (_ _)
>     66:17  0 (%remote-eval _ _)
>
> guix/remote.scm:66:17: In procedure %remote-eval:
> Throw to key `srfi-34' with args `(#<inferior-object #<condition &action-exception-error [service: user-homes action: start key: match-error args: ("match" "no matching pattern" "#<<user-account> name: \"root\" password: \"\" uid: 0 group: \"root\" supplementary-groups: () comment: \"System administrator\" home-directory: #<procedure home-directory (x)> create-home-directory?: #t shell: \"/gnu/store/qn1ax1fkj16x280m1rv7mcimfmn9l2pf-bash-4.4.23/bin/bash\" system?: #f>")] 1837f00>>)'.
>
> Do you know what might be wrong here?

I suspect the bug has nothing to do with ‘remote-eval’.  Namely, if the
machine you’re reconfigure is running Guix from before commit
6061d01512081c93c53fdd1d4302b36696403061 (March 2019), then restart the
‘user-homes’ services fails with this error for obscure reasons that I
forgot.

At any rate, I think the service upgrade phase happens after the
symlink-switch and bootloader installation (right?), in which case
reconfiguration is complete anyway.

HTH,
Ludo’.

Re: "guix deploy" is in git master

[Jakob L. Kreuze]

[-- Attachment #1: Type: text/plain, Size: 1541 bytes --]

Mark H Weaver <mhw@netris.org> writes:

> The problem is that (guix self) has its own logic, independent of the
> *.am files, to determine the set of modules to be compiled and
> installed.  That logic needs to be updated when adding a new module
> directory.
>
> I just pushed commit 079c93e1c1dd93639417095000d5e56d8db62d61, which
> should hopefully fix the problem.
>
>      Thanks,
>        Mark
>
> commit 079c93e1c1dd93639417095000d5e56d8db62d61
> Author: Mark H Weaver <mhw@netris.org>
> Date:   Sat Jul 6 20:01:35 2019 -0400
>
>   self: Ship the (gnu machine …) modules.
>   
>   This is a followup to commit fa9edf09e992db7510c7471486dffc93e1e707e5.
>   
>   * guix/self.scm (compiled-guix)[*system-modules*]: Explicitly add all of
>   gnu/machine/*.
>
> diff --git a/guix/self.scm b/guix/self.scm
> index be90b60863..b8581d01d5 100644
> --- a/guix/self.scm
> +++ b/guix/self.scm
> @@ -769,7 +769,8 @@ Info manual."
>                     (gnu services)
>                     ,@(scheme-modules* source "gnu/bootloader")
>                     ,@(scheme-modules* source "gnu/system")
> -                   ,@(scheme-modules* source "gnu/services"))
> +                   ,@(scheme-modules* source "gnu/services")
> +                   ,@(scheme-modules* source "gnu/machine"))
>                   (list *core-package-modules* *package-modules*
>                         *extra-modules* *core-modules*)
>                   #:extensions dependencies

Thank you so much, Mark!

Regards,
Jakob

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: "guix deploy" is in git master

[Jakob L. Kreuze]

[-- Attachment #1: Type: text/plain, Size: 4199 bytes --]

Ricardo Wurmus <rekado@elephly.net> writes:

> I tried again and it started building things but then aborted like this:
> 
> …
> offloading '/gnu/store/x1q848ra6lm3y0ma9n2i73k8ic1gfyz9-references.drv' to '141.80.167.145'...
> @ build-remote /gnu/store/x1q848ra6lm3y0ma9n2i73k8ic1gfyz9-references.drv 141.80.167.145
> retrieving 1 store item from '141.80.167.145'...
> importing file or directory '/gnu/store/1bm51aa2knvpp4b7kls0m6v19mmhb910-references'...
> found valid signature for '/gnu/store/1bm51aa2knvpp4b7kls0m6v19mmhb910-references'
> done with offloaded '/gnu/store/x1q848ra6lm3y0ma9n2i73k8ic1gfyz9-references.drv'
> sending 47 store items (240 MiB) to '141.80.167.145'...
> [ 1/ 2] Loading './gnu/services/herd.scm'...
> [ 2/ 2] Compiling './gnu/services/herd.scm'...
> sending 3 store items (0 MiB) to '141.80.167.145'...
> Backtrace:
>            7 (primitive-load "/home/rekado/.config/guix/current/bin/��")
> In guix/ui.scm:
>   1655:12  6 (run-guix-command _ . _)
> In guix/store.scm:
>    623:10  5 (call-with-store _)
> In srfi/srfi-1.scm:
>     640:9  4 (for-each #<procedure 3a8e500 at guix/scripts/deploy.s��> ��)
> In guix/store.scm:
>   1803:24  3 (run-with-store #<store-connection 256.99 252e3c0> _ # _ ��)
> In gnu/machine/ssh.scm:
>     339:2  2 (_ _)
> In guix/remote.scm:
>    122:20  1 (_ _)
>     66:17  0 (%remote-eval _ _)
> 
> guix/remote.scm:66:17: In procedure %remote-eval:
> Throw to key `srfi-34' with args `(#<inferior-object #<condition &action-exception-error [service: user-homes action: start key: match-error args: ("match" "no matching pattern" "#<<user-account> name: \"root\" password: \"\" uid: 0 group: \"root\" supplementary-groups: () comment: \"System administrator\" home-directory: #<procedure home-directory (x)> create-home-directory?: #t shell: \"/gnu/store/qn1ax1fkj16x280m1rv7mcimfmn9l2pf-bash-4.4.23/bin/bash\" system?: #f>")] 1837f00>>)'.
> 
> Do you know what might be wrong here?

Ludovic Courtès <ludo@gnu.org> writes:

> I suspect the bug has nothing to do with ‘remote-eval’. Namely, if the
> machine you’re reconfigure is running Guix from before commit
> 6061d01512081c93c53fdd1d4302b36696403061 (March 2019), then restart
> the ‘user-homes’ services fails with this error for obscure reasons
> that I forgot.

The throw is coming from 'upgrade-shepherd-services' making a call to
'start-service', so I think that Ludovic is on the right track. What
version of Guix is the remote running, Ricardo? If it's a recent commit,
I worry that a bad 'user-homes' service file was sent over, or perhaps
not sent over at all...

Regardless, it's undesirable to have the deployment aborted half-way
through. Ludovic -- I've wrapped my calls to 'unload-service' in
'false-if-exception'. Would it make sense to do something similar with
my calls to 'start-service'? Perhaps giving the user a warning that
certain services in the deployment were not started?

> At any rate, I think the service upgrade phase happens after the
> symlink-switch and bootloader installation (right?), in which case
> reconfiguration is complete anyway.

Uh oh. I suppose this is another reason for 'upgrade-shepherd-services'
to be made more robust. 'upgrade-shepherd-services' occurs after the
generation symlinks are switched, but before the bootloader
installation. I copied the order from 'guix system reconfigure', but now
I'm thinking that it may make more sense to upgrade services after
installing the bootloader.

All this is to say that Ricardo has successfully deployed a new system
generation, but there's no bootloader entry for it. So if the machine is
rebooted, it will load the previous generation.

As a side note, congratulations on being the first person to upgrade a
machine with 'guix deploy', Ricardo!

Ricardo Wurmus <rekado@elephly.net> writes:

> Another thing I noticed is that an SSH authentication error prints a
> backtrace. It would be nice if a failure to authenticate would be
> communicated with a clearer error message.

Agreed. It should be a simple enough patch, anyway. Thanks for the
suggestion!

Regards,
Jakob

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: "guix deploy" is in git master

[Ricardo Wurmus]

Hi Jakob,

> Ricardo Wurmus <rekado@elephly.net> writes:
>
>> I tried again and it started building things but then aborted like
>this: […]
>> guix/remote.scm:66:17: In procedure %remote-eval:
>> Throw to key `srfi-34' with args `(#<inferior-object #<condition &action-exception-error [service: user-homes action: start key: match-error args: ("match" "no matching pattern" "#<<user-account> name: \"root\" password: \"\" uid: 0 group: \"root\" supplementary-groups: () comment: \"System administrator\" home-directory: #<procedure home-directory (x)> create-home-directory?: #t shell: \"/gnu/store/qn1ax1fkj16x280m1rv7mcimfmn9l2pf-bash-4.4.23/bin/bash\" system?: #f>")] 1837f00>>)'.
>>
>> Do you know what might be wrong here?
>
> Ludovic Courtès <ludo@gnu.org> writes:
>
>> I suspect the bug has nothing to do with ‘remote-eval’. Namely, if the
>> machine you’re reconfigure is running Guix from before commit
>> 6061d01512081c93c53fdd1d4302b36696403061 (March 2019), then restart
>> the ‘user-homes’ services fails with this error for obscure reasons
>> that I forgot.
>
> The throw is coming from 'upgrade-shepherd-services' making a call to
> 'start-service', so I think that Ludovic is on the right track. What
> version of Guix is the remote running, Ricardo? If it's a recent commit,
> I worry that a bad 'user-homes' service file was sent over, or perhaps
> not sent over at all...

The remote’s Guix is pretty old, so it’s likely that Ludo is right.
So, I actually reconfigured these machines semi-successfully, eh?  Neat!

> Regardless, it's undesirable to have the deployment aborted half-way
> through. Ludovic -- I've wrapped my calls to 'unload-service' in
> 'false-if-exception'.

“false-if-exception” isn’t great because it swallows the errors.  I’d
still very much like to see the errors – I just don’t want to see a
backtrace is all.

Similarly, it would be great if “guix deploy” would continue deploying
other machines in the list even if one of them couldn’t be deployed due
to an error.  I have a list of 24 machines and a few of them fail with
the user-homes error above.  Would be nice if all the other deployments
would still go through, because they are independent.

(On the other hand, the current behaviour might be desirable in case
there’s a problem with “guix deploy” itself.  Rather break just one
machine instead of wrecking the whole site.  Dunno.)

> As a side note, congratulations on being the first person to upgrade a
> machine with 'guix deploy', Ricardo!

Yay!

--
Ricardo

Re: "guix deploy" is in git master

[Jakob L. Kreuze]

[-- Attachment #1: Type: text/plain, Size: 1681 bytes --]

Hi Ricardo,

Ricardo Wurmus <rekado@elephly.net> writes:

> The remote’s Guix is pretty old, so it’s likely that Ludo is right.
> So, I actually reconfigured these machines semi-successfully, eh?
> Neat!

Yeah! Neat for me, too. I find it mind-blowing that this tool I made is
already seeing some use

> “false-if-exception” isn’t great because it swallows the errors. I’d
> still very much like to see the errors – I just don’t want to see a
> backtrace is all.

I agree that we should maintain some level of error reporting, which
inspired my suggestion to warn on services that couldn't be started. But
you're right that we should be catching the exceptions to achieve that
rather than using 'false-if-exception'.

> Similarly, it would be great if “guix deploy” would continue deploying
> other machines in the list even if one of them couldn’t be deployed
> due to an error. I have a list of 24 machines and a few of them fail
> with the user-homes error above. Would be nice if all the other
> deployments would still go through, because they are independent.
>
> (On the other hand, the current behaviour might be desirable in case
> there’s a problem with “guix deploy” itself. Rather break just one
> machine instead of wrecking the whole site. Dunno.)

Well, we can pick and choose which exceptions we continue on. For
services not being started, we can probably carry on with the
deployment. I think, ideally, in the situations where 'guix deploy'
should stop, we should try to clean up a bit? Though I'm not yet sure
how we'd go about that aside from rolling back the system generation.

Regards,
Jakob

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: "guix deploy" is in git master

[Jakob L. Kreuze]

[-- Attachment #1: Type: text/plain, Size: 564 bytes --]

zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) writes:

> Well, we can pick and choose which exceptions we continue on. For
> services not being started, we can probably carry on with the
> deployment. I think, ideally, in the situations where 'guix deploy'
> should stop, we should try to clean up a bit? Though I'm not yet sure
> how we'd go about that aside from rolling back the system generation.

On that note, perhaps I'll hold off for now on a patch to better handle
SSH authentication errors. Error reporting is the common issue here.

Regards,
Jakob

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: "guix deploy" is in git master

[Ricardo Wurmus]

Jakob L. Kreuze <zerodaysfordays@sdf.lonestar.org> writes:

> zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) writes:
>
>> Well, we can pick and choose which exceptions we continue on. For
>> services not being started, we can probably carry on with the
>> deployment. I think, ideally, in the situations where 'guix deploy'
>> should stop, we should try to clean up a bit? Though I'm not yet sure
>> how we'd go about that aside from rolling back the system generation.
>
> On that note, perhaps I'll hold off for now on a patch to better handle
> SSH authentication errors. Error reporting is the common issue here.

Sounds good.  A more general strategy is needed here.

-- 
Ricardo

Re: "guix deploy" is in git master

[Thompson, David]

On Mon, Jul 8, 2019 at 3:29 PM Ricardo Wurmus <rekado@elephly.net> wrote:
>
> Similarly, it would be great if “guix deploy” would continue deploying
> other machines in the list even if one of them couldn’t be deployed due
> to an error.  I have a list of 24 machines and a few of them fail with
> the user-homes error above.  Would be nice if all the other deployments
> would still go through, because they are independent.

Agreed. Also this should be done in parallel eventually because
updating 24 machines serially is silly.

> (On the other hand, the current behaviour might be desirable in case
> there’s a problem with “guix deploy” itself.  Rather break just one
> machine instead of wrecking the whole site.  Dunno.)

I think we need to make 'guix deploy' robust enough so that we can trust it. :)

This does bring up the question of what to do upon failure. Other
deployment systems that I've worked with (mainly AWS CodeDeploy)
provide some options. First, the user can specify what it means for a
deploy to succeed. Does it have to successfully deploy to each of them
or should it allow some amount of failure? Then, upon failure, the
user can specify whether or not a rollback should happen.  My personal
preference for default behavior right now is to update everything
possible and print out a report so users can see what failed, but I
think ultimately we'll need to provide more options.

We need to also keep in mind that in-place updates to machines is just
a primitive initial use-case.  Things will get really fun when we get
to blue-green deployments in cloud environments because "rollback"
takes on a whole new meaning. :)

- Dave

Re: "guix deploy" is in git master

[Jakob L. Kreuze]

[-- Attachment #1: Type: text/plain, Size: 1650 bytes --]

Hi Dave,

"Thompson, David" <dthompson2@worcester.edu> writes:

> Agreed. Also this should be done in parallel eventually because
> updating 24 machines serially is silly.

Good idea. Do we have a Guix-specific API for parallelism, or should I
look to the Guile manual section on Futures?

> This does bring up the question of what to do upon failure. Other
> deployment systems that I've worked with (mainly AWS CodeDeploy)
> provide some options. First, the user can specify what it means for a
> deploy to succeed. Does it have to successfully deploy to each of them
> or should it allow some amount of failure? Then, upon failure, the
> user can specify whether or not a rollback should happen.

Would it make sense to allow failure to be defined on a
machine-by-machine basis? For example, adding some sort of
'behavior-on-failure' field to the 'machine' type?

Other options that come to mind are a '--behavior-on-failure' option for
'guix deploy', or to use a more elaborate type for deployment
specifications, maybe having a 'machines->deployment' function, similar
to 'packages->manifest'.

> My personal preference for default behavior right now is to update
> everything possible and print out a report so users can see what
> failed, but I think ultimately we'll need to provide more options.

Agreed.

> We need to also keep in mind that in-place updates to machines is just
> a primitive initial use-case. Things will get really fun when we get
> to blue-green deployments in cloud environments because "rollback"
> takes on a whole new meaning. :)

Glad to hear that you're already thinking about this sort of thing :)

Regards,
Jakob

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: "guix deploy" is in git master

[Christopher Lemmer Webber]

Jakob L. Kreuze writes:

> Hi Dave,
>
> "Thompson, David" <dthompson2@worcester.edu> writes:
>
>> Agreed. Also this should be done in parallel eventually because
>> updating 24 machines serially is silly.
>
> Good idea. Do we have a Guix-specific API for parallelism, or should I
> look to the Guile manual section on Futures?

You can spawn a bunch of threads, then use Guile's channels to
synchronize with the main thread.  (Not to be confused with Guix's
channels!)

Re: "guix deploy" is in git master

[Ludovic Courtès]

Hey!

Christopher Lemmer Webber <cwebber@dustycloud.org> skribis:

> Jakob L. Kreuze writes:
>
>> Hi Dave,
>>
>> "Thompson, David" <dthompson2@worcester.edu> writes:
>>
>>> Agreed. Also this should be done in parallel eventually because
>>> updating 24 machines serially is silly.
>>
>> Good idea. Do we have a Guix-specific API for parallelism, or should I
>> look to the Guile manual section on Futures?
>
> You can spawn a bunch of threads, then use Guile's channels to
> synchronize with the main thread.  (Not to be confused with Guix's
> channels!)

I think you mean Fibers channels, right?

You could also try to simply use ‘par-map’ and the likes, but be
cautious when you start dreading into that: you’ll have to make sure you
don’t share SSH sessions or ports between threads, that you don’t modify
shared vlists/vhashes, etc.

Ludo’.

Re: "guix deploy" is in git master

[Jakob L. Kreuze]

[-- Attachment #1: Type: text/plain, Size: 896 bytes --]

Hi, Chris + Ludovic!

Ludovic Courtès <ludo@gnu.org> writes:

> Christopher Lemmer Webber <cwebber@dustycloud.org> skribis:
>
>> You can spawn a bunch of threads, then use Guile's channels to
>> synchronize with the main thread.  (Not to be confused with Guix's
>> channels!)
>
> I think you mean Fibers channels, right?
>
> You could also try to simply use ‘par-map’ and the likes, but be
> cautious when you start dreading into that: you’ll have to make sure
> you don’t share SSH sessions or ports between threads, that you don’t
> modify shared vlists/vhashes, etc.
>
> Ludo’.

Thanks for the suggestions, I'll look into these. 'deploy-machine' is
reasonably self-contained, so the only issue that's coming to mind at
the moment is if someone were to explicitly share sessions through the
'session' field of 'machine-ssh-configuration'.

Regards,
Jakob

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]