bug#44999: guix deploy Error reading from the channel

bug#44999: guix deploy Error reading from the channel

[Jérémy Korwin-Zmijowski]

[-- Attachment #1: Type: text/plain, Size: 1853 bytes --]

Hello Guixters,

I am trying to deploy a system on DigitalOcean droplet using `guix
deploy`. But I am facing the following issue :

   $ guix deploy ynm-machine-declaration.scm 
   La (1) machine suivante sera déployée :
     ynm

   guix deploy: déploiement vers ynm...
   ;;; [2020/12/02 00:15:48.683027, 0] read_from_channel_port: [GSSH
   ERROR] Error reading from the channel: #<unknown channel (freed)
   7f3ca3ef80a0>

Then I decided to Ctrl-C the process.


Sometimes I did ssh to the target to see if I can see proof that things
were doing ok (for example if the infect script is created in the tmp
directory...). And I noticed I had to delete the ssh key of the target
on my laptop to ssh again...

   ssh-keygen -f "/home/jeko/.ssh/known_hosts" -R "159.89.21.175"

Then, on the target machine guix seems to be installed. I am able to
run `guix pull`. But the packages I wanted to be globally installed
were missing.

On the laptop I deployed from:

   $ guix --version
   guix (GNU Guix) d848b6caab973f6191143360ca53787a19c5e6e4
   Copyright © 2020 les auteurs de Guix
   Licence GPLv3+ : GNU GPL version 3 ou ultérieure <
   http://www.gnu.org/licenses/gpl.fr.html>
   Ceci est un logiciel libre : vous êtes libre de le modifier et de le
   redistribuer.
   Il n'y a AUCUNE GARANTIE, dans les limites permises par la loi.

On the droplet after guix pull:

   # guix --version
   guix (GNU Guix) 04b83678653fda3c66e600e88f54f5108290ec1c
   Copyright (C) 2020 the Guix authors
   License GPLv3+: GNU GPL version 3 or later <
   http://gnu.org/licenses/gpl.html>
   This is free software: you are free to change and redistribute it.
   There is NO WARRANTY, to the extent permitted by law.

I have no clue how to provide more info (logs, traces, …) but I would
be very happy to do so if anyone can lead me a bit on that.

Cheers,

Jeko

[-- Attachment #2: ynm-machine-declaration.scm --]
[-- Type: text/x-scheme, Size: 912 bytes --]

(add-to-load-path (getcwd))

(use-modules (gnu)
	     (ynm-system-decalaration))

;; (machine
;;  (operating-system %ynm-system-decalaration)
;;  (environment managed-host-environment-type)
;;  (configuration (machine-ssh-configuration
;;                  (host-name "yournextmeal.tech")
;; 		 (host-key "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGXArluyIh9+8Nooz9a7LoQV8ree1jpFLMvp7wPsgiMO root@yournextmeal.tech")
;; 		 (system "x86_64-linux")
;;                  (user "root")
;;                  (identity "/home/jeko/.ssh/id_ed25519.pub")
;;                  (port 2222))))
(list
 (machine
  (operating-system %ynm-system-decalaration)
  (environment digital-ocean-environment-type)
  (configuration (digital-ocean-configuration
                  (region "fra1")
                  (size "s-1vcpu-1gb")
		  (ssh-key "/home/jeko/.ssh/id_rsa")
		  (tags (list "guix" "ynm"))
                  (enable-ipv6? #f)))))

[-- Attachment #3: ynm-system-decalaration.scm --]
[-- Type: text/x-scheme, Size: 1343 bytes --]

(define-module (ynm-system-decalaration))

(use-modules (gnu)
	     (gnu services))

(use-service-modules ssh networking)
(use-package-modules screen ssh certs version-control)

(define-public %ynm-system-decalaration
  (operating-system
   (host-name "ynm")
   (timezone "Europe/Paris")
   (locale "fr_FR.utf8")
   (packages (cons* screen git nss-certs %base-packages))
   (bootloader (bootloader-configuration
		(bootloader grub-bootloader)
		(target "/dev/vda")))
   (file-systems (cons (file-system
			(mount-point "/")
			(device "/dev/vda1")
			(type "ext4"))
                       %base-file-systems))
   (initrd-modules (append (list "virtio_scsi")
                           %base-initrd-modules))
   (services
    (append
     (list      
      (service dhcp-client-service-type)
      (service openssh-service-type
	       (openssh-configuration
		(openssh openssh-sans-x)
		(permit-root-login 'without-password)
		(authorized-keys
		 `(("root" ,(local-file "/home/jeko/.ssh/id_rsa.pub"))))
		(port-number 2222))))
     (modify-services %base-services
		      (guix-service-type ynm-config =>
					 (guix-configuration (inherit ynm-config)
							     (authorized-keys
							      (append
							       (list (local-file "/etc/guix/signing-key.pub"))
							       %default-authorized-guix-keys)))))))))

%ynm-system-decalaration

bug#44999: guix deploy Error reading from the channel

[Ludovic Courtès]

Hi Jérémy,

Jérémy Korwin-Zmijowski <jeremy@korwin-zmijowski.fr> skribis:

> I am trying to deploy a system on DigitalOcean droplet using `guix
> deploy`. But I am facing the following issue :
>
>    $ guix deploy ynm-machine-declaration.scm 
>    La (1) machine suivante sera d.ploy.e :
>      ynm
>
>    guix deploy: d.ploiement vers ynm...
>    ;;; [2020/12/02 00:15:48.683027, 0] read_from_channel_port: [GSSH
>    ERROR] Error reading from the channel: #<unknown channel (freed)
>    7f3ca3ef80a0>
>
> Then I decided to Ctrl-C the process.

Does it fail like this every time?

Your message suggests deployment on the droplet eventually succeeded
since you were able to run ‘guix pull’ there, is that correct?

Thanks,
Ludo’.

bug#44999: guix deploy Error reading from the channel

[Jérémy Korwin-Zmijowski]

Le jeudi 03 décembre 2020 à 17:45 +0100, Ludovic Courtès a écrit :
> Does it fail like this every time?
> 
> Your message suggests deployment on the droplet eventually succeeded
> since you were able to run ‘guix pull’ there, is that correct?
> 
> Thanks,
> Ludo’.

Hello Ludo !

I made some more attempts. I was unable to reproduce the previous
scenario… Looks like with my ssh connections I put myself in an
exceptionnal situation.

All I got is `guix deploy` running forever (I let it more than 2 hours)

   $ guix deploy ynm-droplet-declaration.scm
   La (1) machine suivante sera déployée :
     ynm1607086083

   guix deploy: déploiement vers ynm1607086083...

I have to kill it myself. There is no guix on the target system.

As soon as I get some time I will try to determine where it stops.

Jérémy

bug#44999: guix deploy Error reading from the channel

[Ludovic Courtès]

Hi Jérémy,

Jérémy Korwin-Zmijowski <jeremy@korwin-zmijowski.fr> skribis:

> I made some more attempts. I was unable to reproduce the previous
> scenario… Looks like with my ssh connections I put myself in an
> exceptionnal situation.
>
> All I got is `guix deploy` running forever (I let it more than 2 hours)
>
>    $ guix deploy ynm-droplet-declaration.scm
>    La (1) machine suivante sera déployée :
>      ynm1607086083
>
>    guix deploy: déploiement vers ynm1607086083...
>
> I have to kill it myself. There is no guix on the target system.
>
> As soon as I get some time I will try to determine where it stops.

Did it eventually succeed?  Or do you keep hitting hangs?

Thanks,
Ludo’.

bug#44999: guix deploy Error reading from the channel

[Jérémy Korwin-Zmijowski]

Hey Ludo' !

Thank you for asking !

I apologize to have not taken the time to investigate on this
(understand: put 'pk' commands everywhere haha. I don't know what else
to do).

Just did a retry. The command line still hangs with :

   $ guix deploy ynm-droplet-declaration.scm 
   La (1) machine suivante sera déployée :
     kitchen

   guix deploy: déploiement vers kitchen...

The droplet is created with the right hostname (kitchen in ynm / 1 GB
Memory  / 25 GB Disk  / FRA1 - Debian 9 x64)

So I SSH to the machine and ran some commands :

$ ssh root@165.22.28.15 -p 22 -i /home/jeko/.ssh/id_rsa.pub
The authenticity of host '165.22.28.15 (165.22.28.15)' can't be
established.
ECDSA key fingerprint is
SHA256:7dACwKdFtebnZB/vs/pMcChgsp3yoITOvATZFtXki+c.
Are you sure you want to continue connecting (yes/no/[fingerprint])?
yes
Warning: Permanently added '165.22.28.15' (ECDSA) to the list of known
hosts.
Linux kitchen 4.9.0-13-amd64 #1 SMP Debian 4.9.228-1 (2020-07-05)
x86_64

The programs included with the Debian GNU/Linux system are free
software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
# ls /root/
guix-binary-1.0.1.x86_64-linux.tar.xz
# ls /tmp/
guix-infect.sh	var
# guix --version
guix (GNU Guix) 1.0.1
Copyright (C) 2019 the Guix authors
License GPLv3+: GNU GPL version 3 or later <
http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

While I was writing this message, I've been disconnected from SSH

   root@kitchen:~# Connection to 165.22.28.15 closed by remote host.
   Connection to 165.22.28.15 closed.

And the following trace appeared on the hanging deploy command

   ;;; [2020/12/17 14:10:55.445770, 0] read_from_channel_port: [GSSH
   ERROR] Error reading from the channel: #<unknown channel (freed)
   7f2cf28b86c0>

If I want to SSH again to the machine, I get this message :

   $ ssh root@165.22.28.15 -p 22 -i /home/jeko/.ssh/id_rsa.pub
   @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
   @    WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!     @
   @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
   IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
   Someone could be eavesdropping on you right now (man-in-the-middle
   attack)!
   It is also possible that a host key has just been changed.
   The fingerprint for the ECDSA key sent by the remote host is
   SHA256:52FacP3UGfdq4zggEVW5cbCzlbqSepkZhki5qMo0bnQ.
   Please contact your system administrator.
   Add correct host key in /home/jeko/.ssh/known_hosts to get rid of
   this message.
   Offending ECDSA key in /home/jeko/.ssh/known_hosts:36
     remove with:
     ssh-keygen -f "/home/jeko/.ssh/known_hosts" -R "165.22.28.15"
   ECDSA host key for 165.22.28.15 has changed and you have requested
   strict checking.
   Host key verification failed.

That's all I can bring on the table right know ! 

To be continued…

Jérémy

bug#44999: guix deploy Error reading from the channel

[Jérémy Korwin-Zmijowski]

[-- Attachment #1.1.1.1: Type: text/plain, Size: 156 bytes --]

Hi !

With more recent Guix version the error is slightly different.

I think we could close this issue.

I will open a new one later.

Jérémy


[-- Attachment #1.1.1.2: Type: text/html, Size: 446 bytes --]

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 2515 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 665 bytes --]

bug#44999: guix deploy Error reading from the channel

[Ludovic Courtès]

Hi,

Jérémy Korwin-Zmijowski <jeremy@korwin-zmijowski.fr> skribis:

> With more recent Guix version the error is slightly different.
>
> I think we could close this issue.
>
> I will open a new one later.

Alright, closing this one.

Ludo’.