bug#46481: "guix download" with ftp URL doesn't work on IPv6 network

bug#46481: "guix download" with ftp URL doesn't work on IPv6 network

[Danny Milosavljevic]

[-- Attachment #1: Type: text/plain, Size: 697 bytes --]

I strongly suspect there to be some problem with the ftp client since
that's the second file that doesn't work using guix download but does work
using wget, on the same computer.

$ guix download ftp://ftp.denx.de/pub/u-boot/u-boot-2021.01.tar.bz2

Starting download of /tmp/guix-file.tORPhj
From ftp://ftp.denx.de/pub/u-boot/u-boot-2021.01.tar.bz2...
Throw to key `ftp-error' with args `(#<input-output: socket 16> "PASV" 425 "You cannot use PASV on IPv6 connections. Use EPSV instead.\r")'.
failed to download "/tmp/guix-file.tORPhj" from "ftp://ftp.denx.de/pub/u-boot/u-boot-2021.01.tar.bz2"
guix download: error: ftp://ftp.denx.de/pub/u-boot/u-boot-2021.01.tar.bz2: download failed

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

bug#46481: "guix download" with ftp URL doesn't work on IPv6 network

[宋文武]

[-- Attachment #1: Type: text/plain, Size: 801 bytes --]

Danny Milosavljevic <dannym@scratchpost.org> writes:

> I strongly suspect there to be some problem with the ftp client since
> that's the second file that doesn't work using guix download but does work
> using wget, on the same computer.
>
> $ guix download ftp://ftp.denx.de/pub/u-boot/u-boot-2021.01.tar.bz2
>
> Starting download of /tmp/guix-file.tORPhj
> From ftp://ftp.denx.de/pub/u-boot/u-boot-2021.01.tar.bz2...
> Throw to key `ftp-error' with args `(#<input-output: socket 16> "PASV" 425 "You cannot use PASV on IPv6 connections. Use EPSV instead.\r")'.
> failed to download "/tmp/guix-file.tORPhj" from "ftp://ftp.denx.de/pub/u-boot/u-boot-2021.01.tar.bz2"
> guix download: error: ftp://ftp.denx.de/pub/u-boot/u-boot-2021.01.tar.bz2: download failed

Yes, with this patch I can get it work:

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-ftp-client-Before-PASV-try-EPSV-first-for-IPv6.patch --]
[-- Type: text/x-patch, Size: 2153 bytes --]

From 568ea9cc0e07eab24c7d24e228d7d391f191feca Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?=E5=AE=8B=E6=96=87=E6=AD=A6?= <iyzsong@member.fsf.org>
Date: Sun, 14 Feb 2021 12:02:57 +0800
Subject: [PATCH] ftp-client: Before 'PASV', try 'EPSV' first for IPv6.

This fixes <https://bugs.gnu.org/46481>.

* guix/ftp-client.scm (ftp-epsv, ftp-passive): New procedure.
(ftp-list, ftp-retr): Replace call to 'ftp-pasv' with 'ftp-passive'.
---
 guix/ftp-client.scm | 17 +++++++++++++++--
 1 file changed, 15 insertions(+), 2 deletions(-)

diff --git a/guix/ftp-client.scm b/guix/ftp-client.scm
index 8d5adcb8ed..a72057d3f5 100644
--- a/guix/ftp-client.scm
+++ b/guix/ftp-client.scm
@@ -216,6 +216,19 @@ TIMEOUT, an ETIMEDOUT error is raised."
           (else
            (throw 'ftp-error conn "PASV" 227 message)))))
 
+(define (ftp-epsv conn)
+  (let* ((message (%ftp-command "EPSV" 229 (ftp-connection-socket conn))))
+    (string->number
+     (match:substring
+      (string-match "\\(...([0-9]+).\\)" message) 1))))
+
+(define (ftp-passive conn)
+  "Enter passive mode using EPSV or PASV, return a data connection port on
+success."
+  ;; IPv6 only works with EPSV, so try it first.
+  (or (false-if-exception (ftp-epsv conn))
+      (ftp-pasv conn)))
+
 (define (address-with-port sa port)
   "Return a socket-address object based on SA, but with PORT."
   (let ((fam  (sockaddr:fam sa))
@@ -232,7 +245,7 @@ TIMEOUT, an ETIMEDOUT error is raised."
   (if directory
       (ftp-chdir conn directory))
 
-  (let* ((port (ftp-pasv conn))
+  (let* ((port (ftp-passive conn))
          (ai   (ftp-connection-addrinfo conn))
          (s    (socket (addrinfo:fam ai) (addrinfo:socktype ai)
                        (addrinfo:protocol ai))))
@@ -281,7 +294,7 @@ must be closed before CONN can be used for other purposes."
   ;; Ask for "binary mode".
   (%ftp-command "TYPE I" 200 (ftp-connection-socket conn))
 
-  (let* ((port (ftp-pasv conn))
+  (let* ((port (ftp-passive conn))
          (ai   (ftp-connection-addrinfo conn))
          (s    (with-fluids ((%default-port-encoding #f))
                  (socket (addrinfo:fam ai) (addrinfo:socktype ai)
-- 
2.30.0


[-- Attachment #3: Type: text/plain, Size: 15 bytes --]


Okay to push?

bug#46481: "guix download" with ftp URL doesn't work on IPv6 network

[Danny Milosavljevic]

[-- Attachment #1: Type: text/plain, Size: 7 bytes --]

LGTM!

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

bug#46481: "guix download" with ftp URL doesn't work on IPv6 network

[宋文武]

Danny Milosavljevic <dannym@scratchpost.org> writes:

> LGTM!

Pushed to master, thank you!

bug#46481:

[Léo Le Bouter via Bug reports for GNU Guix]

[-- Attachment #1: Type: text/plain, Size: 1192 bytes --]

Hello!

It looks like the proposed fix at
858898e348eb300a94b74115328ee39191829bda is causing other issues:

$ guix describe
Generation 27	Feb 17 2021 09:39:49	(current)
  guix 861ba52
    repository URL: https://git.savannah.gnu.org/git/guix.git
    branch: master
    commit: 861ba5258399360a8f4c4e7cd08958f46d2c2b1e

$ strace guix refresh libgcrypt
....
read(15, "220-Welcome hacker!\r\n220-.\r\n220-"..., 4096) = 701
write(15, "USER anonymous\r\n", 16)     = 16
read(15, "331 Send e-mail address as passw"..., 4096) = 38
write(15, "PASS guix@example.com\r\n", 23) = 23
read(15, "230 User logged in, proceed.\r\n", 4096) = 30
brk(0x1534000)                          = 0x1534000
write(15, "CWD /\r\n", 7)               = 7
read(15, "250 Directory change successful."..., 4096) = 34
write(15, "CWD gcrypt\r\n", 12)         = 12
read(15, "250-This directory is used as FT"..., 4096) = 1106
write(15, "CWD libgcrypt\r\n", 15)      = 15
read(15, "250-This is the stable version o"..., 4096) = 135
write(15, "EPSV\r\n", 6)                = 6
read(15, 
... hangs ...

Should we explicitly check if we are over an IPv6 connection instead?
Is that possible?

Léo

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

bug#46481:

[Tobias Geerinckx-Rice via Bug reports for GNU Guix]

[-- Attachment #1: Type: text/plain, Size: 449 bytes --]

Léo Le Bouter via Bug reports for GNU Guix 写道：
> Should we explicitly check if we are over an IPv6 connection 
> instead?
> Is that possible?

My FTP knowledge is about two decades out of date: does an IPv6 
server (de facto) have to support EPSV?

The ‘right’ way would be to send the FEAT command, and check 
whether the server actually supports EPSV.  A bit more 
heavy-weight than your heuristic.

Kind regards,

T G-R

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 247 bytes --]

bug#46481:

[Danny Milosavljevic]

[-- Attachment #1: Type: text/plain, Size: 3080 bytes --]

Hi,

On Wed, 17 Feb 2021 11:16:52 +0100
Tobias Geerinckx-Rice via Bug reports for GNU Guix <bug-guix@gnu.org> wrote:

> Léo Le Bouter via Bug reports for GNU Guix 写道：
> > Should we explicitly check if we are over an IPv6 connection 
> > instead?

That sounds very magical.  I mean we can do it as a last resort, I guess.
EPSV is supposed to work with both IPv4 and IPv6.

> > Is that possible?  
> 
> My FTP knowledge is about two decades out of date: does an IPv6 
> server (de facto) have to support EPSV?
> 
> The ‘right’ way would be to send the FEAT command, and check 
> whether the server actually supports EPSV.  A bit more 
> heavy-weight than your heuristic.

Good idea for a workaround, but your suggestion doesn't work with
ftp.gnupg.org:

CWD libgcrypt
250-This is the stable version of Libgcrypt. 
250-For devlopment versions see ../alpha/libgcrypt/.
250 Directory change successful.
FEAT
500 Syntax error, command unrecognized.
HELP
502 Command not implemented.

But this works:

USER anonymous
331 Send e-mail address as password.
PASS a@example.com
230 User logged in, proceed.
EPSV
229 Entering Extended Passive Mode (|||41682|)

And this works:

220 Service ready for new user.
USER anonymous
331 Send e-mail address as password.
PASS a@example.com
230 User logged in, proceed.
CWD /
250 Directory change successful.
EPSV
229 Entering Extended Passive Mode (|||40666|)

And this works:

220 Service ready for new user.
USER anonymous
331 Send e-mail address as password.
PASS a@example.com
230 User logged in, proceed.
CWD /
250 Directory change successful.
CWD gcrypt
[...]
250 Directory change successful.
EPSV
229 Entering Extended Passive Mode (|||41707|)

AND this works:

220 Service ready for new user.
USER anonymous
331 Send e-mail address as password.
PASS a@example.com
230 User logged in, proceed.
CWD /
250 Directory change successful.
CWD gcrypt
[...]
250 Directory change successful.
EPSV
229 Entering Extended Passive Mode (|||41358|)
CWD libgcrypt
250-This is the stable version of Libgcrypt. 
250-For devlopment versions see ../alpha/libgcrypt/.
250 Directory change successful.
EPSV
229 Entering Extended Passive Mode (|||41308|)

But this does not work:

220 Service ready for new user.
USER anonymous
331 Send e-mail address as password.
PASS a@example.com
230 User logged in, proceed.
CWD /
250 Directory change successful.
CWD gcrypt
[...]
250 Directory change successful.
CWD libgcrypt
250-This is the stable version of Libgcrypt. 
250-For devlopment versions see ../alpha/libgcrypt/.
250 Directory change successful.
EPSV
<hangs>

Looks like a straightforward server bug to me.

>250-Please contact ftpmaster@gnupg.org it you have any problems with

Please contact ftpmaster@gnupg.org !

I would not suggest to complicate perfectly valid client code just because the
server does weird stuff--especially when those servers are GNU project servers
anyway.  We can just talk to the server maintainers instead.

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]