Check: missing signature, mirrors

Check: missing signature, mirrors

[Nikita Karetnikov]

Hi,

I'm packaging 'Check'. [1]

There are two problems:

1. There is no signature. [2]

2. 'guix-download' hangs after several redirects.

# ./pre-inst-env guix-download
mirror://sourceforge/check/0.9.9/check-0.9.9.tar.gz
accepted connection from pid 2715, uid 0
starting download of `guix-download.143ZR4' from
`http://prdownloads.sourceforge.net/check/0.9.9/check-0.9.9.tar.gz'...
ERROR: download failed
"http://prdownloads.sourceforge.net/check/0.9.9/check-0.9.9.tar.gz"
301 "Moved Permanently"

[...]

starting download of `guix-download.143ZR4' from
`http://kent.dl.sourceforge.net/sourceforge/check/0.9.9/check-0.9.9.tar.gz'...

To be more specific:

# ./pre-inst-env guile
GNU Guile 2.0.6

[...]

scheme@(guile-user)> ,use (web uri) (web client)
scheme@(guile-user)> (http-get (string->uri
"http://tenet.dl.sourceforge.net/project/check/check/0.9.9/check-0.9.9.tar.gz"))
web/client.scm:109:4: In procedure http-get:
web/client.scm:109:4: Throw to key `bad-response' with args `("EOF
while reading response body: ~a bytes of ~a" (82080 602679))'.

I've found these threads, [3,4] but the proposed solution [5] is a bit
extreme. It's highly possible that I'm missing something. Is there a
workaround?

Nikita

[1] http://check.sourceforge.net/
[2] http://sourceforge.net/projects/check/files/check/0.9.9/
[3] https://lists.gnu.org/archive/html/guile-devel/2012-03/msg00116.html
[4] https://lists.gnu.org/archive/html/guile-devel/2012-03/msg00150.html
[5] https://lists.gnu.org/archive/html/guile-devel/2012-03/msg00167.html

Re: Check: missing signature, mirrors

[Ludovic Courtès]

Hi!

Nikita Karetnikov <nikita.karetnikov@gmail.com> skribis:

> There are two problems:
>
> 1. There is no signature. [2]

Be prepared to encounter this very often for non-GNU packages...

> 2. 'guix-download' hangs after several redirects.
>
> # ./pre-inst-env guix-download
> mirror://sourceforge/check/0.9.9/check-0.9.9.tar.gz

The right URI is actually
<mirror://sourceforge/check/check/0.9.9/check-0.9.9.tar.gz>.
This should solve all your problems.

> scheme@(guile-user)> ,use (web uri) (web client)
> scheme@(guile-user)> (http-get (string->uri
> "http://tenet.dl.sourceforge.net/project/check/check/0.9.9/check-0.9.9.tar.gz"))
> web/client.scm:109:4: In procedure http-get:
> web/client.scm:109:4: Throw to key `bad-response' with args `("EOF
> while reading response body: ~a bytes of ~a" (82080 602679))'.
>
> I've found these threads, [3,4] but the proposed solution [5] is a bit
> extreme. It's highly possible that I'm missing something. Is there a
> workaround?

Hmm, I’m not sure exactly what’s going on here.  Perhaps you could raise
this on bug-guile@gnu.org.  OTOH, it’s not a problem in practice for us,
because the “bad” mirror will normally just be skipped.

HTH,
Ludo’.

Re: Check: missing signature, mirrors

[Nikita Karetnikov]

> The right URI is actually
> <mirror://sourceforge/check/check/0.9.9/check-0.9.9.tar.gz>.
> This should solve all your problems.

Yep, it worked. Thanks.

Re: Check: missing signature, mirrors

[Nikita Karetnikov]

Oh, I can't download 'libusb'.

I've already tried the following:

mirror://sourceforge/libusb/libusb-1.0/libusb-1.0.9/libusb-1.0.9.tar.bz2
mirror://sourceforge/libusb/libusb/1.0.9/libusb-1.0.9.tar.bz2
http://garr.dl.sourceforge.net/project/libusb/libusb-1.0/libusb-1.0.9/libusb-1.0.9.tar.bz2

$ wget --spider
http://garr.dl.sourceforge.net/project/libusb/libusb-1.0/libusb-1.0.9/libusb-1.0.9.tar.bz2

[...]

Remote file exists.

Re: Check: missing signature, mirrors

[Ludovic Courtès]

Works for me:

--8<---------------cut here---------------start------------->8---
$ ./pre-inst-env guix-download mirror://sourceforge/libusb/libusb-1.0/libusb-1.0.9/libusb-1.0.9.tar.bz2
accepted connection from pid 21195, uid 1000
starting download of `guix-download.7ULb5L' from `http://prdownloads.sourceforge.net/libusb/libusb-1.0/libusb-1.0.9/libusb-1.0.9.tar.bz2'...
following redirection to `http://switch.dl.sourceforge.net/project/libusb/libusb-1.0/libusb-1.0.9/libusb-1.0.9.tar.bz2'...
/home/ludo/src/guix/test-tmp/store/a6h8chi4c4pjkhdmaarh1ipvsvq5sqdq-libusb-1.0.9.tar.bz2
16sz34ix6hw2wwl3kqx6rf26fg210iryr68wc439dc065pffw879
--8<---------------cut here---------------end--------------->8---

Ludo’.

Re: Check: missing signature, mirrors

[Nikita Karetnikov]

Hmm, can it be connected with "broken IPv6 connectivity"? [1]

(I'm using 2.0.7.)

# ./pre-inst-env guix-download
mirror://sourceforge/libusb/libusb-1.0/libusb-1.0.9/libusb-1.0.9.tar.bz2
accepted connection from pid 22562, uid 0
starting download of `guix-download.CmTclR' from
`http://prdownloads.sourceforge.net/libusb/libusb-1.0/libusb-1.0.9/libusb-1.0.9.tar.bz2'...
following redirection to
`http://citylan.dl.sourceforge.net/project/libusb/libusb-1.0/libusb-1.0.9/libusb-1.0.9.tar.bz2'...
ERROR: Throw to key `bad-response' with args `("EOF while reading
response body: ~a bytes of ~a" (12731 421971))'.
starting download of `guix-download.CmTclR' from
`http://heanet.dl.sourceforge.net/sourceforge/libusb/libusb-1.0/libusb-1.0.9/libusb-1.0.9.tar.bz2'...
following redirection to
`http://downloads.sourceforge.net/sourceforge/libusb/libusb-1.0/libusb-1.0.9/libusb-1.0.9.tar.bz2?download&failedmirror=heanet.dl.sourceforge.net'...
following redirection to
`http://citylan.dl.sourceforge.net/project/libusb/libusb-1.0/libusb-1.0.9/libusb-1.0.9.tar.bz2'...
ERROR: Throw to key `bad-response' with args `("EOF while reading
response body: ~a bytes of ~a" (12731 421971))'.
starting download of `guix-download.CmTclR' from
`http://surfnet.dl.sourceforge.net/sourceforge/libusb/libusb-1.0/libusb-1.0.9/libusb-1.0.9.tar.bz2'...
following redirection to
`http://downloads.sourceforge.net/sourceforge/libusb/libusb-1.0/libusb-1.0.9/libusb-1.0.9.tar.bz2?download&failedmirror=surfnet.dl.sourceforge.net'...
following redirection to
`http://citylan.dl.sourceforge.net/project/libusb/libusb-1.0/libusb-1.0.9/libusb-1.0.9.tar.bz2'...
ERROR: Throw to key `bad-response' with args `("EOF while reading
response body: ~a bytes of ~a" (12731 421971))'.
starting download of `guix-download.CmTclR' from
`http://dfn.dl.sourceforge.net/sourceforge/libusb/libusb-1.0/libusb-1.0.9/libusb-1.0.9.tar.bz2'...
following redirection to
`http://downloads.sourceforge.net/sourceforge/libusb/libusb-1.0/libusb-1.0.9/libusb-1.0.9.tar.bz2?download&failedmirror=dfn.dl.sourceforge.net'...
following redirection to
`http://citylan.dl.sourceforge.net/project/libusb/libusb-1.0/libusb-1.0.9/libusb-1.0.9.tar.bz2'...
ERROR: Throw to key `bad-response' with args `("EOF while reading
response body: ~a bytes of ~a" (12731 421971))'.
starting download of `guix-download.CmTclR' from
`http://mesh.dl.sourceforge.net/sourceforge/libusb/libusb-1.0/libusb-1.0.9/libusb-1.0.9.tar.bz2'...
ERROR: In procedure getaddrinfo: Name or service not known
starting download of `guix-download.CmTclR' from
`http://ovh.dl.sourceforge.net/sourceforge/libusb/libusb-1.0/libusb-1.0.9/libusb-1.0.9.tar.bz2'...
following redirection to
`http://downloads.sourceforge.net/sourceforge/libusb/libusb-1.0/libusb-1.0.9/libusb-1.0.9.tar.bz2?download&failedmirror=ovh.dl.sourceforge.net'...
following redirection to
`http://citylan.dl.sourceforge.net/project/libusb/libusb-1.0/libusb-1.0.9/libusb-1.0.9.tar.bz2'...
ERROR: Throw to key `bad-response' with args `("EOF while reading
response body: ~a bytes of ~a" (12731 421971))'.
starting download of `guix-download.CmTclR' from
`http://osdn.dl.sourceforge.net/sourceforge/libusb/libusb-1.0/libusb-1.0.9/libusb-1.0.9.tar.bz2'...
following redirection to
`http://downloads.sourceforge.net/sourceforge/libusb/libusb-1.0/libusb-1.0.9/libusb-1.0.9.tar.bz2?download&failedmirror=_'...
following redirection to
`http://citylan.dl.sourceforge.net/project/libusb/libusb-1.0/libusb-1.0.9/libusb-1.0.9.tar.bz2'...
ERROR: Throw to key `bad-response' with args `("EOF while reading
response body: ~a bytes of ~a" (12731 421971))'.
starting download of `guix-download.CmTclR' from
`http://kent.dl.sourceforge.net/sourceforge/libusb/libusb-1.0/libusb-1.0.9/libusb-1.0.9.tar.bz2'...

Then it hangs.

[1] https://lists.gnu.org/archive/html/bug-guix/2012-12/msg00004.html

Re: Check: missing signature, mirrors

[Ludovic Courtès]

I think there are several things going on.  First, unlike me, you don’t
get redirected to a mirror where things just work.

Then there’s this:

> `http://citylan.dl.sourceforge.net/project/libusb/libusb-1.0/libusb-1.0.9/libusb-1.0.9.tar.bz2'...
> ERROR: Throw to key `bad-response' with args `("EOF while reading
> response body: ~a bytes of ~a" (12731 421971))'.

I can reproduce that problem, and this may well be a Guile bug related
to partial responses, which you linked to earlier.

> starting download of `guix-download.CmTclR' from
> `http://kent.dl.sourceforge.net/sourceforge/libusb/libusb-1.0/libusb-1.0.9/libusb-1.0.9.tar.bz2'...
>
> Then it hangs.

I can reproduce that one too.  Here wget fails instantly with “No route
to host”, whereas (web client) is stuck like this:

--8<---------------cut here---------------start------------->8---
scheme@(guile-user)> (http-get (string->uri "http://kent.dl.sourceforge.net/sourceforge/libusb/libusb-1.0/libusb-1.0.9/libusb-1.0.9.tar.bz2"))
^CERROR: In procedure scm-error:
ERROR: User interrupt

Entering a new prompt.  Type `,bt' for a backtrace or `,q' to continue.
scheme@(guile-user) [1]> ,bt
In web/client.scm:
    131:4  6 (http-get #<<uri> scheme: http userinfo: #f host: "kent.dl.sourceforge.net" port: #f path: "/sourceforge/libusb/libusb-1.0/libusb-1.0.9/libusb-1.0.9.tar.bz2" query: #f fragment: #f> #:port #<input-output: soc…> …)
In web/response.scm:
   197:31  5 (read-response #<input-output: socket 9>)
In web/http.scm:
   1116:2  4 (read-response-line #<input-output: socket 9>)
    151:2  3 (read-line* #<input-output: socket 9>)
In unknown file:
           2 (%read-line #<input-output: socket 9>)
In ice-9/boot-9.scm:
   184:14  1 (throw system-error "fport_fill_input" "~A" ("Interrupted system call") (4))
In unknown file:
           0 (scm-error signal #f "User interrupt" () (2))
--8<---------------cut here---------------end--------------->8---

I’ll look into, but you can already post these details to
bug-guile@gnu.org so we don’t lose track of it.

Thanks,
Ludo’.

Re: Check: missing signature, mirrors

[Ludovic Courtès]

Oh, and in the meantime you can use ‘nix-prefetch-url’, or wget +
sha256/base32 from Guix.

Ludo’.

Re: Check: missing signature, mirrors

[Ludovic Courtès]

Hi,

Nikita Karetnikov <nikita.karetnikov@gmail.com> skribis:

> scheme@(guile-user)> ,use (web uri) (web client)
> scheme@(guile-user)> (http-get (string->uri
> "http://tenet.dl.sourceforge.net/project/check/check/0.9.9/check-0.9.9.tar.gz"))
> web/client.scm:109:4: In procedure http-get:
> web/client.scm:109:4: Throw to key `bad-response' with args `("EOF
> while reading response body: ~a bytes of ~a" (82080 602679))'.

Commit b3a53fb adds a workaround for this bug.

Ludo’.

Re: Check: missing signature, mirrors

[Nikita Karetnikov]

> Commit b3a53fb adds a workaround for this bug.

'http://' works, but 'mirror://' hangs.

# ./pre-inst-env guix-download
mirror://sourceforge/check/0.9.9/check-0.9.9.tar.gz
accepted connection from pid 31426, uid 0
starting download of `guix-download.2gcWyG' from
`http://prdownloads.sourceforge.net/check/0.9.9/check-0.9.9.tar.gz'...
ERROR: download failed
"http://prdownloads.sourceforge.net/check/0.9.9/check-0.9.9.tar.gz"
301 "Moved Permanently"
starting download of `guix-download.2gcWyG' from
`http://heanet.dl.sourceforge.net/sourceforge/check/0.9.9/check-0.9.9.tar.gz'...
following redirection to
`http://downloads.sourceforge.net/sourceforge/check/0.9.9/check-0.9.9.tar.gz?download&failedmirror=heanet.dl.sourceforge.net'...
ERROR: download failed
"http://downloads.sourceforge.net/sourceforge/check/0.9.9/check-0.9.9.tar.gz?download&failedmirror=heanet.dl.sourceforge.net"
301 "Moved Permanently"
starting download of `guix-download.2gcWyG' from
`http://surfnet.dl.sourceforge.net/sourceforge/check/0.9.9/check-0.9.9.tar.gz'...
following redirection to
`http://downloads.sourceforge.net/sourceforge/check/0.9.9/check-0.9.9.tar.gz?download&failedmirror=surfnet.dl.sourceforge.net'...
ERROR: download failed
"http://downloads.sourceforge.net/sourceforge/check/0.9.9/check-0.9.9.tar.gz?download&failedmirror=surfnet.dl.sourceforge.net"
301 "Moved Permanently"
starting download of `guix-download.2gcWyG' from
`http://dfn.dl.sourceforge.net/sourceforge/check/0.9.9/check-0.9.9.tar.gz'...
following redirection to
`http://downloads.sourceforge.net/sourceforge/check/0.9.9/check-0.9.9.tar.gz?download&failedmirror=dfn.dl.sourceforge.net'...
ERROR: download failed
"http://downloads.sourceforge.net/sourceforge/check/0.9.9/check-0.9.9.tar.gz?download&failedmirror=dfn.dl.sourceforge.net"
301 "Moved Permanently"
starting download of `guix-download.2gcWyG' from
`http://mesh.dl.sourceforge.net/sourceforge/check/0.9.9/check-0.9.9.tar.gz'...
ERROR: In procedure getaddrinfo: Name or service not known
starting download of `guix-download.2gcWyG' from
`http://ovh.dl.sourceforge.net/sourceforge/check/0.9.9/check-0.9.9.tar.gz'...
following redirection to
`http://downloads.sourceforge.net/sourceforge/check/0.9.9/check-0.9.9.tar.gz?download&failedmirror=ovh.dl.sourceforge.net'...
ERROR: download failed
"http://downloads.sourceforge.net/sourceforge/check/0.9.9/check-0.9.9.tar.gz?download&failedmirror=ovh.dl.sourceforge.net"
301 "Moved Permanently"
starting download of `guix-download.2gcWyG' from
`http://osdn.dl.sourceforge.net/sourceforge/check/0.9.9/check-0.9.9.tar.gz'...
following redirection to
`http://downloads.sourceforge.net/sourceforge/check/0.9.9/check-0.9.9.tar.gz?download&failedmirror=_'...
ERROR: download failed
"http://downloads.sourceforge.net/sourceforge/check/0.9.9/check-0.9.9.tar.gz?download&failedmirror=_"
301 "Moved Permanently"
starting download of `guix-download.2gcWyG' from
`http://kent.dl.sourceforge.net/sourceforge/check/0.9.9/check-0.9.9.tar.gz'...

Nikita

Re: Check: missing signature, mirrors

[Ludovic Courtès]

Nikita Karetnikov <nikita.karetnikov@gmail.com> skribis:

>> Commit b3a53fb adds a workaround for this bug.
>
> 'http://' works, but 'mirror://' hangs.

It’s not a matter or mirror vs. http.  This one hangs:

  guix-download http://kent.dl.sourceforge.net/sourceforge/check/0.9.9/check-0.9.9.tar.gz

Wget reports “No route to host” for this, which comes from here:

  socket(PF_INET, SOCK_STREAM, IPPROTO_IP) = 3
  connect(3, {sa_family=AF_INET, sin_port=htons(80), sin_addr=inet_addr("212.219.56.167")}, 16) = -1 EHOSTUNREACH (No route to host)

Guile’s (web client) instead does this:

  socket(PF_INET, SOCK_STREAM, IPPROTO_TCP) = 7
  connect(7, {sa_family=AF_INET, sin_port=htons(80), sin_addr=inet_addr("212.219.56.167")}, 16) = -1 EHOSTUNREACH (No route to host)

Above is the first attempt, which fails like that of wget.  It is
followed by a second attempt:

  socket(PF_INET, SOCK_DGRAM, IPPROTO_UDP) = 7
  connect(7, {sa_family=AF_INET, sin_port=htons(80), sin_addr=inet_addr("212.219.56.167")}, 16) = 0
  ...
  write(7, "GET /sourceforge/check/0.9.9/che"..., 110) = 110
  read(7, 0x2147000, 4096)          = ? ERESTARTSYS (To be restarted)

And it’s this one that hangs (understandably).

(Time passes...)

OK, fixed in 3c738d6:

  $  ./pre-inst-env guix-download http://kent.dl.sourceforge.net/sourceforge/check/0.9.9/check-0.9.9.tar.gz
  starting download of `guix-download.658Hnj' from `http://kent.dl.sourceforge.net/sourceforge/check/0.9.9/check-0.9.9.tar.gz'...
  ERROR: In procedure connect: No route to host

I’ve also removed kent.dl.* from our mirror list.

Then again, you were looking at the wrong URL.  Try this instead:

  guix-download mirror://sourceforge/check/check/0.9.9/check-0.9.9.tar.gz

Thanks!

Ludo’.