fetching substitutes slow

fetching substitutes slow

[Sree Harsha Totakura]

[-- Attachment #1.1: Type: text/plain, Size: 630 bytes --]

Hi,

Over the last week I tried to fetch substitutes with no success from
hydra.gnu.org.  It always showed the error "server is unresponsive".
However, the nar packages get downloaded with wget.

So, I removed the timeout with the attached patch and the substitute
downloads are working again.  This time I observed that the speed at
which the substitute-binary.scm downloads the nar file is far low than
when the same file is downloaded through wget.  I haven't done extensive
testing on this to confirm firmly, but it looks like the http-fetch
implementation is not efficient.  Can anyone confirm this?

-
Sree


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.2: 0001-remove-timeout.patch --]
[-- Type: text/x-patch; name="0001-remove-timeout.patch", Size: 880 bytes --]

From 50417a01f71f973002171dcd9e5457721b3487ab Mon Sep 17 00:00:00 2001
From: Sree Harsha Totakura <sreeharsha@totakura.in>
Date: Tue, 21 Jan 2014 08:56:37 +0100
Subject: [PATCH] - remove timeout

---
 guix/scripts/substitute-binary.scm |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/guix/scripts/substitute-binary.scm b/guix/scripts/substitute-binary.scm
index 901b3fb..ec620c5 100755
--- a/guix/scripts/substitute-binary.scm
+++ b/guix/scripts/substitute-binary.scm
@@ -142,7 +142,7 @@ provide."
      ;;   sudo tc qdisc del dev eth0 root
      (let ((port #f))
        (with-timeout (if (or timeout? (guile-version>? "2.0.5"))
-                         %fetch-timeout
+                         0
                          0)
          (begin
            (warning (_ "while fetching ~a: server is unresponsive~%")
-- 
1.7.10.4



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 259 bytes --]

Re: fetching substitutes slow

[Ludovic Courtès]

Sree Harsha Totakura <sreeharsha@totakura.in> skribis:

> So, I removed the timeout with the attached patch and the substitute
> downloads are working again.  This time I observed that the speed at
> which the substitute-binary.scm downloads the nar file is far low than
> when the same file is downloaded through wget.  I haven't done extensive
> testing on this to confirm firmly, but it looks like the http-fetch
> implementation is not efficient.  Can anyone confirm this?

What version of Guile are you using?

With Guile > 2.0.5, you get the “is slow” warning, but downloading
proceeds anyway (and there should be a progress report.)  The intent
with 2.0.5 is that there’s no warning at all because its API didn’t
allow us to resume download.

Are you observing something different?

Thanks,
Ludo’.