Speed difference: wget vs guix build

Speed difference: wget vs guix build

[Ricardo Wurmus]

Hi Guix,

a few weeks ago I noticed that the download of binary substitutes is
fairly slow when it is performed from within "guix build" compared with
wget:

    [rwurmus@guix-builder:~] (557) $ time wget http://hydra.gnu.org/nar/h1qjkg8bvcdb485q81lw5844szgywpg7-texlive-texmf-2014
    --2015-05-06 08:20:38--  http://hydra.gnu.org/nar/h1qjkg8bvcdb485q81lw5844szgywpg7-texlive-texmf-2014
    Resolving hydra.gnu.org (hydra.gnu.org)... 18.4.89.46
    Connecting to hydra.gnu.org (hydra.gnu.org)|18.4.89.46|:80... connected.
    HTTP request sent, awaiting response... 200 OK
    Length: unspecified [application/x-nix-archive]
    Saving to: ‘h1qjkg8bvcdb485q81lw5844szgywpg7-texlive-texmf-2014.1’

        [                                         <=>                                      ] 761,111,979 4.17MB/s

The speed ranges from ~4MB/s to 7MB/s with wget.
With guix build, however, I get much less on the same host:

    [rwurmus@guix-builder:/localhome/rwurmus-tmp/guix] (557) $ ./pre-inst-env guix build texlive-texmf-2014
    The following file will be downloaded:
       /gnu/store/h1qjkg8bvcdb485q81lw5844szgywpg7-texlive-texmf-2014
    @ substituter-started /gnu/store/h1qjkg8bvcdb485q81lw5844szgywpg7-texlive-texmf-2014 /usr/libexec/guix/substitute
    found valid signature for '/gnu/store/h1qjkg8bvcdb485q81lw5844szgywpg7-texlive-texmf-2014', from 'http://hydra.gnu.org/nar/h1qjkg8bvcdb485q81lw5844szgywpg7-texlive-texmf-2014'
    downloading `/gnu/store/h1qjkg8bvcdb485q81lw5844szgywpg7-texlive-texmf-2014' (3391.2 MiB installed)...
    http://hydra.gnu.org/nar/h1qjkg8bvcdb485q81lw5844szgywpg7-texlive-texmf-2014	124344.0 KiB transferred (321. KiB/s)

Is this expected?

~~ Ricardo

Re: Speed difference: wget vs guix build

[Ludovic Courtès]

Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de> skribis:

>     [rwurmus@guix-builder:~] (557) $ time wget http://hydra.gnu.org/nar/h1qjkg8bvcdb485q81lw5844szgywpg7-texlive-texmf-2014
>     --2015-05-06 08:20:38--  http://hydra.gnu.org/nar/h1qjkg8bvcdb485q81lw5844szgywpg7-texlive-texmf-2014
>     Resolving hydra.gnu.org (hydra.gnu.org)... 18.4.89.46
>     Connecting to hydra.gnu.org (hydra.gnu.org)|18.4.89.46|:80... connected.
>     HTTP request sent, awaiting response... 200 OK
>     Length: unspecified [application/x-nix-archive]
>     Saving to: ‘h1qjkg8bvcdb485q81lw5844szgywpg7-texlive-texmf-2014.1’
>
>         [                                         <=>                                      ] 761,111,979 4.17MB/s
>
> The speed ranges from ~4MB/s to 7MB/s with wget.
> With guix build, however, I get much less on the same host:
>
>     [rwurmus@guix-builder:/localhome/rwurmus-tmp/guix] (557) $ ./pre-inst-env guix build texlive-texmf-2014
>     The following file will be downloaded:
>        /gnu/store/h1qjkg8bvcdb485q81lw5844szgywpg7-texlive-texmf-2014
>     @ substituter-started /gnu/store/h1qjkg8bvcdb485q81lw5844szgywpg7-texlive-texmf-2014 /usr/libexec/guix/substitute
>     found valid signature for '/gnu/store/h1qjkg8bvcdb485q81lw5844szgywpg7-texlive-texmf-2014', from 'http://hydra.gnu.org/nar/h1qjkg8bvcdb485q81lw5844szgywpg7-texlive-texmf-2014'
>     downloading `/gnu/store/h1qjkg8bvcdb485q81lw5844szgywpg7-texlive-texmf-2014' (3391.2 MiB installed)...
>     http://hydra.gnu.org/nar/h1qjkg8bvcdb485q81lw5844szgywpg7-texlive-texmf-2014	124344.0 KiB transferred (321. KiB/s)

Beware: wget shows the “instantaneous” throughput, whereas Guix shows
the average throughput since the beginning.  So to be sure, you need to
make a full download.  Also, please compare not only against ‘guix
substitute’ but also against ‘guix download’.

Regardless, I don’t see anything like that here.  Commit c822fb8 may
help a bit; could you retry?

Commit 7623848 helps for source derivations and ‘guix download’ for
Guile <= 2.0.11, but it shouldn’t make any difference for ‘guix
substitute’.

Thanks for the report,
Ludo’.