Re: Is it necessary to download the native inputs while installing packages?

[Leo Famulari <leo@famulari.name>]

Sorry for the top-posting.

I need to clarify what --fallback does.

Guix will always build locally if the substitute servers say they lack something Guix needs.

--fallback works when the substitute servers say they have something, and then the substitution fails unexpectedly. For example, the server may become overloaded and drop the connection.


-------- Original Message --------
From: Tobias Geerinckx-Rice <me@tobias.gr>
Sent: January 18, 2018 9:36:50 AM PST
To: ybbs.daans@hotmail.com, guix-devel@gnu.org
Subject: Re: Is it necessary to download the native inputs while installing	packages?

Hullo,

There are quite a few unrelated points mixed together here. I'll try to
address all of them, but I'm not entirely sure what your main question
is. I hope to answer it by accident.

Fis Trivial wrote on 18/01/18 at 16:19:
> The network connections to hydra is bad at day time from where I am,
>  the download speed is about 10kB/s, I can only work with guix at 
> night. And sometimes I use guix --fallback option(like texlive), guix
> will redirect to the upstream download link.

I get your meaning, but ‘redirect’ isn't the right word to use here.
Redirection is something servers do, and that's not the case here.

When installing a package, Guix queries all known substitute servers for
a pre-built binary substitute. What ‘--fallback’ does is tell Guix to
not give up if all servers answer negatively (404), but to build the
package from source locally. For that it needs to download the source
from upstream servers, and of course the native inputs required to
perform the build. There's no way around that.

If any of the substitute servers *can* provide a binary, no
native-inputs will be downloaded at all.

Why are you adding ‘--fallback’?

> Chances are the upstream has even worse network bandwidth, then I 
> have to manually find a mirror of that particular upstream and use 
> `guix download`.

This is an unrelated but valid point. I've previously mused that it
should be possible to let users choose (a set of) their favourite
mirror(s), instead of always downloading sources from the first mirror
queried.

It could be as simple as adding ISO codes to each mirror URI.

> In the case of *texlive-texmf*, it serves mostly as a build 
> dependence. To make things even less convenient, most of the mirror 
> site only store its ISO package, not the tar ball used in guix 
> package.

Oh. That's unfortunate. I'm not familiar with the Guix texlive packages,
and all the ones I happened to look at now have a direct ftp:// link.

Is upstream the one doing the — heh — redirection here? If so, it should
arguably doing a better job. We could still help matters by hard-coding
a shortlist of mirrors known to host the files we need.

> So, I gotta wonder, can we just download the normal inputs and 
> propagated inputs from hydra while installing packages? Would that 
> break something like the functional property of guix?

‘Installing’ is too ambiguous to answer.

- When *substituting* (installing a binary), native-inputs are not
required and should never be downloaded.
- When *building* (from source), they are.

Kind regards,

T G-R

PS: Oh. Right. Sigh... To complicate matters, texlive-texmf is a special
case that's never substituted, because doing so is thought to be worse
than just building it from source[1].

That's the reason you're downloading TexLive from upstream, but it's
*not* the reason you're pulling in TexLive to begin with and doesn't
change the native-inputs situation. I think.

[1]:
https://git.savannah.gnu.org/cgit/guix/maintenance.git/tree/hydra/nginx/hydra.gnu.org-locations.conf#n23