Hydra serving source tarballs

Hydra serving source tarballs

[Sree Harsha Totakura]

Hi,

I just observed that `guix build' is downloading source tarballs from
Hydra.  Is this supposed to be a feature?  If so, what is the rationale
behind not trying to download the source ball from mirrors or upstream
download addresses?

Regards,
Sree

Re: Hydra serving source tarballs

[Andreas Enge]

On Wed, Feb 12, 2014 at 01:34:26PM +0100, Sree Harsha Totakura wrote:
> I just observed that `guix build' is downloading source tarballs from
> Hydra.  Is this supposed to be a feature?  If so, what is the rationale
> behind not trying to download the source ball from mirrors or upstream
> download addresses?

I think it is a feature for tarballs that have disappeared from upstream
(it sounds a bit crazy, but some projects delete old versions, or move them
to a separate location). Some also modify the tarball without giving it
a new name.

I think it is a bug because usually hydra is slower than upstream.
Using "--no-substitutes" prevents the download of tarballs, but also the
maybe desired download of binaries from hydra.

I would be in favour of downloading tarballs from upstream by default, and,
if it fails, keeping tarball downloads from hydra as a last resort.

Andreas

Re: Hydra serving source tarballs

[Ludovic Courtès]

Sree Harsha Totakura <sreeharsha@totakura.in> skribis:

> I just observed that `guix build' is downloading source tarballs from
> Hydra.  Is this supposed to be a feature?  If so, what is the rationale
> behind not trying to download the source ball from mirrors or upstream
> download addresses?

Downloads are normal derivations, like builds¹.  Thus, their result is
cached into the store, and so Hydra serves substitutes for them just
like for any other derivation.

As Andreas said, this is a desirable feature for upstream sites that
remove old source tarballs.

However, hydra.gnu.org is indeed often slower than upstream sites these
days.  This is obviously not inherent to the mechanism, and instead has
to do, I believe, with hydra.gnu.org being an overloaded VM.

I think things will improve when we start offloading builds to other
machines, which I hope will happen Real Soon Now.

Thanks,
Ludo’.

¹ With a slight difference: the hash of the output of a download
  derivation is known in advance.  We call them “fixed-output”
  derivations, because the file name of the output is independent of the
  inputs used to perform the actual download.