GuixSD installation: binary vs source packages

GuixSD installation: binary vs source packages

[Alex Vestgaard]

[-- Attachment #1: Type: text/plain, Size: 801 bytes --]

Hi,

I'm trying to migrate all my machines to GuixSD, which I believe will make some complicated data processing pipelines much easier to deploy and reproduce.

I have one question regarding the installation process, which other people on #guix could not answer. How can I determine what packages will be installed from source and compiled vs just binaries fetched from Hydra?

I tried the --dry-run for guix package -i and guix build, but I didn't find this very informative.

Lengthy compilation took me by surprise twice:

i) When I first started installing GuixSD, Guix got compiled. I expected a binary would be fetched instead.
ii) When installing icecat. I thought in principle there were binary substitutes for all packages in Hydra.

I would appreciate if anyone could clarify these.

Thanks.

[-- Attachment #2: Type: text/html, Size: 1056 bytes --]

Re: GuixSD installation: binary vs source packages

[myglc2]

On 02/24/2017 at 13:39 Alex Vestgaard writes:

> Hi,
>
> I'm trying to migrate all my machines to GuixSD, which I believe will
> make some complicated data processing pipelines much easier to deploy
> and reproduce.
>
> I have one question regarding the installation process, which other
> people on #guix could not answer. How can I determine what packages
> will be installed from source and compiled vs just binaries fetched
> from Hydra?

In my practical experience of running a GuixSD headless server for the
last 12 months, if you use 'guix pull', which is effectively a rolling
release, there is no way to tell. It depends on whether hydra has the
substitute you want, how responsive it is, the version you are
installing, etc.

I run Debian on an identical server, and the Debian updates typically
take 1/10th the time of a GuixSD update. And the Debian updates inspire
confidence in a way the GuixSD updates do not. This used to bother me,
but I got over it ;-)

I mostly use a Guix git source checkout and "re-"build guix from time to
time.  I find, as far as my server goes, that I take a hit each time I
git pull and build Guix, and then the incremental installs of packages
are pretty quick.  But I am doing this primarily because I am interested
in the Guix source and being able to patch guix packages, etc, not
because of update speed. If you don't care about the source, I think you
should probably use guix pull.

I have 3.4GHz, 8 CPUs and 30 MBPS internet and in practice my updates
don't take more than 5-10 min, whether I git pull and build from source
or guix pull, so it is not really a big deal in either case.

I am running a headless server, and not building a bunch of desktop
packages, which may make a difference in all this.

You can keep all of your machines on the same release by using a git
checkout, and share the store among them from a local server. This would
dramatically speed things up.

If you are using guix pull, it will again depend on whether substitutes
in your local cache is relevant.

> I tried the --dry-run for guix package -i and guix build, but I didn't
> find this very informative.
>
> Lengthy compilation took me by surprise twice:
>
> i) When I first started installing GuixSD, Guix got compiled. I
> expected a binary would be fetched instead.

The problem here, as I understand it, is that, by the time a Guix Noob
is using the install image the substitutes for that version of guix may
have "aged off" of hydra. I believe hydra's capacity has been improved
so this situation may be improved.

> ii) When installing icecat. I thought in principle there were binary
> substitutes for all packages in Hydra.

> I would appreciate if anyone could clarify these.

Let me just say that I have found GuixSD to be very solid. There are
occasional glitches with newly ported packages, which is to be
expected. Compared to Debian 8, and practically speaking, as daily
drivers, my GuixSD machine has slower updates and the GuixSD packages
are more up-to-date, which is more important to me. So I plan to switch
the Debian machine over to GuixSD.

HTH - George

Re: GuixSD installation: binary vs source packages

[Leo Famulari]

On Fri, Feb 24, 2017 at 09:01:39PM -0500, myglc2 wrote:
> In my practical experience of running a GuixSD headless server for the
> last 12 months, if you use 'guix pull', which is effectively a rolling
> release, there is no way to tell.

The default action of `guix pull` is like a rolling release, but you can
also use it to deploy any version or Git commit of Guix, using `guix
pull --url=...`:

https://git.savannah.gnu.org/cgit/guix.git/commit/?id=8a9cffb202414b20081910115ba76402924bdcdd

You still can't be sure that we will have binary substitutes for every
package. I just wanted to point out this little-known feature of `guix
pull`, which gives Guix users some more flexibility without requiring
them to maintain a Guix development environment.

Re: GuixSD installation: binary vs source packages

[myglc2]

On 02/25/2017 at 10:52 Leo Famulari writes:

> On Fri, Feb 24, 2017 at 09:01:39PM -0500, myglc2 wrote:
>> In my practical experience of running a GuixSD headless server for the
>> last 12 months, if you use 'guix pull', which is effectively a rolling
>> release, there is no way to tell.
>
> The default action of `guix pull` is like a rolling release, but you can
> also use it to deploy any version or Git commit of Guix, using `guix
> pull --url=...`:
>
> https://git.savannah.gnu.org/cgit/guix.git/commit/?id=8a9cffb202414b20081910115ba76402924bdcdd
>
> You still can't be sure that we will have binary substitutes for every
> package. I just wanted to point out this little-known feature of `guix
> pull`, which gives Guix users some more flexibility without requiring
> them to maintain a Guix development environment.

Thanks. I think the issue with this feature is that there is no guidance
for picking an URL.

Here is an idea... What if hydra provided an URL guidance page?  For
example, it could have a short list of "guix greatest hits URLs" for
which, hopefully, a full set of substitutes is maintained. Obvious
candidates for this list would be the last couple install images. Or, it
could provide a page ranking URLs by release date and showing substitute
coverage.

Re: GuixSD installation: binary vs source packages

[Leo Famulari]

On Sat, Feb 25, 2017 at 03:12:26PM -0500, myglc2 wrote:
> Thanks. I think the issue with this feature is that there is no guidance
> for picking an URL.
>
> Here is an idea... What if hydra provided an URL guidance page?  For
> example, it could have a short list of "guix greatest hits URLs" for
> which, hopefully, a full set of substitutes is maintained. Obvious
> candidates for this list would be the last couple install images. Or, it
> could provide a page ranking URLs by release date and showing substitute
> coverage.

Don't the examples in the manual provide some guidance about how to use
the feature? I meant to make it clear that you can ask the server for a
snapshot of a release tag.

Currently we don't have the capacity to keep substitutes spanning
multiple release tags.

I don't have the knowledge to evaluate the possibility of ranking
commits by substitute coverage.

Re: GuixSD installation: binary vs source packages

[Ludovic Courtès]

Hello,

Alex Vestgaard <birkeroed@protonmail.ch> skribis:

> I have one question regarding the installation process, which other people on #guix could not answer. How can I determine what packages will be installed from source and compiled vs just binaries fetched from Hydra?
>
> I tried the --dry-run for guix package -i and guix build, but I didn't find this very informative.

Normally --dry-run should give a good overview of what will be
downloaded and what will be built.

However, in the presence of “grafts”, it may be that additional things
will be downloaded or built without you being notified beforehand.

I consider this a bug but we don’t have an easy fix at this stage.

Thanks,
Ludo’.

Re: GuixSD installation: binary vs source packages

[dian_cecht]

On Mon, 06 Mar 2017 11:59:51 +0100
ludo@gnu.org (Ludovic Courtès) wrote:
> However, in the presence of “grafts”, it may be that additional things
> will be downloaded or built without you being notified beforehand.
> 
> I consider this a bug but we don’t have an easy fix at this stage.

Is there a number/link for that bug?

Re: GuixSD installation: binary vs source packages

[Ludovic Courtès]

<dian_cecht@zoho.com> skribis:

> On Mon, 06 Mar 2017 11:59:51 +0100
> ludo@gnu.org (Ludovic Courtès) wrote:
>> However, in the presence of “grafts”, it may be that additional things
>> will be downloaded or built without you being notified beforehand.
>> 
>> I consider this a bug but we don’t have an easy fix at this stage.
>
> Is there a number/link for that bug?

I realize there’s no explicit bug for that one, though it kind of
relates to <https://bugs.gnu.org/22990>.

Feel free to open a separate one!

Thanks,
Ludo’.