Package Installation takes a very long time - normal?

Package Installation takes a very long time - normal?

[Niall Dooley]

I'm a user of Guix but not GuixSD. Some time ago I installed Guix following
the instructions in the manual and since then have installed a total of 16
packages. I'm raising this query to better understand why it takes so long for
me to install any one package.

Is this a consequence of my machine being old - it's a refurbished (Libreboot)
ThinkPad X60s running Trisquel 7 with 2.9 GiB memory with a 120 GB SSD?

Is it just simple a consequence of functional package management - as based
purely on my experience this will be a considerable deterrant to most users
despite the obvious advantages that come with functional package management
systems in general and Guix in particular?

Perhaps, it's user error (would be the first time) and there is a step
I missed that
would avoid these long package installation times?

Thanks for Guix and your time.

Re: Package Installation takes a very long time - normal?

[Leo Famulari]

[-- Attachment #1: Type: text/plain, Size: 1151 bytes --]

On Mon, Jul 10, 2017 at 07:52:46PM +0100, Niall Dooley wrote:
> I'm a user of Guix but not GuixSD. Some time ago I installed Guix following
> the instructions in the manual and since then have installed a total of 16
> packages. I'm raising this query to better understand why it takes so long for
> me to install any one package.
> 
> Is this a consequence of my machine being old - it's a refurbished (Libreboot)
> ThinkPad X60s running Trisquel 7 with 2.9 GiB memory with a 120 GB SSD?
> 
> Is it just simple a consequence of functional package management - as based
> purely on my experience this will be a considerable deterrant to most users
> despite the obvious advantages that come with functional package management
> systems in general and Guix in particular?
> 
> Perhaps, it's user error (would be the first time) and there is a step
> I missed that
> would avoid these long package installation times?

Could you give more detail?

For example, what version of Guix are you using (guix --version)? What
command did you run? How long did it take? What was it doing (compiling,
downloading, seemingly nothing, etc)?

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Package Installation takes a very long time - normal?

[Niall Dooley]

Yes, of course. Plus at the end I meant it wouldn't be the first time in my
case of user error.

I'm running Guix 20170710.15 as I did a `guix pull' this afternoon so I
presume the most recent build. I'm currently trying to install several R
related packages (only just started after my first mail) so I'll report back
how long it takes. The command I've used is as follows:

guix package -i r-caret r-ggplot2 r-rpart r-e1071 r-randomforest r-rocr r-class

As I type my terminal window is churning away but occasionally it pauses
seemingly doing nothing and then it will again start churning posting the
various messages. It's certainly past the initial updating of the
substitutes and
downloading phase. If I had to guess based on the messages I can catch it is
currently installing various *source* packages.

On 10 July 2017 at 20:11, Leo Famulari <leo@famulari.name> wrote:
> On Mon, Jul 10, 2017 at 07:52:46PM +0100, Niall Dooley wrote:
>> I'm a user of Guix but not GuixSD. Some time ago I installed Guix following
>> the instructions in the manual and since then have installed a total of 16
>> packages. I'm raising this query to better understand why it takes so long for
>> me to install any one package.
>>
>> Is this a consequence of my machine being old - it's a refurbished (Libreboot)
>> ThinkPad X60s running Trisquel 7 with 2.9 GiB memory with a 120 GB SSD?
>>
>> Is it just simple a consequence of functional package management - as based
>> purely on my experience this will be a considerable deterrant to most users
>> despite the obvious advantages that come with functional package management
>> systems in general and Guix in particular?
>>
>> Perhaps, it's user error (would be the first time) and there is a step
>> I missed that
>> would avoid these long package installation times?
>
> Could you give more detail?
>
> For example, what version of Guix are you using (guix --version)? What
> command did you run? How long did it take? What was it doing (compiling,
> downloading, seemingly nothing, etc)?

Re: Package Installation takes a very long time - normal?

[Leo Famulari]

[-- Attachment #1: Type: text/plain, Size: 1542 bytes --]

On Mon, Jul 10, 2017 at 08:28:15PM +0100, Niall Dooley wrote:
> Yes, of course. Plus at the end I meant it wouldn't be the first time in my
> case of user error.
> 
> I'm running Guix 20170710.15 as I did a `guix pull' this afternoon so I
> presume the most recent build. I'm currently trying to install several R
> related packages (only just started after my first mail) so I'll report back
> how long it takes. The command I've used is as follows:
> 
> guix package -i r-caret r-ggplot2 r-rpart r-e1071 r-randomforest r-rocr r-class
> 
> As I type my terminal window is churning away but occasionally it pauses
> seemingly doing nothing and then it will again start churning posting the
> various messages. It's certainly past the initial updating of the
> substitutes and
> downloading phase. If I had to guess based on the messages I can catch it is
> currently installing various *source* packages.

It sounds like you are building some packages from source rather than
downloading pre-built substitutes.

It's possible these packages have been built on our build farm but not
yet available on the caching mirror. When you request a substitute from
the mirror that is not cached, the mirror will return 404 while checking
if it's available from the build farm.

So, sometimes it's enough to wait a few minutes after the 404 response
and then try again; the mirror might have populated itself in the
meantime. Otherwise, we don't have these packages built currently, and
you'll need to build them yourself.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Package Installation takes a very long time - normal?

[Niall Dooley]

OK, so it has finished but unsuccessfully. The last lines of the output were
as follows but none of the packages were installed. To reach this point it
took about 50 mins (it started around 19:55 and finished at 20:43).

The following packages will be installed:
   r-caret 6.0-76 /gnu/store/dybi25l6wr14y3cxw6xazans9vlqjlji-r-caret-6.0-76
   r-ggplot2 2.2.1 /gnu/store/11jzzbgpg67v3yysiv6pyhi27f75dyh1-r-ggplot2-2.2.1
   r-rpart 4.1-11 /gnu/store/rrsr0q4q4n2wjfg26kyg4klk708f675f-r-rpart-4.1-11
   r-e1071 1.6-8 /gnu/store/1r4if4p10rnyj6q073yjir99mpriv1d0-r-e1071-1.6-8
   r-randomforest 4.6-12
/gnu/store/nkqqls9piqzp7pdgfrka1rz005hxj19c-r-randomforest-4.6-12
   r-rocr 1.0-7 /gnu/store/ldn5y4qcz6kxgqkhld69w00lvbiz9682-r-rocr-1.0-7
   r-class 7.3-14 /gnu/store/bqx4c9r3pzmf12vlfy2hdw8hamk8z45n-r-class-7.3-14

guix package: error: profile contains conflicting entries for r-mass:out
guix package: error:   first entry: r-mass@7.3-47:out
/gnu/store/z0nxbji8yrkdl7n6v3wdsd8iqcj39vyk-r-mass-7.3-47
guix package: error:    ... propagated from r-car@2.1-4
guix package: error:    ... propagated from r-caret@6.0-76
guix package: error:   second entry: r-mass@7.3-47:out
/gnu/store/gnv11ncv2h9ij48jwy86wai3xsm7c9s5-r-mass-7.3-47
guix package: error:    ... propagated from r@3.4.0

I had previously installed successfully R and r-catools via Guix.

Ok, I see. And is there something I need to do to only install the pre-built
substitutes and not to install from source?

Niall


On 10 July 2017 at 21:18, Leo Famulari <leo@famulari.name> wrote:
> On Mon, Jul 10, 2017 at 08:28:15PM +0100, Niall Dooley wrote:
>> Yes, of course. Plus at the end I meant it wouldn't be the first time in my
>> case of user error.
>>
>> I'm running Guix 20170710.15 as I did a `guix pull' this afternoon so I
>> presume the most recent build. I'm currently trying to install several R
>> related packages (only just started after my first mail) so I'll report back
>> how long it takes. The command I've used is as follows:
>>
>> guix package -i r-caret r-ggplot2 r-rpart r-e1071 r-randomforest r-rocr r-class
>>
>> As I type my terminal window is churning away but occasionally it pauses
>> seemingly doing nothing and then it will again start churning posting the
>> various messages. It's certainly past the initial updating of the
>> substitutes and
>> downloading phase. If I had to guess based on the messages I can catch it is
>> currently installing various *source* packages.
>
> It sounds like you are building some packages from source rather than
> downloading pre-built substitutes.
>
> It's possible these packages have been built on our build farm but not
> yet available on the caching mirror. When you request a substitute from
> the mirror that is not cached, the mirror will return 404 while checking
> if it's available from the build farm.
>
> So, sometimes it's enough to wait a few minutes after the 404 response
> and then try again; the mirror might have populated itself in the
> meantime. Otherwise, we don't have these packages built currently, and
> you'll need to build them yourself.

Re: Package Installation takes a very long time - normal?

[Leo Famulari]

[-- Attachment #1: Type: text/plain, Size: 2313 bytes --]

On Mon, Jul 10, 2017 at 09:25:27PM +0100, Niall Dooley wrote:
> OK, so it has finished but unsuccessfully. The last lines of the output were
> as follows but none of the packages were installed. To reach this point it
> took about 50 mins (it started around 19:55 and finished at 20:43).

That sounds annoying :(

> The following packages will be installed:
>    r-caret 6.0-76 /gnu/store/dybi25l6wr14y3cxw6xazans9vlqjlji-r-caret-6.0-76
>    r-ggplot2 2.2.1 /gnu/store/11jzzbgpg67v3yysiv6pyhi27f75dyh1-r-ggplot2-2.2.1
>    r-rpart 4.1-11 /gnu/store/rrsr0q4q4n2wjfg26kyg4klk708f675f-r-rpart-4.1-11
>    r-e1071 1.6-8 /gnu/store/1r4if4p10rnyj6q073yjir99mpriv1d0-r-e1071-1.6-8
>    r-randomforest 4.6-12
> /gnu/store/nkqqls9piqzp7pdgfrka1rz005hxj19c-r-randomforest-4.6-12
>    r-rocr 1.0-7 /gnu/store/ldn5y4qcz6kxgqkhld69w00lvbiz9682-r-rocr-1.0-7
>    r-class 7.3-14 /gnu/store/bqx4c9r3pzmf12vlfy2hdw8hamk8z45n-r-class-7.3-14
> 
> guix package: error: profile contains conflicting entries for r-mass:out
> guix package: error:   first entry: r-mass@7.3-47:out
> /gnu/store/z0nxbji8yrkdl7n6v3wdsd8iqcj39vyk-r-mass-7.3-47
> guix package: error:    ... propagated from r-car@2.1-4
> guix package: error:    ... propagated from r-caret@6.0-76
> guix package: error:   second entry: r-mass@7.3-47:out
> /gnu/store/gnv11ncv2h9ij48jwy86wai3xsm7c9s5-r-mass-7.3-47
> guix package: error:    ... propagated from r@3.4.0

I recommend updating all the packages that are trying to propagate
r-mass. That is, R, r-car, and r-caret. We recently made a change to
throw an error here:

https://bugs.gnu.org/27271

I don't fully understand the impact of the change so I asked Ludovic to
chime in.

> I had previously installed successfully R and r-catools via Guix.
> 
> Ok, I see. And is there something I need to do to only install the pre-built
> substitutes and not to install from source?

Guix is a build-from-source distro that can offer pre-built binary
substitutes as a side-effect of the functional package management model.
Your request has been discussed previously but, AFAIK, it's not being
worked on at the moment.

We are currently bringing some more x86_64 / i686 build machines online,
so the build / substitute ratio should improve soon. We know it should
be better.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Package Installation takes a very long time - normal?

[Ludovic Courtès]

Leo Famulari <leo@famulari.name> skribis:

> On Mon, Jul 10, 2017 at 09:25:27PM +0100, Niall Dooley wrote:
>> OK, so it has finished but unsuccessfully. The last lines of the output were
>> as follows but none of the packages were installed. To reach this point it
>> took about 50 mins (it started around 19:55 and finished at 20:43).
>
> That sounds annoying :(

Indeed…

>> The following packages will be installed:
>>    r-caret 6.0-76 /gnu/store/dybi25l6wr14y3cxw6xazans9vlqjlji-r-caret-6.0-76
>>    r-ggplot2 2.2.1 /gnu/store/11jzzbgpg67v3yysiv6pyhi27f75dyh1-r-ggplot2-2.2.1
>>    r-rpart 4.1-11 /gnu/store/rrsr0q4q4n2wjfg26kyg4klk708f675f-r-rpart-4.1-11
>>    r-e1071 1.6-8 /gnu/store/1r4if4p10rnyj6q073yjir99mpriv1d0-r-e1071-1.6-8
>>    r-randomforest 4.6-12
>> /gnu/store/nkqqls9piqzp7pdgfrka1rz005hxj19c-r-randomforest-4.6-12
>>    r-rocr 1.0-7 /gnu/store/ldn5y4qcz6kxgqkhld69w00lvbiz9682-r-rocr-1.0-7
>>    r-class 7.3-14 /gnu/store/bqx4c9r3pzmf12vlfy2hdw8hamk8z45n-r-class-7.3-14
>> 
>> guix package: error: profile contains conflicting entries for r-mass:out
>> guix package: error:   first entry: r-mass@7.3-47:out
>> /gnu/store/z0nxbji8yrkdl7n6v3wdsd8iqcj39vyk-r-mass-7.3-47
>> guix package: error:    ... propagated from r-car@2.1-4
>> guix package: error:    ... propagated from r-caret@6.0-76
>> guix package: error:   second entry: r-mass@7.3-47:out
>> /gnu/store/gnv11ncv2h9ij48jwy86wai3xsm7c9s5-r-mass-7.3-47
>> guix package: error:    ... propagated from r@3.4.0
>
> I recommend updating all the packages that are trying to propagate
> r-mass. That is, R, r-car, and r-caret. We recently made a change to
> throw an error here:
>
> https://bugs.gnu.org/27271
>
> I don't fully understand the impact of the change so I asked Ludovic to
> chime in.

Previously, Guix would have proceeded with the installation; you’d get
many warning lines just before it completes, but that’s it.

The problem is that it’s a situation where failing is safer: if we let
it proceed, the profile would have to arbitrarily choose between the
files coming from one ‘r-mass’ variant and those coming from the other
one.

The fix is to arrange to have only one ‘r-mass’ in your profile.  Here
the message tells you that one of them is propagated by ‘r’ and the
second one by ‘r-caret’, so you have to get these two in sync,
presumably by adding ‘r’ to your “guix package -i” command line (I’m not
familiar with R, maybe Ricardo can comment on this.)

Does that make sense?

To avoid these problems, you might also want to use a manifest with
“guix package -m”.

I hope this clarifies things.

Thanks for your feedback,
Ludo’.