bug#35012: Can't install icecat

bug#35012: Can't install icecat

[Alberto EFG]

Hello, for the last few weeks I've been unable to upgrade icecat.

I use Guix on top of Fedora 29, I've used it with selinux set to
permissive and set to enforce. I've run guix-daemon manually and using
systemd. Yesterday I removed Guix entirely and installed it from scratch.

It keeps failing no matter what. I have about 200gb of disk space and
8gb of RAM. This is the error I always get:

$ guix package -c 7 --upgrade icecat --substitute-urls="https://ci.guix.info"
substitute: updating substitutes from 'https://ci.guix.info'... 100.0%
substitute: updating substitutes from 'https://ci.guix.info'... 100.0%
downloading from https://ci.guix.info/nar/qlwg5lyxnnj68kw2agx4z5fr2b6vxpdc-icecat-60.6.1-guix1.tar.xz...

icecat-60.6.1-guix1.tar.xz  303.0MiB   1.8MiB/s 02:47 [##################] 100.0%

building /gnu/store/jpg44cnp64hzdi2hislnid40jdr3kkhp-rust-1.24.1.drv...
\ 'install' phasenote: build failure may have been caused by lack of free disk space
builder for `/gnu/store/jpg44cnp64hzdi2hislnid40jdr3kkhp-rust-1.24.1.drv' failed with exit code 1
build of /gnu/store/jpg44cnp64hzdi2hislnid40jdr3kkhp-rust-1.24.1.drv failed
View build log at '/var/log/guix/drvs/jp/g44cnp64hzdi2hislnid40jdr3kkhp-rust-1.24.1.drv.bz2'.
cannot build derivation `/gnu/store/mqx2ad5zkki6s02c6x7gkdzmb2p0zd6s-icecat-60.6.1-guix1.drv': 1 dependencies couldn't be built
guix package: error: build of `/gnu/store/mqx2ad5zkki6s02c6x7gkdzmb2p0zd6s-icecat-60.6.1-guix1.drv' failed

bug#35012: Can't install icecat

[Björn Höfling]

[-- Attachment #1: Type: text/plain, Size: 2122 bytes --]

Hi Alberto,

it should be available now from mirror.hydra.gnu.org. Some detailed
explanations follow:

On Tue, 26 Mar 2019 13:39:09 -0600
Alberto EFG <albertoefg@posteo.mx> wrote:

> Hello, for the last few weeks I've been unable to upgrade icecat.
> 
> I use Guix on top of Fedora 29, I've used it with selinux set to
> permissive and set to enforce. I've run guix-daemon manually and using
> systemd. Yesterday I removed Guix entirely and installed it from
> scratch.
> 

First, you can try to see if your package would be built locally or
will be downloaded from substitute servers with the "-n" == "--dry-run"
option:

$ guix package -n -u icecat

The following package would be upgraded:
   icecat	60.5.0-guix1 → 60.6.1-guix1	/gnu/store/bwhnxvfx4ng5fhc7xi6wijp8qciqjpbk-icecat-60.6.1-guix1

The following derivation would be built:
   /gnu/store/m3i38zbqb4ww4qjnw01jbj0wjxmyp0id-profile.drv
55.7 MB would be downloaded:
   /gnu/store/bwhnxvfx4ng5fhc7xi6wijp8qciqjpbk-icecat-60.6.1-guix1
The following profile hooks would be built:
[..]

In that case, it would be downloaded from substitute servers.

If it does not use the substitute, then it could either not yet be
built on the substitute servers or it could
fail to build.

As you already tried to build it locally, you could look at the log
file and see if there is anything useful in there:

> $ guix package -c 7 --upgrade icecat
[..]

> View build log at
> '/var/log/guix/drvs/jp/g44cnp64hzdi2hislnid40jdr3kkhp-rust-1.24.1.drv.bz2'.
> cannot build derivation

I searched on Hydra for icecat and found it was built only today:

https://hydra.gnu.org/job/gnu/master/icecat-60.6.1-guix1.x86_64-linux/all
3431712 	icecat-60.6.1-guix1 	x86_64-linux 	2019-03-26 14:22:53

I could successfully update from there:

guix package --substitute-urls="https://mirror.hydra.gnu.org/" -u icecat

Especially for big packages like icecat there can always be a gap
between the time when the package definition is updated and when a
substitute is built and ready to be downloaded.

Hope this helps a bit,

Björn

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

bug#35012: Can't install icecat

[Danny Milosavljevic]

[-- Attachment #1: Type: text/plain, Size: 939 bytes --]

Just adding a few suggestions for Guix to do:

> Especially for big packages like icecat there can always be a gap
> between the time when the package definition is updated and when a
> substitute is built and ready to be downloaded.

Especially since icecat depends on Rust now.  Rust has a very long
bootstrap chain and if we change something in the base, the chain
has to be rebuilt.  That takes days on current hardware.

Also, currently, we disable tests parallelization in Rust 1.20 because
they have an error there.  But it would be possible and desireable to
enable them for newer versions again and see whether they work.  That
would entail replacing the "check" phase by one which invokes ./x.py
with

  "-j" (number->string (parallel-job-count))

(instead of the current

  "-j1"
)

And then build like 10 rounds.

If it works, the build of each of the corresponding Rust derivations
would be faster.

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

bug#35012: Can't install icecat

[Ludovic Courtès]

Danny Milosavljevic <dannym@scratchpost.org> skribis:

> Also, currently, we disable tests parallelization in Rust 1.20 because
> they have an error there.  But it would be possible and desireable to
> enable them for newer versions again and see whether they work.  

That’d be great.

Also, I wonder what caused a rebuild of the whole chain of Rust versions
the other day.  It would be great if we could reduce the list of
dependencies of older versions to the minimum so that they rarely need
to be rebuilt.

Thanks,
Ludo’.

bug#35012: Can't install icecat

[Mark H Weaver]

Hi Alberto,

> It keeps failing no matter what. I have about 200gb of disk space and
> 8gb of RAM. This is the error I always get:
>
> $ guix package -c 7 --upgrade icecat --substitute-urls="https://ci.guix.info"
> substitute: updating substitutes from 'https://ci.guix.info'... 100.0%
> substitute: updating substitutes from 'https://ci.guix.info'... 100.0%
> downloading from https://ci.guix.info/nar/qlwg5lyxnnj68kw2agx4z5fr2b6vxpdc-icecat-60.6.1-guix1.tar.xz...
>
> icecat-60.6.1-guix1.tar.xz  303.0MiB   1.8MiB/s 02:47 [##################] 100.0%
>
> building /gnu/store/jpg44cnp64hzdi2hislnid40jdr3kkhp-rust-1.24.1.drv...
> \ 'install' phasenote: build failure may have been caused by lack of free disk space

As other people have mentioned, this substitute should now be available,
but I wanted to add that builds are performed in /tmp by default, and
that's likely the reason you ran out of disk space, even though your
root partition has plenty of space.

If you'd like builds to be performed somewhere else, e.g. /var/tmp, then
you should arrange for the TMPDIR environment variable to be set
appropriately when launching 'guix-daemon'.

    Regards,
      Mark

bug#35012: Can't install icecat

[Alberto EFG]

H, Mark and everyone who has been helpful 

> As other people have mentioned, this substitute should now be
> available,
I keep getting this, I don't know if I am doing something wrong:

$ guix package -n -u .  --substitute-urls="https://mirror.hydra.gnu.org https://ci.guix.info"
[...]
substitute: updating substitutes from 'https://mirror.hydra.gnu.org'... 100.0%
substitute: updating substitutes from 'https://ci.guix.info'... 100.0%
The following derivations would be built:
   /gnu/store/5xs5i2pkf369g2hd26f2sy26r40bzjm6-icecat-60.6.1-guix1.drv
   /gnu/store/4f40wrx514rbly302ha0gvpn5b53d91f-rust-1.24.1.drv
   /gnu/store/yf5fcqp3v4jf038g454h2zqm54x6ijj6-profile.drv
938.6 MB would be downloaded:

> but I wanted to add that builds are performed in /tmp by default, and
> that's likely the reason you ran out of disk space, even though your
> root partition has plenty of space.
>
> If you'd like builds to be performed somewhere else, e.g. /var/tmp, then
> you should arrange for the TMPDIR environment variable to be set
> appropriately when launching 'guix-daemon'.
I will try this!!
Hopefully it will work, I don't mind the compiling I was just worried it
keep failing.

bug#35012: Can't install icecat

[Björn Höfling]

[-- Attachment #1: Type: text/plain, Size: 2720 bytes --]

Hi Alberto,

On Thu, 28 Mar 2019 23:11:22 -0600
Alberto EFG <albertoefg@posteo.mx> wrote:

> H, Mark and everyone who has been helpful 
> 
> > As other people have mentioned, this substitute should now be
> > available,  
> I keep getting this, I don't know if I am doing something wrong:
> 
> $ guix package -n -u .
> --substitute-urls="https://mirror.hydra.gnu.org
> https://ci.guix.info" [...] substitute: updating substitutes from
> 'https://mirror.hydra.gnu.org'... 100.0% substitute: updating
> substitutes from 'https://ci.guix.info'... 100.0% The following
> derivations would be
> built: /gnu/store/5xs5i2pkf369g2hd26f2sy26r40bzjm6-icecat-60.6.1-guix1.drv /gnu/store/4f40wrx514rbly302ha0gvpn5b53d91f-rust-1.24.1.drv
>    /gnu/store/yf5fcqp3v4jf038g454h2zqm54x6ijj6-profile.drv
> 938.6 MB would be downloaded:

I see that your hash of icecat differs from mine:

$ guix package -n -i icecat
--substitute-urls="https://mirror.hydra.gnu.org https://ci.guix.info"
The following package would be upgraded: icecat	60.6.1-guix1 →
60.6.1-guix1	/gnu/store/bwhnxvfx4ng5fhc7xi6wijp8qciqjpbk-icecat-60.6.1-guix1

substitute: updating substitutes from 'https://mirror.hydra.gnu.org'...
100.0% substitute: updating substitutes from 'https://ci.guix.info'...
100.0% The following derivation would be built:
   /gnu/store/f6gfzw25p9bqfwk2y7bdkkw1fhs83b2x-profile.drv
The following profile hooks would be built:
[..]

I.e. yours is 5xs5...-icecat, mine is bwhn...-icecat

When I'm looking at the latest evaluation of hydra, there I find the
bwhn... one:

https://hydra.gnu.org/build/3431712

Source Nix closure of
path /gnu/store/bwhnxvfx4ng5fhc7xi6wijp8qciqjpbk-icecat-60.6.1-guix1 

If you look there at the tab "inputs", you will see that it evaluated
from commit

fd9fe868f818b159b58c6a9811cc68e752a0af48

Looking at the git history, you will find out that this is from:

Committer: Tobias Geerinckx-Rice <me@tobias.gr>  2019-03-25 12:59:39

Your "guix pull" should be on this commit or later (but not too late,
such that firefox-affecting changes don't get in of cause :-)). 
You can directly try to use:

guix pull --commit="fd9fe868f818b159b58c6a9811cc68e752a0af48"

and then try again updating icecat like you did above.

Are you on x86_64? Although when I look at i686 and arm, the hashes are
totally different (we would have a problem with substitutes there
anyway).

For reference, I'm here:

$ guix describe
Generation 55	Mar 26 2019 00:36:00	(current)
  guix-hoebjo 385a225
[..]
  guix b9b177b
    repository URL: https://git.savannah.gnu.org/git/guix.git
    branch: master
    commit: b9b177b31756221837c8b85df786c8cdd7df3ef0

Björn

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

bug#35012: Can't install icecat

[Alberto EFG]

Hi Björn Höfling
> I.e. yours is 5xs5...-icecat, mine is bwhn...-icecat
>
> When I'm looking at the latest evaluation of hydra, there I find the
> bwhn... one:
>
> https://hydra.gnu.org/build/3431712
>
> Source Nix closure of
> path /gnu/store/bwhnxvfx4ng5fhc7xi6wijp8qciqjpbk-icecat-60.6.1-guix1 
>
> If you look there at the tab "inputs", you will see that it evaluated
> from commit
>
> fd9fe868f818b159b58c6a9811cc68e752a0af48
>
> Looking at the git history, you will find out that this is from:
>
> Committer: Tobias Geerinckx-Rice <me@tobias.gr>  2019-03-25 12:59:39
>
> Your "guix pull" should be on this commit or later (but not too late,
> such that firefox-affecting changes don't get in of cause :-)). 
> You can directly try to use:
>
> guix pull --commit="fd9fe868f818b159b58c6a9811cc68e752a0af48"
>
> and then try again updating icecat like you did above.
I did this but it still didn't work and asked me to compile
however I did what Mark Weaver suggested, I changed my TMPDIR
environment variable and I was able to compile without problems \o/ now
my guix is up to date



> Are you on x86_64? Although when I look at i686 and arm, the hashes are
> totally different (we would have a problem with substitutes there
> anyway).
>
I am on x86_64 yes
this is my guix describe


 $ guix describe
guile: warning: failed to install locale
hint: Consider installing the `glibc-utf8-locales' or `glibc-locales' package and defining `GUIX_LOCPATH', along these
lines:

     guix package -i glibc-utf8-locales
     export GUIX_LOCPATH="$HOME/.guix-profile/lib/locale"

See the "Application Setup" section in the manual, for more info.


Generation 11	Mar 29 2019 11:54:26	(current)
  guix 32e0f90
    repository URL: https://git.savannah.gnu.org/git/guix.git
    branch: master
    commit: 32e0f906d775679d8b4e770d267d29cb77724360

bug#35012: Can't install icecat

[Mark H Weaver]

Hi Björn,

Björn Höfling <bjoern.hoefling@bjoernhoefling.de> writes:

> On Thu, 28 Mar 2019 23:11:22 -0600
> Alberto EFG <albertoefg@posteo.mx> wrote:
>
>> H, Mark and everyone who has been helpful 
>> 
>> > As other people have mentioned, this substitute should now be
>> > available,  
>> I keep getting this, I don't know if I am doing something wrong:
>> 
>> $ guix package -n -u .
>> --substitute-urls="https://mirror.hydra.gnu.org
>> https://ci.guix.info" [...] substitute: updating substitutes from
>> 'https://mirror.hydra.gnu.org'... 100.0% substitute: updating
>> substitutes from 'https://ci.guix.info'... 100.0% The following
>> derivations would be
>> built: /gnu/store/5xs5i2pkf369g2hd26f2sy26r40bzjm6-icecat-60.6.1-guix1.drv /gnu/store/4f40wrx514rbly302ha0gvpn5b53d91f-rust-1.24.1.drv
>>    /gnu/store/yf5fcqp3v4jf038g454h2zqm54x6ijj6-profile.drv
>> 938.6 MB would be downloaded:
>
> I see that your hash of icecat differs from mine:
>
> $ guix package -n -i icecat
> --substitute-urls="https://mirror.hydra.gnu.org https://ci.guix.info"
> The following package would be upgraded: icecat	60.6.1-guix1 →
> 60.6.1-guix1	/gnu/store/bwhnxvfx4ng5fhc7xi6wijp8qciqjpbk-icecat-60.6.1-guix1

The hashes don't match because one is a store derivation (.drv) file and
the other is the output store item produced by that derivation.  These
store items are assigned different hashes.

You might expect that the hash of a derivation could be reused for the
outputs of that derivation, but it can't be done for a technical reason:
the hash in the file name of a store derivation is based on the entire
contents of that .drv file.  That .drv file also contains its output
file names.  See the problem?

In order to give these the same hash, you would need to create a
derivation file that contains its own hash, i.e. the result of hashing
its entire contents.  That would involve solving an equation of the
form: H = sha256(X ++ H ++ Y), which is presumably hard :)

     Regards,
       Mark

bug#35012: Can't install icecat

[Tobias Geerinckx-Rice via Bug reports for GNU Guix]

[-- Attachment #1: Type: text/plain, Size: 94 bytes --]

Closing, as this was very likely due to running out of /tmp[fs] 
space.

Kind regards,

T G-R

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 247 bytes --]