bug#33235: Bzip hash error

bug#33235: Bzip hash error

[brettg]

The sha256 has for bzip2 is mismatching on build.

bug#33235: Bzip hash error

[Leo Famulari]

[-- Attachment #1: Type: text/plain, Size: 1538 bytes --]

On Fri, Nov 02, 2018 at 03:05:16AM +0100, brettg@posteo.net wrote:
> The sha256 has for bzip2 is mismatching on build.

Can you give more details? For example, the version of Guix you are
using (`guix describe`), the command you ran, and the command's output?

It's working for me:

$ guix describe
Generation 43	Nov 01 2018 13:58:41	(current)
  guix 361dd8a
    repository URL: https://git.savannah.gnu.org/git/guix.git
    branch: master
    commit: 361dd8a55644be6ee81f3ccd9e55d2951a8693f8
$ guix build --no-grafts -S bzip2 --no-substitutes            
The following derivation will be built:                
   /gnu/store/bp0hyahbn3f60b954rfcz0vqlmc7lhzj-bzip2-1.0.6.tar.gz.drv
building /gnu/store/bp0hyahbn3f60b954rfcz0vqlmc7lhzj-bzip2-1.0.6.tar.gz.drv...

Starting download of /gnu/store/bjpalrv9f49d2k40p7ga0y6dwdys9w5j-bzip2-1.0.6.tar.gz
From https://web.archive.org/web/20180624184806/http://www.bzip.org/1.0.6/bzip2-1.0.6.tar.gz...
following redirection to `https://web.archive.org/web/20180624184835/http://www.bzip.org/1.0.6/bzip2-1.0.6.tar.gz'...
downloading from https://web.archive.org/web/20180624184806/http://www.bzip.org/1.0.6/bzip2-1.0.6.tar.gz...
 bzip2-1.0.6.tar.gz                                                                                                                                          129KiB/s 00:06 | 764KiB transferred
successfully built /gnu/store/bp0hyahbn3f60b954rfcz0vqlmc7lhzj-bzip2-1.0.6.tar.gz.drv
/gnu/store/bjpalrv9f49d2k40p7ga0y6dwdys9w5j-bzip2-1.0.6.tar.gz


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

bug#33235: Bzip hash error

[Leo Famulari]

[-- Attachment #1: Type: text/plain, Size: 845 bytes --]

On Fri, Nov 02, 2018 at 03:29:50AM +0100, brettg@posteo.net wrote:
> Hi Leo,
> 
> I’m coming from an installation using guix init, and am typing from mobile.
> I am using the berlin substitutes. I guess my option is to guix pull the
> live iso?

No, I recommend against doing `guix pull` before initializing. We test
that the installer works, but if you update Guix before initialization,
then you void the warranty ;)

The bzip2.org site is offline; the domain registration expired and the
site is gone:

https://git.savannah.gnu.org/cgit/guix.git/commit/?id=7598b678fad2112ea87a85acf755908a59dd4676

I recommend downloading the missing source code from the archives, using
`guix download`. For your reference, here is the full URL:

https://web.archive.org/web/20180624184835/http://www.bzip.org/1.0.6/bzip2-1.0.6.tar.gz

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

bug#33235: Bzip hash error

[brettg]

Hi Leo,

Are you perhaps mistaken? the URL for bzip2 is bzip.org not bzip2.org

Regardless, even when pulling from the archive I am still being given an 
incorrect hash.

On 02.11.2018 04:09, Leo Famulari wrote:
> On Fri, Nov 02, 2018 at 03:29:50AM +0100, brettg@posteo.net wrote:
>> Hi Leo,
>> 
>> I’m coming from an installation using guix init, and am typing from 
>> mobile.
>> I am using the berlin substitutes. I guess my option is to guix pull 
>> the
>> live iso?
> 
> No, I recommend against doing `guix pull` before initializing. We test
> that the installer works, but if you update Guix before initialization,
> then you void the warranty ;)
> 
> The bzip2.org site is offline; the domain registration expired and the
> site is gone:
> 
> https://git.savannah.gnu.org/cgit/guix.git/commit/?id=7598b678fad2112ea87a85acf755908a59dd4676
> 
> I recommend downloading the missing source code from the archives, 
> using
> `guix download`. For your reference, here is the full URL:
> 
> https://web.archive.org/web/20180624184835/http://www.bzip.org/1.0.6/bzip2-1.0.6.tar.gz

bug#33235: Bzip hash error

[Leo Famulari]

[-- Attachment #1: Type: text/plain, Size: 727 bytes --]

On Fri, Nov 02, 2018 at 04:33:10AM +0100, brettg@posteo.net wrote:
> Hi Leo,
> 
> Are you perhaps mistaken? the URL for bzip2 is bzip.org not bzip2.org

I mentioned the wrong "short" URLs in my message, but the full URL of
the source code on archive.org was correct.

bzip.org is offline and the source is available on archive.org.

> Regardless, even when pulling from the archive I am still being given an
> incorrect hash.

Can you take a photo of your screen and email it with your mobile?

If so, please show the command you run and the failing part of the
output.

Also, do 
`guix download https://web.archive.org/web/20180624184835/http://www.bzip.org/1.0.6/bzip2-1.0.6.tar.gz`
and show the result.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

bug#33235: Bzip hash error

[Björn Höfling]

[-- Attachment #1: Type: text/plain, Size: 2136 bytes --]

Hi brettg,

I tried with a fresh guixsd 0.15 install and can confirm your problem:

On Fri, 02 Nov 2018 04:33:10 +0100
brettg@posteo.net wrote:

> Hi Leo,
> 
> Are you perhaps mistaken? the URL for bzip2 is bzip.org not bzip2.org
> 
> Regardless, even when pulling from the archive I am still being given
> an incorrect hash.

$ guix build -S --no-substitutes bzip2

--> As you see it, there is a hash mismatch
That's because the URL expired and the new "archive.org/..." URL was
added in commit

7598b678fad2112ea87a85acf755908a59dd4676
Committer: Leo Famulari <leo@famulari.name>  2018-08-10 22:42:01

Which follows the 0.15.0 release.

If you then look into the log output and open that downloaded file:

less /gnu/store/...-bzip2-1.0.6.tar.gz

you see plain HTML. Of cause, that mismatches.

If there is an error while downloading (i.e. a 404 - not found) then
guix downloader would fallback and try to download the sources from
subistitute servers or even NixOS servers by hashsum. But as we get a
good HTTP return code here, it assumes everything is fine and saves to
the store, not trying anything else.

Now to download the sources from web.archive.org I needed these
steps:

# find out path of guix:
readlink `which guix`

# Then use that directory to authorize berlin:
guix archive --authorize
< /gnu/store/...-guix-0.15.0.../share/guix/berlin.guixsd.org.pub

#Install nss-certs for HTTPS access:
guix package -i nss-certs
# set environment variables, see 6.2.9 X.509 Certificates
export SSL_CERT_DIR="$HOME/.guix-profile/etc/ssl/certs"
export SSL_CERT_FILE="$HOME/.guix-profile/etc/ssl/certs/ca-certificates.crt"

# Now download --with-source
guix build -S
--with-source="https://web.archive.org/web/20180624184835/http://www.bzip.org/1.0.6/bzip2-1.0.6.tar.gz"

# This fails with a stacktrace "wrong type, expected struc", I can't
# copy it out of VM currently.
# 
# Anyway, it downloads the sources and the next:

guix build -S bzip2

# does succeed.

Sorry for the inconveniences with substitutes missing, hope this brings
you a step further?

Björn


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

bug#33235: Bzip hash error

[swedebugia]

Hi

On 2018-11-02 04:09, Leo Famulari wrote:
> No, I recommend against doing `guix pull` before initializing. We test
> that the installer works, but if you update Guix before initialization,
> then you void the warranty
I suggest we state this VERY clearly in the manual.

-- 
Cheers
Swedebugia

bug#33235: Bzip hash error

[brettg]

Hi bjorn.

Thank you your steps did help in getting it to build with the 
appropairiate hash sums.

However, how do I get the guix system init process to
Use that version of the build we just made? It is still
Wanting to use the incorrect substitute.


On 02.11.2018 12:07, Björn Höfling wrote:
> Hi brettg,
> 
> I tried with a fresh guixsd 0.15 install and can confirm your problem:
> 
> On Fri, 02 Nov 2018 04:33:10 +0100
> brettg@posteo.net wrote:
> 
>> Hi Leo,
>> 
>> Are you perhaps mistaken? the URL for bzip2 is bzip.org not bzip2.org
>> 
>> Regardless, even when pulling from the archive I am still being given
>> an incorrect hash.
> 
> $ guix build -S --no-substitutes bzip2
> 
> --> As you see it, there is a hash mismatch
> That's because the URL expired and the new "archive.org/..." URL was
> added in commit
> 
> 7598b678fad2112ea87a85acf755908a59dd4676
> Committer: Leo Famulari <leo@famulari.name>  2018-08-10 22:42:01
> 
> Which follows the 0.15.0 release.
> 
> If you then look into the log output and open that downloaded file:
> 
> less /gnu/store/...-bzip2-1.0.6.tar.gz
> 
> you see plain HTML. Of cause, that mismatches.
> 
> If there is an error while downloading (i.e. a 404 - not found) then
> guix downloader would fallback and try to download the sources from
> subistitute servers or even NixOS servers by hashsum. But as we get a
> good HTTP return code here, it assumes everything is fine and saves to
> the store, not trying anything else.
> 
> Now to download the sources from web.archive.org I needed these
> steps:
> 
> # find out path of guix:
> readlink `which guix`
> 
> # Then use that directory to authorize berlin:
> guix archive --authorize
> < /gnu/store/...-guix-0.15.0.../share/guix/berlin.guixsd.org.pub
> 
> #Install nss-certs for HTTPS access:
> guix package -i nss-certs
> # set environment variables, see 6.2.9 X.509 Certificates
> export SSL_CERT_DIR="$HOME/.guix-profile/etc/ssl/certs"
> export 
> SSL_CERT_FILE="$HOME/.guix-profile/etc/ssl/certs/ca-certificates.crt"
> 
> # Now download --with-source
> guix build -S
> --with-source="https://web.archive.org/web/20180624184835/http://www.bzip.org/1.0.6/bzip2-1.0.6.tar.gz"
> 
> # This fails with a stacktrace "wrong type, expected struc", I can't
> # copy it out of VM currently.
> #
> # Anyway, it downloads the sources and the next:
> 
> guix build -S bzip2
> 
> # does succeed.
> 
> Sorry for the inconveniences with substitutes missing, hope this brings
> you a step further?
> 
> Björn

bug#33235: Bzip hash error

[Björn Höfling]

[-- Attachment #1: Type: text/plain, Size: 904 bytes --]

Hi brettg,


On Fri, 02 Nov 2018 21:01:05 +0100
brettg@posteo.net wrote:

> Hi bjorn.
> 
> Thank you your steps did help in getting it to build with the 
> appropairiate hash sums.
> 
> However, how do I get the guix system init process to
> Use that version of the build we just made? It is still
> Wanting to use the incorrect substitute.

I don't understand this.

When it is already in the store it will be used by Guix (independently
of its source, substitutes or self-compiled).

But it could be that there are two different versions of it, i.e. with
two different store-hashes. For example, one of its dependencies is
different.

Can you post the output of "guix describe" and the log of "guix system
init"?

To get the logs, you can either copy it on USB stick or get it with
ssh/scp, if you started the ssh daemon as described in the installation
manual.

Björn

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

bug#33235: Bzip hash error

[swedebugia]

On 2018-11-02 14:40, swedebugia wrote:
> Hi
>
> On 2018-11-02 04:09, Leo Famulari wrote:
>> No, I recommend against doing `guix pull` before initializing. We test
>> that the installer works, but if you update Guix before initialization,
>> then you void the warranty
> I suggest we state this VERY clearly in the manual.

Actually guix very loudly warns the user that running guix system init 
before guix pull will downgrade guix.

I solved this by pulling the 0.15 commit and then init.

I suggest we tell binary install users to do this in the manual.

What do you think?

-- 
Cheers
Swedebugia