mismatch of source tar ball hash

mismatch of source tar ball hash

[Marco van Hulten]

Hi—

When I try to install calcurse, the hash of the downloaded file is
different from the one specified in the package:


marco@graviton ~$ guix package --fallback -i calcurse
The following package will be installed:
   calcurse	4.3.0	/gnu/store/a77ap0vw0fnsz138paby8w55rlcd58zi-calcurse-4.3.0

The following derivations will be built:
   /gnu/store/r44sbjgn7gxwl3nxvlnq6946zc05xq0f-profile.drv
   /gnu/store/xzvmj17kgc5z97czlw5nrz7afb47cjyh-ca-certificate-bundle.drv
   /gnu/store/wazkm772ssqj81w9npbfi9531s41rh4w-glib-schemas.drv
   /gnu/store/kwqvdmcpqrqg1q2gk6zc83c50x8w8198-gtk-icon-themes.drv
   /gnu/store/fad05ars0k7lnis4pys6bc5n2mrwywqj-xdg-mime-database.drv
   /gnu/store/dkr1dg622dyaphw66zy2lrb925pkfz0y-xdg-desktop-database.drv
   /gnu/store/9zqzgm8gybh8a8w05g4151k7bk3wkv7x-fonts-dir.drv
   /gnu/store/4ccc9liswa27f56867fnsg4xl657irpg-info-dir.drv
   /gnu/store/35zb2h95ssn189i8y43z54b6jqmngwsp-gtk-im-modules.drv
   /gnu/store/6abn78c8wa5dkks4pnig5i2mijj0f3lf-manual-database.drv
0.3 MB will be downloaded:
   /gnu/store/a77ap0vw0fnsz138paby8w55rlcd58zi-calcurse-4.3.0
substitution of /gnu/store/a77ap0vw0fnsz138paby8w55rlcd58zi-calcurse-4.3.0 failed
building /gnu/store/ya1v2nv0mq8dzkcik64inlyxxk3skz3h-calcurse-4.3.0.tar.gz.drv...
downloading from http://calcurse.org/files/calcurse-4.3.0.tar.gz...
sha256 hash mismatch for /gnu/store/873m2xbqxndbhcdfrngpsj7cwflm48d0-calcurse-4.3.0.tar.gz:
  expected hash: 16jzg0nasnxdlz23i121x41pq5kbxmjzk52c5d863rg117fc7v1i
  actual hash:   11q0r4dbi8vca22x3q1ad07nr1gs4y17cgnplbjzmmz9r9x0h8m2
build of /gnu/store/ya1v2nv0mq8dzkcik64inlyxxk3skz3h-calcurse-4.3.0.tar.gz.drv failed
View build log at '/var/log/guix/drvs/ya/1v2nv0mq8dzkcik64inlyxxk3skz3h-calcurse-4.3.0.tar.gz.drv.bz2'.
\guix package: error: build failed: build of `/gnu/store/r44sbjgn7gxwl3nxvlnq6946zc05xq0f-profile.drv' failed


Should I contact the calcurse people to ask if the file could have been
changed?  Or is there a better hypothesis explaining this issue?

—Marco

Re: mismatch of source tar ball hash

[Marco van Hulten]

update—

Je 28 okt 12:23 skribis Marco:
> Hi—
> 
> When I try to install calcurse, the hash of the downloaded file is
> different from the one specified in the package:
> [...]

I happened to find the same tar ball that I downloaded on 5 April 2018.
As this is quite some time ago, my *guess* is that the issue is related
to Guix instead of calcurse.org.

—Marco

Re: mismatch of source tar ball hash

[Gábor Boskovits]

Hello Marco,

Marco van Hulten <marco@hulten.org> ezt írta (időpont: 2018. okt. 28.,
V, 15:50):
>
> update—
>
> Je 28 okt 12:23 skribis Marco:
> > Hi—
> >
> > When I try to install calcurse, the hash of the downloaded file is
> > different from the one specified in the package:
> > [...]
>
> I happened to find the same tar ball that I downloaded on 5 April 2018.
> As this is quite some time ago, my *guess* is that the issue is related
> to Guix instead of calcurse.org.
>
> —Marco
>

A few things you could check:

1. the tarball is the same as checked by diff or by diffoscope?
Diffoscope might reveal some things, such as metadata differences,
that can't be seen by diff alone.
2. can you have a look at the last modification of the hash value in
question, for example by using
git blame, or simply git log, as there tend to be good log entries, so
this should be easy to find.

Best regards,
g_bor

Re: mismatch of source tar ball hash

[Leo Famulari]

[-- Attachment #1: Type: text/plain, Size: 4411 bytes --]

On Sun, Oct 28, 2018 at 12:23:33PM +0100, Marco van Hulten wrote:
> substitution of /gnu/store/a77ap0vw0fnsz138paby8w55rlcd58zi-calcurse-4.3.0 failed
> building /gnu/store/ya1v2nv0mq8dzkcik64inlyxxk3skz3h-calcurse-4.3.0.tar.gz.drv...
> downloading from http://calcurse.org/files/calcurse-4.3.0.tar.gz...
> sha256 hash mismatch for /gnu/store/873m2xbqxndbhcdfrngpsj7cwflm48d0-calcurse-4.3.0.tar.gz:
>   expected hash: 16jzg0nasnxdlz23i121x41pq5kbxmjzk52c5d863rg117fc7v1i
>   actual hash:   11q0r4dbi8vca22x3q1ad07nr1gs4y17cgnplbjzmmz9r9x0h8m2
> build of /gnu/store/ya1v2nv0mq8dzkcik64inlyxxk3skz3h-calcurse-4.3.0.tar.gz.drv failed
> View build log at '/var/log/guix/drvs/ya/1v2nv0mq8dzkcik64inlyxxk3skz3h-calcurse-4.3.0.tar.gz.drv.bz2'.
> \guix package: error: build failed: build of `/gnu/store/r44sbjgn7gxwl3nxvlnq6946zc05xq0f-profile.drv' failed

Fixed by commit a2717e698619fed3204db978f954a1195e1d2b4b, which makes
Guix download the source code over HTTPS, since <calcurse.org> is no
longer available over HTTP.

Here is how I debugged it:

First, I tried downloading the source code, avoiding substitutes because
I still have the source code on my mirror:

------
$ guix build --source calcurse --no-substitutes
The following derivation will be built:         
   /gnu/store/ya1v2nv0mq8dzkcik64inlyxxk3skz3h-calcurse-4.3.0.tar.gz.drv
building /gnu/store/ya1v2nv0mq8dzkcik64inlyxxk3skz3h-calcurse-4.3.0.tar.gz.drv...

Starting download of /gnu/store/873m2xbqxndbhcdfrngpsj7cwflm48d0-calcurse-4.3.0.tar.gz
From http://calcurse.org/files/calcurse-4.3.0.tar.gz...
following redirection to `https://calcurse.org'...
downloading from http://calcurse.org/files/calcurse-4.3.0.tar.gz...
 calcurse-4.3.0.tar.gz  6KiB                                                                                                                          7.9MiB/s 00:00 [##################] 100.0%
sha256 hash mismatch for /gnu/store/873m2xbqxndbhcdfrngpsj7cwflm48d0-calcurse-4.3.0.tar.gz:
  expected hash: 16jzg0nasnxdlz23i121x41pq5kbxmjzk52c5d863rg117fc7v1i
  actual hash:   11q0r4dbi8vca22x3q1ad07nr1gs4y17cgnplbjzmmz9r9x0h8m2
hash mismatch for store item '/gnu/store/873m2xbqxndbhcdfrngpsj7cwflm48d0-calcurse-4.3.0.tar.gz'
build of /gnu/store/ya1v2nv0mq8dzkcik64inlyxxk3skz3h-calcurse-4.3.0.tar.gz.drv failed
View build log at '/var/log/guix/drvs/ya/1v2nv0mq8dzkcik64inlyxxk3skz3h-calcurse-4.3.0.tar.gz.drv.bz2'.
guix build: error: build failed: build of `/gnu/store/ya1v2nv0mq8dzkcik64inlyxxk3skz3h-calcurse-4.3.0.tar.gz.drv' failed
------

Then, I downloaded the file in question and examined it:

------
$ guix download http://calcurse.org/files/calcurse-4.3.0.tar.gz

Starting download of /tmp/guix-file.nEHs4A
From http://calcurse.org/files/calcurse-4.3.0.tar.gz...
following redirection to `https://calcurse.org'...
 ….3.0.tar.gz  6KiB                                                                                                                                  11.2MiB/s 00:00 [##################] 100.0%
/gnu/store/fr962qdmzziwjwngdm8a70pdkaai72mp-calcurse-4.3.0.tar.gz
11q0r4dbi8vca22x3q1ad07nr1gs4y17cgnplbjzmmz9r9x0h8m2
$ file /gnu/store/fr962qdmzziwjwngdm8a70pdkaai72mp-calcurse-4.3.0.tar.gz
/gnu/store/fr962qdmzziwjwngdm8a70pdkaai72mp-calcurse-4.3.0.tar.gz: HTML document, ASCII text
$ head --lines 10 /gnu/store/fr962qdmzziwjwngdm8a70pdkaai72mp-calcurse-4.3.0.tar.gz
<!DOCTYPE html>
<html lang="en">
  <head>
    <title>calcurse.org</title>

    <meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1" />
    <meta http-equiv="Content-Style-Type" content="text/css" />

    <script type="text/javascript" src="/js/jquery-1.7.2.min.js"></script>
    <script type="text/javascript" src="/js/lightbox.js"></script>
------

So, we see that we are actually getting an HTML file (their homepage),
rather than a tarball. Looking back to the original error message, we
can see some suspicious redirections, from HTTP to HTTPS and then back
to HTTP.

This was my first clue that we were not going to download what we
expected, because its rare to configure a webserver to downgrade from
HTTPS to HTTP in 2018.

From there, I made an educated guess that <calcurse.org> had enabled
HSTS, and that we simply needed to adjust our download URL.

[0]
https://www.gnu.org/software/guix/manual/en/html_node/Building-from-Git.html

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]