downloading a tarball

downloading a tarball

[Catonano]

[-- Attachment #1: Type: text/plain, Size: 1117 bytes --]

This
https://sourceforge.net/projects/libxls/files/

I attempted to build the package several times but every time the hash
results to be wrong.

I correct it and then it's wrong again

For example, I get

@ build-failed
/gnu/store/dcaqrf007jxyi0jzlsakr3j7faxm122f-libxsl-1.4.0.tar.gz.drv - 1
output path
`/gnu/store/v6i85v3myb09nbsacq9ghx6yd0spcr67-libxsl-1.4.0.tar.gz' should
have sha256 hash `1574bcyagix5fkbs0yi2npi59y1zck23y2aia52vdv6ra3i5raid',
instead has `1zmsb0w6qh4vx7n7r3yijc5p4fwljyk5apzi1hwmrr5rkawmqmks'

so I change it to
1zmsb0w6qh4vx7n7r3yijc5p4fwljyk5apzi1hwmrr5rkawmqmks

and then I get

@ build-failed
/gnu/store/vsl7yz0nyklv0705jb5py015jkz3r6dg-libxsl-1.4.0.tar.gz.drv - 1
output path
`/gnu/store/wjyja461cr7kvvryp6v21q3iagf5rd8m-libxsl-1.4.0.tar.gz' should
have sha256 hash `1zmsb0w6qh4vx7n7r3yijc5p4fwljyk5apzi1hwmrr5rkawmqmks',
instead has `0fyd8h4i46qw3xii3pfa12k9k9ndmj59b552pmkpwcq7psyrbf32'

So now it's not 1zmsb0w6qh4vx7n7r3yijc5p4fwljyk5apzi1hwmrr5rkawmqmks
anymore. It's
0fyd8h4i46qw3xii3pfa12k9k9ndmj59b552pmkpwcq7psyrbf32

I'm doing this with a master checkout.

What am I missing ?

[-- Attachment #2: Type: text/html, Size: 1402 bytes --]

Re: downloading a tarball

[Ben Woodcroft]

Hi,

On 24/03/17 22:06, Catonano wrote:
> This
> https://sourceforge.net/projects/libxls/files/
>
> I attempted to build the package several times but every time the hash 
> results to be wrong.
>
> I correct it and then it's wrong again
>
> For example, I get
>
> @ build-failed 
> /gnu/store/dcaqrf007jxyi0jzlsakr3j7faxm122f-libxsl-1.4.0.tar.gz.drv - 
> 1 output path 
> `/gnu/store/v6i85v3myb09nbsacq9ghx6yd0spcr67-libxsl-1.4.0.tar.gz' 
> should have sha256 hash 
> `1574bcyagix5fkbs0yi2npi59y1zck23y2aia52vdv6ra3i5raid', instead has 
> `1zmsb0w6qh4vx7n7r3yijc5p4fwljyk5apzi1hwmrr5rkawmqmks'

I think the mistake is that you are not taking the hash of the 
downloaded file, that is on the previous line. For instance from this 
example

output path 
`/gnu/store/7mfyynbzzi15265z92bdb00j7lxfa70y-libxls-1.4.0.zip' should 
have sha256 hash `0py8hsspvwjlckg2xi7jcpj0frrp2qbmsy9x55fx0knnwbhdg5d2', 
instead has `1g8ds7wbhsa4hdcn77xc2c0l3vvz5bx2hx9ng9c9n7aii92ymfnk'
@ build-failed 
/gnu/store/cv5nmjcwja91151zmxh9g956if0bl2xl-libxls-1.4.0.zip.drv - 1 
output path 
`/gnu/store/7mfyynbzzi15265z92bdb00j7lxfa70y-libxls-1.4.0.zip' should 
have sha256 hash `0py8hsspvwjlckg2xi7jcpj0frrp2qbmsy9x55fx0knnwbhdg5d2', 
instead has `1g8ds7wbhsa4hdcn77xc2c0l3vvz5bx2hx9ng9c9n7aii92ymfnk'

The correct hash is 0py8hsspvwjlckg2xi7jcpj0frrp2qbmsy9x55fx0knnwbhdg5d2.

HTH - speaking from experience making the same mistake here..
ben

Re: downloading a tarball

[Ricardo Wurmus]

Catonano <catonano@gmail.com> writes:

> This
> https://sourceforge.net/projects/libxls/files/
>
> I attempted to build the package several times but every time the hash
> results to be wrong.
>
> I correct it and then it's wrong again
>
> For example, I get
>
> @ build-failed
> /gnu/store/dcaqrf007jxyi0jzlsakr3j7faxm122f-libxsl-1.4.0.tar.gz.drv - 1
> output path
> `/gnu/store/v6i85v3myb09nbsacq9ghx6yd0spcr67-libxsl-1.4.0.tar.gz' should
> have sha256 hash `1574bcyagix5fkbs0yi2npi59y1zck23y2aia52vdv6ra3i5raid',
> instead has `1zmsb0w6qh4vx7n7r3yijc5p4fwljyk5apzi1hwmrr5rkawmqmks'
>
> so I change it to
> 1zmsb0w6qh4vx7n7r3yijc5p4fwljyk5apzi1hwmrr5rkawmqmks
>
> and then I get
>
> @ build-failed
> /gnu/store/vsl7yz0nyklv0705jb5py015jkz3r6dg-libxsl-1.4.0.tar.gz.drv - 1
> output path
> `/gnu/store/wjyja461cr7kvvryp6v21q3iagf5rd8m-libxsl-1.4.0.tar.gz' should
> have sha256 hash `1zmsb0w6qh4vx7n7r3yijc5p4fwljyk5apzi1hwmrr5rkawmqmks',
> instead has `0fyd8h4i46qw3xii3pfa12k9k9ndmj59b552pmkpwcq7psyrbf32'
>
> So now it's not 1zmsb0w6qh4vx7n7r3yijc5p4fwljyk5apzi1hwmrr5rkawmqmks
> anymore. It's
> 0fyd8h4i46qw3xii3pfa12k9k9ndmj59b552pmkpwcq7psyrbf32
>
> I'm doing this with a master checkout.
>
> What am I missing ?

Have you confirmed that the file you get is actually a tarball?
Sometimes you might get an HTML page and Guix reports the hash for that.

In general it is better to download the tarball manually and then use
“guix hash” on the result after confirming that the tarball is in fact
okay (e.g. by validating signatures or inspecting it).

--
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net

Re: downloading a tarball

[Tobias Geerinckx-Rice]

[-- Attachment #1.1: Type: text/plain, Size: 1180 bytes --]

Ben (can I trust you now?),

On 24/03/17 13:26, Ben Woodcroft wrote:
> The correct hash is 0py8hsspvwjlckg2xi7jcpj0frrp2qbmsy9x55fx0knnwbhdg5d2.

I'm afraid it's the other way round. ;-)

> `/gnu/store/7mfyynbzzi15265z92bdb00j7lxfa70y-libxls-1.4.0.zip' should
> have sha256 hash `0py8hsspvwjlckg2xi7jcpj0frrp2qbmsy9x55fx0knnwbhdg5d2',
> instead has `1g8ds7wbhsa4hdcn77xc2c0l3vvz5bx2hx9ng9c9n7aii92ymfnk'

The first hash is what's expected, the second is what was actually
received. What's ‘correct’ is of course not always clear.

‘guix hash’ on my manually downloaded copy returns
1g8ds7wbhsa4hdcn77xc2c0l3vvz5bx2hx9ng9c9n7aii92ymfnk.
I did no further checking.

> HTH - speaking from experience making the same mistake here..

I suspect most people have. The message isn't as clear as it could be.

It doesn't help that IIRC Nix and Guix differ(ed) in which hash they
place first. So much fun when dual-'ixing.

I considered changing the wording of this a long time ago to say
something more like ‘expecting ..., got ...’. Like most people, I just
got used to it. Perhaps I should have sent that patch anyway.

Kind regards,

T G-R


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 476 bytes --]

Re: downloading a tarball

[Catonano]

[-- Attachment #1: Type: text/plain, Size: 2375 bytes --]

2017-03-24 13:50 GMT+01:00 Ricardo Wurmus <rekado@elephly.net>:

>
> Catonano <catonano@gmail.com> writes:
>
> > This
> > https://sourceforge.net/projects/libxls/files/
> >
> > I attempted to build the package several times but every time the hash
> > results to be wrong.
> >
> > I correct it and then it's wrong again
> >
> > For example, I get
> >
> > @ build-failed
> > /gnu/store/dcaqrf007jxyi0jzlsakr3j7faxm122f-libxsl-1.4.0.tar.gz.drv - 1
> > output path
> > `/gnu/store/v6i85v3myb09nbsacq9ghx6yd0spcr67-libxsl-1.4.0.tar.gz' should
> > have sha256 hash `1574bcyagix5fkbs0yi2npi59y1zck23y2aia52vdv6ra3i5raid',
> > instead has `1zmsb0w6qh4vx7n7r3yijc5p4fwljyk5apzi1hwmrr5rkawmqmks'
> >
> > so I change it to
> > 1zmsb0w6qh4vx7n7r3yijc5p4fwljyk5apzi1hwmrr5rkawmqmks
> >
> > and then I get
> >
> > @ build-failed
> > /gnu/store/vsl7yz0nyklv0705jb5py015jkz3r6dg-libxsl-1.4.0.tar.gz.drv - 1
> > output path
> > `/gnu/store/wjyja461cr7kvvryp6v21q3iagf5rd8m-libxsl-1.4.0.tar.gz' should
> > have sha256 hash `1zmsb0w6qh4vx7n7r3yijc5p4fwljyk5apzi1hwmrr5rkawmqmks',
> > instead has `0fyd8h4i46qw3xii3pfa12k9k9ndmj59b552pmkpwcq7psyrbf32'
> >
> > So now it's not 1zmsb0w6qh4vx7n7r3yijc5p4fwljyk5apzi1hwmrr5rkawmqmks
> > anymore. It's
> > 0fyd8h4i46qw3xii3pfa12k9k9ndmj59b552pmkpwcq7psyrbf32
> >
> > I'm doing this with a master checkout.
> >
> > What am I missing ?
>
> Have you confirmed that the file you get is actually a tarball?
> Sometimes you might get an HTML page and Guix reports the hash for that.
>

You nailed it

I catted

/gnu/store/w1wf5d44awk0almrdrbhs8442cnzmw2b-libxsl-1.4.0.zip

which is the downloaded tarball and it contains html !

How can this be ? I'm using

(uri (string-append "mirror://sourceforge/libxsl/libxsl-"
                    version ".zip"))

The url used by icecat is
https://netassist.dl.sourceforge.net/project/libxls/libxls-1.4.0.zip

and the result is a sound tarball

How does it happen that the mirror based url leads to an html page ?


>
> In general it is better to download the tarball manually and then use
> “guix hash” on the result after confirming that the tarball is in fact
> okay (e.g. by validating signatures or inspecting it).
>

I did ! This was my first step, I inspected it ! And I hashed it

But then this whirl of hashes began

Thanks

[-- Attachment #2: Type: text/html, Size: 3702 bytes --]

Re: downloading a tarball

[Hartmut Goebel]

Am 24.03.2017 um 14:14 schrieb Tobias Geerinckx-Rice:
>> > HTH - speaking from experience making the same mistake here..
> I suspect most people have. The message isn't as clear as it could be.

Either way round the message is hard to read: squezzed between many
other message, no line-break and arguable wording. IMHO this should be
improved to empower more people.

-- 
+++hartmut

| Hartmut Goebel            |                       |
| hartmut@goebel-consult.de | www.goebel-consult.de |

Re: downloading a tarball

[Ludovic Courtès]

[-- Attachment #1: Type: text/plain, Size: 2127 bytes --]

Hartmut Goebel <hartmut@goebel-consult.de> skribis:

> Am 24.03.2017 um 14:14 schrieb Tobias Geerinckx-Rice:
>>> > HTH - speaking from experience making the same mistake here..
>> I suspect most people have. The message isn't as clear as it could be.
>
> Either way round the message is hard to read: squezzed between many
> other message, no line-break and arguable wording. IMHO this should be
> improved to empower more people.

Yeah I agree.

What about this (patch below)?

--8<---------------cut here---------------start------------->8---
@ build-started /gnu/store/0rn80kgcbc90hfl2vl54adci7675fwb7-idutils-4.6.tar.xz.drv - x86_64-linux /var/log/guix/drvs/0r//n80kgcbc90hfl2vl54adci7675fwb7-idutils-4.6.tar.xz.drv.bz2

Starting download of /gnu/store/si0rm701sqmi3w69k2b2yzar4p99j66k-idutils-4.6.tar.xz
From http://ftpmirror.gnu.org/idutils/idutils-4.6.tar.xz...
following redirection to `http://mirror.ibcp.fr/pub/gnu/idutils/idutils-4.6.tar.xz'...
 idutils-4.6.tar.xz  978KiB         743KiB/s 00:01 [####################] 100.0%
sha256 hash mismatch for output path `/gnu/store/si0rm701sqmi3w69k2b2yzar4p99j66k-idutils-4.6.tar.xz'
  expected: 1hmai3422iaqnp34kkzxdnywl7n7pvlxp11vrw66ybxn9wxg90c2
  actual:   1hmai3422iaqnp34kkzxdnywl7n7pvlxp11vrw66ybxn9wxg90c1
@ build-failed /gnu/store/0rn80kgcbc90hfl2vl54adci7675fwb7-idutils-4.6.tar.xz.drv - 1 sha256 hash mismatch for output path `/gnu/store/si0rm701sqmi3w69k2b2yzar4p99j66k-idutils-4.6.tar.xz'
  expected: 1hmai3422iaqnp34kkzxdnywl7n7pvlxp11vrw66ybxn9wxg90c2
  actual:   1hmai3422iaqnp34kkzxdnywl7n7pvlxp11vrw66ybxn9wxg90c1
cannot build derivation `/gnu/store/kk172bqfx6ya7w62xm1vmr2c9iyq6cpy-idutils-4.6.tar.xz.drv': 1 dependencies couldn't be built
guix build: error: build failed: build of `/gnu/store/kk172bqfx6ya7w62xm1vmr2c9iyq6cpy-idutils-4.6.tar.xz.drv' failed
--8<---------------cut here---------------end--------------->8---

(Here ‘guix build’ repeats the error message because of
#:print-build-trace, which is #true in ‘guix build’.)

Probably not optimal but hopefully an improvement.

Thanks,
Ludo’.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Type: text/x-patch, Size: 1498 bytes --]

diff --git a/nix/libstore/build.cc b/nix/libstore/build.cc
index 9b7bb5391..a93095dd1 100644
--- a/nix/libstore/build.cc
+++ b/nix/libstore/build.cc
@@ -2449,8 +2449,11 @@ void DerivationGoal::registerOutputs()
             Hash h2 = recursive ? hashPath(ht, actualPath).first : hashFile(ht, actualPath);
             if (h != h2)
                 throw BuildError(
-                    format("output path `%1%' should have %2% hash `%3%', instead has `%4%'")
-                    % path % i->second.hashAlgo % printHash16or32(h) % printHash16or32(h2));
+                    format("%1% hash mismatch for output path `%2%'\n"
+			   "  expected: %3%\n"
+			   "  actual:   %4%")
+                    % i->second.hashAlgo % path
+		    % printHash16or32(h) % printHash16or32(h2));
         }
 
         /* Get rid of all weird permissions.  This also checks that
@@ -3096,7 +3099,9 @@ void SubstitutionGoal::finished()
             Hash expectedHash = parseHash16or32(hashType, string(expectedHashStr, n + 1));
             Hash actualHash = hashType == htSHA256 ? hash.first : hashPath(hashType, destPath).first;
             if (expectedHash != actualHash)
-                throw SubstError(format("hash mismatch in downloaded path `%1%': expected %2%, got %3%")
+                throw SubstError(format("hash mismatch in downloaded path `%1%'\n"
+					"  expected: %2%\n"
+					"  actual:   %3%")
                     % storePath % printHash(expectedHash) % printHash(actualHash));
         }
 

Re: downloading a tarball

[Catonano]

[-- Attachment #1: Type: text/plain, Size: 785 bytes --]

2017-03-24 15:33 GMT+01:00 Hartmut Goebel <hartmut@goebel-consult.de>:

> Am 24.03.2017 um 14:14 schrieb Tobias Geerinckx-Rice:
> >> > HTH - speaking from experience making the same mistake here..
> > I suspect most people have. The message isn't as clear as it could be.
>
> Either way round the message is hard to read: squezzed between many
> other message, no line-break and arguable wording. IMHO this should be
> improved to empower more people.
>

I keep being unable to download this file

Now it downloads it but it can't decompress it - no such file

Here is my definition
http://paste.lisp.org/display/342342#1

thanks for any hint


>
> --
> +++hartmut
>
> | Hartmut Goebel            |                       |
> | hartmut@goebel-consult.de | www.goebel-consult.de |
>
>
>

[-- Attachment #2: Type: text/html, Size: 1635 bytes --]

Re: downloading a tarball

[Ben Woodcroft]

[-- Attachment #1: Type: text/plain, Size: 862 bytes --]

Hi,


On 25/03/17 18:47, Catonano wrote:
> 2017-03-24 15:33 GMT+01:00 Hartmut Goebel <hartmut@goebel-consult.de 
> <mailto:hartmut@goebel-consult.de>>:
>
>     Am 24.03.2017 um 14:14 schrieb Tobias Geerinckx-Rice:
>     >> > HTH - speaking from experience making the same mistake here..
>     > I suspect most people have. The message isn't as clear as it
>     could be.
>
>     Either way round the message is hard to read: squezzed between many
>     other message, no line-break and arguable wording. IMHO this should be
>     improved to empower more people.
>
>
> I keep being unable to download this file
>
> Now it downloads it but it can't decompress it - no such file
>
> Here is my definition
> http://paste.lisp.org/display/342342#1

Looks like a zipbomb, it unpacked for me using "(method 
url-fetch/zipbomb)" rather than "(method url-fetch)".

ben

[-- Attachment #2: Type: text/html, Size: 2233 bytes --]

Re: downloading a tarball

[Hartmut Goebel]

Am 24.03.2017 um 23:10 schrieb Ludovic Courtès:
> What about this (patch below)?

This is definitively an enhancement :-) Thanks! (I did not know,
newlines are usable in log-messages. Good to know.)

-- 
+++hartmut

| Hartmut Goebel            |                       |
| hartmut@goebel-consult.de | www.goebel-consult.de |