Empty file in hydra store?

Empty file in hydra store?

[John Darrington]

[-- Attachment #1: Type: text/plain, Size: 1518 bytes --]

I got this error (twice) when attempting "guix system init":

Downloading https://mirror.hydra.gnu.org/nar/ciqw5z470c8ihl1kfswj1j3ix6hs092d-module-import (1KiB installed)...
^[[K module-import                                      0B/s 00:00 | 0B transferred
bzip2: Compressed file ends unexpectedly;
perhaps it is corrupted?  *Possible* reason follows.
bzip2: Inappropriate ioctl for device
Input file = (stdin), output file = (stdout)
It is possible that the compressed file(s) have become corrupted.
You can use the -tvv option to test integrity of such files.
You can use the `bzip2recover' program to attempt to recover
data from undamaged sections of corrupted files.
guix substitute: error: corrupt input while restoring '/gnu/store/ciqw5z470c8ihl1kfswj1j3ix6hs092d-module-import' from #{read pipe}#
killing process 1013
guix system: error: build failed:
some substitutes for the outputs of derivation `/gnu/store/9h4nc2fdy2dnklc7rnf61d7993dzsnks-module-import.drv' failed
(usually happens due to networking issues); try `--fallback' to build derivation from source


and if I try to manually download https://mirror.hydra.gnu.org/nar/ciqw5z470c8ihl1kfswj1j3ix6hs092d-module-import

I get a zero length file. :(

Can it be that hydra is losing files?

J'

-- 
Avoid eavesdropping.  Send strong encrypted email.
PGP Public key ID: 1024D/2DE827B3 
fingerprint = 8797 A26D 0854 2EAB 0285  A290 8A67 719C 2DE8 27B3
See http://sks-keyservers.net or any PGP keyserver for public key.


[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: Empty file in hydra store?

[Jan Synáček]

On Tue, Jan 3, 2017 at 9:17 AM, John Darrington
<john@darrington.wattle.id.au> wrote:
> I got this error (twice) when attempting "guix system init":
>
> Downloading https://mirror.hydra.gnu.org/nar/ciqw5z470c8ihl1kfswj1j3ix6hs092d-module-import (1KiB installed)...
> ^[[K module-import                                      0B/s 00:00 | 0B transferred
> bzip2: Compressed file ends unexpectedly;
> perhaps it is corrupted?  *Possible* reason follows.
> bzip2: Inappropriate ioctl for device
> Input file = (stdin), output file = (stdout)
> It is possible that the compressed file(s) have become corrupted.
> You can use the -tvv option to test integrity of such files.
> You can use the `bzip2recover' program to attempt to recover
> data from undamaged sections of corrupted files.
> guix substitute: error: corrupt input while restoring '/gnu/store/ciqw5z470c8ihl1kfswj1j3ix6hs092d-module-import' from #{read pipe}#
> killing process 1013
> guix system: error: build failed:
> some substitutes for the outputs of derivation `/gnu/store/9h4nc2fdy2dnklc7rnf61d7993dzsnks-module-import.drv' failed
> (usually happens due to networking issues); try `--fallback' to build derivation from source

I've run into the same issue when trying to install GuixSD.

-- 
Jan Synáček

Zero-length files cached by nginx

[Ludovic Courtès]

John Darrington <john@darrington.wattle.id.au> skribis:

> and if I try to manually download https://mirror.hydra.gnu.org/nar/ciqw5z470c8ihl1kfswj1j3ix6hs092d-module-import
>
> I get a zero length file. :(

Indeed:

--8<---------------cut here---------------start------------->8---
$ wget -q -O - https://mirror.hydra.gnu.org/nar/ciqw5z470c8ihl1kfswj1j3ix6hs092d-module-import |wc -c
0
$ wget -q -O - https://hydra.gnu.org/nar/ciqw5z470c8ihl1kfswj1j3ix6hs092d-module-import |wc -c
0
--8<---------------cut here---------------end--------------->8---

So the problem is that nginx on hydra.gnu.org cached a zero-length file,
and then nginx on mirror.hydra.gnu.org cached that too.

On hydra.gnu.org, we have:

--8<---------------cut here---------------start------------->8---
/var/cache/nginx/nar# cat 09/87d4a93959df3b0c82baed615b578109
X��������:kX`��
KEY: http://127.0.0.1:3000/nar/ciqw5z470c8ihl1kfswj1j3ix6hs092d-module-import
HTTP/1.1 200 OK
Content-Type: application/x-nix-archive
Set-Cookie: hydra_session=437ad1277e047114d0a49212805fd1867c0cf914; path=/; expires=Tue, 10-Jan-2017 05:43:20 GMT; HttpOnly
Transfer-Encoding: chunked
Date: Tue, 03 Jan 2017 05:43:43 GMT
Connection: close

--8<---------------cut here---------------end--------------->8---

Hydra (the software) produces HTTP chunked-encoded nars, as can be seen
above.

Rereading <https://tools.ietf.org/html/rfc2616#section-3.6.1>, it seems
that HTTP clients can detect truncated chunks, but they cannot detect
missing chunks since the total number of chunks is unknown and the
trailer is optional.

Presumably, what happened is that hydra.gnu.org was loaded and Hydra
dropped the connection, and then nginx cached what it got (i.e.,
nothing.)

Seems like the only way to fix it is by using ‘Content-Length’.  At the
very least, we should do that in ‘guix publish’.

For now, I’ve removed the faulty cached item from the nginx cache on
both machines.

Thanks,
Ludo’.