unofficial mirror of guix-devel@gnu.org 
 help / color / mirror / code / Atom feed
* Removing corrupted items from a mirror
@ 2016-07-24  0:26 Leo Famulari
  2016-07-24  7:58 ` Efraim Flashner
  2016-07-24 21:18 ` Ludovic Courtès
  0 siblings, 2 replies; 3+ messages in thread
From: Leo Famulari @ 2016-07-24  0:26 UTC (permalink / raw)
  To: guix-devel

While trying to build the trash-cli patch on top of the master branch
(commit 2a69d4df7d7), I got the following hash mismatch while
downloading a coreutils substitute from my local mirror. When I used
mirror.hydra.gnu.org instead, the correct data was downloaded.

My question is: How can I remove this corrupted file from the mirror? I
don't know how to figure the name of the file to remove.

$ ./pre-inst-env guix build trash-cli
substitute: warning: failed to install locale: Invalid argument
@ substituter-started /gnu/store/vsrcn35rjpgrrak1w31m7yhc12zz3hd1-coreutils-8.24 /gnu/store/hb25mzl9a2qjywzx41bfqgi9znc29h32-guix-0.10.0-0.e901/libexec/guix/substitute
warning: failed to install locale: Invalid argument

Found valid signature for /gnu/store/vsrcn35rjpgrrak1w31m7yhc12zz3hd1-coreutils-8.24
From https://my-mirror/nar/vsrcn35rjpgrrak1w31m7yhc12zz3hd1-coreutils-8.24
Downloading vsrcn3...-coreutils-8.24 (13.8MiB installed)...
 coreutils-8.24                                                                                                                                                        3.8MiB/s 00:01 | 3.5MiB transferred
@ substituter-failed /gnu/store/vsrcn35rjpgrrak1w31m7yhc12zz3hd1-coreutils-8.24 0 hash mismatch in downloaded path `/gnu/store/vsrcn35rjpgrrak1w31m7yhc12zz3hd1-coreutils-8.24': expected a91641b020926f42d7dcbe6e9d82be3b5d39f401b7328ea6d0cd79f7893aabb0, got fc8cf24465d8a97fef7cc41dfccddca158b6258ad0e0929e002552d258dec7da
@ substituter-started /gnu/store/dgwa731j19lj41cds023sspcg9lqgh4j-module-import /gnu/store/hb25mzl9a2qjywzx41bfqgi9znc29h32-guix-0.10.0-0.e901/libexec/guix/substitute
killing process 28390
guix build: error: build failed: some substitutes for the outputs of derivation `/gnu/store/yk4nv2zz40f9k7hr6c9v04rn3pb5l00s-coreutils-8.24.drv' failed (usually happens due to networking issues); try `--fallback' to build derivation from source 

^ permalink raw reply	[flat|nested] 3+ messages in thread

* Re: Removing corrupted items from a mirror
  2016-07-24  0:26 Removing corrupted items from a mirror Leo Famulari
@ 2016-07-24  7:58 ` Efraim Flashner
  2016-07-24 21:18 ` Ludovic Courtès
  1 sibling, 0 replies; 3+ messages in thread
From: Efraim Flashner @ 2016-07-24  7:58 UTC (permalink / raw)
  To: Leo Famulari; +Cc: guix-devel

[-- Attachment #1: Type: text/plain, Size: 2426 bytes --]

On Sat, Jul 23, 2016 at 08:26:20PM -0400, Leo Famulari wrote:
> While trying to build the trash-cli patch on top of the master branch
> (commit 2a69d4df7d7), I got the following hash mismatch while
> downloading a coreutils substitute from my local mirror. When I used
> mirror.hydra.gnu.org instead, the correct data was downloaded.
> 
> My question is: How can I remove this corrupted file from the mirror? I
> don't know how to figure the name of the file to remove.
> 
> $ ./pre-inst-env guix build trash-cli
> substitute: warning: failed to install locale: Invalid argument
> @ substituter-started /gnu/store/vsrcn35rjpgrrak1w31m7yhc12zz3hd1-coreutils-8.24 /gnu/store/hb25mzl9a2qjywzx41bfqgi9znc29h32-guix-0.10.0-0.e901/libexec/guix/substitute
> warning: failed to install locale: Invalid argument
> 
> Found valid signature for /gnu/store/vsrcn35rjpgrrak1w31m7yhc12zz3hd1-coreutils-8.24
> From https://my-mirror/nar/vsrcn35rjpgrrak1w31m7yhc12zz3hd1-coreutils-8.24
> Downloading vsrcn3...-coreutils-8.24 (13.8MiB installed)...
>  coreutils-8.24                                                                                                                                                        3.8MiB/s 00:01 | 3.5MiB transferred
> @ substituter-failed /gnu/store/vsrcn35rjpgrrak1w31m7yhc12zz3hd1-coreutils-8.24 0 hash mismatch in downloaded path `/gnu/store/vsrcn35rjpgrrak1w31m7yhc12zz3hd1-coreutils-8.24': expected a91641b020926f42d7dcbe6e9d82be3b5d39f401b7328ea6d0cd79f7893aabb0, got fc8cf24465d8a97fef7cc41dfccddca158b6258ad0e0929e002552d258dec7da
> @ substituter-started /gnu/store/dgwa731j19lj41cds023sspcg9lqgh4j-module-import /gnu/store/hb25mzl9a2qjywzx41bfqgi9znc29h32-guix-0.10.0-0.e901/libexec/guix/substitute
> killing process 28390
> guix build: error: build failed: some substitutes for the outputs of derivation `/gnu/store/yk4nv2zz40f9k7hr6c9v04rn3pb5l00s-coreutils-8.24.drv' failed (usually happens due to networking issues); try `--fallback' to build derivation from source 
> 

I was trying to build something earlier, got bounced between two
mismatched hashes so neither would perminantly fail. Had to kill it and
build both with --no-subsitutes to continue

-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

^ permalink raw reply	[flat|nested] 3+ messages in thread

* Re: Removing corrupted items from a mirror
  2016-07-24  0:26 Removing corrupted items from a mirror Leo Famulari
  2016-07-24  7:58 ` Efraim Flashner
@ 2016-07-24 21:18 ` Ludovic Courtès
  1 sibling, 0 replies; 3+ messages in thread
From: Ludovic Courtès @ 2016-07-24 21:18 UTC (permalink / raw)
  To: Leo Famulari; +Cc: guix-devel

Hi!

Leo Famulari <leo@famulari.name> skribis:

> My question is: How can I remove this corrupted file from the mirror? I
> don't know how to figure the name of the file to remove.

So first, note that there’s a narinfo cache in
/var/guix/substitute/cache.  First step is to rm -rf that.

Next, you can remove all the narinfo cache of your nginx instance.  If
that’s unreasonable, you can find the file corresponding to the faulty
narinfo by grepping it in the nginx cache.  So in your case:

> Found valid signature for /gnu/store/vsrcn35rjpgrrak1w31m7yhc12zz3hd1-coreutils-8.24
> From https://my-mirror/nar/vsrcn35rjpgrrak1w31m7yhc12zz3hd1-coreutils-8.24

… you would:

  cd /var/cache/nginx/narinfo
  grep -r vsrcn35rjpgrrak1w31m7yhc12zz3hd1 .
  rm the-file-that-matches

Not very convenient!

I realize I’m answering a slightly different question, because your
problem is:

> @ substituter-failed /gnu/store/vsrcn35rjpgrrak1w31m7yhc12zz3hd1-coreutils-8.24 0 hash mismatch in downloaded path `/gnu/store/vsrcn35rjpgrrak1w31m7yhc12zz3hd1-coreutils-8.24': expected a91641b020926f42d7dcbe6e9d82be3b5d39f401b7328ea6d0cd79f7893aabb0, got fc8cf24465d8a97fef7cc41dfccddca158b6258ad0e0929e002552d258dec7da

That is, you got a binary whose content do not match what the narinfo
advertises.

That can happen with non-deterministic build processes, which I think is
the case with Coreutils.  Namely, we can have this scenario:

  1. nginx caches the narinfo for coreutils, which contains the hash of
     a first build;

  2. first build is removed from the machine that nginx mirrors;

  3. a second build is performed on that machine, leading different
     contents (different hash);

  4. Guix client requests coreutils binary, gets narinfo with old hash
     and contents with new hash, and errors out.

I’ve never seen this happen on mirror.hydra.gnu.org, but anyway, our
best protection is reproducible builds.

In the meantime, removing the old narinfo from nginx’s cache would help.

HTH!

Ludo’.

^ permalink raw reply	[flat|nested] 3+ messages in thread

end of thread, other threads:[~2016-07-24 21:18 UTC | newest]

Thread overview: 3+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2016-07-24  0:26 Removing corrupted items from a mirror Leo Famulari
2016-07-24  7:58 ` Efraim Flashner
2016-07-24 21:18 ` Ludovic Courtès

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/guix.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).