Re: Removing corrupted items from a mirror

[ludo@gnu.org (Ludovic Courtès)]

Hi!

Leo Famulari <leo@famulari.name> skribis:

> My question is: How can I remove this corrupted file from the mirror? I
> don't know how to figure the name of the file to remove.

So first, note that there’s a narinfo cache in
/var/guix/substitute/cache.  First step is to rm -rf that.

Next, you can remove all the narinfo cache of your nginx instance.  If
that’s unreasonable, you can find the file corresponding to the faulty
narinfo by grepping it in the nginx cache.  So in your case:

> Found valid signature for /gnu/store/vsrcn35rjpgrrak1w31m7yhc12zz3hd1-coreutils-8.24
> From https://my-mirror/nar/vsrcn35rjpgrrak1w31m7yhc12zz3hd1-coreutils-8.24

… you would:

  cd /var/cache/nginx/narinfo
  grep -r vsrcn35rjpgrrak1w31m7yhc12zz3hd1 .
  rm the-file-that-matches

Not very convenient!

I realize I’m answering a slightly different question, because your
problem is:

> @ substituter-failed /gnu/store/vsrcn35rjpgrrak1w31m7yhc12zz3hd1-coreutils-8.24 0 hash mismatch in downloaded path `/gnu/store/vsrcn35rjpgrrak1w31m7yhc12zz3hd1-coreutils-8.24': expected a91641b020926f42d7dcbe6e9d82be3b5d39f401b7328ea6d0cd79f7893aabb0, got fc8cf24465d8a97fef7cc41dfccddca158b6258ad0e0929e002552d258dec7da

That is, you got a binary whose content do not match what the narinfo
advertises.

That can happen with non-deterministic build processes, which I think is
the case with Coreutils.  Namely, we can have this scenario:

  1. nginx caches the narinfo for coreutils, which contains the hash of
     a first build;

  2. first build is removed from the machine that nginx mirrors;

  3. a second build is performed on that machine, leading different
     contents (different hash);

  4. Guix client requests coreutils binary, gets narinfo with old hash
     and contents with new hash, and errors out.

I’ve never seen this happen on mirror.hydra.gnu.org, but anyway, our
best protection is reproducible builds.

In the meantime, removing the old narinfo from nginx’s cache would help.

HTH!

Ludo’.