bug#19512: Failed substitution yields incomplete store item

bug#19512: Failed substitution yields incomplete store item

[David Thompson]

While working on 'guix publish', I managed to mess up the store.  :(

Here's the first substitution that failed due to an invalid eof marker
for some reason:

  dave@labrys ~$ sudo _NIX_OPTIONS="substitute-urls=http://192.168.1.157:8080" guix substitute-binary --substitute /gnu/store/iw3jn6a1avv78pp5v2cv42vyh0d8zi0g-guile-toxcore-0.1-6a9fbe0 /tmp/guile-toxcore
  
  found valid signature for '/gnu/store/iw3jn6a1avv78pp5v2cv42vyh0d8zi0g-guile-toxcore-0.1-6a9fbe0', from 'http://192.168.1.157:8080/nar/iw3jn6a1avv78pp5v2cv42vyh0d8zi0g-guile-toxcore-0.1-6a9fbe0'
  sha256:xl35awxl6kgl5b2eum7kfgqlnap7a7itczpyxptf4kq25pasafka
  downloading `/gnu/store/iw3jn6a1avv78pp5v2cv42vyh0d8zi0g-guile-toxcore-0.1-6a9fbe0' (0.1 MiB installed)...
  http://192.168.1.157/.../iw3jn6a1avv78pp5v2cv42vyh0d8zi0g-guile-toxcore-0.1-6a9fbe0     100.0% of 127.5 KiBguix substitute-binary: error: invalid nar end-of-file marker


When the same command is run again, there's a new error:

  dave@labrys ~$ sudo _NIX_OPTIONS="substitute-urls=http://192.168.1.157:8080" guix substitute-binary --substitute /gnu/store/iw3jn6a1avv78pp5v2cv42vyh0d8zi0g-guile-toxcore-0.1-6a9fbe0 /tmp/guile-toxcore
  
  found valid signature for '/gnu/store/iw3jn6a1avv78pp5v2cv42vyh0d8zi0g-guile-toxcore-0.1-6a9fbe0', from 'http://192.168.1.157:8080/nar/iw3jn6a1avv78pp5v2cv42vyh0d8zi0g-guile-toxcore-0.1-6a9fbe0'
  sha256:xl35awxl6kgl5b2eum7kfgqlnap7a7itczpyxptf4kq25pasafka
  downloading `/gnu/store/iw3jn6a1avv78pp5v2cv42vyh0d8zi0g-guile-toxcore-0.1-6a9fbe0' (0.1 MiB installed)...
  http://192.168.1.157/.../iw3jn6a1avv78pp5v2cv42vyh0d8zi0g-guile-toxcore-0.1-6a9fbe0       6.3% of 127.5 KiBguix substitute-binary: error: mkdir: File exists

Any idea what could be going wrong?

-- 
David Thompson
Web Developer - Free Software Foundation - http://fsf.org
GPG Key: 0FF1D807
Support the FSF: https://fsf.org/donate

bug#19512: Failed substitution yields incomplete store item

[Ludovic Courtès]

David Thompson <dthompson2@worcester.edu> skribis:

> While working on 'guix publish', I managed to mess up the store.  :(
>
> Here's the first substitution that failed due to an invalid eof marker
> for some reason:
>
>   dave@labrys ~$ sudo _NIX_OPTIONS="substitute-urls=http://192.168.1.157:8080" guix substitute-binary --substitute /gnu/store/iw3jn6a1avv78pp5v2cv42vyh0d8zi0g-guile-toxcore-0.1-6a9fbe0 /tmp/guile-toxcore
>   
>   found valid signature for '/gnu/store/iw3jn6a1avv78pp5v2cv42vyh0d8zi0g-guile-toxcore-0.1-6a9fbe0', from 'http://192.168.1.157:8080/nar/iw3jn6a1avv78pp5v2cv42vyh0d8zi0g-guile-toxcore-0.1-6a9fbe0'
>   sha256:xl35awxl6kgl5b2eum7kfgqlnap7a7itczpyxptf4kq25pasafka
>   downloading `/gnu/store/iw3jn6a1avv78pp5v2cv42vyh0d8zi0g-guile-toxcore-0.1-6a9fbe0' (0.1 MiB installed)...
>   http://192.168.1.157/.../iw3jn6a1avv78pp5v2cv42vyh0d8zi0g-guile-toxcore-0.1-6a9fbe0     100.0% of 127.5 KiBguix substitute-binary: error: invalid nar end-of-file marker
>
>
> When the same command is run again, there's a new error:
>
>   dave@labrys ~$ sudo _NIX_OPTIONS="substitute-urls=http://192.168.1.157:8080" guix substitute-binary --substitute /gnu/store/iw3jn6a1avv78pp5v2cv42vyh0d8zi0g-guile-toxcore-0.1-6a9fbe0 /tmp/guile-toxcore
>   
>   found valid signature for '/gnu/store/iw3jn6a1avv78pp5v2cv42vyh0d8zi0g-guile-toxcore-0.1-6a9fbe0', from 'http://192.168.1.157:8080/nar/iw3jn6a1avv78pp5v2cv42vyh0d8zi0g-guile-toxcore-0.1-6a9fbe0'
>   sha256:xl35awxl6kgl5b2eum7kfgqlnap7a7itczpyxptf4kq25pasafka
>   downloading `/gnu/store/iw3jn6a1avv78pp5v2cv42vyh0d8zi0g-guile-toxcore-0.1-6a9fbe0' (0.1 MiB installed)...
>   http://192.168.1.157/.../iw3jn6a1avv78pp5v2cv42vyh0d8zi0g-guile-toxcore-0.1-6a9fbe0       6.3% of 127.5 KiBguix substitute-binary: error: mkdir: File exists
>
> Any idea what could be going wrong?

Nothing!  :-)

The store item that substitute-binary created on the first run is
actually invalid.  You can see that from the fact that you get an error
if you run, say:

  guix gc --references /gnu/store/iw3jn6a1avv78pp5v2cv42vyh0d8zi0g-guile-toxcore-0.1-6a9fbe0

So you can either “rm -rf” it (use with care!), or, better, run ‘guix
gc’, which will automatically remove it and other invalid items.

(A store item is considered valid iff guix-daemon or guix-register has
added it to db.sqlite.  guix-daemon does that upon successful
substitution or build.)

HTH,
Ludo’.

bug#19512: Failed substitution yields incomplete store item

[Ludovic Courtès]

Hi!

Did you eventually salvage the store?  What lessons should be taken on
the Guix side?

TIA,
Ludo’.

bug#19512: Failed substitution yields incomplete store item

[David Thompson]

Ludovic Courtès <ludo@gnu.org> writes:

> Did you eventually salvage the store?  What lessons should be taken on
> the Guix side?

I haven't gotten to the bottom of it, sorry.

-- 
David Thompson
Web Developer - Free Software Foundation - http://fsf.org
GPG Key: 0FF1D807
Support the FSF: https://fsf.org/donate

bug#19512: Failed substitution yields incomplete store item

[Ludovic Courtès]

Closing this bug, thanks.

Ludo’.