Fixed output derivation hash mismatch - why?

Fixed output derivation hash mismatch - why?

[Chris Marusich]

[-- Attachment #1.1: Type: text/plain, Size: 2977 bytes --]

Hi Guix,

I'm trying to define a fixed output derivation using gexp->derivation,
but I'm getting a hash mismatch, and I can't figure out why.  I've
attached a minimal script that illustrates the problem.  When I run it,
I see the following output:

--8<---------------cut here---------------start------------->8---
$ /tmp/test-fixed-output-derivation.scm 
;;; note: auto-compilation is enabled, set GUILE_AUTO_COMPILE=0
;;;       or pass the --no-auto-compile argument to disable.
;;; compiling /tmp/test-fixed-output-derivation.scm
;;; compiled /home/marusich/.cache/guile/ccache/2.2-LE-8-3.A/tmp/test-fixed-output-derivation.scm.go
The following derivation will be built:
   /gnu/store/fiiz8cxz90q7l0c6jfdrpzkanyyvhpv8-test.drv
building path(s) `/gnu/store/8s4m28x6qmpkqhqasy59lw3hih548kxv-test'
`/gnu/store/kfzwabhn37bg692ayb2czb853drlv2gp-test' -> `/gnu/store/8s4m28x6qmpkqhqasy59lw3hih548kxv-test'
sha256 hash mismatch for output path `/gnu/store/8s4m28x6qmpkqhqasy59lw3hih548kxv-test'
  expected: 0dids4n73p8w77pn9nffs7x0ww1a3f1jbic1p642hwd0jfl3zcg9
  actual:   1r3v22qkypccqifzbww5lrn6hf1chi23m6zzkyvb8bvg457nxslq
Backtrace:
           8 (apply-smob/1 #<catch-closure 2507640>)
In ice-9/boot-9.scm:
    705:2  7 (call-with-prompt _ _ #<procedure default-prompt-handle…>)
In ice-9/eval.scm:
    619:8  6 (_ #(#(#<directory (guile-user) 25be140>)))
In ice-9/boot-9.scm:
   2312:4  5 (save-module-excursion _)
  3822:12  4 (_)
In /tmp/test-fixed-output-derivation.scm:
     30:2  3 (_)
In guix/store.scm:
  1443:24  2 (run-with-store _ _ #:guile-for-build _ #:system _ # _)
  1362:38  1 (_ #<build-daemon 256.97 2bd1eb0>)
   936:15  0 (_ #<build-daemon 256.97 2bd1eb0> _ _)

guix/store.scm:936:15: Throw to key `srfi-34' with args `(#<condition &nix-protocol-error [message: "build of `/gnu/store/fiiz8cxz90q7l0c6jfdrpzkanyyvhpv8-test.drv' failed" status: 1] 33c9030>)'.
--8<---------------cut here---------------end--------------->8---

I don't understand why the Guix daemon thinks that the output
(/gnu/store/8s4m28x6qmpkqhqasy59lw3hih548kxv-test) hashes to
0dids4n73p8w77pn9nffs7x0ww1a3f1jbic1p642hwd0jfl3zcg9.  That doesn't even
match what "guix hash" says:

  $ guix hash /gnu/store/8s4m28x6qmpkqhqasy59lw3hih548kxv-test
  1r3v22qkypccqifzbww5lrn6hf1chi23m6zzkyvb8bvg457nxslq
  $ guix hash --recursive /gnu/store/8s4m28x6qmpkqhqasy59lw3hih548kxv-test
  1xmr8jicvzszfzpz46g37mlpvbzjl2wpwvl2b05psipssyp1sm8h

Note that the result of "guix hash" above DOES match the hash value for
/tmp/test:

  $ guix hash /tmp/test
  1r3v22qkypccqifzbww5lrn6hf1chi23m6zzkyvb8bvg457nxslq
  $ guix hash --recursive /tmp/test
  1xmr8jicvzszfzpz46g37mlpvbzjl2wpwvl2b05psipssyp1sm8h

What's going on here?  No matter what I try (e.g., change recursive? to
#t), I can't seem to escape this hash mismatch.  I think I'm missing
something obvious, but I can't see what.  I'd really appreciate a second
pair of eyes!

-- 
Chris

[-- Attachment #1.2: test-fixed-output-derivation.scm --]
[-- Type: application/octet-stream, Size: 1526 bytes --]

#!/run/current-system/profile/bin/guile -s
!#
(use-modules
 (guix store)
 (guix gexp)
 (guix ui)
 (ice-9 format)
 (guix derivations)
 (guix hash)
 (guix packages)
 (guix monads))

(define file-path "/tmp/test")
(define name "test")
(define hash (sha256
              (base32
               "1r3v22qkypccqifzbww5lrn6hf1chi23m6zzkyvb8bvg457nxslq")))
(define recursive? #f)
(define add-to-store* (store-lift add-to-store))

;; $ guix hash /tmp/test 
;; 1r3v22qkypccqifzbww5lrn6hf1chi23m6zzkyvb8bvg457nxslq
;; $ guix hash --recursive /tmp/test 
;; 1xmr8jicvzszfzpz46g37mlpvbzjl2wpwvl2b05psipssyp1sm8h
(call-with-output-file file-path
  (lambda (port)
    (display "hi\n" port)))

(with-store store
  (run-with-store store
    ;; Add the file to the store, and create a fixed-output derivation
    ;; that copies the file (from the store) to its output.
    (mlet* %store-monad
        ((store-path (add-to-store* name recursive? "sha256" file-path))
         (builder -> (with-imported-modules '((guix build utils))
                       #~(begin
                           (use-modules (guix build utils))
                           (copy-recursively #$store-path #$output))))
         (drv (gexp->derivation name builder
                                #:hash-algo 'sha256
                                #:hash hash
                                #:recursive? recursive?))
         (drvs -> (list drv)))
      (mbegin %store-monad
        (show-what-to-build* drvs #:use-substitutes? #f)
        (built-derivations drvs)))))

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: Fixed output derivation hash mismatch - why?

[Mark H Weaver]

Hi Chris,

Chris Marusich <cmmarusich@gmail.com> writes:

> I'm trying to define a fixed output derivation using gexp->derivation,
> but I'm getting a hash mismatch, and I can't figure out why.  I've
> attached a minimal script that illustrates the problem.  When I run it,
> I see the following output:
>
> $ /tmp/test-fixed-output-derivation.scm 
> ;;; note: auto-compilation is enabled, set GUILE_AUTO_COMPILE=0
> ;;;       or pass the --no-auto-compile argument to disable.
> ;;; compiling /tmp/test-fixed-output-derivation.scm
> ;;; compiled /home/marusich/.cache/guile/ccache/2.2-LE-8-3.A/tmp/test-fixed-output-derivation.scm.go
> The following derivation will be built:
>    /gnu/store/fiiz8cxz90q7l0c6jfdrpzkanyyvhpv8-test.drv
> building path(s) `/gnu/store/8s4m28x6qmpkqhqasy59lw3hih548kxv-test'
> `/gnu/store/kfzwabhn37bg692ayb2czb853drlv2gp-test' -> `/gnu/store/8s4m28x6qmpkqhqasy59lw3hih548kxv-test'
> sha256 hash mismatch for output path `/gnu/store/8s4m28x6qmpkqhqasy59lw3hih548kxv-test'
>   expected: 0dids4n73p8w77pn9nffs7x0ww1a3f1jbic1p642hwd0jfl3zcg9
>   actual:   1r3v22qkypccqifzbww5lrn6hf1chi23m6zzkyvb8bvg457nxslq
> Backtrace:
>            8 (apply-smob/1 #<catch-closure 2507640>)
> In ice-9/boot-9.scm:
>     705:2  7 (call-with-prompt _ _ #<procedure default-prompt-handle…>)
> In ice-9/eval.scm:
>     619:8  6 (_ #(#(#<directory (guile-user) 25be140>)))
> In ice-9/boot-9.scm:
>    2312:4  5 (save-module-excursion _)
>   3822:12  4 (_)
> In /tmp/test-fixed-output-derivation.scm:
>      30:2  3 (_)
> In guix/store.scm:
>   1443:24  2 (run-with-store _ _ #:guile-for-build _ #:system _ # _)
>   1362:38  1 (_ #<build-daemon 256.97 2bd1eb0>)
>    936:15  0 (_ #<build-daemon 256.97 2bd1eb0> _ _)
>
> guix/store.scm:936:15: Throw to key `srfi-34' with args `(#<condition &nix-protocol-error [message: "build of `/gnu/store/fiiz8cxz90q7l0c6jfdrpzkanyyvhpv8-test.drv' failed" status: 1] 33c9030>)'.
>
> I don't understand why the Guix daemon thinks that the output
> (/gnu/store/8s4m28x6qmpkqhqasy59lw3hih548kxv-test) hashes to
> 0dids4n73p8w77pn9nffs7x0ww1a3f1jbic1p642hwd0jfl3zcg9.

It doesn't.  When the daemon writes:

   expected: 0dids4n73p8w77pn9nffs7x0ww1a3f1jbic1p642hwd0jfl3zcg9
   actual:   1r3v22qkypccqifzbww5lrn6hf1chi23m6zzkyvb8bvg457nxslq

that means that the output hashes to
1r3v22qkypccqifzbww5lrn6hf1chi23m6zzkyvb8bvg457nxslq, but you told it to
expect 0dids4n73p8w77pn9nffs7x0ww1a3f1jbic1p642hwd0jfl3zcg9.

> That doesn't even
> match what "guix hash" says:
>
>   $ guix hash /gnu/store/8s4m28x6qmpkqhqasy59lw3hih548kxv-test
>   1r3v22qkypccqifzbww5lrn6hf1chi23m6zzkyvb8bvg457nxslq

Yes it does.  This matches what the daemon wrote was the "actual" hash.

> Note that the result of "guix hash" above DOES match the hash value for
> /tmp/test:
>
>   $ guix hash /tmp/test
>   1r3v22qkypccqifzbww5lrn6hf1chi23m6zzkyvb8bvg457nxslq

It matches both of them.

> What's going on here?

I think the problem is here:

  (define hash (sha256
                (base32
                 "1r3v22qkypccqifzbww5lrn6hf1chi23m6zzkyvb8bvg457nxslq")))

Remove the 'sha256'.  It should just be this:

  (define hash (base32
                "1r3v22qkypccqifzbww5lrn6hf1chi23m6zzkyvb8bvg457nxslq"))

Within 'origin' records, the 'sha256' is the field name, part of the
constructor syntax.  Here, you are not within an 'origin' constructor,
and the 'sha256' is instead a procedure which computes the sha256 hash
of its argument.

      Mark

Re: Fixed output derivation hash mismatch - why?

[Chris Marusich]

[-- Attachment #1: Type: text/plain, Size: 855 bytes --]

Mark H Weaver <mhw@netris.org> writes:

> I think the problem is here:
>
>   (define hash (sha256
>                 (base32
>                  "1r3v22qkypccqifzbww5lrn6hf1chi23m6zzkyvb8bvg457nxslq")))
>
> Remove the 'sha256'.  It should just be this:
>
>   (define hash (base32
>                 "1r3v22qkypccqifzbww5lrn6hf1chi23m6zzkyvb8bvg457nxslq"))
>
> Within 'origin' records, the 'sha256' is the field name, part of the
> constructor syntax.  Here, you are not within an 'origin' constructor,
> and the 'sha256' is instead a procedure which computes the sha256 hash
> of its argument.

This was the problem!  I misunderstood what the daemon was telling me,
and I didn't realize that "sha256" was the field name.

I knew a second pair of eyes would help!  Thank you for helping me to
see more clearly, Mark.  :-)

-- 
Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]