Grafting fails for latest Go release candidate

Grafting fails for latest Go release candidate

[Leo Famulari]

[-- Attachment #1.1: Type: text/plain, Size: 1246 bytes --]

Working on the Go package, I found that the grafting procedure fails in
a way that I don't understand:

------
$ ./pre-inst-env guix build go@1.9
The following derivation will be built:
   /gnu/store/bv30jxzjl7c5z3hk6pmywpzwwrn1hq5m-go-1.9rc1.drv
@ build-started /gnu/store/bv30jxzjl7c5z3hk6pmywpzwwrn1hq5m-go-1.9rc1.drv - x86_64-linux /var/log/guix/drvs/bv//30jxzjl7c5z3hk6pmywpzwwrn1hq5m-go-1.9rc1.drv.bz2
grafting '/gnu/store/8zbi5rr5q2pxksfgpxq37m60nsa21d3v-go-1.9rc1-doc' -> '/gnu/store/6v24sf0b3hgyc80b4w2yvx1aw03zinw6-go-1.9rc1-doc'...
grafting '/gnu/store/73fvsx0qizcrbajgjwn8mkzkpxa6abx4-go-1.9rc1' -> '/gnu/store/ghg75mjw3csiv9hf7q0cxrm1g0fx5d0k-go-1.9rc1'...
ERROR: In procedure put-bytevector: Value out of range: 0
builder for `/gnu/store/bv30jxzjl7c5z3hk6pmywpzwwrn1hq5m-go-1.9rc1.drv' failed with exit code 1
@ build-failed /gnu/store/bv30jxzjl7c5z3hk6pmywpzwwrn1hq5m-go-1.9rc1.drv - 1 builder for `/gnu/store/bv30jxzjl7c5z3hk6pmywpzwwrn1hq5m-go-1.9rc1.drv' failed with exit code 1
guix build: error: build failed: build of `/gnu/store/bv30jxzjl7c5z3hk6pmywpzwwrn1hq5m-go-1.9rc1.drv' failed
------

This doesn't happen with the previous release, go1.9beta2. For both
releases, the ungrafted Go seems to work.

I'll keep digging...

[-- Attachment #1.2: 0001-WIP-go-Update-to-1.9beta2.patch --]
[-- Type: text/plain, Size: 3007 bytes --]

From 1bb9f5846bc530c210ed62e5578559578b939eed Mon Sep 17 00:00:00 2001
From: Leo Famulari <leo@famulari.name>
Date: Fri, 14 Jul 2017 14:15:54 -0400
Subject: [PATCH 1/2] WIP: go: Update to 1.9beta2.

* gnu/packages/golang.scm (go-1.8): Replace with ...
(go-1.9): ... new variable.
[arguments]: Adjust the pre-build phase accordingly.
---
 gnu/packages/golang.scm | 14 ++++++++------
 1 file changed, 8 insertions(+), 6 deletions(-)

diff --git a/gnu/packages/golang.scm b/gnu/packages/golang.scm
index 70cae6d87..a7facf4ed 100644
--- a/gnu/packages/golang.scm
+++ b/gnu/packages/golang.scm
@@ -199,11 +199,11 @@ garbage collection, various safety features and in the style of communicating
 sequential processes (CSP) concurrent programming features added.")
     (license license:bsd-3)))
 
-(define-public go-1.8
+(define-public go-1.9
   (package
     (inherit go-1.4)
     (name "go")
-    (version "1.8.3")
+    (version "1.9beta2")
     (source
      (origin
        (method url-fetch)
@@ -211,7 +211,7 @@ sequential processes (CSP) concurrent programming features added.")
                            name version ".src.tar.gz"))
        (sha256
         (base32
-         "19lzv4lqixj3v2gjaff0fdbbmgsq5r8lrfd61z2zvp778wjflpaz"))))
+         "1hx8vab9qbwpk75lfx1zvj9a6ssalp1hpa1pp0fyzcn3x4lip8ac"))))
     (arguments
      (substitute-keyword-arguments (package-arguments go-1.4)
        ((#:phases phases)
@@ -242,7 +242,7 @@ sequential processes (CSP) concurrent programming features added.")
                  ;; Add libgcc to runpath
                  (substitute* "cmd/link/internal/ld/lib.go"
                    (("!rpath.set") "true"))
-                 (substitute* "cmd/go/build.go"
+                 (substitute* "cmd/go/internal/work/build.go"
                    (("cgoldflags := \\[\\]string\\{\\}")
                     (string-append "cgoldflags := []string{"
                                    "\"-rpath=" gcclib "\""
@@ -265,7 +265,9 @@ sequential processes (CSP) concurrent programming features added.")
                      (substitute* file
                        ((regex all before test_name)
                         (string-append before "Disabled" test_name)))))
-                  '(("net/net_test.go" "(.+)(TestShutdownUnix.+)")
+                  '(;; See https://github.com/golang/go/commit/81ed9ca1d411f0bd8738ea60b3a8045446ad85fc
+                    ("cmd/go/go_test.go" "(.+)(TestExecutableGOROOT.+)")
+                    ("net/net_test.go" "(.+)(TestShutdownUnix.+)")
                     ("net/dial_test.go" "(.+)(TestDialTimeout.+)")
                     ("os/os_test.go" "(.+)(TestHostname.+)")
                     ("time/format_test.go" "(.+)(TestParseInSydney.+)")
@@ -366,4 +368,4 @@ sequential processes (CSP) concurrent programming features added.")
      `(("go" ,go-1.4)
        ,@(package-native-inputs go-1.4)))))
 
-(define-public go go-1.8)
+(define-public go go-1.9)
-- 
2.13.3


[-- Attachment #1.3: 0002-WIP-go-Update-to-1.9rc1.patch --]
[-- Type: text/plain, Size: 1290 bytes --]

From ecff7bddcb63f0d03f0d319a2eddf5ab6db65198 Mon Sep 17 00:00:00 2001
From: Leo Famulari <leo@famulari.name>
Date: Fri, 14 Jul 2017 14:15:54 -0400
Subject: [PATCH 2/2] WIP: go: Update to 1.9rc1.

* gnu/packages/golang.scm (go-1.8): Replace with ...
(go-1.9): ... new variable.
[arguments]: Adjust the pre-build phase accordingly.
---
 gnu/packages/golang.scm | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/gnu/packages/golang.scm b/gnu/packages/golang.scm
index a7facf4ed..7ed9cc750 100644
--- a/gnu/packages/golang.scm
+++ b/gnu/packages/golang.scm
@@ -203,7 +203,7 @@ sequential processes (CSP) concurrent programming features added.")
   (package
     (inherit go-1.4)
     (name "go")
-    (version "1.9beta2")
+    (version "1.9rc1")
     (source
      (origin
        (method url-fetch)
@@ -211,7 +211,7 @@ sequential processes (CSP) concurrent programming features added.")
                            name version ".src.tar.gz"))
        (sha256
         (base32
-         "1hx8vab9qbwpk75lfx1zvj9a6ssalp1hpa1pp0fyzcn3x4lip8ac"))))
+         "0bj5dcvhz48bf6704qhqnybf220kgqgi93x4mx1n3k30xac7awc7"))))
     (arguments
      (substitute-keyword-arguments (package-arguments go-1.4)
        ((#:phases phases)
-- 
2.13.3


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Grafting fails for latest Go release candidate

[Marius Bakke]

On 31 July 2017 16:39:25 CEST, Leo Famulari <leo@famulari.name> wrote:
>Working on the Go package, I found that the grafting procedure fails in
>a way that I don't understand:
>
>------
>$ ./pre-inst-env guix build go@1.9
>The following derivation will be built:
>   /gnu/store/bv30jxzjl7c5z3hk6pmywpzwwrn1hq5m-go-1.9rc1.drv
>@ build-started
>/gnu/store/bv30jxzjl7c5z3hk6pmywpzwwrn1hq5m-go-1.9rc1.drv -
>x86_64-linux
>/var/log/guix/drvs/bv//30jxzjl7c5z3hk6pmywpzwwrn1hq5m-go-1.9rc1.drv.bz2
>grafting '/gnu/store/8zbi5rr5q2pxksfgpxq37m60nsa21d3v-go-1.9rc1-doc' ->
>'/gnu/store/6v24sf0b3hgyc80b4w2yvx1aw03zinw6-go-1.9rc1-doc'...
>grafting '/gnu/store/73fvsx0qizcrbajgjwn8mkzkpxa6abx4-go-1.9rc1' ->
>'/gnu/store/ghg75mjw3csiv9hf7q0cxrm1g0fx5d0k-go-1.9rc1'...
>ERROR: In procedure put-bytevector: Value out of range: 0
>builder for `/gnu/store/bv30jxzjl7c5z3hk6pmywpzwwrn1hq5m-go-1.9rc1.drv'
>failed with exit code 1
>@ build-failed
>/gnu/store/bv30jxzjl7c5z3hk6pmywpzwwrn1hq5m-go-1.9rc1.drv - 1 builder
>for `/gnu/store/bv30jxzjl7c5z3hk6pmywpzwwrn1hq5m-go-1.9rc1.drv' failed
>with exit code 1
>guix build: error: build failed: build of
>`/gnu/store/bv30jxzjl7c5z3hk6pmywpzwwrn1hq5m-go-1.9rc1.drv' failed
>------
>
>This doesn't happen with the previous release, go1.9beta2. For both
>releases, the ungrafted Go seems to work.
>
>I'll keep digging...

I've seen this too in some other context, so it's not go-specific.

I was not able to reproduce it however. If the error is consistent for you, can you try to strace it?


-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Re: Grafting fails for latest Go release candidate

[Mark H Weaver]

[Adding Ludovic to the CC]

Leo Famulari <leo@famulari.name> writes:

> Working on the Go package, I found that the grafting procedure fails in
> a way that I don't understand:
>
> ------
> $ ./pre-inst-env guix build go@1.9
> The following derivation will be built:
>    /gnu/store/bv30jxzjl7c5z3hk6pmywpzwwrn1hq5m-go-1.9rc1.drv
> @ build-started
> /gnu/store/bv30jxzjl7c5z3hk6pmywpzwwrn1hq5m-go-1.9rc1.drv -
> x86_64-linux
> /var/log/guix/drvs/bv//30jxzjl7c5z3hk6pmywpzwwrn1hq5m-go-1.9rc1.drv.bz2
> grafting '/gnu/store/8zbi5rr5q2pxksfgpxq37m60nsa21d3v-go-1.9rc1-doc' -> '/gnu/store/6v24sf0b3hgyc80b4w2yvx1aw03zinw6-go-1.9rc1-doc'...
> grafting '/gnu/store/73fvsx0qizcrbajgjwn8mkzkpxa6abx4-go-1.9rc1' -> '/gnu/store/ghg75mjw3csiv9hf7q0cxrm1g0fx5d0k-go-1.9rc1'...
> ERROR: In procedure put-bytevector: Value out of range: 0

I looked into this a bit.  As far as I can tell, the only way that (guix
build graft) could cause 'put-bytevector' to generate this error is if
it's called with four arguments, and the fourth argument (count) is 0,
and the third argument (start) is greater than the length of the
bytevector.

I don't have time right now to complete my analysis, but my strong
suspicion is that the following commit introduced this bug:

  https://git.savannah.gnu.org/cgit/guix.git/commit/?id=57bdd79e485801ccf405ca7389bd099809fe5d67

Specifically, I believe that it mishandles the case where the
end-of-buffer falls within the human-readable part of the store file
name, i.e. if it's within the "eeeeeeeeeeeeeeee" part of the following
example hash:

  /gnu/store/bv30jxzjl7c5z3hk6pmywpzwwrn1hq5m-eeeeeeeeeeeeeeee

The original grafting code that I wrote only rewrote the hashes, and
ensured that the entire hash was present within the buffer.  This
guaranteed that after writing the replacement hash, that the scan
position was still within the buffer.

After the modifications of the commit cited above, when a hash to be
replaced is found, the entire replacement string, including the
human-readable part, is written.  This may leave the scan pointer beyond
the end of the buffer.  It would take some more thought to determine
what happens after that, but I doubt that it's handled properly and it
seems plausible that it could lead to this error.

       Mark

Re: Grafting fails for latest Go release candidate

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 1505 bytes --]

Leo Famulari <leo@famulari.name> writes:

> Working on the Go package, I found that the grafting procedure fails in
> a way that I don't understand:
>
> ------
> $ ./pre-inst-env guix build go@1.9
> The following derivation will be built:
>    /gnu/store/bv30jxzjl7c5z3hk6pmywpzwwrn1hq5m-go-1.9rc1.drv
> @ build-started /gnu/store/bv30jxzjl7c5z3hk6pmywpzwwrn1hq5m-go-1.9rc1.drv - x86_64-linux /var/log/guix/drvs/bv//30jxzjl7c5z3hk6pmywpzwwrn1hq5m-go-1.9rc1.drv.bz2
> grafting '/gnu/store/8zbi5rr5q2pxksfgpxq37m60nsa21d3v-go-1.9rc1-doc' -> '/gnu/store/6v24sf0b3hgyc80b4w2yvx1aw03zinw6-go-1.9rc1-doc'...
> grafting '/gnu/store/73fvsx0qizcrbajgjwn8mkzkpxa6abx4-go-1.9rc1' -> '/gnu/store/ghg75mjw3csiv9hf7q0cxrm1g0fx5d0k-go-1.9rc1'...
> ERROR: In procedure put-bytevector: Value out of range: 0
> builder for `/gnu/store/bv30jxzjl7c5z3hk6pmywpzwwrn1hq5m-go-1.9rc1.drv' failed with exit code 1
> @ build-failed /gnu/store/bv30jxzjl7c5z3hk6pmywpzwwrn1hq5m-go-1.9rc1.drv - 1 builder for `/gnu/store/bv30jxzjl7c5z3hk6pmywpzwwrn1hq5m-go-1.9rc1.drv' failed with exit code 1
> guix build: error: build failed: build of `/gnu/store/bv30jxzjl7c5z3hk6pmywpzwwrn1hq5m-go-1.9rc1.drv' failed

I just got a similar, but not quite same error:

grafting '/gnu/store/mqlhrs9xj7rdnm3apzj2las4fr84i3gy-git-2.13.4' -> '/gnu/store/7kfijk9k2vq7c9bg6zy2pb2xpzs6s5b0-git-2.13.4'...
ERROR: In procedure put-bytevector: Wrong type argument in position 1 (expecting open output port): #<closed: file 91ef50>

This is on 'core-updates'.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: Grafting fails for latest Go release candidate

[Mark H Weaver]

Marius Bakke <mbakke@fastmail.com> writes:

> Leo Famulari <leo@famulari.name> writes:
>
>> Working on the Go package, I found that the grafting procedure fails in
>> a way that I don't understand:
>>
>> ------
>> $ ./pre-inst-env guix build go@1.9
>> The following derivation will be built:
>>    /gnu/store/bv30jxzjl7c5z3hk6pmywpzwwrn1hq5m-go-1.9rc1.drv
>> @ build-started
>> /gnu/store/bv30jxzjl7c5z3hk6pmywpzwwrn1hq5m-go-1.9rc1.drv -
>> x86_64-linux
>> /var/log/guix/drvs/bv//30jxzjl7c5z3hk6pmywpzwwrn1hq5m-go-1.9rc1.drv.bz2
>> grafting '/gnu/store/8zbi5rr5q2pxksfgpxq37m60nsa21d3v-go-1.9rc1-doc' -> '/gnu/store/6v24sf0b3hgyc80b4w2yvx1aw03zinw6-go-1.9rc1-doc'...
>> grafting '/gnu/store/73fvsx0qizcrbajgjwn8mkzkpxa6abx4-go-1.9rc1' -> '/gnu/store/ghg75mjw3csiv9hf7q0cxrm1g0fx5d0k-go-1.9rc1'...
>> ERROR: In procedure put-bytevector: Value out of range: 0
>> builder for `/gnu/store/bv30jxzjl7c5z3hk6pmywpzwwrn1hq5m-go-1.9rc1.drv' failed with exit code 1
>> @ build-failed
>> /gnu/store/bv30jxzjl7c5z3hk6pmywpzwwrn1hq5m-go-1.9rc1.drv - 1
>> builder for
>> `/gnu/store/bv30jxzjl7c5z3hk6pmywpzwwrn1hq5m-go-1.9rc1.drv' failed
>> with exit code 1
>> guix build: error: build failed: build of `/gnu/store/bv30jxzjl7c5z3hk6pmywpzwwrn1hq5m-go-1.9rc1.drv' failed
>
> I just got a similar, but not quite same error:
>
> grafting '/gnu/store/mqlhrs9xj7rdnm3apzj2las4fr84i3gy-git-2.13.4' -> '/gnu/store/7kfijk9k2vq7c9bg6zy2pb2xpzs6s5b0-git-2.13.4'...
> ERROR: In procedure put-bytevector: Wrong type argument in position 1 (expecting open output port): #<closed: file 91ef50>
>
> This is on 'core-updates'.

I guess this is a completely different issue.  Is it reproducible?

      Mark

Re: Grafting fails for latest Go release candidate

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 1766 bytes --]

Mark H Weaver <mhw@netris.org> writes:

> Marius Bakke <mbakke@fastmail.com> writes:
>
>> Leo Famulari <leo@famulari.name> writes:
>>
>>> Working on the Go package, I found that the grafting procedure fails in
>>> a way that I don't understand:
>>>
>>> ------
>>> $ ./pre-inst-env guix build go@1.9
>>> The following derivation will be built:
>>>    /gnu/store/bv30jxzjl7c5z3hk6pmywpzwwrn1hq5m-go-1.9rc1.drv
>>> @ build-started
>>> /gnu/store/bv30jxzjl7c5z3hk6pmywpzwwrn1hq5m-go-1.9rc1.drv -
>>> x86_64-linux
>>> /var/log/guix/drvs/bv//30jxzjl7c5z3hk6pmywpzwwrn1hq5m-go-1.9rc1.drv.bz2
>>> grafting '/gnu/store/8zbi5rr5q2pxksfgpxq37m60nsa21d3v-go-1.9rc1-doc' -> '/gnu/store/6v24sf0b3hgyc80b4w2yvx1aw03zinw6-go-1.9rc1-doc'...
>>> grafting '/gnu/store/73fvsx0qizcrbajgjwn8mkzkpxa6abx4-go-1.9rc1' -> '/gnu/store/ghg75mjw3csiv9hf7q0cxrm1g0fx5d0k-go-1.9rc1'...
>>> ERROR: In procedure put-bytevector: Value out of range: 0
>>> builder for `/gnu/store/bv30jxzjl7c5z3hk6pmywpzwwrn1hq5m-go-1.9rc1.drv' failed with exit code 1
>>> @ build-failed
>>> /gnu/store/bv30jxzjl7c5z3hk6pmywpzwwrn1hq5m-go-1.9rc1.drv - 1
>>> builder for
>>> `/gnu/store/bv30jxzjl7c5z3hk6pmywpzwwrn1hq5m-go-1.9rc1.drv' failed
>>> with exit code 1
>>> guix build: error: build failed: build of `/gnu/store/bv30jxzjl7c5z3hk6pmywpzwwrn1hq5m-go-1.9rc1.drv' failed
>>
>> I just got a similar, but not quite same error:
>>
>> grafting '/gnu/store/mqlhrs9xj7rdnm3apzj2las4fr84i3gy-git-2.13.4' -> '/gnu/store/7kfijk9k2vq7c9bg6zy2pb2xpzs6s5b0-git-2.13.4'...
>> ERROR: In procedure put-bytevector: Wrong type argument in position 1 (expecting open output port): #<closed: file 91ef50>
>>
>> This is on 'core-updates'.
>
> I guess this is a completely different issue.  Is it reproducible?

It's not reproducible.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: Grafting fails for latest Go release candidate

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 3030 bytes --]

Marius Bakke <mbakke@fastmail.com> writes:

> Leo Famulari <leo@famulari.name> writes:
>
>> Working on the Go package, I found that the grafting procedure fails in
>> a way that I don't understand:
>>
>> ------
>> $ ./pre-inst-env guix build go@1.9
>> The following derivation will be built:
>>    /gnu/store/bv30jxzjl7c5z3hk6pmywpzwwrn1hq5m-go-1.9rc1.drv
>> @ build-started /gnu/store/bv30jxzjl7c5z3hk6pmywpzwwrn1hq5m-go-1.9rc1.drv - x86_64-linux /var/log/guix/drvs/bv//30jxzjl7c5z3hk6pmywpzwwrn1hq5m-go-1.9rc1.drv.bz2
>> grafting '/gnu/store/8zbi5rr5q2pxksfgpxq37m60nsa21d3v-go-1.9rc1-doc' -> '/gnu/store/6v24sf0b3hgyc80b4w2yvx1aw03zinw6-go-1.9rc1-doc'...
>> grafting '/gnu/store/73fvsx0qizcrbajgjwn8mkzkpxa6abx4-go-1.9rc1' -> '/gnu/store/ghg75mjw3csiv9hf7q0cxrm1g0fx5d0k-go-1.9rc1'...
>> ERROR: In procedure put-bytevector: Value out of range: 0
>> builder for `/gnu/store/bv30jxzjl7c5z3hk6pmywpzwwrn1hq5m-go-1.9rc1.drv' failed with exit code 1
>> @ build-failed /gnu/store/bv30jxzjl7c5z3hk6pmywpzwwrn1hq5m-go-1.9rc1.drv - 1 builder for `/gnu/store/bv30jxzjl7c5z3hk6pmywpzwwrn1hq5m-go-1.9rc1.drv' failed with exit code 1
>> guix build: error: build failed: build of `/gnu/store/bv30jxzjl7c5z3hk6pmywpzwwrn1hq5m-go-1.9rc1.drv' failed
>
> I just got a similar, but not quite same error:
>
> grafting '/gnu/store/mqlhrs9xj7rdnm3apzj2las4fr84i3gy-git-2.13.4' -> '/gnu/store/7kfijk9k2vq7c9bg6zy2pb2xpzs6s5b0-git-2.13.4'...
> ERROR: In procedure put-bytevector: Wrong type argument in position 1 (expecting open output port): #<closed: file 91ef50>

I've caught a couple other elusive grafting errors:

grafting '/gnu/store/ccw7wzh9rbflc0fl968dbj2x0x9dn4y5-chromium-60.0.3112.90' -> '/gnu/store/b3f2d6l1c546xwcv4hg8619qh87cixhb-chromium-60.0.3112.90'...
ERROR: Wrong type to apply: "der */\n  max-width: 638px;\n  min" 

Wait, what? And this:

grafting '/gnu/store/r4ylhf3cx6gp4iv1m056i58hqwz7m0n5-git-2.14.0' -> '/gnu/store/vy3rj6sc412kxabfisqiywp5z4by21jn-git-2.14.0'...
ERROR: In procedure vector-ref: Wrong type argument in position 1 (expecting vector): #(#f #f #f #f
 #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f
 #f #f #f #f #f #f #f #f #f #f #f #t #t #t #t #t #t #t #t #t #t #f #f #f #f #f #f #f #f #f #f #f #f
 #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #t #t #t #t #f #t
 #t #t #t #t #t #t #t #t #f #t #t #t #t #f #f #t #t #t #t #t #f #f #f #f #f #f #f #f #f #f #f #f #f
 #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f
 #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f
 #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f
 #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f)

These were sampled from different systems so it's not a local
issue. Clearly something is spilling memory, or reading the wrong
address, but what?


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: Grafting fails for latest Go release candidate

[Andy Wingo]

On Mon 07 Aug 2017 22:08, Marius Bakke <mbakke@fastmail.com> writes:

> I've caught a couple other elusive grafting errors:
>
> grafting '/gnu/store/ccw7wzh9rbflc0fl968dbj2x0x9dn4y5-chromium-60.0.3112.90' -> '/gnu/store/b3f2d6l1c546xwcv4hg8619qh87cixhb-chromium-60.0.3112.90'...
> ERROR: Wrong type to apply: "der */\n  max-width: 638px;\n  min" 

This looks like a Guile 2.2 GC error somehow :-(

Andy

Re: Grafting fails for latest Go release candidate

[Ludovic Courtès]

Hi,

Andy Wingo <wingo@igalia.com> skribis:

> On Mon 07 Aug 2017 22:08, Marius Bakke <mbakke@fastmail.com> writes:
>
>> I've caught a couple other elusive grafting errors:
>>
>> grafting '/gnu/store/ccw7wzh9rbflc0fl968dbj2x0x9dn4y5-chromium-60.0.3112.90' -> '/gnu/store/b3f2d6l1c546xwcv4hg8619qh87cixhb-chromium-60.0.3112.90'...
>> ERROR: Wrong type to apply: "der */\n  max-width: 638px;\n  min" 
>
> This looks like a Guile 2.2 GC error somehow :-(

Marius, Leo: was the failure on ‘core-updates’, where Guile 2.2 is used?
(On ‘master’ 2.0 is used to build derivations.)

It could also be a thread-safety issue or something.

Ludo’.

Re: Grafting fails for latest Go release candidate

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 944 bytes --]

Ludovic Courtès <ludo@gnu.org> writes:

> Hi,
>
> Andy Wingo <wingo@igalia.com> skribis:
>
>> On Mon 07 Aug 2017 22:08, Marius Bakke <mbakke@fastmail.com> writes:
>>
>>> I've caught a couple other elusive grafting errors:
>>>
>>> grafting '/gnu/store/ccw7wzh9rbflc0fl968dbj2x0x9dn4y5-chromium-60.0.3112.90' -> '/gnu/store/b3f2d6l1c546xwcv4hg8619qh87cixhb-chromium-60.0.3112.90'...
>>> ERROR: Wrong type to apply: "der */\n  max-width: 638px;\n  min" 
>>
>> This looks like a Guile 2.2 GC error somehow :-(
>
> Marius, Leo: was the failure on ‘core-updates’, where Guile 2.2 is used?
> (On ‘master’ 2.0 is used to build derivations.)

Pretty sure I've seen these on 'master' at least once, but the majority
of grafting errors have been on 'core-updates'.

However, I haven't had any since (very) recently, sometime between
09ec5a0f0c..2718a9cd09 in my local "fork".  ed464f48ef maybe?

Or I've just been lucky.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: Grafting fails for latest Go release candidate

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 1313 bytes --]

Marius Bakke <mbakke@fastmail.com> writes:

> Ludovic Courtès <ludo@gnu.org> writes:
>
>> Hi,
>>
>> Andy Wingo <wingo@igalia.com> skribis:
>>
>>> On Mon 07 Aug 2017 22:08, Marius Bakke <mbakke@fastmail.com> writes:
>>>
>>>> I've caught a couple other elusive grafting errors:
>>>>
>>>> grafting '/gnu/store/ccw7wzh9rbflc0fl968dbj2x0x9dn4y5-chromium-60.0.3112.90' -> '/gnu/store/b3f2d6l1c546xwcv4hg8619qh87cixhb-chromium-60.0.3112.90'...
>>>> ERROR: Wrong type to apply: "der */\n  max-width: 638px;\n  min" 
>>>
>>> This looks like a Guile 2.2 GC error somehow :-(
>>
>> Marius, Leo: was the failure on ‘core-updates’, where Guile 2.2 is used?
>> (On ‘master’ 2.0 is used to build derivations.)
>
> Pretty sure I've seen these on 'master' at least once, but the majority
> of grafting errors have been on 'core-updates'.
>
> However, I haven't had any since (very) recently, sometime between
> 09ec5a0f0c..2718a9cd09 in my local "fork".  ed464f48ef maybe?
>
> Or I've just been lucky.

Most likely just lucky, new grafting failure today.

I've noticed they are more likely to occur when the host system is busy.
Grafting a large package such as 'texlive' is almost guaranteed to fail
on a fully-utilized machine.

Maybe we should switch back to 2.0 for the time being?

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

bug#28211: Grafting fails for latest Go release candidate

[Ludovic Courtès]

Marius Bakke <mbakke@fastmail.com> skribis:

> Most likely just lucky, new grafting failure today.
>
> I've noticed they are more likely to occur when the host system is busy.
> Grafting a large package such as 'texlive' is almost guaranteed to fail
> on a fully-utilized machine.
>
> Maybe we should switch back to 2.0 for the time being?

Yes, done: <https://bugs.gnu.org/28211>.

I took a look: the code in (guix build graft) is multi-threaded but
there’s no shared state among threads (except for the current
output/error ports, which may partly explain the error messages that
suggest memory corruption.)  So it may be that the problem is in
libguile.

Ludo’.

Re: Grafting fails for latest Go release candidate

[Ludovic Courtès]

Marius Bakke <mbakke@fastmail.com> skribis:

> Most likely just lucky, new grafting failure today.
>
> I've noticed they are more likely to occur when the host system is busy.
> Grafting a large package such as 'texlive' is almost guaranteed to fail
> on a fully-utilized machine.
>
> Maybe we should switch back to 2.0 for the time being?

Yes, done: <https://bugs.gnu.org/28211>.

I took a look: the code in (guix build graft) is multi-threaded but
there’s no shared state among threads (except for the current
output/error ports, which may partly explain the error messages that
suggest memory corruption.)  So it may be that the problem is in
libguile.

Ludo’.

bug#28211: Grafting fails for latest Go release candidate

[Mark H Weaver]

ludo@gnu.org (Ludovic Courtès) writes:

> Marius Bakke <mbakke@fastmail.com> skribis:
>
>> Most likely just lucky, new grafting failure today.
>>
>> I've noticed they are more likely to occur when the host system is busy.
>> Grafting a large package such as 'texlive' is almost guaranteed to fail
>> on a fully-utilized machine.
>>
>> Maybe we should switch back to 2.0 for the time being?
>
> Yes, done: <https://bugs.gnu.org/28211>.
>
> I took a look: the code in (guix build graft) is multi-threaded but
> there’s no shared state among threads (except for the current
> output/error ports,

In Guile, port operations do not include built-in thread
synchronization, so it's not safe for multiple threads to write to the
current output/error ports concurrently.  Those writes should be
protected by a mutex.

However, we had agreed that in 2.2, the port operations should be
implemented in such a way that the worst that could happen on typical
hardware in practice was for the output to be garbled.  I didn't review
the new 2.2 ports code, so I'm not sure whether this was accomplished.

      Mark

Re: bug#28211: Grafting fails for latest Go release candidate

[Mark H Weaver]

ludo@gnu.org (Ludovic Courtès) writes:

> Marius Bakke <mbakke@fastmail.com> skribis:
>
>> Most likely just lucky, new grafting failure today.
>>
>> I've noticed they are more likely to occur when the host system is busy.
>> Grafting a large package such as 'texlive' is almost guaranteed to fail
>> on a fully-utilized machine.
>>
>> Maybe we should switch back to 2.0 for the time being?
>
> Yes, done: <https://bugs.gnu.org/28211>.
>
> I took a look: the code in (guix build graft) is multi-threaded but
> there’s no shared state among threads (except for the current
> output/error ports,

In Guile, port operations do not include built-in thread
synchronization, so it's not safe for multiple threads to write to the
current output/error ports concurrently.  Those writes should be
protected by a mutex.

However, we had agreed that in 2.2, the port operations should be
implemented in such a way that the worst that could happen on typical
hardware in practice was for the output to be garbled.  I didn't review
the new 2.2 ports code, so I'm not sure whether this was accomplished.

      Mark

Re: Grafting fails for latest Go release candidate

[Leo Famulari]

I saw this problem on the master branch.


-------- Original Message --------
From: ludo@gnu.org
Sent: August 21, 2017 10:32:01 AM CDT
To: Andy Wingo <wingo@igalia.com>
Cc: guix-devel@gnu.org
Subject: Re: Grafting fails for latest Go release candidate

Hi,

Andy Wingo <wingo@igalia.com> skribis:

> On Mon 07 Aug 2017 22:08, Marius Bakke <mbakke@fastmail.com> writes:
>
>> I've caught a couple other elusive grafting errors:
>>
>> grafting '/gnu/store/ccw7wzh9rbflc0fl968dbj2x0x9dn4y5-chromium-60.0.3112.90' -> '/gnu/store/b3f2d6l1c546xwcv4hg8619qh87cixhb-chromium-60.0.3112.90'...
>> ERROR: Wrong type to apply: "der */\n  max-width: 638px;\n  min" 
>
> This looks like a Guile 2.2 GC error somehow :-(

Marius, Leo: was the failure on ‘core-updates’, where Guile 2.2 is used?
(On ‘master’ 2.0 is used to build derivations.)

It could also be a thread-safety issue or something.

Ludo’.

Re: Grafting fails for latest Go release candidate

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 1528 bytes --]

On Mon, Jul 31, 2017 at 10:39:25AM -0400, Leo Famulari wrote:
> Working on the Go package, I found that the grafting procedure fails in
> a way that I don't understand:
> 
> ------
> $ ./pre-inst-env guix build go@1.9
> The following derivation will be built:
>    /gnu/store/bv30jxzjl7c5z3hk6pmywpzwwrn1hq5m-go-1.9rc1.drv
> @ build-started /gnu/store/bv30jxzjl7c5z3hk6pmywpzwwrn1hq5m-go-1.9rc1.drv - x86_64-linux /var/log/guix/drvs/bv//30jxzjl7c5z3hk6pmywpzwwrn1hq5m-go-1.9rc1.drv.bz2
> grafting '/gnu/store/8zbi5rr5q2pxksfgpxq37m60nsa21d3v-go-1.9rc1-doc' -> '/gnu/store/6v24sf0b3hgyc80b4w2yvx1aw03zinw6-go-1.9rc1-doc'...
> grafting '/gnu/store/73fvsx0qizcrbajgjwn8mkzkpxa6abx4-go-1.9rc1' -> '/gnu/store/ghg75mjw3csiv9hf7q0cxrm1g0fx5d0k-go-1.9rc1'...
> ERROR: In procedure put-bytevector: Value out of range: 0
> builder for `/gnu/store/bv30jxzjl7c5z3hk6pmywpzwwrn1hq5m-go-1.9rc1.drv' failed with exit code 1
> @ build-failed /gnu/store/bv30jxzjl7c5z3hk6pmywpzwwrn1hq5m-go-1.9rc1.drv - 1 builder for `/gnu/store/bv30jxzjl7c5z3hk6pmywpzwwrn1hq5m-go-1.9rc1.drv' failed with exit code 1
> guix build: error: build failed: build of `/gnu/store/bv30jxzjl7c5z3hk6pmywpzwwrn1hq5m-go-1.9rc1.drv' failed
> ------
> 

I can duplicate this with my WIP go on aarch64, with the same error,
with go-1.8.3.


-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]