* [bug#30579] Go 1.10
@ 2018-02-22 18:25 Leo Famulari
2018-02-22 18:35 ` [bug#30579] [PATCH] gnu: Add go@1.10 Leo Famulari
` (2 more replies)
0 siblings, 3 replies; 5+ messages in thread
From: Leo Famulari @ 2018-02-22 18:25 UTC (permalink / raw)
To: 30579
[-- Attachment #1: Type: text/plain, Size: 951 bytes --]
This patch adds Go 1.10 but leaves Go 1.9 as the default Go.
Go 1.10 changes the way that Go decides whether to re-use built objects
or to recompile them [0]. Reading the release notes, it seems to
implement something like a memoized cache for built objects and test
results.
Go 1.10 works with our go-build-system, but the built objects are not
re-used. Instead, either most or all required objects are rebuilt, so
the build system will need some adjustment.
I think it's useful to have this new version of Go packaged, especially
if somebody wants to decipher how the new compiler decides when to
re-use the built objects :)
Also, there is a data race found in the test suite when bootstrapping
with GCC 5, so this patch changes the entire Go stack to GCC 6. I'll
send a report upstream.
[0] The "build cache" is mentioned in the link below. The mechanism is
called "content-based staleness" in some places.
https://golang.org/doc/go1.10#build
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 5+ messages in thread
* [bug#30579] [PATCH] gnu: Add go@1.10.
2018-02-22 18:25 [bug#30579] Go 1.10 Leo Famulari
@ 2018-02-22 18:35 ` Leo Famulari
2018-02-26 1:14 ` Marius Bakke
2018-02-22 19:00 ` [bug#30579] Go 1.10 Leo Famulari
2018-02-26 1:15 ` Marius Bakke
2 siblings, 1 reply; 5+ messages in thread
From: Leo Famulari @ 2018-02-22 18:35 UTC (permalink / raw)
To: 30579
Building Go 1.10 with the Go 1.4 bootstrap, Thread Sanitizer from GCC 5
finds a data race during the the test suite of Go 1.10. With GCC 6, the
race doesn't seem to be present.
.##### ../misc/cgo/testsanitizers
--- FAIL: TestShared (0.03s)
--- FAIL: TestShared/tsan_shared (2.54s)
cshared_test.go:71: `/tmp/guix-build-go-1.10.drv-0/TestShared367937240/tsan_shared` exited with exit status 66
==================
WARNING: ThreadSanitizer: data race (pid=28778)
Write of size 8 at 0x7ffff5495c30 by thread T1:
#0 pthread_attr_getstacksize <null> (tsan_shared+0x00000041be82)
#1 x_cgo_init <null> (libtsan_shared.so+0x00000008aeaf)
#2 runtime.rt0_go /tmp/guix-build-go-1.10.drv-0/go/src/runtime/asm_amd64.s:199 (libtsan_shared.so+0x000000081586)
Previous write of size 8 at 0x7ffff5495c30 by main thread:
#0 <null> <null> (0x000000000001)
Location is stack of thread T1.
Thread T1 (tid=28786, running) created by main thread at:
#0 pthread_create <null> (tsan_shared+0x00000040c794)
#1 _cgo_try_pthread_create <null> (libtsan_shared.so+0x00000008ad70)
#2 x_cgo_sys_thread_create <null> (libtsan_shared.so+0x00000008ab12)
#3 _rt0_amd64_lib /tmp/guix-build-go-1.10.drv-0/go/src/runtime/asm_amd64.s:56 (libtsan_shared.so+0x0000000813b6)
SUMMARY: ThreadSanitizer: data race ??:0 __interceptor_pthread_attr_getstacksize
* gnu/packages/golang.scm (go-1.10): New variable.
(go-1.4)[native-inputs]: Use gcc:lib version 6.
---
gnu/packages/golang.scm | 115 +++++++++++++++++++++++++++++++++++++++++++++++-
1 file changed, 114 insertions(+), 1 deletion(-)
diff --git a/gnu/packages/golang.scm b/gnu/packages/golang.scm
index f11cc76d9..a542eaba8 100644
--- a/gnu/packages/golang.scm
+++ b/gnu/packages/golang.scm
@@ -187,7 +187,7 @@
(inputs
`(("tzdata" ,tzdata)
("pcre" ,pcre)
- ("gcc:lib" ,gcc "lib")))
+ ("gcc:lib" ,gcc-6 "lib")))
(native-inputs
`(("pkg-config" ,%pkg-config)
("which" ,which)
@@ -372,6 +372,119 @@ in the style of communicating sequential processes (@dfn{CSP}).")
,@(package-native-inputs go-1.4)))
(supported-systems %supported-systems)))
+(define-public go-1.10
+ (package
+ (inherit go-1.9)
+ (name "go")
+ (version "1.10")
+ (source
+ (origin
+ (method url-fetch)
+ (uri (string-append "https://storage.googleapis.com/golang/"
+ name version ".src.tar.gz"))
+ (sha256
+ (base32
+ "093z1h0gmi458kv7smpx0ph6jm7ss9mzxa432kysbz85jhl4kppk"))))
+ (arguments
+ (substitute-keyword-arguments (package-arguments go-1.9)
+ ((#:phases phases)
+ `(modify-phases ,phases
+ (replace 'prebuild
+ (lambda* (#:key inputs outputs #:allow-other-keys)
+ (let* ((gcclib (string-append (assoc-ref inputs "gcc:lib") "/lib"))
+ (ld (string-append (assoc-ref inputs "libc") "/lib"))
+ (loader (car (find-files ld "^ld-linux.+")))
+ (net-base (assoc-ref inputs "net-base"))
+ (tzdata-path
+ (string-append (assoc-ref inputs "tzdata") "/share/zoneinfo"))
+ (output (assoc-ref outputs "out")))
+
+ ;; Removing net/ tests, which fail when attempting to access
+ ;; network resources not present in the build container.
+ (for-each delete-file
+ '("net/listen_test.go"
+ "net/parse_test.go"
+ "net/cgo_unix_test.go"))
+
+ (substitute* "os/os_test.go"
+ (("/usr/bin") (getcwd))
+ (("/bin/pwd") (which "pwd"))
+ (("/bin/sh") (which "sh")))
+
+ ;; Add libgcc to runpath
+ (substitute* "cmd/link/internal/ld/lib.go"
+ (("!rpath.set") "true"))
+ (substitute* "cmd/go/internal/work/gccgo.go"
+ (("cgoldflags := \\[\\]string\\{\\}")
+ (string-append "cgoldflags := []string{"
+ "\"-rpath=" gcclib "\""
+ "}"))
+ (("\"-lgcc_s\", ")
+ (string-append
+ "\"-Wl,-rpath=" gcclib "\", \"-lgcc_s\", ")))
+ (substitute* "cmd/go/internal/work/gc.go"
+ (("ldflags = setextld\\(ldflags, compiler\\)")
+ (string-append
+ "ldflags = setextld(ldflags, compiler)\n"
+ "ldflags = append(ldflags, \"-r\")\n"
+ "ldflags = append(ldflags, \"" gcclib "\")\n")))
+
+ ;; Disable failing tests: these tests attempt to access
+ ;; commands or network resources which are neither available
+ ;; nor necessary for the build to succeed.
+ (for-each
+ (match-lambda
+ ((file regex)
+ (substitute* file
+ ((regex all before test_name)
+ (string-append before "Disabled" test_name)))))
+ '(("net/net_test.go" "(.+)(TestShutdownUnix.+)")
+ ("net/dial_test.go" "(.+)(TestDialTimeout.+)")
+ ("os/os_test.go" "(.+)(TestHostname.+)")
+ ("time/format_test.go" "(.+)(TestParseInSydney.+)")
+ ("time/format_test.go" "(.+)(TestParseInLocation.+)")
+ ("os/exec/exec_test.go" "(.+)(TestEcho.+)")
+ ("os/exec/exec_test.go" "(.+)(TestCommandRelativeName.+)")
+ ("os/exec/exec_test.go" "(.+)(TestCatStdin.+)")
+ ("os/exec/exec_test.go" "(.+)(TestCatGoodAndBadFile.+)")
+ ("os/exec/exec_test.go" "(.+)(TestExitStatus.+)")
+ ("os/exec/exec_test.go" "(.+)(TestPipes.+)")
+ ("os/exec/exec_test.go" "(.+)(TestStdinClose.+)")
+ ("os/exec/exec_test.go" "(.+)(TestIgnorePipeErrorOnSuccess.+)")
+ ("syscall/syscall_unix_test.go" "(.+)(TestPassFD\\(.+)")
+ ("os/exec/exec_test.go" "(.+)(TestExtraFiles/areturn.+)")
+ ("cmd/go/go_test.go" "(.+)(TestCoverageWithCgo.+)")
+ ("os/exec/exec_test.go" "(.+)(TestOutputStderrCapture.+)")
+ ("os/exec/exec_test.go" "(.+)(TestExtraFiles.+)")
+ ("os/exec/exec_test.go" "(.+)(TestExtraFilesRace.+)")
+ ("net/lookup_test.go" "(.+)(TestLookupPort.+)")
+ ("syscall/exec_linux_test.go"
+ "(.+)(TestCloneNEWUSERAndRemapNoRootDisableSetgroups.+)")))
+
+ ;; fix shebang for testar script
+ ;; note the target script is generated at build time.
+ (substitute* "../misc/cgo/testcarchive/carchive_test.go"
+ (("#!/usr/bin/env") (string-append "#!" (which "env"))))
+
+ (substitute* "net/lookup_unix.go"
+ (("/etc/protocols") (string-append net-base "/etc/protocols")))
+ (substitute* "net/port_unix.go"
+ (("/etc/services") (string-append net-base "/etc/services")))
+ (substitute* "time/zoneinfo_unix.go"
+ (("/usr/share/zoneinfo/") tzdata-path))
+ (substitute* (find-files "cmd" "\\.go")
+ (("/lib(64)?/ld-linux.*\\.so\\.[0-9]") loader))
+ #t)))
+ (replace 'set-bootstrap-variables
+ (lambda* (#:key outputs inputs #:allow-other-keys)
+ ;; Tell the build system where to find the bootstrap Go.
+ (let ((go (assoc-ref inputs "go")))
+ (setenv "GOROOT_BOOTSTRAP" go)
+ (setenv "GOGC" "400")
+ ;; Go 1.10 tries to write to $HOME in a test
+ (setenv "HOME" "/tmp")
+ #t)))))))))
+
(define-public go go-1.9)
(define-public go-github-com-alsm-ioprogress
--
2.16.2
^ permalink raw reply related [flat|nested] 5+ messages in thread
* [bug#30579] Go 1.10
2018-02-22 18:25 [bug#30579] Go 1.10 Leo Famulari
2018-02-22 18:35 ` [bug#30579] [PATCH] gnu: Add go@1.10 Leo Famulari
@ 2018-02-22 19:00 ` Leo Famulari
2018-02-26 1:15 ` Marius Bakke
2 siblings, 0 replies; 5+ messages in thread
From: Leo Famulari @ 2018-02-22 19:00 UTC (permalink / raw)
To: 30579
[-- Attachment #1: Type: text/plain, Size: 296 bytes --]
On Thu, Feb 22, 2018 at 01:25:03PM -0500, Leo Famulari wrote:
> Also, there is a data race found in the test suite when bootstrapping
> with GCC 5, so this patch changes the entire Go stack to GCC 6. I'll
> send a report upstream.
Upstream bug report:
https://github.com/golang/go/issues/24046
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 5+ messages in thread
* [bug#30579] [PATCH] gnu: Add go@1.10.
2018-02-22 18:35 ` [bug#30579] [PATCH] gnu: Add go@1.10 Leo Famulari
@ 2018-02-26 1:14 ` Marius Bakke
0 siblings, 0 replies; 5+ messages in thread
From: Marius Bakke @ 2018-02-26 1:14 UTC (permalink / raw)
To: Leo Famulari, 30579
[-- Attachment #1: Type: text/plain, Size: 9123 bytes --]
Leo Famulari <leo@famulari.name> writes:
> Building Go 1.10 with the Go 1.4 bootstrap, Thread Sanitizer from GCC 5
> finds a data race during the the test suite of Go 1.10. With GCC 6, the
> race doesn't seem to be present.
Can you add this comment next to the GCC input? That way it's obvious
for people wondering about it, without having to grep the commit log.
Other than that LGTM.
>
> .##### ../misc/cgo/testsanitizers
> --- FAIL: TestShared (0.03s)
> --- FAIL: TestShared/tsan_shared (2.54s)
> cshared_test.go:71: `/tmp/guix-build-go-1.10.drv-0/TestShared367937240/tsan_shared` exited with exit status 66
> ==================
> WARNING: ThreadSanitizer: data race (pid=28778)
> Write of size 8 at 0x7ffff5495c30 by thread T1:
> #0 pthread_attr_getstacksize <null> (tsan_shared+0x00000041be82)
> #1 x_cgo_init <null> (libtsan_shared.so+0x00000008aeaf)
> #2 runtime.rt0_go /tmp/guix-build-go-1.10.drv-0/go/src/runtime/asm_amd64.s:199 (libtsan_shared.so+0x000000081586)
>
> Previous write of size 8 at 0x7ffff5495c30 by main thread:
> #0 <null> <null> (0x000000000001)
>
> Location is stack of thread T1.
>
> Thread T1 (tid=28786, running) created by main thread at:
> #0 pthread_create <null> (tsan_shared+0x00000040c794)
> #1 _cgo_try_pthread_create <null> (libtsan_shared.so+0x00000008ad70)
> #2 x_cgo_sys_thread_create <null> (libtsan_shared.so+0x00000008ab12)
> #3 _rt0_amd64_lib /tmp/guix-build-go-1.10.drv-0/go/src/runtime/asm_amd64.s:56 (libtsan_shared.so+0x0000000813b6)
>
> SUMMARY: ThreadSanitizer: data race ??:0 __interceptor_pthread_attr_getstacksize
>
> * gnu/packages/golang.scm (go-1.10): New variable.
> (go-1.4)[native-inputs]: Use gcc:lib version 6.
> ---
> gnu/packages/golang.scm | 115 +++++++++++++++++++++++++++++++++++++++++++++++-
> 1 file changed, 114 insertions(+), 1 deletion(-)
>
> diff --git a/gnu/packages/golang.scm b/gnu/packages/golang.scm
> index f11cc76d9..a542eaba8 100644
> --- a/gnu/packages/golang.scm
> +++ b/gnu/packages/golang.scm
> @@ -187,7 +187,7 @@
> (inputs
> `(("tzdata" ,tzdata)
> ("pcre" ,pcre)
> - ("gcc:lib" ,gcc "lib")))
> + ("gcc:lib" ,gcc-6 "lib")))
> (native-inputs
> `(("pkg-config" ,%pkg-config)
> ("which" ,which)
> @@ -372,6 +372,119 @@ in the style of communicating sequential processes (@dfn{CSP}).")
> ,@(package-native-inputs go-1.4)))
> (supported-systems %supported-systems)))
>
> +(define-public go-1.10
> + (package
> + (inherit go-1.9)
> + (name "go")
> + (version "1.10")
> + (source
> + (origin
> + (method url-fetch)
> + (uri (string-append "https://storage.googleapis.com/golang/"
> + name version ".src.tar.gz"))
> + (sha256
> + (base32
> + "093z1h0gmi458kv7smpx0ph6jm7ss9mzxa432kysbz85jhl4kppk"))))
> + (arguments
> + (substitute-keyword-arguments (package-arguments go-1.9)
> + ((#:phases phases)
> + `(modify-phases ,phases
> + (replace 'prebuild
> + (lambda* (#:key inputs outputs #:allow-other-keys)
> + (let* ((gcclib (string-append (assoc-ref inputs "gcc:lib") "/lib"))
> + (ld (string-append (assoc-ref inputs "libc") "/lib"))
> + (loader (car (find-files ld "^ld-linux.+")))
> + (net-base (assoc-ref inputs "net-base"))
> + (tzdata-path
> + (string-append (assoc-ref inputs "tzdata") "/share/zoneinfo"))
> + (output (assoc-ref outputs "out")))
> +
> + ;; Removing net/ tests, which fail when attempting to access
> + ;; network resources not present in the build container.
> + (for-each delete-file
> + '("net/listen_test.go"
> + "net/parse_test.go"
> + "net/cgo_unix_test.go"))
> +
> + (substitute* "os/os_test.go"
> + (("/usr/bin") (getcwd))
> + (("/bin/pwd") (which "pwd"))
> + (("/bin/sh") (which "sh")))
> +
> + ;; Add libgcc to runpath
> + (substitute* "cmd/link/internal/ld/lib.go"
> + (("!rpath.set") "true"))
> + (substitute* "cmd/go/internal/work/gccgo.go"
> + (("cgoldflags := \\[\\]string\\{\\}")
> + (string-append "cgoldflags := []string{"
> + "\"-rpath=" gcclib "\""
> + "}"))
> + (("\"-lgcc_s\", ")
> + (string-append
> + "\"-Wl,-rpath=" gcclib "\", \"-lgcc_s\", ")))
> + (substitute* "cmd/go/internal/work/gc.go"
> + (("ldflags = setextld\\(ldflags, compiler\\)")
> + (string-append
> + "ldflags = setextld(ldflags, compiler)\n"
> + "ldflags = append(ldflags, \"-r\")\n"
> + "ldflags = append(ldflags, \"" gcclib "\")\n")))
> +
> + ;; Disable failing tests: these tests attempt to access
> + ;; commands or network resources which are neither available
> + ;; nor necessary for the build to succeed.
> + (for-each
> + (match-lambda
> + ((file regex)
> + (substitute* file
> + ((regex all before test_name)
> + (string-append before "Disabled" test_name)))))
> + '(("net/net_test.go" "(.+)(TestShutdownUnix.+)")
> + ("net/dial_test.go" "(.+)(TestDialTimeout.+)")
> + ("os/os_test.go" "(.+)(TestHostname.+)")
> + ("time/format_test.go" "(.+)(TestParseInSydney.+)")
> + ("time/format_test.go" "(.+)(TestParseInLocation.+)")
> + ("os/exec/exec_test.go" "(.+)(TestEcho.+)")
> + ("os/exec/exec_test.go" "(.+)(TestCommandRelativeName.+)")
> + ("os/exec/exec_test.go" "(.+)(TestCatStdin.+)")
> + ("os/exec/exec_test.go" "(.+)(TestCatGoodAndBadFile.+)")
> + ("os/exec/exec_test.go" "(.+)(TestExitStatus.+)")
> + ("os/exec/exec_test.go" "(.+)(TestPipes.+)")
> + ("os/exec/exec_test.go" "(.+)(TestStdinClose.+)")
> + ("os/exec/exec_test.go" "(.+)(TestIgnorePipeErrorOnSuccess.+)")
> + ("syscall/syscall_unix_test.go" "(.+)(TestPassFD\\(.+)")
> + ("os/exec/exec_test.go" "(.+)(TestExtraFiles/areturn.+)")
> + ("cmd/go/go_test.go" "(.+)(TestCoverageWithCgo.+)")
> + ("os/exec/exec_test.go" "(.+)(TestOutputStderrCapture.+)")
> + ("os/exec/exec_test.go" "(.+)(TestExtraFiles.+)")
> + ("os/exec/exec_test.go" "(.+)(TestExtraFilesRace.+)")
> + ("net/lookup_test.go" "(.+)(TestLookupPort.+)")
> + ("syscall/exec_linux_test.go"
> + "(.+)(TestCloneNEWUSERAndRemapNoRootDisableSetgroups.+)")))
> +
> + ;; fix shebang for testar script
> + ;; note the target script is generated at build time.
> + (substitute* "../misc/cgo/testcarchive/carchive_test.go"
> + (("#!/usr/bin/env") (string-append "#!" (which "env"))))
> +
> + (substitute* "net/lookup_unix.go"
> + (("/etc/protocols") (string-append net-base "/etc/protocols")))
> + (substitute* "net/port_unix.go"
> + (("/etc/services") (string-append net-base "/etc/services")))
> + (substitute* "time/zoneinfo_unix.go"
> + (("/usr/share/zoneinfo/") tzdata-path))
> + (substitute* (find-files "cmd" "\\.go")
> + (("/lib(64)?/ld-linux.*\\.so\\.[0-9]") loader))
> + #t)))
> + (replace 'set-bootstrap-variables
> + (lambda* (#:key outputs inputs #:allow-other-keys)
> + ;; Tell the build system where to find the bootstrap Go.
> + (let ((go (assoc-ref inputs "go")))
> + (setenv "GOROOT_BOOTSTRAP" go)
> + (setenv "GOGC" "400")
> + ;; Go 1.10 tries to write to $HOME in a test
> + (setenv "HOME" "/tmp")
> + #t)))))))))
> +
> (define-public go go-1.9)
>
> (define-public go-github-com-alsm-ioprogress
> --
> 2.16.2
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]
^ permalink raw reply [flat|nested] 5+ messages in thread
* [bug#30579] Go 1.10
2018-02-22 18:25 [bug#30579] Go 1.10 Leo Famulari
2018-02-22 18:35 ` [bug#30579] [PATCH] gnu: Add go@1.10 Leo Famulari
2018-02-22 19:00 ` [bug#30579] Go 1.10 Leo Famulari
@ 2018-02-26 1:15 ` Marius Bakke
2 siblings, 0 replies; 5+ messages in thread
From: Marius Bakke @ 2018-02-26 1:15 UTC (permalink / raw)
To: Leo Famulari, 30579
[-- Attachment #1: Type: text/plain, Size: 1078 bytes --]
Leo Famulari <leo@famulari.name> writes:
> This patch adds Go 1.10 but leaves Go 1.9 as the default Go.
>
> Go 1.10 changes the way that Go decides whether to re-use built objects
> or to recompile them [0]. Reading the release notes, it seems to
> implement something like a memoized cache for built objects and test
> results.
>
> Go 1.10 works with our go-build-system, but the built objects are not
> re-used. Instead, either most or all required objects are rebuilt, so
> the build system will need some adjustment.
>
> I think it's useful to have this new version of Go packaged, especially
> if somebody wants to decipher how the new compiler decides when to
> re-use the built objects :)
Definitely. Thanks for working on it! :-)
>
> Also, there is a data race found in the test suite when bootstrapping
> with GCC 5, so this patch changes the entire Go stack to GCC 6. I'll
> send a report upstream.
>
> [0] The "build cache" is mentioned in the link below. The mechanism is
> called "content-based staleness" in some places.
> https://golang.org/doc/go1.10#build
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2018-02-26 1:16 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2018-02-22 18:25 [bug#30579] Go 1.10 Leo Famulari
2018-02-22 18:35 ` [bug#30579] [PATCH] gnu: Add go@1.10 Leo Famulari
2018-02-26 1:14 ` Marius Bakke
2018-02-22 19:00 ` [bug#30579] Go 1.10 Leo Famulari
2018-02-26 1:15 ` Marius Bakke
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/guix.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).