[PATCH 0/3] WIP: Go-lang

[PATCH 0/3] WIP: Go-lang

[Efraim Flashner]

It seems there's a lot of interest around getting Go packaged in Guix, so I
figured I should post my work so far to the mailinglist. Currently go-1.4
does not have cgo enabled, the go binary keeps on trying to link against
libgcc_s.so.1 (from gcc:lib) and can't find it in the RUNPATH. Also, the
tests are disabled. Nix has a whole slew of changes to the test suite to
remove tests that call out to the internet and to change tests that
call /bin/{hostname,pwd} and others.

Go-1.4 is built using gccgo-4.9, and go-1.5 uses go-1.4 to build, so I'm
pretty confident that go-1.4 works, and since go-1.5 inherits from go-1.4
it should be working too.

I wanted to write the go-build-system before submitting, but everything I
know about go I learned while packaging it, so it doesn't seem fair to have
everyone wait while I prepare a build system that could be missing crucial
features. I still intend to work on the build system; I'm probably just
psyching myself out about it :).

Efraim Flashner (3):
  gnu: gccgo: Update to 4.9.
  gnu: Add go-1.4.
  gnu: Add go-1.5.

 gnu-system.am           |   1 +
 gnu/packages/gcc.scm    |   4 +-
 gnu/packages/golang.scm | 124 ++++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 127 insertions(+), 2 deletions(-)
 create mode 100644 gnu/packages/golang.scm

--
2.7.0.rc3

[PATCH 1/3] gnu: gccgo: Update to 4.9.

[Efraim Flashner]

* gnu/packages/gcc.scm (gccgo): Update to 4.9.
---
 gnu/packages/gcc.scm | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/gnu/packages/gcc.scm b/gnu/packages/gcc.scm
index 2848e43..e336967 100644
--- a/gnu/packages/gcc.scm
+++ b/gnu/packages/gcc.scm
@@ -440,8 +440,8 @@ using compilers other than GCC."
 (define-public gfortran-5
   (custom-gcc gcc-5 "gfortran" '("fortran")))
 
-(define-public gccgo-4.8
-  (custom-gcc gcc-4.8 "gccgo" '("go")
+(define-public gccgo-4.9
+  (custom-gcc gcc-4.9 "gccgo" '("go")
               ;; Suppress the separate "lib" output, because otherwise the
               ;; "lib" and "out" outputs would refer to each other, creating
               ;; a cyclic dependency.  <http://debbugs.gnu.org/18101>
-- 
2.7.0.rc3

[PATCH 2/3] gnu: Add go-1.4.

[Efraim Flashner]

* gnu/packages/golang.scm: New file.
* gnu-system.am (GNU_SYSTEM_MODULES): Add golang.scm.
---
 gnu-system.am           |  1 +
 gnu/packages/golang.scm | 94 +++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 95 insertions(+)
 create mode 100644 gnu/packages/golang.scm

diff --git a/gnu-system.am b/gnu-system.am
index 7105e13..1b5c1ed 100644
--- a/gnu-system.am
+++ b/gnu-system.am
@@ -137,6 +137,7 @@ GNU_SYSTEM_MODULES =				\
   gnu/packages/gnustep.scm			\
   gnu/packages/gnuzilla.scm			\
   gnu/packages/gnu-pw-mgr.scm			\
+  gnu/packages/golang.scm			\
   gnu/packages/gperf.scm			\
   gnu/packages/gprolog.scm			\
   gnu/packages/gps.scm				\
diff --git a/gnu/packages/golang.scm b/gnu/packages/golang.scm
new file mode 100644
index 0000000..cb1cbc1
--- /dev/null
+++ b/gnu/packages/golang.scm
@@ -0,0 +1,94 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2015 Efraim Flashner <efraim@flashner.co.il>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu packages golang)
+  #:use-module ((guix licenses) #:prefix license:)
+  #:use-module (guix utils)
+  #:use-module (guix download)
+  #:use-module (guix packages)
+  #:use-module (guix build-system gnu)
+  #:use-module (gnu packages gcc)
+  #:use-module (gnu packages perl))
+
+;; According to https://golang.org/doc/install/gccgo, gccgo-4.8.2 includes a
+;; complete go-1.1.2 implementation, gccgo-4.9 includes a complete go-1.2
+;; implementation, and gccgo-5 a complete implementation of go-1.4.  Ultimately
+;; we hope to build go-1.5 with a bootstrap process using gccgo-5.  As of
+;; go-1.5, go cannot be bootstrapped without go-1.4, so we need to use go-1.4 or
+;; gccgo-5.  Mips is not officially supported, but it should work if it is
+;; bootstrapped.
+
+(define-public go-1.4
+  (package
+    (name "go")
+    (version "1.4.3")
+    (source
+      (origin
+        (method url-fetch)
+        (uri (string-append "https://storage.googleapis.com/golang/go"
+                            version ".src.tar.gz"))
+        (sha256
+         (base32
+          "0na9yqilzpvq0bjndbibfp07wr796gf252y471cip10bbdqgqiwr"))))
+    (build-system gnu-build-system)
+    (outputs '("out" "doc" "tests"))
+    (arguments
+     `(#:phases
+       (modify-phases %standard-phases
+        (delete 'configure)
+        (add-after 'patch-generated-file-shebangs 'chdir
+          (lambda _ (chdir "src")))
+        (replace 'build
+          (let* ((bash   (assoc-ref %build-inputs "bash"))
+                 (gccgo  (assoc-ref %build-inputs "gccgo"))
+                 (output (assoc-ref %outputs "out")))
+            (setenv "CC" "gcc")
+            (setenv "GOROOT" (getcwd))
+            (setenv "GOROOT_BOOTSTRAP" gccgo)
+            (setenv "GOROOT_FINAL" output)
+            (setenv "CGO_ENABLED" "0")
+            (lambda _
+              ;; all.bash includes the tests, which fail because they require
+              ;; network access, and access to /bin
+              (zero? (system* (string-append bash "/bin/bash") "make.bash")))))
+        (replace 'install
+          (lambda* (#:key outputs #:allow-other-keys)
+            (let* ((output (assoc-ref outputs "out"))
+                   (docs   (assoc-ref outputs "doc"))
+                   (tests  (assoc-ref outputs "tests")))
+              (copy-recursively "../test" tests)
+              (delete-file-recursively "../test")
+              (copy-recursively "../api" (string-append docs "/api"))
+              (delete-file-recursively "../api")
+              (copy-recursively "../doc" (string-append docs "/doc"))
+              (delete-file-recursively "../doc")
+              (copy-recursively "../" output)))))
+       #:tests? #f))
+    (native-inputs
+     `(("gccgo" ,gccgo-4.9)
+       ("perl" ,perl)))
+    (home-page "https://golang.org/")
+    (synopsis "Compiled, statically typed language developed by Google")
+    (description "Go, also commonly referred to as golang, is a programming
+ language developed at Google.  Designed primarily for systems programming, it
+ is a compiled, statically typed language in the tradition of C and C++, with
+garbage collection, various safety features and CSP-style concurrent programming
+features added.")
+    (license license:bsd-3)))
+
+(define-public go go-1.4)
-- 
2.7.0.rc3

[PATCH 3/3] gnu: Add go-1.5.

[Efraim Flashner]

* gnu/packages/golang.scm (go-1.5): New variable.
---
 gnu/packages/golang.scm | 32 +++++++++++++++++++++++++++++++-
 1 file changed, 31 insertions(+), 1 deletion(-)

diff --git a/gnu/packages/golang.scm b/gnu/packages/golang.scm
index cb1cbc1..6f281cf 100644
--- a/gnu/packages/golang.scm
+++ b/gnu/packages/golang.scm
@@ -91,4 +91,34 @@ garbage collection, various safety features and CSP-style concurrent programming
 features added.")
     (license license:bsd-3)))
 
-(define-public go go-1.4)
+(define-public go-1.5
+  (package (inherit go-1.4)
+    (version "1.5.2")
+    (source
+      (origin
+        (method url-fetch)
+        (uri (string-append "https://storage.googleapis.com/golang/go"
+                            version ".src.tar.gz"))
+        (sha256
+         (base32
+          "0x3sk32ym93hnc0yk6bnra226f92qvixia6kwcf68q84q0jddpgk"))))
+    (arguments
+     (substitute-keyword-arguments (package-arguments go-1.4)
+       ((#:phases phases)
+        `(modify-phases ,phases
+         (replace 'build
+          (let* ((bash   (assoc-ref %build-inputs "bash"))
+                 (go14   (assoc-ref %build-inputs "go-1.4"))
+                 (output (assoc-ref %outputs "out")))
+            (setenv "CC" "gcc")
+            (setenv "GOPATH" (string-append (getcwd) "/go"))
+            (setenv "GOROOT_BOOTSTRAP" go14)
+            (setenv "GOROOT_FINAL" output)
+            (lambda _
+              (zero?
+                (system* (string-append bash "/bin/bash") "make.bash")))))))))
+    (native-inputs
+     `(("go-1.4" ,go-1.4)
+       ("perl" ,perl)))))
+
+(define-public go go-1.5)
-- 
2.7.0.rc3

Re: [PATCH 3/3] gnu: Add go-1.5.

[Ricardo Wurmus]

Efraim Flashner <efraim@flashner.co.il> writes:

> * gnu/packages/golang.scm (go-1.5): New variable.

[...]

> +    (arguments
> +     (substitute-keyword-arguments (package-arguments go-1.4)
> +       ((#:phases phases)
> +        `(modify-phases ,phases
> +         (replace 'build
> +          (let* ((bash   (assoc-ref %build-inputs "bash"))
> +                 (go14   (assoc-ref %build-inputs "go-1.4"))
> +                 (output (assoc-ref %outputs "out")))
> +            (setenv "CC" "gcc")
> +            (setenv "GOPATH" (string-append (getcwd) "/go"))
> +            (setenv "GOROOT_BOOTSTRAP" go14)
> +            (setenv "GOROOT_FINAL" output)
> +            (lambda _
> +              (zero?
> +                (system* (string-append bash "/bin/bash") "make.bash")))))))))

This looks weird.  Shouldn’t the “let” and the “setenv” be inside the
“lambda”?  (BTW: this is a perfect usecase for “M-x
paredit-convolute-sexp”, after placing point before “(zero?”.)

I think you could do this instead:

   `(modify-phases ,phases
      (replace 'build
        (lambda* (#:key inputs outputs #:allow-other-keys)
          (let ((go14   (assoc-ref inputs "go-1.4"))
                (output (assoc-ref outputs "out")))
            (setenv "CC" "gcc")
            (setenv "GOPATH" (string-append (getcwd) "/go"))
            (setenv "GOROOT_BOOTSTRAP" go14)
            (setenv "GOROOT_FINAL" output)
            (zero? (system* "bash" "make.bash"))))))

~~ Ricardo

Re: [PATCH 3/3] gnu: Add go-1.5.

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 2001 bytes --]

On Tue, 12 Jan 2016 21:25:37 +0100
Ricardo Wurmus <rekado@elephly.net> wrote:

> Efraim Flashner <efraim@flashner.co.il> writes:
> 
>  [...]  
> 
> [...]
> 
> > +    (arguments
> > +     (substitute-keyword-arguments (package-arguments go-1.4)
> > +       ((#:phases phases)
> > +        `(modify-phases ,phases
> > +         (replace 'build
> > +          (let* ((bash   (assoc-ref %build-inputs "bash"))
> > +                 (go14   (assoc-ref %build-inputs "go-1.4"))
> > +                 (output (assoc-ref %outputs "out")))
> > +            (setenv "CC" "gcc")
> > +            (setenv "GOPATH" (string-append (getcwd) "/go"))
> > +            (setenv "GOROOT_BOOTSTRAP" go14)
> > +            (setenv "GOROOT_FINAL" output)
> > +            (lambda _
> > +              (zero?
> > +                (system* (string-append bash "/bin/bash") "make.bash")))))))))  
> 
> This looks weird.  Shouldn’t the “let” and the “setenv” be inside the
> “lambda”?  (BTW: this is a perfect usecase for “M-x
> paredit-convolute-sexp”, after placing point before “(zero?”.)
> 
> I think you could do this instead:
> 
>    `(modify-phases ,phases
>       (replace 'build
>         (lambda* (#:key inputs outputs #:allow-other-keys)
>           (let ((go14   (assoc-ref inputs "go-1.4"))
>                 (output (assoc-ref outputs "out")))
>             (setenv "CC" "gcc")
>             (setenv "GOPATH" (string-append (getcwd) "/go"))
>             (setenv "GOROOT_BOOTSTRAP" go14)
>             (setenv "GOROOT_FINAL" output)
>             (zero? (system* "bash" "make.bash"))))))
> 
> ~~ Ricardo
> 

This looks better than before, and having the lambda earlier is
better/safer.  I've applied the same change to go-1.4 also.

-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [PATCH 0/3] WIP: Go-lang

[Ludovic Courtès]

Efraim Flashner <efraim@flashner.co.il> skribis:

> It seems there's a lot of interest around getting Go packaged in Guix, so I
> figured I should post my work so far to the mailinglist. Currently go-1.4
> does not have cgo enabled, the go binary keeps on trying to link against
> libgcc_s.so.1 (from gcc:lib) and can't find it in the RUNPATH.

In GCC, we have a hack that automatically adds gcc:lib to the RUNPATH
(look for “libgcc_s” in gcc.scm.)  I guess we need to do something
similar here.

> Also, the tests are disabled. Nix has a whole slew of changes to the
> test suite to remove tests that call out to the internet and to change
> tests that call /bin/{hostname,pwd} and others.

OTOH, unless I’m mistaken, Nixpkgs does not run the test suite of Go
(and of most packages; specifically, ‘stdenv.mkDerivation’ has a
‘doCheck’ parameter, which is unset by default, meaning that the ‘check’
phase is skipped unless ‘doCheck = true’ is explicitly given.)

> Go-1.4 is built using gccgo-4.9, and go-1.5 uses go-1.4 to build, so I'm
> pretty confident that go-1.4 works, and since go-1.5 inherits from go-1.4
> it should be working too.

Nice!

Thanks for all the work!

Ludo’.

Re: [PATCH 1/3] gnu: gccgo: Update to 4.9.

[Ludovic Courtès]

Efraim Flashner <efraim@flashner.co.il> skribis:

> * gnu/packages/gcc.scm (gccgo): Update to 4.9.

OK.

Ludo'.

Re: [PATCH 2/3] gnu: Add go-1.4.

[Ludovic Courtès]

Efraim Flashner <efraim@flashner.co.il> skribis:

> * gnu/packages/golang.scm: New file.
> * gnu-system.am (GNU_SYSTEM_MODULES): Add golang.scm.

[...]

> +        (replace 'build
> +          (let* ((bash   (assoc-ref %build-inputs "bash"))
> +                 (gccgo  (assoc-ref %build-inputs "gccgo"))
> +                 (output (assoc-ref %outputs "out")))
> +            (setenv "CC" "gcc")
> +            (setenv "GOROOT" (getcwd))
> +            (setenv "GOROOT_BOOTSTRAP" gccgo)
> +            (setenv "GOROOT_FINAL" output)
> +            (setenv "CGO_ENABLED" "0")
> +            (lambda _
> +              ;; all.bash includes the tests, which fail because they require
> +              ;; network access, and access to /bin
> +              (zero? (system* (string-append bash "/bin/bash") "make.bash")))))

Why not:

  (replace 'build
    (lambda* (#:key inputs #:allow-other-keys)
      (let ((bash …) …)
        …
        (zero? (system* …)))))

?

> +    (synopsis "Compiled, statically typed language developed by Google")
> +    (description "Go, also commonly referred to as golang, is a programming
> + language developed at Google.  Designed primarily for systems programming, it
> + is a compiled, statically typed language in the tradition of C and C++, with
> +garbage collection, various safety features and CSP-style concurrent programming
> +features added.")

I would remove “developed by Google” and focus on the technical
characteristics.

Also, what’s CSP-style?  :-)

Thanks!

Ludo’.

Re: [PATCH 3/3] gnu: Add go-1.5.

[Ludovic Courtès]

Efraim Flashner <efraim@flashner.co.il> skribis:

> * gnu/packages/golang.scm (go-1.5): New variable.

I agree with Ricardo’s comment, nothing to add!

Thanks,
Ludo’.

Re: [PATCH 0/3] WIP: Go-lang

[Jeff Mickey]

* Ludovic Courtès <ludo@gnu.org> [2016-01-14 07:08]:
> Efraim Flashner <efraim@flashner.co.il> skribis:
>
>> It seems there's a lot of interest around getting Go packaged in Guix, so I
>> figured I should post my work so far to the mailinglist. Currently go-1.4
>> does not have cgo enabled, the go binary keeps on trying to link against
>> libgcc_s.so.1 (from gcc:lib) and can't find it in the RUNPATH.
>
> In GCC, we have a hack that automatically adds gcc:lib to the RUNPATH
> (look for “libgcc_s” in gcc.scm.)  I guess we need to do something
> similar here.

Aha - ok, I've had the same problem! I've tried injecting it many ways
and it doesn't work due to the intersection of libgcc_s and glibc.

Efraim - can you successfully run programs that link against glibc? As
in anything that uses os/user, net, etc? Running the tests as part of
your build is a big way to pick up on this, I've added the following to
my go-1.4.3 attempts in my build steps:

               
 ;; these all have specific file locations they look for
 ;; or in the case of exec_test resets the environment
 ;; before executing binaries they expect.
 (for-each delete-file '("src/net/multicast_test.go"
                                "src/net/parse_test.go"
                                "src/net/port_test.go"
                                "src/os/exec/exec_test.go"))
 (substitute* "src/os/os_test.go"
   (("/bin/pwd") (which "pwd"))
   (("/usr/bin") (dirname (which "pwd")))
   (("TestHostname") "areturn"))
 
 ;; Disable the unix socket test
 (substitute* "src/net/net_test.go"
   (("TestShutdownUnix") "areturn"))
 
 (substitute* "src/net/lookup_unix.go"
   (("/etc/protocols") (string-append iana "protocols")))
 
 ;; ParseInLocation fails the test
 (substitute* "src/time/format_test.go"
   (("TestParseInSydney") "areturn"))
 
 (substitute* "src/time/zoneinfo_unix.go"
   (("/usr/share/zoneinfo/") tz))
 
 ;; exec.Command on os.Args[0] from go run for whatever
 ;; reason doesn't work right now. libgcc_s.so link
 ;; missing crap occurs here as well, this may require
 ;; that 6l-wrapper for go run to work.
 (substitute* "src/syscall/syscall_unix_test.go"
   (("TestPassFD") "areturn"))

 (substitute* "src/cmd/6l/asm.c"
   (("char linuxdynld.*$") (string-append "char linuxdynld[] = \"" ld-linux "\";\n")))

I'll be trying this later with your packages.

I'd really hesitate on packaging this without runnning the tests, as
there are lots of ugly corner cases in go's build system.

  //  codemac

Re: [PATCH 0/3] WIP: Go-lang

[Jeff Mickey]

* Efraim Flashner <efraim@flashner.co.il> [2016-01-12 08:31]:
> Go-1.4 is built using gccgo-4.9, and go-1.5 uses go-1.4 to build, so I'm
> pretty confident that go-1.4 works, and since go-1.5 inherits from go-1.4
> it should be working too.

I'm a bit confused - go-1.4 is not a bootstrapped release, and requires
a working C compiler. Exactly what go code is being built with go-1.4?

And isn't anything before gcc-5 building a different go runtime than the
standard compilers due to gccgo's lack of support for segmented
stacks. I would not suggest running much go code with pre-gcc-5

  //  codemac

Re: [PATCH 2/3] gnu: Add go-1.4.

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 2313 bytes --]

On Thu, 14 Jan 2016 16:12:35 +0100
ludo@gnu.org (Ludovic Courtès) wrote:

> Efraim Flashner <efraim@flashner.co.il> skribis:
> 
>  [...]  
> 
> [...]
> 
> > +        (replace 'build
> > +          (let* ((bash   (assoc-ref %build-inputs "bash"))
> > +                 (gccgo  (assoc-ref %build-inputs "gccgo"))
> > +                 (output (assoc-ref %outputs "out")))
> > +            (setenv "CC" "gcc")
> > +            (setenv "GOROOT" (getcwd))
> > +            (setenv "GOROOT_BOOTSTRAP" gccgo)
> > +            (setenv "GOROOT_FINAL" output)
> > +            (setenv "CGO_ENABLED" "0")
> > +            (lambda _
> > +              ;; all.bash includes the tests, which fail because they require
> > +              ;; network access, and access to /bin
> > +              (zero? (system* (string-append bash "/bin/bash") "make.bash")))))  
> 
> Why not:
> 
>   (replace 'build
>     (lambda* (#:key inputs #:allow-other-keys)
>       (let ((bash …) …)
>         …
>         (zero? (system* …)))))
> 
> ?

I got that one the same time I changed go-1.5 :)

> > +    (synopsis "Compiled, statically typed language developed by Google")
> > +    (description "Go, also commonly referred to as golang, is a programming
> > + language developed at Google.  Designed primarily for systems programming, it
> > + is a compiled, statically typed language in the tradition of C and C++, with
> > +garbage collection, various safety features and CSP-style concurrent programming
> > +features added.")  
> 
> I would remove “developed by Google” and focus on the technical
> characteristics.
> 
> Also, what’s CSP-style?  :-)

I have no idea :) I scanned golang.org but didn't see anything that looked
usable, so I hit up Wikipedia and that's what it spit out. It seems wikipedia
says: https://en.wikipedia.org/wiki/Communicating_sequential_processes "In
computer science, communicating sequential processes (CSP) is a formal
language for describing patterns of interaction in concurrent systems."

> 
> Thanks!
> 
> Ludo’.

:)

-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [PATCH 0/3] WIP: Go-lang

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 1214 bytes --]

On Thu, 14 Jan 2016 11:17:59 -0800
Jeff Mickey <j@codemac.net> wrote:

> * Efraim Flashner <efraim@flashner.co.il> [2016-01-12 08:31]:
> > Go-1.4 is built using gccgo-4.9, and go-1.5 uses go-1.4 to build, so I'm
> > pretty confident that go-1.4 works, and since go-1.5 inherits from go-1.4
> > it should be working too.  
> 
> I'm a bit confused - go-1.4 is not a bootstrapped release, and requires
> a working C compiler. Exactly what go code is being built with go-1.4?
> 

Reading the google group for go, it looks like go-1.5 uses go-1.4 to
bootstrap itself, and then it uses that to rebuild itself for a final copy. I
was under the impression that the go binary/compiler worked by itself and
only needed the C compiler to compile the go binary.

> And isn't anything before gcc-5 building a different go runtime than the
> standard compilers due to gccgo's lack of support for segmented
> stacks. I would not suggest running much go code with pre-gcc-5
> 
>   //  codemac


-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [PATCH 0/3] WIP: Go-lang

[Leo Famulari]

On Thu, Jan 14, 2016 at 04:08:53PM +0100, Ludovic Courtès wrote:
> Efraim Flashner <efraim@flashner.co.il> skribis:
> 
> > It seems there's a lot of interest around getting Go packaged in Guix, so I
> > figured I should post my work so far to the mailinglist. Currently go-1.4
> > does not have cgo enabled, the go binary keeps on trying to link against
> > libgcc_s.so.1 (from gcc:lib) and can't find it in the RUNPATH.
> 
> In GCC, we have a hack that automatically adds gcc:lib to the RUNPATH
> (look for “libgcc_s” in gcc.scm.)  I guess we need to do something
> similar here.
> 
> > Also, the tests are disabled. Nix has a whole slew of changes to the
> > test suite to remove tests that call out to the internet and to change
> > tests that call /bin/{hostname,pwd} and others.
> 
> OTOH, unless I’m mistaken, Nixpkgs does not run the test suite of Go
> (and of most packages; specifically, ‘stdenv.mkDerivation’ has a
> ‘doCheck’ parameter, which is unset by default, meaning that the ‘check’
> phase is skipped unless ‘doCheck = true’ is explicitly given.)

Wow, this really clears some things up for me! I often refer to nixpkgs
when I get stuck and all this time I wondered why their build logs
didn't mention the tests very often.

> 
> > Go-1.4 is built using gccgo-4.9, and go-1.5 uses go-1.4 to build, so I'm
> > pretty confident that go-1.4 works, and since go-1.5 inherits from go-1.4
> > it should be working too.
> 
> Nice!
> 
> Thanks for all the work!
> 
> Ludo’.
> 

Re: [PATCH 0/3] WIP: Go-lang

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 5314 bytes --]

On Thu, 14 Jan 2016 11:14:50 -0800
Jeff Mickey <j@codemac.net> wrote:

> * Ludovic Courtès <ludo@gnu.org> [2016-01-14 07:08]:
> > Efraim Flashner <efraim@flashner.co.il> skribis:
> >  
> >> It seems there's a lot of interest around getting Go packaged in Guix, so I
> >> figured I should post my work so far to the mailinglist. Currently go-1.4
> >> does not have cgo enabled, the go binary keeps on trying to link against
> >> libgcc_s.so.1 (from gcc:lib) and can't find it in the RUNPATH.  
> >
> > In GCC, we have a hack that automatically adds gcc:lib to the RUNPATH
> > (look for “libgcc_s” in gcc.scm.)  I guess we need to do something
> > similar here.  
> 
> Aha - ok, I've had the same problem! I've tried injecting it many ways
> and it doesn't work due to the intersection of libgcc_s and glibc.
> 
> Efraim - can you successfully run programs that link against glibc? As
> in anything that uses os/user, net, etc? Running the tests as part of
> your build is a big way to pick up on this, I've added the following to
> my go-1.4.3 attempts in my build steps:
> 

I haven't ever touched go before, trying to get it packaged was my first time
looking at it so I have no way to check.

I tried looking at Nix's go-1.4 package for inspiration, and I see they have
a lot of sed-magic to remove or modify tests that don't like being sandboxed.
FreeBSD's ports system was also great for a template.
https://svnweb.freebsd.org/ports/head/lang/go14/Makefile?view=markup

https://github.com/NixOS/nixpkgs/blob/788800e437c4a0a25d95e217540bded68804b25e/pkgs/development/compilers/go/1.4.nix

    # Disabling the 'os/http/net' tests (they want files not available in
    # chroot builds)
    rm src/net/{multicast_test.go,parse_test.go,port_test.go}
    # !!! substituteInPlace does not seems to be effective.
    # The os test wants to read files in an existing path. Just don't let it be /usr/bin.
    sed -i 's,/usr/bin,'"`pwd`", src/os/os_test.go
    sed -i 's,/bin/pwd,'"`type -P pwd`", src/os/os_test.go
    # Disable the unix socket test
    sed -i '/TestShutdownUnix/areturn' src/net/net_test.go
    # Disable the hostname test
    sed -i '/TestHostname/areturn' src/os/os_test.go
    # ParseInLocation fails the test
    sed -i '/TestParseInSydney/areturn' src/time/format_test.go
    sed -i 's,/etc/protocols,${iana_etc}/etc/protocols,' src/net/lookup_unix.go
  '' + lib.optionalString stdenv.isLinux ''
    sed -i 's,/usr/share/zoneinfo/,${tzdata}/share/zoneinfo/,' src/time/zoneinfo_unix.go
    # Find the loader dynamically
    LOADER="$(find ${libc}/lib -name ld-linux\* | head -n 1)"
    # Replace references to the loader
    find src/cmd -name asm.c -exec sed -i "s,/lib/ld-linux.*\.so\.[0-9],$LOADER," {} \;
  '' + lib.optionalString stdenv.isDarwin ''
    sed -i 's,"/etc","'"$TMPDIR"'",' src/os/os_test.go
    sed -i 's,/_go_os_test,'"$TMPDIR"'/_go_os_test,' src/os/path_test.go
    sed -i '/TestRead0/areturn' src/os/os_test.go
    sed -i '/TestSystemRoots/areturn' src/crypto/x509/root_darwin_test.go
    sed -i '/TestDialDualStackLocalhost/areturn' src/net/dial_test.go
    # remove IP resolving tests, on darwin they can find fe80::1%lo while expecting ::1
    sed -i '/TestResolveIPAddr/areturn' src/net/ipraw_test.go
    sed -i '/TestResolveTCPAddr/areturn' src/net/tcp_test.go
    sed -i '/TestResolveUDPAddr/areturn' src/net/udp_test.go


>                
>  ;; these all have specific file locations they look for
>  ;; or in the case of exec_test resets the environment
>  ;; before executing binaries they expect.
>  (for-each delete-file '("src/net/multicast_test.go"
>                                 "src/net/parse_test.go"
>                                 "src/net/port_test.go"
>                                 "src/os/exec/exec_test.go"))
>  (substitute* "src/os/os_test.go"
>    (("/bin/pwd") (which "pwd"))
>    (("/usr/bin") (dirname (which "pwd")))
>    (("TestHostname") "areturn"))
>  
>  ;; Disable the unix socket test
>  (substitute* "src/net/net_test.go"
>    (("TestShutdownUnix") "areturn"))
>  
>  (substitute* "src/net/lookup_unix.go"
>    (("/etc/protocols") (string-append iana "protocols")))
>  
>  ;; ParseInLocation fails the test
>  (substitute* "src/time/format_test.go"
>    (("TestParseInSydney") "areturn"))
>  
>  (substitute* "src/time/zoneinfo_unix.go"
>    (("/usr/share/zoneinfo/") tz))
>  
>  ;; exec.Command on os.Args[0] from go run for whatever
>  ;; reason doesn't work right now. libgcc_s.so link
>  ;; missing crap occurs here as well, this may require
>  ;; that 6l-wrapper for go run to work.
>  (substitute* "src/syscall/syscall_unix_test.go"
>    (("TestPassFD") "areturn"))
> 
>  (substitute* "src/cmd/6l/asm.c"
>    (("char linuxdynld.*$") (string-append "char linuxdynld[] = \"" ld-linux "\";\n")))
> 
> I'll be trying this later with your packages.
> 
> I'd really hesitate on packaging this without runnning the tests, as
> there are lots of ugly corner cases in go's build system.
> 
>   //  codemac


-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [PATCH 0/3] WIP: Go-lang

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 1770 bytes --]

On Thu, 14 Jan 2016 16:08:53 +0100
ludo@gnu.org (Ludovic Courtès) wrote:

> Efraim Flashner <efraim@flashner.co.il> skribis:
> 
> > It seems there's a lot of interest around getting Go packaged in Guix, so I
> > figured I should post my work so far to the mailinglist. Currently go-1.4
> > does not have cgo enabled, the go binary keeps on trying to link against
> > libgcc_s.so.1 (from gcc:lib) and can't find it in the RUNPATH.  
> 
> In GCC, we have a hack that automatically adds gcc:lib to the RUNPATH
> (look for “libgcc_s” in gcc.scm.)  I guess we need to do something
> similar here.

I'll take a look at that. That should also help for gccgo-5.

> 
> > Also, the tests are disabled. Nix has a whole slew of changes to the
> > test suite to remove tests that call out to the internet and to change
> > tests that call /bin/{hostname,pwd} and others.  
> 
> OTOH, unless I’m mistaken, Nixpkgs does not run the test suite of Go
> (and of most packages; specifically, ‘stdenv.mkDerivation’ has a
> ‘doCheck’ parameter, which is unset by default, meaning that the ‘check’
> phase is skipped unless ‘doCheck = true’ is explicitly given.)

They do actually run the tests, they call the build with `./all.bash` which builds and then runs the tests.

> 
> > Go-1.4 is built using gccgo-4.9, and go-1.5 uses go-1.4 to build, so I'm
> > pretty confident that go-1.4 works, and since go-1.5 inherits from go-1.4
> > it should be working too.  
> 
> Nice!
> 
> Thanks for all the work!
> 
> Ludo’.

-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [PATCH 0/3] WIP: Go-lang

[Jeff Mickey]

* Efraim Flashner <efraim@flashner.co.il> [2016-01-14 13:37]:
> On Thu, 14 Jan 2016 11:17:59 -0800
> Jeff Mickey <j@codemac.net> wrote:
>
>> * Efraim Flashner <efraim@flashner.co.il> [2016-01-12 08:31]:
>> > Go-1.4 is built using gccgo-4.9, and go-1.5 uses go-1.4 to build, so I'm
>> > pretty confident that go-1.4 works, and since go-1.5 inherits from go-1.4
>> > it should be working too.  
>> 
>> I'm a bit confused - go-1.4 is not a bootstrapped release, and requires
>> a working C compiler. Exactly what go code is being built with go-1.4?
>> 
>
> Reading the google group for go, it looks like go-1.5 uses go-1.4 to
> bootstrap itself, and then it uses that to rebuild itself for a final copy. I
> was under the impression that the go binary/compiler worked by itself and
> only needed the C compiler to compile the go binary.

That's my understanding as well, in your first reply you say "Go-1.4 is
built using gccgo-4.9" which is not true, it doesn't use a bootstrap
go. It uses gcc and it's own C compiler (8g/6g for x86/x86_64). go-1.5
then uses go-1.4 to bootstrap, which ideally could use gccgo-5 instead
of go-1.4, but unfortunately runs into the same libgcc_s issues with
glibc linking.

  //  codemac

Re: [PATCH 2/3] gnu: Add go-1.4.

[Ludovic Courtès]

Efraim Flashner <efraim@flashner.co.il> skribis:

> On Thu, 14 Jan 2016 16:12:35 +0100
> ludo@gnu.org (Ludovic Courtès) wrote:

[...]

>> Also, what’s CSP-style?  :-)
>
> I have no idea :) I scanned golang.org but didn't see anything that looked
> usable, so I hit up Wikipedia and that's what it spit out. It seems wikipedia
> says: https://en.wikipedia.org/wiki/Communicating_sequential_processes "In
> computer science, communicating sequential processes (CSP) is a formal
> language for describing patterns of interaction in concurrent systems."

Oh, interesting!  It might be worth replacing “CSP-style” in the
description with “in the style of communicating sequential processes
(CSP)”.

Thanks,
Ludo’.