[bug#49019] [PATCH 0/1] gnu: Add gccgo-10.

[bug#49019] [PATCH 0/1] gnu: Add gccgo-10.

[Sarah Morgensen via Guix-patches via]

The primary goal for adding gccgo is to bootstrap the go compiler for arches
that aren't supported by go 1.4. I believe the major one to support here is
powerpc64le, but I am unsure of current arch usage. A secondary goal is to
provide a gccgo-toolchain, as gccgo may provide advantages over gc in time.

I have successfully built gccgo 9, 10 and 11 for x86-64. I additionally tested
gcc 10 using `make check-go`. There are two minor test failures:

1. index0-out.go, previously reported upstream [0]
2. pprof (TestConvertCPUProfile/TestConvertMemProfile)

(I have not yet been able to deterine whether the pprof failures are an upstream
bug or a Guix bug. Should I send details to bug-guix?)

Finally, the regex update in [1] is implemented in CUSTOM-GCC-GCCGO rather than
CUSTOM-GCC to avoid rebuilding 2k+ gfortran packages, and CUSTOM-GCC-GCCGO
should be removed after the change is in master.

[0] https://issues.guix.gnu.org/49010
[1] https://gcc.gnu.org/bugzilla/show_bug.cgi?id=87589

Sarah Morgensen (1):
  gnu: Add gccgo-10.

 gnu/packages/gcc.scm | 77 ++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 77 insertions(+)

-- 
2.31.1

[bug#49019] [PATCH 1/1] gnu: Add gccgo-10.

[Sarah Morgensen via Guix-patches via]

Generate gccgo with MAKE-GCCGO to factorize phases, and to fix the
cyclic dependency between out and lib (caused by libgo embedding the
gotools path) that was worked around in
<https://issues.guix.gnu.org/18101>.

* gnu/packages/gcc.scm (custom-gcc-gccgo): New procedure.
(make-gccgo): New procedure.
(gccgo-10): New variable.
---
 gnu/packages/gcc.scm | 77 ++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 77 insertions(+)

diff --git a/gnu/packages/gcc.scm b/gnu/packages/gcc.scm
index 1fd5710e57..df2ba729d8 100644
--- a/gnu/packages/gcc.scm
+++ b/gnu/packages/gcc.scm
@@ -11,6 +11,7 @@
 ;;; Copyright © 2020 Guy Fleury Iteriteka <gfleury@disroot.org>
 ;;; Copyright © 2020 Simon Tournier <zimon.toutoune@gmail.com>
 ;;; Copyright © 2021 Chris Marusich <cmmarusich@gmail.com>
+;;; Copyright © 2021 Sarah Morgensen <iskarian@mgsn.dev>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -744,6 +745,42 @@ as the 'native-search-paths' field."
                                      ".*(c\\+\\+|cpp|g\\+\\+|gcov|gcc|gcc-.*)"))
                #t))))))))
 
+(define* (custom-gcc-gccgo gcc name languages
+                           #:optional
+                           (search-paths (package-native-search-paths gcc))
+                           #:key (separate-lib-output? #t))
+  ;; TODO: remove CUSTOM-GCC-GCCGO when regex changes for CUSTOM-GCC are
+  ;; merged into master <https://issues.guix.gnu.org/49010>
+  "Return a custom version of GCC that supports LANGUAGES.  Use SEARCH-PATHS
+as the 'native-search-paths' field."
+  (package (inherit gcc)
+    (name name)
+    (outputs (if separate-lib-output?
+                 (package-outputs gcc)
+                 (delete "lib" (package-outputs gcc))))
+    (native-search-paths search-paths)
+    (properties (alist-delete 'hidden? (package-properties gcc)))
+    (arguments
+     (substitute-keyword-arguments (package-arguments gcc)
+       ((#:modules modules %gnu-build-system-modules)
+        `(,@modules
+          (srfi srfi-1)
+          (srfi srfi-26)
+          (ice-9 regex)))
+       ((#:configure-flags flags)
+        `(cons (string-append "--enable-languages="
+                              ,(string-join languages ","))
+               (remove (cut string-match "--enable-languages.*" <>)
+                       ,flags)))
+       ((#:phases phases)
+        `(modify-phases ,phases
+           (add-after 'install 'remove-broken-or-conflicting-files
+             (lambda* (#:key outputs #:allow-other-keys)
+               (for-each
+                delete-file
+                (find-files (string-append (assoc-ref outputs "out") "/bin")
+                            ".*(c\\+\\+|cpp|g\\+\\+|gcov|gcc|lto)(-.*)?$"))))))))))
+
 (define %generic-search-paths
   ;; This is the language-neutral search path for GCC.  Entries in $CPATH are
   ;; not considered "system headers", which means GCC can raise warnings for
@@ -813,6 +850,43 @@ It can also be used for ahead-of-time code generation for building standalone
 compilers.  The just-in-time (jit) part of the name is now something of a
 misnomer.")))
 
+(define (make-gccgo gcc)
+  "Return a gccgo package based on GCC."
+  (let ((gccgo (custom-gcc-gccgo gcc "gccgo" '("go") %generic-search-paths)))
+    (package
+      (inherit gccgo)
+      (synopsis "Go frontend to GCC")
+      (description
+        "This package is part of the GNU Compiler Collection and
+provides the GNU compiler for the Go programming language.")
+      (arguments
+       (substitute-keyword-arguments (package-arguments gccgo)
+         ((#:phases phases)
+          `(modify-phases ,phases
+             (add-after 'install 'wrap-go-with-tool-path
+               (lambda* (#:key outputs #:allow-other-keys)
+                 (let* ((out (assoc-ref outputs "out"))
+                        (exedir (string-append out "/libexec/gcc"))
+                        (tooldir (dirname (car (find-files exedir "^cgo$")))))
+                   (wrap-program (string-append out "/bin/go")
+                     `("GCCGOTOOLDIR" =
+                       (,(string-append "${GCCGOTOOLDIR-" tooldir "}")))
+                     `("GOROOT" =
+                       (,(string-append "${GOROOT-" out "}")))))))
+             (add-before 'configure 'fix-gotools-runpath
+               (lambda _
+                 (substitute* "gotools/Makefile.in"
+                   (("AM_LDFLAGS =" all)
+                    (string-append all " -Wl,-rpath=$(libdir) ")))))
+             (add-before 'configure 'remove-tool-reference-from-libgo
+               (lambda _
+                 (substitute* "libgo/Makefile.in"
+                   (("(GccgoToolDir = \\\")[^\\\"]+" _ start)
+                    (string-append start "/nonexistent"))
+                   (("(DefaultGoroot = \\\")[^\\\"]+" _ start)
+                    (string-append start "/nonexistent"))
+                   (("(defaultGOROOTValue.*?return `)[^`]+" _ start)
+                    (string-append start "/nonexistent"))))))))))))
 
 (define-public gccgo-4.9
   (custom-gcc (package
@@ -828,6 +902,9 @@ provides the GNU compiler for the Go programming language."))
               ;; a cyclic dependency.  <http://debbugs.gnu.org/18101>
               #:separate-lib-output? #f))
 
+(define-public gccgo-10
+  (make-gccgo gcc-10))
+
 (define %objc-search-paths
   (list (search-path-specification
          (variable "OBJC_INCLUDE_PATH")
-- 
2.31.1

bug#49019: [PATCH 1/1] gnu: Add gccgo-10.

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 319 bytes --]

Thank you for the patch! gccgo was one of the things that has taunted me
for a while now. Patch pushed!

-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[bug#49019] [PATCH 0/1] gnu: Add gccgo-10.

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 2120 bytes --]

On Sun, Jun 13, 2021 at 09:16:53PM -0700, Sarah Morgensen via Guix-patches via wrote:
> The primary goal for adding gccgo is to bootstrap the go compiler for arches
> that aren't supported by go 1.4. I believe the major one to support here is
> powerpc64le, but I am unsure of current arch usage. A secondary goal is to
> provide a gccgo-toolchain, as gccgo may provide advantages over gc in time.
> 
> I have successfully built gccgo 9, 10 and 11 for x86-64. I additionally tested
> gcc 10 using `make check-go`. There are two minor test failures:
> 
> 1. index0-out.go, previously reported upstream [0]
> 2. pprof (TestConvertCPUProfile/TestConvertMemProfile)
> 
> (I have not yet been able to deterine whether the pprof failures are an upstream
> bug or a Guix bug. Should I send details to bug-guix?)

I wouldn't worry about the test failures.

> Finally, the regex update in [1] is implemented in CUSTOM-GCC-GCCGO rather than
> CUSTOM-GCC to avoid rebuilding 2k+ gfortran packages, and CUSTOM-GCC-GCCGO
> should be removed after the change is in master.

Makes sense

> [0] https://issues.guix.gnu.org/49010
> [1] https://gcc.gnu.org/bugzilla/show_bug.cgi?id=87589
> 
> Sarah Morgensen (1):
>   gnu: Add gccgo-10.
> 
>  gnu/packages/gcc.scm | 77 ++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 77 insertions(+)
> 
> -- 
> 2.31.1
> 

I threw together a quick gccgo-toolchain package and tried using that to
build keybase and syncthing. One of the errors I came across was:
/home/efraim/go/pkg/mod/golang.org/x/sys@v0.0.0-20200922070232-aee5d888a860/unix/gccgo.go:50:
undefined reference to `gccgoRealSyscallNoError' and also
github.com/minio/sha256-simd@v0.1.1 didn't like the assembly in the
package.

It's definitely possible that I'm missing some flag to tell go that I'm
really using gccgo. Or that it should be built with binutils-gold.

-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[bug#49019] [PATCH 0/1] gnu: Add gccgo-10.

[iskarian--- via Guix-patches via]

Thanks for taking a look at this, Efraim!

June 21, 2021 6:25 AM, "Efraim Flashner" <efraim@flashner.co.il> wrote:

> On Sun, Jun 13, 2021 at 09:16:53PM -0700, Sarah Morgensen via Guix-patches via wrote:
> 
>> The primary goal for adding gccgo is to bootstrap the go compiler for arches
>> that aren't supported by go 1.4. I believe the major one to support here is
>> powerpc64le, but I am unsure of current arch usage. A secondary goal is to
>> provide a gccgo-toolchain, as gccgo may provide advantages over gc in time.
>> 
>> I have successfully built gccgo 9, 10 and 11 for x86-64. I additionally tested
>> gcc 10 using `make check-go`. There are two minor test failures:
>> 
>> 1. index0-out.go, previously reported upstream [0]
>> 2. pprof (TestConvertCPUProfile/TestConvertMemProfile)
>> 
>> (I have not yet been able to deterine whether the pprof failures are an upstream
>> bug or a Guix bug. Should I send details to bug-guix?)
> 
> I wouldn't worry about the test failures.
> 
>> Finally, the regex update in [1] is implemented in CUSTOM-GCC-GCCGO rather than
>> CUSTOM-GCC to avoid rebuilding 2k+ gfortran packages, and CUSTOM-GCC-GCCGO
>> should be removed after the change is in master.
> 
> Makes sense
> 
>> [0] https://issues.guix.gnu.org/49010
>> [1] https://gcc.gnu.org/bugzilla/show_bug.cgi?id=87589
>> 
>> Sarah Morgensen (1):
>> gnu: Add gccgo-10.
>> 
>> gnu/packages/gcc.scm | 77 ++++++++++++++++++++++++++++++++++++++++++++
>> 1 file changed, 77 insertions(+)
>> 
>> --
>> 2.31.1
> 
> I threw together a quick gccgo-toolchain package and tried using that to
> build keybase and syncthing. One of the errors I came across was:
> /home/efraim/go/pkg/mod/golang.org/x/sys@v0.0.0-20200922070232-aee5d888a860/unix/gccgo.go:50:
> undefined reference to `gccgoRealSyscallNoError' and also
> github.com/minio/sha256-simd@v0.1.1 didn't like the assembly in the
> package.

The assembly problem is (unfortunately) a known limitation of gccgo. The gc and
gccgo compilers use different asm syntax, so many extant go programs cannot
currently be compiled with gccgo alone. There are also other limitations with
the go tools provided by gccgo, as only a subset have been implemented.

However, I have successfully built go-1.16 with gccgo with a few tweaks. (Most
dependent packages build with no modification or with a version bump, and I
have minimal patches for the handful of others. Docker will have to be updated
to work with go-1.16, which will likely be out of my depth.) I plan to share
a patchset for this soon.

As for x/sys: most likely updating the module dependency will solve it. I see
x/sys has had some gccgo related commits since that version was released.

> It's definitely possible that I'm missing some flag to tell go that I'm
> really using gccgo. Or that it should be built with binutils-gold.

I must admit that I had not even thought about whether gold might be
necessary. I really am out of my depth on this one. I see that you authored
the patch for go@1.14 to use gold on armhf and aarch64, so you likely know
more about this than me! Presumably that was in regards to [0] and [1]?

Other than that, the main motivation for using gold seems to be the support for
split-stack, which allows the use of many more goroutines before running out of
stack space [2]. Replacing GNU ld (`./configure --with-ld=...') as [2] suggests
may introduce bugs... It appears that in at least one case (ppc64) gccgo
prefers gold even if gcc was not built solely with it [3]. Unfortunately, adding
gold as an optional linker would add ~75MiB to closure size...

I'm not sure what the best way forward is, here. WDYT?

> --
> Efraim Flashner <efraim@flashner.co.il> אפרים פלשנר
> GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351
> Confidentiality cannot be guaranteed on emails sent or received unencrypted

[0]: https://github.com/golang/go/issues/15696
[1]: https://sourceware.org/bugzilla/show_bug.cgi?id=19962
[2]: https://golang.org/doc/install/gccgo#Gold
[3]: https://github.com/gcc-mirror/gcc/commit/402565b61f9783473472fcfd729efa8c1e355d2b

[bug#49019] [PATCH 0/1] gnu: Add gccgo-10.

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 4760 bytes --]

On Mon, Jun 21, 2021 at 09:32:22PM +0000, iskarian@mgsn.dev wrote:
> Thanks for taking a look at this, Efraim!
> 
> June 21, 2021 6:25 AM, "Efraim Flashner" <efraim@flashner.co.il> wrote:
> 
> > On Sun, Jun 13, 2021 at 09:16:53PM -0700, Sarah Morgensen via Guix-patches via wrote:
> > 
> >> The primary goal for adding gccgo is to bootstrap the go compiler for arches
> >> that aren't supported by go 1.4. I believe the major one to support here is
> >> powerpc64le, but I am unsure of current arch usage. A secondary goal is to
> >> provide a gccgo-toolchain, as gccgo may provide advantages over gc in time.
> >> 
> >> I have successfully built gccgo 9, 10 and 11 for x86-64. I additionally tested
> >> gcc 10 using `make check-go`. There are two minor test failures:
> >> 
> >> 1. index0-out.go, previously reported upstream [0]
> >> 2. pprof (TestConvertCPUProfile/TestConvertMemProfile)
> >> 
> >> (I have not yet been able to deterine whether the pprof failures are an upstream
> >> bug or a Guix bug. Should I send details to bug-guix?)
> > 
> > I wouldn't worry about the test failures.
> > 
> >> Finally, the regex update in [1] is implemented in CUSTOM-GCC-GCCGO rather than
> >> CUSTOM-GCC to avoid rebuilding 2k+ gfortran packages, and CUSTOM-GCC-GCCGO
> >> should be removed after the change is in master.
> > 
> > Makes sense
> > 
> >> [0] https://issues.guix.gnu.org/49010
> >> [1] https://gcc.gnu.org/bugzilla/show_bug.cgi?id=87589
> >> 
> >> Sarah Morgensen (1):
> >> gnu: Add gccgo-10.
> >> 
> >> gnu/packages/gcc.scm | 77 ++++++++++++++++++++++++++++++++++++++++++++
> >> 1 file changed, 77 insertions(+)
> >> 
> >> --
> >> 2.31.1
> > 
> > I threw together a quick gccgo-toolchain package and tried using that to
> > build keybase and syncthing. One of the errors I came across was:
> > /home/efraim/go/pkg/mod/golang.org/x/sys@v0.0.0-20200922070232-aee5d888a860/unix/gccgo.go:50:
> > undefined reference to `gccgoRealSyscallNoError' and also
> > github.com/minio/sha256-simd@v0.1.1 didn't like the assembly in the
> > package.
> 
> The assembly problem is (unfortunately) a known limitation of gccgo. The gc and
> gccgo compilers use different asm syntax, so many extant go programs cannot
> currently be compiled with gccgo alone. There are also other limitations with
> the go tools provided by gccgo, as only a subset have been implemented.
> 
> However, I have successfully built go-1.16 with gccgo with a few tweaks. (Most
> dependent packages build with no modification or with a version bump, and I
> have minimal patches for the handful of others. Docker will have to be updated
> to work with go-1.16, which will likely be out of my depth.) I plan to share
> a patchset for this soon.
> 
> As for x/sys: most likely updating the module dependency will solve it. I see
> x/sys has had some gccgo related commits since that version was released.

Sounds good.

> > It's definitely possible that I'm missing some flag to tell go that I'm
> > really using gccgo. Or that it should be built with binutils-gold.
> 
> I must admit that I had not even thought about whether gold might be
> necessary. I really am out of my depth on this one. I see that you authored
> the patch for go@1.14 to use gold on armhf and aarch64, so you likely know
> more about this than me! Presumably that was in regards to [0] and [1]?

That one's come forward with a couple of go versions. Perhaps it'd be
best to add it unconditionally, or at least to switch the logic and only
skip it on x86_64 and i686.

> Other than that, the main motivation for using gold seems to be the support for
> split-stack, which allows the use of many more goroutines before running out of
> stack space [2]. Replacing GNU ld (`./configure --with-ld=...') as [2] suggests
> may introduce bugs... It appears that in at least one case (ppc64) gccgo
> prefers gold even if gcc was not built solely with it [3]. Unfortunately, adding
> gold as an optional linker would add ~75MiB to closure size...
> 
> I'm not sure what the best way forward is, here. WDYT?

It's not a lot of extra closure size, given just how large the go
package itself is. I think we leave it as-is for now and add change it
up later if necessary.

> 
> [0]: https://github.com/golang/go/issues/15696
> [1]: https://sourceware.org/bugzilla/show_bug.cgi?id=19962
> [2]: https://golang.org/doc/install/gccgo#Gold
> [3]: https://github.com/gcc-mirror/gcc/commit/402565b61f9783473472fcfd729efa8c1e355d2b

-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]