bug#39400: Go retains a reference to GCC

bug#39400: Go retains a reference to GCC

[Ludovic Courtès]

Hello!

It seems that Go unduly retains a reference to GCC:

--8<---------------cut here---------------start------------->8---
$ guix size go
store item                                                       total    self
/gnu/store/g4rrz9rnr8ssbnj3gx3dmsxv4glll8nf-go-1.12.15             646.3   355.9  55.1%
/gnu/store/x3jx25cd3q363mr7nbgzrhrv1vza6cf7-gcc-7.4.0              177.4   107.2  16.6%
/gnu/store/ziinjmbnq004866mwjrczsk12wf35qb8-perl-5.30.0            148.2    57.1   8.8%
/gnu/store/ahqgl4h89xqj695lgqvsaf6zh2nhy4pj-glibc-2.29              37.4    35.8   5.5%
/gnu/store/64xzk1kwk30n7arqgmxs2qy7wzcnc8i3-gcc-7.4.0-lib           70.1    32.8   5.1%
/gnu/store/2plcy91lypnbbysb18ymnhaw3zwk8pg1-gcc-7.4.0-lib           70.0    32.6   5.0%
/gnu/store/9kzrrccpzl6i1sfwb0drb00gi2gwk0x0-coreutils-8.31          90.0    16.5   2.6%
/gnu/store/dvs3acxwfnwgc7yma6h3y937ri2li47y-gmp-6.1.2               72.6     2.6   0.4%
/gnu/store/9mmsilz9avdl49i6a6nj5mzfyim8ihv2-tzdata-2019c             2.0     2.0   0.3%
/gnu/store/cp72ncw4prnsga65n3pzll07hpsg524f-bash-static-5.0.7        1.6     1.6   0.2%
/gnu/store/29jhbbg1hf557x8j53f9sxd9imlmf02a-bash-minimal-5.0.7      38.4     1.0   0.2%
/gnu/store/nffbgghxyvrj29lcgxs5fpmi3sx9zzql-acl-2.2.53              70.7     0.5   0.1%
/gnu/store/in1738m2zvhgpz78n2yqa972sdzc42ss-attr-2.4.48             70.3     0.3   0.0%
/gnu/store/y127kw0vmw46zcrvfz0i5s2rynq7ddjv-zlib-1.2.11             37.6     0.2   0.0%
/gnu/store/waw5ci4lazbf2a1x9v6gw1v274nk0gny-libcap-2.27             70.2     0.2   0.0%
/gnu/store/hzk6d08yiih0alc2raciah570plkjxzh-net-base-5.3             0.0     0.0   0.0%
total: 646.3 MiB
$ guix describe
Generacio 126	Jan 27 2020 23:59:19	(nuna)
  guix 93f4511
    repository URL: https://git.savannah.gnu.org/git/guix.git
    branch: master
    commit: 93f4511eb0c9b33f5083c2a04f4148e0a494059c
$ grep -r x3jx25cd3q363mr7nbgzrhrv1vza6cf7 /gnu/store/g4rrz9rnr8ssbnj3gx3dmsxv4glll8nf-go-1.12.15
Duuma dosiero /gnu/store/g4rrz9rnr8ssbnj3gx3dmsxv4glll8nf-go-1.12.15/bin/go kongruas
Duuma dosiero /gnu/store/g4rrz9rnr8ssbnj3gx3dmsxv4glll8nf-go-1.12.15/pkg/tool/linux_amd64/cgo kongruas
--8<---------------cut here---------------end--------------->8---

Ludo’.

bug#39400: Go retains a reference to GCC

[Sarah Morgensen via Bug reports for GNU Guix]

Hello all,

(I have CC'd Tobias since we briefly discussed this on #guix recently.)

Ludovic Courtès <ludo@gnu.org> writes:

> Hello!
>
> It seems that Go unduly retains a reference to GCC:
>
> $ guix size go
> store item                                                       total    self
> /gnu/store/g4rrz9rnr8ssbnj3gx3dmsxv4glll8nf-go-1.12.15             646.3   355.9  55.1%
> /gnu/store/x3jx25cd3q363mr7nbgzrhrv1vza6cf7-gcc-7.4.0              177.4   107.2  16.6%
> [...]

This is still the case:

$ guix size go
store item                                                       total    self
/gnu/store/vvly7zgn981brb37v8y8a7f9psx7c6ay-go-1.16.5              570.0   371.5  65.2%
/gnu/store/rn75fm7adgx3pw5j8pg3bczfqq1y17lk-gcc-7.5.0              178.5   107.3  18.8%
/gnu/store/fa6wj5bxkj5ll1d7292a70knmyl7a0cr-glibc-2.31              38.4    36.7   6.4%
/gnu/store/01b4w3m6mp55y531kyi1g8shh722kwqm-gcc-7.5.0-lib           71.0    32.6   5.7%
/gnu/store/57xj5gcy1jbl9ai2lnrqnpr0dald9i65-coreutils-8.32          88.0    17.0   3.0%
/gnu/store/kl68v5mclwp511xgpsl2h1s9gmsdxpzh-tzdata-2021a             1.9     1.9   0.3%
/gnu/store/mmhimfwmmidf09jw1plw3aw1g1zn2nkh-bash-static-5.0.16       1.6     1.6   0.3%
/gnu/store/pwcp239kjf7lnj5i4lkdzcfcxwcfyk72-bash-minimal-5.0.16     39.4     1.0   0.2%
/gnu/store/g2s5jfkfd4k973wb58476b1bbv9zpm6m-zlib-1.2.11             38.6     0.2   0.0%
/gnu/store/zfbbn61ij7w0bl4wbrwi87x5ghqx968c-net-base-5.3             0.0     0.0   0.0%
total: 570.0 MiB

But... is the "baking-in" of (a particular version of) GCC a bad thing?
I am not sure either way.

Go invokes gcc when compiling with cgo, and cgo (or at least the usage
of standard libraries which use cgo) is getting fairly common. If we do
not provide a default gcc with Go, a plain "go build" will produce an
error if it encounters something which uses cgo and it can't find gcc:

$ go build
# runtime/cgo
cgo: exec gcc: exec: "gcc": executable file not found in $PATH

The user would then have to either install a gcc-toolchain, or figure
out that they must set CGO_ENABLED=0. From this perspective, it is more
convenient to have GCC baked-in for the average user, who likely has no
reason to want a different CC anyway.

On the other hand, currently GCC is baked-in to Go in two ways:

* CC is set to /gnu/store/...-gcc-7.5.0/bin/gcc
* Go is patched so that it adds /gnu/store/...-gcc-7.5.0-lib/lib as a
runpath when linking with cgo files

This means that even if the user provides a different CC, the
gcc-7.5.0-lib dir will also be in the runpath. I do not know if, or how
much, this would conflict with other gcc-lib runpaths.

I have experimented with a couple ways of removing the gcc-7.5.0 reference:

1. Simply set CC=gcc. This works to remove gcc-7.5.0 from references,
but we still get a gcc-7.5.0-lib runpath. We can't remove this runpath
completely, as anything using cgo-enabled parts of the standard library
require it, and Go does not save the library location anywhere.

2. Make Go require external linking for anything using cgo, which would
remove the need to patch internal linking at all. Some platforms do not
support internally linking cgo at all, so Go should have no trouble
handling this. It does break some tests which expect to be able to
internally link, but I have not yet found any actual packages it breaks.

My instinct says that removing the reference, and doing so via #2, is
the way to go, but I am just a newcomer to Guix. WDYT?

--
Sarah

bug#39400: Go retains a reference to GCC

[Sarah Morgensen via Bug reports for GNU Guix]

Hello,

A quick addendum...

Sarah Morgensen <iskarian@mgsn.dev> writes:

> This means that even if the user provides a different CC, the
> gcc-7.5.0-lib dir will also be in the runpath. I do not know if, or how
> much, this would conflict with other gcc-lib runpaths.
>

I have just seen the consequences of this: the binary is unable to load
symbols from the newer library. More info at
<https://issues.guix.gnu.org/36823>.

> I have experimented with a couple ways of removing the gcc-7.5.0 reference:
>
> 1. Simply set CC=gcc. This works to remove gcc-7.5.0 from references,
> but we still get a gcc-7.5.0-lib runpath. We can't remove this runpath
> completely, as anything using cgo-enabled parts of the standard library
> require it, and Go does not save the library location anywhere.
>

--
Sarah

bug#39400: Go retains a reference to GCC

[Ludovic Courtès]

Hi,

Sarah Morgensen <iskarian@mgsn.dev> skribis:

> Ludovic Courtès <ludo@gnu.org> writes:
>
>> Hello!
>>
>> It seems that Go unduly retains a reference to GCC:
>>
>> $ guix size go
>> store item                                                       total    self
>> /gnu/store/g4rrz9rnr8ssbnj3gx3dmsxv4glll8nf-go-1.12.15             646.3   355.9  55.1%
>> /gnu/store/x3jx25cd3q363mr7nbgzrhrv1vza6cf7-gcc-7.4.0              177.4   107.2  16.6%
>> [...]
>
> This is still the case:
>
> $ guix size go
> store item                                                       total    self
> /gnu/store/vvly7zgn981brb37v8y8a7f9psx7c6ay-go-1.16.5              570.0   371.5  65.2%
> /gnu/store/rn75fm7adgx3pw5j8pg3bczfqq1y17lk-gcc-7.5.0              178.5   107.3  18.8%
> /gnu/store/fa6wj5bxkj5ll1d7292a70knmyl7a0cr-glibc-2.31              38.4    36.7   6.4%
> /gnu/store/01b4w3m6mp55y531kyi1g8shh722kwqm-gcc-7.5.0-lib           71.0    32.6   5.7%
> /gnu/store/57xj5gcy1jbl9ai2lnrqnpr0dald9i65-coreutils-8.32          88.0    17.0   3.0%
> /gnu/store/kl68v5mclwp511xgpsl2h1s9gmsdxpzh-tzdata-2021a             1.9     1.9   0.3%
> /gnu/store/mmhimfwmmidf09jw1plw3aw1g1zn2nkh-bash-static-5.0.16       1.6     1.6   0.3%
> /gnu/store/pwcp239kjf7lnj5i4lkdzcfcxwcfyk72-bash-minimal-5.0.16     39.4     1.0   0.2%
> /gnu/store/g2s5jfkfd4k973wb58476b1bbv9zpm6m-zlib-1.2.11             38.6     0.2   0.0%
> /gnu/store/zfbbn61ij7w0bl4wbrwi87x5ghqx968c-net-base-5.3             0.0     0.0   0.0%
> total: 570.0 MiB
>
> But... is the "baking-in" of (a particular version of) GCC a bad thing?
> I am not sure either way.
>
> Go invokes gcc when compiling with cgo, and cgo (or at least the usage
> of standard libraries which use cgo) is getting fairly common. If we do
> not provide a default gcc with Go, a plain "go build" will produce an
> error if it encounters something which uses cgo and it can't find gcc:
>
> $ go build
> # runtime/cgo
> cgo: exec gcc: exec: "gcc": executable file not found in $PATH

Ah, I didn’t know about cgo (a helper for C bindings, right?).

I think it’s a case where “dynamic composition” (i.e., looking for gcc
in $PATH at run time) is preferable, because there are lots of
situations where gcc is not needed at all.

[...]

> I have experimented with a couple ways of removing the gcc-7.5.0 reference:
>
> 1. Simply set CC=gcc. This works to remove gcc-7.5.0 from references,
> but we still get a gcc-7.5.0-lib runpath. We can't remove this runpath
> completely, as anything using cgo-enabled parts of the standard library
> require it, and Go does not save the library location anywhere.

Sounds good to me.  (gcc-7.5.0-lib is always in the RUNPATH of
executables, we don’t have to worry about this one.)

> 2. Make Go require external linking for anything using cgo, which would
> remove the need to patch internal linking at all. Some platforms do not
> support internally linking cgo at all, so Go should have no trouble
> handling this. It does break some tests which expect to be able to
> internally link, but I have not yet found any actual packages it breaks.

What do you mean by “external linking” and “internal linking” in this
context?  (I know very little about Go.)

Thanks!

Ludo’.

bug#39400: Go retains a reference to GCC

[Sarah Morgensen via Bug reports for GNU Guix]

Hi,

Ludovic Courtès <ludo@gnu.org> writes:

>> Go invokes gcc when compiling with cgo, and cgo (or at least the usage
>> of standard libraries which use cgo) is getting fairly common. If we do
>> not provide a default gcc with Go, a plain "go build" will produce an
>> error if it encounters something which uses cgo and it can't find gcc:
>>
>> $ go build
>> # runtime/cgo
>> cgo: exec gcc: exec: "gcc": executable file not found in $PATH
>
> Ah, I didn’t know about cgo (a helper for C bindings, right?).

Yes, cgo allows you to compile Go programs which interface with C, as
well as straight .c files.

>
> I think it’s a case where “dynamic composition” (i.e., looking for gcc
> in $PATH at run time) is preferable, because there are lots of
> situations where gcc is not needed at all.
>
> [...]
>
>> I have experimented with a couple ways of removing the gcc-7.5.0 reference:
>>
>> 1. Simply set CC=gcc. This works to remove gcc-7.5.0 from references,
>> but we still get a gcc-7.5.0-lib runpath. We can't remove this runpath
>> completely, as anything using cgo-enabled parts of the standard library
>> require it, and Go does not save the library location anywhere.
>
> Sounds good to me.  (gcc-7.5.0-lib is always in the RUNPATH of
> executables, we don’t have to worry about this one.)

I recently discovered that there is actually an issue with this
particular approach. If the user uses a newer gcc-toolchain, the
always-added gcc-7.5.0-lib shadows the newer libraries and newer symbols
are unavailable. See <https://issues.guix.gnu.org/36823>.

>
>> 2. Make Go require external linking for anything using cgo, which would
>> remove the need to patch internal linking at all. Some platforms do not
>> support internally linking cgo at all, so Go should have no trouble
>> handling this. It does break some tests which expect to be able to
>> internally link, but I have not yet found any actual packages it breaks.
>
> What do you mean by “external linking” and “internal linking” in this
> context?  (I know very little about Go.)

"external linking" => Go invokes gcc to link object files together
"internal linking" => Go does the linking itself

--
Sarah