Go build system updates and future work?

Go build system updates and future work?

[Leo Famulari]

[-- Attachment #1: Type: text/plain, Size: 1095 bytes --]

I just pushed a revamped Go build system with commit
e3900a4d64e4bf6f426809d6bff058e5a2ae9bc8.

The main change is that instead of putting the list of Go-language
dependencies store paths in the GOPATH environment variable, these store
paths are union-symlinked into the build directory, and GOPATH points
there.

This fixes the issue of spurious store references
<https://bugs.gnu.org/33620> and should dramatically reduce the closure
size of certain Go packages. So now there is less reason to use bundled
libraries for Go programs.

I think we should consider another change in how we build Go software:
let's stop splitting up Go libraries instead sub-packages.

For example, the packages 'go-golang-org-x-net-ipv4' and
'go-golang-org-x-net-ipv6' are both from the same Git repo at
<https://go.googlesource.com/net>. We should just package
'go-golang-org-x-net'. It's unidiomatic to split them up. In practice, a
Go programmer would have cloned the Git repo into their Go workspace,
and would never have a "split view" of the x/net libraries.

Thoughts? I can work on this in the next week.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Go build system updates and future work?

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 571 bytes --]

Hi!

Thank you very much for your work!

> For example, the packages 'go-golang-org-x-net-ipv4' and
> 'go-golang-org-x-net-ipv6' are both from the same Git repo at
> <https://go.googlesource.com/net>. We should just package
> 'go-golang-org-x-net'. It's unidiomatic to split them up. In practice, a
> Go programmer would have cloned the Git repo into their Go workspace,
> and would never have a "split view" of the x/net libraries.

Agreed!  This should also make the Go importer more trivial to write/finish!

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: Go build system updates and future work?

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 216 bytes --]

Erratum: You've packaged go-github-com-kr-text a second time :p

This has happened to me once.  I think we should have a linter to protect
against this.  Thoughts?

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: Go build system updates and future work?

[Leo Famulari]

[-- Attachment #1: Type: text/plain, Size: 388 bytes --]

On Thu, Mar 14, 2019 at 11:54:30PM +0100, Pierre Neidhardt wrote:
> Erratum: You've packaged go-github-com-kr-text a second time :p

Oops! Fixed in 10b30b97735ba9037f4ce58867f47678d78f4970

> This has happened to me once.  I think we should have a linter to protect
> against this.  Thoughts?

I think it's been discussed before but I'm not sure — try searching the
archives.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Go build system updates and future work?

[Leo Famulari]

[-- Attachment #1: Type: text/plain, Size: 681 bytes --]

On Thu, Mar 14, 2019 at 11:51:06PM +0100, Pierre Neidhardt wrote:
> Agreed!  This should also make the Go importer more trivial to write/finish!

Okay, I'll try it.

The only part I'm not sure about is what to "build" or "test" for these
library collections. Currently many of them run test suites, which is
nice because it gives us some confidence the package is in good shape.
We don't install any compiled libraries but it does help get the
dependency graph in good shape.

But the top level of the x/net repo, for example, does not have any Go
files at all, so the package will effectively be source only with no
testing. Maybe it won't be a problem in practice, I don't know.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Go build system updates and future work?

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 166 bytes --]

What about adding and extra key #:test-directories that takes a list of
directories where we would "cd && go test"?

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: Go build system updates and future work?

[Katherine Cox-Buday]

Leo Famulari <leo@famulari.name> writes:

> I just pushed a revamped Go build system with commit
> e3900a4d64e4bf6f426809d6bff058e5a2ae9bc8.
>
> The main change is that instead of putting the list of Go-language
> dependencies store paths in the GOPATH environment variable, these store
> paths are union-symlinked into the build directory, and GOPATH points
> there.

There is a larger change coming that I think we need to account for. Go
modules are here (and in v1.13 will be on by default). This feature
allows Go code to be built outside of a $GOPATH. I'm unsure how this
information intersects with your work, so I'll just leave a link to the
wiki[1] on modules.

Thank you for your contributions!

[1] - https://github.com/golang/go/wiki/Modules

-- 
Katherine

Re: Go build system updates and future work?

[Christopher Baines]

[-- Attachment #1: Type: text/plain, Size: 653 bytes --]


Leo Famulari <leo@famulari.name> writes:

> On Thu, Mar 14, 2019 at 11:54:30PM +0100, Pierre Neidhardt wrote:
>> Erratum: You've packaged go-github-com-kr-text a second time :p
>
> Oops! Fixed in 10b30b97735ba9037f4ce58867f47678d78f4970
>
>> This has happened to me once.  I think we should have a linter to protect
>> against this.  Thoughts?
>
> I think it's been discussed before but I'm not sure — try searching the
> archives.

The linter does detect duplicate definitions in different modules (when
you run it on one of the duplicate definitions), and there is a Guile
warning for duplicate definitions within the same module.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 962 bytes --]

Re: Go build system updates and future work?

[swedebugia]

[-- Attachment #1: Type: text/plain, Size: 292 bytes --]

Pierre Neidhardt <mail@ambrevar.xyz> skrev: (15 mars 2019 07:54:49 CET)
>What about adding and extra key #:test-directories that takes a list of
>directories where we would "cd && go test"?
>
>-- 
>Pierre Neidhardt
>https://ambrevar.xyz/

+1
-- 
Sent from my k-9 mail for Android.

[-- Attachment #2: Type: text/html, Size: 514 bytes --]

Re: Go build system updates and future work?

[Leo Famulari]

[-- Attachment #1: Type: text/plain, Size: 545 bytes --]

On Fri, Mar 15, 2019 at 09:40:15PM -0500, Katherine Cox-Buday wrote:
> There is a larger change coming that I think we need to account for. Go
> modules are here (and in v1.13 will be on by default). This feature
> allows Go code to be built outside of a $GOPATH. I'm unsure how this
> information intersects with your work, so I'll just leave a link to the
> wiki[1] on modules.

Thanks Katherine! Do you have any advice about how we can adjust to Go
modules?

Personally I still need to educate myself about them so I'm basically in
the dark.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]