[bug#41695] [PATCH] Update Go to v1.14.4

[bug#41695] [PATCH] Update Go to v1.14.4

[Katherine Cox-Buday]

[-- Attachment #1: Type: text/plain, Size: 125 bytes --]

There are too many dependent Go packages to test, but I compiled
syncthing which should be a reasonably representative test.

[-- Attachment #2: 0001-gnu-go-Update-to-1.14.4.patch --]
[-- Type: text/x-patch, Size: 3144 bytes --]

From 5db2ffd50f7284a0f18353dc15f637f9eb059504 Mon Sep 17 00:00:00 2001
From: Katherine Cox-Buday <cox.katherine.e@gmail.com>
Date: Wed, 3 Jun 2020 18:31:00 -0500
Subject: [PATCH] gnu: go: Update to 1.14.4.

* gnu/packages/golang.scm (go): Update to 1.14.4.
---
 gnu/packages/golang.scm | 25 ++++++++++++++++---------
 1 file changed, 16 insertions(+), 9 deletions(-)

diff --git a/gnu/packages/golang.scm b/gnu/packages/golang.scm
index ae0b7c6779..f09690aa01 100644
--- a/gnu/packages/golang.scm
+++ b/gnu/packages/golang.scm
@@ -217,19 +217,21 @@ in the style of communicating sequential processes (@dfn{CSP}).")
     (supported-systems '("x86_64-linux" "i686-linux" "armhf-linux" "aarch64-linux"))
     (license license:bsd-3)))
 
-(define-public go-1.13
+(define-public go-1.14
   (package
     (inherit go-1.4)
     (name "go")
-    (version "1.13.9")
+    (version "1.14.4")
     (source
      (origin
-       (method url-fetch)
-       (uri (string-append "https://storage.googleapis.com/golang/"
-                           name version ".src.tar.gz"))
+       (method git-fetch)
+       (uri (git-reference
+             (url "https://github.com/golang/go.git")
+             (commit (string-append "go" version))))
+       (file-name (git-file-name name version))
        (sha256
         (base32
-         "07gksk9194wa90xyd6yhagxfv7syvsx29bh8ypc4mg700vc1kfrl"))))
+         "08bazglmqp123c9dgrxflvxd011xsqfxsgah2kzbvca0mhm6qcm3"))))
     (arguments
      (substitute-keyword-arguments (package-arguments go-1.4)
        ((#:phases phases)
@@ -260,7 +262,13 @@ in the style of communicating sequential processes (@dfn{CSP}).")
                   '("cmd/go/testdata/script/mod_case_cgo.txt"
                     "cmd/go/testdata/script/list_find.txt"
                     "cmd/go/testdata/script/list_compiled_imports.txt"
-                    "cmd/go/testdata/script/cgo_syso_issue29253.txt"))
+                    "cmd/go/testdata/script/cgo_syso_issue29253.txt"
+                    "cmd/go/testdata/script/cover_cgo.txt"
+                    "cmd/go/testdata/script/cover_cgo_xtest.txt"
+                    "cmd/go/testdata/script/cover_cgo_extra_test.txt"
+                    "cmd/go/testdata/script/cover_cgo_extra_file.txt"))
+
+                 (for-each make-file-writable (find-files "."))
 
                  (substitute* "os/os_test.go"
                    (("/usr/bin") (getcwd))
@@ -359,7 +367,6 @@ in the style of communicating sequential processes (@dfn{CSP}).")
                  (setenv "GOROOT_FINAL" output)
                  (setenv "CGO_ENABLED" "1")
                  (invoke "sh" "all.bash"))))
-
            (replace 'install
              ;; TODO: Most of this could be factorized with Go 1.4.
              (lambda* (#:key outputs #:allow-other-keys)
@@ -405,7 +412,7 @@ in the style of communicating sequential processes (@dfn{CSP}).")
        ,@(package-native-inputs go-1.4)))
     (supported-systems %supported-systems)))
 
-(define-public go go-1.13)
+(define-public go go-1.14)
 
 (define-public go-github-com-alsm-ioprogress
   (let ((commit "063c3725f436e7fba0c8f588547bee21ffec7ac5")
-- 
2.26.2

[bug#41695] [PATCH] Update Go to v1.14.4

[Jack Hill]

[-- Attachment #1: Type: text/plain, Size: 2171 bytes --]

Katherine,

On Wed, 3 Jun 2020, Katherine Cox-Buday wrote:

> There are too many dependent Go packages to test, but I compiled
> syncthing which should be a reasonably representative test.

Thanks for working on updating Go.

I have rebuilt all the packages reported by `guix refresh -l go`. The 
three failures were stress-make, which was already broken (build log 
attached), mongodb-tools, which was already broken [0], and docker (build 
log attached).

[0] https://issues.guix.gnu.org/39637

Reading through the Go release notes [1], the following change to the 
net/url package caught my eye:

> When parsing of a URL fails (for example by Parse or ParseRequestURI), 
> the resulting Error message will now quote the unparsable URL. This 
> provides clearer structure and consistency with other parsing errors.

I think this could be the cause of the docker test failure. Should we 
patch docker or perhaps try to update it?

[1] https://golang.org/doc/go1.14

Another item from the release notes about changes to the Go runtime:

> A consequence of the implementation of preemption is that on Unix 
> systems, including Linux and macOS systems, programs built with Go 1.14 
> will receive more signals than programs built with earlier releases. 
> This means that programs that use packages like syscall or 
> golang.org/x/sys/unix will see more slow system calls fail with EINTR 
> errors. Those programs will have to handle those errors in some way, 
> most likely looping to try the system call again. For more information 
> about this see man 7 signal for Linux systems or similar documentation 
> for other systems.

I didn't notice any problems caused by this during package rebuilds and 
testing, but it sounds like something that could be difficult to write 
automated tests for, so we should probably be on the lookout for future 
problems at runtime.

I'm happy to see that that the go modules changes didn't cause us any 
problems with this upgrade.

As far as I can tell, once the problem with docker is addressed, this 
patch could be applied, perhaps after updating Katherine copyright line.

I am curious, why switch to using git-fetch?

Best,
Jack

[-- Attachment #2: Type: application/octet-stream, Size: 540817 bytes --]

[-- Attachment #3: Type: application/octet-stream, Size: 6688 bytes --]

[bug#41695] [PATCH] Update Go to v1.14.4

[Katherine Cox-Buday]

[-- Attachment #1: Type: text/plain, Size: 2765 bytes --]

Jack Hill <jackhill@jackhill.us> writes:

> Katherine,
>
> On Wed, 3 Jun 2020, Katherine Cox-Buday wrote:
>
>> There are too many dependent Go packages to test, but I compiled
>> syncthing which should be a reasonably representative test.
>
> Thanks for working on updating Go.
>
> I have rebuilt all the packages reported by `guix refresh -l go`. The
> three failures were stress-make, which was already broken (build log
> attached), mongodb-tools, which was already broken [0], and docker
> (build log attached).

You're welcome! Thanks for the review. I wish I had more time to
contribute to Guix.

I suppose I should have qualified my statement by saying there's too
many dependent Go packages for me to test. I should invest in a better
computer :)

> Reading through the Go release notes [1], the following change to the
> net/url package caught my eye:
>
>> When parsing of a URL fails (for example by Parse or
>> ParseRequestURI), the resulting Error message will now quote the
>> unparsable URL. This provides clearer structure and consistency with
>> other parsing errors.
>
> I think this could be the cause of the docker test failure. Should we
> patch docker or perhaps try to update it?

Spot on! I've updated docker and docker-cli. hyperledger-fabric depends
on docker-cli and also continues to build. I've attached a patch which
should supersede the prior patch. It updates go, docker, and docker-cli
atomically.

> Another item from the release notes about changes to the Go runtime:
>
>> A consequence of the implementation of preemption is that on Unix
>> systems, including Linux and macOS systems, programs built with Go
>> 1.14 will receive more signals than programs built with earlier
>> releases. This means that programs that use packages like syscall or
>> golang.org/x/sys/unix will see more slow system calls fail with
>> EINTR errors. Those programs will have to handle those errors in
>> some way, most likely looping to try the system call again. For more
>> information about this see man 7 signal for Linux systems or similar
>> documentation for other systems.
>
> I didn't notice any problems caused by this during package rebuilds
> and testing, but it sounds like something that could be difficult to
> write automated tests for, so we should probably be on the lookout for
> future problems at runtime.

Agreed.

> I'm happy to see that that the go modules changes didn't cause us any
> problems with this upgrade.

I believe 1.14 represents a stabilization of the modules feature, so I
would expect future updates to go smoothly as well.

> I am curious, why switch to using git-fetch?

`guix lint` now complains about downloading tarballs. I have also
recently seen some chatter on the mailing list about preferring this.


[-- Attachment #2: 0001-gnu-go-Update-to-1.14.4.patch --]
[-- Type: text/x-patch, Size: 7024 bytes --]

From 600f60e78a19eefc3ea1bf518d658fc8fc47fb36 Mon Sep 17 00:00:00 2001
From: Katherine Cox-Buday <cox.katherine.e@gmail.com>
Date: Wed, 3 Jun 2020 18:31:00 -0500
Subject: [PATCH] gnu: go: Update to 1.14.4.

* gnu/packages/golang.scm (go): Update to 1.14.4.
* gnu/packages/docker.scm (docker, docker-cli): Update to 19.03.11.
---
 gnu/packages/docker.scm | 19 ++++++++++++++++---
 gnu/packages/golang.scm | 27 +++++++++++++++++----------
 2 files changed, 33 insertions(+), 13 deletions(-)

diff --git a/gnu/packages/docker.scm b/gnu/packages/docker.scm
index aeb43a6393..3dd706cbe9 100644
--- a/gnu/packages/docker.scm
+++ b/gnu/packages/docker.scm
@@ -4,6 +4,7 @@
 ;;; Copyright © 2019, 2020 Tobias Geerinckx-Rice <me@tobias.gr>
 ;;; Copyright © 2019 Maxim Cournoyer <maxim.cournoyer@gmail.com>
 ;;; Copyright © 2020 Michael Rohleder <mike@rohleder.de>
+;;; Copyright © 2020 Katherine Cox-Buday <cox.katherine.e@gmail.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -47,7 +48,7 @@
   #:use-module (gnu packages version-control)
   #:use-module (gnu packages virtualization))
 
-(define %docker-version "19.03.9")
+(define %docker-version "19.03.11")
 
 (define-public python-docker-py
   (package
@@ -314,7 +315,7 @@ built-in registry server of Docker.")
              (commit (string-append "v" version))))
        (file-name (git-file-name name version))
        (sha256
-        (base32 "1a9hkprkix5d7lqi88r8svvfpzh1qbzw9nrkp11lxrcf9wdan4hg"))
+        (base32 "1pmbggxbazipl24hxiaccbj32379zv79xba76l78v5131ihx922h"))
        (patches
         (search-patches "docker-fix-tests.patch"))))
     (build-system gnu-build-system)
@@ -390,6 +391,17 @@ built-in registry server of Docker.")
              (substitute* "pkg/archive/archive.go"
                (("string\\{\"xz")
                 (string-append "string{\"" (assoc-ref inputs "xz") "/bin/xz")))
+             ;; TODO: Remove when Docker proper uses v1.14.x to build
+             (substitute* "registry/resumable/resumablerequestreader_test.go"
+               (("I%27m%20not%20an%20url" all)
+                (string-append "\"" all "\"")))
+             ;; TODO: Remove when Docker proper uses v1.14.x to build
+             (substitute* "vendor/gotest.tools/x/subtest/context.go"
+               (("func \\(tc \\*testcase\\) Cleanup\\(" all)
+                (string-append all "func()"))
+               (("tc\\.Cleanup\\(" all)
+                (string-append all "nil")))
+
              (let ((source-files (filter (lambda (name)
                                            (not (string-contains name "test")))
                                          (find-files "." "\\.go$"))))
@@ -488,6 +500,7 @@ built-in registry server of Docker.")
              ;; Timeouts after 5 min.
              (delete-file "plugin/manager_linux_test.go")
              ;; Operation not permitted.
+             (delete-file "daemon/graphdriver/aufs/aufs_test.go")
              (delete-file "daemon/graphdriver/btrfs/btrfs_test.go")
              (delete-file "daemon/graphdriver/overlay/overlay_test.go")
              (delete-file "daemon/graphdriver/overlay2/overlay_test.go")
@@ -592,7 +605,7 @@ provisioning etc.")
             (commit (string-append "v" version))))
       (file-name (git-file-name name version))
       (sha256
-       (base32 "1599ff7699p3m925rdyfg7gl3cga6gy0lli7qh2ybyiw2kwf4gj9"))))
+       (base32 "1y9ymv70r1hndblr64h19q34arxl2f3dqqi2qcrai5zfimcml6lr"))))
     (build-system go-build-system)
     (arguments
      `(#:import-path "github.com/docker/cli"
diff --git a/gnu/packages/golang.scm b/gnu/packages/golang.scm
index ae0b7c6779..94fadd3302 100644
--- a/gnu/packages/golang.scm
+++ b/gnu/packages/golang.scm
@@ -12,7 +12,7 @@
 ;;; Copyright © 2018 Tomáš Čech <sleep_walker@gnu.org>
 ;;; Copyright © 2018 Pierre-Antoine Rouby <pierre-antoine.rouby@inria.fr>
 ;;; Copyright © 2018 Pierre Neidhardt <mail@ambrevar.xyz>
-;;; Copyright @ 2018, 2019 Katherine Cox-Buday <cox.katherine.e@gmail.com>
+;;; Copyright @ 2018, 2019, 2020 Katherine Cox-Buday <cox.katherine.e@gmail.com>
 ;;; Copyright @ 2019 Giovanni Biscuolo <g@xelera.eu>
 ;;; Copyright @ 2019, 2020 Alex Griffin <a@ajgrf.com>
 ;;; Copyright © 2019, 2020 Arun Isaac <arunisaac@systemreboot.net>
@@ -217,19 +217,21 @@ in the style of communicating sequential processes (@dfn{CSP}).")
     (supported-systems '("x86_64-linux" "i686-linux" "armhf-linux" "aarch64-linux"))
     (license license:bsd-3)))
 
-(define-public go-1.13
+(define-public go-1.14
   (package
     (inherit go-1.4)
     (name "go")
-    (version "1.13.9")
+    (version "1.14.4")
     (source
      (origin
-       (method url-fetch)
-       (uri (string-append "https://storage.googleapis.com/golang/"
-                           name version ".src.tar.gz"))
+       (method git-fetch)
+       (uri (git-reference
+             (url "https://github.com/golang/go.git")
+             (commit (string-append "go" version))))
+       (file-name (git-file-name name version))
        (sha256
         (base32
-         "07gksk9194wa90xyd6yhagxfv7syvsx29bh8ypc4mg700vc1kfrl"))))
+         "08bazglmqp123c9dgrxflvxd011xsqfxsgah2kzbvca0mhm6qcm3"))))
     (arguments
      (substitute-keyword-arguments (package-arguments go-1.4)
        ((#:phases phases)
@@ -260,7 +262,13 @@ in the style of communicating sequential processes (@dfn{CSP}).")
                   '("cmd/go/testdata/script/mod_case_cgo.txt"
                     "cmd/go/testdata/script/list_find.txt"
                     "cmd/go/testdata/script/list_compiled_imports.txt"
-                    "cmd/go/testdata/script/cgo_syso_issue29253.txt"))
+                    "cmd/go/testdata/script/cgo_syso_issue29253.txt"
+                    "cmd/go/testdata/script/cover_cgo.txt"
+                    "cmd/go/testdata/script/cover_cgo_xtest.txt"
+                    "cmd/go/testdata/script/cover_cgo_extra_test.txt"
+                    "cmd/go/testdata/script/cover_cgo_extra_file.txt"))
+
+                 (for-each make-file-writable (find-files "."))
 
                  (substitute* "os/os_test.go"
                    (("/usr/bin") (getcwd))
@@ -359,7 +367,6 @@ in the style of communicating sequential processes (@dfn{CSP}).")
                  (setenv "GOROOT_FINAL" output)
                  (setenv "CGO_ENABLED" "1")
                  (invoke "sh" "all.bash"))))
-
            (replace 'install
              ;; TODO: Most of this could be factorized with Go 1.4.
              (lambda* (#:key outputs #:allow-other-keys)
@@ -405,7 +412,7 @@ in the style of communicating sequential processes (@dfn{CSP}).")
        ,@(package-native-inputs go-1.4)))
     (supported-systems %supported-systems)))
 
-(define-public go go-1.13)
+(define-public go go-1.14)
 
 (define-public go-github-com-alsm-ioprogress
   (let ((commit "063c3725f436e7fba0c8f588547bee21ffec7ac5")
-- 
2.26.2


[-- Attachment #3: Type: text/plain, Size: 15 bytes --]


-- 
Katherine

[bug#41695] [PATCH] Update Go to v1.14.4

[Jack Hill]

[-- Attachment #1: Type: text/plain, Size: 1732 bytes --]

On Sat, 6 Jun 2020, Katherine Cox-Buday wrote:

> Jack Hill <jackhill@jackhill.us> writes:
>
> You're welcome! Thanks for the review. I wish I had more time to
> contribute to Guix.
>
> I suppose I should have qualified my statement by saying there's too
> many dependent Go packages for me to test. I should invest in a better
> computer :)

More computing is nice, but in the meantime, I'm happy to be part of a 
community of other Guix people who can provide extra cycles when needed.

I'd like to recognize the work Chris and Danjela are doing on Guix Data 
Service to ease patch review as well.

>> Reading through the Go release notes [1], the following change to the
>> net/url package caught my eye:
>>
>>> When parsing of a URL fails (for example by Parse or
>>> ParseRequestURI), the resulting Error message will now quote the
>>> unparsable URL. This provides clearer structure and consistency with
>>> other parsing errors.
>>
>> I think this could be the cause of the docker test failure. Should we
>> patch docker or perhaps try to update it?
>
> Spot on! I've updated docker and docker-cli. hyperledger-fabric depends
> on docker-cli and also continues to build. I've attached a patch which
> should supersede the prior patch. It updates go, docker, and docker-cli
> atomically.

I'm happy to confirm that the revised path fixes the docker problem here 
as well.

[…]

>> I am curious, why switch to using git-fetch?
>
> `guix lint` now complains about downloading tarballs. I have also
> recently seen some chatter on the mailing list about preferring this.

Right, I remember that thread, thanks!

Hopefully a more experienced reviewer/committer will confirm that 
everything looks good and push soon.

Best,
Jack

bug#41695: [PATCH] Update Go to v1.14.4

[Christopher Baines]

[-- Attachment #1: Type: text/plain, Size: 2084 bytes --]


Jack Hill <jackhill@jackhill.us> writes:

> On Sat, 6 Jun 2020, Katherine Cox-Buday wrote:
>
>> Jack Hill <jackhill@jackhill.us> writes:
>>
>> You're welcome! Thanks for the review. I wish I had more time to
>> contribute to Guix.
>>
>> I suppose I should have qualified my statement by saying there's too
>> many dependent Go packages for me to test. I should invest in a better
>> computer :)
>
> More computing is nice, but in the meantime, I'm happy to be part of a
> community of other Guix people who can provide extra cycles when
> needed.
>
> I'd like to recognize the work Chris and Danjela are doing on Guix
> Data Service to ease patch review as well.
>
>>> Reading through the Go release notes [1], the following change to the
>>> net/url package caught my eye:
>>>
>>>> When parsing of a URL fails (for example by Parse or
>>>> ParseRequestURI), the resulting Error message will now quote the
>>>> unparsable URL. This provides clearer structure and consistency with
>>>> other parsing errors.
>>>
>>> I think this could be the cause of the docker test failure. Should we
>>> patch docker or perhaps try to update it?
>>
>> Spot on! I've updated docker and docker-cli. hyperledger-fabric depends
>> on docker-cli and also continues to build. I've attached a patch which
>> should supersede the prior patch. It updates go, docker, and docker-cli
>> atomically.
>
> I'm happy to confirm that the revised path fixes the docker problem
> here as well.
>
> […]
>
>>> I am curious, why switch to using git-fetch?
>>
>> `guix lint` now complains about downloading tarballs. I have also
>> recently seen some chatter on the mailing list about preferring this.
>
> Right, I remember that thread, thanks!
>
> Hopefully a more experienced reviewer/committer will confirm that
> everything looks good and push soon.

Thanks for the patch Katherine, and thanks to doing some review
Jack. I've had a look now, and pushed (with a small commit message
changes): 6e9f8826087fbd824c546f6daad70f090fa1a8d7

Thanks again,

Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 962 bytes --]