From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0 ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id aG9wNcxLMWGN/AAAgWs5BA (envelope-from ) for ; Fri, 03 Sep 2021 00:10:20 +0200 Received: from aspmx1.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0 with LMTPS id 6LMKMcxLMWHLUgAA1q6Kng (envelope-from ) for ; Thu, 02 Sep 2021 22:10:20 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 1E1FF1497C for ; Fri, 3 Sep 2021 00:10:20 +0200 (CEST) Received: from localhost ([::1]:54532 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mLuuZ-0006DR-4V for larch@yhetil.org; Thu, 02 Sep 2021 18:10:19 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:41206) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLuuJ-0006BY-4V for guix-patches@gnu.org; Thu, 02 Sep 2021 18:10:03 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:58951) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mLuuI-0005wo-Nv for guix-patches@gnu.org; Thu, 02 Sep 2021 18:10:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1mLuuI-0007F8-9V for guix-patches@gnu.org; Thu, 02 Sep 2021 18:10:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#50348] [PATCH] gnu: Add go-1.17. Resent-From: Sarah Morgensen Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Thu, 02 Sep 2021 22:10:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 50348 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 50348@debbugs.gnu.org X-Debbugs-Original-To: guix-patches@gnu.org Received: via spool by submit@debbugs.gnu.org id=B.163062059127818 (code B ref -1); Thu, 02 Sep 2021 22:10:02 +0000 Received: (at submit) by debbugs.gnu.org; 2 Sep 2021 22:09:51 +0000 Received: from localhost ([127.0.0.1]:42264 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mLuu2-0007EY-S7 for submit@debbugs.gnu.org; Thu, 02 Sep 2021 18:09:50 -0400 Received: from lists.gnu.org ([209.51.188.17]:57344) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mLuty-0007EP-UA for submit@debbugs.gnu.org; Thu, 02 Sep 2021 18:09:46 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:41074) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLuty-0005nm-0c for guix-patches@gnu.org; Thu, 02 Sep 2021 18:09:42 -0400 Received: from out2.migadu.com ([2001:41d0:2:aacc::]:47794) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLutt-0005cD-8K for guix-patches@gnu.org; Thu, 02 Sep 2021 18:09:41 -0400 X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mgsn.dev; s=key1; t=1630620570; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding; bh=mtFeGoko2xJrXQos+VNdcmICANgWZJAA6fTGyHdZzk0=; b=ePWvnrIw1ROTpA7QTLcV/BYkuIk7wWEmVkvu4mIBrKsINqzHvcZ/4REeX/DPUZAI2GtDCR ts+8fmNvwzYqJJnEq9dsHj4YL22v0kl8/sEjcy7nCqmmJhN+S7B9Yjkmtk0jmhMif1QUSq qwIwfBSW7/Oh0fJmI10qmynJ3ijCLTI= From: Sarah Morgensen Date: Thu, 2 Sep 2021 15:09:27 -0700 Message-Id: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Migadu-Auth-User: iskarian@mgsn.dev Received-SPF: pass client-ip=2001:41d0:2:aacc::; envelope-from=iskarian@mgsn.dev; helo=out2.migadu.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: "Guix-patches" X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1630620620; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding:resent-cc: resent-from:resent-sender:resent-message-id:list-id:list-help: list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=mtFeGoko2xJrXQos+VNdcmICANgWZJAA6fTGyHdZzk0=; b=EpuM7Vhc5FSlvlRlVY90VJjeDlIVtnir0KCeP7pambnRNdFOYiS8gnjy71uT5DCIkvOEPl bp/gzXmNB8CHWrsbzr2Fo7weDpaGW8baM1dE+hf8PaejGo41bK9yZ2o6lcSx0MtOwlECZS o1750o64hhykMjxrQr9OVEUA49Cy+NfwAlRnPn+mSCkZRS0vwKJ0HRe4YP94+Bm2v3K+r9 G3rtS0uBjoWgmgJvUpSOsliH+wjijOi+wjbqe1CpCtBmZZOnjtellxK6gUxV8mHV6l1E6d Sw9YkyrlVCycQsoKFAlAbi/lJvpnDrkGeObASdAudWc8irxUoHWc0vZ20KuvLA== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1630620620; a=rsa-sha256; cv=none; b=hihr9Uv/zOD9KAFGpa3zkXiTUyHw4EUljkTh8XV4vEcCrUfW6kwzhc3/vEb8o7K8rt3z+r ivNwaTex8c6v/SGF7/swWjN4zeWMg+isqXNPNDx0h2ctIN4TGR1gRgQERyHV3xlTzSW5o6 PBryJ9M6MaKf5VWiMSpo/n0od0MraYioNWbvU5+sIT98kogaYATVhSq79gIMeGRJ6H5Mxu kRBdokN+bZjiO4pQ8RpZA0iY7qBhTlE63vFn50MzdTj0mAw6RFCMo0wWFmmIV0wSLQmfG+ ZPi7oFdCsu24pdkEOhkXB4rmPxvIqKQayX38NOSx3pt+UuoawsE2msVpUSRhZw== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=mgsn.dev header.s=key1 header.b=ePWvnrIw; dmarc=fail reason="SPF not aligned (relaxed)" header.from=mgsn.dev (policy=none); spf=pass (aspmx1.migadu.com: domain of guix-patches-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-patches-bounces@gnu.org X-Migadu-Spam-Score: 3.68 Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=mgsn.dev header.s=key1 header.b=ePWvnrIw; dmarc=fail reason="SPF not aligned (relaxed)" header.from=mgsn.dev (policy=none); spf=pass (aspmx1.migadu.com: domain of guix-patches-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-patches-bounces@gnu.org X-Migadu-Queue-Id: 1E1FF1497C X-Spam-Score: 3.68 X-Migadu-Scanner: scn1.migadu.com X-TUID: k9WRj3GMuz09 * gnu/packages.golang.scm (go-1.17): New variable. --- Hello Guix, This patch (finally) adds Go 1.17, with some further improvements! Highlights: * 43% closure size compared to go-1.16 (~50% output size) * No longer dependent on single GCC version (fixes #36823, #39400)[0][1] * Many tests re-enabled * 'doc' output merged (makes docs usable with `godoc') * Installs in FHS paths rather than directly in $out/ * Builds nearly all our Go packages * Phases refactored For the few Guix Go packages which require an update or patch to build with Go 1.17, I will send those later. But I don't think those should be required to merge this, since go-1.17 won't be used to build Go packages by default. What do you think? (Details below the break, for those interested.) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ This update removes Go's dependence on canonical-gcc by replacing the patch that hardcodes the gcc-lib runpath in Go's own linker with one that tells Go to use the host linker (which knows how to add runpaths) when linking against libraries (glibc, gcc-lib, etc). This removes GCC from Go's closure (fixing [0]), and allows users to build with any GCC library (fixing [1]). In the future, we may be able to drop this patch and instead use GO_EXTLINK_ENABLED=1 or similar when [2] or [3] is resolved. Removing the GCC dependency reduces closure size by ~107MiB. Properly stripping the binaries shaves off ~20MiB. No longer installing pre-compiled standard library archives shaves off another ~180MiB. Quoting my explanatory comment: > Notably, we do not install archives (180M), which Go will happily recompile > quickly (and cache) if needed, almost surely faster than they could be > substituted. > The main motivation for pre-compiled archives is to use libc-linked `net' or > `os' packages without a C compiler, but on Guix a C compiler is necessary to > properly link the final binaries anyway. Many build flags also invalidate > these pre-compiled archives, so in practice Go often recompiles them anyway. > Upstream is also planning to no longer install these archives: > > When necessary, a custom pre-compiled library package can be created > with `#:import-path "std"' and used with `-pkgdir'. That latter step should only be necessary when updating the build-system to use this Go. Finally, rather than installing directly in $out, Go now installs to /bin, /lib/go, /share/go, and /share/doc/go (albeit with symlinks to them from /lib/go). Also, the 'doc' output, with less than 1MB, was merged into 'out', which makes `godoc' work correctly for the standard library. [0] Go retains a reference to GCC. [1] https://issues.guix.gnu.org/39400 [2] cmd/go: spurious error message when external linking a pure Go program [3] cmd/link: provide a way to always use external linking if cgo is used -- Sarah gnu/packages/golang.scm | 182 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 182 insertions(+) diff --git a/gnu/packages/golang.scm b/gnu/packages/golang.scm index b6e8b84749..8da7c9bf85 100644 --- a/gnu/packages/golang.scm +++ b/gnu/packages/golang.scm @@ -608,6 +608,188 @@ (define-public go-1.16 (alist-replace "go" (list gccgo-10) (package-native-inputs go-1.14)) (package-native-inputs go-1.14)))))) +(define-public go-1.17 + (package + (inherit go-1.16) + (name "go") + (version "1.17") + (source + (origin + (method git-fetch) + (uri (git-reference + (url "https://github.com/golang/go") + (commit (string-append "go" version)))) + (file-name (git-file-name name version)) + (sha256 + (base32 + "1psra6j95ws38mx3scc1whky8cwk32mazqs0wzzdwbs8ppl8iwl5")))) + (outputs '("out" "tests")) ; 'tests' contains distribution tests. + (arguments + `(#:modules ((ice-9 match) + (guix build gnu-build-system) + (guix build utils)) + #:phases + (modify-phases %standard-phases + (replace 'configure + (lambda* (#:key inputs outputs #:allow-other-keys) + (let ((output (assoc-ref outputs "out")) + (loader (string-append (assoc-ref inputs "libc") + ,(glibc-dynamic-linker)))) + (setenv "GOOS" "linux") + (setenv "GO_LDSO" loader) + (setenv "GOROOT" (getcwd)) + (setenv "GOROOT_FINAL" (string-append output "/lib/go")) + (setenv "GOGC" "400") + (setenv "GOCACHE" "/tmp/go-cache")))) + (add-after 'unpack 'patch-source + (lambda* (#:key inputs outputs #:allow-other-keys) + (let* ((net-base (assoc-ref inputs "net-base")) + (tzdata-path (string-append (assoc-ref inputs "tzdata") + "/share/zoneinfo"))) + ;; XXX: Remove when #49729 is merged? + (for-each make-file-writable (find-files "src")) + + ;; Having the patch in the 'patches' field of breaks + ;; the 'TestServeContent' test due to the fact that + ;; timestamps are reset. Thus, apply it from here. + (invoke "patch" "-p1" "--force" "-i" + (assoc-ref inputs "go-skip-gc-test.patch")) + (invoke "patch" "-p1" "--force" "-i" + (assoc-ref inputs "go-fix-script-tests.patch")) + + (substitute* "src/os/os_test.go" + (("/usr/bin") (getcwd)) + (("/bin/sh") (which "sh"))) + + (substitute* "src/cmd/go/testdata/script/cgo_path_space.txt" + (("/bin/sh") (which "sh"))) + + ;; fix shebang for testar script + ;; note the target script is generated at build time. + (substitute* "misc/cgo/testcarchive/carchive_test.go" + (("/usr/bin/env bash") (which "bash"))) + + (substitute* "src/net/lookup_unix.go" + (("/etc/protocols") + (string-append net-base "/etc/protocols"))) + (substitute* "src/net/port_unix.go" + (("/etc/services") + (string-append net-base "/etc/services"))) + (substitute* "src/time/zoneinfo_unix.go" + (("/usr/share/zoneinfo/") tzdata-path))))) + (add-after 'patch-source 'disable-failing-tests + (lambda _ + ;; Disable failing tests: these tests attempt to access + ;; commands or network resources which are neither available + ;; nor necessary for the build to succeed. + (for-each + (match-lambda + ((file test) + (let ((regex (string-append "^(func\\s+)(" test "\\()"))) + (substitute* file + ((regex all before test_name) + (string-append before "Disabled" test_name)))))) + '(("src/net/cgo_unix_test.go" "TestCgoLookupPort") + ("src/net/cgo_unix_test.go" "TestCgoLookupPortWithCancel") + ;; 127.0.0.1 doesn't exist + ("src/net/cgo_unix_test.go" "TestCgoLookupPTR") + ("src/net/cgo_unix_test.go" "TestCgoLookupPTRWithCancel") + ;; /etc/services doesn't exist + ("src/net/parse_test.go" "TestReadLine") + ;; The user's directory doesn't exist + ("src/os/os_test.go" "TestUserHomeDir"))) + + ;; These tests fail on aarch64-linux + (substitute* "src/cmd/dist/test.go" + (("t.registerHostTest\\(\"testsanitizers/msan.*") "")))) + (add-after 'patch-source 'enable-external-linking + (lambda _ + ;; Invoke GCC to link any archives created with GCC (that is, any + ;; packages built using 'cgo'), because Go doesn't know how to + ;; handle the runpaths but GCC does. Use substitute* rather than + ;; a patch since these files are liable to change often. + ;; + ;; XXX: Replace with GO_EXTLINK_ENABLED=1 or similar when + ;; and/or + ;; are resolved. + (substitute* "src/cmd/link/internal/ld/config.go" + (("iscgo && externalobj") "iscgo")) + (substitute* '("src/cmd/nm/nm_cgo_test.go" + "src/cmd/dist/test.go") + (("^func.*?nternalLink\\(\\).*" all) + (string-append all "\n\treturn false\n"))))) + (replace 'build + (lambda* (#:key (parallel-build? #t) #:allow-other-keys) + (let* ((njobs (if parallel-build? (parallel-job-count) 1))) + (with-directory-excursion "src" + (setenv "GOMAXPROCS" (number->string njobs)) + (invoke "sh" "make.bash" "--no-banner"))))) + (replace 'check + (lambda* (#:key target (tests? (not target)) (parallel-tests? #t) + #:allow-other-keys) + (let* ((njobs (if parallel-tests? (parallel-job-count) 1))) + (when tests? + (with-directory-excursion "src" + (setenv "GOMAXPROCS" (number->string njobs)) + (invoke "sh" "run.bash" "--no-rebuild")))))) + (add-before 'install 'unpatch-perl-shebangs + (lambda _ + ;; Avoid inclusion of perl in closure by rewriting references + ;; to perl input in sourcecode generators and test scripts + (substitute* (cons "src/net/http/cgi/testdata/test.cgi" + (find-files "src" "\\.pl$")) + (("^#!.*") "#!/usr/bin/env perl\n")))) + (replace 'install + (lambda* (#:key outputs #:allow-other-keys) + ;; Notably, we do not install archives (180M), which Go will + ;; happily recompile quickly (and cache) if needed, almost + ;; surely faster than they could be substituted. + ;; + ;; The main motivation for pre-compiled archives is to use + ;; libc-linked `net' or `os' packages without a C compiler, + ;; but on Guix a C compiler is necessary to properly link the + ;; final binaries anyway. Many build flags also invalidate + ;; these pre-compiled archives, so in practice Go often + ;; recompiles them anyway. + ;; + ;; Upstream is also planning to no longer install these + ;; archives: + ;; + ;; When necessary, a custom pre-compiled library package can + ;; be created with `#:import-path "std"' and used with + ;; `-pkgdir'. + (let* ((out (assoc-ref outputs "out")) + (tests (assoc-ref outputs "tests"))) + (for-each + (lambda (file) + (copy-recursively file (string-append out "/lib/go/" file))) + '("lib" "VERSION" "pkg/include" "pkg/tool")) + + (for-each + (match-lambda + ((file dest output) + ;; Copy to output/dest and symlink from output/lib/go/file. + (let ((file* (string-append output "/lib/go/" file)) + (dest* (string-append output "/" dest))) + (copy-recursively file dest*) + (mkdir-p (dirname file*)) + (symlink (string-append "../../" dest) file*)))) + `(("bin" "bin" ,out) + ("src" "share/go/src" ,out) + ("misc" "share/go/misc" ,out) + ("doc" "share/doc/go/doc" ,out) + ("api" "share/go/api" ,tests) + ("test" "share/go/test" ,tests)))))) + (add-after 'install 'install-doc-files + (lambda* (#:key outputs #:allow-other-keys) + (let ((out (assoc-ref outputs "out"))) + (for-each + (lambda (file) + (install-file file (string-append out "/share/doc/go"))) + '("AUTHORS" "CONTRIBUTORS" "CONTRIBUTING.md" "PATENTS" + "README.md" "SECURITY.md")))))))) + (inputs (alist-delete "gcc:lib" (package-inputs go-1.16))))) + (define-public go go-1.14) (define-public go-0xacab-org-leap-shapeshifter base-commit: 95c29d2746943733cbe8df7013854d45bb0df413 -- 2.31.1