From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id R7zYJLEWKWEwAAEAgWs5BA (envelope-from ) for ; Fri, 27 Aug 2021 18:45:37 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0 with LMTPS id IKQCILEWKWH/ZQAA1q6Kng (envelope-from ) for ; Fri, 27 Aug 2021 16:45:37 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id A825D1423E for ; Fri, 27 Aug 2021 18:45:36 +0200 (CEST) Received: from localhost ([::1]:48494 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mJez1-0007Ag-Bx for larch@yhetil.org; Fri, 27 Aug 2021 12:45:35 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:44520) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mJeyU-0007AX-40 for guix-patches@gnu.org; Fri, 27 Aug 2021 12:45:02 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:41289) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mJeyT-0002Gg-Nd for guix-patches@gnu.org; Fri, 27 Aug 2021 12:45:01 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1mJeyT-0007TC-LO for guix-patches@gnu.org; Fri, 27 Aug 2021 12:45:01 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#50227] [PATCH] build-system/go: Trim store references using the native compiler option. Resent-From: Marius Bakke Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Fri, 27 Aug 2021 16:45:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 50227 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 50227@debbugs.gnu.org Received: via spool by 50227-submit@debbugs.gnu.org id=B50227.163008268228675 (code B ref 50227); Fri, 27 Aug 2021 16:45:01 +0000 Received: (at 50227) by debbugs.gnu.org; 27 Aug 2021 16:44:42 +0000 Received: from localhost ([127.0.0.1]:52835 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mJey5-0007SN-CB for submit@debbugs.gnu.org; Fri, 27 Aug 2021 12:44:41 -0400 Received: from eggs.gnu.org ([209.51.188.92]:56570) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mJexz-0007S6-Bq for 50227@debbugs.gnu.org; Fri, 27 Aug 2021 12:44:36 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:54008) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mJexu-0001wH-3c for 50227@debbugs.gnu.org; Fri, 27 Aug 2021 12:44:26 -0400 Received: from host-37-191-231-185.lynet.no ([37.191.231.185]:38010 helo=localhost) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mJext-0000bo-N8 for 50227@debbugs.gnu.org; Fri, 27 Aug 2021 12:44:25 -0400 From: Marius Bakke Date: Fri, 27 Aug 2021 18:44:23 +0200 Message-Id: <20210827164423.17109-1-marius@gnu.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210827151052.12611-1-marius@gnu.org> References: <20210827151052.12611-1-marius@gnu.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: "Guix-patches" X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1630082737; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding:resent-cc: resent-from:resent-sender:resent-message-id:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post; bh=GVs43IzZEhNgLk3CyxW+A8BDMG5k5UR1zF/rTtOIhSo=; b=tjKm1q7kIFJoRtBmgw1ueopjFTCPj18vORhmDaUSBsPO6Q8mGZ4YOrakRKqETyAUsBwKZP MBwntEpsbQz2iETHMv9sSXyzuX7io0tVsPeC64jxsA3AxM0FwvWL4VDNexhhSfb3I/2Cnq 4qkiwkwYE41JQg/UKmaaNU0GxpbJKLvScJ15T9N4eU97gpwQXCQKE8nTkwBTplen4ic0vf wNOYshUoLbgW0Yesfgaw+ofiVwJZqjk0oiQTHnRPEE2sNtSzZ3Ebyqlp8n1+6F/7cbjPkg Gpt7/nA6BK0HYcSFtI5wcwCYFE3cT12vEbg65BW6+C6/PPwl65zXG1LdWr9hHw== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1630082737; a=rsa-sha256; cv=none; b=EtGmIRfXTnV7WvIA7+Ujkj62cU2Iz6D/hi7pZv82d3pLQcamHkJkEa0p56wYHYUg6g2pdR iX1HcxZXWWIHyvR/XH/+ca36c1LJs3rfWY863COHGR/aH9QejTC4Zkw9IFam35zUiawiNp dFtcDEnXkiYrDQbRPIsH2KG3Xtf7jjTe9e88/shuehj14yQ6Cc/XejJF10T/0WJhe2xFl2 FftNBtWaKnnIizknLde3H95+q4iVJCB5XitBwl9I+OpJSrYJsS+cMya2y9M0/gp/+n0IW7 ZqQEa5EMphIFiV2MSqLKdbFt/Ceo8iNKN5Wf/DK/8d5aVly3IWQXgSfeb/WPMg== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=none; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of guix-patches-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-patches-bounces@gnu.org X-Migadu-Spam-Score: 2.07 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of guix-patches-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-patches-bounces@gnu.org X-Migadu-Queue-Id: A825D1423E X-Spam-Score: 2.07 X-Migadu-Scanner: scn0.migadu.com X-TUID: jkoHigJXfu71 * guix/build/go-build-system.scm (build): Add '-trimpath' to the 'go install' invocation. (remove-store-references, remove-go-references): Remove procedures. (%standard-phases): Don't include remove-go-references. * gnu/packages/docker.scm (docker)[arguments]: Add the '-trimpath' option to the build flags. Remove phase remove-go-references. * gnu/packages/uucp.scm (nncp)[arguments]: Likewise. --- gnu/packages/docker.scm | 7 ++-- gnu/packages/uucp.scm | 8 ++-- guix/build/go-build-system.scm | 70 +++------------------------------- 3 files changed, 14 insertions(+), 71 deletions(-) diff --git a/gnu/packages/docker.scm b/gnu/packages/docker.scm index 8bac1b89ce..108f355aa7 100644 --- a/gnu/packages/docker.scm +++ b/gnu/packages/docker.scm @@ -535,6 +535,8 @@ built-in registry server of Docker.") ;; Respectively, strip the symbol table and debug ;; information, and the DWARF symbol table. (setenv "LDFLAGS" "-s -w") + ;; Trim store references from the compiled binary. + (setenv "BUILDFLAGS" "-trimpath") ;; Make build faster (setenv "GOCACHE" "/tmp") #t)) @@ -568,10 +570,7 @@ built-in registry server of Docker.") (install-file "bundles/dynbinary-daemon/dockerd" out-bin) (install-file (string-append "bundles/dynbinary-daemon/dockerd-" (getenv "VERSION")) - out-bin) - #t))) - (add-after 'install 'remove-go-references - (assoc-ref go:%standard-phases 'remove-go-references))))) + out-bin))))))) (inputs `(("btrfs-progs" ,btrfs-progs) ("containerd" ,containerd) ; for containerd-shim diff --git a/gnu/packages/uucp.scm b/gnu/packages/uucp.scm index 120417dea1..9d39c88fe5 100644 --- a/gnu/packages/uucp.scm +++ b/gnu/packages/uucp.scm @@ -127,6 +127,10 @@ between computers.") (substitute* (list "bin/default.do" "bin/hjson-cli.do" "test.do") ((" -mod=vendor") "") ((" -m") "")) + (substitute* (list "bin/default.do" "bin/hjson-cli.do") + ;; Prevent reference to the Go inputs in the compiled binaries. + (("\\$GO build") + "$GO build -trimpath")) ;; Use the correct module path. `go list` does not report the ;; correct module path since we have moved the source files. (substitute* "bin/default.do" @@ -138,9 +142,7 @@ between computers.") (replace 'check (lambda* (#:key tests? #:allow-other-keys) (when tests? - (invoke "contrib/do" "-c" "test")))) - (add-after 'install 'remove-go-references - (assoc-ref go:%standard-phases 'remove-go-references))))) + (invoke "contrib/do" "-c" "test"))))))) (inputs `(("go-github-com-davecgh-go-xdr" ,go-github-com-davecgh-go-xdr) ("go-github-com-dustin-go-humanize" ,go-github-com-dustin-go-humanize) diff --git a/guix/build/go-build-system.scm b/guix/build/go-build-system.scm index 37936fe5ca..fc5ee39c8d 100644 --- a/guix/build/go-build-system.scm +++ b/guix/build/go-build-system.scm @@ -28,8 +28,6 @@ #:use-module (ice-9 match) #:use-module (ice-9 ftw) #:use-module (srfi srfi-1) - #:use-module (rnrs io ports) - #:use-module (rnrs bytevectors) #:export (%standard-phases go-build)) @@ -47,12 +45,9 @@ ;; structure called a 'workspace' [1]. This workspace can be found by Go via ;; the GOPATH environment variable. Typically, all Go source code and compiled ;; objects are kept in a single workspace, but GOPATH may be a list of -;; directories [2]. In this go-build-system we create a file system union of -;; the Go-language dependencies. Previously, we made GOPATH a list of store -;; directories, but stopped because Go programs started keeping references to -;; these directories in Go 1.11: -;; . -;; +;; directories [2], which we rely on here, with the caveat that the current +;; package must appear first on GOPATH. +; ;; Go software, whether a package or a command, is uniquely named using an ;; 'import path'. The import path is based on the URL of the software's source. ;; Because most source code is provided over the internet, the import path is @@ -88,7 +83,6 @@ ;; a tmpdir when creating the inputs union. ;; * Use Go modules [4] ;; * Re-use compiled packages [5] -;; * Stop needing remove-go-references (-trimpath ? ) ;; * Remove module packages, only offering the full Git repos? This is ;; more idiomatic, I think, because Go downloads Git repos, not modules. ;; What are the trade-offs? @@ -194,6 +188,8 @@ unpacking." (apply invoke "go" "install" "-v" ; print the name of packages as they are compiled "-x" ; print each command as it is invoked + ;; Trim store references from the compiled binaries. + "-trimpath" ;; Respectively, strip the symbol table and debug ;; information, and the DWARF symbol table. "-ldflags=-s -w" @@ -236,59 +232,6 @@ the standard install-license-files phase to first enter the correct directory." unpack-path)) (apply (assoc-ref gnu:%standard-phases 'install-license-files) args))) -(define* (remove-store-reference file file-name - #:optional (store (%store-directory))) - "Remove from FILE occurrences of FILE-NAME in STORE; return #t when FILE-NAME -is encountered in FILE, #f otherwise. This implementation reads FILE one byte at -a time, which is slow. Instead, we should use the Boyer-Moore string search -algorithm; there is an example in (guix build grafts)." - (define pattern - (string-take file-name - (+ 34 (string-length (%store-directory))))) - - (with-fluids ((%default-port-encoding #f)) - (with-atomic-file-replacement file - (lambda (in out) - ;; We cannot use `regexp-exec' here because it cannot deal with - ;; strings containing NUL characters. - (format #t "removing references to `~a' from `~a'...~%" file-name file) - (setvbuf in 'block 65536) - (setvbuf out 'block 65536) - (fold-port-matches (lambda (match result) - (put-bytevector out (string->utf8 store)) - (put-u8 out (char->integer #\/)) - (put-bytevector out - (string->utf8 - "eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-")) - #t) - #f - pattern - in - (lambda (char result) - (put-u8 out (char->integer char)) - result)))))) - -(define* (remove-go-references #:key allow-go-reference? - inputs outputs #:allow-other-keys) - "Remove any references to the Go compiler from the compiled Go executable -files in OUTPUTS." -;; We remove this spurious reference to save bandwidth when installing Go -;; executables. It would be better to not embed the reference in the first -;; place, but I'm not sure how to do that. The subject was discussed at: -;; - (if allow-go-reference? - #t - (let ((go (assoc-ref inputs "go")) - (bin "/bin")) - (for-each (lambda (output) - (when (file-exists? (string-append (cdr output) - bin)) - (for-each (lambda (file) - (remove-store-reference file go)) - (find-files (string-append (cdr output) bin))))) - outputs) - #t))) - (define %standard-phases (modify-phases gnu:%standard-phases (delete 'bootstrap) @@ -299,8 +242,7 @@ files in OUTPUTS." (replace 'build build) (replace 'check check) (replace 'install install) - (replace 'install-license-files install-license-files) - (add-after 'install 'remove-go-references remove-go-references))) + (replace 'install-license-files install-license-files))) (define* (go-build #:key inputs (phases %standard-phases) #:allow-other-keys #:rest args) -- 2.31.1