[bug#68423] [PATCH] guix: go-build-system: use trimpath go flag

[bug#68423] [PATCH] guix: go-build-system: use trimpath go flag

[Ekaitz Zarraga]

Go 1.13 introduced[1] a new "trimpath" flag in charge of removing the
references to the go toolchain in the produced ELFs. We used to remove
these references using the "remove-go-reference" function. This function
was executed after go install. By using this new trimpath flag, we don't
have to remove any store path from the ELFs produced by go install.

We're not using any go older than 1.13 anymore, it's safe to remove
these functions.

[1] https://go.dev/doc/go1.13

    -trimpath
        remove all file system paths from the resulting executable.
        Instead of absolute file system paths, the recorded file names
        will begin either a module path@version (when using modules),
        or a plain import path (when using the standard library, or
        GOPATH).

* guix/build/go-build-system.scm (build): Add -trimpath
(%standard-phases): Remove remove-go-references.
(remove-go-references): Remove.

Change-Id: Idcae366d226da5ce095693f81fd33133fd1d70d6
Co-authored-by: Picnoir <picnoir@alternativebit.fr>
---
 guix/build/go-build-system.scm | 59 +++-------------------------------
 1 file changed, 4 insertions(+), 55 deletions(-)

diff --git a/guix/build/go-build-system.scm b/guix/build/go-build-system.scm
index 7f25e05d0d..70ddcd07cd 100644
--- a/guix/build/go-build-system.scm
+++ b/guix/build/go-build-system.scm
@@ -6,6 +6,8 @@
 ;;; Copyright © 2020 Jakub Kądziołka <kuba@kadziolka.net>
 ;;; Copyright © 2020, 2021 Efraim Flashner <efraim@flashner.co.il>
 ;;; Copyright © 2021 Sarah Morgensen <iskarian@mgsn.dev>
+;;; Copyright © 2024 Ekaitz Zarraga <ekaitz@elenq.tech>
+;;; Copyright © 2024 Picnoir <picnoir@alternativebit.fr>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -90,7 +92,6 @@ (define-module (guix build go-build-system)
 ;; * Use Go modules [4]
 ;; * Re-use compiled packages [5]
 ;; * Avoid the go-inputs hack
-;; * Stop needing remove-go-references (-trimpath ? )
 ;; * Remove module packages, only offering the full Git repos? This is
 ;; more idiomatic, I think, because Go downloads Git repos, not modules.
 ;; What are the trade-offs?
@@ -265,6 +266,7 @@ (define* (build #:key import-path build-flags #:allow-other-keys)
               ;; Respectively, strip the symbol table and debug
               ;; information, and the DWARF symbol table.
               "-ldflags=-s -w"
+              "-trimpath"
               `(,@build-flags ,import-path)))
     (lambda (key . args)
       (display (string-append "Building '" import-path "' failed.\n"
@@ -304,58 +306,6 @@ (define* (install-license-files #:key unpack-path
                                                     unpack-path))
     (apply (assoc-ref gnu:%standard-phases 'install-license-files) args)))
 
-(define* (remove-store-reference file file-name
-                                  #:optional (store (%store-directory)))
-  "Remove from FILE occurrences of FILE-NAME in STORE; return #t when FILE-NAME
-is encountered in FILE, #f otherwise. This implementation reads FILE one byte at
-a time, which is slow. Instead, we should use the Boyer-Moore string search
-algorithm; there is an example in (guix build grafts)."
-  (define pattern
-    (string-take file-name
-                 (+ 34 (string-length (%store-directory)))))
-
-  (with-fluids ((%default-port-encoding #f))
-    (with-atomic-file-replacement file
-      (lambda (in out)
-        ;; We cannot use `regexp-exec' here because it cannot deal with
-        ;; strings containing NUL characters.
-        (format #t "removing references to `~a' from `~a'...~%" file-name file)
-        (setvbuf in 'block 65536)
-        (setvbuf out 'block 65536)
-        (fold-port-matches (lambda (match result)
-                             (put-bytevector out (string->utf8 store))
-                             (put-u8 out (char->integer #\/))
-                             (put-bytevector out
-                                             (string->utf8
-                                              "eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-"))
-                             #t)
-                           #f
-                           pattern
-                           in
-                           (lambda (char result)
-                             (put-u8 out (char->integer char))
-                             result))))))
-
-(define* (remove-go-references #:key allow-go-reference?
-                               inputs outputs #:allow-other-keys)
-  "Remove any references to the Go compiler from the compiled Go executable
-files in OUTPUTS."
-;; We remove this spurious reference to save bandwidth when installing Go
-;; executables. It would be better to not embed the reference in the first
-;; place, but I'm not sure how to do that. The subject was discussed at:
-;; <https://lists.gnu.org/archive/html/guix-devel/2017-10/msg00207.html>
-  (if allow-go-reference?
-    #t
-    (let ((go (assoc-ref inputs "go"))
-          (bin "/bin"))
-      (for-each (lambda (output)
-                  (when (file-exists? (string-append (cdr output)
-                                                     bin))
-                    (for-each (lambda (file)
-                                (remove-store-reference file go))
-                              (find-files (string-append (cdr output) bin)))))
-                outputs)
-      #t)))
 
 (define %standard-phases
   (modify-phases gnu:%standard-phases
@@ -367,8 +317,7 @@ (define %standard-phases
     (replace 'build build)
     (replace 'check check)
     (replace 'install install)
-    (replace 'install-license-files install-license-files)
-    (add-after 'install 'remove-go-references remove-go-references)))
+    (replace 'install-license-files install-license-files)))
 
 (define* (go-build #:key inputs (phases %standard-phases)
                       #:allow-other-keys #:rest args)

base-commit: c0b303aaa3d6154acbe054120d11467eb98e6d33
-- 
2.41.0

[bug#68423] [PATCH] guix: go-build-system: use trimpath go flag

[Sharlatan Hellseher]

[-- Attachment #1: Type: text/plain, Size: 6522 bytes --]

Hi Ekaitz,

Thanks for the patch.

I think this is more suitable for core updates brunch as triggers rebuild
the world for all golang packages as far as I see.

Regards,
Oleg

On Sat, 13 Jan 2024, 11:48 Ekaitz Zarraga, <ekaitz@elenq.tech> wrote:

> Go 1.13 introduced[1] a new "trimpath" flag in charge of removing the
> references to the go toolchain in the produced ELFs. We used to remove
> these references using the "remove-go-reference" function. This function
> was executed after go install. By using this new trimpath flag, we don't
> have to remove any store path from the ELFs produced by go install.
>
> We're not using any go older than 1.13 anymore, it's safe to remove
> these functions.
>
> [1] https://go.dev/doc/go1.13
>
>     -trimpath
>         remove all file system paths from the resulting executable.
>         Instead of absolute file system paths, the recorded file names
>         will begin either a module path@version (when using modules),
>         or a plain import path (when using the standard library, or
>         GOPATH).
>
> * guix/build/go-build-system.scm (build): Add -trimpath
> (%standard-phases): Remove remove-go-references.
> (remove-go-references): Remove.
>
> Change-Id: Idcae366d226da5ce095693f81fd33133fd1d70d6
> Co-authored-by: Picnoir <picnoir@alternativebit.fr>
> ---
>  guix/build/go-build-system.scm | 59 +++-------------------------------
>  1 file changed, 4 insertions(+), 55 deletions(-)
>
> diff --git a/guix/build/go-build-system.scm
> b/guix/build/go-build-system.scm
> index 7f25e05d0d..70ddcd07cd 100644
> --- a/guix/build/go-build-system.scm
> +++ b/guix/build/go-build-system.scm
> @@ -6,6 +6,8 @@
>  ;;; Copyright © 2020 Jakub Kądziołka <kuba@kadziolka.net>
>  ;;; Copyright © 2020, 2021 Efraim Flashner <efraim@flashner.co.il>
>  ;;; Copyright © 2021 Sarah Morgensen <iskarian@mgsn.dev>
> +;;; Copyright © 2024 Ekaitz Zarraga <ekaitz@elenq.tech>
> +;;; Copyright © 2024 Picnoir <picnoir@alternativebit.fr>
>  ;;;
>  ;;; This file is part of GNU Guix.
>  ;;;
> @@ -90,7 +92,6 @@ (define-module (guix build go-build-system)
>  ;; * Use Go modules [4]
>  ;; * Re-use compiled packages [5]
>  ;; * Avoid the go-inputs hack
> -;; * Stop needing remove-go-references (-trimpath ? )
>  ;; * Remove module packages, only offering the full Git repos? This is
>  ;; more idiomatic, I think, because Go downloads Git repos, not modules.
>  ;; What are the trade-offs?
> @@ -265,6 +266,7 @@ (define* (build #:key import-path build-flags
> #:allow-other-keys)
>                ;; Respectively, strip the symbol table and debug
>                ;; information, and the DWARF symbol table.
>                "-ldflags=-s -w"
> +              "-trimpath"
>                `(,@build-flags ,import-path)))
>      (lambda (key . args)
>        (display (string-append "Building '" import-path "' failed.\n"
> @@ -304,58 +306,6 @@ (define* (install-license-files #:key unpack-path
>                                                      unpack-path))
>      (apply (assoc-ref gnu:%standard-phases 'install-license-files) args)))
>
> -(define* (remove-store-reference file file-name
> -                                  #:optional (store (%store-directory)))
> -  "Remove from FILE occurrences of FILE-NAME in STORE; return #t when
> FILE-NAME
> -is encountered in FILE, #f otherwise. This implementation reads FILE one
> byte at
> -a time, which is slow. Instead, we should use the Boyer-Moore string
> search
> -algorithm; there is an example in (guix build grafts)."
> -  (define pattern
> -    (string-take file-name
> -                 (+ 34 (string-length (%store-directory)))))
> -
> -  (with-fluids ((%default-port-encoding #f))
> -    (with-atomic-file-replacement file
> -      (lambda (in out)
> -        ;; We cannot use `regexp-exec' here because it cannot deal with
> -        ;; strings containing NUL characters.
> -        (format #t "removing references to `~a' from `~a'...~%" file-name
> file)
> -        (setvbuf in 'block 65536)
> -        (setvbuf out 'block 65536)
> -        (fold-port-matches (lambda (match result)
> -                             (put-bytevector out (string->utf8 store))
> -                             (put-u8 out (char->integer #\/))
> -                             (put-bytevector out
> -                                             (string->utf8
> -
> "eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-"))
> -                             #t)
> -                           #f
> -                           pattern
> -                           in
> -                           (lambda (char result)
> -                             (put-u8 out (char->integer char))
> -                             result))))))
> -
> -(define* (remove-go-references #:key allow-go-reference?
> -                               inputs outputs #:allow-other-keys)
> -  "Remove any references to the Go compiler from the compiled Go
> executable
> -files in OUTPUTS."
> -;; We remove this spurious reference to save bandwidth when installing Go
> -;; executables. It would be better to not embed the reference in the first
> -;; place, but I'm not sure how to do that. The subject was discussed at:
> -;; <https://lists.gnu.org/archive/html/guix-devel/2017-10/msg00207.html>
> -  (if allow-go-reference?
> -    #t
> -    (let ((go (assoc-ref inputs "go"))
> -          (bin "/bin"))
> -      (for-each (lambda (output)
> -                  (when (file-exists? (string-append (cdr output)
> -                                                     bin))
> -                    (for-each (lambda (file)
> -                                (remove-store-reference file go))
> -                              (find-files (string-append (cdr output)
> bin)))))
> -                outputs)
> -      #t)))
>
>  (define %standard-phases
>    (modify-phases gnu:%standard-phases
> @@ -367,8 +317,7 @@ (define %standard-phases
>      (replace 'build build)
>      (replace 'check check)
>      (replace 'install install)
> -    (replace 'install-license-files install-license-files)
> -    (add-after 'install 'remove-go-references remove-go-references)))
> +    (replace 'install-license-files install-license-files)))
>
>  (define* (go-build #:key inputs (phases %standard-phases)
>                        #:allow-other-keys #:rest args)
>
> base-commit: c0b303aaa3d6154acbe054120d11467eb98e6d33
> --
> 2.41.0
>
>
>
>

[-- Attachment #2: Type: text/html, Size: 8538 bytes --]

[bug#68423] [PATCH] guix: go-build-system: use trimpath go flag

[Ekaitz Zarraga]

Hi,

On 2024-01-13 13:58, Sharlatan Hellseher wrote:
> Hi Ekaitz,
> 
> Thanks for the patch.
> 
> I think this is more suitable for core updates brunch as triggers 
> rebuild the world for all golang packages as far as I see.

Yeah, probably.

We also thought after the patch that would be interesting to provide a 
way to disable this behavior. Or maybe it's ok?

With the previous alternative the packager would just remove the phase, 
but hardcoding the option as we did doesn't provide any way to do 
that...  Some kind of flag should be enough.

Are you ok with the patch, thought?

If this is ok we'll make those small changes and send a v2.

Thanks

[bug#68423] [PATCH] guix: go-build-system: use trimpath go flag

[Sharlatan Hellseher]

[-- Attachment #1: Type: text/plain, Size: 647 bytes --]

Hi Ekaitz,

> With the previous alternative the packager would just remove
> the phase, but hardcoding the option as we did doesn't provide
> any way to do that...  Some kind of flag should be enough.

It would be great to have more control over golang build envars
and build flags, like it happen in gnu-build-system or cmake-build-system
via #:make-flags and #:configure-flags.

Having some common defaults hardcoded and kind of #:build-flags as a way to
path anything else to compiler.

There is a similar thread referring go test flags, I've listed all in here:
https://mail.gnu.org/archive/html/guix-devel/2024-01/msg00235.html

Thanks,
Oleg

[-- Attachment #2: Type: text/html, Size: 1128 bytes --]

bug#68423: [PATCH] guix: go-build-system: use trimpath go flag

[Sharlatan Hellseher]

[-- Attachment #1: Type: text/plain, Size: 73 bytes --]


Pushed as 998588599aa46e384b664ec576ab124b83fc16bd to go-team.

--
Oleg

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]