[bug#63647] [PATCH 0/2] Fix annoyance with "guix import go"

[bug#63647] [PATCH 0/2] Fix annoyance with "guix import go"

[Simon Tournier]

Hi,

As reported in [1], the Go importer returns backtrace.  This patch fixes
that.  The first patch is about sorting the versions.  The second is about
warn but not exit.  It allows to have a first draft when applying recursive
importer.

Compare (first patch),

--8<---------------cut here---------------start------------->8---
$ guix import go go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc@0.41
guix import: error: version v0.41 of go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc is not available

hint: Pick one of the following available versions: 0.26.0 0.36.0 0.15.0 0.41.0-rc.1 0.40.0 0.37.0 0.33.0 0.20.0 0.24.0 0.23.0 0.41.0 0.26.1 0.41.1 0.22.0
0.18.0 0.21.0 0.12.0 0.36.4 0.19.0 0.25.0 0.27.0 0.36.2 0.34.0 0.14.0 0.36.3 0.31.0 0.15.1 0.36.1 0.41.0-rc.2 0.32.0 0.28.0 0.35.0 0.42.0-rc.1 0.13.0
0.17.0 0.29.0 0.30.0 0.39.0 0.38.0 0.16.0.

$ ./pre-inst-env guix import go go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc@0.41
guix import: error: version v0.41 of go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc is not available

hint: Pick one of the following available versions: 0.42.0-rc.1 0.41.1 0.41.0-rc.2 0.41.0-rc.1 0.41.0 0.40.0 0.39.0 0.38.0 0.37.0 0.36.4 0.36.3 0.36.2 0.36.1
0.36.0 0.35.0 0.34.0 0.33.0 0.32.0 0.31.0 0.30.0 0.29.0 0.28.0 0.27.0 0.26.1 0.26.0 0.25.0 0.24.0 0.23.0 0.22.0 0.21.0 0.20.0 0.19.0 0.18.0 0.17.0
0.16.0 0.15.1 0.15.0 0.14.0 0.13.0 0.12.0.
--8<---------------cut here---------------end--------------->8---

Then instead of,

--8<---------------cut here---------------start------------->8---
$ guix import go go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc
Backtrace:
          14 (primitive-load "/home/simon/.config/guix/current/bin/guix")
In guix/ui.scm:
   2300:7 13 (run-guix . _)
  2263:10 12 (run-guix-command _ . _)
In guix/scripts/import.scm:
    89:11 11 (guix-import . _)
In ice-9/boot-9.scm:
  1752:10 10 (with-exception-handler _ _ #:unwind? _ #:unwind-for-type _)
In guix/scripts/import/go.scm:
   116:29  9 (_)
In ice-9/exceptions.scm:
   406:15  8 (go-module->guix-package* . _)
In ice-9/boot-9.scm:
  1752:10  7 (with-exception-handler _ _ #:unwind? _ #:unwind-for-type _)
In guix/import/go.scm:
   511:19  6 (go-module->guix-package "go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc" #:goproxy _ #:version _ # _)
In guix/git.scm:
    291:4  5 (update-cached-checkout _ #:ref _ #:recursive? _ #:check-out? _ #:starting-commit _ #:log-port _ #:cache-directory _)
   277:19  4 (resolve _)
In git/reference.scm:
     60:8  3 (_ _ _)
In git/bindings.scm:
     77:2  2 (raise-git-error _)
In ice-9/boot-9.scm:
  1685:16  1 (raise-exception _ #:continuable? _)
  1683:16  0 (raise-exception _ #:continuable? _)

ice-9/boot-9.scm:1683:16: In procedure raise-exception:
Git error: reference 'refs/tags/v0.41.1' not found
--8<---------------cut here---------------end--------------->8---

Now it returns (second patch),

--8<---------------cut here---------------start------------->8---
$ ./pre-inst-env guix import go go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc
guix import: warning: Git error: reference 'refs/tags/v0.41.1' not found in https://github.com/open-telemetry/opentelemetry-go-contrib
(define-public go-go-opentelemetry-io-contrib-instrumentation-google-golang-org-grpc-otelgrpc
  (package
    (name
     "go-go-opentelemetry-io-contrib-instrumentation-google-golang-org-grpc-otelgrpc")
    (version "0.41.1")
    (source (origin
              (method git-fetch)
              (uri (git-reference
                    (url
                     "https://github.com/open-telemetry/opentelemetry-go-contrib")
                    (commit (string-append "v" version))))
              (file-name (git-file-name name version))
              (sha256
               (base32
                "0000000000000000000000000000000000000000000000000000"))))
    (build-system go-build-system)
    (arguments
     '(#:import-path
       "go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc"))
    (propagated-inputs `(("go-gopkg-in-yaml-v3" ,go-gopkg-in-yaml-v3)
                         ("go-google-golang-org-genproto" ,go-google-golang-org-genproto)
                         ("go-google-golang-org-appengine" ,go-google-golang-org-appengine)
                         ("go-golang-org-x-text" ,go-golang-org-x-text)
                         ("go-golang-org-x-sys" ,go-golang-org-x-sys)
                         ("go-golang-org-x-oauth2" ,go-golang-org-x-oauth2)
                         ("go-golang-org-x-net" ,go-golang-org-x-net)
                         ("go-github-com-pmezard-go-difflib" ,go-github-com-pmezard-go-difflib)
                         ("go-github-com-golang-protobuf" ,go-github-com-golang-protobuf)
                         ("go-github-com-go-logr-stdr" ,go-github-com-go-logr-stdr)
                         ("go-github-com-go-logr-logr" ,go-github-com-go-logr-logr)
                         ("go-github-com-davecgh-go-spew" ,go-github-com-davecgh-go-spew)
                         ("go-cloud-google-com-go-compute-metadata" ,go-cloud-google-com-go-compute-metadata)
                         ("go-cloud-google-com-go-compute" ,go-cloud-google-com-go-compute)
                         ("go-google-golang-org-protobuf" ,go-google-golang-org-protobuf)
                         ("go-google-golang-org-grpc" ,go-google-golang-org-grpc)
                         ("go-go-opentelemetry-io-otel-trace" ,go-go-opentelemetry-io-otel-trace)
                         ("go-go-opentelemetry-io-otel-metric" ,go-go-opentelemetry-io-otel-metric)
                         ("go-go-opentelemetry-io-otel" ,go-go-opentelemetry-io-otel)
                         ("go-github-com-stretchr-testify" ,go-github-com-stretchr-testify)))
    (home-page
     "https://go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc")
    (synopsis #f)
    (description #f)
    (license license:asl2.0)))[env]
--8<---------------cut here---------------end--------------->8---


Cheers,
simon


Simon Tournier (2):
  guix: import: go: Sort hint about available versions.
  guix: import: go: Catch Git error.

 guix/import/go.scm | 30 ++++++++++++++++++++----------
 1 file changed, 20 insertions(+), 10 deletions(-)


base-commit: 849286ba66c96534bddc04df1a47d5692cbc977e
-- 
2.38.1

[bug#63647] [PATCH 1/2] guix: import: go: Sort hint about available versions.

[Simon Tournier]

* guix/import/go.scm (validate-version): Sort available versions.
---
 guix/import/go.scm | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/guix/import/go.scm b/guix/import/go.scm
index 0357e6a1eb..1943869162 100644
--- a/guix/import/go.scm
+++ b/guix/import/go.scm
@@ -34,6 +34,7 @@ (define-module (guix import go)
   #:use-module (guix packages)
   #:use-module (guix http-client)
   #:use-module (guix memoization)
+  #:use-module (guix utils)
   #:autoload   (htmlprag) (html->sxml)            ;from Guile-Lib
   #:autoload   (guix base32) (bytevector->nix-base32-string)
   #:autoload   (guix build utils) (mkdir-p)
@@ -588,8 +589,9 @@ (define (validate-version version available-versions module-path)
         (condition (&fix-hint
                     (hint (format #f (G_ "Pick one of the following \
 available versions:~{ ~a~}.")
-                                  (map strip-v-prefix
-                                       available-versions)))))))))
+                                  (sort (map strip-v-prefix
+                                             available-versions)
+                                        version>?)))))))))
 
 (define* (go-module->guix-package module-path #:key
                                   (goproxy "https://proxy.golang.org")
-- 
2.38.1

[bug#63647] [PATCH 2/2] guix: import: go: Catch Git error.

[Simon Tournier]

* guix/import/go.scm (git-checkout-hash): Catch Git error.

Reported-by: Felix Lechner <felix.lechner@lease-up.com>.
---
 guix/import/go.scm | 24 ++++++++++++++++--------
 1 file changed, 16 insertions(+), 8 deletions(-)

diff --git a/guix/import/go.scm b/guix/import/go.scm
index 1943869162..c6258296f6 100644
--- a/guix/import/go.scm
+++ b/guix/import/go.scm
@@ -6,7 +6,7 @@
 ;;; Copyright © 2021-2022 Ludovic Courtès <ludo@gnu.org>
 ;;; Copyright © 2021 Xinglu Chen <public@yoctocell.xyz>
 ;;; Copyright © 2021 Sarah Morgensen <iskarian@mgsn.dev>
-;;; Copyright © 2021 Simon Tournier <zimon.toutoune@gmail.com>
+;;; Copyright © 2021, 2023 Simon Tournier <zimon.toutoune@gmail.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -36,9 +36,11 @@ (define-module (guix import go)
   #:use-module (guix memoization)
   #:use-module (guix utils)
   #:autoload   (htmlprag) (html->sxml)            ;from Guile-Lib
-  #:autoload   (guix base32) (bytevector->nix-base32-string)
+  #:autoload   (guix base32) (bytevector->nix-base32-string nix-base32-string->bytevector)
   #:autoload   (guix build utils) (mkdir-p)
+  #:autoload   (guix ui) (warning)
   #:autoload   (gcrypt hash) (hash-algorithm sha256)
+  #:autoload   (git structs) (git-error-message)
   #:use-module (ice-9 format)
   #:use-module (ice-9 match)
   #:use-module (ice-9 peg)
@@ -507,12 +509,18 @@ (define* (git-checkout-hash url reference algorithm)
   ;; subsequent "guix import" invocations.
   (mkdir-p cache)
   (chmod cache #o700)
-  (let-values (((checkout commit _)
-                (parameterize ((%repository-cache-directory cache))
-                  (update-cached-checkout url
-                                          #:ref
-                                          `(tag-or-commit . ,reference)))))
-    (file-hash* checkout #:algorithm algorithm #:recursive? #true)))
+  (catch 'git-error
+    (lambda _
+      (let-values (((checkout commit _)
+                    (parameterize ((%repository-cache-directory cache))
+                      (update-cached-checkout url
+                                              #:ref
+                                              `(tag-or-commit . ,reference)))))
+        (file-hash* checkout #:algorithm algorithm #:recursive? #true)))
+    (lambda (key error . rest)
+      (warning (G_ "Git error: ~a in ~a~%") (git-error-message error) url)
+      (nix-base32-string->bytevector
+       "0000000000000000000000000000000000000000000000000000"))))
 
 (define (vcs->origin vcs-type vcs-repo-url version)
   "Generate the `origin' block of a package depending on what type of source
-- 
2.38.1

[bug#63647] [PATCH v2 1/3] guix: import: go: Sort hint about available versions.

[Simon Tournier]

* guix/import/go.scm (validate-version): Sort available versions.
---
 guix/import/go.scm | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/guix/import/go.scm b/guix/import/go.scm
index 0357e6a1eb..1943869162 100644
--- a/guix/import/go.scm
+++ b/guix/import/go.scm
@@ -34,6 +34,7 @@ (define-module (guix import go)
   #:use-module (guix packages)
   #:use-module (guix http-client)
   #:use-module (guix memoization)
+  #:use-module (guix utils)
   #:autoload   (htmlprag) (html->sxml)            ;from Guile-Lib
   #:autoload   (guix base32) (bytevector->nix-base32-string)
   #:autoload   (guix build utils) (mkdir-p)
@@ -588,8 +589,9 @@ (define (validate-version version available-versions module-path)
         (condition (&fix-hint
                     (hint (format #f (G_ "Pick one of the following \
 available versions:~{ ~a~}.")
-                                  (map strip-v-prefix
-                                       available-versions)))))))))
+                                  (sort (map strip-v-prefix
+                                             available-versions)
+                                        version>?)))))))))
 
 (define* (go-module->guix-package module-path #:key
                                   (goproxy "https://proxy.golang.org")

base-commit: 99fc7e5dd3c9c92f9b6c2e51db9dc5b0481a56b2
-- 
2.38.1

[bug#63647] [PATCH v2 2/3] guix: import: go: Catch Git error.

[Simon Tournier]

* guix/import/go.scm (git-checkout-hash): Catch Git error.

Reported-by: Felix Lechner <felix.lechner@lease-up.com>.
---
 guix/import/go.scm | 24 ++++++++++++++++--------
 1 file changed, 16 insertions(+), 8 deletions(-)

diff --git a/guix/import/go.scm b/guix/import/go.scm
index 1943869162..c6258296f6 100644
--- a/guix/import/go.scm
+++ b/guix/import/go.scm
@@ -6,7 +6,7 @@
 ;;; Copyright © 2021-2022 Ludovic Courtès <ludo@gnu.org>
 ;;; Copyright © 2021 Xinglu Chen <public@yoctocell.xyz>
 ;;; Copyright © 2021 Sarah Morgensen <iskarian@mgsn.dev>
-;;; Copyright © 2021 Simon Tournier <zimon.toutoune@gmail.com>
+;;; Copyright © 2021, 2023 Simon Tournier <zimon.toutoune@gmail.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -36,9 +36,11 @@ (define-module (guix import go)
   #:use-module (guix memoization)
   #:use-module (guix utils)
   #:autoload   (htmlprag) (html->sxml)            ;from Guile-Lib
-  #:autoload   (guix base32) (bytevector->nix-base32-string)
+  #:autoload   (guix base32) (bytevector->nix-base32-string nix-base32-string->bytevector)
   #:autoload   (guix build utils) (mkdir-p)
+  #:autoload   (guix ui) (warning)
   #:autoload   (gcrypt hash) (hash-algorithm sha256)
+  #:autoload   (git structs) (git-error-message)
   #:use-module (ice-9 format)
   #:use-module (ice-9 match)
   #:use-module (ice-9 peg)
@@ -507,12 +509,18 @@ (define* (git-checkout-hash url reference algorithm)
   ;; subsequent "guix import" invocations.
   (mkdir-p cache)
   (chmod cache #o700)
-  (let-values (((checkout commit _)
-                (parameterize ((%repository-cache-directory cache))
-                  (update-cached-checkout url
-                                          #:ref
-                                          `(tag-or-commit . ,reference)))))
-    (file-hash* checkout #:algorithm algorithm #:recursive? #true)))
+  (catch 'git-error
+    (lambda _
+      (let-values (((checkout commit _)
+                    (parameterize ((%repository-cache-directory cache))
+                      (update-cached-checkout url
+                                              #:ref
+                                              `(tag-or-commit . ,reference)))))
+        (file-hash* checkout #:algorithm algorithm #:recursive? #true)))
+    (lambda (key error . rest)
+      (warning (G_ "Git error: ~a in ~a~%") (git-error-message error) url)
+      (nix-base32-string->bytevector
+       "0000000000000000000000000000000000000000000000000000"))))
 
 (define (vcs->origin vcs-type vcs-repo-url version)
   "Generate the `origin' block of a package depending on what type of source
-- 
2.38.1

[bug#63647] [PATCH v2 3/3] guix: import: go: Use correct tag for go module in subdirectory.

[Simon Tournier]

From: Stephen Paul Weber <singpolyma@singpolyma.net>

* guix/import/go.scm (vcs->origin): New argument module-path-subdirectory.
Use it to get Git tag as described by <https://go.dev/ref/mod>.

Signed-off-by: Simon Tournier <zimon.toutoune@gmail.com>
---
 guix/import/go.scm | 34 +++++++++++++++++++++++-----------
 1 file changed, 23 insertions(+), 11 deletions(-)

diff --git a/guix/import/go.scm b/guix/import/go.scm
index c6258296f6..f02c098f1c 100644
--- a/guix/import/go.scm
+++ b/guix/import/go.scm
@@ -6,6 +6,7 @@
 ;;; Copyright © 2021-2022 Ludovic Courtès <ludo@gnu.org>
 ;;; Copyright © 2021 Xinglu Chen <public@yoctocell.xyz>
 ;;; Copyright © 2021 Sarah Morgensen <iskarian@mgsn.dev>
+;;; Copyright © 2022 Stephen Paul Weber <singpolyma@singpolyma.net>
 ;;; Copyright © 2021, 2023 Simon Tournier <zimon.toutoune@gmail.com>
 ;;;
 ;;; This file is part of GNU Guix.
@@ -522,28 +523,34 @@ (define* (git-checkout-hash url reference algorithm)
       (nix-base32-string->bytevector
        "0000000000000000000000000000000000000000000000000000"))))
 
-(define (vcs->origin vcs-type vcs-repo-url version)
+(define (vcs->origin vcs-type vcs-repo-url module-path-subdirectory version)
   "Generate the `origin' block of a package depending on what type of source
 control system is being used."
   (case vcs-type
     ((git)
-     (let ((plain-version? (string=? version (go-version->git-ref version)))
-           (v-prefixed?    (string-prefix? "v" version)))
+     (let* ((plain-version? (string=? version (go-version->git-ref version)))
+            (v-prefixed?    (string-prefix? "v" version))
+            ;; A module in a subdirectory has a tag prefixed by this
+            ;; subdirectory.  See <https://go.dev/ref/mod>.
+            (tag-prefix     (if (string=? module-path-subdirectory "") ;
+                                ""
+                                (string-append
+                                 (substring module-path-subdirectory 1) "/")))
+            (git-commit     (if plain-version?
+                                (string-append tag-prefix version)
+                                (go-version->git-ref version))))
        `(origin
           (method git-fetch)
           (uri (git-reference
                 (url ,vcs-repo-url)
-                ;; This is done because the version field of the package,
-                ;; which the generated quoted expression refers to, has been
-                ;; stripped of any 'v' prefixed.
                 (commit ,(if (and plain-version? v-prefixed?)
-                             '(string-append "v" version)
+                             `(string-append ,tag-prefix "v" version)
                              '(go-version->git-ref version)))))
           (file-name (git-file-name name version))
           (sha256
            (base32
             ,(bytevector->nix-base32-string
-              (git-checkout-hash vcs-repo-url (go-version->git-ref version)
+              (git-checkout-hash vcs-repo-url git-commit
                                  (hash-algorithm sha256))))))))
     ((hg)
      `(origin
@@ -621,12 +628,17 @@ (define* (go-module->guix-package module-path #:key
                            dependencies+versions
                            (map car dependencies+versions)))
          (module-path-sans-suffix
-          (match:prefix (string-match "([\\./]v[0-9]+)?$" module-path)))
+           (if (string-prefix? "gopkg.in" module-path)
+               module-path
+               (match:prefix (string-match "([\\./]v[0-9]+)?$" module-path))))
          (guix-name (go-module->guix-package-name module-path))
-         (root-module-path (module-path->repository-root module-path))
+         (root-module-path (module-path->repository-root module-path-sans-suffix))
          ;; The VCS type and URL are not included in goproxy information. For
          ;; this we need to fetch it from the official module page.
          (meta-data (fetch-module-meta-data root-module-path))
+         (module-path-subdirectory
+           (substring module-path-sans-suffix
+             (string-length (module-meta-import-prefix meta-data))))
          (vcs-type (module-meta-vcs meta-data))
          (vcs-repo-url (module-meta-data-repo-url meta-data goproxy))
          (synopsis (go-package-synopsis module-path))
@@ -637,7 +649,7 @@ (define* (go-module->guix-package module-path #:key
         (name ,guix-name)
         (version ,(strip-v-prefix version*))
         (source
-         ,(vcs->origin vcs-type vcs-repo-url version*))
+         ,(vcs->origin vcs-type vcs-repo-url module-path-subdirectory version*))
         (build-system go-build-system)
         (arguments
          '(#:import-path ,module-path
-- 
2.38.1

[bug#63647] [PATCH v3 1/3] guix: import: go: Sort hint about available versions.

[Simon Tournier]

* guix/import/go.scm (validate-version): Sort available versions.
---
 guix/import/go.scm | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/guix/import/go.scm b/guix/import/go.scm
index 0357e6a1eb..1943869162 100644
--- a/guix/import/go.scm
+++ b/guix/import/go.scm
@@ -34,6 +34,7 @@ (define-module (guix import go)
   #:use-module (guix packages)
   #:use-module (guix http-client)
   #:use-module (guix memoization)
+  #:use-module (guix utils)
   #:autoload   (htmlprag) (html->sxml)            ;from Guile-Lib
   #:autoload   (guix base32) (bytevector->nix-base32-string)
   #:autoload   (guix build utils) (mkdir-p)
@@ -588,8 +589,9 @@ (define (validate-version version available-versions module-path)
         (condition (&fix-hint
                     (hint (format #f (G_ "Pick one of the following \
 available versions:~{ ~a~}.")
-                                  (map strip-v-prefix
-                                       available-versions)))))))))
+                                  (sort (map strip-v-prefix
+                                             available-versions)
+                                        version>?)))))))))
 
 (define* (go-module->guix-package module-path #:key
                                   (goproxy "https://proxy.golang.org")

base-commit: 99fc7e5dd3c9c92f9b6c2e51db9dc5b0481a56b2
-- 
2.38.1

[bug#63647] [PATCH v3 2/3] guix: import: go: Catch Git error.

[Simon Tournier]

* guix/import/go.scm (git-checkout-hash): Catch Git error.

Reported-by: Felix Lechner <felix.lechner@lease-up.com>.
---
 guix/import/go.scm | 24 ++++++++++++++++--------
 1 file changed, 16 insertions(+), 8 deletions(-)

diff --git a/guix/import/go.scm b/guix/import/go.scm
index 1943869162..c6258296f6 100644
--- a/guix/import/go.scm
+++ b/guix/import/go.scm
@@ -6,7 +6,7 @@
 ;;; Copyright © 2021-2022 Ludovic Courtès <ludo@gnu.org>
 ;;; Copyright © 2021 Xinglu Chen <public@yoctocell.xyz>
 ;;; Copyright © 2021 Sarah Morgensen <iskarian@mgsn.dev>
-;;; Copyright © 2021 Simon Tournier <zimon.toutoune@gmail.com>
+;;; Copyright © 2021, 2023 Simon Tournier <zimon.toutoune@gmail.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -36,9 +36,11 @@ (define-module (guix import go)
   #:use-module (guix memoization)
   #:use-module (guix utils)
   #:autoload   (htmlprag) (html->sxml)            ;from Guile-Lib
-  #:autoload   (guix base32) (bytevector->nix-base32-string)
+  #:autoload   (guix base32) (bytevector->nix-base32-string nix-base32-string->bytevector)
   #:autoload   (guix build utils) (mkdir-p)
+  #:autoload   (guix ui) (warning)
   #:autoload   (gcrypt hash) (hash-algorithm sha256)
+  #:autoload   (git structs) (git-error-message)
   #:use-module (ice-9 format)
   #:use-module (ice-9 match)
   #:use-module (ice-9 peg)
@@ -507,12 +509,18 @@ (define* (git-checkout-hash url reference algorithm)
   ;; subsequent "guix import" invocations.
   (mkdir-p cache)
   (chmod cache #o700)
-  (let-values (((checkout commit _)
-                (parameterize ((%repository-cache-directory cache))
-                  (update-cached-checkout url
-                                          #:ref
-                                          `(tag-or-commit . ,reference)))))
-    (file-hash* checkout #:algorithm algorithm #:recursive? #true)))
+  (catch 'git-error
+    (lambda _
+      (let-values (((checkout commit _)
+                    (parameterize ((%repository-cache-directory cache))
+                      (update-cached-checkout url
+                                              #:ref
+                                              `(tag-or-commit . ,reference)))))
+        (file-hash* checkout #:algorithm algorithm #:recursive? #true)))
+    (lambda (key error . rest)
+      (warning (G_ "Git error: ~a in ~a~%") (git-error-message error) url)
+      (nix-base32-string->bytevector
+       "0000000000000000000000000000000000000000000000000000"))))
 
 (define (vcs->origin vcs-type vcs-repo-url version)
   "Generate the `origin' block of a package depending on what type of source
-- 
2.38.1

[bug#63647] [PATCH 0/2] Fix annoyance with "guix import go"

[Ludovic Courtès]

Simon Tournier <zimon.toutoune@gmail.com> skribis:

> * guix/import/go.scm (git-checkout-hash): Catch Git error.

[...]

> +  (catch 'git-error
> +    (lambda _
> +      (let-values (((checkout commit _)
> +                    (parameterize ((%repository-cache-directory cache))
> +                      (update-cached-checkout url
> +                                              #:ref
> +                                              `(tag-or-commit . ,reference)))))
> +        (file-hash* checkout #:algorithm algorithm #:recursive? #true)))
> +    (lambda (key error . rest)
> +      (warning (G_ "Git error: ~a in ~a~%") (git-error-message error) url)
> +      (nix-base32-string->bytevector
> +       "0000000000000000000000000000000000000000000000000000"))))

I’d rather let the exception through.  How about adding
‘with-git-error-handling’ at the UI level, in (guix scripts import go)?

Thanks,
Ludo’.

[bug#63647] [PATCH 0/2] Fix annoyance with "guix import go"

[Simon Tournier]

Hi Ludo,

On Mon, 05 Jun 2023 at 14:45, Ludovic Courtès <ludo@gnu.org> wrote:
> Simon Tournier <zimon.toutoune@gmail.com> skribis:
>
>> * guix/import/go.scm (git-checkout-hash): Catch Git error.
>
> [...]
>
>> +  (catch 'git-error
>> +    (lambda _
>> +      (let-values (((checkout commit _)
>> +                    (parameterize ((%repository-cache-directory cache))
>> +                      (update-cached-checkout url
>> +                                              #:ref
>> +                                              `(tag-or-commit . ,reference)))))
>> +        (file-hash* checkout #:algorithm algorithm #:recursive? #true)))
>> +    (lambda (key error . rest)
>> +      (warning (G_ "Git error: ~a in ~a~%") (git-error-message error) url)
>> +      (nix-base32-string->bytevector
>> +       "0000000000000000000000000000000000000000000000000000"))))
>
> I’d rather let the exception through.  How about adding
> ‘with-git-error-handling’ at the UI level, in (guix scripts import go)?

What do you mean by “let the exception through”?  It seems better to be
non-blocking and thus catch the exception then raise a meaningful
warning; it’s required when running with the option recursive.

Well, maybe an improvement could be in the addition of some
’report-git-warning’ and/or ’with-git-error-handling*’, in (guix git);
hum, I do not know.

Last, considering that the module (guix import go) already contains 4 UI
messages (G_), trying to move this warning about Git to (guix scripts
import go) will add some complexity – re-raise the exception 2 or 3
times, IIUC – and thus it will not change much about the UI, IMHO.  I
mean, such move should be for all the messages or nothing.

Cheers,
simon

[bug#63647] [PATCH 0/2] Fix annoyance with "guix import go"

[Ludovic Courtès]

Hello!

Simon Tournier <zimon.toutoune@gmail.com> skribis:

> On Mon, 05 Jun 2023 at 14:45, Ludovic Courtès <ludo@gnu.org> wrote:
>> Simon Tournier <zimon.toutoune@gmail.com> skribis:
>>
>>> * guix/import/go.scm (git-checkout-hash): Catch Git error.
>>
>> [...]
>>
>>> +  (catch 'git-error
>>> +    (lambda _
>>> +      (let-values (((checkout commit _)
>>> +                    (parameterize ((%repository-cache-directory cache))
>>> +                      (update-cached-checkout url
>>> +                                              #:ref
>>> +                                              `(tag-or-commit . ,reference)))))
>>> +        (file-hash* checkout #:algorithm algorithm #:recursive? #true)))
>>> +    (lambda (key error . rest)
>>> +      (warning (G_ "Git error: ~a in ~a~%") (git-error-message error) url)
>>> +      (nix-base32-string->bytevector
>>> +       "0000000000000000000000000000000000000000000000000000"))))
>>
>> I’d rather let the exception through.  How about adding
>> ‘with-git-error-handling’ at the UI level, in (guix scripts import go)?
>
> What do you mean by “let the exception through”?  It seems better to be
> non-blocking and thus catch the exception then raise a meaningful
> warning; it’s required when running with the option recursive.

I thought it would be more appropriate to error out as soon as we have a
Git problem, rather than print a warning and emit an incorrect hash.

I understand that, when using ‘--recursive’, this means aborting the
whole process without producing anything.  But maybe that’s better that
producing an incorrect (origin …) form?

Now, I don’t use ‘guix import go -r’ so it’s possible that I don’t
understand the scenario being considered here!

Thanks,
Ludo’.

[bug#63647] [PATCH v3 3/3] guix: import: go: Use correct tag for go module in subdirectory.

[Simon Tournier]

From: Stephen Paul Weber <singpolyma@singpolyma.net>

* guix/import/go.scm (vcs->origin): New argument module-path-subdirectory.
Use it to get Git tag as described by <https://go.dev/ref/mod>.

Co-authored-by: Simon Tournier <zimon.toutoune@gmail.com>
Signed-off-by: Simon Tournier <zimon.toutoune@gmail.com>
---
 guix/import/go.scm | 35 ++++++++++++++++++++++++-----------
 1 file changed, 24 insertions(+), 11 deletions(-)

diff --git a/guix/import/go.scm b/guix/import/go.scm
index c6258296f6..c2f9065d94 100644
--- a/guix/import/go.scm
+++ b/guix/import/go.scm
@@ -6,6 +6,7 @@
 ;;; Copyright © 2021-2022 Ludovic Courtès <ludo@gnu.org>
 ;;; Copyright © 2021 Xinglu Chen <public@yoctocell.xyz>
 ;;; Copyright © 2021 Sarah Morgensen <iskarian@mgsn.dev>
+;;; Copyright © 2022 Stephen Paul Weber <singpolyma@singpolyma.net>
 ;;; Copyright © 2021, 2023 Simon Tournier <zimon.toutoune@gmail.com>
 ;;;
 ;;; This file is part of GNU Guix.
@@ -522,28 +523,35 @@ (define* (git-checkout-hash url reference algorithm)
       (nix-base32-string->bytevector
        "0000000000000000000000000000000000000000000000000000"))))
 
-(define (vcs->origin vcs-type vcs-repo-url version)
+(define (vcs->origin vcs-type vcs-repo-url module-path-subdirectory version)
   "Generate the `origin' block of a package depending on what type of source
 control system is being used."
   (case vcs-type
     ((git)
-     (let ((plain-version? (string=? version (go-version->git-ref version)))
-           (v-prefixed?    (string-prefix? "v" version)))
+     (let* ((plain-version? (string=? version (go-version->git-ref version)))
+            (v-prefixed?    (string-prefix? "v" version))
+            ;; A module in a subdirectory has a tag prefixed by this
+            ;; subdirectory.  See <https://go.dev/ref/mod>.
+            (tag-prefix     (if (string=? module-path-subdirectory "") ;
+                                ""
+                                (string-append
+                                 (substring module-path-subdirectory 1) "/")))
+            (prefix         (string-append tag-prefix "v"))
+            (git-commit     (if plain-version?
+                                (string-append tag-prefix version)
+                                (go-version->git-ref version))))
        `(origin
           (method git-fetch)
           (uri (git-reference
                 (url ,vcs-repo-url)
-                ;; This is done because the version field of the package,
-                ;; which the generated quoted expression refers to, has been
-                ;; stripped of any 'v' prefixed.
                 (commit ,(if (and plain-version? v-prefixed?)
-                             '(string-append "v" version)
+                             `(string-append ,prefix version)
                              '(go-version->git-ref version)))))
           (file-name (git-file-name name version))
           (sha256
            (base32
             ,(bytevector->nix-base32-string
-              (git-checkout-hash vcs-repo-url (go-version->git-ref version)
+              (git-checkout-hash vcs-repo-url git-commit
                                  (hash-algorithm sha256))))))))
     ((hg)
      `(origin
@@ -621,12 +629,17 @@ (define* (go-module->guix-package module-path #:key
                            dependencies+versions
                            (map car dependencies+versions)))
          (module-path-sans-suffix
-          (match:prefix (string-match "([\\./]v[0-9]+)?$" module-path)))
+           (if (string-prefix? "gopkg.in" module-path)
+               module-path
+               (match:prefix (string-match "([\\./]v[0-9]+)?$" module-path))))
          (guix-name (go-module->guix-package-name module-path))
-         (root-module-path (module-path->repository-root module-path))
+         (root-module-path (module-path->repository-root module-path-sans-suffix))
          ;; The VCS type and URL are not included in goproxy information. For
          ;; this we need to fetch it from the official module page.
          (meta-data (fetch-module-meta-data root-module-path))
+         (module-path-subdirectory
+           (substring module-path-sans-suffix
+             (string-length (module-meta-import-prefix meta-data))))
          (vcs-type (module-meta-vcs meta-data))
          (vcs-repo-url (module-meta-data-repo-url meta-data goproxy))
          (synopsis (go-package-synopsis module-path))
@@ -637,7 +650,7 @@ (define* (go-module->guix-package module-path #:key
         (name ,guix-name)
         (version ,(strip-v-prefix version*))
         (source
-         ,(vcs->origin vcs-type vcs-repo-url version*))
+         ,(vcs->origin vcs-type vcs-repo-url module-path-subdirectory version*))
         (build-system go-build-system)
         (arguments
          '(#:import-path ,module-path
-- 
2.38.1

[bug#63647] [PATCH 0/2] Fix annoyance with "guix import go"

[Ludovic Courtès]

Hi,

Simon Tournier <zimon.toutoune@gmail.com> skribis:

> * guix/import/go.scm (validate-version): Sort available versions.

LGTM!

> From: Stephen Paul Weber <singpolyma@singpolyma.net>
>
> * guix/import/go.scm (vcs->origin): New argument module-path-subdirectory.
> Use it to get Git tag as described by <https://go.dev/ref/mod>.
>
> Co-authored-by: Simon Tournier <zimon.toutoune@gmail.com>
> Signed-off-by: Simon Tournier <zimon.toutoune@gmail.com>

[...]

> -(define (vcs->origin vcs-type vcs-repo-url version)
> +(define (vcs->origin vcs-type vcs-repo-url module-path-subdirectory version)
>    "Generate the `origin' block of a package depending on what type of source
>  control system is being used."

Maybe just ‘subdirectory’?  (In general ‘path’ is used for search paths,
not for file names, but I see this module already used the term with
that meaning.)

Also please mention it in the docstring.

Should there be a test, or is it too tricky?

Thanks,
Ludo’.

[bug#63647] [PATCH 0/2] Fix annoyance with "guix import go"

[Fabio Natali via Guix-patches via]

Hi All,

I've been recently trying to update Restic with 'guix import go
--recursive github.com/restic/restic'. (I wasn't expecting this to be an
easy task, by the way. I knew there are difficulties that have kept us
on an old version for a long time. https://issues.guix.gnu.org/63019)

Apparently, the import fails on those Go repositories that use a
subdirectory/subpackage structure - which brought me to this thread.

I was wondering if any of the patches proposed here (or in related
#52362, #63001, #64035, #64036) might be worth some final clean up and
merge - even if it addresses a subset of the problems?

Thanks, cheers, Fabio.

[bug#63647] [PATCH 0/2] Fix annoyance with "guix import go"

[Fabio Natali via Guix-patches via]

Hi All,

I've been recently trying to update Restic with 'guix import go
--recursive github.com/restic/restic'. (I wasn't expecting this to be an
easy task, by the way. I knew there are difficulties that have kept us
on an old version for a long time. https://issues.guix.gnu.org/63019)

Apparently, the import fails on those Go repositories that use a
subdirectory/subpackage structure - which brought me to this thread.

I was wondering if any of the patches proposed here (or in related
#52362, #63001, #64035, #64036) might be worth some final clean up and
merge - even if it addresses a subset of the problems?

Thanks, cheers, Fabio.

[bug#63647] [PATCH 0/2] Fix annoyance with "guix import go"

[Sharlatan Hellseher]

[-- Attachment #1: Type: text/plain, Size: 108 bytes --]


Hi,

Please see this <https://issues.guix.gnu.org/69827> issue which I try to
merge on go-branch.

--
Oleg

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]