all messages for Guix-related lists mirrored at yhetil.org
 help / color / mirror / code / Atom feed
From: Sarah Morgensen via Guix-patches via <guix-patches@gnu.org>
To: 49196@debbugs.gnu.org
Subject: [bug#49196] [PATCH] import: utils: 'recursive-import' skips unfound packages
Date: Wed, 23 Jun 2021 13:49:05 -0700	[thread overview]
Message-ID: <07d7149fc0f89f7f2d11fba47e1b0b2db5ceb809.1624479231.git.iskarian@mgsn.dev> (raw)

* guix/import/utils.scm (recursive-import): Skip packages when the
package returned by repo->guix-package is false.
* guix/import/go.scm (go-module-recursive-import): Explicitly return
false in 'repo->guix-package' when packages are not found.
* tests/import-utils.scm ("recursive-import: skip false packages"): New
test.
---
Hello Guix,

While trying to recursively import a go package I encountered the following
error when it was unable to find one of the dependencies:

--8<---------------cut here---------------start------------->8---
$ ./pre-inst-env guix import go -r git.sr.ht/~sircmpwn/aerc
guix import: warning: Failed to import package "github.com/brunnre8/go.notmuch".
reason: "https://pkg.go.dev/github.com/brunnre8/go.notmuch" could not be fetched: HTTP error 404 ("Not Found").
This package and its dependencies won't be imported.
following redirection to `https://github.com/ProtonMail/go-crypto?go-get=1'...
following redirection to `https://github.com/jtolio/gls?go-get=1'...
Backtrace:
In srfi/srfi-1.scm:
   586:29 19 (map1 ((package (name "go-github-com-rivo-uniseg") …) …))
   586:29 18 (map1 ((package (name "go-github-com-mattn-go-r…") …) …))
   586:29 17 (map1 ((package (name "go-github-com-miolini-da…") …) …))
   586:29 16 (map1 ((package (name "go-github-com-riywo-logi…") …) …))
   586:29 15 (map1 ((package (name "go-github-com-neelance-a…") …) …))
   586:29 14 (map1 ((package (name "go-github-com-neelance-s…") …) …))
   586:29 13 (map1 ((package (name "go-github-com-shurcool-go") …) …))
   586:29 12 (map1 ((package (name "go-github-com-shurcool-h…") …) …))
   586:29 11 (map1 ((package (name "go-github-com-shurcool-v…") …) …))
   586:29 10 (map1 ((package (name "go-github-com-gopherjs-g…") …) …))
   586:29  9 (map1 ((package (name "go-github-com-jtolds-gls") # …) …))
   586:29  8 (map1 ((package (name "go-github-com-smartystre…") …) …))
   586:29  7 (map1 ((package (name "go-github-com-smartystre…") …) …))
   586:29  6 (map1 ((package (name "go-github-com-google-go-…") …) …))
   586:29  5 (map1 ((package (name "go-google-golang-org-pro…") …) …))
   586:29  4 (map1 ((package (name "go-github-com-golang-pro…") …) …))
   586:29  3 (map1 ((package (name "go-google-golang-org-app…") …) …))
   586:29  2 (map1 ((package (name "go-github-com-protonmail…") …) …))
   586:17  1 (map1 (() (package (name "go-git-sr-ht-~sircmpwn…") …)))
In guix/import/utils.scm:
    280:2  0 (package->definition _ _)

guix/import/utils.scm:280:2: In procedure package->definition:
Throw to key `match-error' with args `("match" "no matching pattern" ())'.
--8<---------------cut here---------------end--------------->8---

It seems like several importers (gem, egg, go) expect to be able to return #f or
'() in #:repo->guix-package when a package is not found (while printing a
warning to the user) but recursive-import doesn't look like it handles it. I
suppose this worked before but was silently broken at some point. This patch
(re-)enables this behavior. I added a test for this as well.

Hope this helps,
Sarah

 guix/import/go.scm     |  3 ++-
 guix/import/utils.scm  | 14 ++++++++------
 tests/import-utils.scm | 18 ++++++++++++++++++
 3 files changed, 28 insertions(+), 7 deletions(-)

diff --git a/guix/import/go.scm b/guix/import/go.scm
index d110954664..c859098492 100644
--- a/guix/import/go.scm
+++ b/guix/import/go.scm
@@ -5,6 +5,7 @@
 ;;; Copyright © 2021 Maxim Cournoyer <maxim.cournoyer@gmail.com>
 ;;; Copyright © 2021 Ludovic Courtès <ludo@gnu.org>
 ;;; Copyright © 2021 Xinglu Chen <public@yoctocell.xyz>
+;;; Copyright © 2021 Sarah Morgensen <iskarian@mgsn.dev>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -651,7 +652,7 @@ This package and its dependencies won't be imported.~%")
                          (uri->string (http-get-error-uri c))
                          (http-get-error-code c)
                          (http-get-error-reason c))
-                (values '() '())))
+                (values #f '())))
        (receive (package-sexp dependencies)
            (go-module->guix-package* name #:goproxy goproxy
                                      #:version version
diff --git a/guix/import/utils.scm b/guix/import/utils.scm
index d817318a91..49f38cfa2a 100644
--- a/guix/import/utils.scm
+++ b/guix/import/utils.scm
@@ -8,6 +8,7 @@
 ;;; Copyright © 2020 Helio Machado <0x2b3bfa0+guix@googlemail.com>
 ;;; Copyright © 2020 Martin Becze <mjbecze@riseup.net>
 ;;; Copyright © 2021 Maxim Cournoyer <maxim.cournoyer@gmail.com>
+;;; Copyright © 2021 Sarah Morgensen <iskarian@mgsn.dev>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -469,16 +470,17 @@ to obtain the Guix package name corresponding to the upstream name."
            (normalized-deps (map (match-lambda
                                    ((name version) (list name version))
                                    (name (list name #f))) dependencies)))
-      (make-node name version package normalized-deps)))
+      (and package
+           (make-node name version package normalized-deps))))
 
   (map node-package
        (topological-sort (list (lookup-node package-name version))
                          (lambda (node)
-                           (map (lambda (name-version)
-                                  (apply lookup-node name-version))
-                                (remove (lambda (name-version)
-                                          (apply exists? name-version))
-                                        (node-dependencies node))))
+                           (filter-map (lambda (name-version)
+                                         (apply lookup-node name-version))
+                                       (remove (lambda (name-version)
+                                                 (apply exists? name-version))
+                                               (node-dependencies node))))
                          (lambda (node)
                            (string-append
                             (node-name node)
diff --git a/tests/import-utils.scm b/tests/import-utils.scm
index 874816442e..1b728104e0 100644
--- a/tests/import-utils.scm
+++ b/tests/import-utils.scm
@@ -2,6 +2,7 @@
 ;;; Copyright © 2015, 2017 Ricardo Wurmus <rekado@elephly.net>
 ;;; Copyright © 2016 Ben Woodcroft <donttrustben@gmail.com>
 ;;; Copyright © 2020 Martin Becze <mjbecze@riseup.net>
+;;; Copyright © 2021 Sarah Morgensen <iskarian@mgsn.dev>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -64,6 +65,23 @@
                                '())))
                     #:guix-name identity))
 
+(test-equal "recursive-import: skip false packages"
+  '((package
+      (name "foo")
+      (inputs `(("bar" ,bar)))))
+  (recursive-import "foo"
+                    #:repo 'repo
+                    #:repo->guix-package
+                    (match-lambda*
+                      (("foo" #:version #f #:repo 'repo)
+                       (values '(package
+                                  (name "foo")
+                                  (inputs `(("bar" ,bar))))
+                               '("bar")))
+                      (("bar" #:version #f #:repo 'repo)
+                       (values #f '())))
+                    #:guix-name identity))
+
 (test-assert "alist->package with simple source"
   (let* ((meta '(("name" . "hello")
                  ("version" . "2.10")

base-commit: acfa55a581ca4e688ee4b8f860fe365b1f153ef9
-- 
2.31.1





             reply	other threads:[~2021-06-23 20:50 UTC|newest]

Thread overview: 22+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-06-23 20:49 Sarah Morgensen via Guix-patches via [this message]
2021-06-24 12:21 ` [bug#49196] [PATCH] import: utils: 'recursive-import' skips unfound packages zimoun
2021-06-25  4:22   ` Sarah Morgensen via Guix-patches via
2021-06-25  6:53     ` zimoun
2021-06-25 16:47     ` zimoun
2021-06-25  4:48 ` [bug#49196] [PATCH v2] " Sarah Morgensen via Guix-patches via
2021-06-25 16:37 ` [bug#49196] [PATCH v3 1/3] import: go: Return false for package not found zimoun
2021-06-25 16:37   ` [bug#49196] [PATCH v3 2/3] import: utils: Skip not found packages zimoun
2021-06-25 16:37   ` [bug#49196] [PATCH v3 3/3] import: go: Improve error handling zimoun
2021-06-27  4:46     ` [bug#49196] [PATCH] import: utils: 'recursive-import' skips unfound packages Sarah Morgensen via Guix-patches via
2021-06-28 11:42       ` zimoun
2021-06-28 17:13         ` Sarah Morgensen via Guix-patches via
2021-06-28 16:20 ` [bug#49196] [PATCH v4 1/3] import: go: Return false for package not found zimoun
2021-06-28 16:20   ` [bug#49196] [PATCH v4 2/3] import: utils: Skip not found packages zimoun
2021-06-28 16:20   ` [bug#49196] [PATCH v4 3/3] import: go: Improve error handling zimoun
2021-06-29 10:52 ` [bug#49196] [PATCH v5 1/3] import: go: Return false for package not found zimoun
2021-06-29 10:52   ` [bug#49196] [PATCH v5 2/3] import: utils: Skip not found packages zimoun
2021-06-29 10:52   ` [bug#49196] [PATCH v5 3/3] import: go: Improve error handling zimoun
2021-08-06 18:04 ` [bug#49196] [PATCH v6 1/3] import: go: Return false for package not found Sarah Morgensen
2021-08-06 18:05   ` [bug#49196] [PATCH v6 2/3] import: utils: Skip not found packages Sarah Morgensen
2021-08-06 18:05   ` [bug#49196] [PATCH v6 3/3] import: go: Improve error handling Sarah Morgensen
2021-09-01 21:39   ` bug#49196: [PATCH] "guix import go" " Ludovic Courtès

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=07d7149fc0f89f7f2d11fba47e1b0b2db5ceb809.1624479231.git.iskarian@mgsn.dev \
    --to=guix-patches@gnu.org \
    --cc=49196@debbugs.gnu.org \
    --cc=iskarian@mgsn.dev \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this external index

	https://git.savannah.gnu.org/cgit/guix.git

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.