[bug#36948] [PATCH 0/2] Fix the CPAN importer

[bug#36948] [PATCH 0/2] Fix the CPAN importer

[Christopher Baines]

[-- Attachment #1: Type: text/plain, Size: 312 bytes --]


Christopher Baines (2):
  import: utils: Add hash-ref*.
  import: cpan: Adapt for the change to guile-json version 3.

 guix/import/cpan.scm  | 30 ++++++++++++++++--------------
 guix/import/utils.scm | 10 ++++++++++
 tests/cpan.scm        | 13 ++++++++-----
 3 files changed, 34 insertions(+), 19 deletions(-)

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 962 bytes --]

[bug#36948] [PATCH 1/2] import: utils: Add hash-ref*.

[Christopher Baines]

With the change to guile-json version 3, JSON objects are represented as hash
tables, rather than alists. The cpan importer uses assoc-ref* on a hash table,
so add an equivalent function for hash tables.

* guix/import/utils.scm (hash-ref*): New procedure.
---
 guix/import/utils.scm | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/guix/import/utils.scm b/guix/import/utils.scm
index 2a3b7341fb..ed6c3ce6af 100644
--- a/guix/import/utils.scm
+++ b/guix/import/utils.scm
@@ -47,6 +47,7 @@
 
             flatten
             assoc-ref*
+            hash-ref*
 
             url-fetch
             guix-hash-url
@@ -116,6 +117,15 @@ recursively apply the procedure to the sub-list."
       (assoc-ref alist key)
       (apply assoc-ref* (assoc-ref alist key) rest)))
 
+(define (hash-ref* hash-table key . rest)
+  "Return the value for KEY from HASH-TABLE.  For each additional key specified,
+recursively apply the procedure to the sub-hash-table."
+  (if (hash-table? hash-table)
+      (if (null? rest)
+          (hash-ref hash-table key)
+          (apply hash-ref* (hash-ref hash-table key) rest))
+      #f))                              ; For consistency with assoc-ref*
+
 (define (url-fetch url file-name)
   "Save the contents of URL to FILE-NAME.  Return #f on failure."
   (parameterize ((current-output-port (current-error-port)))
-- 
2.22.0

[bug#36948] [PATCH 2/2] import: cpan: Adapt for the change to guile-json version 3.

[Christopher Baines]

In guile-json version 3, JSON objects are represented as hash tables, rather
than alists.

* guix/import/cpan.scm (string->license): Change the match expression to match
on lists, rather than vectors.
(module->dist-name, cpan-source-url, cpan-version): Change assoc-ref to
hash-ref.
(cpan-module->sexp): Change assoc-ref to hash-ref, and assoc-ref* to
hash-ref*.
* tests/cpan.scm ("source-url-http", "source-url-https"): Convert the alist to
a hash table.
---
 guix/import/cpan.scm | 30 ++++++++++++++++--------------
 tests/cpan.scm       | 13 ++++++++-----
 2 files changed, 24 insertions(+), 19 deletions(-)

diff --git a/guix/import/cpan.scm b/guix/import/cpan.scm
index ec86f11743..0be37e715e 100644
--- a/guix/import/cpan.scm
+++ b/guix/import/cpan.scm
@@ -34,7 +34,7 @@
   #:use-module (guix ui)
   #:use-module ((guix download) #:select (download-to-store url-fetch))
   #:use-module ((guix import utils) #:select (factorize-uri
-                                              flatten assoc-ref*))
+                                              flatten hash-ref*))
   #:use-module (guix import json)
   #:use-module (guix packages)
   #:use-module (guix upstream)
@@ -76,8 +76,8 @@
    ;; ssleay
    ;; sun
    ("zlib" 'zlib)
-   (#(x) (string->license x))
-   (#(lst ...) `(list ,@(map string->license lst)))
+   ((x) (string->license x))
+   ((lst ...) `(list ,@(map string->license lst)))
    (_ #f)))
 
 (define (module->name module)
@@ -88,11 +88,11 @@
   "Return the base distribution module for a given module.  E.g. the 'ok'
 module is distributed with 'Test::Simple', so (module->dist-name \"ok\") would
 return \"Test-Simple\""
-  (assoc-ref (json-fetch (string-append
-                          "https://fastapi.metacpan.org/v1/module/"
-                          module
-                          "?fields=distribution"))
-             "distribution"))
+  (hash-ref (json-fetch (string-append
+                         "https://fastapi.metacpan.org/v1/module/"
+                         module
+                         "?fields=distribution"))
+            "distribution"))
 
 (define (package->upstream-name package)
   "Return the CPAN name of PACKAGE."
@@ -122,12 +122,12 @@ or #f on failure.  MODULE should be e.g. \"Test::Script\""
 (define (cpan-source-url meta)
   "Return the download URL for a module's source tarball."
   (regexp-substitute/global #f "http[s]?://cpan.metacpan.org"
-                            (assoc-ref meta "download_url")
+                            (hash-ref meta "download_url")
                             'pre "mirror://cpan" 'post))
 
 (define (cpan-version meta)
   "Return the version number from META."
-  (match (assoc-ref meta "version")
+  (match (hash-ref meta "version")
     ((? number? version)
      ;; version is sometimes not quoted in the module json, so it gets
      ;; imported into Guile as a number, so convert it to a string.
@@ -183,7 +183,7 @@ depend on (gnu packages perl)."
   "Return the `package' s-expression for a CPAN module from the metadata in
 META."
   (define name
-    (assoc-ref meta "distribution"))
+    (hash-ref meta "distribution"))
 
   (define (guix-name name)
     (if (string-prefix? "perl-" name)
@@ -198,7 +198,9 @@ META."
     (match (flatten
             (map (lambda (ph)
                    (filter-map (lambda (t)
-                                 (assoc-ref* meta "metadata" "prereqs" ph t))
+                                 (and=> (hash-ref* meta "metadata" "prereqs" ph t)
+                                        (lambda (h)
+                                          (hash-map->list cons h))))
                                '("requires" "recommends" "suggests")))
                  phases))
       (#f
@@ -251,9 +253,9 @@ META."
        ,@(maybe-inputs 'propagated-inputs
                        (convert-inputs '("runtime")))
        (home-page ,(cpan-home name))
-       (synopsis ,(assoc-ref meta "abstract"))
+       (synopsis ,(hash-ref meta "abstract"))
        (description fill-in-yourself!)
-       (license ,(string->license (assoc-ref meta "license"))))))
+       (license ,(string->license (hash-ref meta "license"))))))
 
 (define (cpan->guix-package module-name)
   "Fetch the metadata for PACKAGE-NAME from metacpan.org, and return the
diff --git a/tests/cpan.scm b/tests/cpan.scm
index 189dd027e6..cdd6c0e76a 100644
--- a/tests/cpan.scm
+++ b/tests/cpan.scm
@@ -24,7 +24,8 @@
   #:use-module (guix tests)
   #:use-module (guix grafts)
   #:use-module (srfi srfi-64)
-  #:use-module (ice-9 match))
+  #:use-module (ice-9 match)
+  #:use-module (ice-9 hash-table))
 
 ;; Globally disable grafts because they can trigger early builds.
 (%graft? #f)
@@ -109,14 +110,16 @@
 
 (test-equal "source-url-http"
   ((@@ (guix import cpan) cpan-source-url)
-   `(("download_url" .
-      "http://cpan.metacpan.org/authors/id/T/TE/TEST/Foo-Bar-0.1.tar.gz")))
+   (alist->hash-table
+    `(("download_url" .
+       "http://cpan.metacpan.org/authors/id/T/TE/TEST/Foo-Bar-0.1.tar.gz"))))
   "mirror://cpan/authors/id/T/TE/TEST/Foo-Bar-0.1.tar.gz")
 
 (test-equal "source-url-https"
   ((@@ (guix import cpan) cpan-source-url)
-   `(("download_url" .
-      "https://cpan.metacpan.org/authors/id/T/TE/TEST/Foo-Bar-0.1.tar.gz")))
+   (alist->hash-table
+    `(("download_url" .
+       "https://cpan.metacpan.org/authors/id/T/TE/TEST/Foo-Bar-0.1.tar.gz"))))
   "mirror://cpan/authors/id/T/TE/TEST/Foo-Bar-0.1.tar.gz")
 
 (test-end "cpan")
-- 
2.22.0

bug#36948: [PATCH 0/2] Fix the CPAN importer

[Christopher Baines]

[-- Attachment #1: Type: text/plain, Size: 450 bytes --]


Christopher Baines <mail@cbaines.net> writes:

> Christopher Baines (2):
>   import: utils: Add hash-ref*.
>   import: cpan: Adapt for the change to guile-json version 3.
>
>  guix/import/cpan.scm  | 30 ++++++++++++++++--------------
>  guix/import/utils.scm | 10 ++++++++++
>  tests/cpan.scm        | 13 ++++++++-----
>  3 files changed, 34 insertions(+), 19 deletions(-)

I've pushed these patches as 01ce7af25add55514f737af48ea6c127bedfde67
now.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 962 bytes --]

[bug#36948] [PATCH 1/2] import: utils: Add hash-ref*.

[Ludovic Courtès]

Hello Chris,

Christopher Baines <mail@cbaines.net> skribis:

> With the change to guile-json version 3, JSON objects are represented as hash
> tables, rather than alists. The cpan importer uses assoc-ref* on a hash table,
> so add an equivalent function for hash tables.
>
> * guix/import/utils.scm (hash-ref*): New procedure.

[...]

> In guile-json version 3, JSON objects are represented as hash tables, rather
> than alists.

I seems to me that this is adapting for Guile-JSON v1, not v3: in v3,
JSON arrays map to Scheme vectors, and JSON dictionaries map to alists;
JSON dictionaries used to map to hash tables in v1.

Indeed, ‘tests/cpan.scm’ now fails for me:

--8<---------------cut here---------------start------------->8---
actual-error:
+ (wrong-type-arg
+   "scm_hash_fn_get_handle"
+   "Wrong type argument in position ~A (expecting ~A): ~S"
+   (1
+    "hash-table"
+    (("version" . "0.1")
+     ("author" . "Guix")
+     ("download_url"
+      .
+      "http://example.com/Foo-Bar-0.1.tar.gz")
[…]
--8<---------------cut here---------------end--------------->8---

Could it be that you were testing this in an environment containing v1
and not v3?

Sorry for not noticing earlier!

Thanks,
Ludo’.