[bug#37927]

[bug#37927]

[Brice Waegeneire]

This patch should allow correctly importing most of the licenses from
crates.io. Before it, importing recursively ripgrep would result in ~140
missing licenses; now none seems missing.

[bug#37928] [PATCH] import: crate: Fix licenses.

[Brice Waegeneire]

* guix/import/crate.scm (%dual-license-rx): Removed function.
(crate->guix-package): Handle most of the multi-licensing cases.
---
 guix/import/crate.scm | 15 ++++++---------
 1 file changed, 6 insertions(+), 9 deletions(-)

diff --git a/guix/import/crate.scm b/guix/import/crate.scm
index 8dc014d232..e08028db15 100644
--- a/guix/import/crate.scm
+++ b/guix/import/crate.scm
@@ -178,21 +178,18 @@ and LICENSE."
          (close-port port)
          pkg))
 
-(define %dual-license-rx
-  ;; Dual licensing is represented by a string such as "MIT OR Apache-2.0".
-  ;; This regexp matches that.
-  (make-regexp "^(.*) OR (.*)$"))
-
 (define* (crate->guix-package crate-name #:optional version)
   "Fetch the metadata for CRATE-NAME from crates.io, and return the
 `package' s-expression corresponding to that package, or #f on failure.
 When VERSION is specified, attempt to fetch that version; otherwise fetch the
 latest version of CRATE-NAME."
   (define (string->license string)
-    (match (regexp-exec %dual-license-rx string)
-      (#f (list (spdx-string->license string)))
-      (m  (list (spdx-string->license (match:substring m 1))
-                (spdx-string->license (match:substring m 2))))))
+    (filter
+     (lambda (word)
+       (and (not (string-null? word))
+            (not (any (lambda (elem) (string=? elem word))
+                      '("AND" "OR" "WITH"))) ))
+     (string-split string (string->char-set " /"))))
 
   (define (normal-dependency? dependency)
     (eq? (crate-dependency-kind dependency) 'normal))
-- 
2.19.2

[bug#37927] [bug#37928] [PATCH] import: crate: Fix licenses.

[Ludovic Courtès]

Hi Brice,

Brice Waegeneire <brice@waegenei.re> skribis:

> * guix/import/crate.scm (%dual-license-rx): Removed function.
> (crate->guix-package): Handle most of the multi-licensing cases.

Nice!

>  (define* (crate->guix-package crate-name #:optional version)
>    "Fetch the metadata for CRATE-NAME from crates.io, and return the
>  `package' s-expression corresponding to that package, or #f on failure.
>  When VERSION is specified, attempt to fetch that version; otherwise fetch the
>  latest version of CRATE-NAME."
>    (define (string->license string)
> -    (match (regexp-exec %dual-license-rx string)
> -      (#f (list (spdx-string->license string)))
> -      (m  (list (spdx-string->license (match:substring m 1))
> -                (spdx-string->license (match:substring m 2))))))
> +    (filter
> +     (lambda (word)
> +       (and (not (string-null? word))
> +            (not (any (lambda (elem) (string=? elem word))
> +                      '("AND" "OR" "WITH"))) ))
> +     (string-split string (string->char-set " /"))))

It would be great to have tests for that in tests/crate.scm.  To that
end, I think you could lift ‘string->license’ to the top level (that is,
outside ‘crate->guix-package’), and then have a few tests along these
lines:

  (define string->license
    (@@ (guix import crate) string->license))

  (test-equal "GPL OR LGPL"
    (list license:gpl3+ license:lgpl3+)
    (string->license "GPL OR LGPL"))

(See
<https://guix.gnu.org/manual/en/html_node/Running-the-Test-Suite.html>.)

Let me know if anything is unclear.

Could you send an updated patch?

Thanks,
Ludo’.

[bug#37927] [bug#37928] [PATCH] import: crate: Fix licenses.

[Ludovic Courtès]

Hi Brice,

Did you have a chance to look into this?

(See <https://issues.guix.gnu.org/issue/37927> for context.)

Thanks in advance.  :-)
Ludo’.

Ludovic Courtès <ludo@gnu.org> skribis:

> Hi Brice,
>
> Brice Waegeneire <brice@waegenei.re> skribis:
>
>> * guix/import/crate.scm (%dual-license-rx): Removed function.
>> (crate->guix-package): Handle most of the multi-licensing cases.
>
> Nice!
>
>>  (define* (crate->guix-package crate-name #:optional version)
>>    "Fetch the metadata for CRATE-NAME from crates.io, and return the
>>  `package' s-expression corresponding to that package, or #f on failure.
>>  When VERSION is specified, attempt to fetch that version; otherwise fetch the
>>  latest version of CRATE-NAME."
>>    (define (string->license string)
>> -    (match (regexp-exec %dual-license-rx string)
>> -      (#f (list (spdx-string->license string)))
>> -      (m  (list (spdx-string->license (match:substring m 1))
>> -                (spdx-string->license (match:substring m 2))))))
>> +    (filter
>> +     (lambda (word)
>> +       (and (not (string-null? word))
>> +            (not (any (lambda (elem) (string=? elem word))
>> +                      '("AND" "OR" "WITH"))) ))
>> +     (string-split string (string->char-set " /"))))
>
> It would be great to have tests for that in tests/crate.scm.  To that
> end, I think you could lift ‘string->license’ to the top level (that is,
> outside ‘crate->guix-package’), and then have a few tests along these
> lines:
>
>   (define string->license
>     (@@ (guix import crate) string->license))
>
>   (test-equal "GPL OR LGPL"
>     (list license:gpl3+ license:lgpl3+)
>     (string->license "GPL OR LGPL"))
>
> (See
> <https://guix.gnu.org/manual/en/html_node/Running-the-Test-Suite.html>.)
>
> Let me know if anything is unclear.
>
> Could you send an updated patch?
>
> Thanks,
> Ludo’.

[bug#37928]

[Brice Waegeneire]

I have finally managed to write the requested tests.

string->license has been rewritten to output licenses instead of strings and to avoid swallowing unknown licenses. Now, importing rpigrep recursively give only 3 #f in the licenses fields.

[bug#37928] [PATCH v2] import: crate: Better handle license expressions.

[Brice Waegeneire]

* guix/import/crate.scm (%dual-license-rx): Removed function.
(crate->guix-package): Handle most of the multi-licensing cases.
* tests/crate.scm (licenses): Add tests for some licenses.
---
 guix/import/crate.scm | 17 ++++++++---------
 tests/crate.scm       | 23 +++++++++++++++++++++++
 2 files changed, 31 insertions(+), 9 deletions(-)

diff --git a/guix/import/crate.scm b/guix/import/crate.scm
index 8dc014d232..0915d8295a 100644
--- a/guix/import/crate.scm
+++ b/guix/import/crate.scm
@@ -178,21 +178,20 @@ and LICENSE."
          (close-port port)
          pkg))
 
-(define %dual-license-rx
-  ;; Dual licensing is represented by a string such as "MIT OR Apache-2.0".
-  ;; This regexp matches that.
-  (make-regexp "^(.*) OR (.*)$"))
+(define (string->license string)
+    (map
+     spdx-string->license
+     (filter (lambda (license)
+               (and (not (string-null? license))
+                   (not (any (lambda (elem) (string=? elem license))
+                             '("AND" "OR" "WITH")))))
+            (string-split string (string->char-set " /")))))
 
 (define* (crate->guix-package crate-name #:optional version)
   "Fetch the metadata for CRATE-NAME from crates.io, and return the
 `package' s-expression corresponding to that package, or #f on failure.
 When VERSION is specified, attempt to fetch that version; otherwise fetch the
 latest version of CRATE-NAME."
-  (define (string->license string)
-    (match (regexp-exec %dual-license-rx string)
-      (#f (list (spdx-string->license string)))
-      (m  (list (spdx-string->license (match:substring m 1))
-                (spdx-string->license (match:substring m 2))))))
 
   (define (normal-dependency? dependency)
     (eq? (crate-dependency-kind dependency) 'normal))
diff --git a/tests/crate.scm b/tests/crate.scm
index c14862ad9f..baad2d0c44 100644
--- a/tests/crate.scm
+++ b/tests/crate.scm
@@ -63,6 +63,9 @@
 (define test-source-hash
   "")
 
+(define string->license
+  (@@ (guix import crate) string->license))
+
 (test-begin "crate")
 
 (test-equal "guix-package->crate-name"
@@ -111,4 +114,24 @@
       (x
        (pk 'fail x #f)))))
 
+(test-equal "licenses: MIT OR Apache-2.0"
+  '(license:expat license:asl2.0)
+  (string->license "MIT OR Apache-2.0"))
+
+(test-equal "licenses: Apache-2.0 / MIT"
+  '(license:asl2.0 license:expat)
+  (string->license "Apache-2.0 / MIT"))
+
+(test-equal "licenses: Apache-2.0 WITH LLVM-exception"
+  '(license:asl2.0 #f)
+  (string->license "Apache-2.0 WITH LLVM-exception"))
+
+(test-equal "licenses: MIT/Apache-2.0 AND BSD-2-Clause"
+  '(license:expat license:asl2.0 #f)
+  (string->license "MIT/Apache-2.0 AND BSD-2-Clause"))
+
+(test-equal "licenses: MIT/Apache-2.0"
+  '(license:expat license:asl2.0)
+  (string->license "MIT/Apache-2.0"))
+
 (test-end "crate")
-- 
2.19.2

bug#37927: [bug#37928] [PATCH v2] import: crate: Better handle license expressions.

[Ludovic Courtès]

Hi Brice,

Brice Waegeneire <brice@waegenei.re> skribis:

> * guix/import/crate.scm (%dual-license-rx): Removed function.
> (crate->guix-package): Handle most of the multi-licensing cases.
> * tests/crate.scm (licenses): Add tests for some licenses.

Thanks for the updated patch!

> +(define (string->license string)
> +    (map
> +     spdx-string->license
> +     (filter (lambda (license)
> +               (and (not (string-null? license))
> +                   (not (any (lambda (elem) (string=? elem license))
> +                             '("AND" "OR" "WITH")))))
> +            (string-split string (string->char-set " /")))))

I changed that to use ‘filter-map’ instead and pushed as
263a267b75e472cb84428571580dabd99d5dff0c.

Thanks!

Ludo’.