[bug#37525] [PATCH 0/3] Recursive option for crate importer

[bug#37525] [PATCH 0/3] Recursive option for crate importer

[Martin Becze]

This add recursive functionalty for the crate importer. Currently it doesn't respect the versioning. To do that we will need to add semver functionality as well so that we can look up the correct version of dependencies. 

Martin Becze (3):
  added recusive import functionality to the crate importer
  updated the crate import script to accept recursive option
  updated docs for import crate

 doc/guix.texi                 |  7 +++++++
 guix/import/crate.scm         | 29 +++++++++++++++++++----------
 guix/scripts/import/crate.scm | 35 +++++++++++++++++++++++++----------
 3 files changed, 51 insertions(+), 20 deletions(-)

-- 
2.23.0

[bug#37525] [PATCH 1/3] added recusive import functionality to the crate importer

[Martin Becze]

---
 guix/import/crate.scm | 29 +++++++++++++++++++----------
 1 file changed, 19 insertions(+), 10 deletions(-)

diff --git a/guix/import/crate.scm b/guix/import/crate.scm
index fd1974eae8..8dc014d232 100644
--- a/guix/import/crate.scm
+++ b/guix/import/crate.scm
@@ -40,6 +40,7 @@
   #:use-module (srfi srfi-26)
   #:export (crate->guix-package
             guix-package->crate-name
+            crate-recursive-import
             %crate-updater))
 
 \f
@@ -218,16 +219,24 @@ latest version of CRATE-NAME."
               (cargo-development-inputs
                (sort (map crate-dependency-id dev-dep-crates)
                      string-ci<?)))
-         (make-crate-sexp #:name crate-name
-                          #:version (crate-version-number version*)
-                          #:cargo-inputs cargo-inputs
-                          #:cargo-development-inputs cargo-development-inputs
-                          #:home-page (or (crate-home-page crate)
-                                          (crate-repository crate))
-                          #:synopsis (crate-description crate)
-                          #:description (crate-description crate)
-                          #:license (and=> (crate-version-license version*)
-                                           string->license)))))
+         (values
+          (make-crate-sexp #:name crate-name
+                           #:version (crate-version-number version*)
+                           #:cargo-inputs cargo-inputs
+                           #:cargo-development-inputs cargo-development-inputs
+                           #:home-page (or (crate-home-page crate)
+                                           (crate-repository crate))
+                           #:synopsis (crate-description crate)
+                           #:description (crate-description crate)
+                           #:license (and=> (crate-version-license version*)
+                                            string->license))
+          (append cargo-inputs cargo-development-inputs)))))
+
+(define (crate-recursive-import crate-name)
+  (recursive-import crate-name #f
+                    #:repo->guix-package (lambda (name repo)
+                                           (crate->guix-package name))
+                    #:guix-name crate-name->package-name))
 
 (define (guix-package->crate-name package)
   "Return the crate name of PACKAGE."
-- 
2.23.0

[bug#37525] [PATCH 2/3] updated the crate import script to accept recursive option

[Martin Becze]

---
 guix/scripts/import/crate.scm | 35 +++++++++++++++++++++++++----------
 1 file changed, 25 insertions(+), 10 deletions(-)

diff --git a/guix/scripts/import/crate.scm b/guix/scripts/import/crate.scm
index 7ae8638911..19c8277d14 100644
--- a/guix/scripts/import/crate.scm
+++ b/guix/scripts/import/crate.scm
@@ -28,6 +28,7 @@
   #:use-module (srfi srfi-1)
   #:use-module (srfi srfi-11)
   #:use-module (srfi srfi-37)
+  #:use-module (srfi srfi-41)
   #:use-module (ice-9 match)
   #:use-module (ice-9 format)
   #:export (guix-import-crate))
@@ -46,6 +47,8 @@ Import and convert the crate.io package for PACKAGE-NAME.\n"))
   (display (G_ "
   -h, --help             display this help and exit"))
   (display (G_ "
+  -r, --recursive        import packages recursively"))
+  (display (G_ "
   -V, --version          display version information and exit"))
   (newline)
   (show-bug-report-information))
@@ -59,6 +62,9 @@ Import and convert the crate.io package for PACKAGE-NAME.\n"))
          (option '(#\V "version") #f #f
                  (lambda args
                    (show-version-and-exit "guix import crate")))
+         (option '(#\r "recursive") #f #f
+                 (lambda (opt name arg result)
+                   (alist-cons 'recursive #t result)))
          %standard-import-options))
 
 \f
@@ -79,22 +85,31 @@ Import and convert the crate.io package for PACKAGE-NAME.\n"))
 
   (let* ((opts (parse-options))
          (args (filter-map (match-lambda
-                            (('argument . value)
-                             value)
-                            (_ #f))
+                             (('argument . value)
+                              value)
+                             (_ #f))
                            (reverse opts))))
     (match args
       ((spec)
        (define-values (name version)
          (package-name->name+version spec))
 
-       (let ((sexp (crate->guix-package name version)))
-         (unless sexp
-           (leave (G_ "failed to download meta-data for package '~a'~%")
-                  (if version
-                      (string-append name "@" version)
-                      name)))
-         sexp))
+       (if (assoc-ref opts 'recursive)
+           (map (match-lambda
+                  ((and ('package ('name name) . rest) pkg)
+                   `(define-public ,(string->symbol name)
+                      ,pkg))
+                  (_ #f))
+                (reverse
+                 (stream->list
+                  (crate-recursive-import name))))
+           (let ((sexp (crate->guix-package name version)))
+             (unless sexp
+               (leave (G_ "failed to download meta-data for package '~a'~%")
+                      (if version
+                          (string-append name "@" version)
+                          name)))
+             sexp)))
       (()
        (leave (G_ "too few arguments~%")))
       ((many ...)
-- 
2.23.0

[bug#37525] [PATCH 3/3] updated docs for import crate

[Martin Becze]

---
 doc/guix.texi | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/doc/guix.texi b/doc/guix.texi
index 14c4514b31..3b3645b854 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -9054,6 +9054,13 @@ in Guix.
 Import metadata from the crates.io Rust package repository
 @uref{https://crates.io, crates.io}, as in this example:
 
+@item --recursive
+@itemx -r
+Traverse the dependency graph of the given upstream package recursively
+and generate package expressions for all those packages that are not yet
+in Guix.
+@end table
+
 @example
 guix import crate blake2-rfc
 @end example
-- 
2.23.0

[bug#37525] previous patch had mistake in docs

[Martin Becze]

Hiy yall, the previous patch had mistake in docs. a new patch with
change log incoming!

[bug#37525] [PATCH v2 1/3] added recusive import functionality to the crate importer

[Martin Becze]

* /guix/import/crate.scm (crate-recursive-import crate-name): added recusive import proc
* /guix/import/crate.scm (crate->guix-package): return inputs for use by the recurive importer
---
 guix/import/crate.scm | 29 +++++++++++++++++++----------
 1 file changed, 19 insertions(+), 10 deletions(-)

diff --git a/guix/import/crate.scm b/guix/import/crate.scm
index fd1974eae8..8dc014d232 100644
--- a/guix/import/crate.scm
+++ b/guix/import/crate.scm
@@ -40,6 +40,7 @@
   #:use-module (srfi srfi-26)
   #:export (crate->guix-package
             guix-package->crate-name
+            crate-recursive-import
             %crate-updater))
 
 \f
@@ -218,16 +219,24 @@ latest version of CRATE-NAME."
               (cargo-development-inputs
                (sort (map crate-dependency-id dev-dep-crates)
                      string-ci<?)))
-         (make-crate-sexp #:name crate-name
-                          #:version (crate-version-number version*)
-                          #:cargo-inputs cargo-inputs
-                          #:cargo-development-inputs cargo-development-inputs
-                          #:home-page (or (crate-home-page crate)
-                                          (crate-repository crate))
-                          #:synopsis (crate-description crate)
-                          #:description (crate-description crate)
-                          #:license (and=> (crate-version-license version*)
-                                           string->license)))))
+         (values
+          (make-crate-sexp #:name crate-name
+                           #:version (crate-version-number version*)
+                           #:cargo-inputs cargo-inputs
+                           #:cargo-development-inputs cargo-development-inputs
+                           #:home-page (or (crate-home-page crate)
+                                           (crate-repository crate))
+                           #:synopsis (crate-description crate)
+                           #:description (crate-description crate)
+                           #:license (and=> (crate-version-license version*)
+                                            string->license))
+          (append cargo-inputs cargo-development-inputs)))))
+
+(define (crate-recursive-import crate-name)
+  (recursive-import crate-name #f
+                    #:repo->guix-package (lambda (name repo)
+                                           (crate->guix-package name))
+                    #:guix-name crate-name->package-name))
 
 (define (guix-package->crate-name package)
   "Return the crate name of PACKAGE."
-- 
2.23.0

[bug#37525] [PATCH v2 2/3] updated the crate import script to accept recursive option

[Martin Becze]

* guix/scripts/import/crate.scm (show-help, guix-import-crate): added recursive option
---
 guix/scripts/import/crate.scm | 35 +++++++++++++++++++++++++----------
 1 file changed, 25 insertions(+), 10 deletions(-)

diff --git a/guix/scripts/import/crate.scm b/guix/scripts/import/crate.scm
index 7ae8638911..19c8277d14 100644
--- a/guix/scripts/import/crate.scm
+++ b/guix/scripts/import/crate.scm
@@ -28,6 +28,7 @@
   #:use-module (srfi srfi-1)
   #:use-module (srfi srfi-11)
   #:use-module (srfi srfi-37)
+  #:use-module (srfi srfi-41)
   #:use-module (ice-9 match)
   #:use-module (ice-9 format)
   #:export (guix-import-crate))
@@ -46,6 +47,8 @@ Import and convert the crate.io package for PACKAGE-NAME.\n"))
   (display (G_ "
   -h, --help             display this help and exit"))
   (display (G_ "
+  -r, --recursive        import packages recursively"))
+  (display (G_ "
   -V, --version          display version information and exit"))
   (newline)
   (show-bug-report-information))
@@ -59,6 +62,9 @@ Import and convert the crate.io package for PACKAGE-NAME.\n"))
          (option '(#\V "version") #f #f
                  (lambda args
                    (show-version-and-exit "guix import crate")))
+         (option '(#\r "recursive") #f #f
+                 (lambda (opt name arg result)
+                   (alist-cons 'recursive #t result)))
          %standard-import-options))
 
 \f
@@ -79,22 +85,31 @@ Import and convert the crate.io package for PACKAGE-NAME.\n"))
 
   (let* ((opts (parse-options))
          (args (filter-map (match-lambda
-                            (('argument . value)
-                             value)
-                            (_ #f))
+                             (('argument . value)
+                              value)
+                             (_ #f))
                            (reverse opts))))
     (match args
       ((spec)
        (define-values (name version)
          (package-name->name+version spec))
 
-       (let ((sexp (crate->guix-package name version)))
-         (unless sexp
-           (leave (G_ "failed to download meta-data for package '~a'~%")
-                  (if version
-                      (string-append name "@" version)
-                      name)))
-         sexp))
+       (if (assoc-ref opts 'recursive)
+           (map (match-lambda
+                  ((and ('package ('name name) . rest) pkg)
+                   `(define-public ,(string->symbol name)
+                      ,pkg))
+                  (_ #f))
+                (reverse
+                 (stream->list
+                  (crate-recursive-import name))))
+           (let ((sexp (crate->guix-package name version)))
+             (unless sexp
+               (leave (G_ "failed to download meta-data for package '~a'~%")
+                      (if version
+                          (string-append name "@" version)
+                          name)))
+             sexp)))
       (()
        (leave (G_ "too few arguments~%")))
       ((many ...)
-- 
2.23.0

[bug#37525] [PATCH v2 3/3] updated docs for import crate

[Martin Becze]

* doc/guix.texi update docs
---

This fixes a bug in the prevous patch. And adds better commits to the git log!

 doc/guix.texi | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index 14c4514b31..a49c9d2b26 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -9054,8 +9054,14 @@ in Guix.
 Import metadata from the crates.io Rust package repository
 @uref{https://crates.io, crates.io}, as in this example:
 
+@item --recursive
+@itemx -r
+Traverse the dependency graph of the given upstream package recursively
+and generate package expressions for all those packages that are not yet
+in Guix.
+
 @example
-guix import crate blake2-rfc
+guix import crate -r blake2-rfc
 @end example
 
 The crate importer also allows you to specify a version string:
-- 
2.23.0

bug#37525: [PATCH v2 1/3] added recusive import functionality to the crate importer

[Ludovic Courtès]

Hi,

Applied!

Note that I merged patches #2 and #3, because we usually document
features as we add them, not separately.  I had to fix up the Texinfo
markup so that ‘--recursive’ is described under ‘crate’.

Anyway it seems to work well, thanks for the patches!

Ludo’.

[bug#37525] [PATCH v2 1/3] added recusive import functionality to the crate importer

[Martin Becze]

On 2019-10-01 21:33, Ludovic Courtès wrote:
> Hi,
> 
> Applied!
> 
> Note that I merged patches #2 and #3, because we usually document
> features as we add them, not separately.  I had to fix up the Texinfo
> markup so that ‘--recursive’ is described under ‘crate’.
> 
> Anyway it seems to work well, thanks for the patches!
> 
> Ludo’.

Awesome thanks! 

> features as we add them, not separately.

Do you mean in the same commit that makes the changes to the code? or
when you apply patches to master?

[bug#37525] [PATCH v2 1/3] added recusive import functionality to the crate importer

[Ludovic Courtès]

Martin Becze <mjbecze@riseup.net> skribis:

> On 2019-10-01 21:33, Ludovic Courtès wrote:

[...]

>> features as we add them, not separately.
>
> Do you mean in the same commit that makes the changes to the code? or
> when you apply patches to master?

I mean that when we add a user-visible change, such as a new
command-line option, we should document it in the same commit that
actually adds the feature.

Thanks,
Ludo’.