[bug#38408] [PATCH v6] Semantic version aware recusive importer for crates

[Martin Becze <mjbecze@riseup.net>]

[-- Attachment #1.1: Type: text/plain, Size: 4265 bytes --]

okkkie! finally got this rewote! Patches attached.

> As noted in my previous message, I think this interface is too  > complex, and since it’s used in a single importer, it would be best > 
to have it directly in (guix import crate).
This is now done!

> In the meantime, there are probably semver-related things that could  > naturally to a helper (guix import semver) module, although perhaps > 
most of that is already provided by guile-semver?
guile-semver is pretty easy to work with so I didn't need to use many 
helpers. The one I did use I think I will try to submit to guile-semver 
itself.

On 12/27/19 1:38 PM, Ludovic Courtès wrote:
> Hi Martin,  > > Sorry for the late reply. > > Martin Becze <mjbecze@riseup.net> 
skribis: > >>> Providing an explicit cache bypassing method also sounds 
 >>> worrying to me: the cache is supposed to be transparent and >>> 
semantics-preserving. >>> >>> More generally, I think adding new 
features to an importer >>> shouldn’t require modifications in this 
area, as a matter of >>> separating concerns. >>> >>> WDYT? >>> >>> 
Thanks, Ludo’. >> >> yes I agree, I removed that in the last version! 
Which also >> rebases off the new topological sort procedure. I'll 
attach the >> latest patches here, in case you missed it. > > Thanks! > 
 >> From eeffdf569c4d7fbfd843e0b48404b6a2f3d46343 Mon Sep 17 00:00:00 >> 
2001 From: Martin Becze <mjbecze@riseup.net> Date: Mon, 16 Dec >> 2019 
17:08:16 -0500 Subject: [PATCH v5 1/4] guix: import: added >> 
recusive-import-semver >> >> * guix/import/utils.scm 
(recusive-import-semver): New Varible * >> guix/import/utils.scm 
(package->definition)[arguments]: Add >> append-verions option > > [...] 
 > >> +(define* (recursive-import-semver #:key + name + (range "*") + >> 
name->metadata + metadata->package + metadata-versions + >> 
package-dependencies + dependency-name + dependency-range + >> guix-name 
+ make-sexp) + "Generates a list of package expressions >> for the 
dependencies of the given +NAME and version RANGE. The >> dependencies 
will be resolved using semantic versioning. +This >> procedure makes the 
assumption that most package repositories will, >> for a +given package 
provide some <metadata> on that package that >> includes what +versions 
of the package that are available and a >> list of dependencies for each 
+version. Dependencies are assumed to >> be composed of a NAME, a 
semantic RANGE and +other data. + +This >> procedure takes the following 
keys: + NAME - The name of the >> package to import + RANGE - The 
version range of the package to >> import + NAME->METADATA - A procedure 
that takes a NAME of a >> package and returns that +package's <metadata> 
+ METADATA->PACKAGE >> A procedure that takes a package's <metadata> and 
VERSION +and >> returns the <package> for the given VERSION + 
METADATA-VERSIONS A >> procedure that that takes a packages <metadata> 
and +returns a >> list of version as strings that are available for the 
given package >> + PACKAGE-DEPENDENCIES a procedure that returns a list 
of >> <dependency> given a +<package> + DEPENDENCY-NAME A procedure >> 
that takes a <dependency> and returns the its name + >> DEPENDENCY-RANGE 
A procedure that takes a <dependency> and returns >> that +decency's 
range as a string + GUIX-NAME A procedure that >> take a NAME and 
returns the Guix version of it + MAKE-SEXP A >> procedure that takes 
<metadata>, <package> and a list of pairs >> +containing (EXPORT-NAME 
<dependency>), returning the package >> expression as an +s-expression" 
 > > As noted in my previous message, I think this interface is too > 
complex, and since it’s used in a single importer, it would be best > to 
have it directly in (guix import crate). > > If/when there’s a need to 
share that logic among several importers, > then we can look for ways to 
factorize whatever needs to be > factorized. > > In the meantime, there 
are probably semver-related things that could > naturally to a helper 
(guix import semver) module, although perhaps > most of that is already 
provided by guile-semver? > >> + (define-record-type <node-dependency> > 
 > Also, as a rule of thumb, we wouldn’t want to duplicate these data > 
types and related code. > > WDYT? > > Thanks, Ludo’.

[-- Attachment #1.2: Type: text/html, Size: 5400 bytes --]

[-- Attachment #2: v6-0001-guix-import-recursive-import-Allow-for-version-nu.patch --]
[-- Type: text/x-patch, Size: 16714 bytes --]

From 7747d27ed32069acb193b3811b30a5a3cc8cfd5b Mon Sep 17 00:00:00 2001
From: Martin Becze <mjbecze@riseup.net>
Date: Sat, 18 Jan 2020 05:05:03 -0500
Subject: [PATCH v6 1/2] guix: import: (recursive-import) Allow for version
 numbers

* guix/import/utils.scm (package->definition) [arguments] added optional `append-version?`
* guix/import/utils.scm (recursive-import) [arguments] added key `version` and
  moved `repo` to be a key
* tests/import-utils.scm

* guix/import/cran.scm (cran->guix-package) [argument]: change `repo` to a key
* guix/import/cran.scm (cran-recursive-import) [argument]: change `repo` to a key
* guix/scripts/import/cran.scm: change `repo` to a key
* guix/import/elpa.scm (elpa->guix-pakcage) [argumnets]: change `repo` to a key
* guix/import/elpa.scm (elpa-recursive-import) [argumnets]: change `repo` to a key
* guix/scripts/import/elpa.scm: change `repo` to a key
* guix/import/gem.scm (gem->guix-package) [arguments]: change `repo` to a key
* guix/import/gem.scm (recursive-import) [arguments]: change `repo` to a key
* guix/import/opam.scm (opam-recurive-import) [arguments]: change `repo` to a key
* guix/import/pypi.scm (pypi-recursive-import) [arguments]: change `repo` to a key
* guix/import/stackage.scm (stackage-recursive-import) [arguments]: change `repo` to a key
---
 guix/import/cran.scm         |  8 +++--
 guix/import/elpa.scm         |  6 ++--
 guix/import/gem.scm          |  6 ++--
 guix/import/opam.scm         |  5 ++--
 guix/import/pypi.scm         |  5 ++--
 guix/import/stackage.scm     |  5 ++--
 guix/import/utils.scm        | 57 +++++++++++++++++++++++-------------
 guix/scripts/import/cran.scm |  5 ++--
 guix/scripts/import/elpa.scm |  4 ++-
 tests/import-utils.scm       |  8 +++--
 10 files changed, 69 insertions(+), 40 deletions(-)

diff --git a/guix/import/cran.scm b/guix/import/cran.scm
index bcb37ed250..9e05dfcba8 100644
--- a/guix/import/cran.scm
+++ b/guix/import/cran.scm
@@ -2,6 +2,7 @@
 ;;; Copyright © 2015, 2016, 2017, 2018, 2019 Ricardo Wurmus <rekado@elephly.net>
 ;;; Copyright © 2015, 2016, 2017, 2019, 2020 Ludovic Courtès <ludo@gnu.org>
 ;;; Copyright © 2017 Mathieu Othacehe <m.othacehe@gmail.com>
+;;; Copyright © 2020 Martin Becze <mjbecze@riseup.net>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -506,7 +507,7 @@ from the alist META, which was derived from the R package's DESCRIPTION file."
 
 (define cran->guix-package
   (memoize
-   (lambda* (package-name #:optional (repo 'cran))
+   (lambda* (package-name #:key (repo 'cran) #:allow-other-keys)
      "Fetch the metadata for PACKAGE-NAME from REPO and return the `package'
 s-expression corresponding to that package, or #f on failure."
      (let ((description (fetch-description repo package-name)))
@@ -521,8 +522,9 @@ s-expression corresponding to that package, or #f on failure."
               (cran->guix-package package-name 'cran))
              (else (values #f '()))))))))
 
-(define* (cran-recursive-import package-name #:optional (repo 'cran))
-  (recursive-import package-name repo
+(define* (cran-recursive-import package-name #:key (repo 'cran))
+  (recursive-import package-name
+                    #:repo repo
                     #:repo->guix-package cran->guix-package
                     #:guix-name cran-guix-name))
 
diff --git a/guix/import/elpa.scm b/guix/import/elpa.scm
index 2d4487dba0..9140bcdc34 100644
--- a/guix/import/elpa.scm
+++ b/guix/import/elpa.scm
@@ -2,6 +2,7 @@
 ;;; Copyright © 2015 Federico Beffa <beffa@fbengineering.ch>
 ;;; Copyright © 2015, 2016, 2017, 2018, 2020 Ludovic Courtès <ludo@gnu.org>
 ;;; Copyright © 2018 Oleg Pykhalov <go.wigust@gmail.com>
+;;; Copyright © 2020 Martin Becze <mjbecze@riseup.net>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -245,7 +246,7 @@ type '<elpa-package>'."
         (license ,license))
      dependencies-names)))
 
-(define* (elpa->guix-package name #:optional (repo 'gnu))
+(define* (elpa->guix-package name #:key (repo 'gnu) #:allow-other-keys)
   "Fetch the package NAME from REPO and produce a Guix package S-expression."
   (match (fetch-elpa-package name repo)
     (#f #f)
@@ -301,7 +302,8 @@ type '<elpa-package>'."
 (define elpa-guix-name (cut guix-name "emacs-" <>))
 
 (define* (elpa-recursive-import package-name #:optional (repo 'gnu))
-  (recursive-import package-name repo
+  (recursive-import package-name
+                    #:repo repo
                     #:repo->guix-package elpa->guix-package
                     #:guix-name elpa-guix-name))
 
diff --git a/guix/import/gem.scm b/guix/import/gem.scm
index 0bf9ff2552..e744d9e69d 100644
--- a/guix/import/gem.scm
+++ b/guix/import/gem.scm
@@ -2,6 +2,7 @@
 ;;; Copyright © 2015 David Thompson <davet@gnu.org>
 ;;; Copyright © 2016 Ben Woodcroft <donttrustben@gmail.com>
 ;;; Copyright © 2018 Oleg Pykhalov <go.wigust@gmail.com>
+;;; Copyright © 2020 Martin Becze <mjbecze@riseup.net>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -117,7 +118,7 @@ VERSION, HASH, HOME-PAGE, DESCRIPTION, DEPENDENCIES, and LICENSES."
                  ((license) (license->symbol license))
                  (_ `(list ,@(map license->symbol licenses)))))))
 
-(define* (gem->guix-package package-name #:optional (repo 'rubygems) version)
+(define* (gem->guix-package package-name #:key (repo 'rubygems) version)
   "Fetch the metadata for PACKAGE-NAME from rubygems.org, and return the
 `package' s-expression corresponding to that package, or #f on failure."
   (let ((package (rubygems-fetch package-name)))
@@ -201,6 +202,7 @@ package on RubyGems."
    (latest latest-release)))
 
 (define* (gem-recursive-import package-name #:optional version)
-  (recursive-import package-name '()
+  (recursive-import package-name
+                    #:repo '()
                     #:repo->guix-package gem->guix-package
                     #:guix-name ruby-package-name))
diff --git a/guix/import/opam.scm b/guix/import/opam.scm
index 394415fdd4..87c823a98c 100644
--- a/guix/import/opam.scm
+++ b/guix/import/opam.scm
@@ -1,5 +1,6 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2018 Julien Lepiller <julien@lepiller.eu>
+;;; Copyright © 2020 Martin Becze <mjbecze@riseup.net>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -311,8 +312,8 @@ or #f on failure."
 		      dependencies))))))))
 
 (define (opam-recursive-import package-name)
-  (recursive-import package-name #f
-                    #:repo->guix-package (lambda (name repo)
+  (recursive-import package-name
+                    #:repo->guix-package (lambda (name . _)
                                            (opam->guix-package name))
                     #:guix-name ocaml-name->guix-name))
 
diff --git a/guix/import/pypi.scm b/guix/import/pypi.scm
index 354cae9c4c..f0702d6403 100644
--- a/guix/import/pypi.scm
+++ b/guix/import/pypi.scm
@@ -5,6 +5,7 @@
 ;;; Copyright © 2017 Mathieu Othacehe <m.othacehe@gmail.com>
 ;;; Copyright © 2018 Ricardo Wurmus <rekado@elephly.net>
 ;;; Copyright © 2019 Maxim Cournoyer <maxim.cournoyer@gmail.com>
+;;; Copyright © 2020 Martin Becze <mjbecze@riseup.net>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -415,8 +416,8 @@ VERSION, SOURCE-URL, HOME-PAGE, SYNOPSIS, DESCRIPTION, and LICENSE."
                                 description license))))))))
 
 (define (pypi-recursive-import package-name)
-  (recursive-import package-name #f
-                    #:repo->guix-package (lambda (name repo)
+  (recursive-import package-name
+                    #:repo->guix-package (lambda (name . _)
                                            (pypi->guix-package name))
                     #:guix-name python->package-name))
 
diff --git a/guix/import/stackage.scm b/guix/import/stackage.scm
index 14150201b5..6091cf2c64 100644
--- a/guix/import/stackage.scm
+++ b/guix/import/stackage.scm
@@ -1,6 +1,7 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2017 Federico Beffa <beffa@fbengineering.ch>
 ;;; Copyright © 2018 Ricardo Wurmus <rekado@elephly.net>
+;;; Copyright © 2020 Martin Becze <mjbecze@riseup.net>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -108,8 +109,8 @@ included in the Stackage LTS release."
            (leave-with-message "~a: Stackage package not found" package-name))))))
 
 (define (stackage-recursive-import package-name . args)
-  (recursive-import package-name #f
-                    #:repo->guix-package (lambda (name repo)
+  (recursive-import package-name
+                    #:repo->guix-package (lambda (name . _)
                                            (apply stackage->guix-package (cons name args)))
                     #:guix-name hackage-name->package-name))
 
diff --git a/guix/import/utils.scm b/guix/import/utils.scm
index d17d400ddf..59430d3e66 100644
--- a/guix/import/utils.scm
+++ b/guix/import/utils.scm
@@ -5,6 +5,7 @@
 ;;; Copyright © 2017, 2019 Ricardo Wurmus <rekado@elephly.net>
 ;;; Copyright © 2018 Oleg Pykhalov <go.wigust@gmail.com>
 ;;; Copyright © 2019 Robert Vollmert <rob@vllmrt.net>
+;;; Copyright © 2020 Martin Becze <mjbecze@riseup.net>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -44,6 +45,7 @@
   #:use-module (srfi srfi-9)
   #:use-module (srfi srfi-11)
   #:use-module (srfi srfi-26)
+  #:use-module (srfi srfi-71)
   #:export (factorize-uri
 
             flatten
@@ -258,13 +260,15 @@ package definition."
     ((package-inputs ...)
      `((native-inputs (,'quasiquote ,package-inputs))))))
 
-(define (package->definition guix-package)
+(define* (package->definition guix-package #:optional append-version?)
   (match guix-package
-    (('package ('name (? string? name)) _ ...)
-     `(define-public ,(string->symbol name)
-        ,guix-package))
-    (('let anything ('package ('name (? string? name)) _ ...))
-     `(define-public ,(string->symbol name)
+    ((or
+      ('package ('name name) ('version version) . rest)
+      ('let _ ('package ('name name) ('version version) . rest)))
+
+     `(define-public ,(string->symbol (if append-version?
+                                          (string-append name "-" version)
+                                          version))
         ,guix-package))))
 
 (define (build-system-modules)
@@ -399,32 +403,43 @@ obtain a node's uniquely identifying \"key\"."
                    (cons head result)
                    (set-insert (node-name head) visited))))))))
 
-(define* (recursive-import package-name repo
-                           #:key repo->guix-package guix-name
+(define* (recursive-import package-name
+                           #:key repo->guix-package guix-name version repo
                            #:allow-other-keys)
   "Return a list of package expressions for PACKAGE-NAME and all its
 dependencies, sorted in topological order.  For each package,
-call (REPO->GUIX-PACKAGE NAME REPO), which should return a package expression
-and a list of dependencies; call (GUIX-NAME NAME) to obtain the Guix package
-name corresponding to the upstream name."
+call (REPO->GUIX-PACKAGE NAME :KEYS version repo), which should return a
+package expression and a list of dependencies; call (GUIX-NAME NAME) to
+obtain the Guix package name corresponding to the upstream name."
   (define-record-type <node>
-    (make-node name package dependencies)
+    (make-node name version package dependencies)
     node?
     (name         node-name)
+    (version       node-version)
     (package      node-package)
     (dependencies node-dependencies))
 
-  (define (exists? name)
-    (not (null? (find-packages-by-name (guix-name name)))))
+  (define (exists? name version)
+    (not (null? (find-packages-by-name (guix-name name) version))))
 
-  (define (lookup-node name)
-    (receive (package dependencies) (repo->guix-package name repo)
-      (make-node name package dependencies)))
+  (define (lookup-node name version)
+    (let* ((package dependencies (repo->guix-package name
+                                                     #:version version
+                                                     #:repo repo))
+           (normilizied-deps (map (match-lambda
+                                    ((name version) (list name version))
+                                    (name (list name #f))) dependencies)))
+      (make-node name version package normilizied-deps)))
 
   (map node-package
-       (topological-sort (list (lookup-node package-name))
+       (topological-sort (list (lookup-node package-name version))
                          (lambda (node)
-                           (map lookup-node
-                                (remove exists?
+                           (map (lambda (name-version)
+                                  (apply lookup-node name-version))
+                                (remove (lambda (name-version)
+                                          (apply exists? name-version))
                                         (node-dependencies node))))
-                         node-name)))
+                         (lambda (node)
+                           (string-append
+                            (node-name node)
+                            (or (node-version node) ""))))))
diff --git a/guix/scripts/import/cran.scm b/guix/scripts/import/cran.scm
index d6f371ef3a..bc266ad9da 100644
--- a/guix/scripts/import/cran.scm
+++ b/guix/scripts/import/cran.scm
@@ -1,6 +1,7 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2014 Eric Bavier <bavier@member.fsf.org>
 ;;; Copyright © 2015, 2017, 2019 Ricardo Wurmus <rekado@elephly.net>
+;;; Copyright © 2020 Martin Becze <mjbecze@riseup.net>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -98,10 +99,10 @@ Import and convert the CRAN package for PACKAGE-NAME.\n"))
            ;; Recursive import
            (map package->definition
                 (cran-recursive-import package-name
-                                       (or (assoc-ref opts 'repo) 'cran)))
+                                       #:repo (or (assoc-ref opts 'repo) 'cran)))
            ;; Single import
            (let ((sexp (cran->guix-package package-name
-                                           (or (assoc-ref opts 'repo) 'cran))))
+                                           #:repo (or (assoc-ref opts 'repo) 'cran))))
              (unless sexp
                (leave (G_ "failed to download description for package '~a'~%")
                       package-name))
diff --git a/guix/scripts/import/elpa.scm b/guix/scripts/import/elpa.scm
index d270d2b4bc..07ac07a3d5 100644
--- a/guix/scripts/import/elpa.scm
+++ b/guix/scripts/import/elpa.scm
@@ -1,6 +1,7 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2015 Federico Beffa <beffa@fbengineering.ch>
 ;;; Copyright © 2018 Oleg Pykhalov <go.wigust@gmail.com>
+;;; Copyright © 2020 Martin Becze <mjbecze@riseup.net>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -102,7 +103,8 @@ Import the latest package named PACKAGE-NAME from an ELPA repository.\n"))
                   (_ #f))
                 (elpa-recursive-import package-name
                                        (or (assoc-ref opts 'repo) 'gnu)))
-           (let ((sexp (elpa->guix-package package-name (assoc-ref opts 'repo))))
+           (let ((sexp (elpa->guix-package package-name
+                                           #:repo (assoc-ref opts 'repo))))
              (unless sexp
                (leave (G_ "failed to download package '~a'~%") package-name))
              sexp)))
diff --git a/tests/import-utils.scm b/tests/import-utils.scm
index 87dda3238f..2357ea5c40 100644
--- a/tests/import-utils.scm
+++ b/tests/import-utils.scm
@@ -1,6 +1,7 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2015, 2017 Ricardo Wurmus <rekado@elephly.net>
 ;;; Copyright © 2016 Ben Woodcroft <donttrustben@gmail.com>
+;;; Copyright © 2020 Martin Becze <mjbecze@riseup.net>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -48,15 +49,16 @@
     (package
       (name "foo")
       (inputs `(("bar" ,bar)))))
-  (recursive-import "foo" 'repo
+  (recursive-import "foo"
+                    #:repo 'repo
                     #:repo->guix-package
                     (match-lambda*
-                      (("foo" 'repo)
+                      (("foo" #:version #f #:repo 'repo)
                        (values '(package
                                   (name "foo")
                                   (inputs `(("bar" ,bar))))
                                '("bar")))
-                      (("bar" 'repo)
+                      (("bar" #:version #f #:repo 'repo)
                        (values '(package
                                   (name "bar"))
                                '())))
-- 
2.25.0


[-- Attachment #3: v6-0002-guix-import-crate-Use-semver-to-resovle-module-ve.patch --]
[-- Type: text/x-patch, Size: 28034 bytes --]

From a405e3ffdf2716b9920f6b74e4690c9b7731f67a Mon Sep 17 00:00:00 2001
From: Martin Becze <mjbecze@riseup.net>
Date: Sat, 18 Jan 2020 10:44:18 -0500
Subject: [PATCH v6 2/2] guix: import: crate: Use semver to resovle module
 versions

*  guix/import/crate.scm (make-crate-sexp): formatting, added '#:skip-build?'
   to build system args; added package definition geneation
*  guix/import/crate.scm (crate->guix-package): [arguments] moved `verions` to
   a key. Use semver to resolve the correct module versions
*  guix/import/crate.scm (crate-name->package0name): [arguments] add
   #:optional `version` arguement
*  guix/scripts/import/crate.scm remove package definition generation; changed
   `version` to a key
*  tests/crate.scm: added version data to (recursuve-import) test
---
 guix/import/crate.scm         | 140 +++++++++-------
 guix/scripts/import/crate.scm |  11 +-
 tests/crate.scm               | 290 +++++++++++++++++++---------------
 tests/elpa.scm                |   3 +-
 4 files changed, 258 insertions(+), 186 deletions(-)

diff --git a/guix/import/crate.scm b/guix/import/crate.scm
index 57823c3639..6847a7046b 100644
--- a/guix/import/crate.scm
+++ b/guix/import/crate.scm
@@ -1,7 +1,7 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2016 David Craven <david@craven.ch>
 ;;; Copyright © 2019, 2020 Ludovic Courtès <ludo@gnu.org>
-;;; Copyright © 2019 Martin Becze <mjbecze@riseup.net>
+;;; Copyright © 2019, 2020 Martin Becze <mjbecze@riseup.net>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -28,6 +28,7 @@
   #:use-module (guix import json)
   #:use-module (guix import utils)
   #:use-module ((guix licenses) #:prefix license:)
+  #:use-module (guix memoization)
   #:use-module (guix monads)
   #:use-module (guix packages)
   #:use-module (guix upstream)
@@ -35,9 +36,12 @@
   #:use-module (ice-9 match)
   #:use-module (ice-9 regex)
   #:use-module (json)
+  #:use-module (semver)
+  #:use-module (semver ranges)
   #:use-module (srfi srfi-1)
   #:use-module (srfi srfi-2)
   #:use-module (srfi srfi-26)
+  #:use-module (srfi srfi-71)
   #:export (crate->guix-package
             guix-package->crate-name
             string->license
@@ -86,7 +90,7 @@
   crate-dependency?
   json->crate-dependency
   (id            crate-dependency-id "crate_id")  ;string
-  (kind          crate-dependency-kind "kind"     ;'normal | 'dev
+  (kind          crate-dependency-kind "kind"     ;'normal | 'dev | 'build
                  string->symbol)
   (requirement   crate-dependency-requirement "req")) ;string
 
@@ -105,6 +109,8 @@ record or #f if it was not found."
                (json->crate `(,@alist
                               ("actual_versions" . ,versions))))))))
 
+(define mem-lookup-crate (memoize lookup-crate))
+
 (define (crate-version-dependencies version)
   "Return the list of <crate-dependency> records of VERSION, a
 <crate-version>."
@@ -150,34 +156,40 @@ VERSION, CARGO-INPUTS, CARGO-DEVELOPMENT-INPUTS, HOME-PAGE, SYNOPSIS, DESCRIPTIO
 and LICENSE."
   (let* ((port (http-fetch (crate-uri name version)))
          (guix-name (crate-name->package-name name))
-         (cargo-inputs (map crate-name->package-name cargo-inputs))
-         (cargo-development-inputs (map crate-name->package-name
-                                        cargo-development-inputs))
+         (cargo-inputs
+          (map
+           (lambda (name-version)
+             (apply crate-name->package-name name-version)) cargo-inputs))
+         (cargo-development-inputs
+          (map
+           (lambda (name-version)
+             (apply crate-name->package-name name-version)) cargo-development-inputs))
          (pkg `(package
-                   (name ,guix-name)
-                   (version ,version)
-                   (source (origin
-                             (method url-fetch)
-                             (uri (crate-uri ,name version))
-                             (file-name (string-append name "-" version ".tar.gz"))
-                             (sha256
-                              (base32
-                               ,(bytevector->nix-base32-string (port-sha256 port))))))
-                   (build-system cargo-build-system)
-                   ,@(maybe-arguments (append (maybe-cargo-inputs cargo-inputs)
-                                              (maybe-cargo-development-inputs
+                 (name ,guix-name)
+                 (version ,version)
+                 (source (origin
+                           (method url-fetch)
+                           (uri (crate-uri ,name version))
+                           (file-name (string-append name "-" version ".tar.gz"))
+                           (sha256
+                            (base32
+                             ,(bytevector->nix-base32-string (port-sha256 port))))))
+                 (build-system cargo-build-system)
+                 ,@(maybe-arguments (append `(#:skip-build? #t)
+                                               (maybe-cargo-inputs cargo-inputs)
+                                               (maybe-cargo-development-inputs
                                                 cargo-development-inputs)))
-                   (home-page ,(match home-page
-                                 (() "")
-                                 (_ home-page)))
-                   (synopsis ,synopsis)
-                   (description ,(beautify-description description))
-                   (license ,(match license
-                               (() #f)
-                               ((license) license)
-                               (_ `(list ,@license)))))))
-         (close-port port)
-         pkg))
+                 (home-page ,(match home-page
+                               (() "")
+                               (_ home-page)))
+                 (synopsis ,synopsis)
+                 (description ,(beautify-description description))
+                 (license ,(match license
+                                (() #f)
+                                ((license) license)
+                                (_ `(list ,@license)))))))
+    (close-port port)
+    (package->definition pkg #t)))
 
 (define (string->license string)
   (filter-map (lambda (license)
@@ -188,37 +200,60 @@ and LICENSE."
                          'unknown-license!)))
               (string-split string (string->char-set " /"))))
 
-(define* (crate->guix-package crate-name #:optional version)
+(define* (crate->guix-package crate-name #:key version #:allow-other-keys)
   "Fetch the metadata for CRATE-NAME from crates.io, and return the
 `package' s-expression corresponding to that package, or #f on failure.
 When VERSION is specified, attempt to fetch that version; otherwise fetch the
 latest version of CRATE-NAME."
 
+  (define (semver-range-contains-string? range version)
+    (semver-range-contains? (string->semver-range range)
+                            (string->semver version)))
+
   (define (normal-dependency? dependency)
-    (eq? (crate-dependency-kind dependency) 'normal))
+    (or (eq? (crate-dependency-kind dependency) 'build)
+        (eq? (crate-dependency-kind dependency) 'normal)))
 
   (define crate
-    (lookup-crate crate-name))
+    (mem-lookup-crate crate-name))
 
   (define version-number
     (or version
         (crate-latest-version crate)))
 
-  (define version*
+  (define (find-version crate range)
+    "finds the a vesion of a crate that fulfils the semver <range>"
     (find (lambda (version)
-            (string=? (crate-version-number version)
-                      version-number))
+            (semver-range-contains-string?
+             range
+             (crate-version-number version)))
           (crate-versions crate)))
 
+  (define version*
+    (find-version crate version-number))
+
+  (define (sort-map-deps deps)
+    "sorts the dependencies and maps the dependencies to a list
+     containing pairs of (name version)"
+    (sort (map (lambda (dep)
+                 (let* ((name (crate-dependency-id dep))
+                        (crate (mem-lookup-crate name))
+                        (req (crate-dependency-requirement dep))
+                        (ver (find-version crate req)))
+                   (list name
+                         (crate-version-number ver))))
+               deps)
+          (match-lambda* (((_ name) ...)
+                          (apply string-ci<? name)))))
+
   (and crate version*
-       (let* ((dependencies   (crate-version-dependencies version*))
-              (dep-crates     (filter normal-dependency? dependencies))
-              (dev-dep-crates (remove normal-dependency? dependencies))
-              (cargo-inputs   (sort (map crate-dependency-id dep-crates)
-                                    string-ci<?))
-              (cargo-development-inputs
-               (sort (map crate-dependency-id dev-dep-crates)
-                     string-ci<?)))
+       (let* ((dependencies (crate-version-dependencies version*))
+              (dep-crates dev-dep-crates (partition normal-dependency? dependencies))
+              (cargo-inputs (sort-map-deps dep-crates))
+              ;; for now we are skipping the resolution of the development inputs
+              ;; since most crates are libaries and we only want to test at the
+              ;; app level. This probably should be parameterized though.
+              (cargo-development-inputs '()))
          (values
           (make-crate-sexp #:name crate-name
                            #:version (crate-version-number version*)
@@ -230,15 +265,12 @@ latest version of CRATE-NAME."
                            #:description (crate-description crate)
                            #:license (and=> (crate-version-license version*)
                                             string->license))
-          (append cargo-inputs cargo-development-inputs)))))
+          cargo-inputs))))
 
-(define* (crate-recursive-import crate-name #:optional version)
-  (recursive-import crate-name #f
-                    #:repo->guix-package
-                    (lambda (name repo)
-                      (let ((version (and (string=? name crate-name)
-                                          version)))
-                        (crate->guix-package name version)))
+(define* (crate-recursive-import crate-name #:key version)
+  (recursive-import crate-name
+                    #:repo->guix-package crate->guix-package
+                    #:version version
                     #:guix-name crate-name->package-name))
 
 (define (guix-package->crate-name package)
@@ -252,8 +284,11 @@ latest version of CRATE-NAME."
     (match parts
       ((name _ ...) name))))
 
-(define (crate-name->package-name name)
-  (string-append "rust-" (string-join (string-split name #\_) "-")))
+(define* (crate-name->package-name name #:optional version)
+  (let ((name (guix-name "rust-" name)))
+    (if version
+        (string-append name "-" version)
+        name)))
 
 \f
 ;;;
@@ -288,4 +323,3 @@ latest version of CRATE-NAME."
    (description "Updater for crates.io packages")
    (pred crate-package?)
    (latest latest-release)))
-
diff --git a/guix/scripts/import/crate.scm b/guix/scripts/import/crate.scm
index d834518c18..552628cfc7 100644
--- a/guix/scripts/import/crate.scm
+++ b/guix/scripts/import/crate.scm
@@ -2,7 +2,7 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2014 David Thompson <davet@gnu.org>
 ;;; Copyright © 2016 David Craven <david@craven.ch>
-;;; Copyright © 2019 Martin Becze <mjbecze@riseup.net>
+;;; Copyright © 2019, 2020 Martin Becze <mjbecze@riseup.net>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -95,13 +95,8 @@ Import and convert the crate.io package for PACKAGE-NAME.\n"))
          (package-name->name+version spec))
 
        (if (assoc-ref opts 'recursive)
-           (map (match-lambda
-                  ((and ('package ('name name) . rest) pkg)
-                   `(define-public ,(string->symbol name)
-                      ,pkg))
-                  (_ #f))
-                (crate-recursive-import name version))
-           (let ((sexp (crate->guix-package name version)))
+           (crate-recursive-import name #:version version)
+           (let ((sexp (crate->guix-package name #:version version)))
              (unless sexp
                (leave (G_ "failed to download meta-data for package '~a'~%")
                       (if version
diff --git a/tests/crate.scm b/tests/crate.scm
index aa51faebf9..39561d5745 100644
--- a/tests/crate.scm
+++ b/tests/crate.scm
@@ -2,6 +2,7 @@
 ;;; Copyright © 2014 David Thompson <davet@gnu.org>
 ;;; Copyright © 2016 David Craven <david@craven.ch>
 ;;; Copyright © 2019, 2020 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2020 Martin Becze <mjbecze@riseup.net>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -54,8 +55,9 @@
   "{
   \"dependencies\": [
      {
-       \"crate_id\": \"bar\",
+       \"crate_id\": \"leaf-alice\",
        \"kind\": \"normal\",
+       \"req\": \"1.0.0\",
      }
   ]
 }")
@@ -88,18 +90,22 @@
      {
        \"crate_id\": \"intermediate-1\",
        \"kind\": \"normal\",
+       \"req\": \"1.0.0\",
      },
      {
        \"crate_id\": \"intermediate-2\",
        \"kind\": \"normal\",
+       \"req\": \"1.0.0\",
      }
      {
        \"crate_id\": \"leaf-alice\",
        \"kind\": \"normal\",
+       \"req\": \"1.0.0\",
      },
      {
        \"crate_id\": \"leaf-bob\",
        \"kind\": \"normal\",
+       \"req\": \"1.0.0\",
      },
   ]
 }")
@@ -132,14 +138,17 @@
      {
        \"crate_id\": \"intermediate-2\",
        \"kind\": \"normal\",
+       \"req\": \"1.0.0\",
      },
      {
        \"crate_id\": \"leaf-alice\",
        \"kind\": \"normal\",
+       \"req\": \"1.0.0\",
      },
      {
        \"crate_id\": \"leaf-bob\",
        \"kind\": \"normal\",
+       \"req\": \"1.0.0\",
      }
   ]
 }")
@@ -172,6 +181,7 @@
      {
        \"crate_id\": \"leaf-bob\",
        \"kind\": \"normal\",
+       \"req\": \"1.0.0\",
      },
   ]
 }")
@@ -252,34 +262,48 @@
               (open-input-string test-foo-crate))
              ("https://crates.io/api/v1/crates/foo/1.0.0/download"
               (set! test-source-hash
-                (bytevector->nix-base32-string
-                 (sha256 (string->bytevector "empty file\n" "utf-8"))))
+                    (bytevector->nix-base32-string
+                     (sha256 (string->bytevector "empty file\n" "utf-8"))))
               (open-input-string "empty file\n"))
              ("https://crates.io/api/v1/crates/foo/1.0.0/dependencies"
               (open-input-string test-foo-dependencies))
+             ("https://crates.io/api/v1/crates/leaf-alice"
+              (open-input-string test-leaf-alice-crate))
+             ("https://crates.io/api/v1/crates/leaf-alice/1.0.0/download"
+              (set! test-source-hash
+                    (bytevector->nix-base32-string
+                     (sha256 (string->bytevector "empty file\n" "utf-8"))))
+              (open-input-string "empty file\n"))
+             ("https://crates.io/api/v1/crates/leaf-alice/1.0.0/dependencies"
+              (open-input-string test-leaf-alice-dependencies))
              (_ (error "Unexpected URL: " url)))))
-    (match (crate->guix-package "foo")
-      (('package
-         ('name "rust-foo")
-         ('version "1.0.0")
-         ('source ('origin
-                    ('method 'url-fetch)
-                    ('uri ('crate-uri "foo" 'version))
-                    ('file-name ('string-append 'name "-" 'version ".tar.gz"))
-                    ('sha256
-                     ('base32
-                      (? string? hash)))))
-         ('build-system 'cargo-build-system)
-         ('arguments
-          ('quasiquote
-           ('#:cargo-inputs (("rust-bar" ('unquote rust-bar))))))
-         ('home-page "http://example.com")
-         ('synopsis "summary")
-         ('description "summary")
-         ('license ('list 'license:expat 'license:asl2.0)))
-       (string=? test-source-hash hash))
-      (x
-       (pk 'fail x #f)))))
+
+        (match (crate->guix-package "foo")
+          ((define-public rust-foo-1.0.0
+             (package (name "rust-foo")
+                      (version "1.0.0")
+                      (source
+                       (origin
+                         (method url-fetch)
+                         (uri (crate-uri "foo" 'version))
+                         (file-name (string-append name "-" version ".tar.gz"))
+                         (sha256
+                          (base32
+                           (?  string? hash)))))
+                      (build-system 'cargo-build-system)
+                      (arguments
+                       ('quasiquote
+                        (#:skip-build? #t
+                         #:cargo-inputs
+                         (("rust-leaf-alice-1.0.0" ('unquote rust-leaf-alice-1.0.0))))))
+                      (home-page "http://example.com")
+                      (synopsis "summary")
+                      (description "summary")
+                      (license (list license:expat license:asl2.0))))
+
+           (string=? test-source-hash hash))
+          (x
+           (pk 'fail x #f)))))
 
 (test-assert "cargo-recursive-import"
   ;; Replace network resources with sample data.
@@ -334,105 +358,123 @@
              (_ (error "Unexpected URL: " url)))))
         (match (crate-recursive-import "root")
           ;; rust-intermediate-2 has no dependency on the rust-leaf-alice package, so this is a valid ordering
-          ((('package
-              ('name "rust-leaf-alice")
-              ('version (? string? ver))
-              ('source
-               ('origin
-                 ('method 'url-fetch)
-                 ('uri ('crate-uri "leaf-alice" 'version))
-                 ('file-name
-                  ('string-append 'name "-" 'version ".tar.gz"))
-                 ('sha256
-                  ('base32
-                   (? string? hash)))))
-              ('build-system 'cargo-build-system)
-              ('home-page "http://example.com")
-              ('synopsis "summary")
-              ('description "summary")
-              ('license ('list 'license:expat 'license:asl2.0)))
-            ('package
-              ('name "rust-leaf-bob")
-              ('version (? string? ver))
-              ('source
-               ('origin
-                 ('method 'url-fetch)
-                 ('uri ('crate-uri "leaf-bob" 'version))
-                 ('file-name
-                  ('string-append 'name "-" 'version ".tar.gz"))
-                 ('sha256
-                  ('base32
-                   (? string? hash)))))
-              ('build-system 'cargo-build-system)
-              ('home-page "http://example.com")
-              ('synopsis "summary")
-              ('description "summary")
-              ('license ('list 'license:expat 'license:asl2.0)))
-            ('package
-              ('name "rust-intermediate-2")
-              ('version (? string? ver))
-              ('source
-               ('origin
-                 ('method 'url-fetch)
-                 ('uri ('crate-uri "intermediate-2" 'version))
-                 ('file-name
-                  ('string-append 'name "-" 'version ".tar.gz"))
-                 ('sha256
-                  ('base32
-                   (? string? hash)))))
-              ('build-system 'cargo-build-system)
-              ('arguments
-               ('quasiquote
-                ('#:cargo-inputs (("rust-leaf-bob" ('unquote rust-leaf-bob))))))
-              ('home-page "http://example.com")
-              ('synopsis "summary")
-              ('description "summary")
-              ('license ('list 'license:expat 'license:asl2.0)))
-            ('package
-              ('name "rust-intermediate-1")
-              ('version (? string? ver))
-              ('source
-               ('origin
-                 ('method 'url-fetch)
-                 ('uri ('crate-uri "intermediate-1" 'version))
-                 ('file-name
-                  ('string-append 'name "-" 'version ".tar.gz"))
-                 ('sha256
-                  ('base32
-                   (? string? hash)))))
-              ('build-system 'cargo-build-system)
-              ('arguments
-               ('quasiquote
-                ('#:cargo-inputs (("rust-intermediate-2" ('unquote rust-intermediate-2))
-                                  ("rust-leaf-alice" ('unquote rust-leaf-alice))
-                                  ("rust-leaf-bob" ('unquote rust-leaf-bob))))))
-              ('home-page "http://example.com")
-              ('synopsis "summary")
-              ('description "summary")
-              ('license ('list 'license:expat 'license:asl2.0)))
-            ('package
-              ('name "rust-root")
-              ('version (? string? ver))
-              ('source
-               ('origin
-                 ('method 'url-fetch)
-                 ('uri ('crate-uri "root" 'version))
-                 ('file-name
-                  ('string-append 'name "-" 'version ".tar.gz"))
-                 ('sha256
-                  ('base32
-                   (? string? hash)))))
-              ('build-system 'cargo-build-system)
-              ('arguments
-               ('quasiquote
-                ('#:cargo-inputs (("rust-intermediate-1" ('unquote rust-intermediate-1))
-                                  ("rust-intermediate-2" ('unquote rust-intermediate-2))
-                                  ("rust-leaf-alice" ('unquote rust-leaf-alice))
-                                  ("rust-leaf-bob" ('unquote rust-leaf-bob))))))
-              ('home-page "http://example.com")
-              ('synopsis "summary")
-              ('description "summary")
-              ('license ('list 'license:expat 'license:asl2.0))))
+          (((define-public rust-leaf-alice-1.0.0
+              (package
+                (name "rust-leaf-alice")
+                (version (?  string? ver))
+                (source
+                 (origin
+                   (method url-fetch)
+                   (uri (crate-uri "leaf-alice" version))
+                   (file-name
+                    (string-append name "-" version ".tar.gz"))
+                   (sha256
+                    (base32
+                     (?  string? hash)))))
+                (build-system cargo-build-system)
+                (arguments ('quasiquote (#:skip-build? #t)))
+                (home-page "http://example.com")
+                (synopsis "summary")
+                (description "summary")
+                (license (list license:expat license:asl2.0))))
+            (define-public rust-leaf-bob-1.0.0
+              (package
+                (name "rust-leaf-bob")
+                (version (?  string? ver))
+                (source
+                 (origin
+                   (method url-fetch)
+                   (uri (crate-uri "leaf-bob" version))
+                   (file-name
+                    (string-append name "-" version ".tar.gz"))
+                   (sha256
+                    (base32
+                     (?  string? hash)))))
+                (build-system cargo-build-system)
+                (arguments ('quasiquote (#:skip-build? #t)))
+                (home-page "http://example.com")
+                (synopsis "summary")
+                (description "summary")
+                (license (list license:expat license:asl2.0))))
+            (define-public rust-intermediate-2-1.0.0
+              (package
+                (name "rust-intermediate-2")
+                (version (?  string? ver))
+                (source
+                 (origin
+                   (method url-fetch)
+                   (uri (crate-uri "intermediate-2" version))
+                   (file-name
+                    (string-append name "-" version ".tar.gz"))
+                   (sha256
+                    (base32
+                     (?  string? hash)))))
+                (build-system cargo-build-system)
+                (arguments
+                 ('quasiquote (#:skip-build? #t
+                               #:cargo-inputs
+                               (("rust-leaf-bob-1.0.0"
+                                 ('unquote rust-leaf-bob-1.0.0))))))
+                (home-page "http://example.com")
+                (synopsis "summary")
+                (description "summary")
+                (license (list license:expat license:asl2.0))))
+            (define-public rust-intermediate-1-1.0.0
+              (package
+                (name "rust-intermediate-1")
+                (version (?  string? ver))
+                (source
+                 (origin
+                   (method url-fetch)
+                   (uri (crate-uri "intermediate-1" version))
+                   (file-name
+                    (string-append name "-" version ".tar.gz"))
+                   (sha256
+                    (base32
+                     (?  string? hash)))))
+                (build-system cargo-build-system)
+                (arguments
+                 ('quasiquote (#:skip-build? #t
+                               #:cargo-inputs
+                               (("rust-intermediate-2-1.0.0"
+                                 ,rust-intermediate-2-1.0.0)
+                                ("rust-leaf-alice-1.0.0"
+                                 ('unquote rust-leaf-alice-1.0.0))
+                                ("rust-leaf-bob-1.0.0"
+                                 ('unquote rust-leaf-bob-1.0.0))))))
+                (home-page "http://example.com")
+                (synopsis "summary")
+                (description "summary")
+                (license (list license:expat license:asl2.0))))
+            (define-public rust-root-1.0.0
+              (package
+                (name "rust-root")
+                (version (?  string? ver))
+                (source
+                 (origin
+                   (method url-fetch)
+                   (uri (crate-uri "root" version))
+                   (file-name
+                    (string-append name "-" version ".tar.gz"))
+                   (sha256
+                    (base32
+                     (?  string? hash)))))
+                (build-system cargo-build-system)
+                (arguments
+                 ('quasiquote (#:skip-build?
+                               #t #:cargo-inputs
+                               (("rust-intermediate-1-1.0.0"
+                                 ('unquote rust-intermediate-1-1.0.0))
+                                ("rust-intermediate-2-1.0.0"
+                                 ('unquote rust-intermediate-2-1.0.0))
+                                ("rust-leaf-alice-1.0.0"
+                                 ('unquote rust-leaf-alice-1.0.0))
+                                ("rust-leaf-bob-1.0.0"
+                                 ('unquote rust-leaf-bob-1.0.0))))))
+                (home-page "http://example.com")
+                (synopsis "summary")
+                (description "summary")
+                (license (list license:expat license:asl2.0)))))
            #t)
           (x
            (pk 'fail x #f)))))
diff --git a/tests/elpa.scm b/tests/elpa.scm
index b70539bda6..a008cf993c 100644
--- a/tests/elpa.scm
+++ b/tests/elpa.scm
@@ -1,6 +1,7 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2015 Federico Beffa <beffa@fbengineering.ch>
 ;;; Copyright © 2020 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2020 Martin Becze <mjbecze@riseup.net>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -51,7 +52,7 @@
                       (200 "This is the description.")
                       (200 "fake tarball contents"))
     (parameterize ((current-http-proxy (%local-url)))
-      (match (elpa->guix-package pkg 'gnu/http)
+      (match (elpa->guix-package pkg #:repo 'gnu/http)
         (('package
            ('name "emacs-auctex")
            ('version "11.88.6")
-- 
2.25.0