From: Martin Becze <mjbecze@riseup.net>
To: "Ludovic Courtès" <ludo@gnu.org>
Cc: 38408@debbugs.gnu.org, Efraim Flashner <efraim@flashner.co.il>,
jsoo1@asu.edu, Leo Famulari <leo@famulari.name>
Subject: [bug#38408] [PATCH v9 3/8] Added Guile-Semver as a dependency to guix
Date: Tue, 24 Mar 2020 15:00:01 -0400 [thread overview]
Message-ID: <5ae20de5-ddf8-22ab-2c40-d76715751962@riseup.net> (raw)
In-Reply-To: <87mu86njj3.fsf@gnu.org>
[-- Attachment #1: Type: text/plain, Size: 2574 bytes --]
Ok Ludo, so I think I have worked through everything that you mentioned
and attached is a new patch set.
On 3/24/20 6:18 AM, Ludovic Courtès wrote:
>> +(define mem-lookup-crate (memoize lookup-crate))
>> +
>> (define (crate-version-dependencies version)
>> "Return the list of <crate-dependency> records of VERSION, a
>> <crate-version>."
>> @@ -216,7 +219,7 @@ latest version of CRATE-NAME."
>> (eq? (crate-dependency-kind dependency) 'normal)))
>>
>> (define crate
>> - (lookup-crate crate-name))
>> + (mem-lookup-crate crate-name))
>
> I’d suggest calling ‘mem-lookup-crate’ ‘lookup-crate*’ for instance.
> Can we also make its definition local to ‘crate-version-dependencies’ or
> would that defeat your caching goals?
I think it would, 'crate-version-dependencies' only deal with looking up
the dependencies. If I made it local 'crate->guix-package' it aslo
wouldn't work because we to cache across multiple calls to
'crate->guix-package'.
>> (define version-number
>> (or version
>> @@ -238,7 +241,7 @@ latest version of CRATE-NAME."
>> containing pairs of (name version)"
>> (sort (map (lambda (dep)
>> (let* ((name (crate-dependency-id dep))
>> - (crate (lookup-crate name))
>> + (crate (mem-lookup-crate name))
>> (req (crate-dependency-requirement dep))
>> (ver (find-version crate req)))
>> (list name
>> @@ -265,9 +268,11 @@ latest version of CRATE-NAME."
>> string->license))
>> cargo-inputs))))
>>
>> +(define mem-crate->guix-package (memoize crate->guix-package))
>> +
>> (define* (crate-recursive-import crate-name #:key version)
>> (recursive-import crate-name
>> - #:repo->guix-package crate->guix-package
>> + #:repo->guix-package mem-crate->guix-package
>
> Please make ‘mem-crate->guix-package’ local to ‘crate-recursive-import’
> and call it ‘crate->guix-package*’ for instance.
>
> (Memoization should always be used as a last resort: it’s a neat hack,
> but it’s a hack. :-) In particular, it has the problem that its cache
> cannot be easily invalidated. That said, I realize that other importers
> do this already, so that’s OK.)
Understood. If its any trouble it is easy to drop this commit. Though on
slow networks it can help quite a bit.
Let me know if anything else needs changed!
Thanks,
Martni
[-- Attachment #2: v13-0006-guix-self-Adds-guile-semver-as-a-depenedency.patch --]
[-- Type: text/x-patch, Size: 1898 bytes --]
From 1eefe635b4d0fd42a62bc1b8896f35224416e0f7 Mon Sep 17 00:00:00 2001
From: Martin Becze <mjbecze@riseup.net>
Date: Fri, 21 Feb 2020 10:41:44 -0500
Subject: [PATCH v13 6/6] guix: self: Adds guile-semver as a depenedency.
* guix/self.scm (compiled-guix) Added guile-semver as a depenedency.
---
guix/self.scm | 7 ++++++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/guix/self.scm b/guix/self.scm
index 6b633f9bc0..7da7fdea81 100644
--- a/guix/self.scm
+++ b/guix/self.scm
@@ -1,5 +1,6 @@
;;; GNU Guix --- Functional package management for GNU
;;; Copyright © 2017, 2018, 2019, 2020 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2020 Martin Becze <mjbecze@riseup.net>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -54,6 +55,7 @@
("guile-git" (ref '(gnu packages guile) 'guile3.0-git))
("guile-sqlite3" (ref '(gnu packages guile) 'guile3.0-sqlite3))
("guile-gcrypt" (ref '(gnu packages gnupg) 'guile3.0-gcrypt))
+ ("guile-semver" (ref '(gnu packages guile-xyz) 'guile3.0-semver))
("gnutls" (ref '(gnu packages tls) 'guile3.0-gnutls))
("zlib" (ref '(gnu packages compression) 'zlib))
("lzlib" (ref '(gnu packages compression) 'lzlib))
@@ -682,6 +684,9 @@ Info manual."
(define guile-gcrypt
(specification->package "guile-gcrypt"))
+ (define guile-semver
+ (specification->package "guile-semver"))
+
(define gnutls
(specification->package "gnutls"))
@@ -690,7 +695,7 @@ Info manual."
(cons (list "x" package)
(package-transitive-propagated-inputs package)))
(list guile-gcrypt gnutls guile-git guile-json
- guile-ssh guile-sqlite3))
+ guile-ssh guile-sqlite3 guile-semver))
(((labels packages _ ...) ...)
packages)))
--
2.25.2
[-- Attachment #3: v13-0005-import-crate-Parametrized-importing-of-dev-depen.patch --]
[-- Type: text/x-patch, Size: 6539 bytes --]
From 9a8281e0836a6304db61930490c45a88aea5eb45 Mon Sep 17 00:00:00 2001
From: Martin Becze <mjbecze@riseup.net>
Date: Mon, 3 Feb 2020 16:19:49 -0500
Subject: [PATCH v13 5/6] import: crate: Parametrized importing of dev
dependencies.
This changes the behavoir of the recusive crate importer so that it will
include importing of development dependencies for the top level package
but will not inculded the development dependencies for any other imported
package.
* guix/import/crate.scm (make-crate-sexp): Add the key BUILD?.
(crate->guix-package): Add the key INCLUDE-DEV-DEPS?.
(crate-recursive-import): Likewise.
* guix/scripts/import/crate.scm (guix-import-crate): Likewise.
* tests/crate.scm (cargo-recursive-import): Likewise.
---
guix/import/crate.scm | 27 +++++++++++++++++++--------
guix/scripts/import/crate.scm | 4 ++--
tests/crate.scm | 3 +--
3 files changed, 22 insertions(+), 12 deletions(-)
diff --git a/guix/import/crate.scm b/guix/import/crate.scm
index 96cf8d9b55..622b0c1975 100644
--- a/guix/import/crate.scm
+++ b/guix/import/crate.scm
@@ -152,7 +152,7 @@ record or #f if it was not found."
`((arguments (,'quasiquote ,args))))))
(define* (make-crate-sexp #:key name version cargo-inputs cargo-development-inputs
- home-page synopsis description license)
+ home-page synopsis description license build?)
"Return the `package' s-expression for a rust package with the given NAME,
VERSION, CARGO-INPUTS, CARGO-DEVELOPMENT-INPUTS, HOME-PAGE, SYNOPSIS, DESCRIPTION,
and LICENSE."
@@ -179,7 +179,9 @@ and LICENSE."
(base32
,(bytevector->nix-base32-string (port-sha256 port))))))
(build-system cargo-build-system)
- ,@(maybe-arguments (append '(#:skip-build? #t)
+ ,@(maybe-arguments (append (if build?
+ '()
+ '(#:skip-build? #t))
(maybe-cargo-inputs cargo-inputs)
(maybe-cargo-development-inputs
cargo-development-inputs)))
@@ -204,11 +206,12 @@ and LICENSE."
'unknown-license!)))
(string-split string (string->char-set " /"))))
-(define* (crate->guix-package crate-name #:key version repo)
+(define* (crate->guix-package crate-name #:key version include-dev-deps? repo)
"Fetch the metadata for CRATE-NAME from crates.io, and return the
`package' s-expression corresponding to that package, or #f on failure.
When VERSION is specified, attempt to fetch that version; otherwise fetch the
-latest version of CRATE-NAME."
+latest version of CRATE-NAME. If INCLUDE-DEV-DEPS is true then this
+will also lookup the development dependencs for the given crate."
(define (semver-range-contains-string? range version)
(semver-range-contains? (string->semver-range range)
@@ -254,9 +257,12 @@ latest version of CRATE-NAME."
(let* ((dependencies (crate-version-dependencies version*))
(dep-crates dev-dep-crates (partition normal-dependency? dependencies))
(cargo-inputs (sort-map-dependencies dep-crates))
- (cargo-development-inputs '()))
+ (cargo-development-inputs (if include-dev-deps?
+ (sort-map-dependencies dev-dep-crates)
+ '())))
(values
- (make-crate-sexp #:name crate-name
+ (make-crate-sexp #:build? include-dev-deps?
+ #:name crate-name
#:version (crate-version-number version*)
#:cargo-inputs cargo-inputs
#:cargo-development-inputs cargo-development-inputs
@@ -266,11 +272,16 @@ latest version of CRATE-NAME."
#:description (crate-description crate)
#:license (and=> (crate-version-license version*)
string->license))
- cargo-inputs))))
+ (append cargo-inputs cargo-development-inputs)))))
(define* (crate-recursive-import crate-name #:key version)
(recursive-import crate-name
- #:repo->guix-package (memoize crate->guix-package)
+ #:repo->guix-package (lambda* params
+ ;; only download the development dependencies for the top level package
+ (let ((include-dev-deps? (equal? (car params) crate-name))
+ (crate->guix-package* (memoize crate->guix-package)))
+ (apply crate->guix-package*
+ (append params `(#:include-dev-deps? ,include-dev-deps?)))))
#:version version
#:guix-name crate-name->package-name))
diff --git a/guix/scripts/import/crate.scm b/guix/scripts/import/crate.scm
index 552628cfc7..9252c52dfa 100644
--- a/guix/scripts/import/crate.scm
+++ b/guix/scripts/import/crate.scm
@@ -96,13 +96,13 @@ Import and convert the crate.io package for PACKAGE-NAME.\n"))
(if (assoc-ref opts 'recursive)
(crate-recursive-import name #:version version)
- (let ((sexp (crate->guix-package name #:version version)))
+ (let ((sexp (crate->guix-package name #:version version #:include-dev-deps? #t)))
(unless sexp
(leave (G_ "failed to download meta-data for package '~a'~%")
(if version
(string-append name "@" version)
name)))
- sexp)))
+ (list sexp))))
(()
(leave (G_ "too few arguments~%")))
((many ...)
diff --git a/tests/crate.scm b/tests/crate.scm
index 893dd70fc9..6fb9b772d8 100644
--- a/tests/crate.scm
+++ b/tests/crate.scm
@@ -461,8 +461,7 @@
(? string? hash)))))
(build-system cargo-build-system)
(arguments
- ('quasiquote (#:skip-build?
- #t #:cargo-inputs
+ ('quasiquote (#:cargo-inputs
(("rust-intermediate-1"
('unquote rust-intermediate-1-1.0))
("rust-intermediate-2"
--
2.25.2
[-- Attachment #4: v13-0004-import-utils-Trim-patch-version-from-names.patch --]
[-- Type: text/x-patch, Size: 6843 bytes --]
From 844a1f2009d9267e0b199bd50c086e0d693437aa Mon Sep 17 00:00:00 2001
From: Martin Becze <mjbecze@riseup.net>
Date: Thu, 30 Jan 2020 11:19:13 -0500
Subject: [PATCH v13 4/6] import: utils: Trim patch version from names.
This remove the the patch version from input names. For example
'rust-my-crate-1.1.2' now becomes 'rust-my-crate-1.1'
* guix/import/utils.scm (package->definition): Trim patch version from
generated package names.
* tests/crate.scm: (cargo>guix-package): Likewise.
(cargo-recursive-import): Likewise.
---
guix/import/utils.scm | 7 ++++---
tests/crate.scm | 44 +++++++++++++++++++++----------------------
2 files changed, 26 insertions(+), 25 deletions(-)
diff --git a/guix/import/utils.scm b/guix/import/utils.scm
index 709cd718f6..a0aacdc6de 100644
--- a/guix/import/utils.scm
+++ b/guix/import/utils.scm
@@ -265,9 +265,10 @@ package definition."
('package ('name name) ('version version) . rest)
('let _ ('package ('name name) ('version version) . rest)))
- `(define-public ,(string->symbol (if append-version?
- (string-append name "-" version)
- version))
+ `(define-public ,(string->symbol
+ (if append-version?
+ (string-append name "-" (version-major+minor version))
+ version))
,guix-package))))
(define (build-system-modules)
diff --git a/tests/crate.scm b/tests/crate.scm
index 39561d5745..893dd70fc9 100644
--- a/tests/crate.scm
+++ b/tests/crate.scm
@@ -279,7 +279,7 @@
(_ (error "Unexpected URL: " url)))))
(match (crate->guix-package "foo")
- ((define-public rust-foo-1.0.0
+ ((define-public rust-foo-1.0
(package (name "rust-foo")
(version "1.0.0")
(source
@@ -295,7 +295,7 @@
('quasiquote
(#:skip-build? #t
#:cargo-inputs
- (("rust-leaf-alice-1.0.0" ('unquote rust-leaf-alice-1.0.0))))))
+ (("rust-leaf-alice" ('unquote rust-leaf-alice-1.0))))))
(home-page "http://example.com")
(synopsis "summary")
(description "summary")
@@ -358,7 +358,7 @@
(_ (error "Unexpected URL: " url)))))
(match (crate-recursive-import "root")
;; rust-intermediate-2 has no dependency on the rust-leaf-alice package, so this is a valid ordering
- (((define-public rust-leaf-alice-1.0.0
+ (((define-public rust-leaf-alice-1.0
(package
(name "rust-leaf-alice")
(version (? string? ver))
@@ -377,7 +377,7 @@
(synopsis "summary")
(description "summary")
(license (list license:expat license:asl2.0))))
- (define-public rust-leaf-bob-1.0.0
+ (define-public rust-leaf-bob-1.0
(package
(name "rust-leaf-bob")
(version (? string? ver))
@@ -396,7 +396,7 @@
(synopsis "summary")
(description "summary")
(license (list license:expat license:asl2.0))))
- (define-public rust-intermediate-2-1.0.0
+ (define-public rust-intermediate-2-1.0
(package
(name "rust-intermediate-2")
(version (? string? ver))
@@ -413,13 +413,13 @@
(arguments
('quasiquote (#:skip-build? #t
#:cargo-inputs
- (("rust-leaf-bob-1.0.0"
+ (("rust-leaf-bob"
('unquote rust-leaf-bob-1.0.0))))))
(home-page "http://example.com")
(synopsis "summary")
(description "summary")
(license (list license:expat license:asl2.0))))
- (define-public rust-intermediate-1-1.0.0
+ (define-public rust-intermediate-1-1.0
(package
(name "rust-intermediate-1")
(version (? string? ver))
@@ -436,17 +436,17 @@
(arguments
('quasiquote (#:skip-build? #t
#:cargo-inputs
- (("rust-intermediate-2-1.0.0"
- ,rust-intermediate-2-1.0.0)
- ("rust-leaf-alice-1.0.0"
- ('unquote rust-leaf-alice-1.0.0))
- ("rust-leaf-bob-1.0.0"
- ('unquote rust-leaf-bob-1.0.0))))))
+ (("rust-intermediate-2"
+ ,rust-intermediate-2-1.0)
+ ("rust-leaf-alice"
+ ('unquote rust-leaf-alice-1.0))
+ ("rust-leaf-bob"
+ ('unquote rust-leaf-bob-1.0))))))
(home-page "http://example.com")
(synopsis "summary")
(description "summary")
(license (list license:expat license:asl2.0))))
- (define-public rust-root-1.0.0
+ (define-public rust-root-1.0
(package
(name "rust-root")
(version (? string? ver))
@@ -463,14 +463,14 @@
(arguments
('quasiquote (#:skip-build?
#t #:cargo-inputs
- (("rust-intermediate-1-1.0.0"
- ('unquote rust-intermediate-1-1.0.0))
- ("rust-intermediate-2-1.0.0"
- ('unquote rust-intermediate-2-1.0.0))
- ("rust-leaf-alice-1.0.0"
- ('unquote rust-leaf-alice-1.0.0))
- ("rust-leaf-bob-1.0.0"
- ('unquote rust-leaf-bob-1.0.0))))))
+ (("rust-intermediate-1"
+ ('unquote rust-intermediate-1-1.0))
+ ("rust-intermediate-2"
+ ('unquote rust-intermediate-2-1.0))
+ ("rust-leaf-alice"
+ ('unquote rust-leaf-alice-1.0))
+ ("rust-leaf-bob"
+ ('unquote rust-leaf-bob-1.0))))))
(home-page "http://example.com")
(synopsis "summary")
(description "summary")
--
2.25.2
[-- Attachment #5: v13-0003-import-crate-Memorize-crate-guix-package.patch --]
[-- Type: text/x-patch, Size: 2643 bytes --]
From b5dd33db1f33e44d88aab110dfa373470d73f4ed Mon Sep 17 00:00:00 2001
From: Martin Becze <mjbecze@riseup.net>
Date: Thu, 30 Jan 2020 11:17:00 -0500
Subject: [PATCH v13 3/6] import: crate: Memorize crate->guix-package.
This adds memorization to procedures that involve network lookups.
'lookup-crate*' is used on every dependency of a package to get its version
list. It is also used to lookup a packages metadata. 'crate-recursive-import'
is also memorized since creating the same package twice will trigger a lookup
on in its dependencies.
* guix/import/crate.scm (lookup-crate*): New procedure.
(crate->guix-package): Memorize package metadata lookups.
(crate-recursive-import): Memorize package creation.
---
guix/import/crate.scm | 9 ++++++---
1 file changed, 6 insertions(+), 3 deletions(-)
diff --git a/guix/import/crate.scm b/guix/import/crate.scm
index 633893765c..96cf8d9b55 100644
--- a/guix/import/crate.scm
+++ b/guix/import/crate.scm
@@ -28,6 +28,7 @@
#:use-module (guix import json)
#:use-module (guix import utils)
#:use-module ((guix licenses) #:prefix license:)
+ #:use-module (guix memoization)
#:use-module (guix monads)
#:use-module (guix packages)
#:use-module (guix upstream)
@@ -111,6 +112,8 @@ record or #f if it was not found."
(json->crate `(,@alist
("actual_versions" . ,versions))))))))
+(define lookup-crate* (memoize lookup-crate))
+
(define (crate-version-dependencies version)
"Return the list of <crate-dependency> records of VERSION, a
<crate-version>."
@@ -216,7 +219,7 @@ latest version of CRATE-NAME."
(eq? (crate-dependency-kind dependency) 'normal)))
(define crate
- (lookup-crate crate-name))
+ (lookup-crate* crate-name))
(define version-number
(or version
@@ -238,7 +241,7 @@ latest version of CRATE-NAME."
(define (sort-map-dependencies deps)
(sort (map (lambda (dep)
(let* ((name (crate-dependency-id dep))
- (crate (lookup-crate name))
+ (crate (lookup-crate* name))
(req (crate-dependency-requirement dep))
(ver (find-version crate req)))
(list name
@@ -267,7 +270,7 @@ latest version of CRATE-NAME."
(define* (crate-recursive-import crate-name #:key version)
(recursive-import crate-name
- #:repo->guix-package crate->guix-package
+ #:repo->guix-package (memoize crate->guix-package)
#:version version
#:guix-name crate-name->package-name))
--
2.25.2
[-- Attachment #6: v13-0002-import-crate-Use-guile-semver-to-resolve-module-.patch --]
[-- Type: text/x-patch, Size: 27099 bytes --]
From 950e7888fd73731a54347984755a232bab2068d0 Mon Sep 17 00:00:00 2001
From: Martin Becze <mjbecze@riseup.net>
Date: Tue, 4 Feb 2020 03:50:48 -0500
Subject: [PATCH v13 2/6] import: crate: Use guile-semver to resolve module
versions.
* guix/import/crate.scm (make-crate-sexp): Added '#:skip-build?' to build
system args. Pass a VERSION argument to 'cargo-inputs'. Move
'package-definition' from scripts/import/crate.scm to here.
(crate->guix-package): Use guile-semver to resolve the correct module versions.
(crate-name->package-name): Reuse the procedure 'guix-name' instead of
duplicating its logic.
(module) Added guile-semver as a soft dependency.
* guix/import/utils.scm (package-names->package-inputs): Implemented
handling of (name version) pairs.
* guix/scripts/import/crate.scm (guix-import-crate): Move
'package-definition' from here to guix/import/crate.scm.
* tests/crate.scm: (recursuve-import) Added version data to the test.
---
guix/import/crate.scm | 95 +++++++----
guix/import/utils.scm | 21 ++-
guix/scripts/import/crate.scm | 11 +-
tests/crate.scm | 290 +++++++++++++++++++---------------
4 files changed, 245 insertions(+), 172 deletions(-)
diff --git a/guix/import/crate.scm b/guix/import/crate.scm
index 0b4482e876..633893765c 100644
--- a/guix/import/crate.scm
+++ b/guix/import/crate.scm
@@ -1,7 +1,7 @@
;;; GNU Guix --- Functional package management for GNU
;;; Copyright © 2016 David Craven <david@craven.ch>
;;; Copyright © 2019, 2020 Ludovic Courtès <ludo@gnu.org>
-;;; Copyright © 2019 Martin Becze <mjbecze@riseup.net>
+;;; Copyright © 2019, 2020 Martin Becze <mjbecze@riseup.net>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -38,6 +38,7 @@
#:use-module (srfi srfi-1)
#:use-module (srfi srfi-2)
#:use-module (srfi srfi-26)
+ #:use-module (srfi srfi-71)
#:export (crate->guix-package
guix-package->crate-name
string->license
@@ -86,10 +87,15 @@
crate-dependency?
json->crate-dependency
(id crate-dependency-id "crate_id") ;string
- (kind crate-dependency-kind "kind" ;'normal | 'dev
+ (kind crate-dependency-kind "kind" ;'normal | 'dev | 'build
string->symbol)
(requirement crate-dependency-requirement "req")) ;string
+(module-autoload! (current-module)
+ '(semver) '(string->semver))
+(module-autoload! (current-module)
+ '(semver ranges) '(string->semver-range semver-range-contains?))
+
(define (lookup-crate name)
"Look up NAME on https://crates.io and return the corresopnding <crate>
record or #f if it was not found."
@@ -143,16 +149,22 @@ record or #f if it was not found."
`((arguments (,'quasiquote ,args))))))
(define* (make-crate-sexp #:key name version cargo-inputs cargo-development-inputs
- home-page synopsis description license
- #:allow-other-keys)
+ home-page synopsis description license)
"Return the `package' s-expression for a rust package with the given NAME,
VERSION, CARGO-INPUTS, CARGO-DEVELOPMENT-INPUTS, HOME-PAGE, SYNOPSIS, DESCRIPTION,
and LICENSE."
+ (define (format-inputs inputs)
+ (map
+ (match-lambda
+ ((name version)
+ (list (crate-name->package-name name)
+ (version-major+minor version))))
+ inputs))
+
(let* ((port (http-fetch (crate-uri name version)))
(guix-name (crate-name->package-name name))
- (cargo-inputs (map crate-name->package-name cargo-inputs))
- (cargo-development-inputs (map crate-name->package-name
- cargo-development-inputs))
+ (cargo-inputs (format-inputs cargo-inputs))
+ (cargo-development-inputs (format-inputs cargo-development-inputs))
(pkg `(package
(name ,guix-name)
(version ,version)
@@ -164,9 +176,10 @@ and LICENSE."
(base32
,(bytevector->nix-base32-string (port-sha256 port))))))
(build-system cargo-build-system)
- ,@(maybe-arguments (append (maybe-cargo-inputs cargo-inputs)
+ ,@(maybe-arguments (append '(#:skip-build? #t)
+ (maybe-cargo-inputs cargo-inputs)
(maybe-cargo-development-inputs
- cargo-development-inputs)))
+ cargo-development-inputs)))
(home-page ,(match home-page
(() "")
(_ home-page)))
@@ -177,7 +190,7 @@ and LICENSE."
((license) license)
(_ `(list ,@license)))))))
(close-port port)
- pkg))
+ (package->definition pkg #t)))
(define (string->license string)
(filter-map (lambda (license)
@@ -188,14 +201,19 @@ and LICENSE."
'unknown-license!)))
(string-split string (string->char-set " /"))))
-(define* (crate->guix-package crate-name #:optional version)
+(define* (crate->guix-package crate-name #:key version repo)
"Fetch the metadata for CRATE-NAME from crates.io, and return the
`package' s-expression corresponding to that package, or #f on failure.
When VERSION is specified, attempt to fetch that version; otherwise fetch the
latest version of CRATE-NAME."
+ (define (semver-range-contains-string? range version)
+ (semver-range-contains? (string->semver-range range)
+ (string->semver version)))
+
(define (normal-dependency? dependency)
- (eq? (crate-dependency-kind dependency) 'normal))
+ (or (eq? (crate-dependency-kind dependency) 'build)
+ (eq? (crate-dependency-kind dependency) 'normal)))
(define crate
(lookup-crate crate-name))
@@ -204,21 +222,36 @@ latest version of CRATE-NAME."
(or version
(crate-latest-version crate)))
- (define version*
+ ;; finds the a version of a crate that fulfills the semver <range>
+ (define (find-version crate range)
(find (lambda (version)
- (string=? (crate-version-number version)
- version-number))
+ (semver-range-contains-string?
+ range
+ (crate-version-number version)))
(crate-versions crate)))
+ (define version*
+ (find-version crate version-number))
+
+ ;; sorts the dependencies and maps the dependencies to a list containing
+ ;; pairs of (name version)
+ (define (sort-map-dependencies deps)
+ (sort (map (lambda (dep)
+ (let* ((name (crate-dependency-id dep))
+ (crate (lookup-crate name))
+ (req (crate-dependency-requirement dep))
+ (ver (find-version crate req)))
+ (list name
+ (crate-version-number ver))))
+ deps)
+ (match-lambda* (((_ name) ...)
+ (apply string-ci<? name)))))
+
(and crate version*
- (let* ((dependencies (crate-version-dependencies version*))
- (dep-crates (filter normal-dependency? dependencies))
- (dev-dep-crates (remove normal-dependency? dependencies))
- (cargo-inputs (sort (map crate-dependency-id dep-crates)
- string-ci<?))
- (cargo-development-inputs
- (sort (map crate-dependency-id dev-dep-crates)
- string-ci<?)))
+ (let* ((dependencies (crate-version-dependencies version*))
+ (dep-crates dev-dep-crates (partition normal-dependency? dependencies))
+ (cargo-inputs (sort-map-dependencies dep-crates))
+ (cargo-development-inputs '()))
(values
(make-crate-sexp #:name crate-name
#:version (crate-version-number version*)
@@ -230,15 +263,12 @@ latest version of CRATE-NAME."
#:description (crate-description crate)
#:license (and=> (crate-version-license version*)
string->license))
- (append cargo-inputs cargo-development-inputs)))))
+ cargo-inputs))))
-(define* (crate-recursive-import crate-name #:optional version)
- (recursive-import crate-name #f
- #:repo->guix-package
- (lambda (name repo)
- (let ((version (and (string=? name crate-name)
- version)))
- (crate->guix-package name version)))
+(define* (crate-recursive-import crate-name #:key version)
+ (recursive-import crate-name
+ #:repo->guix-package crate->guix-package
+ #:version version
#:guix-name crate-name->package-name))
(define (guix-package->crate-name package)
@@ -253,7 +283,7 @@ latest version of CRATE-NAME."
((name _ ...) name))))
(define (crate-name->package-name name)
- (string-append "rust-" (string-join (string-split name #\_) "-")))
+ (guix-name "rust-" name))
\f
;;;
@@ -288,4 +318,3 @@ latest version of CRATE-NAME."
(description "Updater for crates.io packages")
(pred crate-package?)
(latest latest-release)))
-
diff --git a/guix/import/utils.scm b/guix/import/utils.scm
index cd92cf7dd8..709cd718f6 100644
--- a/guix/import/utils.scm
+++ b/guix/import/utils.scm
@@ -225,13 +225,20 @@ into a proper sentence and by using two spaces between sentences."
cleaned 'pre ". " 'post)))
(define* (package-names->package-inputs names #:optional (output #f))
- "Given a list of PACKAGE-NAMES, and an optional OUTPUT, tries to generate a
-quoted list of inputs, as suitable to use in an 'inputs' field of a package
-definition."
- (map (lambda (input)
- (cons* input (list 'unquote (string->symbol input))
- (or (and output (list output))
- '())))
+ "Given a list of PACKAGE-NAMES or (PACKAGE-NAME VERSION) pairs, and an
+optional OUTPUT, tries to generate a quoted list of inputs, as suitable to
+use in an 'inputs' field of a package definition."
+ (define (make-input input version)
+ (cons* input (list 'unquote (string->symbol
+ (if version
+ (string-append input "-" version)
+ input)))
+ (or (and output (list output))
+ '())))
+
+ (map (match-lambda
+ ((input version) (make-input input version))
+ (input (make-input input #f)))
names))
(define* (maybe-inputs package-names #:optional (output #f))
diff --git a/guix/scripts/import/crate.scm b/guix/scripts/import/crate.scm
index d834518c18..552628cfc7 100644
--- a/guix/scripts/import/crate.scm
+++ b/guix/scripts/import/crate.scm
@@ -2,7 +2,7 @@
;;; GNU Guix --- Functional package management for GNU
;;; Copyright © 2014 David Thompson <davet@gnu.org>
;;; Copyright © 2016 David Craven <david@craven.ch>
-;;; Copyright © 2019 Martin Becze <mjbecze@riseup.net>
+;;; Copyright © 2019, 2020 Martin Becze <mjbecze@riseup.net>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -95,13 +95,8 @@ Import and convert the crate.io package for PACKAGE-NAME.\n"))
(package-name->name+version spec))
(if (assoc-ref opts 'recursive)
- (map (match-lambda
- ((and ('package ('name name) . rest) pkg)
- `(define-public ,(string->symbol name)
- ,pkg))
- (_ #f))
- (crate-recursive-import name version))
- (let ((sexp (crate->guix-package name version)))
+ (crate-recursive-import name #:version version)
+ (let ((sexp (crate->guix-package name #:version version)))
(unless sexp
(leave (G_ "failed to download meta-data for package '~a'~%")
(if version
diff --git a/tests/crate.scm b/tests/crate.scm
index aa51faebf9..39561d5745 100644
--- a/tests/crate.scm
+++ b/tests/crate.scm
@@ -2,6 +2,7 @@
;;; Copyright © 2014 David Thompson <davet@gnu.org>
;;; Copyright © 2016 David Craven <david@craven.ch>
;;; Copyright © 2019, 2020 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2020 Martin Becze <mjbecze@riseup.net>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -54,8 +55,9 @@
"{
\"dependencies\": [
{
- \"crate_id\": \"bar\",
+ \"crate_id\": \"leaf-alice\",
\"kind\": \"normal\",
+ \"req\": \"1.0.0\",
}
]
}")
@@ -88,18 +90,22 @@
{
\"crate_id\": \"intermediate-1\",
\"kind\": \"normal\",
+ \"req\": \"1.0.0\",
},
{
\"crate_id\": \"intermediate-2\",
\"kind\": \"normal\",
+ \"req\": \"1.0.0\",
}
{
\"crate_id\": \"leaf-alice\",
\"kind\": \"normal\",
+ \"req\": \"1.0.0\",
},
{
\"crate_id\": \"leaf-bob\",
\"kind\": \"normal\",
+ \"req\": \"1.0.0\",
},
]
}")
@@ -132,14 +138,17 @@
{
\"crate_id\": \"intermediate-2\",
\"kind\": \"normal\",
+ \"req\": \"1.0.0\",
},
{
\"crate_id\": \"leaf-alice\",
\"kind\": \"normal\",
+ \"req\": \"1.0.0\",
},
{
\"crate_id\": \"leaf-bob\",
\"kind\": \"normal\",
+ \"req\": \"1.0.0\",
}
]
}")
@@ -172,6 +181,7 @@
{
\"crate_id\": \"leaf-bob\",
\"kind\": \"normal\",
+ \"req\": \"1.0.0\",
},
]
}")
@@ -252,34 +262,48 @@
(open-input-string test-foo-crate))
("https://crates.io/api/v1/crates/foo/1.0.0/download"
(set! test-source-hash
- (bytevector->nix-base32-string
- (sha256 (string->bytevector "empty file\n" "utf-8"))))
+ (bytevector->nix-base32-string
+ (sha256 (string->bytevector "empty file\n" "utf-8"))))
(open-input-string "empty file\n"))
("https://crates.io/api/v1/crates/foo/1.0.0/dependencies"
(open-input-string test-foo-dependencies))
+ ("https://crates.io/api/v1/crates/leaf-alice"
+ (open-input-string test-leaf-alice-crate))
+ ("https://crates.io/api/v1/crates/leaf-alice/1.0.0/download"
+ (set! test-source-hash
+ (bytevector->nix-base32-string
+ (sha256 (string->bytevector "empty file\n" "utf-8"))))
+ (open-input-string "empty file\n"))
+ ("https://crates.io/api/v1/crates/leaf-alice/1.0.0/dependencies"
+ (open-input-string test-leaf-alice-dependencies))
(_ (error "Unexpected URL: " url)))))
- (match (crate->guix-package "foo")
- (('package
- ('name "rust-foo")
- ('version "1.0.0")
- ('source ('origin
- ('method 'url-fetch)
- ('uri ('crate-uri "foo" 'version))
- ('file-name ('string-append 'name "-" 'version ".tar.gz"))
- ('sha256
- ('base32
- (? string? hash)))))
- ('build-system 'cargo-build-system)
- ('arguments
- ('quasiquote
- ('#:cargo-inputs (("rust-bar" ('unquote rust-bar))))))
- ('home-page "http://example.com")
- ('synopsis "summary")
- ('description "summary")
- ('license ('list 'license:expat 'license:asl2.0)))
- (string=? test-source-hash hash))
- (x
- (pk 'fail x #f)))))
+
+ (match (crate->guix-package "foo")
+ ((define-public rust-foo-1.0.0
+ (package (name "rust-foo")
+ (version "1.0.0")
+ (source
+ (origin
+ (method url-fetch)
+ (uri (crate-uri "foo" 'version))
+ (file-name (string-append name "-" version ".tar.gz"))
+ (sha256
+ (base32
+ (? string? hash)))))
+ (build-system 'cargo-build-system)
+ (arguments
+ ('quasiquote
+ (#:skip-build? #t
+ #:cargo-inputs
+ (("rust-leaf-alice-1.0.0" ('unquote rust-leaf-alice-1.0.0))))))
+ (home-page "http://example.com")
+ (synopsis "summary")
+ (description "summary")
+ (license (list license:expat license:asl2.0))))
+
+ (string=? test-source-hash hash))
+ (x
+ (pk 'fail x #f)))))
(test-assert "cargo-recursive-import"
;; Replace network resources with sample data.
@@ -334,105 +358,123 @@
(_ (error "Unexpected URL: " url)))))
(match (crate-recursive-import "root")
;; rust-intermediate-2 has no dependency on the rust-leaf-alice package, so this is a valid ordering
- ((('package
- ('name "rust-leaf-alice")
- ('version (? string? ver))
- ('source
- ('origin
- ('method 'url-fetch)
- ('uri ('crate-uri "leaf-alice" 'version))
- ('file-name
- ('string-append 'name "-" 'version ".tar.gz"))
- ('sha256
- ('base32
- (? string? hash)))))
- ('build-system 'cargo-build-system)
- ('home-page "http://example.com")
- ('synopsis "summary")
- ('description "summary")
- ('license ('list 'license:expat 'license:asl2.0)))
- ('package
- ('name "rust-leaf-bob")
- ('version (? string? ver))
- ('source
- ('origin
- ('method 'url-fetch)
- ('uri ('crate-uri "leaf-bob" 'version))
- ('file-name
- ('string-append 'name "-" 'version ".tar.gz"))
- ('sha256
- ('base32
- (? string? hash)))))
- ('build-system 'cargo-build-system)
- ('home-page "http://example.com")
- ('synopsis "summary")
- ('description "summary")
- ('license ('list 'license:expat 'license:asl2.0)))
- ('package
- ('name "rust-intermediate-2")
- ('version (? string? ver))
- ('source
- ('origin
- ('method 'url-fetch)
- ('uri ('crate-uri "intermediate-2" 'version))
- ('file-name
- ('string-append 'name "-" 'version ".tar.gz"))
- ('sha256
- ('base32
- (? string? hash)))))
- ('build-system 'cargo-build-system)
- ('arguments
- ('quasiquote
- ('#:cargo-inputs (("rust-leaf-bob" ('unquote rust-leaf-bob))))))
- ('home-page "http://example.com")
- ('synopsis "summary")
- ('description "summary")
- ('license ('list 'license:expat 'license:asl2.0)))
- ('package
- ('name "rust-intermediate-1")
- ('version (? string? ver))
- ('source
- ('origin
- ('method 'url-fetch)
- ('uri ('crate-uri "intermediate-1" 'version))
- ('file-name
- ('string-append 'name "-" 'version ".tar.gz"))
- ('sha256
- ('base32
- (? string? hash)))))
- ('build-system 'cargo-build-system)
- ('arguments
- ('quasiquote
- ('#:cargo-inputs (("rust-intermediate-2" ('unquote rust-intermediate-2))
- ("rust-leaf-alice" ('unquote rust-leaf-alice))
- ("rust-leaf-bob" ('unquote rust-leaf-bob))))))
- ('home-page "http://example.com")
- ('synopsis "summary")
- ('description "summary")
- ('license ('list 'license:expat 'license:asl2.0)))
- ('package
- ('name "rust-root")
- ('version (? string? ver))
- ('source
- ('origin
- ('method 'url-fetch)
- ('uri ('crate-uri "root" 'version))
- ('file-name
- ('string-append 'name "-" 'version ".tar.gz"))
- ('sha256
- ('base32
- (? string? hash)))))
- ('build-system 'cargo-build-system)
- ('arguments
- ('quasiquote
- ('#:cargo-inputs (("rust-intermediate-1" ('unquote rust-intermediate-1))
- ("rust-intermediate-2" ('unquote rust-intermediate-2))
- ("rust-leaf-alice" ('unquote rust-leaf-alice))
- ("rust-leaf-bob" ('unquote rust-leaf-bob))))))
- ('home-page "http://example.com")
- ('synopsis "summary")
- ('description "summary")
- ('license ('list 'license:expat 'license:asl2.0))))
+ (((define-public rust-leaf-alice-1.0.0
+ (package
+ (name "rust-leaf-alice")
+ (version (? string? ver))
+ (source
+ (origin
+ (method url-fetch)
+ (uri (crate-uri "leaf-alice" version))
+ (file-name
+ (string-append name "-" version ".tar.gz"))
+ (sha256
+ (base32
+ (? string? hash)))))
+ (build-system cargo-build-system)
+ (arguments ('quasiquote (#:skip-build? #t)))
+ (home-page "http://example.com")
+ (synopsis "summary")
+ (description "summary")
+ (license (list license:expat license:asl2.0))))
+ (define-public rust-leaf-bob-1.0.0
+ (package
+ (name "rust-leaf-bob")
+ (version (? string? ver))
+ (source
+ (origin
+ (method url-fetch)
+ (uri (crate-uri "leaf-bob" version))
+ (file-name
+ (string-append name "-" version ".tar.gz"))
+ (sha256
+ (base32
+ (? string? hash)))))
+ (build-system cargo-build-system)
+ (arguments ('quasiquote (#:skip-build? #t)))
+ (home-page "http://example.com")
+ (synopsis "summary")
+ (description "summary")
+ (license (list license:expat license:asl2.0))))
+ (define-public rust-intermediate-2-1.0.0
+ (package
+ (name "rust-intermediate-2")
+ (version (? string? ver))
+ (source
+ (origin
+ (method url-fetch)
+ (uri (crate-uri "intermediate-2" version))
+ (file-name
+ (string-append name "-" version ".tar.gz"))
+ (sha256
+ (base32
+ (? string? hash)))))
+ (build-system cargo-build-system)
+ (arguments
+ ('quasiquote (#:skip-build? #t
+ #:cargo-inputs
+ (("rust-leaf-bob-1.0.0"
+ ('unquote rust-leaf-bob-1.0.0))))))
+ (home-page "http://example.com")
+ (synopsis "summary")
+ (description "summary")
+ (license (list license:expat license:asl2.0))))
+ (define-public rust-intermediate-1-1.0.0
+ (package
+ (name "rust-intermediate-1")
+ (version (? string? ver))
+ (source
+ (origin
+ (method url-fetch)
+ (uri (crate-uri "intermediate-1" version))
+ (file-name
+ (string-append name "-" version ".tar.gz"))
+ (sha256
+ (base32
+ (? string? hash)))))
+ (build-system cargo-build-system)
+ (arguments
+ ('quasiquote (#:skip-build? #t
+ #:cargo-inputs
+ (("rust-intermediate-2-1.0.0"
+ ,rust-intermediate-2-1.0.0)
+ ("rust-leaf-alice-1.0.0"
+ ('unquote rust-leaf-alice-1.0.0))
+ ("rust-leaf-bob-1.0.0"
+ ('unquote rust-leaf-bob-1.0.0))))))
+ (home-page "http://example.com")
+ (synopsis "summary")
+ (description "summary")
+ (license (list license:expat license:asl2.0))))
+ (define-public rust-root-1.0.0
+ (package
+ (name "rust-root")
+ (version (? string? ver))
+ (source
+ (origin
+ (method url-fetch)
+ (uri (crate-uri "root" version))
+ (file-name
+ (string-append name "-" version ".tar.gz"))
+ (sha256
+ (base32
+ (? string? hash)))))
+ (build-system cargo-build-system)
+ (arguments
+ ('quasiquote (#:skip-build?
+ #t #:cargo-inputs
+ (("rust-intermediate-1-1.0.0"
+ ('unquote rust-intermediate-1-1.0.0))
+ ("rust-intermediate-2-1.0.0"
+ ('unquote rust-intermediate-2-1.0.0))
+ ("rust-leaf-alice-1.0.0"
+ ('unquote rust-leaf-alice-1.0.0))
+ ("rust-leaf-bob-1.0.0"
+ ('unquote rust-leaf-bob-1.0.0))))))
+ (home-page "http://example.com")
+ (synopsis "summary")
+ (description "summary")
+ (license (list license:expat license:asl2.0)))))
#t)
(x
(pk 'fail x #f)))))
--
2.25.2
[-- Attachment #7: v13-0001-import-utils-recursive-import-accepts-an-optiona.patch --]
[-- Type: text/x-patch, Size: 18475 bytes --]
From e1d599345980b3f0a18eeaced2156c0926b7d926 Mon Sep 17 00:00:00 2001
From: Martin Becze <mjbecze@riseup.net>
Date: Tue, 4 Feb 2020 07:18:18 -0500
Subject: [PATCH v13 1/6] import: utils: 'recursive-import' accepts an optional
version parameter.
This adds a key VERSION to 'recursive-import' and move the paramter REPO to a
key. This also changes all the things that rely on 'recursive-import'
* guix/import/utils.scm (recursive-import): Add the VERSION key. Make REPO a
key.
(package->definition): Added optional 'append-version?'.
* guix/import/cran.scm (cran->guix-package): Change the REPO parameter to a key.
(cran-recursive-import): Likewise.
* guix/import/elpa.scm (elpa->guix-pakcage): Likewise.
(elpa-recursive-import): Likewise.
* guix/import/gem.scm (gem->guix-package): Likewise.
(recursive-import): Likewise.
* guix/import/opam.scm (opam-recurive-import): Likewise.
* guix/import/pypi.scm (pypi-recursive-import): Likewise.
* guix/import/stackage.scm (stackage-recursive-import): Likewise.
* guix/scripts/import/cran.scm: (guix-import-cran) Likewise.
* guix/scripts/import/elpa.scm: (guix-import-elpa) Likewise.
* tests/elpa.scm: (eval-test-with-elpa) Likewise.
* tests/import-utils.scm Likewise.
---
guix/import/cran.scm | 8 +++--
guix/import/elpa.scm | 6 ++--
guix/import/gem.scm | 6 ++--
guix/import/opam.scm | 8 ++---
guix/import/pypi.scm | 8 ++---
guix/import/stackage.scm | 5 +--
guix/import/utils.scm | 59 ++++++++++++++++++++++--------------
guix/scripts/import/cran.scm | 5 +--
guix/scripts/import/elpa.scm | 4 ++-
tests/elpa.scm | 3 +-
tests/import-utils.scm | 8 +++--
11 files changed, 74 insertions(+), 46 deletions(-)
diff --git a/guix/import/cran.scm b/guix/import/cran.scm
index bb8226f714..0651796c60 100644
--- a/guix/import/cran.scm
+++ b/guix/import/cran.scm
@@ -2,6 +2,7 @@
;;; Copyright © 2015, 2016, 2017, 2018, 2019, 2020 Ricardo Wurmus <rekado@elephly.net>
;;; Copyright © 2015, 2016, 2017, 2019, 2020 Ludovic Courtès <ludo@gnu.org>
;;; Copyright © 2017 Mathieu Othacehe <m.othacehe@gmail.com>
+;;; Copyright © 2020 Martin Becze <mjbecze@riseup.net>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -511,7 +512,7 @@ from the alist META, which was derived from the R package's DESCRIPTION file."
(define cran->guix-package
(memoize
- (lambda* (package-name #:optional (repo 'cran))
+ (lambda* (package-name #:key (repo 'cran) version)
"Fetch the metadata for PACKAGE-NAME from REPO and return the `package'
s-expression corresponding to that package, or #f on failure."
(let ((description (fetch-description repo package-name)))
@@ -526,8 +527,9 @@ s-expression corresponding to that package, or #f on failure."
(cran->guix-package package-name 'cran))
(else (values #f '()))))))))
-(define* (cran-recursive-import package-name #:optional (repo 'cran))
- (recursive-import package-name repo
+(define* (cran-recursive-import package-name #:key (repo 'cran))
+ (recursive-import package-name
+ #:repo repo
#:repo->guix-package cran->guix-package
#:guix-name cran-guix-name))
diff --git a/guix/import/elpa.scm b/guix/import/elpa.scm
index 2d4487dba0..0e32a6508d 100644
--- a/guix/import/elpa.scm
+++ b/guix/import/elpa.scm
@@ -2,6 +2,7 @@
;;; Copyright © 2015 Federico Beffa <beffa@fbengineering.ch>
;;; Copyright © 2015, 2016, 2017, 2018, 2020 Ludovic Courtès <ludo@gnu.org>
;;; Copyright © 2018 Oleg Pykhalov <go.wigust@gmail.com>
+;;; Copyright © 2020 Martin Becze <mjbecze@riseup.net>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -245,7 +246,7 @@ type '<elpa-package>'."
(license ,license))
dependencies-names)))
-(define* (elpa->guix-package name #:optional (repo 'gnu))
+(define* (elpa->guix-package name #:key (repo 'gnu) version)
"Fetch the package NAME from REPO and produce a Guix package S-expression."
(match (fetch-elpa-package name repo)
(#f #f)
@@ -301,7 +302,8 @@ type '<elpa-package>'."
(define elpa-guix-name (cut guix-name "emacs-" <>))
(define* (elpa-recursive-import package-name #:optional (repo 'gnu))
- (recursive-import package-name repo
+ (recursive-import package-name
+ #:repo repo
#:repo->guix-package elpa->guix-package
#:guix-name elpa-guix-name))
diff --git a/guix/import/gem.scm b/guix/import/gem.scm
index bd5d5b3569..345d6f003c 100644
--- a/guix/import/gem.scm
+++ b/guix/import/gem.scm
@@ -3,6 +3,7 @@
;;; Copyright © 2016 Ben Woodcroft <donttrustben@gmail.com>
;;; Copyright © 2018 Oleg Pykhalov <go.wigust@gmail.com>
;;; Copyright © 2020 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2020 Martin Becze <mjbecze@riseup.net>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -122,7 +123,7 @@ VERSION, HASH, HOME-PAGE, DESCRIPTION, DEPENDENCIES, and LICENSES."
((license) (license->symbol license))
(_ `(list ,@(map license->symbol licenses)))))))
-(define* (gem->guix-package package-name #:optional (repo 'rubygems) version)
+(define* (gem->guix-package package-name #:key (repo 'rubygems) version)
"Fetch the metadata for PACKAGE-NAME from rubygems.org, and return the
`package' s-expression corresponding to that package, or #f on failure."
(let ((gem (rubygems-fetch package-name)))
@@ -200,6 +201,7 @@ package on RubyGems."
(latest latest-release)))
(define* (gem-recursive-import package-name #:optional version)
- (recursive-import package-name '()
+ (recursive-import package-name
+ #:repo '()
#:repo->guix-package gem->guix-package
#:guix-name ruby-package-name))
diff --git a/guix/import/opam.scm b/guix/import/opam.scm
index ae7df8a8b5..81f178e6a9 100644
--- a/guix/import/opam.scm
+++ b/guix/import/opam.scm
@@ -1,5 +1,6 @@
;;; GNU Guix --- Functional package management for GNU
;;; Copyright © 2018 Julien Lepiller <julien@lepiller.eu>
+;;; Copyright © 2020 Martin Becze <mjbecze@riseup.net>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -250,7 +251,7 @@ path to the repository."
(substring version 1)
version)))))
-(define* (opam->guix-package name #:key (repository (get-opam-repository)))
+(define* (opam->guix-package name #:key (repository (get-opam-repository)) version)
"Import OPAM package NAME from REPOSITORY (a directory name) or, if
REPOSITORY is #f, from the official OPAM repository. Return a 'package' sexp
or #f on failure."
@@ -311,9 +312,8 @@ or #f on failure."
dependencies))))))))
(define (opam-recursive-import package-name)
- (recursive-import package-name #f
- #:repo->guix-package (lambda (name repo)
- (opam->guix-package name))
+ (recursive-import package-name
+ #:repo->guix-package opam->guix-package
#:guix-name ocaml-name->guix-name))
(define (guix-name->opam-name name)
diff --git a/guix/import/pypi.scm b/guix/import/pypi.scm
index f93fa8831f..3ec984b1f4 100644
--- a/guix/import/pypi.scm
+++ b/guix/import/pypi.scm
@@ -7,6 +7,7 @@
;;; Copyright © 2019 Maxim Cournoyer <maxim.cournoyer@gmail.com>
;;; Copyright © 2020 Jakub Kądziołka <kuba@kadziolka.net>
;;; Copyright © 2020 Lars-Dominik Braun <ldb@leibniz-psychology.org>
+;;; Copyright © 2020 Martin Becze <mjbecze@riseup.net>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -468,7 +469,7 @@ VERSION, SOURCE-URL, HOME-PAGE, SYNOPSIS, DESCRIPTION, and LICENSE."
(define pypi->guix-package
(memoize
- (lambda* (package-name)
+ (lambda* (package-name #:key repo version)
"Fetch the metadata for PACKAGE-NAME from pypi.org, and return the
`package' s-expression corresponding to that package, or #f on failure."
(let* ((project (pypi-fetch package-name))
@@ -492,9 +493,8 @@ VERSION, SOURCE-URL, HOME-PAGE, SYNOPSIS, DESCRIPTION, and LICENSE."
(project-info-license info)))))))))
(define (pypi-recursive-import package-name)
- (recursive-import package-name #f
- #:repo->guix-package (lambda (name repo)
- (pypi->guix-package name))
+ (recursive-import package-name
+ #:repo->guix-package pypi->guix-package
#:guix-name python->package-name))
(define (string->license str)
diff --git a/guix/import/stackage.scm b/guix/import/stackage.scm
index 14150201b5..767fc491bf 100644
--- a/guix/import/stackage.scm
+++ b/guix/import/stackage.scm
@@ -1,6 +1,7 @@
;;; GNU Guix --- Functional package management for GNU
;;; Copyright © 2017 Federico Beffa <beffa@fbengineering.ch>
;;; Copyright © 2018 Ricardo Wurmus <rekado@elephly.net>
+;;; Copyright © 2020 Martin Becze <mjbecze@riseup.net>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -108,8 +109,8 @@ included in the Stackage LTS release."
(leave-with-message "~a: Stackage package not found" package-name))))))
(define (stackage-recursive-import package-name . args)
- (recursive-import package-name #f
- #:repo->guix-package (lambda (name repo)
+ (recursive-import package-name
+ #:repo->guix-package (lambda* (name #:key repo version)
(apply stackage->guix-package (cons name args)))
#:guix-name hackage-name->package-name))
diff --git a/guix/import/utils.scm b/guix/import/utils.scm
index 94c8cb040b..cd92cf7dd8 100644
--- a/guix/import/utils.scm
+++ b/guix/import/utils.scm
@@ -5,6 +5,7 @@
;;; Copyright © 2017, 2019 Ricardo Wurmus <rekado@elephly.net>
;;; Copyright © 2018 Oleg Pykhalov <go.wigust@gmail.com>
;;; Copyright © 2019 Robert Vollmert <rob@vllmrt.net>
+;;; Copyright © 2020 Martin Becze <mjbecze@riseup.net>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -44,6 +45,7 @@
#:use-module (srfi srfi-9)
#:use-module (srfi srfi-11)
#:use-module (srfi srfi-26)
+ #:use-module (srfi srfi-71)
#:export (factorize-uri
flatten
@@ -250,13 +252,15 @@ package definition."
((package-inputs ...)
`((native-inputs (,'quasiquote ,package-inputs))))))
-(define (package->definition guix-package)
+(define* (package->definition guix-package #:optional append-version?)
(match guix-package
- (('package ('name (? string? name)) _ ...)
- `(define-public ,(string->symbol name)
- ,guix-package))
- (('let anything ('package ('name (? string? name)) _ ...))
- `(define-public ,(string->symbol name)
+ ((or
+ ('package ('name name) ('version version) . rest)
+ ('let _ ('package ('name name) ('version version) . rest)))
+
+ `(define-public ,(string->symbol (if append-version?
+ (string-append name "-" version)
+ version))
,guix-package))))
(define (build-system-modules)
@@ -391,32 +395,43 @@ obtain a node's uniquely identifying \"key\"."
(cons head result)
(set-insert (node-name head) visited))))))))
-(define* (recursive-import package-name repo
- #:key repo->guix-package guix-name
+(define* (recursive-import package-name
+ #:key repo->guix-package guix-name version repo
#:allow-other-keys)
"Return a list of package expressions for PACKAGE-NAME and all its
dependencies, sorted in topological order. For each package,
-call (REPO->GUIX-PACKAGE NAME REPO), which should return a package expression
-and a list of dependencies; call (GUIX-NAME NAME) to obtain the Guix package
-name corresponding to the upstream name."
+call (REPO->GUIX-PACKAGE NAME :KEYS version repo), which should return a
+package expression and a list of dependencies; call (GUIX-NAME NAME) to
+obtain the Guix package name corresponding to the upstream name."
(define-record-type <node>
- (make-node name package dependencies)
+ (make-node name version package dependencies)
node?
(name node-name)
+ (version node-version)
(package node-package)
(dependencies node-dependencies))
- (define (exists? name)
- (not (null? (find-packages-by-name (guix-name name)))))
+ (define (exists? name version)
+ (not (null? (find-packages-by-name (guix-name name) version))))
- (define (lookup-node name)
- (receive (package dependencies) (repo->guix-package name repo)
- (make-node name package dependencies)))
+ (define (lookup-node name version)
+ (let* ((package dependencies (repo->guix-package name
+ #:version version
+ #:repo repo))
+ (normilizied-deps (map (match-lambda
+ ((name version) (list name version))
+ (name (list name #f))) dependencies)))
+ (make-node name version package normilizied-deps)))
(map node-package
- (topological-sort (list (lookup-node package-name))
+ (topological-sort (list (lookup-node package-name version))
+ (lambda (node)
+ (map (lambda (name-version)
+ (apply lookup-node name-version))
+ (remove (lambda (name-version)
+ (apply exists? name-version))
+ (node-dependencies node))))
(lambda (node)
- (map lookup-node
- (remove exists?
- (node-dependencies node))))
- node-name)))
+ (string-append
+ (node-name node)
+ (or (node-version node) ""))))))
diff --git a/guix/scripts/import/cran.scm b/guix/scripts/import/cran.scm
index d6f371ef3a..bc266ad9da 100644
--- a/guix/scripts/import/cran.scm
+++ b/guix/scripts/import/cran.scm
@@ -1,6 +1,7 @@
;;; GNU Guix --- Functional package management for GNU
;;; Copyright © 2014 Eric Bavier <bavier@member.fsf.org>
;;; Copyright © 2015, 2017, 2019 Ricardo Wurmus <rekado@elephly.net>
+;;; Copyright © 2020 Martin Becze <mjbecze@riseup.net>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -98,10 +99,10 @@ Import and convert the CRAN package for PACKAGE-NAME.\n"))
;; Recursive import
(map package->definition
(cran-recursive-import package-name
- (or (assoc-ref opts 'repo) 'cran)))
+ #:repo (or (assoc-ref opts 'repo) 'cran)))
;; Single import
(let ((sexp (cran->guix-package package-name
- (or (assoc-ref opts 'repo) 'cran))))
+ #:repo (or (assoc-ref opts 'repo) 'cran))))
(unless sexp
(leave (G_ "failed to download description for package '~a'~%")
package-name))
diff --git a/guix/scripts/import/elpa.scm b/guix/scripts/import/elpa.scm
index d270d2b4bc..07ac07a3d5 100644
--- a/guix/scripts/import/elpa.scm
+++ b/guix/scripts/import/elpa.scm
@@ -1,6 +1,7 @@
;;; GNU Guix --- Functional package management for GNU
;;; Copyright © 2015 Federico Beffa <beffa@fbengineering.ch>
;;; Copyright © 2018 Oleg Pykhalov <go.wigust@gmail.com>
+;;; Copyright © 2020 Martin Becze <mjbecze@riseup.net>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -102,7 +103,8 @@ Import the latest package named PACKAGE-NAME from an ELPA repository.\n"))
(_ #f))
(elpa-recursive-import package-name
(or (assoc-ref opts 'repo) 'gnu)))
- (let ((sexp (elpa->guix-package package-name (assoc-ref opts 'repo))))
+ (let ((sexp (elpa->guix-package package-name
+ #:repo (assoc-ref opts 'repo))))
(unless sexp
(leave (G_ "failed to download package '~a'~%") package-name))
sexp)))
diff --git a/tests/elpa.scm b/tests/elpa.scm
index b70539bda6..a008cf993c 100644
--- a/tests/elpa.scm
+++ b/tests/elpa.scm
@@ -1,6 +1,7 @@
;;; GNU Guix --- Functional package management for GNU
;;; Copyright © 2015 Federico Beffa <beffa@fbengineering.ch>
;;; Copyright © 2020 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2020 Martin Becze <mjbecze@riseup.net>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -51,7 +52,7 @@
(200 "This is the description.")
(200 "fake tarball contents"))
(parameterize ((current-http-proxy (%local-url)))
- (match (elpa->guix-package pkg 'gnu/http)
+ (match (elpa->guix-package pkg #:repo 'gnu/http)
(('package
('name "emacs-auctex")
('version "11.88.6")
diff --git a/tests/import-utils.scm b/tests/import-utils.scm
index 87dda3238f..2357ea5c40 100644
--- a/tests/import-utils.scm
+++ b/tests/import-utils.scm
@@ -1,6 +1,7 @@
;;; GNU Guix --- Functional package management for GNU
;;; Copyright © 2015, 2017 Ricardo Wurmus <rekado@elephly.net>
;;; Copyright © 2016 Ben Woodcroft <donttrustben@gmail.com>
+;;; Copyright © 2020 Martin Becze <mjbecze@riseup.net>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -48,15 +49,16 @@
(package
(name "foo")
(inputs `(("bar" ,bar)))))
- (recursive-import "foo" 'repo
+ (recursive-import "foo"
+ #:repo 'repo
#:repo->guix-package
(match-lambda*
- (("foo" 'repo)
+ (("foo" #:version #f #:repo 'repo)
(values '(package
(name "foo")
(inputs `(("bar" ,bar))))
'("bar")))
- (("bar" 'repo)
+ (("bar" #:version #f #:repo 'repo)
(values '(package
(name "bar"))
'())))
--
2.25.2
next prev parent reply other threads:[~2020-03-24 19:02 UTC|newest]
Thread overview: 107+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-11-10 21:39 [bug#44560] [PATCH v16 0/6] New take on: Semantic version aware recursive importer for crates Hartmut Goebel
2020-11-10 21:39 ` [bug#38408] [PATCH v16 1/6] import: utils: 'recursive-import' accepts an optional version parameter Hartmut Goebel
2020-11-10 21:39 ` [bug#38408] [PATCH v16 2/6] guix: self: Add guile-semver as a depenedency Hartmut Goebel
2020-11-10 21:39 ` [bug#38408] [PATCH v16 3/6] import: crate: Use guile-semver to resolve module versions Hartmut Goebel
2020-11-10 22:13 ` Hartmut Goebel
2019-11-28 0:13 ` [bug#38408] [PATCH 0/3] (WIP) Semantic version aware recusive importer for crates Martin Becze
2019-11-28 0:16 ` [bug#38408] [PATCH 1/3] gnu: added new function, find-packages-by-name*/direct Martin Becze
2019-11-28 0:16 ` [bug#38408] [PATCH 2/3] gnu: added new procedure, recusive-import-semver Martin Becze
2019-11-28 0:16 ` [bug#38408] [PATCH 3/3] Rewrote some of guix/import/crate.scm to use recursive-import-semver and updated script and test Martin Becze
2019-11-30 16:36 ` Martin Becze
2019-12-01 9:00 ` Efraim Flashner
2019-12-05 20:05 ` [bug#38408] [PATCH v2 0/5] Semantic version aware recusive importer for crates Martin Becze
2019-12-05 20:05 ` [bug#38408] [PATCH v2 1/5] gnu: added new function, find-packages-by-name*/direct Martin Becze
2019-12-05 20:05 ` [bug#38408] [PATCH v2 2/5] gnu: added new procedure, recusive-import-semver Martin Becze
2019-12-05 20:05 ` [bug#38408] [PATCH v2 3/5] Rewrote some of guix/import/crate.scm to use recursive-import-semver and updated script and test Martin Becze
2019-12-05 20:05 ` [bug#38408] [PATCH v2 4/5] added "#:skip-build? #t" to the output of (make-crate-sexp). Most the the packages imported will be libaries and won't need to build. The top level package will build them though Martin Becze
2019-12-05 20:05 ` [bug#38408] [PATCH v2 5/5] guix: crate: Depublicated build and normal dependencies Martin Becze
2019-12-06 18:21 ` [bug#38408] [PATCH v3 0/5] Semantic version aware recusive importer for crates Martin Becze
2019-12-06 18:21 ` [bug#38408] [PATCH v3 1/5] gnu: added new function, find-packages-by-name*/direct Martin Becze
2019-12-06 18:21 ` [bug#38408] [PATCH v3 2/5] gnu: added new procedure, recusive-import-semver Martin Becze
2019-12-06 18:21 ` [bug#38408] [PATCH v3 3/5] Rewrote some of guix/import/crate.scm to use recursive-import-semver and updated script and test Martin Becze
2019-12-06 18:21 ` [bug#38408] [PATCH v3 4/5] added "#:skip-build? #t" to the output of (make-crate-sexp). Most the the packages imported will be libaries and won't need to build. The top level package will build them though Martin Becze
2019-12-06 18:21 ` [bug#38408] [PATCH v3 5/5] guix: crate: Depublicated dependencies Martin Becze
2019-12-10 19:23 ` [bug#38408] [PATCH v4 0/6] Semantic version aware recusive importer for crates Martin Becze
2019-12-10 19:23 ` [bug#38408] [PATCH v4 1/6] gnu: added new function, find-packages-by-name*/direct Martin Becze
2019-12-19 22:00 ` Ludovic Courtès
2019-12-20 18:37 ` Martin Becze
2019-12-27 18:38 ` Ludovic Courtès
2020-01-18 16:40 ` [bug#38408] [PATCH v6] Semantic version aware recusive importer for crates Martin Becze
2019-12-10 19:23 ` [bug#38408] [PATCH v4 2/6] gnu: added new procedure, recusive-import-semver Martin Becze
2019-12-19 22:07 ` Ludovic Courtès
2019-12-20 18:46 ` Martin Becze
2019-12-10 19:23 ` [bug#38408] [PATCH v4 3/6] Rewrote some of guix/import/crate.scm to use recursive-import-semver and updated script and test Martin Becze
2019-12-10 19:23 ` [bug#38408] [PATCH v4 4/6] added "#:skip-build? #t" to the output of (make-crate-sexp). Most the the packages imported will be libaries and won't need to build. The top level package will build them though Martin Becze
2019-12-10 19:23 ` [bug#38408] [PATCH v4 5/6] guix: crate: Depublicated dependencies Martin Becze
2019-12-10 19:23 ` [bug#38408] [PATCH v4 6/6] guix: import: recursive-import-semver: allow the range of a package to be specified when begining import Martin Becze
2019-12-16 23:30 ` [bug#38408] Rewrote recursive-import-semver based on topological-sort Martin Becze
2020-02-04 12:18 ` [bug#38408] [PATCH v9 0/8] recursive semver crate importer! Martin Becze
2020-02-04 12:18 ` [bug#38408] [PATCH v9 1/8] guix: import: (recursive-import) Allow for version numbers Martin Becze
2020-02-17 10:03 ` Efraim Flashner
2020-02-04 12:18 ` [bug#38408] [PATCH v9 2/8] guix: import: crate: Use semver to resovle module versions Martin Becze
2020-02-17 14:35 ` Ludovic Courtès
2020-02-17 14:57 ` Efraim Flashner
2020-02-17 15:37 ` Ludovic Courtès
2020-02-18 8:56 ` Martin Becze
2020-02-04 12:18 ` [bug#38408] [PATCH v9 3/8] Added Guile-Semver as a dependency to guix Martin Becze
2020-02-17 10:03 ` Efraim Flashner
2020-02-17 14:36 ` Ludovic Courtès
2020-02-18 9:30 ` Martin Becze
2020-02-20 9:40 ` Ludovic Courtès
2020-02-20 16:54 ` Martin Becze
2020-02-21 9:01 ` Ludovic Courtès
2020-02-21 16:25 ` Martin Becze
2020-02-21 16:27 ` Leo Famulari
2020-02-23 20:34 ` Martin Becze
2020-02-23 21:05 ` Martin Becze
2020-03-11 20:20 ` Martin Becze
2020-03-21 18:35 ` Martin Becze
2020-03-22 19:26 ` Leo Famulari
2020-03-22 20:10 ` Leo Famulari
2020-03-23 9:50 ` Martin Becze
2020-03-23 16:28 ` Martin Becze
2020-03-24 10:18 ` Ludovic Courtès
2020-03-24 14:19 ` Martin Becze
2020-03-24 19:00 ` Martin Becze [this message]
2020-04-12 15:07 ` Martin Becze
2020-04-12 16:59 ` Ludovic Courtès
2020-04-17 14:57 ` Martin Becze
2020-04-29 19:50 ` Martin Becze
2020-04-29 19:51 ` Martin Becze
2020-05-08 19:57 ` Martin Becze
2020-08-18 9:44 ` Martin Becze
2020-07-05 0:23 ` Jakub Kądziołka
2020-02-04 12:18 ` [bug#38408] [PATCH v9 4/8] guix: import: utils: allow generation of inputs to be version aware Martin Becze
2020-02-04 12:18 ` [bug#38408] [PATCH v9 5/8] guix: import: crate: deduplicate dependencies Martin Becze
2020-02-04 12:18 ` [bug#38408] [PATCH v9 6/8] guix: import: crate: memorize crate->guix-package Martin Becze
2020-02-04 12:18 ` [bug#38408] [PATCH v9 7/8] guix: import: utils: trim patch version from names Martin Becze
2020-02-04 12:18 ` [bug#38408] [PATCH v9 8/8] guix: import: parametrized importing of dev dependencies Martin Becze
2020-02-20 18:53 ` [bug#38408] [PATCH v9 0/8] recursive semver crate importer! Leo Famulari
2020-02-21 8:35 ` Martin Becze
2020-02-21 12:15 ` Efraim Flashner
2020-02-21 16:29 ` Martin Becze
2020-11-07 22:19 ` [bug#38408] [PATCH 0/3] (WIP) Semantic version aware recusive importer for crates Hartmut Goebel
2020-11-07 22:35 ` Marius Bakke
2020-11-09 17:15 ` Hartmut Goebel
2020-11-09 17:27 ` Nicolas Goaziou
2020-11-11 15:06 ` [bug#38408] [PATCH v16 3/6] import: crate: Use guile-semver to resolve module versions Timothy Sample
2020-11-16 19:07 ` [bug#44694] [PATCH v17 0/8] New take continued: Semantic version aware recursive Hartmut Goebel
2020-11-16 19:07 ` [bug#38408] [PATCH v17 1/8] guix: self: Add guile-semver as a depenedency Hartmut Goebel
2020-11-16 19:07 ` [bug#38408] [PATCH v17 2/8] import: utils: 'recursive-import' accepts an optional version parameter Hartmut Goebel
2020-11-16 19:07 ` [bug#38408] [PATCH v17 3/8] import: crate: Use guile-semver to resolve module versions Hartmut Goebel
2020-11-16 19:07 ` [bug#38408] [PATCH v17 4/8] import: crate: Memorize crate->guix-package Hartmut Goebel
2020-11-16 19:07 ` [bug#38408] [PATCH v17 5/8] import: crate: Parameterized importing of dev dependencies Hartmut Goebel
2020-11-16 19:07 ` [bug#38408] [PATCH v17 6/8] import: utils: Trim patch version from names Hartmut Goebel
2020-11-16 19:07 ` [bug#38408] [PATCH v17 7/8] import: crate: Trim version for names after left-most non-zero part Hartmut Goebel
2020-11-16 19:07 ` [bug#38408] [PATCH v17 8/8] import: crate: Use existing package satisfying semver requirement Hartmut Goebel
2020-11-17 21:54 ` [bug#38408] [PATCH v17 0/8] New take continued: Semantic version aware recursive Martin Becze
2020-12-02 21:13 ` bug#38408: " Hartmut Goebel
2020-12-02 21:48 ` [bug#38408] " Leo Famulari
2020-11-17 21:43 ` [bug#38408] [PATCH v16 3/6] import: crate: Use guile-semver to resolve module versions Martin Becze
2020-11-17 21:51 ` Martin Becze
2020-11-18 7:56 ` Hartmut Goebel
2020-11-10 21:39 ` [bug#38408] [PATCH v16 4/6] import: crate: Memorize crate->guix-package Hartmut Goebel
2020-11-10 21:39 ` [bug#38408] [PATCH v16 5/6] import: utils: Trim patch version from names Hartmut Goebel
2020-11-10 21:39 ` [bug#38408] [PATCH v16 6/6] import: crate: Parameterized importing of dev dependencies Hartmut Goebel
2020-11-10 22:21 ` Hartmut Goebel
2020-11-10 22:24 ` [bug#38408] [PATCH v16 0/6] New take on: Semantic version aware recursive importer for crates Hartmut Goebel
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://guix.gnu.org/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=5ae20de5-ddf8-22ab-2c40-d76715751962@riseup.net \
--to=mjbecze@riseup.net \
--cc=38408@debbugs.gnu.org \
--cc=efraim@flashner.co.il \
--cc=jsoo1@asu.edu \
--cc=leo@famulari.name \
--cc=ludo@gnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/guix.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).