* [bug#53828] [PATCH] guix: opam: Allow importing local files.
@ 2022-02-06 21:41 Julien Lepiller
2022-02-06 21:58 ` Maxime Devos
` (2 more replies)
0 siblings, 3 replies; 7+ messages in thread
From: Julien Lepiller @ 2022-02-06 21:41 UTC (permalink / raw)
To: 53828
[-- Attachment #1: Type: text/plain, Size: 473 bytes --]
Hi Guix!
This patch lets you import opam packages from a local file. This is a
feature that was requested a few times, and there it is :)
Now, if you have an OCaml package sources that provides an opam file
(usually called opam, or with the .opam extension), you can import the
package, recursively or not with something like:
guix import opam foo -f foo.opam -r
instead of fetching foo from the repositories, it will fetch the data
directly from the given file. WDYT?
[-- Attachment #2: 0001-guix-opam-Allow-importing-local-files.patch --]
[-- Type: text/x-patch, Size: 11718 bytes --]
From b027391357eef4d6fa1bb8741528cf9650aef568 Mon Sep 17 00:00:00 2001
Message-Id: <b027391357eef4d6fa1bb8741528cf9650aef568.1644183523.git.julien@lepiller.eu>
From: Julien Lepiller <julien@lepiller.eu>
Date: Sun, 6 Feb 2022 22:35:14 +0100
Subject: [PATCH] guix: opam: Allow importing local files.
* guix/scripts/import/opam.scm (guix-import-opam): Support `--file` flag.
* guix/import/utils.scm (recursive-import): Add optional file argument.
Pass it to the first iteration when available.
* guix/import/opam.scm (opam-fetch): Fetch from file when available.
(opam->guix-source): Factorize source out of opam->guix-package.
(opam->guix-package): Add file argument. Ensure it does not fail when
name, version and url are not available.
---
guix/import/opam.scm | 101 +++++++++++++++++++----------------
guix/import/utils.scm | 17 +++---
guix/scripts/import/opam.scm | 11 +++-
3 files changed, 75 insertions(+), 54 deletions(-)
diff --git a/guix/import/opam.scm b/guix/import/opam.scm
index a6f6fe8c9f..05f79110f8 100644
--- a/guix/import/opam.scm
+++ b/guix/import/opam.scm
@@ -310,8 +310,11 @@ (define (dependency-list->inputs lst)
(map string->symbol
(ocaml-names->guix-names lst)))
-(define* (opam-fetch name #:optional (repositories-specs '("opam")))
- (or (fold (lambda (repository others)
+(define* (opam-fetch name #:optional (repositories-specs '("opam")) (file #f))
+ (or (and file (let ((metadata (get-metadata file)))
+ `(("metadata" . ,metadata)
+ ("version" . #f))))
+ (fold (lambda (repository others)
(match (find-latest-version name repository)
((_ version file) `(("metadata" ,@(get-metadata file))
("version" . ,version)))
@@ -320,17 +323,29 @@ (define* (opam-fetch name #:optional (repositories-specs '("opam")))
(filter-map get-opam-repository repositories-specs))
(warning (G_ "opam: package '~a' not found~%") name)))
-(define* (opam->guix-package name #:key (repo 'opam) version)
+(define (opam->guix-source url-dict)
+ (let ((source-url (and url-dict
+ (or (metadata-ref url-dict "src")
+ (metadata-ref url-dict "archive")))))
+ (if source-url
+ (call-with-temporary-output-file
+ (lambda (temp port)
+ (and (url-fetch source-url temp)
+ `(origin
+ (method url-fetch)
+ (uri ,source-url)
+ (sha256 (base32 ,(guix-hash-url temp)))))))
+ 'no-source-information)))
+
+(define* (opam->guix-package name #:key (repo '("opam")) (file #f) version)
"Import OPAM package NAME from REPOSITORY (a directory name) or, if
-REPOSITORY is #f, from the official OPAM repository. Return a 'package' sexp
+REPOSITORY is #f, from the official OPAM repository. When FILE is defined,
+import the packaged defined in the opam FILE instead. Return a 'package' sexp
or #f on failure."
(and-let* ((with-opam (if (member "opam" repo) repo (cons "opam" repo)))
- (opam-file (opam-fetch name with-opam))
- (version (assoc-ref opam-file "version"))
+ (opam-file (opam-fetch name with-opam file))
(opam-content (assoc-ref opam-file "metadata"))
- (url-dict (metadata-ref opam-content "url"))
- (source-url (or (metadata-ref url-dict "src")
- (metadata-ref url-dict "archive")))
+ (source (opam->guix-source (metadata-ref opam-content "url")))
(requirements (metadata-ref opam-content "depends"))
(names (dependency-list->names requirements))
(dependencies (filter-dependencies names))
@@ -344,47 +359,41 @@ (define* (opam->guix-package name #:key (repo 'opam) version)
(not (member name '("dune" "jbuilder"))))
native-dependencies))))
(let ((use-dune? (member "dune" names)))
- (call-with-temporary-output-file
- (lambda (temp port)
- (and (url-fetch source-url temp)
- (values
- `(package
- (name ,(ocaml-name->guix-name name))
- (version ,version)
- (source
- (origin
- (method url-fetch)
- (uri ,source-url)
- (sha256 (base32 ,(guix-hash-url temp)))))
- (build-system ,(if use-dune?
- 'dune-build-system
- 'ocaml-build-system))
- ,@(if (null? inputs)
- '()
- `((propagated-inputs (list ,@inputs))))
- ,@(if (null? native-inputs)
- '()
- `((native-inputs (list ,@native-inputs))))
- ,@(if (equal? name (guix-name->opam-name (ocaml-name->guix-name name)))
- '()
- `((properties
- ,(list 'quasiquote `((upstream-name . ,name))))))
- (home-page ,(metadata-ref opam-content "homepage"))
- (synopsis ,(metadata-ref opam-content "synopsis"))
- (description ,(beautify-description
- (metadata-ref opam-content "description")))
- (license ,(spdx-string->license
- (metadata-ref opam-content "license"))))
- (filter
- (lambda (name)
- (not (member name '("dune" "jbuilder"))))
- dependencies))))))))
+ (values
+ `(package
+ (name ,(and name (ocaml-name->guix-name name)))
+ (version ,(assoc-ref opam-file "version"))
+ (source ,source)
+ (build-system ,(if use-dune?
+ 'dune-build-system
+ 'ocaml-build-system))
+ ,@(if (null? inputs)
+ '()
+ `((propagated-inputs (list ,@inputs))))
+ ,@(if (null? native-inputs)
+ '()
+ `((native-inputs (list ,@native-inputs))))
+ ,@(if (and name (equal? name (guix-name->opam-name (ocaml-name->guix-name name))))
+ '()
+ `((properties
+ ,(list 'quasiquote `((upstream-name . ,name))))))
+ (home-page ,(metadata-ref opam-content "homepage"))
+ (synopsis ,(metadata-ref opam-content "synopsis"))
+ (description ,(beautify-description
+ (metadata-ref opam-content "description")))
+ (license ,(spdx-string->license
+ (metadata-ref opam-content "license"))))
+ (filter
+ (lambda (name)
+ (not (member name '("dune" "jbuilder"))))
+ dependencies)))))
-(define* (opam-recursive-import package-name #:key repo)
+(define* (opam-recursive-import package-name #:key repo file)
(recursive-import package-name
#:repo->guix-package opam->guix-package
#:guix-name ocaml-name->guix-name
- #:repo repo))
+ #:repo repo
+ #:file file))
(define (guix-name->opam-name name)
(if (string-prefix? "ocaml-" name)
diff --git a/guix/import/utils.scm b/guix/import/utils.scm
index 1c3cfa3e0b..ab35b8a4fc 100644
--- a/guix/import/utils.scm
+++ b/guix/import/utils.scm
@@ -470,7 +470,7 @@ (define (topological-sort nodes
(set-insert (node-name head) visited))))))))
(define* (recursive-import package-name
- #:key repo->guix-package guix-name version repo
+ #:key repo->guix-package guix-name version repo file
#:allow-other-keys)
"Return a list of package expressions for PACKAGE-NAME and all its
dependencies, sorted in topological order. For each package,
@@ -488,10 +488,15 @@ (define* (recursive-import package-name
(define (exists? name version)
(not (null? (find-packages-by-name (guix-name name) version))))
- (define (lookup-node name version)
- (let* ((package dependencies (repo->guix-package name
- #:version version
- #:repo repo))
+ (define* (lookup-node name version #:optional file)
+ (let* ((package dependencies (if file
+ (repo->guix-package name
+ #:version version
+ #:repo repo
+ #:file file)
+ (repo->guix-package name
+ #:version version
+ #:repo repo)))
(normalized-deps (map (match-lambda
((name version) (list name version))
(name (list name #f))) dependencies)))
@@ -499,7 +504,7 @@ (define* (recursive-import package-name
(filter-map
node-package
- (topological-sort (list (lookup-node package-name version))
+ (topological-sort (list (lookup-node package-name version file))
(lambda (node)
(map (lambda (name-version)
(apply lookup-node name-version))
diff --git a/guix/scripts/import/opam.scm b/guix/scripts/import/opam.scm
index 834ac34cb0..b58c6eab3d 100644
--- a/guix/scripts/import/opam.scm
+++ b/guix/scripts/import/opam.scm
@@ -50,6 +50,8 @@ (define (show-help)
--repo import packages from this opam repository (name, URL or local path)
can be used more than once"))
(display (G_ "
+ -f, --file import the package defined from a .opam file"))
+ (display (G_ "
-V, --version display version information and exit"))
(newline)
(show-bug-report-information))
@@ -66,6 +68,9 @@ (define %options
(option '(#f "repo") #t #f
(lambda (opt name arg result)
(alist-cons 'repo arg result)))
+ (option '(#\f "file") #t #f
+ (lambda (opt name arg result)
+ (alist-cons 'file arg result)))
(option '(#\r "recursive") #f #f
(lambda (opt name arg result)
(alist-cons 'recursive #t result)))
@@ -100,9 +105,11 @@ (define (guix-import-opam . args)
`(define-public ,(string->symbol name)
,pkg))
(_ #f))
- (opam-recursive-import package-name #:repo repo))
+ (opam-recursive-import package-name #:repo repo
+ #:file (assoc-ref opts 'file)))
;; Single import
- (let ((sexp (opam->guix-package package-name #:repo repo)))
+ (let ((sexp (opam->guix-package package-name #:repo repo
+ #:file (assoc-ref opts 'file))))
(unless sexp
(leave (G_ "failed to download meta-data for package '~a'~%")
package-name))
--
2.34.0
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [bug#53828] [PATCH] guix: opam: Allow importing local files.
2022-02-06 21:41 [bug#53828] [PATCH] guix: opam: Allow importing local files Julien Lepiller
@ 2022-02-06 21:58 ` Maxime Devos
2022-02-07 8:27 ` Xinglu Chen
2022-06-12 6:31 ` [bug#53828] [PATCH v2] import: " Julien Lepiller
2 siblings, 0 replies; 7+ messages in thread
From: Maxime Devos @ 2022-02-06 21:58 UTC (permalink / raw)
To: Julien Lepiller, 53828
[-- Attachment #1: Type: text/plain, Size: 560 bytes --]
Julien Lepiller schreef op zo 06-02-2022 om 22:41 [+0100]:
> Now, if you have an OCaml package sources that provides an opam file
> (usually called opam, or with the .opam extension), you can import
> the
> package, recursively or not with something like:
>
> guix import opam foo -f foo.opam -r
>
> instead of fetching foo from the repositories, it will fetch the data
> directly from the given file. WDYT?
I don't know OCaml at all, but that seems potentially useful.
Can we have some tests for this in tests/opam.scm?
Greetings,
Maxime.
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread
* [bug#53828] [PATCH] guix: opam: Allow importing local files.
2022-02-06 21:41 [bug#53828] [PATCH] guix: opam: Allow importing local files Julien Lepiller
2022-02-06 21:58 ` Maxime Devos
@ 2022-02-07 8:27 ` Xinglu Chen
2022-02-07 8:46 ` Julien Lepiller
2022-06-12 6:31 ` [bug#53828] [PATCH v2] import: " Julien Lepiller
2 siblings, 1 reply; 7+ messages in thread
From: Xinglu Chen @ 2022-02-07 8:27 UTC (permalink / raw)
To: Julien Lepiller, 53828
[-- Attachment #1: Type: text/plain, Size: 10341 bytes --]
Julien schrieb am Sonntag der 06. Februar 2022 um 22:41 +01:
> Hi Guix!
>
> This patch lets you import opam packages from a local file. This is a
> feature that was requested a few times, and there it is :)
>
> Now, if you have an OCaml package sources that provides an opam file
> (usually called opam, or with the .opam extension), you can import the
> package, recursively or not with something like:
>
> guix import opam foo -f foo.opam -r
>
> instead of fetching foo from the repositories, it will fetch the data
> directly from the given file. WDYT?
Why just limit it to local files? What about accepting a tarball on the
internet or any VCS repository? This could generally be useful for any
importer.
Also, since we already know the name of the package, maybe specifying
the directory that contains the OCaml project would be enough?
guix import opam foo --type=local /path/to/directory
guix import opam foo --type=tarball https://example.org/foo.tar.gz
guix import opam foo --type=git https://git.example.org/foo
…
Or am I asking for too much? :-)
> From b027391357eef4d6fa1bb8741528cf9650aef568 Mon Sep 17 00:00:00 2001
> Message-Id: <b027391357eef4d6fa1bb8741528cf9650aef568.1644183523.git.julien@lepiller.eu>
> From: Julien Lepiller <julien@lepiller.eu>
> Date: Sun, 6 Feb 2022 22:35:14 +0100
> Subject: [PATCH] guix: opam: Allow importing local files.
It should be “import: opam: …”.
> * guix/scripts/import/opam.scm (guix-import-opam): Support `--file` flag.
> * guix/import/utils.scm (recursive-import): Add optional file argument.
> Pass it to the first iteration when available.
> * guix/import/opam.scm (opam-fetch): Fetch from file when available.
> (opam->guix-source): Factorize source out of opam->guix-package.
> (opam->guix-package): Add file argument. Ensure it does not fail when
> name, version and url are not available.
> ---
> guix/import/opam.scm | 101 +++++++++++++++++++----------------
> guix/import/utils.scm | 17 +++---
> guix/scripts/import/opam.scm | 11 +++-
> 3 files changed, 75 insertions(+), 54 deletions(-)
>
> diff --git a/guix/import/opam.scm b/guix/import/opam.scm
> index a6f6fe8c9f..05f79110f8 100644
> --- a/guix/import/opam.scm
> +++ b/guix/import/opam.scm
> @@ -310,8 +310,11 @@ (define (dependency-list->inputs lst)
> (map string->symbol
> (ocaml-names->guix-names lst)))
>
> -(define* (opam-fetch name #:optional (repositories-specs '("opam")))
> - (or (fold (lambda (repository others)
> +(define* (opam-fetch name #:optional (repositories-specs '("opam")) (file #f))
> + (or (and file (let ((metadata (get-metadata file)))
> + `(("metadata" . ,metadata)
> + ("version" . #f))))
> + (fold (lambda (repository others)
> (match (find-latest-version name repository)
> ((_ version file) `(("metadata" ,@(get-metadata file))
> ("version" . ,version)))
> @@ -320,17 +323,29 @@ (define* (opam-fetch name #:optional (repositories-specs '("opam")))
> (filter-map get-opam-repository repositories-specs))
> (warning (G_ "opam: package '~a' not found~%") name)))
>
> -(define* (opam->guix-package name #:key (repo 'opam) version)
> +(define (opam->guix-source url-dict)
> + (let ((source-url (and url-dict
> + (or (metadata-ref url-dict "src")
> + (metadata-ref url-dict "archive")))))
> + (if source-url
> + (call-with-temporary-output-file
> + (lambda (temp port)
> + (and (url-fetch source-url temp)
> + `(origin
> + (method url-fetch)
> + (uri ,source-url)
> + (sha256 (base32 ,(guix-hash-url temp)))))))
> + 'no-source-information)))
I would use ‘and-let*’ instead of ‘let’, and drop the ‘if’ form.
The ability to import from a local file seems to mostly be useful when
the package isn’t meant to be submitted to Guix, right? Wouldn’t it
make more sense to make the source a ‘local-file’ object instead of just
setting it to #f?
> +(define* (opam->guix-package name #:key (repo '("opam")) (file #f) version)
> "Import OPAM package NAME from REPOSITORY (a directory name) or, if
> -REPOSITORY is #f, from the official OPAM repository. Return a 'package' sexp
> +REPOSITORY is #f, from the official OPAM repository. When FILE is defined,
> +import the packaged defined in the opam FILE instead. Return a 'package' sexp
s/packaged/package/
> (and-let* ((with-opam (if (member "opam" repo) repo (cons "opam" repo)))
> - (opam-file (opam-fetch name with-opam))
> - (version (assoc-ref opam-file "version"))
> + (opam-file (opam-fetch name with-opam file))
> (opam-content (assoc-ref opam-file "metadata"))
> - (url-dict (metadata-ref opam-content "url"))
> - (source-url (or (metadata-ref url-dict "src")
> - (metadata-ref url-dict "archive")))
> + (source (opam->guix-source (metadata-ref opam-content "url")))
> (requirements (metadata-ref opam-content "depends"))
> (names (dependency-list->names requirements))
> (dependencies (filter-dependencies names))
> @@ -344,47 +359,41 @@ (define* (opam->guix-package name #:key (repo 'opam) version)
> (not (member name '("dune" "jbuilder"))))
> native-dependencies))))
> (let ((use-dune? (member "dune" names)))
> - (call-with-temporary-output-file
> - (lambda (temp port)
> - (and (url-fetch source-url temp)
> - (values
> - `(package
> - (name ,(ocaml-name->guix-name name))
> - (version ,version)
> - (source
> - (origin
> - (method url-fetch)
> - (uri ,source-url)
> - (sha256 (base32 ,(guix-hash-url temp)))))
> - (build-system ,(if use-dune?
> - 'dune-build-system
> - 'ocaml-build-system))
> - ,@(if (null? inputs)
> - '()
> - `((propagated-inputs (list ,@inputs))))
> - ,@(if (null? native-inputs)
> - '()
> - `((native-inputs (list ,@native-inputs))))
> - ,@(if (equal? name (guix-name->opam-name (ocaml-name->guix-name name)))
> - '()
> - `((properties
> - ,(list 'quasiquote `((upstream-name . ,name))))))
> - (home-page ,(metadata-ref opam-content "homepage"))
> - (synopsis ,(metadata-ref opam-content "synopsis"))
> - (description ,(beautify-description
> - (metadata-ref opam-content "description")))
> - (license ,(spdx-string->license
> - (metadata-ref opam-content "license"))))
> - (filter
> - (lambda (name)
> - (not (member name '("dune" "jbuilder"))))
> - dependencies))))))))
> + (values
> + `(package
> + (name ,(and name (ocaml-name->guix-name name)))
> + (version ,(assoc-ref opam-file "version"))
> + (source ,source)
> + (build-system ,(if use-dune?
> + 'dune-build-system
> + 'ocaml-build-system))
> + ,@(if (null? inputs)
> + '()
> + `((propagated-inputs (list ,@inputs))))
> + ,@(if (null? native-inputs)
> + '()
> + `((native-inputs (list ,@native-inputs))))
> + ,@(if (and name (equal? name (guix-name->opam-name (ocaml-name->guix-name name))))
> + '()
> + `((properties
> + ,(list 'quasiquote `((upstream-name . ,name))))))
> + (home-page ,(metadata-ref opam-content "homepage"))
> + (synopsis ,(metadata-ref opam-content "synopsis"))
> + (description ,(beautify-description
> + (metadata-ref opam-content "description")))
> + (license ,(spdx-string->license
> + (metadata-ref opam-content "license"))))
> + (filter
> + (lambda (name)
> + (not (member name '("dune" "jbuilder"))))
> + dependencies)))))
>
> -(define* (opam-recursive-import package-name #:key repo)
> +(define* (opam-recursive-import package-name #:key repo file)
> (recursive-import package-name
> #:repo->guix-package opam->guix-package
> #:guix-name ocaml-name->guix-name
> - #:repo repo))
> + #:repo repo
> + #:file file))
>
> (define (guix-name->opam-name name)
> (if (string-prefix? "ocaml-" name)
> diff --git a/guix/import/utils.scm b/guix/import/utils.scm
> index 1c3cfa3e0b..ab35b8a4fc 100644
> --- a/guix/import/utils.scm
> +++ b/guix/import/utils.scm
> @@ -470,7 +470,7 @@ (define (topological-sort nodes
> (set-insert (node-name head) visited))))))))
>
> (define* (recursive-import package-name
> - #:key repo->guix-package guix-name version repo
> + #:key repo->guix-package guix-name version repo file
> #:allow-other-keys)
> "Return a list of package expressions for PACKAGE-NAME and all its
> dependencies, sorted in topological order. For each package,
The docstrings should also be updated.
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 861 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread
* [bug#53828] [PATCH] guix: opam: Allow importing local files.
2022-02-07 8:27 ` Xinglu Chen
@ 2022-02-07 8:46 ` Julien Lepiller
2022-02-09 13:34 ` Xinglu Chen
0 siblings, 1 reply; 7+ messages in thread
From: Julien Lepiller @ 2022-02-07 8:46 UTC (permalink / raw)
To: Xinglu Chen, 53828
On February 7, 2022 9:27:39 AM GMT+01:00, Xinglu Chen <public@yoctocell.xyz> wrote:
>Julien schrieb am Sonntag der 06. Februar 2022 um 22:41 +01:
>
>> Hi Guix!
>>
>> This patch lets you import opam packages from a local file. This is a
>> feature that was requested a few times, and there it is :)
>>
>> Now, if you have an OCaml package sources that provides an opam file
>> (usually called opam, or with the .opam extension), you can import the
>> package, recursively or not with something like:
>>
>> guix import opam foo -f foo.opam -r
>>
>> instead of fetching foo from the repositories, it will fetch the data
>> directly from the given file. WDYT?
>
>Why just limit it to local files? What about accepting a tarball on the
>internet or any VCS repository? This could generally be useful for any
>importer.
That's a good idea, but I'm not sure how to recognize these cases.
>
>Also, since we already know the name of the package, maybe specifying
>the directory that contains the OCaml project would be enough?
Sure, that could be useful for your suggestion below to use the local directory as the source.
>
> guix import opam foo --type=local /path/to/directory
> guix import opam foo --type=tarball https://example.org/foo.tar.gz
> guix import opam foo --type=git https://git.example.org/foo
> …
>
>Or am I asking for too much? :-)
>
>> From b027391357eef4d6fa1bb8741528cf9650aef568 Mon Sep 17 00:00:00 2001
>> Message-Id: <b027391357eef4d6fa1bb8741528cf9650aef568.1644183523.git.julien@lepiller.eu>
>> From: Julien Lepiller <julien@lepiller.eu>
>> Date: Sun, 6 Feb 2022 22:35:14 +0100
>> Subject: [PATCH] guix: opam: Allow importing local files.
>
>It should be “import: opam: …”.
>
>> * guix/scripts/import/opam.scm (guix-import-opam): Support `--file` flag.
>> * guix/import/utils.scm (recursive-import): Add optional file argument.
>> Pass it to the first iteration when available.
>> * guix/import/opam.scm (opam-fetch): Fetch from file when available.
>> (opam->guix-source): Factorize source out of opam->guix-package.
>> (opam->guix-package): Add file argument. Ensure it does not fail when
>> name, version and url are not available.
>> ---
>> guix/import/opam.scm | 101 +++++++++++++++++++----------------
>> guix/import/utils.scm | 17 +++---
>> guix/scripts/import/opam.scm | 11 +++-
>> 3 files changed, 75 insertions(+), 54 deletions(-)
>>
>> diff --git a/guix/import/opam.scm b/guix/import/opam.scm
>> index a6f6fe8c9f..05f79110f8 100644
>> --- a/guix/import/opam.scm
>> +++ b/guix/import/opam.scm
>> @@ -310,8 +310,11 @@ (define (dependency-list->inputs lst)
>> (map string->symbol
>> (ocaml-names->guix-names lst)))
>>
>> -(define* (opam-fetch name #:optional (repositories-specs '("opam")))
>> - (or (fold (lambda (repository others)
>> +(define* (opam-fetch name #:optional (repositories-specs '("opam")) (file #f))
>> + (or (and file (let ((metadata (get-metadata file)))
>> + `(("metadata" . ,metadata)
>> + ("version" . #f))))
>> + (fold (lambda (repository others)
>> (match (find-latest-version name repository)
>> ((_ version file) `(("metadata" ,@(get-metadata file))
>> ("version" . ,version)))
>> @@ -320,17 +323,29 @@ (define* (opam-fetch name #:optional (repositories-specs '("opam")))
>> (filter-map get-opam-repository repositories-specs))
>> (warning (G_ "opam: package '~a' not found~%") name)))
>>
>> -(define* (opam->guix-package name #:key (repo 'opam) version)
>> +(define (opam->guix-source url-dict)
>> + (let ((source-url (and url-dict
>> + (or (metadata-ref url-dict "src")
>> + (metadata-ref url-dict "archive")))))
>> + (if source-url
>> + (call-with-temporary-output-file
>> + (lambda (temp port)
>> + (and (url-fetch source-url temp)
>> + `(origin
>> + (method url-fetch)
>> + (uri ,source-url)
>> + (sha256 (base32 ,(guix-hash-url temp)))))))
>> + 'no-source-information)))
>
>I would use ‘and-let*’ instead of ‘let’, and drop the ‘if’ form.
That would return #f and make the ocher and-let* in opam->guix-package fail.
>
>The ability to import from a local file seems to mostly be useful when
>the package isn’t meant to be submitted to Guix, right? Wouldn’t it
>make more sense to make the source a ‘local-file’ object instead of just
>setting it to #f?
Excellent idea.
>
>> +(define* (opam->guix-package name #:key (repo '("opam")) (file #f) version)
>> "Import OPAM package NAME from REPOSITORY (a directory name) or, if
>> -REPOSITORY is #f, from the official OPAM repository. Return a 'package' sexp
>> +REPOSITORY is #f, from the official OPAM repository. When FILE is defined,
>> +import the packaged defined in the opam FILE instead. Return a 'package' sexp
>
>s/packaged/package/
>
>> (and-let* ((with-opam (if (member "opam" repo) repo (cons "opam" repo)))
>> - (opam-file (opam-fetch name with-opam))
>> - (version (assoc-ref opam-file "version"))
>> + (opam-file (opam-fetch name with-opam file))
>> (opam-content (assoc-ref opam-file "metadata"))
>> - (url-dict (metadata-ref opam-content "url"))
>> - (source-url (or (metadata-ref url-dict "src")
>> - (metadata-ref url-dict "archive")))
>> + (source (opam->guix-source (metadata-ref opam-content "url")))
>> (requirements (metadata-ref opam-content "depends"))
>> (names (dependency-list->names requirements))
>> (dependencies (filter-dependencies names))
>> @@ -344,47 +359,41 @@ (define* (opam->guix-package name #:key (repo 'opam) version)
>> (not (member name '("dune" "jbuilder"))))
>> native-dependencies))))
>> (let ((use-dune? (member "dune" names)))
>> - (call-with-temporary-output-file
>> - (lambda (temp port)
>> - (and (url-fetch source-url temp)
>> - (values
>> - `(package
>> - (name ,(ocaml-name->guix-name name))
>> - (version ,version)
>> - (source
>> - (origin
>> - (method url-fetch)
>> - (uri ,source-url)
>> - (sha256 (base32 ,(guix-hash-url temp)))))
>> - (build-system ,(if use-dune?
>> - 'dune-build-system
>> - 'ocaml-build-system))
>> - ,@(if (null? inputs)
>> - '()
>> - `((propagated-inputs (list ,@inputs))))
>> - ,@(if (null? native-inputs)
>> - '()
>> - `((native-inputs (list ,@native-inputs))))
>> - ,@(if (equal? name (guix-name->opam-name (ocaml-name->guix-name name)))
>> - '()
>> - `((properties
>> - ,(list 'quasiquote `((upstream-name . ,name))))))
>> - (home-page ,(metadata-ref opam-content "homepage"))
>> - (synopsis ,(metadata-ref opam-content "synopsis"))
>> - (description ,(beautify-description
>> - (metadata-ref opam-content "description")))
>> - (license ,(spdx-string->license
>> - (metadata-ref opam-content "license"))))
>> - (filter
>> - (lambda (name)
>> - (not (member name '("dune" "jbuilder"))))
>> - dependencies))))))))
>> + (values
>> + `(package
>> + (name ,(and name (ocaml-name->guix-name name)))
>> + (version ,(assoc-ref opam-file "version"))
>> + (source ,source)
>> + (build-system ,(if use-dune?
>> + 'dune-build-system
>> + 'ocaml-build-system))
>> + ,@(if (null? inputs)
>> + '()
>> + `((propagated-inputs (list ,@inputs))))
>> + ,@(if (null? native-inputs)
>> + '()
>> + `((native-inputs (list ,@native-inputs))))
>> + ,@(if (and name (equal? name (guix-name->opam-name (ocaml-name->guix-name name))))
>> + '()
>> + `((properties
>> + ,(list 'quasiquote `((upstream-name . ,name))))))
>> + (home-page ,(metadata-ref opam-content "homepage"))
>> + (synopsis ,(metadata-ref opam-content "synopsis"))
>> + (description ,(beautify-description
>> + (metadata-ref opam-content "description")))
>> + (license ,(spdx-string->license
>> + (metadata-ref opam-content "license"))))
>> + (filter
>> + (lambda (name)
>> + (not (member name '("dune" "jbuilder"))))
>> + dependencies)))))
>>
>> -(define* (opam-recursive-import package-name #:key repo)
>> +(define* (opam-recursive-import package-name #:key repo file)
>> (recursive-import package-name
>> #:repo->guix-package opam->guix-package
>> #:guix-name ocaml-name->guix-name
>> - #:repo repo))
>> + #:repo repo
>> + #:file file))
>>
>> (define (guix-name->opam-name name)
>> (if (string-prefix? "ocaml-" name)
>> diff --git a/guix/import/utils.scm b/guix/import/utils.scm
>> index 1c3cfa3e0b..ab35b8a4fc 100644
>> --- a/guix/import/utils.scm
>> +++ b/guix/import/utils.scm
>> @@ -470,7 +470,7 @@ (define (topological-sort nodes
>> (set-insert (node-name head) visited))))))))
>>
>> (define* (recursive-import package-name
>> - #:key repo->guix-package guix-name version repo
>> + #:key repo->guix-package guix-name version repo file
>> #:allow-other-keys)
>> "Return a list of package expressions for PACKAGE-NAME and all its
>> dependencies, sorted in topological order. For each package,
>
>The docstrings should also be updated.
^ permalink raw reply [flat|nested] 7+ messages in thread
* [bug#53828] [PATCH] guix: opam: Allow importing local files.
2022-02-07 8:46 ` Julien Lepiller
@ 2022-02-09 13:34 ` Xinglu Chen
0 siblings, 0 replies; 7+ messages in thread
From: Xinglu Chen @ 2022-02-09 13:34 UTC (permalink / raw)
To: Julien Lepiller, 53828
[-- Attachment #1: Type: text/plain, Size: 1354 bytes --]
Hi,
Julien schrieb am Montag der 07. Februar 2022 um 09:46 +01:
>>> +(define (opam->guix-source url-dict)
>>> + (let ((source-url (and url-dict
>>> + (or (metadata-ref url-dict "src")
>>> + (metadata-ref url-dict "archive")))))
>>> + (if source-url
>>> + (call-with-temporary-output-file
>>> + (lambda (temp port)
>>> + (and (url-fetch source-url temp)
>>> + `(origin
>>> + (method url-fetch)
>>> + (uri ,source-url)
>>> + (sha256 (base32 ,(guix-hash-url temp)))))))
>>> + 'no-source-information)))
>>
>>I would use ‘and-let*’ instead of ‘let’, and drop the ‘if’ form.
>
> That would return #f and make the ocher and-let* in opam->guix-package
> fail.
Right, I missed the ‘and-let*’ in ‘opam->guix-package’. However, seeing
as the ‘source’ variable is only used in the body of the ‘and-let*’, I
think it would make sense to not even bind the result of
‘opam->guix-source’ to any variable, instead using it directly in the
‘package’ form.
`(package
…
(source ,source)
…)
becomes
`(package
…
(source ,(opam->guix-source (metadata-ref opam-content "url")))
…)
WDYT?
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 861 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread
* [bug#53828] [PATCH v2] import: opam: Allow importing local files.
2022-02-06 21:41 [bug#53828] [PATCH] guix: opam: Allow importing local files Julien Lepiller
2022-02-06 21:58 ` Maxime Devos
2022-02-07 8:27 ` Xinglu Chen
@ 2022-06-12 6:31 ` Julien Lepiller
2022-06-12 6:37 ` Julien Lepiller
2 siblings, 1 reply; 7+ messages in thread
From: Julien Lepiller @ 2022-06-12 6:31 UTC (permalink / raw)
To: 53828
* guix/scripts/import/opam.scm (guix-import-opam): Support `--scan-project` flag.
* guix/import/opam.scm (opam-scan-projects): New procedure.
---
guix/import/opam.scm | 32 ++++++++++++++++++++++++++++++--
guix/scripts/import/opam.scm | 11 +++++++++++
2 files changed, 41 insertions(+), 2 deletions(-)
diff --git a/guix/import/opam.scm b/guix/import/opam.scm
index b4b5a6eaad..3989dff58e 100644
--- a/guix/import/opam.scm
+++ b/guix/import/opam.scm
@@ -32,9 +32,11 @@ (define-module (guix import opam)
#:use-module (srfi srfi-2)
#:use-module ((srfi srfi-26) #:select (cut))
#:use-module ((web uri) #:select (string->uri uri->string))
- #:use-module ((guix build utils) #:select (dump-port find-files mkdir-p))
+ #:use-module ((guix build utils) #:select (dump-port find-files mkdir-p
+ delete-file-recursively))
#:use-module (guix build-system)
#:use-module (guix build-system ocaml)
+ #:use-module (guix git)
#:use-module (guix http-client)
#:use-module (guix ui)
#:use-module (guix packages)
@@ -48,7 +50,8 @@ (define-module (guix import opam)
spdx-string->license
url-fetch))
#:use-module ((guix licenses) #:prefix license:)
- #:export (opam->guix-package
+ #:export (opam-scan-projects
+ opam->guix-package
opam-recursive-import
%opam-updater
@@ -178,6 +181,31 @@ (define* (get-opam-repository #:optional (repo "opam"))
;; Prevent Guile 3 from inlining this procedure so we can mock it in tests.
(set! get-opam-repository get-opam-repository)
+(define (opam-scan-dir dir)
+ (let* ((opam-files (find-files dir "\\.opam$"))
+ (dir (opam-cache-directory dir))
+ (packages (string-append dir "/packages")))
+ (when (file-exists? dir)
+ (delete-file-recursively dir))
+ (mkdir-p packages)
+ (for-each
+ (lambda (package)
+ (let* ((name (basename package ".opam"))
+ (version (metadata-ref (get-metadata package) "version"))
+ (file (string-append packages "/" name "." version "/opam")))
+ (mkdir-p (dirname file))
+ (copy-file package file)))
+ opam-files)
+ dir))
+
+(define (opam-scan-project project)
+ (if (file-exists? project)
+ (opam-scan-dir project)
+ (opam-scan-dir (update-cached-checkout project))))
+
+(define (opam-scan-projects projects)
+ (map opam-scan-project projects))
+
(define (get-version-and-file path)
"Analyse a candidate path and return an list containing information for proper
version comparison as well as the source path for metadata."
diff --git a/guix/scripts/import/opam.scm b/guix/scripts/import/opam.scm
index 834ac34cb0..0b15a81541 100644
--- a/guix/scripts/import/opam.scm
+++ b/guix/scripts/import/opam.scm
@@ -50,6 +50,10 @@ (define (show-help)
--repo import packages from this opam repository (name, URL or local path)
can be used more than once"))
(display (G_ "
+ -p, --scan-project import packages from this OCaml project (URL of a
+ Git repository or local path). Can be used more
+ than once."))
+ (display (G_ "
-V, --version display version information and exit"))
(newline)
(show-bug-report-information))
@@ -69,6 +73,9 @@ (define %options
(option '(#\r "recursive") #f #f
(lambda (opt name arg result)
(alist-cons 'recursive #t result)))
+ (option '(#\p "scan-project") #t #f
+ (lambda (opt name arg result)
+ (alist-cons 'project arg result)))
%standard-import-options))
\f
@@ -86,6 +93,10 @@ (define (parse-options)
(repo (filter-map (match-lambda
(('repo . name) name)
(_ #f)) opts))
+ (projects (filter-map (match-lambda
+ (('project . name) name)
+ (_ #f)) opts))
+ (repo (append repo (opam-scan-projects projects)))
(args (filter-map (match-lambda
(('argument . value)
value)
--
2.35.1
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [bug#53828] [PATCH v2] import: opam: Allow importing local files.
2022-06-12 6:31 ` [bug#53828] [PATCH v2] import: " Julien Lepiller
@ 2022-06-12 6:37 ` Julien Lepiller
0 siblings, 0 replies; 7+ messages in thread
From: Julien Lepiller @ 2022-06-12 6:37 UTC (permalink / raw)
To: 53828
As you can see, I've reworked the patch a little. Instead of a special
#:file argument to the importer, I simply create an opam repository out
of a given "project" (if you have a better name...). A "project" is an
upstream source repository, either a local checkout or a remote git
repository, that contain .opam files.
The new version of the patch scans the project for opam packages and
adds them to a local repository. When importing recursively, they
become available, so countrary to the previous patch, you can use this
for recursive import even when multiple packages are not in the opam
repository but in the project (some projects split their code into
multiple opam packages, like coq for instance). You can specify
--scan-project multiple times, so you can import from dependent repos
too.
WDYT? :)
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2022-06-12 6:38 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-02-06 21:41 [bug#53828] [PATCH] guix: opam: Allow importing local files Julien Lepiller
2022-02-06 21:58 ` Maxime Devos
2022-02-07 8:27 ` Xinglu Chen
2022-02-07 8:46 ` Julien Lepiller
2022-02-09 13:34 ` Xinglu Chen
2022-06-12 6:31 ` [bug#53828] [PATCH v2] import: " Julien Lepiller
2022-06-12 6:37 ` Julien Lepiller
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/guix.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).