* [PATCH] import: pypi: add updater
@ 2015-11-03 21:44 Cyril Roelandt
2015-11-03 22:20 ` Ludovic Courtès
2015-11-05 19:58 ` Mathieu Lirzin
0 siblings, 2 replies; 7+ messages in thread
From: Cyril Roelandt @ 2015-11-03 21:44 UTC (permalink / raw)
To: guix-devel
* guix/import/pypi.scm (guix-package->pypi-name,
latest-release): New procedures.
(pypi-updater): New variable.
* guix/scripts/refresh.scm (%updaters): Add %PYPI-UPDATER.
---
guix/import/pypi.scm | 50 +++++++++++++++++++++++++++++++++++++++++++++++-
guix/scripts/refresh.scm | 4 +++-
2 files changed, 52 insertions(+), 2 deletions(-)
diff --git a/guix/import/pypi.scm b/guix/import/pypi.scm
index 647ef61..f1988a7 100644
--- a/guix/import/pypi.scm
+++ b/guix/import/pypi.scm
@@ -30,12 +30,16 @@
#:use-module (guix ui)
#:use-module (guix utils)
#:use-module (guix import utils)
+ #:use-module ((guix download) #:prefix download:)
#:use-module (guix import json)
#:use-module (guix packages)
+ #:use-module (guix upstream)
#:use-module (guix licenses)
#:use-module (guix build-system python)
+ #:use-module (gnu packages)
#:use-module (gnu packages python)
- #:export (pypi->guix-package))
+ #:export (pypi->guix-package
+ %pypi-updater))
(define (pypi-fetch name)
"Return an alist representation of the PyPI metadata for the package NAME,
@@ -60,6 +64,16 @@ package."
(snake-case name)
(string-append "python-" (snake-case name))))
+(define (guix-package->pypi-name package)
+ "Given a Python PACKAGE built from pypi.python.org, return the name of the
+package on PyPI."
+ (let ((source-url (and=> (package-source package) origin-uri)))
+ ;; The URL has the form:
+ ;; 'https://pypi.python.org/packages/source/' +
+ ;; first letter of the package name +
+ ;; '/' + package name + '/' + ...
+ (substring source-url 42 (string-rindex source-url #\/))))
+
(define (maybe-inputs package-inputs)
"Given a list of PACKAGE-INPUTS, tries to generate the 'inputs' field of a
package definition."
@@ -190,3 +204,37 @@ VERSION, SOURCE-URL, HOME-PAGE, SYNOPSIS, DESCRIPTION, and LICENSE."
(license (string->license (assoc-ref* package "info" "license"))))
(make-pypi-sexp name version release home-page synopsis
description license)))))
+
+(define (pypi-package? package)
+ "Return true if PACKAGE is a Python package from PyPI."
+
+ (define (pypi-url? url)
+ (string-prefix? "https://pypi.python.org/" url))
+
+ (let ((source-url (and=> (package-source package) origin-uri))
+ (fetch-method (and=> (package-source package) origin-method)))
+ (and (eq? fetch-method download:url-fetch)
+ (match source-url
+ ((? string?)
+ (pypi-url? source-url))
+ ((source-url ...)
+ (any pypi-url? source-url))))))
+
+(define (latest-release guix-package)
+ "Return an <upstream-source> for the latest release of GUIX-PACKAGE."
+ (let* ((pypi-name (guix-package->pypi-name
+ (specification->package guix-package)))
+ (metadata (pypi-fetch pypi-name))
+ (version (assoc-ref* metadata "info" "version"))
+ (url (assoc-ref (latest-source-release metadata) "url")))
+ (upstream-source
+ (package guix-package)
+ (version version)
+ (urls (list url)))))
+
+(define %pypi-updater
+ (upstream-updater
+ (name 'pypi)
+ (description "Updater for PyPI packages")
+ (pred pypi-package?)
+ (latest latest-release)))
diff --git a/guix/scripts/refresh.scm b/guix/scripts/refresh.scm
index 0df4121..3984a0b 100644
--- a/guix/scripts/refresh.scm
+++ b/guix/scripts/refresh.scm
@@ -30,6 +30,7 @@
#:use-module ((guix gnu-maintenance) #:select (%gnu-updater))
#:use-module (guix import elpa)
#:use-module (guix import cran)
+ #:use-module (guix import pypi)
#:use-module (guix gnupg)
#:use-module (gnu packages)
#:use-module ((gnu packages commencement) #:select (%final-inputs))
@@ -152,7 +153,8 @@ specified with `--select'.\n"))
;; List of "updaters" used by default. They are consulted in this order.
(list %gnu-updater
%elpa-updater
- %cran-updater))
+ %cran-updater
+ %pypi-updater))
(define (lookup-updater name)
"Return the updater called NAME."
--
2.6.2
^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [PATCH] import: pypi: add updater
2015-11-03 21:44 [PATCH] import: pypi: add updater Cyril Roelandt
@ 2015-11-03 22:20 ` Ludovic Courtès
2015-11-05 19:58 ` Mathieu Lirzin
1 sibling, 0 replies; 7+ messages in thread
From: Ludovic Courtès @ 2015-11-03 22:20 UTC (permalink / raw)
To: Cyril Roelandt; +Cc: guix-devel
Cyril Roelandt <tipecaml@gmail.com> skribis:
> * guix/import/pypi.scm (guix-package->pypi-name,
> latest-release): New procedures.
> (pypi-updater): New variable.
^
Missing ‘%’.
> * guix/scripts/refresh.scm (%updaters): Add %PYPI-UPDATER.
Could you add it to the list that appears under ‘--type’ in guix.texi?
OK to push with this change.
Thank you!
Ludo’.
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH] import: pypi: add updater
2015-11-03 21:44 [PATCH] import: pypi: add updater Cyril Roelandt
2015-11-03 22:20 ` Ludovic Courtès
@ 2015-11-05 19:58 ` Mathieu Lirzin
2015-11-05 22:25 ` Cyril Roelandt
2015-11-06 14:07 ` Ludovic Courtès
1 sibling, 2 replies; 7+ messages in thread
From: Mathieu Lirzin @ 2015-11-05 19:58 UTC (permalink / raw)
To: Cyril Roelandt; +Cc: guix-devel
[-- Attachment #1: Type: text/plain, Size: 651 bytes --]
Hi,
Cyril Roelandt <tipecaml@gmail.com> writes:
> * guix/import/pypi.scm (guix-package->pypi-name,
> latest-release): New procedures.
> (pypi-updater): New variable.
> * guix/scripts/refresh.scm (%updaters): Add %PYPI-UPDATER.
> ---
> guix/import/pypi.scm | 50 +++++++++++++++++++++++++++++++++++++++++++++++-
> guix/scripts/refresh.scm | 4 +++-
> 2 files changed, 52 insertions(+), 2 deletions(-)
>
IIUC the commit bab020d7ca50e4153cf24832d119389a37fa8f63 has made
Guile-JSON non optional.
I've succeed a clean build with the previous commit (gnu: Add
python-tempest-lib.) but then when I have done:
git reset bab020d
make -j4
[-- Attachment #2: make.log --]
[-- Type: text/plain, Size: 3169 bytes --]
mthl@godel:~/src/gnu/guix$ make -j4
make all-recursive
make[1]: Entering directory '/home/mthl/src/gnu/guix'
Making all in po/guix
make[2]: Entering directory '/home/mthl/src/gnu/guix/po/guix'
make[2]: Leaving directory '/home/mthl/src/gnu/guix/po/guix'
Making all in po/packages
make[2]: Entering directory '/home/mthl/src/gnu/guix/po/packages'
make[2]: Leaving directory '/home/mthl/src/gnu/guix/po/packages'
make[2]: Entering directory '/home/mthl/src/gnu/guix'
LANGUAGE= ./pre-inst-env \
/bin/bash /home/mthl/src/gnu/guix/build-aux/missing help2man --output="doc/guix.1" guix
LANGUAGE= ./pre-inst-env /bin/bash /home/mthl/src/gnu/guix/build-aux/missing help2man --output="doc/guix-refresh.1" "guix refresh"
GUILEC guix/scripts/refresh.go
MAKEINFO doc/guix.info
Backtrace:
In ice-9/psyntax.scm:
279: 19 [scan ((#(syntax-object let # ...) (#) (# #) ...)) () ...]
In ice-9/eval.scm:
411: 18 [eval # ()]
In ice-9/boot-9.scm:
2951: 17 [define-module* (guix scripts refresh) #:filename ...]
2926: 16 [resolve-imports (((guix ui)) ((guix hash)) ((guix scripts)) ...)]
2864: 15 [resolve-interface (guix import pypi) #:select ...]
2789: 14 [#<procedure b3c360 at ice-9/boot-9.scm:2777:4 (name #:optional autoload version #:key ensure)> # ...]
3065: 13 [try-module-autoload (guix import pypi) #f]
2401: 12 [save-module-excursion #<procedure 1a57bd0 at ice-9/boot-9.scm:3066:17 ()>]
3085: 11 [#<procedure 1a57bd0 at ice-9/boot-9.scm:3066:17 ()>]
In unknown file:
?: 10 [primitive-load-path "guix/import/pypi" ...]
In ice-9/eval.scm:
505: 9 [#<procedure a894a0 at ice-9/eval.scm:499:4 (exp)> (define-module # # ...)]
In ice-9/psyntax.scm:
1106: 8 [expand-top-sequence ((define-module # # # ...)) () ((top)) ...]
989: 7 [scan ((define-module (guix import pypi) #:use-module ...)) () ...]
279: 6 help2man: impossible de récupérer l'information « --help » de guix refresh
Try `--no-discard-stderr' if option outputs to stderr
Makefile:4590: recipe for target 'doc/guix-refresh.1' failed
make[2]: [doc/guix-refresh.1] Error 1 (ignorée)
[scan ((#(syntax-object let # ...) (#) (# #) ...)) () ...]
In ice-9/eval.scm:
411: 5 [eval # ()]
In ice-9/boot-9.scm:
2951: 4 [define-module* (guix import pypi) #:filename ...]
2926: 3 [resolve-imports ((#) (#) (#) (#) ...)]
2867: 2 [resolve-interface (json) #:select ...]
In unknown file:
?: 1 [scm-error misc-error #f "~A ~S" ("no code for module" (json)) #f]
In ice-9/boot-9.scm:
106: 0 [#<procedure e3d9c0 at ice-9/boot-9.scm:97:6 (thrown-k . args)> misc-error ...]
ice-9/boot-9.scm:106:20: In procedure #<procedure e3d9c0 at ice-9/boot-9.scm:97:6 (thrown-k . args)>:
ice-9/boot-9.scm:106:20: no code for module (json)
Makefile:4526: recipe for target 'guix/scripts/refresh.go' failed
make[2]: *** [guix/scripts/refresh.go] Error 1
make[2]: *** Attente des tâches non terminées....
make[2]: Leaving directory '/home/mthl/src/gnu/guix'
Makefile:3688: recipe for target 'all-recursive' failed
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory '/home/mthl/src/gnu/guix'
Makefile:2302: recipe for target 'all' failed
make: *** [all] Error 2
[-- Attachment #3: Type: text/plain, Size: 50 bytes --]
What should we do about this?
--
Mathieu Lirzin
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH] import: pypi: add updater
2015-11-05 19:58 ` Mathieu Lirzin
@ 2015-11-05 22:25 ` Cyril Roelandt
2015-11-06 14:07 ` Ludovic Courtès
1 sibling, 0 replies; 7+ messages in thread
From: Cyril Roelandt @ 2015-11-05 22:25 UTC (permalink / raw)
To: Mathieu Lirzin; +Cc: guix-devel
On 11/05/2015 08:58 PM, Mathieu Lirzin wrote:
>
> IIUC the commit bab020d7ca50e4153cf24832d119389a37fa8f63 has made
> Guile-JSON non optional.
>
That is weird, because the PyPI importer has always been using the
guile-json module. I thought it was already mandatory.
Cyril.
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH] import: pypi: add updater
2015-11-05 19:58 ` Mathieu Lirzin
2015-11-05 22:25 ` Cyril Roelandt
@ 2015-11-06 14:07 ` Ludovic Courtès
2015-11-08 2:24 ` Mathieu Lirzin
1 sibling, 1 reply; 7+ messages in thread
From: Ludovic Courtès @ 2015-11-06 14:07 UTC (permalink / raw)
To: Mathieu Lirzin; +Cc: guix-devel
[-- Attachment #1: Type: text/plain, Size: 643 bytes --]
Mathieu Lirzin <mthl@gnu.org> skribis:
> Cyril Roelandt <tipecaml@gmail.com> writes:
>
>> * guix/import/pypi.scm (guix-package->pypi-name,
>> latest-release): New procedures.
>> (pypi-updater): New variable.
>> * guix/scripts/refresh.scm (%updaters): Add %PYPI-UPDATER.
>> ---
>> guix/import/pypi.scm | 50 +++++++++++++++++++++++++++++++++++++++++++++++-
>> guix/scripts/refresh.scm | 4 +++-
>> 2 files changed, 52 insertions(+), 2 deletions(-)
>>
>
> IIUC the commit bab020d7ca50e4153cf24832d119389a37fa8f63 has made
> Guile-JSON non optional.
Yes, a mistake.
> What should we do about this?
Something like this should work:
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Type: text/x-patch, Size: 1841 bytes --]
diff --git a/guix/scripts/refresh.scm b/guix/scripts/refresh.scm
index 3984a0b..4e9d1c8 100644
--- a/guix/scripts/refresh.scm
+++ b/guix/scripts/refresh.scm
@@ -30,7 +30,6 @@
#:use-module ((guix gnu-maintenance) #:select (%gnu-updater))
#:use-module (guix import elpa)
#:use-module (guix import cran)
- #:use-module (guix import pypi)
#:use-module (guix gnupg)
#:use-module (gnu packages)
#:use-module ((gnu packages commencement) #:select (%final-inputs))
@@ -149,12 +148,35 @@ specified with `--select'.\n"))
;;; Updates.
;;;
+(define-syntax maybe-updater
+ (lambda (s)
+ (syntax-case s (=>)
+ ((_ ((module => updater) rest ...) (result ...))
+ (let ((met? (false-if-exception
+ (resolve-interface (syntax->datum #'module)))))
+ (if met?
+ #'(maybe-updater (rest ...)
+ (result ... (@ module updater)))
+ #'(maybe-updater (rest ...) (result ...)))))
+ ((_ (updater rest ...) (result ...))
+ #'(maybe-updater (rest ...) (result ... updater)))
+ ((_ () result)
+ #'result))))
+
+(define-syntax-rule (list-updaters updaters ...)
+ "Expand to '(list UPDATERS ...)' but only the subset of UPDATERS that are
+either unconditional, or have their requirement met.
+
+This is a way to discard at macro expansion time updaters that depend on
+unavailable optional dependencies such as Guile-JSON."
+ (maybe-updater (updaters ...) (list)))
+
(define %updaters
;; List of "updaters" used by default. They are consulted in this order.
- (list %gnu-updater
+ (list-updaters %gnu-updater
%elpa-updater
%cran-updater
- %pypi-updater))
+ ((guix import pypi) => %pypi-updater)))
(define (lookup-updater name)
"Return the updater called NAME."
[-- Attachment #3: Type: text/plain, Size: 141 bytes --]
Could you test in a JSON-less environment and report back?
Alternately, we could declare Guile-JSON to be mandatory.
TIA,
Ludo’.
^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [PATCH] import: pypi: add updater
2015-11-06 14:07 ` Ludovic Courtès
@ 2015-11-08 2:24 ` Mathieu Lirzin
2015-11-08 20:59 ` Ludovic Courtès
0 siblings, 1 reply; 7+ messages in thread
From: Mathieu Lirzin @ 2015-11-08 2:24 UTC (permalink / raw)
To: Ludovic Courtès; +Cc: guix-devel
ludo@gnu.org (Ludovic Courtès) writes:
> Mathieu Lirzin <mthl@gnu.org> skribis:
>> IIUC the commit bab020d7ca50e4153cf24832d119389a37fa8f63 has made
>> Guile-JSON non optional.
>
> Yes, a mistake.
>
>> What should we do about this?
>
> Something like this should work:
>
> diff --git a/guix/scripts/refresh.scm b/guix/scripts/refresh.scm
> index 3984a0b..4e9d1c8 100644
> --- a/guix/scripts/refresh.scm
> +++ b/guix/scripts/refresh.scm
[...]
> Could you test in a JSON-less environment and report back?
It compiles and ‘guix refresh’ seems to work fine. Thank you very much Ludo!
> Alternately, we could declare Guile-JSON to be mandatory.
Since it is not packaged in Debian IMO it is not a good idea to depend
on it.
--
Mathieu Lirzin
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH] import: pypi: add updater
2015-11-08 2:24 ` Mathieu Lirzin
@ 2015-11-08 20:59 ` Ludovic Courtès
0 siblings, 0 replies; 7+ messages in thread
From: Ludovic Courtès @ 2015-11-08 20:59 UTC (permalink / raw)
To: Mathieu Lirzin; +Cc: guix-devel
Mathieu Lirzin <mthl@gnu.org> skribis:
> ludo@gnu.org (Ludovic Courtès) writes:
>
>> Mathieu Lirzin <mthl@gnu.org> skribis:
>>> IIUC the commit bab020d7ca50e4153cf24832d119389a37fa8f63 has made
>>> Guile-JSON non optional.
>>
>> Yes, a mistake.
>>
>>> What should we do about this?
>>
>> Something like this should work:
>>
>> diff --git a/guix/scripts/refresh.scm b/guix/scripts/refresh.scm
>> index 3984a0b..4e9d1c8 100644
>> --- a/guix/scripts/refresh.scm
>> +++ b/guix/scripts/refresh.scm
> [...]
>> Could you test in a JSON-less environment and report back?
>
> It compiles and ‘guix refresh’ seems to work fine. Thank you very much Ludo!
Committed, thanks!
Ludo’.
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2015-11-08 20:59 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2015-11-03 21:44 [PATCH] import: pypi: add updater Cyril Roelandt
2015-11-03 22:20 ` Ludovic Courtès
2015-11-05 19:58 ` Mathieu Lirzin
2015-11-05 22:25 ` Cyril Roelandt
2015-11-06 14:07 ` Ludovic Courtès
2015-11-08 2:24 ` Mathieu Lirzin
2015-11-08 20:59 ` Ludovic Courtès
Code repositories for project(s) associated with this external index
https://git.savannah.gnu.org/cgit/guix.git
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.