* [PATCH 1/3] import: hackage: Silence download output. @ 2016-03-23 3:54 ericbavier 2016-03-23 3:54 ` [PATCH 2/3] import: hackage: Factorize url synthesis ericbavier ` (2 more replies) 0 siblings, 3 replies; 8+ messages in thread From: ericbavier @ 2016-03-23 3:54 UTC (permalink / raw) To: guix-devel; +Cc: Eric Bavier From: Eric Bavier <bavier@member.fsf.org> * guix/import/hackage.scm (hackage-fetch): Send error output from url-fetch to /dev/null. --- guix/import/hackage.scm | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/guix/import/hackage.scm b/guix/import/hackage.scm index 8725ffa..9401830 100644 --- a/guix/import/hackage.scm +++ b/guix/import/hackage.scm @@ -82,9 +82,14 @@ version." name "-" version "/" name ".cabal") (string-append "http://hackage.haskell.org/package/" name "/" name ".cabal")))) + ;; XXX: We want to silence the download progress report, which is + ;; especially annoying for 'guix refresh', but we have to use a file port. (call-with-temporary-output-file (lambda (temp port) - (and (url-fetch url temp) + (and (call-with-output-file "/dev/null" + (lambda (null) + (with-error-to-port null + (lambda () (url-fetch url temp))))) (call-with-input-file temp (compose read-cabal canonical-newline-port))))))) -- 2.7.3 ^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH 2/3] import: hackage: Factorize url synthesis. 2016-03-23 3:54 [PATCH 1/3] import: hackage: Silence download output ericbavier @ 2016-03-23 3:54 ` ericbavier 2016-03-28 16:08 ` Ludovic Courtès 2016-03-23 3:54 ` [PATCH 3/3] import: Add Hackage updater ericbavier 2016-03-28 16:07 ` [PATCH 1/3] import: hackage: Silence download output Ludovic Courtès 2 siblings, 1 reply; 8+ messages in thread From: ericbavier @ 2016-03-23 3:54 UTC (permalink / raw) To: guix-devel; +Cc: Eric Bavier From: Eric Bavier <bavier@member.fsf.org> * guix/import/hackage.scm (hackage-source-url, hackage-cabal-url): New procedures. (hackage-fetch, hackage-module->sexp): Use them. --- guix/import/hackage.scm | 27 +++++++++++++++++++-------- 1 file changed, 19 insertions(+), 8 deletions(-) diff --git a/guix/import/hackage.scm b/guix/import/hackage.scm index 9401830..3e9df63 100644 --- a/guix/import/hackage.scm +++ b/guix/import/hackage.scm @@ -1,5 +1,6 @@ ;;; GNU Guix --- Functional package management for GNU ;;; Copyright © 2015 Federico Beffa <beffa@fbengineering.ch> +;;; Copyright © 2016 Eric Bavier <bavier@member.fsf.org> ;;; ;;; This file is part of GNU Guix. ;;; @@ -65,6 +66,22 @@ (define package-name-prefix "ghc-") +(define (hackage-source-url name version) + "Given a Hackage package NAME and VERSION, return a url to the source +tarball." + (string-append "http://hackage.haskell.org/package/" name + "/" name "-" version ".tar.gz")) + +(define* (hackage-cabal-url name #:optional version) + "Given a Hackage package NAME and VERSION, return a url to the corresponding +.cabal file on Hackage. If VERSION is #f or missing, the url for the latest +version is returned." + (if version + (string-append "http://hackage.haskell.org/package/" + name "-" version "/" name ".cabal") + (string-append "http://hackage.haskell.org/package/" + name "/" name ".cabal"))) + (define (hackage-name->package-name name) "Given the NAME of a Cabal package, return the corresponding Guix name." (if (string-prefix? package-name-prefix name) @@ -76,12 +93,7 @@ the version part is omitted from the package name, then return the latest version." (let*-values (((name version) (package-name->name+version name-version)) - ((url) - (if version - (string-append "http://hackage.haskell.org/package/" - name "-" version "/" name ".cabal") - (string-append "http://hackage.haskell.org/package/" - name "/" name ".cabal")))) + ((url) (hackage-cabal-url name version))) ;; XXX: We want to silence the download progress report, which is ;; especially annoying for 'guix refresh', but we have to use a file port. (call-with-temporary-output-file @@ -159,8 +171,7 @@ representation of a Cabal file as produced by 'read-cabal'." (cabal-package-version cabal)) (define source-url - (string-append "http://hackage.haskell.org/package/" name - "/" name "-" version ".tar.gz")) + (hackage-source-url name version)) (define dependencies (let ((names -- 2.7.3 ^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [PATCH 2/3] import: hackage: Factorize url synthesis. 2016-03-23 3:54 ` [PATCH 2/3] import: hackage: Factorize url synthesis ericbavier @ 2016-03-28 16:08 ` Ludovic Courtès 0 siblings, 0 replies; 8+ messages in thread From: Ludovic Courtès @ 2016-03-28 16:08 UTC (permalink / raw) To: ericbavier; +Cc: guix-devel, Eric Bavier ericbavier@openmailbox.org skribis: > From: Eric Bavier <bavier@member.fsf.org> > > * guix/import/hackage.scm (hackage-source-url, hackage-cabal-url): New > procedures. > (hackage-fetch, hackage-module->sexp): Use them. OK! Ludo'. ^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH 3/3] import: Add Hackage updater. 2016-03-23 3:54 [PATCH 1/3] import: hackage: Silence download output ericbavier 2016-03-23 3:54 ` [PATCH 2/3] import: hackage: Factorize url synthesis ericbavier @ 2016-03-23 3:54 ` ericbavier 2016-03-28 16:12 ` Ludovic Courtès 2016-03-28 16:07 ` [PATCH 1/3] import: hackage: Silence download output Ludovic Courtès 2 siblings, 1 reply; 8+ messages in thread From: ericbavier @ 2016-03-23 3:54 UTC (permalink / raw) To: guix-devel; +Cc: Eric Bavier From: Eric Bavier <bavier@member.fsf.org> * guix/import/hackage.scm (guix-package->hackage-name, hackage-package?) (latest-release): New procedures. (%hackage-updater): New variable. * guix/scripts/refresh.scm (%updaters): Add it. * doc/guix.texi (Invoking guix refresh): Mention it. --- doc/guix.texi | 2 ++ guix/import/hackage.scm | 65 +++++++++++++++++++++++++++++++++++++++++++++--- guix/scripts/refresh.scm | 2 ++ 3 files changed, 66 insertions(+), 3 deletions(-) diff --git a/doc/guix.texi b/doc/guix.texi index 186b850..57ddf60 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -4764,6 +4764,8 @@ the updater for @uref{https://pypi.python.org, PyPI} packages. the updater for @uref{https://rubygems.org, RubyGems} packages. @item github the updater for @uref{https://github.com, GitHub} packages. +@item hackage +the updater for @uref{https://hackage.haskell.org, Hackage} packages. @end table For instance, the following command only checks for updates of Emacs diff --git a/guix/import/hackage.scm b/guix/import/hackage.scm index 3e9df63..c67ce91 100644 --- a/guix/import/hackage.scm +++ b/guix/import/hackage.scm @@ -19,10 +19,13 @@ (define-module (guix import hackage) #:use-module (ice-9 match) + #:use-module (ice-9 regex) #:use-module (srfi srfi-26) #:use-module (srfi srfi-11) #:use-module (srfi srfi-1) - #:use-module ((guix download) #:select (download-to-store)) + #:use-module (gnu packages) + #:use-module ((guix download) #:select (download-to-store url-fetch) + #:prefix download:) #:use-module ((guix utils) #:select (package-name->name+version canonical-newline-port)) #:use-module (guix import utils) @@ -30,8 +33,11 @@ #:use-module (guix store) #:use-module (guix hash) #:use-module (guix base32) + #:use-module (guix upstream) + #:use-module (guix packages) #:use-module ((guix utils) #:select (call-with-temporary-output-file)) - #:export (hackage->guix-package)) + #:export (hackage->guix-package + %hackage-updater)) (define ghc-standard-libraries ;; List of libraries distributed with ghc (7.10.2). We include GHC itself as @@ -88,6 +94,17 @@ version is returned." (string-downcase name) (string-append package-name-prefix (string-downcase name)))) +(define guix-package->hackage-name + (let ((uri-rx (make-regexp "https?://hackage.haskell.org/package/([^/]+)/.*")) + (name-rx (make-regexp "(.*)-[0-9\\.]+"))) + (lambda (package) + "Given a Guix package name, return the corresponding Hackage name." + (let* ((source-url (and=> (package-source package) origin-uri)) + (name (match:substring (regexp-exec uri-rx source-url) 1))) + (match (regexp-exec name-rx name) + (#f name) + (m (match:substring m 1))))))) + (define (hackage-fetch name-version) "Return the Cabal file for the package NAME-VERSION, or #f on failure. If the version part is omitted from the package name, then return the latest @@ -199,7 +216,7 @@ representation of a Cabal file as produced by 'read-cabal'." '())) (let ((tarball (with-store store - (download-to-store store source-url)))) + (download:download-to-store store source-url)))) `(package (name ,(hackage-name->package-name name)) (version ,version) @@ -241,4 +258,46 @@ respectively." include-test-dependencies?) (cut eval-cabal <> cabal-environment))))) +(define (hackage-package? package) + "Return #t if PACKAGE is a Haskell package from Hackage." + + (define haskell-url? + (let ((hackage-rx (make-regexp "https?://hackage.haskell.org"))) + (lambda (url) + (regexp-exec hackage-rx url)))) + + (let ((source-url (and=> (package-source package) origin-uri)) + (fetch-method (and=> (package-source package) origin-method))) + (and (eq? fetch-method download:url-fetch) + (match source-url + ((? string?) + (haskell-url? source-url)) + ((source-url ...) + (any haskell-url? source-url)))))) + +(define (latest-release guix-package) + "Return an <upstream-source> for the latest release of GUIX-PACKAGE." + (let* ((hackage-name (guix-package->hackage-name + (specification->package guix-package))) + (cabal-meta (hackage-fetch hackage-name))) + (match cabal-meta + (#f + (format (current-error-port) + "warning: failed to parse ~a~%" + (hackage-cabal-url hackage-name)) + #f) + ((_ *** ("version" (version))) + (let ((url (hackage-source-url hackage-name version))) + (upstream-source + (package guix-package) + (version version) + (urls (list url)))))))) + +(define %hackage-updater + (upstream-updater + (name 'hackage) + (description "Updater for Hackage packages") + (pred hackage-package?) + (latest latest-release))) + ;;; cabal.scm ends here diff --git a/guix/scripts/refresh.scm b/guix/scripts/refresh.scm index e541138..0efc190 100644 --- a/guix/scripts/refresh.scm +++ b/guix/scripts/refresh.scm @@ -35,6 +35,7 @@ #:select (%gnu-updater %gnome-updater %xorg-updater)) #:use-module (guix import elpa) #:use-module (guix import cran) + #:use-module (guix import hackage) #:use-module (guix gnupg) #:use-module (gnu packages) #:use-module ((gnu packages commencement) #:select (%final-inputs)) @@ -198,6 +199,7 @@ unavailable optional dependencies such as Guile-JSON." %elpa-updater %cran-updater %bioconductor-updater + %hackage-updater ((guix import pypi) => %pypi-updater) ((guix import gem) => %gem-updater) ((guix import github) => %github-updater))) -- 2.7.3 ^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [PATCH 3/3] import: Add Hackage updater. 2016-03-23 3:54 ` [PATCH 3/3] import: Add Hackage updater ericbavier @ 2016-03-28 16:12 ` Ludovic Courtès 2016-03-30 1:53 ` Eric Bavier 0 siblings, 1 reply; 8+ messages in thread From: Ludovic Courtès @ 2016-03-28 16:12 UTC (permalink / raw) To: ericbavier; +Cc: guix-devel, Eric Bavier ericbavier@openmailbox.org skribis: > From: Eric Bavier <bavier@member.fsf.org> > > * guix/import/hackage.scm (guix-package->hackage-name, hackage-package?) > (latest-release): New procedures. > (%hackage-updater): New variable. > * guix/scripts/refresh.scm (%updaters): Add it. > * doc/guix.texi (Invoking guix refresh): Mention it. [...] > +(define guix-package->hackage-name > + (let ((uri-rx (make-regexp "https?://hackage.haskell.org/package/([^/]+)/.*")) > + (name-rx (make-regexp "(.*)-[0-9\\.]+"))) > + (lambda (package) > + "Given a Guix package name, return the corresponding Hackage name." > + (let* ((source-url (and=> (package-source package) origin-uri)) > + (name (match:substring (regexp-exec uri-rx source-url) 1))) > + (match (regexp-exec name-rx name) > + (#f name) > + (m (match:substring m 1))))))) It might be useful to honor a ‘cabal-package-name’ property in cases where guessing doesn’t work. That can always be added later, though. [...] > %elpa-updater > %cran-updater > %bioconductor-updater > + %hackage-updater Make it: ((guix import hackage) => %hackage-importer) to be on the safe side. If you confirm that ‘guix refresh -t hackage’ doesn’t crash, go for it! Thank you! Ludo’. ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH 3/3] import: Add Hackage updater. 2016-03-28 16:12 ` Ludovic Courtès @ 2016-03-30 1:53 ` Eric Bavier 2016-03-30 21:16 ` Ludovic Courtès 0 siblings, 1 reply; 8+ messages in thread From: Eric Bavier @ 2016-03-30 1:53 UTC (permalink / raw) To: Ludovic Courtès; +Cc: guix-devel On Mon, 28 Mar 2016 18:12:05 +0200 ludo@gnu.org (Ludovic Courtès) wrote: > ericbavier@openmailbox.org skribis: > > > From: Eric Bavier <bavier@member.fsf.org> > > > > * guix/import/hackage.scm (guix-package->hackage-name, hackage-package?) > > (latest-release): New procedures. > > (%hackage-updater): New variable. > > * guix/scripts/refresh.scm (%updaters): Add it. > > * doc/guix.texi (Invoking guix refresh): Mention it. > > [...] > > > +(define guix-package->hackage-name > > + (let ((uri-rx (make-regexp "https?://hackage.haskell.org/package/([^/]+)/.*")) > > + (name-rx (make-regexp "(.*)-[0-9\\.]+"))) > > + (lambda (package) > > + "Given a Guix package name, return the corresponding Hackage name." > > + (let* ((source-url (and=> (package-source package) origin-uri)) > > + (name (match:substring (regexp-exec uri-rx source-url) 1))) > > + (match (regexp-exec name-rx name) > > + (#f name) > > + (m (match:substring m 1))))))) > > It might be useful to honor a ‘cabal-package-name’ property in cases > where guessing doesn’t work. That can always be added later, though. Sure. Let's save that for later. This scheme works for all packages that we have so far, and seems like it should work for most others on Hackage that I've seen. > > [...] > > > %elpa-updater > > %cran-updater > > %bioconductor-updater > > + %hackage-updater > > Make it: > > ((guix import hackage) => %hackage-importer) > > to be on the safe side. I considered that at first, but it seems like that syntax is for updaters that rely on guile-json (or other modules which might be missing). I'd prefer not to unnecessarily confuse the situation. > If you confirm that ‘guix refresh -t hackage’ doesn’t crash, go for it! I have done so, and it seems to work quite well. There are a few packages whose .cabal file fail to parse, so I included the warning output to incite some future bug-fixing. `~Eric ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH 3/3] import: Add Hackage updater. 2016-03-30 1:53 ` Eric Bavier @ 2016-03-30 21:16 ` Ludovic Courtès 0 siblings, 0 replies; 8+ messages in thread From: Ludovic Courtès @ 2016-03-30 21:16 UTC (permalink / raw) To: Eric Bavier; +Cc: guix-devel Eric Bavier <ericbavier@openmailbox.org> skribis: > On Mon, 28 Mar 2016 18:12:05 +0200 > ludo@gnu.org (Ludovic Courtès) wrote: [...] >> It might be useful to honor a ‘cabal-package-name’ property in cases >> where guessing doesn’t work. That can always be added later, though. > > Sure. Let's save that for later. This scheme works for all packages > that we have so far, and seems like it should work for most others on > Hackage that I've seen. Sure. >> >> [...] >> >> > %elpa-updater >> > %cran-updater >> > %bioconductor-updater >> > + %hackage-updater >> >> Make it: >> >> ((guix import hackage) => %hackage-importer) >> >> to be on the safe side. > > I considered that at first, but it seems like that syntax is for > updaters that rely on guile-json (or other modules which might be > missing). I'd prefer not to unnecessarily confuse the situation. I thought (guix import hackage) would depend on guile-json via (guix import utils), but apparently no, so you’re right. >> If you confirm that ‘guix refresh -t hackage’ doesn’t crash, go for it! > > I have done so, and it seems to work quite well. There are a few > packages whose .cabal file fail to parse, so I included the > warning output to incite some future bug-fixing. Awesome! Thanks, Ludo’. ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH 1/3] import: hackage: Silence download output. 2016-03-23 3:54 [PATCH 1/3] import: hackage: Silence download output ericbavier 2016-03-23 3:54 ` [PATCH 2/3] import: hackage: Factorize url synthesis ericbavier 2016-03-23 3:54 ` [PATCH 3/3] import: Add Hackage updater ericbavier @ 2016-03-28 16:07 ` Ludovic Courtès 2 siblings, 0 replies; 8+ messages in thread From: Ludovic Courtès @ 2016-03-28 16:07 UTC (permalink / raw) To: ericbavier; +Cc: guix-devel, Eric Bavier ericbavier@openmailbox.org skribis: > From: Eric Bavier <bavier@member.fsf.org> > > * guix/import/hackage.scm (hackage-fetch): Send error output from > url-fetch to /dev/null. [...] > + ;; XXX: We want to silence the download progress report, which is > + ;; especially annoying for 'guix refresh', but we have to use a file port. > (call-with-temporary-output-file > (lambda (temp port) > - (and (url-fetch url temp) > + (and (call-with-output-file "/dev/null" > + (lambda (null) > + (with-error-to-port null > + (lambda () (url-fetch url temp))))) > (call-with-input-file temp > (compose read-cabal canonical-newline-port))))))) Would it work to directly use: (let* ((port (http-fetch url)) ;from (guix http-client) (result (read-cabal (canonical-newline-port port)))) (close-port port) result) This avoids the creation of a temporary file. TIA, Ludo’. ^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2016-03-30 21:16 UTC | newest] Thread overview: 8+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2016-03-23 3:54 [PATCH 1/3] import: hackage: Silence download output ericbavier 2016-03-23 3:54 ` [PATCH 2/3] import: hackage: Factorize url synthesis ericbavier 2016-03-28 16:08 ` Ludovic Courtès 2016-03-23 3:54 ` [PATCH 3/3] import: Add Hackage updater ericbavier 2016-03-28 16:12 ` Ludovic Courtès 2016-03-30 1:53 ` Eric Bavier 2016-03-30 21:16 ` Ludovic Courtès 2016-03-28 16:07 ` [PATCH 1/3] import: hackage: Silence download output Ludovic Courtès
Code repositories for project(s) associated with this public inbox https://git.savannah.gnu.org/cgit/guix.git This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).