From mboxrd@z Thu Jan 1 00:00:00 1970 From: Marius Bakke Subject: Re: [PATCH 1/4] gnu: Add python-avro. Date: Mon, 26 Sep 2016 20:56:18 +0100 Message-ID: <871t06o2ot.fsf@ike.i-did-not-set--mail-host-address--so-tickle-me> References: <20160925211728.7636-1-mbakke@fastmail.com> <57E8EB04.2040905@crazy-compilers.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:52470) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1boc0z-0005Vc-2Z for guix-devel@gnu.org; Mon, 26 Sep 2016 15:56:39 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1boc0t-0005fW-TC for guix-devel@gnu.org; Mon, 26 Sep 2016 15:56:36 -0400 Received: from out2-smtp.messagingengine.com ([66.111.4.26]:60739) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1boc0r-0005dr-Hd for guix-devel@gnu.org; Mon, 26 Sep 2016 15:56:31 -0400 In-Reply-To: <57E8EB04.2040905@crazy-compilers.com> List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: Hartmut Goebel , guix-devel@gnu.org --=-=-= Content-Type: text/plain Hartmut Goebel writes: > Am 25.09.2016 um 23:17 schrieb Marius Bakke: >> +;;; Avro uses a single source repository for all implementations. The individual >> +;;; released versions often have missing or incomplete test data, so we define >> +;;; the common source here for use in all avro packages. >> +(define (avro-version) "1.8.1") >> +(define (avro-source version) > > With "Avro uses a single source repository for all implementations " you > mean for both Python 2 and Python 3? Well, this is the common case :-) > Any other packages do have this and we never use a separate definition > for source or version. Please have a look at e.g. python-graphql-relay > (picked as example at random). This is a rather special case. I've updated the comment to make it more clear. The Avro repository contains one folder for each language implementation, and py2 and py3 are developed separately: https://www-eu.apache.org/dist/avro/avro-1.8.1/ The source in this case contains all those subfolders. I'm only building the two different Python implementations for now, but expect the other interfaces to be able to use the same source when packaged. There are release tarballs for the subdirectories, but at least the py3 version was missing tests, which was also the case on PyPi: https://pypi.python.org/pypi/avro https://pypi.python.org/pypi/avro-python3 > Am 25.09.2016 um 23:17 schrieb Marius Bakke: >> + `(#:configure-flags '("--single-version-externally-managed" "--root=/") > > Same here, please add a comment, > > BTW: What should --root=/ be goof for? AFAIK this is the default?! When using --single-version-externally-managed without --root, the install phase fails with: error: You must specify --record or --root when building system packages As to why it is needed, there is another side effect in python-build-system that is not mentioned in the bug report, but the same fix works. When not specified, "bin" inputs may end up in the "bin" output of the package! E.g. for pbcommand, $out/bin contains "jsonschema" from the python-jsonschema package, whereas sphinx-bootstrap-theme has "easy_install" in $out/bin and sphinx-argparse/bin contains various executables from python-sphinx. I've updated the patch series addressing these and your other comments. Thanks for the feedback! --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=0001-gnu-Add-python-avro.patch >From 9bb2dac10478efc420a97ac70892be0fe0356080 Mon Sep 17 00:00:00 2001 From: Marius Bakke Date: Sun, 25 Sep 2016 20:23:39 +0100 Subject: [PATCH 1/4] gnu: Add python-avro. * gnu/packages/serialization.scm (avro-version): New variable (private). (avro-source): New variable (private). (python-avro, python2-avro): New variables. --- gnu/packages/serialization.scm | 50 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 50 insertions(+) diff --git a/gnu/packages/serialization.scm b/gnu/packages/serialization.scm index 4a3278f..e80f5fe 100644 --- a/gnu/packages/serialization.scm +++ b/gnu/packages/serialization.scm @@ -25,6 +25,7 @@ #:use-module (guix download) #:use-module (guix build-system cmake) #:use-module (guix build-system gnu) + #:use-module (guix build-system python) #:use-module (gnu packages) #:use-module (gnu packages autotools) #:use-module (gnu packages boost) @@ -200,3 +201,52 @@ it a convenient format to store user input files.") "Cap'n Proto is a very fast data interchange format and capability-based RPC system. Think JSON, except binary. Or think Protocol Buffers, except faster.") (license license:expat))) + +;;; Avro uses a single source repository for all language implementations. The +;;; individual released versions some times have missing or incomplete test data, +;;; so we use the full source tarball for all Avro packages. +(define (avro-version) "1.8.1") +(define (avro-source version) + (origin + (method url-fetch) + (uri (string-append "mirror://apache/avro/avro-" version + "/avro-src-" version ".tar.gz")) + (sha256 (base32 "0bplj4qmh7d6p987qd6a13g59awrc5cbx25n5brcrwq8yjik21av")) + (modules '((guix build utils))) + (snippet + '(begin + ;; Drop bundled dependencies. + (delete-file-recursively "lang/py/lib/simplejson") + #t)))) + +(define-public python-avro + (package + (name "python-avro") + (version (avro-version)) + (source (avro-source version)) + (build-system python-build-system) + (arguments + `(#:phases + (modify-phases %standard-phases + (add-after 'unpack 'enter-source + (lambda _ (chdir "lang/py3") #t))))) + (propagated-inputs + `(("python-simplejson" ,python-simplejson))) + (home-page "https://avro.apache.org/") + (synopsis "Python implementation of the Avro data serialization system") + (description + "Avro is a data serialization system and RPC framework with rich data +structures. This package provides the Python interface.") + (license license:asl2.0) + (properties `((python2-variant . ,(delay python2-avro)))))) + +(define-public python2-avro + (let ((base (package-with-python2 (strip-python2-variant python-avro)))) + (package (inherit base) + (arguments + `(#:tests? #f ; TODO: Enable when Apache Ivy is packaged. + #:python ,python-2 ; Needed when overriding inherited args. + #:phases + (modify-phases %standard-phases + (add-after 'unpack 'enter-source + (lambda _ (chdir "lang/py") #t)))))))) -- 2.10.0 --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=0002-gnu-Add-python-sphinx-bootstrap-theme.patch >From f460c88f853911618e77f869f31f4465923dbdd8 Mon Sep 17 00:00:00 2001 From: Marius Bakke Date: Sun, 25 Sep 2016 20:40:52 +0100 Subject: [PATCH 2/4] gnu: Add python-sphinx-bootstrap-theme. * gnu/packages/python.scm (python-sphinx-bootstrap-theme, python2-sphinx-bootstrap-theme): New variables. --- gnu/packages/python.scm | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/gnu/packages/python.scm b/gnu/packages/python.scm index 688a5d4..c649bee 100644 --- a/gnu/packages/python.scm +++ b/gnu/packages/python.scm @@ -10939,3 +10939,33 @@ with an associated set of resolve methods that know how to fetch data.") provide extendible implementations of common aspects of a cloud so that you can focus on building massively scalable web applications.") (license license:expat))) + +(define-public python-sphinx-bootstrap-theme + (package + (name "python-sphinx-bootstrap-theme") + (version "0.4.12") + (source (origin + (method url-fetch) + (uri (pypi-uri "sphinx-bootstrap-theme" version)) + (sha256 + (base32 + "0wmm292rpfzxaib7zf2j6kdl1dl2xzx303hx8sx8qsdy0pkmrk65")))) + (build-system python-build-system) + (arguments + `(#:tests? #f ; No tests. + ;; Without this flag, various artifacts from the build inputs may end up + ;; in the final output. It also works around https://bugs.gnu.org/20765 . + #:configure-flags '("--single-version-externally-managed" "--root=/"))) + (home-page "https://ryan-roemer.github.io/sphinx-bootstrap-theme/") + (synopsis "Bootstrap theme for Sphinx") + (description "Sphinx theme that integrates the Bootstrap CSS / JavaScript +framework with various layout options, hierarchical menu navigation, and +mobile-friendly responsive design.") + (license license:expat) + (properties `((python2-variant . ,(delay python2-sphinx-bootstrap-theme)))))) + +(define-public python2-sphinx-bootstrap-theme + (package (inherit (package-with-python2 (strip-python2-variant + python-sphinx-bootstrap-theme))) + (native-inputs + `(("python2-setuptools" ,python2-setuptools))))) -- 2.10.0 --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=0003-gnu-Add-python-sphinx-argparse.patch >From e81dcaa9ad6ffd507ddbd19ab8d1583b626d3dc1 Mon Sep 17 00:00:00 2001 From: Marius Bakke Date: Sun, 25 Sep 2016 20:45:07 +0100 Subject: [PATCH 3/4] gnu: Add python-sphinx-argparse. * gnu/packages/python.scm (python-sphinx-argparse, python2-sphinx-argparse): New variables. --- gnu/packages/python.scm | 47 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 47 insertions(+) diff --git a/gnu/packages/python.scm b/gnu/packages/python.scm index c649bee..8ed0bc5 100644 --- a/gnu/packages/python.scm +++ b/gnu/packages/python.scm @@ -10969,3 +10969,50 @@ mobile-friendly responsive design.") python-sphinx-bootstrap-theme))) (native-inputs `(("python2-setuptools" ,python2-setuptools))))) + +(define-public python-sphinx-argparse + (package + (name "python-sphinx-argparse") + (version "0.1.15") + (source (origin + (method url-fetch) + (uri (pypi-uri "sphinx-argparse" version)) + (sha256 + (base32 + "14wdxq379xxnhw0kgf8z6jqdi0rd4k5y20jllyar9mxwwjblayvq")))) + (build-system python-build-system) + (arguments + `(;; Without this flag, various artifacts from the build inputs may end up + ;; in the final output. It also works around https://bugs.gnu.org/20765 . + #:configure-flags '("--single-version-externally-managed" "--root=/") + #:phases + (modify-phases %standard-phases + (add-after 'unpack 'adjust-tests + ;; Two tests compare the output of "py.test --help" and fail + ;; when it gets ".py.test-real" back, so we substitute it here. + (lambda _ + (substitute* "test/test_parser.py" + (("py.test") ".py.test-real")) + #t)) + (delete 'check) + (add-after 'install 'check + (lambda _ (zero? (system* "py.test" "-vv"))))))) + (native-inputs + `(("python-pytest" ,python-pytest))) + (propagated-inputs + `(("python-sphinx" ,python-sphinx) + ("python-docutils" ,python-docutils))) + (home-page "https://github.com/ribozz/sphinx-argparse") + (synopsis "Sphinx extension to document argparse commands") + (description "Sphinx extension that automatically documents @code{argparse} +commands and options.") + (license license:expat) + (properties `((python2-variant . ,(delay python2-sphinx-argparse)))))) + +(define-public python2-sphinx-argparse + (let ((base (package-with-python2 (strip-python2-variant + python-sphinx-argparse)))) + (package (inherit base) + (native-inputs + `(("python2-setuptools" ,python2-setuptools) + ,@(package-native-inputs base)))))) -- 2.10.0 --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=0004-gnu-Add-python2-pbcommand.patch >From 4d49796615cfbd8c34890835689eea74ddcfca60 Mon Sep 17 00:00:00 2001 From: Marius Bakke Date: Sun, 25 Sep 2016 21:11:58 +0100 Subject: [PATCH 4/4] gnu: Add python2-pbcommand. * gnu/packages/bioinformatics.scm (python2-pbcommand): New variable. --- gnu/packages/bioinformatics.scm | 52 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 52 insertions(+) diff --git a/gnu/packages/bioinformatics.scm b/gnu/packages/bioinformatics.scm index 1bf91a9..f5a5756 100644 --- a/gnu/packages/bioinformatics.scm +++ b/gnu/packages/bioinformatics.scm @@ -3373,6 +3373,58 @@ interrupted by stop codons. OrfM finds and prints these ORFs.") (home-page "https://github.com/wwood/OrfM") (license license:lgpl3+))) +(define-public python2-pbcommand + ;; Upstream does not tag git releases and PyPi is out of date. + ;; See https://github.com/PacificBiosciences/pbcommand/issues/116 + (let ((revision "1") + (commit "e83a3443a1cd42b4da0d210201d711ede789917f")) + (package + (name "python2-pbcommand") + (version (string-append "0.4.11-" revision "." (string-take commit 7))) + (source (origin + (method git-fetch) + (uri (git-reference + (url "https://github.com/PacificBiosciences/pbcommand.git") + (commit commit))) + (file-name (string-append name "-" version "-checkout")) + (sha256 + (base32 + "1kwg84f23l7ik65qy7cxa6g5nipc2y23mppigd4j3vlzam18v52h")))) + (build-system python-build-system) + (arguments + `(#:python ,python-2 + ;; Without this flag, various artifacts from the build inputs may end up + ;; in the final output. It also works around https://bugs.gnu.org/20765 . + #:configure-flags '("--single-version-externally-managed" "--root=/") + #:phases + (modify-phases %standard-phases + (add-after 'unpack 'substitute-/bin/bash + (lambda _ + ;; Fully qualify /bin/bash for running external commands. + (substitute* "pbcommand/engine/runner.py" + (("/bin/bash") (which "bash"))) + #t)) + (replace 'check + (lambda _ (zero? (system* "nosetests" "-v"))))))) + (native-inputs + `(("python-nose" ,python2-nose) + ("python-sphinx-bootstrap-theme" ,python2-sphinx-bootstrap-theme) + ("python-sphinx-argparse" ,python2-sphinx-argparse) + ("python-tox" ,python2-tox) + ("python-setuptools" ,python2-setuptools))) + (propagated-inputs + `(("python-functools32" ,python2-functools32) + ("python-jsonschema" ,python2-jsonschema) + ("python-numpy" ,python2-numpy "out") + ("python-avro" ,python2-avro) + ("python-requests" ,python2-requests) + ("python-iso8601" ,python2-iso8601))) + (home-page "https://github.com/PacificBiosciences/pbcommand") + (synopsis "PacBio common models and CLI tool contract interface") + (description "PacBio library for common utils, models, and tools +to interface with pbsmrtpipe workflow engine.") + (license license:bsd-3)))) + (define-public python2-pbcore (package (name "python2-pbcore") -- 2.10.0 --=-=-=--