From mboxrd@z Thu Jan 1 00:00:00 1970 From: Marius Bakke Subject: Re: [PATCH 2/3] gnu: Add python-pyxb. Date: Fri, 23 Sep 2016 16:56:03 +0100 Message-ID: <8737kqzk30.fsf@ike.i-did-not-set--mail-host-address--so-tickle-me> References: <20160917101047.4597-1-mbakke@fastmail.com> <20160917101047.4597-3-mbakke@fastmail.com> <3bd16b58-b3d1-d47c-2433-c3a721681463@uq.edu.au> <39d81c47-a96d-1f1c-ad1d-a80e7b7f109d@uq.edu.au> <1a9d61d6-0ee2-1161-25b4-9ffd32396039@uq.edu.au> <87h999ymzk.fsf@ike.i-did-not-set--mail-host-address--so-tickle-me> <679cc096-9eed-152f-0a01-f4a1d85c422e@uq.edu.au> <87eg4cyni4.fsf@ike.i-did-not-set--mail-host-address--so-tickle-me> <03badac9-3d2d-25ad-b0ed-3695d8a70bc7@uq.edu.au> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:52061) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bnSpq-0008TV-7O for guix-devel@gnu.org; Fri, 23 Sep 2016 11:56:28 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bnSpl-0007e2-7n for guix-devel@gnu.org; Fri, 23 Sep 2016 11:56:21 -0400 Received: from out2-smtp.messagingengine.com ([66.111.4.26]:36986) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bnSpi-0007Zz-O1 for guix-devel@gnu.org; Fri, 23 Sep 2016 11:56:17 -0400 In-Reply-To: <03badac9-3d2d-25ad-b0ed-3695d8a70bc7@uq.edu.au> List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: Ben Woodcroft , "guix-devel@gnu.org" --=-=-= Content-Type: text/plain Ben Woodcroft writes: >> >>> Also, I notice that pysam bundles htslib, bcftools and samtools C code. >>> Hopefully it should be straightforward enough to remove htslib as there >>> are install instructions, I'm not sure about the other two. This >>> shouldn't block the patch here, but would you mind taking a look? >>> http://pysam.readthedocs.io/en/latest/installation.html#installation >> I had a go at this, and also enabled tests since I was reading the build >> system anyway. Samtools and bcftools does not seem possible to un-bundle >> at this time, but htslib was straightforward. > OK. I don't think it needs to be propagated though, right? Also, would > you mind separating the change to modify-phases syntax and unbundling of > htslib into two patches please? Other than that this whole series LGTM. Modify-phases is now a separate patch. Htslib is propagated because it is included by the installed samtools and bcftools headers; at least one of pysams dependencies failed to build when it was a regular input. I added some comments and other cosmetic changes. Updated patch below. Cheers! --=-=-= Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename=pysam-and-pbcore-update.patch Content-Transfer-Encoding: quoted-printable >From eaa38fb4174997f6282ff7d278a6aa4153b75bcf Mon Sep 17 00:00:00 2001 From: Marius Bakke Date: Fri, 16 Sep 2016 18:05:15 +0100 Subject: [PATCH 1/7] gnu: python-pysam: Update to 0.9.1.4. * gnu/packages/bioinformatics.scm (python-pysam, python2-pysam): Update to 0.9.1.4. --- gnu/packages/bioinformatics.scm | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/gnu/packages/bioinformatics.scm b/gnu/packages/bioinformatics.= scm index 1bf91a9..6e83461 100644 --- a/gnu/packages/bioinformatics.scm +++ b/gnu/packages/bioinformatics.scm @@ -1290,13 +1290,13 @@ multiple sequence alignments.") (define-public python-pysam (package (name "python-pysam") - (version "0.8.4") + (version "0.9.1.4") (source (origin (method url-fetch) (uri (pypi-uri "pysam" version)) (sha256 (base32 - "1slx5mb94mzm5qzk52q270sab0sar95j67w1g1k452nz3s9j7krh")))) + "1i1djacqbr88y7w18b4aa78zxnsyr4sz7yqdq2spi7gs0y6pzvjn")))) (build-system python-build-system) (arguments `(#:tests? #f ; tests are excluded in the manifest --=20 2.10.0 >From 5431b05fe66398f5c44a1f0a81e1e90663550ac1 Mon Sep 17 00:00:00 2001 From: Marius Bakke Date: Sat, 17 Sep 2016 10:33:28 +0100 Subject: [PATCH 2/7] gnu: Add python-pyxb. * gnu/packages/xml.scm (python-pyxb, python2-pyxb): New variables. --- gnu/packages/xml.scm | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/gnu/packages/xml.scm b/gnu/packages/xml.scm index 7befad5..cd3426b 100644 --- a/gnu/packages/xml.scm +++ b/gnu/packages/xml.scm @@ -13,6 +13,7 @@ ;;; Copyright =C2=A9 2016 Jan Nieuwenhuizen ;;; Copyright =C2=A9 2016 ng0 ;;; Copyright =C2=A9 2016 Tobias Geerinckx-Rice +;;; Copyright =C2=A9 2016 Marius Bakke ;;; ;;; This file is part of GNU Guix. ;;; @@ -729,6 +730,30 @@ different Unicode encodings which happen automatically= during parsing/saving.") (license license:expat))) =20 +(define-public python-pyxb + (package + (name "python-pyxb") + (version "1.2.5") + (source (origin + (method url-fetch) + (uri (pypi-uri "PyXB" version)) + (sha256 + (base32 + "0rzzwibfqa28gxgcxx4cybx1qcg0g6fand06ykj3gz7z5kp653sf")))) + (build-system python-build-system) + (home-page "http://pyxb.sourceforge.net/") + (synopsis "Python XML Schema Bindings") + (description + "PyXB (\"pixbee\") is a pure Python package that generates Python sou= rce +code for classes that correspond to data structures defined by XMLSchema.") + (license (list license:asl2.0 ; Most files. + license:expat ; pyxb/utils/six.py + license:gpl2 ; bundled jquery in doc is dual MIT/G= PL2 + license:psfl)))) ; pyxb/utils/activestate.py + +(define-public python2-pyxb + (package-with-python2 python-pyxb)) + (define-public xmlto (package (name "xmlto") --=20 2.10.0 >From 923fc43d8c665d1a2795ea2c221003e54544a06b Mon Sep 17 00:00:00 2001 From: Marius Bakke Date: Sat, 17 Sep 2016 10:42:56 +0100 Subject: [PATCH 3/7] gnu: python2-pbcore: Update to 1.2.10. * gnu/packages/bioinformatics.scm (python2-pbcore): Update to 1.2.10. [propagated-inputs]: New field. Add python2-pyxb. --- gnu/packages/bioinformatics.scm | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/gnu/packages/bioinformatics.scm b/gnu/packages/bioinformatics.= scm index 6e83461..a935d5a 100644 --- a/gnu/packages/bioinformatics.scm +++ b/gnu/packages/bioinformatics.scm @@ -3376,13 +3376,13 @@ interrupted by stop codons. OrfM finds and prints = these ORFs.") (define-public python2-pbcore (package (name "python2-pbcore") - (version "1.2.8") + (version "1.2.10") (source (origin (method url-fetch) (uri (pypi-uri "pbcore" version)) (sha256 (base32 - "02pfn5raa3zf739672bg0dkx7z3j2c4nx7vmpfjqy5b12jrqpymk")))) + "1kjmv891d6qbpp4shhhvkl02ff4q5xlpnls2513sm2cjcrs52f1i")))) (build-system python-build-system) (arguments `(#:python ,python-2)) ; pbcore requires Python 2.7 (inputs @@ -3395,6 +3395,8 @@ interrupted by stop codons. OrfM finds and prints th= ese ORFs.") ("python-nose" ,python2-nose) ("python-setuptools" ,python2-setuptools) ("python-sphinx" ,python2-sphinx))) + (propagated-inputs + `(("python-pyxb" ,python2-pyxb))) (home-page "http://pacificbiosciences.github.io/pbcore/") (synopsis "Library for reading and writing PacBio data files") (description --=20 2.10.0 >From 7748677c13242b7edb6ddae1efed7f0699e487bf Mon Sep 17 00:00:00 2001 From: Marius Bakke Date: Thu, 22 Sep 2016 13:47:53 +0100 Subject: [PATCH 4/7] gnu: Add bcftools. * gnu/packages/bioinformatics.scm (bcftools): New variable. --- gnu/packages/bioinformatics.scm | 58 +++++++++++++++++++++++++++++++++++++= ++++ 1 file changed, 58 insertions(+) diff --git a/gnu/packages/bioinformatics.scm b/gnu/packages/bioinformatics.= scm index a935d5a..679502e 100644 --- a/gnu/packages/bioinformatics.scm +++ b/gnu/packages/bioinformatics.scm @@ -265,6 +265,64 @@ instance, it implements several methods to assess cont= ig-wise read coverage.") BAM files.") (license license:expat))) =20 +(define-public bcftools + (package + (name "bcftools") + (version "1.3.1") + (source (origin + (method url-fetch) + (uri (string-append + "https://github.com/samtools/bcftools/releases/downloa= d/" + version "/bcftools-" version ".tar.bz2")) + (sha256 + (base32 + "095ry68vmz9q5s1scjsa698dhgyvgw5aicz24c19iwfbai07mhqj")) + (modules '((guix build utils))) + (snippet + ;; Delete bundled htslib. + '(delete-file-recursively "htslib-1.3.1")))) + (build-system gnu-build-system) + (arguments + `(#:test-target "test" + #:make-flags + (list + "USE_GPL=3D1" + (string-append "prefix=3D" (assoc-ref %outputs "out")) + (string-append "HTSDIR=3D" (assoc-ref %build-inputs "htslib") "/in= clude") + (string-append "HTSLIB=3D" (assoc-ref %build-inputs "htslib") "/li= b/libhts.a") + (string-append "BGZIP=3D" (assoc-ref %build-inputs "htslib") "/bin= /bgzip") + (string-append "TABIX=3D" (assoc-ref %build-inputs "htslib") "/bin= /tabix")) + #:phases + (modify-phases %standard-phases + (add-after 'unpack 'patch-Makefile + (lambda _ + (substitute* "Makefile" + ;; Do not attempt to build htslib. + (("^include \\$\\(HTSDIR\\)/htslib\\.mk") "") + ;; Link against GSL cblas. + (("-lcblas") "-lgslcblas")) + #t)) + (delete 'configure) + (add-before 'check 'patch-tests + (lambda _ + (substitute* "test/test.pl" + (("/bin/bash") (which "bash"))) + #t))))) + (native-inputs + `(("htslib" ,htslib) + ("perl" ,perl))) + (inputs + `(("gsl" ,gsl) + ("zlib" ,zlib))) + (home-page "https://samtools.github.io/bcftools/") + (synopsis "Utilities for variant calling and manipulating VCFs and BCF= s") + (description + "BCFtools is a set of utilities that manipulate variant calls in the +Variant Call Format (VCF) and its binary counterpart BCF. All commands wo= rk +transparently with both VCFs and BCFs, both uncompressed and BGZF-compress= ed.") + ;; The sources are dual MIT/GPL, but becomes GPL-only when USE_GPL=3D1. + (license (list license:gpl3+ license:expat)))) + (define-public bedops (package (name "bedops") --=20 2.10.0 >From d4fd6f8482a76ccfe4d33f80070445d284006166 Mon Sep 17 00:00:00 2001 From: Marius Bakke Date: Fri, 23 Sep 2016 06:40:40 +0100 Subject: [PATCH 5/7] gnu: python-pysam: Use 'modify-phases'. * gnu/packages/bioinformatics.scm (python-pysam)[arguments]: Use 'modify-phases'. --- gnu/packages/bioinformatics.scm | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/gnu/packages/bioinformatics.scm b/gnu/packages/bioinformatics.= scm index 679502e..518dfea 100644 --- a/gnu/packages/bioinformatics.scm +++ b/gnu/packages/bioinformatics.scm @@ -1359,12 +1359,12 @@ multiple sequence alignments.") (arguments `(#:tests? #f ; tests are excluded in the manifest #:phases - (alist-cons-before - 'build 'set-flags - (lambda _ - (setenv "LDFLAGS" "-lncurses") - (setenv "CFLAGS" "-D_CURSES_LIB=3D1")) - %standard-phases))) + (modify-phases %standard-phases + (add-before 'build 'set-flags + (lambda _ + (setenv "LDFLAGS" "-lncurses") + (setenv "CFLAGS" "-D_CURSES_LIB=3D1") + #t))))) (inputs `(("ncurses" ,ncurses) ("zlib" ,zlib))) --=20 2.10.0 >From e4d3befdc33a77f6b3d1d241befd7077c0be7e70 Mon Sep 17 00:00:00 2001 From: Marius Bakke Date: Thu, 22 Sep 2016 14:08:05 +0100 Subject: [PATCH 6/7] gnu: python-pysam: Delete bundled htslib. * gnu/packages/bioinformatics.scm (python-pysam)[source]: Add snippet to delete htslib. [arguments]: Add htslib flags in 'set-flags phase. [propagated-inputs]: New field. Add htslib. --- gnu/packages/bioinformatics.scm | 15 +++++++++++++-- 1 file changed, 13 insertions(+), 2 deletions(-) diff --git a/gnu/packages/bioinformatics.scm b/gnu/packages/bioinformatics.= scm index 518dfea..a712a01 100644 --- a/gnu/packages/bioinformatics.scm +++ b/gnu/packages/bioinformatics.scm @@ -1354,17 +1354,28 @@ multiple sequence alignments.") (uri (pypi-uri "pysam" version)) (sha256 (base32 - "1i1djacqbr88y7w18b4aa78zxnsyr4sz7yqdq2spi7gs0y6pzvjn")))) + "1i1djacqbr88y7w18b4aa78zxnsyr4sz7yqdq2spi7gs0y6pzvjn")) + (modules '((guix build utils))) + (snippet + ;; Drop bundled htslib. TODO: Also remove samtools and bcft= ools. + '(delete-file-recursively "htslib")))) (build-system python-build-system) (arguments `(#:tests? #f ; tests are excluded in the manifest #:phases (modify-phases %standard-phases (add-before 'build 'set-flags - (lambda _ + (lambda* (#:key inputs #:allow-other-keys) + (setenv "HTSLIB_MODE" "external") + (setenv "HTSLIB_LIBRARY_DIR" + (string-append (assoc-ref inputs "htslib") "/lib")) + (setenv "HTSLIB_INCLUDE_DIR" + (string-append (assoc-ref inputs "htslib") "/include"= )) (setenv "LDFLAGS" "-lncurses") (setenv "CFLAGS" "-D_CURSES_LIB=3D1") #t))))) + (propagated-inputs + `(("htslib" ,htslib))) ; Included from installed header fi= les. (inputs `(("ncurses" ,ncurses) ("zlib" ,zlib))) --=20 2.10.0 >From 2afa68286db8cf55cc9dc325bef2f529b7e8f2ba Mon Sep 17 00:00:00 2001 From: Marius Bakke Date: Thu, 22 Sep 2016 14:11:59 +0100 Subject: [PATCH 7/7] gnu: python-pysam: Enable tests. * gnu/packages/bioinformatics.scm (python-pysam)[source]: Change from PyPi to source archive due to missing test data. [arguments]: Add check phase after install. [native-inputs]: Add python-nose, samtools and bcftools. --- gnu/packages/bioinformatics.scm | 36 ++++++++++++++++++++++++++++++------ 1 file changed, 30 insertions(+), 6 deletions(-) diff --git a/gnu/packages/bioinformatics.scm b/gnu/packages/bioinformatics.= scm index a712a01..7a784c8 100644 --- a/gnu/packages/bioinformatics.scm +++ b/gnu/packages/bioinformatics.scm @@ -1351,18 +1351,21 @@ multiple sequence alignments.") (version "0.9.1.4") (source (origin (method url-fetch) - (uri (pypi-uri "pysam" version)) + ;; Test data is missing on PyPi. + (uri (string-append + "https://github.com/pysam-developers/pysam/archive/v" + version ".tar.gz")) + (file-name (string-append name "-" version ".tar.gz")) (sha256 (base32 - "1i1djacqbr88y7w18b4aa78zxnsyr4sz7yqdq2spi7gs0y6pzvjn")) + "0y41ssbg6nvn2jgcbnrvkzblpjcwszaiv1rgyd8dwzjkrbfsgsmc")) (modules '((guix build utils))) (snippet ;; Drop bundled htslib. TODO: Also remove samtools and bcft= ools. '(delete-file-recursively "htslib")))) (build-system python-build-system) (arguments - `(#:tests? #f ; tests are excluded in the manifest - #:phases + `(#:phases (modify-phases %standard-phases (add-before 'build 'set-flags (lambda* (#:key inputs #:allow-other-keys) @@ -1373,7 +1376,24 @@ multiple sequence alignments.") (string-append (assoc-ref inputs "htslib") "/include"= )) (setenv "LDFLAGS" "-lncurses") (setenv "CFLAGS" "-D_CURSES_LIB=3D1") - #t))))) + #t)) + (delete 'check) + (add-after 'install 'check + (lambda* (#:key inputs outputs #:allow-other-keys) + (setenv "PYTHONPATH" + (string-append + (getenv "PYTHONPATH") + ":" (assoc-ref outputs "out") + "/lib/python" + (string-take (string-take-right + (assoc-ref inputs "python") 5) 3) + "/site-packages")) + ;; Step out of source dir so python does not import from CWD. + (chdir "tests") + (setenv "HOME" "/tmp") + (and (zero? (system* "make" "-C" "pysam_data")) + (zero? (system* "make" "-C" "cbcf_data")) + (zero? (system* "nosetests" "-v")))))))) (propagated-inputs `(("htslib" ,htslib))) ; Included from installed header fi= les. (inputs @@ -1381,7 +1401,11 @@ multiple sequence alignments.") ("zlib" ,zlib))) (native-inputs `(("python-cython" ,python-cython) - ("python-setuptools" ,python-setuptools))) + ("python-setuptools" ,python-setuptools) + ;; Depencies below are are for tests only. + ("samtools" ,samtools) + ("bcftools" ,bcftools) + ("python-nose" ,python-nose))) (home-page "https://github.com/pysam-developers/pysam") (synopsis "Python bindings to the SAMtools C API") (description --=20 2.10.0 --=-=-=--