* [bug#44613] [PATCH] Fix build for bedtools
@ 2020-11-13 11:01 Roel Janssen
2020-11-13 12:39 ` zimoun
0 siblings, 1 reply; 7+ messages in thread
From: Roel Janssen @ 2020-11-13 11:01 UTC (permalink / raw)
To: 44613
[-- Attachment #1: Type: text/plain, Size: 556 bytes --]
Dear Guix,
By updating samtools to 1.11, I introduced a build failure for
bedtools. More precisely, the tests for intersect break in precisely
this way:
https://github.com/arq5x/bedtools2/issues/814
With the following patches, I'd like to add samtools-1.9, htslib-1.9
(samtools depends on that) to fix this problem with bedtools.
Alternatively we could add a patch to disable the failing bedtools
tests. I manually inspected the test results, and seem to match
perfectly (indicating that there's no problem with bedtools).
Kind regards,
Roel Janssen
[-- Attachment #2: 0003-gnu-bedtools-Use-samtools-1.9.patch --]
[-- Type: text/x-patch, Size: 1018 bytes --]
From 2541b3c44e695ff1a6b56128b6efa72c463c0b0a Mon Sep 17 00:00:00 2001
From: Roel Janssen <roel@gnu.org>
Date: Fri, 13 Nov 2020 11:29:38 +0100
Subject: [PATCH 3/3] gnu: bedtools: Use samtools-1.9.
The build for bedtools with samtools 1.11 triggers a testsuite
failure which is reported here:
https://github.com/arq5x/bedtools2/issues/814
* gnu/packages/bioinformatics.scm (bedtools): Use samtools-1.9.
---
gnu/packages/bioinformatics.scm | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/gnu/packages/bioinformatics.scm b/gnu/packages/bioinformatics.scm
index e047aebd1d..8ad38ac498 100644
--- a/gnu/packages/bioinformatics.scm
+++ b/gnu/packages/bioinformatics.scm
@@ -437,7 +437,7 @@ computational cluster.")
(native-inputs
`(("python" ,python-wrapper)))
(inputs
- `(("samtools" ,samtools)
+ `(("samtools" ,samtools-1.9)
("zlib" ,zlib)))
(home-page "https://github.com/arq5x/bedtools2")
(synopsis "Tools for genome analysis and arithmetic")
--
2.29.2
[-- Attachment #3: 0002-gnu-Add-samtools-1.9.patch --]
[-- Type: text/x-patch, Size: 1635 bytes --]
From 7881fd95bcddc0fee4717fab30dbb276745825d8 Mon Sep 17 00:00:00 2001
From: Roel Janssen <roel@gnu.org>
Date: Fri, 13 Nov 2020 11:29:20 +0100
Subject: [PATCH 2/3] gnu: Add samtools-1.9.
* gnu/packages/bioinformatics.scm (samtools-1.9): New variable.
---
gnu/packages/bioinformatics.scm | 25 +++++++++++++++++++++++++
1 file changed, 25 insertions(+)
diff --git a/gnu/packages/bioinformatics.scm b/gnu/packages/bioinformatics.scm
index ba86333fc3..e047aebd1d 100644
--- a/gnu/packages/bioinformatics.scm
+++ b/gnu/packages/bioinformatics.scm
@@ -5699,6 +5699,31 @@ variant calling (in conjunction with bcftools), and a simple alignment
viewer.")
(license license:expat)))
+(define-public samtools-1.9
+ (package (inherit samtools)
+ (name "samtools")
+ (version "1.9")
+ (source
+ (origin
+ (method url-fetch)
+ (uri
+ (string-append "mirror://sourceforge/samtools/samtools/"
+ version "/samtools-" version ".tar.bz2"))
+ (sha256
+ (base32
+ "10ilqbmm7ri8z431sn90lvbjwizd0hhkf9rcqw8j823hf26nhgq8"))
+ (modules '((guix build utils)))
+ (snippet '(begin
+ ;; Delete bundled htslib.
+ (delete-file-recursively "htslib-1.9")
+ #t))))
+ (inputs
+ `(("htslib" ,htslib-1.9)
+ ("ncurses" ,ncurses)
+ ("perl" ,perl)
+ ("python" ,python)
+ ("zlib" ,zlib)))))
+
(define-public samtools-0.1
;; This is the most recent version of the 0.1 line of samtools. The input
;; and output formats differ greatly from that used and produced by samtools
--
2.29.2
[-- Attachment #4: 0001-gnu-Add-htslib-1.9.patch --]
[-- Type: text/x-patch, Size: 1291 bytes --]
From 220604159383b09e11a7b0bed51237257c7c0442 Mon Sep 17 00:00:00 2001
From: Roel Janssen <roel@gnu.org>
Date: Fri, 13 Nov 2020 11:28:43 +0100
Subject: [PATCH 1/3] gnu: Add htslib-1.9.
* gnu/packages/bioinformatics.scm (htslib-1.9): New variable.
---
gnu/packages/bioinformatics.scm | 13 +++++++++++++
1 file changed, 13 insertions(+)
diff --git a/gnu/packages/bioinformatics.scm b/gnu/packages/bioinformatics.scm
index 06972dee51..ba86333fc3 100644
--- a/gnu/packages/bioinformatics.scm
+++ b/gnu/packages/bioinformatics.scm
@@ -4250,6 +4250,19 @@ data. It also provides the @command{bgzip}, @command{htsfile}, and
;; the rest is released under the Expat license
(license (list license:expat license:bsd-3))))
+(define-public htslib-1.9
+ (package (inherit htslib)
+ (name "htslib")
+ (version "1.9")
+ (source (origin
+ (method url-fetch)
+ (uri (string-append
+ "https://github.com/samtools/htslib/releases/download/"
+ version "/htslib-" version ".tar.bz2"))
+ (sha256
+ (base32
+ "16ljv43sc3fxmv63w7b2ff8m1s7h89xhazwmbm1bicz8axq8fjz0"))))))
+
;; This package should be removed once no packages rely upon it.
(define htslib-1.3
(package
--
2.29.2
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [bug#44613] [PATCH] Fix build for bedtools
2020-11-13 11:01 [bug#44613] [PATCH] Fix build for bedtools Roel Janssen
@ 2020-11-13 12:39 ` zimoun
2020-11-13 12:55 ` Roel Janssen
0 siblings, 1 reply; 7+ messages in thread
From: zimoun @ 2020-11-13 12:39 UTC (permalink / raw)
To: Roel Janssen, 44613
Hi Roel,
On Fri, 13 Nov 2020 at 12:01, Roel Janssen <roel@gnu.org> wrote:
> With the following patches, I'd like to add samtools-1.9, htslib-1.9
> (samtools depends on that) to fix this problem with bedtools.
Recently, investigating why the substitute of ’python-pysam’ was not
available, I decided then to give a try at fixing the TODO:
--8<---------------cut here---------------start------------->8---
(snippet '(begin
;; Drop bundled htslib. TODO: Also remove samtools
;; and bcftools.
(delete-file-recursively "htslib")
#t))))
--8<---------------cut here---------------end--------------->8---
And the bundled version is 1.9 (if I remember correctly), therefore
because of:
--8<---------------cut here---------------start------------->8---
(native-inputs
`(("python-cython" ,python-cython)
;; Dependencies below are are for tests only.
("samtools" ,samtools)
("bcftools" ,bcftools)
("python-nose" ,python-nose)))
--8<---------------cut here---------------end--------------->8---
some tests are unhappy.
That’s said, these additions seem fine with me. :-)
All the best,
simon
^ permalink raw reply [flat|nested] 7+ messages in thread
* [bug#44613] [PATCH] Fix build for bedtools
2020-11-13 12:39 ` zimoun
@ 2020-11-13 12:55 ` Roel Janssen
2020-11-13 13:34 ` zimoun
0 siblings, 1 reply; 7+ messages in thread
From: Roel Janssen @ 2020-11-13 12:55 UTC (permalink / raw)
To: zimoun, 44613
Hi Simon,
On Fri, 2020-11-13 at 13:39 +0100, zimoun wrote:
> Hi Roel,
>
> On Fri, 13 Nov 2020 at 12:01, Roel Janssen <roel@gnu.org> wrote:
>
> > With the following patches, I'd like to add samtools-1.9, htslib-
> > 1.9
> > (samtools depends on that) to fix this problem with bedtools.
>
> Recently, investigating why the substitute of ’python-pysam’ was not
> available, I decided then to give a try at fixing the TODO:
>
> --8<---------------cut here---------------start------------->8---
> (snippet '(begin
> ;; Drop bundled htslib. TODO: Also remove
> samtools
> ;; and bcftools.
> (delete-file-recursively "htslib")
> #t))))
> --8<---------------cut here---------------end--------------->8---
>
> And the bundled version is 1.9 (if I remember correctly), therefore
> because of:
>
> --8<---------------cut here---------------start------------->8---
> (native-inputs
> `(("python-cython" ,python-cython)
> ;; Dependencies below are are for tests only.
> ("samtools" ,samtools)
> ("bcftools" ,bcftools)
> ("python-nose" ,python-nose)))
> --8<---------------cut here---------------end--------------->8---
>
> some tests are unhappy.
>
> That’s said, these additions seem fine with me. :-)
>
I also tried removing the bundled htslib for bedtools, but didn't go
this route for two reasons:
- The bundled htslib for bedtools seems "slightly modified" (I didn't
investigate further)
- Replacing the references to libhts.a with $(pkg-config htslib --
cflags --libs) produced various linker errors. So I stopped right
there.
I'm sure more tools will likely have failed because of the htslib
upgrade (sorry about this!), so having htslib-1.9 around for some time
may be a good fallback for now.
Just to double-check: Is it OK to push the proposed patches?
Kind regards,
Roel Janssen
^ permalink raw reply [flat|nested] 7+ messages in thread
* [bug#44613] [PATCH] Fix build for bedtools
2020-11-13 12:55 ` Roel Janssen
@ 2020-11-13 13:34 ` zimoun
2020-11-13 14:00 ` bug#44613: " Roel Janssen
2020-11-13 15:08 ` [bug#44613] " Marius Bakke
0 siblings, 2 replies; 7+ messages in thread
From: zimoun @ 2020-11-13 13:34 UTC (permalink / raw)
To: Roel Janssen, 44613
Hi Roel,
On Fri, 13 Nov 2020 at 13:55, Roel Janssen <roel@gnu.org> wrote:
> I also tried removing the bundled htslib for bedtools, but didn't go
> this route for two reasons:
> - The bundled htslib for bedtools seems "slightly modified" (I didn't
> investigate further)
> - Replacing the references to libhts.a with $(pkg-config htslib --
> cflags --libs) produced various linker errors. So I stopped right
> there.
>
> I'm sure more tools will likely have failed because of the htslib
> upgrade (sorry about this!), so having htslib-1.9 around for some time
> may be a good fallback for now.
Thank for your explanations.
> Just to double-check: Is it OK to push the proposed patches?
I have not tried them but they LGTM.
All the best,
simon
PS:
I am always confused if the removal should be done in ’origin’ or in the
’add-after 'unpack’ phase; especially when the bundle is free software.
Other said, what should an user expect when fetching with “guix build -S”?
Anyway! :-)
^ permalink raw reply [flat|nested] 7+ messages in thread
* bug#44613: [PATCH] Fix build for bedtools
2020-11-13 13:34 ` zimoun
@ 2020-11-13 14:00 ` Roel Janssen
2020-11-13 15:08 ` [bug#44613] " Marius Bakke
1 sibling, 0 replies; 7+ messages in thread
From: Roel Janssen @ 2020-11-13 14:00 UTC (permalink / raw)
To: zimoun; +Cc: 44613-done
Hi Simon,
On Fri, 2020-11-13 at 14:34 +0100, zimoun wrote:
> Hi Roel,
>
> On Fri, 13 Nov 2020 at 13:55, Roel Janssen <roel@gnu.org> wrote:
>
> > I also tried removing the bundled htslib for bedtools, but didn't
> > go
> > this route for two reasons:
> > - The bundled htslib for bedtools seems "slightly modified" (I
> > didn't
> > investigate further)
> > - Replacing the references to libhts.a with $(pkg-config htslib --
> > cflags --libs) produced various linker errors. So I stopped right
> > there.
> >
> > I'm sure more tools will likely have failed because of the htslib
> > upgrade (sorry about this!), so having htslib-1.9 around for some
> > time
> > may be a good fallback for now.
>
> Thank for your explanations.
>
>
> > Just to double-check: Is it OK to push the proposed patches?
>
> I have not tried them but they LGTM.
>
Thanks for the quick review. I pushed the patches in
c3232fcc7785abc1057a0d4b5b1832f1e39c9c1b,
da4a38edad52f7bb5a8d41465d09f3f0197fd0b7, and
3ede804f6d4c38ff0b9a5705544a8c35f6827ff1.
Kind regards,
Roel Janssen
^ permalink raw reply [flat|nested] 7+ messages in thread
* [bug#44613] [PATCH] Fix build for bedtools
2020-11-13 13:34 ` zimoun
2020-11-13 14:00 ` bug#44613: " Roel Janssen
@ 2020-11-13 15:08 ` Marius Bakke
2020-11-13 15:51 ` [bug#44613] what “guix build -S” should return? zimoun
1 sibling, 1 reply; 7+ messages in thread
From: Marius Bakke @ 2020-11-13 15:08 UTC (permalink / raw)
To: zimoun, Roel Janssen, 44613
[-- Attachment #1: Type: text/plain, Size: 734 bytes --]
zimoun <zimon.toutoune@gmail.com> writes:
> PS:
> I am always confused if the removal should be done in ’origin’ or in the
> ’add-after 'unpack’ phase; especially when the bundle is free software.
> Other said, what should an user expect when fetching with “guix build -S”?
> Anyway! :-)
Unbundling is always better to do in a snippet. It leads to less
bandwidth usage, and users can more easily inspect the (actual) code.
For other kinds of patching the boundary is less clear. Generally,
Guix-specific tweaks should be in a phase, but "universal" bug fixes may
well be in a snippet.
I sometimes imagine a downstream distribution that use Guix sources, but
not the build scripts, to draw the line.
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 507 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread
* [bug#44613] what “guix build -S” should return?
2020-11-13 15:08 ` [bug#44613] " Marius Bakke
@ 2020-11-13 15:51 ` zimoun
0 siblings, 0 replies; 7+ messages in thread
From: zimoun @ 2020-11-13 15:51 UTC (permalink / raw)
To: Marius Bakke, Roel Janssen, 44613
Hi Marius,
Thank you for the explanations.
On Fri, 13 Nov 2020 at 16:08, Marius Bakke <marius@gnu.org> wrote:
> zimoun <zimon.toutoune@gmail.com> writes:
>
>> PS:
>> I am always confused if the removal should be done in ’origin’ or in the
>> ’add-after 'unpack’ phase; especially when the bundle is free software.
>> Other said, what should an user expect when fetching with “guix build -S”?
>> Anyway! :-)
>
> Unbundling is always better to do in a snippet. It leads to less
> bandwidth usage, and users can more easily inspect the (actual) code.
Well, I do not know. For example, I could do this workflow:
guix environment bedtools
tar -xvf $(guix build -S bedtools)
make
which probably fails because removing the bundles often needs some extra
tweaks. Concretely, see python-pysam for instance:
--8<---------------cut here---------------start------------->8---
(snippet '(begin
;; Drop bundled htslib. TODO: Also remove samtools
;; and bcftools.
(delete-file-recursively "htslib")
#t))))
[...]
#:phases
(modify-phases %standard-phases
(add-before 'build 'set-flags
(lambda* (#:key inputs #:allow-other-keys)
(setenv "HTSLIB_MODE" "external")
(setenv "HTSLIB_LIBRARY_DIR"
(string-append (assoc-ref inputs "htslib") "/lib"))
(setenv "HTSLIB_INCLUDE_DIR"
(string-append (assoc-ref inputs "htslib") "/include"))
--8<---------------cut here---------------end--------------->8---
Then, I am not convince that:
guix build bedtools --with-git-url=http://example.org
works too. Or ’--with-source=’ as well. I remember a discussion
initiated by Mark and Maxim about this: snippet vs phases but I am not
able to reach it.
> For other kinds of patching the boundary is less clear. Generally,
> Guix-specific tweaks should be in a phase, but "universal" bug fixes may
> well be in a snippet.
I agree that non-free and bug fixes should go to snippet. Then I am
still confused and my feelings are mixed about Guix specific tweaks.
> I sometimes imagine a downstream distribution that use Guix sources, but
> not the build scripts, to draw the line.
It seems a good criteria to draw the line. And in the case of bedtools
or python-pysam or many others, ’snippet’ removes (free software)
bundles because of an implicit and non-uniform Guix policy that a
downstream distribution could choose differently.
Well, my mind is not clear about this topic. :-)
All the best,
simon
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2020-11-13 16:02 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2020-11-13 11:01 [bug#44613] [PATCH] Fix build for bedtools Roel Janssen
2020-11-13 12:39 ` zimoun
2020-11-13 12:55 ` Roel Janssen
2020-11-13 13:34 ` zimoun
2020-11-13 14:00 ` bug#44613: " Roel Janssen
2020-11-13 15:08 ` [bug#44613] " Marius Bakke
2020-11-13 15:51 ` [bug#44613] what “guix build -S” should return? zimoun
Code repositories for project(s) associated with this external index
https://git.savannah.gnu.org/cgit/guix.git
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.