[PATCH 0/4] Add sra-tools.

[PATCH 0/4] Add sra-tools.

[Ricardo Wurmus]

Hi Guix,

this patch set adds the sra tools (a collection containing the commonly
encountered "fastq-dump") and its dependencies.

The NCBI stuff uses a frustrating custom configuration script written in
Perl and I had to learn more about how it works and what assumptions it
makes than I hoped for.

More significant changes had to be made to the build of both ncbi-vdb
and sra-tools, because sra-tools expects the sources and the raw build
output of ncbi-vdb to lie in some directory.  It would not build against
an installed version of ncbi-vdb due to missing libraries and missing
includes.

As both of these projects have been split from the same repository in
the recent past and are so closely coupled I took the liberty to let
ncbi-vdb install more headers and more libraries, so that at the time
sra-tools is built we don't need to rebuild ncbi-vdb just to get at the
includes and interface libraries.  The phases do not look pretty but
they are as clean as I could make them.

I had to remove libxml2 from the inputs to sra-tools, resulting in a
failure to build a couple of loader tools.  The reason for this is that
I encountered linker problems like these:


~~~~~~~~~~
...
gcc -static-libstdc++ -static-libgcc -o /buildtmp/nix-build-sra-tools-2.4.5-5.drv-4/sra-tools-2.4.5-5/build/sra-tools/linux/gcc/x86_64/rel/bin/fastq-load.2.4.5 -DNDEBUG -m64 -L/gnu/store/pjjpcai7ckqlric0p0xdrzjsifdx1d44-hdf5-1.8.12/lib -L/gnu/store/3x1qwishrgvsvaddhv2238s27jcg60wp-libxml2-2.9.0/lib -L/gnu/store/xbvqikb6y02zqpvac876sxy4ci937spd-file-5.22/lib -L/gnu/store/35gqycjy0ih7wrjadzhvixya0a905sf5-fuse-2.9.3/lib pstring.o common-xml.o experiment-xml.o run-xml.o loader-file.o loader-fmt.o sra-writer.o loader.o writer-illumina.o writer-454.o writer-ion-torrent.o fastq-fmt.o -L/buildtmp/nix-build-sra-tools-2.4.5-5.drv-4/sra-tools-2.4.5-5/build/sra-tools/linux/gcc/x86_64/rel/lib -L/buildtmp/nix-build-sra-tools-2.4.5-5.drv-4/sra-tools-2.4.5-5/build/sra-tools/linux/gcc/x86_64/rel/ilib -L/gnu/store/xldsqaj0cw6v89karyx1izhhmxihpmb1-ngs-sdk-1.1.0/lib64 -L/gnu/store/fjpl0vm87dzn3hpvqaca625bq8hv1mha-ncbi-vdb-2.4.5-5/ilib -Wl,-Bstatic -lkapp -lload -lncbi-wvdb -lkxfs -lkxml -Wl,-Bdynamic -ldl -lpthread -Wl,-Bstatic -L/gnu/store/3x1qwishrgvsvaddhv2238s27jcg60wp-libxml2-2.9.0/lib -lxml2 -Wl,-Bdynamic -lm
/gnu/store/3x1makeqwishrgvsvaddhv2238s27jcg60wp-libxml2-2.9.0/lib/libxml2.a(xmlIO.o): In function `xmlGzfileOpenW':
(.text+0x2c9): undefined reference to `gzdopen'
/gnu/store/3x1qwishrgvsvaddhv2238s27jcg60wp-libxml2-2.9.0/lib/libxml2.a(xmlIO.o): In function `xmlGzfileOpenW':
(.text+0x2fe): undefined reference to `gzopen64'
/gnu/store/3x1qwishrgvsvaddhv2238s27jcg60wp-libxml2-2.9.0/lib/libxml2.a(xmlIO.o): In function `xmlGzfileClose':
(.text+0xe85): undefined reference to `gzclose'
/gnu/store/3x1qwishrgvsvaddhv2238s27jcg60wp-libxml2-2.9.0/lib/libxml2.a(xmlIO.o): In function `xmlGzfileRead':
(.text+0xeb5): undefined reference to `gzread'
/gnu/store/3x1qwishrgvsvaddhv2238s27jcg60wp-libxml2-2.9.0/lib/libxml2.a(xmlIO.o): In function `xmlGzfileWrite':
(.text+0x1095): undefined reference to `gzwrite'
/gnu/store/3x1qwishrgvsvaddhv2238s27jcg60wp-libxml2-2.9.0/lib/libxml2.a(xmlIO.o): In function `__xmlParserInputBufferCreateFilename':
(.text+0x1d82): undefined reference to `gzdirect'
/gnu/store/3x1qwishrgvsvaddhv2238s27jcg60wp-libxml2-2.9.0/lib/libxml2.a(xmlIO.o): In function `xmlGzfileOpen_real':
(.text+0x171f): undefined reference to `gzdopen'
/gnu/store/3x1qwishrgvsvaddhv2238s27jcg60wp-libxml2-2.9.0/lib/libxml2.a(xmlIO.o): In function `xmlGzfileOpen_real':
(.text+0x175d): undefined reference to `gzopen64'
/gnu/store/3x1qwishrgvsvaddhv2238s27jcg60wp-libxml2-2.9.0/lib/libxml2.a(xzlib.o): In function `xz_decomp':
(.text+0x5a4): undefined reference to `lzma_code'
/gnu/store/3x1qwishrgvsvaddhv2238s27jcg60wp-libxml2-2.9.0/lib/libxml2.a(xzlib.o): In function `xz_make':
(.text+0x983): undefined reference to `lzma_properties_decode'
/gnu/store/3x1qwishrgvsvaddhv2238s27jcg60wp-libxml2-2.9.0/lib/libxml2.a(xzlib.o): In function `xz_make':
(.text+0xbb8): undefined reference to `lzma_auto_decoder'
/gnu/store/3x1qwishrgvsvaddhv2238s27jcg60wp-libxml2-2.9.0/lib/libxml2.a(xzlib.o): In function `__libxml2_xzclose':
(.text+0x1605): undefined reference to `lzma_end'
collect2: error: ld returned 1 exit status

Re: [PATCH 0/4] Add sra-tools.

[Ludovic Courtès]

Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de> skribis:

> The NCBI project uses a frustrating custom configuration script written
> in Perl, requiring a lot of tweaks to make it work.
>
> More significant changes had to be made to the build of both ncbi-vdb
> and sra-tools, because sra-tools expects the sources and the raw build
> output of ncbi-vdb to lie in some directory.  It would not build against
> an installed version of ncbi-vdb due to missing libraries and missing
> includes.
>
> As both of these projects have been split from the same repository in
> the recent past and are so closely coupled I took the liberty to let
> ncbi-vdb install more headers and more libraries, so that at the time
> sra-tools is built we don't need to rebuild ncbi-vdb just to get at the
> includes and interface libraries.  The phases do not look pretty but
> they are as clean as I could make them.

Sounds reasonable.

> I had to remove libxml2 from the inputs to sra-tools, resulting in a
> failure to build a couple of loader tools.  The reason for this is that
> I encountered linker problems like these:

It looks like an upstream bug?  And why is it being statically-linked
anyway?  It would be nice if they would default to dynamic linking,
which is more efficient in terms of memory use, etc.

> From c80c582724cc515b4baa54623c5c691de01ea2fd Mon Sep 17 00:00:00 2001
> From: Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de>
> Date: Tue, 14 Apr 2015 14:23:19 +0200
> Subject: [PATCH 1/4] gnu: Add NGS SDK.
>
> * gnu/packages/bioinformatics.scm (ngs-sdk): New variable.

OK!

> +    (synopsis "API for accessing Next Generation Sequencing data")

I would put quotes around “Next Generation”.  :-)

> From f689906b999164282841baafe3ec705a91103ef8 Mon Sep 17 00:00:00 2001
> From: Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de>
> Date: Tue, 14 Apr 2015 14:24:01 +0200
> Subject: [PATCH 2/4] gnu: Add NGS Java bindings.
>
> * gnu/packages/bioinformatics.scm (ngs-java): New variable.

[...]

> +    (synopsis "Java bindings for NGS SDK")
> +    (description
> +     (string-append (package-description ngs-sdk)
> +                    "\n\nThis package contains the Java bindings."))))

‘description’ should be a string literal, otherwise xgettext won’t catch
it, thereby preventing i18n.

What I would do is just keep ‘description’ unchanged, or rewrite it.
Keeping it unchanged is OK here since the synopsis makes it clear that
these are Java bindings.

> From 9e08ad24aa173320c025e6bf2460d95784c74a40 Mon Sep 17 00:00:00 2001
> From: Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de>
> Date: Tue, 14 Apr 2015 14:42:49 +0200
> Subject: [PATCH 3/4] gnu: Add NCBI VDB.
>
> * gnu/packages/bioinformatics.scm (ncbi-vdb): New variable.

OK.

> From 11acd5405931288923e54298ba4d481c2b55211d Mon Sep 17 00:00:00 2001
> From: Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de>
> Date: Tue, 14 Apr 2015 14:43:32 +0200
> Subject: [PATCH 4/4] gnu: Add sra-tools.
>
> * gnu/packages/bioinformatics.scm (sra-tools): New variable.

OK.

Thanks!

Ludo’.

Re: [PATCH 0/4] Add sra-tools.

[Ricardo Wurmus]

Ludovic Courtès writes:

>> I had to remove libxml2 from the inputs to sra-tools, resulting in a
>> failure to build a couple of loader tools.  The reason for this is that
>> I encountered linker problems like these:
>
> It looks like an upstream bug?  And why is it being statically-linked
> anyway?  It would be nice if they would default to dynamic linking,
> which is more efficient in terms of memory use, etc.

Indeed.  I will file a bug report upstream.

>> +    (synopsis "API for accessing Next Generation Sequencing data")
>
> I would put quotes around “Next Generation”.  :-)

"Next-generation sequencing" is an actual term (although it sounds a
little silly).  It refers to high-throughput sequencing technology.

See http://www.nature.com/subjects/next-generation-sequencing

>> +    (synopsis "Java bindings for NGS SDK")
>> +    (description
>> +     (string-append (package-description ngs-sdk)
>> +                    "\n\nThis package contains the Java bindings."))))
>
> ‘description’ should be a string literal, otherwise xgettext won’t catch
> it, thereby preventing i18n.
>
> What I would do is just keep ‘description’ unchanged, or rewrite it.
> Keeping it unchanged is OK here since the synopsis makes it clear that
> these are Java bindings.

Okay, I'll just keep the inherited description then.  I should note that
"python-wrapper" in python.scm also does not use a string literal.

~~ Ricardo