[bug#54635] [PATCH 0/5] Add wfmash

[bug#54635] [PATCH 0/5] Add wfmash

[Arun Isaac]

This patchset adds wfmash, and ensures that, in addition to x86_64, it also
cross-compiles to riscv64.

Arun Isaac (5):
  gnu: gsl: Force bootstrap when cross-compiling to riscv64-linux.
  gnu: htslib: Add bzip2 and xz to inputs.
  gnu: atomic-queue: Run tests correctly.
  gnu: atomic-queue: Do not depend on boost when cross-compiling.
  gnu: Add wfmash.

 gnu/packages/bioinformatics.scm | 55 +++++++++++++++++++++++++++++++--
 gnu/packages/cpp.scm            | 26 +++++++++++++---
 gnu/packages/maths.scm          | 15 ++++++++-
 3 files changed, 88 insertions(+), 8 deletions(-)

-- 
2.34.0

[bug#54635] [PATCH 1/5] gnu: gsl: Force bootstrap when cross-compiling to riscv64-linux.

[Arun Isaac]

* gnu/packages/maths.scm (gsl)[arguments]: Force autotools bootstrap when
cross-compiling to riscv64-linux.
[native-inputs]: Add autoconf, automake and libtool when cross-compiling to
riscv64-linux.
---
 gnu/packages/maths.scm | 15 ++++++++++++++-
 1 file changed, 14 insertions(+), 1 deletion(-)

diff --git a/gnu/packages/maths.scm b/gnu/packages/maths.scm
index ecb85642ec..f01bf51580 100644
--- a/gnu/packages/maths.scm
+++ b/gnu/packages/maths.scm
@@ -19,7 +19,7 @@
 ;;; Copyright © 2017 Nikita <nikita@n0.is>
 ;;; Copyright © 2017 Ben Woodcroft <donttrustben@gmail.com>
 ;;; Copyright © 2017 Theodoros Foradis <theodoros@foradis.org>
-;;; Copyright © 2017, 2019 Arun Isaac <arunisaac@systemreboot.net>
+;;; Copyright © 2017, 2019, 2022 Arun Isaac <arunisaac@systemreboot.net>
 ;;; Copyright © 2017–2021 Tobias Geerinckx-Rice <me@tobias.gr>
 ;;; Copyright © 2017 Dave Love <me@fx@gnu.org>
 ;;; Copyright © 2018, 2019, 2020, 2021 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
@@ -533,6 +533,13 @@ (define-public gsl
          #:phases
          (modify-phases %standard-phases
            ,@(cond
+              ((target-riscv64?)
+               '((add-after 'unpack 'force-bootstrap
+                   (lambda _
+                     ;; gsl ships with an old configure script that does not
+                     ;; support riscv64. Regenerate it.
+                     (delete-file "configure")))))
+
               ((or (string-prefix? "aarch64" system)
                    (string-prefix? "powerpc" system))
                ;; Some sparse matrix tests are failing on AArch64 and PowerPC:
@@ -568,6 +575,12 @@ (define-public gsl
                         (string-append "exit (77);\n" all)))))))
 
               (else '()))))))
+    (native-inputs
+     (if (target-riscv64?)
+         `(("autoconf" ,autoconf)
+           ("automake" ,automake)
+           ("libtool" ,libtool))
+         '()))
     (home-page "https://www.gnu.org/software/gsl/")
     (synopsis "Numerical library for C and C++")
     (description
-- 
2.34.0

[bug#54635] [PATCH 2/5] gnu: htslib: Add bzip2 and xz to inputs.

[Arun Isaac]

htslib links to libbz2 from bzip2 and liblzma from xz. Therefore, bzip2 and xz
should be listed in inputs even though they are already present implicitly in
native-inputs. Else, cross-compilation will fail.

* gnu/packages/bioinformatics.scm (htslib)[inputs]: Add bzip2 and xz.
---
 gnu/packages/bioinformatics.scm | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/gnu/packages/bioinformatics.scm b/gnu/packages/bioinformatics.scm
index 4e1028b3ce..2f6f3efb66 100644
--- a/gnu/packages/bioinformatics.scm
+++ b/gnu/packages/bioinformatics.scm
@@ -8,7 +8,7 @@
 ;;; Copyright © 2016, 2020, 2022 Marius Bakke <marius@gnu.org>
 ;;; Copyright © 2016, 2018 Raoul Bonnal <ilpuccio.febo@gmail.com>
 ;;; Copyright © 2017, 2018 Tobias Geerinckx-Rice <me@tobias.gr>
-;;; Copyright © 2017, 2021 Arun Isaac <arunisaac@systemreboot.net>
+;;; Copyright © 2017, 2021, 2022 Arun Isaac <arunisaac@systemreboot.net>
 ;;; Copyright © 2018 Joshua Sierles, Nextjournal <joshua@nextjournal.com>
 ;;; Copyright © 2018 Gábor Boskovits <boskovits@gmail.com>
 ;;; Copyright © 2018, 2019, 2020, 2021 Mădălin Ionel Patrașcu <madalinionel.patrascu@mdc-berlin.de>
@@ -4788,7 +4788,7 @@ (define-public htslib
                            "--enable-libcurl"
                            "--enable-s3")))
     (inputs
-     (list curl openssl))
+     (list bzip2 curl openssl xz))
     ;; This is referred to in the pkg-config file as a required library.
     (propagated-inputs
      (list zlib))
-- 
2.34.0

[bug#54635] [PATCH 3/5] gnu: atomic-queue: Run tests correctly.

[Arun Isaac]

* gnu/packages/cpp.scm (atomic-queue)[arguments]: In the check phase, run
tests instead of returning a function to run them.
---
 gnu/packages/cpp.scm | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/gnu/packages/cpp.scm b/gnu/packages/cpp.scm
index bb7f5ab95a..eb48902034 100644
--- a/gnu/packages/cpp.scm
+++ b/gnu/packages/cpp.scm
@@ -1297,8 +1297,7 @@ (define-public atomic-queue
          (replace 'check
            (lambda* (#:key tests? #:allow-other-keys)
              (when tests?
-               (lambda _
-                 (invoke "make" "run_tests")))))
+               (invoke "./tests"))))
          (replace 'install
            (lambda* (#:key outputs #:allow-other-keys)
              (copy-recursively "../source/include/atomic_queue"
-- 
2.34.0

[bug#54635] [PATCH 4/5] gnu: atomic-queue: Do not depend on boost when cross-compiling.

[Arun Isaac]

* gnu/packages/cpp.scm (atomic-queue)[arguments]: When cross-compiling, add
do-not-check-for-boost phase and delete the build phase.
[native-inputs]: Do not include boost when cross-compiling.
---
 gnu/packages/cpp.scm | 23 ++++++++++++++++++++---
 1 file changed, 20 insertions(+), 3 deletions(-)

diff --git a/gnu/packages/cpp.scm b/gnu/packages/cpp.scm
index eb48902034..649a57b0d5 100644
--- a/gnu/packages/cpp.scm
+++ b/gnu/packages/cpp.scm
@@ -1294,6 +1294,21 @@ (define-public atomic-queue
      `(#:configure-flags '("-Dbenchmarks=false")
        #:phases
        (modify-phases %standard-phases
+         ,@(if (%current-target-system)
+               `(;; boost is a test dependency. We don't run tests when
+                 ;; cross-compiling. So, do not check for it.
+                 (add-after 'unpack 'do-not-check-for-boost
+                   (lambda _
+                     (substitute* "meson.build"
+                       (("unit_test_framework =" all)
+                        (string-append "# " all))
+                       ((", unit_test_framework") ""))))
+                 ;; atomic-queue is a header-only library. Excepting the
+                 ;; tests, no building is required. And since we don't run
+                 ;; tests when cross-compiling, delete the build phase
+                 ;; entirely.
+                 (delete 'build))
+               '())
          (replace 'check
            (lambda* (#:key tests? #:allow-other-keys)
              (when tests?
@@ -1303,9 +1318,11 @@ (define-public atomic-queue
              (copy-recursively "../source/include/atomic_queue"
                                (string-append (assoc-ref outputs "out")
                                               "/include/atomic_queue")))))))
-     (native-inputs
-      (list boost
-            pkg-config))
+    (native-inputs
+     (cons pkg-config
+           (if (%current-target-system)
+               '()
+               (list boost))))
     (home-page "https://github.com/max0x7ba/atomic_queue")
     (synopsis "C++ lockless queue")
     (description
-- 
2.34.0

[bug#54635] [PATCH 4/5] gnu: atomic-queue: Do not depend on boost when cross-compiling.

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 682 bytes --]

Arun Isaac schreef op wo 30-03-2022 om 14:53 [+0530]:
> * gnu/packages/cpp.scm (atomic-queue)[arguments]: When cross-compiling, add
> do-not-check-for-boost phase and delete the build phase.
> [native-inputs]: Do not include boost when cross-compiling.

I think it's simpler and less tedious to just always include test
dependencies in native-inputs, without special-casing cross-
compilation.  WYDT?

Also, being header-only does not per-se mean ‘no building’.  E.g., what
if some of the headers are generated?  I think it's a bit simpler and
less likely to cause trouble to just always run the build phase, even
if sometimes it is a no-op.

Greetings,
Maxime.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

[bug#54635] [PATCH 4/5] gnu: atomic-queue: Do not depend on boost when cross-compiling.

[Arun Isaac]

> I think it's simpler and less tedious to just always include test
> dependencies in native-inputs, without special-casing cross-
> compilation.  WYDT?

Sure, agreed!

> Also, being header-only does not per-se mean ‘no building’.  E.g., what
> if some of the headers are generated?  I think it's a bit simpler and
> less likely to cause trouble to just always run the build phase, even
> if sometimes it is a no-op.

Agreed too!

Will address both in the next version of my patchset.

[bug#54635] [PATCH 5/5] gnu: Add wfmash.

[Arun Isaac]

* gnu/packages/bioinformatics.scm (wfmash): New variable.
---
 gnu/packages/bioinformatics.scm | 51 +++++++++++++++++++++++++++++++++
 1 file changed, 51 insertions(+)

diff --git a/gnu/packages/bioinformatics.scm b/gnu/packages/bioinformatics.scm
index 2f6f3efb66..500ba4ac60 100644
--- a/gnu/packages/bioinformatics.scm
+++ b/gnu/packages/bioinformatics.scm
@@ -16107,3 +16107,54 @@ (define-public ccwl
 @acronym{EDSL, Embedded Domain Specific Language} in the Scheme programming
 language.")
     (license license:gpl3+)))
+
+(define-public wfmash
+  (package
+    (name "wfmash")
+    (version "0.8.1")
+    (source
+     (origin
+       (method url-fetch)
+       (uri (string-append "https://github.com/ekg/wfmash/releases/download/v"
+                           version "/wfmash-v" version ".tar.gz"))
+       (sha256
+        (base32
+         "031cm1arpfckvihb28vlk69mirpnmlag81zcscfba1bac58wvr7c"))
+       (modules '((guix build utils)))
+       (snippet
+        '(begin
+           ;; Unbundle atomic-queue.
+           (delete-file-recursively "src/common/atomic_queue")
+           (substitute* "src/align/include/computeAlignments.hpp"
+             (("\"common/atomic_queue/atomic_queue.h\"")
+              "<atomic_queue/atomic_queue.h>"))))))
+    (build-system cmake-build-system)
+    (arguments
+     `(#:tests? #f                      ; no tests
+       #:phases
+       (modify-phases %standard-phases
+         ;; Remove compiler flags and checks specific to x86 when not
+         ;; targeting it.
+         ,@(if (target-x86-64?)
+               '()
+               '((add-after 'unpack 'remove-x86-specific-compile-flags
+                   (lambda _
+                     (substitute* (find-files "." "CMakeLists\\.txt")
+                       (("-mcx16") "")
+                       (("-march=native") ""))
+                     (substitute* "src/common/dset64.hpp"
+                       (("!__x86_64__") "0")))))))))
+    (inputs
+     (list atomic-queue
+           gsl
+           htslib
+           jemalloc
+           zlib))
+    (synopsis "Base-accurate DNA sequence aligner")
+    (description "@code{wfmash} is a DNA sequence read mapper based on mash
+distances and the wavefront alignment algorithm.  It is a fork of MashMap that
+implements base-level alignment via the wflign tiled wavefront global
+alignment algorithm.  It completes MashMap with a high-performance alignment
+module capable of computing base-level alignments for very large sequences.")
+    (home-page "https://github.com/ekg/wfmash")
+    (license license:expat)))
-- 
2.34.0

[bug#54635] [PATCH 1/5] gnu: gsl: Force bootstrap when cross-compiling to riscv64-linux.

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 440 bytes --]

Arun Isaac schreef op wo 30-03-2022 om 14:53 [+0530]:
> +    (native-inputs
> +     (if (target-riscv64?)
> +         `(("autoconf" ,autoconf)
> +           ("automake" ,automake)
> +           ("libtool" ,libtool))
> +         '()))

Nowadays input labels are not required anymore here, you can do

 (native-inputs (if (targer-riscv64?) (list autoconf automake libtool)
'()))

Greetings,
Maxime.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

[bug#54635] [PATCH 1/5] gnu: gsl: Force bootstrap when cross-compiling to riscv64-linux.

[Arun Isaac]

> Nowadays input labels are not required anymore here, you can do
>
>  (native-inputs (if (targer-riscv64?) (list autoconf automake libtool)
> '()))

Yes, indeed! Will remove the input labels in the next version of the
patchset.

[bug#54635] [PATCH 1/5] gnu: gsl: Force bootstrap when cross-compiling to riscv64-linux.

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 852 bytes --]

Arun Isaac schreef op wo 30-03-2022 om 14:53 [+0530]:
>             ,@(cond
> +              ((target-riscv64?)
> +               '((add-after 'unpack 'force-bootstrap
> +                   (lambda _
> +                     ;; gsl ships with an old configure script that does not
> +                     ;; support riscv64. Regenerate it.
> +                     (delete-file "configure")))))
> +

WDYT of making this unconditional?  Two benefits:

  * if Guix is ported to another new architecture,
    then no changes are necessary to the package definition.

  * 'configure' and 'Makefile.in' are not source code,
    and more difficult to audit for things like malware than
    'configure.ac' and 'Makefile.am'.

Greetings,
Maxime.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

[bug#54635] [PATCH 1/5] gnu: gsl: Force bootstrap when cross-compiling to riscv64-linux.

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 1452 bytes --]

On Wed, Mar 30, 2022 at 01:36:34PM +0200, Maxime Devos wrote:
> Arun Isaac schreef op wo 30-03-2022 om 14:53 [+0530]:
> >             ,@(cond
> > +              ((target-riscv64?)
> > +               '((add-after 'unpack 'force-bootstrap
> > +                   (lambda _
> > +                     ;; gsl ships with an old configure script that does not
> > +                     ;; support riscv64. Regenerate it.
> > +                     (delete-file "configure")))))
> > +
> 
> WDYT of making this unconditional?  Two benefits:
> 
>   * if Guix is ported to another new architecture,
>     then no changes are necessary to the package definition.
> 
>   * 'configure' and 'Makefile.in' are not source code,
>     and more difficult to audit for things like malware than
>     'configure.ac' and 'Makefile.am'.

This can be with a TODO for core-updates. gsl itself has about 2000
dependant packages.

That said, I'm not convinced about unilaterally removing configure
unless we make it a policy to remove it. Also, I haven't had trouble
with building gsl on riscv64-linux without this patch.


-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[bug#54635] [PATCH 1/5] gnu: gsl: Force bootstrap when cross-compiling to riscv64-linux.

[Arun Isaac]

>> >             ,@(cond
>> > +              ((target-riscv64?)
>> > +               '((add-after 'unpack 'force-bootstrap
>> > +                   (lambda _
>> > +                     ;; gsl ships with an old configure script that does not
>> > +                     ;; support riscv64. Regenerate it.
>> > +                     (delete-file "configure")))))
>> > +
>> 
>> WDYT of making this unconditional?  Two benefits:
>> 
>>   * if Guix is ported to another new architecture,
>>     then no changes are necessary to the package definition.
>> 
>>   * 'configure' and 'Makefile.in' are not source code,
>>     and more difficult to audit for things like malware than
>>     'configure.ac' and 'Makefile.am'.
>
> This can be with a TODO for core-updates. gsl itself has about 2000
> dependant packages.

I agree. That was my reasoning as well. If we agree that making it
unconditional is the way forward, I can send another patch for
core-updates after this patchset is pushed to master.

> That said, I'm not convinced about unilaterally removing configure
> unless we make it a policy to remove it. Also, I haven't had trouble
> with building gsl on riscv64-linux without this patch.

Without the force-boostrap phase, the configure phase fails during
cross-compilation.

--8<---------------cut here---------------start------------->8---
$ ./pre-inst-env guix build --target=riscv64-linux-gnu gsl
[...]
starting phase `configure'
source directory: "/tmp/guix-build-gsl-2.7.drv-0/gsl-2.7" (relative from build: ".")
build directory: "/tmp/guix-build-gsl-2.7.drv-0/gsl-2.7"
configure flags: ("CC_FOR_BUILD=gcc" "CONFIG_SHELL=/gnu/store/4y5m9lb8k3qkb1y9m02sw9w9a6hacd16-bash-minimal-5.1.8/bin/bash" "SHELL=/gnu/store/4y5m9lb8k3qkb1y9m02sw9w9a6hacd16-bash-minimal-5.1.8/bin/bash" "--prefix=/gnu/store/x7ag3i38ykn2l3f6sfn06bn9356kdk0x-gsl-2.7" "--enable-fast-install" "--build=x86_64-unknown-linux-gnu" "--host=riscv64-linux-gnu" "--disable-static")
checking for a BSD-compatible install... /gnu/store/d251rfgc9nm2clzffzhgiipdvfvzkvwi-coreutils-8.32/bin/install -c
checking whether build environment is sane... yes
checking for riscv64-linux-gnu-strip... riscv64-linux-gnu-strip
checking for a race-free mkdir -p... /gnu/store/d251rfgc9nm2clzffzhgiipdvfvzkvwi-coreutils-8.32/bin/mkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... no
checking whether make supports nested variables... yes
checking whether to enable maintainer-specific portions of Makefiles... no
checking for a sed that does not truncate output... /gnu/store/wxgv6i8g0p24q5gcyzd0yr07s8kn9680-sed-4.8/bin/sed
checking whether make sets $(MAKE)... (cached) no
checking build system type... x86_64-unknown-linux-gnu
checking host system type... Invalid configuration `riscv64-linux-gnu': machine `riscv64' not recognized
configure: error: /gnu/store/4y5m9lb8k3qkb1y9m02sw9w9a6hacd16-bash-minimal-5.1.8/bin/bash ./config.sub riscv64-linux-gnu failed
error: in phase 'configure': uncaught exception:
%exception #<&invoke-error program: "/gnu/store/4y5m9lb8k3qkb1y9m02sw9w9a6hacd16-bash-minimal-5.1.8/bin/bash" arguments: ("./configure" "CC_FOR_BUILD=gcc" "CONFIG_SHELL=/gnu/store/4y5m9lb8k3qkb1y9m02sw9w9a6hacd16-bash-minimal-5.1.8/bin/bash" "SHELL=/gnu/store/4y5m9lb8k3qkb1y9m02sw9w9a6hacd16-bash-minimal-5.1.8/bin/bash" "--prefix=/gnu/store/x7ag3i38ykn2l3f6sfn06bn9356kdk0x-gsl-2.7" "--enable-fast-install" "--build=x86_64-unknown-linux-gnu" "--host=riscv64-linux-gnu" "--disable-static") exit-status: 1 term-signal: #f stop-signal: #f> 
phase `configure' failed after 0.3 seconds
command "/gnu/store/4y5m9lb8k3qkb1y9m02sw9w9a6hacd16-bash-minimal-5.1.8/bin/bash" "./configure" "CC_FOR_BUILD=gcc" "CONFIG_SHELL=/gnu/store/4y5m9lb8k3qkb1y9m02sw9w9a6hacd16-bash-minimal-5.1.8/bin/bash" "SHELL=/gnu/store/4y5m9lb8k3qkb1y9m02sw9w9a6hacd16-bash-minimal-5.1.8/bin/bash" "--prefix=/gnu/store/x7ag3i38ykn2l3f6sfn06bn9356kdk0x-gsl-2.7" "--enable-fast-install" "--build=x86_64-unknown-linux-gnu" "--host=riscv64-linux-gnu" "--disable-static" failed with status 1
builder for `/gnu/store/fwjvvklmswc9midrcdg6qir2knvmraif-gsl-2.7.drv' failed with exit code 1
build of /gnu/store/fwjvvklmswc9midrcdg6qir2knvmraif-gsl-2.7.drv failed
View build log at '/var/log/guix/drvs/fw/jvvklmswc9midrcdg6qir2knvmraif-gsl-2.7.drv.gz'.
guix build: error: build of `/gnu/store/fwjvvklmswc9midrcdg6qir2knvmraif-gsl-2.7.drv' failed
--8<---------------cut here---------------end--------------->8---

[bug#54635] [PATCH 1/5] gnu: gsl: Force bootstrap when cross-compiling to riscv64-linux.

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 891 bytes --]

Arun Isaac schreef op do 31-03-2022 om 12:03 [+0530]:
> > > WDYT of making this unconditional?  Two benefits:
> > > 
> > >    * if Guix is ported to another new architecture,
> > >      then no changes are necessary to the package definition.
> > > 
> > >    * 'configure' and 'Makefile.in' are not source code,
> > >      and more difficult to audit for things like malware than
> > >      'configure.ac' and 'Makefile.am'.
> > 
> > This can be with a TODO for core-updates. gsl itself has about 2000
> > dependant packages.
> 
> I agree. That was my reasoning as well. If we agree that making it
> unconditional is the way forward, I can send another patch for
> core-updates after this patchset is pushed to master.

I started an e-mail thread about this at
<https://lists.gnu.org/archive/html/guix-devel/2022-03/msg00226.html>.

Greetings,
Maxime.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

[bug#54635] [PATCH 1/5] gnu: gsl: Force bootstrap when cross-compiling to riscv64-linux.

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 2213 bytes --]

On Thu, Mar 31, 2022 at 12:03:39PM +0530, Arun Isaac wrote:
> 
> >> >             ,@(cond
> >> > +              ((target-riscv64?)
> >> > +               '((add-after 'unpack 'force-bootstrap
> >> > +                   (lambda _
> >> > +                     ;; gsl ships with an old configure script that does not
> >> > +                     ;; support riscv64. Regenerate it.
> >> > +                     (delete-file "configure")))))
> >> > +
> >> 
> >> WDYT of making this unconditional?  Two benefits:
> >> 
> >>   * if Guix is ported to another new architecture,
> >>     then no changes are necessary to the package definition.
> >> 
> >>   * 'configure' and 'Makefile.in' are not source code,
> >>     and more difficult to audit for things like malware than
> >>     'configure.ac' and 'Makefile.am'.
> >
> > This can be with a TODO for core-updates. gsl itself has about 2000
> > dependant packages.
> 
> I agree. That was my reasoning as well. If we agree that making it
> unconditional is the way forward, I can send another patch for
> core-updates after this patchset is pushed to master.
> 
> > That said, I'm not convinced about unilaterally removing configure
> > unless we make it a policy to remove it. Also, I haven't had trouble
> > with building gsl on riscv64-linux without this patch.
> 
> Without the force-boostrap phase, the configure phase fails during
> cross-compilation.
> 
> --8<---------------cut here---------------start------------->8---
..snip..
> --8<---------------cut here---------------end--------------->8---

I somehow missed that when I was testing it before. It builds fine
natively on riscv64-linux and I haven't tested cross-building from
riscv64-linux to another architecture. I think for now we can tag it as
(target-riscv64?) and (%current-target-system) so it only takes effect
when needed.

-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[bug#54635] [PATCH v2 1/5] gnu: gsl: Force bootstrap when cross-compiling to riscv64-linux.

[Arun Isaac]

* gnu/packages/maths.scm (gsl)[arguments]: Force autotools bootstrap when
cross-compiling to riscv64-linux.
[native-inputs]: Add autoconf, automake and libtool when cross-compiling to
riscv64-linux.
---
 gnu/packages/maths.scm | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/gnu/packages/maths.scm b/gnu/packages/maths.scm
index ecb85642ec..a05f89cb4e 100644
--- a/gnu/packages/maths.scm
+++ b/gnu/packages/maths.scm
@@ -19,7 +19,7 @@
 ;;; Copyright © 2017 Nikita <nikita@n0.is>
 ;;; Copyright © 2017 Ben Woodcroft <donttrustben@gmail.com>
 ;;; Copyright © 2017 Theodoros Foradis <theodoros@foradis.org>
-;;; Copyright © 2017, 2019 Arun Isaac <arunisaac@systemreboot.net>
+;;; Copyright © 2017, 2019, 2022 Arun Isaac <arunisaac@systemreboot.net>
 ;;; Copyright © 2017–2021 Tobias Geerinckx-Rice <me@tobias.gr>
 ;;; Copyright © 2017 Dave Love <me@fx@gnu.org>
 ;;; Copyright © 2018, 2019, 2020, 2021 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
@@ -533,6 +533,13 @@ (define-public gsl
          #:phases
          (modify-phases %standard-phases
            ,@(cond
+              ((target-riscv64?)
+               '((add-after 'unpack 'force-bootstrap
+                   (lambda _
+                     ;; gsl ships with an old configure script that does not
+                     ;; support riscv64. Regenerate it.
+                     (delete-file "configure")))))
+
               ((or (string-prefix? "aarch64" system)
                    (string-prefix? "powerpc" system))
                ;; Some sparse matrix tests are failing on AArch64 and PowerPC:
@@ -568,6 +575,10 @@ (define-public gsl
                         (string-append "exit (77);\n" all)))))))
 
               (else '()))))))
+    (native-inputs
+     (if (target-riscv64?)
+         (list autoconf automake libtool)
+         '()))
     (home-page "https://www.gnu.org/software/gsl/")
     (synopsis "Numerical library for C and C++")
     (description
-- 
2.34.0

[bug#54635] [PATCH v2 2/5] gnu: htslib: Add bzip2 and xz to inputs.

[Arun Isaac]

htslib links to libbz2 from bzip2 and liblzma from xz. Therefore, bzip2 and xz
should be listed in inputs even though they are already present implicitly in
native-inputs. Else, cross-compilation will fail.

* gnu/packages/bioinformatics.scm (htslib)[inputs]: Add bzip2 and xz.
---
 gnu/packages/bioinformatics.scm | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/gnu/packages/bioinformatics.scm b/gnu/packages/bioinformatics.scm
index 4e1028b3ce..2f6f3efb66 100644
--- a/gnu/packages/bioinformatics.scm
+++ b/gnu/packages/bioinformatics.scm
@@ -8,7 +8,7 @@
 ;;; Copyright © 2016, 2020, 2022 Marius Bakke <marius@gnu.org>
 ;;; Copyright © 2016, 2018 Raoul Bonnal <ilpuccio.febo@gmail.com>
 ;;; Copyright © 2017, 2018 Tobias Geerinckx-Rice <me@tobias.gr>
-;;; Copyright © 2017, 2021 Arun Isaac <arunisaac@systemreboot.net>
+;;; Copyright © 2017, 2021, 2022 Arun Isaac <arunisaac@systemreboot.net>
 ;;; Copyright © 2018 Joshua Sierles, Nextjournal <joshua@nextjournal.com>
 ;;; Copyright © 2018 Gábor Boskovits <boskovits@gmail.com>
 ;;; Copyright © 2018, 2019, 2020, 2021 Mădălin Ionel Patrașcu <madalinionel.patrascu@mdc-berlin.de>
@@ -4788,7 +4788,7 @@ (define-public htslib
                            "--enable-libcurl"
                            "--enable-s3")))
     (inputs
-     (list curl openssl))
+     (list bzip2 curl openssl xz))
     ;; This is referred to in the pkg-config file as a required library.
     (propagated-inputs
      (list zlib))
-- 
2.34.0

[bug#54635] [PATCH v2 3/5] gnu: atomic-queue: Run tests correctly.

[Arun Isaac]

* gnu/packages/cpp.scm (atomic-queue)[arguments]: In the check phase, run
tests instead of returning a function to run them.
---
 gnu/packages/cpp.scm | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/gnu/packages/cpp.scm b/gnu/packages/cpp.scm
index bb7f5ab95a..eb48902034 100644
--- a/gnu/packages/cpp.scm
+++ b/gnu/packages/cpp.scm
@@ -1297,8 +1297,7 @@ (define-public atomic-queue
          (replace 'check
            (lambda* (#:key tests? #:allow-other-keys)
              (when tests?
-               (lambda _
-                 (invoke "make" "run_tests")))))
+               (invoke "./tests"))))
          (replace 'install
            (lambda* (#:key outputs #:allow-other-keys)
              (copy-recursively "../source/include/atomic_queue"
-- 
2.34.0

[bug#54635] [PATCH v2 4/5] gnu: atomic-queue: Do not look for boost when cross-compiling.

[Arun Isaac]

* gnu/packages/cpp.scm (atomic-queue)[arguments]: When cross-compiling, add a
do-not-check-for-boost phase.
---
 gnu/packages/cpp.scm | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/gnu/packages/cpp.scm b/gnu/packages/cpp.scm
index eb48902034..b0d4fdeb3e 100644
--- a/gnu/packages/cpp.scm
+++ b/gnu/packages/cpp.scm
@@ -1294,6 +1294,15 @@ (define-public atomic-queue
      `(#:configure-flags '("-Dbenchmarks=false")
        #:phases
        (modify-phases %standard-phases
+         ,@(if (%current-target-system)
+               `(;; boost is a test dependency. We don't run tests when
+                 ;; cross-compiling. Disable all targets that depend on it.
+                 (add-after 'unpack 'do-not-check-for-boost
+                   (lambda _
+                     (substitute* "meson.build"
+                       (("unit_test_framework = [^\n]*" all)
+                        "unit_test_framework = disabler()")))))
+               '())
          (replace 'check
            (lambda* (#:key tests? #:allow-other-keys)
              (when tests?
-- 
2.34.0

[bug#54635] [PATCH v2 5/5] gnu: Add wfmash.

[Arun Isaac]

* gnu/packages/bioinformatics.scm (wfmash): New variable.
---
 gnu/packages/bioinformatics.scm | 51 +++++++++++++++++++++++++++++++++
 1 file changed, 51 insertions(+)

diff --git a/gnu/packages/bioinformatics.scm b/gnu/packages/bioinformatics.scm
index 2f6f3efb66..500ba4ac60 100644
--- a/gnu/packages/bioinformatics.scm
+++ b/gnu/packages/bioinformatics.scm
@@ -16107,3 +16107,54 @@ (define-public ccwl
 @acronym{EDSL, Embedded Domain Specific Language} in the Scheme programming
 language.")
     (license license:gpl3+)))
+
+(define-public wfmash
+  (package
+    (name "wfmash")
+    (version "0.8.1")
+    (source
+     (origin
+       (method url-fetch)
+       (uri (string-append "https://github.com/ekg/wfmash/releases/download/v"
+                           version "/wfmash-v" version ".tar.gz"))
+       (sha256
+        (base32
+         "031cm1arpfckvihb28vlk69mirpnmlag81zcscfba1bac58wvr7c"))
+       (modules '((guix build utils)))
+       (snippet
+        '(begin
+           ;; Unbundle atomic-queue.
+           (delete-file-recursively "src/common/atomic_queue")
+           (substitute* "src/align/include/computeAlignments.hpp"
+             (("\"common/atomic_queue/atomic_queue.h\"")
+              "<atomic_queue/atomic_queue.h>"))))))
+    (build-system cmake-build-system)
+    (arguments
+     `(#:tests? #f                      ; no tests
+       #:phases
+       (modify-phases %standard-phases
+         ;; Remove compiler flags and checks specific to x86 when not
+         ;; targeting it.
+         ,@(if (target-x86-64?)
+               '()
+               '((add-after 'unpack 'remove-x86-specific-compile-flags
+                   (lambda _
+                     (substitute* (find-files "." "CMakeLists\\.txt")
+                       (("-mcx16") "")
+                       (("-march=native") ""))
+                     (substitute* "src/common/dset64.hpp"
+                       (("!__x86_64__") "0")))))))))
+    (inputs
+     (list atomic-queue
+           gsl
+           htslib
+           jemalloc
+           zlib))
+    (synopsis "Base-accurate DNA sequence aligner")
+    (description "@code{wfmash} is a DNA sequence read mapper based on mash
+distances and the wavefront alignment algorithm.  It is a fork of MashMap that
+implements base-level alignment via the wflign tiled wavefront global
+alignment algorithm.  It completes MashMap with a high-performance alignment
+module capable of computing base-level alignments for very large sequences.")
+    (home-page "https://github.com/ekg/wfmash")
+    (license license:expat)))
-- 
2.34.0

[bug#54635] [PATCH v2 5/5] gnu: Add wfmash.

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 333 bytes --]

Arun Isaac schreef op do 31-03-2022 om 12:58 [+0530]:
> +                       (("-march=native") ""))

This is also wrong for x86 systems because it makes the build non-
reproducible.  Also, has upstream been informed about some of the
compiler flags being architecture-specific?

Greetings,
Maxime.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

[bug#54635] [PATCH v2 5/5] gnu: Add wfmash.

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 822 bytes --]

On Thu, Mar 31, 2022 at 01:34:29PM +0200, Maxime Devos wrote:
> Arun Isaac schreef op do 31-03-2022 om 12:58 [+0530]:
> > +                       (("-march=native") ""))
> 
> This is also wrong for x86 systems because it makes the build non-
> reproducible.  Also, has upstream been informed about some of the
> compiler flags being architecture-specific?

I'm pretty sure upstream is aware of it, and the -mcx16 flag. That whole
phase doesn't need to be non-x86_64 only, upstream prefers it that way
to get fater results but IMO it would be fine to move it into a snippet.


-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[bug#54635] [PATCH v2 5/5] gnu: Add wfmash.

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 1096 bytes --]

Efraim Flashner schreef op do 31-03-2022 om 15:18 [+0300]:
> > Arun Isaac schreef op do 31-03-2022 om 12:58 [+0530]:
> > > +                       (("-march=native") ""))
> > 
> > This is also wrong for x86 systems because it makes the build non-
> > reproducible.  Also, has upstream been informed about some of the
> > compiler flags being architecture-specific?
> 
> I'm pretty sure upstream is aware of it, and the -mcx16 flag. That
> whole phase doesn't need to be non-x86_64 only, upstream prefers it
> that way to get fater results

wfmash could be written to detect CPU features at runtime and there is
also --tune.  Also, upstream preferring march=native does not make the
build reproducible.

> but IMO it would be fine to move it into a snippet.

It does not have to be in a snippet, it just needs to be reproducible
(so no march=native, whether on x86 or not).

Upstream seems to be aware of the non-x86
(https://github.com/ekg/wfmash/issues/125) but they do not seem to be
aware of the problems with march=native.

Greetings,
Maxime

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

[bug#54635] [PATCH v2 5/5] gnu: Add wfmash.

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 1499 bytes --]

On Thu, Mar 31, 2022 at 03:07:49PM +0200, Maxime Devos wrote:
> Efraim Flashner schreef op do 31-03-2022 om 15:18 [+0300]:
> > > Arun Isaac schreef op do 31-03-2022 om 12:58 [+0530]:
> > > > +                       (("-march=native") ""))
> > > 
> > > This is also wrong for x86 systems because it makes the build non-
> > > reproducible.  Also, has upstream been informed about some of the
> > > compiler flags being architecture-specific?
> > 
> > I'm pretty sure upstream is aware of it, and the -mcx16 flag. That
> > whole phase doesn't need to be non-x86_64 only, upstream prefers it
> > that way to get fater results
> 
> wfmash could be written to detect CPU features at runtime and there is
> also --tune.  Also, upstream preferring march=native does not make the
> build reproducible.
> 
> > but IMO it would be fine to move it into a snippet.
> 
> It does not have to be in a snippet, it just needs to be reproducible
> (so no march=native, whether on x86 or not).
> 
> Upstream seems to be aware of the non-x86
> (https://github.com/ekg/wfmash/issues/125) but they do not seem to be
> aware of the problems with march=native.

I will let them know. I also added a patch which runs a test suite based
on the github action.


-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

bug#54635: [PATCH v2 5/5] gnu: Add wfmash.

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 1862 bytes --]

On Thu, Mar 31, 2022 at 03:07:49PM +0200, Maxime Devos wrote:
> Efraim Flashner schreef op do 31-03-2022 om 15:18 [+0300]:
> > > Arun Isaac schreef op do 31-03-2022 om 12:58 [+0530]:
> > > > +                       (("-march=native") ""))
> > > 
> > > This is also wrong for x86 systems because it makes the build non-
> > > reproducible.  Also, has upstream been informed about some of the
> > > compiler flags being architecture-specific?
> > 
> > I'm pretty sure upstream is aware of it, and the -mcx16 flag. That
> > whole phase doesn't need to be non-x86_64 only, upstream prefers it
> > that way to get fater results
> 
> wfmash could be written to detect CPU features at runtime and there is
> also --tune.  Also, upstream preferring march=native does not make the
> build reproducible.
> 
> > but IMO it would be fine to move it into a snippet.
> 
> It does not have to be in a snippet, it just needs to be reproducible
> (so no march=native, whether on x86 or not).

I suppose not, but the -mcx16 should be in the snippet, since it adds
compiler flags which only work on some architectures. And while we're at
it we can do the -march=native one too.

> Upstream seems to be aware of the non-x86
> (https://github.com/ekg/wfmash/issues/125) but they do not seem to be
> aware of the problems with march=native.
> 

I've pushed these patches with a few tweaks and an additional patch to
run a test suite based on the github workflow in the repository. The
entire check phase takes ~2 minutes on my pinebook pro, so aarch64
doesn't seem to need to skip some tests like riscv64 does.


-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[bug#54635] [PATCH v2 5/5] gnu: Add wfmash.

[Arun Isaac]

Thanks for reviewing and merging this patchset!

>> wfmash could be written to detect CPU features at runtime and there is
>> also --tune.

wfmash's speed is one of the critical features it brings to the
table. We might have rendered the package more or less unusable by
disabling CPU optimizations.

I could try talking to upstream about detecting and using CPU features
at runtime. But, I don't know too much about the topic. Any quick links
I can share with them?

> I've pushed these patches with a few tweaks and an additional patch to
> run a test suite based on the github workflow in the repository. The
> entire check phase takes ~2 minutes on my pinebook pro, so aarch64
> doesn't seem to need to skip some tests like riscv64 does.

I think it is better if upstream provides us a `make check' target to
run these tests. I have asked them:
https://github.com/ekg/wfmash/issues/130 Hard-coding their CI tests
makes the package fragile. In the future, upstream could change the
tests they run.

[bug#54635] [PATCH v2 5/5] gnu: Add wfmash.

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 872 bytes --]

Arun Isaac schreef op vr 01-04-2022 om 13:10 [+0530]:
> wfmash's speed is one of the critical features it brings to the
> table. We might have rendered the package more or less unusable by
> disabling CPU optimizations.
> 
> I could try talking to upstream about detecting and using CPU features
> at runtime. But, I don't know too much about the topic. Any quick links
> I can share with them?

For the concept, maybe
<https://hpc.guix.info/blog/2022/01/tuning-packages-for-a-cpu-micro-architecture/>
, which is about combining high performance with reproducibility using
the (Guix-specific) --tune package transformation.  To implement run-
time detection, the following could be useful:

* Guix article:
<https://hpc.guix.info/blog/2018/01/pre-built-binaries-vs-performance/>
* LWN article: <https://lwn.net/Articles/691932/>.

Greetings,
Maxime.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

[bug#54635] [PATCH v2 5/5] gnu: Add wfmash.

[Arun Isaac]

Hi Maxime,

> For the concept, maybe
> <https://hpc.guix.info/blog/2022/01/tuning-packages-for-a-cpu-micro-architecture/>
> , which is about combining high performance with reproducibility using
> the (Guix-specific) --tune package transformation.  To implement run-
> time detection, the following could be useful:
>
> * Guix article:
> <https://hpc.guix.info/blog/2018/01/pre-built-binaries-vs-performance/>
> * LWN article: <https://lwn.net/Articles/691932/>.

This is helpful. I will read and pass it on to wfmash upstream.

Thanks,
Arun