[bug#31434] [PATCH 0/2] OpenMPI 3.0

[bug#31434] [PATCH 0/2] OpenMPI 3.0

[Ludovic Courtès]

Hello,

The attached patch updates OpenMPI to 3.0.1.  The ‘superlu-dist’ upgrade
was necessary to get there.

I was able to rebuild everything that depends on it on x86_64, except
‘dealii-openmpi’, where GCC would eat all the memory of the machine
before completing (I suppose it might work with --cores=1 but I haven’t
tried.)

Ludo’.

Ludovic Courtès (2):
  gnu: superlu-dist: Update to 5.3.0.
  gnu: openmpi: Update to 3.0.1.

 gnu/packages/maths.scm | 87 ++++++++++++++++++++++++++++++++++--------
 gnu/packages/mpi.scm   |  8 ++--
 2 files changed, 76 insertions(+), 19 deletions(-)

-- 
2.17.0

[bug#31434] [PATCH 1/2] gnu: superlu-dist: Update to 5.3.0.

[Ludovic Courtès]

From: Ludovic Courtès <ludovic.courtes@inria.fr>

* gnu/packages/maths.scm (superlu-dist): Update to 5.3.0.
[source](snippet): Adapt file name and function names to new version.
Remove 'delete-file' call.
---
 gnu/packages/maths.scm | 13 ++++++-------
 1 file changed, 6 insertions(+), 7 deletions(-)

diff --git a/gnu/packages/maths.scm b/gnu/packages/maths.scm
index 2e6f79ad5..d2edecc54 100644
--- a/gnu/packages/maths.scm
+++ b/gnu/packages/maths.scm
@@ -2109,35 +2109,34 @@ also provides threshold-based ILU factorization preconditioners.")
 (define-public superlu-dist
   (package
     (name "superlu-dist")
-    (version "3.3")
+    (version "5.3.0")
     (source
      (origin
        (method url-fetch)
        (uri (string-append "http://crd-legacy.lbl.gov/~xiaoye/SuperLU/"
                            "superlu_dist_" version ".tar.gz"))
        (sha256
-        (base32 "1hnak09yxxp026blq8zhrl7685yip16svwngh1wysqxf8z48vzfj"))
-              (modules '((guix build utils)))
+        (base32 "0ja5ihqivkda1wd58y4lmzvmwssm9g91f70c5q0fzwhng6580h6y"))
+       (modules '((guix build utils)))
        (snippet
         ;; Replace the non-free implementation of MC64 with a stub
         '(begin
            (use-modules (ice-9 regex)
                         (ice-9 rdelim))
-           (call-with-output-file "SRC/mc64ad.c"
+           (call-with-output-file "SRC/mc64ad_dist.c"
              (lambda (port)
                (display "
 #include <stdio.h>
 #include <stdlib.h>
-void mc64id_(int *a) {
+void mc64id_dist(int *a) {
   fprintf (stderr, \"SuperLU_DIST: non-free MC64 not available.  Aborting.\\n\");
   abort ();
 }
-void mc64ad_ (int *a, int *b, int *c, int *d, int *e, double *f, int *g,
+void mc64ad_dist (int *a, int *b, int *c, int *d, int *e, double *f, int *g,
               int *h, int *i, int *j, int *k, double *l, int *m, int *n) {
   fprintf (stderr, \"SuperLU_DIST: non-free MC64 not available.  Aborting.\\n\");
   abort ();
 }\n" port)))
-           (delete-file "SRC/mc64ad.f.bak")
            (substitute* "SRC/util.c"    ;adjust default algorithm
              (("RowPerm[[:blank:]]*=[[:blank:]]*LargeDiag")
               "RowPerm = NOROWPERM"))))
-- 
2.17.0

[bug#31434] [PATCH 2/2] gnu: openmpi: Update to 3.0.1.

[Ludovic Courtès]

From: Ludovic Courtès <ludovic.courtes@inria.fr>

* gnu/packages/mpi.scm (openmpi): Update to 3.0.1.
[arguments]: In 'remove-absolute' phase, adjust romio file name.
* gnu/packages/maths.scm (scalapack)[arguments]: Add
'set-test-environment' phase.
(slepc-openmpi)[arguments]: Likewise.
(mumps-openmpi)[arguments]: Likewise.
(superlu-dist)[arguments]: In 'check' phase, set
'OMPI_MCA_plm_rsh_agent'.
(scotch)[arguments]: In 'build-esmumps', likewise.
(p4est)[arguments]: Add 'set-test-environment' phase.
(hypre-openmpi)[arguments]: Likewise.
---
 gnu/packages/maths.scm | 74 +++++++++++++++++++++++++++++++++++++-----
 gnu/packages/mpi.scm   |  8 ++---
 2 files changed, 70 insertions(+), 12 deletions(-)

diff --git a/gnu/packages/maths.scm b/gnu/packages/maths.scm
index d2edecc54..bc2e1be37 100644
--- a/gnu/packages/maths.scm
+++ b/gnu/packages/maths.scm
@@ -554,7 +554,16 @@ problems in numerical linear algebra.")
        ("fortran" ,gfortran)
        ("lapack" ,lapack)))             ;for testing only
     (arguments
-     `(#:configure-flags `("-DBUILD_SHARED_LIBS:BOOL=YES")))
+     `(#:configure-flags `("-DBUILD_SHARED_LIBS:BOOL=YES")
+       #:phases (modify-phases %standard-phases
+                  (add-before 'check 'set-test-environment
+                    (lambda _
+                      ;; By default, running the test suite would fail because
+                      ;; 'ssh' could not be found in $PATH.  Define this
+                      ;; variable to placate Open MPI without adding a
+                      ;; dependency on OpenSSH (the agent isn't used anyway.)
+                      (setenv "OMPI_MCA_plm_rsh_agent" (which "cat"))
+                      #t)))))
     (home-page "http://www.netlib.org/scalapack/")
     (synopsis "Library for scalable numerical linear algebra")
     (description
@@ -1786,6 +1795,18 @@ arising after the discretization of partial differential equations.")
 (define-public slepc-openmpi
   (package (inherit slepc)
     (name "slepc-openmpi")
+    (arguments
+     (substitute-keyword-arguments (package-arguments slepc)
+       ((#:phases phases '%standard-phases)
+        `(modify-phases ,phases
+           (add-before 'check 'set-test-environment
+             (lambda _
+               ;; By default, running the test suite would fail because 'ssh'
+               ;; could not be found in $PATH.  Define this variable to
+               ;; placate Open MPI without adding a dependency on OpenSSH (the
+               ;; agent isn't used anyway.)
+               (setenv "OMPI_MCA_plm_rsh_agent" (which "cat"))
+               #t))))))
     (inputs
      `(("mpi" ,openmpi)
        ("arpack" ,arpack-ng-openmpi)
@@ -1955,11 +1976,18 @@ sparse system of linear equations A x = b using Guassian elimination.")
      (substitute-keyword-arguments (package-arguments mumps)
        ((#:phases phases)
         `(modify-phases ,phases
-           (replace
-            'check
-            (lambda _
-              ((assoc-ref ,phases 'check)
-               #:exec-prefix '("mpirun" "-n" "2"))))))))
+           (add-before 'check 'set-test-environment
+             (lambda _
+               ;; By default, running the test suite would fail because 'ssh'
+               ;; could not be found in $PATH.  Define this variable to
+               ;; placate Open MPI without adding a dependency on OpenSSH (the
+               ;; agent isn't used anyway.)
+               (setenv "OMPI_MCA_plm_rsh_agent" (which "cat"))
+               #t))
+           (replace 'check
+             (lambda _
+               ((assoc-ref ,phases 'check)
+                #:exec-prefix '("mpirun" "-n" "2"))))))))
     (synopsis "Multifrontal sparse direct solver (with MPI)")))
 
 (define-public mumps-metis-openmpi
@@ -2201,6 +2229,11 @@ CDEFS       = -DAdd_"
              #t))
          (replace 'check
            (lambda _
+             ;; By default, running the test suite would fail because 'ssh'
+             ;; could not be found in $PATH.  Define this variable to placate
+             ;; Open MPI without adding a dependency on OpenSSH (the agent
+             ;; isn't used anyway.)
+             (setenv "OMPI_MCA_plm_rsh_agent" (which "cat"))
              (with-directory-excursion "EXAMPLE"
                (and
                 (zero? (system* "mpirun" "-n" "2"
@@ -2295,6 +2328,12 @@ YACC = bison -pscotchyy -y -b y
          (add-after
           'build 'build-esmumps
           (lambda _
+            ;; By default, running the test suite would fail because 'ssh'
+            ;; could not be found in $PATH.  Define this variable to placate
+            ;; Open MPI without adding a dependency on OpenSSH (the agent
+            ;; isn't used anyway.)
+            (setenv "OMPI_MCA_plm_rsh_agent" (which "cat"))
+
             (zero? (system* "make"
                             (format #f "-j~a" (parallel-job-count))
                             "esmumps"))))
@@ -2474,7 +2513,16 @@ schemes.")
                                            " -lopenblas")
                            ,(string-append "LAPACK_LIBS=-L"
                                            (assoc-ref %build-inputs "lapack")
-                                           " -llapack"))))
+                                           " -llapack"))
+       #:phases (modify-phases %standard-phases
+                  (add-before 'check 'set-test-environment
+                    (lambda _
+                      ;; By default, running the test suite would fail because
+                      ;; 'ssh' could not be found in $PATH.  Define this
+                      ;; variable to placate Open MPI without adding a
+                      ;; dependency on OpenSSH (the agent isn't used anyway.)
+                      (setenv "OMPI_MCA_plm_rsh_agent" (which "cat"))
+                      #t)))))
     (home-page "http://www.p4est.org")
     (synopsis "Adaptive mesh refinement on forests of octrees")
     (description
@@ -3531,7 +3579,17 @@ problems.")
      (substitute-keyword-arguments (package-arguments hypre)
        ((#:configure-flags flags)
         ``("--with-MPI"
-           ,@(delete "--without-MPI" ,flags)))))
+           ,@(delete "--without-MPI" ,flags)))
+       ((#:phases phases)
+        `(modify-phases ,phases
+           (add-before 'check 'set-test-environment
+             (lambda _
+               ;; By default, running the test suite would fail because 'ssh'
+               ;; could not be found in $PATH.  Define this variable to
+               ;; placate Open MPI without adding a dependency on OpenSSH (the
+               ;; agent isn't used anyway.)
+               (setenv "OMPI_MCA_plm_rsh_agent" (which "cat"))
+               #t))))))
     (synopsis "Parallel solvers and preconditioners for linear equations")
     (description
      "HYPRE is a software library of high performance preconditioners and
diff --git a/gnu/packages/mpi.scm b/gnu/packages/mpi.scm
index 5dfb26ae7..5496ea9c9 100644
--- a/gnu/packages/mpi.scm
+++ b/gnu/packages/mpi.scm
@@ -153,7 +153,7 @@ bind processes, and much more.")
 (define-public openmpi
   (package
     (name "openmpi")
-    (version "1.10.7")
+    (version "3.0.1")
     (source
      (origin
       (method url-fetch)
@@ -162,7 +162,7 @@ bind processes, and much more.")
                           "/downloads/openmpi-" version ".tar.bz2"))
       (sha256
        (base32
-        "142s1vny9gllkq336yafxayjgcirj2jv0ddabj879jgya7hyr2d0"))))
+        "0pbqrm5faf57nasy1s81wqivl7zvxmv8lzjh8hvb0f3qxv8m0d36"))))
     (build-system gnu-build-system)
     (inputs
      `(("hwloc" ,hwloc "lib")
@@ -205,8 +205,8 @@ bind processes, and much more.")
                         (("_ABSOLUTE") ""))
                       ;; Avoid valgrind (which pulls in gdb etc.).
                       (substitute*
-                          '("./ompi/mca/io/romio/src/io_romio_component.c")
-                        (("MCA_io_romio_COMPLETE_CONFIGURE_FLAGS")
+                          '("./ompi/mca/io/romio314/src/io_romio314_component.c")
+                        (("MCA_io_romio314_COMPLETE_CONFIGURE_FLAGS")
                          "\"[elided to reduce closure]\""))
                       #t))
                   (add-before 'build 'scrub-timestamps ;reproducibility
-- 
2.17.0

[bug#31434] [PATCH 0/2] OpenMPI 3.0

[Ludovic Courtès]

Hello Eric,

Ludovic Courtès <ludo@gnu.org> skribis:

> The attached patch updates OpenMPI to 3.0.1.  The ‘superlu-dist’ upgrade
> was necessary to get there.

Did you have a chance to look into it?  You mentioned on IRC that there
might be a better way than setting OMPI_MCA_plm_rsh_agent.

Thanks in advance!

Ludo’.

[bug#31434] [PATCH 0/2] OpenMPI 3.0

[Ludovic Courtès]

A somewhat related issue that a colleague pointed out is whether we
should build with ‘--enable-mpirun-prefix-by-default’, which README
documents like this:

--8<---------------cut here---------------start------------->8---
--enable-mpirun-prefix-by-default
  This option forces the "mpirun" command to always behave as if
  "--prefix $prefix" was present on the command line (where $prefix is
  the value given to the --prefix option to configure).  This prevents
  most rsh/ssh-based users from needing to modify their shell startup
  files to set the PATH and/or LD_LIBRARY_PATH for Open MPI on remote
  nodes.  Note, however, that such users may still desire to set PATH
  -- perhaps even in their shell startup files -- so that executables
  such as mpicc and mpirun can be found without needing to type long
  path names.  --enable-orterun-prefix-by-default is a synonym for
  this option.
--8<---------------cut here---------------end--------------->8---

It looks like something we may want.  Thoughts?

Ludo’.

bug#31434: [PATCH 0/2] OpenMPI 3.0

[Ludovic Courtès]

Hello,

ludovic.courtes@inria.fr (Ludovic Courtès) skribis:

> Ludovic Courtès <ludo@gnu.org> skribis:
>
>> The attached patch updates OpenMPI to 3.0.1.  The ‘superlu-dist’ upgrade
>> was necessary to get there.
>
> Did you have a chance to look into it?  You mentioned on IRC that there
> might be a better way than setting OMPI_MCA_plm_rsh_agent.

I went ahead and apply the patches as they were so we can have a recent
Open MPI.  We can always revisit the OMPI_MCA_plm_rsh_agent issue later.

Thanks,
Ludo’.