bug#63986: Julia is very slow

[Simon Tournier <zimon.toutoune@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 1803 bytes --]

Hi,

On Thu, 22 Jun 2023 at 19:25, Efraim Flashner <efraim@flashner.co.il> wrote:

> (ins)efraim@3900XT ~/workspace/guix$ cat /gnu/store/v6z5ykkjfzbc72x1x900xflspqc5wd5r-openblas-ilp64-0.3.20/lib/pkgconfig/openblas.pc
> libdir=/gnu/store/v6z5ykkjfzbc72x1x900xflspqc5wd5r-openblas-ilp64-0.3.20/lib
> includedir=/gnu/store/v6z5ykkjfzbc72x1x900xflspqc5wd5r-openblas-ilp64-0.3.20/include
> openblas_config= USE_64BITINT= DYNAMIC_ARCH=1 DYNAMIC_OLDER=1 NO_CBLAS= NO_LAPACK= NO_LAPACKE= NO_AFFINITY=1 USE_OPENMP= generic MAX_THREADS=128
> version=0.3.20
> extralib=-lm -lpthread -lgfortran -lm -lpthread -lgfortran
> Name: openblas
> Description: OpenBLAS is an optimized BLAS library based on GotoBLAS2 1.13 BSD version
> Version: ${version}
> URL: https://github.com/xianyi/OpenBLAS
> Libs: -L${libdir} -lopenblas
> Libs.private: ${extralib}
> Cflags: -I${includedir}
>
> Looks like it should be "LIBBLAS=-lopenblas"

I propose to tweak openblas-ilp64.  Currently it looks like:

--8<---------------cut here---------------start------------->8---
$ tree $(guix build openblas-ilp64)/lib
/gnu/store/v6z5ykkjfzbc72x1x900xflspqc5wd5r-openblas-ilp64-0.3.20/lib
├── cmake
│   └── openblas
│       ├── OpenBLASConfig.cmake
│       └── OpenBLASConfigVersion.cmake
├── libopenblas_ilp64p-r0.3.20.so
├── libopenblas_ilp64.so -> libopenblas_ilp64p-r0.3.20.so
├── libopenblas_ilp64.so.0 -> libopenblas_ilp64p-r0.3.20.so
└── pkgconfig
    └── openblas.pc
--8<---------------cut here---------------end--------------->8---

which is inconsistent with pkgconfig as you noticed above.  Therefore, I
am proposing the addition of a symlink of libopenblas_ilp64p.so to
libopenblas.so.  For instance this attached patch.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: p3-openblas-ilp64.patch --]
[-- Type: text/x-diff, Size: 1396 bytes --]

From 0c8c7e9371d11972f4a6012ef503ef3057c91364 Mon Sep 17 00:00:00 2001
Message-Id: <0c8c7e9371d11972f4a6012ef503ef3057c91364.1687459454.git.zimon.toutoune@gmail.com>
From: Simon Tournier <zimon.toutoune@gmail.com>
Date: Thu, 22 Jun 2023 20:31:26 +0200
Subject: [PATCH v3 1/2] gnu: openblas-ilp64: Install symlink to libopenblas.

* gnu/packages/maths.scm (openblas-ilp64)[arguments]: Add phases for
installing symlink to libopenblas.
---
 gnu/packages/maths.scm | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/gnu/packages/maths.scm b/gnu/packages/maths.scm
index f5a2181905..5c39ab8b94 100644
--- a/gnu/packages/maths.scm
+++ b/gnu/packages/maths.scm
@@ -4646,7 +4646,13 @@ (define-public openblas-ilp64
      (substitute-keyword-arguments (package-arguments openblas)
        ((#:make-flags flags #~'())
         #~(append (list "INTERFACE64=1" "LIBNAMESUFFIX=ilp64")
-                 #$flags))))
+                 #$flags))
+       ((#:phases phases)
+        #~(modify-phases #$phases
+            (add-after 'install 'install-symlink
+              (lambda _
+                (symlink "libopenblas_ilp64.so"
+                         (string-append #$output "/lib/libopenblas.so"))))))))
     (synopsis "Optimized BLAS library based on GotoBLAS (ILP64 version)")
     (license license:bsd-3)))
 

base-commit: 37c2e94cec6cb8b5e0e93e7b6c712c3b187ca5db
-- 
2.38.1


[-- Attachment #3: Type: text/plain, Size: 65 bytes --]


Then, the patch for Julia looks like the other attached patch.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #4: p3.patch --]
[-- Type: text/x-diff, Size: 1823 bytes --]

From 8563a738703d133b44ae05b91b7448ca56d280b5 Mon Sep 17 00:00:00 2001
Message-Id: <8563a738703d133b44ae05b91b7448ca56d280b5.1687459454.git.zimon.toutoune@gmail.com>
In-Reply-To: <0c8c7e9371d11972f4a6012ef503ef3057c91364.1687459454.git.zimon.toutoune@gmail.com>
References: <0c8c7e9371d11972f4a6012ef503ef3057c91364.1687459454.git.zimon.toutoune@gmail.com>
From: Simon Tournier <zimon.toutoune@gmail.com>
Date: Thu, 22 Jun 2023 17:45:50 +0200
Subject: [PATCH v3 2/2] gnu: julia: Conditionally use openblas with ILP64
 support.

Fixes <https://bugs.gnu.org/63986>.
Reported by Cayetano Santos <csantosb@inventati.org>.

* gnu/packages/julia.scm (julia)[arguments]<#:make-flags>: Conditionally use
64-bit BLAS for x86-64 target.
[inputs]: Conditionally replace openblas by openblas-ilp64 for x86-64 target.
---
 gnu/packages/julia.scm | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/gnu/packages/julia.scm b/gnu/packages/julia.scm
index ba54175822..17a27f4928 100644
--- a/gnu/packages/julia.scm
+++ b/gnu/packages/julia.scm
@@ -479,9 +479,9 @@ (define-public julia
          "NO_GIT=1"             ; build from release tarball.
          "USE_GPL_LIBS=1"       ; proudly
 
-         ,@(if (target-aarch64?)
-             `("USE_BLAS64=0")
-             '())
+         ,@(if (target-x86-64?)
+               `("USE_BLAS64=1")
+               '())
 
          "LIBBLAS=-lopenblas"
          "LIBBLASNAME=libopenblas"
@@ -513,7 +513,9 @@ (define-public julia
        ("llvm" ,llvm-julia)
        ("mbedtls-apache" ,mbedtls-apache)
        ("mpfr" ,mpfr)
-       ("openblas" ,openblas)
+       ,@(if (target-x86-64?)
+             `(("openblas" ,openblas-ilp64))
+             `(("openblas" ,openblas)))
        ("openlibm" ,openlibm)
        ("p7zip" ,p7zip)
        ("pcre2" ,pcre2)
-- 
2.38.1


[-- Attachment #5: Type: text/plain, Size: 167 bytes --]


WDYT?

Well, I am running all the testsuite for checking if all is correct.
Somehow, I think this is direction.  Otherwise, what would you suggest?

Cheers,
simon