OpenBLAS and INTERFACE64=1

OpenBLAS and INTERFACE64=1

[Eric Brown]

Dear List,

During the course of creating a port for OpenMolcas, I discovered that
it requires the 64-bit interface to OpenBLAS. My understanding is that
this promotes INTEGER to be 64-bit, so that arrays longer than 2**32-1
are supported. It was easy enough to shoe-horn in INTERFACE64=1 to
OpenBLAS:

maths.scm (openblas):
---------------------
 ,@(let ((system (or (%current-target-system) (%current-system))))
      (cond
       ((string-prefix? "i686" system)
        '("DYNAMIC_ARCH=1"))
        ((string-prefix? "x86_64" system)
        '("DYNAMIC_ARCH=1" "INTERFACE64=1"))
       ;; On MIPS we force the "SICORTEX" TARGET, as for the other
       ;; two available MIPS targets special extended instructions
       ;; for Loongson cores are used.
       ((string-prefix? "mips" system)
        '("TARGET=SICORTEX" "INTERFACE64=1"))
       ;; On aarch64 force the generic 'armv8-a' target
       ((string-prefix? "aarch64" system)
        '("TARGET=ARMV8" "INTERFACE64=1"))
       (else '()))))

and OpenMolcas compiles and passes all tests.

I then set about to ensure that dependents also compiled without issue,
and in fact all do (at least, no *new* failures) except for python-numpy
and python-scipy, which Segmentation Fault on at least one test when
linked with an OpenBLAS defined with 64-bit integers.

Of course, it would be nice if each package can build, and also be
possible for Fortran routines to handle large arrays.

A few thoughts come to mind, perhaps in combination:

* add a variant to the current openblas package 

* add compiler flags such as -fdefault-integer-8 to gfortran

* create a new package, e.g. openblas64, perhaps with a suffix to the
  library name which distinguishes a 64-bit version from a 32-bit
  version

(I'd like to have e.g. python-scipy and OpenMolcas on my machine at the
same time, so I'm worried about collisions in case 32-bit must be
present sometimes. Though on my 64-bit bit machines, I can say that I
never want 32-bit limitations of data size.)

On the web, people have mentioned all of the above, even adding a suffix
"_64" to each function in the API.  I just don't see how this would be
workable given the number of packages that Guix aims to deliver.

Just a few thoughts which rekado has asked me to post to this list.
Hopefully someone has some ideas about the optimal way to proceed.

Best regards,
Eric

Re: OpenBLAS and INTERFACE64=1

[Pjotr Prins]

On Sun, Sep 09, 2018 at 05:25:13PM -0500, Eric Brown wrote:
> I then set about to ensure that dependents also compiled without issue,
> and in fact all do (at least, no *new* failures) except for python-numpy
> and python-scipy, which Segmentation Fault on at least one test when
> linked with an OpenBLAS defined with 64-bit integers.

The problem is that applications have to allow for the 64-bit
interface and many have not been designed for that. I still want to
convert GEMMA and it is a pain. Trust C/C++ and the mess around short,
ints and implicit casting.

> Of course, it would be nice if each package can build, and also be
> possible for Fortran routines to handle large arrays.
> 
> A few thoughts come to mind, perhaps in combination:
> 
> * add a variant to the current openblas package 


> * add compiler flags such as -fdefault-integer-8 to gfortran
> 
> * create a new package, e.g. openblas64, perhaps with a suffix to the
>   library name which distinguishes a 64-bit version from a 32-bit
>   version

Yup. That is the way forward. Convert packages one by one and see if
things break. I would not trust numpy/scipy at this point to handle
64-bit correctly unless they explicitly converted.

Pj.