Re: OpenBLAS and performance

[Dave Love <fx@gnu.org>]

Ludovic Courtès <ludovic.courtes@inria.fr> writes:

> Hello,
>
> Dave Love <fx@gnu.org> skribis:
>
>> Fedora sensibly builds separately-named libraries for different flavours
>> <https://apps.fedoraproject.org/packages/openblas/sources/>, but I'd
>> argue also for threaded versions being available with the generic soname
>> in librray sub-directories.  There's some discussion and measurements
>> (apologies if I've referenced it before) at
>> <https://loveshack.fedorapeople.org/blas-subversion.html>
>
> I like the idea of an ‘update-alternative’ kind of approach for
> interchangeable implementations.

/etc/ld.so.conf.d normally provides a clean way to flip the default, but
that isn't available in Guix as far as I remember.

> Unfortunately my understanding is that implementations aren’t entirely
> interchangeable, especially for LAPACK (not sure about BLAS), because
> BLIS, OpenBLAS, etc. implement slightly different subsets of netlib
> LAPACK, AIUI.

LAPACK may add new routines, but you can always link with the vanilla
Netlib version, and openblas is currently only one release behind.  The
LAPACK release notes I've seen aren't very helpful for following that.
The important requirement is fast GEMM from the optimized BLAS.  I
thought BLIS just provided the BLAS layer, which is quite stable, isn't
it?

> Packages also often check for specific implementations in
> their configure/CMakeLists.txt rather than just for “BLAS” or “LAPACK”.

It doesn't matter what they're built against when you dynamically load a
compatible version.  (You'd hope a build system would be able to find
arbitrary BLAS but I'm too familiar with cmake pain.)  The openblas
compatibility hack basically just worked on an RHEL6 cluster when I
maintained it.

> FlexiBLAS, which Eric mentioned, looks interesting because it’s designed
> specifically for that purpose.  Perhaps worth giving it a try.

I see it works by wrapping everything, which I wanted to avoid.  Also
it's GPL, which restricts its use.  What's the advantage over just
having implementations which are directly interchangeable at load time?

> Besides, it would be good to have a BLAS/LAPACK policy in Guix.  We
> should at least agree (1) on default BLAS/LAPACK implementations, (2)
> possibly on a naming scheme for variants based on a different
> implementation.

Yes, but the issue is wider than just linear algebra.  It seems to
reflect tension between Guix' approach (as I understand it) and the late
binding I expect to use.  There are potentially other libraries with
similar micro-architecture-specific issues, and the related one of
profiling/debugging versions.  I don't know how much of a real problem
there really is, and it would be good to know if someone has addressed
this.

It's a reason I'm currently not convinced about the trades-off with
Guix, and don't go along with the "reproducibility" mantra.  Obviously
I'm not writing Guix off, though, and I hope the discussion is useful.

> For #1 we should probably favor implementations that support run-time
> implementation selection such as OpenBLAS (or the coming BLIS release).
>
> Thoughts?
>
> Ludo’.

Yes, but even with dynamic dispatch you need to account for situations
like we currently have on x86_64 with OB not supporting the latest
micro-architecture, and it only works on x86 with OB.  You may also want
to avoid overhead -- see FFTW's advice for packaging.  Oh for SIMD
hwcaps...