R

R

[Andreas Enge]

Yesterday I submitted a bug report for R on arm:
   https://bugs.r-project.org/bugzilla/show_bug.cgi?id=16697https://bugs.r-project.org/bugzilla/show_bug.cgi?id=16697

With all the different R packages failing consequently, I hoped that fixing R
would reduce the number of failures. (On MIPS there is also a problem, but
not the same one.)

Andreas

Re: R

[Andreas Enge]

I did get a reply, but not using R, I do not quite know what to make of it:
   https://bugs.r-project.org/bugzilla/show_bug.cgi?id=16697
It would be nice if someone using R could help me out; if you send me an
ssh key, I could give you access to an arm machine.

To start, a probably simple question: Why do we use "--with-blas=openblas"?

Andreas

Re: R

[Pjotr Prins]

On Fri, Feb 05, 2016 at 11:00:40AM +0100, Andreas Enge wrote:
> I did get a reply, but not using R, I do not quite know what to make of it:
>    https://bugs.r-project.org/bugzilla/show_bug.cgi?id=16697

Anyone here programming in R?

> It would be nice if someone using R could help me out; if you send me an
> ssh key, I could give you access to an arm machine.
> 
> To start, a probably simple question: Why do we use "--with-blas=openblas"?

openblas is fast.

http://blog.nguyenvq.com/blog/2014/11/10/optimized-r-and-python-standard-blas-vs-atlas-vs-openblas-vs-mkl/

Re: R

[Ricardo Wurmus]

Andreas Enge <andreas@enge.fr> writes:

> I did get a reply, but not using R, I do not quite know what to make of it:
>    https://bugs.r-project.org/bugzilla/show_bug.cgi?id=16697
> It would be nice if someone using R could help me out; if you send me an
> ssh key, I could give you access to an arm machine.
>
> To start, a probably simple question: Why do we use "--with-blas=openblas"?

I did this when we switched from ATLAS to OpenBLAS (for substitutability
reasons).  It is possible I have misunderstood the meaning of this
configure flag.

Here’s the installation manual on building with a specific BLAS
implementation:

    https://cran.r-project.org/doc/manuals/r-release/R-admin.html#BLAS

According to this section,

    “R can make use of enhanced BLAS (Basic Linear Algebra Subprograms,
    http://www.netlib.org/blas/faq.html) routines. However, these have
    to be explicitly requested at configure time: R provides an internal
    BLAS which is well-tested and will be adequate for most uses of R.”

There’s a separate section for OpenBLAS, which also mentions that using
a “shared BLAS” is a preferred alternative to linking with a particular
BLAS implementation.  We already pass “--enable-BLAS-shlib”, so we
actually do use a shared BLAS, IIUC.

I guess we could drop the “--with-blas” flag and see if that helps.  I
remember struggling with this a lot in the past.  I’ll try building R
again without that flag and see if it still works.

I’m sorry if that’s what’s responsible for the failures, but it’s pretty
hard for me to understand how linking with a high-performance BLAS
implementation could have caused these build/test failures on other
platforms.  I’ll report back when I have a successful build or an
interesting failure to share.

~~ Ricardo

Re: R

[Ricardo Wurmus]

Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de> writes:

> There’s a separate section for OpenBLAS, which also mentions that using
> a “shared BLAS” is a preferred alternative to linking with a particular
> BLAS implementation.  We already pass “--enable-BLAS-shlib”, so we
> actually do use a shared BLAS, IIUC.
>
> I guess we could drop the “--with-blas” flag and see if that helps.  I
> remember struggling with this a lot in the past.  I’ll try building R
> again without that flag and see if it still works.

I dropped the flag and noticed that OpenBLAS is not mentioned at all (it
does retain a reference, but that’s expected when “openblas” is among
the inputs), so I also dropped “openblas” from the inputs and R built
just fine.

It’s a bit sad to me that we won’t be able to just use this BLAS
implementation, but if it fixes the build on ARM and MIPS I’m all for
it.

I can also confirm that dropping openblas from the R build “fixes” the
segfault when running

    x <- eigen(crossprod(matrix(rnorm(50 * 500), 50, 500)))

as reported here: https://github.com/xianyi/OpenBLAS/issues/703

So, I think it’s a good idea to build R without OpenBLAS on all
architectures for now.

~~ Ricardo

Re: R

[Kyle Meyer]

Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de> writes:

[...]

> I can also confirm that dropping openblas from the R build “fixes” the
> segfault when running
>
>     x <- eigen(crossprod(matrix(rnorm(50 * 500), 50, 500)))
>
> as reported here: https://github.com/xianyi/OpenBLAS/issues/703
>
> So, I think it’s a good idea to build R without OpenBLAS on all
> architectures for now.

Thank you, Ricardo.  I'm sorry I wasn't able make any progress on
figuring out what the underlying issue was there.

-- 
Kyle

Re: R

[Andreas Enge]

On Fri, Feb 05, 2016 at 06:17:04PM +0100, Ricardo Wurmus wrote:
> I dropped the flag and noticed that OpenBLAS is not mentioned at all (it
> does retain a reference, but that’s expected when “openblas” is among
> the inputs), so I also dropped “openblas” from the inputs and R built
> just fine.

Okay. I am giving it a try on arm right now. It looks like we should
then also drop "--with-lapack"; the documentation states that this has only
an effect when "--with-blas" is also given:
   https://cran.r-project.org/doc/manuals/r-release/R-admin.html#LAPACK

If compilation succeeds on arm now, I will see whether a reference to lapack
is retained or not.

Andreas

Re: R

[Andreas Enge]

On Fri, Feb 05, 2016 at 08:02:36PM +0100, Andreas Enge wrote:
> Okay. I am giving it a try on arm right now. It looks like we should
> then also drop "--with-lapack"; the documentation states that this has only
> an effect when "--with-blas" is also given:
>    https://cran.r-project.org/doc/manuals/r-release/R-admin.html#LAPACK
> 
> If compilation succeeds on arm now, I will see whether a reference to lapack
> is retained or not.

Well, compilation succeeded! We will see whether this will also be the
case on mips.

And there is still a reference to lapack, so I am not touching the
"--with-lapack" configure flag for now.

Thanks a lot for your help!

Andreas

Re: R

[Ricardo Wurmus]

Andreas Enge <andreas@enge.fr> writes:

> On Fri, Feb 05, 2016 at 08:02:36PM +0100, Andreas Enge wrote:
>> Okay. I am giving it a try on arm right now. It looks like we should
>> then also drop "--with-lapack"; the documentation states that this has only
>> an effect when "--with-blas" is also given:
>>    https://cran.r-project.org/doc/manuals/r-release/R-admin.html#LAPACK
>> 
>> If compilation succeeds on arm now, I will see whether a reference to lapack
>> is retained or not.
>
> Well, compilation succeeded! We will see whether this will also be the
> case on mips.
>
> And there is still a reference to lapack, so I am not touching the
> "--with-lapack" configure flag for now.

I would suggest to drop “--with-lapack”, too.  (I haven’t found the time
to try building without this flag just now.)  The manual says:

    “However, the likely performance gains are thought to be small (and
    may be negative), and the default is not to search for a suitable
    LAPACK library, and this is definitely not recommended.

    [...]

    Please do bear in mind that using --with-lapack is ‘definitely not
    recommended’: it is provided only because it is necessary on some
    platforms and because some users want to experiment with claimed
    performance improvements. Reporting problems where it is used
    unnecessarily will simply irritate the R helpers.”

I don’t remember why I added it in the first place, so I think it’s best
to drop it, considering that the manual tells us only to do it if we
know what we want to achieve by adding it.

~~ Ricardo

Re: R

[Andreas Enge]

On Fri, Feb 05, 2016 at 09:48:24PM +0100, Ricardo Wurmus wrote:
> I would suggest to drop “--with-lapack”, too.  (I haven’t found the time
> to try building without this flag just now.)  The manual says:
> I don’t remember why I added it in the first place, so I think it’s best
> to drop it, considering that the manual tells us only to do it if we
> know what we want to achieve by adding it.

I did, and it still builds on arm.

Andreas

Re: R

[Ben Woodcroft]

[-- Attachment #1: Type: text/plain, Size: 955 bytes --]

Hi,

I thought I'd respond to this old thread now that the openblas eigen 
error seems to be fixed. I'm interested in using openblas so that the 
(as yet not in master) WGCNA package works faster, among other things:

https://labs.genetics.ucla.edu/horvath/CoexpressionNetwork/Rpackages/WGCNA/faq.html


On 06/02/16 08:24, Andreas Enge wrote:
> On Fri, Feb 05, 2016 at 09:48:24PM +0100, Ricardo Wurmus wrote:
>> I would suggest to drop “--with-lapack”, too.  (I haven’t found the time
>> to try building without this flag just now.)  The manual says:
>> I don’t remember why I added it in the first place, so I think it’s best
>> to drop it, considering that the manual tells us only to do it if we
>> know what we want to achieve by adding it.
> I did, and it still builds on arm.

Shall I push the attached patch to core-updates? I tested it on x86_64. 
Or, would you mind testing it on arm please Andreas?

Thanks,
ben

[-- Attachment #2: 0001-gnu-r-Add-openblas-input.patch --]
[-- Type: text/x-patch, Size: 1099 bytes --]

From 012c013661c739b8d90be8bcbfce13349dcaee89 Mon Sep 17 00:00:00 2001
From: Ben J Woodcroft <donttrustben@gmail.com>
Date: Thu, 8 Sep 2016 21:14:12 +1000
Subject: [PATCH] gnu: r: Add openblas input.

* gnu/packages/statistics.scm (r)[inputs]: Add openblas.
[arguments]: Adapt configure flags.
---
 gnu/packages/statistics.scm | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/gnu/packages/statistics.scm b/gnu/packages/statistics.scm
index 990f2fc..e34bcec 100644
--- a/gnu/packages/statistics.scm
+++ b/gnu/packages/statistics.scm
@@ -136,6 +136,7 @@ be output in text, PostScript, PDF or HTML.")
           (lambda _ (zero? (system* "make" "install-info")))))
        #:configure-flags
        '("--with-cairo"
+         "--with-blas=openblas"
          "--with-libpng"
          "--with-jpeglib"
          "--with-libtiff"
@@ -170,6 +171,7 @@ be output in text, PostScript, PDF or HTML.")
        ("pango" ,pango)
        ("curl" ,curl)
        ("tzdata" ,tzdata)
+       ("openblas" ,openblas)
        ("gfortran" ,gfortran)
        ("icu4c" ,icu4c)
        ("libjpeg" ,libjpeg)
-- 
2.9.2

Re: R

[Ricardo Wurmus]

Ben Woodcroft <b.woodcroft@uq.edu.au> writes:

> Hi,
>
> I thought I'd respond to this old thread now that the openblas eigen 
> error seems to be fixed. I'm interested in using openblas so that the 
> (as yet not in master) WGCNA package works faster, among other things:
>
> https://labs.genetics.ucla.edu/horvath/CoexpressionNetwork/Rpackages/WGCNA/faq.html
>
>
> On 06/02/16 08:24, Andreas Enge wrote:
>> On Fri, Feb 05, 2016 at 09:48:24PM +0100, Ricardo Wurmus wrote:
>>> I would suggest to drop “--with-lapack”, too.  (I haven’t found the time
>>> to try building without this flag just now.)  The manual says:
>>> I don’t remember why I added it in the first place, so I think it’s best
>>> to drop it, considering that the manual tells us only to do it if we
>>> know what we want to achieve by adding it.
>> I did, and it still builds on arm.
>
> Shall I push the attached patch to core-updates? I tested it on x86_64. 
> Or, would you mind testing it on arm please Andreas?

The patch looks fine to me and I’m generally in favour of adding
OpenBLAS to R.  

I should note that the R admin manual[1] says that the correct configure
flag for OpenBLAS is

    --with-blas="-lopenblas"

I’m not sure if this makes a difference.

~~ Ricardo


[1]: https://cran.r-project.org/doc/manuals/r-release/R-admin.html#Goto-and-OpenBLAS

Re: R

[Pjotr Prins]

OpenBLAS works fine for me - hit the same issue with Python before.

Pj.

On Thu, Sep 08, 2016 at 09:41:59PM +1000, Ben Woodcroft wrote:
> Hi,
> 
> I thought I'd respond to this old thread now that the openblas eigen
> error seems to be fixed. I'm interested in using openblas so that
> the (as yet not in master) WGCNA package works faster, among other
> things:
> 
> https://labs.genetics.ucla.edu/horvath/CoexpressionNetwork/Rpackages/WGCNA/faq.html
> 
> 
> On 06/02/16 08:24, Andreas Enge wrote:
> >On Fri, Feb 05, 2016 at 09:48:24PM +0100, Ricardo Wurmus wrote:
> >>I would suggest to drop “--with-lapack”, too.  (I haven’t found the time
> >>to try building without this flag just now.)  The manual says:
> >>I don’t remember why I added it in the first place, so I think it’s best
> >>to drop it, considering that the manual tells us only to do it if we
> >>know what we want to achieve by adding it.
> >I did, and it still builds on arm.
> 
> Shall I push the attached patch to core-updates? I tested it on
> x86_64. Or, would you mind testing it on arm please Andreas?
> 
> Thanks,
> ben

> From 012c013661c739b8d90be8bcbfce13349dcaee89 Mon Sep 17 00:00:00 2001
> From: Ben J Woodcroft <donttrustben@gmail.com>
> Date: Thu, 8 Sep 2016 21:14:12 +1000
> Subject: [PATCH] gnu: r: Add openblas input.
> 
> * gnu/packages/statistics.scm (r)[inputs]: Add openblas.
> [arguments]: Adapt configure flags.
> ---
>  gnu/packages/statistics.scm | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/gnu/packages/statistics.scm b/gnu/packages/statistics.scm
> index 990f2fc..e34bcec 100644
> --- a/gnu/packages/statistics.scm
> +++ b/gnu/packages/statistics.scm
> @@ -136,6 +136,7 @@ be output in text, PostScript, PDF or HTML.")
>            (lambda _ (zero? (system* "make" "install-info")))))
>         #:configure-flags
>         '("--with-cairo"
> +         "--with-blas=openblas"
>           "--with-libpng"
>           "--with-jpeglib"
>           "--with-libtiff"
> @@ -170,6 +171,7 @@ be output in text, PostScript, PDF or HTML.")
>         ("pango" ,pango)
>         ("curl" ,curl)
>         ("tzdata" ,tzdata)
> +       ("openblas" ,openblas)
>         ("gfortran" ,gfortran)
>         ("icu4c" ,icu4c)
>         ("libjpeg" ,libjpeg)
> -- 
> 2.9.2
> 


-- 

Re: R

[Ben Woodcroft]

On 08/09/16 23:02, Ricardo Wurmus wrote:
> Ben Woodcroft <b.woodcroft@uq.edu.au> writes:
>
>> Hi,
>>
>> I thought I'd respond to this old thread now that the openblas eigen
>> error seems to be fixed. I'm interested in using openblas so that the
>> (as yet not in master) WGCNA package works faster, among other things:
>>
>> https://labs.genetics.ucla.edu/horvath/CoexpressionNetwork/Rpackages/WGCNA/faq.html
>>
>>
>> On 06/02/16 08:24, Andreas Enge wrote:
>>> On Fri, Feb 05, 2016 at 09:48:24PM +0100, Ricardo Wurmus wrote:
>>>> I would suggest to drop “--with-lapack”, too.  (I haven’t found the time
>>>> to try building without this flag just now.)  The manual says:
>>>> I don’t remember why I added it in the first place, so I think it’s best
>>>> to drop it, considering that the manual tells us only to do it if we
>>>> know what we want to achieve by adding it.
>>> I did, and it still builds on arm.
>> Shall I push the attached patch to core-updates? I tested it on x86_64.
>> Or, would you mind testing it on arm please Andreas?
> The patch looks fine to me and I’m generally in favour of adding
> OpenBLAS to R.
>
> I should note that the R admin manual[1] says that the correct configure
> flag for OpenBLAS is
>
>      --with-blas="-lopenblas"
>
> I’m not sure if this makes a difference.

Thanks. I changed it to that minus the double quotes, and updated to 
3.3.1, then pushed to core-updated. We'll see how things go.
ta