unofficial mirror of guix-devel@gnu.org 
 help / color / mirror / code / Atom feed
* R
@ 2016-02-04 19:39 Andreas Enge
  2016-02-05 10:00 ` R Andreas Enge
  0 siblings, 1 reply; 14+ messages in thread
From: Andreas Enge @ 2016-02-04 19:39 UTC (permalink / raw)
  To: guix-devel

Yesterday I submitted a bug report for R on arm:
   https://bugs.r-project.org/bugzilla/show_bug.cgi?id=16697https://bugs.r-project.org/bugzilla/show_bug.cgi?id=16697

With all the different R packages failing consequently, I hoped that fixing R
would reduce the number of failures. (On MIPS there is also a problem, but
not the same one.)

Andreas

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: R
  2016-02-04 19:39 R Andreas Enge
@ 2016-02-05 10:00 ` Andreas Enge
  2016-02-05 10:55   ` R Pjotr Prins
  2016-02-05 12:37   ` R Ricardo Wurmus
  0 siblings, 2 replies; 14+ messages in thread
From: Andreas Enge @ 2016-02-05 10:00 UTC (permalink / raw)
  To: guix-devel

I did get a reply, but not using R, I do not quite know what to make of it:
   https://bugs.r-project.org/bugzilla/show_bug.cgi?id=16697
It would be nice if someone using R could help me out; if you send me an
ssh key, I could give you access to an arm machine.

To start, a probably simple question: Why do we use "--with-blas=openblas"?

Andreas

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: R
  2016-02-05 10:00 ` R Andreas Enge
@ 2016-02-05 10:55   ` Pjotr Prins
  2016-02-05 12:37   ` R Ricardo Wurmus
  1 sibling, 0 replies; 14+ messages in thread
From: Pjotr Prins @ 2016-02-05 10:55 UTC (permalink / raw)
  To: Andreas Enge; +Cc: guix-devel

On Fri, Feb 05, 2016 at 11:00:40AM +0100, Andreas Enge wrote:
> I did get a reply, but not using R, I do not quite know what to make of it:
>    https://bugs.r-project.org/bugzilla/show_bug.cgi?id=16697

Anyone here programming in R?

> It would be nice if someone using R could help me out; if you send me an
> ssh key, I could give you access to an arm machine.
> 
> To start, a probably simple question: Why do we use "--with-blas=openblas"?

openblas is fast.

http://blog.nguyenvq.com/blog/2014/11/10/optimized-r-and-python-standard-blas-vs-atlas-vs-openblas-vs-mkl/

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: R
  2016-02-05 10:00 ` R Andreas Enge
  2016-02-05 10:55   ` R Pjotr Prins
@ 2016-02-05 12:37   ` Ricardo Wurmus
  2016-02-05 17:17     ` R Ricardo Wurmus
  1 sibling, 1 reply; 14+ messages in thread
From: Ricardo Wurmus @ 2016-02-05 12:37 UTC (permalink / raw)
  To: Andreas Enge; +Cc: guix-devel


Andreas Enge <andreas@enge.fr> writes:

> I did get a reply, but not using R, I do not quite know what to make of it:
>    https://bugs.r-project.org/bugzilla/show_bug.cgi?id=16697
> It would be nice if someone using R could help me out; if you send me an
> ssh key, I could give you access to an arm machine.
>
> To start, a probably simple question: Why do we use "--with-blas=openblas"?

I did this when we switched from ATLAS to OpenBLAS (for substitutability
reasons).  It is possible I have misunderstood the meaning of this
configure flag.

Here’s the installation manual on building with a specific BLAS
implementation:

    https://cran.r-project.org/doc/manuals/r-release/R-admin.html#BLAS

According to this section,

    “R can make use of enhanced BLAS (Basic Linear Algebra Subprograms,
    http://www.netlib.org/blas/faq.html) routines. However, these have
    to be explicitly requested at configure time: R provides an internal
    BLAS which is well-tested and will be adequate for most uses of R.”

There’s a separate section for OpenBLAS, which also mentions that using
a “shared BLAS” is a preferred alternative to linking with a particular
BLAS implementation.  We already pass “--enable-BLAS-shlib”, so we
actually do use a shared BLAS, IIUC.

I guess we could drop the “--with-blas” flag and see if that helps.  I
remember struggling with this a lot in the past.  I’ll try building R
again without that flag and see if it still works.

I’m sorry if that’s what’s responsible for the failures, but it’s pretty
hard for me to understand how linking with a high-performance BLAS
implementation could have caused these build/test failures on other
platforms.  I’ll report back when I have a successful build or an
interesting failure to share.

~~ Ricardo

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: R
  2016-02-05 12:37   ` R Ricardo Wurmus
@ 2016-02-05 17:17     ` Ricardo Wurmus
  2016-02-05 17:59       ` R Kyle Meyer
  2016-02-05 19:02       ` R Andreas Enge
  0 siblings, 2 replies; 14+ messages in thread
From: Ricardo Wurmus @ 2016-02-05 17:17 UTC (permalink / raw)
  To: Andreas Enge; +Cc: guix-devel


Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de> writes:

> There’s a separate section for OpenBLAS, which also mentions that using
> a “shared BLAS” is a preferred alternative to linking with a particular
> BLAS implementation.  We already pass “--enable-BLAS-shlib”, so we
> actually do use a shared BLAS, IIUC.
>
> I guess we could drop the “--with-blas” flag and see if that helps.  I
> remember struggling with this a lot in the past.  I’ll try building R
> again without that flag and see if it still works.

I dropped the flag and noticed that OpenBLAS is not mentioned at all (it
does retain a reference, but that’s expected when “openblas” is among
the inputs), so I also dropped “openblas” from the inputs and R built
just fine.

It’s a bit sad to me that we won’t be able to just use this BLAS
implementation, but if it fixes the build on ARM and MIPS I’m all for
it.

I can also confirm that dropping openblas from the R build “fixes” the
segfault when running

    x <- eigen(crossprod(matrix(rnorm(50 * 500), 50, 500)))

as reported here: https://github.com/xianyi/OpenBLAS/issues/703

So, I think it’s a good idea to build R without OpenBLAS on all
architectures for now.

~~ Ricardo

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: R
  2016-02-05 17:17     ` R Ricardo Wurmus
@ 2016-02-05 17:59       ` Kyle Meyer
  2016-02-05 19:02       ` R Andreas Enge
  1 sibling, 0 replies; 14+ messages in thread
From: Kyle Meyer @ 2016-02-05 17:59 UTC (permalink / raw)
  To: Ricardo Wurmus; +Cc: guix-devel

Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de> writes:

[...]

> I can also confirm that dropping openblas from the R build “fixes” the
> segfault when running
>
>     x <- eigen(crossprod(matrix(rnorm(50 * 500), 50, 500)))
>
> as reported here: https://github.com/xianyi/OpenBLAS/issues/703
>
> So, I think it’s a good idea to build R without OpenBLAS on all
> architectures for now.

Thank you, Ricardo.  I'm sorry I wasn't able make any progress on
figuring out what the underlying issue was there.

-- 
Kyle

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: R
  2016-02-05 17:17     ` R Ricardo Wurmus
  2016-02-05 17:59       ` R Kyle Meyer
@ 2016-02-05 19:02       ` Andreas Enge
  2016-02-05 19:54         ` R Andreas Enge
  1 sibling, 1 reply; 14+ messages in thread
From: Andreas Enge @ 2016-02-05 19:02 UTC (permalink / raw)
  To: Ricardo Wurmus; +Cc: guix-devel

On Fri, Feb 05, 2016 at 06:17:04PM +0100, Ricardo Wurmus wrote:
> I dropped the flag and noticed that OpenBLAS is not mentioned at all (it
> does retain a reference, but that’s expected when “openblas” is among
> the inputs), so I also dropped “openblas” from the inputs and R built
> just fine.

Okay. I am giving it a try on arm right now. It looks like we should
then also drop "--with-lapack"; the documentation states that this has only
an effect when "--with-blas" is also given:
   https://cran.r-project.org/doc/manuals/r-release/R-admin.html#LAPACK

If compilation succeeds on arm now, I will see whether a reference to lapack
is retained or not.

Andreas

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: R
  2016-02-05 19:02       ` R Andreas Enge
@ 2016-02-05 19:54         ` Andreas Enge
  2016-02-05 20:48           ` R Ricardo Wurmus
  0 siblings, 1 reply; 14+ messages in thread
From: Andreas Enge @ 2016-02-05 19:54 UTC (permalink / raw)
  To: Ricardo Wurmus; +Cc: guix-devel

On Fri, Feb 05, 2016 at 08:02:36PM +0100, Andreas Enge wrote:
> Okay. I am giving it a try on arm right now. It looks like we should
> then also drop "--with-lapack"; the documentation states that this has only
> an effect when "--with-blas" is also given:
>    https://cran.r-project.org/doc/manuals/r-release/R-admin.html#LAPACK
> 
> If compilation succeeds on arm now, I will see whether a reference to lapack
> is retained or not.

Well, compilation succeeded! We will see whether this will also be the
case on mips.

And there is still a reference to lapack, so I am not touching the
"--with-lapack" configure flag for now.

Thanks a lot for your help!

Andreas

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: R
  2016-02-05 19:54         ` R Andreas Enge
@ 2016-02-05 20:48           ` Ricardo Wurmus
  2016-02-05 22:24             ` R Andreas Enge
  0 siblings, 1 reply; 14+ messages in thread
From: Ricardo Wurmus @ 2016-02-05 20:48 UTC (permalink / raw)
  To: Andreas Enge; +Cc: guix-devel


Andreas Enge <andreas@enge.fr> writes:

> On Fri, Feb 05, 2016 at 08:02:36PM +0100, Andreas Enge wrote:
>> Okay. I am giving it a try on arm right now. It looks like we should
>> then also drop "--with-lapack"; the documentation states that this has only
>> an effect when "--with-blas" is also given:
>>    https://cran.r-project.org/doc/manuals/r-release/R-admin.html#LAPACK
>> 
>> If compilation succeeds on arm now, I will see whether a reference to lapack
>> is retained or not.
>
> Well, compilation succeeded! We will see whether this will also be the
> case on mips.
>
> And there is still a reference to lapack, so I am not touching the
> "--with-lapack" configure flag for now.

I would suggest to drop “--with-lapack”, too.  (I haven’t found the time
to try building without this flag just now.)  The manual says:

    “However, the likely performance gains are thought to be small (and
    may be negative), and the default is not to search for a suitable
    LAPACK library, and this is definitely not recommended.

    [...]

    Please do bear in mind that using --with-lapack is ‘definitely not
    recommended’: it is provided only because it is necessary on some
    platforms and because some users want to experiment with claimed
    performance improvements. Reporting problems where it is used
    unnecessarily will simply irritate the R helpers.”

I don’t remember why I added it in the first place, so I think it’s best
to drop it, considering that the manual tells us only to do it if we
know what we want to achieve by adding it.

~~ Ricardo

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: R
  2016-02-05 20:48           ` R Ricardo Wurmus
@ 2016-02-05 22:24             ` Andreas Enge
  2016-09-08 11:41               ` R Ben Woodcroft
  0 siblings, 1 reply; 14+ messages in thread
From: Andreas Enge @ 2016-02-05 22:24 UTC (permalink / raw)
  To: Ricardo Wurmus; +Cc: guix-devel

On Fri, Feb 05, 2016 at 09:48:24PM +0100, Ricardo Wurmus wrote:
> I would suggest to drop “--with-lapack”, too.  (I haven’t found the time
> to try building without this flag just now.)  The manual says:
> I don’t remember why I added it in the first place, so I think it’s best
> to drop it, considering that the manual tells us only to do it if we
> know what we want to achieve by adding it.

I did, and it still builds on arm.

Andreas

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: R
  2016-02-05 22:24             ` R Andreas Enge
@ 2016-09-08 11:41               ` Ben Woodcroft
  2016-09-08 13:02                 ` R Ricardo Wurmus
  2016-09-08 17:01                 ` R Pjotr Prins
  0 siblings, 2 replies; 14+ messages in thread
From: Ben Woodcroft @ 2016-09-08 11:41 UTC (permalink / raw)
  To: Andreas Enge, Ricardo Wurmus; +Cc: guix-devel

[-- Attachment #1: Type: text/plain, Size: 955 bytes --]

Hi,

I thought I'd respond to this old thread now that the openblas eigen 
error seems to be fixed. I'm interested in using openblas so that the 
(as yet not in master) WGCNA package works faster, among other things:

https://labs.genetics.ucla.edu/horvath/CoexpressionNetwork/Rpackages/WGCNA/faq.html


On 06/02/16 08:24, Andreas Enge wrote:
> On Fri, Feb 05, 2016 at 09:48:24PM +0100, Ricardo Wurmus wrote:
>> I would suggest to drop “--with-lapack”, too.  (I haven’t found the time
>> to try building without this flag just now.)  The manual says:
>> I don’t remember why I added it in the first place, so I think it’s best
>> to drop it, considering that the manual tells us only to do it if we
>> know what we want to achieve by adding it.
> I did, and it still builds on arm.

Shall I push the attached patch to core-updates? I tested it on x86_64. 
Or, would you mind testing it on arm please Andreas?

Thanks,
ben

[-- Attachment #2: 0001-gnu-r-Add-openblas-input.patch --]
[-- Type: text/x-patch, Size: 1099 bytes --]

From 012c013661c739b8d90be8bcbfce13349dcaee89 Mon Sep 17 00:00:00 2001
From: Ben J Woodcroft <donttrustben@gmail.com>
Date: Thu, 8 Sep 2016 21:14:12 +1000
Subject: [PATCH] gnu: r: Add openblas input.

* gnu/packages/statistics.scm (r)[inputs]: Add openblas.
[arguments]: Adapt configure flags.
---
 gnu/packages/statistics.scm | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/gnu/packages/statistics.scm b/gnu/packages/statistics.scm
index 990f2fc..e34bcec 100644
--- a/gnu/packages/statistics.scm
+++ b/gnu/packages/statistics.scm
@@ -136,6 +136,7 @@ be output in text, PostScript, PDF or HTML.")
           (lambda _ (zero? (system* "make" "install-info")))))
        #:configure-flags
        '("--with-cairo"
+         "--with-blas=openblas"
          "--with-libpng"
          "--with-jpeglib"
          "--with-libtiff"
@@ -170,6 +171,7 @@ be output in text, PostScript, PDF or HTML.")
        ("pango" ,pango)
        ("curl" ,curl)
        ("tzdata" ,tzdata)
+       ("openblas" ,openblas)
        ("gfortran" ,gfortran)
        ("icu4c" ,icu4c)
        ("libjpeg" ,libjpeg)
-- 
2.9.2


^ permalink raw reply related	[flat|nested] 14+ messages in thread

* Re: R
  2016-09-08 11:41               ` R Ben Woodcroft
@ 2016-09-08 13:02                 ` Ricardo Wurmus
  2016-09-09 12:07                   ` R Ben Woodcroft
  2016-09-08 17:01                 ` R Pjotr Prins
  1 sibling, 1 reply; 14+ messages in thread
From: Ricardo Wurmus @ 2016-09-08 13:02 UTC (permalink / raw)
  To: Ben Woodcroft; +Cc: guix-devel


Ben Woodcroft <b.woodcroft@uq.edu.au> writes:

> Hi,
>
> I thought I'd respond to this old thread now that the openblas eigen 
> error seems to be fixed. I'm interested in using openblas so that the 
> (as yet not in master) WGCNA package works faster, among other things:
>
> https://labs.genetics.ucla.edu/horvath/CoexpressionNetwork/Rpackages/WGCNA/faq.html
>
>
> On 06/02/16 08:24, Andreas Enge wrote:
>> On Fri, Feb 05, 2016 at 09:48:24PM +0100, Ricardo Wurmus wrote:
>>> I would suggest to drop “--with-lapack”, too.  (I haven’t found the time
>>> to try building without this flag just now.)  The manual says:
>>> I don’t remember why I added it in the first place, so I think it’s best
>>> to drop it, considering that the manual tells us only to do it if we
>>> know what we want to achieve by adding it.
>> I did, and it still builds on arm.
>
> Shall I push the attached patch to core-updates? I tested it on x86_64. 
> Or, would you mind testing it on arm please Andreas?

The patch looks fine to me and I’m generally in favour of adding
OpenBLAS to R.  

I should note that the R admin manual[1] says that the correct configure
flag for OpenBLAS is

    --with-blas="-lopenblas"

I’m not sure if this makes a difference.

~~ Ricardo


[1]: https://cran.r-project.org/doc/manuals/r-release/R-admin.html#Goto-and-OpenBLAS

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: R
  2016-09-08 11:41               ` R Ben Woodcroft
  2016-09-08 13:02                 ` R Ricardo Wurmus
@ 2016-09-08 17:01                 ` Pjotr Prins
  1 sibling, 0 replies; 14+ messages in thread
From: Pjotr Prins @ 2016-09-08 17:01 UTC (permalink / raw)
  To: Ben Woodcroft; +Cc: guix-devel

OpenBLAS works fine for me - hit the same issue with Python before.

Pj.

On Thu, Sep 08, 2016 at 09:41:59PM +1000, Ben Woodcroft wrote:
> Hi,
> 
> I thought I'd respond to this old thread now that the openblas eigen
> error seems to be fixed. I'm interested in using openblas so that
> the (as yet not in master) WGCNA package works faster, among other
> things:
> 
> https://labs.genetics.ucla.edu/horvath/CoexpressionNetwork/Rpackages/WGCNA/faq.html
> 
> 
> On 06/02/16 08:24, Andreas Enge wrote:
> >On Fri, Feb 05, 2016 at 09:48:24PM +0100, Ricardo Wurmus wrote:
> >>I would suggest to drop “--with-lapack”, too.  (I haven’t found the time
> >>to try building without this flag just now.)  The manual says:
> >>I don’t remember why I added it in the first place, so I think it’s best
> >>to drop it, considering that the manual tells us only to do it if we
> >>know what we want to achieve by adding it.
> >I did, and it still builds on arm.
> 
> Shall I push the attached patch to core-updates? I tested it on
> x86_64. Or, would you mind testing it on arm please Andreas?
> 
> Thanks,
> ben

> From 012c013661c739b8d90be8bcbfce13349dcaee89 Mon Sep 17 00:00:00 2001
> From: Ben J Woodcroft <donttrustben@gmail.com>
> Date: Thu, 8 Sep 2016 21:14:12 +1000
> Subject: [PATCH] gnu: r: Add openblas input.
> 
> * gnu/packages/statistics.scm (r)[inputs]: Add openblas.
> [arguments]: Adapt configure flags.
> ---
>  gnu/packages/statistics.scm | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/gnu/packages/statistics.scm b/gnu/packages/statistics.scm
> index 990f2fc..e34bcec 100644
> --- a/gnu/packages/statistics.scm
> +++ b/gnu/packages/statistics.scm
> @@ -136,6 +136,7 @@ be output in text, PostScript, PDF or HTML.")
>            (lambda _ (zero? (system* "make" "install-info")))))
>         #:configure-flags
>         '("--with-cairo"
> +         "--with-blas=openblas"
>           "--with-libpng"
>           "--with-jpeglib"
>           "--with-libtiff"
> @@ -170,6 +171,7 @@ be output in text, PostScript, PDF or HTML.")
>         ("pango" ,pango)
>         ("curl" ,curl)
>         ("tzdata" ,tzdata)
> +       ("openblas" ,openblas)
>         ("gfortran" ,gfortran)
>         ("icu4c" ,icu4c)
>         ("libjpeg" ,libjpeg)
> -- 
> 2.9.2
> 


-- 

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: R
  2016-09-08 13:02                 ` R Ricardo Wurmus
@ 2016-09-09 12:07                   ` Ben Woodcroft
  0 siblings, 0 replies; 14+ messages in thread
From: Ben Woodcroft @ 2016-09-09 12:07 UTC (permalink / raw)
  To: Ricardo Wurmus; +Cc: guix-devel

On 08/09/16 23:02, Ricardo Wurmus wrote:
> Ben Woodcroft <b.woodcroft@uq.edu.au> writes:
>
>> Hi,
>>
>> I thought I'd respond to this old thread now that the openblas eigen
>> error seems to be fixed. I'm interested in using openblas so that the
>> (as yet not in master) WGCNA package works faster, among other things:
>>
>> https://labs.genetics.ucla.edu/horvath/CoexpressionNetwork/Rpackages/WGCNA/faq.html
>>
>>
>> On 06/02/16 08:24, Andreas Enge wrote:
>>> On Fri, Feb 05, 2016 at 09:48:24PM +0100, Ricardo Wurmus wrote:
>>>> I would suggest to drop “--with-lapack”, too.  (I haven’t found the time
>>>> to try building without this flag just now.)  The manual says:
>>>> I don’t remember why I added it in the first place, so I think it’s best
>>>> to drop it, considering that the manual tells us only to do it if we
>>>> know what we want to achieve by adding it.
>>> I did, and it still builds on arm.
>> Shall I push the attached patch to core-updates? I tested it on x86_64.
>> Or, would you mind testing it on arm please Andreas?
> The patch looks fine to me and I’m generally in favour of adding
> OpenBLAS to R.
>
> I should note that the R admin manual[1] says that the correct configure
> flag for OpenBLAS is
>
>      --with-blas="-lopenblas"
>
> I’m not sure if this makes a difference.

Thanks. I changed it to that minus the double quotes, and updated to 
3.3.1, then pushed to core-updated. We'll see how things go.
ta

^ permalink raw reply	[flat|nested] 14+ messages in thread

end of thread, other threads:[~2016-09-09 12:08 UTC | newest]

Thread overview: 14+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2016-02-04 19:39 R Andreas Enge
2016-02-05 10:00 ` R Andreas Enge
2016-02-05 10:55   ` R Pjotr Prins
2016-02-05 12:37   ` R Ricardo Wurmus
2016-02-05 17:17     ` R Ricardo Wurmus
2016-02-05 17:59       ` R Kyle Meyer
2016-02-05 19:02       ` R Andreas Enge
2016-02-05 19:54         ` R Andreas Enge
2016-02-05 20:48           ` R Ricardo Wurmus
2016-02-05 22:24             ` R Andreas Enge
2016-09-08 11:41               ` R Ben Woodcroft
2016-09-08 13:02                 ` R Ricardo Wurmus
2016-09-09 12:07                   ` R Ben Woodcroft
2016-09-08 17:01                 ` R Pjotr Prins

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/guix.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).