Re: 01/01: gnu: python-numpy-bootstrap: Reduce matrix size in failing test.

Re: 01/01: gnu: python-numpy-bootstrap: Reduce matrix size in failing test.

[Ludovic Courtès]

Federico Beffa <beffa@fbengineering.ch> skribis:

> +        (alist-cons-before
> +         'check 'fix-failing-tests
> +         (lambda _
> +           (substitute* (find-files "numpy/linalg/tests" 
> +                                    "test_regression\\.py")
> +             (("x = np.eye(1000, 66)")
> +              "x = np.eye(10, 66)")))

Could you add a comment here explaining that this is reducing the input
matrix size (?) and why it matters?  Otherwise it may seem a bit too
magical.  :-)

Also, this kind of change would be a good candidate for ‘snippet’,
because it’s a “static” change.

Thanks,
Ludo’.

Re: 01/01: gnu: python-numpy-bootstrap: Reduce matrix size in failing test.

[Federico Beffa]

On Sun, Dec 14, 2014 at 11:54 PM, Ludovic Courtès <ludo@gnu.org> wrote:
> Federico Beffa <beffa@fbengineering.ch> skribis:
>
>> +        (alist-cons-before
>> +         'check 'fix-failing-tests
>> +         (lambda _
>> +           (substitute* (find-files "numpy/linalg/tests"
>> +                                    "test_regression\\.py")
>> +             (("x = np.eye(1000, 66)")
>> +              "x = np.eye(10, 66)")))
>
> Could you add a comment here explaining that this is reducing the input
> matrix size (?) and why it matters?  Otherwise it may seem a bit too
> magical.  :-)
>
> Also, this kind of change would be a good candidate for ‘snippet’,
> because it’s a “static” change.

OK, I need to look into those :-)

Actually the test is still failing and I do not understand why on
hydra it is failing while on my machine (x86_64) it passes with both
python versions.  So, at least for the moment, I would propose to
remove the above phase and disable the tests.

WDYT?

Fede

Re: 01/01: gnu: python-numpy-bootstrap: Reduce matrix size in failing test.

[Federico Beffa]

On Mon, Dec 15, 2014 at 9:41 AM, Federico Beffa <beffa@ieee.org> wrote:
> On Sun, Dec 14, 2014 at 11:54 PM, Ludovic Courtès <ludo@gnu.org> wrote:
>> Federico Beffa <beffa@fbengineering.ch> skribis:
>>
>>> +        (alist-cons-before
>>> +         'check 'fix-failing-tests
>>> +         (lambda _
>>> +           (substitute* (find-files "numpy/linalg/tests"
>>> +                                    "test_regression\\.py")
>>> +             (("x = np.eye(1000, 66)")
>>> +              "x = np.eye(10, 66)")))
>>
>> Could you add a comment here explaining that this is reducing the input
>> matrix size (?) and why it matters?  Otherwise it may seem a bit too
>> magical.  :-)
>>
>> Also, this kind of change would be a good candidate for ‘snippet’,
>> because it’s a “static” change.
>
> OK, I need to look into those :-)
>
> Actually the test is still failing and I do not understand why on
> hydra it is failing while on my machine (x86_64) it passes with both
> python versions.  So, at least for the moment, I would propose to
> remove the above phase and disable the tests.
>
> WDYT?

One more thing: the function causing the test to fail is implemented
making use of ATLAS. However, the log does not show ATLAS being built
before numpy (as I would expect given the #:substitutable? #f
argument). If the machines to which hydra distributes the jobs are not
homogeneous, then it could be that the used ATLAS is not working
properly (maybe different ISA extensions?). If that is the case, then
why is the #:substitutable argument not honored?

What machines is the build farm made of? Do they all have the same
version of OS and kernel?

Regards,
Fede

Re: 01/01: gnu: python-numpy-bootstrap: Reduce matrix size in failing test.

[Ludovic Courtès]

Federico Beffa <beffa@ieee.org> skribis:

> One more thing: the function causing the test to fail is implemented
> making use of ATLAS. However, the log does not show ATLAS being built
> before numpy (as I would expect given the #:substitutable? #f
> argument). If the machines to which hydra distributes the jobs are not
> homogeneous, then it could be that the used ATLAS is not working
> properly (maybe different ISA extensions?). If that is the case, then
> why is the #:substitutable argument not honored?

I *think* that what happens, is that #:substitutable? also prevents
offloading (because of <http://bugs.gnu.org/18747>.)

So, since hydra.gnu.org currently runs the guix-daemon with
--max-jobs=1, it probably ends up building ATLAS locally (this machine
is a Quad-Core AMD Opteron(tm) Processor 2374 HE, but actually a VM.)

However, numpy itself does get offloaded, so ATLAS from the step above
ends up being transferred to whatever machine is building numpy, which
could explain the problem.

I’m not sure exactly what to do.  Perhaps we should just exclude ATLAS
and whatever refers to it from Hydra?

> What machines is the build farm made of? Do they all have the same
> version of OS and kernel?

No, they’re different.  The CPU and kernel are the only “impurities”
that leak through the build environment though.  It rarely matters, but
ATLAS is clearly a case where it does make a difference.

Thanks,
Ludo’.