Re: 01/01: python: Honor '--cores=...' in tests.

Re: 01/01: python: Honor '--cores=...' in tests.

[Christopher Baines]

[-- Attachment #1: Type: text/plain, Size: 1757 bytes --]


Eric Bavier <ericbavier@centurylink.net> writes:

> bavier pushed a commit to branch core-updates-next
> in repository guix.
>
> commit 5b01b6034aeab32a5011c5757f18bd9772d3497d
> Author: Eric Bavier <bavier@member.fsf.org>
> Date:   Thu Nov 1 21:18:41 2018 -0500
>
>     python: Honor '--cores=...' in tests.
>     
>     * gnu/packages/python.scm (python-2.7)[arguments]: Add #:make-flags.
> ---
>  gnu/packages/python.scm | 3 +++
>  1 file changed, 3 insertions(+)
>
> diff --git a/gnu/packages/python.scm b/gnu/packages/python.scm
> index 2b6a064..46ce0dd 100644
> --- a/gnu/packages/python.scm
> +++ b/gnu/packages/python.scm
> @@ -190,6 +190,9 @@
>               "--enable-unicode=ucs4"
>               (string-append "LDFLAGS=-Wl,-rpath="
>                              (assoc-ref %outputs "out") "/lib"))
> +       ;; With no -j argument tests use all available cpus, so provide one.
> +       #:make-flags
> +       (list (format #f "EXTRATESTOPTS=-j~d" (parallel-job-count)))
>  
>          #:modules ((ice-9 ftw) (ice-9 match)
>                     (guix build utils) (guix build gnu-build-system))

Hi Eric,

I'm having trouble building the python2 package on core-updates (which
was renamed from core-updates-next), and I think it relates to this
change.

This is roughly the issue I'm seeing:

  LD_LIBRARY_PATH=/tmp/guix-build-python2-2.7.15.drv-0/Python-2.7.15 ./python -Wd -3 -E -tt  ./Lib/test/regrtest.py -l -j12

  -l and -j don't go together!


Since all the python packages look to inherit from python-2.7 that's
being changed here, perhaps this worked for some of them (I know you
mentioned python-minimal in one message), but not all of them, or at
least python2?

Thanks,

Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 962 bytes --]

Re: 01/01: python: Honor '--cores=...' in tests.

[Christopher Baines]

[-- Attachment #1: Type: text/plain, Size: 1618 bytes --]


Christopher Baines <mail@cbaines.net> writes:

> Eric Bavier <ericbavier@centurylink.net> writes:
>
>> diff --git a/gnu/packages/python.scm b/gnu/packages/python.scm
>> index 2b6a064..46ce0dd 100644
>> --- a/gnu/packages/python.scm
>> +++ b/gnu/packages/python.scm
>> @@ -190,6 +190,9 @@
>>               "--enable-unicode=ucs4"
>>               (string-append "LDFLAGS=-Wl,-rpath="
>>                              (assoc-ref %outputs "out") "/lib"))
>> +       ;; With no -j argument tests use all available cpus, so provide one.
>> +       #:make-flags
>> +       (list (format #f "EXTRATESTOPTS=-j~d" (parallel-job-count)))
>>  
>>          #:modules ((ice-9 ftw) (ice-9 match)
>>                     (guix build utils) (guix build gnu-build-system))
>
> Hi Eric,
>
> I'm having trouble building the python2 package on core-updates (which
> was renamed from core-updates-next), and I think it relates to this
> change.
>
> This is roughly the issue I'm seeing:
>
>   LD_LIBRARY_PATH=/tmp/guix-build-python2-2.7.15.drv-0/Python-2.7.15 ./python -Wd -3 -E -tt  ./Lib/test/regrtest.py -l -j12
>
>   -l and -j don't go together!
>
>
> Since all the python packages look to inherit from python-2.7 that's
> being changed here, perhaps this worked for some of them (I know you
> mentioned python-minimal in one message), but not all of them, or at
> least python2?

Just got around to testing this, moving this change to python-3.7, from
python-2.7 fixes the python-2.7 build for me, but I haven't checked
how that affects how many cores are actually used when running the tests.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 962 bytes --]

Re: 01/01: python: Honor '--cores=...' in tests.

[Eric Bavier]

[-- Attachment #1: Type: text/plain, Size: 1905 bytes --]

On Fri, 14 Dec 2018 15:50:37 +0000
Christopher Baines <mail@cbaines.net> wrote:

> Christopher Baines <mail@cbaines.net> writes:
> 
> > Eric Bavier <ericbavier@centurylink.net> writes:
> >  
> >> diff --git a/gnu/packages/python.scm b/gnu/packages/python.scm
> >> index 2b6a064..46ce0dd 100644
> >> --- a/gnu/packages/python.scm
> >> +++ b/gnu/packages/python.scm
> >> @@ -190,6 +190,9 @@
> >>               "--enable-unicode=ucs4"
> >>               (string-append "LDFLAGS=-Wl,-rpath="
> >>                              (assoc-ref %outputs "out") "/lib"))
> >> +       ;; With no -j argument tests use all available cpus, so provide one.
> >> +       #:make-flags
> >> +       (list (format #f "EXTRATESTOPTS=-j~d" (parallel-job-count)))
> >>  
> >>          #:modules ((ice-9 ftw) (ice-9 match)
> >>                     (guix build utils) (guix build gnu-build-system))  
> >
> > Hi Eric,
> >
> > I'm having trouble building the python2 package on core-updates (which
> > was renamed from core-updates-next), and I think it relates to this
> > change.
> >
> > This is roughly the issue I'm seeing:
> >
> >   LD_LIBRARY_PATH=/tmp/guix-build-python2-2.7.15.drv-0/Python-2.7.15 ./python -Wd -3 -E -tt  ./Lib/test/regrtest.py -l -j12
> >
> >   -l and -j don't go together!
> >
> >
> > Since all the python packages look to inherit from python-2.7 that's
> > being changed here, perhaps this worked for some of them (I know you
> > mentioned python-minimal in one message), but not all of them, or at
> > least python2?  
> 
> Just got around to testing this, moving this change to python-3.7, from
> python-2.7 fixes the python-2.7 build for me, but I haven't checked
> how that affects how many cores are actually used when running the tests.

Strange.  I recall testing this patch on both python2 and python.  Not
sure what has changed since then.

`~Eric

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: 01/01: python: Honor '--cores=...' in tests.

[Christopher Baines]

[-- Attachment #1: Type: text/plain, Size: 1397 bytes --]


Eric Bavier <ericbavier@centurylink.net> writes:

> On Fri, 14 Dec 2018 15:50:37 +0000
> Christopher Baines <mail@cbaines.net> wrote:
>
>> Christopher Baines <mail@cbaines.net> writes:
>>
>> > Eric Bavier <ericbavier@centurylink.net> writes:
>> >
>> > Since all the python packages look to inherit from python-2.7 that's
>> > being changed here, perhaps this worked for some of them (I know you
>> > mentioned python-minimal in one message), but not all of them, or at
>> > least python2?
>>
>> Just got around to testing this, moving this change to python-3.7, from
>> python-2.7 fixes the python-2.7 build for me, but I haven't checked
>> how that affects how many cores are actually used when running the tests.
>
> Strange.  I recall testing this patch on both python2 and python.  Not
> sure what has changed since then.

Ok, I've checked out the relevant commit and I can reproduce the issue
there. I've also tried to compare the behaviour between the python2 and
python3 packages.

I think the reason why it might work for python3, and not python2, is
that the python2 package seems to include the -l option by default,
whereas the python3 package doesn't. Looking at the -l option, it's
something about detecting tests that leak memory, which doesn't sound
that important, so maybe the thing to do is to patch out the use of -l
in the python2 package.

What do you think?

Thanks,

Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 962 bytes --]