all messages for Guix-related lists mirrored at yhetil.org
 help / color / mirror / code / Atom feed
* [bug#35126] [PATCH] gnu: rust: Enable parallel tests starting with 1.26.
@ 2019-04-03 17:00 Danny Milosavljevic
  2019-04-04  0:52 ` Ivan Petkov
  0 siblings, 1 reply; 9+ messages in thread
From: Danny Milosavljevic @ 2019-04-03 17:00 UTC (permalink / raw)
  To: 35126

* gnu/packages/rust.scm (rust-1.26)[arguments]<#:phases>[check]: Enable
parallel tests.
---
 gnu/packages/rust.scm | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/gnu/packages/rust.scm b/gnu/packages/rust.scm
index 63426a12fc..23b28cce23 100644
--- a/gnu/packages/rust.scm
+++ b/gnu/packages/rust.scm
@@ -724,6 +724,11 @@ jemalloc = \"" jemalloc "/lib/libjemalloc_pic.a" "\"
           `(modify-phases ,phases
              ;; binaryen was replaced with LLD project from LLVM
              (delete 'dont-build-native)
+             (replace 'check
+               (lambda* _
+                 ;; Enable parallel execution.
+                 (invoke "./x.py" (string-append "-j" (number->string (parallel-job-count))) "test" "-vv")
+                 (invoke "./x.py" (string-append "-j" (number->string (parallel-job-count))) "test" "src/tools/cargo")))
              (replace 'remove-unsupported-tests
                (lambda* _
                  ;; Our ld-wrapper cannot process non-UTF8 bytes in LIBRARY_PATH.

^ permalink raw reply related	[flat|nested] 9+ messages in thread

* [bug#35126] [PATCH] gnu: rust: Enable parallel tests starting with 1.26.
  2019-04-03 17:00 [bug#35126] [PATCH] gnu: rust: Enable parallel tests starting with 1.26 Danny Milosavljevic
@ 2019-04-04  0:52 ` Ivan Petkov
  2019-04-04 18:29   ` Danny Milosavljevic
  0 siblings, 1 reply; 9+ messages in thread
From: Ivan Petkov @ 2019-04-04  0:52 UTC (permalink / raw)
  To: Danny Milosavljevic; +Cc: 35126

Functionally, it looks good to me (and should speed up all subsequent builds)!

Just curious, any reason why we’re doing this starting 1.26 and not earlier?

—Ivan

^ permalink raw reply	[flat|nested] 9+ messages in thread

* [bug#35126] [PATCH] gnu: rust: Enable parallel tests starting with 1.26.
  2019-04-04  0:52 ` Ivan Petkov
@ 2019-04-04 18:29   ` Danny Milosavljevic
  2019-04-04 21:59     ` Ivan Petkov
  2019-04-05 14:11     ` Danny Milosavljevic
  0 siblings, 2 replies; 9+ messages in thread
From: Danny Milosavljevic @ 2019-04-04 18:29 UTC (permalink / raw)
  To: Ivan Petkov; +Cc: 35126

[-- Attachment #1: Type: text/plain, Size: 741 bytes --]

Hi Ivan,

On Wed, 3 Apr 2019 17:52:35 -0700
Ivan Petkov <ivanppetkov@gmail.com> wrote:

> Functionally, it looks good to me (and should speed up all subsequent builds)!
> 
> Just curious, any reason why we’re doing this starting 1.26 and not earlier?

There have been problems with it in very old Rust builds which is why it was
disabled in the first place.

Icecat uses Rust 1.24 - I selfishly don't want to change that one because I
use it ;)

And I'm not sure when (in which version) they fixed the parallel tests (if
they fixed them, that is).

So I'm kinda starting in the middle...

(I'm now up to and excluding rust 1.30.1 with enabled parallel tests with -j4)
(rust 1.30.1 is still building for a few rounds)

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

^ permalink raw reply	[flat|nested] 9+ messages in thread

* [bug#35126] [PATCH] gnu: rust: Enable parallel tests starting with 1.26.
  2019-04-04 18:29   ` Danny Milosavljevic
@ 2019-04-04 21:59     ` Ivan Petkov
  2019-04-05 14:11     ` Danny Milosavljevic
  1 sibling, 0 replies; 9+ messages in thread
From: Ivan Petkov @ 2019-04-04 21:59 UTC (permalink / raw)
  To: Danny Milosavljevic; +Cc: 35126

Hi Danny,

> On Apr 4, 2019, at 11:29 AM, Danny Milosavljevic <dannym@scratchpost.org> wrote:
> 
> There have been problems with it in very old Rust builds which is why it was
> disabled in the first place.
> 
> Icecat uses Rust 1.24 - I selfishly don't want to change that one because I
> use it ;)

Can’t say that I fault you there, I really hate having to rebootstrap the entire rust chain ;)

> And I'm not sure when (in which version) they fixed the parallel tests (if
> they fixed them, that is).
> 
> So I'm kinda starting in the middle...

We can always try moving it up to earlier versions as necessary, so I’m fine with landing
this as long as all the later tests build fine!

—Ivan

^ permalink raw reply	[flat|nested] 9+ messages in thread

* [bug#35126] [PATCH] gnu: rust: Enable parallel tests starting with 1.26.
  2019-04-04 18:29   ` Danny Milosavljevic
  2019-04-04 21:59     ` Ivan Petkov
@ 2019-04-05 14:11     ` Danny Milosavljevic
  2019-04-05 15:36       ` Ivan Petkov
  1 sibling, 1 reply; 9+ messages in thread
From: Danny Milosavljevic @ 2019-04-05 14:11 UTC (permalink / raw)
  To: Ivan Petkov; +Cc: 35126

[-- Attachment #1: Type: text/plain, Size: 213 bytes --]

All tests of all Rust >= 1.26 just successfully completed--with parallel tests.

Should we limit the number of parallel jobs?  I think at least armhf is going
to be very overloaded otherwise.  To which value?

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

^ permalink raw reply	[flat|nested] 9+ messages in thread

* [bug#35126] [PATCH] gnu: rust: Enable parallel tests starting with 1.26.
  2019-04-05 14:11     ` Danny Milosavljevic
@ 2019-04-05 15:36       ` Ivan Petkov
  2019-04-06 10:48         ` Danny Milosavljevic
  0 siblings, 1 reply; 9+ messages in thread
From: Ivan Petkov @ 2019-04-05 15:36 UTC (permalink / raw)
  To: Danny Milosavljevic; +Cc: 35126

Hi Danny,

> On Apr 5, 2019, at 7:11 AM, Danny Milosavljevic <dannym@scratchpost.org> wrote:
> 
> Should we limit the number of parallel jobs?  I think at least armhf is going
> to be very overloaded otherwise.  To which value?

Do the armhf builders define that they have fewer cores available?

If not, we could cap the max jobs to 2, although it would be nice to potentially
use as many cores as are available for non-armhf builds.

—Ivan

^ permalink raw reply	[flat|nested] 9+ messages in thread

* [bug#35126] [PATCH] gnu: rust: Enable parallel tests starting with 1.26.
  2019-04-05 15:36       ` Ivan Petkov
@ 2019-04-06 10:48         ` Danny Milosavljevic
  2019-04-06 16:09           ` Ivan Petkov
  0 siblings, 1 reply; 9+ messages in thread
From: Danny Milosavljevic @ 2019-04-06 10:48 UTC (permalink / raw)
  To: Ivan Petkov; +Cc: 35126

[-- Attachment #1: Type: text/plain, Size: 687 bytes --]

On Fri, 5 Apr 2019 08:36:44 -0700
Ivan Petkov <ivanppetkov@gmail.com> wrote:

> Hi Danny,
> 
> > On Apr 5, 2019, at 7:11 AM, Danny Milosavljevic <dannym@scratchpost.org> wrote:
> > 
> > Should we limit the number of parallel jobs?  I think at least armhf is going
> > to be very overloaded otherwise.  To which value?  
> 
> Do the armhf builders define that they have fewer cores available?
> 
> If not, we could cap the max jobs to 2, although it would be nice to potentially
> use as many cores as are available for non-armhf builds.

I think the problem is RAM usage.  Arm machines have little RAM and will choke
if too many memory-expensive things run in parallel.

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

^ permalink raw reply	[flat|nested] 9+ messages in thread

* [bug#35126] [PATCH] gnu: rust: Enable parallel tests starting with 1.26.
  2019-04-06 10:48         ` Danny Milosavljevic
@ 2019-04-06 16:09           ` Ivan Petkov
  2019-04-08  1:43             ` Ivan Petkov
  0 siblings, 1 reply; 9+ messages in thread
From: Ivan Petkov @ 2019-04-06 16:09 UTC (permalink / raw)
  To: Danny Milosavljevic; +Cc: 35126



> On Apr 6, 2019, at 3:48 AM, Danny Milosavljevic <dannym@scratchpost.org> wrote:
> 
> On Fri, 5 Apr 2019 08:36:44 -0700
> Ivan Petkov <ivanppetkov@gmail.com> wrote:
> 
>> Hi Danny,
>> 
>>> On Apr 5, 2019, at 7:11 AM, Danny Milosavljevic <dannym@scratchpost.org> wrote:
>>> 
>>> Should we limit the number of parallel jobs?  I think at least armhf is going
>>> to be very overloaded otherwise.  To which value?  
>> 
>> Do the armhf builders define that they have fewer cores available?
>> 
>> If not, we could cap the max jobs to 2, although it would be nice to potentially
>> use as many cores as are available for non-armhf builds.
> 
> I think the problem is RAM usage.  Arm machines have little RAM and will choke
> if too many memory-expensive things run in parallel.

I’d anticipate that building rustc itself would be the most memory intense operation,
I think most of the individual tests aren’t anywhere nearly as big…

—Ivan

^ permalink raw reply	[flat|nested] 9+ messages in thread

* [bug#35126] [PATCH] gnu: rust: Enable parallel tests starting with 1.26.
  2019-04-06 16:09           ` Ivan Petkov
@ 2019-04-08  1:43             ` Ivan Petkov
  0 siblings, 0 replies; 9+ messages in thread
From: Ivan Petkov @ 2019-04-08  1:43 UTC (permalink / raw)
  To: Danny Milosavljevic; +Cc: 35126

Hi Danny,

When it’s time to merge this in, would you mind also changing the
`generate-checksums` procedure in (guix build cargo-utils) to omit
the `src-name` parameter (as mentioned in #35155)?

If we’re going to be having the CI bootstrap almost the whole rustc
chain for this change, we might as well combine them together and
build all at once!

Thanks,
—Ivan

^ permalink raw reply	[flat|nested] 9+ messages in thread

end of thread, other threads:[~2019-04-08  1:44 UTC | newest]

Thread overview: 9+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2019-04-03 17:00 [bug#35126] [PATCH] gnu: rust: Enable parallel tests starting with 1.26 Danny Milosavljevic
2019-04-04  0:52 ` Ivan Petkov
2019-04-04 18:29   ` Danny Milosavljevic
2019-04-04 21:59     ` Ivan Petkov
2019-04-05 14:11     ` Danny Milosavljevic
2019-04-05 15:36       ` Ivan Petkov
2019-04-06 10:48         ` Danny Milosavljevic
2019-04-06 16:09           ` Ivan Petkov
2019-04-08  1:43             ` Ivan Petkov

Code repositories for project(s) associated with this external index

	https://git.savannah.gnu.org/cgit/guix.git

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.