Using --max-jobs

Using --max-jobs

[Phil]

Hi all,

A few questions about running multiple build jobs in parallel:

According to the manual the number of build users (eg guixbuilder01)
determines how many jobs can be run in parallel.

If we have 10 guix builder accounts and do not specify --max-jobs
explicitly, does this mean that 10 builds can be done in parallel - i.e
10 invocations of "guix build" will run in parallel, for example?

If I set the daemon to --max-jobs=4, will that restrict it to 4 builds
in parallel, even if more guix builder accounts exist?  If yes, then are
further builds queued until an available account becomes available, or do
they fail?

Are these conditions changed by the introduction of a machines.scm?  i.e. if we
have a build farm containing 4 machines, each with 10 guix builder
accounts, will parallel building still be restricted to the 10 guix
builder accounts on the parent machine (the machine with the machines.scm)?

I see that settting --max-jobs to 0 and having machines to offload to
results in no builds taking place on the parent.  I understand if no
machines are contactable the build will fail?  However, what happens if all
machines in the machines.scm are already busy (no slots available)?
Does any further build request fail, or is it queued?  If --max-jobs is
set to 0 on the parent, are builds still restricted by the number of
build users on the parent box?


Thanks!
Phil.

Re: Using --max-jobs

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 4354 bytes --]

On Fri, Jan 28, 2022 at 09:20:24AM +0000, Phil wrote:
> Hi all,
> 
> A few questions about running multiple build jobs in parallel:
> 
> According to the manual the number of build users (eg guixbuilder01)
> determines how many jobs can be run in parallel.
> 
> If we have 10 guix builder accounts and do not specify --max-jobs
> explicitly, does this mean that 10 builds can be done in parallel - i.e
> 10 invocations of "guix build" will run in parallel, for example?

IIRC the default is only 1 job at a time for each invocation. You can,
however, run two different guix commands and each will only use one
builder, for a total of 2.

> If I set the daemon to --max-jobs=4, will that restrict it to 4 builds
> in parallel, even if more guix builder accounts exist?  If yes, then are
> further builds queued until an available account becomes available, or do
> they fail?

Similar to above. If you have --max-jobs=4 and run 'guix build foo' and
'guix upgrade' each guix command will use up to 4 builders. If you try
to use more builders than you have on your system then one (or more) of
the commands will error with 'not enough builders' as it tries to
allocate more builders to hit the 4 allowed.

> Are these conditions changed by the introduction of a machines.scm?  i.e. if we
> have a build farm containing 4 machines, each with 10 guix builder
> accounts, will parallel building still be restricted to the 10 guix
> builder accounts on the parent machine (the machine with the machines.scm)?

If you offload to a machine and in your machines.scm you say that it can
do 4 parallel builds then the guix daemon on the originating machine
will try to pass 4 individual build jobs to that one machine. You can
still run guix commands on that offload machine too, and they don't
count toward the 4 jobs which were offloaded to it.

The originating machine will create its graph of the order of builds and
will pass them off one at a time until the max number of offload builds
is reached for that machine; 4 in the example you gave before. It
doesn't try to offload 4 builds, each of which spawn multiple builds of
their own.

> I see that settting --max-jobs to 0 and having machines to offload to
> results in no builds taking place on the parent.  I understand if no
> machines are contactable the build will fail?  However, what happens if all
> machines in the machines.scm are already busy (no slots available)?
> Does any further build request fail, or is it queued?  If --max-jobs is
> set to 0 on the parent, are builds still restricted by the number of
> build users on the parent box?

This isn't something I have tried before, but I believe you are correct,
if it isn't possible to offload the build (and you don't pass
--no-offload or the derivation isn't flagged as local-only) then it will
fail. If the offload machines are busy then the originating machine will
keep the builds queued and will offload them once the build slots open
up and the machine load decreases.

The --max-jobs on the parent/originator machine determine the number of
jobs which can be built per guix invocation. The max-jobs field in
machines.scm determine the number of jobs which can be passed off to an
offload machine. The --max-jobs flag on the offload machine determine
the number of jobs which can be spawned by a guix command when run on
that machine and doesn't affect the parent/originator machine.

To both complicate the matter and to try to clarify it, lets say you
have your main machine and 4 offload machines. The main machine can
offload to one offload machine, and that one can offload to 3 additional
machines. In a default configuration (max-jobs=1 everywhere), the main
machine will offload exactly one build to the offload machine
(regardless of the number of guix commands run), which will offload that
build to exactly one of its offload machines. A second guix command
which would be offloaded will wait until the first guix command is
completed entirely, since only one build is allowed to be offloaded.

> Thanks!
> Phil.

Hope that helps.

-- 
Efraim Flashner   <efraim@flashner.co.il>   רנשלפ םירפא
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]