From: "Ludovic Courtès" <ludo@gnu.org>
To: Maxim Cournoyer <maxim.cournoyer@gmail.com>
Cc: 43773@debbugs.gnu.org
Subject: bug#43773: [PATCH] offload: Improve load normalization and configurability.
Date: Mon, 05 Oct 2020 16:06:09 +0200 [thread overview]
Message-ID: <87tuv8wywe.fsf@gnu.org> (raw)
In-Reply-To: <20201004032112.5916-1-maxim.cournoyer@gmail.com> (Maxim Cournoyer's message of "Sat, 3 Oct 2020 23:21:12 -0400")
Hi,
Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:
> Fixes <https://issues.guix.gnu.org/43773>.
>
> The computed normalized load was previously obtained by dividing the load
> average as found in /proc/loadavg by the number of parallel builds defined for
> a build machine.
>
> This normalized didn't allow to compare machines with different number of
^
> cores, as the load average reported by can be as high as the number of cores;
^
Missing words.
> thus comparing that value to a fixed threshold of 2.0 would mean machines with
> multiple cores were more likely to be flagged as overloaded compared to single
> core machines.
>
> This can be fixed by normalizing using the available number of cores instead
> of the number of parallel jobs.
Indeed, good catch!
> * guix/scripts/offload.scm (<build-machine>)[overload-threshold]: New field.
> (node-load): Modify to return a normalized load value between 0 and 1, taking
> into account the number of cores available.
> (normalized-load): Remove procedure.
> (report-load): New procedure.
> (choose-build-machine): Adjust to use the modified 'node-load' and the new
> 'report-load' and 'build-machine-overload-threshold' procedures.
> (check-machine-status): Adjust.
> * doc/guix.texi (Daemon Offload Setup): Document the offload scheduler and the
> new 'overload-threshold' field.
>
> doc/guix.texi | 30 +++++++++++++++++++++-
> guix/scripts/offload.scm | 54 ++++++++++++++++++++++++----------------
> 2 files changed, 62 insertions(+), 22 deletions(-)
Nice.
[...]
> (define (node-load node)
> - "Return the load on NODE. Return + if NODE is misbehaving."
> + "Return the load on NODE, a normalized value between 0.0 and 1.0. The value
> +is derived from /proc/loadavg and normalized according to the number of
> +logical cores available, to give a rough estimation of CPU usage. Return
> +1.0 (fully loaded) if NODE is misbehaving."
> (let ((line (inferior-eval '(begin
> (use-modules (ice-9 rdelim))
> (call-with-input-file "/proc/loadavg"
> read-string))
> - node)))
> - (if (eof-object? line)
> - +inf.0 ;MACHINE does not respond, so assume it is infinitely loaded
> + node))
> + (ncores (inferior-eval '(begin
> + (use-modules (ice-9 threads))
> + (current-processor-count))
> + node)))
> + (if (or (eof-object? line) (eof-object? ncores))
> + 1.0 ;MACHINE does not respond, so assume it is fully loaded
Returning 1.0 now is akin to returning +\x1e before, meaning that the
machine will never be picked up, is that right?
What if one sets overload-threshold = 1.0, the machine would still be
picked up, no?
> + (if (and node
> + (or (not threshold) (< load threshold))
I think we can assume that THRESHOLD is always a number, including
possible +inf.0.
Thanks,
Ludo\x19.
next prev parent reply other threads:[~2020-10-05 14:07 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-10-03 3:05 bug#43773: guix offload scheduler/load balancer throttles itself Maxim Cournoyer
2020-10-04 3:21 ` bug#43773: [PATCH] offload: Improve load normalization and configurability Maxim Cournoyer
2020-10-04 7:59 ` Andreas Enge
2020-10-05 14:06 ` Ludovic Courtès [this message]
2020-10-05 17:07 ` Maxim Cournoyer
2020-10-08 15:04 ` Maxim Cournoyer
2020-10-05 21:00 ` bug#43773: guix offload scheduler/load balancer throttles itself zimoun
2020-10-06 3:44 ` Maxim Cournoyer
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://guix.gnu.org/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87tuv8wywe.fsf@gnu.org \
--to=ludo@gnu.org \
--cc=43773@debbugs.gnu.org \
--cc=maxim.cournoyer@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/guix.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).