From: "Ludovic Courtès" <ludo@gnu.org>
To: Christopher Baines <mail@cbaines.net>
Cc: Josselin Poiret <dev@jpoiret.xyz>,
70494@debbugs.gnu.org, Simon Tournier <zimon.toutoune@gmail.com>,
Mathieu Othacehe <othacehe@gnu.org>,
Tobias Geerinckx-Rice <me@tobias.gr>,
Ricardo Wurmus <rekado@elephly.net>,
Christopher Baines <guix@cbaines.net>
Subject: [bug#70494] [PATCH 11/23] scripts: substitute: Untangle selecting fast vs small compressions.
Date: Thu, 16 May 2024 18:08:43 +0200 [thread overview]
Message-ID: <87le49agdw.fsf@gnu.org> (raw)
In-Reply-To: <1fbbc6d1b99d423ed58bdb126f30a309a1e99117.1713692561.git.mail@cbaines.net> (Christopher Baines's message of "Sun, 21 Apr 2024 10:42:29 +0100")
Christopher Baines <mail@cbaines.net> skribis:
> Pulling the logic up to the script makes this code more portable and not
> reliant on setting a global variable.
>
> * guix/scripts/substitute.scm (%prefer-fast-decompression?): Rename to…
> (%default-prefer-fast-decompression?): this.
> (call-with-cpu-usage-monitoring): Use multiple values to return the results
> from the thunk as well as the cpu usage.
> (display-narinfo-data): Update accordingly.
> (download-nar): Add prefer-fast-decompression? as a keyword argument, remove
> code to set! it and monitor the cpu-usage.
> (process-substitution, process-substitution/fallback): Accept and pass through
> prefer-fast-decompression? to download-nar.
> (guix-substitute): Move the cpu usage monitoring and prefer fast decompression
> switching logic here.
>
> Change-Id: I4e80b457b55bcda8c0ff4ee224dd94a55e1b24fb
[...]
> + ;; Create a hysteresis: depending on CPU usage, favor
> + ;; compression methods with faster decompression (like ztsd)
> + ;; or methods with better compression ratios (like lzip).
> + ;; This stems from the observation that substitution can be
> + ;; CPU-bound when high-speed networks are used:
> + ;; <https://lists.gnu.org/archive/html/guix-devel/2020-12/msg00177.html>.
> + ;; To simulate "slow" networking or changing conditions, run:
> + ;; sudo tc qdisc add dev eno1 root tbf rate 512kbit latency
> + ;; 50ms burst 1540 and then cancel with: sudo tc qdisc del
> + ;; dev eno1 root
This comment’s formatting was broken (by M-q I guess).
Otherwise LGTM!
next prev parent reply other threads:[~2024-05-16 16:09 UTC|newest]
Thread overview: 45+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-04-21 9:35 [bug#70494] [PATCH 00/23] Groundwork for the Guile guix-daemon Christopher Baines
2024-04-21 9:42 ` [bug#70494] [PATCH 01/23] store: database: Register derivation outputs Christopher Baines
2024-05-07 14:30 ` Ludovic Courtès
2024-04-21 9:42 ` [bug#70494] [PATCH 02/23] gnu: linux-container: Make it more suitable for derivation-building Christopher Baines
2024-05-07 14:28 ` Ludovic Courtès
2024-04-21 9:42 ` [bug#70494] [PATCH 03/23] syscalls: Add missing pieces for derivation build environment Christopher Baines
2024-05-07 14:27 ` Ludovic Courtès
2024-04-21 9:42 ` [bug#70494] [PATCH 04/23] guix: store: environment: New module Christopher Baines
2024-05-13 15:10 ` Ludovic Courtès
2024-04-21 9:42 ` [bug#70494] [PATCH 05/23] store: build-derivations: " Christopher Baines
2024-05-13 15:22 ` Ludovic Courtès
2024-04-21 9:42 ` [bug#70494] [PATCH 06/23] store: Export protocol related constants Christopher Baines
2024-05-13 15:58 ` Ludovic Courtès
2024-04-21 9:42 ` [bug#70494] [PATCH 07/23] serialization: Export read-byte-string Christopher Baines
2024-05-13 15:58 ` Ludovic Courtès
2024-04-21 9:42 ` [bug#70494] [PATCH 08/23] store: Add text-output-path and text-output-path-from-hash Christopher Baines
2024-05-13 15:59 ` Ludovic Courtès
2024-04-21 9:42 ` [bug#70494] [PATCH 09/23] store: Add validate-store-name Christopher Baines
2024-05-13 16:04 ` Ludovic Courtès
2024-04-21 9:42 ` [bug#70494] [PATCH 10/23] store: database: Add procedures for querying valid paths Christopher Baines
2024-05-16 16:04 ` Ludovic Courtès
2024-04-21 9:42 ` [bug#70494] [PATCH 11/23] scripts: substitute: Untangle selecting fast vs small compressions Christopher Baines
2024-05-16 16:08 ` Ludovic Courtès [this message]
2024-04-21 9:42 ` [bug#70494] [PATCH 12/23] scripts: substitute: Extract script specific output from download-nar Christopher Baines
2024-05-16 16:13 ` Ludovic Courtès
2024-04-21 9:42 ` [bug#70494] [PATCH 13/23] syscalls: Add unshare Christopher Baines
2024-05-16 16:14 ` Ludovic Courtès
2024-04-21 9:42 ` [bug#70494] [PATCH 14/23] scripts: perform-download: Support configuring the %store-prefix Christopher Baines
2024-05-16 16:17 ` Ludovic Courtès
2024-04-21 9:42 ` [bug#70494] [PATCH 15/23] store: Export operation-id Christopher Baines
2024-05-16 16:18 ` Ludovic Courtès
2024-04-21 9:42 ` [bug#70494] [PATCH 16/23] store: database: Log when aborting transactions Christopher Baines
2024-05-16 16:20 ` Ludovic Courtès
2024-04-21 9:42 ` [bug#70494] [PATCH 17/23] store: database: Export transaction helpers Christopher Baines
2024-05-16 16:21 ` Ludovic Courtès
2024-04-21 9:42 ` [bug#70494] [PATCH 18/23] guix: http-client: Add network-error? Christopher Baines
2024-05-16 16:23 ` Ludovic Courtès
2024-04-21 9:42 ` [bug#70494] [PATCH 19/23] http-client: Include EPIPE in network-error? Christopher Baines
2024-05-16 16:23 ` Ludovic Courtès
2024-04-21 9:42 ` [bug#70494] [PATCH 20/23] scripts: substitute: Simplify with-timeout usage Christopher Baines
2024-05-16 16:27 ` Ludovic Courtès
2024-04-21 9:42 ` [bug#70494] [PATCH 21/23] scripts: substitute: Don't enforce cached connections in download-nar Christopher Baines
2024-04-21 9:42 ` [bug#70494] [PATCH 22/23] substitutes: Move download-nar from substitutes script to here Christopher Baines
2024-04-21 9:42 ` [bug#70494] [PATCH 23/23] substitutes: Add #:keep-alive? keyword argument to download-nar Christopher Baines
2024-05-16 16:29 ` Ludovic Courtès
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87le49agdw.fsf@gnu.org \
--to=ludo@gnu.org \
--cc=70494@debbugs.gnu.org \
--cc=dev@jpoiret.xyz \
--cc=guix@cbaines.net \
--cc=mail@cbaines.net \
--cc=me@tobias.gr \
--cc=othacehe@gnu.org \
--cc=rekado@elephly.net \
--cc=zimon.toutoune@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this external index
https://git.savannah.gnu.org/cgit/guix.git
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.