From: "Ludovic Courtès" <ludo@gnu.org>
To: 52283@debbugs.gnu.org
Cc: "Ludovic Courtès" <ludo@gnu.org>
Subject: [bug#52283] [PATCH 03/10] ci: Add extra jobs for tunable packages.
Date: Sat, 4 Dec 2021 21:49:17 +0100 [thread overview]
Message-ID: <20211204204924.15581-3-ludo@gnu.org> (raw)
In-Reply-To: <20211204204924.15581-1-ludo@gnu.org>
This allows us to provide substitutes for tuned package variants.
* gnu/ci.scm (package-job): Add #:suffix and honor it.
(package->job): Add #:suffix and honor it.
(%x86-64-micro-architectures): New variable.
(tuned-package-jobs): New procedure.
(cuirass-jobs): Add jobs for tunable packages.
---
gnu/ci.scm | 43 ++++++++++++++++++++++++++++++++++---------
1 file changed, 34 insertions(+), 9 deletions(-)
diff --git a/gnu/ci.scm b/gnu/ci.scm
index e1011355db..2f56554d93 100644
--- a/gnu/ci.scm
+++ b/gnu/ci.scm
@@ -28,6 +28,7 @@ (define-module (gnu ci)
#:use-module (guix grafts)
#:use-module (guix profiles)
#:use-module (guix packages)
+ #:autoload (guix transformations) (tunable-package? tuned-package)
#:use-module (guix channels)
#:use-module (guix config)
#:use-module (guix derivations)
@@ -108,9 +109,9 @@ (define* (derivation->job name drv
(#:timeout . ,timeout)))
(define* (package-job store job-name package system
- #:key cross? target)
+ #:key cross? target (suffix ""))
"Return a job called JOB-NAME that builds PACKAGE on SYSTEM."
- (let ((job-name (string-append job-name "." system)))
+ (let ((job-name (string-append job-name "." system suffix)))
(parameterize ((%graft? #f))
(let* ((drv (if cross?
(package-cross-derivation store package target system
@@ -395,21 +396,39 @@ (define package->job
(((_ inputs _ ...) ...)
inputs))))
(%final-inputs)))))
- (lambda (store package system)
+ (lambda* (store package system #:key (suffix ""))
"Return a job for PACKAGE on SYSTEM, or #f if this combination is not
-valid."
+valid. Append SUFFIX to the job name."
(cond ((member package base-packages)
(package-job store (string-append "base." (job-name package))
- package system))
+ package system #:suffix suffix))
((supported-package? package system)
(let ((drv (package-derivation store package system
#:graft? #f)))
(and (substitutable-derivation? drv)
(package-job store (job-name package)
- package system))))
+ package system #:suffix suffix))))
(else
#f)))))
+(define %x86-64-micro-architectures
+ ;; Micro-architectures for which we build tuned variants.
+ '("westmere" "ivybridge" "haswell" "skylake" "skylake-avx512"))
+
+(define (tuned-package-jobs store package system)
+ "Return a list of jobs for PACKAGE tuned for SYSTEM's micro-architectures."
+ (filter-map (lambda (micro-architecture)
+ (define suffix
+ (string-append "." micro-architecture))
+
+ (package->job store
+ (tuned-package package micro-architecture)
+ system
+ #:suffix suffix))
+ (match system
+ ("x86_64-linux" %x86-64-micro-architectures)
+ (_ '()))))
+
(define (all-packages)
"Return the list of packages to build."
(define (adjust package result)
@@ -527,10 +546,16 @@ (define source
('all
;; Build everything, including replacements.
(let ((all (all-packages))
- (job (lambda (package)
- (package->job store package system))))
+ (jobs (lambda (package)
+ (match (package->job store package system)
+ (#f '())
+ (main-job
+ (cons main-job
+ (if (tunable-package? package)
+ (tuned-package-jobs store package system)
+ '())))))))
(append
- (filter-map job all)
+ (append-map jobs all)
(cross-jobs store system))))
('core
;; Build core packages only.
--
2.33.0
next prev parent reply other threads:[~2021-12-04 20:50 UTC|newest]
Thread overview: 41+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-12-04 20:34 [bug#52283] [PATCH 00/10] Tuning packages for CPU micro-architectures Ludovic Courtès
2021-12-04 20:49 ` [bug#52283] [PATCH 01/10] Add (guix cpu) Ludovic Courtès
2021-12-04 20:49 ` [bug#52283] [PATCH 02/10] transformations: Add '--tune' Ludovic Courtès
2021-12-06 23:18 ` Thiago Jung Bauermann via Guix-patches via
2021-12-07 8:04 ` Ludovic Courtès
2021-12-07 10:32 ` zimoun
2021-12-07 14:52 ` Ludovic Courtès
2021-12-07 15:52 ` zimoun
2021-12-09 9:19 ` Ludovic Courtès
2021-12-09 10:35 ` zimoun
2021-12-10 8:49 ` Ludovic Courtès
2021-12-04 20:49 ` Ludovic Courtès [this message]
2021-12-04 20:49 ` [bug#52283] [PATCH 04/10] gnu: Add eigen-benchmarks Ludovic Courtès
2021-12-04 20:49 ` [bug#52283] [PATCH 05/10] gnu: Add xsimd-benchmark Ludovic Courtès
2021-12-04 20:49 ` [bug#52283] [PATCH 06/10] gnu: Add xtensor-benchmark Ludovic Courtès
2021-12-04 20:49 ` [bug#52283] [PATCH 07/10] gnu: ceres-solver: Mark as tunable Ludovic Courtès
2021-12-04 20:49 ` [bug#52283] [PATCH 08/10] gnu: Add ceres-solver-benchmarks Ludovic Courtès
2021-12-04 20:49 ` [bug#52283] [PATCH 09/10] gnu: libfive: Mark as tunable Ludovic Courtès
2021-12-04 20:49 ` [bug#52283] [PATCH 10/10] gnu: prusa-slicer: " Ludovic Courtès
2021-12-05 8:37 ` [bug#52283] [PATCH 00/10] Tuning packages for CPU micro-architectures Mathieu Othacehe
2021-12-06 10:38 ` Ludovic Courtès
2021-12-06 12:47 ` zimoun
2021-12-07 8:39 ` Mathieu Othacehe
2021-12-07 9:02 ` Ludovic Courtès
2021-12-06 16:48 ` Ludovic Courtès
2021-12-04 21:11 ` Ludovic Courtès
2021-12-07 9:13 ` Ludovic Courtès
2021-12-16 17:58 ` [bug#52283] [PATCH v2 00/12] " Ludovic Courtès
2021-12-16 17:58 ` [bug#52283] [PATCH v2 01/12] Add (guix cpu) Ludovic Courtès
2021-12-16 17:58 ` [bug#52283] [PATCH v2 02/12] gnu: gcc: Add 'compiler-cpu-architectures' property Ludovic Courtès
2021-12-16 17:58 ` [bug#52283] [PATCH v2 03/12] gnu: clang: " Ludovic Courtès
2021-12-16 17:58 ` [bug#52283] [PATCH v2 04/12] transformations: Add '--tune' Ludovic Courtès
2021-12-16 17:58 ` [bug#52283] [PATCH v2 05/12] ci: Add extra jobs for tunable packages Ludovic Courtès
2021-12-16 17:58 ` [bug#52283] [PATCH v2 06/12] gnu: Add eigen-benchmarks Ludovic Courtès
2021-12-16 17:58 ` [bug#52283] [PATCH v2 07/12] gnu: Add xsimd-benchmark Ludovic Courtès
2021-12-16 17:58 ` [bug#52283] [PATCH v2 08/12] gnu: Add xtensor-benchmark Ludovic Courtès
2021-12-16 17:58 ` [bug#52283] [PATCH v2 09/12] gnu: ceres-solver: Mark as tunable Ludovic Courtès
2021-12-16 17:58 ` [bug#52283] [PATCH v2 10/12] gnu: Add ceres-solver-benchmarks Ludovic Courtès
2021-12-16 17:58 ` [bug#52283] [PATCH v2 11/12] gnu: libfive: Mark as tunable Ludovic Courtès
2021-12-16 17:58 ` [bug#52283] [PATCH v2 12/12] gnu: prusa-slicer: " Ludovic Courtès
2022-01-01 14:59 ` bug#52283: [PATCH 00/10] Tuning packages for CPU micro-architectures Ludovic Courtès
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20211204204924.15581-3-ludo@gnu.org \
--to=ludo@gnu.org \
--cc=52283@debbugs.gnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this external index
https://git.savannah.gnu.org/cgit/guix.git
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.