From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id 0G8dG4Z+u2GVEwAAgWs5BA (envelope-from ) for ; Thu, 16 Dec 2021 18:59:34 +0100 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0 with LMTPS id UEHfFoZ+u2EDFAAA1q6Kng (envelope-from ) for ; Thu, 16 Dec 2021 17:59:34 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id F1538F837 for ; Thu, 16 Dec 2021 18:59:30 +0100 (CET) Received: from localhost ([::1]:42816 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mxv2Q-0008Sg-4j for larch@yhetil.org; Thu, 16 Dec 2021 12:59:30 -0500 Received: from eggs.gnu.org ([209.51.188.92]:54124) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mxv1z-0007g7-R7 for guix-patches@gnu.org; Thu, 16 Dec 2021 12:59:03 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:53381) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mxv1z-0007zG-IG for guix-patches@gnu.org; Thu, 16 Dec 2021 12:59:03 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1mxv1z-0006ka-Hw for guix-patches@gnu.org; Thu, 16 Dec 2021 12:59:03 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#52283] [PATCH v2 05/12] ci: Add extra jobs for tunable packages. Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Thu, 16 Dec 2021 17:59:03 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 52283 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 52283@debbugs.gnu.org Cc: Ludovic =?UTF-8?Q?Court=C3=A8s?= Received: via spool by 52283-submit@debbugs.gnu.org id=B52283.163967753625884 (code B ref 52283); Thu, 16 Dec 2021 17:59:03 +0000 Received: (at 52283) by debbugs.gnu.org; 16 Dec 2021 17:58:56 +0000 Received: from localhost ([127.0.0.1]:36682 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mxv1r-0006jJ-KB for submit@debbugs.gnu.org; Thu, 16 Dec 2021 12:58:56 -0500 Received: from eggs.gnu.org ([209.51.188.92]:53234) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mxv1m-0006hq-WD for 52283@debbugs.gnu.org; Thu, 16 Dec 2021 12:58:51 -0500 Received: from [2001:470:142:3::e] (port=44368 helo=fencepost.gnu.org) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mxv1h-0007tb-9Z; Thu, 16 Dec 2021 12:58:45 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:References:In-Reply-To:Date:Subject:To: From; bh=KkmRNdObKbMR8mxqnuQObOQAbzMbid4vqB6zPpNAsqY=; b=N3mi0GJ+hOegqmg37KaB /r274tIWOeHAw4lB3wGzyXqGgeRKLS7Y0pW2PbT3EE8VWR8K4sGcyDxaC+owYy7FrcV5v/xQZIFJS iAWKxskTUD77DMbbVmS8CDQC8qX7ft7LgNOFSZTLG+OGB04O2JvKXcHAb5ZY6SH3quURIqLCmYesx Ag8oL8vydtHn20pwEQ7PWBBoBc3AuUZBkd2vCHSC8IBw5K8ACQyUw5o/p45voHgTbkg2BLSHIPfJK AQYSSRho2vaNsGu+gu00DSCAX48tTfb13Bv0oZ6CJ02wYODOIW3GTIIlV+PlvwGYTRvP3hMHwVQPP ZxDTOgCN4xH3ow==; Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=42202 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mxv1h-0000YX-8y; Thu, 16 Dec 2021 12:58:45 -0500 From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Date: Thu, 16 Dec 2021 18:58:20 +0100 Message-Id: <20211216175827.2077-6-ludo@gnu.org> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20211216175827.2077-1-ludo@gnu.org> References: <87lf0wkcam.fsf@gnu.org> <20211216175827.2077-1-ludo@gnu.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: "Guix-patches" X-Migadu-Flow: FLOW_IN X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1639677571; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding:resent-cc: resent-from:resent-sender:resent-message-id:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=KkmRNdObKbMR8mxqnuQObOQAbzMbid4vqB6zPpNAsqY=; b=N4knFXZSXiWkzbATQ6ZHB9nah3yh+L5etTbyWUYUS07ZSTfhsfZQpVRnI93VNiauC3ZjQP zr5MMIA8oR7mFFzP24lV/sr2DRu8SFlrudOB1LBVSGgnfl5wQpuQTx4OvOtuSjptka5yfB c3t4q4VyfeKLZCMMSVN00Fm6yGq9+VciAj2iaUFcHsfG5j2UBqlh53B4lqh5/hwUs/yIhz ozfzZKLKcOtoRqG9MXbgzUgitIk07lu8b7s3/v3FD7d09M0krAX+bZny+pC16TSr/GDrni 5qvxOWg5BLXIU1Hi3o+YRbOcIOOCdVDFhMAmYcB/cEoUWB/GucfEbOCuWaDlVA== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1639677571; a=rsa-sha256; cv=none; b=oaPvvlwhF9NZVupAMzRdNWpW7k3+7u1wdpX3d3KSUmT20GAuEac3WsTdG/TFgdd2609nbR 848k6bW4E6ZYQvxtSwRNl8jxX6oFzRZ3Z1HqmUfJYusQDAeOqRJY7aXb8VVm81nj1dIJOk z9q3FowEUfB6J1Omv9dzrU2UqgfRdmnKEO4te54C4eWrXSDlw2+syiNt92azWK+Rc3GTUw w6MYtWoHv+eRW129eg2jA+xzXGO3orUfTyYa+/8TtK5UNDORim2gBKm7FEDa9v9yE6pN4U uf260yrdnKlwQN+CAgYoZ/M54LQYIChyzxi4Bj48kC9YHDSRoLg7e4MdR5V95w== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=gnu.org header.s=fencepost-gnu-org header.b=N3mi0GJ+; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -2.69 Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=gnu.org header.s=fencepost-gnu-org header.b=N3mi0GJ+; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: F1538F837 X-Spam-Score: -2.69 X-Migadu-Scanner: scn0.migadu.com X-TUID: /AvaB3rKb6mt This allows us to provide substitutes for tuned package variants. * gnu/ci.scm (package-job): Add #:suffix and honor it. (package->job): Add #:suffix and honor it. (%x86-64-micro-architectures): New variable. (tuned-package-jobs): New procedure. (cuirass-jobs): Add jobs for tunable packages. --- gnu/ci.scm | 43 ++++++++++++++++++++++++++++++++++--------- 1 file changed, 34 insertions(+), 9 deletions(-) diff --git a/gnu/ci.scm b/gnu/ci.scm index 6039af8f07..35fd583f75 100644 --- a/gnu/ci.scm +++ b/gnu/ci.scm @@ -28,6 +28,7 @@ (define-module (gnu ci) #:use-module (guix grafts) #:use-module (guix profiles) #:use-module (guix packages) + #:autoload (guix transformations) (tunable-package? tuned-package) #:use-module (guix channels) #:use-module (guix config) #:use-module (guix derivations) @@ -107,9 +108,9 @@ (define* (derivation->job name drv (#:timeout . ,timeout))) (define* (package-job store job-name package system - #:key cross? target) + #:key cross? target (suffix "")) "Return a job called JOB-NAME that builds PACKAGE on SYSTEM." - (let ((job-name (string-append job-name "." system))) + (let ((job-name (string-append job-name "." system suffix))) (parameterize ((%graft? #f)) (let* ((drv (if cross? (package-cross-derivation store package target system @@ -395,21 +396,39 @@ (define package->job (((_ inputs _ ...) ...) inputs)))) (%final-inputs))))) - (lambda (store package system) + (lambda* (store package system #:key (suffix "")) "Return a job for PACKAGE on SYSTEM, or #f if this combination is not -valid." +valid. Append SUFFIX to the job name." (cond ((member package base-packages) (package-job store (string-append "base." (job-name package)) - package system)) + package system #:suffix suffix)) ((supported-package? package system) (let ((drv (package-derivation store package system #:graft? #f))) (and (substitutable-derivation? drv) (package-job store (job-name package) - package system)))) + package system #:suffix suffix)))) (else #f))))) +(define %x86-64-micro-architectures + ;; Micro-architectures for which we build tuned variants. + '("westmere" "ivybridge" "haswell" "skylake" "skylake-avx512")) + +(define (tuned-package-jobs store package system) + "Return a list of jobs for PACKAGE tuned for SYSTEM's micro-architectures." + (filter-map (lambda (micro-architecture) + (define suffix + (string-append "." micro-architecture)) + + (package->job store + (tuned-package package micro-architecture) + system + #:suffix suffix)) + (match system + ("x86_64-linux" %x86-64-micro-architectures) + (_ '())))) + (define (all-packages) "Return the list of packages to build." (define (adjust package result) @@ -527,10 +546,16 @@ (define source ('all ;; Build everything, including replacements. (let ((all (all-packages)) - (job (lambda (package) - (package->job store package system)))) + (jobs (lambda (package) + (match (package->job store package system) + (#f '()) + (main-job + (cons main-job + (if (tunable-package? package) + (tuned-package-jobs store package system) + '()))))))) (append - (filter-map job all) + (append-map jobs all) (cross-jobs store system)))) ('core ;; Build core packages only. -- 2.33.0