From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id AMuHEUnRq2FAbwEAgWs5BA (envelope-from ) for ; Sat, 04 Dec 2021 21:36:25 +0100 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0 with LMTPS id iOhKDUnRq2ETSwAA1q6Kng (envelope-from ) for ; Sat, 04 Dec 2021 20:36:25 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id AD7EB3E0A for ; Sat, 4 Dec 2021 21:36:24 +0100 (CET) Received: from localhost ([::1]:57480 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mtble-0007wA-Hv for larch@yhetil.org; Sat, 04 Dec 2021 15:36:22 -0500 Received: from eggs.gnu.org ([209.51.188.92]:59310) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mtblL-0007vj-4I for guix-patches@gnu.org; Sat, 04 Dec 2021 15:36:03 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:43606) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mtblK-0005o9-Sx for guix-patches@gnu.org; Sat, 04 Dec 2021 15:36:02 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1mtblK-0000yZ-EA for guix-patches@gnu.org; Sat, 04 Dec 2021 15:36:02 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#52283] [PATCH 00/10] Tuning packages for CPU micro-architectures Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sat, 04 Dec 2021 20:36:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 52283 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 52283@debbugs.gnu.org Cc: Ludovic =?UTF-8?Q?Court=C3=A8s?= X-Debbugs-Original-To: guix-patches@gnu.org Received: via spool by submit@debbugs.gnu.org id=B.16386501143688 (code B ref -1); Sat, 04 Dec 2021 20:36:02 +0000 Received: (at submit) by debbugs.gnu.org; 4 Dec 2021 20:35:14 +0000 Received: from localhost ([127.0.0.1]:55152 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mtbkS-0000xE-CZ for submit@debbugs.gnu.org; Sat, 04 Dec 2021 15:35:14 -0500 Received: from lists.gnu.org ([209.51.188.17]:58576) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mtbkQ-0000x7-JO for submit@debbugs.gnu.org; Sat, 04 Dec 2021 15:35:07 -0500 Received: from eggs.gnu.org ([209.51.188.92]:59090) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mtbkN-0007Ui-Ou for guix-patches@gnu.org; Sat, 04 Dec 2021 15:35:05 -0500 Received: from [2001:470:142:3::e] (port=44842 helo=fencepost.gnu.org) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mtbkK-0005b1-5M; Sat, 04 Dec 2021 15:35:03 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:Date:Subject:To:From:in-reply-to: references; bh=kMeqSPIHiZoI+FvIU1i0vtGJ1hrPsS2V61xXqZYteqw=; b=jBxANkV0ePfCwk PLwIWAya18r6OBdJpATPZKuzUiqQdgmcH5mcoXJz4UbDZY2Akcf7es76rvFpmXrx/R8tY9RvuHOUS LHzSZ5wi/rVkd8sreXg+Dkw5j9dvtCJgK8fFxNRzuZcATO4+ULF8dUgk/jiJ6XoodGETgziERhvBH nMRMmfYGw1124slycZTITf4fY8vyOaqfY85K1NPRPb8M0qDpGRW1nUoHSJvAJgs7WVv3eBow4iVsf R/qfNukkhydEFeGOXBpCQ05iQxZ5a79Rnp+vm2gllHNEsMNP+MvUr5r5Jfbpdlg1xIYkJR3uxTgLY rGtK2Sifjpqe7gnBUbTw==; Received: from 91-160-117-201.subs.proxad.net ([91.160.117.201]:54570 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mtbkK-0003SD-1y; Sat, 04 Dec 2021 15:35:00 -0500 From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Date: Sat, 4 Dec 2021 21:34:47 +0100 Message-Id: <20211204203447.15200-1-ludo@gnu.org> X-Mailer: git-send-email 2.33.0 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: "Guix-patches" X-Migadu-Flow: FLOW_IN X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1638650184; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:resent-cc: resent-from:resent-sender:resent-message-id:list-id:list-help: list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=kMeqSPIHiZoI+FvIU1i0vtGJ1hrPsS2V61xXqZYteqw=; b=nuTcAVliWdWkr0avc5KOCJUgVS91Hj9GjMNTHmvQs6sBsdUpJjTxNJv2Hn+bqSWuGYCnmm 6mSETwmo0fc3RhorU3+uJhpXlYUT4WK+OqaygMpoO0naQHiURbQiA+K8GS0QffZrQwvl9g IinRyb2sW4jCzespQtGQaEMB3WvgD2KZGY5TbWvvT4aUA+NMxypXGm00CKdxzGQEIBW2XS wPRllV8ShSxkD5yFlYmjdJ+TEifI/4FIqXIdqRLyrkK7yubdk0cf/fa5CphQO/jzAwv0ob aYRoroK2A0aX9QkIqVRQBN83gCtRQ1q+7n0AiA4psj70AeBl3/XyUMAa386dbQ== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1638650184; a=rsa-sha256; cv=none; b=raqdJ7vStviwoRjnXs8ps6PM522aE4M0C3y/uZ8UILhWxlvkLnsqoAtYTe9o1H367AY6F+ tGs1y+Rusbawx3feqG3yKhy9r2J4XQwPmH3xh042vwBV1iWOZKGRDusZrrjMg2rZreLDJb g0zOuNhagFWsApIvfxVqS+ox6PZ8I3jyQi8WNqmYGj5sA/ZKSEj4TvORDkMhYG/zhY4dk9 OG8LG/Q7vV9eQuei9Gaj3ptzQ4nqPrgfvZEMmRvP3JcuvEEVcRflaDu1xLcqDWVFqbV5kJ SxrpaVYwkibirTKmFK1NNSmaBPqi1lwxAG4t0N6s7ZQUlCRFfGXhMcWmxxSOOg== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=gnu.org header.s=fencepost-gnu-org header.b=jBxANkV0; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -2.83 Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=gnu.org header.s=fencepost-gnu-org header.b=jBxANkV0; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: AD7EB3E0A X-Spam-Score: -2.83 X-Migadu-Scanner: scn0.migadu.com X-TUID: OYr21s1SRavP Hello Guix! This patch series is an attempt to allow users to build or substitute packages for the very CPU they are using, as opposed to using a generic binary that targets the baseline architecture—e.g., x86_64 without AVX extensions. As a reminder, my take on this is that The Right Thing is for code to select optimized implementations for the host CPU at load time, using (possibly hand-crafted) “function multi-versioning”: https://hpc.guix.info/blog/2018/01/pre-built-binaries-vs-performance/ Now, there’s at least one situation where developers don’t do “the right thing”: C++ header-only libraries. It turns out header-only libraries with #ifdef’d SIMD code are quite common: Eigen, xsimd, xtensor, etc. Every user of those libs has to be compiled with ‘-march=native’ to take advantage of those SIMD-optimized routines and there’s little hope of seeing those libraries implement load-time or run-time selection¹. This patch set implements “package multi-versioning”, where a package can have different variants users may choose from: baseline, haswell, skylake, etc. This is implemented as a package transformation option, ‘--tune’. Without any argument, ‘--tune’ grafts tuned package variants for each package that has the ‘tunable?’ property. For example: guix shell eigen-benchmarks --tune -- benchBlasGemm 16 16 16 100 100 runs one of the Eigen benchmarks tuned for the host CPU, because ‘eigen-benchmarks’ is marked as “tunable”. This is achieved not by passing ‘-march=native’, because the daemon might be running on a separate machine with a different CPU, but by identifying the ‘-march’ value corresponding to the host CPU and passing ‘-march’ to the compiler, via a wrapper. On my skylake laptop, that gives a noticeable difference on the GEMM benchmark of Eigen and good results on the xtensor benchmarks too, unsurprisingly. I don’t have figures for higher-level applications, but it’d be nice to benchmark some of Eigen’s dependents for instance, as shown by: guix graph -M2 -t reverse-package eigen | xdot -f fdp - If you could run such benchmarks, that’d be great! :-) Things like Fenics may benefit from it. Nix people chose to introduce separate system types for the various x86_64 micro-architecture levels: x86_64-linux-v1, x86_64-linux-v2, etc.² I think this is somewhat wasteful and unpractical though. It’s also unclear whether those levels, defined in the new x86_64 psABI³, are a viable abstraction: vendors seem to be mixing features rather than really following the accumulative pattern that those levels imply. Thoughts? Ludo’. ¹ https://listengine.tuxfamily.org/lists.tuxfamily.org/eigen/2021/11/msg00006.html ² https://discourse.nixos.org/t/nix-2-4-released/15822 ³ https://gitlab.com/x86-psABIs/x86-64-ABI/-/blob/master/x86-64-ABI/low-level-sys-info.tex Ludovic Courtès (10): Add (guix cpu). transformations: Add '--tune'. ci: Add extra jobs for tunable packages. gnu: Add eigen-benchmarks. gnu: Add xsimd-benchmark. gnu: Add xtensor-benchmark. gnu: ceres-solver: Mark as tunable. gnu: Add ceres-solver-benchmarks. gnu: libfive: Mark as tunable. gnu: prusa-slicer: Mark as tunable. Makefile.am | 1 + doc/guix.texi | 54 ++++++++++++++ gnu/ci.scm | 43 ++++++++--- gnu/packages/algebra.scm | 79 ++++++++++++++++++++ gnu/packages/cpp.scm | 23 ++++++ gnu/packages/engineering.scm | 10 ++- gnu/packages/maths.scm | 49 ++++++++++++- guix/cpu.scm | 137 +++++++++++++++++++++++++++++++++++ guix/transformations.scm | 134 ++++++++++++++++++++++++++++++++++ tests/transformations.scm | 20 +++++ 10 files changed, 538 insertions(+), 12 deletions(-) create mode 100644 guix/cpu.scm base-commit: 052f56e5a614854636563278ee5a2248b3609d87 prerequisite-patch-id: 7e5c2bb5942496daf01a7f6dfc1b0b5b214f1584 -- 2.33.0