From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp12.migadu.com ([2001:41d0:306:2d92::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms9.migadu.com with LMTPS id COjbAfp0tmT4rAAASxT56A (envelope-from ) for ; Tue, 18 Jul 2023 13:18:18 +0200 Received: from aspmx1.migadu.com ([2001:41d0:306:2d92::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp12.migadu.com with LMTPS id UHxuAfp0tmTXAAAAauVa8A (envelope-from ) for ; Tue, 18 Jul 2023 13:18:18 +0200 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 32DBD45C58 for ; Tue, 18 Jul 2023 13:18:17 +0200 (CEST) Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=gmail.com header.s=20221208 header.b=aOdSU+ij; spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org"; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1689679097; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:resent-cc:resent-from:resent-sender: resent-message-id:in-reply-to:in-reply-to:references:references: list-id:list-help:list-unsubscribe:list-subscribe:list-post: dkim-signature; bh=+8e90LHLplUzpPF2kyU1dYM7L5uw2vEi6Sezqe98+m8=; b=oRY/8MMnXFoDDmCMuOa9aXiS21GxQbVJfCeg/oYV+ggp8iLGGS4UkyfinVPMbR1co6VeHy Kean7fBY/p3dk3T2735cT65kdPF2tk/yjDO1NpAntWucDfpVKW06l6dJ3k8NxqaX9hRP2s 1eSKC+8BGmZoo0gVz5eA5/4ExqYo8qdyAu8it7w9Pyy46yoUFgP1hWYVDVZOrTZEf9O4dN WV7DTygOj+O4KhqZmZQkIky3973kK1ct5orV+8bu0obp504C1ySuWEEgYo6jEpwrAmlaXZ DzC4I5eWfH9404kxaxemJNoHU7CmljYvk75jdmU8SmsJ4maYcK8Ei81338tHuQ== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1689679097; a=rsa-sha256; cv=none; b=Tk1PxyY4b5D8YJ6+qky1wUDhZi5fKf0X/VMpNgCWPEZ5iQ6HcajEqm8/lbsc2W8g4TgoF2 PCLMUMS+CguT7Wog3bJu+MvAfNQHkYY5xBB6hiXwSasgRYc7P/+bqOr0loKkCKe/GlZ1WU py0LyJ6yO2KSrbC66czG/OrNDFlNJ14jXpg2GC8cSzIgChQEa5MFI/eBA2SRG3BrECwbYM M1tTDaUTvrWYMmnsIdE9n9MeRVCwDs6vaFtBpYnPCbHeFN8vEi8kJS/xwZlqFwyoDNH/48 QGXloj30V+ynKRWZzVvdf10/JylW41/Ck+jgCIpxNrcMSdnDMUrHQ2pJLDuedw== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=gmail.com header.s=20221208 header.b=aOdSU+ij; spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org"; dmarc=none Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qLiiT-0007ch-Ky; Tue, 18 Jul 2023 07:18:05 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qLiiQ-0007cJ-Np for guix-patches@gnu.org; Tue, 18 Jul 2023 07:18:03 -0400 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qLiiQ-0005LC-Fx for guix-patches@gnu.org; Tue, 18 Jul 2023 07:18:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1qLiiQ-0006KR-Ag for guix-patches@gnu.org; Tue, 18 Jul 2023 07:18:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#64188] [PATCH 0/8] More package tuning Resent-From: Efraim Flashner Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Tue, 18 Jul 2023 11:18:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 64188 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Cc: Josselin Poiret , Tobias Geerinckx-Rice , Simon Tournier , Mathieu Othacehe , Christopher Baines , 64188@debbugs.gnu.org, Ricardo Wurmus Received: via spool by 64188-submit@debbugs.gnu.org id=B64188.168967907124308 (code B ref 64188); Tue, 18 Jul 2023 11:18:02 +0000 Received: (at 64188) by debbugs.gnu.org; 18 Jul 2023 11:17:51 +0000 Received: from localhost ([127.0.0.1]:51704 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qLiiE-0006K0-O2 for submit@debbugs.gnu.org; Tue, 18 Jul 2023 07:17:51 -0400 Received: from mail-wm1-x32d.google.com ([2a00:1450:4864:20::32d]:52553) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qLiiC-0006Jk-60 for 64188@debbugs.gnu.org; Tue, 18 Jul 2023 07:17:49 -0400 Received: by mail-wm1-x32d.google.com with SMTP id 5b1f17b1804b1-3fba8e2aa52so57886585e9.1 for <64188@debbugs.gnu.org>; Tue, 18 Jul 2023 04:17:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1689679062; x=1692271062; h=in-reply-to:content-disposition:mime-version:references :mail-followup-to:message-id:subject:cc:to:from:date:sender:from:to :cc:subject:date:message-id:reply-to; bh=+8e90LHLplUzpPF2kyU1dYM7L5uw2vEi6Sezqe98+m8=; b=aOdSU+ijn37+vB8Bl7iCg7S6pKMROmF8QE58YTlX4fUP6TiRNyACeA1Xrdt1wgO0eN doDOT8K/voVc7008jhjWh77rKqdPX0+DVcidRGyyScd2alVuFqHgipMiB9h78//e4n4K XkR6RJHraxEdJiDu4gRwJ6MQ9LnHSta8J2OjCW7ycHvctTDtADxY5e8YAcVe7TVcT5gN T3T5sm6LnfxqUA7kmq0dSJ3+/1TXoEG/KyWhgoYD8RBkUUJui90BeGexWnp+8DG4ERVC hmsq44EMivzTxGgGYQg8AMUpPb8/gSt2Rd25KeX5MeiR1bGc3fIMf2XizeKEM1sWN4cr QHaQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689679062; x=1692271062; h=in-reply-to:content-disposition:mime-version:references :mail-followup-to:message-id:subject:cc:to:from:date:sender :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=+8e90LHLplUzpPF2kyU1dYM7L5uw2vEi6Sezqe98+m8=; b=THXeuh6OUKgDv7LMjdt4FrLt5jQBuAjAGW8X3HG0elwnFoUEvjtm/+Uk5EZveAmiik bcQmMw8o6tTdgNj6S2yQJQdrIrkpOvlNICbUAsCn/8td9gudFtJ7EVo9o2CKVacGgaM/ ME9R8ir+hCv8dZSb1ME7WxVQzyu5p8FB4Jn9s4LPHJ3JuasLbESMF1PMaw5qr6ckE38C RxcQno38lN5nPN3CnAeiGTlB9xnJuJHVPktr8U/5YXlSqcJVWhJtCISQHcfU/Z832hWz MvJt+FRcGYHx4L87m/sSzod6pgmXIsNXfTFTtziIMFsi2LtJ2MJyF1wfh8UTxEB2vMiv Wkzg== X-Gm-Message-State: ABy/qLaNg/Jo/rPdXC5If1L3Udnghhk9+B6fmaI9pKmCtoD4LeEfePJV zhxUmm3o4WFJ78PrPZIu5xI= X-Google-Smtp-Source: APBJJlGKI5tTVRZdXoGgBmbKMavwhGjjmljIK4F5g/9fo6Cijc4JKJCxv7yski8/7o2HQgRjd5tCLw== X-Received: by 2002:a1c:e915:0:b0:3fb:e054:903f with SMTP id q21-20020a1ce915000000b003fbe054903fmr1677004wmc.36.1689679062110; Tue, 18 Jul 2023 04:17:42 -0700 (PDT) Received: from localhost ([2a02:ed3:914:a500:bdef:3269:70ba:bfc8]) by smtp.gmail.com with ESMTPSA id 25-20020a05600c025900b003fc00702f65sm9998426wmj.46.2023.07.18.04.17.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 18 Jul 2023 04:17:41 -0700 (PDT) Date: Tue, 18 Jul 2023 14:17:39 +0300 From: Efraim Flashner Message-ID: Mail-Followup-To: Efraim Flashner , Ludovic =?UTF-8?Q?Court=C3=A8s?= , 64188@debbugs.gnu.org, Christopher Baines , Josselin Poiret , Mathieu Othacehe , Ricardo Wurmus , Simon Tournier , Tobias Geerinckx-Rice References: <87352fz2m9.fsf_-_@gnu.org> <871qhbzvae.fsf_-_@gnu.org> <87wmyypmts.fsf@inria.fr> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="6MEHSWBLO7koTwxw" Content-Disposition: inline In-Reply-To: <87wmyypmts.fsf@inria.fr> X-PGP-Key-ID: 0x41AAE7DCCA3D8351 X-PGP-Key: https://flashner.co.il/~efraim/efraim_flashner.asc X-PGP-Fingerprint: A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: guix-patches-bounces+larch=yhetil.org@gnu.org X-Migadu-Flow: FLOW_IN X-Migadu-Country: US X-Migadu-Scanner: mx2.migadu.com X-Migadu-Spam-Score: 2.26 X-Spam-Score: 2.26 X-Migadu-Queue-Id: 32DBD45C58 X-TUID: mEX0me0nEFNE --6MEHSWBLO7koTwxw Content-Type: multipart/mixed; boundary="3aqIM1pGO/veOiQe" Content-Disposition: inline --3aqIM1pGO/veOiQe Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Jul 17, 2023 at 05:41:35PM +0200, Ludovic Court=C3=A8s wrote: > Hi, >=20 > Efraim Flashner skribis: >=20 > > On Thu, Jul 13, 2023 at 05:27:21PM +0200, Ludovic Court=C3=A8s wrote: >=20 > [...] >=20 > >> It looks like we=E2=80=99re now adding the =E2=80=98set-microarchitect= ure=E2=80=99 phase > >> unconditionally, not just for go. For example: > >>=20 > >> --8<---------------cut here---------------start------------->8--- > >> $ ./pre-inst-env guix build --tune eigen-benchmarks --log-file > >> guix build: tuning eigen-benchmarks@3.4.0 for CPU skylake > >> https://ci.guix.gnu.org/log/djwka1jhzhk08yb23as83yk5hysn0pky-eigen-ben= chmarks-3.4.0 > >> $ wget -qO- https://ci.guix.gnu.org/log/djwka1jhzhk08yb23as83yk5hysn0p= ky-eigen-benchmarks-3.4.0 |gunzip -c| grep -C3 set-micro > >> phase `reset-gzip-timestamps' succeeded after 0.0 seconds > >> starting phase `compress-documentation' > >> phase `compress-documentation' succeeded after 0.0 seconds > >> starting phase `set-microarchitecture' > >> Setting GOAMD to "v3". > >> phase `set-microarchitecture' succeeded after 0.0 seconds > >> @ build-succeeded /gnu/store/pdz0g9q2yd9i1jkbhk2rnbfa88ngvffw-eigen-be= nchmarks-3.4.0.drv - > >> --8<---------------cut here---------------end--------------->8--- > >>=20 > >> What I had in mind was to have a procedure similar to =E2=80=98tuning-= compiler=E2=80=99 > >> that would return a wrapper around the =E2=80=9Cgo=E2=80=9D binary tha= t would set > >> =E2=80=98GOAMD=E2=80=99 (or similar). That way the change would be we= ll isolated. > >>=20 > >> Could you look into providing a patch for that? > >>=20 > >> Thanks in advance! > >>=20 > >> Ludo=E2=80=99. > > > > That's actually really surprising to me. I thought that if you tried to > > add a phase after a non-existent phase then it just wouldn't be added. >=20 > Actually I thought so too. :-) >=20 > But anyway, the point is that we=E2=80=99re modifying phases unconditiona= lly > (whether or not this has an effect), and it would be nicer to avoid it > IMO. >=20 > > I tried just wrapping the call to the 'go' binary itself so that every > > time 'go' was called it would also set the environment variable setting > > the optimization level but I was having a hard time working that. While > > experimenting I did change what I had written to check for the > > 'setup-go-environment phase, and if it existed to add the optimization > > at the end of that phase. > > > > I have the part with wrapping the go binary as a WIP, and when it's > > ready I'll post both parts so we can choose which one seems better. I > > like the idea of go being wrapped, it makes it easier to just add in the > > optimizations whenever go is added to a package. On the other hand I > > like the extra phase, since it's already done :) >=20 > Not a valid argument! :-) We can discuss the implementation on IRC if > you want. It might be that we can slightly generalize =E2=80=98tuning-co= mpiler=E2=80=99 > so that it works for go (perhaps there=E2=80=99s an option like =E2=80=98= -march=E2=80=99 that we > could use instead of setting =E2=80=98GOAMD=E2=80=99?). I found -goarch, but it's for cross-compiling and wouldn't take x86_64-v3 as an input. The attached diff has 2 parts, the first wraps the go binary (and only the go binary) with GOAMD or the like. The second part is commented out, but is how I would've fixed the extra 'set-microarchitecture phase. I'm pretty certain that I have the logic correct, but I'm not certain that it's being applied. It probably needs (system* "export" "GOAMD" #$psabi) or something similar, when I tried adjusting syncthing to display (getenv "GOAMD") I was getting #f. --=20 Efraim Flashner =D7=A8=D7=A0=D7=A9=D7=9C=D7=A4 = =D7=9D=D7=99=D7=A8=D7=A4=D7=90 GPG key =3D A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351 Confidentiality cannot be guaranteed on emails sent or received unencrypted --3aqIM1pGO/veOiQe Content-Type: text/plain; charset=utf-8 Content-Disposition: attachment; filename="wrap-go-binary.diff" Content-Transfer-Encoding: quoted-printable diff --git a/guix/transformations.scm b/guix/transformations.scm index 92d9c89c0e..0665f33178 100644 --- a/guix/transformations.scm +++ b/guix/transformations.scm @@ -441,6 +441,9 @@ (define tuning-compiler #~(begin (use-modules (ice-9 match)) =20 + (define psabi #$(gcc-architecture->micro-architecture-level + micro-architecture)) + (define* (search-next command #:optional (path (string-split (getenv "PATH") @@ -469,10 +472,25 @@ (define tuning-compiler (match (search-next (basename command)) (#f (exit 127)) (next - (apply execl next + (if (and (search-next "go") + (string=3D? next (search-next "go"))) + (cond + ((string-prefix? "arm" psabi) + (setenv "GOARM" (string-take-right psabi 1))) + ((string-prefix? "powerpc" psabi) + (setenv "GOPPC64" psabi)) + ((string-prefix? "x86_64" psabi) + (setenv "GOAMD" (string-take-right psabi 2))) + (else #t)) + '()) + (apply + execl next (append (cons next arguments) + (if (and (search-next "go") + (string=3D? next (search-next "go"))) + '() (list (string-append "-march=3D" - #$micro-architecture))))))= )))) + #$micro-architecture))))))= ))))) =20 (define program (program-file (string-append "tuning-compiler-wrapper-" micro-archit= ecture) @@ -489,7 +507,8 @@ (define tuning-compiler (for-each (lambda (program) (symlink #$program (string-append bin "/" progr= am))) - '("cc" "gcc" "clang" "g++" "c++" "clang= ++"))))))) + '("cc" "gcc" "clang" "g++" "c++" "clang= ++" + "go"))))))) =20 (define (build-system-with-tuning-compiler bs micro-architecture) "Return a variant of BS, a build system, that ensures that the compiler = that @@ -564,27 +583,31 @@ (define (build-system-with-tuning-compiler bs micro-a= rchitecture) =20 (bag (inherit lowered) - (arguments + #;(arguments (substitute-keyword-arguments (bag-arguments lowered) ;; We add the tuning parameter after the default GO flags are se= t. ((#:phases phases '%standard-phases) - #~(modify-phases #$phases - (add-after 'setup-go-environment 'set-microarchitecture - (lambda _ - (cond - ((string-prefix? "arm" #$psabi) - (setenv "GOARM" (string-take-right #$psabi 1)) - (format #t "Setting GOARM to ~s." - (getenv "GOARM"))) - ((string-prefix? "powerpc" #$psabi) - (setenv "GOPPC64" #$psabi) - (format #t "Setting GOPPC64 to ~s." - (getenv "GOPPC64"))) - ((string-prefix? "x86_64" #$psabi) - (setenv "GOAMD" (string-take-right #$psabi 2)) - (format #t "Setting GOAMD to ~s.\n" - (getenv "GOAMD"))) - (else #t)))))))) + ;; This phase is only in the go-build-system. + #~(if (assoc-ref #$phases 'setup-go-environment) + (modify-phases #$phases + (replace 'setup-go-environment + (lambda* args + (apply (assoc-ref #$phases 'setup-go-environment) a= rgs) + (cond + ((string-prefix? "arm" #$psabi) + (setenv "GOARM" (string-take-right #$psabi 1)) + (format #t "Setting GOARM to ~s." + (getenv "GOARM"))) + ((string-prefix? "powerpc" #$psabi) + (setenv "GOPPC64" #$psabi) + (format #t "Setting GOPPC64 to ~s." + (getenv "GOPPC64"))) + ((string-prefix? "x86_64" #$psabi) + (setenv "GOAMD" (string-take-right #$psabi 2)) + (format #t "Setting GOAMD to ~s.\n" + (getenv "GOAMD"))) + (else #t))))) + #$phases)))) (build-inputs ;; Arrange so that the compiler wrapper comes first in $PATH. `(("tuning-compiler" ,(tuning-compiler micro-architecture)) --3aqIM1pGO/veOiQe-- --6MEHSWBLO7koTwxw Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAABCAAdFiEEoov0DD5VE3JmLRT3Qarn3Mo9g1EFAmS2dNAACgkQQarn3Mo9 g1FeRRAApJ8+8JI5fKn8Bgxt5cqle/sbgsr3eDJo50jLAH+/D/jFLNqqoAtgN87+ 5RKiqOOBSQgT6Xkxj9DJIOyjEnGyX2xvGCVC4iyTD6fNA1FbzbTYBGsnFjE68Axn Y3xl/ghr+oTRxMYOFjoifREJh+PjpGKpSmIR4Vp0TxKymzGOS8TMKr+RCf9l9BYr ONpfqHPD37Ekvo/6KztGEq2lbF8A3gKPPU86PMc291oJhnv+OZcyPBAQHMCYXKln ce0Mz5ogF6y+eWlmrY45sP5+T/3ucX81lOjG0aplmHyEvrKv6rXnrD4I7mi36rbX mthbN4qfZjuFAepkArn01KaRv+aYRUo4Rj9/MNwsIUnatIxM0oeRVAEWtLpHAQ/f 25j3pgIRfDfpf/Mii7M7xtr+7u4HrI0mdoTe+h2SrIojv3QHxUhYzGGS463NCA2U i//55OUgXpn8/5ttYJAQhlVGiXAeVOo1gAV5vPEMOIn/MWSpvxYerDXyFmtBGFVe /ic1wa+T/wDledgfsBFl7x2kCNdx1LVw1QsljKKFtzBRwS4OY0imuDPA5e0UAPyY z10i8s2FnstK9GyOVi5rnKlRpViSqBo4jUiEn1QsF3Hx4WLQT4VDMUoMRpBMZtMS cv6F0t/l0bpCtxTdZgLZDdC80tTfU4Cp518rpA8YNuUTCFCqeN0= =eyLC -----END PGP SIGNATURE----- --6MEHSWBLO7koTwxw--