From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp11.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id eALQN7em5mHWyAAAgWs5BA (envelope-from ) for ; Tue, 18 Jan 2022 12:38:31 +0100 Received: from aspmx1.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp11.migadu.com with LMTPS id wCZXNbem5mG/BwEA9RJhRA (envelope-from ) for ; Tue, 18 Jan 2022 12:38:31 +0100 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 69CC62CB08 for ; Tue, 18 Jan 2022 12:38:31 +0100 (CET) Received: from localhost ([::1]:48524 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1n9moo-00086v-8C for larch@yhetil.org; Tue, 18 Jan 2022 06:38:30 -0500 Received: from eggs.gnu.org ([209.51.188.92]:45662) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1n9mkU-0001jl-Sb for guix-patches@gnu.org; Tue, 18 Jan 2022 06:34:04 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:56212) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1n9mkU-0001J1-HB for guix-patches@gnu.org; Tue, 18 Jan 2022 06:34:02 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1n9mkU-00042F-BY for guix-patches@gnu.org; Tue, 18 Jan 2022 06:34:02 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#48463] gnu: Add j. Resent-From: Liliana Marie Prikler Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Tue, 18 Jan 2022 11:34:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 48463 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: elaexuotee@wilsonb.com Cc: Maxime Devos , 48463@debbugs.gnu.org Received: via spool by 48463-submit@debbugs.gnu.org id=B48463.164250563215496 (code B ref 48463); Tue, 18 Jan 2022 11:34:02 +0000 Received: (at 48463) by debbugs.gnu.org; 18 Jan 2022 11:33:52 +0000 Received: from localhost ([127.0.0.1]:49115 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1n9mkK-00041r-Eg for submit@debbugs.gnu.org; Tue, 18 Jan 2022 06:33:52 -0500 Received: from mail-wm1-f65.google.com ([209.85.128.65]:53043) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1n9mkI-00041e-Mi for 48463@debbugs.gnu.org; Tue, 18 Jan 2022 06:33:51 -0500 Received: by mail-wm1-f65.google.com with SMTP id v123so26976412wme.2 for <48463@debbugs.gnu.org>; Tue, 18 Jan 2022 03:33:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:subject:from:to:cc:date:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=gMG1u0isEcr54Dajof74VgxUhwYhT4ewYlz3afAvhbU=; b=UTuCjGgWSmG5T2lyGOkCdiwDxRxcuEpmuVDEh4cucrmOb7eLdZ+MYcs4lcyt653unU lrcY4jRuSE4UYJGBmiv9/fzWbJx0CnoNobTLiT38kY7KkfYZ1nx9g6WN/MPrrao1Yi2y xl0raOJU/to4taSzLLuO7yMI0AaqPS1dShmBYMSF8uhfVTOBO+X6s4tQTbPwgcQZV9MH h6ZFbnVBRZggc/poF2wrMVWWeYvUe9VMX5hDZIwravZ4L2WisnwroF3m6Simc3/Qqasp 6hMSVOrmKFqEfjdewMEBUD5WpM7+6umaD40Eloee7dzoIsEIIUPNpVQH47fRKVuvIKhk ca1Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:subject:from:to:cc:date:in-reply-to :references:user-agent:mime-version:content-transfer-encoding; bh=gMG1u0isEcr54Dajof74VgxUhwYhT4ewYlz3afAvhbU=; b=fB6ztLUVXUtnEGnEFS5XS+9H2PHyeaI3pMaUe+8+EckHsd36hnZ9VzNNc9lcwKPlNo lOqHidHNxL/hP7Ys6Hao3jeBr09+14djbC7bykBD4pWs5yTNO8bKMsRt54hE+9bpHEY6 LEmn7NrHMul1LgGCXKsuyOZtCW70grdwqvibj57XfemIXgIfO49hmxibpK6Hrdonbi4c BGFMnCqF+InpWmqx7vbVgbkW8xZMT9CxGCCKCq1yiOwJ9n9UR5NZe2e2RnjdfohqYIOU YWVfAXPirkoLwQG7yztFl5ZlC4RYtbHKm3QMUSmByUuM/Ch77j6QtJeWFsoVDLt07+gq W6ew== X-Gm-Message-State: AOAM5339v3HHhfXeld3H2UQzEgBQuW5HkTnPzg9aEnQWAu7SvuvtP8lf P8COT0IoknH2s9OlVHTvlbA= X-Google-Smtp-Source: ABdhPJzDwTR2aRR0zyVcqDC29i6U8+iLee9/KWIe8o/L0n/oDtLt8FktQbHU0Xd+IB/IFFwxjlKVqQ== X-Received: by 2002:a5d:6f14:: with SMTP id ay20mr24427127wrb.164.1642505624585; Tue, 18 Jan 2022 03:33:44 -0800 (PST) Received: from nijino.fritz.box (85-127-52-93.dsl.dynamic.surfer.at. [85.127.52.93]) by smtp.gmail.com with ESMTPSA id h11sm2117561wmb.12.2022.01.18.03.33.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 18 Jan 2022 03:33:44 -0800 (PST) Message-ID: From: Liliana Marie Prikler Date: Tue, 18 Jan 2022 12:33:38 +0100 In-Reply-To: <2GLLHI6YET96V.279HCOFQSZLLF@wilsonb.com> References: <3LOAUDT0FLL4U.2SOD925YP915T@wilsonb.com> <8b853d0585505ce29c9afc638b644fa34805e6c0.camel@student.tugraz.at> <293L8YPQS4CLB.3VK1B1A36XNAY@wilsonb.com> <5d30160bd2a4592459cd407f99cbd3edadb1db1b.camel@student.tugraz.at> <27DCD25Y68ZWJ.2HRC4G65PWIA7@wilsonb.com> <94f4625dcb0479d873cf60449631527e841fd457.camel@gmail.com> <2JQJMV0O718S1.31FZE8GKCTLPF@wilsonb.com> <90704c2259f576a14fb1268219e8c0dc2b3bf289.camel@telenet.be> <2P322C327XW0Q.21O5A4IFGMNDI@wilsonb.com> <72aff035c93f9f91afa54ef5b51c7381b0b02ccb.camel@gmail.com> <3MMTDZQJQ8IR6.334ZWY8AD0487@wilsonb.com> <62d37956f16c08bc4ce26e44da16dce704ddd0f8.camel@gmail.com> <24ZUUMG4QYSHN.2OS7YAMCKREUA@wilsonb.com> <83aba994536bec60f79900d551d4801c967742bd.camel@gmail.com> <25Z6NGGGNJYD1.3S7A1QLFX7I9Y@wilsonb.com> <24FU0VP6N4ZZ7.3PE5LG30BSNUQ@wilsonb.com> <2EZU214MJAIBY.3EXSPSUMS5WW5@wilsonb.com> <3T4YAXDRKZCAW.2LXSUL7VDA46J@wilsonb.com> <2GLLHI6YET96V.279HCOFQSZLLF@wilsonb.com> Content-Type: text/plain; charset="UTF-8" User-Agent: Evolution 3.42.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: "Guix-patches" X-Migadu-Flow: FLOW_IN X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1642505911; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:resent-cc: resent-from:resent-sender:resent-message-id:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=gMG1u0isEcr54Dajof74VgxUhwYhT4ewYlz3afAvhbU=; b=gT0guh8/U2izefQnyMs6mSa7w1to3WAs1ithecldOJ1cqYelwnnf9Qmv5Nw/AMJHUIiis/ 0IUgneczfMOXeyMX4C0kfebx6kuOVjnHP/KfRWWFmkUxkTlJemNp2UE1IgPpbiSpQuIIr+ /eeD3jPeKu2tvR1dJOckJllC/FHlCyLZkuG6w4BBJHp0Stc3K49k/WHDJ6QpyK0Ajrxmb0 DhCtLJM61PDMSdf+uG5oiNPyuq0wGVPCqeJaxw7IIeHXF/n20PyYTA8C0ekLpzxE1t+iMN EkkGUu3Kpj64PGb/S8GwErZLIFZvoheSUPTJcggFcbdBu2FYV7vEOrUWBfrJAw== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1642505911; a=rsa-sha256; cv=none; b=mMAgvv/wTOpXjBb2zeXXx2gcGvFrf90MLkgkNj5vRleV920V0umXPn7lPPN401nHOzIZBr xv0pLZvvzeedoLyn6PiqoB72pqImIrojnIha8B5iIlEQQ60qmH8cp5OnDejJeMheVUXKbk k1JRG92AsMKea2ebFKuzGVUvq4sISGoxAj35iLSh9xhGaRiuB4dlcg8XPs3pOyDZP0CGYd QIvHvHdcca/vqF+42MTrywp2OQWAFQTlua7Ovd2CKvjKqWC90F2/Wb//HHUJ1my32PaB5F 2ArJScyBWwCiCJxmi7pkJqZ9I+CPS7T7BOrMI5Pzb3n73rnZVJ+i6oPe7JTbjA== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=gmail.com header.s=20210112 header.b=UTuCjGgW; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none); spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -2.02 Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=gmail.com header.s=20210112 header.b=UTuCjGgW; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none); spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: 69CC62CB08 X-Spam-Score: -2.02 X-Migadu-Scanner: scn1.migadu.com X-TUID: wcKGNYIp3Mtb Am Dienstag, dem 18.01.2022 um 19:40 +0900 schrieb elaexuotee@wilsonb.com: > > FWIW I don't quite think that fat binaries are the issue here, but > > that building AVX blows up the check phase in the way we currently > > do. It's a similar issue w.r.t. check? for cross-compiling.  In my > > opinion only the base feature set can be checked unless the CPU > > supports the same features as the target. > > Oh, good. Apparently, I mis-interpreted your original concerns. > > > I think if we want to do fat binaries, the solution would be to > > build all three variants from make-j and tests only the base > > variants, and then have a non-substitutable wrapper package, that > > takes that as an input and simply provides a wrapper tuned to the > > current CPU features. > > If you want, you can then rerun the correct tests in the wrapper > > package.  The package returned by make-j could itself also provide > > three binaries just in case someone doesn't want to generate the > > wrapper package, but knows they can run ijconsole-avx2 just fine. > > WDYT? > > Slick. Let me check my understanding against yours with some > specifics: > > - Name make-j's package 'jsoftware-j-lib' which >   1. Builds all upstream stuff, and >   2. Only runs libj.so (non-avx) tests; > > - Create non-substitutable 'jsoftware-j' wrapper package which >   1. Detects SSE extensions at build time and specializes the > 'ijconsole' >      script accordingly, and >   2. Runs tests if and only if an avx or avx2 variant. > > Does those points jibe with your thoughts on the fat binaries > approach? That's roughly what I was suggesting, yes. > Glancing around at the CPU tuning stuff, it looks like tunable > packages end up getting a 'cpu-tuning' property which holds the > microartecture name passed to -march. AVX first landed in Sandy > Bridge, and AVX2 in Haswell, so assuming cpu-tuning is accessible at > build time, it should be easy enough to condition the build phase on > those. > > Regarding the check phase, here's a relevant comment in > guix/transformations.scm(tuned-package):552: > >     ;; The machine building this package may or may not be able to > run code >     ;; for MICRO-ARCHITECTURE.  Because of that, skip tests; they are > run for >     ;; the "baseline" variant anyway. > > Which I read to mean that the check phase should explicitly use > libj.so, ignoring libj-avx.so and libj-avx2.so. Running specialized > tests in a non-substitutable wrapper seems potentially better in this > particular case. > > Thoughts? I can't say that I agree with your reasoning here. Essentially, what you (or rather the J authors) are doing here is trying to outsmart the compiler, while Guix assumes, that -march=WHATEVER ought to both suffice in terms of performance and still be correct. I'm leaning more towards the assumption that Guix makes here. If you want to enable tests on your local machine, you can still do so by inheriting the transformed package. Perhaps we should add a "-- with-tests" transformation as a counterpart to the already existing -- without-tests, which would forcefully enable checks that have been disabled elsewhere. Furthermore, tuned-package would still support substitution, you just need to tell your CI to build for particular micro-architectures. If you're part of a sufficiently large dev team that uses J with Guix tooling, that makes a difference between everyone running checks Monday in the morning or simply fetching a build that has already succeeded on Sunday. Cheers