From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp10.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id +KPbKP105mGnSAAAgWs5BA (envelope-from ) for ; Tue, 18 Jan 2022 09:06:21 +0100 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp10.migadu.com with LMTPS id eL2jIf105mHfBwAAG6o9tA (envelope-from ) for ; Tue, 18 Jan 2022 09:06:21 +0100 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id F278F2B0C2 for ; Tue, 18 Jan 2022 09:06:20 +0100 (CET) Received: from localhost ([::1]:53506 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1n9jVU-0002QE-1X for larch@yhetil.org; Tue, 18 Jan 2022 03:06:20 -0500 Received: from eggs.gnu.org ([209.51.188.92]:52924) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1n9jUF-0002NM-IV for guix-patches@gnu.org; Tue, 18 Jan 2022 03:05:05 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:55960) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1n9jUF-0001dM-4T for guix-patches@gnu.org; Tue, 18 Jan 2022 03:05:03 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1n9jUF-0004oL-02 for guix-patches@gnu.org; Tue, 18 Jan 2022 03:05:03 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#48463] gnu: Add j. Resent-From: Liliana Marie Prikler Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Tue, 18 Jan 2022 08:05:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 48463 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: elaexuotee@wilsonb.com Cc: Maxime Devos , 48463@debbugs.gnu.org Received: via spool by 48463-submit@debbugs.gnu.org id=B48463.164249309218442 (code B ref 48463); Tue, 18 Jan 2022 08:05:02 +0000 Received: (at 48463) by debbugs.gnu.org; 18 Jan 2022 08:04:52 +0000 Received: from localhost ([127.0.0.1]:48856 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1n9jU4-0004nN-2E for submit@debbugs.gnu.org; Tue, 18 Jan 2022 03:04:52 -0500 Received: from mail-wm1-f65.google.com ([209.85.128.65]:54884) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1n9jU2-0004nA-Fe for 48463@debbugs.gnu.org; Tue, 18 Jan 2022 03:04:51 -0500 Received: by mail-wm1-f65.google.com with SMTP id p18so25716537wmg.4 for <48463@debbugs.gnu.org>; Tue, 18 Jan 2022 00:04:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:subject:from:to:cc:date:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=JusHME9BgNr6IpWixmAdx89c2EIPvb734qbKttLES9I=; b=Px8oiKbDsH0wVwzCUhoMFUjfojbncIYC5Yoc/EMaVXTPwFiFMK/gjRzz6gDsvKbEgS j2/UMAIogt/xk4WGRrUZFxWTrPyXaayAoGeBV3Jqsc1O6zpszOKyVtusZQycfVwdjYaH cc9X1DgWOxK1i3pPB99PkhXEHoNCtxhn76lKD3mfz+1XX9x3CHy8MUOxoZGrvB23Z6w8 eRqTUTStR1DYr5DZWUUDve6/6VWt6muhyjX6e2YUGvsEGApiQSxmFZUjmg/KnCD9Kn3x nxYsgVEzFURNOAFLbR0cH0vTopEbI4+fgGL95juS4o5lEKy1uG+QcN4KRDSjTpvrWHbd P2RA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:subject:from:to:cc:date:in-reply-to :references:user-agent:mime-version:content-transfer-encoding; bh=JusHME9BgNr6IpWixmAdx89c2EIPvb734qbKttLES9I=; b=DrtsRFEMlkodkTUhL+OGy7SXaxhi0lutA7IbQYR/XKzQ57/xaMoORjYGiqo84mq+au ZrdaFgX0yk5H8E9qoefxZ35GbZ/Zld3SZ9MuMq2+bVbVMSV8OCEe5SVZiVoOCdF0hO9Z 5DlAYVXzcUM20iMAawYNqcoJtdybNb4QjQaDW0LJ5DfE918zokvlK1XxyhT9YlrtYI4m dDTSx1+r99hggOo29wbv1mIXnYm+I0QwrWEXHEyHju+U0EyHYZ6UPUpB1YVmZqyzTB8O nEQzOn8OW5GmjQZLbmlYvPL8goNDuvyxJ9QCz7NnXb8UfXOV9BeiGhZ8RJthJl5H7+uF zxcw== X-Gm-Message-State: AOAM530yH6/PenT/6SP6ba+2UJ+zUL2IvtbyQzoyi64ym9Ubxm1eABFo Iz6Y0MCtJXn60y2WKhFSG8s= X-Google-Smtp-Source: ABdhPJzbodP138YZbQVWcVqQP+6ofkGzID6eZwglGHuMgDnKnU6U6UJRpP6p8Y8sVa+L3fI5FUF/mw== X-Received: by 2002:a05:6000:1d91:: with SMTP id bk17mr22278992wrb.684.1642493084532; Tue, 18 Jan 2022 00:04:44 -0800 (PST) Received: from nijino.fritz.box (85-127-52-93.dsl.dynamic.surfer.at. [85.127.52.93]) by smtp.gmail.com with ESMTPSA id b15sm12498179wrr.50.2022.01.18.00.04.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 18 Jan 2022 00:04:43 -0800 (PST) Message-ID: From: Liliana Marie Prikler Date: Tue, 18 Jan 2022 09:04:42 +0100 In-Reply-To: <3T4YAXDRKZCAW.2LXSUL7VDA46J@wilsonb.com> References: <3LOAUDT0FLL4U.2SOD925YP915T@wilsonb.com> <8b853d0585505ce29c9afc638b644fa34805e6c0.camel@student.tugraz.at> <293L8YPQS4CLB.3VK1B1A36XNAY@wilsonb.com> <5d30160bd2a4592459cd407f99cbd3edadb1db1b.camel@student.tugraz.at> <27DCD25Y68ZWJ.2HRC4G65PWIA7@wilsonb.com> <94f4625dcb0479d873cf60449631527e841fd457.camel@gmail.com> <2JQJMV0O718S1.31FZE8GKCTLPF@wilsonb.com> <90704c2259f576a14fb1268219e8c0dc2b3bf289.camel@telenet.be> <2P322C327XW0Q.21O5A4IFGMNDI@wilsonb.com> <72aff035c93f9f91afa54ef5b51c7381b0b02ccb.camel@gmail.com> <3MMTDZQJQ8IR6.334ZWY8AD0487@wilsonb.com> <62d37956f16c08bc4ce26e44da16dce704ddd0f8.camel@gmail.com> <24ZUUMG4QYSHN.2OS7YAMCKREUA@wilsonb.com> <83aba994536bec60f79900d551d4801c967742bd.camel@gmail.com> <25Z6NGGGNJYD1.3S7A1QLFX7I9Y@wilsonb.com> <24FU0VP6N4ZZ7.3PE5LG30BSNUQ@wilsonb.com> <2EZU214MJAIBY.3EXSPSUMS5WW5@wilsonb.com> <3T4YAXDRKZCAW.2LXSUL7VDA46J@wilsonb.com> Content-Type: text/plain; charset="UTF-8" User-Agent: Evolution 3.42.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: "Guix-patches" X-Migadu-Flow: FLOW_IN X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1642493181; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:resent-cc: resent-from:resent-sender:resent-message-id:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=JusHME9BgNr6IpWixmAdx89c2EIPvb734qbKttLES9I=; b=fKt+9nxhJy3D02IQZJgCbC8yDnPaRGMBNolfVbiN/0rNYEGMEg8baykP7XhuceeDzxWUEV VmXXOPcEPrinB0ErGZsgmfErRkojyNnxPGGbqz4xsCsNrhngNhiMfy9VtBiKw+JVKyD7O4 I13yLi5LDakz+Qp4UJKsbz62R6p0G1Sjdl2qdd8nzYP1J2we0gMCMQYb8TxCyXBUpHds2t VOlMNFG7ryOfHWMmyvOto9wYkuu6TjRNlNZg7JXDSqBmyuYNzfIPo5N6U/CyMueImwHONh tkvWljpXkss2Rl6Vo4OfRD7lt1RHqIUlsMyzt/Z5ETZsnutrNRvY+7AOmRIYkQ== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1642493181; a=rsa-sha256; cv=none; b=lTy/8RTK9vPbl7Q3q5Kxy0VzxFTeky2tj6K3NLdLJ1JLoa888L7LhGAYkeBMh+LpxVsOFa qFzYkbW0RqkESmb+Jkdwa6p1rt0OrIOH7L1uTZuvtyq5FYTLYnTiN8rMSxVZ62VRagwkHN eTgNsr9buZSJzTqYDAZ6GJAySXMjkA3hGUZjJ21Kb9PxWfBo+nnqa4v9RoZ7GFFom1riZs 3rX/Rn8WXGKGdtY3pZPTViaHH43AgiCgoHMY0XiKJwAqd65sPWflzkjp3VlwTwIzXdWhmL /AgQ5/JIETH9ov+cr+u0yUQcEpKB5YBqQ294JzvOyUJnmfPjoijleH9pXEyGsQ== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=gmail.com header.s=20210112 header.b=Px8oiKbD; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none); spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -2.02 Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=gmail.com header.s=20210112 header.b=Px8oiKbD; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none); spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: F278F2B0C2 X-Spam-Score: -2.02 X-Migadu-Scanner: scn1.migadu.com X-TUID: UesUaSrD4rbN Hi, Am Dienstag, dem 18.01.2022 um 13:01 +0900 schrieb elaexuotee@wilsonb.com: > > > > The (guix cpu) code is not meant to be used at runtime as the > > comment to ijconsole implies it wants it to be used.  Rather, you > > should use it at compile time to bake in the correct tuning > > parameters, and it should ideally also support "cross-tuning" > > (though in that case you probably need to disable the tests because > > your CPU won't make sense of the code). > > Hrm... Mind if I ask about the rationale? We could go back to > scraping /proc/cpuinfo, but I suspect that's not addressing the issue > you have in mind. > > > ijconsole not only fails that, but it also fails at an even more > > basic task in Guix' launcher scripts, which is actually knowing the > > thing you launch. (We hardcode everything, period.) > > Okay. I see the logic in that. > > The original patch indeed hard-coded paths in the ijconsole script, > but that required having ijconsole be a wrapper package. I thought I > was being clever by eliminating a "needless" wrapper, but it sounds > like we can just bring it back. As far as I can see, you still need a wrapper to provide a symlink stripped of the version. I think you can combine the CPU detection code with that (see below), so you can have a wrapper procedure, which strips the version LOGICAL OR detects the CPU. > > Now I could just disable everything AVX-related in J and push > > something that works on x86 and amd64, but since you do claim that > > AVX is important to J, there are also other options one could take > > here.  One is to implement tuning for this package the correct way, > > though since it reinvents build systems, that's probably going to > > be a hard one. The other would be to define package variants (e.g. > > jsoftware-j-with-avx) and use those -- there ought to be an old > > blog post on Guix HPC detailing the rationale behind doing that. > > Not including AVX would be a hard hit. > > Morally, the ijconsole script just wraps the libj*.so libs into a > "fat binary". From what I understand, fat binaries are (were?) an > accepted solution in the HPC world. Heck, the libj*.so total up to a > whopping 15Mb, which feels more like slim that fat, anyway. > > Questions floating around in my head: > > - What is wrong with detecting CPU features in a wrapper vs in the > binary? > - If a wrapper is theoretically okay, how can we fix the current one? > - Is such a "proper fix" massively more subtle and bug-prone than the > naive solution? Why? FWIW I don't quite think that fat binaries are the issue here, but that building AVX blows up the check phase in the way we currently do. It's a similar issue w.r.t. check? for cross-compiling. In my opinion only the base feature set can be checked unless the CPU supports the same features as the target. I think if we want to do fat binaries, the solution would be to build all three variants from make-j and tests only the base variants, and then have a non-substitutable wrapper package, that takes that as an input and simply provides a wrapper tuned to the current CPU features. If you want, you can then rerun the correct tests in the wrapper package. The package returned by make-j could itself also provide three binaries just in case someone doesn't want to generate the wrapper package, but knows they can run ijconsole-avx2 just fine. WDYT? > > WDYT?  Is AVX worth the trouble? > > Personally, I really do not like the jsoftware-j-with-* variants > workaround. If you say that tuning support is the only other > acceptable solution, then I'll bite the bullet and see what is > possible. > > P.S. Just throwing this out there, but if you'd like, I'd be willing > to do a mob session together. It seems like we are in compatible > timezones. I'm in UTC+1 and have a day job from 8am to 4pm (plus delta and bus routes), which normally make me unable to reply from roughly 6:30am to 8pm. I'm also a little shy when it comes to letting strangers hear my voice and experiencing healthy vaccine side effects atm. Cheers