From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id WG3yBvd4eWGNQgAAgWs5BA (envelope-from ) for ; Wed, 27 Oct 2021 18:06:15 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1 with LMTPS id WJGHAvd4eWFzMgAAbx9fmQ (envelope-from ) for ; Wed, 27 Oct 2021 16:06:15 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 912011D98 for ; Wed, 27 Oct 2021 18:06:14 +0200 (CEST) Received: from localhost ([::1]:45494 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mflRN-0002Jk-PG for larch@yhetil.org; Wed, 27 Oct 2021 12:06:13 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35922) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mfl18-0003nA-24 for guix-patches@gnu.org; Wed, 27 Oct 2021 11:39:06 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:38714) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mfl14-0002uA-0d for guix-patches@gnu.org; Wed, 27 Oct 2021 11:39:05 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1mfl13-0001IG-VS for guix-patches@gnu.org; Wed, 27 Oct 2021 11:39:01 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#51198] [PATCH] gnu: Add b2sum. Resent-From: Leo Famulari Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Wed, 27 Oct 2021 15:39:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 51198 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: =?UTF-8?Q?Nicol=C3=B2?= Balzarotti Cc: 51198@debbugs.gnu.org Received: via spool by 51198-submit@debbugs.gnu.org id=B51198.16353491014910 (code B ref 51198); Wed, 27 Oct 2021 15:39:01 +0000 Received: (at 51198) by debbugs.gnu.org; 27 Oct 2021 15:38:21 +0000 Received: from localhost ([127.0.0.1]:50260 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mfl0O-0001H8-Nn for submit@debbugs.gnu.org; Wed, 27 Oct 2021 11:38:21 -0400 Received: from out4-smtp.messagingengine.com ([66.111.4.28]:52151) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mfl0L-0001Gr-Ia for 51198@debbugs.gnu.org; Wed, 27 Oct 2021 11:38:18 -0400 Received: from compute2.internal (compute2.nyi.internal [10.202.2.42]) by mailout.nyi.internal (Postfix) with ESMTP id EA8375C0153; Wed, 27 Oct 2021 11:38:11 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute2.internal (MEProxy); Wed, 27 Oct 2021 11:38:11 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=famulari.name; h=date:from:to:cc:subject:message-id:references:mime-version :content-type:content-transfer-encoding:in-reply-to; s=mesmtp; bh=Fbexfu9LxdTTdNV54VJuw169A+bPHPidHJHdVEKkwM8=; b=eisJAP1xEJtf g/IMvN2U9J/rJ9TYG2OqmvoNXu8dCXvFQyVugchj5NVp6yfhhtgCyHu545Dmps8A o8i1hMbLp5d09YdAtg/5UzpMQbKMGnuKOVj1zpQpH+WBA+K9W9uBhApM1ANbRLad nCiluEAr2cfbCt8BPACEHn6oMQCvgh0= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm1; bh=Fbexfu9LxdTTdNV54VJuw169A+bPHPidHJHdVEKkw M8=; b=jkhL41SGRlQclyoYLLtCbPbHBQyNhIKlhXnBqCkKLNDJ7FAK2k7y5rCDc 9CYlw6jJAJAnn5k+/ZB0jifbdDIi6VvtfzQk0pSHyj8T/B5bNdVVVMTujxWc+rZb mLphesdPdq2a+IbFyQ7cSLX1G5ZkROwgWPDP7EuifDNyTht+YRufxRAOdzPOMeTN arN2fbKC1V21e6BILUZ55ybVjE2gy64PfIZFBE1Kee1k9dn2aNLaHX0gmRFOPwGt 07HX9mpqeb8H/cSDNuYQhC3AaFR13p80V/TmqybQOp3hbg5Fl35bkdyWWDZMPvYT whye87Ak9QuEyLsTUBzrnnO2yloIA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvtddrvdegtddgkeekucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpeffhffvuffkfhggtggugfgjsehmke erredttddunecuhfhrohhmpefnvghoucfhrghmuhhlrghrihcuoehlvghosehfrghmuhhl rghrihdrnhgrmhgvqeenucggtffrrghtthgvrhhnpeffgeekgeelfffftdevhffhtddvud evveduuddvudfgvefgheeutdejffeuleevhfenucffohhmrghinhepghhuihigrdhinhhf ohdpghhnuhdrohhrghenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrih hlfhhrohhmpehlvghosehfrghmuhhlrghrihdrnhgrmhgv X-ME-Proxy: Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed, 27 Oct 2021 11:38:11 -0400 (EDT) Date: Wed, 27 Oct 2021 11:38:08 -0400 From: Leo Famulari Message-ID: References: <878ryfjudi.fsf@guix.i-did-not-set--mail-host-address--so-tickle-me> <874k92kgaq.fsf@guix.i-did-not-set--mail-host-address--so-tickle-me> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="CSb/+FphNSCOEKlG" Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <874k92kgaq.fsf@guix.i-did-not-set--mail-host-address--so-tickle-me> X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: "Guix-patches" X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1635350774; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:resent-cc: resent-from:resent-sender:resent-message-id:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=Fbexfu9LxdTTdNV54VJuw169A+bPHPidHJHdVEKkwM8=; b=X2/AxgBWXMcJsLEwUQaOgTMNNYm4/GshifPWXS7hbkSIOJ3RFtuOoFrowmwAH+PT1EWGZQ UFfTLEJJ9/mFc6bUZJKb/KaKjvLaBtWJmtfIdJunztYt8GnAQWPMITQV7f0RdaazdnKUbw zdvoK4FTvorj+zaCLCBf6LbvojYDR9Q+kQNhXnNldG5akdC2L14MtUBD6vukSkrws9w6wo J72HoXP5FDHSPHB7iQroN17l+JQZfwFloSaCYJW3gocr403uFvX+Q8mmKcdXgrX2Gp5su4 wv7NHt25P1Th0UF9+bHRtg0YR3+zIo7RkxGizoV/GUIo1mtjRoYEdZ0y2/7ZCw== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1635350774; a=rsa-sha256; cv=none; b=koJZGThzylmf7eVX2DIgsfjDbU3GBa4sdkQ1Be3u6Ir016vPeuTolSohT5MsEInOQqtcul o/+ui5EE5F5AM4Daq7zRxxYjWv0jnUKrrkNbI4ct26/GC7L/TlovuG0nld1kfRjWBeRmuk +8KMh5R+aTk+phVKLwutlj2Yz8GyopqPPO8gfAJfzwg0NvbClr4noqdLlXvd6SHlnkxeLK rTTUOUMVfO4Db1YTB8WAP3UDXNaBaEQPmFCebAaRuL8wlAmaeOxjd7TWAa6TRDwqBpjJ/f rAEex4MA84tvUfPgoE3wY+r9hZcIMJ2FH458yU8upQj+53+MGm/hJxh9h0j8eA== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=famulari.name header.s=mesmtp header.b=eisJAP1x; dkim=fail ("headers rsa verify failed") header.d=messagingengine.com header.s=fm1 header.b=jkhL41SG; spf=pass (aspmx1.migadu.com: domain of guix-patches-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-patches-bounces@gnu.org X-Migadu-Spam-Score: 1.07 Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=famulari.name header.s=mesmtp header.b=eisJAP1x; dkim=fail ("headers rsa verify failed") header.d=messagingengine.com header.s=fm1 header.b=jkhL41SG; dmarc=none; spf=pass (aspmx1.migadu.com: domain of guix-patches-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-patches-bounces@gnu.org X-Migadu-Queue-Id: 912011D98 X-Spam-Score: 1.07 X-Migadu-Scanner: scn0.migadu.com X-TUID: K5TnCiTZerC4 --CSb/+FphNSCOEKlG Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit On Wed, Oct 27, 2021 at 10:44:45AM +0200, Nicoḷ Balzarotti wrote: > This blog post [fn:1] on guix-hpc address the "Pre-built binaries > vs. performance" dilemma. > > [fn:1] https://hpc.guix.info/blog/2018/01/pre-built-binaries-vs-performance/ It's helpful, but it doesn't look like we are using that method in Guix anymore. It was removed in this commit: https://git.savannah.gnu.org/cgit/guix.git/commit/?id=969adb235ee34decb65255e1ea821ff0e221ed3d I guess it learned how to do runtime feature detection? > I guess the easiest way is to provide a variant (b2sum-avx or something > like that) with avx enabled. Else, I'd just go with the unoptimized > version as it happens for many other packages, but let's hear from > others. The easiest thing is require local building, since it's an extremely cheap build. Only 1.5 seconds on my laptop, total (not just the build phase). With blis, one had to consider the lengthy build time. Because performance is critical for a tool like this, and because it's cheap to build, I've attached a patch to require local building. This is easier for me than creating a set of package variants that will need to be expanded for each new generation of CPUs :) --CSb/+FphNSCOEKlG Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="0001-gnu-b2sum-Build-on-the-local-machine.patch" >From 8b862425310cf631c30c823eb0fa2bfd79d36823 Mon Sep 17 00:00:00 2001 From: Leo Famulari Date: Wed, 27 Oct 2021 11:29:40 -0400 Subject: [PATCH] gnu: b2sum: Build on the local machine. * gnu/packages/crypto.scm (b2sum)[arguments]: Set '#:substitutable? #f'. --- gnu/packages/crypto.scm | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-) diff --git a/gnu/packages/crypto.scm b/gnu/packages/crypto.scm index 3acd147f25..de8bcf1d70 100644 --- a/gnu/packages/crypto.scm +++ b/gnu/packages/crypto.scm @@ -858,9 +858,18 @@ (define-public b2sum (base32 "04z631v0vzl52g73v390ask5fnzi5wg83lcjkjhpmmymaz0jn152")))) (build-system gnu-build-system) (arguments - `(#:make-flags (list (string-append "CC=" ,(cc-for-target)) - (string-append "PREFIX=" (assoc-ref %outputs "out"))) + `(;; By default, b2sum uses the compiler to generate instructions + ;; tailored to the CPU of the running machine, using "-march=native". + ;; This gives a ~1.5x speedup on a Core i5-6300U with a large dataset + ;; paged in, whereas compilation of b2sum takes ~1.5 seconds. + ;; b2sum does not support run-time feature detection: + ;; https://github.com/BLAKE2/BLAKE2/issues/1 + ;; For more information, see the discussion beginning here: + ;; https://debbugs.gnu.org/cgi/bugreport.cgi?bug=51198#13 + #:substitutable? #f #:tests? #f ; No test suite + #:make-flags (list (string-append "CC=" ,(cc-for-target)) + (string-append "PREFIX=" (assoc-ref %outputs "out"))) #:phases (modify-phases %standard-phases (add-before 'build 'change-directory -- 2.33.1 --CSb/+FphNSCOEKlG--