From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp12.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id sGxuAUfC+2EFCwAAgWs5BA (envelope-from ) for ; Thu, 03 Feb 2022 12:53:43 +0100 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp12.migadu.com with LMTPS id uFDROUbC+2HXtwAAauVa8A (envelope-from ) for ; Thu, 03 Feb 2022 12:53:42 +0100 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 8832129CB9 for ; Thu, 3 Feb 2022 12:53:42 +0100 (CET) Received: from localhost ([::1]:60546 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nFagH-00082z-NE for larch@yhetil.org; Thu, 03 Feb 2022 06:53:41 -0500 Received: from eggs.gnu.org ([209.51.188.92]:49432) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nFag4-00082i-Tr for guix-devel@gnu.org; Thu, 03 Feb 2022 06:53:28 -0500 Received: from sender4-of-o51.zoho.com ([136.143.188.51]:21169) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nFag2-00068G-Hs for guix-devel@gnu.org; Thu, 03 Feb 2022 06:53:28 -0500 ARC-Seal: i=1; a=rsa-sha256; t=1643889202; cv=none; d=zohomail.com; s=zohoarc; b=S6uRCkdIrNa4PS19YkTU76+h+QKWT+5rkJYo8oqs7YygMrlKr/e2jL/8ouoUMG46PnRR/9ckPrciSERe/pdFAio49u/n1gnuSuuMRhmWQZ+EN0+cX4dpIPfo3uX3p071M/2j8vz9O9ERGbsjgH83Kml0pKXiVUtm5fKtMX4m/RA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1643889202; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:To; bh=oD1Q5PufKZDW/W2RG3ZdtbLkwGqHC21ZrmS7k9bfhEc=; b=P6h/JseHIXtgqJ/K2L2sX41lPKCU2+FVxxOGCqz7a3YVd2R5oMfDgPrqNDI1hxTZNlBx0rsHdsAVQEftVaM/4HzPvS4zqXwJGLeQELbTJvhkUEgALyXF8e3bD2/y6d+UzpFnxD6iRFe7DhfJRsnZJ39gXmVG2xtDsTIanhYe6ew= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=elephly.net; spf=pass smtp.mailfrom=rekado@elephly.net; dmarc=pass header.from= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1643889202; s=zoho; d=elephly.net; i=rekado@elephly.net; h=References:From:To:Cc:Subject:Date:In-reply-to:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding; bh=oD1Q5PufKZDW/W2RG3ZdtbLkwGqHC21ZrmS7k9bfhEc=; b=EE868Ht/wMooEBCBC9SbLFvX6ych0yYpNf0y/zDaft9wBWug5KCMqPbeBfTI2uWJ HfbshsF8VxUyzTTUsHjlXe//XYEXSmwU652Hr98FxLwD2HXsuL0+1QNjgUakfTv2eJc Gb21ZPc/s50mtD3FlFHcoY123LrVKnk4hGLe/eqE= Received: from localhost (p54ad4fd5.dip0.t-ipconnect.de [84.173.79.213]) by mx.zohomail.com with SMTPS id 1643889199066258.0330425346914; Thu, 3 Feb 2022 03:53:19 -0800 (PST) References: <871r0l9fd1.fsf@gmail.com> User-agent: mu4e 1.6.10; emacs 27.2 From: Ricardo Wurmus To: Konrad Hinsen Subject: Re: Investigating a reproducibility failure Date: Thu, 03 Feb 2022 12:41:58 +0100 In-reply-to: X-URL: https://elephly.net X-PGP-Key: https://elephly.net/rekado.pubkey X-PGP-Fingerprint: BCA6 89B6 3655 3801 C3C6 2150 197A 5888 235F ACAC Message-ID: <87a6f8415g.fsf@elephly.net> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-ZohoMailClient: External Received-SPF: pass client-ip=136.143.188.51; envelope-from=rekado@elephly.net; helo=sender4-of-o51.zoho.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Guix Devel , zimoun Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: "Guix-devel" X-Migadu-Flow: FLOW_IN X-Migadu-Country: US ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1643889222; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=oD1Q5PufKZDW/W2RG3ZdtbLkwGqHC21ZrmS7k9bfhEc=; b=CP2UTv/5wdtKv8MX1x9+GPsTjdNXPzFFVMPftxyZD006rpkPuOsDG72hGF57fuaL7KpvCZ g9dJTNohihLDObla2ZZMkOL6XVuTLU1+kSPhbmsGb+Sq398iiBvJkr3WEiqBPc4BBJYJqf qTUSrJgHxX3/Rez74VBHUw5McEJzJt1tAXpJrycatGiSbxTg2X6Xuz9FjHoSnSPx8JtjpJ l5jPDgyb2K6kdRkoyFnnLs7h4doarTzL+BWMe5tZFhxJF0XWVY17ZuKO1LRk/NfO0xH+C2 jBNjME1YljFJIYJou2J+cBPojMx9fJJVb7cj4vU6aoBt//xJh0XY1/kYtbtycA== ARC-Seal: i=2; s=key1; d=yhetil.org; t=1643889222; a=rsa-sha256; cv=fail; b=A/5Q2+XejiQGwD2DSq1NXoA6MRuF1IqZj0R/o/4B3PtxVxHK09Ah3AXOTPhGV8TRivK5IQ CHC/SfZOdaNHNzHfveitxaxQ6gsd//GYUt7QaIwFFD1OjZYRnFwUFTKB3f8Cz/P18/LOkC Wf0hKKuAu6Xp73+rg6Q/bIIzw0ZNDWjLenJG5peTw/xMxoGTOpWSSHVoL26dYisIfe4Q+G 6znP68DwgtdbYRSQzGrl+AbIYI1KgqF/Kc1D/m4ExJ4t5LZZWC5iL9NUCJEsqKJXPKDqAp sUWPtzXqMWmxBXkT38kWJ4JtCe+L++jgEtuz59ULNHGipIY2jHZdeDVdcxD4/A== ARC-Authentication-Results: i=2; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=elephly.net header.s=zoho header.b="EE868Ht/"; arc=reject ("signature check failed: fail, {[1] = sig:zohomail.com:reject}"); dmarc=none; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -1.13 Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=elephly.net header.s=zoho header.b="EE868Ht/"; arc=reject ("signature check failed: fail, {[1] = sig:zohomail.com:reject}"); dmarc=none; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: 8832129CB9 X-Spam-Score: -1.13 X-Migadu-Scanner: scn1.migadu.com X-TUID: iKOyPOEBK6xw Hi Konrad, >> CPU detection is a bottomless can of worms. > > That sounds very credible. But what can we do about this? > > There is obviously a trade-off between reproducibility and performance > here. Can we support both, in a way that users can understand and manage? So far our default approach has been to use the lowest common set of CPU instructions, which generally leads to poorly performing code. Some packages are smarter and provide different code paths for different CPUs. The resulting binary is built the same, but at runtime different parts of the code run dependent on the features the CPU reports. The case of OpenBLAS is an anomaly in that this mechanism seems to produce different binaries dependent on where it is built. When I first encountered this problem I guessed that perhaps it can only build these different code paths up to the feature set of the CPU on the build machine, so if you=E2=80=99re building with an older CPU your binary will l= ack components that would be used on newer CPUs. This is just a guess, though. Your problem is that the OpenBLAS build system doesn=E2=80=99t recognize yo= ur modern CPU. Ideally, it wouldn=E2=80=99t need to know anything about the build-time CPU to build all the different code paths for different CPU features. The only way around this =E2=80=94 retroactively =E2=80=94 is to= pretend to have an older CPU, e.g. by using qemu. In the long term it would be great if we could patch OpenBLAS to not attempt to detect CPU features at build time. I=E2=80=99m not sure this wi= ll work if it does indeed use the currently available CPU features to determine =E2=80=9Chow far up=E2=80=9D to build modules in support of certa= in CPU features / instruction sets. > There is of course the issue that we can never be sure if a build will > be reproducible in the future. But we can at least take care of the > cases where the packager is aware of non-reproducibility issues, and > make them transparent and manageable. The new =E2=80=9C--tune=E2=80=9D feature is supposed to take care of cases = like this. We would still patch the code so that by default you=E2=80=99d get a package that is reproducible (=3D you get the same exact binary no matter when or where you build it) but that many not have optimal performance. With =E2=80=9C--tune=E2=80=9D you could opt to replace that generic build with o= ne that uses features of your current CPU, using grafts to swap the generic library for the more performant library. --=20 Ricardo