From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp11.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id WOvUEqKd+2HEuwAAgWs5BA (envelope-from ) for ; Thu, 03 Feb 2022 10:17:22 +0100 Received: from aspmx1.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp11.migadu.com with LMTPS id 6FwpD6Kd+2HkdwEA9RJhRA (envelope-from ) for ; Thu, 03 Feb 2022 10:17:22 +0100 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id D1997320EC for ; Thu, 3 Feb 2022 10:17:21 +0100 (CET) Received: from localhost ([::1]:46730 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nFYEy-0001VV-Vb for larch@yhetil.org; Thu, 03 Feb 2022 04:17:21 -0500 Received: from eggs.gnu.org ([209.51.188.92]:37402) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nFYEj-0001VL-4L for guix-devel@gnu.org; Thu, 03 Feb 2022 04:17:05 -0500 Received: from wout2-smtp.messagingengine.com ([64.147.123.25]:54769) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nFYEg-0003cj-TW for guix-devel@gnu.org; Thu, 03 Feb 2022 04:17:04 -0500 Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.west.internal (Postfix) with ESMTP id BAD3332020BE; Thu, 3 Feb 2022 04:16:59 -0500 (EST) Received: from mailfrontend1 ([10.202.2.162]) by compute4.internal (MEProxy); Thu, 03 Feb 2022 04:17:00 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastmail.net; h= cc:content-type:date:date:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to:to:to; s=fm2; bh=h2zRuDVTbpmZCBrMI3RwQgAuLHyoZdKB kS2u/q634ns=; b=XBa6HwM6JwdCLVCjz9usMn5/Aeqh/6LI2v5e46T6s4Vpx8JQ P7E8+mbQmsPN3UOmZVNHNI7eOXrV8vbqi124bKJOnUYWmU1c65vtJI2jfRVtgLyy zOtJYZFOgREGHfCII/uaXcC6aPSsVCkAZ19meA3/VTkQo+Id0DWNcGJPVOWcu4Ya rRDLYx4llBhsGqG540GJV3U6oujnS7bzWUdpa7absAx9zXkeIpYB1KMJuRtedjos nooIF97ziNXxPuwSFcQC7doMOYvFjEZbT8vtLYHe7d9hoDsh6mDu2YANoEwXEuq5 MYFm6yT+C07SiTkKkRXA3ByviDsSEkyBKBPGHQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:date:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:sender:subject:subject:to:to:to:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; bh=h2zRuD VTbpmZCBrMI3RwQgAuLHyoZdKBkS2u/q634ns=; b=HZN4oC2uEtI3kfT+TTxcMt d1C2HenfYcUUWLUPnzVd6A63Sa4lQXV/fMQHVzKjoDSm5UMEgNwBmINslpS0JJVK +odNoMb1bc+KPFdpN/MviC2Ky9ONnEN6SOR6wefyPuaIxR6mZV0AF/hO9aKZC+On NVBKClCCAT4FG/OeM80YLbvTGN98L6h5M+LIwDaEgDldzODbf6ypBo+tYWrknAi0 gu+eqrBgss+kdYaeRiahVwFwWOYqwrsC2WaCTAnzKbagd4jfl3z3wA270qjFuyCW eIgVtgHze1qKdIyK5p2eyacBWs5ocyTWxr4gKEaQsQjprZU1uR+YTeaBfZ787wNA == X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvvddrgeejucetufdoteggodetrfdotffvucfrrh hofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgenuceurghi lhhouhhtmecufedttdenucgotffhvedqqdfjughrqdfuuhhsphgvtghtqdhlohifucdlhe dmnecujfgurhephffvvffvufgjfhffkfggtgesthdtredttddttdenucfhrhhomhepmfho nhhrrgguucfjihhnshgvnhcuoehkohhnrhgrugdrhhhinhhsvghnsehfrghsthhmrghilh drnhgvtheqnecuggftrfgrthhtvghrnhepffeuiefftddvffekueejgfevteevteehjedu udevkedvteeijedtgeffvefhveejnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrg hmpehmrghilhhfrhhomhepkhhonhhrrggurdhhihhnshgvnhesfhgrshhtmhgrihhlrdhn vght X-ME-Proxy: Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 3 Feb 2022 04:16:58 -0500 (EST) From: Konrad Hinsen To: zimoun To: Guix Devel To: Ricardo Wurmus Subject: Re: Investigating a reproducibility failure In-Reply-To: References: <871r0l9fd1.fsf@gmail.com> Date: Thu, 03 Feb 2022 10:16:55 +0100 Message-ID: MIME-Version: 1.0 Content-Type: text/plain Received-SPF: pass client-ip=64.147.123.25; envelope-from=konrad.hinsen@fastmail.net; helo=wout2-smtp.messagingengine.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: "Guix-devel" X-Migadu-Flow: FLOW_IN X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1643879841; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:to:to:cc:mime-version:mime-version: content-type:content-type:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=h2zRuDVTbpmZCBrMI3RwQgAuLHyoZdKBkS2u/q634ns=; b=dYj+PCpDboM0hgsaNJb0DJ+7ooYTv650WqTR3bQ9J+ThkRcdQVTqbD/jpGJx1Ettfvy5PX 6/yD8GLPR86VApt1zxqdtFzrSBS101q9ejREEt336BJkHtPLub4f3ijLYuj/yd2M4sQGgv tyGGCphgUy/rLgV+8FWxoES33UfLGGgnCVAhqTikFtUs7VlRnPDqtq8KgfxkXLOajmAdHO ZdCPJZ7SxmmYAymNPWfXw5kentWxjJWHvcMrhTI85u8wtcRyL0h9JzPRuBhUE2LS5cga+7 0ZJdrym5mVrCqzB3r1dooOGy9pPfkCyQQCAWfqn63XjN6fRb9x2irzTAah9WWg== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1643879841; a=rsa-sha256; cv=none; b=C8rwiyH24+Y+pY1iDMFp+z9VUhSdJUgP3qfxaIRz+Qopgm2JY1bIHjxsO/0X65+bCQ3v/1 WUfRWL663wE7CswNKfOghDe3Mxtgo1holrf3kY+vVr4MlraOGhm4aBdaj6FfzH9l/IVT2J YKLxLuNKJE73pQdyy6jd5PjmPBU+NN0sHAbo4cxBda50/6egI7tY4bH//pybPwoaAf03Lk FKHwAyLwH1xLrp5rIYtGZz8g8M8eCNeUJL3xenTuz1/2cQDWbGGMWGr8cGOFWD3+6nLyQp uOmWLD7KtQUMqhZ7V9+M9Rzyd6V70gjmktywidGsgFcPUI3UlwoF9z/BzTxZ8g== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=fastmail.net header.s=fm2 header.b=XBa6HwM6; dkim=fail ("headers rsa verify failed") header.d=messagingengine.com header.s=fm2 header.b=HZN4oC2u; dmarc=fail reason="SPF not aligned (relaxed)" header.from=fastmail.net (policy=none); spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -2.13 Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=fastmail.net header.s=fm2 header.b=XBa6HwM6; dkim=fail ("headers rsa verify failed") header.d=messagingengine.com header.s=fm2 header.b=HZN4oC2u; dmarc=fail reason="SPF not aligned (relaxed)" header.from=fastmail.net (policy=none); spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: D1997320EC X-Spam-Score: -2.13 X-Migadu-Scanner: scn0.migadu.com X-TUID: XCQRztJXGPUJ Hi Ricardo and Simon, Thanks for your insight! I didn't even know about lscpu. The output for my laptop is shown below. I tried building on a virtual machine, and that works fine. > CPU detection is a bottomless can of worms. That sounds very credible. But what can we do about this? There is obviously a trade-off between reproducibility and performance here. Can we support both, in a way that users can understand and manage? The OpenBlas package in Guix is (or at least was, back then) written for performance. Can I, as a user, ask for a reproducible version? That could either be a generic version for any x86 architecture (preferably), or one that always builds for a given sub-architecture and then fails at runtime if the CPU doesn't match. Next: can I, as a user of dependent code, ask for reproducible versions of all my dependencies? In my case, I was packaging Python code that calls OpenBlas via NumPy. Many people in that situation don't even know what OpenBlas is. I did know, but wasn't aware of the build-time CPU detection. There is of course the issue that we can never be sure if a build will be reproducible in the future. But we can at least take care of the cases where the packager is aware of non-reproducibility issues, and make them transparent and manageable. Cheers, Konrad --8<---------------cut here---------------start------------->8--- $ lscpu Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Byte Order: Little Endian Address sizes: 39 bits physical, 48 bits virtual CPU(s): 8 On-line CPU(s) list: 0-7 Thread(s) per core: 2 Core(s) per socket: 4 Socket(s): 1 NUMA node(s): 1 Vendor ID: GenuineIntel CPU family: 6 Model: 140 Model name: 11th Gen Intel(R) Core(TM) i7-1185G7 @ 3.00GHz Stepping: 1 CPU MHz: 1800.000 CPU max MHz: 4800,0000 CPU min MHz: 400,0000 BogoMIPS: 3609.60 Virtualization: VT-x L1d cache: 192 KiB L1i cache: 128 KiB L2 cache: 5 MiB L3 cache: 12 MiB NUMA node0 CPU(s): 0-7 Vulnerability Itlb multihit: Not affected Vulnerability L1tf: Not affected Vulnerability Mds: Not affected Vulnerability Meltdown: Not affected Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via p rctl and seccomp Vulnerability Spectre v1: Mitigation; usercopy/swapgs barriers and __user poi nter sanitization Vulnerability Spectre v2: Mitigation; Enhanced IBRS, IBPB conditional, RSB fi lling Vulnerability Srbds: Not affected Vulnerability Tsx async abort: Not affected Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pg e mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm cons tant_tsc art arch_perfmon pebs bts rep_good nopl xt opology nonstop_tsc cpuid aperfmperf tsc_known_freq pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm 2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave av x f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l2 invpcid_single cdp_l2 ssbd ibrs ibpb st ibp ibrs_enhanced tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid rdt_a avx512f avx512dq rdseed adx sma p avx512ifma clflushopt clwb intel_pt avx512cd sha_ ni avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves split_lock_detect dtherm ida arat pln pts hwp hwp_ notify hwp_act_window hwp_epp hwp_pkg_req avx512vbm i umip pku ospke avx512_vbmi2 gfni vaes vpclmulqdq avx512_vnni avx512_bitalg tme avx512_vpopcntdq rdpi d movdiri movdir64b fsrm avx512_vp2intersect md_cle ar flush_l1d arch_capabilities --8<---------------cut here---------------end--------------->8---