From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp10.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms5.migadu.com with LMTPS id oMfTMtEQuGLpsAAAbAwnHQ (envelope-from ) for ; Sun, 26 Jun 2022 09:54:57 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp10.migadu.com with LMTPS id MJ7cMdEQuGJHWwEAG6o9tA (envelope-from ) for ; Sun, 26 Jun 2022 09:54:57 +0200 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 934A63F7AC for ; Sun, 26 Jun 2022 09:54:57 +0200 (CEST) Received: from localhost ([::1]:50078 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1o5N6e-0006rd-Pr for larch@yhetil.org; Sun, 26 Jun 2022 03:54:56 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38886) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1o5N6J-0006rJ-I1 for guix-devel@gnu.org; Sun, 26 Jun 2022 03:54:35 -0400 Received: from xavier.telenet-ops.be ([2a02:1800:120:4::f00:14]:44482) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1o5N6H-0003RM-2m for guix-devel@gnu.org; Sun, 26 Jun 2022 03:54:35 -0400 Received: from ptr-bvsjgyhxw7psv60dyze.18120a2.ip6.access.telenet.be ([IPv6:2a02:1811:8c09:9d00:3c5f:2eff:feb0:ba5a]) by xavier.telenet-ops.be with bizsmtp id njuV2700C4UW6Th01juVFJ; Sun, 26 Jun 2022 09:54:30 +0200 Message-ID: <60c35901007af0f3bf02f23814d49973e0c13772.camel@telenet.be> Subject: Rust reprodubility -- .rmeta and shadow-rs From: Maxime Devos To: guix-devel@gnu.org Date: Sun, 26 Jun 2022 09:54:29 +0200 Content-Type: multipart/signed; micalg="pgp-sha512"; protocol="application/pgp-signature"; boundary="=-9r7FDaC2wbBQZMKQxIdo" User-Agent: Evolution 3.38.3-1 MIME-Version: 1.0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=telenet.be; s=r22; t=1656230070; bh=lzTuAdUGT4j2XhYOqc1/HpsX5uk4pUcKfBRz9NGmO6U=; h=Subject:From:To:Date; b=PaFnlGRQ1W4XYaTPQSEI6pIC6If4wCH40VLgUZsqDe2APPQRmpkwgnR/HerP83pXL GH4dT9hamQSs16Y17FhX/s59venYEObSTfa1WnxbFf+azWSvMeb8tm2X083zRef2ux giDaIQqlJhxsfnmv/dr8EQ9aXL4W0gTCWPkdbqbBuzovyB3YMrVMp6WW5BbOJrGHxI koss+SJonAKn50C8sQPfPblWfTMZLogofueAzLJRq/ORK0LR8j3/uvejIWl3IWVcd4 n/oMcmJIJBGPZ2R54dAslXxAZE3Kd5RR3Ghyy5PepNzUOW6zSl9AkICE2KUHUUO08+ MH3bHzDW4rn4Q== Received-SPF: pass client-ip=2a02:1800:120:4::f00:14; envelope-from=maximedevos@telenet.be; helo=xavier.telenet-ops.be X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: "Guix-devel" X-Migadu-Flow: FLOW_IN X-Migadu-To: larch@yhetil.org X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1656230097; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=lzTuAdUGT4j2XhYOqc1/HpsX5uk4pUcKfBRz9NGmO6U=; b=KNYqG4YYZG8+wZcY1ni5mytv74Z89w9ubGvcAHQSdc+hTxjqxvinkuXU44i/y0W/fS+cKh Hl/pv7aHBN9I2l73APtmphjiRPdWIERH1gZGm0ED14Pa6SUccKPrmxCNvuUedM+KWBbwZd Tpx8cwQvv56YKN8R44yPcxZaqDJ2dpF81HarrphCa0ReRGNXN0VJXTK1HIALe8DPe3huDZ 5ukxoYRAnxKfuYrKwIE2cw8N7mmh+8wToXQ3OW3RIPZRzRH4WTLI+cx3gV7SjmfQbxj3BY +ncAeE4P0pphGyB4Lis6SGAeTiYlYdvfvsCvTxG2QdJUss+QAkj7OMn1dPOBwg== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1656230097; a=rsa-sha256; cv=none; b=dYHLnzn96EZ4orGvioGCAxLoXkEEPFReLU2E9P0lJSrXpYHe7Iqjk+9O1Ds+YfDKTv4637 /msjMfZcOxfhnucJ49k+8OlTWsJkXIB1cWrAMb7SMQj0qXRdo1rbZjxZMpgr8Kn5rhgW/G 246psjwikWD9eDt8/u2hZVlom9/PYes/t+5a7aaSZOiixMVxENiv6CskeHBq622eit3nVe R9B4tv+3H9wyiIJZHRDS9mnt3xpqD+4h3R//9y4kO5OO/gTh6GTql7uYdBuMOIV0OorQFa AODSXSqSXIuWzPrR/zlYkIjYsudYMox5j3moGndRb5byuoV7CiRGO+8vcecMrA== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=telenet.be header.s=r22 header.b=PaFnlGRQ; dmarc=pass (policy=none) header.from=telenet.be; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -9.56 Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=telenet.be header.s=r22 header.b=PaFnlGRQ; dmarc=pass (policy=none) header.from=telenet.be; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: 934A63F7AC X-Spam-Score: -9.56 X-Migadu-Scanner: scn0.migadu.com X-TUID: BEsfZ9Lj+ZdI --=-9r7FDaC2wbBQZMKQxIdo Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi, There was some mail about irreproducibility in Rust, but I couldn't find it anymore. Anyway, I found a potential cause: rust-shadow-rs embeds timestamps (even though it nominally respects SOURCE_DATE_EPOCH???) and the ordering of definitions it generates is based on a hash map (and hence, irreproducible). The crate id is based on a hash over the source code, so this irreproducibility can cause build failures if substitutes are used. By removing the time stamp and sorting the definitions, 'nushell' successfully built on ci.guix.gnu.org whereas it previously failed to build on ci.guix.gnu.org but built successfully locally (with antioxidant), and IIRC the (antioxidated) 'rust-nu-command' is now reproducible: Anyway, here the patch I used: ("rust-shadow-rs" ,#~((add-after 'unpack 'fixup-source-date-epoch (lambda _ ;; TODO: it nominally supports SOURCE_DATE_EPOCH, yet something thing= s go wrong, ;; as the shadow.rs still contains the unnormalised time stamp ... ;; For now, do a work-around. (substitute* '("src/lib.rs" "src/env.rs") (("BuildTime::Local\\(Local::now\\(\\)\\)\\.human_format\\(\\)") (object->string "[timestamp expunged for reproducibility]")) (("time\\.human_format\\(\\)") "\"[timestamp expunged for reproducibility]\".to_string()") (("time\\.to_rfc3339_opts\\(SecondsFormat::Secs, true)") "\"[timestamp expunged for reproducibility]\".to_string()") (("time\\.to_rfc2822\\(\\)") "\"[timestamp expunged for reproducibility]\".to_string()")))) (add-after 'unpack 'more-reproducibility ;; by default, it uses a hashmap= , leading to an irreproducible ordering in shadow.rs and hence an irreprodu= cible .rmeta (TODO: upstream?) (lambda _ (substitute* "src/lib.rs" ; sort (("\\(k, v\\) in self\\.map\\.clone\\(\\)") "(k, v) in std::collections::BTreeMap::from_iter(self.map.clone().iter())= ") (("self\\.write_const\\(k, v\\)") "self.write_const(k, v.clone())") (("self\\.map\\.keys\\(\\)") "std::collections::BTreeSet::from_iter= (self.map.keys())")))))) Maybe that was the cause? Greetings, Maxime. --=-9r7FDaC2wbBQZMKQxIdo Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYrgQtRccbWF4aW1lZGV2 b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7uMlAQD14NlFQzeihVFaWs+TImF88onk yajsCrFkOXsytyCBEwEA6dq0nYzMX0io3xWxJnsnSSYmOM3qjWxK+8VSJQAa3Ak= =1MOo -----END PGP SIGNATURE----- --=-9r7FDaC2wbBQZMKQxIdo--