From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1 ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id aAcWOf+YC2FWGgAAgWs5BA (envelope-from ) for ; Thu, 05 Aug 2021 09:53:35 +0200 Received: from aspmx1.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1 with LMTPS id gKqqNP+YC2EjNAAAbx9fmQ (envelope-from ) for ; Thu, 05 Aug 2021 07:53:35 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 330DD247B9 for ; Thu, 5 Aug 2021 09:53:35 +0200 (CEST) Received: from localhost ([::1]:50820 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mBYC5-0007zQ-Q8 for larch@yhetil.org; Thu, 05 Aug 2021 03:53:33 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55294) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mBYBv-0007yv-TQ for help-guix@gnu.org; Thu, 05 Aug 2021 03:53:23 -0400 Received: from flashner.co.il ([178.62.234.194]:59968) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mBYBu-00062t-0a for help-guix@gnu.org; Thu, 05 Aug 2021 03:53:23 -0400 Received: from localhost (unknown [141.226.169.107]) by flashner.co.il (Postfix) with ESMTPSA id D429540037; Thu, 5 Aug 2021 07:52:47 +0000 (UTC) Date: Thu, 5 Aug 2021 10:52:09 +0300 From: Efraim Flashner To: Julien Lepiller Subject: Re: Non deterministic package Message-ID: Mail-Followup-To: Julien Lepiller , phodina , phodina via References: <25E731BD-DF0D-4A60-9440-E53DE1FC6FE0@lepiller.eu> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="bd/DeQ9X3Ap0l8Mf" Content-Disposition: inline In-Reply-To: <25E731BD-DF0D-4A60-9440-E53DE1FC6FE0@lepiller.eu> X-PGP-Key-ID: 0x41AAE7DCCA3D8351 X-PGP-Key: https://flashner.co.il/~efraim/efraim_flashner.asc X-PGP-Fingerprint: A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351 Received-SPF: pass client-ip=178.62.234.194; envelope-from=efraim@flashner.co.il; helo=flashner.co.il X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: help-guix@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: phodina via Errors-To: help-guix-bounces+larch=yhetil.org@gnu.org Sender: "Help-Guix" X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1628150015; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post; bh=x/IHM1cXPaio1XpMF78y0L52T2oIRZtHedzROFqpQRs=; b=Rk+oN+ywyeBVkVtHjhNjO7+OQMzfuXLyxw74X9orGTfJpV0uLKU1Ttc9lkjFcrS/SXL1iX r3lCUiH0jTzGGTMA2Hrpg7lvh0ck+KgHcDiQu5WlhokwIG3uQ+Bsz87P7t8SMjIDprW2W/ 5LMadmvu/6GAV9k6rlFk1Kh1p8mHYiiVubHQkuVwVlHcoPaU/lBUFSCtqo5E9/VJgKa6Jk NhGsn6YYivGr9PBBH/6XssNPP64kV7wLdZKf+Z5DPhx7VE5btqglJX8Wid/7VpqveDGv2T oXN6fpSRKWvTlW6u5pjiO3fsKHWjfs3jS4W57qLEv6npMHY6LaRn7i5BkJT27Q== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1628150015; a=rsa-sha256; cv=none; b=eBhUAG/yzAVC8qqfz7jkPo5tLVkjBL9kc61lFlLAysshxdwOQ4uy0K6GIwBgwRtjq5gVmL sJpAw5cMGzRDOP2s1rmSBnNAff4WhONe23OcBSilH+xPcOpOS/g1z3p/oaG8VnSNKt+ZQT nBvVBU3LF4NG3Zo7nHPyTSot2Lw4CuzLd6QSk65z7bztBRAqeYHbhb2PSCcSPY55+OgUWv J/gCN837SZrviIZIriRLy1oAiSTP8r8d/IpGP43wQ0opaADjzBnsbFcesneR6Fn8dHcRM5 g5inO3EGrgi/WULoJIrFgN0mf5M4ZdKbaZOTCmWOBxO6Jo+VMbIiPVzeyLS2eQ== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=none; spf=pass (aspmx1.migadu.com: domain of help-guix-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=help-guix-bounces@gnu.org X-Migadu-Spam-Score: -4.02 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of help-guix-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=help-guix-bounces@gnu.org X-Migadu-Queue-Id: 330DD247B9 X-Spam-Score: -4.02 X-Migadu-Scanner: scn0.migadu.com X-TUID: 6DvLZWNB734N --bd/DeQ9X3Ap0l8Mf Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable I've become involved in the rust packaging effort so I'll jump in with some specifics. On Wed, Aug 04, 2021 at 03:50:00PM -0400, Julien Lepiller wrote: >=20 >=20 > Le 4 ao=C3=BBt 2021 15:14:16 GMT-04:00, phodina via a= =C3=A9crit=C2=A0: > >Hi, > > > >I came across a package in the Guix repository which fails to build. The= package in question here is rust-brotli-sys-0.3. So the obvious question c= omes up. How does this happen? > > > >I just ran =3Dguix build rust-brotli-sys=3D with the following output: > > > >... > >phase `reset-gzip-timestamps' succeeded after 0.0 seconds > >starting phase `compress-documentation' > >phase `compress-documentation' succeeded after 0.0 seconds > >guix build: error: derivation `/gnu/store/kf8kwl7yn8jqpwc9p97bw8vs5g5bva= s4-rust-brotli-sys-0.3.2.drv' may not be deterministic: output `/gnu/store/= yrrdr6rrc1s4b9lf7wz40c2gnv5kvxff-rust-brotli-sys-0.3.2' differs > >Also this is my guix channel: > > > >guix: > >repository URL: https://git.savannah.gnu.org/git/guix.git > >branch: master > >commit: b95fb85f593d377864fc1bfb3c118e54d03783be > > > >I tried to run the build with the flag --check and multiple rounds, but = got the same result. > >So could it be caused by not running =3D./pre-inst-env guix build --roun= ds=3D2 =3D when submitting the patch? >=20 > You need to use. /pre-inst-env to test your patches, otherwise you'll onl= y run your current guix. For an added package, it will be missing, for an u= pdated package it will not build the newer version, etc. >=20 > > > >Or when somebody updates some pkg that this pkg depends on and does not = run =3D./pre-inst-env guix --refresh --list-dependent=3D and then check the= dependent pkgs? >=20 > It can also happen when only the current day is recorded in the output: i= f you use =E2=80=94rounds=3D2, it is likely run on the same day, so the out= put will be the same (don't bother though, last time I found such an issue = it was in january because only the year was recorded ^^") >=20 > You should now use -K to keep the different version and run diffoscope to= analyze the differences. Then we should be able to fix the non determinism. >=20 > > > >Could also sombody please share what kind of CI pipeline is there implem= ented on applying the patch on the mainter's side? >=20 > The maentainer is responsible for testing the patch (at a minimum that it= builds), and sometimes we forget to check it's reproducible. The only auto= mated thing is to check the patch is signed. >=20 > > > >Thanks > >Petr > > > >PS: Yes, I do have more questions regading how Guix works lately. I trie= d to search the mailing list but haven't found answers to them >=20 I'm not always great with diffoscope, but it turns out this one isn't too bad to read. (ins)efraim@3900XT ~ [env]$ diffoscope /gnu/store/yrrdr6rrc1s4b9lf7wz40c2gn= v5kvxff-rust-brotli-sys-0.3.2 /gnu/store/yrrdr6rrc1s4b9lf7wz40c2gnv5kvxff-r= ust-brotli-sys-0.3.2-check --- /gnu/store/yrrdr6rrc1s4b9lf7wz40c2gnv5kvxff-rust-brotli-sys-0.3.2 +++ /gnu/store/yrrdr6rrc1s4b9lf7wz40c2gnv5kvxff-rust-brotli-sys-0.3.2-check =E2=94=82 --- /gnu/store/yrrdr6rrc1s4b9lf7wz40c2gnv5kvxff-rust-brotli-sys= -0.3.2/share =E2=94=9C=E2=94=80=E2=94=80 +++ /gnu/store/yrrdr6rrc1s4b9lf7wz40c2gnv5kvxff= -rust-brotli-sys-0.3.2-check/share =E2=94=82 =E2=94=82 --- /gnu/store/yrrdr6rrc1s4b9lf7wz40c2gnv5kvxff-rust-= brotli-sys-0.3.2/share/cargo =E2=94=82 =E2=94=9C=E2=94=80=E2=94=80 +++ /gnu/store/yrrdr6rrc1s4b9lf7wz40c= 2gnv5kvxff-rust-brotli-sys-0.3.2-check/share/cargo =E2=94=82 =E2=94=82 =E2=94=82 --- /gnu/store/yrrdr6rrc1s4b9lf7wz40c2gnv5k= vxff-rust-brotli-sys-0.3.2/share/cargo/registry =E2=94=82 =E2=94=82 =E2=94=9C=E2=94=80=E2=94=80 +++ /gnu/store/yrrdr6rrc1s4= b9lf7wz40c2gnv5kvxff-rust-brotli-sys-0.3.2-check/share/cargo/registry =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 --- /gnu/store/yrrdr6rrc1s4b9lf7w= z40c2gnv5kvxff-rust-brotli-sys-0.3.2/share/cargo/registry/brotli-sys-0.3.2.= crate =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=9C=E2=94=80=E2=94=80 +++ /gnu/store/yr= rdr6rrc1s4b9lf7wz40c2gnv5kvxff-rust-brotli-sys-0.3.2-check/share/cargo/regi= stry/brotli-sys-0.3.2.crate =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=9C=E2=94=80=E2=94=80 brotli-= sys-0.3.2.crate-content =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=9C=E2=94=80=E2=94= =80 file list =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 @@ -1,8 +1,8 @@ =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 --rw-r--r-- 0= 0 0 1123 2021-08-05 07:38:54.000000 brotli-sys-0.3.2/Car= go.toml =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 +-rw-r--r-- 0= 0 0 1123 2021-08-05 07:38:41.000000 brotli-sys-0.3.2/Car= go.toml =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 -rw-r--r-- 0= 999 30000 1122 1970-01-01 00:00:00.000000 brotli-sys-0.3.2/Car= go.toml.orig =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 -rw-r--r-- 0= 999 30000 2168 2017-06-30 05:59:10.000000 brotli-sys-0.3.2/bro= tli/common/constants.h =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 -rw-r--r-- 0= 999 30000 471278 2017-09-19 18:16:36.000000 brotli-sys-0.3.2/bro= tli/common/dictionary.c =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 -rw-r--r-- 0= 999 30000 1499 2017-09-19 18:16:36.000000 brotli-sys-0.3.2/bro= tli/common/dictionary.h =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 -rwxr-xr-x 0= 999 30000 595 2017-09-19 18:16:36.000000 brotli-sys-0.3.2/bro= tli/common/version.h =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 -rw-r--r-- 0= 999 30000 1259 2017-06-30 05:59:10.000000 brotli-sys-0.3.2/bro= tli/dec/bit_reader.c =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 -rw-r--r-- 0= 999 30000 12006 2017-06-30 05:59:10.000000 brotli-sys-0.3.2/bro= tli/dec/bit_reader.h =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=9C=E2=94=80=E2=94=80 stat {} =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 @@ -1,8 +1,8 @@ =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 - Size: 406369 Blo= cks: 800 IO Block: 4096 regular file =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 + Size: 406370 Blo= cks: 800 IO Block: 4096 regular file =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 Links: 1 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 Access: (0444/-r--r--r--= ) Uid: ( 0/ root) Gid: ( 0/ root) =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 =E2=94=82 Modify: 1970-01-01 00:00= :01.000000000 +0000 Here the difference is the timestamp on Cargo.toml. Cargo.toml gets regenerated during the build process (likely when we run `cargo build`) and then during the install phase we create a tarball of the source and put that and an unpacked version in %out/share/cargo as a step toward switching from #:cargo-inputs to just 'inputs'. Ideally we'd modify the install phase or add a new phase to reset some of the timestamps. Unfortunately, modifying the cargo-build-system would result in the rebuild of ~2500 packages, or ~150, depending on how you count. I can see about queueing up something for core-updates-frozen, which will make its way back to the master branch soonish=E2=84=A2. --=20 Efraim Flashner =D7=90=D7=A4=D7=A8=D7=99=D7=9D = =D7=A4=D7=9C=D7=A9=D7=A0=D7=A8 GPG key =3D A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351 Confidentiality cannot be guaranteed on emails sent or received unencrypted --bd/DeQ9X3Ap0l8Mf Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAABCgAdFiEEoov0DD5VE3JmLRT3Qarn3Mo9g1EFAmELmKYACgkQQarn3Mo9 g1Hn6hAAqOe92zKwZNmhwUK/SXwr3ytj23JET0eW6ztj1GjHQICI4vRZuVXctQna qiL4+uZnOJszOzfyGceI9T0zfDUzSMFzUhe4GWe4g7W5e9cs5CULO+Rr5uhisiQh j4OXw2OiIKWXUw1TwFEtCJlVI45+dNUEpH7MmwqI2QULNhtlCD8X4ObU2k6dbTCr vIiJmKh9oSIuI6lQV0294ysOe6ijE3Omc+3A4v8t/A1lv+e5+nBnRNwXZo/Er48i xbiYMfn+RtfEpHo876ABgb6/C+IqY2hM+hZzgGoS+TLQ8rGp1toZVK14EhBI+NHO CHaUIFN7pWFWD9rSGwcXoaqRyCR2JTjgHAgE+mus5OyxHAZtHuV+UpItCureCv4N haSxs2T3qjl6cemw1YkMsjaFwpb3zoHSR8U6DCccmDIDgNpzpr3qdNHdjq0uaVaE wVuOKXu1GLJHJz+yMDemSOKgiQBlqqDDWmoJM6MhlF6hJHprQwdQbJAG043e940Y iZf99if89m/oN8rrUoF3wvBeIuCfaCxTrl8S/VPrK/U4CPIQN1tLh6BF3QT+0+5K 3JLNiS2bPo6vSKv+yid+1tBla/2heZ1dNeARMoAoBKm5b+PbB6gADl16DW73eHXj ZZbt6k3XXmAmHQ+yGreifqSTaiB9D7lM9F76s2y5gGW/OjSoFTk= =SgRG -----END PGP SIGNATURE----- --bd/DeQ9X3Ap0l8Mf--