From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id WDB1IBdy117ZbgAA0tVLHw (envelope-from ) for ; Wed, 03 Jun 2020 09:49:11 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id 6ERcHBdy1172QQAAB5/wlQ (envelope-from ) for ; Wed, 03 Jun 2020 09:49:11 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 0061C940669 for ; Wed, 3 Jun 2020 09:49:10 +0000 (UTC) Received: from localhost ([::1]:50628 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jgQ1F-0008DA-E7 for larch@yhetil.org; Wed, 03 Jun 2020 05:49:09 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51202) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jgQ18-0008Cd-RV for bug-guix@gnu.org; Wed, 03 Jun 2020 05:49:02 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:58691) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jgQ18-0002EX-IV for bug-guix@gnu.org; Wed, 03 Jun 2020 05:49:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1jgQ18-00029J-Hi for bug-guix@gnu.org; Wed, 03 Jun 2020 05:49:02 -0400 X-Loop: help-debbugs@gnu.org Subject: bug#41669: Cross-compiled powerpc64-linux bootstrap-tarballs not reproducible Resent-From: Chris Marusich Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Wed, 03 Jun 2020 09:49:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 41669 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: 41669@debbugs.gnu.org Received: via spool by 41669-submit@debbugs.gnu.org id=B41669.15911777148225 (code B ref 41669); Wed, 03 Jun 2020 09:49:02 +0000 Received: (at 41669) by debbugs.gnu.org; 3 Jun 2020 09:48:34 +0000 Received: from localhost ([127.0.0.1]:42004 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jgQ0g-00028b-HK for submit@debbugs.gnu.org; Wed, 03 Jun 2020 05:48:34 -0400 Received: from mail-pg1-f182.google.com ([209.85.215.182]:37580) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jgQ0e-00028L-HG for 41669@debbugs.gnu.org; Wed, 03 Jun 2020 05:48:32 -0400 Received: by mail-pg1-f182.google.com with SMTP id d10so1412517pgn.4 for <41669@debbugs.gnu.org>; Wed, 03 Jun 2020 02:48:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version; bh=Q18QBCNrvylD3ooY6bLE5vwc+rjT7BPMYahw7tGb2u0=; b=iqlXdEM8yhWXwpcYsC0IwU5p5GzcUbwaV0S+7px+8FP5FMwwaXHF9cp/3ZZ5LnZ532 iL/1QXNuW/zcA3LUGoi+AdTJPCVyxwJ0EQcidzJqO2pZSCNsBe5hr3A37z3tY0q6JxF8 ORVffEsRUS47bHiyx0fgh1XcT6swO23zzoRI61G1iIZwo3nWLuhTqk9xlXYHd101jRH9 5GGGEsK4ZqQLHFS1UFwgJP/a78/R3UjXlPygTlc0+/s2IFJ7ZNiVUrZ7cXWQtujik8FK beiR9Qxh9rkNGT2nFYzJivc9S5tg4K62lgOEPnjGVH/Wi20Aow78Y0Zd17W2100ug/tS R1RA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version; bh=Q18QBCNrvylD3ooY6bLE5vwc+rjT7BPMYahw7tGb2u0=; b=HuWIKiUrOgJtFnoMUF8MBDlhnfya9hFviMqRdpbhtXQAgOJdH89hSgrCLYvsMEJZj5 kPfUiIweoYbZZVedR4IGdmSarzlJR1ext4pTpKPKzWlMrBLK04C0nPv5H2DzlcwVW1yT 2xCecYRk7oB4lSlcpelr/bXs6sOiurzUG7fK4AhmosTtvXvfsTrn1cYB6tDcFtQuKxRM MnayPYk2w5uf38EAABq9tARTZW6K/Ynoz5+wSjJaj6GDt22zR6KZSokH5GLLqgXpC7XY 9NBjcXQ3zTAZGB899dB+lLksEy5NAN3GUc1UW+N6hmFJu6T4atfrbJuW2jFx+fsGClEW aegw== X-Gm-Message-State: AOAM530nfhOj97I7tkVFJuMJeFoeS9UjHKaD1eRiB2X6FxqZGI7R4aHl KtbmUgTcP5fIG1m3vjN9+0Q= X-Google-Smtp-Source: ABdhPJylWJr0BXh5vycpb64yv141d9EBVZ93/em3KOeO+2vNgRl2PdvXqnrs7GuAq/S+a40hLF6Y8g== X-Received: by 2002:a17:90a:1642:: with SMTP id x2mr4433137pje.149.1591177706367; Wed, 03 Jun 2020 02:48:26 -0700 (PDT) Received: from garuda-lan (c-73-97-103-127.hsd1.wa.comcast.net. [73.97.103.127]) by smtp.gmail.com with ESMTPSA id n2sm1573113pfd.125.2020.06.03.02.48.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 03 Jun 2020 02:48:25 -0700 (PDT) From: Chris Marusich References: <874krtnvk8.fsf@gmail.com> Date: Wed, 03 Jun 2020 02:48:21 -0700 In-Reply-To: <874krtnvk8.fsf@gmail.com> (Chris Marusich's message of "Tue, 02 Jun 2020 11:59:03 -0700") Message-ID: <87y2p4mqe2.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-Spam-Score: 0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-Spam-Score: -1.0 (-) X-BeenThere: bug-guix@gnu.org List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Vincent Legoll , =?UTF-8?Q?L=C3=A9o?= Le Bouter , Maxim Cournoyer Errors-To: bug-guix-bounces+larch=yhetil.org@gnu.org Sender: "bug-Guix" X-Scanner: scn0 Authentication-Results: aspmx1.migadu.com; dkim=fail (rsa verify failed) header.d=gmail.com header.s=20161025 header.b=iqlXdEM8; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none); spf=pass (aspmx1.migadu.com: domain of bug-guix-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=bug-guix-bounces@gnu.org X-Spam-Score: 1.99 X-TUID: /2aJJ8UhaDna --=-=-= Content-Type: text/plain Content-Transfer-Encoding: quoted-printable Hi all, Chris Marusich writes: > The derivation that produced the differing output was: > > /gnu/store/pygln3lr6qbxcps3kmn3w4bc0d0nlpd3-gcc-stripped-tarball-5.5.0.drv > > On my x86_64-linux system, twice I tried running "guix build --check" on > this derivation, but each time it surprisingly reported no difference. This derivation corresponds to %gcc-bootstrap-tarball from (gnu packages make-bootstrap), which just creates a tarball of the output of %gcc-stripped. Therefore, it's not too surprising that it's reproducible. Similarly, %gcc-stripped just strips some store references from the output of %gcc-static. The %gcc-static package is more interesting: it's where we actually build the statically linked bootstrap GCC (which is then stripped and packed into a tarball as mentioned above), so I thought that it might be where the non-determinism is coming from. However, I haven't yet pinpointed the problem. If you examine the derivations and their inputs, you'll find that they depend upon each other in the following order: guix build --target=3Dpowerpc64-linux-gnu -d -e '(@ (gnu packages make-boot= strap) %gcc-bootstrap-tarball)' /gnu/store/pygln3lr6qbxcps3kmn3w4bc0d0nlpd3-gcc-stripped-tarball-5.5.0.drv guix build --target=3Dpowerpc64-linux-gnu -d -e '(@@ (gnu packages make-boo= tstrap) %gcc-stripped)' /gnu/store/kcv3ja1rfr93hw6ly51878zjhdwpgv7z-gcc-stripped-5.5.0.drv guix build --target=3Dpowerpc64-linux-gnu -d -e '(@@ (gnu packages make-boo= tstrap) %gcc-static)' /gnu/store/m9hfwppla8lph0vxa15lfkp81s2bbjjs-gcc-static-5.5.0.drv In other words, gcc-static-5.5.0.drv is an input of gcc-stripped-5.5.0.drv, which is an input of gcc-stripped-tarball-5.5.0.drv. Above, I've included example guix commands you can use to obtain each derivation. Using "guix build =2D-check", I confirmed that all three of these derivations build reproducibly on my machine. I hoped to find more information by invoking "guix build --check" on every input of gcc-static-5.5.0.drv. When I tried that, what I found was that all of its inputs build reproducibly except the following two: /gnu/store/x32cnfkd50fnxs10xp1jdn24h7ai2gxr-guile-3.0.2.drv /gnu/store/g9fpkg2qa27mka1znqsvx8vxqyabsj2y-gcc-7.5.0.drv I haven't investigated guile-3.0.2.drv. However, gcc-7.5.0.drv felt more suspicious to me, and it is actually the derivation that builds gcc-final from (gnu packages commencement), which you can see via: guix build -d -e '(@@ (gnu packages commencement) gcc-final)' Using "guix gc", I deleted the outputs of gcc-stripped-tarball-5.5.0.drv, gcc-stripped-5.5.0.drv, gcc-static-5.5.0.drv, and gcc-7.5.0.drv. I then tried building these four derivations again (without substitutes, the same as before). Before doing this, I stored the SHA-512 hashes of their output files, and after the build succeeded, I compared the hashes of the new files with the previous values. I found that these derivations' newly rebuilt outputs were identical to their original values, except for gcc-7.5.0.drv, which produced some different files. Most significantly, this means that gcc-stripped-tarball-5.5.0.drv produced the exact same tarball for me as it did the first time I built it, even though some of its inputs are not themselves reproducible. At present, it seems possible that within the context of a single machine, gcc-stripped-tarball-5.5.0.drv builds reproducibly, but on a different machine, it may (reproducibly) build a different output. I'm a bit paranoid about making mistakes, so I'll perform another full GC and then try yet again to build gcc-stripped-tarball-5.5.0.drv in order to verify whether it truly produces the same output when all (or nearly all) of its inputs are rebuilt from scratch. Some people have also shared their differing copies of the binaries on the email list. It could be productive to compare the contents with diffoscope, although I suspect the diff might be too large to be useful. =2D-=20 Chris --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEy/WXVcvn5+/vGD+x3UCaFdgiRp0FAl7XceUACgkQ3UCaFdgi Rp0EgRAA0OmlaDC8TFlxSXsGjlbPVhV5kO7mplWnYNAbROMWrFWoRU4okIISdCeV pklnsneNtg9UN+OagjGMNsgw6w/iaiftMduLS17AqWeUoaCi7HUTGVgwg6AZD2z3 nWFFgjPdYy8NO0uSA4wxoqGXHIXZsmy6m8HlNkQ5Wxv4ILrMLW1aAloAwLuicnD7 xszXKVfC38UyVwdjVyTbdVqUcl8QPY8jYdQtk+RtK6Yb1aWba6xyFW6FZhgNl4PW DcEIBD6lKewXYT8saDJkkD1dBhkGiHjMPWkDPKiDuPR7dfDhrm4QHxRZHifv5Fu3 wGoya2yxyCtaB2O/xH8YLiQiLbWjGE+pZdbGN3hIwf8ie2lilK97895hTIhMRasi 1q6XMpRW27X8cZG7nv/spMyo3Zu/PudOghzOk/h16W0Ey24hLv0IO4xAtUxCprnB 4gtU/hKxbKYS8mN4K9AmkgnCDlibfAuEqRAwAcDpA3oxobjZLTih/BBlulUn3pxN lf4/xIn9DCaZpi9pckjYnucByP0wv/JjA4aJGBpPM9rfau805E8XeHtBBX1djsL0 ekPaT9BkEYVxILT9eHkrIp3JO5P1H0dRqhtT6XUFShKwe9FC3BgxaJeW8v7DsLnv bsKJ/0nXMjrcjJBqR5FhuAy/ZZXi+ILivSHQ5QlG/D1uBg8kEPM= =kMiK -----END PGP SIGNATURE----- --=-=-=--