From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: ludo@gnu.org (Ludovic =?UTF-8?Q?Court=C3=A8s?=) Newsgroups: gmane.lisp.guile.bugs Subject: bug#20272: Support reproducible builds Date: Thu, 04 Feb 2016 10:35:48 +0100 Message-ID: <878u30g6mj.fsf@gnu.org> References: <87twws88hr.fsf@gnu.org> <87d1sdqjs2.fsf@netris.org> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Trace: ger.gmane.org 1454578643 22742 80.91.229.3 (4 Feb 2016 09:37:23 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Thu, 4 Feb 2016 09:37:23 +0000 (UTC) Cc: 20272@debbugs.gnu.org To: Mark H Weaver Original-X-From: bug-guile-bounces+guile-bugs=m.gmane.org@gnu.org Thu Feb 04 10:37:12 2016 Return-path: Envelope-to: guile-bugs@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1aRGLf-0001hS-3Y for guile-bugs@m.gmane.org; Thu, 04 Feb 2016 10:37:11 +0100 Original-Received: from localhost ([::1]:40531 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aRGLe-0002zB-Fe for guile-bugs@m.gmane.org; Thu, 04 Feb 2016 04:37:10 -0500 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:50171) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aRGLY-0002yL-3u for bug-guile@gnu.org; Thu, 04 Feb 2016 04:37:07 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1aRGLW-0005ds-RA for bug-guile@gnu.org; Thu, 04 Feb 2016 04:37:03 -0500 Original-Received: from debbugs.gnu.org ([208.118.235.43]:50338) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aRGLW-0005do-NA for bug-guile@gnu.org; Thu, 04 Feb 2016 04:37:02 -0500 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84) (envelope-from ) id 1aRGLW-0006z9-Gq for bug-guile@gnu.org; Thu, 04 Feb 2016 04:37:02 -0500 X-Loop: help-debbugs@gnu.org Resent-From: ludo@gnu.org (Ludovic =?UTF-8?Q?Court=C3=A8s?=) Original-Sender: "Debbugs-submit" Resent-CC: bug-guile@gnu.org Resent-Date: Thu, 04 Feb 2016 09:37:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 20272 X-GNU-PR-Package: guile X-GNU-PR-Keywords: Original-Received: via spool by 20272-submit@debbugs.gnu.org id=B20272.145457856126775 (code B ref 20272); Thu, 04 Feb 2016 09:37:02 +0000 Original-Received: (at 20272) by debbugs.gnu.org; 4 Feb 2016 09:36:01 +0000 Original-Received: from localhost ([127.0.0.1]:58927 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84) (envelope-from ) id 1aRGKX-0006xg-CL for submit@debbugs.gnu.org; Thu, 04 Feb 2016 04:36:01 -0500 Original-Received: from eggs.gnu.org ([208.118.235.92]:58696) by debbugs.gnu.org with esmtp (Exim 4.84) (envelope-from ) id 1aRGKW-0006xN-Ab for 20272@debbugs.gnu.org; Thu, 04 Feb 2016 04:36:00 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1aRGKN-00058l-Sz for 20272@debbugs.gnu.org; Thu, 04 Feb 2016 04:35:55 -0500 Original-Received: from fencepost.gnu.org ([2001:4830:134:3::e]:46234) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aRGKN-00058h-PH; Thu, 04 Feb 2016 04:35:51 -0500 Original-Received: from pluto.bordeaux.inria.fr ([193.50.110.57]:32966 helo=pluto) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.82) (envelope-from ) id 1aRGKM-0008SG-QF; Thu, 04 Feb 2016 04:35:51 -0500 X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 16 =?UTF-8?Q?Pluvi=C3=B4se?= an 224 de la =?UTF-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x3D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-unknown-linux-gnu In-Reply-To: <87d1sdqjs2.fsf@netris.org> (Mark H. Weaver's message of "Wed, 03 Feb 2016 21:41:33 -0500") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.5 (gnu/linux) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 208.118.235.43 X-BeenThere: bug-guile@gnu.org List-Id: "Bug reports for GUILE, GNU's Ubiquitous Extension Language" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-guile-bounces+guile-bugs=m.gmane.org@gnu.org Original-Sender: bug-guile-bounces+guile-bugs=m.gmane.org@gnu.org Xref: news.gmane.org gmane.lisp.guile.bugs:7950 Archived-At: Mark H Weaver skribis: > ludo@gnu.org (Ludovic Court=C3=A8s) writes: > >> Currently .go files embed randomly-generated symbols stemming from >> =E2=80=98syntax-session-id=E2=80=99, which prevents reproducible builds = (see >> .) >> >> One way to fix it would be to allow users to specify a random seed used >> when generating session ids, and to make that available as a >> command-line option to =E2=80=98guild compile=E2=80=99. (GCC does somet= hing similar >> with its =E2=80=98-frandom-seed=E2=80=99 option.) > > We could add this, but it is not analogous to the -frandom-seed option > where it is okay to give it the same value everywhere. Users would need > to ensure that distinct session-ids are used for every invocation of > Guile. With GCC the common idiom is to use =E2=80=98-frandom-seed=3D$source_file= =E2=80=99. However, it would be best if =E2=80=98guild compile=E2=80=99 would choose t= he seed deterministically by default somehow, because we cannot expect all users to add the new flag and use properly. What about having =E2=80=98guild compile=E2=80=99 use the canonical file na= me of the source being compiled (or a hash thereof) as the seed? > More precisely, users of this feature would need to observe the > following restriction, or else unspecified behavior may result: > > If A.go is generated by a Guile session with session-id A, and > B.go is generated by a Guile session with session-id B, and > they are both loaded into a Guile session with session-id C, then > A, B, and C must all be distinct session-ids. Right. I wonder if we could detect collisions. Ideally each .go could record its session ID, but that=E2=80=99s probably not feasible in 2.0. > One more thing: even with a deterministic session-id, the multi-threaded > compiling of *.go files recently added to Guix will lead to > non-deterministic outputs. I'm not sure how to make this work > deterministically with concurrency. Even if we switched to a fixed > number of concurrent processes with one thread each, when process A > attempts to load a .go file that is produced by process B, it is not > deterministic whether it will be there. If it is there, the macros in > that .go file will contain B's session-id, and if not, they will contain > A's session-id. Hmm, OK. Well, let=E2=80=99s keep this use case aside for now. >> Probably, =E2=80=98syntax-session-id=E2=80=99 would have to be a SRFI-39= parameter > > This might adversely affect the efficiency of our macro expander on > platforms with slow thread local variables, and I'm not sure what it > would buy us. If the idea is that it would allow us to build things in > multiple threads, I think that won't work anyway, for the reasons given > above. I was just thinking that it would be a convenient interface for (scripts compile) to specify the session ID to use. I think the first thing to do is to change =E2=80=98fresh_syntax_session_id= =E2=80=99 so that they can use a user-specified seed, when available, instead of =E2=80=98scm_i_random_bytes_from_platform=E2=80=99. WDYT? Thanks for working on it! :-) Ludo=E2=80=99.