From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: ludo@gnu.org (Ludovic =?UTF-8?Q?Court=C3=A8s?=) Newsgroups: gmane.lisp.guile.bugs Subject: bug#20272: Support reproducible builds Date: Thu, 15 Dec 2016 00:42:22 +0100 Message-ID: <87mvfygjs1.fsf@gnu.org> References: <87bmxxnjd7.fsf@gnu.org> <87shqtwxcb.fsf@gnu.org> <87twa6iikd.fsf@gnu.org> <87vaumgk80.fsf@gnu.org> NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Trace: blaine.gmane.org 1481758995 3988 195.159.176.226 (14 Dec 2016 23:43:15 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Wed, 14 Dec 2016 23:43:15 +0000 (UTC) User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.1 (gnu/linux) Cc: 20272@debbugs.gnu.org To: Jan Nieuwenhuizen Original-X-From: bug-guile-bounces+guile-bugs=m.gmane.org@gnu.org Thu Dec 15 00:43:11 2016 Return-path: Envelope-to: guile-bugs@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cHJCW-00089h-Kj for guile-bugs@m.gmane.org; Thu, 15 Dec 2016 00:43:08 +0100 Original-Received: from localhost ([::1]:50874 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cHJCa-00014m-NE for guile-bugs@m.gmane.org; Wed, 14 Dec 2016 18:43:12 -0500 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:59252) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cHJCT-0000xb-Jx for bug-guile@gnu.org; Wed, 14 Dec 2016 18:43:07 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cHJCQ-0003os-E6 for bug-guile@gnu.org; Wed, 14 Dec 2016 18:43:05 -0500 Original-Received: from debbugs.gnu.org ([208.118.235.43]:55020) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1cHJCQ-0003om-AN for bug-guile@gnu.org; Wed, 14 Dec 2016 18:43:02 -0500 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1cHJCQ-0005ON-4L for bug-guile@gnu.org; Wed, 14 Dec 2016 18:43:02 -0500 X-Loop: help-debbugs@gnu.org Resent-From: ludo@gnu.org (Ludovic =?UTF-8?Q?Court=C3=A8s?=) Original-Sender: "Debbugs-submit" Resent-CC: bug-guile@gnu.org Resent-Date: Wed, 14 Dec 2016 23:43:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 20272 X-GNU-PR-Package: guile X-GNU-PR-Keywords: patch Original-Received: via spool by 20272-submit@debbugs.gnu.org id=B20272.148175897420713 (code B ref 20272); Wed, 14 Dec 2016 23:43:02 +0000 Original-Received: (at 20272) by debbugs.gnu.org; 14 Dec 2016 23:42:54 +0000 Original-Received: from localhost ([127.0.0.1]:42186 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cHJCG-0005Ny-5y for submit@debbugs.gnu.org; Wed, 14 Dec 2016 18:42:52 -0500 Original-Received: from eggs.gnu.org ([208.118.235.92]:39994) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cHJCE-0005Nl-B5 for 20272@debbugs.gnu.org; Wed, 14 Dec 2016 18:42:50 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cHJC5-0003g1-Rb for 20272@debbugs.gnu.org; Wed, 14 Dec 2016 18:42:45 -0500 Original-Received: from fencepost.gnu.org ([2001:4830:134:3::e]:40710) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cHJBq-0003Zx-R2; Wed, 14 Dec 2016 18:42:26 -0500 Original-Received: from [37.120.80.33] (port=43650 helo=pluto) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1cHJBp-0004Zw-UX; Wed, 14 Dec 2016 18:42:26 -0500 In-Reply-To: <87vaumgk80.fsf@gnu.org> ("Ludovic \=\?utf-8\?Q\?Court\=C3\=A8s\=22'\?\= \=\?utf-8\?Q\?s\?\= message of "Thu, 15 Dec 2016 00:32:47 +0100") X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 208.118.235.43 X-BeenThere: bug-guile@gnu.org List-Id: "Bug reports for GUILE, GNU's Ubiquitous Extension Language" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-guile-bounces+guile-bugs=m.gmane.org@gnu.org Original-Sender: "bug-guile" Xref: news.gmane.org gmane.lisp.guile.bugs:8477 Archived-At: --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable ludo@gnu.org (Ludovic Court=C3=A8s) skribis: > ludo@gnu.org (Ludovic Court=C3=A8s) skribis: > >> Jan Nieuwenhuizen skribis: >> >>> I have been building quite some versions of Guile the past week. There >>> re a couple of .go files that keep giving me problems wrt >>> nondeterminism: >>> >>> web/server/http.go >>> >>> (define *timer* (gettimeofday)) seemed suspect, but seems harmless. >>> >>> rnrs/arithmetic/fixnums.go >>> >>> make-implementation-restriction-violation seemed suspect, but commenting >>> it out does not seem to help. >>> >>> After commenting out all of http.scm and fixnums.scm, yesterday I had 3 >>> Guix builds on two machines (GuixSD and one Ubuntu+Guix) that were >>> bit-identical, but one build showed differences in >>> >>> system/repl/commands.go >>> >>> Apparently most of the time this file builds deterministicly, but some >>> times it does not!? Ideas? >> >> Did you look at these differences with diffoscope? That would be >> useful. > > I built it with =E2=80=98guix build --rounds=3D2 -K=E2=80=99 and got 17 d= iffering .go > files. I looked at srfi/srfi-9/gnu.go with ediff + hexl-mode in Emacs > and found a dozen of =E2=80=9Ct-XYZ=E2=80=9D and =E2=80=9Ctmp-XYZ=E2=80= =9D identifiers with a different > value for XYZ (an integer). To demonstrate non-reproducibility (with the attached patch, which is a rebased version of Mark=E2=80=99s), just build the same module twice: once = with its dependency evaluated, and then with its dependency compiled. The results differ: --8<---------------cut here---------------start------------->8--- ludo@pluto ~/src/guile$ rm -f module/srfi/srfi-9/gnu.go module/srfi/srfi-9.= go ludo@pluto ~/src/guile$ make -C module srfi/srfi-9/gnu.go make: Entering directory '/home/ludo/src/guile/module' GUILEC srfi/srfi-9/gnu.go wrote `srfi/srfi-9/gnu.go' make: Leaving directory '/home/ludo/src/guile/module' ludo@pluto ~/src/guile$ mv module/srfi/srfi-9/gnu.go{,.v1} ludo@pluto ~/src/guile$ make -C module srfi/srfi-9.go make: Entering directory '/home/ludo/src/guile/module' GUILEC srfi/srfi-9.go wrote `srfi/srfi-9.go' make: Leaving directory '/home/ludo/src/guile/module' ludo@pluto ~/src/guile$ make -C module srfi/srfi-9/gnu.go make: Entering directory '/home/ludo/src/guile/module' GUILEC srfi/srfi-9/gnu.go wrote `srfi/srfi-9/gnu.go' make: Leaving directory '/home/ludo/src/guile/module' ludo@pluto ~/src/guile$ mv module/srfi/srfi-9/gnu.go{,.v2} ludo@pluto ~/src/guile$ diff -u module/srfi/srfi-9/gnu.go.{v1,v2} Ciferecaj dosieroj module/srfi/srfi-9/gnu.go.v1 kaj module/srfi/srfi-9/gnu.= go.v2 estas malsamaj --8<---------------cut here---------------end--------------->8--- In gnu.go.v2 the integers appended to generated symbols are lower because fewer symbols had to be generated. Ludo=E2=80=99. --=-=-= Content-Type: text/x-patch; charset=utf-8 Content-Disposition: inline; filename=0001-psyntax-Remove-uses-of-syntax-session-id.patch Content-Transfer-Encoding: quoted-printable Content-Description: the patch >From 4301d4b53d7243012bbf6ee4d7558910dd712330 Mon Sep 17 00:00:00 2001 From: Mark H Weaver Date: Fri, 12 Feb 2016 11:19:38 -0500 Subject: [PATCH] psyntax: Remove uses of syntax-session-id. MIME-Version: 1.0 Content-Type: text/plain; charset=3DUTF-8 Content-Transfer-Encoding: 8bit This is an attempted fix for , but I'm not yet confident in its correctness. * module/ice-9/boot-9.scm (module-generate-unique-id!): New procedure. (module): Add 'next-unique-id' field. (the-root-module): Inherit 'next-unique-id' value from early stub. (make-module, make-autoload-interface): Adjust calls to module-constructor. * module/ice-9/psyntax.scm (gen-label, new-mark): Generate unique identifiers from the module name and the per-module unique-id. * module/ice-9/psyntax-pp.scm: Regenerate. Signed-off-by: Ludovic Court=C3=A8s --- module/ice-9/boot-9.scm | 25 +++++++++++++++++++++---- module/ice-9/psyntax-pp.scm | 15 +++++++++++++-- module/ice-9/psyntax.scm | 11 +++++++++-- 3 files changed, 43 insertions(+), 8 deletions(-) diff --git a/module/ice-9/boot-9.scm b/module/ice-9/boot-9.scm index 426d831..8ef7e5f 100644 --- a/module/ice-9/boot-9.scm +++ b/module/ice-9/boot-9.scm @@ -1,6 +1,6 @@ ;;; -*- mode: scheme; coding: utf-8; -*- =20 -;;;; Copyright (C) 1995-2014 Free Software Foundation, Inc. +;;;; Copyright (C) 1995-2014, 2016 Free Software Foundation, Inc. ;;;; ;;;; This library is free software; you can redistribute it and/or ;;;; modify it under the terms of the GNU Lesser General Public @@ -384,6 +384,12 @@ If there is no handler at all, Guile prints an error a= nd then exits." (define (module-ref module sym) (let ((v (module-variable module sym))) (if v (variable-ref v) (error "badness!" (pk module) (pk sym))))) +(define module-generate-unique-id! + (let ((next-id 0)) + (lambda (m) + (let ((i next-id)) + (set! next-id (+ i 1)) + i)))) (define (resolve-module . args) #f) =20 @@ -2021,7 +2027,8 @@ VALUE." submodules submodule-binder public-interface - filename))) + filename + next-unique-id))) =20 =20 ;; make-module &opt size uses binder @@ -2049,7 +2056,7 @@ VALUE." (make-hash-table %default-import-size) '() (make-weak-key-hash-table 31) #f - (make-hash-table 7) #f #f #f)) + (make-hash-table 7) #f #f #f 0)) =20 =20 @@ -2656,6 +2663,11 @@ VALUE." (let ((m (make-module 0))) (set-module-obarray! m (%get-pre-modules-obarray)) (set-module-name! m '(guile)) + + ;; Inherit next-unique-id from preliminary stub of + ;; %module-get-next-unique-id! defined above. + (set-module-next-unique-id! m (module-generate-unique-id! #f)) + m)) =20 ;; The root interface is a module that uses the same obarray as the @@ -2684,6 +2696,11 @@ VALUE." the-root-module (error "unexpected module to resolve during module boot" name))) =20 +(define (module-generate-unique-id! m) + (let ((i (module-next-unique-id m))) + (set-module-next-unique-id! m (+ i 1)) + i)) + ;; Cheat. These bindings are needed by modules.c, but we don't want ;; to move their real definition here because that would be unnatural. ;; @@ -3013,7 +3030,7 @@ VALUE." #:warning "Failed to autoload ~a in ~a:\n" sym name)))) (module-constructor (make-hash-table 0) '() b #f #f name 'autoload #f (make-hash-table 0) '() (make-weak-value-hash-tabl= e 31) #f - (make-hash-table 0) #f #f #f))) + (make-hash-table 0) #f #f #f 0))) =20 (define (module-autoload! module . args) "Have @var{module} automatically load the module named @var{name} when o= ne diff --git a/module/ice-9/psyntax-pp.scm b/module/ice-9/psyntax-pp.scm index 1ec5107..c81b69e 100644 --- a/module/ice-9/psyntax-pp.scm +++ b/module/ice-9/psyntax-pp.scm @@ -312,7 +312,12 @@ (values x (car w))))) (gen-label (lambda () - (string-append "l-" (session-id) (symbol->string (gensym "-"))))) + (let ((mod (current-module))) + (simple-format + #f + "l-~s~s" + (module-generate-unique-id! mod) + (module-name mod))))) (gen-labels (lambda (ls) (if (null? ls) '() (cons (gen-label) (gen-labels (cdr ls)))))) @@ -960,7 +965,13 @@ ((transformer-environment (lambda (k) (k e r w s rib mod)))) (rebuild-macro-output (p (source-wrap e (anti-mark w) s mod)) - (gensym (string-append "m-" (session-id) "-"))))))) + (let ((mod (current-module))) + (string->symbol + (simple-format + #f + "m-~s~s" + (module-generate-unique-id! mod) + (module-name mod))))))))) (expand-body (lambda (body outer-form r w mod) (let* ((r (cons '("placeholder" placeholder) r)) diff --git a/module/ice-9/psyntax.scm b/module/ice-9/psyntax.scm index 79b353d..7d12469 100644 --- a/module/ice-9/psyntax.scm +++ b/module/ice-9/psyntax.scm @@ -642,7 +642,10 @@ ;; labels must be comparable with "eq?", have read-write invariance, ;; and distinct from symbols. (define (gen-label) - (string-append "l-" (session-id) (symbol->string (gensym "-")))) + (let ((mod (current-module))) + (simple-format #f "l-~s~s" + (module-generate-unique-id! mod) + (module-name mod)))) =20 (define gen-labels (lambda (ls) @@ -671,7 +674,11 @@ (cons 'shift (wrap-subst w))))) =20 (define-syntax-rule (new-mark) - (gensym (string-append "m-" (session-id) "-"))) + (let ((mod (current-module))) + (string->symbol + (simple-format #f "m-~s~s" + (module-generate-unique-id! mod) + (module-name mod))))) =20 ;; make-empty-ribcage and extend-ribcage maintain list-based ribcages = for ;; internal definitions, in which the ribcages are built incrementally --=20 2.10.2 --=-=-=--