From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jan Nieuwenhuizen Subject: bug#36747: Official MesCC bootstrap binaries differ from my locally built ones Date: Sun, 21 Jul 2019 15:34:10 +0200 Message-ID: <87ef2j1pgt.fsf@gnu.org> References: <875znwcoo9.fsf@netris.org> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Return-path: Received: from eggs.gnu.org ([2001:470:142:3::10]:47545) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hpBzU-0007n4-Re for bug-guix@gnu.org; Sun, 21 Jul 2019 09:35:06 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hpBzS-0002uD-T0 for bug-guix@gnu.org; Sun, 21 Jul 2019 09:35:04 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:49399) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1hpBzS-0002u6-Mv for bug-guix@gnu.org; Sun, 21 Jul 2019 09:35:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1hpBzS-0003ev-I8 for bug-guix@gnu.org; Sun, 21 Jul 2019 09:35:02 -0400 Sender: "Debbugs-submit" Resent-Message-ID: In-Reply-To: <875znwcoo9.fsf@netris.org> (Mark H. Weaver's message of "Sat, 20 Jul 2019 18:43:07 -0400") List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-guix-bounces+gcggb-bug-guix=m.gmane.org@gnu.org Sender: "bug-Guix" To: Mark H Weaver Cc: 36747@debbugs.gnu.org --=-=-= Content-Type: text/plain Mark H Weaver writes: > I'm working to independently verify the new bootstrap binaries. Toward > that end, I locally built the new bootstrap tarballs by running the > following command from a git checkout at commit > ef809e3ac036eccc5f9c9edd8fb661d14ae15f2f. > > ./pre-inst-env guix build bootstrap-tarballs --system=i686-linux > > This produces a mescc-tools-static-0.5.2-0.bb062b0-i686-linux.tar.xz > tarball that differs from the official one. See below for the precise > differences, but what they boil down to is that the official tarball > contains several references to: > > /gnu/store/xjr9rv1p4vgg9sclwzsai17gqi1c1d9j-glibc-2.28 > > Whereas in my locally built tarball, these references are instead to: > > /gnu/store/ggj9164ahhn1kmdcyl3sm27qsxk46h01-glibc-2.28 Aww, that's not good :( > First of all, I hope we can agree that our bootstrap binaries shouldn't > include any references to the store. I think that is our policy and it is probably safest not to do so; at least in code. A piece of documentation can include a string /gnu/store/.... as long at it is static and not really a reference. Store references in bootstrap binaries or scripts cannot be used or depended on: if our bootstrap build is clean those references will not exist when the bootstrap binaries run. In any case, if store references are present in bootstrap binaries, then they should be reproducible. Removing store references is one way to make them reproducible but I'm not sure if that's the best way? Making a reference invalid helps making sure that it isn't used. But if it cannot be used anyway, removing it could hide the fact that it may differs accros builds, as we are finding out right now. Thoughts? > However, it also raises the curious question of how I ended up with > references to a different glibc than the one referenced by the official > tarball. Any idea what happened here? Yes, that's something that I would like to know... Anyway, I have built a new set of bootstrap binaries for mescc-tools and mes using attched patches that remove store references (also on my https://gitlab.com/janneke/guix core-updates). I have just verified that tcc-boot0 builds by using this local modification --8<---------------cut here---------------start------------->8--- 15:05:56 janneke@dundal:~/src/guix/core-updates [env] $ git diff diff --git a/gnu/packages/bootstrap.scm b/gnu/packages/bootstrap.scm index 73fc7d3e5e..bca90bec94 100644 --- a/gnu/packages/bootstrap.scm +++ b/gnu/packages/bootstrap.scm @@ -288,7 +288,8 @@ or false to signal an error." (define %bootstrap-base-urls ;; This is where the initial binaries come from. - '("https://alpha.gnu.org/gnu/guix/bootstrap" + '("http://lilypond.org/janneke/guix" + "https://alpha.gnu.org/gnu/guix/bootstrap" "http://alpha.gnu.org/gnu/guix/bootstrap" "ftp://alpha.gnu.org/gnu/guix/bootstrap" "http://www.fdn.fr/~lcourtes/software/guix/packages" --8<---------------cut here---------------end--------------->8--- and running ./pre-inst-env guix build --system=i686-linux -e '(@@ (gnu packages commencement) tcc-boot0)' Thanks again for looking into this! Greetings, janneke --=-=-= Content-Type: text/x-patch Content-Disposition: inline; filename=0001-bootstrap-mescc-tools-static-stripped-Remove-store-r.patch >From 11c12e30bbdb1a10fd56dc9d5c14548e49b63c12 Mon Sep 17 00:00:00 2001 From: Jan Nieuwenhuizen Date: Sun, 21 Jul 2019 11:40:54 +0200 Subject: [PATCH 1/4] bootstrap: mescc-tools-static-stripped: Remove store references. * gnu/packages/make-bootstrap.scm (%mescc-tools-static-stripped): Rename from %mescc-tools-static; update users. Remove store references. --- gnu/packages/make-bootstrap.scm | 27 +++++++++++++++++++-------- 1 file changed, 19 insertions(+), 8 deletions(-) diff --git a/gnu/packages/make-bootstrap.scm b/gnu/packages/make-bootstrap.scm index 2163b646f6..8cc4eef430 100644 --- a/gnu/packages/make-bootstrap.scm +++ b/gnu/packages/make-bootstrap.scm @@ -585,16 +585,27 @@ for `sh' in $PATH, and without nscd, and with static NSS modules." #t)))) (inputs `(("gcc" ,%gcc-static))))) -(define %mescc-tools-static - ;; A statically linked MesCC Tools for bootstrap. +(define %mescc-tools-static-stripped + ;; A statically linked Mescc Tools with store references removed, for + ;; bootstrap. (package (inherit mescc-tools) - (name "mescc-tools-static") + (name "mescc-tools-static-stripped") (arguments - `(#:system "i686-linux" - ,@(substitute-keyword-arguments (package-arguments mescc-tools) - ((#:make-flags flags) - `(cons "CC=gcc -static" ,flags))))))) + `(#:make-flags (list (string-append "PREFIX=" (assoc-ref %outputs "out")) + "CC=gcc -static") + #:test-target "test" + #:phases (modify-phases %standard-phases + (delete 'configure) + (add-after 'install 'strip-store-references + (lambda _ + (let* ((out (assoc-ref %outputs "out")) + (bin (string-append out "/bin"))) + (for-each (lambda (file) + (let ((target (string-append bin "/" file))) + (format #t "strippingg `~a'...~%" target) + (remove-store-references target))) + '( "M1" "blood-elf" "hex2")))))))))) (define-public %mes-minimal-stripped ;; A minimal Mes without documentation dependencies, for bootstrap. @@ -791,7 +802,7 @@ for `sh' in $PATH, and without nscd, and with static NSS modules." (define %mescc-tools-bootstrap-tarball ;; A tarball with MesCC binary seed. - (tarball-package %mescc-tools-static)) + (tarball-package %mescc-tools-static-stripped)) (define %mes-bootstrap-tarball ;; A tarball with Mes ASCII Seed and binary Mes C Library. -- 2.21.0 --=-=-= Content-Type: text/x-patch Content-Disposition: inline; filename=0002-bootstrap-mes-minimal-stripped-Remove-store-referenc.patch >From 2e13b6fff94027158b3de5d3a1469dc9f89b0c39 Mon Sep 17 00:00:00 2001 From: Jan Nieuwenhuizen Date: Sun, 21 Jul 2019 14:27:07 +0200 Subject: [PATCH 2/4] bootstrap: mes-minimal-stripped: Remove store references. * gnu/packages/make-bootstrap.scm (%mes-minimal-stripped): Remove store references. --- gnu/packages/make-bootstrap.scm | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-) diff --git a/gnu/packages/make-bootstrap.scm b/gnu/packages/make-bootstrap.scm index 8cc4eef430..af89ca8c3c 100644 --- a/gnu/packages/make-bootstrap.scm +++ b/gnu/packages/make-bootstrap.scm @@ -621,6 +621,7 @@ for `sh' in $PATH, and without nscd, and with static NSS modules." #:configure-flags '("--mes") #:phases (modify-phases %standard-phases + (delete 'patch-shebangs) (add-after 'install 'strip-install (lambda _ (let* ((out (assoc-ref %outputs "out")) @@ -628,10 +629,16 @@ for `sh' in $PATH, and without nscd, and with static NSS modules." (delete-file-recursively (string-append out "/lib/guile")) (delete-file-recursively (string-append share "/guile")) (delete-file-recursively (string-append share "/mes/scaffold")) - (for-each - delete-file - (find-files (string-append share "/mes/lib") - "\\.(h|c)"))))))))))) + + (for-each delete-file + (find-files + (string-append share "/mes/lib") "\\.(h|c)")) + + (for-each (lambda (dir) + (for-each remove-store-references + (find-files (string-append out "/" dir) + ".*"))) + '("bin" "share/mes"))))))))))) (define %guile-static ;; A statically-linked Guile that is relocatable--i.e., it can search -- 2.21.0 --=-=-= Content-Type: text/x-patch Content-Disposition: inline; filename=0003-bootstrap-bootstrap-mescc-tools-Update.patch >From 172ee1045689cd42d2de837ee560d32c43730cc3 Mon Sep 17 00:00:00 2001 From: Jan Nieuwenhuizen Date: Sun, 21 Jul 2019 14:38:52 +0200 Subject: [PATCH 3/4] bootstrap: bootstrap-mescc-tools: Update. Built with 2e13b6fff94027158b3de5d3a1469dc9f89b0c39 bootstrap: mes-minimal-stripped: Remove store references. * gnu/packages/bootstrap.scm (%bootstrap-mescc-tools): Update. --- gnu/packages/bootstrap.scm | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/gnu/packages/bootstrap.scm b/gnu/packages/bootstrap.scm index 8dbe52435e..a8a5c93e01 100644 --- a/gnu/packages/bootstrap.scm +++ b/gnu/packages/bootstrap.scm @@ -703,7 +703,7 @@ exec ~a/bin/.gcc-wrapped -B~a/lib \ ;; %MESCC-TOOLS-BOOTSTRAP-TARBALL. (package (name "bootstrap-mescc-tools") - (version "0.5.2") + (version "1") (source #f) (build-system trivial-build-system) (arguments @@ -735,12 +735,12 @@ exec ~a/bin/.gcc-wrapped -B~a/lib \ (method url-fetch) (uri (map (cute string-append <> - "/i686-linux/20181020/" - "mescc-tools-static-0.5.2-0.bb062b0-i686-linux.tar.xz") + "/i686-linux/20190721/" + "mescc-tools-static-stripped-0.5.2-0.bb062b0-i686-linux.tar.xz") %bootstrap-base-urls)) (sha256 (base32 - "11lniw0vg61kmyhvnwkmcnkci9ym6hbmiksiqggd0hkipbq7hvlz"))))))) + "0h7sbc5im74gxjx3xybqavq7hacbg7gcl4sznc1mpw67yja49f75"))))))) (synopsis "Bootstrap binaries of MesCC Tools") (description synopsis) (home-page #f) -- 2.21.0 --=-=-= Content-Type: text/x-patch Content-Disposition: inline; filename=0004-bootstrap-bootstrap-mes-Update.patch >From e316146f23dce4156fc8c5b3b28cd5e180487cda Mon Sep 17 00:00:00 2001 From: Jan Nieuwenhuizen Date: Sun, 21 Jul 2019 14:42:35 +0200 Subject: [PATCH 4/4] bootstrap: bootstrap-mes: Update. Built with 2e13b6fff94027158b3de5d3a1469dc9f89b0c39 bootstrap: mes-minimal-stripped: Remove store references. * gnu/packages/bootstrap.scm (%bootstrap-mes): Update. --- gnu/packages/bootstrap.scm | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/gnu/packages/bootstrap.scm b/gnu/packages/bootstrap.scm index a8a5c93e01..73fc7d3e5e 100644 --- a/gnu/packages/bootstrap.scm +++ b/gnu/packages/bootstrap.scm @@ -752,7 +752,7 @@ exec ~a/bin/.gcc-wrapped -B~a/lib \ ;; %MES-BOOTSTRAP-TARBALL. (package (name "bootstrap-mes") - (version "0") + (version "1") (source #f) (build-system trivial-build-system) (arguments @@ -784,12 +784,12 @@ exec ~a/bin/.gcc-wrapped -B~a/lib \ (method url-fetch) (uri (map (cute string-append <> - "/i686-linux/20181020/" + "/i686-linux/20190721/" "mes-minimal-stripped-0.19-i686-linux.tar.xz") %bootstrap-base-urls)) (sha256 (base32 - "0k7kkl68a6xaadv47ij0nr9jm5ca1ffj38n7f2lg80y72wdkwr9h"))))))) + "0c8j9p84vx4w8fz6bkxf57wsarq3ng1ka09h18lv3lfxxvnpiak6"))))))) (supported-systems '("i686-linux" "x86_64-linux")) (synopsis "Bootstrap binaries of Mes") (description synopsis) -- 2.21.0 --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable --=20 Jan Nieuwenhuizen | GNU LilyPond http://lilypond.org Freelance IT http://JoyofSource.com | Avatar=C2=AE http://AvatarAcademy.com --=-=-=--