From mboxrd@z Thu Jan 1 00:00:00 1970 From: Mark H Weaver Subject: bug#36747: Official MesCC bootstrap binaries differ from my locally built ones Date: Wed, 14 Aug 2019 14:35:47 -0400 Message-ID: <87v9uz4msh.fsf@netris.org> References: <875znwcoo9.fsf@netris.org> <87ef2j1pgt.fsf@gnu.org> <87ftmy51kk.fsf@netris.org> <87muh6sib4.fsf@gnu.org> <877e8a79mz.fsf@netris.org> <87pnm2ufv1.fsf@gnu.org> <87lfwpqpb7.fsf@netris.org> <875znt2hlc.fsf@gnu.org> <87zhke97xj.fsf@netris.org> <87h86mdaex.fsf@gnu.org> <8736i5a7mb.fsf@netris.org> <87mugdbc9r.fsf@gnu.org> <8736i3iyas.fsf@devup.no> <87zhkbhd07.fsf@devup.no> Mime-Version: 1.0 Content-Type: text/plain Return-path: Received: from eggs.gnu.org ([2001:470:142:3::10]:44439) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1hxy8u-0004dX-IP for bug-guix@gnu.org; Wed, 14 Aug 2019 14:37:06 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hxy8s-0007sw-Is for bug-guix@gnu.org; Wed, 14 Aug 2019 14:37:04 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:40832) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1hxy8s-0007sk-DR for bug-guix@gnu.org; Wed, 14 Aug 2019 14:37:02 -0400 Sender: "Debbugs-submit" Resent-Message-ID: In-Reply-To: <87zhkbhd07.fsf@devup.no> (Marius Bakke's message of "Wed, 14 Aug 2019 19:29:12 +0200") List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-guix-bounces+gcggb-bug-guix=m.gmane.org@gnu.org Sender: "bug-Guix" To: Marius Bakke Cc: 36747@debbugs.gnu.org Hi Marius, Marius Bakke writes: > Marius Bakke writes: > >> Jan Nieuwenhuizen writes: >> >>> Mark H Weaver writes: >>> >>> Hi Mark, >>> >>>>> I called that `wip-binaries', @master from three weeks ago. >>>> >>>> Thank you, that was a good start. I found that some additional patches >>>> were needed to match the bootstrap binaries that 'core-updates' is >>>> currently based on. >>>> >>>> I ended up deleting and repushing a revised 'wip-binaries' to Savannah. >>>> It includes slightly modified versions of the two commits you had >>>> included, as well as some additional cherry-picked commits of yours to >>>> update mescc-tools and add linux-libre-headers-bootstrap-tarball, and a >>>> few of my own. >>> >>> Very nice. >>> >>>> I built the new bootstrap tarballs at the new 'wip-binaries', commit >>>> c67becb31c30a5cd7685f166970ac4793e3a34a9, and here's what I got: >>>> >>>> mhw@jojen ~/guix-wip-binaries$ git describe >>>> v1.0.1-2404-gc67becb31c >>>> mhw@jojen ~/guix-wip-binaries$ ./pre-inst-env guix build --system=i686-linux bootstrap-tarballs >>>> /gnu/store/bg086i2qw1fn2jgbd15d9v91hyjrjsb2-bootstrap-tarballs-0 >>>> mhw@jojen ~/guix-wip-binaries$ cd /gnu/store/bg086i2qw1fn2jgbd15d9v91hyjrjsb2-bootstrap-tarballs-0 >>>> mhw@jojen /gnu/store/bg086i2qw1fn2jgbd15d9v91hyjrjsb2-bootstrap-tarballs-0$ sha256sum * >>>> 3e50c070a100b6bcf84c4bf5c868f9cd0a9fd1570f5d82fbfb78f8411959091b guile-static-stripped-2.2.4-i686-linux.tar.xz >>>> 1acd8f83e27d2fac311a5ca78e9bf11a9a1638b82469870d5c854c4e7afaa26a linux-libre-headers-stripped-4.14.67-i686-linux.tar.xz >>>> 021543d9bb6af55f39e68d69692e3cb74646ced2cad0bb9ac0047ef81e9d7330 mescc-tools-static-stripped-0.5.2-0.bb062b0-i686-linux.tar.xz >>>> fb32090071b39fc804fb9a7fba96f0bc5eb844a0efd268fb24c42e6bfa959de0 mes-minimal-stripped-0.19-i686-linux.tar.xz >>>> c80cdd17b0a24eebdd75570ff72c4ec06e129bd702ac008186b57f6301c448e7 static-binaries-0-i686-linux.tar.xz >>> >>>> Can you try "guix build --system=i686-linux bootstrap-tarballs" at the >>>> new 'wip-binaries' branch and see if you get the same results? >>> >>> Yes, on c67becb31c30a5cd7685f166970ac4793e3a34a9 running >>> "./pre-inst-env guix build --system=i686-linux bootstrap-tarballs" gives me exactly this, >>> also for guile-static-stripped! \o/ >>> >>>> Also, I have a question: One of the changes I made to 'wip-binaries' was >>>> to update mescc-tools to 0.5.2-0.bb062b0, to match the >>>> %bootstrap-mescc-tools that's currently being used in 'core-updates'. >>>> >>>> However, I noticed that you have also apparently built the official >>>> release of mescc-tools-0.5.2, which is on your site: >>>> >>>> http://lilypond.org/janneke/guix/20190722/mescc-tools-static-stripped-0.5.2-i686-linux.tar.xz >>>> >>>> and that this tarball is identical to the build output of the later git >>>> commit: mescc-tools-static-stripped-0.5.2-0.bb062b0-i686-linux.tar.xz. >>>> >>>> With this in mind, could we just use 0.5.2? What changed between 0.5.2 >>>> and 0.5.2-0.bb062b0, and what was the rationale for updating to bb062b0? >>> >>> Good catch. We probably can, we might try that. >>> >>> I think the need for updating to bb062b0 has been removed during the >>> review of the integration of the reduced binary seed bootstrap into >>> core-updates by Ludovic. >>> >>> For historical reasons, I think this mescc-tools commit >>> >>> --8<---------------cut here---------------start------------->8--- >>> commit c184e95096881a13f29ebd7fc507fe305d3d8de5 (gitlab/janneke, janneke) >>> Author: Jan Nieuwenhuizen >>> Date: Thu Oct 4 22:03:31 2018 +0200 >>> >>> build.sh: Update for mes 0.18. >>> --8<---------------cut here---------------end--------------->8--- >>> >>> was needed at a time that we did not have mescc-tools or mes in >>> bootstrap tarballs. We built bootstrap variants of mescc-tools and mes >>> using a externally (outside fo Guix) built mescc-tools-seed and >>> (an almost pure ASCII) mes-seed. >> >> I tried building the i686 bootstrap tarballs from wip-binaries with this >> additional patch: >> >> diff --git a/gnu/packages/mes.scm b/gnu/packages/mes.scm >> index e298cb05c1..380cac6c88 100644 >> --- a/gnu/packages/mes.scm >> +++ b/gnu/packages/mes.scm >> @@ -139,33 +139,31 @@ Guile.") >> (license gpl3+))) >> >> (define-public mescc-tools >> - (let ((commit "bb062b0da7bf2724ca40f9002b121579898d4ef7") >> - (revision "0") >> - (version "0.5.2")) >> - (package >> - (name "mescc-tools") >> - (version (string-append version "-" revision "." (string-take commit 7))) >> - (source (origin >> - (method url-fetch) >> - (uri (string-append >> - "https://git.savannah.nongnu.org/cgit/mescc-tools.git/snapshot/" >> - name "-" commit >> - ".tar.gz")) >> - (sha256 >> - (base32 >> - "1h6j57wyf91i42b26f8msbv6451cw3nm4nmpl1fckp9c7vi8mwkh")))) >> - (build-system gnu-build-system) >> - (supported-systems '("i686-linux" "x86_64-linux")) >> - (arguments >> - `(#:make-flags (list (string-append "PREFIX=" (assoc-ref %outputs "out"))) >> - #:test-target "test" >> - #:phases (modify-phases %standard-phases >> - (delete 'configure)))) >> - (synopsis "Tools for the full source bootstrapping process") >> - (description >> - "Mescc-tools is a collection of tools for use in a full source >> + (package >> + (name "mescc-tools") >> + (version "0.5.2") >> + (source (origin >> + (method url-fetch) >> + (uri (string-append >> + "https://git.savannah.nongnu.org/cgit/mescc-tools.git/snapshot/" >> + name "-Release_" version >> + ".tar.gz")) >> + (file-name (string-append name "-" version ".tar.gz")) >> + (sha256 >> + (base32 >> + "01x7bhmgwyf6mc2g1hcvibhps98nllacqm4f0j5l51b1mbi18pc2")))) >> + (build-system gnu-build-system) >> + (supported-systems '("i686-linux" "x86_64-linux")) >> + (arguments >> + `(#:make-flags (list (string-append "PREFIX=" (assoc-ref %outputs "out"))) >> + #:test-target "test" >> + #:phases (modify-phases %standard-phases >> + (delete 'configure)))) >> + (synopsis "Tools for the full source bootstrapping process") >> + (description >> + "Mescc-tools is a collection of tools for use in a full source >> bootstrapping process. It consists of the M1 macro assembler, the hex2 >> linker, the blood-elf symbol table generator, the kaem shell, exec_enable and >> get_machine.") >> (home-page "https://savannah.nongnu.org/projects/mescc-tools") >> - (license gpl3+)))) >> + (license gpl3+))) I guess this is equivalent to reverting commit 78ced7975b0665e810834391d826c9f0ef7277e1 on the 'wip-binaries' branch, yes? >> And got this result: >> >> $ cd /gnu/store/rdwyr8mh7dvhfkb5g4cws6q40hp23rbi-bootstrap-tarballs-0 >> $ sha256sum * >> 3e50c070a100b6bcf84c4bf5c868f9cd0a9fd1570f5d82fbfb78f8411959091b guile-static-stripped-2.2.4-i686-linux.tar.xz >> 1acd8f83e27d2fac311a5ca78e9bf11a9a1638b82469870d5c854c4e7afaa26a linux-libre-headers-stripped-4.14.67-i686-linux.tar.xz >> 021543d9bb6af55f39e68d69692e3cb74646ced2cad0bb9ac0047ef81e9d7330 mescc-tools-static-stripped-0.5.2-i686-linux.tar.xz >> fb32090071b39fc804fb9a7fba96f0bc5eb844a0efd268fb24c42e6bfa959de0 mes-minimal-stripped-0.19-i686-linux.tar.xz >> c80cdd17b0a24eebdd75570ff72c4ec06e129bd702ac008186b57f6301c448e7 static-binaries-0-i686-linux.tar.xz Great, looks good to me! >> I also merged the branch to core-updates and reverted the bash patch, I think that 'bash-4.4-linux-pgrp-pipe.patch', adapted to 5.0, should be applied to all 'bash' packages in 'core-updates'. To be more precise, the patch should be applied to the main 'bash' package in bash.scm, inherited from all other bash packages, so that PGRP_PIPE is set unconditionally set on systems based on Linux (the kernel), regardless of the kernel version running on the build machine. >> which produced this derivation for "guix build -d -s i686-linux >> bootstrap-tarballs": >> >> /gnu/store/ld82vli1msfrlimjaryznrqcwm0jc5ii-bootstrap-tarballs-0.drv > > Here are the hashes from this derivation: > > $ cd /gnu/store/srsqilb3g70r8c7vma0gpam21z1zmg2w-bootstrap-tarballs-0 > $ sha256sum * > 4e5b219be4d9ad4d125f17b8a8a991e78be3908aadc8d22d1a115e96ec856279 guile-static-stripped-2.2.6-i686-linux.tar.xz > 9e34b26526b184912b0d97df96e316151fd97ede24aec2f22dd13ed33438b2e4 linux-libre-headers-stripped-4.19.56-i686-linux.tar.xz > 7db07a7097a7920e17f6f1794098b9f29bc3522c36a33cf797123f64581cdc34 mescc-tools-static-0.5.2-i686-linux.tar.xz > 3a9b050c1a2b61bb1cbed16ee07aecdc83e9097ce4a914623aa1b9808abe57f4 mes-minimal-stripped-0.19-i686-linux.tar.xz > 1ccb4f39656eb977706db7a4aa811461785200b0a11e847035bb89fef91b85ab static-binaries-0-i686-linux.tar.xz What was the purpose of building these? 'core-updates' is built upon on the earlier, unverified, reduced binary seed bootstrap binaries. > @Mark: do you think we are ready to merge this now? Can you do it? I think what needs to be done is the following: (1) commit 78ced7975b0665e810834391d826c9f0ef7277e1 on 'wip-binaries' should be reverted, to downgrade mescc-tools to the 0.5.2 release. (2) The 'wip-binaries' tarballs should be uploaded to a new subdirectory of , along with digital signatures, of course. I'm talking about these in particular: 3e50c070a100b6bcf84c4bf5c868f9cd0a9fd1570f5d82fbfb78f8411959091b guile-static-stripped-2.2.4-i686-linux.tar.xz 1acd8f83e27d2fac311a5ca78e9bf11a9a1638b82469870d5c854c4e7afaa26a linux-libre-headers-stripped-4.14.67-i686-linux.tar.xz 021543d9bb6af55f39e68d69692e3cb74646ced2cad0bb9ac0047ef81e9d7330 mescc-tools-static-stripped-0.5.2-i686-linux.tar.xz fb32090071b39fc804fb9a7fba96f0bc5eb844a0efd268fb24c42e6bfa959de0 mes-minimal-stripped-0.19-i686-linux.tar.xz c80cdd17b0a24eebdd75570ff72c4ec06e129bd702ac008186b57f6301c448e7 static-binaries-0-i686-linux.tar.xz (3) The following bootstrap packages in 'core-updates' bootstrap.scm should be updated to use the new binaries above: (a) %bootstrap-linux-libre-headers (b) %bootstrap-mescc-tools (c) %bootstrap-mes (4) Berlin should start rebuilding 'core-updates'. If desired, steps (3) and (4) could come before (2) if someone temporarily uploads the new binaries somewhere else, and adjusts '%bootstrap-base-urls' accordingly. The key is for the hashes and file names to match what we've agreed on here, as I listed in (2) above. What do you think? It will be a couple of days before I can do more solid work on this, but I'd be grateful if others could carry it forward in the meantime. Thanks! Mark