From mboxrd@z Thu Jan 1 00:00:00 1970 From: Marius Bakke Subject: bug#36747: Official MesCC bootstrap binaries differ from my locally built ones Date: Wed, 14 Aug 2019 21:56:38 +0200 Message-ID: <87woffh66h.fsf@devup.no> References: <875znwcoo9.fsf@netris.org> <87ef2j1pgt.fsf@gnu.org> <87ftmy51kk.fsf@netris.org> <87muh6sib4.fsf@gnu.org> <877e8a79mz.fsf@netris.org> <87pnm2ufv1.fsf@gnu.org> <87lfwpqpb7.fsf@netris.org> <875znt2hlc.fsf@gnu.org> <87zhke97xj.fsf@netris.org> <87h86mdaex.fsf@gnu.org> <8736i5a7mb.fsf@netris.org> <87mugdbc9r.fsf@gnu.org> <8736i3iyas.fsf@devup.no> <87zhkbhd07.fsf@devup.no> <87v9uz4msh.fsf@netris.org> Mime-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" Return-path: Received: from eggs.gnu.org ([2001:470:142:3::10]:57329) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1hxzOK-0005mF-Ai for bug-guix@gnu.org; Wed, 14 Aug 2019 15:57:06 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hxzOI-0004xt-6S for bug-guix@gnu.org; Wed, 14 Aug 2019 15:57:04 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:40907) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1hxzOI-0004xb-3O for bug-guix@gnu.org; Wed, 14 Aug 2019 15:57:02 -0400 Sender: "Debbugs-submit" Resent-Message-ID: In-Reply-To: <87v9uz4msh.fsf@netris.org> List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-guix-bounces+gcggb-bug-guix=m.gmane.org@gnu.org Sender: "bug-Guix" To: Mark H Weaver Cc: 36747@debbugs.gnu.org --=-=-= Content-Type: text/plain Content-Transfer-Encoding: quoted-printable Mark H Weaver writes: > Hi Marius, > > Marius Bakke writes: > >> Marius Bakke writes: >> >>> Jan Nieuwenhuizen writes: >>> >>>> Mark H Weaver writes: >>>> >>>> Hi Mark, >>>> >>>>>> I called that `wip-binaries', @master from three weeks ago. >>>>> >>>>> Thank you, that was a good start. I found that some additional patch= es >>>>> were needed to match the bootstrap binaries that 'core-updates' is >>>>> currently based on. >>>>> >>>>> I ended up deleting and repushing a revised 'wip-binaries' to Savanna= h. >>>>> It includes slightly modified versions of the two commits you had >>>>> included, as well as some additional cherry-picked commits of yours to >>>>> update mescc-tools and add linux-libre-headers-bootstrap-tarball, and= a >>>>> few of my own. >>>> >>>> Very nice. >>>> >>>>> I built the new bootstrap tarballs at the new 'wip-binaries', commit >>>>> c67becb31c30a5cd7685f166970ac4793e3a34a9, and here's what I got: >>>>> >>>>> mhw@jojen ~/guix-wip-binaries$ git describe >>>>> v1.0.1-2404-gc67becb31c >>>>> mhw@jojen ~/guix-wip-binaries$ ./pre-inst-env guix build --system=3Di= 686-linux bootstrap-tarballs >>>>> /gnu/store/bg086i2qw1fn2jgbd15d9v91hyjrjsb2-bootstrap-tarballs-0 >>>>> mhw@jojen ~/guix-wip-binaries$ cd /gnu/store/bg086i2qw1fn2jgbd15d9v91= hyjrjsb2-bootstrap-tarballs-0 >>>>> mhw@jojen /gnu/store/bg086i2qw1fn2jgbd15d9v91hyjrjsb2-bootstrap-tarba= lls-0$ sha256sum * >>>>> 3e50c070a100b6bcf84c4bf5c868f9cd0a9fd1570f5d82fbfb78f8411959091b gui= le-static-stripped-2.2.4-i686-linux.tar.xz >>>>> 1acd8f83e27d2fac311a5ca78e9bf11a9a1638b82469870d5c854c4e7afaa26a lin= ux-libre-headers-stripped-4.14.67-i686-linux.tar.xz >>>>> 021543d9bb6af55f39e68d69692e3cb74646ced2cad0bb9ac0047ef81e9d7330 mes= cc-tools-static-stripped-0.5.2-0.bb062b0-i686-linux.tar.xz >>>>> fb32090071b39fc804fb9a7fba96f0bc5eb844a0efd268fb24c42e6bfa959de0 mes= -minimal-stripped-0.19-i686-linux.tar.xz >>>>> c80cdd17b0a24eebdd75570ff72c4ec06e129bd702ac008186b57f6301c448e7 sta= tic-binaries-0-i686-linux.tar.xz >>>> >>>>> Can you try "guix build --system=3Di686-linux bootstrap-tarballs" at = the >>>>> new 'wip-binaries' branch and see if you get the same results? >>>> >>>> Yes, on c67becb31c30a5cd7685f166970ac4793e3a34a9 running >>>> "./pre-inst-env guix build --system=3Di686-linux bootstrap-tarballs" g= ives me exactly this, >>>> also for guile-static-stripped! \o/ >>>> >>>>> Also, I have a question: One of the changes I made to 'wip-binaries' = was >>>>> to update mescc-tools to 0.5.2-0.bb062b0, to match the >>>>> %bootstrap-mescc-tools that's currently being used in 'core-updates'. >>>>> >>>>> However, I noticed that you have also apparently built the official >>>>> release of mescc-tools-0.5.2, which is on your site: >>>>> >>>>> http://lilypond.org/janneke/guix/20190722/mescc-tools-static-stripp= ed-0.5.2-i686-linux.tar.xz >>>>> >>>>> and that this tarball is identical to the build output of the later g= it >>>>> commit: mescc-tools-static-stripped-0.5.2-0.bb062b0-i686-linux.tar.xz. >>>>> >>>>> With this in mind, could we just use 0.5.2? What changed between 0.5= .2 >>>>> and 0.5.2-0.bb062b0, and what was the rationale for updating to bb062= b0? >>>> >>>> Good catch. We probably can, we might try that. >>>> >>>> I think the need for updating to bb062b0 has been removed during the >>>> review of the integration of the reduced binary seed bootstrap into >>>> core-updates by Ludovic. >>>> >>>> For historical reasons, I think this mescc-tools commit >>>> >>>> --8<---------------cut here---------------start------------->8--- >>>> commit c184e95096881a13f29ebd7fc507fe305d3d8de5 (gitlab/janneke, janne= ke) >>>> Author: Jan Nieuwenhuizen >>>> Date: Thu Oct 4 22:03:31 2018 +0200 >>>> >>>> build.sh: Update for mes 0.18. >>>> --8<---------------cut here---------------end--------------->8--- >>>> >>>> was needed at a time that we did not have mescc-tools or mes in >>>> bootstrap tarballs. We built bootstrap variants of mescc-tools and mes >>>> using a externally (outside fo Guix) built mescc-tools-seed and >>>> (an almost pure ASCII) mes-seed. >>> >>> I tried building the i686 bootstrap tarballs from wip-binaries with this >>> additional patch: >>> >>> diff --git a/gnu/packages/mes.scm b/gnu/packages/mes.scm >>> index e298cb05c1..380cac6c88 100644 >>> --- a/gnu/packages/mes.scm >>> +++ b/gnu/packages/mes.scm >>> @@ -139,33 +139,31 @@ Guile.") >>> (license gpl3+))) >>>=20=20 >>> (define-public mescc-tools >>> - (let ((commit "bb062b0da7bf2724ca40f9002b121579898d4ef7") >>> - (revision "0") >>> - (version "0.5.2")) >>> - (package >>> - (name "mescc-tools") >>> - (version (string-append version "-" revision "." (string-take co= mmit 7))) >>> - (source (origin >>> - (method url-fetch) >>> - (uri (string-append >>> - "https://git.savannah.nongnu.org/cgit/mescc-tool= s.git/snapshot/" >>> - name "-" commit >>> - ".tar.gz")) >>> - (sha256 >>> - (base32 >>> - "1h6j57wyf91i42b26f8msbv6451cw3nm4nmpl1fckp9c7vi8mwk= h")))) >>> - (build-system gnu-build-system) >>> - (supported-systems '("i686-linux" "x86_64-linux")) >>> - (arguments >>> - `(#:make-flags (list (string-append "PREFIX=3D" (assoc-ref %out= puts "out"))) >>> - #:test-target "test" >>> - #:phases (modify-phases %standard-phases >>> - (delete 'configure)))) >>> - (synopsis "Tools for the full source bootstrapping process") >>> - (description >>> - "Mescc-tools is a collection of tools for use in a full source >>> + (package >>> + (name "mescc-tools") >>> + (version "0.5.2") >>> + (source (origin >>> + (method url-fetch) >>> + (uri (string-append >>> + "https://git.savannah.nongnu.org/cgit/mescc-tools.= git/snapshot/" >>> + name "-Release_" version >>> + ".tar.gz")) >>> + (file-name (string-append name "-" version ".tar.gz")) >>> + (sha256 >>> + (base32 >>> + "01x7bhmgwyf6mc2g1hcvibhps98nllacqm4f0j5l51b1mbi18pc2"= )))) >>> + (build-system gnu-build-system) >>> + (supported-systems '("i686-linux" "x86_64-linux")) >>> + (arguments >>> + `(#:make-flags (list (string-append "PREFIX=3D" (assoc-ref %outpu= ts "out"))) >>> + #:test-target "test" >>> + #:phases (modify-phases %standard-phases >>> + (delete 'configure)))) >>> + (synopsis "Tools for the full source bootstrapping process") >>> + (description >>> + "Mescc-tools is a collection of tools for use in a full source >>> bootstrapping process. It consists of the M1 macro assembler, the hex2 >>> linker, the blood-elf symbol table generator, the kaem shell, exec_ena= ble and >>> get_machine.") >>> (home-page "https://savannah.nongnu.org/projects/mescc-tools") >>> - (license gpl3+)))) >>> + (license gpl3+))) > > I guess this is equivalent to reverting commit > 78ced7975b0665e810834391d826c9f0ef7277e1 on the 'wip-binaries' branch, > yes? Indeed. >>> And got this result: >>> >>> $ cd /gnu/store/rdwyr8mh7dvhfkb5g4cws6q40hp23rbi-bootstrap-tarballs-0 >>> $ sha256sum * >>> 3e50c070a100b6bcf84c4bf5c868f9cd0a9fd1570f5d82fbfb78f8411959091b guile= -static-stripped-2.2.4-i686-linux.tar.xz >>> 1acd8f83e27d2fac311a5ca78e9bf11a9a1638b82469870d5c854c4e7afaa26a linux= -libre-headers-stripped-4.14.67-i686-linux.tar.xz >>> 021543d9bb6af55f39e68d69692e3cb74646ced2cad0bb9ac0047ef81e9d7330 mescc= -tools-static-stripped-0.5.2-i686-linux.tar.xz >>> fb32090071b39fc804fb9a7fba96f0bc5eb844a0efd268fb24c42e6bfa959de0 mes-m= inimal-stripped-0.19-i686-linux.tar.xz >>> c80cdd17b0a24eebdd75570ff72c4ec06e129bd702ac008186b57f6301c448e7 stati= c-binaries-0-i686-linux.tar.xz > > Great, looks good to me! > >>> I also merged the branch to core-updates and reverted the bash patch, > > I think that 'bash-4.4-linux-pgrp-pipe.patch', adapted to 5.0, should be > applied to all 'bash' packages in 'core-updates'. To be more precise, > the patch should be applied to the main 'bash' package in bash.scm, > inherited from all other bash packages, so that PGRP_PIPE is set > unconditionally set on systems based on Linux (the kernel), regardless > of the kernel version running on the build machine. Got it. >>> which produced this derivation for "guix build -d -s i686-linux >>> bootstrap-tarballs": >>> >>> /gnu/store/ld82vli1msfrlimjaryznrqcwm0jc5ii-bootstrap-tarballs-0.drv >> >> Here are the hashes from this derivation: >> >> $ cd /gnu/store/srsqilb3g70r8c7vma0gpam21z1zmg2w-bootstrap-tarballs-0 >> $ sha256sum * >> 4e5b219be4d9ad4d125f17b8a8a991e78be3908aadc8d22d1a115e96ec856279 guile-= static-stripped-2.2.6-i686-linux.tar.xz >> 9e34b26526b184912b0d97df96e316151fd97ede24aec2f22dd13ed33438b2e4 linux-= libre-headers-stripped-4.19.56-i686-linux.tar.xz >> 7db07a7097a7920e17f6f1794098b9f29bc3522c36a33cf797123f64581cdc34 mescc-= tools-static-0.5.2-i686-linux.tar.xz >> 3a9b050c1a2b61bb1cbed16ee07aecdc83e9097ce4a914623aa1b9808abe57f4 mes-mi= nimal-stripped-0.19-i686-linux.tar.xz >> 1ccb4f39656eb977706db7a4aa811461785200b0a11e847035bb89fef91b85ab static= -binaries-0-i686-linux.tar.xz > > What was the purpose of building these? 'core-updates' is built upon on > the earlier, unverified, reduced binary seed bootstrap binaries. I wanted to check that the bootstrap-tarballs machinery still worked after merging the branch, since it was non-trivial. Mainly to make the commit that created them reachable forever, but maybe we don't need it. >> @Mark: do you think we are ready to merge this now? Can you do it? > > I think what needs to be done is the following: > > (1) commit 78ced7975b0665e810834391d826c9f0ef7277e1 on 'wip-binaries' > should be reverted, to downgrade mescc-tools to the 0.5.2 release. > > (2) The 'wip-binaries' tarballs should be uploaded to a new subdirectory > of , along > with digital signatures, of course. I'm talking about these in > particular: > > 3e50c070a100b6bcf84c4bf5c868f9cd0a9fd1570f5d82fbfb78f8411959091b gui= le-static-stripped-2.2.4-i686-linux.tar.xz > 1acd8f83e27d2fac311a5ca78e9bf11a9a1638b82469870d5c854c4e7afaa26a lin= ux-libre-headers-stripped-4.14.67-i686-linux.tar.xz > 021543d9bb6af55f39e68d69692e3cb74646ced2cad0bb9ac0047ef81e9d7330 mes= cc-tools-static-stripped-0.5.2-i686-linux.tar.xz > fb32090071b39fc804fb9a7fba96f0bc5eb844a0efd268fb24c42e6bfa959de0 mes= -minimal-stripped-0.19-i686-linux.tar.xz > c80cdd17b0a24eebdd75570ff72c4ec06e129bd702ac008186b57f6301c448e7 sta= tic-binaries-0-i686-linux.tar.xz > > (3) The following bootstrap packages in 'core-updates' bootstrap.scm > should be updated to use the new binaries above: > > (a) %bootstrap-linux-libre-headers > (b) %bootstrap-mescc-tools > (c) %bootstrap-mes > > (4) Berlin should start rebuilding 'core-updates'. > > If desired, steps (3) and (4) could come before (2) if someone > temporarily uploads the new binaries somewhere else, and adjusts > '%bootstrap-base-urls' accordingly. The key is for the hashes and file > names to match what we've agreed on here, as I listed in (2) above. > > What do you think? Thank you for the excellent summary. I can look into adjusting the bash fix for 5.0, and updating the bootstrap binary URLs and hashes. I will do this in a 'core-updates-next' branch. I would also like to merge wip-binaries into it as a final step, unless someone has objections. Ludovic should be back in a couple of days and can hopefully take care of the uploads. Ricardo: can you instruct Cuirass to add a reduced jobset for 'core-updates-next', that only builds builds the "core" package subset? --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQEzBAEBCgAdFiEEu7At3yzq9qgNHeZDoqBt8qM6VPoFAl1UZ3YACgkQoqBt8qM6 VPrb4Qf/W6gi8ZSes1eH2Wd6Fn9R7kgLIeFK1mpVJc4aaXIfB+0M5BJol7k882qh oLpkVyEH/cWwmaOdhegmIPeDztQKNhDDHQibOJis2/Eg9KIhi5JlI4A9HrUE6zsJ t3L1ID7l4Ykojlx2RBVbvtHDKqf69rc8UxTyaR7hPV8eYCTc0aRsqueYpx4XYToq L2ADyEEXjrY6oVcG5wlTqr5XBWN3afU+g/4d4oB3FLMQ44jwXz+w8f9ulp0IPE1+ CxLCIFp57n5Ef65sYlPmzdyT8SXEezcla0/CHr8vGccXnPb/6l0SsR6yYtPfyFF1 Ttd7wGTyGEbJB+vbJOxYpje7SjvhfQ== =6vry -----END PGP SIGNATURE----- --=-=-=--