From: Marius Bakke <mbakke@fastmail.com>
To: Mark H Weaver <mhw@netris.org>
Cc: 36747@debbugs.gnu.org
Subject: bug#36747: Official MesCC bootstrap binaries differ from my locally built ones
Date: Wed, 14 Aug 2019 21:56:38 +0200 [thread overview]
Message-ID: <87woffh66h.fsf@devup.no> (raw)
In-Reply-To: <87v9uz4msh.fsf@netris.org>
[-- Attachment #1: Type: text/plain, Size: 12146 bytes --]
Mark H Weaver <mhw@netris.org> writes:
> Hi Marius,
>
> Marius Bakke <mbakke@fastmail.com> writes:
>
>> Marius Bakke <mbakke@fastmail.com> writes:
>>
>>> Jan Nieuwenhuizen <janneke@gnu.org> writes:
>>>
>>>> Mark H Weaver writes:
>>>>
>>>> Hi Mark,
>>>>
>>>>>> I called that `wip-binaries', @master from three weeks ago.
>>>>>
>>>>> Thank you, that was a good start. I found that some additional patches
>>>>> were needed to match the bootstrap binaries that 'core-updates' is
>>>>> currently based on.
>>>>>
>>>>> I ended up deleting and repushing a revised 'wip-binaries' to Savannah.
>>>>> It includes slightly modified versions of the two commits you had
>>>>> included, as well as some additional cherry-picked commits of yours to
>>>>> update mescc-tools and add linux-libre-headers-bootstrap-tarball, and a
>>>>> few of my own.
>>>>
>>>> Very nice.
>>>>
>>>>> I built the new bootstrap tarballs at the new 'wip-binaries', commit
>>>>> c67becb31c30a5cd7685f166970ac4793e3a34a9, and here's what I got:
>>>>>
>>>>> mhw@jojen ~/guix-wip-binaries$ git describe
>>>>> v1.0.1-2404-gc67becb31c
>>>>> mhw@jojen ~/guix-wip-binaries$ ./pre-inst-env guix build --system=i686-linux bootstrap-tarballs
>>>>> /gnu/store/bg086i2qw1fn2jgbd15d9v91hyjrjsb2-bootstrap-tarballs-0
>>>>> mhw@jojen ~/guix-wip-binaries$ cd /gnu/store/bg086i2qw1fn2jgbd15d9v91hyjrjsb2-bootstrap-tarballs-0
>>>>> mhw@jojen /gnu/store/bg086i2qw1fn2jgbd15d9v91hyjrjsb2-bootstrap-tarballs-0$ sha256sum *
>>>>> 3e50c070a100b6bcf84c4bf5c868f9cd0a9fd1570f5d82fbfb78f8411959091b guile-static-stripped-2.2.4-i686-linux.tar.xz
>>>>> 1acd8f83e27d2fac311a5ca78e9bf11a9a1638b82469870d5c854c4e7afaa26a linux-libre-headers-stripped-4.14.67-i686-linux.tar.xz
>>>>> 021543d9bb6af55f39e68d69692e3cb74646ced2cad0bb9ac0047ef81e9d7330 mescc-tools-static-stripped-0.5.2-0.bb062b0-i686-linux.tar.xz
>>>>> fb32090071b39fc804fb9a7fba96f0bc5eb844a0efd268fb24c42e6bfa959de0 mes-minimal-stripped-0.19-i686-linux.tar.xz
>>>>> c80cdd17b0a24eebdd75570ff72c4ec06e129bd702ac008186b57f6301c448e7 static-binaries-0-i686-linux.tar.xz
>>>>
>>>>> Can you try "guix build --system=i686-linux bootstrap-tarballs" at the
>>>>> new 'wip-binaries' branch and see if you get the same results?
>>>>
>>>> Yes, on c67becb31c30a5cd7685f166970ac4793e3a34a9 running
>>>> "./pre-inst-env guix build --system=i686-linux bootstrap-tarballs" gives me exactly this,
>>>> also for guile-static-stripped! \o/
>>>>
>>>>> Also, I have a question: One of the changes I made to 'wip-binaries' was
>>>>> to update mescc-tools to 0.5.2-0.bb062b0, to match the
>>>>> %bootstrap-mescc-tools that's currently being used in 'core-updates'.
>>>>>
>>>>> However, I noticed that you have also apparently built the official
>>>>> release of mescc-tools-0.5.2, which is on your site:
>>>>>
>>>>> http://lilypond.org/janneke/guix/20190722/mescc-tools-static-stripped-0.5.2-i686-linux.tar.xz
>>>>>
>>>>> and that this tarball is identical to the build output of the later git
>>>>> commit: mescc-tools-static-stripped-0.5.2-0.bb062b0-i686-linux.tar.xz.
>>>>>
>>>>> With this in mind, could we just use 0.5.2? What changed between 0.5.2
>>>>> and 0.5.2-0.bb062b0, and what was the rationale for updating to bb062b0?
>>>>
>>>> Good catch. We probably can, we might try that.
>>>>
>>>> I think the need for updating to bb062b0 has been removed during the
>>>> review of the integration of the reduced binary seed bootstrap into
>>>> core-updates by Ludovic.
>>>>
>>>> For historical reasons, I think this mescc-tools commit
>>>>
>>>> --8<---------------cut here---------------start------------->8---
>>>> commit c184e95096881a13f29ebd7fc507fe305d3d8de5 (gitlab/janneke, janneke)
>>>> Author: Jan Nieuwenhuizen <janneke@gnu.org>
>>>> Date: Thu Oct 4 22:03:31 2018 +0200
>>>>
>>>> build.sh: Update for mes 0.18.
>>>> --8<---------------cut here---------------end--------------->8---
>>>>
>>>> was needed at a time that we did not have mescc-tools or mes in
>>>> bootstrap tarballs. We built bootstrap variants of mescc-tools and mes
>>>> using a externally (outside fo Guix) built mescc-tools-seed and
>>>> (an almost pure ASCII) mes-seed.
>>>
>>> I tried building the i686 bootstrap tarballs from wip-binaries with this
>>> additional patch:
>>>
>>> diff --git a/gnu/packages/mes.scm b/gnu/packages/mes.scm
>>> index e298cb05c1..380cac6c88 100644
>>> --- a/gnu/packages/mes.scm
>>> +++ b/gnu/packages/mes.scm
>>> @@ -139,33 +139,31 @@ Guile.")
>>> (license gpl3+)))
>>>
>>> (define-public mescc-tools
>>> - (let ((commit "bb062b0da7bf2724ca40f9002b121579898d4ef7")
>>> - (revision "0")
>>> - (version "0.5.2"))
>>> - (package
>>> - (name "mescc-tools")
>>> - (version (string-append version "-" revision "." (string-take commit 7)))
>>> - (source (origin
>>> - (method url-fetch)
>>> - (uri (string-append
>>> - "https://git.savannah.nongnu.org/cgit/mescc-tools.git/snapshot/"
>>> - name "-" commit
>>> - ".tar.gz"))
>>> - (sha256
>>> - (base32
>>> - "1h6j57wyf91i42b26f8msbv6451cw3nm4nmpl1fckp9c7vi8mwkh"))))
>>> - (build-system gnu-build-system)
>>> - (supported-systems '("i686-linux" "x86_64-linux"))
>>> - (arguments
>>> - `(#:make-flags (list (string-append "PREFIX=" (assoc-ref %outputs "out")))
>>> - #:test-target "test"
>>> - #:phases (modify-phases %standard-phases
>>> - (delete 'configure))))
>>> - (synopsis "Tools for the full source bootstrapping process")
>>> - (description
>>> - "Mescc-tools is a collection of tools for use in a full source
>>> + (package
>>> + (name "mescc-tools")
>>> + (version "0.5.2")
>>> + (source (origin
>>> + (method url-fetch)
>>> + (uri (string-append
>>> + "https://git.savannah.nongnu.org/cgit/mescc-tools.git/snapshot/"
>>> + name "-Release_" version
>>> + ".tar.gz"))
>>> + (file-name (string-append name "-" version ".tar.gz"))
>>> + (sha256
>>> + (base32
>>> + "01x7bhmgwyf6mc2g1hcvibhps98nllacqm4f0j5l51b1mbi18pc2"))))
>>> + (build-system gnu-build-system)
>>> + (supported-systems '("i686-linux" "x86_64-linux"))
>>> + (arguments
>>> + `(#:make-flags (list (string-append "PREFIX=" (assoc-ref %outputs "out")))
>>> + #:test-target "test"
>>> + #:phases (modify-phases %standard-phases
>>> + (delete 'configure))))
>>> + (synopsis "Tools for the full source bootstrapping process")
>>> + (description
>>> + "Mescc-tools is a collection of tools for use in a full source
>>> bootstrapping process. It consists of the M1 macro assembler, the hex2
>>> linker, the blood-elf symbol table generator, the kaem shell, exec_enable and
>>> get_machine.")
>>> (home-page "https://savannah.nongnu.org/projects/mescc-tools")
>>> - (license gpl3+))))
>>> + (license gpl3+)))
>
> I guess this is equivalent to reverting commit
> 78ced7975b0665e810834391d826c9f0ef7277e1 on the 'wip-binaries' branch,
> yes?
Indeed.
>>> And got this result:
>>>
>>> $ cd /gnu/store/rdwyr8mh7dvhfkb5g4cws6q40hp23rbi-bootstrap-tarballs-0
>>> $ sha256sum *
>>> 3e50c070a100b6bcf84c4bf5c868f9cd0a9fd1570f5d82fbfb78f8411959091b guile-static-stripped-2.2.4-i686-linux.tar.xz
>>> 1acd8f83e27d2fac311a5ca78e9bf11a9a1638b82469870d5c854c4e7afaa26a linux-libre-headers-stripped-4.14.67-i686-linux.tar.xz
>>> 021543d9bb6af55f39e68d69692e3cb74646ced2cad0bb9ac0047ef81e9d7330 mescc-tools-static-stripped-0.5.2-i686-linux.tar.xz
>>> fb32090071b39fc804fb9a7fba96f0bc5eb844a0efd268fb24c42e6bfa959de0 mes-minimal-stripped-0.19-i686-linux.tar.xz
>>> c80cdd17b0a24eebdd75570ff72c4ec06e129bd702ac008186b57f6301c448e7 static-binaries-0-i686-linux.tar.xz
>
> Great, looks good to me!
>
>>> I also merged the branch to core-updates and reverted the bash patch,
>
> I think that 'bash-4.4-linux-pgrp-pipe.patch', adapted to 5.0, should be
> applied to all 'bash' packages in 'core-updates'. To be more precise,
> the patch should be applied to the main 'bash' package in bash.scm,
> inherited from all other bash packages, so that PGRP_PIPE is set
> unconditionally set on systems based on Linux (the kernel), regardless
> of the kernel version running on the build machine.
Got it.
>>> which produced this derivation for "guix build -d -s i686-linux
>>> bootstrap-tarballs":
>>>
>>> /gnu/store/ld82vli1msfrlimjaryznrqcwm0jc5ii-bootstrap-tarballs-0.drv
>>
>> Here are the hashes from this derivation:
>>
>> $ cd /gnu/store/srsqilb3g70r8c7vma0gpam21z1zmg2w-bootstrap-tarballs-0
>> $ sha256sum *
>> 4e5b219be4d9ad4d125f17b8a8a991e78be3908aadc8d22d1a115e96ec856279 guile-static-stripped-2.2.6-i686-linux.tar.xz
>> 9e34b26526b184912b0d97df96e316151fd97ede24aec2f22dd13ed33438b2e4 linux-libre-headers-stripped-4.19.56-i686-linux.tar.xz
>> 7db07a7097a7920e17f6f1794098b9f29bc3522c36a33cf797123f64581cdc34 mescc-tools-static-0.5.2-i686-linux.tar.xz
>> 3a9b050c1a2b61bb1cbed16ee07aecdc83e9097ce4a914623aa1b9808abe57f4 mes-minimal-stripped-0.19-i686-linux.tar.xz
>> 1ccb4f39656eb977706db7a4aa811461785200b0a11e847035bb89fef91b85ab static-binaries-0-i686-linux.tar.xz
>
> What was the purpose of building these? 'core-updates' is built upon on
> the earlier, unverified, reduced binary seed bootstrap binaries.
I wanted to check that the bootstrap-tarballs machinery still worked
after merging the branch, since it was non-trivial. Mainly to make the
commit that created them reachable forever, but maybe we don't need it.
>> @Mark: do you think we are ready to merge this now? Can you do it?
>
> I think what needs to be done is the following:
>
> (1) commit 78ced7975b0665e810834391d826c9f0ef7277e1 on 'wip-binaries'
> should be reverted, to downgrade mescc-tools to the 0.5.2 release.
>
> (2) The 'wip-binaries' tarballs should be uploaded to a new subdirectory
> of <https://alpha.gnu.org/gnu/guix/bootstrap/i686-linux/>, along
> with digital signatures, of course. I'm talking about these in
> particular:
>
> 3e50c070a100b6bcf84c4bf5c868f9cd0a9fd1570f5d82fbfb78f8411959091b guile-static-stripped-2.2.4-i686-linux.tar.xz
> 1acd8f83e27d2fac311a5ca78e9bf11a9a1638b82469870d5c854c4e7afaa26a linux-libre-headers-stripped-4.14.67-i686-linux.tar.xz
> 021543d9bb6af55f39e68d69692e3cb74646ced2cad0bb9ac0047ef81e9d7330 mescc-tools-static-stripped-0.5.2-i686-linux.tar.xz
> fb32090071b39fc804fb9a7fba96f0bc5eb844a0efd268fb24c42e6bfa959de0 mes-minimal-stripped-0.19-i686-linux.tar.xz
> c80cdd17b0a24eebdd75570ff72c4ec06e129bd702ac008186b57f6301c448e7 static-binaries-0-i686-linux.tar.xz
>
> (3) The following bootstrap packages in 'core-updates' bootstrap.scm
> should be updated to use the new binaries above:
>
> (a) %bootstrap-linux-libre-headers
> (b) %bootstrap-mescc-tools
> (c) %bootstrap-mes
>
> (4) Berlin should start rebuilding 'core-updates'.
>
> If desired, steps (3) and (4) could come before (2) if someone
> temporarily uploads the new binaries somewhere else, and adjusts
> '%bootstrap-base-urls' accordingly. The key is for the hashes and file
> names to match what we've agreed on here, as I listed in (2) above.
>
> What do you think?
Thank you for the excellent summary. I can look into adjusting the bash
fix for 5.0, and updating the bootstrap binary URLs and hashes. I will
do this in a 'core-updates-next' branch. I would also like to merge
wip-binaries into it as a final step, unless someone has objections.
Ludovic should be back in a couple of days and can hopefully take care
of the uploads.
Ricardo: can you instruct Cuirass to add a reduced jobset for
'core-updates-next', that only builds builds the "core" package subset?
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]
next prev parent reply other threads:[~2019-08-14 19:57 UTC|newest]
Thread overview: 58+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-07-20 22:43 bug#36747: Official MesCC bootstrap binaries differ from my locally built ones Mark H Weaver
2019-07-21 13:34 ` Jan Nieuwenhuizen
2019-07-22 0:56 ` Mark H Weaver
2019-07-22 6:18 ` Jan Nieuwenhuizen
2019-07-22 6:26 ` Jan Nieuwenhuizen
2019-07-22 8:26 ` Jan Nieuwenhuizen
2019-07-22 8:31 ` Mark H Weaver
2019-07-22 17:41 ` Jan Nieuwenhuizen
2019-07-23 5:42 ` Mark H Weaver
2019-07-23 6:28 ` Jan Nieuwenhuizen
2019-08-12 0:21 ` Mark H Weaver
2019-08-12 4:11 ` Mark H Weaver
2019-07-23 10:03 ` Ludovic Courtès
2019-08-12 7:08 ` Mark H Weaver
2019-08-12 9:01 ` Jan Nieuwenhuizen
2019-08-13 6:42 ` Mark H Weaver
2019-08-13 10:17 ` Jan Nieuwenhuizen
2019-08-14 15:03 ` Marius Bakke
2019-08-14 17:29 ` Marius Bakke
2019-08-14 18:35 ` Mark H Weaver
2019-08-14 18:43 ` Mark H Weaver
2019-08-14 19:56 ` Marius Bakke [this message]
2019-08-14 20:43 ` Mark H Weaver
2019-08-15 19:44 ` Mark H Weaver
2019-08-15 21:19 ` Marius Bakke
2019-08-15 23:16 ` Mark H Weaver
2019-08-15 20:56 ` Mark H Weaver
2019-08-16 7:42 ` Mark H Weaver
2019-08-17 16:49 ` Mark H Weaver
2019-08-16 10:49 ` Ludovic Courtès
2019-08-16 16:59 ` Mark H Weaver
2019-08-17 21:38 ` Ludovic Courtès
2019-08-18 1:17 ` Mark H Weaver
2019-08-18 9:26 ` Ludovic Courtès
2019-08-20 18:40 ` Mark H Weaver
2019-08-21 20:15 ` Mark H Weaver
2019-08-21 21:38 ` Ludovic Courtès
2019-08-21 22:57 ` Mark H Weaver
2019-08-22 10:09 ` Ludovic Courtès
2019-08-24 13:31 ` Ludovic Courtès
2019-08-24 20:34 ` Mark H Weaver
2019-08-26 8:25 ` Ludovic Courtès
2019-08-26 18:36 ` Mark H Weaver
2019-08-27 9:38 ` Ludovic Courtès
2019-08-29 22:28 ` Bengt Richter
2019-08-27 3:58 ` Mark H Weaver
2019-08-27 9:40 ` Ludovic Courtès
2019-08-27 14:27 ` Mark H Weaver
2019-08-27 16:04 ` Ludovic Courtès
2019-08-27 16:46 ` Mark H Weaver
2019-08-28 0:55 ` Mark H Weaver
2019-08-28 22:12 ` Ludovic Courtès
2019-08-29 5:46 ` Ricardo Wurmus
2019-08-29 6:32 ` Ricardo Wurmus
2019-08-29 19:28 ` Mark H Weaver
2019-08-29 23:23 ` Ludovic Courtès
2019-08-30 19:52 ` Mark H Weaver
2019-08-31 12:44 ` Ludovic Courtès
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://guix.gnu.org/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87woffh66h.fsf@devup.no \
--to=mbakke@fastmail.com \
--cc=36747@debbugs.gnu.org \
--cc=mhw@netris.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/guix.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).