Re: Preparing the reduced bootstrap tarballs

Re: Preparing the reduced bootstrap tarballs

[Jeremiah]

> Indeed.  "mes-seed" and "tinycc-seed" are remnants of the past; the only
> things we need are
>
>> What we would need here is something to build the things listed in
>> ‘%bootstrap-inputs’, namely:
>> ‘linux-libre-headers-stripped-4.14.26-i686-linux.tar.xz’ (easy :-)),
>> ‘mescc-tools-seed-XYZ.tar.gz’, and
>> ‘mes-stripped-0.18-0.08f04f5-i686-linux.tar.xz’

That is correct (With mescc-tools-seed adding steps and removing
binaries over time)

> So if you like, please make that change.  There is only one little
> thing: I have no (scripted) recipe to create mescc-tools-seed-XYZ.  But
> wait: I have a great excuse for that...I was too lazy or too sloppy.

I do, in mescc-tools-seed; the script bootstrap.sh when run with the
option "sin" will build the mescc-tools-seed binaries using mescc-tools.
The .M1 files are always generated by cc_x86.s using the C source files.


> The thing is, I used to build mescc-tools-seed, mes-seed and tinycc-seed
> manually from the mes+mescc+tinycc source trees.
I've been building them from the stage0 bootstrap tree (which as you can
see is rather trivial:
https://github.com/oriansj/talk-notes/blob/master/Current%20bootstrap%20map.dot
)


> Jeremiah Orians is
> working to remove any need for mescc-tools-seed (esp. the forward
> dependency on Mes) but I don't think we're there yet.
We have eliminated the forward dependency on Mes for the creation of the
mescc-tools-seed already


> Anyway, I think we/I will have to put some work into scripting
> mescc-tools-seed or otherwise changing the mescc-tools-boot build.
Already done in bash and kaem but not in guix yet (Should be trivial)

> WDYT?
I think we will end up having several versions of mescc-tools-seed; as
each architecture guix supports will end up needing a variant if we plan
on keeping them small. (I also have no idea how to make a multi-arch fat
elf binary)

I am also curious if there is any demand for the stripped versions of
mescc-tools-seed as those binaries are nearly half the size.

>> (do we really need an x86_64 version of this Mes?).
> No, I don't think so.  I added it esp. to get a preview and enable
> future development of pure x86_64 bootstrap; but dependency-wise we
> should be able to drop it!
also, AMD64 does support i386 binaries without issue

- Jeremiah

Re: Preparing the reduced bootstrap tarballs

[Ludovic Courtès]

Jeremiah@pdp10.guru skribis:

>> So if you like, please make that change.  There is only one little
>> thing: I have no (scripted) recipe to create mescc-tools-seed-XYZ.  But
>> wait: I have a great excuse for that...I was too lazy or too sloppy.
>
> I do, in mescc-tools-seed; the script bootstrap.sh when run with the
> option "sin" will build the mescc-tools-seed binaries using mescc-tools.
> The .M1 files are always generated by cc_x86.s using the C source files.

I saw this script but it’s not entirely clear to me how to package the
whole thing.  We don’t have a “stage0” package for instance in Guix, do
we?

>> WDYT?
> I think we will end up having several versions of mescc-tools-seed; as
> each architecture guix supports will end up needing a variant if we plan
> on keeping them small. (I also have no idea how to make a multi-arch fat
> elf binary)

For now let’s focus on x86_64/i686.  :-)

IMO we should change the seeds as rarely as possible because they are
managed “out-of-band” and verifying them is difficult (you need to fetch
the right Guix commit, run “guix build bootstrap-tarballs”, and compare
the result—assuming this is all bit-reproducible.)

The one we’re using today in Guix date back to 2013.

Thanks,
Ludo’.

Re: Preparing the reduced bootstrap tarballs

[Jan Nieuwenhuizen]

[-- Attachment #1: Type: text/plain, Size: 1701 bytes --]

Ludovic Courtès writes:

> Jeremiah@pdp10.guru skribis:
>
>>> So if you like, please make that change.  There is only one little
>>> thing: I have no (scripted) recipe to create mescc-tools-seed-XYZ.  But
>>> wait: I have a great excuse for that...I was too lazy or too sloppy.
>>
>> I do, in mescc-tools-seed; the script bootstrap.sh when run with the
>> option "sin" will build the mescc-tools-seed binaries using mescc-tools.
>> The .M1 files are always generated by cc_x86.s using the C source files.
>
> I saw this script but it’s not entirely clear to me how to package the
> whole thing.  We don’t have a “stage0” package for instance in Guix, do
> we?

I had a look but it seems we need either binary seeds for stage0, or
binary seeds for mescc-tools and I'd rather build them within Guix; so
that's what I did now.

>>> WDYT?
>> I think we will end up having several versions of mescc-tools-seed; as
>> each architecture guix supports will end up needing a variant if we plan
>> on keeping them small. (I also have no idea how to make a multi-arch fat
>> elf binary)
>
> For now let’s focus on x86_64/i686.  :-)
>
> IMO we should change the seeds as rarely as possible because they are
> managed “out-of-band” and verifying them is difficult (you need to fetch
> the right Guix commit, run “guix build bootstrap-tarballs”, and compare
> the result—assuming this is all bit-reproducible.)
>
> The one we’re using today in Guix date back to 2013.

Okay, I am proposing the attached patches, also available on my 
core-updates-next at http://gitlab.com/janneke/guix

Still I hope these won't last for five more years ;-)
janneke.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-bootstrap-Add-mescc-tools-static-mescc-tools-static-.patch --]
[-- Type: text/x-patch, Size: 1884 bytes --]

From 5702107a31f52a615c516084b7a82d9f5e2967e0 Mon Sep 17 00:00:00 2001
From: Jan Nieuwenhuizen <janneke@gnu.org>
Date: Fri, 16 Nov 2018 19:29:55 +0100
Subject: [PATCH 1/3] bootstrap: Add mescc-tools-static,
 mescc-tools-static-tarball.

* gnu/packages/make-bootstrap.scm (%mescc-tools-static,
%mescc-tools-bootstrap-tarball):  New variable.
---
 gnu/packages/make-bootstrap.scm | 15 +++++++++++++++
 1 file changed, 15 insertions(+)

diff --git a/gnu/packages/make-bootstrap.scm b/gnu/packages/make-bootstrap.scm
index 7bfba3c14..36a5f6904 100644
--- a/gnu/packages/make-bootstrap.scm
+++ b/gnu/packages/make-bootstrap.scm
@@ -48,6 +48,7 @@
             %glibc-bootstrap-tarball
             %gcc-bootstrap-tarball
             %guile-bootstrap-tarball
+            %mescc-tools-bootstrap-tarball
             %mes-bootstrap-tarball
             %bootstrap-tarballs
 
@@ -535,6 +536,16 @@ for `sh' in $PATH, and without nscd, and with static NSS modules."
            #t))))
     (inputs `(("gcc" ,%gcc-static)))))
 
+(define %mescc-tools-static
+  ;; A statically linked MesCC Tools for bootstrap.
+  (package
+    (inherit mescc-tools)
+    (name "mescc-tools-static")
+    (arguments
+     (substitute-keyword-arguments (package-arguments mescc-tools)
+       ((#:make-flags flags)
+        `(cons "CC=gcc -static" ,flags))))))
+
 (define %mes-stripped
   ;; The subset of Mes files needed for bootstrap.
   (package
@@ -731,6 +742,10 @@ for `sh' in $PATH, and without nscd, and with static NSS modules."
   ;; A tarball with the statically-linked, relocatable Guile.
   (tarball-package %guile-static-stripped))
 
+(define %mescc-tools-bootstrap-tarball
+  ;; A tarball with MesCC binary seed.
+  (tarball-package %mescc-tools-static))
+
 (define %mes-bootstrap-tarball
   ;; A tarball with Mes ASCII Seed and binary Mes C Library.
   (tarball-package %mes-stripped))
-- 
2.19.1


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #3: 0002-bootstrap-Add-bootstrap-mescc-tools.patch --]
[-- Type: text/x-patch, Size: 3163 bytes --]

From cc3782c5f9d2c606a265be0dfdca44d79ec25622 Mon Sep 17 00:00:00 2001
From: Jan Nieuwenhuizen <janneke@gnu.org>
Date: Fri, 16 Nov 2018 21:30:00 +0100
Subject: [PATCH 2/3] bootstrap: Add %bootstrap-mescc-tools.

Built with
    5702107a31f52a615c516084b7a82d9f5e2967e0 bootstrap: Add mescc-tools-static, mescc-tools-static-tarball.

* gnu/packages/bootstrap.scm (%bootstrap-mescc-tools): New variable.
---
 gnu/packages/bootstrap.scm | 49 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 49 insertions(+)

diff --git a/gnu/packages/bootstrap.scm b/gnu/packages/bootstrap.scm
index 7e6200c57..1075325dc 100644
--- a/gnu/packages/bootstrap.scm
+++ b/gnu/packages/bootstrap.scm
@@ -49,6 +49,7 @@
             %bootstrap-gcc
             %bootstrap-glibc
             %bootstrap-inputs
+            %bootstrap-mescc-tools
             %bootstrap-mes
             %mescc-tools-seed
             %srfi-43))
@@ -615,6 +616,54 @@ exec ~a/bin/.gcc-wrapped -B~a/lib \
     (home-page #f)
     (license gpl3+)))
 
+(define %bootstrap-mescc-tools
+  ;; The initial MesCC tools.  Uses binaries from a tarball typically built by
+  ;; %MESCC-TOOLS-BOOTSTRAP-TARBALL.
+  (package
+    (name "bootstrap-mescc-tools")
+    (version "0.5.2")
+    (source #f)
+    (build-system trivial-build-system)
+    (arguments
+     `(#:guile ,%bootstrap-guile
+       #:modules ((guix build utils))
+       #:builder
+       (begin
+         (use-modules (guix build utils)
+                      (ice-9 popen))
+         (let ((out     (assoc-ref %outputs "out"))
+               (tar     (assoc-ref %build-inputs "tar"))
+               (xz      (assoc-ref %build-inputs "xz"))
+               (tarball (assoc-ref %build-inputs "tarball")))
+
+           (mkdir out)
+           (copy-file tarball "binaries.tar.xz")
+           (invoke xz "-d" "binaries.tar.xz")
+           (let ((builddir (getcwd))
+                 (bindir   (string-append out "/bin")))
+             (with-directory-excursion out
+               (invoke tar "xvf"
+                       (string-append builddir "/binaries.tar"))))))))
+    (inputs
+     `(("tar" ,(search-bootstrap-binary "tar" (%current-system)))
+       ("xz"  ,(search-bootstrap-binary "xz" (%current-system)))
+       ("tarball" ,(bootstrap-origin
+                    (origin
+                      (method url-fetch)
+                      (uri (string-append
+                            "http://lilypond.org/janneke/mes/"
+                            (match (%current-system)
+                              ((or "i686-linux" "x86_64-linux")
+                               "mescc-tools-static-0.5.2-0.bb062b0-i686-linux.tar.xz"))))
+                      (sha256
+                       (match (%current-system)
+                         ((or "i686-linux" "x86_64-linux")
+                          (base32 "1qcqz5lfl2blbfvh5ccg08pcxfhqrmlrsh6cyaxvm0hxsr1rvlm1")))))))))
+    (synopsis "Bootstrap binaries of MesCC Tools")
+    (description synopsis)
+    (home-page #f)
+    (license gpl3+)))
+
 (define %bootstrap-mes
   ;; The initial Mes.  Uses binaries from a tarball typically built by
   ;; %MES-BOOTSTRAP-TARBALL.
-- 
2.19.1


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #4: 0003-bootstrap-Replace-mescc-tools-seed-with-bootstrap-me.patch --]
[-- Type: text/x-patch, Size: 6662 bytes --]

From daa9ff9463917687aabed8330eacb304a7e4deae Mon Sep 17 00:00:00 2001
From: Jan Nieuwenhuizen <janneke@gnu.org>
Date: Fri, 16 Nov 2018 21:30:28 +0100
Subject: [PATCH 3/3] bootstrap: Replace %mescc-tools-seed with
 %bootstrap-mescc-tools.

* gnu/packages/bootstrap.scm (%mescc-tools-seed): Remove.
(%bootstrap-inputs): Replace %mescc-tools-seed with %bootstrap-mescc-tools.
* gnu/packages/commencement.scm (mes-boot): Likewise.
(tcc-boot0): Likewise.
(mescc-tools-boot): Remove.
---
 gnu/packages/bootstrap.scm    | 14 +------
 gnu/packages/commencement.scm | 79 ++---------------------------------
 2 files changed, 4 insertions(+), 89 deletions(-)

diff --git a/gnu/packages/bootstrap.scm b/gnu/packages/bootstrap.scm
index 1075325dc..2eac08cb4 100644
--- a/gnu/packages/bootstrap.scm
+++ b/gnu/packages/bootstrap.scm
@@ -51,7 +51,6 @@
             %bootstrap-inputs
             %bootstrap-mescc-tools
             %bootstrap-mes
-            %mescc-tools-seed
             %srfi-43))
 
 ;;; Commentary:
@@ -716,17 +715,6 @@ exec ~a/bin/.gcc-wrapped -B~a/lib \
     (home-page #f)
     (license gpl3+)))
 
-(define %mescc-tools-seed ; todo: add tarballs to alpha.gnu.org/pub/mes/bootstrap/
-  (let ((commit "dc4e20e74924a5c80a2b7a77b4d7b927234fa71c"))
-    (origin
-      (method url-fetch)
-      (uri (string-append "https://gitlab.com/janneke/mescc-tools-seed"
-                          "/-/archive/" commit
-                          "/mescc-tools-seed-" commit ".tar.gz"))
-      (sha256
-       (base32
-        "1lj7df73vxanmffmiwkhcn83r7yd9n8568nkki06bqq5zg526nyz")))))
-
 (define %srfi-43
   (origin
     (method url-fetch)
@@ -742,7 +730,7 @@ exec ~a/bin/.gcc-wrapped -B~a/lib \
   `(,@(match (%current-system)
         ((or "i686-linux" "x86_64-linux")
          `(("linux-libre-headers" ,%bootstrap-linux-libre-headers)
-           ("mescc-tools-seed" ,%mescc-tools-seed)
+           ("bootstrap-mescc-tools" ,%bootstrap-mescc-tools)
            ("mes" ,%bootstrap-mes)
            ("srfi-43" ,%srfi-43 )))
         (_
diff --git a/gnu/packages/commencement.scm b/gnu/packages/commencement.scm
index 4def178a8..43373b591 100644
--- a/gnu/packages/commencement.scm
+++ b/gnu/packages/commencement.scm
@@ -106,79 +106,6 @@
       (native-inputs '())
       (propagated-inputs '()))))
 
-(define mescc-tools-boot
-  (package-with-bootstrap-guile
-   (package
-     (inherit mescc-tools)
-     (name "mescc-tools-boot")
-     (version "0.5.2")
-     (source (origin
-               (method url-fetch)
-               (uri (string-append
-                     "http://git.savannah.nongnu.org/cgit/mescc-tools.git/snapshot/"
-                     name "-Release_" version
-                     ".tar.gz"))
-               (patches (search-patches "mescc-tools-boot.patch"))
-               (file-name (string-append "mescc-tools" "-" version ".tar.gz"))
-               (sha256
-                (base32
-                 "01x7bhmgwyf6mc2g1hcvibhps98nllacqm4f0j5l51b1mbi18pc2"))))
-     (inputs '())
-     (propagated-inputs '())
-     (native-inputs
-      `(("mescc-tools-seed" ,%mescc-tools-seed)
-        ("mes-source" ,(package-source mes-boot0))
-
-        ("bootstrap-mes" ,%bootstrap-mes)
-        ("coreutils" ,%bootstrap-coreutils&co)))
-     (build-system gnu-build-system)
-     (arguments
-      `(#:implicit-inputs? #f
-        #:guile ,%bootstrap-guile
-        #:strip-binaries? #f   ; binutil's strip b0rkes MesCC/M1/hex2 binaries
-        #:phases
-        (modify-phases %standard-phases
-          (add-after 'unpack 'unpack-seeds
-            (lambda* (#:key outputs #:allow-other-keys)
-              (let* ((coreutils (assoc-ref %build-inputs "coreutils"))
-                     (mescc-tools-seed (assoc-ref %build-inputs "mescc-tools-seed"))
-                     (mes-source (assoc-ref %build-inputs "mes-source"))
-                     (out (assoc-ref %outputs "out")))
-                (with-directory-excursion ".."
-                  (and
-                   (mkdir-p "mescc-tools-seed")
-                   (invoke "tar" "--strip=1" "-C" "mescc-tools-seed"
-                           "-xvf" mescc-tools-seed)
-                   (mkdir-p "mes-source")
-                   (invoke "tar" "--strip=1" "-C" "mes-source"
-                           "-xvf" mes-source)
-                  #t)))))
-          (replace 'configure
-            (lambda* (#:key outputs #:allow-other-keys)
-              (let ((coreutils (assoc-ref %build-inputs "coreutils"))
-                    (bootstrap-mes (assoc-ref %build-inputs "bootstrap-mes"))
-                    (out (assoc-ref %outputs "out")))
-                (setenv "PATH" (string-append coreutils "/bin"
-                                              ":" "../mescc-tools-seed"))
-                (format (current-error-port) "PATH=~s\n" (getenv "PATH"))
-                (setenv "PREFIX" out)
-                (setenv "MES_PREFIX" "../mes-source")
-                (setenv "MESCC_TOOLS_SEED" "../mescc-tools-seed")
-                (setenv "MES_SEED" (string-append bootstrap-mes "/lib"))
-                #t)))
-          (replace 'build
-            (lambda _
-              (invoke "sh" "build.sh")))
-          (replace 'check
-            (lambda _
-              ;; bootstrap build.sh lacks exec_enable, get_machine, and
-              ;; kaem_machine
-              ;; (invoke "sh" "check.sh")
-              #t))
-          (replace 'install
-            (lambda _
-              (invoke "sh" "install.sh")))))))))
-
 (define nyacc-boot
   (let ((version "0.86.0")
         (revision "0")
@@ -211,7 +138,7 @@
      (inputs '())
      (propagated-inputs '())
      (native-inputs
-      `(("mescc-tools" ,mescc-tools-boot)
+      `(("mescc-tools" ,%bootstrap-mescc-tools)
         ("nyacc-source" ,(package-source nyacc-boot))
 
         ("coreutils" , %bootstrap-coreutils&co)
@@ -284,7 +211,7 @@
                (setenv "MES_ARENA" "100000000")
                (setenv "DIFF" "sh scripts/diff.scm")
                ;; fail fast tests
-               ;; (invoke "sh" "-x" "build-aux/test.sh" "scaffold/tests/t")
+               (invoke "sh" "-x" "build-aux/test.sh" "scaffold/tests/t")
                ;; (invoke "sh" "-x" "build-aux/test.sh" "scaffold/tests/63-struct-cell")
                (invoke "sh" "check.sh"))))
           (replace 'install
@@ -332,7 +259,7 @@
        (propagated-inputs '())
        (native-inputs
         `(("mes" ,mes-boot)
-          ("mescc-tools" ,mescc-tools-boot)
+          ("mescc-tools" ,%bootstrap-mescc-tools)
           ("nyacc-source" ,(package-source nyacc-boot))
 
           ("coreutils" , %bootstrap-coreutils&co)
-- 
2.19.1


[-- Attachment #5: Type: text/plain, Size: 152 bytes --]


-- 
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com

Re: Preparing the reduced bootstrap tarballs

[Ludovic Courtès]

Hello,

Jan Nieuwenhuizen <janneke@gnu.org> skribis:

>> IMO we should change the seeds as rarely as possible because they are
>> managed “out-of-band” and verifying them is difficult (you need to fetch
>> the right Guix commit, run “guix build bootstrap-tarballs”, and compare
>> the result—assuming this is all bit-reproducible.)
>>
>> The one we’re using today in Guix date back to 2013.
>
> Okay, I am proposing the attached patches, also available on my 
> core-updates-next at http://gitlab.com/janneke/guix
>
> Still I hope these won't last for five more years ;-)

Well I hope these will last for at least a couple of years, or at least
this is what we should aim for.

Replacing bootstrap binaries is not anecdotal as I wrote because these
are “binary blobs” that need a special procedure to verify them; that
they are smaller blobs is of course a crucial improvement, but it
doesn’t change the fact that one may want to verify them.

> From 5702107a31f52a615c516084b7a82d9f5e2967e0 Mon Sep 17 00:00:00 2001
> From: Jan Nieuwenhuizen <janneke@gnu.org>
> Date: Fri, 16 Nov 2018 19:29:55 +0100
> Subject: [PATCH 1/3] bootstrap: Add mescc-tools-static,
>  mescc-tools-static-tarball.
>
> * gnu/packages/make-bootstrap.scm (%mescc-tools-static,
> %mescc-tools-bootstrap-tarball):  New variable.

Neat; I’m glad it was as simple as this.  :-)

As discussed yesterday on IRC, please also make sure make-bootstrap.scm
does not refer to (gnu packages bootstrap) at all since that would be
like running in circles.

As discussed just now on IRC, we probably also need #:system
"i686-linux" in the appropriate places such that “guix build
bootstrap-tarballs” gives the same result on i686 and on x86_64.

> From cc3782c5f9d2c606a265be0dfdca44d79ec25622 Mon Sep 17 00:00:00 2001
> From: Jan Nieuwenhuizen <janneke@gnu.org>
> Date: Fri, 16 Nov 2018 21:30:00 +0100
> Subject: [PATCH 2/3] bootstrap: Add %bootstrap-mescc-tools.
>
> Built with
>     5702107a31f52a615c516084b7a82d9f5e2967e0 bootstrap: Add mescc-tools-static, mescc-tools-static-tarball.
>
> * gnu/packages/bootstrap.scm (%bootstrap-mescc-tools): New variable.

OK.

If you want, before pushing this patch, you can first push the one
above.  From there I (and possibly others!) will run “guix build
bootstrap-tarballs” and make sure we get the same result as you did.

> From daa9ff9463917687aabed8330eacb304a7e4deae Mon Sep 17 00:00:00 2001
> From: Jan Nieuwenhuizen <janneke@gnu.org>
> Date: Fri, 16 Nov 2018 21:30:28 +0100
> Subject: [PATCH 3/3] bootstrap: Replace %mescc-tools-seed with
>  %bootstrap-mescc-tools.
>
> * gnu/packages/bootstrap.scm (%mescc-tools-seed): Remove.
> (%bootstrap-inputs): Replace %mescc-tools-seed with %bootstrap-mescc-tools.
> * gnu/packages/commencement.scm (mes-boot): Likewise.

Also: (mescc-tools-boot): Remove.

> (tcc-boot0): Likewise.
> (mescc-tools-boot): Remove.

I suppose this is not directly related to the other changes, but anyway,
LGTM.

Thanks for the quick reply!

Ludo’.

Re: Preparing the reduced bootstrap tarballs

[Jan Nieuwenhuizen]

Ludovic Courtès writes:

Hi!

>> Still I hope these won't last for five more years ;-)
>
> Well I hope these will last for at least a couple of years, or at least
> this is what we should aim for.

Yes, we should aim for that while creating them.  I'm possibly a bit too
eager to replace the reduced binary seed bootstrap by a scheme only
bootstrap and reduce all of our seeds even further.

> Replacing bootstrap binaries is not anecdotal as I wrote because these
> are “binary blobs” that need a special procedure to verify them; that
> they are smaller blobs is of course a crucial improvement, but it
> doesn’t change the fact that one may want to verify them.

Fully agree.

> As discussed yesterday on IRC, please also make sure make-bootstrap.scm
> does not refer to (gnu packages bootstrap) at all since that would be
> like running in circles.

Thank you.  Right, that's fixed now.

> As discussed just now on IRC, we probably also need #:system
> "i686-linux" in the appropriate places such that “guix build
> bootstrap-tarballs” gives the same result on i686 and on x86_64.

I'm looking into that right now.

>> From cc3782c5f9d2c606a265be0dfdca44d79ec25622 Mon Sep 17 00:00:00 2001
>> From: Jan Nieuwenhuizen <janneke@gnu.org>
>> Date: Fri, 16 Nov 2018 21:30:00 +0100
>> Subject: [PATCH 2/3] bootstrap: Add %bootstrap-mescc-tools.
>>
>> Built with
>>     5702107a31f52a615c516084b7a82d9f5e2967e0 bootstrap: Add mescc-tools-static, mescc-tools-static-tarball.
>>
>> * gnu/packages/bootstrap.scm (%bootstrap-mescc-tools): New variable.
>
> OK.
>
> If you want, before pushing this patch, you can first push the one
> above.  From there I (and possibly others!) will run “guix build
> bootstrap-tarballs” and make sure we get the same result as you did.

Great, done.  I missed

    5b01b6034aeab32a5011c5757f18bd9772d3497d python: Honor '--cores=...' in tests.

happening yesterday; so pushed as

    fae3c0d953a7b0b552b4ce654ed9b5a51186f491
    bootstrap: Add mescc-tools-static, mescc-tools-static-tarball.

Note you'll also need something like these three extra commits

--8<---------------cut here---------------start------------->8---
2d8f7e2ac bootstrap: Update %bootstrap-tarballs.
57fbf077a bootstrap: Replace %mescc-tools-seed with %bootstrap-mescc-tools.
706574ee4 bootstrap: Add %bootstrap-mescc-tools.
--8<---------------cut here---------------end--------------->8---

from core-updates-next @ https://gitlab.com/janneke/guix to be able to
build `guix --system=i686-linux bootstrap-tarballs'.  Until then, the
recipe is

--8<---------------cut here---------------start------------->8---
./pre-inst-env guix build --system=i686-linux mes-stripped-tarball mescc-tools-static-tarball
--8<---------------cut here---------------end--------------->8---

Rebuilding that right now, let's compare what we get.

Thanks for all your input!
janneke

-- 
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com

Re: Preparing the reduced bootstrap tarballs

[Jan Nieuwenhuizen]

Jan Nieuwenhuizen writes:

>> If you want, before pushing this patch, you can first push the one
>> above.  From there I (and possibly others!) will run “guix build
>> bootstrap-tarballs” and make sure we get the same result as you did.
>
> Great, done.  I missed
>
>     5b01b6034aeab32a5011c5757f18bd9772d3497d python: Honor '--cores=...' in tests.
>
> happening yesterday; so pushed as
>
>     fae3c0d953a7b0b552b4ce654ed9b5a51186f491
>     bootstrap: Add mescc-tools-static, mescc-tools-static-tarball.
>
> Note you'll also need something like these three extra commits
>
> 2d8f7e2ac bootstrap: Update %bootstrap-tarballs.
> 57fbf077a bootstrap: Replace %mescc-tools-seed with %bootstrap-mescc-tools.
> 706574ee4 bootstrap: Add %bootstrap-mescc-tools.
>
>
> from core-updates-next @ https://gitlab.com/janneke/guix to be able to
> build `guix --system=i686-linux bootstrap-tarballs'.  Until then, the
> recipe is
>
> ./pre-inst-env guix build --system=i686-linux mes-stripped-tarball mescc-tools-static-tarball
>
> Rebuilding that right now, let's compare what we get.

Just to give you a heads up: left to build this last night to find that
the python patch does not build for me.  I reverted the python patch on
my personal branch and start a new build.

Not sure what to do about this, YMMV but this is what I get

--8<---------------cut here---------------start------------->8---
building /gnu/store/k3h2iyv4b36a3d5ji7azgi8bcd8czyrz-python-minimal-3.7.0.drv...
[..]
-l and -j don't go together!
make: *** [Makefile:878: test] Error 2

Test suite failed, dumping logs.
Backtrace:
           4 (primitive-load "/gnu/store/9lz3rpgyrm1y97g6k9qg0f7a1h6?")
In ice-9/eval.scm:
   191:35  3 (_ _)
In srfi/srfi-1.scm:
   863:16  2 (every1 #<procedure 80fce20 at /gnu/store/4nhwwas62jab?> ?)
In /gnu/store/4nhwwas62jabgi8lmhimpg4v3fk2ynk1-module-import/guix/build/gnu-build-system.scm:
   799:28  1 (_ _)
    369:6  0 (check #:target _ #:make-flags _ #:tests? _ # _ # _ # _)

/gnu/store/4nhwwas62jabgi8lmhimpg4v3fk2ynk1-module-import/guix/build/gnu-build-system.scm:369:6: In procedure check:
Throw to key `srfi-34' with args `(#<condition &invoke-error [program: "make" arguments: ("test" "-j" "16" "EXTRATESTOPTS=-j16") exit-status: 2 term-signal: #f stop-signal: #f] 822ea20>)'.
builder for `/gnu/store/llvsk2pszk1cc7qwf6azi9877nhks59f-python2-2.7.15.drv' failed with exit code 1
build of /gnu/store/llvsk2pszk1cc7qwf6azi9877nhks59f-python2-2.7.15.drv failed
--8<---------------cut here---------------end--------------->8---

janneke

-- 
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com

Preparing the reduced bootstrap tarballs, take 2

[Jan Nieuwenhuizen]

[-- Attachment #1: Type: text/plain, Size: 1168 bytes --]

Hello!

Meanwhile, I have updated core-updates-next to include

    d9ed0a463 bootstrap: Update %bootstrap-tarballs.
    4fd4efc81 bootstrap: Replace %mescc-tools-seed with %bootstrap-mescc-tools.
    562c37c7b bootstrap: Update %bootstrap-mes.
    b00a95be2 bootstrap: Add %bootstrap-mescc-tools.

With these, I can build

   ./pre-inst-env guix build hello
   ./pre-inst-env guix build --system=i686-linux hello
   ./pre-inst-env guix build --system=i686-linux bootstrap-tarballs

If these reproduce, we're getting real close.  However...

On my core-updates-next at gitlab I also have (see attached)

    5c2209ad4 bootstrap: Force i686-linux for bootstrap-tarballs.

which tries to support

   ./pre-inst-env guix build bootstrap-tarballs

which fails when attempting to build gcc-cross-sans-libc-i686-unknown-linux-gnu-5.5.0

    /tmp/guix-build-gcc-cross-sans-libc-i686-unknown-linux-gnu-5.5.0.drv-0/build/./gcc/cc1: error while loading shared libraries: libisl.so.15: cannot open shared object file: No such file or directory

I do not understand the dependency chain of why exactly this is being
rebuilt and what %current-system is, or why it fails.

janneke


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-bootstrap-Force-i686-linux-for-bootstrap-tarballs.patch --]
[-- Type: text/x-patch, Size: 2406 bytes --]

From 5c2209ad4edd198f600dbf1fef58d434cd3bd92a Mon Sep 17 00:00:00 2001
From: Jan Nieuwenhuizen <janneke@gnu.org>
Date: Tue, 20 Nov 2018 08:12:09 +0100
Subject: [PATCH] bootstrap: Force i686-linux for bootstrap-tarballs.

* gnu/packages/make-bootstrap.scm (%mescc-tools-static): Force i686-linux for
bootstrap-tarballs.
(%mes-minimal): Likewise.
* gnu/packages/commencement.scm (mes-boot): Support strict i686-linux build on
x86_64.
---
 gnu/packages/commencement.scm   |  2 ++
 gnu/packages/make-bootstrap.scm | 10 ++++++----
 2 files changed, 8 insertions(+), 4 deletions(-)

diff --git a/gnu/packages/commencement.scm b/gnu/packages/commencement.scm
index ed8a429c2..3d3f8e665 100644
--- a/gnu/packages/commencement.scm
+++ b/gnu/packages/commencement.scm
@@ -174,6 +174,8 @@
                 (symlink (string-append "../nyacc-source/module") "nyacc")
                 (setenv "GUILE_LOAD_PATH" "nyacc")
                 (setenv "GUILE_TOOLS" "true") ; no tools in bootstrap-guile
+                (substitute* "configure.sh"
+                  (("^arch=.*") "arch=i686\n"))
                 (invoke "bash" "configure.sh"
                         (string-append "--prefix=" out))
                 (setenv "MES" "src/mes")
diff --git a/gnu/packages/make-bootstrap.scm b/gnu/packages/make-bootstrap.scm
index fa3914cde..d75289b45 100644
--- a/gnu/packages/make-bootstrap.scm
+++ b/gnu/packages/make-bootstrap.scm
@@ -542,9 +542,10 @@ for `sh' in $PATH, and without nscd, and with static NSS modules."
     (inherit mescc-tools)
     (name "mescc-tools-static")
     (arguments
-     (substitute-keyword-arguments (package-arguments mescc-tools)
-       ((#:make-flags flags)
-        `(cons "CC=gcc -static" ,flags))))))
+     `(#:system "i686-linux"
+       ,@(substitute-keyword-arguments (package-arguments mescc-tools)
+           ((#:make-flags flags)
+            `(cons "CC=gcc -static" ,flags)))))))
 
 (define %mes-minimal
   ;; A minimal Mes without documentation dependencies, for bootstrap.
@@ -562,7 +563,8 @@ for `sh' in $PATH, and without nscd, and with static NSS modules."
                  ("i686-linux-gcc" ,(cross-gcc triplet)))
                '())))
       (arguments
-       `(#:strip-binaries? #f
+       `(#:system "i686-linux"
+         #:strip-binaries? #f
          #:phases
          (modify-phases %standard-phases
            (add-before 'configure 'optional-dot
-- 
2.19.1


[-- Attachment #3: Type: text/plain, Size: 152 bytes --]


-- 
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com

Preparing the reduced bootstrap tarballs, take 3

[Jan Nieuwenhuizen]

[-- Attachment #1: Type: text/plain, Size: 3383 bytes --]

Jan Nieuwenhuizen writes:

Hello again!

The last problem left for me before we can verify the new bootstrap
tarballs and move them to ftp.gnu.org:guix was for

   ./pre-inst-env guix bootstrap-tarballs

to succeed.  We needed to force #:system "i686-linux" for
mescc-tools-static and mes-minimal.  This triggered a build of
gcc-cross-sans-libc-i686-unknown-linux-gnu, that did not succeed.

I found that the mes-minimal recipe used #:system "i686-linux" and on
x86_64-linux still depended on a i686-linux cross compiler.  With that
removed, I now have for you the tarballs as I produced them.

I have updated core-updates-next with this fix, a documentation update
and to also include Marius' mes bootstrap cleanups from his wip-gcc7
branch.

So, finally for core-updates-next 4ae7dc7b9af64794081b1913740b97acd89c91bc
here are my lucky numbers

--8<---------------cut here---------------start------------->8---
08:21:04 janneke@dundal:~/src/guix-boot 
$ for i in $(./pre-inst-env guix build --system=i686-linux bootstrap-tarballs)/*; do echo $i;guix hash $i; done
/gnu/store/i8rrdivr5ab56kqwbasigp41wqpj1pyp-bootstrap-tarballs-0/guile-static-stripped-2.2.4-i686-linux.tar.xz
17p28xv0y1q729idvf2k1dpcfd469akxm8lnf7gd5a7hmnrb28l5
/gnu/store/i8rrdivr5ab56kqwbasigp41wqpj1pyp-bootstrap-tarballs-0/linux-libre-headers-stripped-4.14.67-i686-linux.tar.xz
0sm2z9x4wk45bh6qfs94p0w1d6hsy6dqx9sw38qsqbvxwa1qzk8s
/gnu/store/i8rrdivr5ab56kqwbasigp41wqpj1pyp-bootstrap-tarballs-0/mescc-tools-static-0.5.2-0.bb062b0-i686-linux.tar.xz
11lniw0vg61kmyhvnwkmcnkci9ym6hbmiksiqggd0hkipbq7hvlz
/gnu/store/i8rrdivr5ab56kqwbasigp41wqpj1pyp-bootstrap-tarballs-0/mes-minimal-stripped-0.18-0.08f04f5-i686-linux.tar.xz
0qwpby91hp6afmg5ibdrrk3fw85zxdazfk7rhrdsihsfzqwmfhfx
/gnu/store/i8rrdivr5ab56kqwbasigp41wqpj1pyp-bootstrap-tarballs-0/static-binaries-0-i686-linux.tar.xz
1rs8qh0n6zxmhs0h1b02sydi4vn09qngf3spfpfynkm2n0bxs368
--8<---------------cut here---------------end--------------->8---

--8<---------------cut here---------------start------------->8---
08:21:10 janneke@dundal:~/src/guix-boot 
$ for i in $(./pre-inst-env guix build bootstrap-tarballs)/*; do echo $i;guix hash $i; done
/gnu/store/n8lrszv7qf0rwqhibm57d8jykj253bw2-bootstrap-tarballs-0/guile-static-stripped-2.2.4-x86_64-linux.tar.xz
1285hlbfahp414hy3jjdghxjfgi94aj7y3fw0pz7z27421b7z2f9
/gnu/store/n8lrszv7qf0rwqhibm57d8jykj253bw2-bootstrap-tarballs-0/linux-libre-headers-stripped-4.14.67-x86_64-linux.tar.xz
0sm2z9x4wk45bh6qfs94p0w1d6hsy6dqx9sw38qsqbvxwa1qzk8s
/gnu/store/n8lrszv7qf0rwqhibm57d8jykj253bw2-bootstrap-tarballs-0/mescc-tools-static-0.5.2-0.bb062b0-x86_64-linux.tar.xz
11lniw0vg61kmyhvnwkmcnkci9ym6hbmiksiqggd0hkipbq7hvlz
/gnu/store/n8lrszv7qf0rwqhibm57d8jykj253bw2-bootstrap-tarballs-0/mes-minimal-stripped-0.18-0.08f04f5-x86_64-linux.tar.xz
0qwpby91hp6afmg5ibdrrk3fw85zxdazfk7rhrdsihsfzqwmfhfx
/gnu/store/n8lrszv7qf0rwqhibm57d8jykj253bw2-bootstrap-tarballs-0/static-binaries-0-x86_64-linux.tar.xz
0bqyjdbas67wgfwv4rcmr5a5b31l8kk8z6gsidxay7liih5rp5hn
--8<---------------cut here---------------end--------------->8---

We may want to change the bootstrap downloads for x86_64 in
bootstrap.scm, mescc-tools-static and mes-minimal-stripped and
linux-libre-headers-stripped to use the x86_64-linux variants (currently
we use the i686-linux tarballs); but as you can see their content is
identical.

Happy verifying ;-)

janneke


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-bootstrap-Force-i686-linux-for-bootstrap-tarballs.patch --]
[-- Type: text/x-patch, Size: 2982 bytes --]

From da91723cc5bba433f59ac40ee669d97e4cbc0e45 Mon Sep 17 00:00:00 2001
From: Jan Nieuwenhuizen <janneke@gnu.org>
Date: Tue, 20 Nov 2018 08:12:09 +0100
Subject: [PATCH] bootstrap: Force i686-linux for bootstrap-tarballs.

* gnu/packages/make-bootstrap.scm (%mescc-tools-static): Force i686-linux for
bootstrap-tarballs.
(%mes-minimal): Likewise.  Remove i686-linux cross-compiler dependency.
* gnu/packages/commencement.scm (mes-boot): Support strict i686-linux build on
x86_64.
---
 gnu/packages/commencement.scm   |  2 ++
 gnu/packages/make-bootstrap.scm | 19 +++++++------------
 2 files changed, 9 insertions(+), 12 deletions(-)

diff --git a/gnu/packages/commencement.scm b/gnu/packages/commencement.scm
index ed8a429c23..3d3f8e6650 100644
--- a/gnu/packages/commencement.scm
+++ b/gnu/packages/commencement.scm
@@ -174,6 +174,8 @@
                 (symlink (string-append "../nyacc-source/module") "nyacc")
                 (setenv "GUILE_LOAD_PATH" "nyacc")
                 (setenv "GUILE_TOOLS" "true") ; no tools in bootstrap-guile
+                (substitute* "configure.sh"
+                  (("^arch=.*") "arch=i686\n"))
                 (invoke "bash" "configure.sh"
                         (string-append "--prefix=" out))
                 (setenv "MES" "src/mes")
diff --git a/gnu/packages/make-bootstrap.scm b/gnu/packages/make-bootstrap.scm
index fa3914cde7..761420af67 100644
--- a/gnu/packages/make-bootstrap.scm
+++ b/gnu/packages/make-bootstrap.scm
@@ -542,9 +542,10 @@ for `sh' in $PATH, and without nscd, and with static NSS modules."
     (inherit mescc-tools)
     (name "mescc-tools-static")
     (arguments
-     (substitute-keyword-arguments (package-arguments mescc-tools)
-       ((#:make-flags flags)
-        `(cons "CC=gcc -static" ,flags))))))
+     `(#:system "i686-linux"
+       ,@(substitute-keyword-arguments (package-arguments mescc-tools)
+           ((#:make-flags flags)
+            `(cons "CC=gcc -static" ,flags)))))))
 
 (define %mes-minimal
   ;; A minimal Mes without documentation dependencies, for bootstrap.
@@ -553,16 +554,10 @@ for `sh' in $PATH, and without nscd, and with static NSS modules."
       (inherit mes)
       (name "mes-minimal")
       (native-inputs
-       `(("guile" ,guile-2.2)
-         ,@(if (not (string-prefix? "i686-linux" (or (%current-target-system)
-                                                     (%current-system))))
-               ;; Use cross-compiler rather than #:system "i686-linux" to get
-               ;; MesCC 64 bit .go files installed ready for use with Guile.
-               `(("i686-linux-binutils" ,(cross-binutils triplet))
-                 ("i686-linux-gcc" ,(cross-gcc triplet)))
-               '())))
+       `(("guile" ,guile-2.2)))
       (arguments
-       `(#:strip-binaries? #f
+       `(#:system "i686-linux"
+         #:strip-binaries? #f
          #:phases
          (modify-phases %standard-phases
            (add-before 'configure 'optional-dot
-- 
2.19.1


[-- Attachment #3: Type: text/plain, Size: 152 bytes --]


-- 
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com

Re: Preparing the reduced bootstrap tarballs, take 3

[Ludovic Courtès]

[-- Attachment #1: Type: text/plain, Size: 4425 bytes --]

Hello janneke,

Jan Nieuwenhuizen <janneke@gnu.org> skribis:

> I have updated core-updates-next with this fix, a documentation update
> and to also include Marius' mes bootstrap cleanups from his wip-gcc7
> branch.
>
> So, finally for core-updates-next 4ae7dc7b9af64794081b1913740b97acd89c91bc
> here are my lucky numbers
>
> 08:21:04 janneke@dundal:~/src/guix-boot 
> $ for i in $(./pre-inst-env guix build --system=i686-linux bootstrap-tarballs)/*; do echo $i;guix hash $i; done
> /gnu/store/i8rrdivr5ab56kqwbasigp41wqpj1pyp-bootstrap-tarballs-0/guile-static-stripped-2.2.4-i686-linux.tar.xz
> 17p28xv0y1q729idvf2k1dpcfd469akxm8lnf7gd5a7hmnrb28l5
> /gnu/store/i8rrdivr5ab56kqwbasigp41wqpj1pyp-bootstrap-tarballs-0/linux-libre-headers-stripped-4.14.67-i686-linux.tar.xz
> 0sm2z9x4wk45bh6qfs94p0w1d6hsy6dqx9sw38qsqbvxwa1qzk8s
> /gnu/store/i8rrdivr5ab56kqwbasigp41wqpj1pyp-bootstrap-tarballs-0/mescc-tools-static-0.5.2-0.bb062b0-i686-linux.tar.xz
> 11lniw0vg61kmyhvnwkmcnkci9ym6hbmiksiqggd0hkipbq7hvlz
> /gnu/store/i8rrdivr5ab56kqwbasigp41wqpj1pyp-bootstrap-tarballs-0/mes-minimal-stripped-0.18-0.08f04f5-i686-linux.tar.xz
> 0qwpby91hp6afmg5ibdrrk3fw85zxdazfk7rhrdsihsfzqwmfhfx
> /gnu/store/i8rrdivr5ab56kqwbasigp41wqpj1pyp-bootstrap-tarballs-0/static-binaries-0-i686-linux.tar.xz
> 1rs8qh0n6zxmhs0h1b02sydi4vn09qngf3spfpfynkm2n0bxs368
>
> 08:21:10 janneke@dundal:~/src/guix-boot 
> $ for i in $(./pre-inst-env guix build bootstrap-tarballs)/*; do echo $i;guix hash $i; done
> /gnu/store/n8lrszv7qf0rwqhibm57d8jykj253bw2-bootstrap-tarballs-0/guile-static-stripped-2.2.4-x86_64-linux.tar.xz
> 1285hlbfahp414hy3jjdghxjfgi94aj7y3fw0pz7z27421b7z2f9
> /gnu/store/n8lrszv7qf0rwqhibm57d8jykj253bw2-bootstrap-tarballs-0/linux-libre-headers-stripped-4.14.67-x86_64-linux.tar.xz
> 0sm2z9x4wk45bh6qfs94p0w1d6hsy6dqx9sw38qsqbvxwa1qzk8s
> /gnu/store/n8lrszv7qf0rwqhibm57d8jykj253bw2-bootstrap-tarballs-0/mescc-tools-static-0.5.2-0.bb062b0-x86_64-linux.tar.xz
> 11lniw0vg61kmyhvnwkmcnkci9ym6hbmiksiqggd0hkipbq7hvlz
> /gnu/store/n8lrszv7qf0rwqhibm57d8jykj253bw2-bootstrap-tarballs-0/mes-minimal-stripped-0.18-0.08f04f5-x86_64-linux.tar.xz
> 0qwpby91hp6afmg5ibdrrk3fw85zxdazfk7rhrdsihsfzqwmfhfx
> /gnu/store/n8lrszv7qf0rwqhibm57d8jykj253bw2-bootstrap-tarballs-0/static-binaries-0-x86_64-linux.tar.xz
> 0bqyjdbas67wgfwv4rcmr5a5b31l8kk8z6gsidxay7liih5rp5hn
>
> We may want to change the bootstrap downloads for x86_64 in
> bootstrap.scm, mescc-tools-static and mes-minimal-stripped and
> linux-libre-headers-stripped to use the x86_64-linux variants (currently
> we use the i686-linux tarballs); but as you can see their content is
> identical.
>
> Happy verifying ;-)

So, on x86_64-linux, here’s what I got:

--8<---------------cut here---------------start------------->8---
ludo@ribbon ~/src/guix/+core-updates-next$ ./pre-inst-env guix build bootstrap-tarballs
/gnu/store/n8lrszv7qf0rwqhibm57d8jykj253bw2-bootstrap-tarballs-0
ludo@ribbon ~/src/guix/+core-updates-next$ (cd /gnu/store/n8lrszv7qf0rwqhibm57d8jykj253bw2-bootstrap-tarballs-0; for i in * ; do echo $i && guix hash $i ; done )
guile-static-stripped-2.2.4-x86_64-linux.tar.xz
06ygv0b3dpvy93jji5yl4hdz17w4hiqm64m7ypf6g2vi4nfn9xl9
linux-libre-headers-stripped-4.14.67-x86_64-linux.tar.xz
0sm2z9x4wk45bh6qfs94p0w1d6hsy6dqx9sw38qsqbvxwa1qzk8s
mescc-tools-static-0.5.2-0.bb062b0-x86_64-linux.tar.xz
11lniw0vg61kmyhvnwkmcnkci9ym6hbmiksiqggd0hkipbq7hvlz
mes-minimal-stripped-0.18-0.08f04f5-x86_64-linux.tar.xz
0qwpby91hp6afmg5ibdrrk3fw85zxdazfk7rhrdsihsfzqwmfhfx
static-binaries-0-x86_64-linux.tar.xz
0bqyjdbas67wgfwv4rcmr5a5b31l8kk8z6gsidxay7liih5rp5hn
ludo@ribbon ~/src/guix/+core-updates-next$ git describe
v0.15.0-3121-g4ae7dc7b9a
--8<---------------cut here---------------end--------------->8---

So only ‘guile-static-stripped’ differs from yours.  This may be due to
non-determinism in bindings introduced by psyntax (something we
addressed with <https://bugs.gnu.org/20272>, but maybe there are
remnants around.)  I’ve uploaded mine here:

  <http://web.fdn.fr/~lcourtes/tmp/guile-static-stripped-2.2.4-x86_64-linux.tar.xz>

Could you share yours so we can compare?

Anyway, I’ll run the “-s i686-linux” build because right now I only have
the 64-bit ‘guile-static-stripped’, which is not what we want, silly me…
Should we force #:system for this one as well?

Thanks,
Ludo’.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: Preparing the reduced bootstrap tarballs, take 3

[Jan Nieuwenhuizen]

Ludovic Courtès writes:

Hello Ludo',

> Jan Nieuwenhuizen <janneke@gnu.org> skribis:
>
>> I have updated core-updates-next with this fix, a documentation update
>> and to also include Marius' mes bootstrap cleanups from his wip-gcc7
>> branch.
>>
>> $ for i in $(./pre-inst-env guix build bootstrap-tarballs)/*; do echo $i;guix hash $i; done
>> /gnu/store/n8lrszv7qf0rwqhibm57d8jykj253bw2-bootstrap-tarballs-0/guile-static-stripped-2.2.4-x86_64-linux.tar.xz
>> 1285hlbfahp414hy3jjdghxjfgi94aj7y3fw0pz7z27421b7z2f9
>> /gnu/store/n8lrszv7qf0rwqhibm57d8jykj253bw2-bootstrap-tarballs-0/linux-libre-headers-stripped-4.14.67-x86_64-linux.tar.xz

> So, on x86_64-linux, here’s what I got:
>
> ludo@ribbon ~/src/guix/+core-updates-next$ ./pre-inst-env guix build bootstrap-tarballs
> /gnu/store/n8lrszv7qf0rwqhibm57d8jykj253bw2-bootstrap-tarballs-0
> ludo@ribbon ~/src/guix/+core-updates-next$ (cd /gnu/store/n8lrszv7qf0rwqhibm57d8jykj253bw2-bootstrap-tarballs-0; for i in * ; do echo $i && guix hash $i ; done )
> guile-static-stripped-2.2.4-x86_64-linux.tar.xz
> 06ygv0b3dpvy93jji5yl4hdz17w4hiqm64m7ypf6g2vi4nfn9xl9

> So only ‘guile-static-stripped’ differs from yours.

Ah, beautiful!

> This may be due to
> non-determinism in bindings introduced by psyntax (something we
> addressed with <https://bugs.gnu.org/20272>, but maybe there are
> remnants around.)  I’ve uploaded mine here:
>
>   <http://web.fdn.fr/~lcourtes/tmp/guile-static-stripped-2.2.4-x86_64-linux.tar.xz>
>
> Could you share yours so we can compare?

Mine is here

    http://lilypond.org/janneke/mes/x86_64-linux/guile-static-stripped-2.2.4-x86_64-linux.tar.xz

...so it looks like we're walking a pretty big round , for --rounds=2 ;-)


> Anyway, I’ll run the “-s i686-linux” build because right now I only have
> the 64-bit ‘guile-static-stripped’, which is not what we want, silly me…

Oh!  Eh, is that right?  I wasn't aware...

> Should we force #:system for this one as well?

We could but then should make sure that any go files in x86_64-linux
bootstrap binaries (mes, and in time gash) are i686-linux too.

janneke

-- 
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com

Re: Preparing the reduced bootstrap tarballs, take 3

[Jan Nieuwenhuizen]

Jan Nieuwenhuizen writes:

>>   <http://web.fdn.fr/~lcourtes/tmp/guile-static-stripped-2.2.4-x86_64-linux.tar.xz>
>>
>> Could you share yours so we can compare?
>
> Mine is here
>
>     http://lilypond.org/janneke/mes/x86_64-linux/guile-static-stripped-2.2.4-x86_64-linux.tar.xz
>
> ...so it looks like we're walking a pretty big round , for --rounds=2 ;-)

Diffoscope output is huge, here is the file diff list (manually edited)
--8<---------------cut here---------------start------------->8---
18:40:36 janneke@dundal:~/src/ludo [env]
$ PYTHONIOENCODING=utf-8 diffoscope ~/guile-static-stripped-2.2.4-x86_64-linux.tar.xz guile-static-stripped-2.2.4-x86_64-linux.tar.xz |grep .*.go
│ ├── ./lib/guile/2.2/ccache/ice-9/suspendable-ports.go
│ ├── ./lib/guile/2.2/ccache/ice-9/vlist.go
│ ├── ./lib/guile/2.2/ccache/language/cps/effects-analysis.go
│ ├── ./lib/guile/2.2/ccache/language/cps/intmap.go
│ ├── ./lib/guile/2.2/ccache/language/cps/intset.go
│ ├── ./lib/guile/2.2/ccache/language/cps/slot-allocation.go
│ ├── ./lib/guile/2.2/ccache/language/cps/types.go
│ ├── ./lib/guile/2.2/ccache/language/cps/utils.go
│ ├── ./lib/guile/2.2/ccache/srfi/srfi-18.go
│ ├── ./lib/guile/2.2/ccache/srfi/srfi-19.go
│ ├── ./lib/guile/2.2/ccache/srfi/srfi-27.go
--8<---------------cut here---------------end--------------->8---

Not sure if that's useful...hoping the specific names may ring a bell.

Greetings,
janneke

Re: Preparing the reduced bootstrap tarballs, take 3

[Ludovic Courtès]

Hello,

Jan Nieuwenhuizen <janneke@gnu.org> skribis:

> Jan Nieuwenhuizen writes:
>
>>>   <http://web.fdn.fr/~lcourtes/tmp/guile-static-stripped-2.2.4-x86_64-linux.tar.xz>
>>>
>>> Could you share yours so we can compare?
>>
>> Mine is here
>>
>>     http://lilypond.org/janneke/mes/x86_64-linux/guile-static-stripped-2.2.4-x86_64-linux.tar.xz
>>
>> ...so it looks like we're walking a pretty big round , for --rounds=2 ;-)
>
> Diffoscope output is huge, here is the file diff list (manually edited)

Here’s an excerpt of the differences in the .go files:

--8<---------------cut here---------------start------------->8---
$ diffoscope --exclude-directory-metadata janneke ludo
--- janneke
+++ ludo
├── lib
│ ├── guile
│ │ ├── 2.2
│ │ │ ├── ccache
│ │ │ │ ├── language
│ │ │ │ │ ├── tree-il
│ │ │ │ │ │ ├── effects.go
│ │ │ │ │ │ │ ├── /gnu/store/02iklp4swqs0ipxhg5x9b2shmj6b30h1-binutils-2.31.1/bin/readelf --wide --decompress --hex-dump=.rodata {}
│ │ │ │ │ │ │ │ @@ -473,38 +473,38 @@
│ │ │ │ │ │ │ │    0x00016d78 6c2d3137 63356138 34616166 34633833 l-17c5a84aaf4c83
│ │ │ │ │ │ │ │    0x00016d88 30612d31 38620000 27000000 00000000 0a-18b..'.......
│ │ │ │ │ │ │ │    0x00016d98 09000000 00000000 70726f63 2d6e616d ........proc-nam
│ │ │ │ │ │ │ │    0x00016da8 65000000 00000000 27000000 00000000 e.......'.......
│ │ │ │ │ │ │ │    0x00016db8 04000000 00000000 61726773 00000000 ........args....
│ │ │ │ │ │ │ │    0x00016dc8 27000000 00000000 16000000 00000000 '...............
│ │ │ │ │ │ │ │    0x00016dd8 6c2d3638 30623737 35666233 37613436 l-680b775fb37a46
│ │ │ │ │ │ │ │ -  0x00016de8 332d3136 62630000 27000000 00000000 3-16bc..'.......
│ │ │ │ │ │ │ │ +  0x00016de8 332d3166 65300000 27000000 00000000 3-1fe0..'.......
│ │ │ │ │ │ │ │    0x00016df8 16000000 00000000 6c2d3638 30623737 ........l-680b77
│ │ │ │ │ │ │ │ -  0x00016e08 35666233 37613436 332d3136 62640000 5fb37a463-16bd..
│ │ │ │ │ │ │ │ +  0x00016e08 35666233 37613436 332d3166 65310000 5fb37a463-1fe1..
│ │ │ │ │ │ │ │    0x00016e18 27000000 00000000 07000000 00000000 '...............
│ │ │ │ │ │ │ │    0x00016e28 666f726d 616c7300 27000000 00000000 formals.'.......
│ │ │ │ │ │ │ │    0x00016e38 16000000 00000000 6c2d3638 30623737 ........l-680b77
│ │ │ │ │ │ │ │ -  0x00016e48 35666233 37613436 332d3136 62330000 5fb37a463-16b3..
│ │ │ │ │ │ │ │ +  0x00016e48 35666233 37613436 332d3166 64370000 5fb37a463-1fd7..
│ │ │ │ │ │ │ │    0x00016e58 27000000 00000000 16000000 00000000 '...............
│ │ │ │ │ │ │ │    0x00016e68 6c2d3638 30623737 35666233 37613436 l-680b775fb37a46
│ │ │ │ │ │ │ │ -  0x00016e78 332d3136 62340000 27000000 00000000 3-16b4..'.......
│ │ │ │ │ │ │ │ +  0x00016e78 332d3166 64380000 27000000 00000000 3-1fd8..'.......
│ │ │ │ │ │ │ │    0x00016e88 16000000 00000000 6c2d3638 30623737 ........l-680b77
│ │ │ │ │ │ │ │ -  0x00016e98 35666233 37613436 332d3136 62350000 5fb37a463-16b5..
│ │ │ │ │ │ │ │ +  0x00016e98 35666233 37613436 332d3166 64390000 5fb37a463-1fd9..
--8<---------------cut here---------------end--------------->8---

It clearly looks like the psyntax non-deterministic gensym bug described
at <https://bugs.gnu.org/20272>.  I guess we’ll have to give it another
go.

Ludo’.

Re: Preparing the reduced bootstrap tarballs, take 3

[Ludovic Courtès]

Hello,

Jan Nieuwenhuizen <janneke@gnu.org> skribis:

> Jan Nieuwenhuizen writes:
>
>>>   <http://web.fdn.fr/~lcourtes/tmp/guile-static-stripped-2.2.4-x86_64-linux.tar.xz>
>>>
>>> Could you share yours so we can compare?
>>
>> Mine is here
>>
>>     http://lilypond.org/janneke/mes/x86_64-linux/guile-static-stripped-2.2.4-x86_64-linux.tar.xz
>>
>> ...so it looks like we're walking a pretty big round , for --rounds=2 ;-)
>
> Diffoscope output is huge, here is the file diff list (manually edited)

Here’s an excerpt of the differences in the .go files:

--8<---------------cut here---------------start------------->8---
$ diffoscope --exclude-directory-metadata janneke ludo
--- janneke
+++ ludo
├── lib
│ ├── guile
│ │ ├── 2.2
│ │ │ ├── ccache
│ │ │ │ ├── language
│ │ │ │ │ ├── tree-il
│ │ │ │ │ │ ├── effects.go
│ │ │ │ │ │ │ ├── /gnu/store/02iklp4swqs0ipxhg5x9b2shmj6b30h1-binutils-2.31.1/bin/readelf --wide --decompress --hex-dump=.rodata {}
│ │ │ │ │ │ │ │ @@ -473,38 +473,38 @@
│ │ │ │ │ │ │ │    0x00016d78 6c2d3137 63356138 34616166 34633833 l-17c5a84aaf4c83
│ │ │ │ │ │ │ │    0x00016d88 30612d31 38620000 27000000 00000000 0a-18b..'.......
│ │ │ │ │ │ │ │    0x00016d98 09000000 00000000 70726f63 2d6e616d ........proc-nam
│ │ │ │ │ │ │ │    0x00016da8 65000000 00000000 27000000 00000000 e.......'.......
│ │ │ │ │ │ │ │    0x00016db8 04000000 00000000 61726773 00000000 ........args....
│ │ │ │ │ │ │ │    0x00016dc8 27000000 00000000 16000000 00000000 '...............
│ │ │ │ │ │ │ │    0x00016dd8 6c2d3638 30623737 35666233 37613436 l-680b775fb37a46
│ │ │ │ │ │ │ │ -  0x00016de8 332d3136 62630000 27000000 00000000 3-16bc..'.......
│ │ │ │ │ │ │ │ +  0x00016de8 332d3166 65300000 27000000 00000000 3-1fe0..'.......
│ │ │ │ │ │ │ │    0x00016df8 16000000 00000000 6c2d3638 30623737 ........l-680b77
│ │ │ │ │ │ │ │ -  0x00016e08 35666233 37613436 332d3136 62640000 5fb37a463-16bd..
│ │ │ │ │ │ │ │ +  0x00016e08 35666233 37613436 332d3166 65310000 5fb37a463-1fe1..
│ │ │ │ │ │ │ │    0x00016e18 27000000 00000000 07000000 00000000 '...............
│ │ │ │ │ │ │ │    0x00016e28 666f726d 616c7300 27000000 00000000 formals.'.......
│ │ │ │ │ │ │ │    0x00016e38 16000000 00000000 6c2d3638 30623737 ........l-680b77
│ │ │ │ │ │ │ │ -  0x00016e48 35666233 37613436 332d3136 62330000 5fb37a463-16b3..
│ │ │ │ │ │ │ │ +  0x00016e48 35666233 37613436 332d3166 64370000 5fb37a463-1fd7..
│ │ │ │ │ │ │ │    0x00016e58 27000000 00000000 16000000 00000000 '...............
│ │ │ │ │ │ │ │    0x00016e68 6c2d3638 30623737 35666233 37613436 l-680b775fb37a46
│ │ │ │ │ │ │ │ -  0x00016e78 332d3136 62340000 27000000 00000000 3-16b4..'.......
│ │ │ │ │ │ │ │ +  0x00016e78 332d3166 64380000 27000000 00000000 3-1fd8..'.......
│ │ │ │ │ │ │ │    0x00016e88 16000000 00000000 6c2d3638 30623737 ........l-680b77
│ │ │ │ │ │ │ │ -  0x00016e98 35666233 37613436 332d3136 62350000 5fb37a463-16b5..
│ │ │ │ │ │ │ │ +  0x00016e98 35666233 37613436 332d3166 64390000 5fb37a463-1fd9..
--8<---------------cut here---------------end--------------->8---

It clearly looks like the psyntax non-deterministic gensym bug described
at <https://bugs.gnu.org/20272>.  I guess we’ll have to give it another
go.

Ludo’.

Re: Preparing the reduced bootstrap tarballs, take 3

[Ludovic Courtès]

[-- Attachment #1: Type: text/plain, Size: 1794 bytes --]

Hello!

ludo@gnu.org (Ludovic Courtès) skribis:

> So, on x86_64-linux, here’s what I got:
>
> ludo@ribbon ~/src/guix/+core-updates-next$ ./pre-inst-env guix build bootstrap-tarballs
> /gnu/store/n8lrszv7qf0rwqhibm57d8jykj253bw2-bootstrap-tarballs-0
> ludo@ribbon ~/src/guix/+core-updates-next$ (cd /gnu/store/n8lrszv7qf0rwqhibm57d8jykj253bw2-bootstrap-tarballs-0; for i in * ; do echo $i && guix hash $i ; done )
> guile-static-stripped-2.2.4-x86_64-linux.tar.xz
> 06ygv0b3dpvy93jji5yl4hdz17w4hiqm64m7ypf6g2vi4nfn9xl9
> linux-libre-headers-stripped-4.14.67-x86_64-linux.tar.xz
> 0sm2z9x4wk45bh6qfs94p0w1d6hsy6dqx9sw38qsqbvxwa1qzk8s
> mescc-tools-static-0.5.2-0.bb062b0-x86_64-linux.tar.xz
> 11lniw0vg61kmyhvnwkmcnkci9ym6hbmiksiqggd0hkipbq7hvlz
> mes-minimal-stripped-0.18-0.08f04f5-x86_64-linux.tar.xz
> 0qwpby91hp6afmg5ibdrrk3fw85zxdazfk7rhrdsihsfzqwmfhfx
> static-binaries-0-x86_64-linux.tar.xz
> 0bqyjdbas67wgfwv4rcmr5a5b31l8kk8z6gsidxay7liih5rp5hn
> ludo@ribbon ~/src/guix/+core-updates-next$ git describe
> v0.15.0-3121-g4ae7dc7b9a

I’ve just uploaded these to
<https://alpha.gnu.org/gnu/guix/bootstrap/i686-linux/20181020>:

  linux-libre-headers-stripped-4.14.67-i686-linux.tar.xz
  linux-libre-headers-stripped-4.14.67-i686-linux.tar.xz.sig
  mescc-tools-static-0.5.2-0.bb062b0-i686-linux.tar.xz
  mescc-tools-static-0.5.2-0.bb062b0-i686-linux.tar.xz.sig
  mes-minimal-stripped-0.18-0.08f04f5-i686-linux.tar.xz
  mes-minimal-stripped-0.18-0.08f04f5-i686-linux.tar.xz.sig

Could you adjust bootstrap.scm to refer to this URL?  Currently I see
bootstrap.scm refers to a different version of
linux-libre-headers-stripped so it should be the only one whose hash
needs to be changed.

Thanks, and let me know if we need anything else!

Ludo’.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: Preparing the reduced bootstrap tarballs, take 3

[Jan Nieuwenhuizen]

[-- Attachment #1: Type: text/plain, Size: 930 bytes --]

Ludovic Courtès writes:

Hi!

> I’ve just uploaded these to
> <https://alpha.gnu.org/gnu/guix/bootstrap/i686-linux/20181020>:
>
>   linux-libre-headers-stripped-4.14.67-i686-linux.tar.xz
>   linux-libre-headers-stripped-4.14.67-i686-linux.tar.xz.sig
>   mescc-tools-static-0.5.2-0.bb062b0-i686-linux.tar.xz
>   mescc-tools-static-0.5.2-0.bb062b0-i686-linux.tar.xz.sig
>   mes-minimal-stripped-0.18-0.08f04f5-i686-linux.tar.xz
>   mes-minimal-stripped-0.18-0.08f04f5-i686-linux.tar.xz.sig

Great!

> Could you adjust bootstrap.scm to refer to this URL?  Currently I see
> bootstrap.scm refers to a different version of
> linux-libre-headers-stripped so it should be the only one whose hash
> needs to be changed.

I don't see that...and the hash matches.

> Thanks, and let me know if we need anything else!

See attached patch.  Push to my gitlab core-updates-next, not to
savannah yet.

janneke


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-bootstrap-Switch-to-official-bootstrap-urls.patch --]
[-- Type: text/x-patch, Size: 3524 bytes --]

From 3cc2f27208de71f9dd0b37593db974b7c1a305f3 Mon Sep 17 00:00:00 2001
From: Jan Nieuwenhuizen <janneke@gnu.org>
Date: Sun, 9 Dec 2018 19:05:45 +0100
Subject: [PATCH] bootstrap: Switch to official bootstrap urls.

* gnu/packages/bootstrap.scm (%bootstrap-linux-libre-headers): Switch to
official bootstrap urls.
(%bootstrap-mescc-tools): Likewise.
(%bootstrap-mes): Likewise.
---
 gnu/packages/bootstrap.scm | 41 +++++++++++++-------------------------
 1 file changed, 14 insertions(+), 27 deletions(-)

diff --git a/gnu/packages/bootstrap.scm b/gnu/packages/bootstrap.scm
index a835cbc206..eac729f785 100644
--- a/gnu/packages/bootstrap.scm
+++ b/gnu/packages/bootstrap.scm
@@ -405,17 +405,10 @@ $out/bin/guile --version~%"
    (lambda (system)
      (origin
        (method url-fetch)
-       (uri (match system
-              ("i686-linux"
-               (string-append
-                "http://lilypond.org/janneke/guix/20181124/"
-                "linux-libre-headers-stripped-4.14.67-i686-linux.tar.xz"))
-              ("x86_64-linux"
-               (string-append
-                "http://lilypond.org/janneke/guix/20181124/"
-                "linux-libre-headers-stripped-4.14.67-x86_64-linux.tar.xz"))
-              (_ (error "linux-libre-headers-bootstrap: system not supported:"
-                        (%current-system)))))
+       (uri (map (cute string-append <>
+                       "/i686-linux/20181020/"
+                       "linux-libre-headers-stripped-4.14.67-i686-linux.tar.xz")
+                 %bootstrap-base-urls))
        (sha256
         (base32
          "0sm2z9x4wk45bh6qfs94p0w1d6hsy6dqx9sw38qsqbvxwa1qzk8s"))))
@@ -656,15 +649,11 @@ exec ~a/bin/.gcc-wrapped -B~a/lib \
         ,(bootstrap-origin
           (origin
             (method url-fetch)
-            (uri (string-append
-                  "http://lilypond.org/janneke/guix/20181124/"
-                  (match (%current-system)
-                    ("i686-linux"
-                     "mescc-tools-static-0.5.2-0.bb062b0-i686-linux.tar.xz")
-                    ("x86_64-linux"
-                     "mescc-tools-static-0.5.2-0.bb062b0-x86_64-linux.tar.xz")
-                    (_ (error "bootstrap-mescc-tools: system not supported:"
-                              (%current-system))))))
+            (uri (map
+                  (cute string-append <>
+                        "/i686-linux/20181020/"
+                        "mescc-tools-static-0.5.2-0.bb062b0-i686-linux.tar.xz")
+                  %bootstrap-base-urls))
             (sha256
              (base32
               "11lniw0vg61kmyhvnwkmcnkci9ym6hbmiksiqggd0hkipbq7hvlz")))))))
@@ -708,13 +697,11 @@ exec ~a/bin/.gcc-wrapped -B~a/lib \
         ,(bootstrap-origin
           (origin
             (method url-fetch)
-            (uri (string-append
-                  "http://lilypond.org/janneke/guix/20181124/"
-                  (match (%current-system)
-                    ("i686-linux"
-                     "mes-minimal-stripped-0.18-0.08f04f5-i686-linux.tar.xz")
-                    ("x86_64-linux"
-                     "mes-minimal-stripped-0.18-0.08f04f5-x86_64-linux.tar.xz"))))
+            (uri (map
+                  (cute string-append <>
+                        "/i686-linux/20181020/"
+                        "mes-minimal-stripped-0.18-0.08f04f5-i686-linux.tar.xz")
+                 %bootstrap-base-urls))
             (sha256
              (match (%current-system)
                ((or "i686-linux" "x86_64-linux")
-- 
2.19.2

Re: Preparing the reduced bootstrap tarballs, take 3

[Ludovic Courtès]

Jan Nieuwenhuizen <janneke@gnu.org> skribis:

> Ludovic Courtès writes:
>
> Hi!
>
>> I’ve just uploaded these to
>> <https://alpha.gnu.org/gnu/guix/bootstrap/i686-linux/20181020>:
>>
>>   linux-libre-headers-stripped-4.14.67-i686-linux.tar.xz
>>   linux-libre-headers-stripped-4.14.67-i686-linux.tar.xz.sig
>>   mescc-tools-static-0.5.2-0.bb062b0-i686-linux.tar.xz
>>   mescc-tools-static-0.5.2-0.bb062b0-i686-linux.tar.xz.sig
>>   mes-minimal-stripped-0.18-0.08f04f5-i686-linux.tar.xz
>>   mes-minimal-stripped-0.18-0.08f04f5-i686-linux.tar.xz.sig
>
> Great!
>
>> Could you adjust bootstrap.scm to refer to this URL?  Currently I see
>> bootstrap.scm refers to a different version of
>> linux-libre-headers-stripped so it should be the only one whose hash
>> needs to be changed.
>
> I don't see that...and the hash matches.

We’ve discussed it in person, and now I think we’re all set!  :-)

Ludo’.

Re: Preparing the reduced bootstrap tarballs, take 3

[Jan Nieuwenhuizen]

Ludovic Courtès writes:

> Jan Nieuwenhuizen <janneke@gnu.org> skribis:
>
>> Ludovic Courtès writes:
>>
>> Hi!
>>
>>> I’ve just uploaded these to
>>> <https://alpha.gnu.org/gnu/guix/bootstrap/i686-linux/20181020>:
>>>
>>>   linux-libre-headers-stripped-4.14.67-i686-linux.tar.xz
>>>   linux-libre-headers-stripped-4.14.67-i686-linux.tar.xz.sig
>>>   mescc-tools-static-0.5.2-0.bb062b0-i686-linux.tar.xz
>>>   mescc-tools-static-0.5.2-0.bb062b0-i686-linux.tar.xz.sig
>>>   mes-minimal-stripped-0.18-0.08f04f5-i686-linux.tar.xz
>>>   mes-minimal-stripped-0.18-0.08f04f5-i686-linux.tar.xz.sig
>>
>> Great!
>>
>>> Could you adjust bootstrap.scm to refer to this URL?  Currently I see
>>> bootstrap.scm refers to a different version of
>>> linux-libre-headers-stripped so it should be the only one whose hash
>>> needs to be changed.
>>
>> I don't see that...and the hash matches.
>
> We’ve discussed it in person, and now I think we’re all set!  :-)

Yes, that helped!  Pushed to core-updates as

da3c6a7f19ef1243af725f63c16c8fd92fde33b4
bootstrap: Switch to official bootstrap urls.

janneke

Re: Preparing the reduced bootstrap tarballs, take 3

[Mark H Weaver]

Ludovic Courtès <ludo@gnu.org> writes:

> Jan Nieuwenhuizen <janneke@gnu.org> skribis:
>
>> Ludovic Courtès writes:
>>
>> Hi!
>>
>>> I’ve just uploaded these to
>>> <https://alpha.gnu.org/gnu/guix/bootstrap/i686-linux/20181020>:
>>>
>>>   linux-libre-headers-stripped-4.14.67-i686-linux.tar.xz
>>>   linux-libre-headers-stripped-4.14.67-i686-linux.tar.xz.sig
>>>   mescc-tools-static-0.5.2-0.bb062b0-i686-linux.tar.xz
>>>   mescc-tools-static-0.5.2-0.bb062b0-i686-linux.tar.xz.sig
>>>   mes-minimal-stripped-0.18-0.08f04f5-i686-linux.tar.xz
>>>   mes-minimal-stripped-0.18-0.08f04f5-i686-linux.tar.xz.sig
>>
>> Great!
>>
>>> Could you adjust bootstrap.scm to refer to this URL?  Currently I see
>>> bootstrap.scm refers to a different version of
>>> linux-libre-headers-stripped so it should be the only one whose hash
>>> needs to be changed.
>>
>> I don't see that...and the hash matches.
>
> We’ve discussed it in person, and now I think we’re all set!  :-)

Does our documentation include instructions on how to reproducibly build
these new bootstrap binaries, to independently verify them?

     Thanks,
       Mark

Re: Preparing the reduced bootstrap tarballs, take 3

[Ludovic Courtès]

Hi Mark,

Mark H Weaver <mhw@netris.org> skribis:

> Ludovic Courtès <ludo@gnu.org> writes:
>
>> Jan Nieuwenhuizen <janneke@gnu.org> skribis:
>>
>>> Ludovic Courtès writes:
>>>
>>> Hi!
>>>
>>>> I’ve just uploaded these to
>>>> <https://alpha.gnu.org/gnu/guix/bootstrap/i686-linux/20181020>:
>>>>
>>>>   linux-libre-headers-stripped-4.14.67-i686-linux.tar.xz
>>>>   linux-libre-headers-stripped-4.14.67-i686-linux.tar.xz.sig
>>>>   mescc-tools-static-0.5.2-0.bb062b0-i686-linux.tar.xz
>>>>   mescc-tools-static-0.5.2-0.bb062b0-i686-linux.tar.xz.sig
>>>>   mes-minimal-stripped-0.18-0.08f04f5-i686-linux.tar.xz
>>>>   mes-minimal-stripped-0.18-0.08f04f5-i686-linux.tar.xz.sig
>>>
>>> Great!
>>>
>>>> Could you adjust bootstrap.scm to refer to this URL?  Currently I see
>>>> bootstrap.scm refers to a different version of
>>>> linux-libre-headers-stripped so it should be the only one whose hash
>>>> needs to be changed.
>>>
>>> I don't see that...and the hash matches.
>>
>> We’ve discussed it in person, and now I think we’re all set!  :-)
>
> Does our documentation include instructions on how to reproducibly build
> these new bootstrap binaries, to independently verify them?

The build procedure remains unchanged (info "(guix) Bootstrapping"):

  guix build bootstrap-tarballs

To verify them, you can do what I described in this thread, which is to
use the same commit as Janneke and myself used, run “guix build
bootstrap-tarballs”, and compare the three tarballs listed above.

Ludo’.

Re: Preparing the reduced bootstrap tarballs, take 3

[Jan Nieuwenhuizen]

Ludovic Courtès writes:

>> Does our documentation include instructions on how to reproducibly build
>> these new bootstrap binaries, to independently verify them?
>
> The build procedure remains unchanged (info "(guix) Bootstrapping"):
>
>   guix build bootstrap-tarballs
>
> To verify them, you can do what I described in this thread, which is to
> use the same commit as Janneke and myself used, run “guix build
> bootstrap-tarballs”, and compare the three tarballs listed above.

Note however that while you're very welcome to do so now, Ricardo and
are working on at least a new mes-minimal-stripped-tarball that I
expect any day now.

janneke

-- 
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com

Re: Preparing the reduced bootstrap tarballs, take 3

[Mark H Weaver]

Ludovic Courtès <ludo@gnu.org> writes:

> Mark H Weaver <mhw@netris.org> skribis:
>
>> Ludovic Courtès <ludo@gnu.org> writes:
>>
>>> Jan Nieuwenhuizen <janneke@gnu.org> skribis:
>>>
>>>> Ludovic Courtès writes:
>>>>
>>>> Hi!
>>>>
>>>>> I’ve just uploaded these to
>>>>> <https://alpha.gnu.org/gnu/guix/bootstrap/i686-linux/20181020>:
>>>>>
>>>>>   linux-libre-headers-stripped-4.14.67-i686-linux.tar.xz
>>>>>   linux-libre-headers-stripped-4.14.67-i686-linux.tar.xz.sig
>>>>>   mescc-tools-static-0.5.2-0.bb062b0-i686-linux.tar.xz
>>>>>   mescc-tools-static-0.5.2-0.bb062b0-i686-linux.tar.xz.sig
>>>>>   mes-minimal-stripped-0.18-0.08f04f5-i686-linux.tar.xz
>>>>>   mes-minimal-stripped-0.18-0.08f04f5-i686-linux.tar.xz.sig
>>>>
>>>> Great!
>>>>
>>>>> Could you adjust bootstrap.scm to refer to this URL?  Currently I see
>>>>> bootstrap.scm refers to a different version of
>>>>> linux-libre-headers-stripped so it should be the only one whose hash
>>>>> needs to be changed.
>>>>
>>>> I don't see that...and the hash matches.
>>>
>>> We’ve discussed it in person, and now I think we’re all set!  :-)
>>
>> Does our documentation include instructions on how to reproducibly build
>> these new bootstrap binaries, to independently verify them?
>
> The build procedure remains unchanged (info "(guix) Bootstrapping"):
>
>   guix build bootstrap-tarballs
>
> To verify them, you can do what I described in this thread, which is to
> use the same commit as Janneke and myself used, run “guix build
> bootstrap-tarballs”, and compare the three tarballs listed above.

What commit did you use?  It would be good to document this, so that
others can independently verify our bootstrap binaries, now or in the
future.

     Thanks,
       Mark

Re: Preparing the reduced bootstrap tarballs, take 3

[Ludovic Courtès]

Mark H Weaver <mhw@netris.org> skribis:

> Ludovic Courtès <ludo@gnu.org> writes:
>
>> Mark H Weaver <mhw@netris.org> skribis:
>>
>>> Ludovic Courtès <ludo@gnu.org> writes:
>>>
>>>> Jan Nieuwenhuizen <janneke@gnu.org> skribis:
>>>>
>>>>> Ludovic Courtès writes:
>>>>>
>>>>> Hi!
>>>>>
>>>>>> I’ve just uploaded these to
>>>>>> <https://alpha.gnu.org/gnu/guix/bootstrap/i686-linux/20181020>:
>>>>>>
>>>>>>   linux-libre-headers-stripped-4.14.67-i686-linux.tar.xz
>>>>>>   linux-libre-headers-stripped-4.14.67-i686-linux.tar.xz.sig
>>>>>>   mescc-tools-static-0.5.2-0.bb062b0-i686-linux.tar.xz
>>>>>>   mescc-tools-static-0.5.2-0.bb062b0-i686-linux.tar.xz.sig
>>>>>>   mes-minimal-stripped-0.18-0.08f04f5-i686-linux.tar.xz
>>>>>>   mes-minimal-stripped-0.18-0.08f04f5-i686-linux.tar.xz.sig
>>>>>
>>>>> Great!
>>>>>
>>>>>> Could you adjust bootstrap.scm to refer to this URL?  Currently I see
>>>>>> bootstrap.scm refers to a different version of
>>>>>> linux-libre-headers-stripped so it should be the only one whose hash
>>>>>> needs to be changed.
>>>>>
>>>>> I don't see that...and the hash matches.
>>>>
>>>> We’ve discussed it in person, and now I think we’re all set!  :-)
>>>
>>> Does our documentation include instructions on how to reproducibly build
>>> these new bootstrap binaries, to independently verify them?
>>
>> The build procedure remains unchanged (info "(guix) Bootstrapping"):
>>
>>   guix build bootstrap-tarballs
>>
>> To verify them, you can do what I described in this thread, which is to
>> use the same commit as Janneke and myself used, run “guix build
>> bootstrap-tarballs”, and compare the three tarballs listed above.
>
> What commit did you use?  It would be good to document this, so that
> others can independently verify our bootstrap binaries, now or in the
> future.

The commit appears in the commit log and in this very thread, no?

Ludo’.

Re: Preparing the reduced bootstrap tarballs, take 3

[Mark H Weaver]

Hi Ludovic,

Ludovic Courtès <ludo@gnu.org> writes:

> Mark H Weaver <mhw@netris.org> skribis:
>
>> What commit did you use?  It would be good to document this, so that
>> others can independently verify our bootstrap binaries, now or in the
>> future.
>
> The commit appears in the commit log and in this very thread, no?

Yes, thanks.  I'm glad to see it in the commit log.  I thought it might
also be worthy of documentation in our source tree, possibly in HACKING,
for the sake of those who wish to independently verify our bootstrap
binaries in the future.  However, it's not something that I feel
strongly about.

     Thanks,
       Mark

Re: Preparing the reduced bootstrap tarballs

[Mark H Weaver]

ludo@gnu.org (Ludovic Courtès) writes:

> Jeremiah@pdp10.guru skribis:
>
>>> So if you like, please make that change.  There is only one little
>>> thing: I have no (scripted) recipe to create mescc-tools-seed-XYZ.  But
>>> wait: I have a great excuse for that...I was too lazy or too sloppy.
>>
>> I do, in mescc-tools-seed; the script bootstrap.sh when run with the
>> option "sin" will build the mescc-tools-seed binaries using mescc-tools.
>> The .M1 files are always generated by cc_x86.s using the C source files.
>
> I saw this script but it’s not entirely clear to me how to package the
> whole thing.  We don’t have a “stage0” package for instance in Guix, do
> we?
>
>>> WDYT?
>> I think we will end up having several versions of mescc-tools-seed; as
>> each architecture guix supports will end up needing a variant if we plan
>> on keeping them small. (I also have no idea how to make a multi-arch fat
>> elf binary)
>
> For now let’s focus on x86_64/i686.  :-)
>
> IMO we should change the seeds as rarely as possible because they are
> managed “out-of-band” and verifying them is difficult (you need to fetch
> the right Guix commit, run “guix build bootstrap-tarballs”, and compare
> the result—assuming this is all bit-reproducible.)
>
> The one we’re using today in Guix date back to 2013.

I think it's important that the new bootstrap-tarballs be
bit-reproducible, such that they can be independently verified by anyone
who wishes to do so.

In particular, *I* would like to independently verify them, on my own
laptops where I have avoided using binary substitutes for a long time,
and which I keep with me at all times.

My hope until now is that when we generated our existing bootstrap
binaries in 2013, Guix was too marginal a project to attract the
attention of hackers who might wish to compromise our bootstrap.  In
2018, as Guix has become more popular, we might well be considered a
worthy target of such efforts.

      Mark