Why do we not cross-compile bootstrap binaries?

Why do we not cross-compile bootstrap binaries?

[Chris Marusich]

[-- Attachment #1: Type: text/plain, Size: 1240 bytes --]

Hi,

We have bootstrap binaries for various platforms.  For example, commit
3b88f3767d9f3ad2cc64173525cd53d429bfe7e7 adds bootstrap binaries for
aarch64-linux.  Judging by the commit message, my guess is that these
bootstrap binaries were built (cross-compiled, I presume) using Guix at
commit 8f8f250bdca917b3ce38aa0902f01b19081859a4 and then checked into
the Git repository.

My understanding is that we could choose, in theory, to cross-compile
the bootstrap binaries from a "base" platform.  This choice would reduce
the number of bootstrap binaries we need to check into the Git
repository.  However, my guess is that we do not want to treat one
particular platform as special: we would like to be able to bootstrap
everything from any supported architecture.  For example, it would be
unfortunate if bootstrapping an armhf-linux machine required the use of
an x86_64-linux machine.  To avoid this "asymmetry" in the bootstrap
path, we choose to check in the built artifacts instead of
cross-compiling them.

Is my understanding correct?  I couldn't find an explicit explanation of
the motivation for this choice by looking at email archives, source
code, or the manual, so I thought I'd ask here.

-- 
Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: Why do we not cross-compile bootstrap binaries?

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 3103 bytes --]

On Thu, Nov 23, 2017 at 04:47:51PM -0800, Chris Marusich wrote:
> Hi,
> 
> We have bootstrap binaries for various platforms.  For example, commit
> 3b88f3767d9f3ad2cc64173525cd53d429bfe7e7 adds bootstrap binaries for
> aarch64-linux.  Judging by the commit message, my guess is that these
> bootstrap binaries were built (cross-compiled, I presume) using Guix at
> commit 8f8f250bdca917b3ce38aa0902f01b19081859a4 and then checked into
> the Git repository.
> 

That's correct. The aarch64 bootstrap binaries were cross compiled from
x86_64 using the commit listed and then added in. Looking back at the
early commit history, I believe x86_64 and i686 were bootstrapped from
Nix and then rebootstrapped later.

> My understanding is that we could choose, in theory, to cross-compile
> the bootstrap binaries from a "base" platform.  This choice would reduce
> the number of bootstrap binaries we need to check into the Git
> repository.  However, my guess is that we do not want to treat one
> particular platform as special: we would like to be able to bootstrap
> everything from any supported architecture.  For example, it would be
> unfortunate if bootstrapping an armhf-linux machine required the use of
> an x86_64-linux machine.  To avoid this "asymmetry" in the bootstrap
> path, we choose to check in the built artifacts instead of
> cross-compiling them.
> 

The diverse double compilation project which aims to allow different
starting points arrive at bit-identical binaries, which would make it so
that it wouldn't matter which architecture the bootstrap binaries were
built on. Additionally, between mes and stage0 there are efforts to
bootstrap from human readable hex code, which would allow literally
bootstrapping from zero, without the need of blessed bootstrap binaries.

> Is my understanding correct?  I couldn't find an explicit explanation of
> the motivation for this choice by looking at email archives, source
> code, or the manual, so I thought I'd ask here.
> 

The way I look at it, one architecture shouldn't have to depend on
another architecture for its packages. Currently we only have support
for haskell on Intel systems, those are the only systems which GHC has
binary releases for and we need a binary bootstrap to build GHC. If we
want to support Haskell on other architectures we can do like Debian
(and I'm sure many other distributions) and cross-compile the bootstrap
GHC, which would make aarch64's haskell dependant on x86_64. Gentoo has
a couple of their own blessed bootstrap binaries for GHC that must've
been cross compiled some time in the past, and those are used when
bootstrapping GHC on a new install.

There was also a talk from FOSDEM 2017 about the size of the bootstrap
binaries once they're unpacked (200+ MB?) and the work up to then with
bootstrapping from a lower base.



-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Why do we not cross-compile bootstrap binaries?

[Ludovic Courtès]

Hello,

In practice, apart from x86_64-linux which was bootstrapped on NixOS, we
always cross-compile to bootstrap new architectures:

  https://www.gnu.org/software/guix/manual/html_node/Porting.html

That was the case for all the non-Intel arches and for GNU/Hurd.

Is that what you meant?

Ludo’.

Re: Why do we not cross-compile bootstrap binaries?

[Chris Marusich]

[-- Attachment #1: Type: text/plain, Size: 809 bytes --]

ludo@gnu.org (Ludovic Courtès) writes:

> Hello,
>
> In practice, apart from x86_64-linux which was bootstrapped on NixOS, we
> always cross-compile to bootstrap new architectures:
>
>   https://www.gnu.org/software/guix/manual/html_node/Porting.html
>
> That was the case for all the non-Intel arches and for GNU/Hurd.
>
> Is that what you meant?

My question was more about: why do we check in bootstrap binaries for
different architectures, instead of checking in bootstrap binaries for
one "base" architecture and then generating the bootstrap binaries for
other architectures from that "base" architecture.  I think Efraim's
answer confirmed my understanding, which is that basically we want to
avoid treating any single platform as special.

Thank you for the reply!

-- 
Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]