Cross-building GuixSD (and maybe using pre-built toolchains)

Cross-building GuixSD (and maybe using pre-built toolchains)

[Paul Boddie]

Hello,

I've been looking at FSF-endorsed Free Software distributions and found 
GuixSD, and I have a few questions about bootstrapping the distribution for 
another architecture.

I was looking at the documentation, which is rather more helpful than most 
distributions when it comes to describing how the actual distribution archives 
are built, and I figured a few things out about guix-daemon. However, I'm 
doing all my work in a chroot - I don't want to "make install" and populate 
various locations on my main system - and this doesn't seem to be entirely 
compatible with the recommended way the daemon is used. Specifically, the 
daemon does not seem to be able to build anything unless I specify the --
disable-chroot flag.

Are there any recommended methods of running guix-daemon in a chroot and have 
it create new chroots, or do I have to use some kind of virtualisation or 
container technology? Is any kind of fakeroot/fakechroot mechanism supported?

One thing that looks very promising is the ability to cross-bootstrap the 
system, and I did manage to get that started for a new architecture (mipsel) 
by performing some modifications suggested in the documentation [1], running 
guix-daemon with the --disable-chroot flag, and then running this:

guix build --target=mipsel-linux-gnu bootstrap-tarballs

However, not wanting to leave my computer switched on all night, I stopped the 
build after a while. I can understand the need to bootstrap things like 
toolchains, but given that I am running Debian which has cross-toolchains for 
mipsel, I wondered if I could short-circuit this process by employing pre-
built toolchains. Would this be possible?

Finally, it seems to be the case that the workflow involves building a 
bootstrapped minimal system and then natively building packages. Or have I 
misunderstood the process? Is it possible to cross-build all of the packages 
in the distribution?

I hope I haven't missed anything that tells me the answers to all of these 
questions.

Thanks in advance for any answers you may have!

Paul

[1] https://www.gnu.org/software/guix/manual/html_node/Porting.html#Porting

Re: Cross-building GuixSD (and maybe using pre-built toolchains)

[Ludovic Courtès]

Hello,

Paul Boddie <paul@boddie.org.uk> skribis:

> I was looking at the documentation, which is rather more helpful than most 
> distributions when it comes to describing how the actual distribution archives 
> are built, and I figured a few things out about guix-daemon. However, I'm 
> doing all my work in a chroot - I don't want to "make install" and populate 
> various locations on my main system - and this doesn't seem to be entirely 
> compatible with the recommended way the daemon is used. Specifically, the 
> daemon does not seem to be able to build anything unless I specify the --
> disable-chroot flag.

How does guix-daemon fail exactly?  If needed, could you strace it to
get more details on what’s going on?

> Are there any recommended methods of running guix-daemon in a chroot and have 
> it create new chroots, or do I have to use some kind of virtualisation or 
> container technology? Is any kind of fakeroot/fakechroot mechanism supported?

I think few people do this, on the grounds that Guix is rather
non-intrusive: it stores all its stuff in /gnu/store and /var/guix.  So
if you want to get rid of it, all you have to do is delete those two
directories.  The rest of the system is untouched.

Now, if you really want it, I can’t think of any reason why guix-daemon
wouldn’t run in a chroot.  It currently requires root privileges,
precisely so that it can set up a chroot, separate name spaces, and so
on, but that could work in a chroot too.

> One thing that looks very promising is the ability to cross-bootstrap the 
> system, and I did manage to get that started for a new architecture (mipsel) 
> by performing some modifications suggested in the documentation [1], running 
> guix-daemon with the --disable-chroot flag, and then running this:
>
> guix build --target=mipsel-linux-gnu bootstrap-tarballs
>
> However, not wanting to leave my computer switched on all night, I stopped the 
> build after a while. I can understand the need to bootstrap things like 
> toolchains, but given that I am running Debian which has cross-toolchains for 
> mipsel, I wondered if I could short-circuit this process by employing pre-
> built toolchains. Would this be possible?

No.  The approach we’re taking is to have a self-contained distro, and
that’s where we get all the fruits of the “functional paradigm”
described in the manual.

It’s surely an annoyance when bootstrapping an architecture for the
first time.  Once it’s in place though, it’s OK.  You could have build
machines for that architecture publishing binaries to users (via ‘guix
publish’).

> Finally, it seems to be the case that the workflow involves building a 
> bootstrapped minimal system and then natively building packages. Or have I 
> misunderstood the process?

Correct, this is the process to port the distro to a new platform.

> Is it possible to cross-build all of the packages in the distribution?

That’s also an option: you can simply cross-build the packages that you
need and copy them to the target machine.

We make sure core packages can be cross-built to the architectures we
support (currently x86_64, i686, armhf, and mips64el).  However, there’s
no guarantee that cross-building works for other targets, or that
non-core packages cross-build at all.  We’d definitely welcome patches
in this area, though.  :-)

HTH!

Ludo’.

Re: Cross-building GuixSD (and maybe using pre-built toolchains)

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 1772 bytes --]

On Sun, Jul 03, 2016 at 04:51:27PM +0200, Paul Boddie wrote:
> Hello,
> 
...
>
> One thing that looks very promising is the ability to cross-bootstrap the 
> system, and I did manage to get that started for a new architecture (mipsel) 
> by performing some modifications suggested in the documentation [1], running 
> guix-daemon with the --disable-chroot flag, and then running this:
> 
> guix build --target=mipsel-linux-gnu bootstrap-tarballs
> 
> However, not wanting to leave my computer switched on all night, I stopped the 
> build after a while. I can understand the need to bootstrap things like 
> toolchains, but given that I am running Debian which has cross-toolchains for 
> mipsel, I wondered if I could short-circuit this process by employing pre-
> built toolchains. Would this be possible?

I built a set of bootstrap-tarballs on my very underpowered laptop
which took about 15 hours. The longest single package was gcc-4.9, which
took me about 5 hours. On a more powerful machine it should go much
faster.

> Finally, it seems to be the case that the workflow involves building a 
> bootstrapped minimal system and then natively building packages. Or have I 
> misunderstood the process? Is it possible to cross-build all of the packages 
> in the distribution?
> 
> I hope I haven't missed anything that tells me the answers to all of these 
> questions.
> 
> Thanks in advance for any answers you may have!
> 
> Paul
> 
> [1] https://www.gnu.org/software/guix/manual/html_node/Porting.html#Porting
> 

-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]