* Re: Cross-building GuixSD (and maybe using pre-built toolchains)
@ 2016-08-30 21:26 Paul Boddie
2016-08-31 20:04 ` Ludovic Courtès
0 siblings, 1 reply; 19+ messages in thread
From: Paul Boddie @ 2016-08-30 21:26 UTC (permalink / raw)
To: help-guix
Hello,
(Quoting a discussion on the development list, which proved rather too active
for me to follow. Sending to this list at Ludo's suggestion.)
> Paul Boddie <paul@boddie.org.uk> skribis:
> > I was looking at the documentation, which is rather more helpful than
> > most distributions when it comes to describing how the actual
> > distribution archives are built, and I figured a few things out about
> > guix-daemon. However, I'm doing all my work in a chroot - I don't want
> > to "make install" and populate various locations on my main system - and
> > this doesn't seem to be entirely compatible with the recommended way the
> > daemon is used. Specifically, the daemon does not seem to be able to
> > build anything unless I specify the -- disable-chroot flag.
>
> How does guix-daemon fail exactly? If needed, could you strace it to
> get more details on what’s going on?
I could try and take a look at this.
> > Are there any recommended methods of running guix-daemon in a chroot and
> > have it create new chroots, or do I have to use some kind of
> > virtualisation or container technology? Is any kind of
> > fakeroot/fakechroot mechanism supported?
>
> I think few people do this, on the grounds that Guix is rather
> non-intrusive: it stores all its stuff in /gnu/store and /var/guix. So
> if you want to get rid of it, all you have to do is delete those two
> directories. The rest of the system is untouched.
>
> Now, if you really want it, I can’t think of any reason why guix-daemon
> wouldn’t run in a chroot. It currently requires root privileges,
> precisely so that it can set up a chroot, separate name spaces, and so
> on, but that could work in a chroot too.
OK. I tend to run things in chroots for basic protection against things
deciding to install stuff in places I would rather keep "clean", and to use
different distro versions and packages. I guess I could instead have separate
partitions for /usr and other things and make them read-only, and so on, but
I'm not sure many people do this these days.
Then again, I do see people wanting to run Guix on other distros, so perhaps I
should just accept that it doesn't need to run inside a chroot itself.
> > One thing that looks very promising is the ability to cross-bootstrap the
> > system, and I did manage to get that started for a new architecture
> > (mipsel) by performing some modifications suggested in the documentation
> > [1], running guix-daemon with the --disable-chroot flag, and then
> > running this:
> >
> > guix build --target=mipsel-linux-gnu bootstrap-tarballs
> >
> > However, not wanting to leave my computer switched on all night, I
> > stopped the build after a while. I can understand the need to bootstrap
> > things like toolchains, but given that I am running Debian which has
> > cross-toolchains for mipsel, I wondered if I could short-circuit this
> > process by employing pre- built toolchains. Would this be possible?
>
> No. The approach we’re taking is to have a self-contained distro, and
> that’s where we get all the fruits of the “functional paradigm”
> described in the manual.
>
> It’s surely an annoyance when bootstrapping an architecture for the
> first time. Once it’s in place though, it’s OK. You could have build
> machines for that architecture publishing binaries to users (via ‘guix
> publish’).
I was really only looking for a short cut here, I admit. And if you have a
robust way of building toolchains, that's actually a great thing, because it
can be difficult to do repeatably (tracking the evolution of the software,
dealing with platform issues), and people still download random binary
toolchains for such purposes even today because it is a scary thought to build
their own from source.
I noticed that Arch Linux and its relative Parabola GNU/Linux support the
target machine using distcc as a client and having distcc servers cross-
compile code for the target architecture. That seems to require the
coordinating host to be running Arch/Parabola, which then means that some
bootstrapping needs to be done already. Could a similar thing be done with
GuixSD?
I do understand the "self-contained" motivation: it's exactly what Debian and
other distros insist upon, although I'd argue that it's also what holds such
distros back from properly supporting cross-compilation. (I did follow the
Emdebian project for a while, and it just seemed like very punishing work that
tended to interest very few people, sadly.)
> > Finally, it seems to be the case that the workflow involves building a
> > bootstrapped minimal system and then natively building packages. Or have
> > I misunderstood the process?
>
> Correct, this is the process to port the distro to a new platform.
Understood.
> > Is it possible to cross-build all of the packages in the distribution?
>
> That’s also an option: you can simply cross-build the packages that you
> need and copy them to the target machine.
>
> We make sure core packages can be cross-built to the architectures we
> support (currently x86_64, i686, armhf, and mips64el). However, there’s
> no guarantee that cross-building works for other targets, or that
> non-core packages cross-build at all. We’d definitely welcome patches
> in this area, though. :-)
Right. So you'd cross-build things like the kernel, libc, gcc, binutils, maybe
some other things (the bootstrapping discussed above). Then bring up the
target machine and hopefully have enough to start building packages somehow.
Does that sound correct?
> HTH!
Yes, I think it does. Thanks for the response and sorry once again for not
replying before now!
Regards,
Paul
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Cross-building GuixSD (and maybe using pre-built toolchains)
@ 2016-08-31 16:15 David Craven
2016-08-31 20:08 ` Ludovic Courtès
0 siblings, 1 reply; 19+ messages in thread
From: David Craven @ 2016-08-31 16:15 UTC (permalink / raw)
To: help-guix
Hi,
So one issue that happens when cross building is for things that have
the #:no-substitutes #t enabled. This is mainly found in service
derivations for generating configuration files. I guess if you aren't
using substitutes at all and want to cross-compile everything, it will
take a while till you hit this problem. I have not found a solution to
this problem yet, but I think it should be possible to get
gexp->script and gexp->file to ignore the #:system and #:target, since
I don't think there's a reason why configuration files can't be
cross-built.
David
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Cross-building GuixSD (and maybe using pre-built toolchains)
2016-08-30 21:26 Paul Boddie
@ 2016-08-31 20:04 ` Ludovic Courtès
2016-08-31 21:49 ` Paul Boddie
0 siblings, 1 reply; 19+ messages in thread
From: Ludovic Courtès @ 2016-08-31 20:04 UTC (permalink / raw)
To: Paul Boddie; +Cc: help-guix
Hi Paul,
Paul Boddie <paul@boddie.org.uk> skribis:
>> > Are there any recommended methods of running guix-daemon in a chroot and
>> > have it create new chroots, or do I have to use some kind of
>> > virtualisation or container technology? Is any kind of
>> > fakeroot/fakechroot mechanism supported?
>>
>> I think few people do this, on the grounds that Guix is rather
>> non-intrusive: it stores all its stuff in /gnu/store and /var/guix. So
>> if you want to get rid of it, all you have to do is delete those two
>> directories. The rest of the system is untouched.
>>
>> Now, if you really want it, I can’t think of any reason why guix-daemon
>> wouldn’t run in a chroot. It currently requires root privileges,
>> precisely so that it can set up a chroot, separate name spaces, and so
>> on, but that could work in a chroot too.
>
> OK. I tend to run things in chroots for basic protection against things
> deciding to install stuff in places I would rather keep "clean", and to use
> different distro versions and packages.
Precisely: Guix only ever touches /gnu/store, /var/guix, and optionally
/etc/guix; nothing else, I swear. ;-)
> I noticed that Arch Linux and its relative Parabola GNU/Linux support the
> target machine using distcc as a client and having distcc servers cross-
> compile code for the target architecture. That seems to require the
> coordinating host to be running Arch/Parabola, which then means that some
> bootstrapping needs to be done already. Could a similar thing be done with
> GuixSD?
Yes, using offloading:
https://www.gnu.org/software/guix/manual/html_node/Daemon-Offload-Setup.html
If your master node is x86_64 and its machines.scm lists a mips64el
machine as in the example above, then “guix build foo -s mips64el-linux”
automatically offloads to that mips machine. (This is the mechanism our
hydra.gnu.org build farm uses.)
>> That’s also an option: you can simply cross-build the packages that you
>> need and copy them to the target machine.
>>
>> We make sure core packages can be cross-built to the architectures we
>> support (currently x86_64, i686, armhf, and mips64el). However, there’s
>> no guarantee that cross-building works for other targets, or that
>> non-core packages cross-build at all. We’d definitely welcome patches
>> in this area, though. :-)
>
> Right. So you'd cross-build things like the kernel, libc, gcc, binutils, maybe
> some other things (the bootstrapping discussed above). Then bring up the
> target machine and hopefully have enough to start building packages somehow.
> Does that sound correct?
When bootstrapping a new architecture, we cross-build the relevant
“bootstrap binaries”, which are statically-linked tools such as GCC,
Guile, as well as libc, and from there we can start building natively:
https://www.gnu.org/software/guix/manual/html_node/Porting.html
Ludo’.
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Cross-building GuixSD (and maybe using pre-built toolchains)
2016-08-31 16:15 Cross-building GuixSD (and maybe using pre-built toolchains) David Craven
@ 2016-08-31 20:08 ` Ludovic Courtès
2016-08-31 22:01 ` David Craven
0 siblings, 1 reply; 19+ messages in thread
From: Ludovic Courtès @ 2016-08-31 20:08 UTC (permalink / raw)
To: David Craven; +Cc: help-guix
Hi David,
David Craven <david@craven.ch> skribis:
> So one issue that happens when cross building is for things that have
> the #:no-substitutes #t enabled. This is mainly found in service
> derivations for generating configuration files.
I think there’s currently no easy way to cross build a full GuixSD
anyway, like ‘guix system build --target=mips64el-linux-gnu’ (the
--target flag doesn’t exist yet). Or did you experiment in this area?
I’d be curious. It’s designed to allow this (in particular because
gexps are cross-compilation-aware), but I’ve never tried.
Ludo’.
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Cross-building GuixSD (and maybe using pre-built toolchains)
2016-08-31 20:04 ` Ludovic Courtès
@ 2016-08-31 21:49 ` Paul Boddie
2016-09-01 8:10 ` Ludovic Courtès
0 siblings, 1 reply; 19+ messages in thread
From: Paul Boddie @ 2016-08-31 21:49 UTC (permalink / raw)
To: help-guix
On Wednesday 31. August 2016 22.04.35 Ludovic Courtès wrote:
>
> Paul Boddie <paul@boddie.org.uk> skribis:
> >
> > OK. I tend to run things in chroots for basic protection against things
> > deciding to install stuff in places I would rather keep "clean", and to
> > use different distro versions and packages.
>
> Precisely: Guix only ever touches /gnu/store, /var/guix, and optionally
> /etc/guix; nothing else, I swear. ;-)
Right! Well, I may have another try outside a chroot and take your word for
it. ;-)
[...]
> > I noticed that Arch Linux and its relative Parabola GNU/Linux support the
> > target machine using distcc as a client and having distcc servers cross-
> > compile code for the target architecture. That seems to require the
> > coordinating host to be running Arch/Parabola, which then means that some
> > bootstrapping needs to be done already. Could a similar thing be done
> > with GuixSD?
>
> Yes, using offloading:
>
>
> https://www.gnu.org/software/guix/manual/html_node/Daemon-Offload-Setup.html
>
> If your master node is x86_64 and its machines.scm lists a mips64el
> machine as in the example above, then “guix build foo -s mips64el-linux”
> automatically offloads to that mips machine. (This is the mechanism our
> hydra.gnu.org build farm uses.)
But this seems like the opposite of the approach in Arch/Parabola: there, the
target machine (mips64el) coordinates the build, with the farm machines
(x86_64) cross-compiling the sources. The benefit is that architectures with
less powerful hardware can have their packages built for them, but this would
presumably be contrary to the principles employed by Guix, where native builds
are mandatory for viability and, I guess, reproducibility reasons.
[...]
> When bootstrapping a new architecture, we cross-build the relevant
> “bootstrap binaries”, which are statically-linked tools such as GCC,
> Guile, as well as libc, and from there we can start building natively:
>
> https://www.gnu.org/software/guix/manual/html_node/Porting.html
Understood, thanks!
Paul
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Cross-building GuixSD (and maybe using pre-built toolchains)
2016-08-31 20:08 ` Ludovic Courtès
@ 2016-08-31 22:01 ` David Craven
2016-08-31 22:13 ` David Craven
2016-09-01 8:17 ` Ludovic Courtès
0 siblings, 2 replies; 19+ messages in thread
From: David Craven @ 2016-08-31 22:01 UTC (permalink / raw)
To: Ludovic Courtès; +Cc: help-guix
Hi Ludo,
>> So one issue that happens when cross building is for things that have
>> the #:no-substitutes #t enabled. This is mainly found in service
>> derivations for generating configuration files.
>
> I think there’s currently no easy way to cross build a full GuixSD
> anyway, like ‘guix system build --target=mips64el-linux-gnu’ (the
> --target flag doesn’t exist yet). Or did you experiment in this area?
>
> I’d be curious. It’s designed to allow this (in particular because
> gexps are cross-compilation-aware), but I’ve never tried.
I'm currently not trying to cross-compile everything. The
--system=mips64el-linux gets us pretty far already. The missing parts
are only the ones where no substitutes are available.
What I've done so far is remove grub from the system/vm.scm file which
is added as a hard dependency (even when run with --no-grub). That
gets guix system build arm-minimal.scm --system=armhf-linux to
download all the substitutable dependencies but then fails at the end.
The other thing I've tried is guix system vm arm-minimal.scm
--system=armhf-linux. This fails when it tries to fetch
gstreamer-plugins-bad. This is required to build spice-gtk-viewer
which is required to run the tests for spice. This looks like qemu is
added as an input somewhere when it should be a native-input.
Other than these two issues I don't think that getting guixsd arm
running in qemu is too far off. This of course doesn't help
bootstrapping a new architecture, but it's nice for developers to be
able to test stuff on arm or mips.
Thanks,
David
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Cross-building GuixSD (and maybe using pre-built toolchains)
2016-08-31 22:01 ` David Craven
@ 2016-08-31 22:13 ` David Craven
2016-09-01 8:17 ` Ludovic Courtès
1 sibling, 0 replies; 19+ messages in thread
From: David Craven @ 2016-08-31 22:13 UTC (permalink / raw)
To: Ludovic Courtès; +Cc: help-guix
>> I think there’s currently no easy way to cross build a full GuixSD
>> anyway, like ‘guix system build --target=mips64el-linux-gnu’ (the
>> --target flag doesn’t exist yet). Or did you experiment in this area?
I'm operating under the assumption that when using the --target flag
packages aren't substituteable (since cross-compiling a package will
likely not yield the same result as natively compiling it) in which
case the --target flag would only be useful for either bootstrapping
or optimizing guixsd for an embedded system or something like that.
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Cross-building GuixSD (and maybe using pre-built toolchains)
2016-08-31 21:49 ` Paul Boddie
@ 2016-09-01 8:10 ` Ludovic Courtès
2016-09-01 15:51 ` Paul Boddie
0 siblings, 1 reply; 19+ messages in thread
From: Ludovic Courtès @ 2016-09-01 8:10 UTC (permalink / raw)
To: Paul Boddie; +Cc: help-guix
Paul Boddie <paul@boddie.org.uk> skribis:
> On Wednesday 31. August 2016 22.04.35 Ludovic Courtès wrote:
[...]
>> > I noticed that Arch Linux and its relative Parabola GNU/Linux support the
>> > target machine using distcc as a client and having distcc servers cross-
>> > compile code for the target architecture. That seems to require the
>> > coordinating host to be running Arch/Parabola, which then means that some
>> > bootstrapping needs to be done already. Could a similar thing be done
>> > with GuixSD?
>>
>> Yes, using offloading:
>>
>>
>> https://www.gnu.org/software/guix/manual/html_node/Daemon-Offload-Setup.html
>>
>> If your master node is x86_64 and its machines.scm lists a mips64el
>> machine as in the example above, then “guix build foo -s mips64el-linux”
>> automatically offloads to that mips machine. (This is the mechanism our
>> hydra.gnu.org build farm uses.)
>
> But this seems like the opposite of the approach in Arch/Parabola: there, the
> target machine (mips64el) coordinates the build, with the farm machines
> (x86_64) cross-compiling the sources. The benefit is that architectures with
> less powerful hardware can have their packages built for them, but this would
> presumably be contrary to the principles employed by Guix, where native builds
> are mandatory for viability and, I guess, reproducibility reasons.
I should mention that native builds are also subject to offloading. So
if your main machine is x86_64 and has a machines.scm file that lists
other x86_64 machines, then native builds can be offloaded to them.
If you cross-compile from x86_64, it’s a native build that can be
offloaded to another x86_64 machine.
Hope this clarifies things!
Ludo’.
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Cross-building GuixSD (and maybe using pre-built toolchains)
2016-08-31 22:01 ` David Craven
2016-08-31 22:13 ` David Craven
@ 2016-09-01 8:17 ` Ludovic Courtès
2016-09-01 10:11 ` David Craven
2016-09-01 11:59 ` Ludovic Courtès
1 sibling, 2 replies; 19+ messages in thread
From: Ludovic Courtès @ 2016-09-01 8:17 UTC (permalink / raw)
To: David Craven; +Cc: help-guix
Hi David,
David Craven <david@craven.ch> skribis:
>>> So one issue that happens when cross building is for things that have
>>> the #:no-substitutes #t enabled. This is mainly found in service
>>> derivations for generating configuration files.
>>
>> I think there’s currently no easy way to cross build a full GuixSD
>> anyway, like ‘guix system build --target=mips64el-linux-gnu’ (the
>> --target flag doesn’t exist yet). Or did you experiment in this area?
>>
>> I’d be curious. It’s designed to allow this (in particular because
>> gexps are cross-compilation-aware), but I’ve never tried.
>
> I'm currently not trying to cross-compile everything. The
> --system=mips64el-linux gets us pretty far already. The missing parts
> are only the ones where no substitutes are available.
Aaah OK. That’s not cross-compilation though, so it works if and only
if substitutes are available or builds can be offloaded to a machine of
that architecture. Otherwise it cannot really work.
Now, in most cases, the tiny Guile derivations created via
‘gexp->derivation’ et al. in GuixSD (e.g., the derivation that builds
the initrd) could use, say, an x86_64 Guile, even if building for
mips64el. However, it would be difficult to take advantage of this
AFAICS.
Ludo’.
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Cross-building GuixSD (and maybe using pre-built toolchains)
2016-09-01 8:17 ` Ludovic Courtès
@ 2016-09-01 10:11 ` David Craven
2016-09-01 11:16 ` Ludovic Courtès
2016-09-01 11:59 ` Ludovic Courtès
1 sibling, 1 reply; 19+ messages in thread
From: David Craven @ 2016-09-01 10:11 UTC (permalink / raw)
To: Ludovic Courtès; +Cc: help-guix
> Now, in most cases, the tiny Guile derivations created via
> ‘gexp->derivation’ et al. in GuixSD (e.g., the derivation that builds
> the initrd) could use, say, an x86_64 Guile, even if building for
> mips64el. However, it would be difficult to take advantage of this
> AFAICS.
So maybe I am taking the wrong approach. If an hypothetical --target
flag would exist, could it use natively built substitutes?
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Cross-building GuixSD (and maybe using pre-built toolchains)
2016-09-01 10:11 ` David Craven
@ 2016-09-01 11:16 ` Ludovic Courtès
0 siblings, 0 replies; 19+ messages in thread
From: Ludovic Courtès @ 2016-09-01 11:16 UTC (permalink / raw)
To: David Craven; +Cc: help-guix
David Craven <david@craven.ch> skribis:
>> Now, in most cases, the tiny Guile derivations created via
>> ‘gexp->derivation’ et al. in GuixSD (e.g., the derivation that builds
>> the initrd) could use, say, an x86_64 Guile, even if building for
>> mips64el. However, it would be difficult to take advantage of this
>> AFAICS.
>
> So maybe I am taking the wrong approach. If an hypothetical --target
> flag would exist, could it use natively built substitutes?
It could use substitutes for cross-compiled things, just like “guix
build coreutils --target=mips64el-linux-gnu” can use substitutes.
Ludo’.
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Cross-building GuixSD (and maybe using pre-built toolchains)
2016-09-01 8:17 ` Ludovic Courtès
2016-09-01 10:11 ` David Craven
@ 2016-09-01 11:59 ` Ludovic Courtès
1 sibling, 0 replies; 19+ messages in thread
From: Ludovic Courtès @ 2016-09-01 11:59 UTC (permalink / raw)
To: David Craven; +Cc: help-guix
[-- Attachment #1: Type: text/plain, Size: 1418 bytes --]
ludo@gnu.org (Ludovic Courtès) skribis:
> David Craven <david@craven.ch> skribis:
>
>>>> So one issue that happens when cross building is for things that have
>>>> the #:no-substitutes #t enabled. This is mainly found in service
>>>> derivations for generating configuration files.
>>>
>>> I think there’s currently no easy way to cross build a full GuixSD
>>> anyway, like ‘guix system build --target=mips64el-linux-gnu’ (the
>>> --target flag doesn’t exist yet). Or did you experiment in this area?
>>>
>>> I’d be curious. It’s designed to allow this (in particular because
>>> gexps are cross-compilation-aware), but I’ve never tried.
>>
>> I'm currently not trying to cross-compile everything. The
>> --system=mips64el-linux gets us pretty far already. The missing parts
>> are only the ones where no substitutes are available.
>
> Aaah OK. That’s not cross-compilation though, so it works if and only
> if substitutes are available or builds can be offloaded to a machine of
> that architecture. Otherwise it cannot really work.
>
> Now, in most cases, the tiny Guile derivations created via
> ‘gexp->derivation’ et al. in GuixSD (e.g., the derivation that builds
> the initrd) could use, say, an x86_64 Guile, even if building for
> mips64el. However, it would be difficult to take advantage of this
> AFAICS.
There’s an untested hack to illustrate:
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Type: text/x-patch, Size: 2531 bytes --]
diff --git a/guix/gexp.scm b/guix/gexp.scm
index 302879f..53239ac 100644
--- a/guix/gexp.scm
+++ b/guix/gexp.scm
@@ -523,6 +523,8 @@ The other arguments are as for 'derivation'."
(cons file-name thing)))
graphs))
+ (define gsystem "x86_64-linux")
+
(mlet* %store-monad (;; The following binding forces '%current-system' and
;; '%current-target-system' to be looked up at >>=
;; time.
@@ -546,13 +548,13 @@ The other arguments are as for 'derivation'."
(object->string sexp)))
(modules (if (pair? %modules)
(imported-modules %modules
- #:system system
+ #:system gsystem
#:module-path module-path
#:guile guile-for-build)
(return #f)))
(compiled (if (pair? %modules)
(compiled-modules %modules
- #:system system
+ #:system gsystem
#:module-path module-path
#:guile guile-for-build)
(return #f)))
@@ -587,7 +589,7 @@ The other arguments are as for 'derivation'."
,builder)
#:outputs outputs
#:env-vars env-vars
- #:system system
+ #:system gsystem
#:inputs `((,guile)
(,builder)
,@(if modules
diff --git a/guix/packages.scm b/guix/packages.scm
index 3646b9b..fe23d35 100644
--- a/guix/packages.scm
+++ b/guix/packages.scm
@@ -1104,7 +1104,7 @@ symbolic output name, such as \"out\". Note that this procedure calls
"This monadic procedure changes the Guile currently used to run the build
code of derivations to GUILE, a package object."
(lambda (store)
- (let ((guile (package-derivation store guile)))
+ (let ((guile (package-derivation store guile "x86_64-linux")))
(values (%guile-for-build guile) store))))
(define* (package-file package
[-- Attachment #3: Type: text/plain, Size: 12 bytes --]
Ludo’.
^ permalink raw reply related [flat|nested] 19+ messages in thread
* Re: Cross-building GuixSD (and maybe using pre-built toolchains)
2016-09-01 8:10 ` Ludovic Courtès
@ 2016-09-01 15:51 ` Paul Boddie
2016-09-01 19:43 ` Ludovic Courtès
2016-09-02 18:52 ` David Craven
0 siblings, 2 replies; 19+ messages in thread
From: Paul Boddie @ 2016-09-01 15:51 UTC (permalink / raw)
To: help-guix
On Thursday 1. September 2016 10.10.15 Ludovic Courtès wrote:
> Paul Boddie <paul@boddie.org.uk> skribis:
> >
[Offloading]
> > But this seems like the opposite of the approach in Arch/Parabola: there,
> > the target machine (mips64el) coordinates the build, with the farm
> > machines (x86_64) cross-compiling the sources. The benefit is that
> > architectures with less powerful hardware can have their packages built
> > for them, but this would presumably be contrary to the principles
> > employed by Guix, where native builds are mandatory for viability and, I
> > guess, reproducibility reasons.
>
> I should mention that native builds are also subject to offloading. So
> if your main machine is x86_64 and has a machines.scm file that lists
> other x86_64 machines, then native builds can be offloaded to them.
>
> If you cross-compile from x86_64, it’s a native build that can be
> offloaded to another x86_64 machine.
>
> Hope this clarifies things!
Perhaps. Here come the stupid questions: ;-)
When you write "cross-compile from x86_64" do you mean run a cross-compiler on
an x86_64 host that targets another architecture? (I'm guessing "yes".)
From previous questions, it appears that any compiler must have been built
specifically for Guix to use (and cannot be those those pre-built toolchains
provided by, say, Debian), so would these cross-compilers be the ones normally
used to generate the "bootstrap binaries" (that would normally be used to then
build packages on the target system)?
Or do I first bootstrap Guix on my (x86_64, in this example) "build system",
then generate special cross-compilers for my target architecture (that are
then exposed via offloading)?
And can I provide a foreign architecture build daemon on the same machine as
the one performing the build?
Sorry if this is all obvious or if I'm not assuming that what is possible is
actually possible! I'm used to discovering limitations with build systems. (To
me it is remarkable that given all the portable things like shell scripts that
do a lot of the lifting in build processes, many build systems have an
unreasonable affinity to the build host's architecture.)
Thanks for all the help!
Paul
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Cross-building GuixSD (and maybe using pre-built toolchains)
2016-09-01 15:51 ` Paul Boddie
@ 2016-09-01 19:43 ` Ludovic Courtès
2016-09-02 18:52 ` David Craven
1 sibling, 0 replies; 19+ messages in thread
From: Ludovic Courtès @ 2016-09-01 19:43 UTC (permalink / raw)
To: Paul Boddie; +Cc: help-guix
Paul Boddie <paul@boddie.org.uk> skribis:
> When you write "cross-compile from x86_64" do you mean run a cross-compiler on
> an x86_64 host that targets another architecture? (I'm guessing "yes".)
Yes.
> From previous questions, it appears that any compiler must have been built
> specifically for Guix to use (and cannot be those those pre-built toolchains
> provided by, say, Debian), so would these cross-compilers be the ones normally
> used to generate the "bootstrap binaries" (that would normally be used to then
> build packages on the target system)?
Yes, Guix packages only depend on other Guix packages; there cannot be
dependencies on external packages.
> Or do I first bootstrap Guix on my (x86_64, in this example) "build system",
> then generate special cross-compilers for my target architecture (that are
> then exposed via offloading)?
Offloading and cross-compilation are unrelated. Offloading is just a
mechanism that allows distributed builds.
Guix supports x86_64-linux-gnu, so, IOW, the distro is already
“bootstrapped” for that platform. There’s nothing special to be done.
I’m trying to find ways to rephrase this, but I can’t think of other
ways than what the “Bootstrapping” and “Porting” sections of the manuals
say. :-)
> And can I provide a foreign architecture build daemon on the same machine as
> the one performing the build?
No.
> Sorry if this is all obvious or if I'm not assuming that what is possible is
> actually possible! I'm used to discovering limitations with build systems. (To
> me it is remarkable that given all the portable things like shell scripts that
> do a lot of the lifting in build processes, many build systems have an
> unreasonable affinity to the build host's architecture.)
>
> Thanks for all the help!
No problem! I realize it can be difficult to convey the big picture of
Guix here to someone already familiar with other build systems.
Ludo’.
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Cross-building GuixSD (and maybe using pre-built toolchains)
2016-09-01 15:51 ` Paul Boddie
2016-09-01 19:43 ` Ludovic Courtès
@ 2016-09-02 18:52 ` David Craven
2016-09-03 12:56 ` Ludovic Courtès
1 sibling, 1 reply; 19+ messages in thread
From: David Craven @ 2016-09-02 18:52 UTC (permalink / raw)
To: Ludovic Courtès; +Cc: help-guix
> There’s an untested hack to illustrate:
Thanks! I think I got it kind of working - but we'll have to see after
there's a armhf kernel available.
When adding cross-compilation support to the linux-libre package I
noticed that native-inputs are in #:key native-inputs when
cross-compiling but not when compiling natively. I think this is a
problem because we can't just use #:key native-inputs instead of #:key
inputs when adding support for cross-compilation.
WDYT?
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Cross-building GuixSD (and maybe using pre-built toolchains)
2016-09-02 18:52 ` David Craven
@ 2016-09-03 12:56 ` Ludovic Courtès
2016-09-03 12:59 ` David Craven
0 siblings, 1 reply; 19+ messages in thread
From: Ludovic Courtès @ 2016-09-03 12:56 UTC (permalink / raw)
To: David Craven; +Cc: help-guix
Hi,
David Craven <david@craven.ch> skribis:
> When adding cross-compilation support to the linux-libre package I
> noticed that native-inputs are in #:key native-inputs when
> cross-compiling but not when compiling natively. I think this is a
> problem because we can't just use #:key native-inputs instead of #:key
> inputs when adding support for cross-compilation.
>
> WDYT?
It’s on purpose, and packages rarely (if even) need to refer to
#:native-inputs; only a couple of phases in gnu-build-system.scm refer
to #:native-inputs.
In which context do you need to refer to #:native-inputs?
Ludo’.
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Cross-building GuixSD (and maybe using pre-built toolchains)
2016-09-03 12:56 ` Ludovic Courtès
@ 2016-09-03 12:59 ` David Craven
2016-09-03 21:18 ` Ludovic Courtès
0 siblings, 1 reply; 19+ messages in thread
From: David Craven @ 2016-09-03 12:59 UTC (permalink / raw)
To: Ludovic Courtès; +Cc: help-guix
> It’s on purpose, and packages rarely (if even) need to refer to
> #:native-inputs; only a couple of phases in gnu-build-system.scm refer
> to #:native-inputs.
>
> In which context do you need to refer to #:native-inputs?
Cross-compiling the linux kernel.
> ;; Apply the neat patch.
> (system* "patch" "-p1" "--force"
> "-i" (assoc-ref inputs "patch/freedo+gnu"))
and
> (mit (assoc-ref inputs "module-init-tools")))
fail for no "real" reason. I could add a (if target native-inputs
inputs) but it seems ugly and think it might be a common problem?
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Cross-building GuixSD (and maybe using pre-built toolchains)
2016-09-03 12:59 ` David Craven
@ 2016-09-03 21:18 ` Ludovic Courtès
2016-09-03 21:27 ` David Craven
0 siblings, 1 reply; 19+ messages in thread
From: Ludovic Courtès @ 2016-09-03 21:18 UTC (permalink / raw)
To: David Craven; +Cc: help-guix
David Craven <david@craven.ch> skribis:
>> It’s on purpose, and packages rarely (if even) need to refer to
>> #:native-inputs; only a couple of phases in gnu-build-system.scm refer
>> to #:native-inputs.
>>
>> In which context do you need to refer to #:native-inputs?
>
> Cross-compiling the linux kernel.
>
>> ;; Apply the neat patch.
>> (system* "patch" "-p1" "--force"
>> "-i" (assoc-ref inputs "patch/freedo+gnu"))
This looks like a remnant of the past; nowadays it would work to add
‘%boot-logo’ to ‘patches’ in ‘origin’. We should do that.
> and
>
>> (mit (assoc-ref inputs "module-init-tools")))
>
> fail for no "real" reason. I could add a (if target native-inputs
> inputs) but it seems ugly and think it might be a common problem?
Hmm, I can see two occurrences of
(assoc-ref (or native-inputs inputs) "bash")
in gnu/packages/*.scm.
Not ideal, but I’d suggest sticking to that for now. :-)
Ludo’.
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Cross-building GuixSD (and maybe using pre-built toolchains)
2016-09-03 21:18 ` Ludovic Courtès
@ 2016-09-03 21:27 ` David Craven
0 siblings, 0 replies; 19+ messages in thread
From: David Craven @ 2016-09-03 21:27 UTC (permalink / raw)
To: Ludovic Courtès; +Cc: help-guix
> (assoc-ref (or native-inputs inputs) "bash")
Yes, that's not too bad. Thanks!
I'll wait for someone to comment the patches I've already submitted to
the mailing list (to see if they are cosher :) before making the
changes.
FYI: I rebuilt linux-libre-headers but the tar test-suite kept
crashing my computer, don't know what that's about. I don't think it's
related to my changes...
^ permalink raw reply [flat|nested] 19+ messages in thread
end of thread, other threads:[~2016-09-03 21:27 UTC | newest]
Thread overview: 19+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2016-08-31 16:15 Cross-building GuixSD (and maybe using pre-built toolchains) David Craven
2016-08-31 20:08 ` Ludovic Courtès
2016-08-31 22:01 ` David Craven
2016-08-31 22:13 ` David Craven
2016-09-01 8:17 ` Ludovic Courtès
2016-09-01 10:11 ` David Craven
2016-09-01 11:16 ` Ludovic Courtès
2016-09-01 11:59 ` Ludovic Courtès
-- strict thread matches above, loose matches on Subject: below --
2016-08-30 21:26 Paul Boddie
2016-08-31 20:04 ` Ludovic Courtès
2016-08-31 21:49 ` Paul Boddie
2016-09-01 8:10 ` Ludovic Courtès
2016-09-01 15:51 ` Paul Boddie
2016-09-01 19:43 ` Ludovic Courtès
2016-09-02 18:52 ` David Craven
2016-09-03 12:56 ` Ludovic Courtès
2016-09-03 12:59 ` David Craven
2016-09-03 21:18 ` Ludovic Courtès
2016-09-03 21:27 ` David Craven
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).