Cross-compilation, Guix "system", and GNU "triplet"

Cross-compilation, Guix "system", and GNU "triplet"

[Chris Marusich]

[-- Attachment #1: Type: text/plain, Size: 3253 bytes --]

Hi,

As mentioned in bug 29402 [1], Guix uses strings like "armhf-linux" to
identify a system platform.  However, to cross-compile a package using
Guix, you need to specify the target platform's "GNU triplet", which
looks like "arm-linux-gnueabihf" (e.g. by invoking "guix build
--target=arm-linux-gnueabihf hello").  These are not the same thing.

OK, so Guix system "armhf-linux" maps to GNU triplet
"arm-linux-gnueabihf".  Or perhaps a better way to think of this is that
Guix system "armhf-linux" refers to a very specific platform choice -
and it happens to be the case that that platform can also be identified
by the GNU triplet "arm-linux-gnueabihf".  What about other ARM
variants?  When we eventually get Guix running on two variants of ARM,
will we just define another special string, like "the-second-arm-linux",
to identify its Guix system?

My understanding is that multiple GNU triplets can refer to the same
platform.  For example, I believe "arm-linux-gnueabihf" is the same as
"arm-unknown-linux-gnueabihf" because config.sub in Autoconf translates
the former to the latter.  Do they refer to the same platform?

Guix seems to treat them differently:

--8<---------------cut here---------------start------------->8---
[0] [env] marusich@garuda.local:~/guix
$ ./pre-inst-env guix build --no-substitutes --target=arm-linux-gnueabihf hello
/gnu/store/g7cmg23ssqwknpmbbrvap60iv524i9fw-hello-2.10
[0] [env] marusich@garuda.local:~/guix
$ ./pre-inst-env guix build --no-substitutes --target=arm-unknown-linux-gnueabihf hello
@ build-started /gnu/store/9kk7b46s836n5r0jh0sb00v8p3i5vlhg-gcc-cross-arm-unknown-linux-gnueabihf-5.4.0.drv - x86_64-linux /var/log/guix/drvs/9k//k7b46s836n5r0jh0sb00v8p3i5vlhg-gcc-cross-arm-unknown-linux-gnueabihf-5.4.0.drv.bz2
...
--8<---------------cut here---------------end--------------->8---

As you can see, Guix initiated a build even though the targets are "the
same" as far as Autoconf is concerned.  Perhaps this is because the
strings "arm-linux-gnueabihf" and "arm-unknown-linux-gnueabihf" differ,
which causes the derivations' output paths to differ?  I don't know, but
if that's the case, then it would explain why the second invocation
above resulted in a build, even though these two targets refer to the
same platform.

Finally, I want to ask for some clarification regarding the "GNU
triplet" concept in general.  Using the language of the section titled
"Specifying Target Triplets" in the Autoconf manual, I understand that a
GNU triplet takes the following form:

    `CPU-VENDOR-OS', where OS can be `SYSTEM' or `KERNEL-SYSTEM'

It seems to me that "SYSTEM" here always denotes an ABI, such as
"gnueabihf".  Can SYSTEM ever refer to something that is NOT an ABI?
The reason I ask is because in the Autoconf manual, this part of the
"triplet" is sometimes referred to as the SYSTEM (in the section
mentioned above), and sometimes it is referred to as the OS (in the
section titled "System Type").  But it seems to me that it is neither a
"system" nor an "os"; it is always an ABI.  Am I right?

Thank you for taking the time to help me understand!

Footnotes: 
[1]  https://debbugs.gnu.org/cgi/bugreport.cgi?bug=29402

-- 
Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: Cross-compilation, Guix "system", and GNU "triplet"

[Ludovic Courtès]

Hi,

Chris Marusich <cmmarusich@gmail.com> skribis:

> As mentioned in bug 29402 [1], Guix uses strings like "armhf-linux" to
> identify a system platform.  However, to cross-compile a package using
> Guix, you need to specify the target platform's "GNU triplet", which
> looks like "arm-linux-gnueabihf" (e.g. by invoking "guix build
> --target=arm-linux-gnueabihf hello").  These are not the same thing.
>
> OK, so Guix system "armhf-linux" maps to GNU triplet
> "arm-linux-gnueabihf".  Or perhaps a better way to think of this is that
> Guix system "armhf-linux" refers to a very specific platform choice -
> and it happens to be the case that that platform can also be identified
> by the GNU triplet "arm-linux-gnueabihf".  What about other ARM
> variants?  When we eventually get Guix running on two variants of ARM,
> will we just define another special string, like "the-second-arm-linux",
> to identify its Guix system?

The Guix system string has the form ARCHITECTURE-KERNEL, because these
are the two things not explicitly captured by the derivation graph.

The “hf” in “armhf-linux” denotes an ARMv7 processor with a hardware
floating point unit; it does not denote the ABI since the ABI is
something that’s software-defined.

As it turns out, we map “armhf-linux” to the HF ABI (“eabihf”), because
that’s the most sensible thing to do, but we could just as well map it
to the old soft-float ABI (“eabi”).

> My understanding is that multiple GNU triplets can refer to the same
> platform.  For example, I believe "arm-linux-gnueabihf" is the same as
> "arm-unknown-linux-gnueabihf" because config.sub in Autoconf translates
> the former to the latter.  Do they refer to the same platform?

Usually yes, because user-land software doesn’t care about the “vendor”
part.  It could make a difference to, say, GRUB, for which the “vendor”
part is important.

> Guix seems to treat them differently:
>
> [0] [env] marusich@garuda.local:~/guix
> $ ./pre-inst-env guix build --no-substitutes --target=arm-linux-gnueabihf hello
> /gnu/store/g7cmg23ssqwknpmbbrvap60iv524i9fw-hello-2.10
> [0] [env] marusich@garuda.local:~/guix
> $ ./pre-inst-env guix build --no-substitutes --target=arm-unknown-linux-gnueabihf hello
> @ build-started /gnu/store/9kk7b46s836n5r0jh0sb00v8p3i5vlhg-gcc-cross-arm-unknown-linux-gnueabihf-5.4.0.drv - x86_64-linux /var/log/guix/drvs/9k//k7b46s836n5r0jh0sb00v8p3i5vlhg-gcc-cross-arm-unknown-linux-gnueabihf-5.4.0.drv.bz2
> ...
>
> As you can see, Guix initiated a build even though the targets are "the
> same" as far as Autoconf is concerned.  Perhaps this is because the
> strings "arm-linux-gnueabihf" and "arm-unknown-linux-gnueabihf" differ,
> which causes the derivations' output paths to differ?

Guix doesn’t try to interpret GNU triplets in any way.  So when you pass
a triplet to --target, it goes ahead and creates a cross-toolchain for
that triplet.

If turns out that Hydra has substitutes for the arm-linux-gnueabihf
cross-toolchain, but not for arm-whatever-linux-gnueabihf.

> Finally, I want to ask for some clarification regarding the "GNU
> triplet" concept in general.  Using the language of the section titled
> "Specifying Target Triplets" in the Autoconf manual, I understand that a
> GNU triplet takes the following form:
>
>     `CPU-VENDOR-OS', where OS can be `SYSTEM' or `KERNEL-SYSTEM'
>
> It seems to me that "SYSTEM" here always denotes an ABI, such as
> "gnueabihf".  Can SYSTEM ever refer to something that is NOT an ABI?

The triplet idea predates GNU/Linux, I think.  Back then, it was quite
clear what the OS was: SunOS, AIX, GNU, etc.

With glibc ported to the kernel Linux, it became useful to differentiate
between “gnu” (i.e., GNU/Hurd) and “linux-gnu”.  User-land software
often only cares about the “-gnu” part, but sometimes it needs to know
about the kernel as well.

Then people had the idea to abuse the triplet to piggyback information
about the ABI being targeted.  There are bits in the toolchain that
interpret it meaningfully.

I hope it makes sense.  :-)

Ludo’.

Re: Cross-compilation, Guix "system", and GNU "triplet"

[Chris Marusich]

[-- Attachment #1: Type: text/plain, Size: 4880 bytes --]

Hi Ludo,

As always, thank you for entertaining my questions!

ludo@gnu.org (Ludovic Courtès) writes:

> The Guix system string has the form ARCHITECTURE-KERNEL, because these
> are the two things not explicitly captured by the derivation graph.

I see.  When you say that something is "captured by the derivation
graph," do you mean that it is part of the hash calculation, so it
influences a derivation's store path as well as the derivation's
outputs' store paths?  I think that's what you mean, but I'd like to
make sure.  My understanding is that this is precisely the reason why we
make the Guix system string a field of each derivation: it ensures that
different systems use different store paths, which is good because an
x86_64-linux system probably can't build derivations that are intended
to be built on an armhf-linux system.

Other than the Guix system string, what things are "captured by the
derivation graph" (either explicitly or implicitly)?  For example, is
the ABI captured by the derivation graph?  What about the vendor?  What
about the executable file format (e.g., ELF vs. some other possibility)?
Anything else?

> The “hf” in “armhf-linux” denotes an ARMv7 processor with a hardware
> floating point unit; it does not denote the ABI since the ABI is
> something that’s software-defined.
>
> As it turns out, we map “armhf-linux” to the HF ABI (“eabihf”), because
> that’s the most sensible thing to do, but we could just as well map it
> to the old soft-float ABI (“eabi”).

My understanding is that when a derivation's system is "armhf-linux", it
means that the derivation itself (as opposed to its output) is intended
to be built (i.e., the derivation's builder is intended to be run) on
the "armhf-linux" platform.  If the Guix system string does not contain
details like the ABI, then how does Guix differentiate between the case
in which the derivation is built on an ARM system that uses the HF ABI,
and the case in which it is built on an ARM system that uses the
soft-float ABI?  Are you saying that we should expect the derivation to
build correctly and produce the same output regardless of which ABI is
used?  Or is there some other mechanism by which Guix differentiates
these two cases that I am not aware of?

>> My understanding is that multiple GNU triplets can refer to the same
>> platform.  For example, I believe "arm-linux-gnueabihf" is the same as
>> "arm-unknown-linux-gnueabihf" because config.sub in Autoconf translates
>> the former to the latter.  Do they refer to the same platform?
>
> Usually yes, because user-land software doesn’t care about the “vendor”
> part.  It could make a difference to, say, GRUB, for which the “vendor”
> part is important.

What is the recommended way to package software in Guix when the
software's behavior or the output of the derivation that builds it can
be influenced by the value of the "vendor" part?

For example, suppose I purchase an x86_64-linux server from a vendor
called FooCorp specifically so that I can take advantage of some super
cool feature of a piece of software that has been packaged in Guix.  It
would be unfortunate if I installed that software using a substitute
from Hydra, only to find that the super cool feature does not work on my
FooCorp machine because the software was not explicitly compiled for the
x86_64-foocorp-linux-gnu platform.  This is probably a theoretical edge
case, but I'm curious to know what the plan is for dealing with
situations like this.

> Guix doesn’t try to interpret GNU triplets in any way.  So when you pass
> a triplet to --target, it goes ahead and creates a cross-toolchain for
> that triplet.
>
> If turns out that Hydra has substitutes for the arm-linux-gnueabihf
> cross-toolchain, but not for arm-whatever-linux-gnueabihf.

I see.  That makes sense.

> The triplet idea predates GNU/Linux, I think.  Back then, it was quite
> clear what the OS was: SunOS, AIX, GNU, etc.
>
> With glibc ported to the kernel Linux, it became useful to differentiate
> between “gnu” (i.e., GNU/Hurd) and “linux-gnu”.  User-land software
> often only cares about the “-gnu” part, but sometimes it needs to know
> about the kernel as well.
>
> Then people had the idea to abuse the triplet to piggyback information
> about the ABI being targeted.  There are bits in the toolchain that
> interpret it meaningfully.
>
> I hope it makes sense.  :-)

Yes, I understand.  The GNU triplet concept makes more sense to me now.
However, if information like the vendor and the OS/ABI are important
enough to put into the GNU triplet, I still don't understand why we can
get away with omitting those details from the Guix system string.

Hopefully you can help me to understand what I'm missing.

-- 
Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: Cross-compilation, Guix "system", and GNU "triplet"

[Ludovic Courtès]

Heya,

Chris Marusich <cmmarusich@gmail.com> skribis:

> ludo@gnu.org (Ludovic Courtès) writes:
>
>> The Guix system string has the form ARCHITECTURE-KERNEL, because these
>> are the two things not explicitly captured by the derivation graph.
>
> I see.  When you say that something is "captured by the derivation
> graph," do you mean that it is part of the hash calculation, so it
> influences a derivation's store path as well as the derivation's
> outputs' store paths?

What I meant is that, in the functional model as implemented in
Nix/Guix, the derivation graph captures all user-land software that
comes into play.  But it does not capture the hardware and kernel, both
of which are taken from granted.

Thus, this system string allows us to represent the dependency on a
kernel and hardware architecture.  In turn, this allows us to
distinguish between, say, the Guile derivation for x86_64 and that for
MIPS.

> Other than the Guix system string, what things are "captured by the
> derivation graph" (either explicitly or implicitly)?  For example, is
> the ABI captured by the derivation graph?  What about the vendor?  What
> about the executable file format (e.g., ELF vs. some other possibility)?
> Anything else?

The ABI and file format are entirely (or almost entirely) the
responsibility of user-land software (how you configure the toolchain
determines what ABI you use, for instance.)  Thus they’re necessarily
captured by the dependency graph; no need to store that information
elsewhere.

The vendor doesn’t matter on Intel machines.  It matters on ARM, but
only for specific things like the bootloader and the kernel.

>> The “hf” in “armhf-linux” denotes an ARMv7 processor with a hardware
>> floating point unit; it does not denote the ABI since the ABI is
>> something that’s software-defined.
>>
>> As it turns out, we map “armhf-linux” to the HF ABI (“eabihf”), because
>> that’s the most sensible thing to do, but we could just as well map it
>> to the old soft-float ABI (“eabi”).
>
> My understanding is that when a derivation's system is "armhf-linux", it
> means that the derivation itself (as opposed to its output) is intended
> to be built (i.e., the derivation's builder is intended to be run) on
> the "armhf-linux" platform.  If the Guix system string does not contain
> details like the ABI, then how does Guix differentiate between the case
> in which the derivation is built on an ARM system that uses the HF ABI,
> and the case in which it is built on an ARM system that uses the
> soft-float ABI?

It’s the toolchain that shows up in the graph that determines what ABI
is targeted.

>> Usually yes, because user-land software doesn’t care about the “vendor”
>> part.  It could make a difference to, say, GRUB, for which the “vendor”
>> part is important.
>
> What is the recommended way to package software in Guix when the
> software's behavior or the output of the derivation that builds it can
> be influenced by the value of the "vendor" part?

There are two or three examples of that, which are U-Boot, GRUB, and
Linux-libre.  We essentially define several package variants for each of
these, depending on the target hardware.

> For example, suppose I purchase an x86_64-linux server from a vendor
> called FooCorp

Nothing to worry about in this case.  :-)

> Yes, I understand.  The GNU triplet concept makes more sense to me now.
> However, if information like the vendor and the OS/ABI are important
> enough to put into the GNU triplet, I still don't understand why we can
> get away with omitting those details from the Guix system string.

I hope I answered that question!

That said, you may get better-informed answers on the GCC or Binutils
mailing lists.  :-)

Ludo’.

Re: Cross-compilation, Guix "system", and GNU "triplet"

[Chris Marusich]

[-- Attachment #1: Type: text/plain, Size: 2587 bytes --]

ludo@gnu.org (Ludovic Courtès) writes:

> What I meant is that, in the functional model as implemented in
> Nix/Guix, the derivation graph captures all user-land software that
> comes into play.  But it does not capture the hardware and kernel, both
> of which are taken from granted.
>
> Thus, this system string allows us to represent the dependency on a
> kernel and hardware architecture.  In turn, this allows us to
> distinguish between, say, the Guile derivation for x86_64 and that for
> MIPS.
>
> ...
> 
> The ABI and file format are entirely (or almost entirely) the
> responsibility of user-land software (how you configure the toolchain
> determines what ABI you use, for instance.)  Thus they’re necessarily
> captured by the dependency graph; no need to store that information
> elsewhere.
> 
> ...
>
> It’s the toolchain that shows up in the graph that determines what ABI
> is targeted.

I think I understand now.  I was missing the fact that what ABI you use
is determined by how you configure the toolchain.  I don't (yet!) know
as much about compilation, cross-compilation, and the GNU toolchain as
I'd like, so I appreciate you taking the time to explain to me what must
have seemed obvious to you.

>> What is the recommended way to package software in Guix when the
>> software's behavior or the output of the derivation that builds it can
>> be influenced by the value of the "vendor" part?
>
> There are two or three examples of that, which are U-Boot, GRUB, and
> Linux-libre.  We essentially define several package variants for each of
> these, depending on the target hardware.

OK, that makes sense.

>> For example, suppose I purchase an x86_64-linux server from a vendor
>> called FooCorp
>
> Nothing to worry about in this case.  :-)

Do you say that because if it mattered, we could just define a separate
package that takes advantage of the vendor-specific feature, like we
might do for U-Boot and GRUB?

>> Yes, I understand.  The GNU triplet concept makes more sense to me now.
>> However, if information like the vendor and the OS/ABI are important
>> enough to put into the GNU triplet, I still don't understand why we can
>> get away with omitting those details from the Guix system string.
>
> I hope I answered that question!

Yes, I think you did.  I really appreciate it!

> That said, you may get better-informed answers on the GCC or Binutils
> mailing lists.  :-)

Thank you for the suggestion.  I'll keep it in mind.  It's good to know
where to go for help!

-- 
Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: Cross-compilation, Guix "system", and GNU "triplet"

[Chris Marusich]

[-- Attachment #1: Type: text/plain, Size: 1233 bytes --]

ludo@gnu.org (Ludovic Courtès) writes:

> The ABI and file format are entirely (or almost entirely) the
> responsibility of user-land software (how you configure the toolchain
> determines what ABI you use, for instance.)  Thus they’re necessarily
> captured by the dependency graph; no need to store that information
> elsewhere.
>
> ...
> 
> It’s the toolchain that shows up in the graph that determines what ABI
> is targeted.

The Guix manual makes it sound like the Guix system string can identify
more than just the CPU architecture and kernel.  Specifically, in the
section titled "GNU Distribution", it says the following:

‘armhf-linux’
     ARMv7-A architecture with hard float, Thumb-2 and NEON, using the
     EABI hard-float application binary interface (ABI), and Linux-Libre
     kernel.

‘aarch64-linux’
     little-endian 64-bit ARMv8-A processors, Linux-Libre kernel.  This
     is currently in an experimental stage, with limited support.  *Note
     Contributing::, for how to help!

‘mips64el-linux’
     little-endian 64-bit MIPS processors, specifically the Loongson
     series, n32 ABI, and Linux-Libre kernel.


Is this langugae too specific?

-- 
Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: Cross-compilation, Guix "system", and GNU "triplet"

[Ludovic Courtès]

Chris Marusich <cmmarusich@gmail.com> skribis:

> ludo@gnu.org (Ludovic Courtès) writes:
>
>> The ABI and file format are entirely (or almost entirely) the
>> responsibility of user-land software (how you configure the toolchain
>> determines what ABI you use, for instance.)  Thus they’re necessarily
>> captured by the dependency graph; no need to store that information
>> elsewhere.
>>
>> ...
>> 
>> It’s the toolchain that shows up in the graph that determines what ABI
>> is targeted.
>
> The Guix manual makes it sound like the Guix system string can identify
> more than just the CPU architecture and kernel.  Specifically, in the
> section titled "GNU Distribution", it says the following:
>
> ‘armhf-linux’
>      ARMv7-A architecture with hard float, Thumb-2 and NEON, using the
>      EABI hard-float application binary interface (ABI), and Linux-Libre
>      kernel.
>
> ‘aarch64-linux’
>      little-endian 64-bit ARMv8-A processors, Linux-Libre kernel.  This
>      is currently in an experimental stage, with limited support.  *Note
>      Contributing::, for how to help!
>
> ‘mips64el-linux’
>      little-endian 64-bit MIPS processors, specifically the Loongson
>      series, n32 ABI, and Linux-Libre kernel.
>
>
> Is this langugae too specific?

I think so, yes.

Then again we could use different system strings to different ABIs, but
we don’t have to.

Ludo’.