Installing cross-compiled packages alongside a cross-compile toolchain

Installing cross-compiled packages alongside a cross-compile toolchain

[Christoph Buck]

Hi!

Total guix newbie here. Currently i am evaluating if guix could be a
better alternative to yocto. My use case is as follows. I want to
declarative define a system (for aarch64), generate an image and deploy
it by flashing it to an sd card. Additionally i want to setup a
development environment on my x64 dev machine, which i want to use to
develop native applications for the aarch64 target. For this i need a
cross-compile gcc toolchain (host: x64 target: aarch64) and
cross-compiled packages of all libraries my application will use. Sounds
like the perfect use-case for the `guix shell` command.

The first part (generating the system image) works. I managed to get the
second part running as well, but only after alot of struggle and i am
wondering if there is a simpler approach than the one i came up with.

The first thing i tried was using the `--system=aarch64-linux` flag in
the `guix shell` command, i.e `guix shell --system=aarch64-linux -m
manfest.scm` where `manfest.scm` defines the compiler and all libraries
my application will use. In theory this works but not only the libraries
are installed as binaries for `aarch64` but the compiler as well. This
means that in order to compile my application, the compiler must be run
via qemu, which is too slow for my use case. Next, i tried to setup a
native (x64) `aarch64-linux-gnu-` cross-compile toolchain, using the
`cross-gcc-toolchain` api in `cross-base.scm`. Besides running into the
bug https://issues.guix.gnu.org/68058#1-lineno32 i got it to work with
the following definition im my manifest file:


> (define gcc-cross-aarch64-linux-toolchain
>   (let ((chain (cross-gcc-toolchain "aarch64-linux-gnu")))
>     (package
>       (inherit chain)
>       (native-search-paths
>        (package-search-paths
>         (lookup-package-input chain "gcc-cross-aarch64-linux-gnu"))))))
>
> (packages->manifest `(,gcc-cross-aarch64-linux-toolchain ,libpng))

This gives me a cross-compile toolchain but my dependency libpng is
installed as x86-64bit version and i am unable to link againt it. This
raises the question, how to install a native cross-compile toolchain
next to libraries for the target platform. After a lot of struggle, i
came up with the following solution which is inspired by the gnu guix
cookbook (the section about ci, i can't link atm because gnu.org seems
to be down).

> (define* (enable-cross-compilation entry system target)
>   (manifest-entry
>     (inherit entry)
>     (name (string-append (manifest-entry-name entry) "." system
>                          (if target
>                              (string-append "." target)
>                              "")))
>     (item (with-parameters ((%current-system system)
>                             (%current-target-system target))
>             (manifest-entry-item entry)))
>     (dependencies (if (not (null? (manifest-entry-dependencies entry)))
>                       (map (lambda (dep) (enable-cross-compilation dep system target))
>                            (manifest-entry-dependencies entry))
>                       '()))))


> (define (cross-build-package package)
>   (manifest (list (enable-cross-compilation (package->manifest-entry package) "x86_64-linux" "aarch64-linux-gnu"))))

> (concatenate-manifests (list (specifications->manifest '("cmake"))
>                              (cross-build-package libpng)
>                              (packages->manifest `(,gcc-cross-aarch64-linux-toolchain)))) 

From my rudimentary understanding of guile, this goes recursively
through all manifest entries of the wrapped dependency and resets the
`target` to "aarch64-linux-gnu". And indeed, this seems to work:

> file /gnu/store/kh7kl57h5i3vzx9hbbairnkkgnx7kf61-gcc-cross-aarch64-linux-gnu-11.3.0/bin/aarch64-linux-gnu-gcc
> /gnu/store/kh7kl57h5i3vzx9hbbairnkkgnx7kf61-gcc-cross-aarch64-linux-gnu-11.3.0/bin/aarch64-linux-gnu-gcc: ELF 64-bit LSB executable, x86-64, version 1 (GNU/Linux), dynamically linked, interpreter /gnu/store/gsjczqir1wbz8p770zndrpw4rnppmxi3-glibc-2.35/lib/ld-linux-x86-64.so.2, for GNU/Linux 2.6.32, stripped

> file /gnu/store/798dl8m0mxigjc8w3fh36iqiq0sfm8ah-libpng-1.6.37/lib/libpng16.so.16.37.0
> /gnu/store/798dl8m0mxigjc8w3fh36iqiq0sfm8ah-libpng-1.6.37/lib/libpng16.so.16.37.0: ELF 64-bit LSB shared object, ARM aarch64, version 1 (SYSV), dynamically linked, stripped

Is this the correct way to set this up? Is there a simpler way? It is
such a common problem and my solution looks rather complicated. Are
there any insights and tips of people with similiar uses cases?

Best regards

Christoph 







-- 

Re: Installing cross-compiled packages alongside a cross-compile toolchain

[Thiago Jung Bauermann]

Hello Christoph,

Christoph Buck <buck.christoph@googlemail.com> writes:

> Total guix newbie here. Currently i am evaluating if guix could be a
> better alternative to yocto. My use case is as follows. I want to
> declarative define a system (for aarch64), generate an image and deploy
> it by flashing it to an sd card. Additionally i want to setup a
> development environment on my x64 dev machine, which i want to use to
> develop native applications for the aarch64 target. For this i need a
> cross-compile gcc toolchain (host: x64 target: aarch64) and
> cross-compiled packages of all libraries my application will use. Sounds
> like the perfect use-case for the `guix shell` command.

I'm very interested in the second part as well but unfortunately it's
not something supported by Guix without resorting to hacks, as you found
out. I never had the time and/or energy to dig deep into it. You went a
lot farther than I did!

This is a pity, because it doesn't look like this use-case would be hard
to support in Guix and I think it would be a killer feature for
developers, allowing it to gain more popularity as a development tool.

> Is this the correct way to set this up? Is there a simpler way? It is
> such a common problem and my solution looks rather complicated. Are
> there any insights and tips of people with similiar uses cases?

I don't know the answer to these questions, but your solution is
ingenious!

-- 
Thiago

Re: Installing cross-compiled packages alongside a cross-compile toolchain

[Christoph Buck]

Thiago Jung Bauermann <bauermann@kolabnow.com> writes:

> Hello Christoph,

Hi Thiago!

> This is a pity, because it doesn't look like this use-case would be hard
> to support in Guix and I think it would be a killer feature for
> developers, allowing it to gain more popularity as a development tool.

I think so too. Compaired to the horrible mess of a "language" yocto
uses, guile scheme is really an improvement here. Unfortunatly
cross-compiling still seems to be experimental. I ran into a lot of
issues getting even the basic setup to work (for a list look at the
response to Richard, even though my mail seems to be lost). Definitly
not production ready but the potential is there.


-- 
Best regards

Christoph

Re: Installing cross-compiled packages alongside a cross-compile toolchain

[Richard Sent]

Hi Christoph!

Christoph Buck <buck.christoph@googlemail.com> writes:

>> (define gcc-cross-aarch64-linux-toolchain
>>   (let ((chain (cross-gcc-toolchain "aarch64-linux-gnu")))
>>     (package
>>       (inherit chain)
>>       (native-search-paths
>>        (package-search-paths
>>         (lookup-package-input chain "gcc-cross-aarch64-linux-gnu"))))))
>>
>> (packages->manifest `(,gcc-cross-aarch64-linux-toolchain ,libpng))
>> 
>> (define* (enable-cross-compilation entry system target)
>>   (manifest-entry
>>     (inherit entry)
>>     (name (string-append (manifest-entry-name entry) "." system
>>                          (if target
>>                              (string-append "." target)
>>                              "")))
>>     (item (with-parameters ((%current-system system)
>>                             (%current-target-system target))
>>             (manifest-entry-item entry)))
>>     (dependencies (if (not (null? (manifest-entry-dependencies entry)))
>>                       (map (lambda (dep) (enable-cross-compilation dep system target))
>>                            (manifest-entry-dependencies entry))
>>                       '()))))
>> 
>> (define (cross-build-package package)
>>   (manifest (list (enable-cross-compilation (package->manifest-entry package) "x86_64-linux" "aarch64-linux-gnu"))))
>
>> (concatenate-manifests (list (specifications->manifest '("cmake"))
>>                              (cross-build-package libpng)
>>                              (packages->manifest `(,gcc-cross-aarch64-linux-toolchain)))) 
>
> From my rudimentary understanding of guile, this goes recursively
> through all manifest entries of the wrapped dependency and resets the
> `target` to "aarch64-linux-gnu". And indeed, this seems to work:
> 
> Is this the correct way to set this up? Is there a simpler way? It is
> such a common problem and my solution looks rather complicated. Are
> there any insights and tips of people with similiar uses cases?

I also unfortunately don't know of a better solution. I'm not fluent
with how cross-compilation works in Guix, but I think it would be
valuable proposing this (or something like it) as a patch and/or
cookbook entry! It seems too useful to leave hidden away in the mailing
list. (I assume your inspiration was [1], but in my opinion this is
different enough in both context and content to deserve its own
section.)

The closest I can think of is wrapping your Guix software as a package
and using --target, but this is often too much of a barrier to entry
when all you want is a simple development environment. Anything making
cross-compilation easier is a good thing in my eyes.

[1]: https://guix.gnu.org/en/cookbook/en/guix-cookbook.html#Build-Manifest

-- 
Take it easy,
Richard Sent
Making my computer weirder one commit at a time.

Re: Installing cross-compiled packages alongside a cross-compile toolchain

[Christoph Buck]

Richard Sent <richard@freakingpenguin.com> writes:

> Hi Christoph!

Hi Richard!

Christoph here (writing from a different mail address to up my mailing
list game).

> (I assume your inspiration was [1], but in my opinion this is
> different enough in both context and content to deserve its own
> section.)
> [...]
> [1]: https://guix.gnu.org/en/cookbook/en/guix-cookbook.html#Build-Manifest

Exactly. This was my inspiration. 

> I also unfortunately don't know of a better solution. I'm not fluent
> with how cross-compilation works in Guix, but I think it would be
> valuable proposing this (or something like it) as a patch and/or
> cookbook entry! It seems too useful to leave hidden away in the mailing
> list.

I agree. If my hack is the technical correct way to achieve this, it
should probably be documented somewhere. I took me (a newbe) some time
to figure it out. I certainly can help writing an entry for the
guix cookbook if desired, but i am not sure if i am the correct person
to do so. My technical understanding and jargon of guix is currently
quite limited. I don't even understand my hack in full, it was more or
less try and error and guestimate engineering.

> The closest I can think of is wrapping your Guix software as a package
> and using --target, but this is often too much of a barrier to entry
> when all you want is a simple development environment. 

Interestingly there is no `--target` option for the `guix shell`
command, only the `--system` option, which result in using qemu to
emulate the target platform. I guess because it usually does not make
sense to instantiate a development environment with cross-compiled
packages which can't be used/executed on the host platfom if host !=
target platform?

If i had to come up with a feature to support native cross-build
toolchains in `guix shell` i would probably add the `target` option.
This option would instantiate the gcc toolchain with a cross-compile
configuration for the host platform, but treat any package as
cross-compiled version, so that i can specify the additional
dependencies in a manifest file like i normally would.

> Anything making cross-compilation easier is a good thing in my eyes.

Yes i agree. Unfortunately cross-compilation still seems to be very
"experimental". I stumpled upon multiple bugs ([1],[2],[3]) in order to
get it to work at all. Luckily they were well known and documented
([1][2]) or easy to fix. ([3]).

All in all not a production ready replacement for yocto, but imho the
potential is there :)

Best regards

Christoph


[1] https://issues.guix.gnu.org/68058
[2] https://issues.guix.gnu.org/66866,
[3] https://issues.guix.gnu.org/71174