* Re: Cross compilation status
2023-09-10 9:14 Cross compilation status Mathieu Othacehe
@ 2023-09-10 9:20 ` Efraim Flashner
2023-09-10 9:27 ` Efraim Flashner
` (4 subsequent siblings)
5 siblings, 0 replies; 10+ messages in thread
From: Efraim Flashner @ 2023-09-10 9:20 UTC (permalink / raw)
To: Mathieu Othacehe
Cc: guix-devel, andrew, bjoern.hoefling, guix, julien,
cox.katherine.e+guix, lars, liliana.prikler, rekado, jgart
[-- Attachment #1: Type: text/plain, Size: 907 bytes --]
On Sun, Sep 10, 2023 at 11:14:29AM +0200, Mathieu Othacehe wrote:
>
> I have CC'ed members of the python, java, ruby, rust, r, haskell and
> emacs teams. Any plans adding cross-compilation support to your
> build-system, barriers to overcome?
Speaking as the one who added the rakudo build system and the perl6
packages, IMO all the perl6 packages should be removed and also the
rakudo-build-system. I don't believe most of the packages have built for
more than a year and I don't believe they have ever had any adoption. If
it turns out that Raku (no longer perl6 for several years now) would
make use of the current rakudo-build-system then we can always revert
removing it.
--
Efraim Flashner <efraim@flashner.co.il> רנשלפ םירפא
GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Cross compilation status
2023-09-10 9:14 Cross compilation status Mathieu Othacehe
2023-09-10 9:20 ` Efraim Flashner
@ 2023-09-10 9:27 ` Efraim Flashner
2023-09-10 12:55 ` Maxim Cournoyer
` (3 subsequent siblings)
5 siblings, 0 replies; 10+ messages in thread
From: Efraim Flashner @ 2023-09-10 9:27 UTC (permalink / raw)
To: Mathieu Othacehe
Cc: guix-devel, andrew, bjoern.hoefling, guix, julien,
cox.katherine.e+guix, lars, liliana.prikler, rekado, jgart
[-- Attachment #1: Type: text/plain, Size: 2647 bytes --]
On Sun, Sep 10, 2023 at 11:14:29AM +0200, Mathieu Othacehe wrote:
>
> Hello,
>
> In order for Guix to become an alternative to tools such as Yocto and
> Buildroot, having most or all our packages cross-compiling is a
> prerequisite.
>
> Here is a status of cross-compilation in Guix. For cross-compilation to
> work, the build-system needs to support cross-compilation.
>
> The following build-systems explicitly refuse cross-compiling packages:
>
<snip>
>
> ocaml: 61
> haskell: 812
> maven: 2
> chicken: 12
> node: 57
> emacs: 1370
> dune: 289
> android-ndk: 12
> waf: 21
> julia: 300
> pyproject: 433
> r: 2441
> cargo: 3535
> ruby: 597
> rebar: 22
> scons: 15
> rakudo: 22
> agda: 6
> ant: 559
> python: 2488
> Cross-compilation KO: 13054
> --8<---------------cut here---------------end--------------->8---
>
> Over the 28076 packages in Guix, 15065 are part of build-systems with
> cross-compilation support and 13054 are part of build-systems without
> cross-compilation support.
>
> Overall 46.5% of our packages will refuse to cross-compile with errors
> such as:
>
> --8<---------------cut here---------------start------------->8---
> mathieu@meije ~$ guix build --target=aarch64-linux-gnu librsvg
> guix build: error: gnu/packages/gnome.scm:3500:2: librsvg@2.54.5: build system `cargo' does not support cross builds
> --8<---------------cut here---------------end--------------->8---
>
> I'd like to help converge towards the situation where all build-systems
> support cross-compilation.
>
> I have CC'ed members of the python, java, ruby, rust, r, haskell and
> emacs teams. Any plans adding cross-compilation support to your
> build-system, barriers to overcome?
I suspect librsvg is especially a problem since it is almost always
pulled in with any sort of image building. I haven't looked at it
closely yet, but I believe that in order to support cross-compilation
support in rust we'll need to enable more targets, which will increase
the build time for the final rust and the final output size. Considering
that nothing built with rust links to it, I'm not convinced this is a
problem.
We will likely see people confused that we have a rust with
cross-compiling abilities but with windows- and mac-specific
(pre-compiled) libraries removed, so it wouldn't be able to
cross-compile to those (or some other) platforms without changes.
--
Efraim Flashner <efraim@flashner.co.il> רנשלפ םירפא
GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Cross compilation status
2023-09-10 9:14 Cross compilation status Mathieu Othacehe
2023-09-10 9:20 ` Efraim Flashner
2023-09-10 9:27 ` Efraim Flashner
@ 2023-09-10 12:55 ` Maxim Cournoyer
2023-09-10 14:59 ` Liliana Marie Prikler
` (2 subsequent siblings)
5 siblings, 0 replies; 10+ messages in thread
From: Maxim Cournoyer @ 2023-09-10 12:55 UTC (permalink / raw)
To: Mathieu Othacehe
Cc: guix-devel, andrew, bjoern.hoefling, guix, efraim, julien,
cox.katherine.e+guix, lars, liliana.prikler, rekado, jgart
Hi Mathieu!
Mathieu Othacehe <othacehe@gnu.org> writes:
> Hello,
>
> In order for Guix to become an alternative to tools such as Yocto and
> Buildroot, having most or all our packages cross-compiling is a
> prerequisite.
>
> Here is a status of cross-compilation in Guix. For cross-compilation to
> work, the build-system needs to support cross-compilation.
>
> The following build-systems explicitly refuse cross-compiling packages:
>
> haskell, agda, waf, chicken, rakudo, julia, python, emacs, rebar,
> cargo, ruby, renpy, dub, android-ndk, scons, dune, ant, pyproject,
> maven, asdf, r, ocaml, node
For Python, it's already reported at: https://issues.guix.gnu.org/60733.
And even implemented here: https://issues.guix.gnu.org/60849.
Why it hasn't been merged yet is because it's coupled with a more daring
change (unified build system for Python) and there were comments that
this should be an all or nothing change, so I'd need to workout all the
build systems and see what breakage happens at the global scale, which
is a rather large undertaking.
Maybe it could be decoupled from that change and tested in isolation,
pushed first; we'd loose the benefit of the later change which is the
ability to avoid wrapping native-inputs in our wrappers.
--
Thanks,
Maxim
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Cross compilation status
2023-09-10 9:14 Cross compilation status Mathieu Othacehe
` (2 preceding siblings ...)
2023-09-10 12:55 ` Maxim Cournoyer
@ 2023-09-10 14:59 ` Liliana Marie Prikler
2023-09-18 14:17 ` Efraim Flashner
2023-09-11 16:19 ` Simon Tournier
2023-09-12 15:05 ` Guillaume Le Vaillant
5 siblings, 1 reply; 10+ messages in thread
From: Liliana Marie Prikler @ 2023-09-10 14:59 UTC (permalink / raw)
To: Mathieu Othacehe, guix-devel
Cc: andrew, bjoern.hoefling, guix, efraim, julien,
cox.katherine.e+guix, lars, rekado, jgart
Hi Mathieu,
Am Sonntag, dem 10.09.2023 um 11:14 +0200 schrieb Mathieu Othacehe:
> I have CC'ed members of the python, java, ruby, rust, r, haskell and
> emacs teams. Any plans adding cross-compilation support to your
> build-system, barriers to overcome?
For Emacs, there are layers of problems. First things first, we are
only doing byte compilation atm, which may or may not be arch-dependant
based on stuff like endianness. For native compilation, there is the
thing that even a slightly differently configured emacs won't load the
packages you've just compiled without compiling it natively on its own.
Granted, if we want to do cross-compilation, having just the bytecode
might already be "good enough", but I wanted to throw this out there.
I think we'd need some support from Emacs proper before we can start
implementing cross builds.
Cheers
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Cross compilation status
2023-09-10 14:59 ` Liliana Marie Prikler
@ 2023-09-18 14:17 ` Efraim Flashner
2023-09-18 17:01 ` Liliana Marie Prikler
0 siblings, 1 reply; 10+ messages in thread
From: Efraim Flashner @ 2023-09-18 14:17 UTC (permalink / raw)
To: Liliana Marie Prikler
Cc: Mathieu Othacehe, guix-devel, andrew, bjoern.hoefling, guix,
julien, cox.katherine.e+guix, lars, rekado, jgart
[-- Attachment #1: Type: text/plain, Size: 1333 bytes --]
On Sun, Sep 10, 2023 at 04:59:35PM +0200, Liliana Marie Prikler wrote:
> Hi Mathieu,
>
> Am Sonntag, dem 10.09.2023 um 11:14 +0200 schrieb Mathieu Othacehe:
> > I have CC'ed members of the python, java, ruby, rust, r, haskell and
> > emacs teams. Any plans adding cross-compilation support to your
> > build-system, barriers to overcome?
> For Emacs, there are layers of problems. First things first, we are
> only doing byte compilation atm, which may or may not be arch-dependant
> based on stuff like endianness. For native compilation, there is the
> thing that even a slightly differently configured emacs won't load the
> packages you've just compiled without compiling it natively on its own.
> Granted, if we want to do cross-compilation, having just the bytecode
> might already be "good enough", but I wanted to throw this out there.
>
> I think we'd need some support from Emacs proper before we can start
> implementing cross builds.
I will note that the only big-endian architecture which we "support" is
32-bit powerpc, so we don't have to worry that much about wrong
endianness.
--
Efraim Flashner <efraim@flashner.co.il> רנשלפ םירפא
GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Cross compilation status
2023-09-18 14:17 ` Efraim Flashner
@ 2023-09-18 17:01 ` Liliana Marie Prikler
0 siblings, 0 replies; 10+ messages in thread
From: Liliana Marie Prikler @ 2023-09-18 17:01 UTC (permalink / raw)
To: Efraim Flashner
Cc: Mathieu Othacehe, guix-devel, andrew, bjoern.hoefling, guix,
julien, cox.katherine.e+guix, lars, rekado, jgart
Am Montag, dem 18.09.2023 um 17:17 +0300 schrieb Efraim Flashner:
> On Sun, Sep 10, 2023 at 04:59:35PM +0200, Liliana Marie Prikler
> wrote:
> > Hi Mathieu,
> >
> > Am Sonntag, dem 10.09.2023 um 11:14 +0200 schrieb Mathieu Othacehe:
> > > I have CC'ed members of the python, java, ruby, rust, r, haskell
> > > and
> > > emacs teams. Any plans adding cross-compilation support to your
> > > build-system, barriers to overcome?
> > For Emacs, there are layers of problems. First things first, we
> > are only doing byte compilation atm, which may or may not be
> > arch-dependant based on stuff like endianness. For native
> > compilation, there is the thing that even a slightly differently
> > configured emacs won't load the packages you've just compiled
> > without compiling it natively on its own.
> > Granted, if we want to do cross-compilation, having just the
> > bytecode might already be "good enough", but I wanted to throw this
> > out there.
> >
> > I think we'd need some support from Emacs proper before we can
> > start implementing cross builds.
>
> I will note that the only big-endian architecture which we "support"
> is 32-bit powerpc, so we don't have to worry that much about wrong
> endianness.
Fine, let's add word size to point out the obvious :)
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Cross compilation status
2023-09-10 9:14 Cross compilation status Mathieu Othacehe
` (3 preceding siblings ...)
2023-09-10 14:59 ` Liliana Marie Prikler
@ 2023-09-11 16:19 ` Simon Tournier
2023-09-11 17:10 ` Liliana Marie Prikler
2023-09-12 15:05 ` Guillaume Le Vaillant
5 siblings, 1 reply; 10+ messages in thread
From: Simon Tournier @ 2023-09-11 16:19 UTC (permalink / raw)
To: Mathieu Othacehe, guix-devel
Cc: andrew, bjoern.hoefling, guix, efraim, julien,
cox.katherine.e+guix, lars, liliana.prikler, rekado, jgart
Hi Mathieu,
On Sun, 10 Sep 2023 at 11:14, Mathieu Othacehe <othacehe@gnu.org> wrote:
> Here is a status of cross-compilation in Guix. For cross-compilation to
> work, the build-system needs to support cross-compilation.
>
> The following build-systems explicitly refuse cross-compiling packages:
>
> haskell, agda, waf, chicken, rakudo, julia, python, emacs, rebar, cargo, ruby, renpy, dub, android-ndk, scons, dune, ant, pyproject, maven, asdf, r, ocaml, node
>
[...]
> --8<---------------cut here---------------start------------->8---
> Total packages: 28076
>
> clojure: 13
> qt: 317
> copy: 134
> minetest-mod: 22
> tree-sitter: 35
> raw: 2
> linux-module: 17
> glib-or-gtk: 139
> asdf/source: 794
> go: 632
> cmake: 1278
> minify: 12
> perl: 839
> trivial: 250
> guile: 49
> elm: 30
> font: 101
> gnu: 4178
> asdf/ecl: 753
> asdf/sbcl: 814
> meson: 504
> mozilla: 2
> texlive: 4150
> Cross-compilation OK: 15065
>
> ocaml: 61
> haskell: 812
> maven: 2
> chicken: 12
> node: 57
> emacs: 1370
> dune: 289
> android-ndk: 12
> waf: 21
> julia: 300
> pyproject: 433
> r: 2441
> cargo: 3535
> ruby: 597
> rebar: 22
> scons: 15
> rakudo: 22
> agda: 6
> ant: 559
> python: 2488
> Cross-compilation KO: 13054
> --8<---------------cut here---------------end--------------->8---
The build-systems renpy, dub is listed in “refuse“ but not then in
Cross-compilation KO. Is it expected?
The build-system asdf is listed as refuse but appears in the list
Cross-compilation OK. Is it expected?
> Over the 28076 packages in Guix, 15065 are part of build-systems with
> cross-compilation support and 13054 are part of build-systems without
> cross-compilation support.
>
> Overall 46.5% of our packages will refuse to cross-compile with errors
> such as:
>
> --8<---------------cut here---------------start------------->8---
> mathieu@meije ~$ guix build --target=aarch64-linux-gnu librsvg
> guix build: error: gnu/packages/gnome.scm:3500:2: librsvg@2.54.5: build system `cargo' does not support cross builds
> --8<---------------cut here---------------end--------------->8---
>
> I'd like to help converge towards the situation where all build-systems
> support cross-compilation.
Considering Julia, the “julia compiler” support of upstream [1] reads:
+ ARMv8 (64-bit): Tier 2
+ ARMv7 (32-bit): Tier 3
+ PowerPC (64-bit): Tier 3
+ then stuff about CUDA, ROCM, oneAPI
where:
+ Tier 2: Julia is guaranteed to build from source using the default
build options, but may or may not pass all tests. Official binaries are
available on a case-by-case basis.
+ Tier 3: Julia may or may not build. If it does, it is unlikely to
pass tests. Binaries may be available in some cases. When they are,
they should be considered experimental. Ongoing support is dependent on
community efforts.
But I guess, these are about native compilation. To my tiny knowledge
on that topic, Julia supports some cross-compilation. However, Julia
depends on heavy packages as ’suitesparse’ or ’openblas’ and I do not
know if we already cross-compile them. Since Julia is used for
scientific computations, I do not know if the effort is worth.
1: https://julialang.org/downloads/
> I have CC'ed members of the python, java, ruby, rust, r, haskell and
> emacs teams. Any plans adding cross-compilation support to your
> build-system, barriers to overcome?
About Haskell, from experience, we are already not able to maintain an
usable ecosystem for i686, but that’s another story. :-)
Well, I do not know if cross-compilation is well-supported by the
Haskell compiler GHC.
Maybe some resources seems there:
https://github.com/input-output-hk/haskell.nix
About R, similarly as Julia, since they are mainly used for scientific
computations, I do not know if supporting cross-compilation is worth our
effort.
Cheers,
simon
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Cross compilation status
2023-09-11 16:19 ` Simon Tournier
@ 2023-09-11 17:10 ` Liliana Marie Prikler
0 siblings, 0 replies; 10+ messages in thread
From: Liliana Marie Prikler @ 2023-09-11 17:10 UTC (permalink / raw)
To: Simon Tournier, Mathieu Othacehe, guix-devel
Cc: andrew, bjoern.hoefling, guix, efraim, julien,
cox.katherine.e+guix, lars, rekado, jgart
Am Montag, dem 11.09.2023 um 18:19 +0200 schrieb Simon Tournier:
> Hi Mathieu,
>
> On Sun, 10 Sep 2023 at 11:14, Mathieu Othacehe <othacehe@gnu.org>
> wrote:
>
> > Here is a status of cross-compilation in Guix. For cross-
> > compilation to
> > work, the build-system needs to support cross-compilation.
> >
> > The following build-systems explicitly refuse cross-compiling
> > packages:
> >
> > haskell, agda, waf, chicken, rakudo, julia, python, emacs, rebar,
> > cargo, ruby, renpy, dub, android-ndk, scons, dune, ant, pyproject,
> > maven, asdf, r, ocaml, node
> >
>
> [...]
>
> > --8<---------------cut here---------------start------------->8---
> > Total packages: 28076
> >
> > clojure: 13
> > qt: 317
> > copy: 134
> > minetest-mod: 22
> > tree-sitter: 35
> > raw: 2
> > linux-module: 17
> > glib-or-gtk: 139
> > asdf/source: 794
> > go: 632
> > cmake: 1278
> > minify: 12
> > perl: 839
> > trivial: 250
> > guile: 49
> > elm: 30
> > font: 101
> > gnu: 4178
> > asdf/ecl: 753
> > asdf/sbcl: 814
> > meson: 504
> > mozilla: 2
> > texlive: 4150
> > Cross-compilation OK: 15065
> >
> > ocaml: 61
> > haskell: 812
> > maven: 2
> > chicken: 12
> > node: 57
> > emacs: 1370
> > dune: 289
> > android-ndk: 12
> > waf: 21
> > julia: 300
> > pyproject: 433
> > r: 2441
> > cargo: 3535
> > ruby: 597
> > rebar: 22
> > scons: 15
> > rakudo: 22
> > agda: 6
> > ant: 559
> > python: 2488
> > Cross-compilation KO: 13054
> > --8<---------------cut here---------------end--------------->8---
>
> The build-systems renpy, dub is listed in “refuse“ but not then in
> Cross-compilation KO. Is it expected?
On the renpy side, there currently are no packages to cross-compile.
"The Question", which ships with renpy, is built as an output rather
than with renpy-build-system and was only used as a proof of concept
back then. If I do encounter a renpy-based game that I actually can
get the sources for, I'll gladly package it.
HTH
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Cross compilation status
2023-09-10 9:14 Cross compilation status Mathieu Othacehe
` (4 preceding siblings ...)
2023-09-11 16:19 ` Simon Tournier
@ 2023-09-12 15:05 ` Guillaume Le Vaillant
5 siblings, 0 replies; 10+ messages in thread
From: Guillaume Le Vaillant @ 2023-09-12 15:05 UTC (permalink / raw)
To: Mathieu Othacehe; +Cc: guix-devel
[-- Attachment #1: Type: text/plain, Size: 1657 bytes --]
Mathieu Othacehe <othacehe@gnu.org> skribis:
> In order for Guix to become an alternative to tools such as Yocto and
> Buildroot, having most or all our packages cross-compiling is a
> prerequisite.
>
> Here is a status of cross-compilation in Guix. For cross-compilation to
> work, the build-system needs to support cross-compilation.
>
> The following build-systems explicitly refuse cross-compiling packages:
>
> haskell, agda, waf, chicken, rakudo, julia, python, emacs, rebar, cargo, ruby, renpy, dub, android-ndk, scons, dune, ant, pyproject, maven, asdf, r, ocaml, node
>
> while the rest of the build-systems do accept cross-compiling packages:
>
> clojure, qt, copy, minetest-mod, tree-sitter, raw, linux-module, glib-or-gtk,
> asdf/source, go, cmake, minify, perl, trivial, guile, elm, font, gnu, asdf/ecl,
> asdf/sbcl, meson, mozilla, texlive
Hi.
I'm surprised to see asdf/* in the list of build systems accepting
to cross-compile packages. How did you test them?
Because I get (on a x86-64 machine):
--8<---------------cut here---------------start------------->8---
$ guix build --target=aarch64-linux-gnu sbcl-alexandria
guix build: error: gnu/packages/lisp-xyz.scm:168:2: sbcl-alexandria@1.4: build system `asdf/sbcl' does not support cross builds
$ guix build --target=aarch64-linux-gnu ecl-alexandria
guix build: error: gnu/packages/lisp-xyz.scm:168:2: ecl-alexandria@1.4: build system `asdf/ecl' does not support cross builds
--8<---------------cut here---------------end--------------->8---
I'm not even sure if sbcl and ecl have the ability to compile Common
Lisp code for an architecture different from the one they are running
on.
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 247 bytes --]
^ permalink raw reply [flat|nested] 10+ messages in thread