Re: A different way to build GCC to overcome issues, especially with C++ for embedded systems

Re: A different way to build GCC to overcome issues, especially with C++ for embedded systems

[Sergio Pastor Pérez]

Hello. I'm very interested in the code for ZMK provided in this
thread[1].

I've tried it locally and it compiles successfully. Does anyone know if
this is available in a public repository? Or if it has been moved
forward?

Anyone here have tried to do something with ZMK? I'm interested in what
would be the Guix approach for ZMK development.

The blog post of Zephyr[2] was a very interesting read, does anyone know
of other resources regarding this topic?

Have a great day!
Sergio.

[1]: https://lists.gnu.org/archive/html/guix-devel/2023-10/msg00014.html
[2]: https://guix.gnu.org/en/blog/2023/building-toolchains-with-guix

Re: A different way to build GCC to overcome issues, especially with C++ for embedded systems

[Stefan]

[-- Attachment #1: Type: text/plain, Size: 1558 bytes --]

Hi Sergio!

> I'm very interested in the code for ZMK provided in this thread[1].
> 
> I've tried it locally and it compiles successfully. 

That's nice to hear!

> Does anyone know if this is available in a public repository? Or if it has been moved forward?

There is no public repository for it. I moved it a bit further. There is now a GCC with picolibc. GCC in general is more self-contained and is completely usable by itself (including paths to specs; Attila, this may solve the issue you were facing). There is no real need for GCC-toolchain packages anymore. However, as some projects invoke programs of Binutils directly and because of locales, there are still GCC-toolchain packages. The Binutils for arm-none-eabi get build separately as well now, as the one from Guix has failing tests (if activated) due to the selected configure flags. GCC is bumped to 13.2.0. Zephyr is updated to 3.5 and ZMK to end of April. There is also more ZMK support for combos, conditional-layers and a bit more.

A problem I figured for arm-none-eabi-g++ is that it always links the generic libstdc++.a, instead of the existing CPU and FPU specific multilib variant from a sub-directory. I'd appreciate any GCC configuration hint to solve this.

> Anyone here have tried to do something with ZMK? I'm interested in what would be the Guix approach for ZMK development.
> 
> The blog post of Zephyr[2] was a very interesting read, does anyone know of other resources regarding this topic?

I'm not aware of anything else regarding ZMK or Zephyr in Guix.


Bye

Stefan

[-- Attachment #2: embedded.tar.gz --]
[-- Type: application/gzip, Size: 29929 bytes --]

Re: A different way to build GCC to overcome issues, especially with C++ for embedded systems

[Attila Lendvai]

hi Stefan,


> > Does anyone know if this is available in a public repository? Or if it has been moved forward?
>
>
> There is no public repository for it.


this is a much more valuable piece of contribution than to allow it to hang in the void indefinitely!

if the only reason that you have not made a channel for this yet is that you've never done it before, then i'd be happy to walk you through it off list.

or i can help you setting up a guix fork where you can push your own signed commits, and guix pull from.

or if you don't mind that the initial commit will be signed by me, then i can even set up a guix channel on codeberg, give you commit access, and then you can push any further changes there.

just let me know.

--
• attila lendvai
• PGP: 963F 5D5F 45C7 DFCD 0A39
--
“Fear is not in the habit of speaking truth; when perfect sincerity is expected, perfect freedom must be allowed; nor has anyone, who is apt to be angry when he hears the truth, any cause to wonder that he does not hear it.”
	— Cornelius Tacitus (ca. 56–117)

Re: A different way to build GCC to overcome issues, especially with C++ for embedded systems

[Sergio Pastor Pérez]

Hi both of you.

I want to echo Attila's sentiments. This is a valuable contribution, and
creating a channel would serve as a central hub for other contributors.

Thanks for sharing, Stefan.

Have nice day.
Sergio.

Re: A different way to build GCC to overcome issues, especially with C++ for embedded systems

[Jean-Pierre De Jesus Diaz]

Hello,

Adding to this conversation I have been making a arm-none-eabi
toolchain with Newlib included as the default C standard library like
in the official Arm toolchain but I'm not quite happy with the patches
yet.[1]

I haven't contributed it yet because it has been a bit hard adapting axoloti-*
packages to use a modern toolchain because I'm also intending to remove
the old toolchains in `gnu/packages/embedded.scm' eventually and
instead use the ones in `cross-gcc-toolchain'.

[1]: https://github.com/Foundation-Devices/guix-mirror/compare/master...wip/arm-none-eabi

On Fri, May 24, 2024 at 3:53 PM Sergio Pastor Pérez
<sergio.pastorperez@outlook.es> wrote:
>
> Hi both of you.
>
> I want to echo Attila's sentiments. This is a valuable contribution, and
> creating a channel would serve as a central hub for other contributors.
>
> Thanks for sharing, Stefan.
>
> Have nice day.
> Sergio.
>

Re: A different way to build GCC to overcome issues, especially with C++ for embedded systems

[Ricardo Wurmus]

Jean-Pierre De Jesus Diaz <jean@foundation.xyz> writes:

> I haven't contributed it yet because it has been a bit hard adapting axoloti-*
> packages to use a modern toolchain because I'm also intending to remove
> the old toolchains in `gnu/packages/embedded.scm' eventually and
> instead use the ones in `cross-gcc-toolchain'.

Could you explain why it is difficult to adjust the axoloti/ksoloti
packages?  They only need an arm-none-eabi cross-compiler with newlib.
Since I have axoloti boards here, I could test your changes.

-- 
Ricardo

Re: A different way to build GCC to overcome issues, especially with C++ for embedded systems

[Jean-Pierre De Jesus Diaz]

>> I haven't contributed it yet because it has been a bit hard adapting axoloti-*
>> packages to use a modern toolchain because I'm also intending to remove
>> the old toolchains in `gnu/packages/embedded.scm' eventually and
>> instead use the ones in `cross-gcc-toolchain'.
>
>Could you explain why it is difficult to adjust the axoloti/ksoloti
>packages?  They only need an arm-none-eabi cross-compiler with newlib.
>Since I have axoloti boards here, I could test your changes.

I'm mostly having issues with refactoring cross-base to use the gcc
4.9 branch for
ARM to compile one of the axoloti packages that need a 4.9 toolchain
as trying to
use a newer one gave me some errors and the default gcc 4.9 doesn't have all of
the multilib targets that axoloti needs IIRC.

On Sat, May 25, 2024 at 10:20 PM Ricardo Wurmus <rekado@elephly.net> wrote:
>
> Jean-Pierre De Jesus Diaz <jean@foundation.xyz> writes:
>
> > I haven't contributed it yet because it has been a bit hard adapting axoloti-*
> > packages to use a modern toolchain because I'm also intending to remove
> > the old toolchains in `gnu/packages/embedded.scm' eventually and
> > instead use the ones in `cross-gcc-toolchain'.
>
> Could you explain why it is difficult to adjust the axoloti/ksoloti
> packages?  They only need an arm-none-eabi cross-compiler with newlib.
> Since I have axoloti boards here, I could test your changes.
>
> --
> Ricardo

A different way to bootstrap and build GCC

[Stefan]

Hi!

I got a step further with the different way to build GCC.  The problem not using the proper multilib variant for embedded systems is solved now and I updated to GCC 14.2.

To cite myself, my gut feeling is that the whole GCC version chain starting in (gnu packages commencement) should be build this way.  So I started to play around with the bootstrapping of GCC.

Starting with only few packages from (gnu packages commencement) I so far managed to build a static GCC 10.4.0 using a static musl 1.2.5.

The initial packages I reuse are:

%bootstrap-guile
gash-boot
bootar
gash-utils-boot
tcc-boot0
gnu-make-mesboot0
gmp-boot
mpfr-boot
mpc-boot


Using only these I build a recent TCC from 2024-08-20 with Mes from tcc-boot0 as C library, and then MUSL 1.2.5. Then (only) three iterations of TCC with musl are needed to get a stable TCC with working floating point support.

The chain continues with GNU Make 4.4.1, Binutils 2.42, Findutils 4.10.0, GCC 4.6.4 with gmp-boot, mpfr-boot, mpc-boot (the version with the RISC-V patches may just work), M4 1.4.19, GMP 6.3.0, MPFR 4.2.1, MPC 1.3.1, GCC 10.4.0.

These are all static builds so far.  I'm using latest versions of all packages, except Binutils, whose version 2.43 does not link the object files from TCC.  I avoid to use --build=i686-unknown-linux-gnu to make it possible to build for other architectures as well.

I think the next step should be GCC 14, then glibc with shared library support and GCC 14 again.

I need several small patches to work around shortcomings in Mes, gash, gash-utils, missing functionality of version 3.8.0 of gnu-make-mesboot0 (version 3.81 would have it), bugs in TCC.  They are all described in the comments.  Maybe gash and gash-utils could be improved in future.  The most annoying thing is that only one core can be used for the builds, otherwise they hang.  I guess it is related to gash in combination with %bootstrap-guile, at least using Make 4.4.1 makes no difference.

I published a git repository at <https://git.pub.solar/stefan/embedded-channel>.  Unfortunately it's not a proper channel yet.  If someone likes to give that bootstrap path a try, use this command:

guix build --cores=1 -L <your-checkout-here> GCC-10-bootstrap

The working GCCs can be build with

guix build -L <your-checkout-here> GCC GCC-cross-picolibc-arm-none-eabi, GCC-cross-newlib-arm-none-eabi

There are GCC…-toolchain package as well, which propagate ld and all the other tools from Binutils.  But if these tools are only used indirectly through the gcc or g++ drivers, these GCC…-toolchain packages are not needed at all, as the GCC packages are standalone.  There are also GCC…-c-toolchain variables for use with package-with-c-toolchain.  There is no separate libstdc++ package yet, the library is just part of GCC.  A separate package will only be needed for Clang or other compilers (maybe Zig), but I'm not sure yet, if clang actually needs a dependency to GCC.

I hope this will be useful, maybe it can help the RISC-V bootstrap effort.  I'm open for suggestions how to proceed from here.


Bye

Stefan

Re: A different way to bootstrap and build GCC

[Ekaitz Zarraga]

Hi!

On 2024-11-24 00:20, Stefan wrote:
> Hi!
> 
> I got a step further with the different way to build GCC.  The problem 
> not using the proper multilib variant for embedded systems is solved now 
> and I updated to GCC 14.2.
> 
> To cite myself, my gut feeling is that the whole GCC version chain 
> starting in (gnu packages commencement) should be build this way.  So I 
> started to play around with the bootstrapping of GCC.
> 
> Starting with only few packages from (gnu packages commencement) I so 
> far managed to build a static GCC 10.4.0 using a static musl 1.2.5.
> 
> The initial packages I reuse are:
> 
> %bootstrap-guile
> gash-boot
> bootar
> gash-utils-boot
> tcc-boot0
> gnu-make-mesboot0
> gmp-boot
> mpfr-boot
> mpc-boot
> 
> 
> Using only these I build a recent TCC from 2024-08-20 with Mes from 
> tcc-boot0 as C library, and then MUSL 1.2.5. Then (only) three 
> iterations of TCC with musl are needed to get a stable TCC with working 
> floating point support.
> 
> The chain continues with GNU Make 4.4.1, Binutils 2.42, Findutils 
> 4.10.0, GCC 4.6.4 with gmp-boot, mpfr-boot, mpc-boot (the version with 
> the RISC-V patches may just work), M4 1.4.19, GMP 6.3.0, MPFR 4.2.1, MPC 
> 1.3.1, GCC 10.4.0.
> 
> These are all static builds so far.  I'm using latest versions of all 
> packages, except Binutils, whose version 2.43 does not link the object 
> files from TCC.  I avoid to use --build=i686-unknown-linux-gnu to make 
> it possible to build for other architectures as well.
> 
> I think the next step should be GCC 14, then glibc with shared library 
> support and GCC 14 again.
> 
> I need several small patches to work around shortcomings in Mes, gash, 
> gash-utils, missing functionality of version 3.8.0 of gnu-make-mesboot0 
> (version 3.81 would have it), bugs in TCC.  They are all described in 
> the comments.  Maybe gash and gash-utils could be improved in future.  
> The most annoying thing is that only one core can be used for the 
> builds, otherwise they hang.  I guess it is related to gash in 
> combination with %bootstrap-guile, at least using Make 4.4.1 makes no 
> difference.
> 
> I published a git repository at 
> <https://git.pub.solar/stefan/embedded-channel>.  Unfortunately it's not 
> a proper channel yet.  If someone likes to give that bootstrap path a 
> try, use this command:
> 
> guix build --cores=1 -L <your-checkout-here> GCC-10-bootstrap
> 
> The working GCCs can be build with
> 
> guix build -L <your-checkout-here> GCC GCC-cross-picolibc-arm-none-eabi, 
> GCC-cross-newlib-arm-none-eabi
> 
> There are GCC…-toolchain package as well, which propagate ld and all the 
> other tools from Binutils.  But if these tools are only used indirectly 
> through the gcc or g++ drivers, these GCC…-toolchain packages are not 
> needed at all, as the GCC packages are standalone.  There are also 
> GCC…-c-toolchain variables for use with package-with-c-toolchain.  There 
> is no separate libstdc++ package yet, the library is just part of GCC.  
> A separate package will only be needed for Clang or other compilers 
> (maybe Zig), but I'm not sure yet, if clang actually needs a dependency 
> to GCC.
> 
> I hope this will be useful, maybe it can help the RISC-V bootstrap 
> effort.  I'm open for suggestions how to proceed from here.
> 
> 
> Bye
> 
> Stefan


Very interesting work. I'll read it with more detail tomorrow but at the 
moment it feels very similar to what we did for RISC-V (which is more or 
less what live-bootstrap does):

https://github.com/ekaitz-zarraga/commencement.scm

Is there any obvious difference in the beginning of the chain that I'm 
missing?

We also agreed in the RISC-V port that using musl makes everything 
easier, mostly because of the RISC-V support is a complicated in 
glibc+gcc couples, but also because Musl is very easy to compile (and 
also read and patch, when needed).

We also went straight for a modern GCC and left many other packages in 
the way.

Efraim is working adapt that into Guix so he probably would have more to 
say about it.

Interesting work, thanks for sharing!

Ekaitz

Re: A different way to bootstrap and build GCC

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 5521 bytes --]

On Sun, Nov 24, 2024 at 12:46:21AM +0100, Ekaitz Zarraga wrote:
> Hi!
> 
> On 2024-11-24 00:20, Stefan wrote:
> > Hi!
> > 
> > I got a step further with the different way to build GCC.  The problem
> > not using the proper multilib variant for embedded systems is solved now
> > and I updated to GCC 14.2.
> > 
> > To cite myself, my gut feeling is that the whole GCC version chain
> > starting in (gnu packages commencement) should be build this way.  So I
> > started to play around with the bootstrapping of GCC.
> > 
> > Starting with only few packages from (gnu packages commencement) I so
> > far managed to build a static GCC 10.4.0 using a static musl 1.2.5.
> > 
> > The initial packages I reuse are:
> > 
> > %bootstrap-guile
> > gash-boot
> > bootar
> > gash-utils-boot
> > tcc-boot0
> > gnu-make-mesboot0
> > gmp-boot
> > mpfr-boot
> > mpc-boot
> > 
> > 
> > Using only these I build a recent TCC from 2024-08-20 with Mes from
> > tcc-boot0 as C library, and then MUSL 1.2.5. Then (only) three
> > iterations of TCC with musl are needed to get a stable TCC with working
> > floating point support.
> > 
> > The chain continues with GNU Make 4.4.1, Binutils 2.42, Findutils
> > 4.10.0, GCC 4.6.4 with gmp-boot, mpfr-boot, mpc-boot (the version with
> > the RISC-V patches may just work), M4 1.4.19, GMP 6.3.0, MPFR 4.2.1, MPC
> > 1.3.1, GCC 10.4.0.
> > 
> > These are all static builds so far.  I'm using latest versions of all
> > packages, except Binutils, whose version 2.43 does not link the object
> > files from TCC.  I avoid to use --build=i686-unknown-linux-gnu to make
> > it possible to build for other architectures as well.
> > 
> > I think the next step should be GCC 14, then glibc with shared library
> > support and GCC 14 again.
> > 
> > I need several small patches to work around shortcomings in Mes, gash,
> > gash-utils, missing functionality of version 3.8.0 of gnu-make-mesboot0
> > (version 3.81 would have it), bugs in TCC.  They are all described in
> > the comments.  Maybe gash and gash-utils could be improved in future.
> > The most annoying thing is that only one core can be used for the
> > builds, otherwise they hang.  I guess it is related to gash in
> > combination with %bootstrap-guile, at least using Make 4.4.1 makes no
> > difference.
> > 
> > I published a git repository at
> > <https://git.pub.solar/stefan/embedded-channel>.  Unfortunately it's not
> > a proper channel yet.  If someone likes to give that bootstrap path a
> > try, use this command:
> > 
> > guix build --cores=1 -L <your-checkout-here> GCC-10-bootstrap
> > 
> > The working GCCs can be build with
> > 
> > guix build -L <your-checkout-here> GCC GCC-cross-picolibc-arm-none-eabi,
> > GCC-cross-newlib-arm-none-eabi
> > 
> > There are GCC…-toolchain package as well, which propagate ld and all the
> > other tools from Binutils.  But if these tools are only used indirectly
> > through the gcc or g++ drivers, these GCC…-toolchain packages are not
> > needed at all, as the GCC packages are standalone.  There are also
> > GCC…-c-toolchain variables for use with package-with-c-toolchain.  There
> > is no separate libstdc++ package yet, the library is just part of GCC.
> > A separate package will only be needed for Clang or other compilers
> > (maybe Zig), but I'm not sure yet, if clang actually needs a dependency
> > to GCC.
> > 
> > I hope this will be useful, maybe it can help the RISC-V bootstrap
> > effort.  I'm open for suggestions how to proceed from here.
> > 
> > 
> > Bye
> > 
> > Stefan
> 
> 
> Very interesting work. I'll read it with more detail tomorrow but at the
> moment it feels very similar to what we did for RISC-V (which is more or
> less what live-bootstrap does):
> 
> https://github.com/ekaitz-zarraga/commencement.scm
> 
> Is there any obvious difference in the beginning of the chain that I'm
> missing?
> 
> We also agreed in the RISC-V port that using musl makes everything easier,
> mostly because of the RISC-V support is a complicated in glibc+gcc couples,
> but also because Musl is very easy to compile (and also read and patch, when
> needed).
> 
> We also went straight for a modern GCC and left many other packages in the
> way.
> 
> Efraim is working adapt that into Guix so he probably would have more to say
> about it.
> 
> Interesting work, thanks for sharing!

Indeed, thank you very much!

In terms of integrating the riscv64 bootstrap work I had gotten stuck
building gcc-7 using Ekaitz's gcc-4.6.4 with riscv64 support patched in.

In terms of what I've seen so far from your repo and reading your email
I would suggest just trying to bump make from 3.80 to 3.82, it Just
Worked™ for me, even on riscv64.

Also I had been using musl-1.1.24 as an intermediate step, but with some
of your code I currently have 1.2.5 built and am testing that out.

I'd also recommend checking out the wip-riscv-bootstrap branch on
savannah https://git.savannah.gnu.org/cgit/guix.git/log/?h=wip-riscv-bootstrap
or one of my mirrors https://git.sr.ht/~efraim/guix/tree/wip-riscv-bootstrap
to see some of the changes I've made to commencement.scm. For example I
ended up rewriting the musl install phase.

-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: A different way to bootstrap and build GCC

[Stefan]

Hi Efraim!

> In terms of what I've seen so far from your repo and reading your email
> I would suggest just trying to bump make from 3.80 to 3.82, it Just
> Worked™ for me, even on riscv64.

Yes, you are right, I should drop the use of gnu-make-mesboot0.  This would allow me to drop four substitutions in the TCC package.  In the original commencement.scm it is anyway bound to i686-unknown-linux-gnu, which I want to avoid.  I think I should try version 4.4.1, which I build with the latest TCC anyway.

> Also I had been using musl-1.1.24 as an intermediate step, but with some
> of your code I currently have 1.2.5 built and am testing that out.

Nice!

> For example I ended up rewriting the musl install phase.

Ah, I know about the trouble.  I patch the install script a bit.  Look for “;; Gash does not support getopts, replace its use.”  With that patch the default install phase of musl is working just fine.  Maybe I should try to send that patch upstream.


Bye

Stefan

Re: A different way to bootstrap and build GCC

[Stefan]

Hi Ekaitz!

> Very interesting work. I'll read it with more detail tomorrow but at the moment it feels very similar to what we did for RISC-V
> Is there any obvious difference in the beginning of the chain that I'm missing?

I use tcc-boot0 only to build the latest TCC with the latest musl, with much less handcrafting (using make and normal installation). There are less packages involved (no need for muls 1.1.24, patch, gzip, xz, flex, bison).

An important difference from my point of view is that I avoid (package (inherit …) …).  I believe that the bootstrap packages need to be the independent base.  For example in commencement.scm gcc-core-mesboot0 (version 2.95.3) inherits from gcc (version 11).  So changes to a newer gcc will change the way how to build an older version.  This is fragile.  This tears in stuff, which is useless or even problematic for the bootstrap.

Well, finally my actual goal is to build GCC differently: There is no need to patch in CROSS_C_INCLUDE_PATH etc.  The include paths to standard header files must not be provided through environment variables at all.  This is the cause of all the troubles people have with GCC in Guix for both native and cross building.  In the end only the one gcc-12-strmov-store-file-names.patch is necessary, and this not even for static builds during bootstrapping.  Keeping (package (inherit …) …) will tear in all the mistakes, which I try hard to avoid.


Bye

Stefan

Re: A different way to bootstrap and build GCC

[Ekaitz Zarraga]

On 2024-11-24 13:02, Stefan wrote:
> Hi Ekaitz!
> 
>> Very interesting work. I'll read it with more detail tomorrow but at 
>> the moment it feels very similar to what we did for RISC-V
>> Is there any obvious difference in the beginning of the chain that I'm 
>> missing?
> 
> I use tcc-boot0 only to build the latest TCC with the latest musl, with 
> much less handcrafting (using make and normal installation). There are 
> less packages involved (no need for muls 1.1.24, patch, gzip, xz, flex, 
> bison).

I'll check in detail why we need those, but some are for the later GCC 
build.

> An important difference from my point of view is that I avoid (package 
> (inherit …) …).  I believe that the bootstrap packages need to be the 
> independent base.  For example in commencement.scm gcc-core-mesboot0 
> (version 2.95.3) inherits from gcc (version 11).  So changes to a newer 
> gcc will change the way how to build an older version.  This is 
> fragile.  This tears in stuff, which is useless or even problematic for 
> the bootstrap.

Oh yes! I strongly agree with this. Our GCC's are too complex and that 
gave us a lot of trouble. It's really hard to know where things are 
coming from.

It's very fragile, indeed.

> Well, finally my actual goal is to build GCC differently: There is no 
> need to patch in CROSS_C_INCLUDE_PATH etc.  The include paths to 
> standard header files must not be provided through environment variables 
> at all.  This is the cause of all the troubles people have with GCC in 
> Guix for both native and cross building.  In the end only the one 
> gcc-12-strmov-store-file-names.patch is necessary, and this not even for 
> static builds during bootstrapping.  Keeping (package (inherit …) …) 
> will tear in all the mistakes, which I try hard to avoid.

Yes. Very interesting. We can try to make something on this line.

> 
> Bye
> 
> Stefan
> 

Thanks for the explanation.

Ekaitz