Core-updates and cross-compilation

Core-updates and cross-compilation

[Josselin Poiret]

[-- Attachment #1: Type: text/plain, Size: 1999 bytes --]

Hi everyone,

I've been looking at the state of most failures for the CI jobset for
core-updates, and we have a couple of problems:

- gcc < 9 and gcc == 12 never cross-compile.  This is because we just
  don't do the right thing: suppose I have by default GCC version X
  (here 11) and I want to cross-compile a GCC version Y.  What we do is,
  we first a GCC version X cross-compiler (as part of the default
  toolchain used when cross-compiling), then cross-compile GCC version Y
  and then cross-compile its supporting libraries.  This doesn't work
  because the supporting libraries might use features only available in
  the same GCC version.  In the native case, the supporting libraries
  are built with the new compiler!  What we should do instead is build a
  GCC version Y cross-compiler and use that to build the cross-compiled
  GCC.  This will require non-trivial changes, since we'd need to
  specify in the package definition of gcc-12 that it needs to be
  cross-compiled by ... gcc-12 :/

  gcc version between 9 and 11 work by sheer luck.

- we can't build the cross toolchain for the hurd, because the glibc
  upgrade to 2.35 would require newer gnumach headers, itself with a
  newer mig.  All these upgrades would be local and pretty ok if they
  didn't also require a glibc patch to make the configure script of
  glibc work (right now it would check for presence of headers without
  -ffreestanding, even though we clearly don't have the glibc built
  yet!).  This would cause a world-rebuild as well.  I don't know how
  much work fixing the rest would be, but that's probably the only glibc
  patch that's needed.

  Also note that Hurd now seems to have some quite recent git tags,
  which are also used by Debian, so we can expect less random commit
  combinations not working.

Should we consider these blockers for a core-updates merge?  Should we
somehow stop supporting the first use-case?

WDYT?

Best,
-- 
Josselin Poiret

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 682 bytes --]

Re: Core-updates and cross-compilation

[Andreas Enge]

Hello,

Am Sat, Mar 11, 2023 at 01:56:27PM +0100 schrieb Josselin Poiret:
> I've been looking at the state of most failures for the CI jobset for
> core-updates, and we have a couple of problems:
> - gcc < 9 and gcc == 12 never cross-compile.
> - we can't build the cross toolchain for the hurd, because the glibc
>   upgrade to 2.35 would require newer gnumach headers

are these new issues compared to master? There we also have gcc@9 and @12,
so my guess would be no, at least for the first one.

> Should we consider these blockers for a core-updates merge?

If the situation is not worse than on master, my answer will be a firm "no".
Otherwise I am less sure. I think we might still want to merge core-updates
first and handle these cross compilation issues in a later feature branch
of the core team.

Thanks for your work and thorough analysis!

Andreas

Re: Core-updates and cross-compilation

[Josselin Poiret]

[-- Attachment #1: Type: text/plain, Size: 1434 bytes --]


-- 

Andreas Enge <andreas@enge.fr> writes:

> Hello,
>
> Am Sat, Mar 11, 2023 at 01:56:27PM +0100 schrieb Josselin Poiret:
>> I've been looking at the state of most failures for the CI jobset for
>> core-updates, and we have a couple of problems:
>> - gcc < 9 and gcc == 12 never cross-compile.
>> - we can't build the cross toolchain for the hurd, because the glibc
>>   upgrade to 2.35 would require newer gnumach headers
>
> are these new issues compared to master? There we also have gcc@9 and @12,
> so my guess would be no, at least for the first one.

The first one no, the second one yes.

>> Should we consider these blockers for a core-updates merge?
>
> If the situation is not worse than on master, my answer will be a firm "no".
> Otherwise I am less sure. I think we might still want to merge core-updates
> first and handle these cross compilation issues in a later feature branch
> of the core team.

I'm working on building Hurd atm, and it's been taking longer than I
would've liked.  We needed an update to mig, which cascaded into updates
to gnumach and hurd, but they also shouldn't be too recent as we'd need
a newer libc.  I thought the git tags in the repos would be a safe bet,
but they don't build by themselves, and lots of patches need to be
backported as well :) maybe after this upgrade is over we could offer
some help with releases/CI.

Best,
-- 
Josselin Poiret

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 682 bytes --]

Re: Core-updates and cross-compilation

[Andreas Enge]

Hello Josselin,

Am Tue, Mar 14, 2023 at 06:48:16PM +0100 schrieb Josselin Poiret:
> I'm working on building Hurd atm, and it's been taking longer than I
> would've liked.  We needed an update to mig, which cascaded into updates
> to gnumach and hurd, but they also shouldn't be too recent as we'd need
> a newer libc.  I thought the git tags in the repos would be a safe bet,
> but they don't build by themselves, and lots of patches need to be
> backported as well :) maybe after this upgrade is over we could offer
> some help with releases/CI.

thanks for your update and working on the hurd integration! And many thanks
also for trying to find a core-updates friendly solution... If it turns out
to be too much work, maybe you could also postpone it until after the merge
and try to get a feature branch going quickly thereafter.

Suggesting help to the hurd people concerning CI sounds like it could be
beneficial to both our communities.

Andreas

Re: Core-updates and cross-compilation

[Ludovic Courtès]

Hi,

Josselin Poiret <dev@jpoiret.xyz> skribis:

> I've been looking at the state of most failures for the CI jobset for
> core-updates, and we have a couple of problems:
>
> - gcc < 9 and gcc == 12 never cross-compile.

I agree with Andreas, I think that’s okay.

[...]

> - we can't build the cross toolchain for the hurd, because the glibc
>   upgrade to 2.35 would require newer gnumach headers, itself with a
>   newer mig.  All these upgrades would be local and pretty ok if they
>   didn't also require a glibc patch to make the configure script of
>   glibc work (right now it would check for presence of headers without
>   -ffreestanding, even though we clearly don't have the glibc built
>   yet!).  This would cause a world-rebuild as well.  I don't know how
>   much work fixing the rest would be, but that's probably the only glibc
>   patch that's needed.

It’s really great that you’ve been looking at this!  If I can be of any
help, of if you feel desperate ;-), please don’t hesitate to ping me on
IRC.

>   Also note that Hurd now seems to have some quite recent git tags,
>   which are also used by Debian, so we can expect less random commit
>   combinations not working.

The good news is that Samuel Thibault is now officially a (actually:
the) Hurd maintainer, which means they should be able to get upload
permission to ftp.gnu.org—a step in the right direction.

> Should we consider these blockers for a core-updates merge?  Should we
> somehow stop supporting the first use-case?

In the past, I’d use ‘etc/release-manifest.scm’ as a way to check for
merge (or release) blockers.

When it comes to cross-compilation, we must be able to cross-compile the
bootstrap tarballs, as in:

  ./pre-inst-env guix build bootstrap-tarballs \
     --target=aarch64-linux-gnu -n

That works well on ‘core-updates’ so far, except for i586-pc-gnu.

Thanks,
Ludo’.