bug#36882: QEMU 4 fails to build for x86_64-linux

bug#36882: QEMU 4 fails to build for x86_64-linux

[Leo Famulari]

While testing the upgrade of our QEMU package to 4.0.0 or 4.1.0-rc3 on
x86_64-linux, the build fails when the header 'gnu/stubs-32.h' can't be
found:

------
In file included from /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/include/features.h:452:0,
                 from /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/include/bits/libc-header-start.h:33,
                 from /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/include/stdint.h:26,
                 from linuxboot_dma.c:65:
/gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/include/gnu/stubs.h:7:27: fatal error: gnu/stubs-32.h: No such file or directory
compilation terminated.
make[1]: *** [/tmp/guix-build-qemu-4.1.0-rc3.drv-0/qemu-4.1.0-rc3/rules.mak:69: linuxboot_dma.o] Error 1
make[1]: Leaving directory '/tmp/guix-build-qemu-4.1.0-rc3.drv-0/qemu-4.1.0-rc3/pc-bios/optionrom'
make: *** [Makefile:519: pc-bios/optionrom/all] Error 2
------

And of course, this header is not installed by glibc when it is built
for 64-bit systems.

The header in question, 'stubs.h', looks like this:

------
#if !defined __x86_64__
# include <gnu/stubs-32.h>
#endif
#if defined __x86_64__ && defined __LP64__
# include <gnu/stubs-64.h>
#endif
#if defined __x86_64__ && defined __ILP32__
# include <gnu/stubs-x32.h>
#endif
------

When I build for i686-linux, it works as expected.

Any advice? I'm not really sure what's going on here.

bug#36882: QEMU 4 fails to build for x86_64-linux

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 1737 bytes --]

Leo Famulari <leo@famulari.name> writes:

> While testing the upgrade of our QEMU package to 4.0.0 or 4.1.0-rc3 on
> x86_64-linux, the build fails when the header 'gnu/stubs-32.h' can't be
> found:
>
> ------
> In file included from /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/include/features.h:452:0,
>                  from /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/include/bits/libc-header-start.h:33,
>                  from /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/include/stdint.h:26,
>                  from linuxboot_dma.c:65:
> /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/include/gnu/stubs.h:7:27: fatal error: gnu/stubs-32.h: No such file or directory
> compilation terminated.
> make[1]: *** [/tmp/guix-build-qemu-4.1.0-rc3.drv-0/qemu-4.1.0-rc3/rules.mak:69: linuxboot_dma.o] Error 1
> make[1]: Leaving directory '/tmp/guix-build-qemu-4.1.0-rc3.drv-0/qemu-4.1.0-rc3/pc-bios/optionrom'
> make: *** [Makefile:519: pc-bios/optionrom/all] Error 2
> ------
>
> And of course, this header is not installed by glibc when it is built
> for 64-bit systems.
>
> The header in question, 'stubs.h', looks like this:
>
> ------
> #if !defined __x86_64__
> # include <gnu/stubs-32.h>
> #endif
> #if defined __x86_64__ && defined __LP64__
> # include <gnu/stubs-64.h>
> #endif
> #if defined __x86_64__ && defined __ILP32__
> # include <gnu/stubs-x32.h>
> #endif
> ------
>
> When I build for i686-linux, it works as expected.
>
> Any advice? I'm not really sure what's going on here.

I don't know why it fails, but it works if you give it a newer GCC such
as the one on 'core-updates'.

I tried it with 4.0.0, but got stuck on two test failures.  Hopefully
4.1.0 is easier to debug...  :-)

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

bug#36882: QEMU 4 fails to build for x86_64-linux

[Ludovic Courtès]

Hi,

Leo Famulari <leo@famulari.name> skribis:

> While testing the upgrade of our QEMU package to 4.0.0 or 4.1.0-rc3 on
> x86_64-linux, the build fails when the header 'gnu/stubs-32.h' can't be
> found:
>
> ------
> In file included from /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/include/features.h:452:0,
>                  from /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/include/bits/libc-header-start.h:33,
>                  from /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/include/stdint.h:26,
>                  from linuxboot_dma.c:65:
> /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/include/gnu/stubs.h:7:27: fatal error: gnu/stubs-32.h: No such file or directory
> compilation terminated.
> make[1]: *** [/tmp/guix-build-qemu-4.1.0-rc3.drv-0/qemu-4.1.0-rc3/rules.mak:69: linuxboot_dma.o] Error 1
> make[1]: Leaving directory '/tmp/guix-build-qemu-4.1.0-rc3.drv-0/qemu-4.1.0-rc3/pc-bios/optionrom'
> make: *** [Makefile:519: pc-bios/optionrom/all] Error 2
> ------
>
> And of course, this header is not installed by glibc when it is built
> for 64-bit systems.

That’s a “multilib header”, which our ‘glibc’ package doesn’t provide.
Normally it’s only needed when doing “gcc -m32” on x86_64 or similar.

Can you show the GCC command that leads to this error?

Thanks,
Ludo’.

bug#36882: Qemu 4.2.0 build for x86_64-linux fails

[Mathieu Othacehe]

Hello,

On core-updates, qemu-minimal (4.2.0), fails to build. This seems to be
the same issue as this bug. The error is:

--8<---------------cut here---------------start------------->8---
In file included from /gnu/store/jsjsczgr8xdnbdminl7lm2v56b7dq7lq-glibc-2.31/include/features.h:489:0,
                 from /gnu/store/jsjsczgr8xdnbdminl7lm2v56b7dq7lq-glibc-2.31/include/bits/libc-header-start.h:33,
                 from /gnu/store/jsjsczgr8xdnbdminl7lm2v56b7dq7lq-glibc-2.31/include/stdint.h:26,
                 from linuxboot_dma.c:65:
/gnu/store/jsjsczgr8xdnbdminl7lm2v56b7dq7lq-glibc-2.31/include/gnu/stubs.h:7:11: fatal error: gnu/stubs-32.h: No such file or directory
 # include <gnu/stubs-32.h>
           ^~~~~~~~~~~~~~~~

Because of this gcc command:

--8<---------------cut here---------------start------------->8---
/gnu/store/dcnp1h3q6qyipwkm0g7l7r1bkvlqvaqa-gcc-7.5.0/bin/gcc -iquote /tmp/guix-build-qemu-minimal-4.2.0.drv-0/qemu-4.2.0/. -iquote . -iquote /tmp/guix-build-qemu-minimal-4.2.0.drv-0/qemu-4.2.0/tcg -iquote /tmp/guix-build-qemu-minimal-4.2.0.drv-0/qemu-4.2.0/tcg/i386 -I/tmp/guix-build-qemu-minimal-4.2.0.drv-0/qemu-4.2.0/linux-headers -I/tmp/guix-build-qemu-minimal-4.2.0.drv-0/qemu-4.2.0/linux-headers -iquote . -iquote /tmp/guix-build-qemu-minimal-4.2.0.drv-0/qemu-4.2.0 -iquote /tmp/guix-build-qemu-minimal-4.2.0.drv-0/qemu-4.2.0/accel/tcg -iquote /tmp/guix-build-qemu-minimal-4.2.0.drv-0/qemu-4.2.0/include -I/tmp/guix-build-qemu-minimal-4.2.0.drv-0/qemu-4.2.0 -Wstrict-prototypes -Wredundant-decls -Wall -Wundef -Wwrite-strings -Wmissing-prototypes -Wexpansion-to-defined -Wendif-labels -Wno-shift-negative-value -Wno-missing-include-dirs -Wempty-body -Wnested-externs -Wformat-security -Wformat-y2k -Winit-self -Wignored-qualifiers -Wold-style-declaration -Wold-style-definition -Wtype-limits -fno-pie -ffreestanding   -fno-stack-protector   -m16   -Wa,-32 -MMD -MP -MT linuxboot_dma.o -MF ./linuxboot_dma.d -O2 -march=i486  -c -o linuxboot_dma.o linuxboot_dma.c
--8<---------------cut here---------------end--------------->8---

Any idea? Do we still need to add "multilib header" support to our
glibc build?

Thanks,

Mathieu

bug#36882: Qemu 4.2.0 build for x86_64-linux fails

[Ludovic Courtès]

Hi,

Mathieu Othacehe <m.othacehe@gmail.com> skribis:

> On core-updates, qemu-minimal (4.2.0), fails to build. This seems to be
> the same issue as this bug. The error is:
>
> In file included from /gnu/store/jsjsczgr8xdnbdminl7lm2v56b7dq7lq-glibc-2.31/include/features.h:489:0,
>                  from /gnu/store/jsjsczgr8xdnbdminl7lm2v56b7dq7lq-glibc-2.31/include/bits/libc-header-start.h:33,
>                  from /gnu/store/jsjsczgr8xdnbdminl7lm2v56b7dq7lq-glibc-2.31/include/stdint.h:26,
>                  from linuxboot_dma.c:65:
> /gnu/store/jsjsczgr8xdnbdminl7lm2v56b7dq7lq-glibc-2.31/include/gnu/stubs.h:7:11: fatal error: gnu/stubs-32.h: No such file or directory
>  # include <gnu/stubs-32.h>
>            ^~~~~~~~~~~~~~~~
>
> Because of this gcc command:
>
> /gnu/store/dcnp1h3q6qyipwkm0g7l7r1bkvlqvaqa-gcc-7.5.0/bin/gcc -iquote /tmp/guix-build-qemu-minimal-4.2.0.drv-0/qemu-4.2.0/. -iquote . -iquote /tmp/guix-build-qemu-minimal-4.2.0.drv-0/qemu-4.2.0/tcg -iquote /tmp/guix-build-qemu-minimal-4.2.0.drv-0/qemu-4.2.0/tcg/i386 -I/tmp/guix-build-qemu-minimal-4.2.0.drv-0/qemu-4.2.0/linux-headers -I/tmp/guix-build-qemu-minimal-4.2.0.drv-0/qemu-4.2.0/linux-headers -iquote . -iquote /tmp/guix-build-qemu-minimal-4.2.0.drv-0/qemu-4.2.0 -iquote /tmp/guix-build-qemu-minimal-4.2.0.drv-0/qemu-4.2.0/accel/tcg -iquote /tmp/guix-build-qemu-minimal-4.2.0.drv-0/qemu-4.2.0/include -I/tmp/guix-build-qemu-minimal-4.2.0.drv-0/qemu-4.2.0 -Wstrict-prototypes -Wredundant-decls -Wall -Wundef -Wwrite-strings -Wmissing-prototypes -Wexpansion-to-defined -Wendif-labels -Wno-shift-negative-value -Wno-missing-include-dirs -Wempty-body -Wnested-externs -Wformat-security -Wformat-y2k -Winit-self -Wignored-qualifiers -Wold-style-declaration -Wold-style-definition -Wtype-limits -fno-pie -ffreestanding   -fno-stack-protector   -m16   -Wa,-32 -MMD -MP -MT linuxboot_dma.o -MF ./linuxboot_dma.d -O2 -march=i486  -c -o linuxboot_dma.o linuxboot_dma.c

How was this bug initially reported against QEMU 4.0.0 fixed?

On master there’s pretty much the same command as above, with ‘-m16’,
and “yet it works”.

  https://ci.guix.gnu.org/log/ymzp5yz2r3zfw4xczwwlykyjv2kqcqs0-qemu-4.2.0

Ludo’.

bug#36882: Qemu 4.2.0 build for x86_64-linux fails

[Mathieu Othacehe]

Hello,

> On master there’s pretty much the same command as above, with ‘-m16’,
> and “yet it works”.
>
>   https://ci.guix.gnu.org/log/ymzp5yz2r3zfw4xczwwlykyjv2kqcqs0-qemu-4.2.0

On master, when building qemu, this is the gcc include path (obtained
with "gcc -v -x c -E /dev/null"):

--8<---------------cut here---------------start------------->8---
 /gnu/store/adm2cx3ayabn1sp84nnjsk0672m800ip-flex-2.6.4/include
 /gnu/store/l86azr7r3p5631wj3kk329jl1y1mpjgy-bzip2-1.0.6/include
 ...
 /gnu/store/2plcy91lypnbbysb18ymnhaw3zwk8pg1-gcc-7.4.0-lib/lib/gcc/x86_64-unknown-linux-gnu/7.4.0/include [1]
 /gnu/store/2plcy91lypnbbysb18ymnhaw3zwk8pg1-gcc-7.4.0-lib/lib/gcc/x86_64-unknown-linux-gnu/7.4.0/include-fixed
 /gnu/store/ahqgl4h89xqj695lgqvsaf6zh2nhy4pj-glibc-2.29/include [2]
--8<---------------cut here---------------end--------------->8---

We see that glibc comes last. So when building a program with "gcc
-ffreestanding -m16", the first <stdint.h> header picked is the one from
gcc (marked [1] above). This header contains:

--8<---------------cut here---------------start------------->8---
#ifndef _GCC_WRAP_STDINT_H
  #if __STDC_HOSTED__ <-- Set to 0 because of -ffreestanding option.
    # if defined __cplusplus && __cplusplus >= 201103L
      #  undef __STDC_LIMIT_MACROS
      #  define __STDC_LIMIT_MACROS
      #  undef __STDC_CONSTANT_MACROS
      #  define __STDC_CONSTANT_MACROS
    # endif
    # include_next <stdint.h>
  #else
    # include "stdint-gcc.h" <-- This one gets included.
  #endif
  #define _GCC_WRAP_STDINT_H
#endif
--8<---------------cut here---------------end--------------->8---

So the <stdint.h> from glibc (marked [2] above) is neved included. And
that's good because this header would try to include <gnu/stubs-32.h>
that our glibc does not provide.

Now the question is, why glibc comes last, whereas it is a part of
CPATH? The "gcc -v -x c -E /dev/null" command says:

--8<---------------cut here---------------start------------->8---
ignoring duplicate directory "/gnu/store/ahqgl4h89xqj695lgqvsaf6zh2nhy4pj-glibc-2.29/include"
  as it is a non-system directory that duplicates a system directory
--8<---------------cut here---------------end--------------->8---

So the glibc from CPATH is ignored and the last glibc, hardcoded into
gcc is kept.

Now, what's happening on core-updates? Because of CPATH ->
C_INCLUDE_PATH transition, we have a different behaviour. The same
command as above states:

--8<---------------cut here---------------start------------->8---
ignoring duplicate directory "/gnu/store/jsjsczgr8xdnbdminl7lm2v56b7dq7lq-glibc-2.31/include"
...
 /gnu/store/kyypmcn2miyzllqqmcx28gvj89qnvhpi-flex-2.6.4/include
 ...
 /gnu/store/jsjsczgr8xdnbdminl7lm2v56b7dq7lq-glibc-2.31/include [1]
 ...
 /gnu/store/wl7zanqf0s9bfppggmv8qxqan71fjfw8-gcc-7.5.0-lib/lib/gcc/x86_64-unknown-linux-gnu/7.5.0/include [2]
 /gnu/store/wl7zanqf0s9bfppggmv8qxqan71fjfw8-gcc-7.5.0-lib/lib/gcc/x86_64-unknown-linux-gnu/7.5.0/include-fixed
--8<---------------cut here---------------end--------------->8---

The glibc ignored seems to be the hardcoded one. The one from the
C_INCLUDE_PATH is kept. So the <stdint.h> included is the one from glibc
(marked [1] above), which triggers <gnu/stubs-32.h> inclusion and makes
build fail.

So here's how it fails, but I'm not sure how to fix it. Removing glibc
from C_INCLUDE_PATH, forces gcc to use the hardcoded one that comes last
and fixes the build. However, there are maybe other implications?

Thanks,

Mathieu

bug#36882: Qemu 4.2.0 build for x86_64-linux fails

[Ludovic Courtès]

Hi Mathieu,

Mathieu Othacehe <m.othacehe@gmail.com> skribis:

> On master, when building qemu, this is the gcc include path (obtained
> with "gcc -v -x c -E /dev/null"):
>
>  /gnu/store/adm2cx3ayabn1sp84nnjsk0672m800ip-flex-2.6.4/include
>  /gnu/store/l86azr7r3p5631wj3kk329jl1y1mpjgy-bzip2-1.0.6/include
>  ...
>  /gnu/store/2plcy91lypnbbysb18ymnhaw3zwk8pg1-gcc-7.4.0-lib/lib/gcc/x86_64-unknown-linux-gnu/7.4.0/include [1]
>  /gnu/store/2plcy91lypnbbysb18ymnhaw3zwk8pg1-gcc-7.4.0-lib/lib/gcc/x86_64-unknown-linux-gnu/7.4.0/include-fixed
>  /gnu/store/ahqgl4h89xqj695lgqvsaf6zh2nhy4pj-glibc-2.29/include [2]
>
>
> We see that glibc comes last. So when building a program with "gcc
> -ffreestanding -m16", the first <stdint.h> header picked is the one from
> gcc (marked [1] above). This header contains:
>
> #ifndef _GCC_WRAP_STDINT_H
>   #if __STDC_HOSTED__ <-- Set to 0 because of -ffreestanding option.
>     # if defined __cplusplus && __cplusplus >= 201103L
>       #  undef __STDC_LIMIT_MACROS
>       #  define __STDC_LIMIT_MACROS
>       #  undef __STDC_CONSTANT_MACROS
>       #  define __STDC_CONSTANT_MACROS
>     # endif
>     # include_next <stdint.h>
>   #else
>     # include "stdint-gcc.h" <-- This one gets included.
>   #endif
>   #define _GCC_WRAP_STDINT_H
> #endif
>
>
> So the <stdint.h> from glibc (marked [2] above) is neved included. And
> that's good because this header would try to include <gnu/stubs-32.h>
> that our glibc does not provide.

[...]

> Now, what's happening on core-updates? Because of CPATH ->
> C_INCLUDE_PATH transition, we have a different behaviour. The same
> command as above states:
>
> ignoring duplicate directory "/gnu/store/jsjsczgr8xdnbdminl7lm2v56b7dq7lq-glibc-2.31/include"
> ...
>  /gnu/store/kyypmcn2miyzllqqmcx28gvj89qnvhpi-flex-2.6.4/include
>  ...
>  /gnu/store/jsjsczgr8xdnbdminl7lm2v56b7dq7lq-glibc-2.31/include [1]
>  ...
>  /gnu/store/wl7zanqf0s9bfppggmv8qxqan71fjfw8-gcc-7.5.0-lib/lib/gcc/x86_64-unknown-linux-gnu/7.5.0/include [2]
>  /gnu/store/wl7zanqf0s9bfppggmv8qxqan71fjfw8-gcc-7.5.0-lib/lib/gcc/x86_64-unknown-linux-gnu/7.5.0/include-fixed
>
> The glibc ignored seems to be the hardcoded one. The one from the
> C_INCLUDE_PATH is kept. So the <stdint.h> included is the one from glibc
> (marked [1] above), which triggers <gnu/stubs-32.h> inclusion and makes
> build fail.

Woow, thanks for the analysis.

It’s a bug on ‘core-updates’ that GCC’s headers come after libc’s.  This
is what should be fixed IMO.

So is it expected that C_INCLUDE_PATH comes before the hard-coded GCC
include directory?  How can we work around that?

Thanks!

Ludo’.

bug#36882: Qemu 4.2.0 build for x86_64-linux fails

[Mathieu Othacehe]

Hey,

> So is it expected that C_INCLUDE_PATH comes before the hard-coded GCC
> include directory?  How can we work around that?

Turns out, the best source of documentation here is
gcc/incpath.c. Here's a summary of my understanding.

Header search list:

* QUOTE
 -> -iquote 
* BRACKET
 -> -I goes here
 -> CPATH goes here
* SYSTEM
 -> -isystem goes here
 -> C_INCLUDE_PATH goes here
 -> Hardcoded includes (gcc + glibc) goes here
* AFTER
-> -idirafter goes here.

Duplicates inside SYSTEM are deleted, the first occurence is kept. So as
long as we have glibc in C_INCLUDE_PATH it will trigger deletion of
glibc in hardcoded includes and AFTER section.

So I can see only two solutions here.

1. Go back to using CPATH (sad!), because when there is duplication
between BRACKET and SYSTEM, the include from SYSTEM is kept (why it
works on master).

2. Find a way to remove glibc from C_INCLUDE_PATH, but I have no clue
how to do this properly. Maybe using some kind of filter on
search-path-specifications.

WDYT?

Mathieu

bug#36882: Qemu 4.2.0 build for x86_64-linux fails

[Ludovic Courtès]

Hi,

Mathieu Othacehe <m.othacehe@gmail.com> skribis:

>> So is it expected that C_INCLUDE_PATH comes before the hard-coded GCC
>> include directory?  How can we work around that?
>
> Turns out, the best source of documentation here is
> gcc/incpath.c. Here's a summary of my understanding.
>
> Header search list:
>
> * QUOTE
>  -> -iquote 
> * BRACKET
>  -> -I goes here
>  -> CPATH goes here
> * SYSTEM
>  -> -isystem goes here
>  -> C_INCLUDE_PATH goes here
>  -> Hardcoded includes (gcc + glibc) goes here
> * AFTER
> -> -idirafter goes here.
>
> Duplicates inside SYSTEM are deleted, the first occurence is kept. So as
> long as we have glibc in C_INCLUDE_PATH it will trigger deletion of
> glibc in hardcoded includes and AFTER section.
>
> So I can see only two solutions here.
>
> 1. Go back to using CPATH (sad!), because when there is duplication
> between BRACKET and SYSTEM, the include from SYSTEM is kept (why it
> works on master).
>
> 2. Find a way to remove glibc from C_INCLUDE_PATH, but I have no clue
> how to do this properly. Maybe using some kind of filter on
> search-path-specifications.

I’d rather go for #2.  To do that, we could modify the ‘set-paths’ phase
to manually remove glibc from C_INCLUDE_PATH (fragile), or we could
modify GCC (perhaps removing the ‘remove_duplicates’ call for SYSTEM).

Either way, this wouldn’t work well with ‘guix environment’, where glibc
ends up in /gnu/store/…-profile, so it does not appear as duplicate to
GCC.

On ‘core-updates’, I see:

--8<---------------cut here---------------start------------->8---
$ git log | head -3
commit 5afcb5caa53615c0a432e0c1781155398d747218
Author: Ludovic Courtès <ludo@gnu.org>
Date:   Sat Feb 22 21:39:27 2020 +0100
$ ./pre-inst-env guix environment -C -e '(@@ (gnu packages commencement) coreutils-final)'
[env]$ gcc -v -x c -E /dev/null
Using built-in specs.
COLLECT_GCC=gcc
Target: x86_64-unknown-linux-gnu
Configured with: 
Thread model: posix
gcc version 7.5.0 (GCC) 
COLLECT_GCC_OPTIONS='-v' '-E' '-mtune=generic' '-march=x86-64'
 /gnu/store/0pjrnr8vhp94ykvarysd9wg7hcvfqkl5-gcc-7.5.0/libexec/gcc/x86_64-unknown-linux-gnu/7.5.0/cc1 -E -quiet -v /dev/null -mtune=generic -march=x86-64
ignoring duplicate directory "/gnu/store/k9l4v4530p1a69j8qs0aijbmn8lwak20-profile/include"
ignoring nonexistent directory "/no-gcc-local-prefix/include"
ignoring nonexistent directory "/gnu/store/adrw71v03nawqwyblxc0mdhcc41j5vnn-gcc-7.5.0-lib/lib/gcc/x86_64-unknown-linux-gnu/7.5.0/../../../../../../../x86_64-unknown-linux-gnu/include"
#include "..." search starts here:
#include <...> search starts here:
 /gnu/store/k9l4v4530p1a69j8qs0aijbmn8lwak20-profile/include
 /gnu/store/adrw71v03nawqwyblxc0mdhcc41j5vnn-gcc-7.5.0-lib/lib/gcc/x86_64-unknown-linux-gnu/7.5.0/include
 /gnu/store/adrw71v03nawqwyblxc0mdhcc41j5vnn-gcc-7.5.0-lib/lib/gcc/x86_64-unknown-linux-gnu/7.5.0/include-fixed
 /gnu/store/ccjj5wg0qkjs1zfjr98nhb8cpr434izw-glibc-2.31/include
End of search list.
# 1 "/dev/null"
# 1 "<built-in>"
# 1 "<command-line>"
# 31 "<command-line>"
# 1 "/gnu/store/k9l4v4530p1a69j8qs0aijbmn8lwak20-profile/include/stdc-predef.h" 1 3
# 32 "<command-line>" 2
# 1 "/dev/null"
COMPILER_PATH=/gnu/store/0pjrnr8vhp94ykvarysd9wg7hcvfqkl5-gcc-7.5.0/libexec/gcc/x86_64-unknown-linux-gnu/7.5.0/:/gnu/store/0pjrnr8vhp94ykvarysd9wg7hcvfqkl5-gcc-7.5.0/libexec/gcc/x86_64-unknown-linux-gnu/7.5.0/:/gnu/store/0pjrnr8vhp94ykvarysd9wg7hcvfqkl5-gcc-7.5.0/libexec/gcc/x86_64-unknown-linux-gnu/:/gnu/store/adrw71v03nawqwyblxc0mdhcc41j5vnn-gcc-7.5.0-lib/lib/gcc/x86_64-unknown-linux-gnu/7.5.0/:/gnu/store/adrw71v03nawqwyblxc0mdhcc41j5vnn-gcc-7.5.0-lib/lib/gcc/x86_64-unknown-linux-gnu/
LIBRARY_PATH=/gnu/store/k9l4v4530p1a69j8qs0aijbmn8lwak20-profile/lib/:/gnu/store/k9l4v4530p1a69j8qs0aijbmn8lwak20-profile/lib/:/gnu/store/adrw71v03nawqwyblxc0mdhcc41j5vnn-gcc-7.5.0-lib/lib/gcc/x86_64-unknown-linux-gnu/7.5.0/:/gnu/store/adrw71v03nawqwyblxc0mdhcc41j5vnn-gcc-7.5.0-lib/lib/gcc/x86_64-unknown-linux-gnu/7.5.0/../../../:/gnu/store/ccjj5wg0qkjs1zfjr98nhb8cpr434izw-glibc-2.31/lib
COLLECT_GCC_OPTIONS='-v' '-E' '-mtune=generic' '-march=x86-64'
--8<---------------cut here---------------end--------------->8---

Looking at ‘cppdefault.c’ in GCC, I don’t see where glibc-2.31/include
comes from; it seems that ‘INCLUDE_DEFAULTS’ is undefined on glibc
systems.

Thoughts?

Incidentally, do we have problems building anything other than QEMU?

Thanks,
Ludo’.

bug#36882: Qemu 4.2.0 build for x86_64-linux fails

[Mathieu Othacehe]

Hey!

> I’d rather go for #2.  To do that, we could modify the ‘set-paths’ phase
> to manually remove glibc from C_INCLUDE_PATH (fragile), or we could
> modify GCC (perhaps removing the ‘remove_duplicates’ call for SYSTEM).
>
> Either way, this wouldn’t work well with ‘guix environment’, where glibc
> ends up in /gnu/store/…-profile, so it does not appear as duplicate to
> GCC.

[...]

> Looking at ‘cppdefault.c’ in GCC, I don’t see where glibc-2.31/include
> comes from; it seems that ‘INCLUDE_DEFAULTS’ is undefined on glibc
> systems.

It's indeed undefined and glibc comes from NATIVE_SYSTEM_HEADER_DIR at
the end of the file you mentioned. It is a consequence of passing
--with-native-system-header-dir=glibc in (gnu packages gcc).

About the environment issue, we have the same problem on master. You can
run the following command:


>
> Thoughts?
>
> Incidentally, do we have problems building anything other than QEMU?
>
> Thanks,
> Ludo’.

bug#36882: Qemu 4.2.0 build for x86_64-linux fails

[Mathieu Othacehe]

Oops wrong shortcut, sorry!

> I’d rather go for #2.  To do that, we could modify the ‘set-paths’ phase
> to manually remove glibc from C_INCLUDE_PATH (fragile), or we could
> modify GCC (perhaps removing the ‘remove_duplicates’ call for SYSTEM).
>
> Either way, this wouldn’t work well with ‘guix environment’, where glibc
> ends up in /gnu/store/…-profile, so it does not appear as duplicate to
> GCC.

[...]

> Looking at ‘cppdefault.c’ in GCC, I don’t see where glibc-2.31/include
> comes from; it seems that ‘INCLUDE_DEFAULTS’ is undefined on glibc
> systems.

It's indeed undefined and glibc comes from NATIVE_SYSTEM_HEADER_DIR at
the end of the file you mentioned. It is a consequence of passing
--with-native-system-header-dir=glibc in (gnu packages gcc).

About the environment issue, we have the same problem on master. You can
run the following command:

--8<---------------cut here---------------start------------->8---
 ./pre-inst-env guix environment -C -e '(@@ (gnu packages commencement)
 coreutils-final)' -- echo -e '#include <stdint.h>\n int main() {return
 0;}' > test.c && gcc -m16 -ffreestanding test.c
--8<---------------cut here---------------end--------------->8---

and see that in takes stdint.h from the profile glibc header:

--8<---------------cut here---------------start------------->8---
In file included from /gnu/store/nl6zndkx4115laq50qmqcvnzinfz5rk0-profile/include/features.h:474:0,
                 from /gnu/store/nl6zndkx4115laq50qmqcvnzinfz5rk0-profile/include/bits/libc-header-start.h:33,
                 from /gnu/store/nl6zndkx4115laq50qmqcvnzinfz5rk0-profile/include/stdint.h:26,
                 from test.c:1:
/gnu/store/nl6zndkx4115laq50qmqcvnzinfz5rk0-profile/include/gnu/stubs.h:7:11: fatal error: gnu/stubs-32.h: No such file or directory
 # include <gnu/stubs-32.h>
           ^~~~~~~~~~~~~~~~
--8<---------------cut here---------------end--------------->8---

So if it's ok for you, I'll try to implement a GCC hack so that we can
keep using C_INCLUDE_PATH on core-updates and have QEMU building, as you
proposed.

About the environment use-case, it's getting really tricky, but as it is
not a regression, we can maybe postpone the resolution.

> Incidentally, do we have problems building anything other than QEMU?

I don't know, but potentially any program building with -m<something>
and -ffreestanding fails on core-updates.

Thanks for your help :),

Mathieu

bug#36882: Qemu 4.2.0 build for x86_64-linux fails

[Ludovic Courtès]

Hi,

Mathieu Othacehe <m.othacehe@gmail.com> skribis:

> About the environment issue, we have the same problem on master. You can
> run the following command:
>
>  ./pre-inst-env guix environment -C -e '(@@ (gnu packages commencement)
>  coreutils-final)' -- echo -e '#include <stdint.h>\n int main() {return
>  0;}' > test.c && gcc -m16 -ffreestanding test.c
>
>
> and see that in takes stdint.h from the profile glibc header:
>
> In file included from /gnu/store/nl6zndkx4115laq50qmqcvnzinfz5rk0-profile/include/features.h:474:0,
>                  from /gnu/store/nl6zndkx4115laq50qmqcvnzinfz5rk0-profile/include/bits/libc-header-start.h:33,
>                  from /gnu/store/nl6zndkx4115laq50qmqcvnzinfz5rk0-profile/include/stdint.h:26,
>                  from test.c:1:
> /gnu/store/nl6zndkx4115laq50qmqcvnzinfz5rk0-profile/include/gnu/stubs.h:7:11: fatal error: gnu/stubs-32.h: No such file or directory
>  # include <gnu/stubs-32.h>
>            ^~~~~~~~~~~~~~~~

Indeed.

> So if it's ok for you, I'll try to implement a GCC hack so that we can
> keep using C_INCLUDE_PATH on core-updates and have QEMU building, as you
> proposed.
>
> About the environment use-case, it's getting really tricky, but as it is
> not a regression, we can maybe postpone the resolution.

Yes, both make sense to me.

>> Incidentally, do we have problems building anything other than QEMU?
>
> I don't know, but potentially any program building with -m<something>
> and -ffreestanding fails on core-updates.

The evil idea I was getting at was that, if that’s just a couple of
packages, we can fix them locally.  Evil plan in case the better hack
turns out to be tricky.  :-)

Ludo’.

bug#36882: Qemu 4.2.0 build for x86_64-linux fails

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 605 bytes --]

Ludovic Courtès <ludo@gnu.org> writes:

> Incidentally, do we have problems building anything other than QEMU?

The only other regression I've noticed with the C_INCLUDE_PATH change is
that GHC 8.4 fails to build -- previously we at least got to GHC 8.6.

The error message does not make much sense to me (something about
libffi.so not found), but it must be related to the C_INCLUDE_PATH
change since it built before.

So I think we can work around these problems for now and try to fix it
properly in the next core-updates cycle (rumor has it that it includes a
switch to GCC 9 as well).

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

bug#36882: Qemu 4.2.0 build for x86_64-linux fails

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 527 bytes --]

Hi Mathieu,

Mathieu Othacehe <m.othacehe@gmail.com> writes:

> So if it's ok for you, I'll try to implement a GCC hack so that we can
> keep using C_INCLUDE_PATH on core-updates and have QEMU building, as you
> proposed.

Did you get anywhere with this?  As Ludovic mentioned, it might make
sense to work around it in gnu-build-system too if patching GCC turns
out to be difficult.

If we can't find a workaround in the coming weeks, perhaps we should
postpone the issue and add package-specific workarounds in the mean
time?

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

bug#36882: Qemu 4.2.0 build for x86_64-linux fails

[Mathieu Othacehe]

Hey Marius,

> Did you get anywhere with this?  As Ludovic mentioned, it might make
> sense to work around it in gnu-build-system too if patching GCC turns
> out to be difficult.

Yup, turned out patching GCC was too difficult. I'm experimenting a
filter over inputs passed to set-path-environment-variable in set-paths.

I hope to send this hack to review later this week.

Thanks,

Mathieu

bug#36882: Qemu 4.2.0 build for x86_64-linux fails

[Mathieu Othacehe]

[-- Attachment #1: Type: text/plain, Size: 421 bytes --]


> Yup, turned out patching GCC was too difficult. I'm experimenting a
> filter over inputs passed to set-path-environment-variable in set-paths.

This is also quite tricky, because the "libc" input passed to set-paths
must not be removed from C_INCLUDE_PATH in (gnu packages commencement)
for mes packages.

So, maybe the best option is to patch QEMU directly, as proposed in
the attached patch. WDYT?

Thanks,

Mathieu

[-- Attachment #2: 0001-gnu-qemu-minimal-Fix-build.patch --]
[-- Type: text/x-diff, Size: 3012 bytes --]

From 6d5dea746ce7b501cdb5ba41c5eff0c9ea7eb0b0 Mon Sep 17 00:00:00 2001
From: Mathieu Othacehe <m.othacehe@gmail.com>
Date: Tue, 3 Mar 2020 12:45:33 +0100
Subject: [PATCH] gnu: qemu-minimal: Fix build.

This fixes <https://issues.guix.info/issue/36882>.

* gnu/packages/virtualization.scm (qemu-minimal)[arguments]: Add a
remove-glibc phase to remove glibc from C_INCLUDE_PATH.
---
 gnu/packages/virtualization.scm | 23 ++++++++++++++++++++++-
 1 file changed, 22 insertions(+), 1 deletion(-)

diff --git a/gnu/packages/virtualization.scm b/gnu/packages/virtualization.scm
index e16405fc9c..6ab8c3802c 100644
--- a/gnu/packages/virtualization.scm
+++ b/gnu/packages/virtualization.scm
@@ -11,6 +11,7 @@
 ;;; Copyright © 2018 Sou Bunnbu <iyzsong@member.fsf.org>
 ;;; Copyright © 2018 Julien Lepiller <julien@lepiller.eu>
 ;;; Copyright © 2019 Guy Fleury Iteriteka <hoonandon@gmail.com>
+;;; Copyright © 2020 Mathieu Othacehe <m.othacehe@gmail.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -124,7 +125,7 @@
                "1w38hzlw7xp05gcq1nhga7hxvndxy6dfcnzi7q2il8ff110isj6k"))))
     (build-system gnu-build-system)
     (arguments
-     '(;; Running tests in parallel can occasionally lead to failures, like:
+     `(;; Running tests in parallel can occasionally lead to failures, like:
        ;; boot_sector_test: assertion failed (signature == SIGNATURE): (0x00000000 == 0x0000dead)
        #:parallel-tests? #f
        #:configure-flags (list "--enable-usb-redir" "--enable-opengl"
@@ -135,8 +136,28 @@
                                "--audio-drv-list=alsa,pa,sdl")
        ;; Make build and test output verbose to facilitate investigation upon failure.
        #:make-flags '("V=1")
+       #:modules ((srfi srfi-1)
+                  (ice-9 match)
+                  ,@%gnu-build-system-modules)
        #:phases
        (modify-phases %standard-phases
+         ;; Work around https://issues.guix.info/issue/36882.  We need to
+         ;; remove glibc from C_INCLUDE_PATH so that the one hardcoded in GCC,
+         ;; at the bottom of GCC include search-path is used.
+         (add-after 'set-paths 'remove-glibc
+          (lambda* (#:key inputs #:allow-other-keys)
+            (let* ((filters '("libc"))
+                   (input-directories
+                    (filter-map (lambda (input)
+                                  (match input
+                                    ((name . dir)
+                                     (and (not (member name filters))
+                                          dir))))
+                                inputs)))
+              (set-path-environment-variable "C_INCLUDE_PATH"
+                                             '("include")
+                                             input-directories)
+              #t)))
          (replace 'configure
            (lambda* (#:key inputs outputs (configure-flags '())
                            #:allow-other-keys)
-- 
2.25.1

bug#36882: Qemu 4.2.0 build for x86_64-linux fails

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 846 bytes --]

Hello Mathieu & Ludo,

(+ janneke)

Mathieu Othacehe <m.othacehe@gmail.com> writes:

>> Yup, turned out patching GCC was too difficult. I'm experimenting a
>> filter over inputs passed to set-path-environment-variable in set-paths.
>
> This is also quite tricky, because the "libc" input passed to set-paths
> must not be removed from C_INCLUDE_PATH in (gnu packages commencement)
> for mes packages.

If that is the only issue preventing us from solving this problem for
all build systems, I'd say let's fix that.  We could rename it obviously,
though there are a few places that expect the "libc" key so it may become
unwieldly.

Can we find another way to detect the Mes toolchain, perhaps directly in
gnu-build-system where we apply this workaround?  E.g. look for inputs
where (string-contains (package-name input) "mesboot")?

Suggestions?

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

bug#36882: Qemu 4.2.0 build for x86_64-linux fails

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 1199 bytes --]

Marius Bakke <mbakke@fastmail.com> writes:

> Hello Mathieu & Ludo,
>
> (+ janneke)
>
> Mathieu Othacehe <m.othacehe@gmail.com> writes:
>
>>> Yup, turned out patching GCC was too difficult. I'm experimenting a
>>> filter over inputs passed to set-path-environment-variable in set-paths.
>>
>> This is also quite tricky, because the "libc" input passed to set-paths
>> must not be removed from C_INCLUDE_PATH in (gnu packages commencement)
>> for mes packages.
>
> If that is the only issue preventing us from solving this problem for
> all build systems, I'd say let's fix that.  We could rename it obviously,
> though there are a few places that expect the "libc" key so it may become
> unwieldly.
>
> Can we find another way to detect the Mes toolchain, perhaps directly in
> gnu-build-system where we apply this workaround?  E.g. look for inputs
> where (string-contains (package-name input) "mesboot")?
>
> Suggestions?

I realize we don't have access to PACKAGE-NAME in the builder, though we
could check for the string in the store item names.  'mesboot' contains
an 'e' and is thus guaranteed not to end up in the hash.

Not a very elegant solution though, hoping for other suggestions.  :-)

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

bug#36882: Qemu 4.2.0 build for x86_64-linux fails

[Jan (janneke) Nieuwenhuizen]

Mathieu Othacehe <m.othacehe@gmail.com> writes:

Hello!

>> Yup, turned out patching GCC was too difficult. I'm experimenting a
>> filter over inputs passed to set-path-environment-variable in set-paths.
>
> This is also quite tricky, because the "libc" input passed to set-paths
> must not be removed from C_INCLUDE_PATH in (gnu packages commencement)
> for mes packages.

Do you have a patch/branch where I can see this related error?  If mes
builds are an exception, it could be nice to solve it there?  Or is
this a Qemu thing and is Qemu the only package that suffers from this?

Trying to help here, but I'm not sure if I understand what's going on...

Greetings,
janneke

-- 
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com

bug#36882: Qemu 4.2.0 build for x86_64-linux fails

[Mathieu Othacehe]

Hello Janneke,

> Do you have a patch/branch where I can see this related error?  If mes
> builds are an exception, it could be nice to solve it there?  Or is
> this a Qemu thing and is Qemu the only package that suffers from this?
>
> Trying to help here, but I'm not sure if I understand what's going on...

To summarize the situation, we are considering adding a dirty hack in
gnu-build-system so that "libc" input is removed from
C_INCLUDE_PATH. The reason is that GCC has already a libc hardcoded in
its include search-path. Having both libc, causes build failures for
QEMU on core-updates and possibly other programs playing around
-m<something> and -ffreestanding in GCC.

Anyway, Mes does not break anything, it's just that removing "libc"
input unconditionally in set-paths breaks Mes packages. That's why
Marius was suggesting renaming "libc" to "mes-libc" or something, so
that we can proceed.

Thanks,

Mathieu

bug#36882: Qemu 4.2.0 build for x86_64-linux fails

[Ludovic Courtès]

Hi!

Mathieu Othacehe <m.othacehe@gmail.com> skribis:

>> Yup, turned out patching GCC was too difficult. I'm experimenting a
>> filter over inputs passed to set-path-environment-variable in set-paths.
>
> This is also quite tricky, because the "libc" input passed to set-paths
> must not be removed from C_INCLUDE_PATH in (gnu packages commencement)
> for mes packages.

Yeah, fiddling with ‘set-paths’ sounds like opening a can of worms.

> So, maybe the best option is to patch QEMU directly, as proposed in
> the attached patch. WDYT?

I’m all in favor of that for now.  It’s likely that QEMU is the only
package having problems, as Marius noted (the GHC error sounds like it
could be something different), so I’d say go for the package-specific
solution for now.

Thanks,
Ludo’.

bug#36882: Qemu 4.2.0 build for x86_64-linux fails

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 1103 bytes --]

Ludovic Courtès <ludo@gnu.org> writes:

> Hi!
>
> Mathieu Othacehe <m.othacehe@gmail.com> skribis:
>
>>> Yup, turned out patching GCC was too difficult. I'm experimenting a
>>> filter over inputs passed to set-path-environment-variable in set-paths.
>>
>> This is also quite tricky, because the "libc" input passed to set-paths
>> must not be removed from C_INCLUDE_PATH in (gnu packages commencement)
>> for mes packages.
>
> Yeah, fiddling with ‘set-paths’ sounds like opening a can of worms.
>
>> So, maybe the best option is to patch QEMU directly, as proposed in
>> the attached patch. WDYT?
>
> I’m all in favor of that for now.  It’s likely that QEMU is the only
> package having problems, as Marius noted (the GHC error sounds like it
> could be something different), so I’d say go for the package-specific
> solution for now.

Indeed, I will push Mathieus commit shortly.

clang-runtime requires a similar fix for CPLUS_INCLUDE_PATH.  No idea
what's up with GHC@8.4 yet, but unsurprisingly removing libc from the
include paths does not make a difference.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

bug#36882: Qemu 4.2.0 build for x86_64-linux fails

[Mathieu Othacehe]

Hey,

> clang-runtime requires a similar fix for CPLUS_INCLUDE_PATH.  No idea
> what's up with GHC@8.4 yet, but unsurprisingly removing libc from the
> include paths does not make a difference.

Closing this one, thanks for pushing!

Mathieu