Problem with removing <config.h> from gen-scmconfig when cross-compiling

Problem with removing <config.h> from gen-scmconfig when cross-compiling

[Mark H Weaver]

Hi Ludovic,

Commit 8cb0d6d7fa9aaac316c29a64c541336b51b6f93d "build: Don't include
<config.h> in native programs when cross-compiling." apparently broke
cross-compiling.  Madsy on #guile, who successfully cross-compiled
e1bb79f for mingw, ran into this problem with 21a7ba9:

--8<---------------cut here---------------start------------->8---
make[1]: Entering directory `/home/madsy/mingw/home/madsy/test/guile-2.0.9.f239-21a7b-dirty/libguile'
  GEN      gen-scmconfig.o
gen-scmconfig.c: In function 'main':
gen-scmconfig.c:245:39: error: 'SIZEOF_CHAR' undeclared (first use in this function)
gen-scmconfig.c:245:39: note: each undeclared identifier is reported only once for each function it appears in
gen-scmconfig.c:246:48: error: 'SIZEOF_UNSIGNED_CHAR' undeclared (first use in this function)
gen-scmconfig.c:247:40: error: 'SIZEOF_SHORT' undeclared (first use in this function)
gen-scmconfig.c:248:49: error: 'SIZEOF_UNSIGNED_SHORT' undeclared (first use in this function)
gen-scmconfig.c:249:39: error: 'SIZEOF_LONG' undeclared (first use in this function)
gen-scmconfig.c:250:48: error: 'SIZEOF_UNSIGNED_LONG' undeclared (first use in this function)
gen-scmconfig.c:251:38: error: 'SIZEOF_INT' undeclared (first use in this function)
gen-scmconfig.c:252:47: error: 'SIZEOF_UNSIGNED_INT' undeclared (first use in this function)
gen-scmconfig.c:253:41: error: 'SIZEOF_SIZE_T' undeclared (first use in this function)
gen-scmconfig.c:258:44: error: 'SIZEOF_LONG_LONG' undeclared (first use in this function)
gen-scmconfig.c:259:53: error: 'SIZEOF_UNSIGNED_LONG_LONG' undeclared (first use in this function)
gen-scmconfig.c:275:43: error: 'SIZEOF_INTMAX_T' undeclared (first use in this function)
gen-scmconfig.c:279:43: error: 'SIZEOF___INT64' undeclared (first use in this function)
gen-scmconfig.c:296:50: error: 'SIZEOF_PTRDIFF_T' undeclared (first use in this function)
gen-scmconfig.c:300:43: error: 'SIZEOF_INTPTR_T' undeclared (first use in this function)
gen-scmconfig.c:302:44: error: 'SIZEOF_UINTPTR_T' undeclared (first use in this function)
make[1]: *** [gen-scmconfig.o] Error 1
make[1]: Leaving directory `/home/madsy/mingw/home/madsy/test/guile-2.0.9.239-21a7b-dirty/libguile'
make: *** [libguile/scmconfig.h] Error 2
--8<---------------cut here---------------end--------------->8---

      Mark

Re: Problem with removing <config.h> from gen-scmconfig when cross-compiling

[Ludovic Courtès]

Mark H Weaver <mhw@netris.org> skribis:

> Commit 8cb0d6d7fa9aaac316c29a64c541336b51b6f93d "build: Don't include
> <config.h> in native programs when cross-compiling." apparently broke
> cross-compiling.  Madsy on #guile, who successfully cross-compiled
> e1bb79f for mingw, ran into this problem with 21a7ba9:
>
> make[1]: Entering directory `/home/madsy/mingw/home/madsy/test/guile-2.0.9.f239-21a7b-dirty/libguile'
>   GEN      gen-scmconfig.o
> gen-scmconfig.c: In function 'main':
> gen-scmconfig.c:245:39: error: 'SIZEOF_CHAR' undeclared (first use in this function)
> gen-scmconfig.c:245:39: note: each undeclared identifier is reported only once for each function it appears in

Arrgggh, indeed, sorry!

We should revert that patch, but then I’m not sure how to properly fix
the initial problem, which is that gen-scmconfig.c ends up using
Gnulib’s <stdio.h> etc.

Could Madsy check, after reverting the patch, if it works when doing
things in this order:

  /* Include the “real” headers.  */
  #include <stdio.h>
  #include <stdlib.h>

  /* Get the useful definitions.  */
  #include <config.h>

It has to be tested when compiling to a different OS, like MinGW as
Madsy did.

Ludo’.

Re: Problem with removing <config.h> from gen-scmconfig when cross-compiling

[Mark H Weaver]

ludo@gnu.org (Ludovic Courtès) writes:

> Mark H Weaver <mhw@netris.org> skribis:
>
>> Commit 8cb0d6d7fa9aaac316c29a64c541336b51b6f93d "build: Don't include
>> <config.h> in native programs when cross-compiling." apparently broke
>> cross-compiling.  Madsy on #guile, who successfully cross-compiled
>> e1bb79f for mingw, ran into this problem with 21a7ba9:
>>
>> make[1]: Entering directory `/home/madsy/mingw/home/madsy/test/guile-2.0.9.f239-21a7b-dirty/libguile'
>>   GEN      gen-scmconfig.o
>> gen-scmconfig.c: In function 'main':
>> gen-scmconfig.c:245:39: error: 'SIZEOF_CHAR' undeclared (first use in this function)
>> gen-scmconfig.c:245:39: note: each undeclared identifier is reported only once for each function it appears in
>
> Arrgggh, indeed, sorry!
>
> We should revert that patch, but then I’m not sure how to properly fix
> the initial problem, which is that gen-scmconfig.c ends up using
> Gnulib’s <stdio.h> etc.

Madsy told me that the only problem he had when cross-compiling e1bb79f
was in c-tokenize.c.  He worked around it by simply removing "#include
<config.h>" from that file.  gen-scmconfig.c was not a problem.

The specific problem was that c-tokenize.c includes <stdlib.h>,
but gen-scmconfig.c doesn't include <stdlib.h>.

I agree that this is fragile and should be reworked somehow, but for
2.0.10, I wonder if we could just revert the part of 8cb0d6d having to
do with gen-scmconfig.c.

What do you think?

> Could Madsy check, after reverting the patch, if it works when doing
> things in this order:
>
>   /* Include the “real” headers.  */
>   #include <stdio.h>
>   #include <stdlib.h>
>
>   /* Get the useful definitions.  */
>   #include <config.h>

My impression is that we can't rely on this level of help from Madsy, at
least not in the short term.

Didn't hydra.nixos.org do MinGW cross-builds of Guile in the past?
If so, what happened to those?

     Mark

Re: Problem with removing <config.h> from gen-scmconfig when cross-compiling

[Ludovic Courtès]

Mark H Weaver <mhw@netris.org> skribis:

> ludo@gnu.org (Ludovic Courtès) writes:
>
>> Mark H Weaver <mhw@netris.org> skribis:
>>
>>> Commit 8cb0d6d7fa9aaac316c29a64c541336b51b6f93d "build: Don't include
>>> <config.h> in native programs when cross-compiling." apparently broke
>>> cross-compiling.  Madsy on #guile, who successfully cross-compiled
>>> e1bb79f for mingw, ran into this problem with 21a7ba9:
>>>
>>> make[1]: Entering directory `/home/madsy/mingw/home/madsy/test/guile-2.0.9.f239-21a7b-dirty/libguile'
>>>   GEN      gen-scmconfig.o
>>> gen-scmconfig.c: In function 'main':
>>> gen-scmconfig.c:245:39: error: 'SIZEOF_CHAR' undeclared (first use in this function)
>>> gen-scmconfig.c:245:39: note: each undeclared identifier is reported only once for each function it appears in
>>
>> Arrgggh, indeed, sorry!
>>
>> We should revert that patch, but then I’m not sure how to properly fix
>> the initial problem, which is that gen-scmconfig.c ends up using
>> Gnulib’s <stdio.h> etc.
>
> Madsy told me that the only problem he had when cross-compiling e1bb79f
> was in c-tokenize.c.  He worked around it by simply removing "#include
> <config.h>" from that file.  gen-scmconfig.c was not a problem.
>
> The specific problem was that c-tokenize.c includes <stdlib.h>,
> but gen-scmconfig.c doesn't include <stdlib.h>.
>
> I agree that this is fragile and should be reworked somehow, but for
> 2.0.10, I wonder if we could just revert the part of 8cb0d6d having to
> do with gen-scmconfig.c.
>
> What do you think?

You’re right.  So for 2.0.10, just revert, and then remove #include
<config.h> altogether in c-tokenize.lex with a comment saying why.

Could you do that?

> Didn't hydra.nixos.org do MinGW cross-builds of Guile in the past?
> If so, what happened to those?

Yes, we had that, but I removed it in
<http://git.sv.gnu.org/cgit/hydra-recipes.git/commit/?id=aea7182a62f8c6f7a04a7e89fbdd59155d5f34ff>
because the MinGW cross toolchain in Nixpkgs had bitrot.  :-/

(BTW, we can test cross-compilation to GNU/Linux with
guix build guile --with-source=guile-2.0.9.xyz.tar.xz --target=mips64el-linux-gnu.)

Ludo’.

Re: Problem with removing <config.h> from gen-scmconfig when cross-compiling

[Mark H Weaver]

ludo@gnu.org (Ludovic Courtès) writes:

> Mark H Weaver <mhw@netris.org> skribis:
>
>> I agree that this is fragile and should be reworked somehow, but for
>> 2.0.10, I wonder if we could just revert the part of 8cb0d6d having to
>> do with gen-scmconfig.c.
>>
>> What do you think?
>
> You’re right.  So for 2.0.10, just revert, and then remove #include
> <config.h> altogether in c-tokenize.lex with a comment saying why.

I don't think we can remove it altogether.  On some non-GNU systems,
some Gnulib's headers complain if we haven't yet included <config.h>.
This was the motivation for e1bb79fde62e678c0f8ceb32c7edd2dab0201a5c,
where you moved the "#include <config.h>" to the top of the c-tokenize.

Dale Evans pointed out that GCC runs the autoconf tests twice when
cross-compiling: once for the build machine and once for the host
machine.  I suspect that this is the proper solution for us, so we'd
end up with two config.h files.

> (BTW, we can test cross-compilation to GNU/Linux with
> guix build guile --with-source=guile-2.0.9.xyz.tar.xz --target=mips64el-linux-gnu.)

Nice!  I can use this to test any proposed solution(s) we come up with.

    Thanks,
      Mark

Re: Problem with removing <config.h> from gen-scmconfig when cross-compiling

[Ludovic Courtès]

Mark H Weaver <mhw@netris.org> skribis:

> ludo@gnu.org (Ludovic Courtès) writes:
>
>> Mark H Weaver <mhw@netris.org> skribis:
>>
>>> I agree that this is fragile and should be reworked somehow, but for
>>> 2.0.10, I wonder if we could just revert the part of 8cb0d6d having to
>>> do with gen-scmconfig.c.
>>>
>>> What do you think?
>>
>> You’re right.  So for 2.0.10, just revert, and then remove #include
>> <config.h> altogether in c-tokenize.lex with a comment saying why.
>
> I don't think we can remove it altogether.  On some non-GNU systems,
> some Gnulib's headers complain if we haven't yet included <config.h>.
> This was the motivation for e1bb79fde62e678c0f8ceb32c7edd2dab0201a5c,
> where you moved the "#include <config.h>" to the top of the c-tokenize.

Right.

So, should we keep just the c-tokenize.lex part of
8cb0d6d7fa9aaac316c29a64c541336b51b6f93d, and revert the rest?

> Dale Evans pointed out that GCC runs the autoconf tests twice when
> cross-compiling: once for the build machine and once for the host
> machine.  I suspect that this is the proper solution for us, so we'd
> end up with two config.h files.

Yes, but GCC’s configure machinery is really the next level...

Ludo’.

Re: Problem with removing <config.h> from gen-scmconfig when cross-compiling

[Mark H Weaver]

ludo@gnu.org (Ludovic Courtès) writes:

> So, should we keep just the c-tokenize.lex part of
> 8cb0d6d7fa9aaac316c29a64c541336b51b6f93d, and revert the rest?

Sounds good.  I did this in 17d4daa8bd11176c2ebe0d35ac48da3d247094ff.

    Thanks,
      Mark

Re: Problem with removing <config.h> from gen-scmconfig when cross-compiling

[Doug Evans]

On Thu, Mar 13, 2014 at 11:05 AM, Ludovic Courtès <ludo@gnu.org> wrote:
> Mark H Weaver <mhw@netris.org> skribis:
>
>> Dale Evans pointed out that GCC runs the autoconf tests twice when
>> cross-compiling: once for the build machine and once for the host
>> machine.  I suspect that this is the proper solution for us, so we'd
>> end up with two config.h files.
>
> Yes, but GCC's configure machinery is really the next level...

What does "the next level" mean?

Re: Problem with removing <config.h> from gen-scmconfig when cross-compiling

[Ludovic Courtès]

Doug Evans <xdje42@gmail.com> skribis:

> On Thu, Mar 13, 2014 at 11:05 AM, Ludovic Courtès <ludo@gnu.org> wrote:
>> Mark H Weaver <mhw@netris.org> skribis:
>>
>>> Dale Evans pointed out that GCC runs the autoconf tests twice when
>>> cross-compiling: once for the build machine and once for the host
>>> machine.  I suspect that this is the proper solution for us, so we'd
>>> end up with two config.h files.
>>
>> Yes, but GCC's configure machinery is really the next level...
>
> What does "the next level" mean?

I mean it’s sophisticated: creates build sub-directories for the host
and build, runs configure and make in there, etc.

Given that Guile has only two files to be compiled for the build machine
and that they use only stdio.h and stdlib.h, I think that we can find a
way to do what we want without resorting to something as sophisticated.

Ludo’.

Re: Problem with removing <config.h> from gen-scmconfig when cross-compiling

[Doug Evans]

On Mon, Mar 17, 2014 at 1:32 PM, Ludovic Courtès <ludo@gnu.org> wrote:
> Doug Evans <xdje42@gmail.com> skribis:
>
>> On Thu, Mar 13, 2014 at 11:05 AM, Ludovic Courtès <ludo@gnu.org> wrote:
>>> Mark H Weaver <mhw@netris.org> skribis:
>>>
>>>> Dale Evans pointed out that GCC runs the autoconf tests twice when
>>>> cross-compiling: once for the build machine and once for the host
>>>> machine.  I suspect that this is the proper solution for us, so we'd
>>>> end up with two config.h files.
>>>
>>> Yes, but GCC's configure machinery is really the next level...
>>
>> What does "the next level" mean?
>
> I mean it's sophisticated: creates build sub-directories for the host
> and build, runs configure and make in there, etc.
>
> Given that Guile has only two files to be compiled for the build machine
> and that they use only stdio.h and stdlib.h, I think that we can find a
> way to do what we want without resorting to something as sophisticated.

It's your call (or at least it's not mine), but I wouldn't call it
sophisticated.
It's pretty straightforward and it does have nice properties.

Re: Problem with removing <config.h> from gen-scmconfig when cross-compiling

[Mark H Weaver]

ludo@gnu.org (Ludovic Courtès) writes:

> (BTW, we can test cross-compilation to GNU/Linux with
> guix build guile --with-source=guile-2.0.9.xyz.tar.xz --target=mips64el-linux-gnu.)

I tried this on my x86_64 box with guix master (v0.5-355-g9037ea2),
freshly built (autoreconf -vfi, configure, make clean, etc) and got
this:

--8<---------------cut here---------------start------------->8---
mhw@tines:~/guix$ ./pre-inst-env guix build -K guile --with-source=/home/mhw/guile-2.0.9.239-21a7b-dirty.tar.xz --target=mips64el-linux-gnu
guix build: warning: ambiguous package specification `guile'
guix build: warning: choosing guile-2.0.9 from gnu/packages/base.scm:1044:33
guix build: error: gnu/packages/bootstrap.scm:201:3: guile-bootstrap-2.0: build system `raw' does not support cross builds
--8<---------------cut here---------------end--------------->8---

I also tried --target=mips64el-linux, but that made no difference.
Any ideas?

     Mark

Re: Problem with removing <config.h> from gen-scmconfig when cross-compiling

[Ludovic Courtès]

Mark H Weaver <mhw@netris.org> skribis:

> ludo@gnu.org (Ludovic Courtès) writes:
>
>> (BTW, we can test cross-compilation to GNU/Linux with
>> guix build guile --with-source=guile-2.0.9.xyz.tar.xz --target=mips64el-linux-gnu.)
>
> I tried this on my x86_64 box with guix master (v0.5-355-g9037ea2),
> freshly built (autoreconf -vfi, configure, make clean, etc) and got
> this:
>
> mhw@tines:~/guix$ ./pre-inst-env guix build -K guile --with-source=/home/mhw/guile-2.0.9.239-21a7b-dirty.tar.xz --target=mips64el-linux-gnu
> guix build: warning: ambiguous package specification `guile'
> guix build: warning: choosing guile-2.0.9 from gnu/packages/base.scm:1044:33
> guix build: error: gnu/packages/bootstrap.scm:201:3: guile-bootstrap-2.0: build system `raw' does not support cross builds

Oh, funny.  Here’s what I get:

--8<---------------cut here---------------start------------->8---
$ ./pre-inst-env guix build -K guile --with-source=$HOME/src/guile/guile-2.0.9.239-21a7b.tar.xz --target=mips64el-linux-gnu -n
guix build: warning: ambiguous package specification `guile'
guix build: warning: choosing guile-2.0.9 from gnu/packages/guile.scm:183:43
The following derivations would be built:
   /nix/store/ygji7r8cw9kq53r3y00fyksdck9rbs2l-guile-2.0.9.239-21a7b.drv
   /nix/store/h1f4d4l69svgj4rszz6pdyhz7rahcf2z-binutils-2.23.2.tar.bz2.drv
   /nix/store/gclaxgj109jmrrx795cg254vfykqwmn4-binutils-2.23.2.tar.xz.drv
   /nix/store/zj9nd1ysr1ga8w775l0b0q9m2jncvj5l-libelf-0.8.13.tar.gz.drv
   /nix/store/9zz4g05zagndaklff8jn695r46q85mf4-gcc-4.8.2.tar.xz.drv
   /nix/store/yivpp34iw8f13sf40mrxx0f92mpzmazs-libelf-0.8.13.drv
   /nix/store/9yccsrz3h41wd8nilnlyg4nva5gr9184-bash-light-4.2.drv
   /nix/store/ipy04pgr0r23nprwh39ijzwmikklxvyj-glibc-2.18.tar.xz.drv
   /nix/store/shm0vybdx80h4s5sagk1pk5mfvhkfa7l-gcc-cross-sans-libc-mips64el-linux-gnu-4.8.2.drv
   /nix/store/6f572clhayc3689i2jz3djqcslh4dn88-libtool-2.4.2.tar.xz.drv
   /nix/store/yifk89n5yn9ibjslkk4s0xn4giq9z6k5-readline-6.2.tar.xz.drv
   /nix/store/mrv5y4brg6cbx7kvib2f2l99v0wispyn-libffi-3.0.13.tar.xz.drv
   /nix/store/ycpg9709lzcvb8mxj50ims0kfs9h2bn2-libffi-3.0.13.drv
   /nix/store/v0jaav7l1vk0bjb9wm6vhac4kpj72cvl-readline-6.2.drv
   /nix/store/a6l4wslz8fh4fvb7xz2dnlb951nrgsbs-ncurses-5.9.drv
   /nix/store/p1k3b0xc1fjfdyy68vjzpbadysbjqzyc-bash-4.2.drv
   /nix/store/9w09gdiz6wabv3r82w2fk67plnck5zm1-libunistring-0.9.3.drv
   /nix/store/qx2a27higncxn4yybcx9402b9fwlm590-libtool-2.4.2.drv
   /nix/store/0lbhn2gifq7bdls5jafjk1gzkcp3zdc0-libgc-7.2d.drv
   /nix/store/kjg76c3wjpy5nx22jbp14ii9j55irs4i-gmp-5.1.3.drv
   /nix/store/2vrxad9jb73d4x5vicnpbbj638cvm0iw-glibc-cross-mips64el-linux-gnu-2.18.drv
   /nix/store/pm908g3hjclq5zvzri2n58icb1l2q8zm-linux-libre-headers-cross-mips64el-linux-gnu-3.3.8.drv
   /nix/store/293bbw8x4vmpg1kdjps31sw8r55527ml-guile-2.0.9.239-21a7b.drv
   /nix/store/ys42q97jxxylrgg9a021zih6pnwphszd-pkg-config-mips64el-linux-gnu-0.27.1.drv
   /nix/store/j4c0j5annyf1gvrpr9jfwmxan0ggv5w5-gcc-cross-mips64el-linux-gnu-4.8.2.drv
   /nix/store/x0fvh7r5nmac8r5f621vhdxw5mqgpk95-binutils-cross-mips64el-linux-gnu-2.23.2.drv
--8<---------------cut here---------------end--------------->8---

The solution is to specify the “right” Guile, unambiguously, with:

  -e '(@ (gnu packages guile) guile-2.0)'

and things should work as expected (with Guix commit 257b934 or later,
that is ;-)).

Thanks,
Ludo’.