MIPS bootstrap problem in core-updates

MIPS bootstrap problem in core-updates

[Mark H Weaver]

In core-updates, 'glibc-final-with-bootstrap-bash' fails to build on
MIPS, because 'gcc-boot0' doesn't work.  The problem is that we've
switched to gcc-4.9, which on MIPS now emits a new ".nan" assembler
directive which is not understood by the assembler in binutils-2.23.2,
the version in our binutils-bootstrap-0.

The assembler in binutils-2.25 understands ".nan", and that version of
binutils is an input to 'glibc-final-with-bootstrap-bash' (via
%boot1-inputs), so one might expect this to work.  There's just one
problem: the assembler in 'binutils-boot0' is named
'mips64el-guix-linux-gnu-as', but the compiler in 'gcc-boot0' runs 'as'.

Historically, we've worked around this by including the bootstrap
binutils in %boot1-inputs:

(define %boot1-inputs
  ;; 2nd stage inputs.
  `(("gcc" ,gcc-boot0)
    ("binutils-cross" ,binutils-boot0)

    ;; Keep "binutils" here because the cross-gcc invokes `as', not the
    ;; cross-`as'.
    ,@%boot0-inputs))

The first solution that comes to mind is to remove this workaround, and
configure 'gcc-boot0' to use 'mips64el-guix-linux-gnu-as' instead of
'as' as its assembler.  This seems doable since 'binutils-boot0' is an
input to 'gcc-boot0'.

Alternatively, we could make a little 'wrapper' package that makes an
'as' symlink that points to 'mips64el-guix-linux-gnu-as', and then add
it to %boot1-inputs (instead of %boot0-inputs).

Ludovic, how do you think we should fix this?
Feel free to just do it :)

See below for an excerpted transcript of my investigation.

     Thanks,
       Mark


--8<---------------cut here---------------start------------->8---
mhw@librenote:/tmp/nix-build-glibc-intermediate-2.21.drv-0$ env -i bash
mhw@librenote:/tmp/nix-build-glibc-intermediate-2.21.drv-0$ source environment-variables 
mhw@librenote:/tmp/nix-build-glibc-intermediate-2.21.drv-0/build$ cd /tmp/nix-build-glibc-intermediate-2.21.drv-0/build
mhw@librenote:/tmp/nix-build-glibc-intermediate-2.21.drv-0/build$ mips64el-guix-linux-gnu-gcc -c   conftest.c
/tmp/nix-build-glibc-intermediate-2.21.drv-0/cc7dcByb.s: Assembler messages:
/tmp/nix-build-glibc-intermediate-2.21.drv-0/cc7dcByb.s:4: Error: unknown pseudo-op: `.nan'
mhw@librenote:/tmp/nix-build-glibc-intermediate-2.21.drv-0/build$ mips64el-guix-linux-gnu-gcc -v -c   conftest.c
Using built-in specs.
COLLECT_GCC=mips64el-guix-linux-gnu-gcc
Target: mips64el-guix-linux-gnu
Configured with: 
Thread model: single
gcc version 4.9.2 (GCC) 
COLLECT_GCC_OPTIONS='-v' '-c' '-mllsc' '-mno-shared' '-EL' '-mabi=n32'
 /gnu/store/qj7hbr1lnxw540vqi1vc07npy5dgli3w-gcc-cross-boot0-4.9.2/libexec/gcc/mips64el-guix-linux-gnu/4.9.2/cc1 -quiet -v conftest.c -mel -quiet -dumpbase conftest.c -mllsc -mno-shared -mabi=n32 -auxbase conftest -version -o /tmp/nix-build-glibc-intermediate-2.21.drv-0/ccDIBdKd.s
GNU C (GCC) version 4.9.2 (mips64el-guix-linux-gnu)
	compiled by GNU C version 4.8.2, GMP version 6.0.0, MPFR version 3.1.2, MPC version 1.0.3
GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
ignoring nonexistent directory "/gnu/store/h7ilhyrmbm1k73dp9d4rcv4hxgac36by-gcc-cross-boot0-4.9.2-lib/lib/gcc/mips64el-guix-linux-gnu/4.9.2/../../../../../../../mips64el-guix-linux-gnu/sys-include"
ignoring nonexistent directory "/gnu/store/h7ilhyrmbm1k73dp9d4rcv4hxgac36by-gcc-cross-boot0-4.9.2-lib/lib/gcc/mips64el-guix-linux-gnu/4.9.2/../../../../../../../mips64el-guix-linux-gnu/include"
#include "..." search starts here:
#include <...> search starts here:
 /gnu/store/h7ilhyrmbm1k73dp9d4rcv4hxgac36by-gcc-cross-boot0-4.9.2-lib/lib/gcc/mips64el-guix-linux-gnu/4.9.2/include
 /gnu/store/h7ilhyrmbm1k73dp9d4rcv4hxgac36by-gcc-cross-boot0-4.9.2-lib/lib/gcc/mips64el-guix-linux-gnu/4.9.2/include-fixed
End of search list.
GNU C (GCC) version 4.9.2 (mips64el-guix-linux-gnu)
	compiled by GNU C version 4.8.2, GMP version 6.0.0, MPFR version 3.1.2, MPC version 1.0.3
GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
Compiler executable checksum: dea46b113a5433c6a02c028045bcda23
COLLECT_GCC_OPTIONS='-v' '-c' '-mllsc' '-mno-shared' '-EL' '-mabi=n32'
 as -v -EL -O1 -no-mdebug -mabi=n32 -mno-shared -KPIC -o conftest.o /tmp/nix-build-glibc-intermediate-2.21.drv-0/ccDIBdKd.s
GNU assembler version 2.23.2 (mips64el-unknown-linux-gnu) using BFD version (GNU Binutils) 2.23.2
/tmp/nix-build-glibc-intermediate-2.21.drv-0/ccDIBdKd.s: Assembler messages:
/tmp/nix-build-glibc-intermediate-2.21.drv-0/ccDIBdKd.s:4: Error: unknown pseudo-op: `.nan'
mhw@librenote:/tmp/nix-build-glibc-intermediate-2.21.drv-0/build$ echo $PATH
/gnu/store/hb0j1ga45cd3vwnb1swyvv1gbaqsjika-texinfo-5.2/bin:/gnu/store/nmkffi8pjcf5aqimsv633qjbwl3pai4m-perl-5.16.1/bin:/gnu/store/qj7hbr1lnxw540vqi1vc07npy5dgli3w-gcc-cross-boot0-4.9.2/bin:/gnu/store/gnznbcg2cnwlmdi0mdj7ix8vfyl3i563-binutils-cross-boot0-2.25/bin:/gnu/store/cx944g8rh4gx2ar3nivk3i8imfcxqfgb-make-boot0-4.1/bin:/gnu/store/qhc2xv9n7gvsa3q38dmi6yvi987s70ys-diffutils-3.3/bin:/gnu/store/vbchmyzy7cylrzyfff04aixkss9a1hh8-findutils-4.4.2/bin:/gnu/store/zkxpl3ak7amfzz9473zkfp6wifp8v2a6-file-5.22/bin:/gnu/store/xn4c8dsj4cxrhk09nznv2xwvjj7f2rxy-gcc-bootstrap-0/bin:/gnu/store/g48d71zxfb7w7iskz35hijyxsxmly0m5-binutils-bootstrap-0/bin:/gnu/store/knz4nib6zrq1wy4bnqw55x6bln3nvqra-bootstrap-binaries-0/bin
mhw@librenote:/tmp/nix-build-glibc-intermediate-2.21.drv-0/build$ type as
as is /gnu/store/g48d71zxfb7w7iskz35hijyxsxmly0m5-binutils-bootstrap-0/bin/as
mhw@librenote:/tmp/nix-build-glibc-intermediate-2.21.drv-0/build$ ls /gnu/store/gnznbcg2cnwlmdi0mdj7ix8vfyl3i563-binutils-cross-boot0-2.25/bin
mips64el-guix-linux-gnu-addr2line  mips64el-guix-linux-gnu-nm
mips64el-guix-linux-gnu-ar	   mips64el-guix-linux-gnu-objcopy
mips64el-guix-linux-gnu-as	   mips64el-guix-linux-gnu-objdump
mips64el-guix-linux-gnu-c++filt    mips64el-guix-linux-gnu-ranlib
mips64el-guix-linux-gnu-elfedit    mips64el-guix-linux-gnu-readelf
mips64el-guix-linux-gnu-gprof	   mips64el-guix-linux-gnu-size
mips64el-guix-linux-gnu-ld	   mips64el-guix-linux-gnu-strings
mips64el-guix-linux-gnu-ld.bfd	   mips64el-guix-linux-gnu-strip
mhw@librenote:/tmp/nix-build-glibc-intermediate-2.21.drv-0/build$ mkdir tmp-bin
mhw@librenote:/tmp/nix-build-glibc-intermediate-2.21.drv-0/build$ ln -s /gnu/store/gnznbcg2cnwlmdi0mdj7ix8vfyl3i563-binutils-cross-boot0-2.25/bin/mips64el-guix-linux-gnu-as tmp-bin/as
mhw@librenote:/tmp/nix-build-glibc-intermediate-2.21.drv-0/build$ PATH=$(pwd)/tmp-bin:$PATH
mhw@librenote:/tmp/nix-build-glibc-intermediate-2.21.drv-0/build$ mips64el-guix-linux-gnu-gcc -c   conftest.c
mhw@librenote:/tmp/nix-build-glibc-intermediate-2.21.drv-0/build$
--8<---------------cut here---------------end--------------->8---

Re: MIPS bootstrap problem in core-updates

[Ludovic Courtès]

[-- Attachment #1: Type: text/plain, Size: 1118 bytes --]

Mark H Weaver <mhw@netris.org> skribis:

> In core-updates, 'glibc-final-with-bootstrap-bash' fails to build on
> MIPS, because 'gcc-boot0' doesn't work.  The problem is that we've
> switched to gcc-4.9, which on MIPS now emits a new ".nan" assembler
> directive which is not understood by the assembler in binutils-2.23.2,
> the version in our binutils-bootstrap-0.
>
> The assembler in binutils-2.25 understands ".nan", and that version of
> binutils is an input to 'glibc-final-with-bootstrap-bash' (via
> %boot1-inputs), so one might expect this to work.  There's just one
> problem: the assembler in 'binutils-boot0' is named
> 'mips64el-guix-linux-gnu-as', but the compiler in 'gcc-boot0' runs 'as'.
>
> Historically, we've worked around this by including the bootstrap
> binutils in %boot1-inputs:
>
> (define %boot1-inputs
>   ;; 2nd stage inputs.
>   `(("gcc" ,gcc-boot0)
>     ("binutils-cross" ,binutils-boot0)
>
>     ;; Keep "binutils" here because the cross-gcc invokes `as', not the
>     ;; cross-`as'.
>     ,@%boot0-inputs))

Thanks for the detailed analysis!

This (lightly tested) patch may work:


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Type: text/x-patch, Size: 1433 bytes --]

diff --git a/gnu/packages/commencement.scm b/gnu/packages/commencement.scm
index ab16660..6dd4d55 100644
--- a/gnu/packages/commencement.scm
+++ b/gnu/packages/commencement.scm
@@ -139,6 +139,19 @@
      (arguments
       `(#:guile ,%bootstrap-guile
         #:implicit-inputs? #f
+
+        #:phases (alist-cons-after
+                  'install 'add-gas-symlink
+                  (lambda* (#:key outputs #:allow-other-keys)
+                    ;; The cross-gcc invokes 'as', not the cross-'as', so add
+                    ;; an 'as' symlink.
+                    (let ((out (assoc-ref outputs "out")))
+                      (with-directory-excursion (string-append out "/bin")
+                        (symlink (string-append ,(boot-triplet) "-as")
+                                 "as")
+                        #t)))
+                  %standard-phases)
+
         ,@(substitute-keyword-arguments (package-arguments binutils)
             ((#:configure-flags cf)
              `(cons ,(string-append "--target=" (boot-triplet))
@@ -274,10 +287,7 @@
   ;; 2nd stage inputs.
   `(("gcc" ,gcc-boot0)
     ("binutils-cross" ,binutils-boot0)
-
-    ;; Keep "binutils" here because the cross-gcc invokes `as', not the
-    ;; cross-`as'.
-    ,@%boot0-inputs))
+    ,@(alist-delete "binutils" %boot0-inputs)))
 
 (define glibc-final-with-bootstrap-bash
   ;; The final libc, "cross-built".  If everything went well, the resulting

[-- Attachment #3: Type: text/plain, Size: 120 bytes --]


Could you try it on MIPS, on top of ‘core-updates’?

It looks like the simplest solution.

Thanks,
Ludo’.

Re: MIPS bootstrap problem in core-updates

[Mark H Weaver]

ludo@gnu.org (Ludovic Courtès) writes:

> This (lightly tested) patch may work:
>
> diff --git a/gnu/packages/commencement.scm b/gnu/packages/commencement.scm
> index ab16660..6dd4d55 100644
> --- a/gnu/packages/commencement.scm
> +++ b/gnu/packages/commencement.scm
> @@ -139,6 +139,19 @@
>       (arguments
>        `(#:guile ,%bootstrap-guile
>          #:implicit-inputs? #f
> +
> +        #:phases (alist-cons-after
> +                  'install 'add-gas-symlink
> +                  (lambda* (#:key outputs #:allow-other-keys)
> +                    ;; The cross-gcc invokes 'as', not the cross-'as', so add
> +                    ;; an 'as' symlink.
> +                    (let ((out (assoc-ref outputs "out")))
> +                      (with-directory-excursion (string-append out "/bin")
> +                        (symlink (string-append ,(boot-triplet) "-as")
> +                                 "as")
> +                        #t)))
> +                  %standard-phases)
> +
>          ,@(substitute-keyword-arguments (package-arguments binutils)
>              ((#:configure-flags cf)
>               `(cons ,(string-append "--target=" (boot-triplet))
> @@ -274,10 +287,7 @@
>    ;; 2nd stage inputs.
>    `(("gcc" ,gcc-boot0)
>      ("binutils-cross" ,binutils-boot0)
> -
> -    ;; Keep "binutils" here because the cross-gcc invokes `as', not the
> -    ;; cross-`as'.
> -    ,@%boot0-inputs))
> +    ,@(alist-delete "binutils" %boot0-inputs)))
>  
>  (define glibc-final-with-bootstrap-bash
>    ;; The final libc, "cross-built".  If everything went well, the resulting

Sounds good.

> Could you try it on MIPS, on top of ‘core-updates’?

I tried this and it turned out that 'ld' was also needed.  I added an
'ld' symlink and then discovered than 'ar' was needed as well.  I'm now
trying it with symlinks for _all_ of the programs in binutils' /bin
directory.

To be continued...

     Thanks,
       Mark

Re: MIPS bootstrap problem in core-updates

[Mark H Weaver]

Mark H Weaver <mhw@netris.org> writes:

> ludo@gnu.org (Ludovic Courtès) writes:
>
>> This (lightly tested) patch may work:
>>
>> diff --git a/gnu/packages/commencement.scm b/gnu/packages/commencement.scm
>> index ab16660..6dd4d55 100644
>> --- a/gnu/packages/commencement.scm
>> +++ b/gnu/packages/commencement.scm
>> @@ -139,6 +139,19 @@
>>       (arguments
>>        `(#:guile ,%bootstrap-guile
>>          #:implicit-inputs? #f
>> +
>> +        #:phases (alist-cons-after
>> +                  'install 'add-gas-symlink
>> +                  (lambda* (#:key outputs #:allow-other-keys)
>> +                    ;; The cross-gcc invokes 'as', not the cross-'as', so add
>> +                    ;; an 'as' symlink.
>> +                    (let ((out (assoc-ref outputs "out")))
>> +                      (with-directory-excursion (string-append out "/bin")
>> +                        (symlink (string-append ,(boot-triplet) "-as")
>> +                                 "as")
>> +                        #t)))
>> +                  %standard-phases)
>> +
>>          ,@(substitute-keyword-arguments (package-arguments binutils)
>>              ((#:configure-flags cf)
>>               `(cons ,(string-append "--target=" (boot-triplet))
>> @@ -274,10 +287,7 @@
>>    ;; 2nd stage inputs.
>>    `(("gcc" ,gcc-boot0)
>>      ("binutils-cross" ,binutils-boot0)
>> -
>> -    ;; Keep "binutils" here because the cross-gcc invokes `as', not the
>> -    ;; cross-`as'.
>> -    ,@%boot0-inputs))
>> +    ,@(alist-delete "binutils" %boot0-inputs)))
>>  
>>  (define glibc-final-with-bootstrap-bash
>>    ;; The final libc, "cross-built".  If everything went well, the resulting
>
> Sounds good.
>
>> Could you try it on MIPS, on top of ‘core-updates’?
>
> I tried this and it turned out that 'ld' was also needed.  I added an
> 'ld' symlink and then discovered than 'ar' was needed as well.  I'm now
> trying it with symlinks for _all_ of the programs in binutils' /bin
> directory.

... and that finally fixed it.  Pushed to core-updates in commit
f8badf151b4d2d966a1ebeba0345dfc9e4564d01.

     Thanks,
       Mark

Re: MIPS bootstrap problem in core-updates

[Ludovic Courtès]

Mark H Weaver <mhw@netris.org> skribis:

> Mark H Weaver <mhw@netris.org> writes:
>
>> ludo@gnu.org (Ludovic Courtès) writes:
>>
>>> This (lightly tested) patch may work:
>>>
>>> diff --git a/gnu/packages/commencement.scm b/gnu/packages/commencement.scm
>>> index ab16660..6dd4d55 100644
>>> --- a/gnu/packages/commencement.scm
>>> +++ b/gnu/packages/commencement.scm
>>> @@ -139,6 +139,19 @@
>>>       (arguments
>>>        `(#:guile ,%bootstrap-guile
>>>          #:implicit-inputs? #f
>>> +
>>> +        #:phases (alist-cons-after
>>> +                  'install 'add-gas-symlink
>>> +                  (lambda* (#:key outputs #:allow-other-keys)
>>> +                    ;; The cross-gcc invokes 'as', not the cross-'as', so add
>>> +                    ;; an 'as' symlink.
>>> +                    (let ((out (assoc-ref outputs "out")))
>>> +                      (with-directory-excursion (string-append out "/bin")
>>> +                        (symlink (string-append ,(boot-triplet) "-as")
>>> +                                 "as")
>>> +                        #t)))
>>> +                  %standard-phases)
>>> +
>>>          ,@(substitute-keyword-arguments (package-arguments binutils)
>>>              ((#:configure-flags cf)
>>>               `(cons ,(string-append "--target=" (boot-triplet))
>>> @@ -274,10 +287,7 @@
>>>    ;; 2nd stage inputs.
>>>    `(("gcc" ,gcc-boot0)
>>>      ("binutils-cross" ,binutils-boot0)
>>> -
>>> -    ;; Keep "binutils" here because the cross-gcc invokes `as', not the
>>> -    ;; cross-`as'.
>>> -    ,@%boot0-inputs))
>>> +    ,@(alist-delete "binutils" %boot0-inputs)))
>>>  
>>>  (define glibc-final-with-bootstrap-bash
>>>    ;; The final libc, "cross-built".  If everything went well, the resulting
>>
>> Sounds good.
>>
>>> Could you try it on MIPS, on top of ‘core-updates’?
>>
>> I tried this and it turned out that 'ld' was also needed.  I added an
>> 'ld' symlink and then discovered than 'ar' was needed as well.  I'm now
>> trying it with symlinks for _all_ of the programs in binutils' /bin
>> directory.
>
> ... and that finally fixed it.  Pushed to core-updates in commit
> f8badf151b4d2d966a1ebeba0345dfc9e4564d01.

Great, thanks!

Ludo’.