ld-wrapper is broken in core-updates due to guile-2.2 (host running guile-2.0)

ld-wrapper is broken in core-updates due to guile-2.2 (host running guile-2.0)

[Sergei Trofimovich]

[-- Attachment #1: Type: text/plain, Size: 4187 bytes --]

Efraim noticed core-updates breakage on package libgpg-error.

TL;DR:

  ld-wrapper package embeds guile-2.2 interpreter
  to interpret ld.go bytecode built by guile-2.0.
  I have no idea how to fix it but still decided to
  share my findings so far.

Longer story:

The sympthon is broken ld:

    $ ./pre-inst-env guix build libgpg-error
    ...
    checking whether the C compiler works... no
    configure: error: in `/tmp/guix-build-libgpg-error-1.26.drv-0/libgpg-error-1.26':
    configure: error: C compiler cannot create executables
    See `config.log' for more details

Namely minimal linking (from config.log) fails as:

  configure:3724: checking whether the C compiler works
  configure:3746: gcc    conftest.c  >&5
  Backtrace:
           5 (apply-smob/1 #<catch-closure 1d14100>)
  In ice-9/boot-9.scm:
    710:2  4 (call-with-prompt ("prompt") #<procedure 1d286a0 at ic?> ?)
  In ice-9/eval.scm:
    619:8  3 (_ #(#(#<directory (guile-user) 1dd01e0>)))
  In ice-9/command-line.scm:
   181:18  2 (_ #<input: string 1dfff50>)
  In unknown file:
           1 (eval (load-compiled "/gnu/store/qv0w20gdi57ybganx82?") #)
           0 (load-compiled/vm "/gnu/store/qv0w20gdi57ybganx82w8ymk1?")

  ERROR: In procedure load-compiled/vm:
  ERROR: In procedure load-thunk-from-memory: No such file or directory
  collect2: error: ld returned 1 exit status
  configure:3750: $? = 1
  configure:3788: result: no
  configure: failed program was:
  | /* confdefs.h */
  | int
  | main ()
  | {
  |   return 0;
  | }
  configure:3793: error: in `/tmp/guix-build-libgpg-error-1.26.drv-0/libgpg-error-1.26':
  configure:3795: error: C compiler cannot create executables
  See `config.log' for more details

That's right: it's the guile's backtrace from gcc invocation.
  /tmp/guix-build-libgpg-error-1.26.drv-0/libgpg-error-1.26 $ ld
  Backtrace:
           5 (apply-smob/1 #<catch-closure ae2c20>)
  In ice-9/boot-9.scm:
    710:2  4 (call-with-prompt ("prompt") #<procedure aefa40 at ice…> …)
  In ice-9/eval.scm:
    619:8  3 (_ #(#(#<directory (guile-user) b781e0>)))
  In ice-9/command-line.scm:
   181:18  2 (_ #<input: string ba5f50>)
  In unknown file:
           1 (eval (load-compiled "/gnu/store/qv0w20gdi57ybganx82…") #)
           0 (load-compiled/vm "/gnu/store/qv0w20gdi57ybganx82w8ymk1…")

  ERROR: In procedure load-compiled/vm:
  ERROR: In procedure load-thunk-from-memory:

Backtrace tries to tell us that guile-2.2 cannot load ld.go file. Snippet from `which ld`:

  """
  exec /gnu/store/kri8a2dy6dy0cdg0z41djn0pnn08515z-guile-2.2.0/bin/guile -c "(load-compiled \"/gnu/store/qv0w20gdi57ybganx82w8ymk1cwv7n2c-ld-wrapper-0/bin/ld.go\") (apply $main (cdr (command-line)))" "$@"
  """

File confirms bytecode is from incompatible 2.0 version:

  $ file /gnu/store/qv0w20gdi57ybganx82w8ymk1cwv7n2c-ld-wrapper-0/bin/ld.go
  /gnu/store/qv0w20gdi57ybganx82w8ymk1cwv7n2c-ld-wrapper-0/bin/ld.go: Guile Object, little endian, 64bit, bytecode v2.0

I think the problem comes from 'gnu/packages/base.scm:make-ld-wrapper' function
which uses host's guile to compile ld.go, not bootstrapped guile:

http://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/base.scm#n458

(define* (make-ld-wrapper name #:key
                          (target (const #f))
                          binutils
                          (guile (canonical-package guile-2.0))
                          (bash (canonical-package bash))
                          (guile-for-build guile))
...
    (arguments
     (let ((target (target (%current-system))))
       `(#:guile ,guile-for-build
         #:modules ((guix build utils))
         #:builder (begin
...
                       (chmod ld #o555)
                       (compile-file ld #:output-file go))))))

Which guile is being used here for compile-file? 'build-for-build'
or current host's guile? Looks like the requirement here is that
both mush be of the same version.

If it really is a requirement to run guile-2.2 perhaps adding early
assertion would be nicer? I don't know how to add it either :)

Thanks!

-- 

  Sergei

[-- Attachment #2: Цифровая подпись OpenPGP --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: ld-wrapper is broken in core-updates due to guile-2.2 (host running guile-2.0)

[Sergei Trofimovich]

[-- Attachment #1: Type: text/plain, Size: 3617 bytes --]

On Fri, 14 Apr 2017 21:59:47 +0100
Sergei Trofimovich <slyfox@inbox.ru> wrote:

> Efraim noticed core-updates breakage on package libgpg-error.
> 
> TL;DR:
> 
>   ld-wrapper package embeds guile-2.2 interpreter
>   to interpret ld.go bytecode built by guile-2.0.
>   I have no idea how to fix it but still decided to
>   share my findings so far.

Confirmed guile-2.2 is triggering failure:

reverting commit 34d624cef51f0e4eb4e888622b60fc2befe31fa7
("gnu: Build derivations with Guile 2.2 by default.") fixes libgpg-error build.

Another less obvious problem caused by guile-2.2 is triggered in bash-static:

  guix-core-updates $ ./pre-inst-env guix build bash-static
  The following derivation will be built:
     /gnu/store/hr3g5avmb9fmgr047990r9mc6gmnq674-bash-static-4.4.12.drv
  @ build-started /gnu/store/hr3g5avmb9fmgr047990r9mc6gmnq674-bash-static-4.4.12.drv - x86_64-linux /var/log/guix/drvs/hr//3g5avmb9fmgr047990r9mc6gmnq674-bash-static-4.4.12.drv.bz2
  ice-9/psyntax.scm:1534:32: In procedure expand-macro:
  ice-9/psyntax.scm:1534:32: Syntax error:
  /gnu/store/bc7pyl426zaa2489qp97bqaycv2ir2v1-bash-static-4.4.12-guile-builder:1:2300: source expression failed to match any pattern in form (%modify-phases phases* (delete (quote move-development-files)))
  builder for `/gnu/store/hr3g5avmb9fmgr047990r9mc6gmnq674-bash-static-4.4.12.drv' failed with exit code 1
  @ build-failed /gnu/store/hr3g5avmb9fmgr047990r9mc6gmnq674-bash-static-4.4.12.drv - 1 builder for `/gnu/store/hr3g5avmb9fmgr047990r9mc6gmnq674-bash-static-4.4.12.drv' failed with exit code 1
  guix build: error: build failed: build of `/gnu/store/hr3g5avmb9fmgr047990r9mc6gmnq674-bash-static-4.4.12.drv' failed

It looks like guile-2.2 has different syntax checking rules which
is triggered by the following modify-phases call site:

[ http://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/bash.scm#n204 ]
  ...
         ((#:phases phases)
          `(modify-phases ,phases
             ;; No loadable modules.
             (delete 'move-development-files))))))))

[ http://git.savannah.gnu.org/cgit/guix.git/tree/guix/build/utils.scm#n550 ]
  ...
  (define-syntax-rule (modify-phases phases mod-spec ...)
    "Modify PHASES sequentially as per each MOD-SPEC, which may have one of the
  following forms:

    (delete <old-phase-name>)
    (replace <old-phase-name> <new-phase>)
    (add-before <old-phase-name> <new-phase-name> <new-phase>)
    (add-after <old-phase-name> <new-phase-name> <new-phase>)

  Where every <*-phase-name> is an expression evaluating to a symbol, and
  <new-phase> an expression evaluating to a procedure."
    (let* ((phases* phases)
           (phases* (%modify-phases phases* mod-spec))
           ...)
      phases*))

  (define-syntax %modify-phases
    (syntax-rules (delete replace add-before add-after)
      ((_ phases (delete old-phase-name))
       (alist-delete old-phase-name phases))
      ((_ phases (replace old-phase-name new-phase))
       (alist-replace old-phase-name new-phase phases))
      ((_ phases (add-before old-phase-name new-phase-name new-phase))
       (alist-cons-before old-phase-name new-phase-name new-phase phases))
      ((_ phases (add-after old-phase-name new-phase-name new-phase))
       (alist-cons-after old-phase-name new-phase-name new-phase phases))))

CCed Andy in case it's an obvious bug in guix or guile syntax rewriter.

Reverting commit 34d624cef51f0e4eb4e888622b60fc2befe31fa7
("gnu: Build derivations with Guile 2.2 by default.") fixes bash-static as well.

-- 

  Sergei

[-- Attachment #2: Цифровая подпись OpenPGP --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: ld-wrapper is broken in core-updates due to guile-2.2 (host running guile-2.0)

[Ludovic Courtès]

Hello,

Sergei Trofimovich <slyfox@inbox.ru> skribis:

> Efraim noticed core-updates breakage on package libgpg-error.
>
> TL;DR:
>
>   ld-wrapper package embeds guile-2.2 interpreter
>   to interpret ld.go bytecode built by guile-2.0.
>   I have no idea how to fix it but still decided to
>   share my findings so far.
>
> Longer story:

[...]

> http://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/base.scm#n458
>
> (define* (make-ld-wrapper name #:key
>                           (target (const #f))
>                           binutils
>                           (guile (canonical-package guile-2.0))
>                           (bash (canonical-package bash))
>                           (guile-for-build guile))
> ...
>     (arguments
>      (let ((target (target (%current-system))))
>        `(#:guile ,guile-for-build
>          #:modules ((guix build utils))
>          #:builder (begin
> ...
>                        (chmod ld #o555)
>                        (compile-file ld #:output-file go))))))
>
> Which guile is being used here for compile-file? 'build-for-build'
> or current host's guile? Looks like the requirement here is that
> both mush be of the same version.

Indeed, good catch!  You were on the right track.

I believe this is fixed by this commit:

  https://git.savannah.gnu.org/cgit/guix.git/commit/?h=core-updates&id=78dea6f1d4a85dd9571ccbd604239912ba3a18b8

The problem here is that ‘ld-wrapper’ in commencement.scm was defined in
such a way that #:guile and #:guile-for-build did not match, so we ended
up compiling with 2.0 (%bootstrap-guile) and running with 2.2
(guile-final).

Thank you!

Ludo’.