bug#32894: Exception in validate-runpath phase

bug#32894: Exception in validate-runpath phase

[Julien Lepiller]

Hi,

I'm trying to create a new package for openjdk versions we don't have 
yet. While building openjdk 10 on top of core-updates (because gcc on 
master has a bug that prevents building openjdk 9 and 10), I get a 
stacktrace at the end of the validate-runpath phase:




starting phase `validate-runpath'
validating RUNPATH of 74 binaries in 
"/gnu/store/bdgbs6nsb1kzxpqmcxajjkvvkmk5kn72-openjdk-10+46/lib"...
Backtrace:
           11 (primitive-load "/gnu/store/cyxf063m59nb288xnpy94gr4chq…")
In ice-9/eval.scm:
    191:35 10 (_ _)
In srfi/srfi-1.scm:
    863:16  9 (every1 #<procedure 77aaa0 at /gnu/store/p9wwyq2jfq2pi…> …)
In 
/gnu/store/p9wwyq2jfq2piwyc01qgsxm3hsxg2bnv-module-import/guix/build/gnu-build-system.scm:
    799:28  8 (_ _)
    557:16  7 (validate-runpath #:validate-runpath? _ # _ #:outputs _)
In 
/gnu/store/p9wwyq2jfq2piwyc01qgsxm3hsxg2bnv-module-import/guix/build/utils.scm:
    536:23  6 (every* #<procedure validate (directory)> _)
    536:23  5 (every* #<procedure validate-needed-in-runpath (file #…> …)
In ice-9/boot-9.scm:
     829:9  4 (catch srfi-34 #<procedure 1023d40 at /gnu/store/p9wwy…> …)
In 
/gnu/store/p9wwyq2jfq2piwyc01qgsxm3hsxg2bnv-module-import/guix/build/gremlin.scm:
    305:26  3 (_)
In unknown file:
            2 (remove #<procedure libc-library? (lib)> (#))
            1 (find #<procedure 1023b80 at /gnu/store/p9wwyq2jfq2piw…> …)
            0 (string-prefix? "libanl.so" 3659183287175258 #<undefin…> …)

ERROR: In procedure string-prefix?:
In procedure string-prefix?: Wrong type argument in position 2 
(expecting string): 3659183287175258




I tried to investigate the issue and I have found that that running 
(validate-needed-in-runpath "/gnu/...") sometimes fails on some files. 
More specifically, it always succeeds on *.so files and on most 
*.debuginfo files, but it fails on these files:

/gnu/store/bdgbs6nsb1kzxpqmcxajjkvvkmk5kn72-openjdk-10+46/lib/libjsound.debuginfo
/gnu/store/bdgbs6nsb1kzxpqmcxajjkvvkmk5kn72-openjdk-10+46/lib/libjimage.debuginfo
/gnu/store/bdgbs6nsb1kzxpqmcxajjkvvkmk5kn72-openjdk-10+46/lib/libjaas_unix.debuginfo
/gnu/store/bdgbs6nsb1kzxpqmcxajjkvvkmk5kn72-openjdk-10+46/lib/libawt_xawt.debuginfo

with the following backtrace:



scheme@(guix build gremlin)> (validate-needed-in-runpath 
"/gnu/store/bdgbs6nsb1kzxpqmcxajjkvvkmk5kn72-openjdk-10+46/lib/libjsound.debuginfo")
ERROR: In procedure string-prefix?:
In procedure string-prefix?: Wrong type argument in position 2 
(expecting string): 1

Entering a new prompt.  Type `,bt' for a backtrace or `,q' to continue.
scheme@(guix build gremlin) [1]> ,bt
In ice-9/boot-9.scm:
     829:9  4 (catch srfi-34 #<procedure 374dd40 at 
guix/build/gremlin.scm:285:2 ()> #<procedure 374dd20 at 
guix/build/gremlin.scm:285:2 (key c)> _)
In guix/build/gremlin.scm:
    305:26  3 (_)
In unknown file:
            2 (remove #<procedure libc-library? (lib)> (1))
            1 (find #<procedure 374db80 at guix/build/gremlin.scm:251:8 
(libc-lib)> ("libanl.so" "libcrypt.so" "libc.so" "libdl.so" "libm.so" 
"libnsl.so" "libp?" ?))
            0 (string-prefix? "libanl.so" 1 #<undefined> #<undefined> 
#<undefined> #<undefined>)



Similarly for libawt_xawt, with the following error message:



In procedure string-prefix?: Wrong type argument in position 2 
(expecting string): 3659183287175258




Then, running:

(elf-dynamic-info (call-with-input-file "libjsoundalsa.debuginfo" 
(compose parse-elf get-bytevector-all)))
$79 = #<<elf-dynamic-info> soname: #f needed: () rpath: () runpath: ()>

(elf-dynamic-info (call-with-input-file "libjsound.debuginfo" (compose 
parse-elf get-bytevector-all)))
$80 = #<<elf-dynamic-info> soname: #f needed: (1) rpath: () runpath: ()>

(elf-dynamic-info (call-with-input-file "libawt_xawt.debuginfo" (compose 
parse-elf get-bytevector-all)))
$81 = #<<elf-dynamic-info> soname: #f needed: (3659183287175258) rpath: 
() runpath: ()>



shows that the number in the exception comes from the needed field. I 
think it should be empty. You can find these three files for comparison 
at:

http://89.234.186.109/guix/libawt_xawt.debuginfo
http://89.234.186.109/guix/libjsoundalsa.debuginfo
http://89.234.186.109/guix/libjsound.debuginfo

Thank you.

bug#32894: Exception in validate-runpath phase

[Ludovic Courtès]

Hello Julien,

Julien Lepiller <julien@lepiller.eu> skribis:

> Then, running:
>
> (elf-dynamic-info (call-with-input-file "libjsoundalsa.debuginfo"
> (compose parse-elf get-bytevector-all)))
> $79 = #<<elf-dynamic-info> soname: #f needed: () rpath: () runpath: ()>
>
> (elf-dynamic-info (call-with-input-file "libjsound.debuginfo" (compose
> parse-elf get-bytevector-all)))
> $80 = #<<elf-dynamic-info> soname: #f needed: (1) rpath: () runpath: ()>
>
> (elf-dynamic-info (call-with-input-file "libawt_xawt.debuginfo"
> (compose parse-elf get-bytevector-all)))
> $81 = #<<elf-dynamic-info> soname: #f needed: (3659183287175258)
> rpath: () runpath: ()>

The reason we get numbers here (rather than strings) is because the
PT_DYNAMIC segment lacks a string table (DT_STRTAB), and thus there’s
nowhere the DT_NEEDED strings can be looked for, AIUI.

That gremlin.scm lets the number through comes from this bit:

           (if string-table-offset
               (pointer->string
                (bytevector->pointer (elf-bytes elf)
                                     (vma->offset
                                      elf
                                      (+ string-table-offset value))))
               value))

This is a questionable choice, but the crux of the problem is that these
ELF files appear to be corrupt or at least non-conventional.

Even BFD (from Binutils) fails to make sense of it:

--8<---------------cut here---------------start------------->8---
$ readelf -a /tmp/libawt_xawt.debuginfo | grep NEED
$ readelf -a /tmp/libawt_xawt.debuginfo | grep PATH
--8<---------------cut here---------------end--------------->8---

Compare with this (random example):

--8<---------------cut here---------------start------------->8---
$ readelf -a ~/.guix-profile/lib/libEGL.so.1 | grep NEED
  [ 6] .gnu.version_r    VERNEED          0000000000004718  00004718
 0x0000000000000001 (NEEDED)             Shared library: [libxcb-dri2.so.0]
 0x0000000000000001 (NEEDED)             Shared library: [libX11-xcb.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libX11.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libxcb-dri3.so.0]
 0x0000000000000001 (NEEDED)             Shared library: [libxcb-xfixes.so.0]
 0x0000000000000001 (NEEDED)             Shared library: [libxcb-present.so.0]
 0x0000000000000001 (NEEDED)             Shared library: [libxcb-sync.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libxcb.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libXau.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libXdmcp.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libbsd.so.0]
 0x0000000000000001 (NEEDED)             Shared library: [libxshmfence.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libwayland-client.so.0]
 0x0000000000000001 (NEEDED)             Shared library: [libgbm.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libz.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libwayland-server.so.0]
 0x0000000000000001 (NEEDED)             Shared library: [libffi.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [librt.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libexpat.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libdl.so.2]
 0x0000000000000001 (NEEDED)             Shared library: [libdrm.so.2]
 0x0000000000000001 (NEEDED)             Shared library: [libm.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libglapi.so.0]
 0x0000000000000001 (NEEDED)             Shared library: [libpthread.so.0]
 0x0000000000000001 (NEEDED)             Shared library: [libgcc_s.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]
 0x000000006ffffffe (VERNEED)            0x4718
 0x000000006fffffff (VERNEEDNUM)         5
ludo@ribbon ~/src/guix$ readelf -a ~/.guix-profile/lib/libEGL.so.1 | grep PATH
 0x000000000000001d (RUNPATH)            Library runpath: [/gnu/store/xmdk5z05kqxpwgagxhlv375x3f82dxb3-libxcb-1.13/lib:/gnu/store/bid7hvpnm8nq04vm4dszywxsw9g2kmf2-libx11-1.6.6/lib:/gnu/store/b9aapwjz2nhri24imzy491fx86ng8jvz-libxau-1.0.8/lib:/gnu/store/07gpi7dx2rjs5v5n12q5b2sk7gxsliih-libxdmcp-1.1.2/lib:/gnu/store/cy16rapipmypb7qj49ncphjkkj9nqkzx-libbsd-0.8.7/lib:/gnu/store/ycj27z17m3n5qj1rwnyxdqkrk7li9712-libxshmfence-1.3/lib:/gnu/store/8w2i20d3gh80x0a7hvbkww8yyn5ky2j8-wayland-1.15.0/lib:/gnu/store/hp2j1cjrca3ghi14ikzhanx8yl41kihp-mesa-18.1.5/lib:/gnu/store/ppsylkcpw2fk2lkzhwjd60xyr9gjl70v-libffi-3.2.1/lib:/gnu/store/70825hjil6070g7cs3mmdnfwmhxgga36-expat-2.2.5/lib:/gnu/store/66h0jfk3k4wavw951ydkzv4x3wwgapkm-libdrm-2.4.92/lib:/gnu/store/l4lr0f5cjd0nbsaaf8b5dmcw1a1yypr3-glibc-2.27/lib:/gnu/store/vla5j7pbkpcp39lsdfsmz7m9azn48lr4-gcc-5.5.0-lib/lib:/gnu/store/fxiwj2wpp11sif613axdax7gmwzsg6kp-zlib-1.2.11/lib:/gnu/store/vla5j7pbkpcp39lsdfsmz7m9azn48lr4-gcc-5.5.0-lib/lib/gcc/x86_64-unknown-linux-gnu/5.5.0/../../..]
--8<---------------cut here---------------end--------------->8---

Then again, these are “.debuginfo” files so perhaps they are the result
of home-made ELF stripping (we don’t have this problem with “.debug”
files created with objcopy & co.).

I can see two short-term “solutions”:

  1. Remove those .debuginfo files prior to the ‘validate-runpath’
     phase.

  2. Set #:validate-runpath? #f.

Could you check in your build logs how those .debuginfo files are produced?

HTH!

Ludo’.

bug#32894: Exception in validate-runpath phase

[Julien Lepiller]

Le 2018-10-02 14:31, ludo@gnu.org a écrit :
> Hello Julien,
> 
> [...]
> 
> Then again, these are “.debuginfo” files so perhaps they are the result
> of home-made ELF stripping (we don’t have this problem with “.debug”
> files created with objcopy & co.).
> 
> I can see two short-term “solutions”:
> 
>   1. Remove those .debuginfo files prior to the ‘validate-runpath’
>      phase.
> 
>   2. Set #:validate-runpath? #f.
> 
> Could you check in your build logs how those .debuginfo files are 
> produced?
> 
> HTH!
> 
> Ludo’.

As a workaround, I found that passing --with-native-debug-symbols=zipped 
to the configure script allowed validate-runpath to complete 
successfully. This option is documented as the prefered value for 
distributions, so I think it's a good thing to have. I creates zipped 
versions of these .debuginfo files, so validate-runpath doesn't look at 
them.

There is no mention of debuginfo until the install phase in my build 
log, so I don't know how they are built.

bug#32894: Exception in validate-runpath phase

[Ludovic Courtès]

Hello,

Julien Lepiller <julien@lepiller.eu> skribis:

> Le 2018-10-02 14:31, ludo@gnu.org a écrit :
>> Hello Julien,
>>
>> [...]
>>
>> Then again, these are “.debuginfo” files so perhaps they are the result
>> of home-made ELF stripping (we don’t have this problem with “.debug”
>> files created with objcopy & co.).
>>
>> I can see two short-term “solutions”:
>>
>>   1. Remove those .debuginfo files prior to the ‘validate-runpath’
>>      phase.
>>
>>   2. Set #:validate-runpath? #f.
>>
>> Could you check in your build logs how those .debuginfo files are
>> produced?
>>
>> HTH!
>>
>> Ludo’.
>
> As a workaround, I found that passing
> --with-native-debug-symbols=zipped to the configure script allowed
> validate-runpath to complete successfully. This option is documented
> as the prefered value for distributions, so I think it's a good thing
> to have. I creates zipped versions of these .debuginfo files, so
> validate-runpath doesn't look at them.

Hmm OK.  :-)

Is it the files that are zipped, or just .debug sections that are
gzipped?  (Binutils, GDB, etc. support the latter.)

> There is no mention of debuginfo until the install phase in my build
> log, so I don't know how they are built.

OK.

Thanks,
Ludo’.

bug#32894: Exception in validate-runpath phase

[Julien Lepiller]

[-- Attachment #1: Type: text/plain, Size: 149 bytes --]

Since then, we have openjdk 10, and much more recent versions. I don't remember what fixed the issue, but I haven't seen it since then, so closing :)

[-- Attachment #2: Type: text/html, Size: 190 bytes --]