IcedTea is not linking correctly with libjvm.so

IcedTea is not linking correctly with libjvm.so

[Roel Janssen]

Dear Guix,

I ran into a problem with a Java program running with Guix's icedtea-3.
The error message looks like this:

Exception in thread "main" java.lang.UnsatisfiedLinkError: /gnu/store/q9ad5zvxpm2spiddcj01sw3jkm5vpgva-icedtea-3.5.1/lib/amd64/libnet.so: /gnu/store/q9ad5zvxpm2spiddcj01sw3jkm5vpgva-icedtea-3.5.1/lib/amd64/libnet.so: failed to map segment from shared object
        at java.lang.ClassLoader$NativeLibrary.load(Native Method)
        at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1941)
        at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1845)
        at java.lang.Runtime.loadLibrary0(Runtime.java:870)
        at java.lang.System.loadLibrary(System.java:1122)
        at java.net.InetAddress$1.run(InetAddress.java:294)
        at java.net.InetAddress$1.run(InetAddress.java:292)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.InetAddress.<clinit>(InetAddress.java:291)
        at org.apache.logging.log4j.core.util.NetUtils.getLocalHostname(NetUtils.java:53)
        at org.apache.logging.log4j.core.LoggerContext.setConfiguration(LoggerContext.java:539)
        at org.apache.logging.log4j.core.LoggerContext.reconfigure(LoggerContext.java:617)
        at org.apache.logging.log4j.core.LoggerContext.reconfigure(LoggerContext.java:634)
        at org.apache.logging.log4j.core.LoggerContext.start(LoggerContext.java:229)
        at org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:152)
        at org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:45)
        at org.apache.logging.log4j.LogManager.getContext(LogManager.java:194)
        at org.apache.logging.log4j.LogManager.getLogger(LogManager.java:551)
        ...

Looking into this shared object, I found that it cannot find libjvm.so:
$ ldd /gnu/store/q9ad5zvxpm2spiddcj01sw3jkm5vpgva-icedtea-3.5.1/lib/amd64/libnet.so
        linux-vdso.so.1 =>  (0x00007ffe355ab000)
        libdl.so.2 => /gnu/store/20jhhjzgyqkiw1078cyy3891amqm8d4f-glibc-2.25/lib/libdl.so.2 (0x00007f3984931000)
        libjvm.so => not found
        libpthread.so.0 => /gnu/store/20jhhjzgyqkiw1078cyy3891amqm8d4f-glibc-2.25/lib/libpthread.so.0 (0x00007f39846f8000)
        libjava.so => /gnu/store/q9ad5zvxpm2spiddcj01sw3jkm5vpgva-icedtea-3.5.1/lib/amd64/libjava.so (0x00007f39844cc000)
        libgcc_s.so.1 => /gnu/store/0ss2akh5grfdfqnik6mm3lj4yyyb08np-gcc-5.4.0-lib/lib/libgcc_s.so.1 (0x00007f39842b4000)
        libc.so.6 => /gnu/store/20jhhjzgyqkiw1078cyy3891amqm8d4f-glibc-2.25/lib/libc.so.6 (0x00007f3983f15000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f3984d4d000)
        libjvm.so => not found
        libverify.so => /gnu/store/q9ad5zvxpm2spiddcj01sw3jkm5vpgva-icedtea-3.5.1/lib/amd64/libverify.so (0x00007f3983d05000)
        libjvm.so => not found

It seems that libjvm.so is in the lib/amd64/server folder of the icedtea
package, so after setting LD_LIBRARY_PATH, the application runs fine:
$ export LD_LIBRARY_PATH=/gnu/store/q9ad5zvxpm2spiddcj01sw3jkm5vpgva-icedtea-3.5.1/lib/amd64/server

I looked into the package recipe, but I cannot find a place where
a change in the compilation process could fix this problem.  My only
guess is that in the 'patch-jni-libs phase, we could change the way
dynamically loaded libraries are found.

The way I see it, we could do three things:
1. Fix the recipe to make sure libjvm.so is found, and thus libnet.so is
linked correctly.

2. Copy or make a symlink of libjvm.so to the parent directory
   (lib/amd64), where the other libraries are.  Maybe then libnet.so can
   find libjvm.so.

3. Propagate LD_LIBRARY_PATH with a path to lib/amd64/server.  It would
   work like in my quick test, but this is my least favorite solution.

Which way would be preferable?  I can prepare a patch for option 2, and
see if that works.  But maybe option 1 would be better.

Thanks for your time.

Kind regards,
Roel Janssen

Re: IcedTea is not linking correctly with libjvm.so

[Ludovic Courtès]

Hi!

Roel Janssen <roel@gnu.org> skribis:

> Looking into this shared object, I found that it cannot find libjvm.so:
> $ ldd /gnu/store/q9ad5zvxpm2spiddcj01sw3jkm5vpgva-icedtea-3.5.1/lib/amd64/libnet.so
>         linux-vdso.so.1 =>  (0x00007ffe355ab000)
>         libdl.so.2 => /gnu/store/20jhhjzgyqkiw1078cyy3891amqm8d4f-glibc-2.25/lib/libdl.so.2 (0x00007f3984931000)
>         libjvm.so => not found
>         libpthread.so.0 => /gnu/store/20jhhjzgyqkiw1078cyy3891amqm8d4f-glibc-2.25/lib/libpthread.so.0 (0x00007f39846f8000)
>         libjava.so => /gnu/store/q9ad5zvxpm2spiddcj01sw3jkm5vpgva-icedtea-3.5.1/lib/amd64/libjava.so (0x00007f39844cc000)
>         libgcc_s.so.1 => /gnu/store/0ss2akh5grfdfqnik6mm3lj4yyyb08np-gcc-5.4.0-lib/lib/libgcc_s.so.1 (0x00007f39842b4000)
>         libc.so.6 => /gnu/store/20jhhjzgyqkiw1078cyy3891amqm8d4f-glibc-2.25/lib/libc.so.6 (0x00007f3983f15000)
>         /lib64/ld-linux-x86-64.so.2 (0x00007f3984d4d000)
>         libjvm.so => not found
>         libverify.so => /gnu/store/q9ad5zvxpm2spiddcj01sw3jkm5vpgva-icedtea-3.5.1/lib/amd64/libverify.so (0x00007f3983d05000)
>         libjvm.so => not found

Oh, bad!  :-)

The package has this:

       ;; The DSOs use $ORIGIN to refer to each other, but (guix build
       ;; gremlin) doesn't support it yet, so skip this phase.
       #:validate-runpath? #f

The comment was first added in fb799cb72e, when it was true, but shortly
after (guix build gremlin) gained support for that.  So we should
probably set this to #t once the package is fixed.

Ludo’.

Re: IcedTea is not linking correctly with libjvm.so

[Chris Marusich]

[-- Attachment #1: Type: text/plain, Size: 694 bytes --]

Roel Janssen <roel@gnu.org> writes:

> 1. Fix the recipe to make sure libjvm.so is found, and thus libnet.so is
> linked correctly.
>
> 2. Copy or make a symlink of libjvm.so to the parent directory
>    (lib/amd64), where the other libraries are.  Maybe then libnet.so can
>    find libjvm.so.

(1) seems better than (2) if possible, but either of those solutions
seem OK to me.  But to be honest, I don't understand why this isn't a
problem for icedtea outside of Guix.  What are we doing that is
different which prevents the library from being found?

Solution (3) feels like more of a hack than (2), so I'm not sure about
it.  Maybe other have other opinions?

-- 
Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: IcedTea is not linking correctly with libjvm.so

[Roel Janssen]

Chris Marusich writes:

> Roel Janssen <roel@gnu.org> writes:
>
>> 1. Fix the recipe to make sure libjvm.so is found, and thus libnet.so is
>> linked correctly.
>>
>> 2. Copy or make a symlink of libjvm.so to the parent directory
>>    (lib/amd64), where the other libraries are.  Maybe then libnet.so can
>>    find libjvm.so.
>
> (1) seems better than (2) if possible, but either of those solutions
> seem OK to me.  But to be honest, I don't understand why this isn't a
> problem for icedtea outside of Guix.  What are we doing that is
> different which prevents the library from being found?

Thanks for your reply!

I agree that (1) would be better than (2).  The only problem I see with
this is that I don't see how to achieve (1), but I do see how to achieve
(2).

I tried running that Java app with CentOS's Java (openjdk 1.7.0), and it
has the exact same problem:

$ ldd /usr/lib/jvm/java-1.7.0-openjdk/jre/lib/amd64/libnet.so 
        linux-vdso.so.1 =>  (0x00007ffcc153d000)
        libjvm.so => not found
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f0b70d7e000)
        libgconf-2.so.4 => /lib64/libgconf-2.so.4 (0x00007f0b70b4d000)
        libglib-2.0.so.0 => /lib64/libglib-2.0.so.0 (0x00007f0b70816000)
        libgobject-2.0.so.0 => /lib64/libgobject-2.0.so.0 (0x00007f0b705c5000)
        libgio-2.0.so.0 => /lib64/libgio-2.0.so.0 (0x00007f0b70245000)
        libjava.so => /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.121-2.6.8.0.el7_3.x86_64/jre/lib/amd64/./libjava.so (0x00007f0b70019000)
        libc.so.6 => /lib64/libc.so.6 (0x00007f0b6fc57000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f0b711ce000)
        libgmodule-2.0.so.0 => /lib64/libgmodule-2.0.so.0 (0x00007f0b6fa53000)
        libdbus-glib-1.so.2 => /lib64/libdbus-glib-1.so.2 (0x00007f0b6f82b000)
        libdbus-1.so.3 => /lib64/libdbus-1.so.3 (0x00007f0b6f5e2000)
        libffi.so.6 => /lib64/libffi.so.6 (0x00007f0b6f3da000)
        libdl.so.2 => /lib64/libdl.so.2 (0x00007f0b6f1d6000)
        libz.so.1 => /lib64/libz.so.1 (0x00007f0b6efbf000)
        libselinux.so.1 => /lib64/libselinux.so.1 (0x00007f0b6ed98000)
        libresolv.so.2 => /lib64/libresolv.so.2 (0x00007f0b6eb7e000)
        libjvm.so => not found
        libverify.so => /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.121-2.6.8.0.el7_3.x86_64/jre/lib/amd64/./libverify.so (0x00007f0b6e96e000)
        librt.so.1 => /lib64/librt.so.1 (0x00007f0b6e765000)
        libpcre.so.1 => /lib64/libpcre.so.1 (0x00007f0b6e504000)
        libjvm.so => not found

>
> Solution (3) feels like more of a hack than (2), so I'm not sure about
> it.  Maybe other have other opinions?

I agree that we should avoid solution (3).

Kind regards,
Roel Janssen

Re: IcedTea is not linking correctly with libjvm.so

[Chris Marusich]

[-- Attachment #1: Type: text/plain, Size: 2896 bytes --]

Roel Janssen <roel@gnu.org> writes:

> Chris Marusich writes:
>
>> Roel Janssen <roel@gnu.org> writes:
>>
>>> 1. Fix the recipe to make sure libjvm.so is found, and thus libnet.so is
>>> linked correctly.
>>>
>>> 2. Copy or make a symlink of libjvm.so to the parent directory
>>>    (lib/amd64), where the other libraries are.  Maybe then libnet.so can
>>>    find libjvm.so.
>>
>> (1) seems better than (2) if possible, but either of those solutions
>> seem OK to me.  But to be honest, I don't understand why this isn't a
>> problem for icedtea outside of Guix.  What are we doing that is
>> different which prevents the library from being found?
>
> Thanks for your reply!
>
> I agree that (1) would be better than (2).  The only problem I see with
> this is that I don't see how to achieve (1), but I do see how to achieve
> (2).
>
> I tried running that Java app with CentOS's Java (openjdk 1.7.0), and it
> has the exact same problem:
>
> $ ldd /usr/lib/jvm/java-1.7.0-openjdk/jre/lib/amd64/libnet.so 
>         linux-vdso.so.1 =>  (0x00007ffcc153d000)
>         libjvm.so => not found
>         libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f0b70d7e000)
>         libgconf-2.so.4 => /lib64/libgconf-2.so.4 (0x00007f0b70b4d000)
>         libglib-2.0.so.0 => /lib64/libglib-2.0.so.0 (0x00007f0b70816000)
>         libgobject-2.0.so.0 => /lib64/libgobject-2.0.so.0 (0x00007f0b705c5000)
>         libgio-2.0.so.0 => /lib64/libgio-2.0.so.0 (0x00007f0b70245000)
>         libjava.so => /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.121-2.6.8.0.el7_3.x86_64/jre/lib/amd64/./libjava.so (0x00007f0b70019000)
>         libc.so.6 => /lib64/libc.so.6 (0x00007f0b6fc57000)
>         /lib64/ld-linux-x86-64.so.2 (0x00007f0b711ce000)
>         libgmodule-2.0.so.0 => /lib64/libgmodule-2.0.so.0 (0x00007f0b6fa53000)
>         libdbus-glib-1.so.2 => /lib64/libdbus-glib-1.so.2 (0x00007f0b6f82b000)
>         libdbus-1.so.3 => /lib64/libdbus-1.so.3 (0x00007f0b6f5e2000)
>         libffi.so.6 => /lib64/libffi.so.6 (0x00007f0b6f3da000)
>         libdl.so.2 => /lib64/libdl.so.2 (0x00007f0b6f1d6000)
>         libz.so.1 => /lib64/libz.so.1 (0x00007f0b6efbf000)
>         libselinux.so.1 => /lib64/libselinux.so.1 (0x00007f0b6ed98000)
>         libresolv.so.2 => /lib64/libresolv.so.2 (0x00007f0b6eb7e000)
>         libjvm.so => not found
>         libverify.so => /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.121-2.6.8.0.el7_3.x86_64/jre/lib/amd64/./libverify.so (0x00007f0b6e96e000)
>         librt.so.1 => /lib64/librt.so.1 (0x00007f0b6e765000)
>         libpcre.so.1 => /lib64/libpcre.so.1 (0x00007f0b6e504000)
>         libjvm.so => not found

Can you share a minimal program that reproduces the issue?  If it
happens on Guix's Java, built with Icedtea, and also on another
distro's, too, then maybe it's a genuine bug that can be fixed upstream.

-- 
Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: IcedTea is not linking correctly with libjvm.so

[Roel Janssen]

[-- Attachment #1: Type: text/plain, Size: 6587 bytes --]


Chris Marusich writes:

> Roel Janssen <roel@gnu.org> writes:
>
>> Chris Marusich writes:
>>
>>> Roel Janssen <roel@gnu.org> writes:
>>>
>>>> 1. Fix the recipe to make sure libjvm.so is found, and thus libnet.so is
>>>> linked correctly.
>>>>
>>>> 2. Copy or make a symlink of libjvm.so to the parent directory
>>>>    (lib/amd64), where the other libraries are.  Maybe then libnet.so can
>>>>    find libjvm.so.
>>>
>>> (1) seems better than (2) if possible, but either of those solutions
>>> seem OK to me.  But to be honest, I don't understand why this isn't a
>>> problem for icedtea outside of Guix.  What are we doing that is
>>> different which prevents the library from being found?
>>
>> Thanks for your reply!
>>
>> I agree that (1) would be better than (2).  The only problem I see with
>> this is that I don't see how to achieve (1), but I do see how to achieve
>> (2).
>>
>> I tried running that Java app with CentOS's Java (openjdk 1.7.0), and it
>> has the exact same problem:
>>
>> $ ldd /usr/lib/jvm/java-1.7.0-openjdk/jre/lib/amd64/libnet.so 
>>         linux-vdso.so.1 =>  (0x00007ffcc153d000)
>>         libjvm.so => not found
>>         libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f0b70d7e000)
>>         libgconf-2.so.4 => /lib64/libgconf-2.so.4 (0x00007f0b70b4d000)
>>         libglib-2.0.so.0 => /lib64/libglib-2.0.so.0 (0x00007f0b70816000)
>>         libgobject-2.0.so.0 => /lib64/libgobject-2.0.so.0 (0x00007f0b705c5000)
>>         libgio-2.0.so.0 => /lib64/libgio-2.0.so.0 (0x00007f0b70245000)
>>         libjava.so => /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.121-2.6.8.0.el7_3.x86_64/jre/lib/amd64/./libjava.so (0x00007f0b70019000)
>>         libc.so.6 => /lib64/libc.so.6 (0x00007f0b6fc57000)
>>         /lib64/ld-linux-x86-64.so.2 (0x00007f0b711ce000)
>>         libgmodule-2.0.so.0 => /lib64/libgmodule-2.0.so.0 (0x00007f0b6fa53000)
>>         libdbus-glib-1.so.2 => /lib64/libdbus-glib-1.so.2 (0x00007f0b6f82b000)
>>         libdbus-1.so.3 => /lib64/libdbus-1.so.3 (0x00007f0b6f5e2000)
>>         libffi.so.6 => /lib64/libffi.so.6 (0x00007f0b6f3da000)
>>         libdl.so.2 => /lib64/libdl.so.2 (0x00007f0b6f1d6000)
>>         libz.so.1 => /lib64/libz.so.1 (0x00007f0b6efbf000)
>>         libselinux.so.1 => /lib64/libselinux.so.1 (0x00007f0b6ed98000)
>>         libresolv.so.2 => /lib64/libresolv.so.2 (0x00007f0b6eb7e000)
>>         libjvm.so => not found
>>         libverify.so => /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.121-2.6.8.0.el7_3.x86_64/jre/lib/amd64/./libverify.so (0x00007f0b6e96e000)
>>         librt.so.1 => /lib64/librt.so.1 (0x00007f0b6e765000)
>>         libpcre.so.1 => /lib64/libpcre.so.1 (0x00007f0b6e504000)
>>         libjvm.so => not found
>
> Can you share a minimal program that reproduces the issue?  If it
> happens on Guix's Java, built with Icedtea, and also on another
> distro's, too, then maybe it's a genuine bug that can be fixed upstream.

Unfortunately, I don't have a simple example to reproduce it.  However,
it looks a lot like this bug:
https://bugzilla.redhat.com/show_bug.cgi?id=1212151

The program producing the error can be found here:
https://github.com/hartwigmedical/hmftools

For which I have a package here:
https://github.com/UMCUGenetics/guix-additions/blob/master/umcu/packages/hmf.scm#L117

However, it needs various data inputs before the tool will run.  Hence,
the problem with making a simple example to reproduce it.

I attached a patch for solution (2), which seems to work.  On the output
directory:
$ $ ldd /gnu/store/0p35h1dq956h4axal8cc9as1y7qxchqv-icedtea-2.6.11/lib/amd64/libnet.so
        linux-vdso.so.1 (0x00007ffdda9ab000)
        libjvm.so => /gnu/store/0p35h1dq956h4axal8cc9as1y7qxchqv-icedtea-2.6.11/lib/amd64/./libjvm.so (0x00007f2def98c000)
        libpthread.so.0 => /gnu/store/n6nvxlk2j8ysffjh3jphn1k5silnakh6-glibc-2.25/lib/libpthread.so.0 (0x00007f2def76e000)
        libdl.so.2 => /gnu/store/n6nvxlk2j8ysffjh3jphn1k5silnakh6-glibc-2.25/lib/libdl.so.2 (0x00007f2def56a000)
        libgio-2.0.so.0 => /gnu/store/qzmyyj0jx6n14vsffa66jgsnnvwhby3n-glib-2.52.3/lib/libgio-2.0.so.0 (0x00007f2def1d3000)
        libgobject-2.0.so.0 => /gnu/store/qzmyyj0jx6n14vsffa66jgsnnvwhby3n-glib-2.52.3/lib/libgobject-2.0.so.0 (0x00007f2deef81000)
        libglib-2.0.so.0 => /gnu/store/qzmyyj0jx6n14vsffa66jgsnnvwhby3n-glib-2.52.3/lib/libglib-2.0.so.0 (0x00007f2deec6e000)
        libjava.so => /gnu/store/0p35h1dq956h4axal8cc9as1y7qxchqv-icedtea-2.6.11/lib/amd64/./libjava.so (0x00007f2deea42000)
        libc.so.6 => /gnu/store/n6nvxlk2j8ysffjh3jphn1k5silnakh6-glibc-2.25/lib/libc.so.6 (0x00007f2dee6a3000)
        libgcc_s.so.1 => /gnu/store/3x53yv4v144c9xp02rs64z7j597kkqax-gcc-5.4.0-lib/lib/libgcc_s.so.1 (0x00007f2dee48c000)
        libstdc++.so.6 => /gnu/store/3x53yv4v144c9xp02rs64z7j597kkqax-gcc-5.4.0-lib/lib/libstdc++.so.6 (0x00007f2dee112000)
        libm.so.6 => /gnu/store/n6nvxlk2j8ysffjh3jphn1k5silnakh6-glibc-2.25/lib/libm.so.6 (0x00007f2dede00000)
        /gnu/store/n6nvxlk2j8ysffjh3jphn1k5silnakh6-glibc-2.25/lib/ld-linux-x86-64.so.2 (0x00007f2df09b4000)
        libffi.so.6 => /gnu/store/jnbb8ffxxvrw2b4z18zn0g08kqk9rsgl-libffi-3.2.1/lib/libffi.so.6 (0x00007f2dedbf5000)
        libgmodule-2.0.so.0 => /gnu/store/qzmyyj0jx6n14vsffa66jgsnnvwhby3n-glib-2.52.3/lib/libgmodule-2.0.so.0 (0x00007f2ded9f1000)
        libpcre.so.1 => /gnu/store/3amb8hw38k2jv604pb87am8v9r17fczi-pcre-8.41/lib/libpcre.so.1 (0x00007f2ded780000)
        libz.so.1 => /gnu/store/sfx1wh27i6gsrk21p87rdyikc64v7d51-zlib-1.2.11/lib/libz.so.1 (0x00007f2ded565000)
        libresolv.so.2 => /gnu/store/n6nvxlk2j8ysffjh3jphn1k5silnakh6-glibc-2.25/lib/libresolv.so.2 (0x00007f2ded34f000)
        libmount.so.1 => /gnu/store/zbywrj6klakskj0sppq56viqh9l56jl0-util-linux-2.30.1/lib/libmount.so.1 (0x00007f2ded0fd000)
        libblkid.so.1 => /gnu/store/zbywrj6klakskj0sppq56viqh9l56jl0-util-linux-2.30.1/lib/libblkid.so.1 (0x00007f2deceb5000)
        libuuid.so.1 => /gnu/store/zbywrj6klakskj0sppq56viqh9l56jl0-util-linux-2.30.1/lib/libuuid.so.1 (0x00007f2deccb0000)
        librt.so.1 => /gnu/store/n6nvxlk2j8ysffjh3jphn1k5silnakh6-glibc-2.25/lib/librt.so.1 (0x00007f2decaa8000)
        libverify.so => /gnu/store/0p35h1dq956h4axal8cc9as1y7qxchqv-icedtea-2.6.11/lib/amd64/./libverify.so (0x00007f2dec899000)

I'd like to put this solution in place until we find out how to apply
solution (1).  The difference on the output is one symbolic link, so the
impact is low.

This patch fixes this issue for icedtea-7 and icedtea-8.

Is it OK to push this patch?

Thanks for your time!

Kind regards,
Roel Janssen


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-gnu-java-Fix-libjvm.so-linkage-problem-in-icedtea-7.patch --]
[-- Type: text/x-patch, Size: 1794 bytes --]

From a9deda0a7b6162b037ff9ffb08135275c06b340c Mon Sep 17 00:00:00 2001
From: Roel Janssen <roel@gnu.org>
Date: Tue, 24 Oct 2017 12:20:26 +0200
Subject: [PATCH] gnu: java: Fix libjvm.so linkage problem in icedtea-7.

* gnu/packages/java.scm (icedtea-7): Add phase to create a symbolic link to
  libjvm.so.
---
 gnu/packages/java.scm | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/gnu/packages/java.scm b/gnu/packages/java.scm
index 95fba20e8..81cfdc132 100644
--- a/gnu/packages/java.scm
+++ b/gnu/packages/java.scm
@@ -1404,6 +1404,18 @@ bootstrapping purposes.")
                  (copy-recursively "openjdk.build/j2re-image" jre)
                  (copy-recursively "openjdk.build/j2sdk-image" jdk))
                #t))
+           ;; Some of the libraries in the lib/amd64 folder link to libjvm.so.  But that
+           ;; shared object is located in the server/ folder, so it cannot be found.
+           ;; This phase creates a symbolic link in the lib/amd64 folder so that the
+           ;; other libraries can find it.
+           ;;
+           ;; See https://lists.gnu.org/archive/html/guix-devel/2017-10/msg00169.html
+           (add-after 'install 'install-libjvm
+             (lambda* (#:key inputs outputs #:allow-other-keys)
+               (let* ((lib-path (string-append (assoc-ref outputs "out") "/lib/amd64")))
+                 (system* "ln" "--symbolic"
+                          (string-append lib-path "/server/libjvm.so")
+                          (string-append lib-path "/libjvm.so")))))
            ;; By default IcedTea only generates an empty keystore.  In order to
            ;; be able to use certificates in Java programs we need to generate a
            ;; keystore from a set of certificates.  For convenience we use the
-- 
2.14.2

Re: IcedTea is not linking correctly with libjvm.so

[Roel Janssen]

Ludovic Courtès writes:

> Hi!
>
> Roel Janssen <roel@gnu.org> skribis:
>
>> Looking into this shared object, I found that it cannot find libjvm.so:
>> $ ldd /gnu/store/q9ad5zvxpm2spiddcj01sw3jkm5vpgva-icedtea-3.5.1/lib/amd64/libnet.so
>>         linux-vdso.so.1 =>  (0x00007ffe355ab000)
>>         libdl.so.2 => /gnu/store/20jhhjzgyqkiw1078cyy3891amqm8d4f-glibc-2.25/lib/libdl.so.2 (0x00007f3984931000)
>>         libjvm.so => not found
>>         libpthread.so.0 => /gnu/store/20jhhjzgyqkiw1078cyy3891amqm8d4f-glibc-2.25/lib/libpthread.so.0 (0x00007f39846f8000)
>>         libjava.so => /gnu/store/q9ad5zvxpm2spiddcj01sw3jkm5vpgva-icedtea-3.5.1/lib/amd64/libjava.so (0x00007f39844cc000)
>>         libgcc_s.so.1 => /gnu/store/0ss2akh5grfdfqnik6mm3lj4yyyb08np-gcc-5.4.0-lib/lib/libgcc_s.so.1 (0x00007f39842b4000)
>>         libc.so.6 => /gnu/store/20jhhjzgyqkiw1078cyy3891amqm8d4f-glibc-2.25/lib/libc.so.6 (0x00007f3983f15000)
>>         /lib64/ld-linux-x86-64.so.2 (0x00007f3984d4d000)
>>         libjvm.so => not found
>>         libverify.so => /gnu/store/q9ad5zvxpm2spiddcj01sw3jkm5vpgva-icedtea-3.5.1/lib/amd64/libverify.so (0x00007f3983d05000)
>>         libjvm.so => not found
>
> Oh, bad!  :-)
>
> The package has this:
>
>        ;; The DSOs use $ORIGIN to refer to each other, but (guix build
>        ;; gremlin) doesn't support it yet, so skip this phase.
>        #:validate-runpath? #f
>
> The comment was first added in fb799cb72e, when it was true, but shortly
> after (guix build gremlin) gained support for that.  So we should
> probably set this to #t once the package is fixed.
>
> Ludo’.

With the patch I submitted earlier today, I tried to set the
#:validate-runpath? to #t, but the build failed.

It seems there are more libraries with linking errors:
$ ldd /gnu/store/fy8krxbj60z9kx9hwsp4f08qjgr2cb20-icedtea-2.6.11/lib/amd64/xawt/libmawt.so
        linux-vdso.so.1 (0x00007ffc94ed3000)
        ...
        libawt.so => not found
        libjava.so => not found
        libjvm.so => not found
        ...

So, again libjvm.so, because this is in different directory.
libmawt.so is used by libjawt.so, (and libjawt.so cannot find libmawt.so
either).

So there's definitely more to investigate/fix here.

Nevertheless, my immediate need is to fix that libjvm.so linkage error
from libjvm.so.  So I would like to go ahead with the patch I submitted
earlier.

I am investigating the remaining linkage errors and whether there's a
better fix than creating symbolic links.

Kind regards,
Roel Janssen

Re: IcedTea is not linking correctly with libjvm.so

[Ricardo Wurmus]

Roel Janssen <roel@gnu.org> writes:

>  gnu/packages/java.scm | 12 ++++++++++++
>  1 file changed, 12 insertions(+)
>
> diff --git a/gnu/packages/java.scm b/gnu/packages/java.scm
> index 95fba20e8..81cfdc132 100644
> --- a/gnu/packages/java.scm
> +++ b/gnu/packages/java.scm
> @@ -1404,6 +1404,18 @@ bootstrapping purposes.")
>                   (copy-recursively "openjdk.build/j2re-image" jre)
>                   (copy-recursively "openjdk.build/j2sdk-image" jdk))
>                 #t))
> +           ;; Some of the libraries in the lib/amd64 folder link to libjvm.so.  But that
> +           ;; shared object is located in the server/ folder, so it cannot be found.
> +           ;; This phase creates a symbolic link in the lib/amd64 folder so that the
> +           ;; other libraries can find it.
> +           ;;
> +           ;; See https://lists.gnu.org/archive/html/guix-devel/2017-10/msg00169.html
> +           (add-after 'install 'install-libjvm
> +             (lambda* (#:key inputs outputs #:allow-other-keys)
> +               (let* ((lib-path (string-append (assoc-ref outputs "out") "/lib/amd64")))
> +                 (system* "ln" "--symbolic"
> +                          (string-append lib-path "/server/libjvm.so")
> +                          (string-append lib-path "/libjvm.so")))))

Please use (symlink foo bar) instead of calling the “ln” tool.  Also end
the phase with #t.

Other than that I think it’s fine as a workaround.  Please also add a
FIXME to the comment, so that we can revisit this later.

Thanks!

-- 
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net

Re: IcedTea is not linking correctly with libjvm.so

[Roel Janssen]

Ricardo Wurmus writes:

> Roel Janssen <roel@gnu.org> writes:
>
>>  gnu/packages/java.scm | 12 ++++++++++++
>>  1 file changed, 12 insertions(+)
>>
>> diff --git a/gnu/packages/java.scm b/gnu/packages/java.scm
>> index 95fba20e8..81cfdc132 100644
>> --- a/gnu/packages/java.scm
>> +++ b/gnu/packages/java.scm
>> @@ -1404,6 +1404,18 @@ bootstrapping purposes.")
>>                   (copy-recursively "openjdk.build/j2re-image" jre)
>>                   (copy-recursively "openjdk.build/j2sdk-image" jdk))
>>                 #t))
>> +           ;; Some of the libraries in the lib/amd64 folder link to libjvm.so.  But that
>> +           ;; shared object is located in the server/ folder, so it cannot be found.
>> +           ;; This phase creates a symbolic link in the lib/amd64 folder so that the
>> +           ;; other libraries can find it.
>> +           ;;
>> +           ;; See https://lists.gnu.org/archive/html/guix-devel/2017-10/msg00169.html
>> +           (add-after 'install 'install-libjvm
>> +             (lambda* (#:key inputs outputs #:allow-other-keys)
>> +               (let* ((lib-path (string-append (assoc-ref outputs "out") "/lib/amd64")))
>> +                 (system* "ln" "--symbolic"
>> +                          (string-append lib-path "/server/libjvm.so")
>> +                          (string-append lib-path "/libjvm.so")))))
>
> Please use (symlink foo bar) instead of calling the “ln” tool.  Also end
> the phase with #t.
>
> Other than that I think it’s fine as a workaround.  Please also add a
> FIXME to the comment, so that we can revisit this later.
>
> Thanks!

I used (symlink ...), added a FIXME, rebuilt to see if it worked, and
pushed in 491dc2fb1.

Thanks!

Kind regards,
Roel Janssen

Re: IcedTea is not linking correctly with libjvm.so

[Ricardo Wurmus]

Hi Roel,

> I used (symlink ...), added a FIXME, rebuilt to see if it worked, and
> pushed in 491dc2fb1.

Did you successfully build icedtea@2 with this commit?  I failed to
build it on my laptop:

--8<---------------cut here---------------start------------->8---
…
make[5]: Leaving directory '/tmp/guix-build-icedtea-2.6.11.drv-0/icedtea-2.6.11/openjdk/jdk/make/java/version'
make[5]: Entering directory '/tmp/guix-build-icedtea-2.6.11.drv-0/icedtea-2.6.11/openjdk/jdk/make/java/jvm'
logname: no login name
INFO: ENABLE_FULL_DEBUG_SYMBOLS=1
INFO: ALT_OBJCOPY=/gnu/store/nnykzgwfy8mwh2gmxm715sjxykg8qjwn-binutils-2.28/bin/objcopy
INFO: /gnu/store/nnykzgwfy8mwh2gmxm715sjxykg8qjwn-binutils-2.28/bin/objcopy cmd found so will create .debuginfo files.
INFO: STRIP_POLICY=no_strip
INFO: ZIP_DEBUGINFO_FILES=1
/gnu/store/m9l0j7apf9ac7shqwi5sh4hsn12x4dnk-coreutils-8.27/bin/mkdir -p /tmp/guix-build-icedtea-2.6.11.drv-0/icedtea-2.6.11/openjdk.build/include
rm -f /tmp/guix-build-icedtea-2.6.11.drv-0/icedtea-2.6.11/openjdk.build/include/jni.h
/gnu/store/m9l0j7apf9ac7shqwi5sh4hsn12x4dnk-coreutils-8.27/bin/cp ../../../src/share/javavm/export/jni.h /tmp/guix-build-icedtea-2.6.11.drv-0/icedtea-2.6.11/openjdk.build/include/jni.h
make[5]: *** No rule to make target '/tmp/guix-build-icedtea-2.6.11.drv-0/icedtea-2.6.11/openjdk.build/include/linux/jni_md.h', needed by 'build'.  Stop.
make[5]: Leaving directory '/tmp/guix-build-icedtea-2.6.11.drv-0/icedtea-2.6.11/openjdk/jdk/make/java/jvm'
make[4]: *** [Makefile:63: all] Error 1
make[4]: Leaving directory '/tmp/guix-build-icedtea-2.6.11.drv-0/icedtea-2.6.11/openjdk/jdk/make/java'
make[3]: *** [Makefile:253: all] Error 1
make[3]: Leaving directory '/tmp/guix-build-icedtea-2.6.11.drv-0/icedtea-2.6.11/openjdk/jdk/make'
make[2]: *** [make/jdk-rules.gmk:93: jdk-build] Error 2
make[2]: Leaving directory '/tmp/guix-build-icedtea-2.6.11.drv-0/icedtea-2.6.11/openjdk'
make[1]: *** [Makefile:251: build_product_image] Error 2
make[1]: Leaving directory '/tmp/guix-build-icedtea-2.6.11.drv-0/icedtea-2.6.11/openjdk'
make: *** [Makefile:2463: stamps/icedtea.stamp] Error 2
phase `build' failed after 12663.3 seconds
…
--8<---------------cut here---------------end--------------->8---

-- 
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net

Re: IcedTea is not linking correctly with libjvm.so

[Roel Janssen]

Hi Ricardo,

Ricardo Wurmus writes:

> Hi Roel,
>
>> I used (symlink ...), added a FIXME, rebuilt to see if it worked, and
>> pushed in 491dc2fb1.
>
> Did you successfully build icedtea@2 with this commit?  I failed to
> build it on my laptop:
>
> --8<---------------cut here---------------start------------->8---
> …
> make[5]: Leaving directory '/tmp/guix-build-icedtea-2.6.11.drv-0/icedtea-2.6.11/openjdk/jdk/make/java/version'
> make[5]: Entering directory '/tmp/guix-build-icedtea-2.6.11.drv-0/icedtea-2.6.11/openjdk/jdk/make/java/jvm'
> logname: no login name
> INFO: ENABLE_FULL_DEBUG_SYMBOLS=1
> INFO: ALT_OBJCOPY=/gnu/store/nnykzgwfy8mwh2gmxm715sjxykg8qjwn-binutils-2.28/bin/objcopy
> INFO: /gnu/store/nnykzgwfy8mwh2gmxm715sjxykg8qjwn-binutils-2.28/bin/objcopy cmd found so will create .debuginfo files.
> INFO: STRIP_POLICY=no_strip
> INFO: ZIP_DEBUGINFO_FILES=1
> /gnu/store/m9l0j7apf9ac7shqwi5sh4hsn12x4dnk-coreutils-8.27/bin/mkdir -p /tmp/guix-build-icedtea-2.6.11.drv-0/icedtea-2.6.11/openjdk.build/include
> rm -f /tmp/guix-build-icedtea-2.6.11.drv-0/icedtea-2.6.11/openjdk.build/include/jni.h
> /gnu/store/m9l0j7apf9ac7shqwi5sh4hsn12x4dnk-coreutils-8.27/bin/cp ../../../src/share/javavm/export/jni.h /tmp/guix-build-icedtea-2.6.11.drv-0/icedtea-2.6.11/openjdk.build/include/jni.h
> make[5]: *** No rule to make target '/tmp/guix-build-icedtea-2.6.11.drv-0/icedtea-2.6.11/openjdk.build/include/linux/jni_md.h', needed by 'build'.  Stop.
> make[5]: Leaving directory '/tmp/guix-build-icedtea-2.6.11.drv-0/icedtea-2.6.11/openjdk/jdk/make/java/jvm'
> make[4]: *** [Makefile:63: all] Error 1
> make[4]: Leaving directory '/tmp/guix-build-icedtea-2.6.11.drv-0/icedtea-2.6.11/openjdk/jdk/make/java'
> make[3]: *** [Makefile:253: all] Error 1
> make[3]: Leaving directory '/tmp/guix-build-icedtea-2.6.11.drv-0/icedtea-2.6.11/openjdk/jdk/make'
> make[2]: *** [make/jdk-rules.gmk:93: jdk-build] Error 2
> make[2]: Leaving directory '/tmp/guix-build-icedtea-2.6.11.drv-0/icedtea-2.6.11/openjdk'
> make[1]: *** [Makefile:251: build_product_image] Error 2
> make[1]: Leaving directory '/tmp/guix-build-icedtea-2.6.11.drv-0/icedtea-2.6.11/openjdk'
> make: *** [Makefile:2463: stamps/icedtea.stamp] Error 2
> phase `build' failed after 12663.3 seconds
> …
> --8<---------------cut here---------------end--------------->8---

Yes I built icedtea@2 succesfully with this commit:
~ λ guix build icedtea@2 --no-grafts
/gnu/store/cbbn89cggf86fq57h7ya7jb70qckq49j-icedtea-2.6.11-doc
/gnu/store/xcaxjgafjip9pkfrnnrj18wfyykyjcrw-icedtea-2.6.11-jdk
/gnu/store/vk6llk5zmvwysc9jcixj7hvxprazmri0-icedtea-2.6.11

And to confirm:
~ λ ls -lh /gnu/store/vk6llk5zmvwysc9jcixj7hvxprazmri0-icedtea-2.6.11/lib/amd64/ | grep libjvm
lrwxrwxrwx   2 root           root       85 1970-01-01  1970 libjvm.so -> /gnu/store/vk6llk5zmvwysc9jcixj7hvxprazmri0-icedtea-2.6.11/lib/amd64/server/libjvm.so

Kind regards,
Roel Janssen

Re: IcedTea is not linking correctly with libjvm.so

[Chris Marusich]

[-- Attachment #1: Type: text/plain, Size: 4193 bytes --]

Roel Janssen <roel@gnu.org> writes:

> Hi Ricardo,
>
> Ricardo Wurmus writes:
>
>> Hi Roel,
>>
>>> I used (symlink ...), added a FIXME, rebuilt to see if it worked, and
>>> pushed in 491dc2fb1.
>>
>> Did you successfully build icedtea@2 with this commit?  I failed to
>> build it on my laptop:
>>
>> --8<---------------cut here---------------start------------->8---
>> …
>> make[5]: Leaving directory '/tmp/guix-build-icedtea-2.6.11.drv-0/icedtea-2.6.11/openjdk/jdk/make/java/version'
>> make[5]: Entering directory '/tmp/guix-build-icedtea-2.6.11.drv-0/icedtea-2.6.11/openjdk/jdk/make/java/jvm'
>> logname: no login name
>> INFO: ENABLE_FULL_DEBUG_SYMBOLS=1
>> INFO: ALT_OBJCOPY=/gnu/store/nnykzgwfy8mwh2gmxm715sjxykg8qjwn-binutils-2.28/bin/objcopy
>> INFO: /gnu/store/nnykzgwfy8mwh2gmxm715sjxykg8qjwn-binutils-2.28/bin/objcopy cmd found so will create .debuginfo files.
>> INFO: STRIP_POLICY=no_strip
>> INFO: ZIP_DEBUGINFO_FILES=1
>> /gnu/store/m9l0j7apf9ac7shqwi5sh4hsn12x4dnk-coreutils-8.27/bin/mkdir -p /tmp/guix-build-icedtea-2.6.11.drv-0/icedtea-2.6.11/openjdk.build/include
>> rm -f /tmp/guix-build-icedtea-2.6.11.drv-0/icedtea-2.6.11/openjdk.build/include/jni.h
>> /gnu/store/m9l0j7apf9ac7shqwi5sh4hsn12x4dnk-coreutils-8.27/bin/cp
>> ../../../src/share/javavm/export/jni.h
>> /tmp/guix-build-icedtea-2.6.11.drv-0/icedtea-2.6.11/openjdk.build/include/jni.h
>> make[5]: *** No rule to make target '/tmp/guix-build-icedtea-2.6.11.drv-0/icedtea-2.6.11/openjdk.build/include/linux/jni_md.h', needed by 'build'.  Stop.
>> make[5]: Leaving directory '/tmp/guix-build-icedtea-2.6.11.drv-0/icedtea-2.6.11/openjdk/jdk/make/java/jvm'
>> make[4]: *** [Makefile:63: all] Error 1
>> make[4]: Leaving directory '/tmp/guix-build-icedtea-2.6.11.drv-0/icedtea-2.6.11/openjdk/jdk/make/java'
>> make[3]: *** [Makefile:253: all] Error 1
>> make[3]: Leaving directory '/tmp/guix-build-icedtea-2.6.11.drv-0/icedtea-2.6.11/openjdk/jdk/make'
>> make[2]: *** [make/jdk-rules.gmk:93: jdk-build] Error 2
>> make[2]: Leaving directory '/tmp/guix-build-icedtea-2.6.11.drv-0/icedtea-2.6.11/openjdk'
>> make[1]: *** [Makefile:251: build_product_image] Error 2
>> make[1]: Leaving directory '/tmp/guix-build-icedtea-2.6.11.drv-0/icedtea-2.6.11/openjdk'
>> make: *** [Makefile:2463: stamps/icedtea.stamp] Error 2
>> phase `build' failed after 12663.3 seconds
>> …
>> --8<---------------cut here---------------end--------------->8---
>
> Yes I built icedtea@2 succesfully with this commit:
> ~ λ guix build icedtea@2 --no-grafts
> /gnu/store/cbbn89cggf86fq57h7ya7jb70qckq49j-icedtea-2.6.11-doc
> /gnu/store/xcaxjgafjip9pkfrnnrj18wfyykyjcrw-icedtea-2.6.11-jdk
> /gnu/store/vk6llk5zmvwysc9jcixj7hvxprazmri0-icedtea-2.6.11
>
> And to confirm:
> ~ λ ls -lh /gnu/store/vk6llk5zmvwysc9jcixj7hvxprazmri0-icedtea-2.6.11/lib/amd64/ | grep libjvm
> lrwxrwxrwx 2 root root 85 1970-01-01 1970 libjvm.so ->
> /gnu/store/vk6llk5zmvwysc9jcixj7hvxprazmri0-icedtea-2.6.11/lib/amd64/server/libjvm.so

I also successfully built icedtea@2 using commit
491dc2fb16a5ae1d30d1607322e19922d8dec5c3.  Specifically, the following
command succeeded:

  ./pre-inst-env guix build --keep-failed --no-grafts --fallback icedtea@2

I also noticed that the following command seems to successfully build
the package, but it reports that the output is non-deterministic (which
was probably already the case):

  ./pre-inst-env guix build --check --keep-failed --no-grafts --fallback icedtea@2


Last few lines of output:

--8<---------------cut here---------------start------------->8---
phase `compress-documentation' succeeded after 0.1 seconds
note: keeping build directory `/tmp/guix-build-icedtea-2.6.11.drv-0'
guix build: error: build failed: derivation `/gnu/store/8ai4v71grgnigkzyb96a55flac8ii402-icedtea-2.6.11.drv' may not be deterministic: output `/gnu/store/cbbn89cggf86fq57h7ya7jb70qckq49j-icedtea-2.6.11-doc' differs from ‘/gnu/store/cbbn89cggf86fq57h7ya7jb70qckq49j-icedtea-2.6.11-doc-check’
--8<---------------cut here---------------end--------------->8---

I wasn't able to respond sooner because it took a few hours to do each
build.

-- 
Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]