bug#36276: Not found .so files in bundle made by guix pack

bug#36276: Not found .so files in bundle made by guix pack

[Andréas Livet]

Hi,

I try to bundle scribus with guix pack in order to install it on a 
centos system on which admin sys doesn't want us to install guix.

So I created a tarball with a symlink to '/usr/bin' (don't know it's 
really the recommanded way), this way scribus binary will automatically 
be in the $PATH.

Here is how I created the bundle :

guix pack -S /usr/bin=bin scribus

I tried it on my Xubuntu 18.04 system and it worked like a charm.

But, when deployed on the centos machine, it did not found 
"libQt5Core.so.5" depsite the file exists.

ldd 
/gnu/store/5xqmac50axcgc8vrrn30jvzndjvrqd6a-scribus-1.5.4/bin/.scribus-real 
| grep Qt
     libQt5Xml.so.5 => 
/gnu/store/890yw9i7lcjnnxhlza8r121352xpb1hi-qtbase-5.11.3/lib/libQt5Xml.so.5 
(0x00007f60dd9f6000)
     libQt5Network.so.5 => 
/gnu/store/890yw9i7lcjnnxhlza8r121352xpb1hi-qtbase-5.11.3/lib/libQt5Network.so.5 
(0x00007f60dd7a4000)
     libQt5OpenGL.so.5 => 
/gnu/store/890yw9i7lcjnnxhlza8r121352xpb1hi-qtbase-5.11.3/lib/libQt5OpenGL.so.5 
(0x00007f60dd99a000)
     libQt5PrintSupport.so.5 => 
/gnu/store/890yw9i7lcjnnxhlza8r121352xpb1hi-qtbase-5.11.3/lib/libQt5PrintSupport.so.5 
(0x00007f60dd72b000)
     libQt5Widgets.so.5 => 
/gnu/store/890yw9i7lcjnnxhlza8r121352xpb1hi-qtbase-5.11.3/lib/libQt5Widgets.so.5 
(0x00007f60da93b000)
     libQt5Gui.so.5 => 
/gnu/store/890yw9i7lcjnnxhlza8r121352xpb1hi-qtbase-5.11.3/lib/libQt5Gui.so.5 
(0x00007f60da3a6000)
     libQt5Core.so.5 => not found

I don't understand why it founds libQt5Gui.so.5 and not libQt5Core.so.5 
which is in the same directory !

I tried to copy the .so inside the bin directory, to add qbase lib path 
to $LD_LIBRARY_PATH, doesn't seem to work.

Can't find any further information on guix manual, so I'm asking here.

Thanks in advance for your help,

Andréas

bug#36276: Not found .so files in bundle made by guix pack

[Ludovic Courtès]

Hi,

Andréas Livet <andreas@livet.me> skribis:

> Here is how I created the bundle :
>
> guix pack -S /usr/bin=bin scribus
>
> I tried it on my Xubuntu 18.04 system and it worked like a charm.

You might want to pass -RR to get a relocatable binary.  :-)

> But, when deployed on the centos machine, it did not found
> "libQt5Core.so.5" depsite the file exists.
>
> ldd
> /gnu/store/5xqmac50axcgc8vrrn30jvzndjvrqd6a-scribus-1.5.4/bin/.scribus-real
> | grep Qt

[...]

>     libQt5Core.so.5 => not found
>

Could you make sure that LD_LIBRARY_PATH is unset?

HTH,
Ludo’.

bug#36276: Not found .so files in bundle made by guix pack

[Andréas Livet]

Hi Ludo,
> You might want to pass -RR to get a relocatable binary.  :-)

Yes, I'm uploading the new tarball on the server, it's a bit slow...

I didn't understand what did -RR option at first, but now it's seems to 
be a good (maybe the only?) option for installation outside the / 
directory (which is not a problem for me because I have root privilege 
on the server, just that admin sys doesn't want us to install tools like 
Docker, guix, etc. because he'll have to maintain the server after) .

> Could you make sure that LD_LIBRARY_PATH is unset?

Yes it is unset. This problem is driving me crazy, I can't understand 
why it can find libQt5Gui.so.5 and not libQt5Core.so.5

As far as I understand how dynamic linking is done under linux, the 
binary looks for .so files that it depends on inside folders define in 
the binary itself and on the operating system (/lib, /lib64).

And when it founds an .so with the corresponding name it stops, then 
search for another .so (if there is one).

Then how can it not found this .so file ?

* I've check it with sha1sum, it's clean.

* I check it with "nm -D" it contains all the symbols.

* I've check the binary with "chrpath -l 
/gnu/store/5xqmac50axcgc8vrrn30jvzndjvrqd6a-scribus-1.5.4/bin/.scribus-real", 
it contains 
"/gnu/store/890yw9i7lcjnnxhlza8r121352xpb1hi-qtbase-5.11.3/lib" which 
contains libQt5Core.so.5


I really can't see what's the problem here... Hope it will work with the 
tarball generated with -RR, but it will leave the problem unsolved...


Again, thanks for your help and what you're doing for guix,

Andréas

bug#36276: Not found .so files in bundle made by guix pack

[Andréas Livet]

Hi,

I still got the problem with a pack build with -RR, I started to think 
about a centos bug...

If you have any ideas, let me know.

Andréas

bug#36276: Not found .so files in bundle made by guix pack

[Ludovic Courtès]

Hi,

Andréas Livet <andreas@livet.me> skribis:

>> You might want to pass -RR to get a relocatable binary.  :-)
>
> Yes, I'm uploading the new tarball on the server, it's a bit slow...
>
> I didn't understand what did -RR option at first, but now it's seems
> to be a good (maybe the only?) option for installation outside the /
> directory

Yes.

<https://www.gnu.org/software/guix/blog/2018/tarballs-the-ultimate-container-image-format/>
explains ‘-R’, and
<https://www.gnu.org/software/guix/blog/2019/gnu-guix-1.0.0-released/>
mentions ‘-RR’.

>> Could you make sure that LD_LIBRARY_PATH is unset?
>
> Yes it is unset. This problem is driving me crazy, I can't understand
> why it can find libQt5Gui.so.5 and not libQt5Core.so.5

Is there a /etc/ld.so.preload file?

Did you try running the pack on another non-Guix machine?

Thanks,
Ludo’.

bug#36276: Not found .so files in bundle made by guix pack

[Andréas Livet]

Hi,
> <https://www.gnu.org/software/guix/blog/2018/tarballs-the-ultimate-container-image-format/>
> explains ‘-R’, and
> <https://www.gnu.org/software/guix/blog/2019/gnu-guix-1.0.0-released/>
> mentions ‘-RR’.
Thanks for the documentation :)
> Is there a /etc/ld.so.preload file?
No. Only some very simples ld.so.config in /etc/ld.so.conf.d
> Did you try running the pack on another non-Guix machine?

I tried yesterday I works as expected on a Xubuntu 16.04 machine without 
guix installed on it.

Don't have much time to investigate on it.

For information, it's a CentOS 7.6.1810 server which I did not configure 
from the beginning, so I don't know precisely which changes has been 
made on it...

I think, maybe there is a conflict of another libQt5Core (not 5) 
installed on the machine ?

Anyway, thanks for your help,

Andréas

bug#36276: Not found .so files in bundle made by guix pack

[Ludovic Courtès]

Hello,

Andréas Livet <andreas@livet.me> skribis:

>> Is there a /etc/ld.so.preload file?
> No. Only some very simples ld.so.config in /etc/ld.so.conf.d

Hmm, can you try temporarily moving them out of the way to see if it
makes a difference?

AFAICS only /etc/ld.so.preload is accessed, but who knows.

> For information, it's a CentOS 7.6.1810 server which I did not
> configure from the beginning, so I don't know precisely which changes
> has been made on it...
>
> I think, maybe there is a conflict of another libQt5Core (not 5)
> installed on the machine ?

That could be the case, but only if there’s LD_LIBRARY_PATH,
ld.so.preload, or something like that interfering.

What if you run:

  env -i …/bin/scribus

where you replace “…” with the full file name of your extracted pack?

HTH,
Ludo’.

bug#36276: Not found .so files in bundle made by guix pack

[Andréas Livet]

Hi !

Found a solution !

https://superuser.com/a/1348051

Don't really understand why I got this problem which seems to be WSL only...

Do you think it is related to guix pack in some way ?

Thank you,

Andréas

bug#36276: Not found .so files in bundle made by guix pack

[Ludovic Courtès]

Hello,

Andréas Livet <andreas@livet.me> skribis:

> Found a solution !
>
> https://superuser.com/a/1348051

Good!  So you ran:

  strip --remove-section=.note.ABI-tag \
     ./gnu/store/…/lib/libQt5Core.so.5

right?

Indeed the .so file produced by Guix has this section:

--8<---------------cut here---------------start------------->8---
$ file -L /gnu/store/y1nlilwa34wqvmvmraggwv12jfdp0kya-qtbase-5.11.3/lib/libQt5Core.so
/gnu/store/y1nlilwa34wqvmvmraggwv12jfdp0kya-qtbase-5.11.3/lib/libQt5Core.so: ELF 64-bit LSB pie executable x86-64, version 1 (GNU/Linux), dynamically linked, interpreter /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/ld-linux-x86-64.so.2, for GNU/Linux 3.16.0, stripped
--8<---------------cut here---------------end--------------->8---

… which reflects in the “for GNU/Linux 3.16.0” that we see above.

What does “uname -r” return on this CentOS machine?  I’m guessing it’s
older than 3.16.0.

The effect of the ‘strip’ command is that libQt5Base.so can be loaded,
but some functionality maybe missing (things that use the ‘renameat2’
system call specifically; see below.)

> Do you think it is related to guix pack in some way ?

I researched it a bit and in qtbase, in
‘src/corelib/global/minimum-linux_p.h’, we can see this:

--8<---------------cut here---------------start------------->8---
#if QT_CONFIG(getentropy)
#  define MINLINUX_MAJOR        3
#  define MINLINUX_MINOR        17
#  define MINLINUX_PATCH        0
#elif QT_CONFIG(renameat2)
#  define MINLINUX_MAJOR        3
#  define MINLINUX_MINOR        16
#  define MINLINUX_PATCH        0
#else
#  define MINLINUX_MAJOR        2
#  define MINLINUX_MINOR        6
#  define MINLINUX_PATCH        28
#endif
--8<---------------cut here---------------end--------------->8---

Since we build Qt with ‘renameat2’ support, Qt effectively requires a
Linux kernel >= 3.16.0.

We could build it without that requirement; 3.16.0 is already rather
old, so I’m not sure we should do it.

Thoughts?

(Note that the Guix package already has a patch to allow it to work with
kernels < 4.11.)

Ludo’.

bug#36276: Not found .so files in bundle made by guix pack

[Andréas Livet]

Hello,

> Good!  So you ran:
>
>    strip --remove-section=.note.ABI-tag \
>       ./gnu/store/…/lib/libQt5Core.so.5
>
> right?
Yes exactly :).
>
> Indeed the .so file produced by Guix has this section:
>
> --8<---------------cut here---------------start------------->8---
> $ file -L /gnu/store/y1nlilwa34wqvmvmraggwv12jfdp0kya-qtbase-5.11.3/lib/libQt5Core.so
> /gnu/store/y1nlilwa34wqvmvmraggwv12jfdp0kya-qtbase-5.11.3/lib/libQt5Core.so: ELF 64-bit LSB pie executable x86-64, version 1 (GNU/Linux), dynamically linked, interpreter /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/ld-linux-x86-64.so.2, for GNU/Linux 3.16.0, stripped
> --8<---------------cut here---------------end--------------->8---
>
> … which reflects in the “for GNU/Linux 3.16.0” that we see above.
>
> What does “uname -r” return on this CentOS machine?  I’m guessing it’s
> older than 3.16.0.
 > 3.10.0-957.12.2.el7.x86_64

What a shame, this kernel was release in 2013 !

I start to think that our VM was not installed properly because regular 
centos install seems to have a much more recent kernel...

> The effect of the ‘strip’ command is that libQt5Base.so can be loaded,
> but some functionality maybe missing (things that use the ‘renameat2’
> system call specifically; see below.)
>
>> Do you think it is related to guix pack in some way ?
> I researched it a bit and in qtbase, in
> ‘src/corelib/global/minimum-linux_p.h’, we can see this:
>
> --8<---------------cut here---------------start------------->8---
> #if QT_CONFIG(getentropy)
> #  define MINLINUX_MAJOR        3
> #  define MINLINUX_MINOR        17
> #  define MINLINUX_PATCH        0
> #elif QT_CONFIG(renameat2)
> #  define MINLINUX_MAJOR        3
> #  define MINLINUX_MINOR        16
> #  define MINLINUX_PATCH        0
> #else
> #  define MINLINUX_MAJOR        2
> #  define MINLINUX_MINOR        6
> #  define MINLINUX_PATCH        28
> #endif
> --8<---------------cut here---------------end--------------->8---

Well done :D, I didn't have time to investigated as far as you did.

> Since we build Qt with ‘renameat2’ support, Qt effectively requires a
> Linux kernel >= 3.16.0.
>
> We could build it without that requirement; 3.16.0 is already rather
> old, so I’m not sure we should do it.
>
> Thoughts?

IMHO, don't bother, as you say 3.16.0 is already very old so it's OK to 
let it this way.

Maybe it could be worth to mention in guix pack manual that bundle may 
not always works on too old kernel ?

Difficult to be precise because it'll depends on projects, don't really 
know how to write it down in a way it'll help people to understand 
problems like this one.

Thanks again for your hard work,

Andréas

bug#36276: Not found .so files in bundle made by guix pack

[Ricardo Wurmus]

Hi Andréas,

>> What does “uname -r” return on this CentOS machine?  I’m guessing it’s
>> older than 3.16.0.
>> 3.10.0-957.12.2.el7.x86_64
>
> What a shame, this kernel was release in 2013 !
>
> I start to think that our VM was not installed properly because
> regular centos install seems to have a much more recent kernel...

I’m afraid not.  This is the kernel version on an arbitrary node running
CentOS 7.4.1708: 3.10.0-693.21.1.el7.x86_64

RHEL 7.x won’t ever include any kernel version higher than 3.10.0.
Until it goes EOL they’ll merely backport fixes and sometimes APIs to
what they call 3.10.0, but which won’t be anywhere close to what a
vanilla 3.10.0 kernel provides.

The same happened on RHEL 6, which still provides what they call 2.6.32,
which is nothing like vanilla 2.6.32.

Sometimes that heavily patched kernel will include an interface that
later versions introduced and thus the kernel version checks will be
inaccurate.  We had this problem with the RHEL 6 kernel and the GNU C
library, which is why we patched the glibc to make an exception for
Linux version 2.6.32 (which only in its RHEL 6 variant provides the
required interfaces).

--
Ricardo

bug#36276: Not found .so files in bundle made by guix pack

[Ludovic Courtès]

Hello,

Andréas Livet <andreas@livet.me> skribis:

>> What does “uname -r” return on this CentOS machine?  I’m guessing it’s
>> older than 3.16.0.
>> 3.10.0-957.12.2.el7.x86_64
>
> What a shame, this kernel was release in 2013 !

Yeah.

>> ‘src/corelib/global/minimum-linux_p.h’, we can see this:
>>
>> --8<---------------cut here---------------start------------->8---
>> #if QT_CONFIG(getentropy)
>> #  define MINLINUX_MAJOR        3
>> #  define MINLINUX_MINOR        17
>> #  define MINLINUX_PATCH        0
>> #elif QT_CONFIG(renameat2)
>> #  define MINLINUX_MAJOR        3
>> #  define MINLINUX_MINOR        16
>> #  define MINLINUX_PATCH        0
>> #else
>> #  define MINLINUX_MAJOR        2
>> #  define MINLINUX_MINOR        6
>> #  define MINLINUX_PATCH        28
>> #endif
>> --8<---------------cut here---------------end--------------->8---
>
> Well done :D, I didn't have time to investigated as far as you did.

I didn’t “have” the time, I took it.  :-)

> IMHO, don't bother, as you say 3.16.0 is already very old so it's OK
> to let it this way.

Sounds good.

> Maybe it could be worth to mention in guix pack manual that bundle may
> not always works on too old kernel ?
>
> Difficult to be precise because it'll depends on projects, don't
> really know how to write it down in a way it'll help people to
> understand problems like this one.

Yes, it’s a bit hard to document it because it’ll vary from package to
package, and the vast majority of packages don’t care about the kernel
version.

Anyway, thanks for your report, and I hope you can find a suitable
workaround!

Ludo’.