Why is Guile now setting LD_LIBRARY_PATH?

Why is Guile now setting LD_LIBRARY_PATH?

[Bruce Korb]

> $ guile --version
> guile (GNU Guile) 2.0.5
> Copyright (C) 2011 Free Software Foundation, Inc.

I added a couple of debug printf's after several hours chasing down
why my program was crashing:

> in main() LD_LIBRARY_PATH='EMPTY'
> in inner_main() LD_LIBRARY_PATH='/usr/lib64:/usr/lib64/guile/2.0/extensions'

It was crashing because it was linking against a library in /usr/lib64
instead of /usr/local/lib64.  Now I know why.

1.  Why was this done?
2.  Why no notice?  This is *BROKEN*.

libguile needs to be linked with -Wl,-rpath -Wl,/usr/lib64/guile/2.0/extensions
instead of messing up your client's link/load.  ("/usr/lib64" should not be needed.)

Distribution:  openSuSE 12.2  (Just in case they are not following your
build procedures and I need to harass them.)

Re: Why is Guile now setting LD_LIBRARY_PATH?

[Bruce Korb]

On 09/10/12 16:48, Bruce Korb wrote:
>> $ guile --version
>> guile (GNU Guile) 2.0.5
>> Copyright (C) 2011 Free Software Foundation, Inc.
> 
> I added a couple of debug printf's after several hours chasing down
> why my program was crashing:
> 
>> in main() LD_LIBRARY_PATH='EMPTY'
>> in inner_main() LD_LIBRARY_PATH='/usr/lib64:/usr/lib64/guile/2.0/extensions'
> 
> It was crashing because it was linking against a library in /usr/lib64
> instead of /usr/local/lib64.  Now I know why.
> 
> 1.  Why was this done?
> 2.  Why no notice?  This is *BROKEN*.
> 
> libguile needs to be linked with -Wl,-rpath -Wl,/usr/lib64/guile/2.0/extensions
> instead of messing up your client's link/load.  ("/usr/lib64" should not be needed.)
> 
> Distribution:  openSuSE 12.2  (Just in case they are not following your
> build procedures and I need to harass them.)


P.S.  the fix seems to be this, though, naturally, I hate it.

static void
inner_main(void * closure, int argc, char ** argv)
{
    atexit(done_check);
    initialize(argc, argv);

    {
        char const * p = getenv("LD_LIBRARY_PATH");
        if (p != NULL)
            putenv("LD_LIBRARY_PATH=");
    }

Re: Why is Guile now setting LD_LIBRARY_PATH?

[Greg Troxel]

[-- Attachment #1: Type: text/plain, Size: 1703 bytes --]


Bruce Korb <bkorb@gnu.org> writes:

> On 09/10/12 16:48, Bruce Korb wrote:
>>> $ guile --version
>>> guile (GNU Guile) 2.0.5
>>> Copyright (C) 2011 Free Software Foundation, Inc.
>> 
>> I added a couple of debug printf's after several hours chasing down
>> why my program was crashing:
>> 
>>> in main() LD_LIBRARY_PATH='EMPTY'
>>> in inner_main() LD_LIBRARY_PATH='/usr/lib64:/usr/lib64/guile/2.0/extensions'
>> 
>> It was crashing because it was linking against a library in /usr/lib64
>> instead of /usr/local/lib64.  Now I know why.
>> 
>> 1.  Why was this done?
>> 2.  Why no notice?  This is *BROKEN*.
>> 
>> libguile needs to be linked with -Wl,-rpath
>> -Wl,/usr/lib64/guile/2.0/extensions
>> instead of messing up your client's link/load.  ("/usr/lib64" should
>> not be needed.)
>> 
>> Distribution:  openSuSE 12.2  (Just in case they are not following your
>> build procedures and I need to harass them.)
>
>
> P.S.  the fix seems to be this, though, naturally, I hate it.
>
> static void
> inner_main(void * closure, int argc, char ** argv)
> {
>     atexit(done_check);
>     initialize(argc, argv);
>
>     {
>         char const * p = getenv("LD_LIBRARY_PATH");
>         if (p != NULL)
>             putenv("LD_LIBRARY_PATH=");
>     }

I don't follow why you think this is the right fix.  It seems that guile
(main) and libguile (library) should both not set LD_LIBRARY_PATH at
all.

Also, lib64 is a linuxism; BSDs have the architecture's libs in /usr/lib
and alternate ones in e.g /emul/netbsd32 (such as i386 libs on amd64
machines).  So setting LD_LIBRARY_PATH has to be done in an os and
arch-dependent way, which is complicated.

[-- Attachment #2: Type: application/pgp-signature, Size: 194 bytes --]

Re: Why is Guile now setting LD_LIBRARY_PATH?

[Ludovic Courtès]

Hi Bruce,

Bruce Korb <bkorb@gnu.org> skribis:

>> in main() LD_LIBRARY_PATH='EMPTY'
>> in inner_main() LD_LIBRARY_PATH='/usr/lib64:/usr/lib64/guile/2.0/extensions'

The reason provided in ‘sysdep_dynl_init’ is:

    /* Add SCM_LIB_DIR and SCM_EXTENSIONS_DIR to the loader's search
       path.  `lt_dladdsearchdir' and $LTDL_LIBRARY_PATH can't be used
       for that because they are searched before the system-dependent
       search path, which is the one `libtool --mode=execute -dlopen'
       fiddles with (info "(libtool) Libltdl Interface").  See
       <http://lists.gnu.org/archive/html/guile-devel/2010-11/msg00095.html>
       for details.  */
    augment_env (SHARED_LIBRARY_PATH_VARIABLE, SCM_LIB_DIR);
    augment_env (SHARED_LIBRARY_PATH_VARIABLE, SCM_EXTENSIONS_DIR);

> libguile needs to be linked with -Wl,-rpath -Wl,/usr/lib64/guile/2.0/extensions
> instead of messing up your client's link/load.  ("/usr/lib64" should not be needed.)

Problem is that (1) -rpath is not always available, and (2) when using
GNU ld with --enable-new-dtags, it produces a RUNPATH (not an RPATH),
which can be overridden by $LD_LIBRARY_PATH.

I agree that using setenv is ugly, but finding an option that’s portable
and doesn’t break existing programs seems tricky too.

Do you have any solutions meeting this criteria in mind?

Thanks,
Ludo’.

Re: Why is Guile now setting LD_LIBRARY_PATH?

[Bruce Korb]

Hi Ludo,
>> in main() LD_LIBRARY_PATH='EMPTY'
>> in inner_main() LD_LIBRARY_PATH='/usr/lib64:/usr/lib64/guile/2.0/extensions'
>
>The reason provided in ‘sysdep_dynl_init’ is:
>
>/* Add SCM_LIB_DIR and ... See
>   <http://lists.gnu.org/archive/html/guile-devel/2010-11/msg00095.html>
>   for details. */

I am sure you are not arguing that I ought to have known due to a comment
in source code and an email from a couple of years ago.  My point is that
messing with the library search path in an effectively secret manner
caused me to spend hours chasing the problem.

>> libguile needs to be linked with -Wl,-rpath
>> -Wl,/usr/lib64/guile/2.0/extensions
>> instead of messing up your client's link/load. ("/usr/lib64" should not be
>> needed.)
>
>Problem is that (1) -rpath is not always available, and (2) when using
>GNU ld with --enable-new-dtags, it produces a RUNPATH (not an RPATH),
>which can be overridden by $LD_LIBRARY_PATH.
>
>I agree that using setenv is ugly, but finding an option that’s portable
>and doesn’t break existing programs seems tricky too.
>
>Do you have any solutions meeting this criteria in mind?

Wrap your dlopen's in code that pushes the needed values to the environment
and then removes them when dlopen is done.  You still have multi-thread
issues when someone is calling dlopen or exec while a libguile thread
is messing with the environment.  Perhaps you could document a global
lock variable that could single-thread these operations for multi-threaded
applications.  Either way, fiddling the environment only while using dlopen
is far better than breaking people's applications in unexpected ways.

At the end:  messing with LD_LIBRARY_PATH in a library breaks programs
that use the library and make calls to exec or dlopen.  Please find
a solution that does not do this, or at least minimizes the issues.

Thank you.  Regards, Bruce

C.F. Greg Troxel:
> I don't follow why you think this is the right fix.  It seems that guile
>  (main) and libguile (library) should both not set LD_LIBRARY_PATH at
> all.
To be clear, I do not think this the right fix, but since my application has to
cope with un-fixed guile releases, I have to do something.  Nuking the
variable is the easiest.   WRT guile the program and libguile the library, you
are correct:  neither should set LD_LIBRARY_PATH.  It needs to be a
user selection to adjust to specific issues in a user's environment.

Re: Why is Guile now setting LD_LIBRARY_PATH?

[Bruce Korb]

On 09/11/12 09:21, Bruce Korb wrote:
>> Do you have any solutions meeting this criteria in mind?
> 
> Wrap your dlopen's in code that pushes the needed values to the environment

I have a better solution that does not require serializing environment
variable access:  do your own search.  Do full path dlopen-s first,
using the paths you otherwise would have put into LD_LIBRARY_PATH.
If they fail, then fall back to whatever the glibc would otherwise
do (maybe even using LD_LIBRARY_PATH -- it hasn't been touched).

Re: Why is Guile now setting LD_LIBRARY_PATH?

[Mark H Weaver]

On 09/13/2012 07:30 PM, Bruce Korb wrote:
> On 09/11/12 09:21, Bruce Korb wrote:
>>> Do you have any solutions meeting this criteria in mind?
>>
>> Wrap your dlopen's in code that pushes the needed values to the environment
>
> I have a better solution that does not require serializing environment
> variable access:  do your own search.  Do full path dlopen-s first,
> using the paths you otherwise would have put into LD_LIBRARY_PATH.
> If they fail, then fall back to whatever the glibc would otherwise
> do (maybe even using LD_LIBRARY_PATH -- it hasn't been touched).

I haven't looked closely at the relevant code, but I tend to agree with 
Bruce here.  We should not change LD_LIBRARY_PATH.  We should do the 
search ourselves.

      Mark