Re: [PATCH] Don't augment LD_LIBRARY_PATH (was Re: [PATCH] do not augment environment)

[Mark H Weaver <mhw@netris.org>]

ludo@gnu.org (Ludovic Courtès) writes:
> Mark H Weaver <mhw@netris.org> skribis:
>
>> As I understand it, the reason given for why we cannot use that approach
>> is that 'libtool --mode=execute -dlopen' would not work properly,
>
> Exactly.
>
>> but why do we have to do it that way?
>
> It is a fact that some projects (at least some of mine) have been using
> that idiom,

Do these projects also set GUILE_SYSTEM_EXTENSIONS_PATH?

> because that’s the Libtool way to say “hey, load this
> particular file, not one that may be found in the search path.”

I would refine this to "load this particular file, not one that may be
found in LD_LIBRARY_PATH or the default system library search paths,
*unless* the library is found in either LTDL_LIBRARY_PATH or in some
path added via 'lt_dladdsearchdir' et al."

In other words, libtool's own preferred mechanism for programs to
augment their internal search paths gives those internal paths priority
over the files specified by 'libtool --mode=execute -dlopen'.

So it seems to me that either this is a bug in libtool that should be
fixed, or it is not appropriate to expect this idiom to override a
program's extensions.

Having said that, I acknowledge that the existence of programs that
depend upon this idiom to override Guile's SCM_EXTENSIONS_DIR force us
to accommodate them somehow, at least in 2.0.

> Ideally, I would accept any solution that (1) gets rid of the
> LD_LIBRARY_PATH export, and (2) can be shown with strace to preserve the
> extension search order.

As I pointed out in my recent followup, the current extension search
order is fundamentally broken, because in the common case of
SCM_LIB_DIR=/usr/lib, it typically puts /usr/lib before /usr/local/lib
in the search path.  This will cause problems for users that depend upon
/usr/local/lib preceding /usr/lib when loading libraries from Guile.

Therefore, I don't think we should try to exactly preserve the current
broken search order.  At the very least, I think we need to avoid adding
SCM_LIB_DIR anywhere other than as a last resort (which means a manual
search), and perhaps not even then.

For SCM_EXTENSIONS_DIR, if we need to allow programs to override it
using 'libtool --mode=execute -dlopen' (and without setting
GUILE_SYSTEM_EXTENSIONS_PATH) then I guess the only good option is to
manually search for SCM_EXTENSIONS_DIR after all else has failed.

Does that make sense?

The one remaining question is where to put a non-empty
GUILE_SYSTEM_EXTENSIONS_PATH in the library search paths.  It's hard for
me to answer that question without knowing how this variable is supposed
to be used.  It's not documented, but its name suggests that it should
override the value of SCM_EXTENSIONS_DIR, and therefore be put where
SCM_EXTENSIONS_DIR would have been put.

To my mind, a sensible use for this variable is to allow the user to
place Guile extensions in directories other than
$(libdir)/guile/2.0/extensions/, for example somewhere within their home
directory.  So a user might reasonably set this variable in their shell
startup files.  However, if they set the variable to anything at all,
then they would also need to include $(libdir)/guile/2.0/extensions/
somewhere within it.  If we want your idiom to override that directory,
then GUILE_SYSTEM_EXTENSIONS_PATH also needs to be placed where
SCM_EXTENSIONS_DIR would have been put.

I realize that backward compatibility is important, but it's also
important to fix broken behavior and improve things over time, when it
can be done without significantly burdening users in practice.

What do you think?

    Mark