Re: INSTALL matters

[Neil Jerram <neil@ossau.uklinux.net>]

Bruno Haible <bruno@clisp.org> writes:

> Hi Neil,
>
> Neil Jerram wrote in reply to
> <http://lists.gnu.org/archive/html/bug-guile/2008-05/msg00012.html>:
>> This issue, and the related (in
>> my mind) one about having multiple copies of a library installed in
>> different places
>
> Having multiple copies of a package installed in different places is
> the most normal thing of the world. That's what --prefix is for.

OK...

>> First, a couple of assumptions: as far as I understand,
>> 
>> - in general, there is no portable way of telling the compiler and
>>   linker to look in one set of prefixes for library A, and in a
>>   different set of prefixes for library B
>
> There is. It consists in passing absolute file names for .h and .a or .so
> files to the linker. If the input you have is a _set_ of of prefixes,
> this means you need to search for the library in the same way as the
> compiler or linker would do, and then pass the absolute file names to the
> compiler or linker.
>
> It is a bit far-fetched, but the mentioned macros in gnulib actually do this.

Thanks for explaining.  I've seen these macros in gnulib now.

> Correct. Even if you pass absolute filenames and '-rpath', on some systems,
> the information written into the executable by the linker does not contain
> sufficient information for choosing library A from /usr/lib (and not
> /usr/local/lib) while at the same time choosing library B from /usr/local/lib
> (and not /usr/lib).

I think you're saying here that the new gnulib macros don't work on all
platforms.  Is that correct?

>> - worse (at least with the GNU toolchain), if you have multiple
>>   versions of library A in /usr/lib and /usr/local/lib, you can't get
>>   a consistent build at all, because gcc prefers /usr/include when
>>   looking for headers, but ld and ld.so prefer /usr/local/lib when
>>   linking.  (Or perhaps the other way round.
>
> This is a problem on some BSD systems, IIRC. Also one specific GCC release
> had this problem on Linux systems, a couple of years ago, and it was fixed
> in the next GCC release. So I've come to view this as an inconsistency in
> the GCC installation. The workaround to this situation is to never use
> /usr/local, but rather pass a different --prefix always.

OK, that's good news.  After looking at the latest GCC and binutils
docs, I agree that this old issue seems to be fixed now.  (-I and -L
directories are now always searched before the system-standard ones.)

>> > I'm trying to install guile-1.8.5 on MacOS X 10.5. I've already installed
>> > gmp-4.2.2 with the same --prefix option as I use for guile,
>> 
>> This suggests a possibility, of automatically adding $prefix/include
>> to CPPFLAGS and $prefix/lib to LDFLAGS.
>
> Yes. If I have installed gmp with the same --prefix option, then this
> is needed and desired. Autoconf does not do it automatically.

But you are happy for this to require saying `--with-gmp-prefix=/opt'
(or whatever) when configuring Guile, right?

>> In some specific contexts - e.g. an admin setting up a box with all
>> 3rd party software under /opt - I think that this would indeed be a
>> convenient option.
>
> I don't think so. The users of this admin will wonder why their programs
> behave differently when built with GCC than when built with the vendor's
> compiler. (The reason would be that GCC links with different versions of
> thirs-party libraries.)

In any case, this debate isn't at all Guile-specific; more on that
below.

>> I don't like these options because they seem bogus to me.  Providing
>> `--with-libreadline-prefix' makes it sound like the build and runtime
>> will look specifically (and only) in that prefix when building/linking
>> readline, and in the default places for other libraries.  And that is
>> not true
>
> It is correct that the option does not mean "hey configure, please
> look for libreadline *exactly* here". It means "hey configure, I have
> installed libreadline here".
>
> Yes there are weird cases which cannot be supported, like the case
> with libraries A and B in both /usr and /usr/local, which you mentioned
> before. But the majority of the cases is not so weird, and in these
> cases the --with-...-prefix options are helpful.

Now I'm confused.  I thought that doing what the new gnulib macros do
(i.e. specifying full paths to the linker, and using rpath) means that
cases like these _can_ be supported - at least on the platforms where
the full paths are stored and respected.

>> Using CPPFLAGS and LDFLAGS is better, in my view, because it
>> corresponds to what the toolchain can actually do - i.e. search in one
>> global set of places for all headers, and in one global set of places
>> for all libraries.
>
> I find a configure option better, because when the user installs a
> package, he's not thinking at the level of linker details, but rather
> "I have installed gmp here. Now how do I tell mpfr or guile about it?"

OK, I agree now that that makes sense.

> Before I came to use --with-...-prefix options, I used CPPFLAGS and
> LDFLAGS like you say. And quite often, the build failed because
>   - I had not been aware of all dependencies before building the package, or
>   - I had used LDFLAGS="-L/opt/gnu", rather than LDFLAGS="-L/opt/gnu/lib",
>     (Simple typo/thinko, but frustrating anyway.)
>   - I had used LDFLAGS="-L/opt/gnu/lib" but the build needs the 64-bit libraries
>     so I should have used LDFLAGS="-L/opt/gnu/lib64".
>
> The --with-...-prefix options do reduce frustration in these cases.
>
>> > As an installer, I'm not going to spend time trying to see which LDFLAGS I need
>> > to set for a package to recognize the libraries that I have installed. I expect
>> > the package to do this by itself.
>> 
>> I don't get this last paragraph.  You have to add a path either to
>> --with-xxx-prefix or to CPPFLAGS and LDFLAGS, so what difference does
>> it make?
>
> The difference that I meant is when I use the same --prefix for a couple of
> related packages. Suppose my goal is to install 'guile'. I learn that it needs
> 'gmp', so I download and install that, with the same expected prefix. Then I
> install guile with the same prefix and it refuses to acknowledge the previously
> installed packages, that it frustrating.

OK, so you _do_ think that $prefix/lib should be added to LDFLAGS by
default, and $prefix/include to CPPFLAGS.  Is there an autoconf macro
to do that?  If there is, does it support a ./configure option for
disabling it, for any cases where it isn't wanted?

Overall, then...

1. I think I agree now that using the gnulib macros to provide
--with-xxx-prefix options is a good idea.  I'm nervous about adding a
build dependency on so much new code (e.g. AC_LIB_LINKFLAGS_BODY is
nearly 500 lines long), but I suppose that must be the right thing to
do for the GNU project as a whole, and that if there are problems
we'll just work through them.

2. There is a lot of subtlety involved in understanding how these
options work, and what can be relied on on different platforms, and
explaining that to users; and none of that is Guile-specific.  So
rather than explaining bits of this in Guile's FAQ/web pages/whatever
it would be better to point to a reliable place where it's explained
more generally.  Is there already such a place?  If not, any
suggestions for where it should be?  I'm happy to draft an initial
text, if nothing already exists.

(Ludovic, do you have a view on all this?  Any one else?)

Many thanks again for your input on this stuff.

Regards,
        Neil