Another load path idea

Another load path idea

[Neil Jerram]

There has not been a resounding consensus for any of the proposals
discussed so far, so here is another possibility.

The basic idea behind all of the previous proposals was to create some
kind of config when a package was installed, such that other Guile
code would be able to find that package automatically.  But what if we
do things the other way round?  When a package that has dependencies
on other Guile modules is installed, we do some work at install time
to locate the dependencies that the package needs, and save the
results in a config file under the name of the _using_ package.  Then
when that package is run, it sets up the environment that it needs by
reading the saved config file.

I have no idea how to do this yet, but is it a good idea in principle?

Regards,
        Neil



_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-user

Re: Another load path idea

[Ludovic Courtès]

Hi,

Neil Jerram <neil@ossau.uklinux.net> writes:

> The basic idea behind all of the previous proposals was to create some
> kind of config when a package was installed, such that other Guile
> code would be able to find that package automatically.  But what if we
> do things the other way round?  When a package that has dependencies
> on other Guile modules is installed, we do some work at install time
> to locate the dependencies that the package needs, and save the
> results in a config file under the name of the _using_ package.  Then
> when that package is run, it sets up the environment that it needs by
> reading the saved config file.
>
> I have no idea how to do this yet, but is it a good idea in principle?

Yes, but I think that's (almost) a different issue.  I think it solves
the issue of finding the right dependency, not that of finding the right
module.  IOW, it's similar to Libtool's `.la' files (which include
information about a library's dependencies), not to /etc/ld.so.conf .

Speaking of that, the analogy of `ld.so.conf' would be something along
the lines of what you originally proposed[0]: storing the initial value
of `%load-path' somewhere in a file (or bunch of file -- the `init.d'
approach[1]), instead of having it hard-wired.

The `init.d' approach was ruled out because of the increased startup
time[2].  However, the single file approach had been considered
unpractical because it is harder to add/remove directories from there,
and to keep track of which packages exactly relied on it[1].

So, what if we just went ahead with the `init.d' approach, except that
each file would only contain a single Scheme string?  This constraint is
critical to address the startup time concern.

Or what about a single init file (again without any code, only data),
somewhat enhanced to keep track of which package rely on a each
particular load path:

  ((guile-gnome . "/opt/guile-gnome/")
   (guile-chbouib . "/usr/local/share/guile-chbouib")
   ... )

I'm sorry for getting back to this kind of proposal, but I think I'm
getting confused.  Since the discussion spanned over several months,
summarizing the key arguments seems important at this point.

Thanks,
Ludovic.

[0] http://lists.gnu.org/archive/html/guile-user/2005-10/msg00036.html
[1] http://lists.gnu.org/archive/html/guile-user/2005-10/msg00109.html
[2] http://lists.gnu.org/archive/html/guile-user/2005-12/msg00000.html


_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-user

Re: Another load path idea

[Neil Jerram]

ludovic.courtes@laas.fr (Ludovic Courtès) writes:

> Hi,

Hi, and thanks for your input.

> Yes, but I think that's (almost) a different issue.  I think it solves
> the issue of finding the right dependency, not that of finding the right
> module.  IOW, it's similar to Libtool's `.la' files (which include
> information about a library's dependencies), not to /etc/ld.so.conf .

Well, in the big picture of Guile package development, both of these
concepts could be useful.  If I'm understanding you correctly, I would
make the analogies that

- a .la file is a bit like a formal list of all a package's
  dependencies (which in practice might have to be declared by the
  package author, or could be detected automatically by code analysis)

- ld.so.conf is the environment information needed to be able to load
  all those dependencies, which is distilled from the .la file.

I'm imagining that the distillation process may take some time - in
the worst case, for example, it could require searching the whole file
system - and so it makes sense to do it once at package installation
time and cache the ld.so.conf-like results.  I'm currently thinking of
those results as just %load-path components, but they could (and
probably should) be extended in future to cover LD_LIBRARY_PATH and/or
the LTDL search path, for loading .so's.

Overall, therefore, I'd say this idea is more about the ld.so.conf
than about the .la files.  Perhaps with this expanded explanation
you'd now agree?

> Speaking of that, the analogy of `ld.so.conf' would be something along
> the lines of what you originally proposed[0]: storing the initial value
> of `%load-path' somewhere in a file (or bunch of file -- the `init.d'
> approach[1]), instead of having it hard-wired.

Yes, exactly.  The difference is which packages the information is
associated with.

Suppose you have an application foo which uses guile-gnome and
guile-pg.  guile-gnome is installed under /opt/gnome; foo and guile-pg
are installed under /usr/local; guile is in /usr.

Under previous proposals, there would have been config (under /etc)
associating:

  guile-gnome ... /opt/gnome/share/guile
  guile-pg ...... /usr/local/share/guile

and nothing for "foo" because it is an application, not a library.

Under this proposal, the config would be:

  foo ........... /opt/gnome/share/guile, /usr/local/share/guile

> The `init.d' approach was ruled out because of the increased startup
> time[2].  However, the single file approach had been considered
> unpractical because it is harder to add/remove directories from there,
> and to keep track of which packages exactly relied on it[1].

Those are both true, but in my mind (at least) there was one more
factor, namely that it is not neat for every Guile application to
start up with a load path that covers all installed Guile packages.
It feels neater to me if each Guile application runs with exactly the
environment that it needs.

I think this factor is the stumbling block.  As I type this, however,
I can also think of some objections to it:

1. A minimal load path is not really a minimal environment, because
   there can be other Guile packages installed under that load path's
   components which the application does not use.

2. The idea of a minimal environment is somewhat contrary to the idea
   of experimentation, and users being able to combine available
   modules in ways that the application developer might not have
   thought of.

3. There's one particular application, called "guile" (used
   interactively), where we really would like all installed packages
   to be available, even though they are not dependencies. :-)

4. Running with a non-minimal load path probably doesn't incur any
   significant runtime overhead.

Therefore, if no one else shares this neatness concern, I'm happy to
drop it... which then opens the door for your following suggestions.

> So, what if we just went ahead with the `init.d' approach, except that
> each file would only contain a single Scheme string?  This constraint is
> critical to address the startup time concern.
>
> Or what about a single init file (again without any code, only data),
> somewhat enhanced to keep track of which package rely on a each
> particular load path:
>
>   ((guile-gnome . "/opt/guile-gnome/")
>    (guile-chbouib . "/usr/local/share/guile-chbouib")
>    ... )

I think this would be my preference.

> I'm sorry for getting back to this kind of proposal, but I think I'm
> getting confused.  Since the discussion spanned over several months,
> summarizing the key arguments seems important at this point.

No need to apologize; you're quite right that summarizing is useful.

I'd still like to hear a little more feedback about my latest idea
above, in case someone else can see compelling pros or cons in it that
I haven't identified yet.  But overall it looks like we are almost
settled on the single init file solution.

Regards,
        Neil



_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-user

Re: Another load path idea

[Ludovic Courtès]

Hi,

Neil Jerram <neil@ossau.uklinux.net> writes:

> Well, in the big picture of Guile package development, both of these
> concepts could be useful.  If I'm understanding you correctly, I would
> make the analogies that
>
> - a .la file is a bit like a formal list of all a package's
>   dependencies (which in practice might have to be declared by the
>   package author, or could be detected automatically by code analysis)
>
> - ld.so.conf is the environment information needed to be able to load
>   all those dependencies, which is distilled from the .la file.

Right.

> I'm imagining that the distillation process may take some time - in
> the worst case, for example, it could require searching the whole file
> system - and so it makes sense to do it once at package installation
> time and cache the ld.so.conf-like results.  I'm currently thinking of
> those results as just %load-path components, but they could (and
> probably should) be extended in future to cover LD_LIBRARY_PATH and/or
> the LTDL search path, for loading .so's.
>
> Overall, therefore, I'd say this idea is more about the ld.so.conf
> than about the .la files.  Perhaps with this expanded explanation
> you'd now agree?

I don't see `.la' files as a cache of `ld.so.conf'.  Libtool's `.la'
files are only used at compile-time, in order to find out library
dependencies, while `ld.so.conf' is used to locate dynamic libraries at
run-time, in a way similar to `%load-path'.

A dependency cache in Guile would make it possible to bypass
`%load-path'.  I.e., instead of `(load-from-path "some-file.scm")',
which needs to go through `%load-path', it would allow the right file to
be directly loaded as in `(load "/some/path/some-file.scm")'.  IOW, this
would be an /optimization/.

Personally, I don't think this optimization is worth it (more below).

> Those are both true, but in my mind (at least) there was one more
> factor, namely that it is not neat for every Guile application to
> start up with a load path that covers all installed Guile packages.
> It feels neater to me if each Guile application runs with exactly the
> environment that it needs.

You gave several good arguments against try to minimize `%load-path'.

In fact, I think that the optimization that consists of
minimizing/bypassing `%load-path' is not very valuable:

1. Practically, it seems that traversing even tens of directories to
   locate a file is cheap compared to actually evaluating code, and, for
   instance, marking and sweeping;  ;-)

2. Just like `$PATH', `$LD_LIBRARY_PATH' and the likes, `%load-path'
   should is not supposed to contain a lot of directories; currently, it
   contains 4 items by default, and I guess people will rarely have more
   than 10 items in it.

   For the record, in Debian, most (if not all) Python packages get
   installed in `/usr/lib/pythonX.X' and
   `/usr/lib/pythonX.X/site-packages'.  The same goes for Perl, etc.


As a conclusion, I'd still be in favor of a single file like this:

>> Or what about a single init file (again without any code, only data),
>> somewhat enhanced to keep track of which package rely on a each
>> particular load path:
>>
>>   ((guile-gnome . "/opt/guile-gnome/")
>>    (guile-chbouib . "/usr/local/share/guile-chbouib")
>>    ... )

At startup-time, Guile would simply:

  (set! %load-path
        (append (map cdr (with-input-from-file "config.scm" read))
                %load-path))

Actually, it's likely that several packages will rely on the same load
path, the format should rather be:

    (((guile-gnome) . "/opt/guile-gnome")
     ((guile-chbouib guile-foo guile-bar) . "/usr/local/share/guile")
     ... )

Then, we need a `guile-update-load-path' script that does the right
thing with this file at (un)installation-time.

Thanks,
Ludovic.


_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-user

Re: Another load path idea

[Kevin Ryde]

ludovic.courtes@laas.fr (Ludovic Courtès) writes:
>
> Libtool's `.la' files are only used at compile-time, in order to
> find out library dependencies,

They're also read at runtime for dlopens (see ltdl.c try_dlopen) to
get depencency info (for the benefit of non-ELF systems I think).


_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-user

Re: Another load path idea

[Neil Jerram]

ludovic.courtes@laas.fr (Ludovic Courtès) writes:

> Hi,
>
> Neil Jerram <neil@ossau.uklinux.net> writes:
>
>> Well, in the big picture of Guile package development, both of these
>> concepts could be useful.  If I'm understanding you correctly, I would
>> make the analogies that
>>
>> - a .la file is a bit like a formal list of all a package's
>>   dependencies (which in practice might have to be declared by the
>>   package author, or could be detected automatically by code analysis)
>>
>> - ld.so.conf is the environment information needed to be able to load
>>   all those dependencies, which is distilled from the .la file.

[...]

> I don't see `.la' files as a cache of `ld.so.conf'.

Nor do I.  It's the other way round.  (If anything.  Let's not get
hung up on the analogy, because it's not a very good one anyway.)

> You gave several good arguments against try to minimize `%load-path'.
>
> In fact, I think that the optimization that consists of
> minimizing/bypassing `%load-path' is not very valuable:
>
> 1. Practically, it seems that traversing even tens of directories to
>    locate a file is cheap compared to actually evaluating code, and, for
>    instance, marking and sweeping;  ;-)
>
> 2. Just like `$PATH', `$LD_LIBRARY_PATH' and the likes, `%load-path'
>    should is not supposed to contain a lot of directories; currently, it
>    contains 4 items by default, and I guess people will rarely have more
>    than 10 items in it.

Yes, agreed.

> As a conclusion, I'd still be in favor of a single file like this:
>
> [...]

Basically agreed, but what I now plan precisely is as follows.

1. /etc/guile/${GUILE_EFFECTIVE_VERSION}/load-path.scm

Contains the calculated load path, e.g.

("/usr/share/guile/1.6"
 "/usr/share/guile/site"
 "/usr/share/guile"
 "/opt/gnome/share/guile") 

2. Each Guile package installs a file under /etc/guile which contains
   its load path as a single string.

3. guile-config is enhanced so that "guile-config calculate-load-path"
   will recalculate load-path.scm from the installed package files.

Reasons for this arrangement:

1. I think it's nice to make what Guile does at startup as simple as
   possible (even if it's not significant in performance terms).

2. Some distros/installers (such as that on the Nokia 770, to pick an
   example out of the blue :-)) don't support the automatic execution
   of a post-install script.  The above arrangement makes things as
   easy as possible for such platforms: they just need to run
   guile-config manually after the install.

3. Advanced distros/installers may support coalescing the post-install
   steps for multiple packages, which in this case would work very
   nicely.

Regards,
        Neil



_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-user

Re: Another load path idea

[Andy Wingo]

Hi Neil,

On Sat, 2006-01-21 at 09:01 +0000, Neil Jerram wrote:
> Basically agreed, but what I now plan precisely is as follows.

Oh good, a summary of this thread of many moons :)

A question though. What is the problem which is being solved here?

In guile-gnome I only really use the load path in one place. That is
that (use-modules (gnome-0)) adds the path for version 0 of the
guile-gnome API to the load path. Before that you cannot import (gnome
gobject), for example, because it's not in the path. Afterwards you can.

This was done this way to allow me to break API in the future, but leave
existing programs still working.

In that sense a global path that adds gnome isn't terribly interesting,
because you have to select the API version in the first place.

> 1. /etc/guile/${GUILE_EFFECTIVE_VERSION}/load-path.scm

> Contains the calculated load path, e.g.
> 
> ("/usr/share/guile/1.6"
>  "/usr/share/guile/site"
>  "/usr/share/guile"
>  "/opt/gnome/share/guile") 

As generated by guile-config ... ?

> 2. Each Guile package installs a file under /etc/guile which contains
>    its load path as a single string.

What about packages that depend on each other? Is that out of the scope
of this proposal?

I assume you mean /etc/guile/$effective_version

I think requiring users to run a command to fix the cache is an
invitation for bugs. I think it would be sufficient to have a cache in
~/.guile.d/cache, that would effectively hold the output of running
guile-config. That way the normal case is that one only stats the cache,
reads the system path dir, statting entries there, and then if
everything is up to date just load the cache file. That's one readdir,
one file read, and about 5 stats. Not expensive at all.

Perhaps I'm just burned by the fiasco that was gstreamer's gst-register.
Now having gotten rid of it, making the "registry" user-local and with
strict cache semantics, we get no bugs about it.

Sorry to bring up issues this late in the game.

Regards,
-- 
Andy Wingo
http://wingolog.org/


_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-user

Re: Another load path idea

[Ludovic Courtès]

Hi,

Neil Jerram <neil@ossau.uklinux.net> writes:

> 1. /etc/guile/${GUILE_EFFECTIVE_VERSION}/load-path.scm
>
> Contains the calculated load path, e.g.
>
> ("/usr/share/guile/1.6"
>  "/usr/share/guile/site"
>  "/usr/share/guile"
>  "/opt/gnome/share/guile") 
>
> 2. Each Guile package installs a file under /etc/guile which contains
>    its load path as a single string.
>
> 3. guile-config is enhanced so that "guile-config calculate-load-path"
>    will recalculate load-path.scm from the installed package files.

That sounds good to me!

Thanks,
Ludovic.


_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-user

Re: Another load path idea

[Neil Jerram]

ludovic.courtes@laas.fr (Ludovic Courtès) writes:

> Hi,
>
> Neil Jerram <neil@ossau.uklinux.net> writes:
>
>> 1. /etc/guile/${GUILE_EFFECTIVE_VERSION}/load-path.scm
>>
>> Contains the calculated load path, e.g.
>>
>> ("/usr/share/guile/1.6"
>>  "/usr/share/guile/site"
>>  "/usr/share/guile"
>>  "/opt/gnome/share/guile") 
>>
>> 2. Each Guile package installs a file under /etc/guile which contains
>>    its load path as a single string.
>>
>> 3. guile-config is enhanced so that "guile-config calculate-load-path"
>>    will recalculate load-path.scm from the installed package files.
>
> That sounds good to me!

Great; let's do it!  (I'll work on this after enhanced catch.)

    Neil



_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-user

Re: Another load path idea

[Kevin Ryde]

Andy Wingo <wingo@pobox.com> writes:
>
> A question though. What is the problem which is being solved here?

That's escaped me too.

(I have a feeling the present "site" dir is modelled on emacs, I
haven't been able to follow why it's not enough ...)


_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-user

Re: Another load path idea

[Ludovic Courtès]

Hi,

Kevin Ryde <user42@zip.com.au> writes:

> Andy Wingo <wingo@pobox.com> writes:
>>
>> A question though. What is the problem which is being solved here?
>
> That's escaped me too.
>
> (I have a feeling the present "site" dir is modelled on emacs, I
> haven't been able to follow why it's not enough ...)

Roughly, the point is that some packages may want to install elsewhere,
not in `site'.  Therefore, the _default_ value of `%load-path' needs to
be configurable.  Typically, a package that installs to `/opt/foo' may
want to tell Guile to add this directory to its default `%load-path'.

Then, for a summary of the important stages of this everlasting
discussion, you may want to look at:
http://lists.gnu.org/archive/html/guile-user/2006-01/msg00064.html .
;-)

Thanks,
Ludovic.


_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-user

Re: Another load path idea

[Neil Jerram]

Andy Wingo <wingo@pobox.com> writes:

> Hi Neil,
>
> On Sat, 2006-01-21 at 09:01 +0000, Neil Jerram wrote:
>> Basically agreed, but what I now plan precisely is as follows.
>
> Oh good, a summary of this thread of many moons :)

I'm sorry it's gone on for so long.  Every time we appear to near a
consensus, some new issue pops up.

> A question though. What is the problem which is being solved here?

The basic scenario is this: someone has Guile installed (probably by
their distro) in /usr, and then builds and installs an additional
package using ./configure && make && sudo make install, which installs
with a different prefix than /usr (usually /usr/local).  Then they try
to do a (use-modules (newly-installed-package)), and it doesn't work,
because the installed Guile's load path doesn't include anything
beginning with /usr/local/share/guile.

So the question is: can we define a mechanism so that the use-modules
would just work?

Some time back, I thought that the right thing to do might be for the
package to discover the installed Guile's site directory and install
its modules there.  But there was a strong consensus that that was
wrong, and that when a package is configured with a given prefix, it
must install all its files under that prefix (except for config files
which go under the configured sysconfdir).

> In guile-gnome I only really use the load path in one place. That is
> that (use-modules (gnome-0)) adds the path for version 0 of the
> guile-gnome API to the load path. Before that you cannot import (gnome
> gobject), for example, because it's not in the path. Afterwards you
> can.

Then the question in your case would be how do you get the initial
(use-modules (gnome-0)) to work?

> This was done this way to allow me to break API in the future, but leave
> existing programs still working.
>
> In that sense a global path that adds gnome isn't terribly interesting,
> because you have to select the API version in the first place.

The way I see it, the global path would be the one that allowed
(use-modules (gnome-0)) or (use-modules (gnome-1)) to work.  It's fine
if (gnome-0) or (gnome-1) then extends the load path further.

>> 2. Each Guile package installs a file under /etc/guile which contains
>>    its load path as a single string.
>
> What about packages that depend on each other? Is that out of the scope
> of this proposal?

No; the idea is that each installed package does whatever is needed to
allow Guile (or any other Guile package) to bootstrap its modules.

> I assume you mean /etc/guile/$effective_version

Actually I meant to be vague, hence "under" :-).  My guess is that
many Guile packages are version-independent, so could install their
paths under /etc/guile/common.  Packages that know they are
version-dependent could install under one (or more)
$effective_version's.  I'm not really sure on the details here yet.

> I think requiring users to run a command to fix the cache is an
> invitation for bugs. I think it would be sufficient to have a cache in
> ~/.guile.d/cache, that would effectively hold the output of running
> guile-config. That way the normal case is that one only stats the cache,
> reads the system path dir, statting entries there, and then if
> everything is up to date just load the cache file. That's one readdir,
> one file read, and about 5 stats. Not expensive at all.

Sorry, I understand your overall idea about the cache, but I don't get
the details; can you be more precise about the reads and stats?

> Perhaps I'm just burned by the fiasco that was gstreamer's gst-register.
> Now having gotten rid of it, making the "registry" user-local and with
> strict cache semantics, we get no bugs about it.

That sounds like useful experience.  Can I go somewhere to read more
about it?

> Sorry to bring up issues this late in the game.

No problem; thanks for your input.

Regards,
     Neil



_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-user

Re: Another load path idea

[Kevin Ryde]

Neil Jerram <neil@ossau.uklinux.net> writes:
>
> The basic scenario is this: someone has Guile installed (probably by
> their distro) in /usr, and then builds and installs an additional
> package using ./configure && make && sudo make install, which installs
> with a different prefix than /usr (usually /usr/local).  Then they try
> to do a (use-modules (newly-installed-package)), and it doesn't work,
> because the installed Guile's load path doesn't include anything
> beginning with /usr/local/share/guile.

/usr/local should probably be in the defaults.  Of course there's
nothing to stop a distro packaged guile from doing that right now, if
/usr/local is the preferred location for non-distro stuff.

> No; the idea is that each installed package does whatever is needed to
> allow Guile (or any other Guile package) to bootstrap its modules.

I think that's the wrong way around, that it should be a job for the
sysadmin.

Basically, if someone installs in an unusual location then they're
doing something unusual; and consequently will need to tell some,
maybe all, of their installed guiles to look there.  Perhaps for all
users, perhaps just for themself, etc.

I reckon there's much more benefit in getting the guile recommended
locations better described, some sample automakery or whatever,
ie. better to define and assist normal setups, than to try to make
arbitrary arrangements work.  I doubt anybody will want completely
arbitrary anyway, surely there's only a handful of different cases.

I think it will be enough to,

1. Add /usr/local into the default %load-path.

2. Put a note in the manual inviting package builders to augment
   %load-path further if they wish, eg. for /opt.  (By patching
   boot-9.scm I would think.)

3. Put a note in the manual encouraging the use of /site by sysadmins,
   but with an invitation to extend %load-path if they've got good
   reason for violating the usual setup.

And a bit later (but actually needs doing either way),

4. Describe better in the manual how a .scm module should hit its own
   installed C code modules using load-extension.


_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-user

Re: Another load path idea

[Ludovic Courtès]

Hi,

Kevin Ryde <user42@zip.com.au> writes:

> I think it will be enough to,
>
> 1. Add /usr/local into the default %load-path.
>
> 2. Put a note in the manual inviting package builders to augment
>    %load-path further if they wish, eg. for /opt.  (By patching
>    boot-9.scm I would think.)
>
> 3. Put a note in the manual encouraging the use of /site by sysadmins,
>    but with an invitation to extend %load-path if they've got good
>    reason for violating the usual setup.

As I understand it, the whole point of this discussion is about
providing mechanisms to allow the initial value of `%load-path' to be
easily and cleanly customized, i.e., not by "patching boot-9.scm".  ;-)

Then there is another question which is: _who_ performs this
customization and _when_?  Neil (and I) initially suggested that this
could be done automatically by packages themselves at installation-time.
You suggest, OTOH, that this should be done manually by the sysadmin,
after they installed a package in some "unusual" path.

In either case, we still need (i) a config file holding the default
value of `%load-path', (ii) a script to manipulate that file (i.e., to
add/remove directories from it).

Thanks,
Ludovic.


_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-user

Re: Another load path idea

[Kevin Ryde]

ludovic.courtes@laas.fr (Ludovic Courtès) writes:
>
> (i) a config file holding the default value of `%load-path',

If you really want to add something, just load an /etc/guilerc at
startup.  That'd be all the flexibility and wouldn't force add-on
modules to jump through hoops (not until the hoops are much clearer in
everybody's minds).

> (ii) a script to manipulate that file (i.e., to add/remove
> directories from it).

Leave that to later.  See if anyone is actually doing stuff so much
crazy stuff it needs automation.


_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-user

Re: Another load path idea

[Greg Troxel]

Kevin Ryde <user42@zip.com.au> writes:

> Neil Jerram <neil@ossau.uklinux.net> writes:
> >
> > The basic scenario is this: someone has Guile installed (probably by
> > their distro) in /usr, and then builds and installs an additional
> > package using ./configure && make && sudo make install, which installs
> > with a different prefix than /usr (usually /usr/local).  Then they try
> > to do a (use-modules (newly-installed-package)), and it doesn't work,
> > because the installed Guile's load path doesn't include anything
> > beginning with /usr/local/share/guile.
> 
> /usr/local should probably be in the defaults.  Of course there's
> nothing to stop a distro packaged guile from doing that right now, if
> /usr/local is the preferred location for non-distro stuff.

I concur with Neil, but his example does not fully cover the
possibilities.  It's true that on a Linux system guile will be in /usr
and some random hand-built package with no --prefix will end up in
/usr/local.  Yes, adding /usr/local will fix this, but also perhaps
pick up unintended stuff.

On NetBSD (and other pkgsrc systems), guile will be ok /usr/pkg, and
hand-built programs may be in /usr/local, or someplace else.  I tend
to build program foo (from CVS) into /usr/foo.  So IMHO a general
mechanism is required.

> doing something unusual; and consequently will need to tell some,
> maybe all, of their installed guiles to look there.  Perhaps for all
> users, perhaps just for themself, etc.

Exactly, and that's the issue.

> I reckon there's much more benefit in getting the guile recommended
> locations better described, some sample automakery or whatever,
> ie. better to define and assist normal setups, than to try to make
> arbitrary arrangements work.  I doubt anybody will want completely
> arbitrary anyway, surely there's only a handful of different cases.

Yes, but there are two sane cases:

  * put stuff in the same prefix as guile, following standards for
    $(prefix)/share/guile/site etc.

  * put stuff in some other prefix, still following standards,
    e.g. $(otherprefix)/share/guile/site

I have run into the second case a lot myself.

> I think it will be enough to,
> 
> 1. Add /usr/local into the default %load-path.

I'm not sure I like this.  NetBSD philosophy is not to include
/usr/local (include, lib) by default in system tools.  But it seems
reasonable.

> 2. Put a note in the manual inviting package builders to augment
>    %load-path further if they wish, eg. for /opt.  (By patching
>    boot-9.scm I would think.)

Sure, this makes sense, but doesn't address what I see as the dominant
practical issue of someone installing something with a new prefix.
But certainly if a particular system has a notion of two or more
prefixes that should generally be searched, then packagers for that
system should make guile do that.  NetBSD pkgsrc does this routinely
(to look in /usr/pkgsrc, not two places).

> 3. Put a note in the manual encouraging the use of /site by sysadmins,
>    but with an invitation to extend %load-path if they've got good
>    reason for violating the usual setup.

/site?

> And a bit later (but actually needs doing either way),
> 
> 4. Describe better in the manual how a .scm module should hit its own
>    installed C code modules using load-extension.

Yes - in particular, I think a scm module should cons up an absolute
path for loading binaries based on it's own $prefix at configure
time.  I've done this with foo.scm.in and autoconf.


An important point is that some people (including me) believe that
only files managed by a pkg system belong in the prefix assigned to
the system.  I think it's been accepted that this view is reasonable
and that guile should accomodate those who run their systems this way
(as well as those who put things in the same prefix).

-- 
        Greg Troxel <gdt@ir.bbn.com>


_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-user

Re: Another load path idea

[Ludovic Courtès]

Hi,

Kevin Ryde <user42@zip.com.au> writes:

> ludovic.courtes@laas.fr (Ludovic Courtès) writes:
>>
>> (i) a config file holding the default value of `%load-path',
>
> If you really want to add something, just load an /etc/guilerc at
> startup.

Sorry, I don't really understand how what you says differs from what's
written above.

>> (ii) a script to manipulate that file (i.e., to add/remove
>> directories from it).
>
> Leave that to later.  See if anyone is actually doing stuff so much
> crazy stuff it needs automation.

Really, manipulating a list or an alist, reading and writing to/from a
file is something achievable I think.  ;-)

Thanks,
Ludovic.


_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-user

Re: Another load path idea

[Kevin Ryde]

Greg Troxel <gdt@ir.bbn.com> writes:
>
> I tend to build program foo (from CVS) into /usr/foo.

I like that too except usually a subdir of wherever I keep the source.
Nice and easy to rm -r when you're sick of it.  But I presume you, as
I, don't want a package build trying to modify /etc/profile etc to
hook itself into $PATH.  I might well add it in myself, even setup to
test a range of locations and use what exists, I like that in my
.emacs a lot too for instance.  Anyway, the main point would be that I
don't think it's the package's business to try to join itself to the
world when installed in an out-of-the-way place.


_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-user

Re: Another load path idea

[Greg Troxel]

  I like that too except usually a subdir of wherever I keep the source.
  Nice and easy to rm -r when you're sick of it.  But I presume you, as
  I, don't want a package build trying to modify /etc/profile etc to
  hook itself into $PATH.  I might well add it in myself, even setup to
  test a range of locations and use what exists, I like that in my
  .emacs a lot too for instance.  Anyway, the main point would be that I
  don't think it's the package's business to try to join itself to the
  world when installed in an out-of-the-way place.

No, but it shoudl be easy for someone to add.  so I'd like to be able
to do something like:
 
# guile-admin --add-prefix /usr/foo

# guile-admin --remove-prefix /usr/foo

# guile-admin --show-prefixes

-- 
        Greg Troxel <gdt@ir.bbn.com>


_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-user

Re: Another load path idea

[Neil Jerram]

ludovic.courtes@laas.fr (Ludovic Courtès) writes:

> Hi,
>
> Kevin Ryde <user42@zip.com.au> writes:
>
>> ludovic.courtes@laas.fr (Ludovic Courtès) writes:
>>>
>>> (i) a config file holding the default value of `%load-path',
>>
>> If you really want to add something, just load an /etc/guilerc at
>> startup.
>
> Sorry, I don't really understand how what you says differs from what's
> written above.

OK, let's proceed with the minimal first step of supporting an
optional config file,
$sysconfdir/guile/${EFFECTIVE-VERSION}/load-path.scm, which just
contains the load path (as a list of strings).

If this file doesn't exist, behaviour is unchanged from what it is
now.  If it does exist, it overrides the built-in load path.

That will allow sysadmins to set whatever load path they want, and
whoever wants to to play with coding more automatic mechanisms.

>>> (ii) a script to manipulate that file (i.e., to add/remove
>>> directories from it).
>>
>> Leave that to later.  See if anyone is actually doing stuff so much
>> crazy stuff it needs automation.
>
> Really, manipulating a list or an alist, reading and writing to/from a
> file is something achievable I think.  ;-)

True, but we don't lose anything by taking one step at a time.

      Neil



_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-user