Where to install files?

Where to install files?

[Neil Jerram]

Can someone tell me the Makefile.am incantations for determining where
a package should install Scheme and Elisp files?

Also, does it follow that if a package installs nothing other than
Scheme and Elisp files, its own $prefix is meaningless?

Thanks,
        Neil



_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-user

Re: Where to install files?

[Kevin Ryde]

Neil Jerram <neil@ossau.uklinux.net> writes:
>
> Can someone tell me the Makefile.am incantations for determining where
> a package should install Scheme and Elisp files?

For the lisp bits, AM_PATH_LISPDIR in configure.ac then

	dist_lisp_LISP = foo.el

or if you don't want it byte-compiled,

	dist_lisp_DATA = foo-autoloads.el foo-devel.el


_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-user

Re: Where to install files?

[Neil Jerram]

Kevin Ryde <user42@zip.com.au> writes:

> Neil Jerram <neil@ossau.uklinux.net> writes:
>>
>> Can someone tell me the Makefile.am incantations for determining where
>> a package should install Scheme and Elisp files?
>
> For the lisp bits, AM_PATH_LISPDIR in configure.ac then
>
> 	dist_lisp_LISP = foo.el
>
> or if you don't want it byte-compiled,
>
> 	dist_lisp_DATA = foo-autoloads.el foo-devel.el

Thanks.  I realized after posting that my package already does this,
but I'd still like to probe a bit further about whether it is right,
and hence if something similar should apply to Scheme files.

With Emacs under Debian, for example, use of AM_PATH_LISPDIR means
that stuff installed by "./configure; make; make install" goes under
/usr/share/emacs (and so not somewhere like /usr/local/share/emacs).

Is that appropriate?  Do distributions like Debian have policy on
where files should go in such cases?  (i.e. files from packages for a
distribution-managed program, but where the package is not itself
distribution-managed.)

I guess you could say that AM_PATH_LISPDIR defines the policy, and
that if Debian (or other distribution) wanted such files to go under
/usr/local, they could have arranged for AM_PATH_LISPDIR to return
/usr/local/share/emacs.  (Could they in fact do this?)

If the last para is correct, and similar arguments apply to Scheme
files, I think the implications are that

- Guile's configure should allow separate specification of

  (1) overall installation prefix (e.g. /usr)

  (2) default install location for distribution-managed packages
      (e.g. /usr/share/guile, default=(1)/share/guile)

  (3) default install location for non-distribution-managed packages
      (e.g. /usr/local/share/guile, default=(2))

  (4) a location for site-local files
      (e.g. /usr/local/share/guile/site, default=(3)/site)

  (5) location of the system init file (e.g. /etc/guile/1.6/init.scm,
      default=init.scm)

- At runtime, Guile would initialize its load path using (1), (2), (3)
  and (4), and read the init file from (5).

- When a package's configure is run, guile.m4 should be able to
  provide either (2) or (3) as the value of a Makefile.am variable
  GUILE_SCHEME_DIR:

  - GUILE_SCHEME_DIR would give (2) if a package's ./configure was run
    with a --with-dist-managed-install-dir option  (as a convenience
    for distribution packagers)

  - GUILE_SCHEME_DIR would give (3) if a package's ./configure was run
    without this option  (the normal case for people building tarballs)

Guile's guile.m4 currently provides GUILE_SITE_DIR, but that feels
wrong to me both because it doesn't handle the (2)/(3) distinction
above, and because of the /site ending - I think of site as being for
code put there by the local sysadmin, not for code from packages at
all.

Thoughts?  (People commenting on this may also want to review the
thread from last year that began at
http://lists.gnu.org/archive/html/guile-devel/2004-11/msg00016.html)

      Neil



_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-user

Re: Where to install files?

[Greg Troxel]

I concur with most of your comments, but would like to make a
metapoint: various OS/distributions/packaging systems have rules about
what goes where, and guile should be such that it can be made to
behave 'right' for these varying definitions of right.  I think this
is implicit in your comments, but wanted to raise it explicitly.

All that said, most packaging systems (pkgsrc, FreeBSD ports, typical
Linux) put all programs under the same prefix, so using "guile's
prefix" vs "our prefix" amounts to the same thing.  So it's only
non-distribution-managed that's an issue.

My own view is that a package configured with --prefix=/usr/foo should
install files only under /usr/foo.  This leads me with guile to want
to extend load-path to include guile directories in other prefixes.

  (5) location of the system init file (e.g. /etc/guile/1.6/init.scm,
      default=init.scm)

Using --sysconfdir to specify this is/would be nice, and would make it
easy to hook in a new prefix.

  Guile's guile.m4 currently provides GUILE_SITE_DIR, but that feels
  wrong to me both because it doesn't handle the (2)/(3) distinction
  above, and because of the /site ending - I think of site as being for
  code put there by the local sysadmin, not for code from packages at
  all.

I agree, but really we need three levels:
  pkgsrc [wrong word, but distribution-managed]
  site   [managed for a group of machines]
  local  [just this machine]


With respect to /usr/local, I believe that guile should by default
only look in its own prefix.  Whether /usr/local should be searched
e.g by programs in /usr is a distribution-specific decision.  It may
be worthwhile to have a configure argument to add prefixes whose guile
directories should be searched, so that

  ./configure --prefix=/usr/pkg --searchprefix=/usr/bar --searchprefix=/usr/baz

would result in a guile installed in /usr/pkg that not only looks in
/usr/pkg/share/guile/1.6 but also /usr/bar/share/guile/1.6.

It might also be nice to have a run-time method to 

  guile-configer --addsearchprefix=/usr/bar

so that someone building another package to /usr/bar can invoke this
on the system guile to hook in the new path.

-- 
        Greg Troxel <gdt@ir.bbn.com>


_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-user

Re: Where to install files?

[Neil Jerram]

Greg Troxel <gdt@ir.bbn.com> writes:

> I concur with most of your comments, but would like to make a
> metapoint: various OS/distributions/packaging systems have rules about
> what goes where, and guile should be such that it can be made to
> behave 'right' for these varying definitions of right.  I think this
> is implicit in your comments, but wanted to raise it explicitly.

Yes, thanks.  This is exactly what was in my mind too, but I didn't
make that clear.

Basically, when I'm using a distribution (Debian in my case), I like
to do things as much as possible in the way that that distribution
designed.  And I would hope that distributions are broad-minded (or
control-freak!) enough to have a policy on where
non-distribution-managed stuff should go.

> All that said, most packaging systems (pkgsrc, FreeBSD ports, typical
> Linux) put all programs under the same prefix, so using "guile's
> prefix" vs "our prefix" amounts to the same thing.  So it's only
> non-distribution-managed that's an issue.

Yes, exactly.  And the non-distribution-managed part is foremost in my
mind right now, because the load-path issue just came up with someone
installing my guile-debugging package, which is so far only available
as a tarball.

> My own view is that a package configured with --prefix=/usr/foo should
> install files only under /usr/foo.  This leads me with guile to want
> to extend load-path to include guile directories in other prefixes.

On this point I think I disagree with you.  I think there is value in
forcing all packages to be loaded - or at least bootstrapped - from a
known load-path that is defined by the Guile installation.

Note that this doesn't _force_ all Scheme files to go under one of the
load-path locations.  A big application like Gnucash could put a
bootstrap file like this in one of the standard locations ...

(define-module (gnucash))

(set! %load-path (cons "/opt/gnucash" %load-path))

(use-modules (gnucash account)
             (gnucash ui)
             ...)

... and then all its other Scheme files under /opt/gnucash.

IMO this is good because everything is explicit and there is a nice
trail for the investigative free software developer to follow when
they want to understand how stuff works.

>   (5) location of the system init file (e.g. /etc/guile/1.6/init.scm,
>       default=init.scm)
>
> Using --sysconfdir to specify this is/would be nice, and would make it
> easy to hook in a new prefix.

I'm sure you're right, but what is it that defines sysconfdir, and
where can I read about how it behaves?

>   Guile's guile.m4 currently provides GUILE_SITE_DIR, but that feels
>   wrong to me both because it doesn't handle the (2)/(3) distinction
>   above, and because of the /site ending - I think of site as being for
>   code put there by the local sysadmin, not for code from packages at
>   all.
>
> I agree, but really we need three levels:
>   pkgsrc [wrong word, but distribution-managed]
>   site   [managed for a group of machines]
>   local  [just this machine]

Are you sure you're not a plant in the audience?  Your levels idea
connects perfectly with a generalization of the scheme I described
yesterday, which occurred to me after posting:

- At Guile configuration time, we allow the builder to specify an
  arbitrary set of load path directories, each with a tag and a
  description, something (semantically speaking) like this:

  '(("/usr/share/guile/1.6"
     "1.6"
     "Install location for Guile 1.6's own Scheme files")

    ("/usr/share/guile/site"
     "site"
     "Install location for site-specific Scheme files")

    ("/opt/gnome/guile"
     "gnome"
     "Install location for GNOME-related Scheme files")

    ("/usr/local/share/guile"
     "local"
     "Version-independent non-distribution-managed Scheme files")

    ("/usr/local/share/guile/1.6"
     "local-1.6"
     "1.6-dependent non-distribution-managed Scheme files"))

- Guile would always then initialize its %load-path to the union of
  all these locations.

- guile.m4 would provide a --with-guile-scheme-dir=TAG option to
  ./configure, which would set GUILE_SCHEME_DIR to the location for
  TAG, and a macro

    GUILE_SCHEME(default-tag)

  which a package would put in its configure.in, with default-tag
  specifying the tag to use if --with-guile-scheme-dir is not
  specified.

  --with-guile-scheme-dir could also specify a directory explicitly.

Actually it may not be necessary to specify the location list above at
Guile configuration time; the list could perhaps go into init.scm, or
another init file (in sysconfdir) which is always read, even if Guile
is given a -no-init-file option.

> With respect to /usr/local, I believe that guile should by default
> only look in its own prefix.  Whether /usr/local should be searched
> e.g by programs in /usr is a distribution-specific decision.  It may
> be worthwhile to have a configure argument to add prefixes whose guile
> directories should be searched, so that
>
>   ./configure --prefix=/usr/pkg --searchprefix=/usr/bar --searchprefix=/usr/baz
>
> would result in a guile installed in /usr/pkg that not only looks in
> /usr/pkg/share/guile/1.6 but also /usr/bar/share/guile/1.6.

Yes; I think this is equivalent to what I've just described, though.

> It might also be nice to have a run-time method to 
>
>   guile-configer --addsearchprefix=/usr/bar
>
> so that someone building another package to /usr/bar can invoke this
> on the system guile to hook in the new path.

In my view the behaviour that this allows is sufficiently covered by
the combination of

- being able to edit init.scm (or other init file, per comment above)

- the bootstrapping approach described above.

So I don't think we need this as well, at least not in the core
mechanism.  (A distribution or sysadmin could choose to implement this
for their distribution/systems by organizing their init.scm
appropriately.)

Regards,
     Neil



_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-user

Re: Where to install files?

[Greg Troxel]

Neil Jerram <neil@ossau.uklinux.net> writes:

> Greg Troxel <gdt@ir.bbn.com> writes:
> 
> Yes, thanks.  This is exactly what was in my mind too, but I didn't
> make that clear.

Glad I was helpful - was feeling redundant but posted anyway.

> Basically, when I'm using a distribution (Debian in my case), I like
> to do things as much as possible in the way that that distribution
> designed.

Agreed (for me it's NetBSD and pkgsrc).

> And I would hope that distributions are broad-minded (or
> control-freak!) enough to have a policy on where
> non-distribution-managed stuff should go.

NetBSD's take (speaking for myself, not The NetBSD Foundation - I am a
NetBSD developer) is that 

  /usr proper belongs to the base OS, and nothing else belongs there

  /usr/pkg belongs to pkgsrc, and only files installed/managed by
  pkgsrc belong there

  /usr/local is for people to do whatever they want with.  It's not in
  default includes/lib search path.

> Yes, exactly.  And the non-distribution-managed part is foremost in my
> mind right now, because the load-path issue just came up with someone
> installing my guile-debugging package, which is so far only available
> as a tarball.

So specifically I think you mean

  guile installed under one prefix, probably by a packaging system

  some other guile-using package, e.g. guile-debugging, installed
  under some other prefix, but not using the packaging system.

If you configure guile-debugging with the same prefix as guile, these
issues become moot.  Thus my restriction to the different prefix case.

> > My own view is that a package configured with --prefix=/usr/foo should
> > install files only under /usr/foo.  This leads me with guile to want
> > to extend load-path to include guile directories in other prefixes.
> 
> On this point I think I disagree with you.  I think there is value in
> forcing all packages to be loaded - or at least bootstrapped - from a
> known load-path that is defined by the Guile installation.

To me it's not a question of value.  /usr/pkg belongs to pkgsrc, and
to first order other stuff (meaning unmanaged and unrecord stuff)
should not be there.

> Note that this doesn't _force_ all Scheme files to go under one of the
> load-path locations.  A big application like Gnucash could put a
> bootstrap file like this in one of the standard locations ...
> 
> (define-module (gnucash))
> 
> (set! %load-path (cons "/opt/gnucash" %load-path))
> 
> (use-modules (gnucash account)
>              (gnucash ui)
>              ...)
> 
> ... and then all its other Scheme files under /opt/gnucash.

Sure that makes sense, and another way is to have a standard, easy to
follow way to register additional load-path directories with the base
guile installation.

> IMO this is good because everything is explicit and there is a nice
> trail for the investigative free software developer to follow when
> they want to understand how stuff works.

good point.

> >   (5) location of the system init file (e.g. /etc/guile/1.6/init.scm,
> >       default=init.scm)
> >
> > Using --sysconfdir to specify this is/would be nice, and would make it
> > easy to hook in a new prefix.
> 
> I'm sure you're right, but what is it that defines sysconfdir, and
> where can I read about how it behaves?

It's an autoconf (configure script really) directive, or perhaps one
that's commonly added, meant to specify where config files go,
defaulting to $(prefix)/etc or $(prefix)/etc/PACKAGE, but commonly
used to force things to /etc/PACKAGE, especially on systems where
people want that.

> >   Guile's guile.m4 currently provides GUILE_SITE_DIR, but that feels
> >   wrong to me both because it doesn't handle the (2)/(3) distinction
> >   above, and because of the /site ending - I think of site as being for
> >   code put there by the local sysadmin, not for code from packages at
> >   all.
> >
> > I agree, but really we need three levels:
> >   pkgsrc [wrong word, but distribution-managed]
> >   site   [managed for a group of machines]
> >   local  [just this machine]
> 
> Are you sure you're not a plant in the audience?  Your levels idea
> connects perfectly with a generalization of the scheme I described
> yesterday, which occurred to me after posting:

This notion of package/site/local is really just the obvious
generalization of the old site/local split, once one gets over
FreeBSD's misusing /usr/local for their ports system.

> - At Guile configuration time, we allow the builder to specify an
>   arbitrary set of load path directories, each with a tag and a
>   description, something (semantically speaking) like this:
> 
>   '(("/usr/share/guile/1.6"
>      "1.6"
>      "Install location for Guile 1.6's own Scheme files")
> 
>     ("/usr/share/guile/site"
>      "site"
>      "Install location for site-specific Scheme files")
> 
>     ("/opt/gnome/guile"
>      "gnome"
>      "Install location for GNOME-related Scheme files")
> 
>     ("/usr/local/share/guile"
>      "local"
>      "Version-independent non-distribution-managed Scheme files")
> 
>     ("/usr/local/share/guile/1.6"
>      "local-1.6"
>      "1.6-dependent non-distribution-managed Scheme files"))

> - Guile would always then initialize its %load-path to the union of
>   all these locations.

I kind of like having several standard places within a prefix, as seen
on NetBSD now:

guile> %load-path
("/usr/pkg/share/guile/site" "/usr/pkg/share/guile/1.6"
 "/usr/pkg/share/guile" ".")

Or rather, if that continues, I'd like to specify extra prefixes, with
the load path being the cross product of within-prefix places and
prefixes.  But perhaps also single dirs.

> - guile.m4 would provide a --with-guile-scheme-dir=TAG option to
>   ./configure, which would set GUILE_SCHEME_DIR to the location for
>   TAG, and a macro
> 
>     GUILE_SCHEME(default-tag)
> 
>   which a package would put in its configure.in, with default-tag
>   specifying the tag to use if --with-guile-scheme-dir is not
>   specified.
> 
>   --with-guile-scheme-dir could also specify a directory explicitly.
> 
> Actually it may not be necessary to specify the location list above at
> Guile configuration time; the list could perhaps go into init.scm, or
> another init file (in sysconfdir) which is always read, even if Guile
> is given a -no-init-file option.

I don't follow that at all.

I'd like a load-path directory, with files whose names are irrelevant
but whose contets are dirs to add to the load path.  This is
pkg-friendly since they can be deleted and added automatically without
merge/editing issues, using the fs to represent set semantics.

> > It might also be nice to have a run-time method to 
> >
> >   guile-configer --addsearchprefix=/usr/bar
> >
> > so that someone building another package to /usr/bar can invoke this
> > on the system guile to hook in the new path.
> 
> In my view the behaviour that this allows is sufficiently covered by
> the combination of
> 
> - being able to edit init.scm (or other init file, per comment above)

sure, this is a way to avoid editing, so that a makefile can invoke the
'add prefix' method which would either be refcounted or idempotent.
All of this needs to be automation friendly.

-- 
        Greg Troxel <gdt@ir.bbn.com>


_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-user

Re: Where to install files?

[Ludovic Courtès]

Hi,

Neil Jerram <neil@ossau.uklinux.net> writes:

> - At Guile configuration time, we allow the builder to specify an
>   arbitrary set of load path directories, each with a tag and a
>   description, something (semantically speaking) like this:
>
>   '(("/usr/share/guile/1.6"
>      "1.6"
>      "Install location for Guile 1.6's own Scheme files")
>
>     ("/usr/share/guile/site"
>      "site"
>      "Install location for site-specific Scheme files")
>
>     ("/opt/gnome/guile"
>      "gnome"
>      "Install location for GNOME-related Scheme files")
>
>     ("/usr/local/share/guile"
>      "local"
>      "Version-independent non-distribution-managed Scheme files")
>
>     ("/usr/local/share/guile/1.6"
>      "local-1.6"
>      "1.6-dependent non-distribution-managed Scheme files"))
>
> - Guile would always then initialize its %load-path to the union of
>   all these locations.
>
> - guile.m4 would provide a --with-guile-scheme-dir=TAG option to
>   ./configure, which would set GUILE_SCHEME_DIR to the location for
>   TAG, and a macro
>
>     GUILE_SCHEME(default-tag)

I like this approach quite well.  It's generic, and it can be used
handle both Guile versioning and module versioning.  Having it in
`init.scm' or some such rather than hardcoded at configuration time
sounds like a good idea too.

Just a couple of notes.  I might look better to use either an alist or
keywords (like `((:tag "1.6" :dir "/usr..."))') to store this
information, but this is really nitpicking.  As for the contents of this
list, I believe the `site' and `local' tags should by default be
configured as `site-MAJOR.MINOR' and `local-MAJOR.MINOR' so that nothing
ends up being installed in unversioned directories.

Thanks,
Ludovic.


_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-user

Re: Where to install files?

[Neil Jerram]

Greg Troxel <gdt@ir.bbn.com> writes:

>> > My own view is that a package configured with --prefix=/usr/foo should
>> > install files only under /usr/foo.  This leads me with guile to want
>> > to extend load-path to include guile directories in other prefixes.
>> 
>> On this point I think I disagree with you.  I think there is value in
>> forcing all packages to be loaded - or at least bootstrapped - from a
>> known load-path that is defined by the Guile installation.
>
> To me it's not a question of value.  /usr/pkg belongs to pkgsrc, and
> to first order other stuff (meaning unmanaged and unrecord stuff)
> should not be there.

I'm not sure we're actually disagreeing here.  My scheme would support
locations other than under /usr/pkg, so it doesn't conflict with your
last sentence.

That said, I've now modified my proposal to allow locations to be
added dynamically as you've suggested.  I'll post that a little later.

> It's an autoconf (configure script really) directive, or perhaps one
> that's commonly added, meant to specify where config files go,
> defaulting to $(prefix)/etc or $(prefix)/etc/PACKAGE, but commonly
> used to force things to /etc/PACKAGE, especially on systems where
> people want that.

Thanks.

> I kind of like having several standard places within a prefix, as seen
> on NetBSD now:
>
> guile> %load-path
> ("/usr/pkg/share/guile/site" "/usr/pkg/share/guile/1.6"
>  "/usr/pkg/share/guile" ".")
>
> Or rather, if that continues, I'd like to specify extra prefixes, with
> the load path being the cross product of within-prefix places and
> prefixes.  But perhaps also single dirs.

That sounds too complicated to me!  And my way already allows you to
have .../1.6 and .../site when you want them.

> I'd like a load-path directory, with files whose names are irrelevant
> but whose contets are dirs to add to the load path.  This is
> pkg-friendly since they can be deleted and added automatically without
> merge/editing issues, using the fs to represent set semantics.
>
>> > It might also be nice to have a run-time method to 
>> >
>> >   guile-configer --addsearchprefix=/usr/bar
>> >
>> > so that someone building another package to /usr/bar can invoke this
>> > on the system guile to hook in the new path.

My new proposal isn't organized exactly as you suggest, but it allows
the same operations.

Regards,
        Neil



_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-user

Re: Where to install files?

[Neil Jerram]

ludovic.courtes@laas.fr (Ludovic Courtès) writes:

> I like this approach quite well.  It's generic, and it can be used
> handle both Guile versioning and module versioning.  Having it in
> `init.scm' or some such rather than hardcoded at configuration time
> sounds like a good idea too.

Thanks.  I'm about to post a slightly modified version, so please let
me know your thoughts on that too.

> Just a couple of notes.  I might look better to use either an alist or
> keywords (like `((:tag "1.6" :dir "/usr..."))') to store this
> information, but this is really nitpicking.

Yes, perhaps, although it would make the sample code in my modified
proposal a bit more complex.

>  As for the contents of this
> list, I believe the `site' and `local' tags should by default be
> configured as `site-MAJOR.MINOR' and `local-MAJOR.MINOR' so that nothing
> ends up being installed in unversioned directories.

Well in general that's exactly the kind of thing that a distribution
or site will be able to decide for themselves, under my scheme, by
modifying the config file.  But are you saying that the default
settings in this file should be

$prefix/share/guile/1.6
$prefix/share/guile/site-1.6
$prefix/share/guile

instead of

$prefix/share/guile/1.6
$prefix/share/guile/site
$prefix/share/guile

?

I think there's a place for both unversioned and versioned site and
local directories, because its not that hard to kind code that handles
multiple Guile versions in the same files.

Thanks for your input,

       Neil



_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-user