modules

modules

[Ian Zimmerman]

Hi, I am fairly new to guile.  So far I am just exploring it as just
another Lisp/Scheme, and writing toy programs in Scheme.

In that connection, I have two questions related to the module system.

1/ How can I customize a module during loading?  In Emacs Lisp, 
I can either:

a) set a global variable, for instance 

(setq foo-module:bar 'baz)

prior to loading a library "foo-module".  If that library itself does

(defvar foo-module:bar)

it picks up the value that was previously set.  But in Scheme, set! only
works on variables that have already been define'd, and a define creates
a new variable totally unrelated to the existing value.

b) use eval-after-load; is there any equivalent in guile?

2/ The documentation for eval says the second argument can be a module.
But how can I get an actual module object?  I tried

(eval foo '(ice-9 rw))

but I get 

Wrong type argument in position 2 (expecting MODULEP): (ice-9 rw)
ABORT: (wrong-type-arg)

Neither the section on using modules or the section on defining them seems
to list any API for getting a module "thing".

-- 
Nothing can be explained to a stone.
Or to a stoned person, either.


_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-user

Re: modules

[Andreas Rottmann]

Ian Zimmerman <itz@buug.org> writes:

> 1/ How can I customize a module during loading?  In Emacs Lisp, 
> I can either:
>
> a) set a global variable, for instance 
>
> (setq foo-module:bar 'baz)
>
> prior to loading a library "foo-module".  If that library itself does
>
> (defvar foo-module:bar)
>
> it picks up the value that was previously set.  But in Scheme, set! only
> works on variables that have already been define'd, and a define creates
> a new variable totally unrelated to the existing value.
>
What exactly do you want to accomplish? The point of modules is to
isolate mostly independent code from each other, and using a global
variable would just prevent that isolation. If you need a global
module setting, you can provide accessor functions:

(define *bar* "The bar")

(define (set-bar! val)
  (set! *bar* val))

(define (get-bar)
  *bar)

Andy
-- 
Andreas Rottmann         | Rotty@ICQ      | 118634484@ICQ | a.rottmann@gmx.at
http://yi.org/rotty      | GnuPG Key: http://yi.org/rotty/gpg.asc
Fingerprint              | DFB4 4EB4 78A4 5EEE 6219  F228 F92F CFC5 01FD 5B62

Make free software, not war!



_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-user

Re: modules

[Thien-Thi Nguyen]

   From: Ian Zimmerman <itz@buug.org>
   Date: 20 Mar 2004 21:40:28 -0800

   How can I customize a module during loading?

for the present, you have to fake it w/ `load' and closures.

for the future, you can propose a design for a (sub-)clause for
`use-modules' or `define-module' that supports parameterized modules,
and then work w/ the module system that implements your design (after
some likely-to-be-lengthly period of debugging ;-).

   how can I get an actual module object?

use `resolve-module'.  for example:

(eval-in-module '(current-module) (resolve-module '(ice-9 rw)))
=> #<directory (ice-9 rw) 4024bd38>

i have discovered there are actually three kinds of "actual module
object" in the context of guile -- one kind you can see in the above
example (the "directory"), the other two normally more transient but
nonetheless exposed/exposable.  the word "module" is the chicken of
programming terms -- everything tastes like it...

thi


_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-user

Re: modules

[Ian Zimmerman]

Andreas> What exactly do you want to accomplish? The point of modules is
Andreas> to isolate mostly independent code from each other, and using a
Andreas> global variable would just prevent that isolation.

This would be true if the access to the global was direct and
unrestricted.  It is not.  The setting is simply supposed to modify what
the module does when loaded.

The actual situation is this: my module internally uses a global hash
table.  This is invisible to users, except that there is no way for the
module itself to guess the table size in advance, so there has to be a
way for users to customize it.  This is pretty much the prototypical
situation where you'd use "functors" or "parametrized modules" in
languages that have them, like ML.

The workaround I am settling on is this: make the hash table a promise
(with @code{delay}) depending on a size variable, and provide an @code{init}
function to modify that variable.  It's a hack, but it works and is the
best that can be done now.

-- 
Nothing can be explained to a stone.
Or to a stoned person, either.


_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-user

Re: modules

[Robert Uhl]

Ian Zimmerman <itz@buug.org> writes:
> 
> The actual situation is this: my module internally uses a global hash
> table.  This is invisible to users, except that there is no way for the
> module itself to guess the table size in advance, so there has to be a
> way for users to customize it.

Wouldn't the Right Thing in this case be that hash tables be dynamic?
In other words, to remove the need for the user guess at all (which is
not all that likely to be on-the-money anyway).

-- 
Robert Uhl <ruhl@4dv.net>
If you're a politician, bureaucrat, or cop whose livelihood depends on
the drug war, you're fully as contemptible as any pusher, smuggler, or
cocaine baron--more so, because, unlike them, you profit directly by
destroying what was once the greatest freedom ever known to mankind.
                              --Mirelle Stein, The Productive Class


_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-user

Modules

[Marek Kubica]

Hi,

I tried executing the modules example:

$ cd guile-git/examples/module
$ guile -s main

(whereas guile = 1.9.13)

but I get an error:

;;; note: autocompilation is enabled, set GUILE_AUTO_COMPILE=0
;;;       or pass the --no-autocompile argument to disable.
;;; compiling main
;;; WARNING: compilation of main failed:
;;; key misc-error, throw_args (#f "~A ~S" ("no code for
module" (module-0)) #f) Backtrace:
In ice-9/boot-9.scm:
 170: 17 [catch #t #<catch-closure 1b750e0> ...]
In unknown file:
   ?: 16 [catch-closure]
In ice-9/boot-9.scm:
  62: 15 [call-with-prompt prompt0 ...]
In ice-9/eval.scm:
 389: 14 [eval # #]
In ice-9/boot-9.scm:
1840: 13 [save-module-excursion #<procedure 1cac990 at
ice-9/boot-9.scm:1854:3 ()>] 1149: 12 [load "main" #f]
1047: 11 [%start-stack load-stack ...]
1052: 10 [#<procedure 22c2fc0 ()>]
In unknown file:
   ?: 9 [primitive-load "main"]
In ice-9/eval.scm:
 458: 8 [#<procedure 1a838c0 at ice-9/eval.scm:452:4 (exp)> #]
In ice-9/psyntax.scm:
 908: 7 [chi-top-sequence ((#(syntax-object let # ...) (#) (# #) ...))
() ...] 1159: 6 [chi-top (#(syntax-object let # ...) (#) (# #) ...)
() ...] In ice-9/eval.scm:
 374: 5 [eval # ()]
In ice-9/boot-9.scm:
2389: 4 [process-define-module ((main) #:filename "main" ...)]
2307: 3 [resolve-interface (module-0) #:select ...]
In unknown file:
   ?: 2 [scm-error misc-error #f "~A ~S" ("no code for
module" (module-0)) #f] In ice-9/boot-9.scm:
 115: 1 [#<procedure 1c698c0 at ice-9/boot-9.scm:110:6 (thrown-k .
args)> misc-error ...] In unknown file:
   ?: 0 [catch-closure misc-error #f "~A ~S" ("no code for
module" (module-0)) #f]

ERROR: In procedure catch-closure:
ERROR: no code for module (module-0)

Is there a way to add the current directory to the search path? I think
this should be default, just like in Python, otherwise creating modules
is a really big hassle.

regards,
Marek

Re: Modules

[Andreas Rottmann]

Marek Kubica <marek@xivilization.net> writes:

> Hi,
>
> I tried executing the modules example:
>
> $ cd guile-git/examples/module
> $ guile -s main
>
[...]
> ERROR: no code for module (module-0)
>
> Is there a way to add the current directory to the search path?
>
% guile --help | grep -i directory
  -L DIRECTORY   add DIRECTORY to the front of the module load path

So it's "guile -L . -s main" for the case above.

> I think this should be default, just like in Python, otherwise
> creating modules is a really big hassle.
>
I'm not sure this should be made the default; for an installed program
implemented in Guile, having "." implicitly in the load path would be
unneccessary and even a security issue.

Regards, Rotty
-- 
Andreas Rottmann -- <http://rotty.yi.org/>

Re: Modules

[Nala Ginrut]

[-- Attachment #1: Type: text/plain, Size: 84 bytes --]

Maybe you can modify %load-path in your code. But I haven't  tested yet.
Good luck!

[-- Attachment #2: Type: text/html, Size: 94 bytes --]

Re: Modules

[Andy Wingo]

On Wed 08 Dec 2010 12:45, Marek Kubica <marek@xivilization.net> writes:

> Is there a way to add the current directory to the search path? I think
> this should be default, just like in Python, otherwise creating modules
> is a really big hassle.

As Andreas mentions, -L is the thing.  We don't add the current
directory to the search path because it is a security issue, for the
same reason that "." is not in $PATH.

Andy
-- 
http://wingolog.org/

Re: Modules

[Marek Kubica]

On Fri, 28 Jan 2011 17:26:07 +0100
Andy Wingo <wingo@pobox.com> wrote:

> On Wed 08 Dec 2010 12:45, Marek Kubica <marek@xivilization.net>
> writes:
> 
> > Is there a way to add the current directory to the search path? I
> > think this should be default, just like in Python, otherwise
> > creating modules is a really big hassle.
> 
> As Andreas mentions, -L is the thing.  We don't add the current
> directory to the search path because it is a security issue, for the
> same reason that "." is not in $PATH.

What about "the same directory that the file is in"? The point is, when
writing scripts that become larger than one file, splitting them into
modules becomes immensely painful because the modules cannot find each
other.

regards,
Marek

Re: Modules

[Neil Jerram]

Marek Kubica <marek@xivilization.net> writes:

> What about "the same directory that the file is in"? The point is, when
> writing scripts that become larger than one file, splitting them into
> modules becomes immensely painful because the modules cannot find each
> other.

I agree that this is a bit awkward.  My current solution is

(load "setup-load-path.scm")

at the start of each top-level script - which relies on the fact that
`load' will look in the same directory as the script file - with
setup-load-path.scm containing:

(cond-expand (guile-2
	      (eval-when (load compile)
			 (let* ((bindir (dirname (car (command-line))))
				(absdir (cond ((string=? bindir ".")
					       (getcwd))
					      ((string-match "^/" bindir)
					       bindir)
					      (else
					       (in-vicinity (getcwd) bindir)))))
			   (set! %load-path (cons (in-vicinity absdir "..")
						  %load-path)))))
	     (else
	      (let* ((bindir (dirname (car (command-line))))
		     (absdir (cond ((string=? bindir ".")
				    (getcwd))
				   ((string-match "^/" bindir)
				    bindir)
				   (else
				    (in-vicinity (getcwd) bindir)))))
		(set! %load-path (cons (in-vicinity absdir "..")
				       %load-path)))))

This has the effect of adding the parent of the script-containing
directory to the load path.

Part of the complexity is supporting both 1.8 and 1.9/2.0.  For 1.9/2.0
support only, I believe it could be simplified by changing `load' to
`include', and retaining only the body of the `eval-when'.

Regards,
        Neil

Re: Modules

[Neil Jerram]

Neil Jerram <neil@ossau.uklinux.net> writes:

> Marek Kubica <marek@xivilization.net> writes:
>
>> What about "the same directory that the file is in"? The point is, when
>> writing scripts that become larger than one file, splitting them into
>> modules becomes immensely painful because the modules cannot find each
>> other.
>
> I agree that this is a bit awkward.  My current solution is
>
> (load "setup-load-path.scm")
>
> at the start of each top-level script - which relies on the fact that
> `load' will look in the same directory as the script file - with
> setup-load-path.scm containing:
>
> (cond-expand (guile-2
> 	      (eval-when (load compile)

It's amazing how writing an email sets you thinking about whether
something is really correct...

In fact I think the top level probably needs to be

(cond-expand (guile-2 (include "setup-load-path.scm"))
             (else (load "setup-load-path.scm")))

so that the path is set up for 1.9/2.0 compilation time.  I wonder if it
works to write that as

((cond-expand (guile-2 include) (else load)) "setup-load-path.scm")

And then setup-load-path.scm can be just

      (let* ((bindir (dirname (car (command-line))))
	     (absdir (cond ((string=? bindir ".")
			    (getcwd))
			   ((string-match "^/" bindir)
			    bindir)
			   (else
			    (in-vicinity (getcwd) bindir)))))
	(set! %load-path (cons (in-vicinity absdir "..")
			       %load-path)))

Which isn't so bad.

Re: Modules

[Andy Wingo]

Hi Marek,

On Sat 29 Jan 2011 13:13, Marek Kubica <marek@xivilization.net> writes:

> On Fri, 28 Jan 2011 17:26:07 +0100
> Andy Wingo <wingo@pobox.com> wrote:
>
>> We don't add the current directory to the search path because it is a
>> security issue, for the same reason that "." is not in $PATH.
>
> What about "the same directory that the file is in"? The point is, when
> writing scripts that become larger than one file, splitting them into
> modules becomes immensely painful because the modules cannot find each
> other.

For what purpose?  If you are using modules, they are already in one
global namespace, the various roots of which are in the %load-path
and/or %load-compiled-path.  (resolve-module '(foo bar)) should work
regardless of what file calls it.

Perhaps you are interested in `load', which is problematic regarding
compiled files; for example when loading from a compiled file, how will
`load' know what is the current directory?  Note that .go files are
installed to e.g. /usr/lib64/guile, while source is in /usr/share/guile.

My suggestion is to use modules by name, or, equivalently, to use
`load-from-path'.  There might be a "proper" solution here, but I
haven't found it yet.

See also the bug at http://savannah.gnu.org/bugs/?30480.

Regards,

Andy
-- 
http://wingolog.org/

Re: Modules

[Andy Wingo]

On Sat 29 Jan 2011 13:13, Marek Kubica <marek@xivilization.net> writes:

> What about "the same directory that the file is in"? The point is, when
> writing scripts that become larger than one file, splitting them into
> modules becomes immensely painful because the modules cannot find each
> other.

I would also mention the approach from the skeleton package, which you
can fetch from http://wingolog.org/git/skeleton.git.  `autoreconf -vif',
`./configure', and `make'.  It has a toplevel `env' script, similar to
other environment scripts needed for other languages that tweak
LD_LIBRARY_PATH.

Cheers,

Andy
-- 
http://wingolog.org/

Re: Modules

[Marek Kubica]

On Sat, 29 Jan 2011 18:04:14 +0100
Andy Wingo <wingo@pobox.com> wrote:

> For what purpose?  If you are using modules, they are already in one
> global namespace, the various roots of which are in the %load-path
> and/or %load-compiled-path.  (resolve-module '(foo bar)) should work
> regardless of what file calls it.

If I want to split my single-file script into multiple modules, they
are not part of the global namespace and therefore I need a way to
access them.

regards,
Marek

Re: Modules

[Marek Kubica]

On Sat, 29 Jan 2011 18:07:04 +0100
Andy Wingo <wingo@pobox.com> wrote:

> I would also mention the approach from the skeleton package, which you
> can fetch from http://wingolog.org/git/skeleton.git.  `autoreconf
> -vif', `./configure', and `make'.  It has a toplevel `env' script,
> similar to other environment scripts needed for other languages that
> tweak LD_LIBRARY_PATH.

But isn't this rather complicated for something as simple as "split
one file into multiple modules"? I would think it is a trivial thing
and my experience from Python, Ruby and Clojure tells me the same.

So far, the solution from Neil looks most practical to me, actually.

regards,
Marek

Re: Modules

[Neil Jerram]

Andy Wingo <wingo@pobox.com> writes:

> Hi Marek,
>
> On Sat 29 Jan 2011 13:13, Marek Kubica <marek@xivilization.net> writes:
>
>> What about "the same directory that the file is in"? The point is, when
>> writing scripts that become larger than one file, splitting them into
>> modules becomes immensely painful because the modules cannot find each
>> other.
>
> For what purpose?  If you are using modules, they are already in one
> global namespace, the various roots of which are in the %load-path
> and/or %load-compiled-path.  (resolve-module '(foo bar)) should work
> regardless of what file calls it.

If the modules are installed, that's true.  What if they are not?

Even if all of the non-installed modules are in a single tree, things
are a bit fiddly.  For work from a REPL or from Geiser, you can put

(set! %load-path (cons "whatever" %load-path))

in ~/.guile, and 

(load (in-vicinity (getenv "HOME") ".guile"))

in ~/.guile-geiser, but this doesn't do the job for scripts that want to
use those modules.  (Because scripts do not load ~/.guile.)

For scripts that use uninstalled modules, then, some kind of solution is
needed; ideally one that works for both 1.8 and 1.9/2.0, allows the code
needed to live in a single common file, rather than duplicated at the
top of each script; and continues to work if the script+module tree as a
whole is moved to a different place in the filesystem.  Also I think
it's preferable if the solution is a Guile one, as opposed to based on
the #! line, or needing a shell script wrapper.

> Perhaps you are interested in `load', which is problematic regarding
> compiled files; for example when loading from a compiled file, how will
> `load' know what is the current directory?  Note that .go files are
> installed to e.g. /usr/lib64/guile, while source is in /usr/share/guile.

Good point, thanks for the reminder about that.  But (for 1.9/2.0)
`include' will always be well-defined and reliably relative to the
containing file's name, won't it?

Regards,
        Neil

Re: Modules

[Neil Jerram]

Andy Wingo <wingo@pobox.com> writes:

> I would also mention the approach from the skeleton package, which you
> can fetch from http://wingolog.org/git/skeleton.git.  `autoreconf -vif',
> `./configure', and `make'.  It has a toplevel `env' script, similar to
> other environment scripts needed for other languages that tweak
> LD_LIBRARY_PATH.

Thanks.  It seems a shame, and less portable, for us to rely on shell
script wrappers.  But given that we're talking about uninstalled stuff,
and that it seems extremely unlikely for a development machine not to
have a decent shell, I guess that's actually OK.

      Neil

Re: Modules

[Thien-Thi Nguyen]

() Neil Jerram <neil@ossau.uklinux.net>
() Sat, 29 Jan 2011 23:17:08 +0000

   If the modules are installed, that's true.  What if they are not?

   [...]

   For scripts that use uninstalled modules, then, some kind of
   solution is needed; ideally one that works for both 1.8 and
   1.9/2.0, allows the code needed to live in a single common
   file, rather than duplicated at the top of each script; and
   continues to work if the script+module tree as a whole is moved
   to a different place in the filesystem.

This is what "module catalogs" from Guile 1.4.x provides.

[verbose explanation follows, skip to end for summary]

Basically, you associate with each element in ‘%load-path’ a
sub-association between module name (list of symbols) and
implementation resolution method.  For Guile 1.4.x, there are two
types of modules supported:
 - scheme source (text)
 - shared object library

Of course the implementation must provide the appropriate
interface (as defined by the support in ‘resolve-module’) for
loading; this system cannot be used for arbitrary scheme source or
shared object libraries.

At load-time, the interpreter consults these catalogs
preferentially, falling back to old-style filesystem groping on
lookup failure.  A module (of any form) need not include its
location information.

To see how this helps, i use two small scripts:

st:

  #!/bin/sh
  exec strace -f -e open "$@" 2>&1 | grep -v o.such.file

sta:

  #!/bin/sh
  exec strace -f -e open "$@" 2>&1

Here is a run of loading (database postgres-table) [scheme code]
from the Guile-PG build tree.  Note that it in turn requires
(database postgres) [shared object library].  Also note ‘-L .’
which is explained further down.

  $ cd ~/build/guile-pg/.b
  $ st guile -L . -c '(use-modules (database postgres-table))'
  open("/home/ttn/local/lib/libguile.so.9", O_RDONLY) = 3
  open("/home/ttn/local/lib/libltdl.so.7", O_RDONLY) = 3
  open("/etc/ld.so.cache", O_RDONLY)      = 3
  open("/lib/i686/cmov/libm.so.6", O_RDONLY) = 3
  open("/lib/i686/cmov/libdl.so.2", O_RDONLY) = 3
  open("/lib/i686/cmov/libc.so.6", O_RDONLY) = 3
  open("/home/ttn/local/lib/guile/site/.module-catalog", O_RDONLY) = 3
  open("/home/ttn/local/share/guile/site/.module-catalog", O_RDONLY) = 3
  open("/home/ttn/local/lib/guile/1.4.1.122/.module-catalog", O_RDONLY) = 3
  open("/home/ttn/local/lib/guile/site/init.scm", O_RDONLY) = 3
  open("./.module-catalog", O_RDONLY)     = 3
  open("/home/ttn/build/guile-pg/src/postgres-table.scm", O_RDONLY) = 3
  open("/home/ttn/local/lib/guile/1.4.1.122/ice-9/common-list.scm", O_RDONLY) = 4
  open("/home/ttn/build/guile-pg/.b/src/.libs/postgres.so.0.0.0", O_RDONLY) = 4
  open("/home/ttn/local/lib/libpq.so.3", O_RDONLY) = 4
  open("/etc/ld.so.cache", O_RDONLY)      = 4
  open("/lib/i686/cmov/libcrypt.so.1", O_RDONLY) = 4
  open("/lib/i686/cmov/libresolv.so.2", O_RDONLY) = 4
  open("/lib/i686/cmov/libnsl.so.1", O_RDONLY) = 4
  open("/home/ttn/build/guile-pg/src/postgres-types.scm", O_RDONLY) = 4
  open("/home/ttn/build/guile-pg/src/postgres-col-defs.scm", O_RDONLY) = 4
  open("/home/ttn/build/guile-pg/src/postgres-qcons.scm", O_RDONLY) = 4
  open("/home/ttn/build/guile-pg/src/postgres-resx.scm", O_RDONLY) = 4

If i use ‘sta’, there are many "no such file" messages, but NOT
with the modules.  Strictly speaking, open(2) savings proves
improvement only for shared object libraries, so you'll have to
trust me (or confirm by looking at the source code) that the
scheme source modules are treated analogously (i.e., no probing).

  $ cd ~/build/guile-pg/.b
  $ st  guile -L . -c '(use-modules (database postgres-table))' > A
  $ sta guile -L . -c '(use-modules (database postgres-table))' > B
  $ diff -u A B | uniq
  --- A	2011-01-30 10:41:25.000000000 +0100
  +++ B	2011-01-30 10:41:32.000000000 +0100
  @@ -1,8 +1,29 @@
  +open("/home/ttn/local/lib/tls/i686/sse2/cmov/libguile.so.9", O_RDONLY) = -1 ENOENT (No such file or directory)
  +open("/home/ttn/local/lib/tls/i686/sse2/libguile.so.9", O_RDONLY) = -1 ENOENT (No such file or directory)
  +open("/home/ttn/local/lib/tls/i686/cmov/libguile.so.9", O_RDONLY) = -1 ENOENT (No such file or directory)
  +open("/home/ttn/local/lib/tls/i686/libguile.so.9", O_RDONLY) = -1 ENOENT (No such file or directory)
  +open("/home/ttn/local/lib/tls/sse2/cmov/libguile.so.9", O_RDONLY) = -1 ENOENT (No such file or directory)
  +open("/home/ttn/local/lib/tls/sse2/libguile.so.9", O_RDONLY) = -1 ENOENT (No such file or directory)
  +open("/home/ttn/local/lib/tls/cmov/libguile.so.9", O_RDONLY) = -1 ENOENT (No such file or directory)
  +open("/home/ttn/local/lib/tls/libguile.so.9", O_RDONLY) = -1 ENOENT (No such file or directory)
  +open("/home/ttn/local/lib/i686/sse2/cmov/libguile.so.9", O_RDONLY) = -1 ENOENT (No such file or directory)
  +open("/home/ttn/local/lib/i686/sse2/libguile.so.9", O_RDONLY) = -1 ENOENT (No such file or directory)
  +open("/home/ttn/local/lib/i686/cmov/libguile.so.9", O_RDONLY) = -1 ENOENT (No such file or directory)
  +open("/home/ttn/local/lib/i686/libguile.so.9", O_RDONLY) = -1 ENOENT (No such file or directory)
  +open("/home/ttn/local/lib/sse2/cmov/libguile.so.9", O_RDONLY) = -1 ENOENT (No such file or directory)
  +open("/home/ttn/local/lib/sse2/libguile.so.9", O_RDONLY) = -1 ENOENT (No such file or directory)
  +open("/home/ttn/local/lib/cmov/libguile.so.9", O_RDONLY) = -1 ENOENT (No such file or directory)
   open("/home/ttn/local/lib/libguile.so.9", O_RDONLY) = 3
   open("/home/ttn/local/lib/libltdl.so.7", O_RDONLY) = 3
  +open("/home/ttn/local/lib/libm.so.6", O_RDONLY) = -1 ENOENT (No such file or directory)
   open("/etc/ld.so.cache", O_RDONLY)      = 3
   open("/lib/i686/cmov/libm.so.6", O_RDONLY) = 3
  +open("/home/ttn/local/lib/libdl.so.2", O_RDONLY) = -1 ENOENT (No such file or directory)
   open("/lib/i686/cmov/libdl.so.2", O_RDONLY) = 3
  +open("/home/ttn/local/lib/libc.so.6", O_RDONLY) = -1 ENOENT (No such file or directory)
   open("/lib/i686/cmov/libc.so.6", O_RDONLY) = 3
   open("/home/ttn/local/lib/guile/site/.module-catalog", O_RDONLY) = 3
   open("/home/ttn/local/share/guile/site/.module-catalog", O_RDONLY) = 3
  @@ -13,9 +34,21 @@
   open("/home/ttn/local/lib/guile/1.4.1.122/ice-9/common-list.scm", O_RDONLY) = 4
   open("/home/ttn/build/guile-pg/.b/src/.libs/postgres.so.0.0.0", O_RDONLY) = 4
   open("/home/ttn/local/lib/libpq.so.3", O_RDONLY) = 4
  +open("/home/ttn/local/lib/libcrypt.so.1", O_RDONLY) = -1 ENOENT (No such file or directory)
   open("/etc/ld.so.cache", O_RDONLY)      = 4
   open("/lib/i686/cmov/libcrypt.so.1", O_RDONLY) = 4
  +open("/home/ttn/local/lib/libresolv.so.2", O_RDONLY) = -1 ENOENT (No such file or directory)
   open("/lib/i686/cmov/libresolv.so.2", O_RDONLY) = 4
  +open("/home/ttn/local/lib/libnsl.so.1", O_RDONLY) = -1 ENOENT (No such file or directory)
   open("/lib/i686/cmov/libnsl.so.1", O_RDONLY) = 4
   open("/home/ttn/build/guile-pg/src/postgres-types.scm", O_RDONLY) = 4
   open("/home/ttn/build/guile-pg/src/postgres-col-defs.scm", O_RDONLY) = 4

The ‘-L .’ means that ~/build/guile-pg/.b/.module-catalog is
consulted:

  ;;; .module-catalog
  ;;; generated 2010-12-18 20:35:15 UTC -- do not edit!
  
  (
   ((database postgres-resdisp) . "/home/ttn/build/guile-pg/src/postgres-resdisp.scm")
   ((database postgres-col-defs) . "/home/ttn/build/guile-pg/src/postgres-col-defs.scm")
   ((database postgres-table) . "/home/ttn/build/guile-pg/src/postgres-table.scm")
   ((database postgres-meta) . "/home/ttn/build/guile-pg/src/postgres-meta.scm")
   ((database postgres-gxrepl) . "/home/ttn/build/guile-pg/src/postgres-gxrepl.scm")
   ((database postgres) scm_init_module "scm_init_database_postgres_module" () . "/home/ttn/build/guile-pg/.b/src/.libs/postgres.so.0.0.0")
   ((database postgres-types) . "/home/ttn/build/guile-pg/src/postgres-types.scm")
   ((database postgres-qcons) . "/home/ttn/build/guile-pg/src/postgres-qcons.scm")
   ((database postgres-resx) . "/home/ttn/build/guile-pg/src/postgres-resx.scm")
  )
  
  ;;; .module-catalog ends here

This (version 1) format is not very compact.  For version 2, we
add a magic number, factor the root and all intervening parent
directories, provide other meta info in a "header" section, and
include other meta info (exports list being the most interesting)
in the body.  In some sense version 2 is like /etc/ld.so.cache.

I imagine Guile 2.x could adopt module catalogs, extending to
handle also compiled scheme (.go) and the (future) JIT files as
well, but perhaps it's too late.  I wonder how module catalogs
would fit w/ name canonicalization, which IIUC is specific to
compiled scheme.

In sum, build-time caching plus run-time indirection as a means to
facile module relocatability (plus some performance gain).

Re: Modules

[Andy Wingo]

Hi Neil,

On Sun 30 Jan 2011 00:17, Neil Jerram <neil@ossau.uklinux.net> writes:

> Andy Wingo <wingo@pobox.com> writes:
>
>> If you are using modules, they are already in one global namespace,
>> the various roots of which are in the %load-path and/or
>> %load-compiled-path.  (resolve-module '(foo bar)) should work
>> regardless of what file calls it.
>
> If the modules are installed, that's true.  What if they are not?

In that case you have to modify the load path, somehow.  I have been
doing this with scripts that add to $GUILE_LOAD_PATH and
$GUILE_LOAD_COMPILED_PATH.  (Which is another thing to note; if you have
an autotooled project, the .go files go in the $builddir.)

So if I'm hacking against an uninstalled tekuti, for example, I run my
code within ~/src/tekuti/env.

> For scripts that use uninstalled modules, then, some kind of solution is
> needed; ideally one that works for both 1.8 and 1.9/2.0, allows the code
> needed to live in a single common file, rather than duplicated at the
> top of each script; and continues to work if the script+module tree as a
> whole is moved to a different place in the filesystem.  Also I think
> it's preferable if the solution is a Guile one, as opposed to based on
> the #! line, or needing a shell script wrapper.

How would it look?

I guess I am unclear on the actual problem being solved here :)  Let's
consider that I am hacking on my "analysis" script, which lives at
~/src/foo/analysis.  I guess it's helping me in my "foo" project.  Now
the script gets too big; time to split into modules.  How to do that?

We have basically one option of a path to automatically add to the load
path: ~/src/foo.  It seems tractable, unless we start to consider
installing the script to /usr/bin, combined with the presence of "" in
the default load-extensions list, in which case the unexpected
interactions with other members of that path look daunting.

No, I think that the script itself will need to indicate some path to
add to the load path.

What you can do, perhaps, is add a macro:

(define-syntax add-relative-load-path
  (lambda (x)
    (syntax-case x ()
      ((_ path) (string? (syntax->datum #'path))
       (let* ((src (syntax-source #'x))
              (current-file (or (and src (assq-ref src 'filename))
                                (error "Could not determine current file name")))
              (vicinity (dirname (canonicalize-path current-file)))
              (path-elt (in-vicinity vicinity (syntax->datum #'path))))
         #`(eval-when (compile load eval)
             (set! %load-path (cons #,path-elt %load-path))))))))

Then in "analysis", you could `(add-relative-load-path ".")'.

But...  If you compile a .go, then install both .scm and .go somewhere,
the installed .go file will reference the build path, which was inserted
into the source via the current-file business.

(You could argue that this is precisely the case that we are _not_
interested in, given that currently we only install .go files for files
that are in the load path.  But I guess we should figure out a better
autocompilation story for files in $bindir.)

It _is_ true that we need to figure out what's going on with
(current-load-port) in 1.9, but it's not clear what to do, exactly, when
you have compiled files; you could not have the corresponding .scm at
all, or in any case when you've loaded the .go you don't actually have a
port to the .scm, and in fact if the file was loaded via
`load-from-path' you don't know exactly where the .scm is at all.

Perhaps we should residualize into the .go whether the file was compiled
for `load' or for `load-from-path', and what was the original path of
the file.

> Good point, thanks for the reminder about that.  But (for 1.9/2.0)
> `include' will always be well-defined and reliably relative to the
> containing file's name, won't it?

Yes, at expansion time.  But note the .scm/.go installation case; and
also note that the code that chooses when to use a .go over a .scm
doesn't know anything about dependencies, currently, so a change to an
included file doesn't trigger recompilation of the includer.

Andy
-- 
http://wingolog.org/

Re: Modules

[Andy Wingo]

On Sun 30 Jan 2011 00:35, Neil Jerram <neil@ossau.uklinux.net> writes:

> Andy Wingo <wingo@pobox.com> writes:
>
>> I would also mention the approach from the skeleton package, which you
>> can fetch from http://wingolog.org/git/skeleton.git.  `autoreconf -vif',
>> `./configure', and `make'.  It has a toplevel `env' script, similar to
>> other environment scripts needed for other languages that tweak
>> LD_LIBRARY_PATH.
>
> Thanks.  It seems a shame, and less portable, for us to rely on shell
> script wrappers.

Humm, indeed.

While I'm here in this email thing, I should say that I didn't mean to
brush off the original problem so directly.  Apologies if that's how it
came across, Marek.  I just meant to communicate that I don't fully
understand the problem, nor do I know exactly what to do about it.

If there is something that it makes sense for Guile can do to help this
use case, we should do it.


Regards,

Andy
-- 
http://wingolog.org/

Re: Modules

[Jon Wilson]

On 01/29/2011 06:35 PM, Neil Jerram wrote:
> Andy Wingo <wingo@pobox.com> writes:
> 
>> I would also mention the approach from the skeleton package, which you
>> can fetch from http://wingolog.org/git/skeleton.git.  `autoreconf -vif',
>> `./configure', and `make'.  It has a toplevel `env' script, similar to
>> other environment scripts needed for other languages that tweak
>> LD_LIBRARY_PATH.
> 
> Thanks.  It seems a shame, and less portable, for us to rely on shell
> script wrappers.  But given that we're talking about uninstalled stuff,
> and that it seems extremely unlikely for a development machine not to
> have a decent shell, I guess that's actually OK.
> 
>       Neil
> 

Hi all,
I don't post to this list very often, since I'm not using guile much
these days.  I use python by necessity (necessity=that or C++) for work.

My work, however, involves principally writing and running single-use
code for physics analyses.  Installing this code would simply not make
any sense.  Hence I use uninstalled code quite extensively, almost
exclusively.  This seems to work quite well in python, which further
seems to have unified package-internal references with uninstalled-code
references very neatly.  I can't comment on the security or reliability
implications of Guido's arrangement, however.

Just thought I'd weigh in with a use-case in the wild.
Regards,
Jon

Re: Modules

[Andy Wingo]

Hello list,

A couple weeks ago I wrote about figuring out relative paths for `load'
in 1.9:

On Sun 30 Jan 2011 12:42, Andy Wingo <wingo@pobox.com> writes:

> it's not clear what to do, exactly, when
> you have compiled files; you could not have the corresponding .scm at
> all, or in any case when you've loaded the .go you don't actually have a
> port to the .scm, and in fact if the file was loaded via
> `load-from-path' you don't know exactly where the .scm is at all.
>
> Perhaps we should residualize into the .go whether the file was compiled
> for `load' or for `load-from-path', and what was the original path of
> the file.

It turns out we already have this information on the Scheme level, in
the form of source properties of the expressions.  Macros have access to
this via `syntax-source'.  Also, with "relative" filename
canonicalization, the `filename' property of a syntax source will be a
relative path if it was found in the load path, and an absolute path
otherwise.

So, I ended up reimplementing `load' as a macro (!) that expands out to
a call to `load-in-vicinity'.  Relative paths are looked up against the
dirname of the file being expanded.  If the dirname is relative, then
load-from-path is used, and otherwise load is used.

A bare reference to `load' returns a closure that will invoke
`load-in-vicinity' with an appropriate vicinity.

Hopefully this fully fixes bug 30480, discussed in
http://thread.gmane.org/gmane.lisp.guile.bugs/4359, and *part* of this
use case.  We still need an (add-to-load-path "."), as discussed in a
previous mail, to allow use-modules for uninstalled modules to work
reliably.

Cheers,

Andy
-- 
http://wingolog.org/