R6RS Libraries

R6RS Libraries

[Ludovic Courtès]

Hi,

FWIW, I started an implementation of R6RS' bytevector API.  It is
available in my Arch archive [0], in the branch named
`guile-r6rs-libs--devo--0'.

Just letting you know so that we avoid duplicating this (tedious) work.
;-)

Thanks,
Ludovic.

[0] http://www.laas.fr/~lcourtes/software/arch-2006/


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-devel

Re: r6rs libraries

[Julian Graham]

Hi everyone,

(Switching this conversation to guile-devel from guile-user, since it
seems more appropriate to this list...)

Alright, so I've been studying the van Tonder and Dybvig-Ghuloum
implementations and banging my head against chapter 7 of R6RS, all
with an eye towards mapping them onto Guile's module system, and I
can't for the life of me figure out why the existing implementations
are as complicated as they are.  Maybe some more advanced Schemers
than I can shed some light on the following:

* Import and export levels seem to be a fancy way of notifying the
library system of the time at which a library needs to be
loaded/evaluated -- that is, if you import something from [library
foo] for the "expand phase" of [library bar] you've got to evaluate
(i.e., convert to a Guile module) the S-exp for [library foo] before
you can evaluate the S-exp for [library bar].  The levels system is
simply a numerical way of encapsulating this information, but the
proper order of evaluation can also be inferred by inspecting the
import- and export-specs of the libraries being loaded -- i.e., if the
header of [library bar] specifies an import of anything from [library
foo], no matter at what "level," it's a safe move to evaluate [library
foo] (if you haven't already done so) before finishing the evaluation
of [library bar].  Is that right?

* R6RS says that a library's imports need to be visited/instantiated
at the time the bindings they export are "referenced."  Why?  As
above, why can't they be visited/instantiated at the time the imports
for the importing library are processed?  Is there any noticeable
difference to the user?  Or do you guys read R6RS 7.2 to mean that the
side-effects of top-level expressions absolutely need to happen at a
time determined by the import level?

* R6RS also says that implementations are free to visit/instantiate
libraries more or less often than is required by the import-export
graph.  Why would you want to visit/instantiate a library more than
once?  Why not just do it once, turn it into a module, and cache it?
Andy Wingo noted that some implementations do a fresh visit for every
phase (and that it's problematic), but I can't even see why you'd want
to if the spec lets you off the hook for it.

I understand that the authors of the reference implementation
re-created a lot of machinery out of whole cloth since they were
avoiding assumptions about features of their target Scheme platforms,
but, man, both van Tonder and Dybvig-Ghuloum look like overkill for
Guile.  Am I missing a major piece of understanding here?


Regards,
Julian

Re: r6rs libraries

[Andy Wingo]

Hi Julian,

On Mon 26 Jan 2009 01:27, Julian Graham <joolean@gmail.com> writes:

> Maybe some more advanced Schemers than I can shed some light on the
> following:

Well, that's not me, but I'll join you in fumbling for a solution :-)

> The levels system is simply a numerical way of encapsulating this
> information, but the proper order of evaluation can also be inferred
> by inspecting the import- and export-specs of the libraries being
> loaded

I think you're right, yes. I think that the approach that you describe
has been called "Implicit phasing" by Ghuloum and Dybvig. They have a
paper about it, "Implicit phasing in R6RS libraries" -- but I haven't
been able to find it freely on the web. ACM fail.

> * R6RS says that a library's imports need to be visited/instantiated
> at the time the bindings they export are "referenced."  Why?  As
> above, why can't they be visited/instantiated at the time the imports
> for the importing library are processed?

I could not find the quote that you referred to here -- I think what I
can tell (from 7.2):

    If any of a library’s definitions are referenced at phase 0 in the
    expanded form of a program, then an instance of the referenced
    library is created for phase 0 before the program’s definitions and
    expressions are evaluated. This rule applies transitively: if the
    expanded form of one library references at phase 0 an identifier
    from another library, then before the referencing library is
    instantiated at phase n, the referenced library must be instantiated
    at phase n. When an identifier is referenced at any phase n greater
    than 0, in contrast, then the defining library is instantiated at
    phase n at some unspecified time before the reference is evaluated.
    Similarly, when a macro keyword is referenced at phase n during the
    expansion of a library, then the defining library is visited at
    phase n at some unspecified time before the reference is evaluated.

So what this says to me is that:

  (1) At phase 0, libraries that you need to run a /program/ are
      instantiated before the program is run.

  (2) At phase n > 0, we do not specify when libraries are imported.

> Is there any noticeable difference to the user?

Dunno, to me it sounds like a concession, that side effects from loading
libraries occur before side effects from running a program; but that for
meta-levels things are left unspecified.

> Or do you guys read R6RS 7.2 to mean that the side-effects of
> top-level expressions absolutely need to happen at a time determined
> by the import level?

No.

So, for some of your other questions here's section 7.5 of the
rationale:

    7.5. Instantiation and initialization                     
                                                              
    Opinions vary on how libraries should be instantiated and
    initialized during the expansion and execution of library bodies,
    whether library instances should be distinguished across phases, and
    whether levels should be declared so that they constrain identifier
    uses to particular phases.
                                                              
As I read this, it means that at least PLT wanted the separate
instantiation model, and Chez wanted single-instantiation, implicit
phasing.

    This report therefore leaves considerable latitude to
    implementations, while attempting to provide enough guarantees to
    make portable libraries feasible.

So from 7.2 of R6RS itself:

    An implementation may distinguish instances/visits of a library for
    different phases or to use an instance/visit at any phase as an
    instance/visit at any other phase.

Which is to say, "we allow single instantiation" -- as Guile modules
are.

    An implementation may further expand each library form with distinct
    visits of libraries in any phase and/or instances of libraries in
    phases above 0.

Which is to say, "we also allow the PLT model, explicitly."

    An implementation may create instances/visits of more libraries at
    more phases than required to satisfy references.

This is an odd one. I suppose what it means is that if you need a macro
from library A to expand library B, but you don't need library A at
runtime, the spec allows library A to be /instantiated/ at runtime.

    When an identifier appears as an expression in a phase that is
    inconsistent with the identifier’s level, then an implementation
    may raise an exception either at expand time or run time, or it may
    allow the reference.

So, furthermore, it seems that not only may library A be instantiated at
runtime, /it may be in library B's "import list" as well/. This is what
happens with Guile's current module semantics.

    Thus, a library whose meaning depends on whether the instances of a
    library are distinguished or shared across phases or library
    expansions may be unportable.

Indeed, indeed.

> I understand that the authors of the reference implementation
> re-created a lot of machinery out of whole cloth since they were
> avoiding assumptions about features of their target Scheme platforms,
> but, man, both van Tonder and Dybvig-Ghuloum look like overkill for
> Guile.  Am I missing a major piece of understanding here?

I don't think you're missing anything big, no. I hadn't fully poked into
these implementations -- if it is easier just to build something on top
of Guile's modules than to retrofit guile's modules into one of their
implementations, then that's all the better, right?

Cheers,

Andy
-- 
http://wingolog.org/

Re: r6rs libraries

[Ludovic Courtès]

Hello!

Andy Wingo <wingo@pobox.com> writes:

> So from 7.2 of R6RS itself:
>
>     An implementation may distinguish instances/visits of a library for
>     different phases or to use an instance/visit at any phase as an
>     instance/visit at any other phase.
>
> Which is to say, "we allow single instantiation" -- as Guile modules
> are.

[...]

>
>     When an identifier appears as an expression in a phase that is
>     inconsistent with the identifier’s level, then an implementation
>     may raise an exception either at expand time or run time, or it may
>     allow the reference.
>
> So, furthermore, it seems that not only may library A be instantiated at
> runtime, /it may be in library B's "import list" as well/. This is what
> happens with Guile's current module semantics.

These both come as a surprise to me, as I had always thought R6RS was
much stricter about phase separation.  That's good news for "pure
interpreters"; extra work would still be needed to handle phases > 0 in
the compiler.

Thanks for the detailed reading!

Ludo'.

Re: r6rs libraries

[Julian Graham]

Hi Andy,

Thanks so much for helping me parse!


> I think you're right, yes. I think that the approach that you describe
> has been called "Implicit phasing" by Ghuloum and Dybvig. They have a
> paper about it, "Implicit phasing in R6RS libraries" -- but I haven't
> been able to find it freely on the web. ACM fail.

Oooh -- if you do find it, let me know.  That sounds interesting.


> I could not find the quote that you referred to here -- I think what I
> can tell (from 7.2):

(Yes, that was the bit I was referring to.  Sorry about that.)

One thing that occurred to me, though, while I was brooding on this,
is the case in which you've got two bindings that share the same name
that are both imported (from different libraries) into library foo --
one's a syntax transformer used to expand library foo, the other's a
function used by library foo at runtime.  R6RS seems designed to
address cases like this so that they work out, you know, hygienically.
 And I think Guile's module system can handle this as well, though it
may require some slightly weird designs.

E.g., one thing I was thinking of trying was mapping an R6RS library
onto *two* generated Guile modules: One to represent the expansion
environment, and one to represent the environment of the expanded
library in which the library bindings and top-level expressions are
actually evaluated (I guess this is the same as the runtime
environment?).  The first module's imports would include bindings
required for expansion; the second's would include the stuff for
runtime.


> I don't think you're missing anything big, no. I hadn't fully poked into
> these implementations -- if it is easier just to build something on top
> of Guile's modules than to retrofit guile's modules into one of their
> implementations, then that's all the better, right?

Strongly agree.


Regards,
Julian

Re: r6rs libraries

[Julian Graham]

Hi Guilers,

Just wanted to give an update on the status of the library system I've
been working on:

* As per the discussion above, my implementation uses an implicit
phasing approach (i.e., it doesn't do multiple instantiation) along
with `(ice-9 syncase)' to handle expand-time evaluation --
specifically, by way of evaluation environments based on on-the-fly
creation of custom module interfaces.  I wasn't able to find
Abdulaziz's ACM paper, but his "formal comment" on the subject of R6RS
phasing [1] was informative.  (In fact, a lot of the formal comments
were handy: [2].)  I haven't yet delved into how severe the breakage
is where there are differences between Guile's `syntax-case' and
R6RS's, but I'm hoping it's, you know, mild.

* Given the above, my current implementation is hovering at about 200
lines, whereas the current reference implementations, because they
don't presume an existing macro or module system, are several orders
of magnitude larger.  Thanks are due to Tom Lord (and everyone else
who designed Guile's module / interface system) for making this
possible, if not downright easy.

* At the moment, R6RS libraries can "import" Guile modules directly,
but the reverse is not true, partly because I can't figure out a good
way to map version information onto the filesystem in a way that
Guile's module system can understand.  My initial thought was to do it
such that a library's version would map to its module name, a la:

(library (foo bar baz (6))

would live in

/foo/bar/6/baz.scm

...but Guile doesn't like numbers in the module name.  Right now I'm
trying to think of way to do a transformation on a library expression
such that multiple versions of the same library could live in the same
Guile module and be accessed individually by some kind of manifest at
"use" time, but I can't figure out what that module name and the
extra, associated API should look like.

* It would be cool if the module system had hooks of some sort to
allow it to receive hints about locating modules.

* The "party balloons" example from the R6Rs is working, at least
intermittently.  The "quotient-and-remainder" example is almost there.


Regards,
Julian

[1] - http://www.r6rs.org/formal-comments/comment-123.txt
[2] - http://www.r6rs.org/formal-comments/comment-92.txt

Re: r6rs libraries

[Andreas Rottmann]

Julian Graham <joolean@gmail.com> writes:

> * At the moment, R6RS libraries can "import" Guile modules directly,
> but the reverse is not true, partly because I can't figure out a good
> way to map version information onto the filesystem in a way that
> Guile's module system can understand.  My initial thought was to do it
> such that a library's version would map to its module name, a la:
>
> (library (foo bar baz (6))
>
> would live in
>
> /foo/bar/6/baz.scm
>
> ...but Guile doesn't like numbers in the module name.
>
All R6RS implementations I'm somewhat familiar with (Ikarus, PLT,
Ypsilon) do some kind of name mangling when library names contain
"special" characters; e.g. (srfi :6 and-let*) maps to
"srfi/%3a2/and-let%2a.sls", at least for Ikarus (%-escaped hex of the
utf-8 encoding, IIRC).

Nice to see Guile is on the way to R6RS interoperability!

Regards, Rotty

Re: r6rs libraries

[Julian Graham]

Hi Rotty,

> All R6RS implementations I'm somewhat familiar with (Ikarus, PLT,
> Ypsilon) do some kind of name mangling when library names contain
> "special" characters; e.g. (srfi :6 and-let*) maps to
> "srfi/%3a2/and-let%2a.sls", at least for Ikarus (%-escaped hex of the
> utf-8 encoding, IIRC).

Hey, wait, I'm dumb -- why does :6 map to UTF-8 0x3A2?

On a sort of related note, though, I was experimenting with acceptable
symbol characters / filenames, and .6 seems to work as either.  Any
reason Guile couldn't map the R6RS library reference (a b c (1 2 3))
to Guile module (a b .1 .2 .3 c)?


Regards,
Julian





>
> Nice to see Guile is on the way to R6RS interoperability!
>
> Regards, Rotty
>

Re: r6rs libraries

[Andreas Rottmann]

Julian Graham <joolean@gmail.com> writes:

> Hi Rotty,
>
>> All R6RS implementations I'm somewhat familiar with (Ikarus, PLT,
>> Ypsilon) do some kind of name mangling when library names contain
>> "special" characters; e.g. (srfi :6 and-let*) maps to
>> "srfi/%3a2/and-let%2a.sls", at least for Ikarus (%-escaped hex of the
>> utf-8 encoding, IIRC).
>
> Hey, wait, I'm dumb -- why does :6 map to UTF-8 0x3A2?
>
Oh, sorry, typo: it should have been (srfi :2 and-let*).

Rotty

Re: r6rs libraries

[Ludovic Courtès]

Hi Julian,

Thanks for the nice report!

Julian Graham <joolean@gmail.com> writes:

> * At the moment, R6RS libraries can "import" Guile modules directly,
> but the reverse is not true, partly because I can't figure out a good
> way to map version information onto the filesystem in a way that
> Guile's module system can understand.  My initial thought was to do it
> such that a library's version would map to its module name, a la:
>
> (library (foo bar baz (6))
>
> would live in
>
> /foo/bar/6/baz.scm
>
> ...but Guile doesn't like numbers in the module name.

There are many other things that it doesn't like in module names,
typically any standard top-level binding (`eval', `+', etc.) can't be
used as a module name component.  That plus "special" chars means you
really need some name mangling, as Andreas suggested.

As for version numbers, you could have a mapping like the following:

  R6RS Name             Guile Name

  (foo bar)      -->    (foo bar)
  (foo bar (6))  -->    (foo bar version-6)

(Given the recursive name space, this would allow the implementation of
several versions in a single file.  In this example, `version-6' in
`(foo bar)' just has to be bound to a module.)

Thanks,
Ludo'.

Re: r6rs libraries

[Julian Graham]

>  R6RS Name             Guile Name
>
>  (foo bar)      -->    (foo bar)
>  (foo bar (6))  -->    (foo bar version-6)
>
> (Given the recursive name space, this would allow the implementation of
> several versions in a single file.  In this example, `version-6' in
> `(foo bar)' just has to be bound to a module.)

Wait, you've lost me -- the recursive name space?  I thought Guile
requires that the "tail car" of the module name map to an actual
filename on disk when searching for modules.

Re: r6rs libraries

[Ludovic Courtès]

Hi,

Julian Graham <joolean@gmail.com> writes:

>>  R6RS Name             Guile Name
>>
>>  (foo bar)      -->    (foo bar)
>>  (foo bar (6))  -->    (foo bar version-6)
>>
>> (Given the recursive name space, this would allow the implementation of
>> several versions in a single file.  In this example, `version-6' in
>> `(foo bar)' just has to be bound to a module.)
>
> Wait, you've lost me -- the recursive name space?  I thought Guile
> requires that the "tail car" of the module name map to an actual
> filename on disk when searching for modules.

Look for `nested-ref' in `boot-9.scm'.  That's why standard binding
names can't be used in module names (e.g., a module can't be named
`(foo bar eval)').

Thanks,
Ludo'.

Re: r6rs libraries

[Andy Wingo]

Hi,

On Wed 18 Feb 2009 09:53, ludo@gnu.org (Ludovic Courtès) writes:

> Julian Graham <joolean@gmail.com> writes:
>
>> Wait, you've lost me -- the recursive name space?  I thought Guile
>> requires that the "tail car" of the module name map to an actual
>> filename on disk when searching for modules.
>
> Look for `nested-ref' in `boot-9.scm'.  That's why standard binding
> names can't be used in module names (e.g., a module can't be named
> `(foo bar eval)').

We should really fix this -- there's no reason that modules and
variables share the same obarray. This nastiness has bit me a number
of times. (Of course, a module could be in a variable obarray -- but it
shouldn't be by default.)

Andy
-- 
http://wingolog.org/

Re: r6rs libraries

[Julian Graham]

Hi Guilers,

R6RS libraries updates!

* I'm pleased to report that both the balloons example and the
quotient+remainder macro example are working, which I'm going to take
as more or less a vindication of the module environment-based implicit
phasing approach I've been using.  (I didn't quite trust it at first,
but it turns out that when you call `syncase' in the right context, it
does exactly what you want it to, even when it comes to modules!)

* The on-disk directory organization I've been using for the interim
(and around which I've written a rudimentary library load mechanism)
prefixes version numbers with periods.  E.g., `(foo bar baz (6))' ->
foo/bar/.6/baz.scm.  This seems relatively portable and would work
reasonably well for a mapping of library hierarchies onto Guile module
directories -- but I'm no longer sure that libraries and modules need
to co-exist directly.  It might be saner to offer a method of
tranforming a library expression into a regular Guile module by
"wrapping" it with a module declaration and then storing it on disk in
a traditional location -- e.g., the library name:

(foo bar baz (6))

...would be live in

$GUILE_LOAD_PATH/ice-9/r6rs-libraries/foo/bar/baz-6.scm

...and could be imported using either `(import (foo bar baz (6))' or
`(use-modules (ice-9 r6rs-libraries foo bar baz-6))'.  What do people
think?  (Of course, this still leaves open the question of how to
handle version-matching...)

I'm going to clean things up a bit and then I'll post some code.


Regards,
Julian

Re: r6rs libraries

[Ludovic Courtès]

Hi Julian,

Julian Graham <joolean@gmail.com> writes:

> * I'm pleased to report that both the balloons example and the
> quotient+remainder macro example are working, which I'm going to take
> as more or less a vindication of the module environment-based implicit
> phasing approach I've been using.  (I didn't quite trust it at first,
> but it turns out that when you call `syncase' in the right context, it
> does exactly what you want it to, even when it comes to modules!)

Good news!

> * The on-disk directory organization I've been using for the interim
> (and around which I've written a rudimentary library load mechanism)
> prefixes version numbers with periods.  E.g., `(foo bar baz (6))' ->
> foo/bar/.6/baz.scm.

But dot files are traditionally "hidden" on Unices.  Why not go with
`foo/bar/6/baz.scm'?

> (foo bar baz (6))
>
> ...would be live in
>
> $GUILE_LOAD_PATH/ice-9/r6rs-libraries/foo/bar/baz-6.scm
>
> ...and could be imported using either `(import (foo bar baz (6))' or
> `(use-modules (ice-9 r6rs-libraries foo bar baz-6))'.  What do people
> think?  (Of course, this still leaves open the question of how to
> handle version-matching...)

The `r6rs-libraries' part of the module name is just a matter of load
path.  The important thing here is that both `import' and `use-modules'
do the right thing.  Then I would not force R6RS module writers to store
them under `ice-9/r6rs-libraries' as this is inconvenient and doesn't
provide any advantage (AFAICS).

Thanks!

Ludo'.

Re: r6rs libraries

[Julian Graham]

Hi Ludovic,

> But dot files are traditionally "hidden" on Unices.  Why not go with
> `foo/bar/6/baz.scm'?

Because my initial plan was to make it possible to have R6RS libraries
coexist with Guile modules, using the same directory layout and load
system -- and, as per our earlier discussion, numbers can't be part of
module names.

> The `r6rs-libraries' part of the module name is just a matter of load
> path.  The important thing here is that both `import' and `use-modules'
> do the right thing.  Then I would not force R6RS module writers to store
> them under `ice-9/r6rs-libraries' as this is inconvenient and doesn't
> provide any advantage (AFAICS).

Well, it depends on what we plan to support.  Here's what I've got
implemented so far:

`import' (and the `import' specification in the header of a library
expression) can locate Guile modules by delegating to Guile's module
system.  If this fails, the system will attempt to fulfill the import
requirement by loading library expressions from disk as per the
directory layout I described earlier.

What's not clear to me is:

* Should `use-modules' be able to locate and load (and evaluate) R6RS
library expressions?  If so, where should these modules be stores and
how should they be named -- specifically with regard to version
specifiers?

* If both `use-modules' and `import' can load the same R6RS library,
the one of them will need to be extended to understand the syntax of
the other -- i.e., `use-modules' would need to be able to understand
the `library' form or `import' would need to be able to strip off any
"module wrapper" we'd add to a library to make it loadable via
`use-modules'.


Regards,
Julian

Re: r6rs libraries

[Ludovic Courtès]

Hello!

Julian Graham <joolean@gmail.com> writes:

>> But dot files are traditionally "hidden" on Unices.  Why not go with
>> `foo/bar/6/baz.scm'?
>
> Because my initial plan was to make it possible to have R6RS libraries
> coexist with Guile modules, using the same directory layout and load
> system -- and, as per our earlier discussion, numbers can't be part of
> module names.

Hmm, yes, but how about `foo/bar/baz-6.scm'?  Is there a reason to
reject it?

Thanks,
Ludo'..

Re: r6rs libraries

[Julian Graham]

> Hmm, yes, but how about `foo/bar/baz-6.scm'?  Is there a reason to
> reject it?

Well, the part of an R6RS library name that comes before its version
is only restricted in that that it must be an identifier -- so a
system that relied on filenames to locate libraries could have trouble
determining whether a file called ice-9-9.scm contained a library
named `(ice-9 (9))' or one named `(ice (9 9))'.

And this also doesn't address the problem of doing version matching as
part of hybrid module / library load process.  After all, I don't
think there are any restrictions in R6RS on where libraries need to be
stored or how they can be located / loaded, so one thing we could do
is ditch the idea that version needs to go into the filename.  Guile
could require that all installed versions of a particular library
would have to reside in the same file, the name of which would be
constructed of the non-version parts of the library name:

foo/bar/baz.scm would contain the code for:

`(foo bar baz (6))'
`(foo bar baz (6 1))'

and

`(foo bar baz (7))'

Using the module system to load that file, a la `(use-modules (foo bar
baz))' would cause, via a hook in the body of that file, all the
library expressions in that file to be stored in a data structure
accessible by the library system for the purposes of subsequent import
and expansion.


Regards,
Julian

Re: r6rs libraries

[Julian Graham]

[-- Attachment #1: Type: text/plain, Size: 3378 bytes --]

Hi Guilers,

Find attached a very early, definitely broken first draft of R6RS
library support.  I was hoping to send this before / during Libre
Planet '09, but AC outlets were scarce (and boozing at The Red Line so
much easier than hacking) so I'm attempting to send this using the
free WiFi on the Acela back to NYC.

As discussed, this version uses the following approach for mapping
Guile's module search / autoloading mechanism onto locating R6RS
libraries: When a user or a library calls `import' with an import
spec,

1. The import spec is unwrapped until an actual library reference is found.
2. If a matching library reference has already been loaded, an
interface meeting the import spec's requirements is created and
returned.
3. Otherwise, `resolve-interface' is called on the name portion of the
library reference.  If a satisficing interface cannot be loaded, the
import fails.
4. Otherwise, the import system expects that either:

   a. The loaded Guile module has registered one or more versions of a
corresponding R6RS library under the library name in the internal
library registry via the `register-library' or `register-from-path'
functions in `(ice-9 r6rs-libraries)'.  If any versions of the library
can be found in the registry, their versions are matched against the
version in the library reference as per R6RS.

   b. The loaded Guile module is a non-R6RS Guile module (e.g.,
`(guile)' or `(ice-9 syncase)').  In this case, it is wrapped in an
R6RS compatibility layer to make it accessible to the library system.

What this means is that `(ice-9 r6rs-libraries)' can automatically
load normal Guile modules, and that R6RS library expressions don't
need to be modified in order to be loadable.  It does require,
however, that every installed set of versions of an R6RS library have
a Guile module that serves as a catalog of sorts.  This catalog module
file might use `register-library' to register the entire library
expression:

(define-module (mystuff mylibrary)
  #:use-module (ice-9 r6rs-libraries))

(register-library
 '(library (mystuff mylibrary (1 2))
    (export foo)
    (import (mystuff myotherlibrary))

    (define (foo) (display "Hello, world!"))))

...or it could use the search path to register a library expression
from an external file:

(define-module (mystuff mylibrary)
  #:use-module (ice-9 r6rs-libraries))

(register-from-path "mystuff/mylibrary.scm.1")
(register-from-path "mystuff/mylibrary.scm.1.2")


This mechanism might need some tweaking, but I think it resolves some
of the issues we were discussing earlier re: users installing R6RS
libraries having to keep them separate from other Guile modules.

If anyone's interested, I've created catalog modules and library
expressions for the bits and pieces of the core set of R6RS libraries
(including `(rnrs base)') necessary for testing, and I'd be happy to
tar them up and make them available somewhere.


Some other points:

* This draft doesn't do any real validation of the `library' form.

* Phased imports for the `define-syntax' form sort of work, but there
are issues with cross-module visibility of bindings for symbols
included in the output of transformers, as discussed here [1].

* Phased imports for the `letrec-syntax' and `let-syntax' forms don't
work at all yet.


Regards,
Julian


[1] - http://lists.gnu.org/archive/html/guile-user/2009-03/msg00015.html

[-- Attachment #2: r6rs-libraries.scm --]
[-- Type: text/x-scheme, Size: 10861 bytes --]

;;; r6rs-libraries.scm --- Support for R6RS libraries

;; Copyright (C) 2009 Free Software Foundation, Inc.
;;
;; This library is free software; you can redistribute it and/or
;; modify it under the terms of the GNU Lesser General Public
;; License as published by the Free Software Foundation; either
;; version 2.1 of the License, or (at your option) any later version.
;; 
;; This library is distributed in the hope that it will be useful,
;; but WITHOUT ANY WARRANTY; without even the implied warranty of
;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
;; Lesser General Public License for more details.
;; 
;; You should have received a copy of the GNU Lesser General Public
;; License along with this library; if not, write to the Free Software
;; Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA

;;; Author: Julian Graham <julian.graham@aya.yale.edu>
;;; Date: 2009-03-05

;;; Commentary:

(define-module (ice-9 r6rs-libraries)
  :use-module (ice-9 optargs)
  :use-module (ice-9 receive)
  :use-module (ice-9 syncase)
  :use-module (srfi srfi-1)
  :use-module (srfi srfi-2)
  :use-module (srfi srfi-9)
  :use-module (srfi srfi-11)

  :export (import 
	   register-library
	   register-from-path))

(define (library-name library) (list-ref library 1))
(define (library-exports library) (cdr (list-ref library 2)))
(define (library-imports library) (cdr (list-ref library 3)))
(define (library-body library) (cddddr library))

(define library-registry (make-hash-table))
(define interface-registry (make-hash-table))

(define-record-type r6rs-library-type
  (make-r6rs-library name)
  r6rs-library?
  (name r6rs-library-name)
  (exports r6rs-library-exports set-r6rs-library-exports!)
  (module r6rs-library-module set-r6rs-library-module!)
  (interface r6rs-library-interface set-r6rs-library-interface!)
  (parent r6rs-library-parent set-r6rs-library-parent!))

(define* (create-empty-module #:optional name)
  (define module (make-module))
  (set-module-name! module (or name (list (gensym))))
  (let ((interface (make-module)))
    (set-module-name! interface (module-name module))
    (set-module-kind! interface 'interface)
    (set-module-public-interface! module interface))
  module)

(define* (create-module-interface module exports #:optional name)
  (let ((i (make-module)))
    (set-module-name! i (or name (list (gensym))))
    (set-module-kind! i 'custom-interface)
    (for-each 
     (lambda (import)
       (if (pair? import)
	   (module-add! i (cadr import) (module-variable module (car import)))
	   (module-add! i import (module-variable module import))))
     exports)
    i))

(define* (create-library-interface lib exports #:optional name)
  (let* ((name (or name (list (gensym))))
	 (l (make-r6rs-library name)))
    (set-r6rs-library-interface! 
     l (create-module-interface (r6rs-library-interface lib) exports name))
    (set-r6rs-library-exports! 
     l (map (lambda (x) (if (pair? x) (cadr x) x)) exports))
    (set-r6rs-library-parent! l lib)
    l))

(define (instantiate-library lib-expr)
  (define name (library-name lib-expr))
  (define imports (library-imports lib-expr))
  (define exports 
    (fold (lambda (x lst) (append lst (if (pair? x) (cdr x) (list x))))
	  '()
	  (library-exports lib-expr)))

  (define library (make-r6rs-library (library-name lib-expr)))
  (define module (create-empty-module))

  (define (binding-name binding) (if (list? binding) (car binding) binding))

  (define local-definitions (list 'exports))

  (define phase-library-cache (make-hash-table))
  (define (inject-bindings-for-phase! m phase)
    (and=> (hashv-ref phase-library-cache phase)
	   (lambda (ifaces) (for-each (lambda (i) (module-use! m i))
				      (map r6rs-library-interface ifaces)))))

  (define (expand expr m p)
    (if (pair? expr)
	(let ((ce (car expr)))
	  (cond ((eq? ce 'define) 
		 (eval (cons* ce (cadr expr) (expand (cddr expr) m p)) m))
		((eq? ce 'define-syntax)
		 (let ((m+ (create-empty-module))
		       (p (+ p 1)))
		   (inject-bindings-for-phase! m+ p)
		   (let ((rhs (eval (expand (caddr expr) m+ p) m+)))
		     (let ((e `(define-syntax ,(cadr expr) ,rhs)))
		       (eval e m)))))
		(else (map (lambda (x) (expand x m p)) expr))))
	expr))
    
  (define (hashv-append! h k v)
    (or (and=> (hashv-ref h k) (lambda (ov) (hashv-set! h k (append ov `(,v)))))
	(hashv-set! h k `(,v))))

  (for-each 
   (lambda (import) 
     (let* ((import-set (if (eq? (car import) 'for) (cadr import) import))
	    (interface (import-library import-set)))
       (if (equal? (r6rs-library-name interface) '(rnrs (6)))
	   (begin (hashv-append! phase-library-cache 0 interface)
		  (hashv-append! phase-library-cache 1 interface)))
       (if (eq? (car import) 'for)
	   (for-each
	    (lambda (phase)
	      (cond ((eq? phase 'run)
		     (hashv-append! phase-library-cache 0 interface))
		    ((eq? phase 'expand)
		     (hashv-append! phase-library-cache 1 interface))
		    ((and (list? phase) (eq? (car phase 'meta)))
		     (hashv-append! phase-library-cache (cadr phase) interface))
		    (else (error "Invalid import level specification"))))
	    (cddr import))
	   (hashv-append! phase-library-cache 0 interface))))
   imports)

  (inject-bindings-for-phase! module 0)
  (for-each (lambda (expr)
	      (if (and (list? expr) (memq (car expr) '(define define-syntax)))
		  (begin (expand expr module 0)
			 (append! local-definitions 
				  (list (binding-name (cadr expr)))))
		  (eval expr module)))
	    (library-body lib-expr))

  (let ((locals (cdr local-definitions)))
    (receive 
      (export-vars export-names)
      (let f ((vars (list)) (names (list)) (lst exports))
	(cond ((null? lst) (values vars names))
	      ((pair? (car lst))
	       (f (cons (caar lst) vars) (cons (cadar lst) names) (cdr lst)))
	      (else (f (cons (car lst) vars)
		       (cons (car lst) names)
		       (cdr lst)))))
      (if (not (null? locals))
	  (module-export! module locals))
      (module-re-export! module (lset-difference eq? export-vars locals))
      (let ((module-interface (create-module-interface module exports name)))
	(set-module-public-interface! module module-interface)
	(set-r6rs-library-module! library module)
	(set-r6rs-library-interface! library module-interface)
	(set-r6rs-library-exports! library export-names))))
  (hash-set! interface-registry name library))

(define (version-matches? version-ref target)
  (define (sub-versions-match? v-refs t)
    (define (sub-version-matches? v-ref t)
      (define (curried-sub-version-matches? v) (sub-version-matches? v t))
      (cond ((number? v-ref) (eqv? v-ref t))
	    ((list? v-ref)
	     (let ((cv (car v-ref)))
	       (cond ((eq? cv '>=) (>= t (cadr v-ref)))
		     ((eq? cv '<=) (<= t (cadr v-ref)))
		     ((eq? cv 'and) 
		      (every curried-sub-version-matches? (cdr v-ref)))
		     ((eq? cv 'or)
		      (any curried-sub-version-matches? (cdr v-ref)))
		     ((eq? cv 'not) (not (sub-version-matches? (cadr v-ref) t)))
		     (else (error "Incompatible sub-version reference" cv)))))
	    (else (error "Incompatible sub-version reference" v-ref))))
    (or (null? v-refs)
	(and (not (null? t))
	     (sub-version-matches? (car v-refs) (car t))
	     (sub-versions-match? (cdr v-refs) (cdr t)))))
  (define (curried-version-matches? v) (version-matches? v target))
  (or (null? version-ref)
      (let ((cv (car version-ref)))
	(cond ((eq? cv 'and) (every curried-version-matches? (cdr version-ref)))
	      ((eq? cv 'or) (any curried-version-matches? (cdr version-ref)))
	      ((eq? cv 'not) (not version-matches? (cadr version-ref) target))
	      (else (sub-versions-match? version-ref target))))))

(define (import-library import-spec)
  (define (wrap-guile-module module)
    (let ((li (make-r6rs-library (module-name module))))
      (set-r6rs-library-module! li module)
      (set-r6rs-library-interface! li module)
      (set-r6rs-library-exports! 
       li (hash-map->list (lambda (x y) x) (module-obarray module)))
      li))

  (define (locate-library library-reference)
    (receive
      (name version)
      (partition symbol? library-reference)
      (let ((interface (false-if-exception (resolve-interface name))))
	(or (hash-ref interface-registry library-reference)
	    (and-let* ((version-table (hash-ref library-registry name))
		       (cversion (if (null? version) version (car version))))
	      (or (and=> (assoc cversion version-table version-matches?)
			 (lambda (x) (instantiate-library (cdr x))))
		  (error "No version of library found to match version-ref"
			 name
			 cversion
			 version-table)))
            (and interface (hash-set! interface-registry 
				      library-reference 
				      (wrap-guile-module interface)))
	    (error "Unable to resolve interface for library" 
		   library-reference)))))

  (define (resolve-library-interface import)
    (let ((ci (car import)))
      (cond 
       ((eq? ci 'library) (locate-library (cadr import)))
       ((or (eq? ci 'only) (eq? ci 'rename))
	(create-library-interface (resolve-library-interface (cadr import))
				  (cddr import)))
       ((eq? ci 'except) 
	(let ((i (resolve-library-interface (cadr import))))
	  (create-library-interface 
	   i (lset-difference eq? (r6rs-library-exports i) (cddr import)))))
       ((eq? ci 'prefix)
	(let* ((i (resolve-library-interface (cadr import)))
	       (prefix-str (symbol->string (caddr import))))
	  (create-library-interface
	   i (map (lambda (x) 
		    (cons x (list (string->symbol 
				   (string-append prefix-str 
						  (symbol->string x))))))
		  (r6rs-library-exports i)))))
       (else (locate-library import)))))
  (resolve-library-interface import-spec))

(define (import import-spec)
  (let ((lib (import-library import-spec)))
    (or lib (error "Unable to import library for import spec " import-spec))
    (module-use! (current-module) (r6rs-library-interface lib))))

(define (register-library library-expr)
  (define (version-less? x y)
    (cond ((null? x) #f)
	  ((null? y) #t)
	  (else (let ((cx (car x))
		      (cy (car y)))
		  (cond ((< cx cy) #t)
			((> cx cy) #f)
			(else (version-less? (cdr x) (cdr y))))))))
  (receive 
    (name version)
    (partition symbol? (library-name library-expr))
    (let ((cversion (if (null? version) version (car version))))
      (or (and=> (hash-ref library-registry name)
		 (lambda (version-table)
		   (merge! version-table 
			   `(,(cons cversion library-expr))
			   version-less?)))
	  (hash-set! library-registry name `(,(cons cversion library-expr)))))))

(define (register-from-path filename)
  (define (quoting-read port)
    (let ((library-expr (read port)))
      (if (eof-object? library-expr)
	  library-expr
	  (begin (register-library library-expr) *unspecified*))))
  (with-fluids ((current-reader quoting-read)) (load-from-path filename)))