r6rs libraries (was Re: role of guile-lib)

r6rs libraries (was Re: role of guile-lib)

[Julian Graham]

Hey Neil,

>> Hey, sure -- although we'd still need a way to locate modules.  And
>> could we actually rely on other distributions?  My outsider impression
>> is that there was a minor revolt when R6RS was passed (but maybe the
>> library system was less offensive to people?).
>
> IMO no; I'd say the library system is one of the more offensive parts
> of R6RS.  Even so, I wouldn't object to someone trying to hack in a
> way of loading R6RS libraries into Guile.

My inclination is that because Guile's module system already has
almost all of the functionality of R6RS libraries, it might make sense
to support libraries by transforming them into module definitions.
Unfortunately, there's that nasty import levels thing to deal with.
It seems like that aspect of the library spec is attempting to resolve
some issues that (I think) Guile struggled with a while ago, re: the
handling of symbols in macros across module boundaries.  What's the
status of that?


Regards,
Julian

Re: r6rs libraries

[Andy Wingo]

Hi Julian,

On Sun 28 Dec 2008 02:05, "Julian Graham" <joolean@gmail.com> writes:

> My inclination is that because Guile's module system already has
> almost all of the functionality of R6RS libraries, it might make sense
> to support libraries by transforming them into module definitions.
> Unfortunately, there's that nasty import levels thing to deal with.
> It seems like that aspect of the library spec is attempting to resolve
> some issues that (I think) Guile struggled with a while ago, re: the
> handling of symbols in macros across module boundaries.  What's the
> status of that?

Yes, some thought is required here.

 1) psyntax.ss should probably be replaced (in part?) with an
    implementation of R6RS macros, which have some extensions relative
    to R5RS, but otherwise are compatible afaik

 2) that replacement should Do The Right Thing regarding expansion of
    the r6rs `library' form: i.e. expand `library' into something based
    on Guile's modules if run in r6rs mode, and leave it alone otherwise

 3) that replacement should be module-aware, expanding unbound symbols
    to (@@ MOD SYM) forms as appropriate

There are two implementations of r6's macros afaik; we should probably
use the one that Ikarus and Chez use, because it doesn't require
multiple instantiation for modules. (The PLT one is problematic in this
regard; see
http://groups.google.com/group/comp.lang.scheme/browse_thread/thread/8927053ede92fd27
for a long ramble on this).

Here's a reasonably straigtforward explanation of the syncase algorithm:

    http://www.cs.indiana.edu/~dyb/pubs/bc-syntax-case.pdf

Here's a link to Aziz' implementation:

    http://www.cs.indiana.edu/~aghuloum/r6rs-libraries/

I'd like to get around to this at some point, if you decide to punt. But
that's probably not directly relevant to your question on guile-lib ;-)

Andy
-- 
http://wingolog.org/

Re: r6rs libraries

[Julian Graham]

Hi Andy,

>  1) psyntax.ss should probably be replaced (in part?) with an
>    implementation of R6RS macros, which have some extensions relative
>    to R5RS, but otherwise are compatible afaik

Yikes -- I'm in a bit of a maze of twisty little passages here.
Aziz's implementation doesn't support Guile out of the box, and, while
I was certainly prepared to hack Guile support in, it seems like it
assumes a fair amount of R6RS to be already present in the host
system.  Neither `(rnrs base)' nor any of the other libraries it's
asking for look to be too difficult to package out of Guile's existing
functionality, but it seems like we also need full `import' semantics
as well just to bootstrap it.

So maybe I'm being dense here, but it looks like the way forward might
be to whip up a quick module-to-library mapping system (i.e.,
something like (foo bar (version)) -> /foo/bar/version) and then start
working on piecemeal attempts to get imports working?  Like, in the
following order, I'd say we could tackle:

1) Mapping the existing features of Guile's module system onto their
R6RS counterparts
2) Implementing version selection
3) Implementing import levels


> I'd like to get around to this at some point, if you decide to punt. But
> that's probably not directly relevant to your question on guile-lib ;-)

Right -- well, as far as that goes, Neil did seem amenable to adding
it to "Guile proper."  Any opinion on how much of guile-lib should be
included (and what should be done with the rest) and how it should be
organized?  As I've mentioned, I've got some personal "must-haves"
(statprof, SSAX, etc.).

(And I was just trying to fork this discussion, not execve it!  ;-))


Regards,
Julian

Re: r6rs libraries

[Julian Graham]

Happy New Year, Guilers!


> So maybe I'm being dense here, but it looks like the way forward might
> be to whip up a quick module-to-library mapping system (i.e.,
> something like (foo bar (version)) -> /foo/bar/version) and then start
> working on piecemeal attempts to get imports working?  Like, in the
> following order, I'd say we could tackle:
>
> 1) Mapping the existing features of Guile's module system onto their
> R6RS counterparts
> 2) Implementing version selection
> 3) Implementing import levels

That last email was written between drinking at a bar and going to
bed.  The *proper* mapping for versions would be more like:

(foo bar (version)) -> (foo version bar)

So if your R6RS library name was `(my r6rs library (1 2 3))', then the
corresponding Guile module name would be `(my r6rs 1 2 3 library)'.

I had a little time this afternoon to play around with some syntax
transformations for converting between libraries and modules, which
led to the following observations:

* Any library-to-module transformations we could do would be one-way
(i.e., libraries to modules), since R6RS requires that any non-binding
expressions have to come after the bindings in a library; rewriting a
module to meet that requirement would be non-trivial.

* Module definitions created from R6RS libraries need to use the #:pure keyword.

* There's a somewhat perplexing R6RS requirement to allow renaming
symbols on export, similar to the way Guile currently supports
renaming them on import.  I don't see a way to do this without
modifications to the core.

* I think it'd be possible to implement R6RS "import levels" using
Guile's existing #:export, #:export-syntax, #:reexpport, and
#:reexport-syntax keywords -- if it weren't for the `(meta <level>)'
option for import specifiers.  It might still be possible, given the
allowances of section 7.2.  I don't know, I'm still trying to figure
that out.

Other than the above, it looks like a lot of the heavy lifting has
already been done by the module system.


Regards,
Julian

Re: r6rs libraries

[Ludovic Courtès]

Hello and happy new year!  :-)

"Julian Graham" <joolean@gmail.com> writes:

> * I think it'd be possible to implement R6RS "import levels" using
> Guile's existing #:export, #:export-syntax, #:reexpport, and
> #:reexport-syntax keywords -- if it weren't for the `(meta <level>)'
> option for import specifiers.  It might still be possible, given the
> allowances of section 7.2.  I don't know, I'm still trying to figure
> that out.

The problem is that Guile has *no* phase separation whatsoever: macros
are evaluated at run-time, in the same environment as actual code that's
executed.

This is different in the `vm' branch where, of course, macros are
expanded at compile-time in a compile-time environment.  However, I
don't think the compiler supports phase specifiers in `define-module'
(e.g., `#:export-syntax' and `#:use-syntax' to export/import for
compile-time, etc.).  Andy?

Thanks,
Ludo'.

Re: r6rs libraries

[Andy Wingo]

Hello,

On Tue 06 Jan 2009 00:40, ludo@gnu.org (Ludovic Courtès) writes:

> Hello and happy new year!  :-)

Yes yes! 

> "Julian Graham" <joolean@gmail.com> writes:
>
>> * I think it'd be possible to implement R6RS "import levels" using
>> Guile's existing #:export, #:export-syntax, #:reexpport, and
>> #:reexport-syntax keywords -- if it weren't for the `(meta <level>)'
>> option for import specifiers.  It might still be possible, given the
>> allowances of section 7.2.  I don't know, I'm still trying to figure
>> that out.
>
> The problem is that Guile has *no* phase separation whatsoever: macros
> are evaluated at run-time, in the same environment as actual code that's
> executed.

I think we still have what we need for R6RS.

PLT's implementation provides a fresh environment at each meta-level;
thus side effects from one level can't affect other levels. This sounds
nice in theory, but it's not required by R6, and the reason is this:

  Consider an R6RS library that implements `current-output-port'.
  Normally these ports are buffered, so as not to incur syscall overhead
  on each character written. So there is some buffer that is
  side-effected when the port is written to.

  Now consider one library that writes things on the
  `current-output-port' *when the module is initialized*, and one main
  program that writes things on `current-output-port' from level 0. If
  each level has its *own* `current-output-port', there will be multiple
  buffers -- thus (in the absence of level finalizers -- and that can
  get quite complicated) the buffers will be flushed as the ports get
  collected, meaning out-of-order output for something that was clearly
  deterministic.

This might seems contrived, but the need to share state between import
levels is something fundamental, I think, when you get to the level of
language *implementation*. And if you can't implement a
multiple-instantiation R6RS in pure multiple-instantiation R6RS, well,
what a language...

Neither Dybvig & Ghuloum's nor van Tonder's library implementations do
multiple instantiation, AFAIK. PLT's is the only one that does.

Regards,

Andy
-- 
http://wingolog.org/

Re: r6rs libraries

[Julian Graham]

> Neither Dybvig & Ghuloum's nor van Tonder's library implementations do
> multiple instantiation, AFAIK. PLT's is the only one that does.


Sort of pursuant to this, I'm looking at André van Tonder's
implementation with an eye towards modifying it to convert R6RS
library definitions to Guile module definitions at load time.  (I find
his pitch [1] for his code over the Dybvig-Ghuloum implementation
compelling -- plus the code is an order of magnitude smaller.)

One small thing I noticed while playing around is that he uses
`scheme-report-environment' for looking up certain R5RS functions, but
that function isn't available in Guile outside of the REPL without
doing `(use-modules (ice-9 r5rs))' first.  This limitation has been
acknowledged before [2]; any reason this module isn't loaded all the
time?


Regards,
Julian


[1] - http://lists.r6rs.org/pipermail/r6rs-discuss/2007-October/003387.html
[2] - http://www.mail-archive.com/bug-guile@gnu.org/msg04366.html

Re: r6rs libraries

[Ludovic Courtès]

Hi!

"Julian Graham" <joolean@gmail.com> writes:

> One small thing I noticed while playing around is that he uses
> `scheme-report-environment' for looking up certain R5RS functions, but
> that function isn't available in Guile outside of the REPL without
> doing `(use-modules (ice-9 r5rs))' first.  This limitation has been
> acknowledged before [2]; any reason this module isn't loaded all the
> time?

There are other missing R5RS pieces by default, notably `syntax-rules'.
However, the manual clearly states it in (info "(guile) Environments")
and (info "(guile) Syntax Rules").  So I would say that the current
behavior is OK.

Thanks,
Ludo'.