r6rs standard libraries

r6rs standard libraries

[Julian Graham]

Hi Guilers,

As mentioned in an earlier email [0], I've been working on
implementations for the libraries that make up the so-called R6RS
Standard Libraries [1], along with test suites.  Where possible, I've
tried to wrap existing Guile functionality instead of writing things
from scratch.  Speaking optimistically, I think I'm about 80% of the
way there (minus the test cases).  There are quite a few files, so, to
facilitate review, I've uploaded them to my web site [2].  There's
also an updated copy of the `(ice-9 r6rs-libraries)' module there,
which resolves a couple of issues present in the most recent version I
submitted to the list.

What do people think?

One point that may be of interest is that I've chosen to provide the
code in the form of R6RS libraries rather than Guile modules.  My
thinking is that the primary utility of most of these libraries lies
in the cross-Scheme compatibility they provide, and thus they're not
of particular interest to users writing code intended only for Guile.
In cases where there's actual new functionality, such as with the
bytevector features, a Guile module representation is probably more
useful.


Regards,
Julian

[0] - http://www.mail-archive.com/guile-devel@gnu.org/msg04884.html
[1] - http://www.r6rs.org/final/html/r6rs-lib/r6rs-lib.html
[2] - http://undecidable.net/joolean/code/scheme

Re: r6rs standard libraries

[Andy Wingo]

Hi,

On Sun 24 Jan 2010 18:53, Julian Graham <joolean@gmail.com> writes:

> As mentioned in an earlier email [0], I've been working on
> implementations for the libraries that make up the so-called R6RS
> Standard Libraries [1], along with test suites.  Where possible, I've
> tried to wrap existing Guile functionality instead of writing things
> from scratch.  Speaking optimistically, I think I'm about 80% of the
> way there (minus the test cases).  There are quite a few files, so, to
> facilitate review, I've uploaded them to my web site [2].  There's
> also an updated copy of the `(ice-9 r6rs-libraries)' module there,
> which resolves a couple of issues present in the most recent version I
> submitted to the list.
>
> What do people think?

Excellent hacking! I think this is fantastic. I want to get on
http://www.ccs.neu.edu/home/will/Larceny/benchmarksGenuineR6Linux.html.
I did some tests with Clinger's R5RS benchmarks, showing pretty good
performance for Guile, but I think he's only really working on the R6RS
ones now. I would like to smoke a couple of other Schemes ;-)

> One point that may be of interest is that I've chosen to provide the
> code in the form of R6RS libraries rather than Guile modules.  My
> thinking is that the primary utility of most of these libraries lies
> in the cross-Scheme compatibility they provide, and thus they're not
> of particular interest to users writing code intended only for Guile.
> In cases where there's actual new functionality, such as with the
> bytevector features, a Guile module representation is probably more
> useful.

Agreed.

Personally I'm ready to merge this code whenever you want, as long as it
has tests. A section in the manual about what we implement of R6RS and
what the differences are would be useful as well.

I also think we should have `library' and `import' in the basic
environment. In order to do this I would like for them in boot-9.scm,
not in a separate module, and implemented in terms of syntax-case.

Implementing `library' in terms of syntax-case will allow `library' to
expand out to (begin (define-module (foo) #:pure) (import a ...) ...),
which allows imports to be processed only in the `import' macro. The
`import' macro will be resolved in `(guile)' even though the module is
pure, due to hygiene. No toplevel-defined helper functions will be
necessary.

Does this sound sensible to you?

If so, can you make a git branch that (1) adds `library' and `import' to
boot-9, then adds the other modules, one-by-one with tests? I've got my
finger on the merge trigger :)

Cheers,

Andy
-- 
http://wingolog.org/

Re: r6rs standard libraries

[Ludovic Courtès]

Hello Guilers,

Andy Wingo <wingo@pobox.com> writes:

> On Sun 24 Jan 2010 18:53, Julian Graham <joolean@gmail.com> writes:
>
>> As mentioned in an earlier email [0], I've been working on
>> implementations for the libraries that make up the so-called R6RS
>> Standard Libraries [1], along with test suites.  Where possible, I've
>> tried to wrap existing Guile functionality instead of writing things
>> from scratch.  Speaking optimistically, I think I'm about 80% of the
>> way there (minus the test cases).  There are quite a few files, so, to
>> facilitate review, I've uploaded them to my web site [2].  There's
>> also an updated copy of the `(ice-9 r6rs-libraries)' module there,
>> which resolves a couple of issues present in the most recent version I
>> submitted to the list.
>>
>> What do people think?
>
> Excellent hacking!

+1 :-)

>> One point that may be of interest is that I've chosen to provide the
>> code in the form of R6RS libraries rather than Guile modules.  My
>> thinking is that the primary utility of most of these libraries lies
>> in the cross-Scheme compatibility they provide, and thus they're not
>> of particular interest to users writing code intended only for Guile.
>> In cases where there's actual new functionality, such as with the
>> bytevector features, a Guile module representation is probably more
>> useful.
>
> Agreed.

Though if ‘library’ expands to ‘define-module’, R6RS modules will also
be readily usable as if they were “regular” guile modules, right?

> I also think we should have `library' and `import' in the basic
> environment.

The global environment is already a crowded place, and having these
forms globally may make it harder to maintain backward compatibility in
the future, should R15RS have an incompatible ‘library’ form, for
instance.

How about having a special module and a command-line option, just like
we have ‘--use-srfi’?

Thanks,
Ludo’.

Re: r6rs standard libraries

[Andy Wingo]

Greets,

On Mon 25 Jan 2010 18:37, ludo@gnu.org (Ludovic Courtès) writes:

> Andy Wingo <wingo@pobox.com> writes:
>
>> On Sun 24 Jan 2010 18:53, Julian Graham <joolean@gmail.com> writes:
>>
>>> One point that may be of interest is that I've chosen to provide the
>>> code in the form of R6RS libraries rather than Guile modules.
>
> Though if ‘library’ expands to ‘define-module’, R6RS modules will also
> be readily usable as if they were “regular” guile modules, right?

`library' does expand out to `define-module', and yes R6RS modules are
usable as "regular" guile modules.

>> I also think we should have `library' and `import' in the basic
>> environment.
>
> The global environment is already a crowded place, and having these
> forms globally may make it harder to maintain backward compatibility in
> the future, should R15RS have an incompatible ‘library’ form, for
> instance.
>
> How about having a special module and a command-line option, just like
> we have ‘--use-srfi’?

I'm sympathetic, but I think you're overstating the situation. New
Scheme revisions don't come but every 5 or 10 years. I'm sure we've
already broken with R3RS. We can always provide specific
back-compatibility modules, but the default should work, if possible,
with the current state of Scheme. (Assuming we can implement R6RS, that
is.)

Beyond that, we're just talking about two identifiers. R6RS modules
themselves *explicitly* declare what they import, so maintaining compat
for them is much easier than maintaining compat for Guile modules, at
least Guile modules that aren't #:pure.

Yes, the names are a burden on the namespace, but I don't think they're
"creeping", in the sense that they will expand in the future.

If you're dead-set against r6rs support by default, that's cool. But if
not, have I managed to convince you that we should have these by
default? :)

Cheers,

Andy
-- 
http://wingolog.org/

Re: r6rs standard libraries

[Ludovic Courtès]

Hello,

Andy Wingo <wingo@pobox.com> writes:

[...]

> Beyond that, we're just talking about two identifiers.

[...]

> Yes, the names are a burden on the namespace, but I don't think they're
> "creeping", in the sense that they will expand in the future.
>
> If you're dead-set against r6rs support by default, that's cool. But if
> not, have I managed to convince you that we should have these by
> default? :)

Well, yes.  :-)

I don’t have a strong opinion, and you’re probably right on these.

Thanks,
Ludo’.

Re: r6rs standard libraries

[Andy Wingo]

Hi,

On Tue 26 Jan 2010 15:35, ludo@gnu.org (Ludovic Courtès) writes:

> Andy Wingo <wingo@pobox.com> writes:
>
>> have I managed to convince you that we should have these by
>> default? :)
>
> Well, yes.  :-)

Rereading my mail I have to apologize for the "hard sell". I do think
`library' and `import' in the default namespace is the right thing,
though.

Cheers,

Andy
-- 
http://wingolog.org/

Re: r6rs standard libraries

[Neil Jerram]

Andy Wingo <wingo@pobox.com> writes:

> Rereading my mail I have to apologize for the "hard sell". I do think
> `library' and `import' in the default namespace is the right thing,
> though.

FWIW, I agree that this `out of the box' support for R6RS libraries is
pretty cool.  And as you say, 2 identifiers is not very many.  There's
plenty other stuff in boot-9 that's less important, and which I can
imagine us moving into a module, and telling people `now you have to add
a (use-modules (ice-9 so-and-so))', or maybe use autoload.

      Neil

Re: r6rs standard libraries

[Grant Rettke]

On Thu, Jan 28, 2010 at 3:54 PM, Neil Jerram <neil@ossau.uklinux.net> wrote:
> Andy Wingo <wingo@pobox.com> writes:
>
>> Rereading my mail I have to apologize for the "hard sell". I do think
>> `library' and `import' in the default namespace is the right thing,
>> though.
>
> FWIW, I agree that this `out of the box' support for R6RS libraries is
> pretty cool.

It is very cool!

Re: r6rs standard libraries

[Julian Graham]

Hi Andy,


> I also think we should have `library' and `import' in the basic
> environment. In order to do this I would like for them in boot-9.scm,
> not in a separate module, and implemented in terms of syntax-case.
>
> Implementing `library' in terms of syntax-case will allow `library' to
> expand out to (begin (define-module (foo) #:pure) (import a ...) ...),
> which allows imports to be processed only in the `import' macro. The
> `import' macro will be resolved in `(guile)' even though the module is
> pure, due to hygiene. No toplevel-defined helper functions will be
> necessary.
>
> Does this sound sensible to you?

It sure does.  (Sorry about the delay in replying -- I haven't had a
lot of time for Guile this week, plus I wanted to let everyone resolve
their thinking about boot-9.)  Just a warning, though: I may need some
syntax-case hand-holding.


> If so, can you make a git branch that (1) adds `library' and `import' to
> boot-9, then adds the other modules, one-by-one with tests? I've got my
> finger on the merge trigger :)

Will do.


Regards,
Julian

Re: r6rs standard libraries

[Julian Graham]

Hi everyone,


> If so, can you make a git branch that (1) adds `library' and `import' to
> boot-9, then adds the other modules, one-by-one with tests? I've got my
> finger on the merge trigger :)

Done.  Take a look at 'wip-r6rs-libraries'.

Here's a summary of the changes so far:

* The `library' and `import' macros have been added to boot-9.scm.
* `library' has been rewritten to include library imports via the
`import' macro.
* The two macros no longer share code in the form of external helper functions.
* They no longer use the `defmacro' wrapper around `syntax-case',
which makes them slightly more elegant.

Apologies for the delay -- the above, combined with the fact the set
of functions hygienically available in boot-9 is more limited than
what you can pull into a module, added some time to this.

Please take a look!  I'm not totally convinced that this
implementation plays as nicely as the old one with multi-level
dependency graphs.  I'll be adding some of the more complete R6RS
standard libraries and test cases soon.


Regards,
Julian

Re: r6rs standard libraries

[Andy Wingo]

On Sat 06 Feb 2010 18:50, Julian Graham <joolean@gmail.com> writes:

> Hi everyone,
>
>
>> If so, can you make a git branch that (1) adds `library' and `import' to
>> boot-9, then adds the other modules, one-by-one with tests? I've got my
>> finger on the merge trigger :)
>
> Done.  Take a look at 'wip-r6rs-libraries'.
>
> Please take a look!

Will do, and thanks!!

Andy
-- 
http://wingolog.org/

Re: r6rs standard libraries

[Julian Graham]

Hi Andy,


> Will do, and thanks!!

No problem!

I'm running into some trouble, though, and I could use your (or
somebody's) help.  I'm a little shaky on the details with regard to
the syntax-case expansion process, but I think the issue is that
during the expansion of a single top-level expression, the expansion
environment is constant.  So the following (which is approximately
what "native" Guile modules do):

(define-module (my-module))
(use-modules ((my-syntax-transformer)))
[transformable-syntax]

...will properly expand and compile, since the expansion environment
for [transformable-syntax] reflects the updates to the module-uses
list made by `use-modules', whereas the following (which is
approximately what our R6RS library transformer produces):

(begin
  (define-module (my-module))
  (use-modules ((my-syntax-transformer)))
  [transformable-syntax])

...won't properly expand macros whose transformers are defined in the
`(my-syntax-transformer)' module.

Does this sound plausible?  I spent most of this past weekend trying
to grok this and work around it (via various `eval'-based
incantations) without much luck.  (The weird part is, I don't recall
this happening with my old `(ice-9 r6rs-libraries)' module
implementation, but I could be wrong.)


Regards,
Julian

Re: r6rs standard libraries

[Julian Graham]

Hello again,


> Does this sound plausible?  I spent most of this past weekend trying
> to grok this and work around it (via various `eval'-based
> incantations) without much luck.  (The weird part is, I don't recall
> this happening with my old `(ice-9 r6rs-libraries)' module
> implementation, but I could be wrong.)

In case it's helpful, I've distilled the problem down to the following
self-contained example:

(define-module (foo))
(define-syntax foo
  (lambda (stx)
    (syntax-case ((_ x) (syntax (display x))))))
(foo 'bar)

...which has the side-effect of outputting the string "'bar" when
loaded from disk, whereas the following:

(begin
  (define-module (foo))
  (define-syntax foo
    (lambda (stx)
      (syntax-case ((_ x) (syntax (display x))))))
  (foo 'bar))

...produces the following error:

  ERROR: In procedure vm-debug-engine:
  ERROR: Wrong type to apply: #<syntax-transformer foo>

Note that the `define-module' form at the top is related to the error
-- without it, both examples work just fine.  (Wild speculation: Is it
possible that in the context of changes to the "current module" that
`define-syntax' puts bindings in a place different from where the
expander is looking for them?)


Regards,
Julian

Re: r6rs standard libraries

[Julian Graham]

Hi all,


> ...
>
> Note that the `define-module' form at the top is related to the error
> -- without it, both examples work just fine.  (Wild speculation: Is it
> possible that in the context of changes to the "current module" that
> `define-syntax' puts bindings in a place different from where the
> expander is looking for them?)

Some (potentially misguided) digging around in psyntax.scm suggests
that changes to the current module (via `eval-when' forms, e.g.)
indeed are not properly reflected in the expansion environment within
"chi-top-sequence" because of re-use of the hygiene structure (or
"mod" as it's often referred to in psyntax).  The problem goes away
when I construct a fresh mod for expansion of each expression in the
sequence:

(define chi-top-sequence
  (lambda (body r w s m esew mod)
    (build-sequence s
      (let dobody ((body body) (r r) (w w) (m m) (esew esew) (mod mod))
        (if (null? body)
          '()
          (let ((first (chi-top (car body) r w m esew (cons 'hygiene
(module-name (current-module))))))
            (cons first (dobody (cdr body) r w m esew (cons 'hygiene
(module-name (current-module)))))))))))

But that doesn't strike me as being the whole story.  Anyone have some
cycles to help me figure what the right thing to do is here?


Thanks,
Julian