PEG Parser Updates/Questions

PEG Parser Updates/Questions

[Michael Lucy]

I've officially eliminated the last define-macro expression.

However, I get the feeling that things may not be exactly as desired.
The original program made extensive use of functions in building the
macros, and I originally tried to replace these with macros.  This
turned out to be a little difficult to debug, however (read: I was
unable to make the code actually work).  I eventually abandoned this
and just made datum->syntax calls.

On the one hand, this works.  I also find it easier to debug, and I
think it looks cleaner.

The downside is that one doesn't get all the same benefits of
referential transparency, so I still have gensyms in the functions
etc.  Is this a problem?

If so, I can definitely replace everything with macros, but I might
not be able to do that and get everything else done by the GSOC
project deadline.  I'd like to hang around after the project is
officially done from Google's point of view to polish things up, so I
could also do it then.

Another question about module namespaces:  I have some syntax that I'd
like to be available to code generated by macros in my module, but
which I'd rather not export to the user (to avoid clobbering their
functions).  Is there a standard way of doing this?  I can't seem to
find anything in the module documentation regarding giving namespaces
to things in modules except for :renamer, which has to be done by the
user--the only options appear to be not exporting it at all, or
exporting it straight into the user's namespace.  The best fix I can
think of is naming the syntax things the user is unlikely to ever take
(or maybe using gensyms to make sure it isn't a name they take).

Re: PEG Parser Updates/Questions

[No Itisnt]

On Wed, Jul 28, 2010 at 12:13 AM, Michael Lucy <MichaelGLucy@gmail.com> wrote:

> Another question about module namespaces:  I have some syntax that I'd
> like to be available to code generated by macros in my module, but
> which I'd rather not export to the user (to avoid clobbering their
> functions).  Is there a standard way of doing this?  I can't seem to
> find anything in the module documentation regarding giving namespaces
> to things in modules except for :renamer, which has to be done by the
> user--the only options appear to be not exporting it at all, or
> exporting it straight into the user's namespace.  The best fix I can
> think of is naming the syntax things the user is unlikely to ever take
> (or maybe using gensyms to make sure it isn't a name they take).

I can't speak on the namespace issues and perhaps there is a better
solution, but you can explicitly reference a binding in a module using
@ (exported bindings) or @@ (private bindings) like (@ (guile) list)
to get the function LIST from (guile).

Re: PEG Parser Updates/Questions

[Michael Lucy]

On Wed, Jul 28, 2010 at 12:41 AM, No Itisnt <theseaisinhere@gmail.com> wrote:
> On Wed, Jul 28, 2010 at 12:13 AM, Michael Lucy <MichaelGLucy@gmail.com> wrote:
>
>> Another question about module namespaces:  I have some syntax that I'd
>> like to be available to code generated by macros in my module, but
>> which I'd rather not export to the user (to avoid clobbering their
>> functions).  Is there a standard way of doing this?  I can't seem to
>> find anything in the module documentation regarding giving namespaces
>> to things in modules except for :renamer, which has to be done by the
>> user--the only options appear to be not exporting it at all, or
>> exporting it straight into the user's namespace.  The best fix I can
>> think of is naming the syntax things the user is unlikely to ever take
>> (or maybe using gensyms to make sure it isn't a name they take).
>
> I can't speak on the namespace issues and perhaps there is a better
> solution, but you can explicitly reference a binding in a module using
> @ (exported bindings) or @@ (private bindings) like (@ (guile) list)
> to get the function LIST from (guile).

Thanks!  That works perfectly.

>

Re: PEG Parser Updates/Questions

[Michael Lucy]

On Wed, Jul 28, 2010 at 12:13 AM, Michael Lucy <MichaelGLucy@gmail.com> wrote:
> I've officially eliminated the last define-macro expression.
>
> However, I get the feeling that things may not be exactly as desired.
> The original program made extensive use of functions in building the
> macros, and I originally tried to replace these with macros.  This
> turned out to be a little difficult to debug, however (read: I was
> unable to make the code actually work).  I eventually abandoned this
> and just made datum->syntax calls.

I've left this alone since then; it would be nice to get some
confirmation that this was the right choice though.  If it isn't, I
think I still have time to change it before the GSOC deadline (and
like I said, I'll hang around after it).

Anyway, I just pushed up everything I have to the git repository (I
think; I may have screwed up the syntax).  I'm basically done except
that the documentation/comments could use some more touching up, I
don't have a set of benchmarks yet and I haven't really optimized
things all that much.

>
> On the one hand, this works.  I also find it easier to debug, and I
> think it looks cleaner.
>
> The downside is that one doesn't get all the same benefits of
> referential transparency, so I still have gensyms in the functions
> etc.  Is this a problem?
>
> If so, I can definitely replace everything with macros, but I might
> not be able to do that and get everything else done by the GSOC
> project deadline.  I'd like to hang around after the project is
> officially done from Google's point of view to polish things up, so I
> could also do it then.
>
> Another question about module namespaces:  I have some syntax that I'd
> like to be available to code generated by macros in my module, but
> which I'd rather not export to the user (to avoid clobbering their
> functions).  Is there a standard way of doing this?  I can't seem to
> find anything in the module documentation regarding giving namespaces
> to things in modules except for :renamer, which has to be done by the
> user--the only options appear to be not exporting it at all, or
> exporting it straight into the user's namespace.  The best fix I can
> think of is naming the syntax things the user is unlikely to ever take
> (or maybe using gensyms to make sure it isn't a name they take).
>

Re: PEG Parser Updates/Questions

[Michael Lucy]

On Fri, Aug 6, 2010 at 1:40 AM, Michael Lucy <MichaelGLucy@gmail.com> wrote:
> On Wed, Jul 28, 2010 at 12:13 AM, Michael Lucy <MichaelGLucy@gmail.com> wrote:
>> I've officially eliminated the last define-macro expression.
>>
>> However, I get the feeling that things may not be exactly as desired.
>> The original program made extensive use of functions in building the
>> macros, and I originally tried to replace these with macros.  This
>> turned out to be a little difficult to debug, however (read: I was
>> unable to make the code actually work).  I eventually abandoned this
>> and just made datum->syntax calls.
>
> I've left this alone since then; it would be nice to get some
> confirmation that this was the right choice though.  If it isn't, I
> think I still have time to change it before the GSOC deadline (and
> like I said, I'll hang around after it).

So, we're essentially at the GSOC deadline.  I pushed up the finished
form of what I have to the git repository, plus some benchmarks.  I
also added a variant on packrat parsing using a cache instead of a
hash (the hash performs poorly on flat text files).  This turned out
to be well worth it; on the relatively limited benchmarking suite it
was about a 50x performance boost.

Quick summary of files added (all in origin/mlucy):
Actual module: module/ice-9/peg.scm
Test suite: test-suite/tests/peg.test
Benchmarks: test-suite/tests/peg.bench (wasn't quite sure where to put this)
Documentation: doc/ref/api-peg.texi (also updated guile.texi to include it)

The macros haven't been touched (except I fixed up one define-macro
macro that had slipped past my radar).  I'll be hanging around after
the deadline tidying up a bit more (I'd like some more benchmarks); if
the macro situation isn't what you guys want I'll fix it.

>
> Anyway, I just pushed up everything I have to the git repository (I
> think; I may have screwed up the syntax).  I'm basically done except
> that the documentation/comments could use some more touching up, I
> don't have a set of benchmarks yet and I haven't really optimized
> things all that much.
>
>>
>> On the one hand, this works.  I also find it easier to debug, and I
>> think it looks cleaner.
>>
>> The downside is that one doesn't get all the same benefits of
>> referential transparency, so I still have gensyms in the functions
>> etc.  Is this a problem?
>>
>> If so, I can definitely replace everything with macros, but I might
>> not be able to do that and get everything else done by the GSOC
>> project deadline.  I'd like to hang around after the project is
>> officially done from Google's point of view to polish things up, so I
>> could also do it then.
>>
>> Another question about module namespaces:  I have some syntax that I'd
>> like to be available to code generated by macros in my module, but
>> which I'd rather not export to the user (to avoid clobbering their
>> functions).  Is there a standard way of doing this?  I can't seem to
>> find anything in the module documentation regarding giving namespaces
>> to things in modules except for :renamer, which has to be done by the
>> user--the only options appear to be not exporting it at all, or
>> exporting it straight into the user's namespace.  The best fix I can
>> think of is naming the syntax things the user is unlikely to ever take
>> (or maybe using gensyms to make sure it isn't a name they take).
>>
>

Re: PEG Parser Updates/Questions

[Andy Wingo]

Hello, Mr. Lucy!

At some point I might escape the need to apologize at every mail I send,
but until then: sorry for the late response!

On Thu 05 Aug 2010 23:40, Michael Lucy <MichaelGLucy@Gmail.com> writes:

> On Wed, Jul 28, 2010 at 12:13 AM, Michael Lucy <MichaelGLucy@gmail.com> wrote:
>> I've officially eliminated the last define-macro expression.
>>
>> However, I get the feeling that things may not be exactly as desired.
>> The original program made extensive use of functions in building the
>> macros, and I originally tried to replace these with macros.  This
>> turned out to be a little difficult to debug, however (read: I was
>> unable to make the code actually work).  I eventually abandoned this
>> and just made datum->syntax calls.

I'll have to check and see what the deal is. However note that with
procedural macros you can still use helper functions that operate on
syntax objects, destructing them via syntax and building up syntax
objects using `syntax'. Think of a procedural macro as consisting of one
helper function :)

>> The downside is that one doesn't get all the same benefits of
>> referential transparency, so I still have gensyms in the functions
>> etc.  Is this a problem?

Yep! But it probably won't be a big deal to fix.

>> Another question about module namespaces:  I have some syntax that I'd
>> like to be available to code generated by macros in my module, but
>> which I'd rather not export to the user (to avoid clobbering their
>> functions).  Is there a standard way of doing this?

Phil mentioned @ and @@, but the normal case is that things Just Work,
due to the referential-transparency-preserving properties of
syntax-case.

For example:

    (define-module (a)
      #:export (b))

    (define-syntax b
      (lambda (x)
        (syntax-case x ()
          ((_ exp)
           #'(c exp)))))

    (define-syntax c
      (syntax-rules ()
        ((_ exp) (car exp))))

    (define-module (d)
      #:use-module (a))

    (b '(1 2 3))
     => 1

You see that the expansion of `(b '(1 2 3))' in the module `(d)'
produced a reference to `c' -- but `c' is private in the `(a)'
module. Barring the use of datum->syntax, syntax-case macros *scope free
identifiers within the lexical conext and module in which they
appear*. That's what "hygiene" is.

Anyway, I hope to have time to poke this next week. I'm very much
looking forward to having a good PEG parser!

Cheers,

Andy
-- 
http://wingolog.org/

Re: PEG Parser Updates/Questions

[Phil]

So is there a sort of ETA on when this will be part of Guile?

On Fri, Aug 20, 2010 at 4:30 PM, Andy Wingo <wingo@pobox.com> wrote:
> Hello, Mr. Lucy!
>
> At some point I might escape the need to apologize at every mail I send,
> but until then: sorry for the late response!
>
> On Thu 05 Aug 2010 23:40, Michael Lucy <MichaelGLucy@Gmail.com> writes:
>
>> On Wed, Jul 28, 2010 at 12:13 AM, Michael Lucy <MichaelGLucy@gmail.com> wrote:
>>> I've officially eliminated the last define-macro expression.
>>>
>>> However, I get the feeling that things may not be exactly as desired.
>>> The original program made extensive use of functions in building the
>>> macros, and I originally tried to replace these with macros.  This
>>> turned out to be a little difficult to debug, however (read: I was
>>> unable to make the code actually work).  I eventually abandoned this
>>> and just made datum->syntax calls.
>
> I'll have to check and see what the deal is. However note that with
> procedural macros you can still use helper functions that operate on
> syntax objects, destructing them via syntax and building up syntax
> objects using `syntax'. Think of a procedural macro as consisting of one
> helper function :)
>
>>> The downside is that one doesn't get all the same benefits of
>>> referential transparency, so I still have gensyms in the functions
>>> etc.  Is this a problem?
>
> Yep! But it probably won't be a big deal to fix.
>
>>> Another question about module namespaces:  I have some syntax that I'd
>>> like to be available to code generated by macros in my module, but
>>> which I'd rather not export to the user (to avoid clobbering their
>>> functions).  Is there a standard way of doing this?
>
> Phil mentioned @ and @@, but the normal case is that things Just Work,
> due to the referential-transparency-preserving properties of
> syntax-case.
>
> For example:
>
>    (define-module (a)
>      #:export (b))
>
>    (define-syntax b
>      (lambda (x)
>        (syntax-case x ()
>          ((_ exp)
>           #'(c exp)))))
>
>    (define-syntax c
>      (syntax-rules ()
>        ((_ exp) (car exp))))
>
>    (define-module (d)
>      #:use-module (a))
>
>    (b '(1 2 3))
>     => 1
>
> You see that the expansion of `(b '(1 2 3))' in the module `(d)'
> produced a reference to `c' -- but `c' is private in the `(a)'
> module. Barring the use of datum->syntax, syntax-case macros *scope free
> identifiers within the lexical conext and module in which they
> appear*. That's what "hygiene" is.
>
> Anyway, I hope to have time to poke this next week. I'm very much
> looking forward to having a good PEG parser!
>
> Cheers,
>
> Andy
> --
> http://wingolog.org/
>
>

Re: PEG Parser Updates/Questions

[Andy Wingo]

Hi,

On Sun 29 Aug 2010 00:45, Phil <theseaisinhere@gmail.com> writes:

> So is there a sort of ETA on when [PEG and other SoC projecsts] will
> be part of Guile?

Mmm, not really. I hope it will be soon, but I want to go through the
code patch-by-patch, and that will take at least 3 or 4 hours, if
everything goes well -- if there are problems it could take longer...

So I don't want to make myself look silly again by mentioning a date :)

Cheers,

Andy
-- 
http://wingolog.org/