Nyacc question: where are the actions bound?

Nyacc question: where are the actions bound?

[tomas]

[-- Attachment #1: Type: text/plain, Size: 1606 bytes --]

Hi,

I'm playing around with Nyacc: I found a first little use case
to get my feet wet.

First of all, than you, Matt, for this impressive package.

Shamelessly stolen from the minimal example, playground looks
roughly like this:

#+begin_source scheme
  (use-modules (nyacc lalr))
  (use-modules (nyacc lex))
  (use-modules (nyacc parse))
  
  ;; to be used in some ($$ ...) actions:
  (define (collect arg) (display arg))
  
  (define my-grammar
    (lalr-spec
     (start my-file)
     (grammar
      (my-file
       (elt-list))
      ;; more productions, calling out to ($$... collect)
      (name
       ($ident)))))
  
  (define mach (make-lalr-machine aq-grammar))
  (define mtab (lalr-match-table mach))
  (define gen-lexer (make-lexer-generator mtab))
  (define raw-parse (make-lalr-parser mach))
  (define (parse) (raw-parse (gen-lexer)))
  (parse)
#+end_source

So I defined some function =collect= which will be called from
actions in the grammar.

My question is: where is the stuff resolved which is mentioned
in grammar actions? My first experiments indicate that it's
looked up at (module) top level, as if (grammar ...) greedily
weaved that in at compile time.

My idea would be to (pre-) define a grammar and to exchange
the actions later as needed. Or would I have to re-define the
grammar whenever I change my mind about actions?

Note that I'm still very much in exploratory mode: "you're
holding it wrong" would be a perfectly adequate answer, as
would be "your mumblings are pretty unintelligible" :-)

Cheers & thanks
-- tomás

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Nyacc question: where are the actions bound?

[Matt Wette]

On 3/8/20 3:14 AM, tomas@tuxteam.de wrote:
> Hi,
>
> I'm playing around with Nyacc: I found a first little use case
> to get my feet wet.
>
> First of all, than you, Matt, for this impressive package.
>
> Shamelessly stolen from the minimal example, playground looks
> roughly like this:
>
> #+begin_source scheme
>    (use-modules (nyacc lalr))
>    (use-modules (nyacc lex))
>    (use-modules (nyacc parse))
>    
>    ;; to be used in some ($$ ...) actions:
>    (define (collect arg) (display arg))
>    
>    (define my-grammar
>      (lalr-spec
>       (start my-file)
>       (grammar
>        (my-file
>         (elt-list))
>        ;; more productions, calling out to ($$... collect)
>        (name
>         ($ident)))))
>    
>    (define mach (make-lalr-machine aq-grammar))
>    (define mtab (lalr-match-table mach))
>    (define gen-lexer (make-lexer-generator mtab))
>    (define raw-parse (make-lalr-parser mach))
>    (define (parse) (raw-parse (gen-lexer)))
>    (parse)
> #+end_source
>
> So I defined some function =collect= which will be called from
> actions in the grammar.
>
> My question is: where is the stuff resolved which is mentioned
> in grammar actions? My first experiments indicate that it's
> looked up at (module) top level, as if (grammar ...) greedily
> weaved that in at compile time.
>
> My idea would be to (pre-) define a grammar and to exchange
> the actions later as needed. Or would I have to re-define the
> grammar whenever I change my mind about actions?
>
> Note that I'm still very much in exploratory mode: "you're
> holding it wrong" would be a perfectly adequate answer, as
> would be "your mumblings are pretty unintelligible" :-)
>
> Cheers & thanks
> -- tomás
Nyacc was envisioned with the paradigm you propose, by using tags.
But I have not tried that in a while.  If you look at the example calc
in examples/nyacc/lang/calc/mach.scm you will see

(define (gen-calc-files)
   (write-lalr-actions
    full-mach "mach.d/calc-full-act.scm" #:prefix "calc-full-")
   (write-lalr-tables
    full-mach "mach.d/calc-full-tab.scm" #:prefix "calc-full-")
   (write-lalr-actions
    stmt-mach "mach.d/calc-stmt-act.scm" #:prefix "calc-stmt-")
   (write-lalr-tables
    stmt-mach "mach.d/calc-stmt-tab.scm" #:prefix "calc-stmt-")
   )

You can run this, then you can use the generated files to provide
the parser (don't need to run make-lalr-machine anymore.  There
are two files generated: one with all the parser tables and one
with the actions (only).   You could (copy and) modify the actions
file to get different parser actions.   That file includes comments
to help:

(define calc-full-act-v
   (vector
    ;; $start => prog
    (lambda ($1 . $rest) $1)
    ;; prog => stmt-list
    (lambda ($1 . $rest) (tl->list $1))
    ;; stmt-list => stmt
    (lambda ($1 . $rest) (make-tl 'stmt-list $1))
    ;; stmt-list => stmt-list stmt
    (lambda ($2 $1 . $rest) (tl-append $1 $2))
    ;; stmt => "\n"
    (lambda ($1 . $rest) `(empty-stmt))
    ...

Hope this helps.

Matt

Re: Nyacc question: where are the actions bound?

[tomas]

[-- Attachment #1: Type: text/plain, Size: 1311 bytes --]

On Sun, Mar 08, 2020 at 08:10:50AM -0700, Matt Wette wrote:
> 
> 
> On 3/8/20 3:14 AM, tomas@tuxteam.de wrote:
> >Hi,
> >
> >I'm playing around with Nyacc [...]

> >So I defined some function =collect= which will be called from
> >actions in the grammar.
> >
> >My question is: where is the stuff resolved which is mentioned
> >in grammar actions? [...]

Thanks for your quick response, and for taking the time to make
sense of my diffuse text.

> Nyacc was envisioned with the paradigm you propose, by using tags.
> But I have not tried that in a while.  If you look at the example calc
> in examples/nyacc/lang/calc/mach.scm you will see

Note that I'm not yet proposing a paradigm. What you (graciously)
raise to the level of "paradigm" is still just a set of prejudices
as I try to find a way through the bushes :)

I'm ready & willing to change my way of seeing things.

Care to explain why you havent tried your first paradigm in a
while? After all you're probably the one having the most practical
experience with Nyacc.

> (define (gen-calc-files)
  [...]

> You can run this, then you can use the generated files to provide
> the parser [...]

> (define calc-full-act-v
  [...]

Thanks for the hint. I'll definitely give that angle a try.

Cheers
-- tomás

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Nyacc question: [found] where are the actions bound?

[tomas]

[-- Attachment #1: Type: text/plain, Size: 2033 bytes --]

On Sun, Mar 08, 2020 at 08:10:50AM -0700, Matt Wette wrote:
> 
> 
> On 3/8/20 3:14 AM, tomas@tuxteam.de wrote:
> >Hi,

[...]

> >My question is: where is the stuff resolved which is mentioned
> >in grammar actions?

Hah. Managed to answer my own question by reading the source.

For the benefits of others who might have a similar question
(without knowing how to articulate it, like it happened to
me), here's the answer:

Yes, the actions are "resolved" wrt the (calling) module's
top level environment. The magic happens here (that's wrt the
all-fresh V1.01.2), around lines 47 ff, in module/nyacc/parse.scm:

  (define (make-xct av)
    (if (procedure? (vector-ref av 0))
        av
        (vector-map (lambda (ix f) (eval f (current-module)))
      (vector-map (lambda (ix actn) (wrap-action actn)) av))))

It's the (eval f (current-module)) which does the trick. The
trick happens in make-lalr-parser.

Background: I envisioned something like

  (let ( ...some environment for the actions...)
    ...make-lalr-parser...)

and have make-lalr-parser pick up the bindings in the lexical
environment.

but had to realize that make-lalr-parser ignores the lexical
environment. That's now clear to me, because (eval ... (current-module))
looks at the caller's module's top-level bindings.

One would have to call local-eval (from (ice-9 local-eval)) and
explicitly pass (the-environment) (from the same module) to
achieve what I had in mind.

Not that I'm proposing that, mind you. I still barely know what
I'm doing at this point. Probably there are very good reasons
to resort to the (top-level) module bindings.

I just wanted to understand, and I think I do now :-)

As to your other proposal (writing out the pre-compiled parser
with write-lalr-*), I think it's orthogonal to my issue. This
might come in handy when the parser is heavy enough that it
takes a significant time constructing it (which it is not in my
little use case for now)

Thanks & cheers
-- tomás

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Nyacc question: [found] where are the actions bound?

[Matt Wette]

On 3/14/20 4:59 AM, tomas@tuxteam.de wrote:
> On Sun, Mar 08, 2020 at 08:10:50AM -0700, Matt Wette wrote:
>>
>> On 3/8/20 3:14 AM, tomas@tuxteam.de wrote:
>>> Hi,
> [...]
>
>>> My question is: where is the stuff resolved which is mentioned
>>> in grammar actions?
> Hah. Managed to answer my own question by reading the source.
>
> For the benefits of others who might have a similar question
> (without knowing how to articulate it, like it happened to
> me), here's the answer:
>
> Yes, the actions are "resolved" wrt the (calling) module's
> top level environment. The magic happens here (that's wrt the
> all-fresh V1.01.2), around lines 47 ff, in module/nyacc/parse.scm:
>
>    (define (make-xct av)
>      (if (procedure? (vector-ref av 0))
>          av
>          (vector-map (lambda (ix f) (eval f (current-module)))
>        (vector-map (lambda (ix actn) (wrap-action actn)) av))))
>
> It's the (eval f (current-module)) which does the trick. The
> trick happens in make-lalr-parser.
>
> Background: I envisioned something like
>
>    (let ( ...some environment for the actions...)
>      ...make-lalr-parser...)
>
> and have make-lalr-parser pick up the bindings in the lexical
> environment.
>
> but had to realize that make-lalr-parser ignores the lexical
> environment. That's now clear to me, because (eval ... (current-module))
> looks at the caller's module's top-level bindings.
>
> One would have to call local-eval (from (ice-9 local-eval)) and
> explicitly pass (the-environment) (from the same module) to
> achieve what I had in mind.
>
> Not that I'm proposing that, mind you. I still barely know what
> I'm doing at this point. Probably there are very good reasons
> to resort to the (top-level) module bindings.
>
> I just wanted to understand, and I think I do now :-)
>
> As to your other proposal (writing out the pre-compiled parser
> with write-lalr-*), I think it's orthogonal to my issue. This
> might come in handy when the parser is heavy enough that it
> takes a significant time constructing it (which it is not in my
> little use case for now)
>
> Thanks & cheers
> -- tomás
I get it now.  What you expect makes much sense.
  I will think about that.

Matt

Re: Nyacc question: [found] where are the actions bound?

[tomas]

[-- Attachment #1: Type: text/plain, Size: 679 bytes --]

On Sat, Mar 14, 2020 at 07:31:10AM -0700, Matt Wette wrote:
> On 3/14/20 4:59 AM, tomas@tuxteam.de wrote:

[...]

> >and have make-lalr-parser pick up the bindings in the lexical
> >environment.

[...]

> I get it now.  What you expect makes much sense.
>  I will think about that.

It might come at a price, I don't know. At least, the calling site
has to capture the (lexical) environment, so make-lalr-parser has
either to get the env as an extra parameter or it has to become
a macro.

As I wrote in the other mail, a doc fix could be more than enough.

I repeat: I barely know what I'm doing, so I might be dangerous ;-)

Cheers & thanks
-- tomás

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Nyacc question: where are the actions bound?

[Matt Wette]

On 3/8/20 3:14 AM, tomas@tuxteam.de wrote:
> Hi,
>
> I'm playing around with Nyacc: I found a first little use case
> to get my feet wet.
>
> First of all, than you, Matt, for this impressive package.
>
> Shamelessly stolen from the minimal example, playground looks
> roughly like this:
>
> #+begin_source scheme
>    (use-modules (nyacc lalr))
>    (use-modules (nyacc lex))
>    (use-modules (nyacc parse))
>    
>    ;; to be used in some ($$ ...) actions:
>    (define (collect arg) (display arg))
>    
>    (define my-grammar
>      (lalr-spec
>       (start my-file)
>       (grammar
>        (my-file
>         (elt-list))
>        ;; more productions, calling out to ($$... collect)
>        (name
>         ($ident)))))
>    
>    (define mach (make-lalr-machine aq-grammar))
>    (define mtab (lalr-match-table mach))
>    (define gen-lexer (make-lexer-generator mtab))
>    (define raw-parse (make-lalr-parser mach))
>    (define (parse) (raw-parse (gen-lexer)))
>    (parse)
> #+end_source
>
> So I defined some function =collect= which will be called from
> actions in the grammar.
>
> My question is: where is the stuff resolved which is mentioned
> in grammar actions? My first experiments indicate that it's
> looked up at (module) top level, as if (grammar ...) greedily
> weaved that in at compile time.
>
> My idea would be to (pre-) define a grammar and to exchange
> the actions later as needed. Or would I have to re-define the
> grammar whenever I change my mind about actions?
>
> Note that I'm still very much in exploratory mode: "you're
> holding it wrong" would be a perfectly adequate answer, as
> would be "your mumblings are pretty unintelligible" :-)
>
> Cheers & thanks
> -- tomás
Hi Tomas, (I apologize for the ascii spelling)
You may be able to do what you want with the following:

(define (parse)
  (let ((raw-parser (make-lalr-parser)))
    (raw-parser (gen-lexer))))

Re: Nyacc question: where are the actions bound?

[tomas]

[-- Attachment #1: Type: text/plain, Size: 891 bytes --]

On Thu, Jul 30, 2020 at 06:50:08AM -0700, Matt Wette wrote:
> On 3/8/20 3:14 AM, tomas@tuxteam.de wrote:
> >Hi,
> >
> >I'm playing around with Nyacc: I found a first little use case
> >to get my feet wet.

[...]

> Hi Tomas, (I apologize for the ascii spelling)

Hey, no worries. I don't take such things too seriously ;)

> You may be able to do what you want with the following:
> 
> (define (parse)
>  (let ((raw-parser (make-lalr-parser)))
>    (raw-parser (gen-lexer))))

Hm. Have to wrap my head around it (at the moment, I'm fine
having the "semantic" functions bound in the module, and
your last patch lets pass another module/namespace explicitly,
so, short of local-eval, of which we don't know whether it's
a good idea [1] anyway, all my wishes came true.

So thanks, again

Cheers

[1] I'm still to pose a question on that in guile-user.

-- t

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Nyacc question: where are the actions bound?

[tomas]

[-- Attachment #1: Type: text/plain, Size: 1352 bytes --]

On Thu, Jul 30, 2020 at 06:50:08AM -0700, Matt Wette wrote:
> On 3/8/20 3:14 AM, tomas@tuxteam.de wrote:
> >Hi,
> >
> >I'm playing around with Nyacc: I found a first little use case

[...]

> >So I defined some function =collect= which will be called from
> >actions in the grammar.
> >
> >My question is: where is the stuff resolved which is mentioned
> >in grammar actions?

[...]

> Hi Tomas, (I apologize for the ascii spelling)
> You may be able to do what you want with the following:
> 
> (define (parse)
>  (let ((raw-parser (make-lalr-parser)))
>    (raw-parser (gen-lexer))))

OK, got it now. It doesn't actually address my problem above, but
you answered my question above anyway, and even made the environment
more flexible (that was the commit:

  commit 9f45ea29bfc22e53cf08c27a6fb9d2de581b2092
  Author: Matt Wette <mwette@alumni.caltech.edu>
  Date:   Mon Mar 23 17:38:30 2020 -0700

      Author: Matt Wette
      Date:   Mar 23, 2020
    
              Add #:env option to specify module for evaluating parser actions
              * module/nyacc/parse.scm (make-lalr-parser): add #env
                (make-lalr-parser/num , /sym) use (current-module) as default
              * test-suite/nyacc/lalr-01.test: add test case

so thanks a lot for that). I'm a happy camper :)

Cheers
 - t

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Nyacc question: where are the actions bound?

[Matt Wette]

On 8/3/20 1:42 PM, tomas@tuxteam.de wrote:
> On Thu, Jul 30, 2020 at 06:50:08AM -0700, Matt Wette wrote:
>> On 3/8/20 3:14 AM, tomas@tuxteam.de wrote:
>>> Hi,
>>>
>>> I'm playing around with Nyacc: I found a first little use case
> [...]
>
>>> So I defined some function =collect= which will be called from
>>> actions in the grammar.
>>>
>>> My question is: where is the stuff resolved which is mentioned
>>> in grammar actions?
> [...]
>
>> Hi Tomas, (I apologize for the ascii spelling)
>> You may be able to do what you want with the following:
>>
>> (define (parse)
>>   (let ((raw-parser (make-lalr-parser)))
>>     (raw-parser (gen-lexer))))
> OK, got it now. It doesn't actually address my problem above, but
> you answered my question above anyway, and even made the environment
> more flexible (that was the commit:
>
>    commit 9f45ea29bfc22e53cf08c27a6fb9d2de581b2092
>    Author: Matt Wette <mwette@alumni.caltech.edu>
>    Date:   Mon Mar 23 17:38:30 2020 -0700
>
>        Author: Matt Wette
>        Date:   Mar 23, 2020
>      
>                Add #:env option to specify module for evaluating parser actions
>                * module/nyacc/parse.scm (make-lalr-parser): add #env
>                  (make-lalr-parser/num , /sym) use (current-module) as default
>                * test-suite/nyacc/lalr-01.test: add test case
>
> so thanks a lot for that). I'm a happy camper :)
>
> Cheers
>   - t
Good to hear!  Thanks for the feedback.
Matt