* Nyacc: how to lex comments?
@ 2020-03-16 21:39 tomas
2020-03-17 0:11 ` Matt Wette
0 siblings, 1 reply; 5+ messages in thread
From: tomas @ 2020-03-16 21:39 UTC (permalink / raw)
To: Guile User
[-- Attachment #1: Type: text/plain, Size: 1650 bytes --]
Hi,
in my quest to play parsing, I'm failing when I try to cope with
comments.
Here's an excerpt from my current file
(define my-grammar
(lalr-spec
(start my-file)
(grammar
;; boring grammar details elided
)))
(define mach (make-lalr-machine aq-grammar))
(define mtab (lalr-match-table mach))
(define gen-lexer (make-lexer-generator mtab
#:comm-reader (make-comm-reader '(("#" . "\n")))))
(define raw-parse (make-lalr-parser mach))
(define (parse) (raw-parse (gen-lexer)))
;; (gen-lexer)
(parse)
Comments are shell-like, from "#" to end of line. Grammar
works fine so-far, but when (parse) sees a hash-comment
(even just a one-line input
# this is a comment
it panics:
Backtrace:
In ice-9/boot-9.scm:
1736:10 6 (with-exception-handler _ _ #:unwind? _ # _)
In unknown file:
5 (apply-smob/0 #<thunk 557754319740>)
In ice-9/boot-9.scm:
718:2 4 (call-with-prompt ("prompt") #<procedure 5577543308e0 …> …)
In ice-9/eval.scm:
619:8 3 (_ #(#(#<directory (guile-user) 5577543c1f00>)))
In ice-9/boot-9.scm:
2806:4 2 (save-module-excursion #<procedure 5577543b31b0 at ice-…>)
4351:12 1 (_)
In nyacc/parse.scm:
78:4 0 (_ _ #:debug _)
nyacc/parse.scm:78:4: Throw to key `nyacc-error' with args `("~A:~A: parse failed at state ~A, on input ~S" "(unknown)" 1 0 " a comment")'.
I feel I'm doing something stupid. What can I do to understand
what's going on?
Thanks & cheers
-- tomás
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: Nyacc: how to lex comments?
2020-03-16 21:39 Nyacc: how to lex comments? tomas
@ 2020-03-17 0:11 ` Matt Wette
2020-03-17 7:46 ` tomas
0 siblings, 1 reply; 5+ messages in thread
From: Matt Wette @ 2020-03-17 0:11 UTC (permalink / raw)
To: guile-user
On 3/16/20 2:39 PM, tomas@tuxteam.de wrote:
> Hi,
>
> in my quest to play parsing, I'm failing when I try to cope with
> comments.
>
> Here's an excerpt from my current file
>
> (define my-grammar
> (lalr-spec
> (start my-file)
> (grammar
> ;; boring grammar details elided
> )))
>
> (define mach (make-lalr-machine aq-grammar))
> (define mtab (lalr-match-table mach))
> (define gen-lexer (make-lexer-generator mtab
> #:comm-reader (make-comm-reader '(("#" . "\n")))))
> (define raw-parse (make-lalr-parser mach))
> (define (parse) (raw-parse (gen-lexer)))
>
> ;; (gen-lexer)
> (parse)
>
> Comments are shell-like, from "#" to end of line. Grammar
> works fine so-far, but when (parse) sees a hash-comment
> (even just a one-line input
>
> # this is a comment
>
> it panics:
>
> Backtrace:
> In ice-9/boot-9.scm:
> 1736:10 6 (with-exception-handler _ _ #:unwind? _ # _)
> In unknown file:
> 5 (apply-smob/0 #<thunk 557754319740>)
> In ice-9/boot-9.scm:
> 718:2 4 (call-with-prompt ("prompt") #<procedure 5577543308e0 …> …)
> In ice-9/eval.scm:
> 619:8 3 (_ #(#(#<directory (guile-user) 5577543c1f00>)))
> In ice-9/boot-9.scm:
> 2806:4 2 (save-module-excursion #<procedure 5577543b31b0 at ice-…>)
> 4351:12 1 (_)
> In nyacc/parse.scm:
> 78:4 0 (_ _ #:debug _)
>
> nyacc/parse.scm:78:4: Throw to key `nyacc-error' with args `("~A:~A: parse failed at state ~A, on input ~S" "(unknown)" 1 0 " a comment")'.
>
> I feel I'm doing something stupid. What can I do to understand
> what's going on?
>
> Thanks & cheers
> -- tomás
>
>
make-comm-reader will not eat newlines at end of comment by default.
If you want the comment # ... \n to include the newline try adding
#:eat-newline #t.
Matt
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: Nyacc: how to lex comments?
2020-03-17 0:11 ` Matt Wette
@ 2020-03-17 7:46 ` tomas
2020-03-17 12:31 ` Matt Wette
0 siblings, 1 reply; 5+ messages in thread
From: tomas @ 2020-03-17 7:46 UTC (permalink / raw)
To: guile-user
[-- Attachment #1: Type: text/plain, Size: 669 bytes --]
On Mon, Mar 16, 2020 at 05:11:59PM -0700, Matt Wette wrote:
> On 3/16/20 2:39 PM, tomas@tuxteam.de wrote:
> >Hi,
> >
> >in my quest to play parsing, I'm failing when I try to cope with
> >comments.
> >
[...]
> make-comm-reader will not eat newlines at end of comment by default.
> If you want the comment # ... \n to include the newline try adding
> #:eat-newline #t.
D'oh, thanks. Also, I noticed I've to set #:comm-skipper in 'gen-lexer',
instead of #:comm-reader.
I didn't quite understand the difference between both. Would I have
to include comments explicitly in the grammar if the lexer uses
#:comment-reader?
Thanks again
-- tomás
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: Nyacc: how to lex comments?
2020-03-17 7:46 ` tomas
@ 2020-03-17 12:31 ` Matt Wette
2020-03-17 12:42 ` tomas
0 siblings, 1 reply; 5+ messages in thread
From: Matt Wette @ 2020-03-17 12:31 UTC (permalink / raw)
To: guile-user
On 3/17/20 12:46 AM, tomas@tuxteam.de wrote:
> On Mon, Mar 16, 2020 at 05:11:59PM -0700, Matt Wette wrote:
>> On 3/16/20 2:39 PM, tomas@tuxteam.de wrote:
>>> Hi,
>>>
>>> in my quest to play parsing, I'm failing when I try to cope with
>>> comments.
>>>
> [...]
>
>> make-comm-reader will not eat newlines at end of comment by default.
>> If you want the comment # ... \n to include the newline try adding
>> #:eat-newline #t.
> D'oh, thanks. Also, I noticed I've to set #:comm-skipper in 'gen-lexer',
> instead of #:comm-reader.
>
> I didn't quite understand the difference between both. Would I have
> to include comments explicitly in the grammar if the lexer uses
> #:comment-reader?
>
> Thanks again
> -- tomás
Yes. Use comm-reader if your grammar accepts comments.
Use comm-skipper if your grammar does not.
The ffi-helper does read comments in the C grammar and
dumps them in the output files, for example.
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: Nyacc: how to lex comments?
2020-03-17 12:31 ` Matt Wette
@ 2020-03-17 12:42 ` tomas
0 siblings, 0 replies; 5+ messages in thread
From: tomas @ 2020-03-17 12:42 UTC (permalink / raw)
To: guile-user
[-- Attachment #1: Type: text/plain, Size: 616 bytes --]
On Tue, Mar 17, 2020 at 05:31:32AM -0700, Matt Wette wrote:
>
>
> On 3/17/20 12:46 AM, tomas@tuxteam.de wrote:
[...]
> >I didn't quite understand the difference between both. Would I have
> >to include comments explicitly in the grammar if the lexer uses
> >#:comment-reader?
> >
> >Thanks again
> >-- tomás
> Yes. Use comm-reader if your grammar accepts comments.
> Use comm-skipper if your grammar does not.
>
> The ffi-helper does read comments in the C grammar and
> dumps them in the output files, for example.
Ah, I see, thanks. Thank you for your invaluable help!
Cheers
-- t
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2020-03-17 12:42 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-03-16 21:39 Nyacc: how to lex comments? tomas
2020-03-17 0:11 ` Matt Wette
2020-03-17 7:46 ` tomas
2020-03-17 12:31 ` Matt Wette
2020-03-17 12:42 ` tomas
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).