Re: SMIE examples and questions

[Stefan Monnier <monnier@iro.umontreal.ca>]

> Are there *simple* complete examples of SMIE out there? I find the
> manual not that helpful to get started and existing modes look pretty
> complex :(

The only simple and complete ones are for language that are probably
too simple.

Maybe the rnc-mode and dts-mode GNU ELPA packages?  You might also want
to look at my SMIE paper (https://programming-journal.org/2021/5/1/)
which walks through a small part of the rnc-mode SMIE code.

> Is the forward lexer required or is the backward one enough?

Both are needed.

> What are the assumptions of SMIE regarding the lexer?
> - What should it return when there is nothing to read (eg backward and
>   begining of buffer)?

nil, usually.

> - I assume if point is in the middle of a token, it should return
>   the complete current token (including text after point) and place the
>   point at token start?

The caller of the lexer should normally make sure it's never in the
middle of a token.  This doesn't work if you have tokens that can span
multiple lines (e.g. when LF itself is significant, such as the
implicit ; in Javascript).

> - Does point position matter regarding spaces (assuming they have no
>   meaning)?  Is it ok if the lexer places the point sometime after and
>   sometime before them?

It should place point right "after" (in the direction of movement) the
token it has read.

> - Can I safely call it manually without involving SMIE?

Yes.

> - I assume backward and forward lexer should always have the same
>   behaviour, even for spaces?

Yes and no, since one should stop to the left of the spaces and the
other to the right.

> I understand there is no way to dump the AST after parsing, and that
> grammar errors not being reported is a feature.

There's no AST, indeed, and there are no possible errors (IOW errors
can't be detected).

> How am I supposed to debug parsing/indenting?

I mostly use M-C-f and M-C-b for parsing.  For the indentation rules you
can use `M-x smie-edebug`.

> If you have any tips or workflow, like the more practical order to
> implement things, or how to debug that would be greatly appreciated.

Start with the simple cases and then work your way up.

If you want more concrete help, don't hesitate to send me your current
code along with some sample source showing how it should be indented and
the problems you're facing.


        Stefan