Re: Working around the limitations of SMIE

[Philip Kaludercic <philipk@posteo.net>]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>>>> I am writing a major mode for a little language I am using at
>>>> university, and wanted to try using SMIE for indentation and all the
>>>> other things.  The issue I find myself confronted with is that functions
>>>> are defined as in the following example:
>>>>
>>>>     func funktion(x : int): float
>>>>       x := x * x;
>>>>       return x;
>>>>     end
>>>>
>>>> where there is no delimiter between the return type (float), and the
>>>> rest of the body (such as "begin" or something like that).
>>>
>>> How is the separation between the function's return type and the
>>> function's body defined?  Is it based on the newline that follows the
>>> type, or is the language constrained to have types that are
>>> a single identifiers?
>>
>> The latter.  This is the grammar production:
>>
>> functionDeclaration: ' func ' identifier '( ' ( parameterDeclaration ( ', '
>> parameterDeclaration ) * ) ? ') '
>>     ( ': ' typeName ) ? block ' end ' ;
>
> Hmm... so the only "reliable" separator token is the close parenthesis, huh?
> I think I'd go with a hack in the lexer which checks if this is "the
> close paren of a function definition" and make it include the subsequent
> type annotation (if present).  I.e. that new token would cover the whole of
>
>     ')' ( ':' typeName ) ?
>
> Then again, that wouldn't work with the usual handling of parens in SMIE
> (IOW, you couldn't rely on syntax tables for them any more) :-(
>
> Maybe instead you can try and make the lexer recognize just ": typeName"
> (treating it as a special token) and then tweak the indentation rules so
> as to align the subsequent instruction with it.

I have managed to solve by making the tokenizer distinguish if it is
looking at the function type with or without any whitespace inbetween.
If there is, a ghost token is generated, if not the type token is
returned.  This appears to work well.

Thanks for the hint, once again!