SMIE: if-elseif-else constructs

SMIE: if-elseif-else constructs

[Andreas Matthias]

I'm trying to teach SMIE to handle if-elseif-else constructs correctly.
In the following example the indentation of the last line, i.e. "xoxox;",
is wrong. I suppose something's wrong with my definition of elseif
in the grammar. But what is it? How can I fix it?

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
if a > 5 then
    foo;
    bar;
else
    foo;
    bar;
end;

xoxox; // correct indentation

if a > 5 then
    foo;
    bar;
elseif a > 5 then
    foo;
    bar;
else
    foo;
    bar;
end;

    xoxox; // wrong indentation
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;


Here is the code:

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
(require 'smie)

(setq foobar-grammar
  (smie-prec2->grammar
   (smie-bnf->prec2
    `((insts (insts ";" insts)
             (inst))
      (inst (if))
      (exp)
      (if ("if" exp "then" else "end"))
      (else (insts)
            (insts "else" insts)
            (insts "elseif" exp "then" else)))
    '((assoc ";")
      (assoc "then" "end")
      ))))

(defun foobar-rules (kind token)
  (message "rule: (%s . %s) at point %d" kind token (point))
  (pcase (cons kind token)

    (`(:before . "then") (smie-rule-parent 0))

    (`(:before . "else") (smie-rule-parent 0))
    (`(:after . "else") (smie-rule-parent 4))

    (`(:before . "elseif") (smie-rule-parent 0))
    (`(:after . "elseif") (smie-rule-parent 4))

    (`(:before . "end") (smie-rule-parent 0))
    ))

(define-derived-mode foobar-mode prog-mode "foobar"
  :syntax-table nil
  (modify-syntax-entry ?/ ". 124")
  (modify-syntax-entry ?* ". 23b")
  (modify-syntax-entry ?\n ">")
  (setq comment-start "//")
  (smie-setup foobar-grammar #'foobar-rules)
  (font-lock-add-keywords
   nil
   `((,(regexp-opt '("if" "then" "elseif" "else" "end")) .
font-lock-keyword-face)))
  (font-lock-mode)
  (font-lock-ensure (point-min) (point-max)))
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

Re: SMIE: if-elseif-else constructs

[Stefan Monnier]

>       (assoc "then" "end")

This says that

    a then b then c then d

is acceptable and that `a`, `b`, and `c` should be considered as
siblings (i.e. at the same depth level).  Same for `end`.
It's clearly not what you want and is the source of your problem.


        Stefan

Re: SMIE: if-elseif-else constructs

[Andreas Matthias]

Stefan Monnier wrote:
>
> >       (assoc "then" "end")
>
> This says that
>
>     a then b then c then d
>
> is acceptable and that `a`, `b`, and `c` should be considered as
> siblings (i.e. at the same depth level).  Same for `end`.

I see. Using assoc seemed to be the easy way but was not reasonable.

I think I've found a better way. I revised the grammar and adjusted
the indentation rules.
The following seems to work.


(require 'smie)

(setq foobar-grammar
  (smie-prec2->grammar
   (smie-bnf->prec2
    `((insts (insts ";" insts)
             (inst))
      (inst (if))
      (exp)
    (if ("if" if-body1 "end"))
    (if-body1 (exp "then" if-body2))
    (if-body2 (insts "elseif" if-body1)
              (insts "else" insts)
              (insts)))

    '((assoc ";")
      ))))

(defun foobar-rules (kind token)
  (message "rule: (%s . %s) at point %d" kind token (point))
  (pcase (cons kind token)
    (`(:elem . basic) foobar-indent-basic)

    (`(:before . "then") 0)
    (`(:after . "then") foobar-indent-basic)

    (`(:before . "else") 0)
    (`(:after . "else") foobar-indent-basic)

    (`(:before . "elseif") 0)
    (`(:after . "elseif") foobar-indent-basic)
    ))

(setq foobar-indent-basic 4)

(define-derived-mode foobar-mode prog-mode "foobar"
  :syntax-table nil
  (modify-syntax-entry ?/ ". 124")
  (modify-syntax-entry ?* ". 23b")
  (modify-syntax-entry ?\n ">")
  (setq comment-start "//")
  (smie-setup foobar-grammar #'foobar-rules)
  (font-lock-add-keywords
   nil
   `((,(regexp-opt '("if" "then" "elseif" "else" "end")) .
font-lock-keyword-face)))
  (font-lock-mode)
  (font-lock-ensure (point-min) (point-max)))

Re: SMIE: if-elseif-else constructs

[Stefan Monnier]

> I think I've found a better way. I revised the grammar and adjusted
> the indentation rules.
> The following seems to work.

It looks OK, now (tho I think it implies that "elsif" is nested within
the right hand side of "then", which I think is wrong in the sense that
it doesn't reflect the natural shape of the abstract syntax tree).
It seems similar to the approach I've used in such cases, except I would
have used something like:

    (inst ("if" if-body "end"))
    (if-body (exp "then" insts)
             (if-body "elseif" if-body)
             (if-body "else" exp))

with an ((assoc "elseif") (nonassoc "else")).  I believe this better
reflects the intended abstract syntax tree, so it should behave a bit
better w.r.t indentation and navigation.

BTW, another way to define the grammar can be:

     [...]
     (inst ("if" exp "then" insts "end")
           ("if" exp "then" insts "else" insts "end")
           ("if" exp "then" insts "elsif" exp "then" insts "else" insts "end")
     [...]

You don't need to list all the (infinite number of) combinations, the
above is sufficient for SMIE to figure out the needed relative
constraints between the precedences of the keywords.  But IIRC this
approach tends to behave less robustly in cases of syntax error (by
which I mean either when the source code is incorrect or when SMIE
parses incorrectly because its grammar or lexer isn't good enough).


        Stefan