Re: SMIE documentation

["Johan Bockgård" <bojohan@gnu.org>]

Stefan Monnier <monnier@IRO.UMontreal.CA> writes:

> While Savannah is down, maybe someone will feel like checking my attempt
> at documenting SMIE.

I have a few comments on the documentation and the functionality itself.

First, here's a patch to make smie-next-sexp correctly return (t POS
TOKEN) when bumping into a closing thingy in the forward direction.

--- a/lisp/emacs-lisp/smie.el
+++ b/lisp/emacs-lisp/smie.el
@@ -653,7 +653,8 @@ Possible return values:
                 (if (and halfsexp (numberp (funcall op-forw toklevels)))
                     (push toklevels levels)
                   (throw 'return
-                         (prog1 (list (or (car toklevels) t) (point) token)
+                         (prog1 (list (or (funcall op-forw toklevels) t)
+                                      (point) token)
                            (goto-char pos)))))
                (t
                 (let ((lastlevels levels))


There's a problem with the indentation when there's comment at the start
of a line before a keyword, like

    (* Comment *) IF ...                      <-- Press TAB

The virtual indent of the following keyword is computed as its current
column. The comment indentation function tries to align the comment with
this column, making the line wander farther and farther to the right for
every press of the TAB key.

> +Calling this function is sufficient to make commands such as
> +@code{forward-sexp}, @code{backward-sexp}, and @code{transpose-sexps}
> +be able to properly handle structural elements other than just the paired
> +parentheses already handled by syntax tables.  E.g. if the provided
> +grammar is precise enough, @code{transpose-sexps} can correctly
> +transpose the two arguments of a @code{+} operator, taking into account
> +the precedence rules of the language.

This makes C-M-f and friends behave very differently from most other
major modes, which doesn't really feel right.

> +To describe the lexing rules of your language to SMIE, you will need
> +2 functions, one to fetch the next token, and another to fetch the
> +previous token.  Those functions will usually first skip whitespace and
> +comments and then look at the next chunk of text to see if it
> +is a special token, if so it should skip it and return a description of
> +this token.  Usually this is simply the string extracted from the
> +buffer, but this is not necessarily the case.

It would be good if users could hook their own functions into all places
that extract text from the buffer ("buffer-substring"), and not just
smie-forward/backward-token-function; e.g. to use interned token strings
or to handle some kind of banana brackets using the syntax table.

> +@code{:elem}, in which case the function should return either the offset
> +to use to indent function arguments (if @var{arg} is the symbol
> +@code{arg})

Either there's a bug in the code, or this should be the symbol "args"
and likewise for the doc string.

> +@defun smie-rule-parent &optional offset
> +Return the proper offset to align with the parent.
> +If non-@code{nil}, @var{offset} should be an integer giving an
> +additional offset to apply.
> +@end defun

The function returns an absolute column, not an offset.

> +    (:before
> +     (cond
> +      ((equal token ",") (smie-rule-separator kind))
> +      ((member token '("begin" "(" "@{"))
> +       (if (smie-rule-hanging-p) (smie-rule-parent)))

Does this really work for "("? Most of the time smie-indent--parent
seems to returns nil before a paren, which breaks smie-rule-parent.