Re: asking for advice for changing the cfengine.el progmode to support CFEngine 3.x

[Stefan Monnier <monnier@iro.umontreal.ca>]

SM> The patch below will at least give you better feedback for the immediate
SM> problem in your grammar.
> I hope you can commit it eventually.

Real soon now.

> I tried, but I can't examine the parser state at a buffer point or do
> anything else to debug the SMIE grammar.  How do you do it?

I use C-M-f and C-M-b (SMIE hooks into these commands so that it jumps
not only over (..) but also pays attention to the defined grammar), as
well as looking at smie-grammar.

> I'm also surprised that Emacs doesn't have a decent LALR parser outside
> Semantic, and Semantic itself is poorly documented as far as writing new
> language support.

LALR only works forward, so it requires parsing from the beginning of
the file, which basically forces you to use a cache or something
like that.  OPG on the other hand are bidirectional.

>>> (bundle_body (bundle_section) (bundle_section "\n" bundle_section))
SM> I'm surprised to see \n here (and in the previous rule).  Are newlines
SM> really significant in cfengine's syntax?

> I thought they were the best way to separate sections.

I'm not sure if that answers my question: are they syntactically
significant or are they simply "usually present"?

> As far as the syntax, newlines are separators but not operators, so
> I didn't do this right.

I'm not sure what's the difference between separators and operators,
from a syntax point of view.

SM> That means that

SM> bundle edit_line block_append_if_no_line(data)
SM> {
SM> insert_lines:
SM> any::
SM> "$(data)" insert_type => "preserve_block";
SM> }

SM> will be parsed as "bundle" followed by 4 <things> (one of them being
SM> "(data)"), which is structurally incorrect (IIUC "(data)" is actually
SM> structurally a child of "block_append_if_no_line" rather than of
SM> "bundle"), but we can hope that it won't matter for indentation.

> Can you do a SMIE parser for the above to get me started?  It would
> really help.  Or point me to an example that's similar to it.

I tried the appended code for a start and bumped into the following
problems:

- the indentation of {...} is only done relative to the first token
  after "bundle".  This can be addressed with a special indentation rule
  for (:before . "{") which walks back over sexps until finding
  a "bundle" or similar thingy, but I haven't taken the trouble to
  write it.
- more problematic: the ";" has various precedences, as in Pascal and
  Modula-2, because it can separate elements within "a:: b;c;d", and it
  can separate between "a::b ; c::d".  In Modula-2 I use a hack in the
  lexer which looks at the token after the ";" to decide what kind of
  ";" we're dealing with (see m2-smie-refine-semi), so I guess we could
  try something similar, but we may have to skip a whole
  "class_condition" before finding a "::", which is not great.
- code like

    "$(init) restart" 
    comment        =>    ...

  is parsed as
  
    ( "$(init) restart"
    comment )       =>    ...

  rather than
    
    "$(init) restart" 
    (comment        =>    ...)

  even though it appears that the second parse should be the right one
  (if I understand the intention behind the indentation used in your
  example).
  I don't understand enough of cfengine's grammar to know how best to
  handle this problem (what makes "$(init) restart" into something
  special, is it because it's a string, is it the \n that follows it,
  ...?).

    
        Stefan


(defvar foo-grammar
  (smie-prec2->grammar
   (smie-merge-prec2s
   (smie-bnf->prec2
    '((token)
      (decls (decls "body" decls)
             (decls "bundle" decls))
      (insts (token ":" insts)))
    '((assoc "body" "bundle")))
   (smie-precs->prec2
    '((right ":")
      (right "::")
      (assoc ";")
      (assoc ",")
      (right "=>"))))))

(defun foo-smie-rules (kind token)
  (pcase (cons kind token)
    (`(:elem . basic) 2)
    (`(:list-intro . ,(or `"body" `"bundle")) t)
    (`(:after . ":") 2)
    (`(:after . "::") 2)))

(define-derived-mode foo-mode prog-mode "Foo"
  "Docstring."
  (set (make-local-variable 'comment-start) "#")
  (smie-setup foo-grammar #'foo-smie-rules))