help with smie

help with smie

[Yagnesh Raghava Yakkala]

Hello,

I like to use SMIE for indentation in one of my package called ncl-mode¹. Ncl
syntax is some what similar to Fortran. But it seems kind of big for me to
implement now.

So I decided to first try SMIE work for another small major mode called
namelist-mode². Namelist files are sort of configuration files Fortran
programs (you may already know that).

Anyway, namelist files are very simple, look like the following,

--8<---------------cut here---------------start------------->8---
 &group1
   key1 = value1, value2
   key2 = value1,
   ...
 $end
 &group2
   key1 = value1
   ...
 $end
--8<---------------cut here---------------end--------------->8---

I wrote smie rules in the mode, tried to see how it works. No surprise they
didn't work.

So, actual questions are,

- is there any way I can check the correctness of the rules.?
- can I look at the parsed tokens by SMIE to find out how SMIE sees the text
  in the buffer.

and also I would be glad if any of you can review my code (second link in the
footnote), let me know the blunders I made in writing it.

Thanks,

PS: my understanding of language parser is very elementary. So please excuse
me if I sound stupid.

¹  https://github.com/yyr/ncl-mode
²  https://github.com/yyr/namelist-mode

--
ఎందరో మహానుభావులు అందరికి వందనములు.
YYR

Re: help with smie

[Josh]

On Thu, Feb 21, 2013 at 11:40 AM, Yagnesh Raghava Yakkala
<hi@yagnesh.org> wrote:
> So I decided to first try SMIE work for another small major mode called
> namelist-mode². Namelist files are sort of configuration files Fortran
> programs (you may already know that).
> [...]
> I wrote smie rules in the mode, tried to see how it works. No surprise they
> didn't work.

I tried a similar exercise recently, with similar results.  I read in
(info "(elisp) SMIE setup")
that SMIE's "main entry point is `smie-setup' which is a function
typically called while setting
up a major mode."  Would it make sense to add an example of a simple
SMIE-based major
mode to (info "(elisp) Example Major Modes") ?

Re: help with smie

[Yagnesh Raghava Yakkala]

Hello ,

Thanks for the reply,

On Feb 22 2013, Josh <josh@foxtail.org> wrote:

> On Thu, Feb 21, 2013 at 11:40 AM, Yagnesh Raghava Yakkala
> <hi@yagnesh.org> wrote:
>> So I decided to first try SMIE work for another small major mode called
>> namelist-mode². Namelist files are sort of configuration files Fortran
>> programs (you may already know that).
>> [...]
>> I wrote smie rules in the mode, tried to see how it works. No surprise they
>> didn't work.
>
> I tried a similar exercise recently, with similar results.  I read in
> (info "(elisp) SMIE setup")
> that SMIE's "main entry point is `smie-setup' which is a function
> typically called while setting
> up a major mode."  Would it make sense to add an example of a simple
> SMIE-based major
> mode to (info "(elisp) Example Major Modes") ?

Yes, I read that, and I have set smie-setup in my code. may I should have
pasted the code here.


Part of the smie related code: 

--8<---------------cut here---------------start------------->8---
(require 'smie)

(defconst namelist-use-smie nil
  "Whether to use SMIE for indentation.")

(defvar namelist-keywords-regexp
  (regexp-opt '("," "/"))
  "For SMIE.")

(defconst namelist-smie-tokens
  '(("BEGIN" . "&")
    ("END" . '("/" "$END"))
    ("OP" .  "=")))

(defvar namelist-smie-grammer
  (smie-prec2->grammar
   (smie-bnf->prec2
    '((id)
      (inst ("BEGIN" exp "END")
            (exp))
      (exp (id "," id)
           (id "=" exp)))
    '((assoc ","))))
  "SMIE grammar for namelist mode.")

(defun namelist-smie-rules (kind token)
  (case kind
    (:after
     (cond
      ((equal token ",") (smie-rule-separator kind))
      ((equal token "BEGIN") namelist-indent-offset)))
    (:before
     (cond
      ((equal token ",") (smie-rule-separator kind))
      ))
    (:elem
     (cond
      ((equal "BEGIN") '(column . 0))
      ((equal "END") '(column . 0))))))

(defun namelist-smie-forward-token ()
  (forward-comment (point-max))
  (cond
   ((looking-at namelist-keywords-regexp)
    (goto-char (match-end 0))
    (match-string-no-properties 0))
   (t (buffer-substring-no-properties
       (point)
       (progn (skip-syntax-forward "w_")
              (point))))))

(defun namelist-smie-backward-token ()
  (forward-comment (- (point)))
  (cond
   ((looking-back namelist-keywords-regexp (- (point) 2) t)
    (goto-char (match-beginning 0))
    (match-string-no-properties 0))
   (t (buffer-substring-no-properties
       (point)
       (progn (skip-syntax-backward "w_")
              (point))))))


(define-derived-mode namelist-mode prog-mode "namelist"
  "Major mode for editing f90 namelist files.

\\{namelist-mode-map}"
  ;; (set (make-local-variable 'indent-line-function) 'namelist-indent-line)
  (set (make-local-variable 'indent-line-function) 'smie-indent-line)
  (set (make-local-variable 'comment-start) namelist-comment-char)
  (setq indent-tabs-mode nil)
  (set (make-local-variable 'imenu-generic-expression)
       namelist-imenu-generic-expression)
  (set (make-local-variable 'font-lock-defaults)
       '(namelist-font-lock-keywords))
  (smie-setup namelist-smie-grammer
              #'namelist-smie-rules
              :forward-token #'namelist-smie-forward-token
              :backward-token #'namelist-smie-backward-token))
              
--8<---------------cut here---------------end--------------->8---


Thanks.,
-- 
ఎందరో మహానుభావులు అందరికి వందనములు.
YYR

Re: help with smie

[Stephen Leake]

Josh <josh@foxtail.org> writes:

> On Thu, Feb 21, 2013 at 11:40 AM, Yagnesh Raghava Yakkala
> <hi@yagnesh.org> wrote:
>> So I decided to first try SMIE work for another small major mode called
>> namelist-mode². Namelist files are sort of configuration files Fortran
>> programs (you may already know that).
>> [...]
>> I wrote smie rules in the mode, tried to see how it works. No surprise they
>> didn't work.
>
> I tried a similar exercise recently, with similar results.  I read in
> (info "(elisp) SMIE setup")
> that SMIE's "main entry point is `smie-setup' which is a function
> typically called while setting
> up a major mode."  Would it make sense to add an example of a simple
> SMIE-based major
> mode to (info "(elisp) Example Major Modes") ?

There are some fairly simple language modes using SMIE in the Emacs 24
sources; modula2, octave, prolog, sh-script. Do a 'grep' in
emacs-24.2.92/lisp/progmodes.

I've implemented Ada using SMIE; see
http://stephe-leake.org/emacs/ada-mode/emacs-ada-mode.html

I found it most helpful to have a set of Ada source code files that I
could run the indentation engine on, using edebug to step thru the code
when things didn't work.

I didn't see any source code files at https://github.com/yyr/namelist-mode

I glanced at the code in
https://github.com/yyr/namelist-mode/blob/master/namelist-mode.el

namlist-smie-tokens isn't used anywhere. It appears to be trying to
create aliases for the actual tokens. That will probably be more
confusing than it is worth.

You need an expression list, and a group list in your grammar; your
current grammar only allows one group per file, and one expression per
group.

Try:

(exp-list 
 (exp)
 (exp-list exp))

Once you get used to it, the SMIE way of doing an indentation engine is
quite nice, for a small language.

-- 
-- Stephe

Re: help with smie

[Stefan Monnier]

> I wrote smie rules in the mode, tried to see how it works. No surprise they
> didn't work.

"didn't work" is not very informative.

> - is there any way I can check the correctness of the rules?

Try'em out?

> - can I look at the parsed tokens by SMIE to find out how SMIE sees the text
>   in the buffer.

Try to Edebug (i.e. hit C-u C-M-x when inside a function's code, so as
to cause stepping through it next time it's run), or use M-x trace-function.

> and also I would be glad if any of you can review my code (second link in the
> footnote), let me know the blunders I made in writing it.

I took a very brief&superficial look at the code and it looks sane,
other than trivial details like "grammer" instead of "grammar" or the
"\\.namelist" regexp which should probably be "\\.namelist\\'" instead.


        Stefan "SMIE author"

Re: help with smie

[Stefan Monnier]

> (exp-list 
>  (exp)
>  (exp-list exp))

Of course, SMIE won't accept this (you need a terminal between each
non-terminals).


        Stefan