Better indentation for elisp

Better indentation for elisp

[Sergey Mozgovoy]

The problem

Standard indentation algorithm for Emacs Lisp is not perfect in some
respects.  For example, it fails to indent code like this:
*
(flet ((sum (x y)
                (+ x y))
        (diff (x y)
               (- x y)))
  ...)
*
or like this:
*
(func-1 (or arg-1
                arg-2) weird-arg
                another-arg)
*
and this:
*
(let (var-1 var-2
               (x 10))
  (+ var-1 var-2))
*

SLIME indentation

Those who use SLIME know that there's a contrib *slime-cl-indent*
which does some nice job of indenting Common Lisp code.  Aside from
some CL-specific things (like the loop macro or tagbody), SLIME
indentation is different from that of Elisp in that it supports
arbitrarily-nesting indentation patterns rather than simple values.

For example, here's indentation spec for *case* (quoted from
docstring for *common-lisp-indent-function*):
/
... the function `case' has an indent property (4 &rest (&whole 2 &rest 1)),
meaning:
  * indent the first argument by 4.
  * arguments after the first should be lists, and there may be any number
    of them.  The first list element has an offset of 2, all the rest
    have an offset of 2+1=3.
/


SLIME => Emacs Lisp

I think we could just reuse some SLIME indentation ideas for Emacs
Lisp.  The algorithm is smart enough to handle any typical Elisp
indentation, and it is kind of a generalization of the present one.
General idea is this: walk nested lists up not more than some depth
limit (*lisp-indent-maximum-backtracking*) and see what head
symbols (*car*s) of ancestor lists say about indentation.  Upper
specs override what was said by more local ones.  Whatever the current
indentation we have when the depth is exhausted becomes our end
result.

One thing that I found odd in SLIME indentation is that they specify
absolute offsets (in spaces).  I don't see why such a precision might
be really needed.. to my mind, indentation should better be specified
in terms of number of distinguished forms, like it is implemented now
for Elisp.  Maybe I just don't get it.

So I propose the following indentation patterns, roughly:
  
  * any valid value for *lisp-indent-function* symprop, meaning
    is preserved.  This includes:
      + natural number (# of distinguished args);
      + *defun* symbol which means to indent in defun-style;
      + *nil* stands for "usual" indentation, i.e. indent like a
      	normal function call form;
      + *:flat* -- new thing which tells that list should be
      	flat, no indentation for any args;
  
  * (*S* . (*P-1* *P-2* ... *P-N*)), where *S* is a simple pattern
    (any of above), and all *P-i* are any other patterns (recursive
definition).
    *S* specifies indentation for the enclosing list. b>P-1*, ..., *P-n*
specify
    indentation for sublists, beginning from the second one ("first
    function argument");

  * (*S* . (*P-1* ... *P-N* &rest *P*)), same as above but *P*
    applies to all the other sublists after *N*.

So for example, we'd give indentation for the above-mentioned forms
like this:
*
(put 'flet 'lisp-indent-function '(1 (:flat &rest defun)))
(put 'let 'lisp-indent-function '(1 (:flat &rest 1)))
(put 'cond 'lisp-indent-function '(nil &rest :flat))
*
(Yes, I do agree that *cond* is indented perfectly now, this was
just for the sake of example.)

What is done so far

I decided to just go ahead and try to write some Elisp indentation
machinery.

Actually, it seems to work fine on my laptop for me.  I'd be happy if
Emacs developers find my indentation actually useful and incorporate
some of it into Emacs.

If someone gets really interested in the idea of improving Emacs Lisp
indentation, I'm always eager to share my work.




--
View this message in context: http://emacs.1067599.n5.nabble.com/Better-indentation-for-elisp-tp278668.html
Sent from the Emacs - Dev mailing list archive at Nabble.com.

Re: Better indentation for elisp

[Thierry Volpiatto]

Sergey Mozgovoy <egnartsms@gmail.com> writes:

> The problem
>
> Standard indentation algorithm for Emacs Lisp is not perfect in some
> respects.  For example, it fails to indent code like this:
> *
> (flet ((sum (x y)
>                 (+ x y))
>         (diff (x y)
>                (- x y)))
>   ...)
> *
> or like this:
> *
> (func-1 (or arg-1
>                 arg-2) weird-arg
>                 another-arg)
> *
> and this:
> *
> (let (var-1 var-2
>                (x 10))
>   (+ var-1 var-2))
> *
>
> SLIME indentation
>
> Those who use SLIME know that there's a contrib *slime-cl-indent*
> which does some nice job of indenting Common Lisp code.  Aside from
> some CL-specific things (like the loop macro or tagbody), SLIME
> indentation is different from that of Elisp in that it supports
> arbitrarily-nesting indentation patterns rather than simple values.
>
> For example, here's indentation spec for *case* (quoted from
> docstring for *common-lisp-indent-function*):
> /
> ... the function `case' has an indent property (4 &rest (&whole 2 &rest 1)),
> meaning:
>   * indent the first argument by 4.
>   * arguments after the first should be lists, and there may be any number
>     of them.  The first list element has an offset of 2, all the rest
>     have an offset of 2+1=3.
> /
>
>
> SLIME => Emacs Lisp
>
> I think we could just reuse some SLIME indentation ideas for Emacs
> Lisp.  The algorithm is smart enough to handle any typical Elisp
> indentation, and it is kind of a generalization of the present one.
> General idea is this: walk nested lists up not more than some depth
> limit (*lisp-indent-maximum-backtracking*) and see what head
> symbols (*car*s) of ancestor lists say about indentation.  Upper
> specs override what was said by more local ones.  Whatever the current
> indentation we have when the depth is exhausted becomes our end
> result.
>
> One thing that I found odd in SLIME indentation is that they specify
> absolute offsets (in spaces).  I don't see why such a precision might
> be really needed.. to my mind, indentation should better be specified
> in terms of number of distinguished forms, like it is implemented now
> for Elisp.  Maybe I just don't get it.
>
> So I propose the following indentation patterns, roughly:
>   
>   * any valid value for *lisp-indent-function* symprop, meaning
>     is preserved.  This includes:
>       + natural number (# of distinguished args);
>       + *defun* symbol which means to indent in defun-style;
>       + *nil* stands for "usual" indentation, i.e. indent like a
>       	normal function call form;
>       + *:flat* -- new thing which tells that list should be
>       	flat, no indentation for any args;
>   
>   * (*S* . (*P-1* *P-2* ... *P-N*)), where *S* is a simple pattern
>     (any of above), and all *P-i* are any other patterns (recursive
> definition).
>     *S* specifies indentation for the enclosing list. b>P-1*, ..., *P-n*
> specify
>     indentation for sublists, beginning from the second one ("first
>     function argument");
>
>   * (*S* . (*P-1* ... *P-N* &rest *P*)), same as above but *P*
>     applies to all the other sublists after *N*.
>
> So for example, we'd give indentation for the above-mentioned forms
> like this:
> *
> (put 'flet 'lisp-indent-function '(1 (:flat &rest defun)))
> (put 'let 'lisp-indent-function '(1 (:flat &rest 1)))
> (put 'cond 'lisp-indent-function '(nil &rest :flat))
> *
> (Yes, I do agree that *cond* is indented perfectly now, this was
> just for the sake of example.)
>
> What is done so far
>
> I decided to just go ahead and try to write some Elisp indentation
> machinery.
>
> Actually, it seems to work fine on my laptop for me.  I'd be happy if
> Emacs developers find my indentation actually useful and incorporate
> some of it into Emacs.
>
> If someone gets really interested in the idea of improving Emacs Lisp
> indentation, I'm always eager to share my work.

See:

http://article.gmane.org/gmane.emacs.devel/155910/match=lisp+indent+function

-- 
Thierry
Get my Gnupg key:
gpg --keyserver pgp.mit.edu --recv-keys 59F29997 

Re: Better indentation for elisp

[Sergey Mozgovoy]

Thierry Volpiatto-2 wrote
> See: 
> 
> http://article.gmane.org/gmane.emacs.devel/155910/match=lisp+indent+function

Thank you for response and the link.

I guess you're sure that Emacs Lisp indentation is not so bad as to be
worth the effort of introducing substantial changes.  OK, I agree
that's probably my own censoriousness which bothers me, rather than
Elisp indentation. :)

However, please do note that aside from *flet*, there were some other
examples.  I guess they don't annoy pretty much anyone except for me,
too.

Actually, what struck me the most was the indentation of constructs
like this:
*
(let-like-macro (a 10
		           b 20
		           c 30)
  ... BODY ...)
*
Well, this is actually a let-like construct, but bindings are not
explicitly taken into additional parenthesis.  They are "unrolled".

What I would want here is /flat/ indentation, but present Elisp
indenting algorithm treats the binding list as a normal function call.

Yeah, I agree that the easiest way to handle this would be to just
give up and live with default indentation, or use some more
conventional syntactic constructs.



--
View this message in context: http://emacs.1067599.n5.nabble.com/Better-indentation-for-elisp-tp278668p278696.html
Sent from the Emacs - Dev mailing list archive at Nabble.com.

Re: Better indentation for elisp

[Thierry Volpiatto]

Sergey Mozgovoy <egnartsms@gmail.com> writes:

> Thierry Volpiatto-2 wrote
>> See: 
>> 
>> http://article.gmane.org/gmane.emacs.devel/155910/match=lisp+indent+function
>
> Thank you for response and the link.
>
> I guess you're sure that Emacs Lisp indentation is not so bad as to be
> worth the effort of introducing substantial changes.  OK, I agree
> that's probably my own censoriousness which bothers me, rather than
> Elisp indentation. :)
>
> However, please do note that aside from *flet*, there were some other
> examples.  I guess they don't annoy pretty much anyone except for me,
> too.
>
> Actually, what struck me the most was the indentation of constructs
> like this:
> *
> (let-like-macro (a 10
> 		           b 20
> 		           c 30)
>   ... BODY ...)
> *
> Well, this is actually a let-like construct, but bindings are not
> explicitly taken into additional parenthesis.  They are "unrolled".
>
> What I would want here is /flat/ indentation, but present Elisp
> indenting algorithm treats the binding list as a normal function call.
>
> Yeah, I agree that the easiest way to handle this would be to just
> give up and live with default indentation, or use some more
> conventional syntactic constructs.

Here my settings, if that help:

(eval-after-load "cl-indent.el"
  (let ((l '((flet ((&whole 4 &rest (&whole 1 &lambda &body)) &body))
             (cl-flet* . flet)
             (labels . flet)
             (cl-flet . flet)
             (cl-labels . flet)
             (cl-macrolet . flet))))
    (dolist (el l)
      (put (car el) 'common-lisp-indent-function
           (if (symbolp (cdr el))
               (get (cdr el) 'common-lisp-indent-function)
               (car (cdr el)))))))

(dolist (mode '(emacs-lisp-mode lisp-interaction-mode))
  (font-lock-add-keywords
   mode
   '(("(\\<\\(cl-flet[*]?\\|cl-labels\\|cl-macrolet\\)\\>" 1 font-lock-keyword-face)
     ("(\\<\\(cl-loop\\|cl-dolist\\)\\>" 1 font-lock-keyword-face))))

-- 
Thierry
Get my Gnupg key:
gpg --keyserver pgp.mit.edu --recv-keys 59F29997 

Re: Better indentation for elisp

[Stefan Monnier]

> I think we could just reuse some SLIME indentation ideas for Emacs
> Lisp.  The algorithm is smart enough to handle any typical Elisp
> indentation, and it is kind of a generalization of the present one.

OT1H I like the idea, but OTOH I also like the fact that the simple
indentation strategy used in Elisp works well enough for almost all
of Elisp.

So, I think I could be convinced, but only if the new indentation
algorithm is sufficiently simple.  I generally like the indentation
specifications you suggest (except for `defun': why do we need it?).


        Stefan

Re: Better indentation for elisp

[João Távora]

On Tue, Feb 19, 2013 at 1:53 PM, Stefan Monnier
<monnier@iro.umontreal.ca> wrote:
>> I think we could just reuse some SLIME indentation ideas for Emacs
>> Lisp.  [...]
> So, I think I could be convinced, but only if the new indentation
> algorithm is sufficiently simple.  [...]

I don't think this is in the original proposal, but the most
interesting indentation feature in SLIME, at least for me, is that is
automatically discovers how to indent user macros. See
http://common-lisp.net/project/slime/doc/html/Semantic-indentation.html

--
João Távora

Re: Better indentation for elisp

[Sergey Mozgovoy]

>  I generally like the indentation specifications you suggest (except
> for `defun': why do we need it?). 

For backward compatibility.  Aside from this, I'm sure we do not, at
all.


> So, I think I could be convinced, but only if the new indentation 
> algorithm is sufficiently simple.

Well, my algorithm is essentially the same in logic as the standard
one, plus some code for matching indentation patterns and walking up
through containing sexps.  This is 300-350 LOC (including heavy
comments) which would supersede existing 265 lines in "lisp-mode.el".

Obviously, it is not tested and verified thoroughly.

If you're interested, I can just post here a diff against "lisp-mode.el".



--
View this message in context: http://emacs.1067599.n5.nabble.com/Better-indentation-for-elisp-tp278668p278835.html
Sent from the Emacs - Dev mailing list archive at Nabble.com.

Re: Better indentation for elisp

[Stefan Monnier]

>> I generally like the indentation specifications you suggest (except
>> for `defun': why do we need it?). 
> For backward compatibility.  Aside from this, I'm sure we do not, at all.

OK, that makes sense.

>> So, I think I could be convinced, but only if the new indentation 
>> algorithm is sufficiently simple.
> Well, my algorithm is essentially the same in logic as the standard
> one, plus some code for matching indentation patterns and walking up
> through containing sexps.  This is 300-350 LOC (including heavy
> comments) which would supersede existing 265 lines in "lisp-mode.el".
> Obviously, it is not tested and verified thoroughly.
> If you're interested, I can just post here a diff against "lisp-mode.el".

Please do,


        Stefan

Re: Better indentation for elisp

[Sergey Mozgovoy]

There are 400 lines of code, after I added some better documentation and
comments.  I decided to share it via  pastebin
<http://pastebin.com/QXGGU0rW>  , rather than to paste
the whole code directly in a message.. Excuse me if that's not an acceptable
way to share code here, it's just the first time I do it.

Some points to note:
  1. New functions and macros were given "msv-" prefix (author's initials).
      This should be changed.
  
  2. As we cannot use CL package in "lisp-mode.el", I decided to
"reimplement"
      loop and block/return-from.  The other option would be to complicate
      code with deeper nesting by using only primitive Elisp control flow
      structures.  I don't like it myself.. I just cannot afford anything
better
      at the moment.  AFAIK, the problem of partial reimplementation of CL
      package in every Emacs library is well-known.



--
View this message in context: http://emacs.1067599.n5.nabble.com/Better-indentation-for-elisp-tp278668p278923.html
Sent from the Emacs - Dev mailing list archive at Nabble.com.

Re: Better indentation for elisp

[Stefan Monnier]

> There are 400 lines of code, after I added some better documentation and
> comments.  I decided to share it via  pastebin
> <http://pastebin.com/QXGGU0rW>  , rather than to paste
> the whole code directly in a message.. Excuse me if that's not an acceptable
> way to share code here, it's just the first time I do it.

It's perfectly OK to send a 400-line chunk of code/patch here.
It's also OK to send it via pastebin (tho I personally like it less,
because when you try to find this code via a search engine, you
typically only find the pastebin page and then have trouble finding the
corresponding context (e.g. email in which the patch was mentioned)).


        Stefan "Who hasn't yet had time to look at it"

Re: Better indentation for elisp

[Didier Verna]

Sergey Mozgovoy <egnartsms@gmail.com> wrote:

> Standard indentation algorithm for Emacs Lisp is not perfect in some
> respects.

> Those who use SLIME know that there's a contrib *slime-cl-indent*
> which does some nice job of indenting Common Lisp code.  SLIME
> indentation is different from that of Elisp in that it supports
> arbitrarily-nesting indentation patterns rather than simple values.

> I decided to just go ahead and try to write some Elisp indentation
> machinery.

  Given that you think (and I agree) that the current Slime indentation
  contrib is superior to the current one for Emacs Lisp, I don't quite
  understand why you would want to write some new machinery.

  The Slime contrib provides customizable indentation through the notion
  of styles. Can't you just write an Emacs Lisp style that suits your
  needs?

-- 
ELS 2013, June 3/4, Madrid, Spain:  http://els2013.european-lisp-symposium.org

Scientific site:   http://www.lrde.epita.fr/~didier
Music (Jazz) site: http://www.didierverna.com

Re: Better indentation for elisp

[Sergey Mozgovoy]

Didier Verna-3 wrote
> Given that you think (and I agree) that the current Slime indentation
> contrib is superior to the current one for Emacs Lisp, I don't quite
> understand why you would want to write some new machinery.

Well, there are basically 2 reasons:

    * Emacs Lisp indentation mustn't in any way depend on SLIME.
      Emacs Lisp is a vital language and mode for Emacs, whereas SLIME
      is just an ordinary extension package like, for instance,
      Python or JavaScript mode.

      So doing something like *(require 'slime)* from "lisp-mode.el" is
      definitely wrong.  Thus, we can either steal some ideas from
      SLIME (and probably code, as well), or revert this dependency:
      implement the whole customizable, pluggable and generally quite
      complex algorithm in "lisp-mode.el" and adjust SLIME and other
      lisp modes into this model.

      I just chose the first path, for my personal use (at the time
      being).

    * I don't like the fact that they specify absolute indentation in
      spaces, in SLIME. :) I'd agree that this is my subjective
      attitude and may not be a general problem for most other people,
      though.

On the whole, the main reason why I started this post is that I'd
want to see Emacs Lisp have really better indentation support.  This
wouldn't require too much effort to implement, yet this would make it
much more convenient to use *macrolet*, *flet*, and other high-level
macros in Elisp.



--
View this message in context: http://emacs.1067599.n5.nabble.com/Better-indentation-for-elisp-tp278668p279172.html
Sent from the Emacs - Dev mailing list archive at Nabble.com.