bug#1012: calculate-lisp-indent

bug#1012: calculate-lisp-indent

[Markus Sauermann]

[-- Attachment #1: Type: text/plain, Size: 2912 bytes --]

Hello,

I found a bug concerning the function calculate-lisp-indent in lisp-mode.el.
It is triggered in the following scenarios (-!- marks the point):

---begin scenario 1 buffer *scratch*---
(,@foo
 :bar)-!-
---end scenario 1 buffer *scratch*---

---begin scenario 2 buffer *scratch*---
( foo
 :bar)-!-
---end scenario 2 buffer *scratch*---

Using <tab> (bound to the function lisp-indent-line) triggers the
following error in both scenarios:

---begin error---
forward-sexp: Scan error: "Containing expression ends prematurely", 192, 192
---end error---



I traced this bug to the function calculate-lisp-indent.

As far as I understand the code, the characters ",@" in scenario 1 and
the spaces in scenario 2 are not treated correctly.

In this E-Mail I included a patch, that solves the problems for me, and
was created with the command

mhoram@revelstone:~/emacs/trunk/emacs/lisp$ cvs diff -c >
~/emacs-bugreport.txt



The patch works for scenario 1 by an additional call of the
(backward-prefix-chars) function, which sets point back to the beginning
of ",@".
For scenario 2 the addition of "\\|([ \t]+" solves the problem, by not
entering the while-loop (and thus calling the function (forward-sexp
-1)), if there are only spaces between the "(" and the first object.



Regards,
Markus Sauermann



In GNU Emacs 22.3.1 (i386-mingw-nt6.0.6001)
 of 2008-09-06 on SOFT-MJASON
Windowing system distributor `Microsoft Corp.', version 6.0.6001
configured using `configure --with-gcc (3.4)'

Important settings:
  value of $LC_ALL: nil
  value of $LC_COLLATE: nil
  value of $LC_CTYPE: nil
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: nil
  value of $LANG: DEU
  locale-coding-system: cp1252
  default-enable-multibyte-characters: t

Major mode: Lisp Interaction

Minor modes in effect:
  encoded-kbd-mode: t
  tooltip-mode: t
  tool-bar-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  unify-8859-on-encoding-mode: t
  utf-translate-cjk-mode: t
  auto-compression-mode: t
  line-number-mode: t

Recent input:
<help-echo> <help-echo> <help-echo> <help-echo> q (
, @ f o o <return> <tab> : b a r ) <tab> M-x r e p
o r t <tab> <return>

Recent messages:
("D:\\Uninstalled\\emacs-22.3\\bin\\emacs.exe" "-q")
Loading encoded-kb...done
For information about GNU Emacs and the GNU system, type C-h C-a.
forward-sexp: Scan error: "Containing expression ends prematurely", 192, 192
Loading emacsbug...
Loading regexp-opt...done
Loading emacsbug...done


-- 
Markus Sauermann      E-Mail: info@sauermann-consulting.de
Clemensstr. 55 Rgb.   Web: http://www.sauermann-consulting.de
80803 München         Tel: 089/337707, 0179/9879005, Fax: 089/38476434


[-- Attachment #2: emacs-bugreport.txt --]
[-- Type: text/plain, Size: 2011 bytes --]

Index: ChangeLog
===================================================================
RCS file: /sources/emacs/emacs/lisp/ChangeLog,v
retrieving revision 1.14493
diff -c -r1.14493 ChangeLog
*** ChangeLog	20 Sep 2008 22:09:39 -0000	1.14493
--- ChangeLog	21 Sep 2008 17:08:59 -0000
***************
*** 1,3 ****
--- 1,9 ----
+ 2008-09-21  Markus Sauermann <markus@sauermann-consulting.de>
+ 
+ 	* emacs-lisp/lisp-mode.el (calculate-lisp-indent):
+ 	Fix indentation problem with keyword symbols when a list starts
+ 	with ,@ or spaces.
+ 
  2008-09-20  Vincent Belaïche  <vincent.b.1@hotmail.fr>
  
  	* calc/calc-vec.el (calcFunc-venum): Properly handle intervals.
Index: emacs-lisp/lisp-mode.el
===================================================================
RCS file: /sources/emacs/emacs/lisp/emacs-lisp/lisp-mode.el,v
retrieving revision 1.226
diff -c -r1.226 lisp-mode.el
*** emacs-lisp/lisp-mode.el	20 Sep 2008 21:54:44 -0000	1.226
--- emacs-lisp/lisp-mode.el	21 Sep 2008 17:09:04 -0000
***************
*** 1027,1033 ****
                       ;; where it begins, so find that one, instead.
                       (save-excursion
                         (goto-char calculate-lisp-indent-last-sexp)
!                        (while (and (not (looking-back "^[ \t]*"))
                                     (or (not containing-sexp)
                                         (< (1+ containing-sexp) (point))))
                           (forward-sexp -1)
--- 1027,1034 ----
                       ;; where it begins, so find that one, instead.
                       (save-excursion
                         (goto-char calculate-lisp-indent-last-sexp)
!                        (backward-prefix-chars)
!                        (while (and (not (looking-back "^[ \t]*\\|([ \t]+"))
                                     (or (not containing-sexp)
                                         (< (1+ containing-sexp) (point))))
                           (forward-sexp -1)

bug#1012: marked as done (calculate-lisp-indent)

[Emacs bug Tracking System]

[-- Attachment #1: Type: text/plain, Size: 811 bytes --]


Your message dated Wed, 24 Sep 2008 10:16:33 +0200
with message-id <48D9F761.1000805@gmx.at>
and subject line Re: bug#1012: calculate-lisp-indent
has caused the Emacs bug report #1012,
regarding calculate-lisp-indent
to be marked as done.

This means that you claim that the problem has been dealt with.
If this is not the case it is now your responsibility to reopen the
bug report if necessary, and/or fix the problem forthwith.

(NB: If you are a system administrator and have no idea what this
message is talking about, this may indicate a serious mail system
misconfiguration somewhere. Please contact don@donarmstrong.com
immediately.)


-- 
1012: http://emacsbugs.donarmstrong.com/cgi-bin/bugreport.cgi?bug=1012
Emacs Bug Tracking System
Contact don@donarmstrong.com with problems

[-- Attachment #2: Type: message/rfc822, Size: 7698 bytes --]

[-- Attachment #2.1.1: Type: text/plain, Size: 2912 bytes --]

Hello,

I found a bug concerning the function calculate-lisp-indent in lisp-mode.el.
It is triggered in the following scenarios (-!- marks the point):

---begin scenario 1 buffer *scratch*---
(,@foo
 :bar)-!-
---end scenario 1 buffer *scratch*---

---begin scenario 2 buffer *scratch*---
( foo
 :bar)-!-
---end scenario 2 buffer *scratch*---

Using <tab> (bound to the function lisp-indent-line) triggers the
following error in both scenarios:

---begin error---
forward-sexp: Scan error: "Containing expression ends prematurely", 192, 192
---end error---



I traced this bug to the function calculate-lisp-indent.

As far as I understand the code, the characters ",@" in scenario 1 and
the spaces in scenario 2 are not treated correctly.

In this E-Mail I included a patch, that solves the problems for me, and
was created with the command

mhoram@revelstone:~/emacs/trunk/emacs/lisp$ cvs diff -c >
~/emacs-bugreport.txt



The patch works for scenario 1 by an additional call of the
(backward-prefix-chars) function, which sets point back to the beginning
of ",@".
For scenario 2 the addition of "\\|([ \t]+" solves the problem, by not
entering the while-loop (and thus calling the function (forward-sexp
-1)), if there are only spaces between the "(" and the first object.



Regards,
Markus Sauermann



In GNU Emacs 22.3.1 (i386-mingw-nt6.0.6001)
 of 2008-09-06 on SOFT-MJASON
Windowing system distributor `Microsoft Corp.', version 6.0.6001
configured using `configure --with-gcc (3.4)'

Important settings:
  value of $LC_ALL: nil
  value of $LC_COLLATE: nil
  value of $LC_CTYPE: nil
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: nil
  value of $LANG: DEU
  locale-coding-system: cp1252
  default-enable-multibyte-characters: t

Major mode: Lisp Interaction

Minor modes in effect:
  encoded-kbd-mode: t
  tooltip-mode: t
  tool-bar-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  unify-8859-on-encoding-mode: t
  utf-translate-cjk-mode: t
  auto-compression-mode: t
  line-number-mode: t

Recent input:
<help-echo> <help-echo> <help-echo> <help-echo> q (
, @ f o o <return> <tab> : b a r ) <tab> M-x r e p
o r t <tab> <return>

Recent messages:
("D:\\Uninstalled\\emacs-22.3\\bin\\emacs.exe" "-q")
Loading encoded-kb...done
For information about GNU Emacs and the GNU system, type C-h C-a.
forward-sexp: Scan error: "Containing expression ends prematurely", 192, 192
Loading emacsbug...
Loading regexp-opt...done
Loading emacsbug...done


-- 
Markus Sauermann      E-Mail: info@sauermann-consulting.de
Clemensstr. 55 Rgb.   Web: http://www.sauermann-consulting.de
80803 München         Tel: 089/337707, 0179/9879005, Fax: 089/38476434


[-- Attachment #2.1.2: emacs-bugreport.txt --]
[-- Type: text/plain, Size: 2011 bytes --]

Index: ChangeLog
===================================================================
RCS file: /sources/emacs/emacs/lisp/ChangeLog,v
retrieving revision 1.14493
diff -c -r1.14493 ChangeLog
*** ChangeLog	20 Sep 2008 22:09:39 -0000	1.14493
--- ChangeLog	21 Sep 2008 17:08:59 -0000
***************
*** 1,3 ****
--- 1,9 ----
+ 2008-09-21  Markus Sauermann <markus@sauermann-consulting.de>
+ 
+ 	* emacs-lisp/lisp-mode.el (calculate-lisp-indent):
+ 	Fix indentation problem with keyword symbols when a list starts
+ 	with ,@ or spaces.
+ 
  2008-09-20  Vincent Belaïche  <vincent.b.1@hotmail.fr>
  
  	* calc/calc-vec.el (calcFunc-venum): Properly handle intervals.
Index: emacs-lisp/lisp-mode.el
===================================================================
RCS file: /sources/emacs/emacs/lisp/emacs-lisp/lisp-mode.el,v
retrieving revision 1.226
diff -c -r1.226 lisp-mode.el
*** emacs-lisp/lisp-mode.el	20 Sep 2008 21:54:44 -0000	1.226
--- emacs-lisp/lisp-mode.el	21 Sep 2008 17:09:04 -0000
***************
*** 1027,1033 ****
                       ;; where it begins, so find that one, instead.
                       (save-excursion
                         (goto-char calculate-lisp-indent-last-sexp)
!                        (while (and (not (looking-back "^[ \t]*"))
                                     (or (not containing-sexp)
                                         (< (1+ containing-sexp) (point))))
                           (forward-sexp -1)
--- 1027,1034 ----
                       ;; where it begins, so find that one, instead.
                       (save-excursion
                         (goto-char calculate-lisp-indent-last-sexp)
!                        (backward-prefix-chars)
!                        (while (and (not (looking-back "^[ \t]*\\|([ \t]+"))
                                     (or (not containing-sexp)
                                         (< (1+ containing-sexp) (point))))
                           (forward-sexp -1)

[-- Attachment #3: Type: message/rfc822, Size: 1688 bytes --]

From: martin rudalics <rudalics@gmx.at>
To: 1012-done@emacsbugs.donarmstrong.com
Cc: Markus Sauermann <markus@sauermann-consulting.de>
Subject: Re: bug#1012: calculate-lisp-indent
Date: Wed, 24 Sep 2008 10:16:33 +0200
Message-ID: <48D9F761.1000805@gmx.at>

Fixed as:

2008-09-23  Markus Sauermann <markus@sauermann-consulting.de>  (tiny change)

	* emacs-lisp/emacslisp-mode.el (calculate-lisp-indent): Fix
	indentation problem with keyword symbols when a list starts with
	,@ or spaces.  (Bug#1012)

Thanks for the patch, martin

bug#1012: Indentation of constant symbols

[Markus Sauermann]

[-- Attachment #1: Type: text/plain, Size: 4771 bytes --]

Hello,

my bug fix was not farsighted enough. Basically I fixed a part of the
source code, which does nothing at all.
I dug a bit deeper and noticed, that the following "source code 1" of
the function calculate-lisp-indent in lisp-mode.el is a little bugged.

--- BEGIN source code 1 ---
                ;; If the function has no special alignment
		;; or it does not apply to this argument,
		;; try to align a constant-symbol under the last
                ;; preceding constant symbol, if there is such one of
                ;; the last 2 preceding symbols, in the previous
                ;; uncommented line.
                (and (save-excursion
                       (goto-char indent-point)
                       (skip-chars-forward " \t")
                       (looking-at ":"))
                     ;; The last sexp may not be at the indentation
                     ;; where it begins, so find that one, instead.
                     (save-excursion
                       (goto-char calculate-lisp-indent-last-sexp)
		       ;; Handle prefix characters and whitespace
		       ;; following an open paren.  (Bug#1012)
                       (backward-prefix-chars)
                       (while (and (not (looking-back "^[ \t]*\\|([ \t]+"))
                                   (or (not containing-sexp)
                                       (< (1+ containing-sexp) (point))))
                         (forward-sexp -1)
                         (backward-prefix-chars))
                       (setq calculate-lisp-indent-last-sexp (point)))
                     (> calculate-lisp-indent-last-sexp
                        (save-excursion
                          (goto-char (1+ containing-sexp))
                          (parse-partial-sexp (point)
calculate-lisp-indent-last-sexp 0 t)
                          (point)))
                     (let ((parse-sexp-ignore-comments t)
                           indent)
                       (goto-char calculate-lisp-indent-last-sexp)
                       (or (and (looking-at ":")
                                (setq indent (current-column)))
                           (and (< (save-excursion (beginning-of-line)
(point))
                                   (prog2 (backward-sexp) (point)))
                                (looking-at ":")
                                (setq indent (current-column))))
                       indent))
--- END source code 1 ---

One part of that code implements the functionality as described in the
comment at the beginning of the code. The other part tries to fix a
problem, that is caused by this functionality, which is denoted in
example 1, where :baz should be aligned under :foo and not under :foo2.

--- BEGIN example 1 ---
(:foo bar :foo2 bar2
          :baz ar)
--- END example 1 ---

But this other part makes constant symbols to be indented in the same
way as the usual indentation does. So discarding the whole code does not
change any indentation except in some rare, mostly unused cases. See
examples 2 for code snippets that are indented in the same way with or
without this code.

--- BEGIN examples 2 ---
(:foo bar
      :baz)

(foo bar :baz
     :foo2)

(foo bar :baz bor
     :foo2)

(:foo bar (:bar baz
                :asd))
--- END examples 2 ---

These examples also contradict the description of the source code. I
even think that the description of the code is not what the special
handling of constant symbols should be like, as it causes the
indentation described in example 1.

So I suggest to treat constant symbols in the following way:

If the sexp, preceding the constant symbol, begins in the same line as
the containing sexp, then align it under the first appearance of a
constant symbol between the beginning of the containing sexp and
the the begin of the preceding sexp.

If the sexp, preceding the constant symbol, does not begin in the same
line as the containing sexp, then special treatment is not necessary,
because it is handled by the usual indentation.

Do not treat a constant symbol in a special way, if the containing sexp
is a vector, since this would cause indentations in unwanted places.

After applying "patch-1.txt", which implements this suggestions, the
code in examples 2 is indented as shown in examples 3.

--- BEGIN examples 3 ---
(:foo bar
 :baz)

(foo bar :baz
         :foo2)

(foo bar :baz bor
         :foo2)

(:foo bar (:bar baz
           :asd))
--- END examples 3 ---

This implementation still has shortcomings in several unusual
conditions, as examples 4 show, but I think this patch covers all common
usages.

--- BEGIN examples 4 ---
(:foo cons :foo2
      (bar baz :asd asd)
      :sad)

(:foo (:bar baz
       :asd asd) :con con2
       :sad)
--- END examples 4 ---



Regards,
Markus Sauermann

[-- Attachment #2: patch-1.txt --]
[-- Type: text/plain, Size: 4681 bytes --]

Index: lisp-mode.el
===================================================================
RCS file: /sources/emacs/emacs/lisp/emacs-lisp/lisp-mode.el,v
retrieving revision 1.228
diff -c -r1.228 lisp-mode.el
*** lisp-mode.el	23 Sep 2008 17:59:28 -0000	1.228
--- lisp-mode.el	1 Oct 2008 19:10:51 -0000
***************
*** 1014,1055 ****
                       (not retry)
                       (funcall lisp-indent-function indent-point state))
                  ;; If the function has no special alignment
! 		;; or it does not apply to this argument,
! 		;; try to align a constant-symbol under the last
!                 ;; preceding constant symbol, if there is such one of
!                 ;; the last 2 preceding symbols, in the previous
!                 ;; uncommented line.
                  (and (save-excursion
                         (goto-char indent-point)
                         (skip-chars-forward " \t")
                         (looking-at ":"))
!                      ;; The last sexp may not be at the indentation
!                      ;; where it begins, so find that one, instead.
!                      (save-excursion
!                        (goto-char calculate-lisp-indent-last-sexp)
! 		       ;; Handle prefix characters and whitespace
! 		       ;; following an open paren.  (Bug#1012)
!                        (backward-prefix-chars)
!                        (while (and (not (looking-back "^[ \t]*\\|([ \t]+"))
!                                    (or (not containing-sexp)
!                                        (< (1+ containing-sexp) (point))))
!                          (forward-sexp -1)
!                          (backward-prefix-chars))
!                        (setq calculate-lisp-indent-last-sexp (point)))
!                      (> calculate-lisp-indent-last-sexp
!                         (save-excursion
!                           (goto-char (1+ containing-sexp))
!                           (parse-partial-sexp (point) calculate-lisp-indent-last-sexp 0 t)
!                           (point)))
                       (let ((parse-sexp-ignore-comments t)
                             indent)
!                        (goto-char calculate-lisp-indent-last-sexp)
!                        (or (and (looking-at ":")
!                                 (setq indent (current-column)))
!                            (and (< (save-excursion (beginning-of-line) (point))
!                                    (prog2 (backward-sexp) (point)))
!                                 (looking-at ":")
!                                 (setq indent (current-column))))
                         indent))
                  ;; another symbols or constants not preceded by a constant
                  ;; as defined above.
--- 1014,1046 ----
                       (not retry)
                       (funcall lisp-indent-function indent-point state))
                  ;; If the function has no special alignment
!                 ;; or it does not apply to this argument,
!                 ;; try to treat a constant symbol in a special way.
                  (and (save-excursion
                         (goto-char indent-point)
                         (skip-chars-forward " \t")
                         (looking-at ":"))
!                      ;; Treat it only special, if the sexp preceding it begins
!                      ;; in the same line as the containing sexp
!                      (< calculate-lisp-indent-last-sexp
!                         (save-excursion (goto-char containing-sexp)
!                                         (end-of-line) (point)))
!                      ;; Don't treat it special, if the containing sexp is a
!                      ;; vector
!                      (save-excursion (goto-char containing-sexp)
!                                      (not (looking-at "\\[")))
!                      ;; Align it under the first constant symbol, if any,
!                      ;; between the beginning of the containing sexp and
!                      ;; the preceding sexp
                       (let ((parse-sexp-ignore-comments t)
                             indent)
!                        (goto-char (1+ containing-sexp))
!                        (while (and (<= (point) calculate-lisp-indent-last-sexp)
!                                    (not indent))
!                          (skip-syntax-forward "-")
!                          (if (looking-at ":")
!                              (setq indent (current-column))
!                            (forward-sexp)))
                         indent))
                  ;; another symbols or constants not preceded by a constant
                  ;; as defined above.

bug#1012: calculate-lisp-indent

[martin rudalics]

 > In this E-Mail I included a patch, that solves the problems for me, and
 > was created with the command
 >
 > mhoram@revelstone:~/emacs/trunk/emacs/lisp$ cvs diff -c >
 > ~/emacs-bugreport.txt
 >
 > The patch works for scenario 1 by an additional call of the
 > (backward-prefix-chars) function, which sets point back to the beginning
 > of ",@".
 > For scenario 2 the addition of "\\|([ \t]+" solves the problem, by not
 > entering the while-loop (and thus calling the function (forward-sexp
 > -1)), if there are only spaces between the "(" and the first object.

Thank you, applied.

martin

bug#1012: calculate-lisp-indent

[Andrew Hyatt]

martin rudalics <rudalics@gmx.at> writes:

>> In this E-Mail I included a patch, that solves the problems for me, and
>> was created with the command
>>
>> mhoram@revelstone:~/emacs/trunk/emacs/lisp$ cvs diff -c >
>> ~/emacs-bugreport.txt
>>
>> The patch works for scenario 1 by an additional call of the
>> (backward-prefix-chars) function, which sets point back to the beginning
>> of ",@".
>> For scenario 2 the addition of "\\|([ \t]+" solves the problem, by not
>> entering the while-loop (and thus calling the function (forward-sexp
>> -1)), if there are only spaces between the "(" and the first object.
>
> Thank you, applied.
>
> martin

This bug is still marked open, but I can't reproduce either of the
reported issues against Emacs 25. Can either of you still reproduce the
issue? If not, I'll close the bug.

bug#1012: calculate-lisp-indent

[martin rudalics]

 > This bug is still marked open, but I can't reproduce either of the
 > reported issues against Emacs 25. Can either of you still reproduce the
 > issue?

Did you look into the snippets from Example 2 and Example 3 in the
follow-up message?  All issues reported by Markus still apply, at least
here.  Unfortunately, I rarely indent constant symbols, so someone else
would have had to chime in.  But I suppose nobody really cared.

Note also that the second patch is probably too large to qualify as a
tiny change and I have no idea whether Markus is still around to do the
paperwork.

martin

bug#1012: calculate-lisp-indent

[Markus Sauermann]

Hi,

Am 30.12.2015 um 08:27 schrieb martin rudalics:
> Did you look into the snippets from Example 2 and Example 3 in the
> follow-up message?  All issues reported by Markus still apply, at least
> here.  Unfortunately, I rarely indent constant symbols, so someone else
> would have had to chime in.  But I suppose nobody really cared.
> 
> Note also that the second patch is probably too large to qualify as a
> tiny change and I have no idea whether Markus is still around to do the
> paperwork.

still around and still using emacs, but I haven't experienced this issue
during the last 7 years. So I am not very inclined to review my old
patches. Sorry.

Best wishes
Markus

bug#1012: calculate-lisp-indent

[martin rudalics]

 > still around and still using emacs,

That's nice to hear.

 > but I haven't experienced this issue
 > during the last 7 years. So I am not very inclined to review my old
 > patches. Sorry.

I close the bug then.  Whenever you feel an urge to work on this again,
please reopen it.

Thanks, martin

bug#1012: calculate-lisp-indent

[Andrew Hyatt]

Markus Sauermann <markus@sauermann-consulting.de> writes:

> Hi,
>
> Am 30.12.2015 um 08:27 schrieb martin rudalics:
>> Did you look into the snippets from Example 2 and Example 3 in the
>> follow-up message?  All issues reported by Markus still apply, at least
>> here.  Unfortunately, I rarely indent constant symbols, so someone else
>> would have had to chime in.  But I suppose nobody really cared.
>> 
>> Note also that the second patch is probably too large to qualify as a
>> tiny change and I have no idea whether Markus is still around to do the
>> paperwork.
>
> still around and still using emacs, but I haven't experienced this issue
> during the last 7 years. So I am not very inclined to review my old
> patches. Sorry.
>
> Best wishes
> Markus

Thanks for the update.  Martin, the original bug was fairly serious - an
error while trying to indent.  Unless I misunderstand, the followup
issues reported were issues of not indenting "properly", which are much
less serious.  Also, I'm not clear as to what should be a bug here - is
there some documentation on how the lisp indenting should work, so that
we know how things should ideally indent?

In any case, the original bug seems to be fixed.  Martin, if you have
issues with lisp indenting style, it probably would be best to create a
separate bug on it.  I'll mark this one as fixed in the meantime.
Thanks, everyone!