bug#30891: 26.0.91; indent of malformed lisp

bug#30891: 26.0.91; indent of malformed lisp

[Noam Postavsky]

[-- Attachment #1: Type: text/plain, Size: 1188 bytes --]

I recently got a private report about a problem with the new lisp indent
functions: they don't work well when the buffer contains syntax errors.
For example, indent-region in a buffer containing:

)
(+ 2
   3)

gives

Debugger entered--Lisp error: (wrong-type-argument consp nil)
  lisp-indent-calc-next(#s(lisp-indent-state :stack (nil) :ppss (0 nil nil nil nil nil 0 nil nil nil nil) :ppss-point 146))
  lisp-indent-region(146 158)
  indent-region(146 158 nil)
  funcall-interactively(indent-region 146 158 nil)
  call-interactively(indent-region nil nil)
  command-execute(indent-region)

The error can be fixed with the patch below, I think it's safe for
emacs-26.  But, it still gives wrong indentation if you do indent-region
on the latter two lines.  That could be fixed with

    (advice-add 'lisp-ppss :override
                #'lisp-ppss-open-paren-in-column-0-is-defun-start)

The lisp-ppss-open-paren-in-column-0-is-defun-start function is included
(but uncalled) in the patch.  Using it in the override suggested above
will bring back Bug#27920 though.  We could include the function and
suggest to use it (with the aforementioned caveat) in etc/NEWS or
PROBLEMS perhaps?



[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: patch --]
[-- Type: text/x-diff, Size: 1941 bytes --]

From da0916b591a983c9b4ee9c8201600a77ba506b92 Mon Sep 17 00:00:00 2001
From: Noam Postavsky <npostavs@gmail.com>
Date: Sat, 17 Mar 2018 21:14:11 -0400
Subject: [PATCH v1] Handle indentation of malformed Lisp

* lisp/emacs-lisp/lisp-mode.el
(lisp-ppss-open-paren-in-column-0-is-defun-start): New function.
* lisp/emacs-lisp/lisp-mode.el (lisp-indent-calc-next): If we run out
of indent stack, reset the parse state.
---
 lisp/emacs-lisp/lisp-mode.el | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)

diff --git a/lisp/emacs-lisp/lisp-mode.el b/lisp/emacs-lisp/lisp-mode.el
index f082983d48..58710e9876 100644
--- a/lisp/emacs-lisp/lisp-mode.el
+++ b/lisp/emacs-lisp/lisp-mode.el
@@ -794,6 +794,21 @@ lisp-ppss
           (parse-partial-sexp sexp-start pos nil nil (syntax-ppss sexp-start)))
       pss)))
 
+(defun lisp-ppss-open-paren-in-column-0-is-defun-start (&optional pos)
+ (save-excursion
+   (if pos
+       (goto-char pos)
+     (setq pos (point)))
+   (or (looking-at-p "^(")
+       (re-search-backward "^(" nil t)
+       (goto-char (point-min)))
+   (let ((ppss (parse-partial-sexp (point) pos)))
+     (if (< (car ppss) 0)
+         ;; Too many close parens, probably syntax error.  Give a
+         ;; fresh state.
+         (parse-partial-sexp (point) (point))
+       ppss))))
+
 (cl-defstruct (lisp-indent-state
                (:constructor nil)
                (:constructor lisp-indent-initial-state
@@ -844,6 +859,10 @@ lisp-indent-calc-next
     (prog1
         (let (indent)
           (cond ((= (forward-line 1) 1) nil)
+                ;; Negative depth, probably some kind of syntax error.
+                ((null indent-stack)
+                 ;; Reset state.
+                 (setq ppss (parse-partial-sexp (point) (point))))
                 ((car indent-stack))
                 ((integerp (setq indent (calculate-lisp-indent ppss)))
                  (setf (car indent-stack) indent))
-- 
2.11.0

bug#30891: 26.0.91; indent of malformed lisp

[Eli Zaretskii]

> From: Noam Postavsky <npostavs@gmail.com>
> Date: Tue, 20 Mar 2018 21:19:35 -0400
> 
> I recently got a private report about a problem with the new lisp indent
> functions: they don't work well when the buffer contains syntax errors.
> For example, indent-region in a buffer containing:
> 
> )
> (+ 2
>    3)
> 
> gives
> 
> Debugger entered--Lisp error: (wrong-type-argument consp nil)
>   lisp-indent-calc-next(#s(lisp-indent-state :stack (nil) :ppss (0 nil nil nil nil nil 0 nil nil nil nil) :ppss-point 146))
>   lisp-indent-region(146 158)
>   indent-region(146 158 nil)
>   funcall-interactively(indent-region 146 158 nil)
>   call-interactively(indent-region nil nil)
>   command-execute(indent-region)
> 
> The error can be fixed with the patch below, I think it's safe for
> emacs-26.  But, it still gives wrong indentation if you do indent-region
> on the latter two lines.  That could be fixed with
> 
>     (advice-add 'lisp-ppss :override
>                 #'lisp-ppss-open-paren-in-column-0-is-defun-start)
> 
> The lisp-ppss-open-paren-in-column-0-is-defun-start function is included
> (but uncalled) in the patch.  Using it in the override suggested above
> will bring back Bug#27920 though.  We could include the function and
> suggest to use it (with the aforementioned caveat) in etc/NEWS or
> PROBLEMS perhaps?

I'd rather go back to the old code, and let the new Lisp indent
functions mature on master.  Alternatively, we could leave this
problem alone and fix it later.  But making all these changes at the
95th minute before Emacs 26 is released makes very little sense to me.

Thanks.

bug#30891: 26.0.91; indent of malformed lisp

[Noam Postavsky]

Eli Zaretskii <eliz@gnu.org> writes:

> I'd rather go back to the old code, and let the new Lisp indent
> functions mature on master.  Alternatively, we could leave this
> problem alone and fix it later.  But making all these changes at the
> 95th minute before Emacs 26 is released makes very little sense to me.

Okay.  As a paredit use who never has malformed lisp in my buffers, I'm
inclined towards the 2nd option.

bug#30891: 26.0.91; indent of malformed lisp

[Noam Postavsky]

[-- Attachment #1: Type: text/plain, Size: 700 bytes --]

Eli Zaretskii <eliz@gnu.org> writes:

>> For example, indent-region in a buffer containing:
>> 
>> )
>> (+ 2
>>    3)
>> 
>> gives
>> 
>> Debugger entered--Lisp error: (wrong-type-argument consp nil)

> I'd rather go back to the old code, and let the new Lisp indent
> functions mature on master.  Alternatively, we could leave this
> problem alone and fix it later.  But making all these changes at the
> 95th minute before Emacs 26 is released makes very little sense to me.

Okay to push just the error avoidance part to emacs-26 now?  (I'm not
really sure how useful the
lisp-ppss-open-paren-in-column-0-is-defun-start thing would be anyway,
and we haven't seen any other complaints about this)


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: patch --]
[-- Type: text/x-diff, Size: 1099 bytes --]

From 823fadf0a29a5cb01559f40faac2a88cd0defeb8 Mon Sep 17 00:00:00 2001
From: Noam Postavsky <npostavs@gmail.com>
Date: Sat, 17 Mar 2018 21:14:11 -0400
Subject: [PATCH] Don't error when indenting malformed Lisp (Bug#30891)

* lisp/emacs-lisp/lisp-mode.el (lisp-indent-calc-next): If we run out
of indent stack, reset the parse state.
---
 lisp/emacs-lisp/lisp-mode.el | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/lisp/emacs-lisp/lisp-mode.el b/lisp/emacs-lisp/lisp-mode.el
index 205c810b97..13ad06e4ae 100644
--- a/lisp/emacs-lisp/lisp-mode.el
+++ b/lisp/emacs-lisp/lisp-mode.el
@@ -827,6 +827,10 @@ lisp-indent-calc-next
     (prog1
         (let (indent)
           (cond ((= (forward-line 1) 1) nil)
+                ;; Negative depth, probably some kind of syntax error.
+                ((null indent-stack)
+                 ;; Reset state.
+                 (setq ppss (parse-partial-sexp (point) (point))))
                 ((car indent-stack))
                 ((integerp (setq indent (calculate-lisp-indent ppss)))
                  (setf (car indent-stack) indent))
-- 
2.11.0

bug#30891: 26.0.91; indent of malformed lisp

[Eli Zaretskii]

> From: Noam Postavsky <npostavs@gmail.com>
> Cc: 30891@debbugs.gnu.org
> Date: Wed, 24 Oct 2018 19:39:44 -0400
> 
> > I'd rather go back to the old code, and let the new Lisp indent
> > functions mature on master.  Alternatively, we could leave this
> > problem alone and fix it later.  But making all these changes at the
> > 95th minute before Emacs 26 is released makes very little sense to me.
> 
> Okay to push just the error avoidance part to emacs-26 now?

Yes, please.

bug#30891: 26.0.91; indent of malformed lisp

[Noam Postavsky]

close 30891 26.2
quit

Eli Zaretskii <eliz@gnu.org> writes:

>> Okay to push just the error avoidance part to emacs-26 now?
>
> Yes, please.

Done.

[1: 92de44fa1f]: 2018-10-25 17:55:49 -0400
  Don't error when indenting malformed Lisp (Bug#30891)
  https://git.savannah.gnu.org/cgit/emacs.git/commit/?id=92de44fa1fdeda74a9b8254f968829df4c957da0