From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: npostavs@users.sourceforge.net Newsgroups: gmane.emacs.bugs Subject: bug#21343: 24.5; parse-partial-sexp mistakes string for a comment Date: Sun, 05 Mar 2017 01:12:45 -0500 Message-ID: <87d1dw9rqa.fsf@users.sourceforge.net> References: <87lhczjyx5.fsf@gmail.com> <55DCA9C7.7070609@easy-emacs.de> <87pp2awaqp.fsf@gmail.com> <87lh1jfnpj.fsf@users.sourceforge.net> NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Trace: blaine.gmane.org 1488694334 17631 195.159.176.226 (5 Mar 2017 06:12:14 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Sun, 5 Mar 2017 06:12:14 +0000 (UTC) User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.2 (gnu/linux) Cc: 21343@debbugs.gnu.org To: Oleh Krehel Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Sun Mar 05 07:12:09 2017 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ckPOq-0003jh-De for geb-bug-gnu-emacs@m.gmane.org; Sun, 05 Mar 2017 07:12:08 +0100 Original-Received: from localhost ([::1]:37693 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ckPOu-0003ee-Mz for geb-bug-gnu-emacs@m.gmane.org; Sun, 05 Mar 2017 01:12:12 -0500 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:60742) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ckPOn-0003dc-Eb for bug-gnu-emacs@gnu.org; Sun, 05 Mar 2017 01:12:07 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ckPOk-0004lJ-9K for bug-gnu-emacs@gnu.org; Sun, 05 Mar 2017 01:12:05 -0500 Original-Received: from debbugs.gnu.org ([208.118.235.43]:42286) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1ckPOk-0004l8-1Y for bug-gnu-emacs@gnu.org; Sun, 05 Mar 2017 01:12:02 -0500 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1ckPOj-0005Nl-Q3 for bug-gnu-emacs@gnu.org; Sun, 05 Mar 2017 01:12:01 -0500 X-Loop: help-debbugs@gnu.org Resent-From: npostavs@users.sourceforge.net Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sun, 05 Mar 2017 06:12:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 21343 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: confirmed Original-Received: via spool by 21343-submit@debbugs.gnu.org id=B21343.148869430120654 (code B ref 21343); Sun, 05 Mar 2017 06:12:01 +0000 Original-Received: (at 21343) by debbugs.gnu.org; 5 Mar 2017 06:11:41 +0000 Original-Received: from localhost ([127.0.0.1]:40484 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ckPOO-0005N3-Iv for submit@debbugs.gnu.org; Sun, 05 Mar 2017 01:11:41 -0500 Original-Received: from mail-io0-f172.google.com ([209.85.223.172]:35365) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ckPOM-0005Mh-6e; Sun, 05 Mar 2017 01:11:38 -0500 Original-Received: by mail-io0-f172.google.com with SMTP id z13so52424623iof.2; Sat, 04 Mar 2017 22:11:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version; bh=dWJHT+A495pU95Zt6t4mih16CWkdbdq0owEDEW4eG80=; b=RvIR2+L/hPvR6Zg7W7cxCHxud7WRR4RbmXxJ0Wq5nqqZce79mhc5xveuJzHpW/8E7Z 4D4IPedA+nOLmI6QowIK7KOY3+6QAnyTRJ6ZZdqd+jlgSAJbCo7yUa1/I7whoaDDd/Fa JUUGUoIKz+puZmmT6yEwB7JVP8YJDdaFKaN8OlNxtRBdSSjKR2k6Lkt8LA7OaZn+IyLY EimnGTjqOiKhzmluhzOqHFlIsji1we+lew2zY4jwi2R/d4zNoZtA+pG/wqHAawGNAok6 5JMADqPvceS2z2BPFGORwWQhJczkD6c4YAgEg3NoDATuP9pd7WxZYWw7cz7XdxR23FNH wpfA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:references:date :in-reply-to:message-id:user-agent:mime-version; bh=dWJHT+A495pU95Zt6t4mih16CWkdbdq0owEDEW4eG80=; b=jFpkvuV0fc1d3It6oykbSkQZw/CJ9DTF7eaMz6oVNeLg03u5iP6fS7+2DVa0aPKOvG bKpR20i19LNzApTplEwryG992ujGieQ1e2D0H16DCfgx3eeKTSM9BHRM/DCuvlJY5URz B/I2o3yD/ebpB2apzdp3jfhbXK/pq0qS+WYWVJW2nfz1+EFD1HXS5VPrWlgw8ZdZjgwj uIrxOKgBF5Pai29Hihv/RIszsfevFKasFCifgT3Z4RPE3iHe0kyx0lrHnKWthYq4eoPf lrdGUaZL5xflLxkLB1gPN8G8wRfCbRUvHNtFgP2o89uq9tLuV94GsSg/tKHjaV1XkOHh lRRA== X-Gm-Message-State: AMke39koyJ8IgsqypxI4jZNoxCe8lje64VfojMD43q830aKWyoAwsd7we8+QCvBY74Bb7Q== X-Received: by 10.107.5.137 with SMTP id 131mr10270585iof.87.1488694291615; Sat, 04 Mar 2017 22:11:31 -0800 (PST) Original-Received: from zony ([45.2.7.65]) by smtp.googlemail.com with ESMTPSA id 62sm4555653itl.1.2017.03.04.22.11.30 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Sat, 04 Mar 2017 22:11:30 -0800 (PST) In-Reply-To: <87lh1jfnpj.fsf@users.sourceforge.net> (npostavs@users.sourceforge.net's message of "Sat, 02 Jul 2016 22:40:56 -0400") X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 208.118.235.43 X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.org gmane.emacs.bugs:130203 Archived-At: --=-=-= Content-Type: text/plain tags 21343 patch quit npostavs@users.sourceforge.net writes: > A solution could be to use syntax-ppss to go to start of string before > parsing (though I wonder if syntax-ppss should be used more for the > parsing itself in that case, it seems like there's a lot of work going > that probably duplicates what's already being done): Actually, it turns out using syntax-ppss in the loop is slower, at least in the case of calling `pp' on a big list (which is a pessimal case for a cache based parser). Here is a patch (the first one just simplifies the code). --=-=-= Content-Type: text/plain Content-Disposition: attachment; filename=v1-0001-lisp-emacs-lisp-lisp-mode.el-indent-sexp-Simplify.patch Content-Description: patch >From bada5e84d79e38ff386685f9b6b80fdf4e3b0988 Mon Sep 17 00:00:00 2001 From: Noam Postavsky Date: Sun, 5 Mar 2017 00:16:13 -0500 Subject: [PATCH v1 1/2] * lisp/emacs-lisp/lisp-mode.el (indent-sexp): Simplify. * test/lisp/emacs-lisp/lisp-mode-tests.el (indent-sexp): (indent-subsexp, indent-sexp-in-string): New tests. --- lisp/emacs-lisp/lisp-mode.el | 169 ++++++++++++++------------------ test/lisp/emacs-lisp/lisp-mode-tests.el | 92 +++++++++++++++++ 2 files changed, 165 insertions(+), 96 deletions(-) create mode 100644 test/lisp/emacs-lisp/lisp-mode-tests.el diff --git a/lisp/emacs-lisp/lisp-mode.el b/lisp/emacs-lisp/lisp-mode.el index d720e0bc57..de0e66039a 100644 --- a/lisp/emacs-lisp/lisp-mode.el +++ b/lisp/emacs-lisp/lisp-mode.el @@ -1056,103 +1056,80 @@ indent-sexp If optional arg ENDPOS is given, indent each line, stopping when ENDPOS is encountered." (interactive) - (let ((indent-stack (list nil)) - (next-depth 0) - ;; If ENDPOS is non-nil, use nil as STARTING-POINT - ;; so that calculate-lisp-indent will find the beginning of - ;; the defun we are in. - ;; If ENDPOS is nil, it is safe not to scan before point - ;; since every line we indent is more deeply nested than point is. - (starting-point (if endpos nil (point))) - (last-point (point)) - last-depth bol outer-loop-done inner-loop-done state this-indent) - (or endpos - ;; Get error now if we don't have a complete sexp after point. - (save-excursion (forward-sexp 1))) + (let* ((indent-stack (list nil)) + ;; If ENDPOS is non-nil, use beginning of defun as STARTING-POINT. + ;; If ENDPOS is nil, it is safe not to scan before point + ;; since every line we indent is more deeply nested than point is. + (starting-point (save-excursion (if endpos (beginning-of-defun)) + (point))) + (state nil) + (init-depth 0) + (next-depth 0) + (last-depth 0) + (last-syntax-point (point))) + (unless endpos + ;; Get error now if we don't have a complete sexp after point. + (save-excursion (forward-sexp 1) + ;; We need a marker because we modify the buffer + ;; text preceding endpos. + (setq endpos (point-marker)))) (save-excursion - (setq outer-loop-done nil) - (while (if endpos (< (point) endpos) - (not outer-loop-done)) - (setq last-depth next-depth - inner-loop-done nil) - ;; Parse this line so we can learn the state - ;; to indent the next line. - ;; This inner loop goes through only once - ;; unless a line ends inside a string. - (while (and (not inner-loop-done) - (not (setq outer-loop-done (eobp)))) - (setq state (parse-partial-sexp (point) (progn (end-of-line) (point)) - nil nil state)) - (setq next-depth (car state)) - ;; If the line contains a comment other than the sort - ;; that is indented like code, - ;; indent it now with indent-for-comment. - ;; Comments indented like code are right already. - ;; In any case clear the in-comment flag in the state - ;; because parse-partial-sexp never sees the newlines. - (if (car (nthcdr 4 state)) - (progn (indent-for-comment) - (end-of-line) - (setcar (nthcdr 4 state) nil))) - ;; If this line ends inside a string, - ;; go straight to next line, remaining within the inner loop, - ;; and turn off the \-flag. - (if (car (nthcdr 3 state)) - (progn - (forward-line 1) - (setcar (nthcdr 5 state) nil)) - (setq inner-loop-done t))) - (and endpos - (<= next-depth 0) - (progn - (setq indent-stack (nconc indent-stack - (make-list (- next-depth) nil)) - last-depth (- last-depth next-depth) - next-depth 0))) - (forward-line 1) - ;; Decide whether to exit. - (if endpos - ;; If we have already reached the specified end, - ;; give up and do not reindent this line. - (if (<= endpos (point)) - (setq outer-loop-done t)) - ;; If no specified end, we are done if we have finished one sexp. - (if (<= next-depth 0) - (setq outer-loop-done t))) - (unless outer-loop-done - (while (> last-depth next-depth) - (setq indent-stack (cdr indent-stack) - last-depth (1- last-depth))) - (while (< last-depth next-depth) - (setq indent-stack (cons nil indent-stack) - last-depth (1+ last-depth))) - ;; Now indent the next line according - ;; to what we learned from parsing the previous one. - (setq bol (point)) - (skip-chars-forward " \t") - ;; But not if the line is blank, or just a comment - ;; (except for double-semi comments; indent them as usual). - (if (or (eobp) (looking-at "\\s<\\|\n")) - nil - (if (and (car indent-stack) - (>= (car indent-stack) 0)) - (setq this-indent (car indent-stack)) - (let ((val (calculate-lisp-indent - (if (car indent-stack) (- (car indent-stack)) - starting-point)))) - (if (null val) - (setq this-indent val) - (if (integerp val) - (setcar indent-stack - (setq this-indent val)) - (setcar indent-stack (- (car (cdr val)))) - (setq this-indent (car val)))))) - (if (and this-indent (/= (current-column) this-indent)) - (progn (delete-region bol (point)) - (indent-to this-indent))))) - (or outer-loop-done - (setq outer-loop-done (= (point) last-point)) - (setq last-point (point))))))) + (while (< (point) endpos) + ;; Parse this line so we can learn the state to indent the + ;; next line. + (setq state (parse-partial-sexp + last-syntax-point (progn (end-of-line) (point)) + nil nil state)) + ;; If a string continues past end of line, skip to its end. + (when (nth 3 state) + (setq state (parse-partial-sexp (point) (point-max) + nil nil state 'syntax-table))) + (setq next-depth (car state)) + ;; If the line contains a comment indent it now with + ;; `indent-for-comment'. + (when (nth 4 state) + (indent-for-comment) + (end-of-line)) + (setq last-syntax-point (point)) + (when (< next-depth init-depth) + (setq indent-stack (nconc indent-stack + (make-list (- init-depth next-depth) nil)) + last-depth (- last-depth next-depth) + next-depth init-depth)) + (forward-line 1) + (when (< (point) endpos) + (let ((depth-delta (- next-depth last-depth))) + (cond ((< depth-delta 0) + (setq indent-stack (nthcdr (- depth-delta) indent-stack))) + ((> depth-delta 0) + (setq indent-stack (nconc (make-list depth-delta nil) + indent-stack)))) + (setq last-depth next-depth)) + ;; Now indent the next line according + ;; to what we learned from parsing the previous one. + (let ((bol (point))) + (skip-chars-forward " \t") + ;; But not if the line is blank, or just a comment (we + ;; already called `indent-for-comment' above). + (unless (or (eolp) (eq (char-syntax (char-after)) ?<)) + (let ((this-indent (car indent-stack))) + (when (listp this-indent) + (let ((val (calculate-lisp-indent + (or (car this-indent) starting-point)))) + (setq + this-indent + (cond ((integerp val) + (setf (car indent-stack) val)) + ((consp val) ; (COLUMN CONTAINING-SEXP-START) + (setf (car indent-stack) (cdr val)) + (car val)) + ;; `calculate-lisp-indent' only returns nil + ;; when we're in a string, but this won't + ;; happen because we skip strings above. + (t (error "This shouldn't happen!")))))) + (when (/= (current-column) this-indent) + (delete-region bol (point)) + (indent-to this-indent)))))))))) (defun indent-pp-sexp (&optional arg) "Indent each line of the list starting just after point, or prettyprint it. diff --git a/test/lisp/emacs-lisp/lisp-mode-tests.el b/test/lisp/emacs-lisp/lisp-mode-tests.el new file mode 100644 index 0000000000..1972396112 --- /dev/null +++ b/test/lisp/emacs-lisp/lisp-mode-tests.el @@ -0,0 +1,92 @@ +;;; lisp-mode-tests.el --- Test Lisp editing commands -*- lexical-binding: t; -*- + +;; Copyright (C) 2017 Free Software Foundation, Inc. + +;; GNU Emacs is free software: you can redistribute it and/or modify +;; it under the terms of the GNU General Public License as published by +;; the Free Software Foundation, either version 3 of the License, or +;; (at your option) any later version. + +;; GNU Emacs is distributed in the hope that it will be useful, +;; but WITHOUT ANY WARRANTY; without even the implied warranty of +;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;; GNU General Public License for more details. + +;; You should have received a copy of the GNU General Public License +;; along with GNU Emacs. If not, see . + +;;; Code: + +(require 'ert) +(require 'cl-lib) +(require 'lisp-mode) + +(ert-deftest indent-sexp () + "Test basics of \\[indent-sexp]." + (with-temp-buffer + (insert "\ +\(a + (prog1 + (prog1 + 1 + 2) + 2) + (1 + \"string +ending\" + 2) ; comment + ;; comment + b)") + (goto-char (point-min)) + (let ((indent-tabs-mode nil) + (correct (buffer-string))) + (dolist (mode '(fundamental-mode emacs-lisp-mode)) + (funcall mode) + (indent-sexp) + ;; Don't mess up correctly indented code. + (should (string= (buffer-string) correct)) + ;; Correctly add indentation. + (save-excursion + (while (not (eobp)) + (delete-horizontal-space) + (forward-line))) + (indent-sexp) + (should (equal (buffer-string) correct)) + ;; Correctly remove indentation. + (save-excursion + (let ((n 0)) + (while (not (eobp)) + (insert (make-string n ?\s)) + (cl-incf n) + (back-to-indentation) + (forward-line (if (eq (char-after) ?\") 2 1))))) + (indent-sexp) + (should (equal (buffer-string) correct)))))) + +(ert-deftest indent-subsexp () + "Make sure calling `indent-sexp' inside a sexp works." + (with-temp-buffer + (insert "\ +\(d1 xx + (d2 yy + zz) + 11)") + (let ((correct (buffer-string))) + (search-backward "d2") + (up-list -1) + (indent-sexp) + (should (equal (buffer-string) correct))))) + +(ert-deftest indent-sexp-in-string () + "Make sure calling `indent-sexp' inside a string works." + ;; See https://debbugs.gnu.org/cgi/bugreport.cgi?bug=21343. + (with-temp-buffer + (emacs-lisp-mode) + (insert "\";\"") + (let ((correct (buffer-string))) + (search-backward ";") + (indent-sexp) + (should (equal (buffer-string) correct))))) + +(provide 'lisp-mode-tests) +;;; lisp-mode-tests.el ends here -- 2.11.1 --=-=-= Content-Type: text/x-diff Content-Disposition: inline; filename=v1-0002-Fix-indent-sexp-when-called-from-inside-a-string-.patch Content-Description: patch >From b318e8c0bdd0f08b49f7f8c635bf886d967bc742 Mon Sep 17 00:00:00 2001 From: Noam Postavsky Date: Sun, 5 Mar 2017 00:53:58 -0500 Subject: [PATCH v1 2/2] Fix indent-sexp when called from inside a string (Bug#21343) * lisp/emacs-lisp/lisp-mode.el (indent-sexp): Get initial syntax parse state from `syntax-ppss'. --- lisp/emacs-lisp/lisp-mode.el | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/lisp/emacs-lisp/lisp-mode.el b/lisp/emacs-lisp/lisp-mode.el index de0e66039a..ce61c69f3c 100644 --- a/lisp/emacs-lisp/lisp-mode.el +++ b/lisp/emacs-lisp/lisp-mode.el @@ -1062,10 +1062,14 @@ indent-sexp ;; since every line we indent is more deeply nested than point is. (starting-point (save-excursion (if endpos (beginning-of-defun)) (point))) - (state nil) - (init-depth 0) - (next-depth 0) - (last-depth 0) + ;; Use `syntax-ppss' to get initial state so we don't get + ;; confused by starting inside a string. We don't use + ;; `syntax-ppss' in the loop, because this is measurably + ;; slower when we're called on a long list. + (state (syntax-ppss)) + (init-depth (car state)) + (next-depth init-depth) + (last-depth init-depth) (last-syntax-point (point))) (unless endpos ;; Get error now if we don't have a complete sexp after point. -- 2.11.1 --=-=-=--