From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Daniel Colascione Newsgroups: gmane.emacs.devel Subject: Re: Proposal: make up-list escape strings Date: Tue, 08 Apr 2014 23:30:35 -0700 Message-ID: <5344E90B.8080108@dancol.org> References: <534482F2.6010504@dancol.org> <5344A698.6080201@dancol.org> <5344C898.7010904@dancol.org> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="F3ImKwhnreUI02cUcwhqA9Eart5fWobGT" X-Trace: ger.gmane.org 1397025064 6951 80.91.229.3 (9 Apr 2014 06:31:04 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Wed, 9 Apr 2014 06:31:04 +0000 (UTC) Cc: Emacs developers To: Stefan Monnier Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Wed Apr 09 08:30:59 2014 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1WXm28-0000Ut-5v for ged-emacs-devel@m.gmane.org; Wed, 09 Apr 2014 08:30:52 +0200 Original-Received: from localhost ([::1]:44557 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WXm27-0003VX-O1 for ged-emacs-devel@m.gmane.org; Wed, 09 Apr 2014 02:30:51 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:58990) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WXm1z-0003IP-Q7 for emacs-devel@gnu.org; Wed, 09 Apr 2014 02:30:48 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1WXm1u-0002sz-5l for emacs-devel@gnu.org; Wed, 09 Apr 2014 02:30:43 -0400 Original-Received: from dancol.org ([2600:3c01::f03c:91ff:fedf:adf3]:38596) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WXm1t-0002qz-Oo for emacs-devel@gnu.org; Wed, 09 Apr 2014 02:30:38 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=dancol.org; s=x; h=Content-Type:In-Reply-To:References:Subject:CC:To:MIME-Version:From:Date:Message-ID; bh=nDKlzLEhAH/fRFMGopk075apyDp47tYgT7ZH9s4jwsw=; b=mdAKAS+zdYKVm52n4blyHz9H4iRHwmXsiZcUfd1+vOay0csyuN5xZs9yxdWUHnp21RibJMPYyYG46RA6O/wc/ipx10NqDJ48o/Hot3lmlB8P1uSZ4QHeuBxwAZbYvqK8NIjQUl4uBODr1qpq2c5J9b9/Ow60wcmgCMbpYxy3e4P2tSdS9IS9EQXnG0orxMQMYMYTb4WRcBZJPTZqNPMIY4Ud5JHO6fkFoiO8BXxnaLvoByVaNbRSjRxIh7CzKKU3axbc5Vu0nmrk70afVnsAltXlCWwJbhG1BC8VSzBWz1aMgedOaq35JPRF7VBNZXyB9nEdcb+x8ONj9SZ+t/6vpA==; Original-Received: from c-67-161-115-61.hsd1.wa.comcast.net ([67.161.115.61] helo=[192.168.1.50]) by dancol.org with esmtpsa (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:128) (Exim 4.82) (envelope-from ) id 1WXm1s-0006uB-GM; Tue, 08 Apr 2014 23:30:36 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 In-Reply-To: <5344C898.7010904@dancol.org> X-Enigmail-Version: 1.6 X-detected-operating-system: by eggs.gnu.org: Error: Malformed IPv6 address (bad octet value). X-Received-From: 2600:3c01::f03c:91ff:fedf:adf3 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.devel:171361 Archived-At: This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --F3ImKwhnreUI02cUcwhqA9Eart5fWobGT Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On 04/08/2014 09:12 PM, I blathered: > On 04/08/2014 08:56 PM, Stefan Monnier wrote: >>>> Why limit this to strings? It makes just as much sense to do it for= >>>> comments, doesn't it? >>> It's a bit harder to implement for comments (as in, rewrite up-list >>> using parse-partial-sexp), and I'm not sure it's as useful. Probably = not >>> too bad though. >> >> I don't follow you. It doesn't seem hard: just use forward-comment to= >> skip over the comment instead of forward-sexp to skip over the string.= >> Otherwise, the code should be pretty much identical for comments as >> for strings. >=20 > Right now, the main case is based on scan-lists. Say we have something > like this JavaScript code: >=20 > [ 1, 2, /* NO: [ 3.1, 3.3, 3.3 ] */ 3, 4 ] > abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOP >=20 > If I understand what you mean by "do it for comments", if point begins > at `D' (well, between `C' and `D'), up-list should take us to `p', then= > `i', then `a'. The problem is that `scan-lists' called with FROM `p', > COUNT -1, and DEPTH 1 will move directly from `p' to `a', bypassing > position `i'. We're going to need fancier logic to make that work corre= ctly. >=20 > Anyway, even if you did stop at position `i', you couldn't do much: you= > can't manipulate that comment as a sexp. The use case that motivated my= > change is quickly replacing the string that happens to contain point, > like vim can with `di"' command. Since we don't have any commands that > manipulate comments as first-class lexical objects, making up-list stop= > on comments seems less useful to me. I'd rather just treat comments as > pseudo-whitespace like the rest of Emacs does. >=20 > It'd be nice to treat comments as sexps, but that change is out of scop= e. Here's an updated patch that addresses most of the feedback so far. It's still oblivious to comments, as IMHO, it should be. I've added some tests; I've also tweaked the documentation for scan-error: we depend on its error data, but we don't seem to document the required contents anywhere. =3D=3D=3D modified file 'doc/lispref/errors.texi' --- doc/lispref/errors.texi 2014-01-01 07:43:34 +0000 +++ doc/lispref/errors.texi 2014-04-09 04:41:38 +0000 @@ -157,7 +157,10 @@ @item scan-error The message is @samp{Scan error}. This happens when certain syntax-parsing functions find invalid syntax or mismatched -parentheses. @xref{List Motion}, and @xref{Parsing Expressions}. +parentheses. Conventionally raised with three argument: a +human-readable error message, the start of the obstacle that cannot be +moved over, and the end of the obstacle. @xref{List Motion}, and +@xref{Parsing Expressions}. @item search-failed The message is @samp{Search failed}. @xref{Searching and Matching}. =3D=3D=3D modified file 'lisp/emacs-lisp/lisp.el' --- lisp/emacs-lisp/lisp.el 2014-02-26 02:31:27 +0000 +++ lisp/emacs-lisp/lisp.el 2014-04-09 06:28:57 +0000 @@ -57,10 +57,14 @@ (defun forward-sexp (&optional arg) "Move forward across one balanced expression (sexp). -With ARG, do it that many times. Negative arg -N means -move backward across N balanced expressions. -This command assumes point is not in a string or comment. -Calls `forward-sexp-function' to do the work, if that is non-nil." +With ARG, do it that many times. Negative arg -N means move +backward across N balanced expressions. This command assumes +point is not in a string or comment. Calls +`forward-sexp-function' to do the work, if that is non-nil. If +unable to move over a sexp, signal `scan-error' with three +arguments: a message, the start of the obstacle (usually a +parenthesis or list marker of some kind), and end of the +obstacle." (interactive "^p") (or arg (setq arg 1)) (if forward-sexp-function @@ -140,38 +144,74 @@ (goto-char (or (scan-lists (point) inc -1) (buffer-end arg))) (setq arg (- arg inc))))) -(defun backward-up-list (&optional arg) +(defun backward-up-list (&optional arg escape-strings no-string-crossing= ) "Move backward out of one level of parentheses. This command will also work on other parentheses-like expressions -defined by the current language mode. -With ARG, do this that many times. -A negative argument means move forward but still to a less deep spot. -This command assumes point is not in a string or comment." - (interactive "^p") - (up-list (- (or arg 1)))) +defined by the current language mode. With ARG, do this that +many times. A negative argument means move forward but still to +a less deep spot. This command assumes point is not in a string +or comment. If ESCAPE-STRINGS is non-nil (as it is +interactively), move out of enclosing strings as well. If +NO-STRING-CROSSING is non-nil (as it is interactively), prefer to +break out of any enclosing string instead of moving to the start +of a list broken across multiple strings." + (interactive "^p\nd\nd") + (up-list (- (or arg 1)) escape-strings no-string-crossing)) -(defun up-list (&optional arg) +(defun up-list (&optional arg escape-strings no-string-crossing) "Move forward out of one level of parentheses. This command will also work on other parentheses-like expressions -defined by the current language mode. -With ARG, do this that many times. -A negative argument means move backward but still to a less deep spot. -This command assumes point is not in a string or comment." - (interactive "^p") +defined by the current language mode. With ARG, do this that +many times. A negative argument means move backward but still to +a less deep spot. If ESCAPE-STRINGS is non-nil (as it is +interactively), move out of enclosing strings as well. If +NO-STRING-CROSSING is non-nil (as it is interactively), prefer to +break out of any enclosing string instead of moving to the start +of a list broken across multiple strings." + (interactive "^p\nd\nd") (or arg (setq arg 1)) (let ((inc (if (> arg 0) 1 -1)) - pos) + (pos nil)) (while (/=3D arg 0) - (if (null forward-sexp-function) - (goto-char (or (scan-lists (point) inc 1) (buffer-end arg))) - (condition-case err - (while (progn (setq pos (point)) - (forward-sexp inc) - (/=3D (point) pos))) - (scan-error (goto-char (nth (if (> arg 0) 3 2) err)))) - (if (=3D (point) pos) - (signal 'scan-error - (list "Unbalanced parentheses" (point) (point))))) + (condition-case err + (save-restriction + ;; If we've been asked not to cross string boundaries + ;; and we're inside a string, narrow to that string so + ;; that scan-lists doesn't find a match in a different + ;; string. + (when no-string-crossing + (let ((syntax (syntax-ppss))) + (when (nth 3 syntax) ; Inside string + (save-excursion + (goto-char (nth 8 syntax)) ; String start + (narrow-to-region + (point) + (condition-case nil + (progn (forward-sexp) (point)) + (scan-error (point-max)))))))) + (if (null forward-sexp-function) + (goto-char (or (scan-lists (point) inc 1) + (buffer-end arg))) + (condition-case err + (while (progn (setq pos (point)) + (forward-sexp inc) + (/=3D (point) pos))) + (scan-error (goto-char (nth (if (> arg 0) 3 2) err)))) + (if (=3D (point) pos) + (signal 'scan-error + (list "Unbalanced parentheses" (point) (point)))))) + (scan-error + ;; If we bumped up against the end of a list, see whether + ;; we're inside a string: if so, just go to the beginning or + ;; end of that string. + (or (and escape-strings + (let ((syntax (syntax-ppss))) + (and (nth 3 syntax) + (goto-char (nth 8 syntax)) + (progn (when (> inc 0) + (forward-sexp)) + t)))) + (signal (car err) (cdr err))))) (setq arg (- arg inc))))) (defun kill-sexp (&optional arg) =3D=3D=3D added file 'test/automated/syntax-tests.el' --- test/automated/syntax-tests.el 1970-01-01 00:00:00 +0000 +++ test/automated/syntax-tests.el 2014-04-09 06:22:03 +0000 @@ -0,0 +1,97 @@ +;;; syntax-tests.el --- Testing syntax rules and basic movement -*- lexical-binding: t -*- + +;; Copyright (C) 2014 Free Software Foundation, Inc. + +;; Author: Daniel Colascione +;; Keywords: + +;; This program is free software; you can redistribute it and/or modify +;; it under the terms of the GNU General Public License as published by +;; the Free Software Foundation, either version 3 of the License, or +;; (at your option) any later version. + +;; This program is distributed in the hope that it will be useful, +;; but WITHOUT ANY WARRANTY; without even the implied warranty of +;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;; GNU General Public License for more details. + +;; You should have received a copy of the GNU General Public License +;; along with this program. If not, see .= + +;;; Commentary: + +;; + +;;; Code: +(require 'ert) +(require 'cl-lib) + +(defun run-up-list-test (fn data start instructions) + (cl-labels ((posof (thing) + (and (symbolp thing) + (=3D (length (symbol-name thing)) 1) + (- (aref (symbol-name thing) 0) ?a -1)))) + (with-temp-buffer + (set-syntax-table (make-syntax-table)) + ;; Use a syntax table in which single quote is a string + ;; character so that we can embed the test data in a lisp string + ;; literal. + (modify-syntax-entry ?\' "\"") + (insert data) + (goto-char (posof start)) + (dolist (instruction instructions) + (cond ((posof instruction) + (funcall fn) + (should (eql (point) (posof instruction)))) + ((symbolp instruction) + (should-error (funcall fn) + :type instruction)) + (t (cl-assert nil nil "unknown ins"))))))) + +(defmacro define-up-list-test (name fn data start &rest expected) + `(ert-deftest ,name () + (run-up-list-test ,fn ,data ',start ',expected))) + +(define-up-list-test up-list-basic + (lambda () (up-list)) + (or "(1 1 (1 1) 1 (1 1) 1)") + ;; abcdefghijklmnopqrstuv + i k v scan-error) + +(define-up-list-test up-list-with-forward-sexp-function + (lambda () + (let ((forward-sexp-function + (lambda (&optional arg) + (let ((forward-sexp-function nil)) + (forward-sexp arg))))) + (up-list))) + (or "(1 1 (1 1) 1 (1 1) 1)") + ;; abcdefghijklmnopqrstuv + i k v scan-error) + +(define-up-list-test up-list-out-of-string + (lambda () (up-list 1 t)) + (or "1 (1 '2 2 (2 2 2' 1) 1") + ;; abcdefghijklmnopqrstuvwxy + o r u scan-error) + +(define-up-list-test up-list-cross-string + (lambda () (up-list 1 t)) + (or "(1 '2 ( 2' 1 '2 ) 2' 1)") + ;; abcdefghijklmnopqrstuvwxy + i r u x scan-error) + +(define-up-list-test up-list-no-cross-string + (lambda () (up-list 1 t t)) + (or "(1 '2 ( 2' 1 '2 ) 2' 1)") + ;; abcdefghijklmnopqrstuvwxy + i k x scan-error) + +(define-up-list-test backward-up-list-basic + (lambda () (backward-up-list)) + (or "(1 1 (1 1) 1 (1 1) 1)") + ;; abcdefghijklmnopqrstuv + i f a scan-error) + +(provide 'syntax-tests) +;;; syntax-tests.el ends here --F3ImKwhnreUI02cUcwhqA9Eart5fWobGT Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.14 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBAgAGBQJTROkLAAoJEMAaIROpHW7IZ+wP/R2bsDg7RFG7cLo75x80auCq 4MS9VuJGbr1BcfmEGqlTtBfyTLbFtn1Y/x2dCKudtChw/TFTOBsCupcJXf3n3gzt exF9NcsQg6YtnsqfKoEhIQwLdb6Xag4YFfJvqBFoMWUnzUzAbi/or+qq9VNXhDCf gYM6/O+ecu2qwK0Tombsd0QnrpalyFZbYOzG7RH8OiIfloGQqOoRp32BTQkxuRJT dZmSPhBuKTYAj5Gk44S8m5UTfdkF6adj6SMH8EInd3wZDQ3R7Oy9fLIKLMF9vyx+ UmI7NNdfXii2A5a1gWuGWCoyIWGrhgYxUisNX0/8TKChQAzoulBX7Qa8aiDbLIiJ adD7dLYYZ98PIO5utWxF2gmXMi8mb54XvsDz724FZ0a2jcm2I+MU2QyHVpcJwwjL YxtFVhNQz2RMrO3yWJQPjGryw1wFI7CIYpiqhsytBEFpd9T12NdFcF0sGgoFRM9y iRJNz5SeUZ0ljEHdo4j3mNCtbZgEkaV9BV8FTG0WJ0kmPmo7BTedlS/7SdPm15eo +7jomBDznIJfBE+gA3sYzLk4atO5KcIryiXYh6vFHLHVUpg3jtgZUFd5f5qkf3yw gyX5o3qgPGMpdTmBZ2SaitD8cMNR7JZJMdCuneErMeimqwL5DQM2BQJVKwF2kSbu Q3V1+Ewv/AbV8Z6EUkdt =FHKX -----END PGP SIGNATURE----- --F3ImKwhnreUI02cUcwhqA9Eart5fWobGT--