Proposal: make up-list escape strings

Proposal: make up-list escape strings

[Daniel Colascione]

[-- Attachment #1: Type: text/plain, Size: 3476 bytes --]

Here's a small patch that makes backward-up-list and up-list escape
strings as well as explicit lists. This behavior is active only when
these functions are called interactively or when lisp explicitly asks
for this behavior, so the change shouldn't pose a compatibility risk.
Backing out of strings is generally a useful thing to do when you want
to operate on the string as a whole.

=== modified file 'lisp/emacs-lisp/lisp.el'
--- lisp/emacs-lisp/lisp.el	2014-02-26 02:31:27 +0000
+++ lisp/emacs-lisp/lisp.el	2014-04-08 23:12:16 +0000
@@ -140,38 +140,58 @@
       (goto-char (or (scan-lists (point) inc -1) (buffer-end arg)))
       (setq arg (- arg inc)))))

-(defun backward-up-list (&optional arg)
+(defun backward-up-list (&optional arg escape-strings)
   "Move backward out of one level of parentheses.
 This command will also work on other parentheses-like expressions
 defined by the current language mode.
 With ARG, do this that many times.
 A negative argument means move forward but still to a less deep spot.
-This command assumes point is not in a string or comment."
-  (interactive "^p")
-  (up-list (- (or arg 1))))
+This command assumes point is not in a string or comment.
+If ESCAPE-STRINGS is non-nil (as it is interactively), treat
+encoding strings as sexps."
+  (interactive "^p\nd")
+  (up-list (- (or arg 1)) escape-strings))

-(defun up-list (&optional arg)
+(defun up-list (&optional arg escape-strings)
   "Move forward out of one level of parentheses.
 This command will also work on other parentheses-like expressions
 defined by the current language mode.
 With ARG, do this that many times.
 A negative argument means move backward but still to a less deep spot.
-This command assumes point is not in a string or comment."
-  (interactive "^p")
+If ESCAPE-STRINGS is non-nil (as it is interactively), treat
+encoding strings as sexps."
+  (interactive "^p\nd")
   (or arg (setq arg 1))
   (let ((inc (if (> arg 0) 1 -1))
         pos)
     (while (/= arg 0)
       (if (null forward-sexp-function)
-          (goto-char (or (scan-lists (point) inc 1) (buffer-end arg)))
+          (condition-case err
+              (goto-char (or (scan-lists (point) inc 1) (buffer-end arg)))
+            (scan-error
+             (or (and escape-strings
+                      (let ((syntax (syntax-ppss)))
+                        (and (nth 3 syntax)
+                             (nth 8 syntax))
+                        (goto-char (nth 8 syntax))
+                        (when (> arg 0) (forward-sexp))
+                        t))
+                 (signal (car err) (cdr err)))))
 	(condition-case err
 	    (while (progn (setq pos (point))
 			  (forward-sexp inc)
 			  (/= (point) pos)))
 	  (scan-error (goto-char (nth (if (> arg 0) 3 2) err))))
 	(if (= (point) pos)
-            (signal 'scan-error
-                    (list "Unbalanced parentheses" (point) (point)))))
+            (or (and escape-strings
+                     (let ((syntax (syntax-ppss)))
+                       (and (nth 3 syntax)
+                            (nth 8 syntax))
+                       (goto-char (nth 8 syntax))
+                       (when (> arg 0) (forward-sexp))
+                       t))
+             (signal 'scan-error
+                     (list "Unbalanced parentheses" (point) (point))))))
       (setq arg (- arg inc)))))

 (defun kill-sexp (&optional arg)



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 901 bytes --]

Re: Proposal: make up-list escape strings

[Stefan Monnier]

> Here's a small patch that makes backward-up-list and up-list escape
> strings as well as explicit lists. This behavior is active only when
> these functions are called interactively or when lisp explicitly asks
> for this behavior, so the change shouldn't pose a compatibility risk.
> Backing out of strings is generally a useful thing to do when you want
> to operate on the string as a whole.

Thanks, this is long overdue.  See comments below:

> +If ESCAPE-STRINGS is non-nil (as it is interactively), treat
> +encoding strings as sexps."
   ^^^^^^^^
   enclosing, right?  ;-)

Why limit this to strings?  It makes just as much sense to do it for
comments, doesn't it?

>      (while (/= arg 0)
>        (if (null forward-sexp-function)
> -          (goto-char (or (scan-lists (point) inc 1) (buffer-end arg)))
> +          (condition-case err
> +              (goto-char (or (scan-lists (point) inc 1) (buffer-end arg)))
> +            (scan-error
> +             (or (and escape-strings
> +                      (let ((syntax (syntax-ppss)))
> +                        (and (nth 3 syntax)
> +                             (nth 8 syntax))
> +                        (goto-char (nth 8 syntax))
> +                        (when (> arg 0) (forward-sexp))
> +                        t))
> +                 (signal (car err) (cdr err)))))

If @ is point, I think that from

      (a "b (c (d @ e) " "f) g" h)

up-list should move to

      (a "b (c (d e)@ " "f) g" h)

and then to

      (a "b (c (d e) "@ "f) g" h)

whereas I think your code would move to

      (a "b (c (d e) " "f)@ g" h)

> +            (or (and escape-strings
> +                     (let ((syntax (syntax-ppss)))
> +                       (and (nth 3 syntax)
> +                            (nth 8 syntax))
> +                       (goto-char (nth 8 syntax))
> +                       (when (> arg 0) (forward-sexp))
> +                       t))

This should be factored out into a separate function to avoid the
code duplication.


        Stefan

Re: Proposal: make up-list escape strings

[Daniel Colascione]

[-- Attachment #1: Type: text/plain, Size: 1715 bytes --]

On 04/08/2014 06:11 PM, Stefan Monnier wrote:
>> +If ESCAPE-STRINGS is non-nil (as it is interactively), treat
>> +encoding strings as sexps."
>    ^^^^^^^^
>    enclosing, right?  ;-)

That'll learn me some.

> Why limit this to strings?  It makes just as much sense to do it for
> comments, doesn't it?

It's a bit harder to implement for comments (as in, rewrite up-list
using parse-partial-sexp), and I'm not sure it's as useful. Probably not
too bad though.

> 
>>      (while (/= arg 0)
>>        (if (null forward-sexp-function)
>> -          (goto-char (or (scan-lists (point) inc 1) (buffer-end arg)))
>> +          (condition-case err
>> +              (goto-char (or (scan-lists (point) inc 1) (buffer-end arg)))
>> +            (scan-error
>> +             (or (and escape-strings
>> +                      (let ((syntax (syntax-ppss)))
>> +                        (and (nth 3 syntax)
>> +                             (nth 8 syntax))
>> +                        (goto-char (nth 8 syntax))
>> +                        (when (> arg 0) (forward-sexp))
>> +                        t))
>> +                 (signal (car err) (cdr err)))))
> 
> If @ is point, I think that from
> 
>       (a "b (c (d @ e) " "f) g" h)
> 
> up-list should move to
> 
>       (a "b (c (d e)@ " "f) g" h)
> 
> and then to
> 
>       (a "b (c (d e) "@ "f) g" h)
> 
> whereas I think your code would move to
> 
>       (a "b (c (d e) " "f)@ g" h)

I'm not sure which behavior is more useful. Your proposed behavior is a
departure from what we have today, and one could make a case for either
approach. I haven't heard anyone complain about up-list crossing string
boundaries.



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 901 bytes --]

Re: Proposal: make up-list escape strings

[Stefan Monnier]

> I'm not sure which behavior is more useful. Your proposed behavior is a
> departure from what we have today, and one could make a case for either
> approach.  I haven't heard anyone complain about up-list crossing string
> boundaries.

Indeed, it's a problem, but crossing string boundaries makes the
behavior less deterministic.  I agree it can be useful, but I'm not
convinced it's worth the unpredictability: up-list can end up moving to
some very far away string, that's completely unrelated.  Currently noone
complains because when the cross-string jump is not useful, it's not
really worse than getting an error, but when the user intended to get
out of the current string, she'll be very disappointed.


        Stefan

Re: Proposal: make up-list escape strings

[Stefan Monnier]

>> Why limit this to strings?  It makes just as much sense to do it for
>> comments, doesn't it?
> It's a bit harder to implement for comments (as in, rewrite up-list
> using parse-partial-sexp), and I'm not sure it's as useful. Probably not
> too bad though.

I don't follow you.  It doesn't seem hard: just use forward-comment to
skip over the comment instead of forward-sexp to skip over the string.
Otherwise, the code should be pretty much identical for comments as
for strings.


        Stefan

Re: Proposal: make up-list escape strings

[Daniel Colascione]

[-- Attachment #1: Type: text/plain, Size: 1680 bytes --]

On 04/08/2014 08:56 PM, Stefan Monnier wrote:
>>> Why limit this to strings?  It makes just as much sense to do it for
>>> comments, doesn't it?
>> It's a bit harder to implement for comments (as in, rewrite up-list
>> using parse-partial-sexp), and I'm not sure it's as useful. Probably not
>> too bad though.
> 
> I don't follow you.  It doesn't seem hard: just use forward-comment to
> skip over the comment instead of forward-sexp to skip over the string.
> Otherwise, the code should be pretty much identical for comments as
> for strings.

Right now, the main case is based on scan-lists. Say we have something
like this JavaScript code:

  [ 1, 2, /* NO: [ 3.1, 3.3, 3.3 ] */ 3, 4 ]
  abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOP

If I understand what you mean by "do it for comments", if point begins
at `D' (well, between `C' and `D'), up-list should take us to `p', then
`i', then `a'. The problem is that `scan-lists' called with FROM `p',
COUNT -1, and DEPTH 1 will move directly from `p' to `a', bypassing
position `i'. We're going to need fancier logic to make that work correctly.

Anyway, even if you did stop at position `i', you couldn't do much: you
can't manipulate that comment as a sexp. The use case that motivated my
change is quickly replacing the string that happens to contain point,
like vim can with `di"' command. Since we don't have any commands that
manipulate comments as first-class lexical objects, making up-list stop
on comments seems less useful to me. I'd rather just treat comments as
pseudo-whitespace like the rest of Emacs does.

It'd be nice to treat comments as sexps, but that change is out of scope.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 901 bytes --]

Re: Proposal: make up-list escape strings

[Andreas Röhler]

Am 09.04.2014 01:14, schrieb Daniel Colascione:
> Here's a small patch that makes backward-up-list and up-list escape
> strings as well as explicit lists. This behavior is active only when
> these functions are called interactively or when lisp explicitly asks
> for this behavior, so the change shouldn't pose a compatibility risk.
> Backing out of strings is generally a useful thing to do when you want
> to operate on the string as a whole.
>
> === modified file 'lisp/emacs-lisp/lisp.el'
> --- lisp/emacs-lisp/lisp.el	2014-02-26 02:31:27 +0000
> +++ lisp/emacs-lisp/lisp.el	2014-04-08 23:12:16 +0000
> @@ -140,38 +140,58 @@
>         (goto-char (or (scan-lists (point) inc -1) (buffer-end arg)))
>         (setq arg (- arg inc)))))
>
> -(defun backward-up-list (&optional arg)
> +(defun backward-up-list (&optional arg escape-strings)
>     "Move backward out of one level of parentheses.
>   This command will also work on other parentheses-like expressions
>   defined by the current language mode.
>   With ARG, do this that many times.
>   A negative argument means move forward but still to a less deep spot.
> -This command assumes point is not in a string or comment."
> -  (interactive "^p")
> -  (up-list (- (or arg 1))))
> +This command assumes point is not in a string or comment.
> +If ESCAPE-STRINGS is non-nil (as it is interactively), treat
> +encoding strings as sexps."
> +  (interactive "^p\nd")
> +  (up-list (- (or arg 1)) escape-strings))
>
> -(defun up-list (&optional arg)
> +(defun up-list (&optional arg escape-strings)
>     "Move forward out of one level of parentheses.
>   This command will also work on other parentheses-like expressions
>   defined by the current language mode.
>   With ARG, do this that many times.
>   A negative argument means move backward but still to a less deep spot.
> -This command assumes point is not in a string or comment."
> -  (interactive "^p")
> +If ESCAPE-STRINGS is non-nil (as it is interactively), treat
> +encoding strings as sexps."
> +  (interactive "^p\nd")
>     (or arg (setq arg 1))
>     (let ((inc (if (> arg 0) 1 -1))
>           pos)
>       (while (/= arg 0)
>         (if (null forward-sexp-function)
> -          (goto-char (or (scan-lists (point) inc 1) (buffer-end arg)))
> +          (condition-case err
> +              (goto-char (or (scan-lists (point) inc 1) (buffer-end arg)))
> +            (scan-error
> +             (or (and escape-strings
> +                      (let ((syntax (syntax-ppss)))
> +                        (and (nth 3 syntax)
> +                             (nth 8 syntax))
> +                        (goto-char (nth 8 syntax))
> +                        (when (> arg 0) (forward-sexp))
> +                        t))
> +                 (signal (car err) (cdr err)))))
>   	(condition-case err
>   	    (while (progn (setq pos (point))
>   			  (forward-sexp inc)
>   			  (/= (point) pos)))
>   	  (scan-error (goto-char (nth (if (> arg 0) 3 2) err))))
>   	(if (= (point) pos)
> -            (signal 'scan-error
> -                    (list "Unbalanced parentheses" (point) (point)))))
> +            (or (and escape-strings
> +                     (let ((syntax (syntax-ppss)))
> +                       (and (nth 3 syntax)
> +                            (nth 8 syntax))
> +                       (goto-char (nth 8 syntax))
> +                       (when (> arg 0) (forward-sexp))
> +                       t))
> +             (signal 'scan-error
> +                     (list "Unbalanced parentheses" (point) (point))))))
>         (setq arg (- arg inc)))))
>
>   (defun kill-sexp (&optional arg)
>
>

Probably the most ancient bug :-)

IMO you don't need "and" going out of comments/strings

just (goto-char (nth 8 syntax))


Here is the form I'm using for ages:

(defun ar-up-list (arg)
   "Move forward out of one level of parentheses.
With ARG, do this that many times.
A negative argument means move backward but still to a less deep spot."
   (interactive "p")
   (let ((orig (point))
         (pps (syntax-ppss))
         erg)
     (and (nth 8 pps) (goto-char (nth 8 pps)))
     (ignore-errors (up-list arg))
     (and (< orig (point)) (setq erg (point)))
     (when (interactive-p) (message "%s" erg))
     erg))

Cheers,

Andreas

Re: Proposal: make up-list escape strings

[Daniel Colascione]

[-- Attachment #1: Type: text/plain, Size: 11969 bytes --]

On 04/08/2014 09:12 PM, I blathered:
> On 04/08/2014 08:56 PM, Stefan Monnier wrote:
>>>> Why limit this to strings?  It makes just as much sense to do it for
>>>> comments, doesn't it?
>>> It's a bit harder to implement for comments (as in, rewrite up-list
>>> using parse-partial-sexp), and I'm not sure it's as useful. Probably not
>>> too bad though.
>>
>> I don't follow you.  It doesn't seem hard: just use forward-comment to
>> skip over the comment instead of forward-sexp to skip over the string.
>> Otherwise, the code should be pretty much identical for comments as
>> for strings.
> 
> Right now, the main case is based on scan-lists. Say we have something
> like this JavaScript code:
> 
>   [ 1, 2, /* NO: [ 3.1, 3.3, 3.3 ] */ 3, 4 ]
>   abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOP
> 
> If I understand what you mean by "do it for comments", if point begins
> at `D' (well, between `C' and `D'), up-list should take us to `p', then
> `i', then `a'. The problem is that `scan-lists' called with FROM `p',
> COUNT -1, and DEPTH 1 will move directly from `p' to `a', bypassing
> position `i'. We're going to need fancier logic to make that work correctly.
> 
> Anyway, even if you did stop at position `i', you couldn't do much: you
> can't manipulate that comment as a sexp. The use case that motivated my
> change is quickly replacing the string that happens to contain point,
> like vim can with `di"' command. Since we don't have any commands that
> manipulate comments as first-class lexical objects, making up-list stop
> on comments seems less useful to me. I'd rather just treat comments as
> pseudo-whitespace like the rest of Emacs does.
> 
> It'd be nice to treat comments as sexps, but that change is out of scope.

Here's an updated patch that addresses most of the feedback so far. It's
still oblivious to comments, as IMHO, it should be.

I've added some tests; I've also tweaked the documentation for
scan-error: we depend on its error data, but we don't seem to document
the required contents anywhere.

=== modified file 'doc/lispref/errors.texi'
--- doc/lispref/errors.texi	2014-01-01 07:43:34 +0000
+++ doc/lispref/errors.texi	2014-04-09 04:41:38 +0000
@@ -157,7 +157,10 @@
 @item scan-error
 The message is @samp{Scan error}.  This happens when certain
 syntax-parsing functions find invalid syntax or mismatched
-parentheses.  @xref{List Motion}, and @xref{Parsing Expressions}.
+parentheses.  Conventionally raised with three argument: a
+human-readable error message, the start of the obstacle that cannot be
+moved over, and the end of the obstacle.  @xref{List Motion}, and
+@xref{Parsing Expressions}.

 @item search-failed
 The message is @samp{Search failed}.  @xref{Searching and Matching}.

=== modified file 'lisp/emacs-lisp/lisp.el'
--- lisp/emacs-lisp/lisp.el	2014-02-26 02:31:27 +0000
+++ lisp/emacs-lisp/lisp.el	2014-04-09 06:28:57 +0000
@@ -57,10 +57,14 @@

 (defun forward-sexp (&optional arg)
   "Move forward across one balanced expression (sexp).
-With ARG, do it that many times.  Negative arg -N means
-move backward across N balanced expressions.
-This command assumes point is not in a string or comment.
-Calls `forward-sexp-function' to do the work, if that is non-nil."
+With ARG, do it that many times.  Negative arg -N means move
+backward across N balanced expressions.  This command assumes
+point is not in a string or comment.  Calls
+`forward-sexp-function' to do the work, if that is non-nil.  If
+unable to move over a sexp, signal `scan-error' with three
+arguments: a message, the start of the obstacle (usually a
+parenthesis or list marker of some kind), and end of the
+obstacle."
   (interactive "^p")
   (or arg (setq arg 1))
   (if forward-sexp-function
@@ -140,38 +144,74 @@
       (goto-char (or (scan-lists (point) inc -1) (buffer-end arg)))
       (setq arg (- arg inc)))))

-(defun backward-up-list (&optional arg)
+(defun backward-up-list (&optional arg escape-strings no-string-crossing)
   "Move backward out of one level of parentheses.
 This command will also work on other parentheses-like expressions
-defined by the current language mode.
-With ARG, do this that many times.
-A negative argument means move forward but still to a less deep spot.
-This command assumes point is not in a string or comment."
-  (interactive "^p")
-  (up-list (- (or arg 1))))
+defined by the current language mode.  With ARG, do this that
+many times.  A negative argument means move forward but still to
+a less deep spot.  This command assumes point is not in a string
+or comment.  If ESCAPE-STRINGS is non-nil (as it is
+interactively), move out of enclosing strings as well.  If
+NO-STRING-CROSSING is non-nil (as it is interactively), prefer to
+break out of any enclosing string instead of moving to the start
+of a list broken across multiple strings."
+  (interactive "^p\nd\nd")
+  (up-list (- (or arg 1)) escape-strings no-string-crossing))

-(defun up-list (&optional arg)
+(defun up-list (&optional arg escape-strings no-string-crossing)
   "Move forward out of one level of parentheses.
 This command will also work on other parentheses-like expressions
-defined by the current language mode.
-With ARG, do this that many times.
-A negative argument means move backward but still to a less deep spot.
-This command assumes point is not in a string or comment."
-  (interactive "^p")
+defined by the current language mode.  With ARG, do this that
+many times.  A negative argument means move backward but still to
+a less deep spot.  If ESCAPE-STRINGS is non-nil (as it is
+interactively), move out of enclosing strings as well. If
+NO-STRING-CROSSING is non-nil (as it is interactively), prefer to
+break out of any enclosing string instead of moving to the start
+of a list broken across multiple strings."
+  (interactive "^p\nd\nd")
   (or arg (setq arg 1))
   (let ((inc (if (> arg 0) 1 -1))
-        pos)
+        (pos nil))
     (while (/= arg 0)
-      (if (null forward-sexp-function)
-          (goto-char (or (scan-lists (point) inc 1) (buffer-end arg)))
-	(condition-case err
-	    (while (progn (setq pos (point))
-			  (forward-sexp inc)
-			  (/= (point) pos)))
-	  (scan-error (goto-char (nth (if (> arg 0) 3 2) err))))
-	(if (= (point) pos)
-            (signal 'scan-error
-                    (list "Unbalanced parentheses" (point) (point)))))
+      (condition-case err
+          (save-restriction
+            ;; If we've been asked not to cross string boundaries
+            ;; and we're inside a string, narrow to that string so
+            ;; that scan-lists doesn't find a match in a different
+            ;; string.
+            (when no-string-crossing
+              (let ((syntax (syntax-ppss)))
+                (when (nth 3 syntax)    ; Inside string
+                  (save-excursion
+                    (goto-char (nth 8 syntax)) ; String start
+                    (narrow-to-region
+                     (point)
+                     (condition-case nil
+                         (progn (forward-sexp) (point))
+                       (scan-error (point-max))))))))
+            (if (null forward-sexp-function)
+                (goto-char (or (scan-lists (point) inc 1)
+                               (buffer-end arg)))
+              (condition-case err
+                  (while (progn (setq pos (point))
+                                (forward-sexp inc)
+                                (/= (point) pos)))
+                (scan-error (goto-char (nth (if (> arg 0) 3 2) err))))
+              (if (= (point) pos)
+                  (signal 'scan-error
+                          (list "Unbalanced parentheses" (point)
(point))))))
+        (scan-error
+         ;; If we bumped up against the end of a list, see whether
+         ;; we're inside a string: if so, just go to the beginning or
+         ;; end of that string.
+         (or (and escape-strings
+                  (let ((syntax (syntax-ppss)))
+                    (and (nth 3 syntax)
+                         (goto-char (nth 8 syntax))
+                         (progn (when (> inc 0)
+                                  (forward-sexp))
+                                t))))
+             (signal (car err) (cdr err)))))
       (setq arg (- arg inc)))))

 (defun kill-sexp (&optional arg)

=== added file 'test/automated/syntax-tests.el'
--- test/automated/syntax-tests.el	1970-01-01 00:00:00 +0000
+++ test/automated/syntax-tests.el	2014-04-09 06:22:03 +0000
@@ -0,0 +1,97 @@
+;;; syntax-tests.el --- Testing syntax rules and basic movement -*-
lexical-binding: t -*-
+
+;; Copyright (C) 2014 Free Software Foundation, Inc.
+
+;; Author: Daniel Colascione <dancol@dancol.org>
+;; Keywords:
+
+;; This program is free software; you can redistribute it and/or modify
+;; it under the terms of the GNU General Public License as published by
+;; the Free Software Foundation, either version 3 of the License, or
+;; (at your option) any later version.
+
+;; This program is distributed in the hope that it will be useful,
+;; but WITHOUT ANY WARRANTY; without even the implied warranty of
+;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;; GNU General Public License for more details.
+
+;; You should have received a copy of the GNU General Public License
+;; along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+;;; Commentary:
+
+;;
+
+;;; Code:
+(require 'ert)
+(require 'cl-lib)
+
+(defun run-up-list-test (fn data start instructions)
+  (cl-labels ((posof (thing)
+                (and (symbolp thing)
+                     (= (length (symbol-name thing)) 1)
+                     (- (aref (symbol-name thing) 0) ?a -1))))
+    (with-temp-buffer
+      (set-syntax-table (make-syntax-table))
+      ;; Use a syntax table in which single quote is a string
+      ;; character so that we can embed the test data in a lisp string
+      ;; literal.
+      (modify-syntax-entry ?\' "\"")
+      (insert data)
+      (goto-char (posof start))
+      (dolist (instruction instructions)
+        (cond ((posof instruction)
+               (funcall fn)
+               (should (eql (point) (posof instruction))))
+              ((symbolp instruction)
+               (should-error (funcall fn)
+                             :type instruction))
+              (t (cl-assert nil nil "unknown ins")))))))
+
+(defmacro define-up-list-test (name fn data start &rest expected)
+  `(ert-deftest ,name ()
+     (run-up-list-test ,fn ,data ',start ',expected)))
+
+(define-up-list-test up-list-basic
+  (lambda () (up-list))
+  (or "(1 1 (1 1) 1 (1 1) 1)")
+  ;;   abcdefghijklmnopqrstuv
+  i k v scan-error)
+
+(define-up-list-test up-list-with-forward-sexp-function
+  (lambda ()
+    (let ((forward-sexp-function
+           (lambda (&optional arg)
+             (let ((forward-sexp-function nil))
+               (forward-sexp arg)))))
+      (up-list)))
+  (or "(1 1 (1 1) 1 (1 1) 1)")
+  ;;   abcdefghijklmnopqrstuv
+  i k v scan-error)
+
+(define-up-list-test up-list-out-of-string
+  (lambda () (up-list 1 t))
+  (or "1 (1 '2 2 (2 2 2' 1) 1")
+  ;;   abcdefghijklmnopqrstuvwxy
+  o r u scan-error)
+
+(define-up-list-test up-list-cross-string
+  (lambda () (up-list 1 t))
+  (or "(1 '2 ( 2' 1 '2 ) 2' 1)")
+  ;;   abcdefghijklmnopqrstuvwxy
+  i r u x scan-error)
+
+(define-up-list-test up-list-no-cross-string
+  (lambda () (up-list 1 t t))
+  (or "(1 '2 ( 2' 1 '2 ) 2' 1)")
+  ;;   abcdefghijklmnopqrstuvwxy
+  i k x scan-error)
+
+(define-up-list-test backward-up-list-basic
+  (lambda () (backward-up-list))
+  (or "(1 1 (1 1) 1 (1 1) 1)")
+  ;;   abcdefghijklmnopqrstuv
+  i f a scan-error)
+
+(provide 'syntax-tests)
+;;; syntax-tests.el ends here



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 901 bytes --]

Re: Proposal: make up-list escape strings

[Stefan Monnier]

>> If I understand what you mean by "do it for comments", if point begins
>> at `D' (well, between `C' and `D'), up-list should take us to `p', then
>> `i', then `a'.

Right.

>> The problem is that `scan-lists' called with FROM `p', COUNT -1, and
>> DEPTH 1 will move directly from `p' to `a', bypassing position `i'.

Not if you narrow to the comment before.

>> We're going to need fancier logic to make that work correctly.

No, you already have the logic for strings.

>> Anyway, even if you did stop at position `i', you couldn't do much: you
>> can't manipulate that comment as a sexp.

Good point.  It would call for things like mark-sexp to handle comments
not as white space.  But part of the reason is to avoid comment-crossing
and things like that, e.g. with

     x = '(';  // That's comment.

If you start from the end of line (and assume a no-string-crossing
behavior), you don't want C-M-u to jump just before the open paren.

> +            (when no-string-crossing
> +              (let ((syntax (syntax-ppss)))
> +                (when (nth 3 syntax)    ; Inside string
                              ^
                              8

> +                  (save-excursion
> +                    (goto-char (nth 8 syntax)) ; String start
> +                    (narrow-to-region
> +                     (point)
> +                     (condition-case nil
> +                         (progn (forward-sexp) (point))
                                   ^^^^^^^^^^^^^^
                           use (forward-comment) if (nth 4 syntax)

But then no-string-crossing needs to be renamed maybe "no lexical crossing"?

Other than that, looks good, feel free to install.


        Stefan

Re: Proposal: make up-list escape strings

[Daniel Colascione]

[-- Attachment #1: Type: text/plain, Size: 583 bytes --]

On 04/09/2014 05:41 AM, Stefan Monnier wrote:
> Good point.  It would call for things like mark-sexp to handle comments
> not as white space.  But part of the reason is to avoid comment-crossing
> and things like that, e.g. with
> 
>      x = '(';  // That's comment.
             a           b
> 
> If you start from the end of line (and assume a no-string-crossing
> behavior), you don't want C-M-u to jump just before the open paren.

Good point --- I didn't expect that to happen. Also surprising is that
backward-sexp at b jumps to a. I wish we wouldn't do that.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 901 bytes --]