bug#1406: backward-up-list reports scan error incorrectly?

bug#1406: backward-up-list reports scan error incorrectly?

[xah lee]

This appears to be a bug of backward-up-list.

Summary: when i do backward-up-list in a particular lisp file, when  
the cursor is inside a double quote, it tell me Unbalanced  
parentheses, but there does not seems to be any unbalanced paren. The  
file byte-compiles fine.

steps to reproduce:

Save the following function to a file.

(defun replace-keybinding-notation (p1 p2)
   "Do a bunch of find/replace on current text selection."
   (interactive "r")

   (let (mystr (case-fold-search nil) (case-replace nil))
     (setq mystr (buffer-substring p1 p2))

     (setq mystr
           (with-temp-buffer
             (insert mystr)

             ;; C-
             (goto-char (point-min))
             (while
                 (search-forward-regexp "C-" nil t)
               (replace-match "Ctlr+" t) )

             ;; M-
             (goto-char (point-min))
             (while
                 (search-forward-regexp "M-" nil t)
               (replace-match "Alt+" t) )

             ;; Ctrl+Alt+t
             (goto-char (point-min))
             (while
                 (search-forward-regexp "Ctrl\\+Alt\\+\\([[:graph:]]\ 
\)" nil t)
               (replace-match "<span class=\"kbd\"><span class=\"key 
\">Ctrl</span>+<span class=\"key\">Alt</span>+<span class=\"key\">\ 
\1</span></span>" t) )

             ;; Cmd+Shift+t
             (goto-char (point-min))
             (while
                 (search-forward-regexp "Cmd\\+Shift\\+\\([[:graph:]]\ 
\)" nil t)
               (replace-match "<span class=\"kbd\"><span class=\"key 
\">Cmd</span>+<span class=\"key\">Shift</span>+<span class=\"key\">\ 
\1</span></span>" t) )

             ;; Alt+Shift+t
             (goto-char (point-min))
             (while
                 (search-forward-regexp "Alt\\+Shift\\+\\([[:graph:]]\ 
\)" nil t)
               (replace-match "<span class=\"kbd\"><span class=\"key 
\">Alt</span>+<span class=\"key\">Shift</span>+<span class=\"key\">\ 
\1</span></span>" t) )

             ;; Ctrl+t Ctrl+t
             (goto-char (point-min))
             (while
                 (search-forward-regexp "Ctrl\\+\\([[:graph:]]\\) Ctrl 
\\+\\([[:graph:]]\\)" nil t)
               (replace-match "<span class=\"kbd\"><span class=\"key 
\">Ctrl</span>+<span class=\"key\">\\1</span> <span class=\"key 
\">Ctrl</span>+<span class=\"key\">\\2</span></span>" t) )

             ;; Ctrl+t
             (goto-char (point-min))
             (while
                 (search-forward-regexp "Ctrl\\+\\([[:graph:]]\\)"  
nil t)
               (replace-match "<span class=\"kbd\"><span class=\"key 
\">Ctrl</span>+<span class=\"key\">\\1</span></span>" t) )

             ;; Alt+t
             (goto-char (point-min))
             (while
                 (search-forward-regexp "Alt\\+\\([[:graph:]]\\)" nil t)
               (replace-match "<span class=\"kbd\"><span class=\"key 
\">Alt</span>+<span class=\"key\">\\1</span></span>" t) )

             ;; Meta+t
             (goto-char (point-min))
             (while
                 (search-forward-regexp "Meta\\+\\([[:graph:]]\\)"  
nil t)
               (replace-match "<span class=\"kbd\"><span class=\"key 
\">Meta</span>+<span class=\"key\">\\1</span></span>" t) )

             ;; Cmd+t
             (goto-char (point-min))
             (while
                 (search-forward-regexp "Cmd\\+\\([[:graph:]]\\)" nil t)
               (replace-match "<span class=\"kbd\"><span class=\"key 
\">Cmd</span>+<span class=\"key\">\\1</span></span>" t) )

             (buffer-string)
             )
           )

     (delete-region p1 p2)
     (insert mystr)
     )
   )

• start emacs by /Applications/Emacs.app/Contents/MacOS/Emacs -q

• open the file.

• interactive search for “M-”, do it twice so that your cursor  
is in between the double quote on the line: (search-forward-regexp  
"M-" nil t). Press return so that your cursor is just before the  
closing quote.

• press Ctrl+Alt+↑ (invoke backward-up-list).

• emacs says: “up-list: Scan error: "Unbalanced parentheses", 582,  
1”

---------------------------------

In GNU Emacs 22.2.1 (powerpc-apple-darwin8.11.0, Carbon Version 1.6.0)
  of 2008-04-05 on g5.tokyo.stp.isas.jaxa.jp
Windowing system distributor `Apple Inc.', version 10.4.11
configured using `configure  '--prefix=/Applications/Emacs.app/ 
Contents/Resources' '--with-carbon' '--without-x' '--libexecdir=/ 
Volumes/Emacs/Emacs.app/Contents/MacOS/libexec' 'CFLAGS=-Os -arch  
i386 -arch ppc -isysroot /Developer/SDKs/MacOSX10.4u.sdk -DUSE_ATSUI - 
DUSE_MAC_TSM''

   Xah
∑ http://xahlee.org/

☄

bug#1406: Acknowledgement (backward-up-list reports scan error incorrectly?)

[xah lee]

addendum:

the problem does not occur if you switch to text-mode.

It occur if you switch to emacs-lisp-mode or fundamental-mode.

(so, the bug seems having to do with syntax table)

   Xah
∑ http://xahlee.org/

☄


On Nov 21, 2008, at 1:30 PM, Emacs bug Tracking System wrote:


Thank you for filing a new bug report with Emacs.

This is an automatically generated reply to let you know your message
has been received.

Your message is being forwarded to the package maintainers and other
interested parties for their attention; they will reply in due course.

Your message has been sent to the package maintainer(s):
  Emacs Bugs <bug-gnu-emacs@gnu.org>

If you wish to submit further information on this problem, please
send it to 1406@emacsbugs.donarmstrong.com, as before.

Please do not send mail to don@donarmstrong.com unless you wish
to report a problem with the Bug-tracking system.


-- 
1406: http://emacsbugs.donarmstrong.com/cgi-bin/bugreport.cgi?bug=1406
Emacs Bug Tracking System
Contact don@donarmstrong.com with problems

☄

bug#1406: backward-up-list reports scan error incorrectly?

[Alan Mackenzie]

'Evening, Xah!

On Fri, Nov 21, 2008 at 01:24:30PM -0800, xah lee wrote:
> This appears to be a bug of backward-up-list.

> Summary: when i do backward-up-list in a particular lisp file, when
> the cursor is inside a double quote, it tell me Unbalanced
> parentheses, but there does not seems to be any unbalanced paren. The
> file byte-compiles fine.

What do you mean, "the cursor is inside a double quote"?  Functions like
backward-up-list don't know whether or not they start inside a string or
comment; the only way they could find out would be by doing a
partial-parse-sexp from some "safe" position earlier on in the file.

So these functions assume they're NOT in a string or comment.
backward-up-list moves back to the quote mark, and tries to jump over the
"string it's just found", possibly taking it to the beginning of the
buffer.

The bug appears to be in the doc string of the function, which forgets to
mention this assumption.

If you really, really, need a function which works the way you've
pictured, it's not too difficult to write.  But it will be SLOW, SLOW,
SLOW.

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#1406: backward-up-list reports scan error incorrectly?

[xah lee]

Hi Alan,

it works if you switch to text mode though.

It doesn't work while in emacs-lisp-mode and fundamental mode.

In any case, normally it works when inside string too. Just not in  
this particular case.

   Xah
∑ http://xahlee.org/

☄

On Nov 21, 2008, at 3:19 PM, Alan Mackenzie wrote:

'Evening, Xah!

On Fri, Nov 21, 2008 at 01:24:30PM -0800, xah lee wrote:
> This appears to be a bug of backward-up-list.

> Summary: when i do backward-up-list in a particular lisp file, when
> the cursor is inside a double quote, it tell me Unbalanced
> parentheses, but there does not seems to be any unbalanced paren. The
> file byte-compiles fine.

What do you mean, "the cursor is inside a double quote"?  Functions like
backward-up-list don't know whether or not they start inside a string or
comment; the only way they could find out would be by doing a
partial-parse-sexp from some "safe" position earlier on in the file.

So these functions assume they're NOT in a string or comment.
backward-up-list moves back to the quote mark, and tries to jump over  
the
"string it's just found", possibly taking it to the beginning of the
buffer.

The bug appears to be in the doc string of the function, which  
forgets to
mention this assumption.

If you really, really, need a function which works the way you've
pictured, it's not too difficult to write.  But it will be SLOW, SLOW,
SLOW.

-- 
Alan Mackenzie (Nuremberg, Germany).

☄

bug#1406: backward-up-list reports scan error incorrectly?

[Alan Mackenzie]

Hi, Xah!

On Fri, Nov 21, 2008 at 03:26:03PM -0800, xah lee wrote:
> Hi Alan,

> it works if you switch to text mode though.

> It doesn't work while in emacs-lisp-mode and fundamental mode.

> In any case, normally it works when inside string too. Just not in  
> this particular case.

OK, I think half of the problem here is your prolixity, the other half
is my prolixity.  :-)

Try instead the following example:

(defun foo (arg)
  "foo does nothing."
  (interactive "P"))

Put point inside "P", and do C-M-u `backward-up-list'.  You get the
error.  Now modify the above function by inserting an open paren into
the doc string,

(defun foo (arg)
  "foo does nothing. ("
    (interactive "P"))

, and do the same again.  It finds that paren.  Why?  Because
backward-up-list assumes its starting point is NOT in a string.  It
jumps backwards over (what it thinks is) the string

    "\n(interactive "

, and then finds the ?\(. 

This is exactly what is happening in your somewhat larger example.

In text mode, presumably the syntax table doesn't define ?\" as a string
quote.  In fundamental mode, presumably it does.  "(syntax-after
(point))" is your friend here.

>   Xah

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#1406: backward-up-list reports scan error incorrectly?

[xah lee]

hi Alan,

> (defun foo (arg)
>   "foo does nothing."
>   (interactive "P"))
>
> Put point inside "P", and do C-M-u `backward-up-list'.  You get the
> error.


i couldn't duplicate this. It seems to work for me.

am using
GNU Emacs 22.2.1 (powerpc-apple-darwin8.11.0, Carbon Version 1.6.0)  
of 2008-04-05 on g5.tokyo.stp.isas.jaxa.jp

i tried aquamacs. I can duplicate it with -Q, but not with -q.

Now i try carbon emacs again with -q, and i _can_ duplicate it.

... umm... not sure what is the problem now... since i'm pretty sure  
i don't have any customization for emacs-lisp-mode ...

am kinda tired now. Maybe i'll do some more testing to see what i  
find out.

   Xah
∑ http://xahlee.org/

☄

On Nov 22, 2008, at 4:36 AM, Alan Mackenzie wrote:

Hi, Xah!

On Fri, Nov 21, 2008 at 03:26:03PM -0800, xah lee wrote:
> Hi Alan,

> it works if you switch to text mode though.

> It doesn't work while in emacs-lisp-mode and fundamental mode.

> In any case, normally it works when inside string too. Just not in
> this particular case.

OK, I think half of the problem here is your prolixity, the other half
is my prolixity.  :-)

Try instead the following example:

(defun foo (arg)
   "foo does nothing."
   (interactive "P"))

Put point inside "P", and do C-M-u `backward-up-list'.  You get the
error.  Now modify the above function by inserting an open paren into
the doc string,

(defun foo (arg)
   "foo does nothing. ("
     (interactive "P"))

, and do the same again.  It finds that paren.  Why?  Because
backward-up-list assumes its starting point is NOT in a string.  It
jumps backwards over (what it thinks is) the string

     "\n(interactive "

, and then finds the ?\(.

This is exactly what is happening in your somewhat larger example.

In text mode, presumably the syntax table doesn't define ?\" as a string
quote.  In fundamental mode, presumably it does.  "(syntax-after
(point))" is your friend here.

>   Xah

-- 
Alan Mackenzie (Nuremberg, Germany).

☄

bug#1406: backward-up-list reports scan error incorrectly?

[xah lee]

Sorry never mind.

i was able to duplicate your error. Some of my testing in previous  
post went wrong because i was in text-mode.

though, isn't this something easy to fix? I mean, it surely is easy  
to determine if cursor is in inside double quotes, therefore it can  
simply move out the quote then do backward-up-list?

In any case, thanks.

   Xah
∑ http://xahlee.org/

☄

On Nov 22, 2008, at 4:53 AM, xah lee wrote:

hi Alan,

> (defun foo (arg)
>   "foo does nothing."
>   (interactive "P"))
>
> Put point inside "P", and do C-M-u `backward-up-list'.  You get the
> error.


i couldn't duplicate this. It seems to work for me.

am using
GNU Emacs 22.2.1 (powerpc-apple-darwin8.11.0, Carbon Version 1.6.0)  
of 2008-04-05 on g5.tokyo.stp.isas.jaxa.jp

i tried aquamacs. I can duplicate it with -Q, but not with -q.

Now i try carbon emacs again with -q, and i _can_ duplicate it.

... umm... not sure what is the problem now... since i'm pretty sure  
i don't have any customization for emacs-lisp-mode ...

am kinda tired now. Maybe i'll do some more testing to see what i  
find out.

   Xah
∑ http://xahlee.org/

☄

On Nov 22, 2008, at 4:36 AM, Alan Mackenzie wrote:

Hi, Xah!

On Fri, Nov 21, 2008 at 03:26:03PM -0800, xah lee wrote:
> Hi Alan,

> it works if you switch to text mode though.

> It doesn't work while in emacs-lisp-mode and fundamental mode.

> In any case, normally it works when inside string too. Just not in
> this particular case.

OK, I think half of the problem here is your prolixity, the other half
is my prolixity.  :-)

Try instead the following example:

(defun foo (arg)
   "foo does nothing."
   (interactive "P"))

Put point inside "P", and do C-M-u `backward-up-list'.  You get the
error.  Now modify the above function by inserting an open paren into
the doc string,

(defun foo (arg)
   "foo does nothing. ("
     (interactive "P"))

, and do the same again.  It finds that paren.  Why?  Because
backward-up-list assumes its starting point is NOT in a string.  It
jumps backwards over (what it thinks is) the string

     "\n(interactive "

, and then finds the ?\(.

This is exactly what is happening in your somewhat larger example.

In text mode, presumably the syntax table doesn't define ?\" as a string
quote.  In fundamental mode, presumably it does.  "(syntax-after
(point))" is your friend here.

>   Xah

-- 
Alan Mackenzie (Nuremberg, Germany).

☄



☄

bug#1406: backward-up-list reports scan error incorrectly?

[Alan Mackenzie]

Hi, Xah!

On Sat, Nov 22, 2008 at 05:01:10AM -0800, xah lee wrote:
> i was able to duplicate your error. Some of my testing in previous
> post went wrong because i was in text-mode.

> though, isn't this something easy to fix?

No, because it isn't a bug.  It's the way the function is meant to work.
If there is a bug, it's that the doc-string (and maybe the elisp manual,
I haven't looked) is vague and incomplete.

It seems that you really want a slightly different function.

> I mean, it surely is easy  to determine if cursor is in inside double
> quotes, therefore it can simply move out the quote then do
> backward-up-list?

Well, it's easy, but very slow.  You first have to define what "inside"
and "outside" mean.

For example, you might scan from BOB, and point is inside a string if
there've been an odd number of string-quotes up to now (taking comments,
etc. into account).

Emacs Lisp Mode defines "inside a string" as an odd number of
string-quotes since the last "(" in column-0.  Try going into a large
Elisp file and deleting one of the "s.  It throws the fontification out
for the rest of the function, but then "recovers" for the rest of the
file.

> In any case, thanks.

You're welcome!

>   Xah
> ??? http://xahlee.org/

-- 
Alan Mackenzie (Nuremberg, Germany).


> On Nov 22, 2008, at 4:53 AM, xah lee wrote:
 
> hi Alan,

> >(defun foo (arg)
> >  "foo does nothing."
> >  (interactive "P"))

> >Put point inside "P", and do C-M-u `backward-up-list'.  You get the
> >error.

> (defun foo (arg)
>   "foo does nothing. ("
>     (interactive "P"))

> , and do the same again.  It finds that paren.  Why?  Because
> backward-up-list assumes its starting point is NOT in a string.  It
> jumps backwards over (what it thinks is) the string

>     "\n(interactive "

> , and then finds the ?\(.

> This is exactly what is happening in your somewhat larger example.

> In text mode, presumably the syntax table doesn't define ?\" as a string
> quote.  In fundamental mode, presumably it does.  "(syntax-after
> (point))" is your friend here.

bug#1406: backward-up-list reports scan error incorrectly?

[Glenn Morris]

I don't see a need to keep open this particular report.
The doc of backward-up-list says:
"This command assumes point is not in a string or comment."