bug#33940: 27.0.50; ?\^c syntax confuses scanning

bug#33940: 27.0.50; ?\^c syntax confuses scanning

[Michael Heerdegen]

Hello,

in 20b858ef13f8f71fae6cbce5cdac31c4dd130600 "Prefer \... to control
chars in .el literals" woman.el has been changed to contain reader
syntaxes like "?\^]" - which apparently confuses `scan-sexps', e.g.

(scan-sexps (point-min) (point-max))
|- (scan-error "Containing expression ends prematurely" 81997 81998)

Obviously `scan-sexps' doesn't handle this kind of syntax correctly.

This confuses paren highlighting, by-sexp and by-list movement, prevents
enabling paredit mode, and such unpleasant things.

(CC'ing the author of the commit, Paul Eggert)

Thanks and regards,

Michael.



In GNU Emacs 27.0.50 (build 22, x86_64-pc-linux-gnu, GTK+ Version 3.24.2)
 of 2018-12-31 built on drachen
Repository revision: 174d64d4cefb3c49e260a2eb3d9015b0f092f922
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.12003000
System Description: Debian GNU/Linux buster/sid

bug#33940: 27.0.50; ?\^c syntax confuses scanning

[Paul Eggert]

Michael Heerdegen wrote:
> in 20b858ef13f8f71fae6cbce5cdac31c4dd130600 "Prefer \... to control
> chars in .el literals" woman.el has been changed to contain reader
> syntaxes like "?\^]" - which apparently confuses `scan-sexps', e.g.
> 
> (scan-sexps (point-min) (point-max))
> |- (scan-error "Containing expression ends prematurely" 81997 81998)
> 
> Obviously `scan-sexps' doesn't handle this kind of syntax correctly.

The second argument to scan-sexps is a count, not position, so you might want to 
rethink that example.

Anyway, I'm not seeing any problems. I built the emacs-26 branch and ran the 
shell command:

src/emacs -Q lisp/woman.el

and then typed:

M-: (scan-sexps (point-min) (point-max)) RET

and it returned nil, which is what I'd expect. Can you reproduce the problem 
with a simple, self-contained test case like that?

bug#33940: 27.0.50; ?\^c syntax confuses scanning

[Michael Heerdegen]

Paul Eggert <eggert@cs.ucla.edu> writes:

> The second argument to scan-sexps is a count, not position, so you
> might want to rethink that example.

Indeed, thanks.  But let's keep the example for now, since it provokes
the scan error.

> Anyway, I'm not seeing any problems. I built the emacs-26 branch and
> ran the shell command:
>
> src/emacs -Q lisp/woman.el
>
> and then typed:
>
> M-: (scan-sexps (point-min) (point-max)) RET
>
> and it returned nil, which is what I'd expect. Can you reproduce the
> problem with a simple, self-contained test case like that?

Yes, with exactly that recipe, but with emacs built from the master
branch, please.  Oh, and happy new year, if you not are still in the old
one.


Thanks,

Michael.

bug#33940: 27.0.50; ?\^c syntax confuses scanning

[Paul Eggert]

[-- Attachment #1: Type: text/plain, Size: 276 bytes --]

Michael Heerdegen wrote:
> Yes, with exactly that recipe, but with emacs built from the master
> branch, please.  Oh, and happy new year, if you not are still in the old
> one.

The same for you! Thanks for pointing me in the right direction; I installed the 
attached patch.

[-- Attachment #2: 0001-Fix-woman.el-to-not-confuse-scan-sexps.patch --]
[-- Type: text/x-patch, Size: 895 bytes --]

From 504457673dabfa89cb2eac89f093f10736c52ca7 Mon Sep 17 00:00:00 2001
From: Paul Eggert <eggert@cs.ucla.edu>
Date: Tue, 1 Jan 2019 11:35:17 -0800
Subject: [PATCH] Fix woman.el to not confuse scan-sexps

* lisp/woman.el (woman-unpadded-space-char):
Escape the close-bracket (Bug#33940).
---
 lisp/woman.el | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lisp/woman.el b/lisp/woman.el
index 13aea5d91c..9548fdc6b3 100644
--- a/lisp/woman.el
+++ b/lisp/woman.el
@@ -2078,7 +2078,7 @@ woman-escaped-escape-string
   (char-to-string woman-escaped-escape-char)
   "Internal string representation of escaped escape characters.")
 
-(defconst woman-unpadded-space-char ?\^]
+(defconst woman-unpadded-space-char ?\^\]
   ;; An arbitrary unused control character
   "Internal character representation of unpadded space characters.")
 (defconst woman-unpadded-space-string
-- 
2.17.1

bug#33940: 27.0.50; ?\^c syntax confuses scanning

[Michael Heerdegen]

Paul Eggert <eggert@cs.ucla.edu> writes:

> Thanks for pointing me in the right direction; I installed the
> attached patch.

Great, thanks.

BTW, do you know whether the original "?\^]" is considered correct read
syntax?  Does the reader only accept it by accident, or should
`scan-sexps' also be able to handle it?

Anyway, probably a corner case that doesn't matter much.  Thanks for
your fix,


Michael.

bug#33940: 27.0.50; ?\^c syntax confuses scanning

[Paul Eggert]

Michael Heerdegen wrote:
> BTW, do you know whether the original "?\^]" is considered correct read
> syntax?  Does the reader only accept it by accident, or should
> `scan-sexps' also be able to handle it?

Sorry, I don't know, so I played it safe.

bug#33940: 27.0.50; ?\^c syntax confuses scanning

[Andreas Schwab]

On Jan 02 2019, Michael Heerdegen <michael_heerdegen@web.de> wrote:

> BTW, do you know whether the original "?\^]" is considered correct read
> syntax?  Does the reader only accept it by accident, or should
> `scan-sexps' also be able to handle it?

It's as valid as `?]', which has the same problem wrt scan-sexps.
That's why prin1-char adds redundant backslashes.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

bug#33940: 27.0.50; ?\^c syntax confuses scanning

[Michael Heerdegen]

Andreas Schwab <schwab@linux-m68k.org> writes:

> It's as valid as `?]', which has the same problem wrt scan-sexps.
> That's why prin1-char adds redundant backslashes.

Ok, thanks, so it's nothing new.

Michael.