bug#39134: 28.0.50; forward-sexp fails on curved singlequotes

bug#39134: 28.0.50; forward-sexp fails on curved singlequotes

[Andreas Röhler]

Please consider output of "M-x describe-function RET defun RET".

Go in first line to beginning of "‘byte-run.el’". From "Char: ‘ (8216, 
#o20030, #x2018,"
"M-x forward-sexp RET". Cursor stops at dot, but should go until Char: ’ 
(8217, #o20031, #x2019)

Later on in buffer from start of "‘interactive’" it works as expected.
Also on ‘defun-declarations-alist’.

Thanks developing Emacs,
Andreas

In GNU Emacs 28.0.50 (build 1, i686-pc-linux-gnu, GTK+ Version 3.14.5)
  of 2020-01-10 built on sprache
Repository revision: 17cfd708575c351d030f8b05c5921d1867028d79
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.11604000
System Description: Debian GNU/Linux 8 (jessie)

Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.
Type C-x 1 to delete the help window, C-M-v to scroll help.
You can run the command ‘describe-function’ with C-h f
Type C-x 1 to delete the help window, C-M-v to scroll help.

Configured using:
  'configure --with-modules --with-mailutils'

Configured features:
XPM JPEG TIFF GIF PNG SOUND DBUS GSETTINGS GLIB NOTIFY INOTIFY ACL
GNUTLS LIBXML2 FREETYPE XFT ZLIB TOOLKIT_SCROLL_BARS GTK3 X11 XDBE XIM
MODULES THREADS PDUMPER GMP

Important settings:
   value of $LANG: de_DE.utf8
   locale-coding-system: utf-8-unix

Major mode: Help

Minor modes in effect:
   tooltip-mode: t
   global-eldoc-mode: t
   electric-indent-mode: t
   mouse-wheel-mode: t
   tool-bar-mode: t
   menu-bar-mode: t
   file-name-shadow-mode: t
   global-font-lock-mode: t
   font-lock-mode: t
   blink-cursor-mode: t
   auto-composition-mode: t
   auto-encryption-mode: t
   auto-compression-mode: t
   buffer-read-only: t
   line-number-mode: t
   transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr misearch multi-isearch emacsbug message rmc puny
dired dired-loaddefs format-spec rfc822 mml mml-sec password-cache epa
derived epg epg-config gnus-util rmail rmail-loaddefs
text-property-search time-date subr-x mm-decode mm-bodies mm-encode
mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail rfc2047
rfc2045 ietf-drums mm-util mail-prsvr mail-utils cl-extra seq byte-opt
gv bytecomp byte-compile cconv mode-local find-func thingatpt help-fns
radix-tree help-mode easymenu cl-loaddefs cl-lib tooltip eldoc electric
uniquify ediff-hook vc-hooks lisp-float-type mwheel term/x-win x-win
term/common-win x-dnd tool-bar dnd fontset image regexp-opt fringe
tabulated-list replace newcomment text-mode elisp-mode lisp-mode
prog-mode register page tab-bar menu-bar rfn-eshadow isearch timer
select scroll-bar mouse jit-lock font-lock syntax facemenu font-core
term/tty-colors frame minibuffer cl-generic cham georgian utf-8-lang
misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms
cp51932 hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese composite charscript charprop case-table epa-hook
jka-cmpr-hook help simple abbrev obarray cl-preloaded nadvice loaddefs
button faces cus-face macroexp files text-properties overlay sha1 md5
base64 format env code-pages mule custom widget hashtable-print-readable
backquote threads dbusbind inotify dynamic-setting system-font-setting
font-render-setting move-toolbar gtk x-toolkit x multi-tty
make-network-process emacs)

Memory information:
((conses 8 144084 8387)
  (symbols 24 28481 1)
  (strings 16 62818 2702)
  (string-bytes 1 973737)
  (vectors 8 10943)
  (vector-slots 4 145597 16200)
  (floats 8 30 32)
  (intervals 28 273 10)
  (buffers 568 13)
  (heap 1024 11552 1297))

bug#39134: 28.0.50; forward-sexp fails on curved singlequotes

[Eli Zaretskii]

> From: Andreas Röhler <andreas.roehler@easy-emacs.de>
> Date: Tue, 14 Jan 2020 14:35:34 +0100
> 
> Please consider output of "M-x describe-function RET defun RET".
> 
> Go in first line to beginning of "‘byte-run.el’". From "Char: ‘ (8216, 
> #o20030, #x2018,"
> "M-x forward-sexp RET". Cursor stops at dot, but should go until Char: ’ 
> (8217, #o20031, #x2019)

Why do you think it should go until ’ ?

bug#39134: 28.0.50; forward-sexp fails on curved singlequotes

[Andreas Röhler]

On 14.01.20 16:29, Eli Zaretskii wrote:
>> From: Andreas Röhler <andreas.roehler@easy-emacs.de>
>> Date: Tue, 14 Jan 2020 14:35:34 +0100
>>
>> Please consider output of "M-x describe-function RET defun RET".
>>
>> Go in first line to beginning of "‘byte-run.el’". From "Char: ‘ (8216,
>> #o20030, #x2018,"
>> "M-x forward-sexp RET". Cursor stops at dot, but should go until Char: ’
>> (8217, #o20031, #x2019)
> Why do you think it should go until ’ ?


Consider it a balanced expression closed by that char.

bug#39134: 28.0.50; forward-sexp fails on curved singlequotes

[Eli Zaretskii]

> Cc: 39134@debbugs.gnu.org
> From: Andreas Röhler <andreas.roehler@easy-emacs.de>
> Date: Tue, 14 Jan 2020 17:06:54 +0100
> 
> >> "M-x forward-sexp RET". Cursor stops at dot, but should go until Char: ’
> >> (8217, #o20031, #x2019)
> > Why do you think it should go until ’ ?
> 
> 
> Consider it a balanced expression closed by that char.

I'm not sure I understand how that makes sense in a buffer not in Lisp
mode, but maybe I'm missing something.

bug#39134: 28.0.50; forward-sexp fails on curved singlequotes

[Andreas Röhler]

On 14.01.20 17:20, Eli Zaretskii wrote:
>> Cc: 39134@debbugs.gnu.org
>> From: Andreas Röhler <andreas.roehler@easy-emacs.de>
>> Date: Tue, 14 Jan 2020 17:06:54 +0100
>>
>>>> "M-x forward-sexp RET". Cursor stops at dot, but should go until Char: ’
>>>> (8217, #o20031, #x2019)
>>> Why do you think it should go until ’ ?
>>
>> Consider it a balanced expression closed by that char.
> I'm not sure I understand how that makes sense in a buffer not in Lisp
> mode, but maybe I'm missing something.


IMO the sense comes from the markup, from the editing point of view.

bug#39134: 28.0.50; forward-sexp fails on curved singlequotes

[mvar]

Andreas Röhler <andreas.roehler@easy-emacs.de> writes:

> Please consider output of "M-x describe-function RET defun RET".
>
> Go in first line to beginning of "‘byte-run.el’". From "Char: ‘ (8216, #o20030,
> #x2018,"
> "M-x forward-sexp RET". Cursor stops at dot, but should go until Char: ’ (8217,
> #o20031, #x2019)
>
> Later on in buffer from start of "‘interactive’" it works as expected.
> Also on ‘defun-declarations-alist’.
>
> Thanks developing Emacs,
> Andreas

hi Andreas,

even for ‘interactive’ it does not work as expected - the cursor moves
right on top of the second curly singlequote instead of the character to
its right (the dot in that output). But this happens in the help
text. In scratch buffer for the below string:

boing ‘forward-sexp.test’ boing

forward-sexp acknowledges 3 sexps. But adding an extra space

boing ‘forward-sexp.test ’ boing

will break it and the second sexp ends on the newly added space.
Looks like that these specific quotation marks are not considered as equivalents
of parentheses.

do you see this as a general problem or just for the help-mode text?

thank you in advance,
Michalis

bug#39134: 28.0.50; forward-sexp fails on curved singlequotes

[Andreas Röhler]

On 16.08.21 08:20, mvar wrote:
> Andreas Röhler <andreas.roehler@easy-emacs.de> writes:
>
>> Please consider output of "M-x describe-function RET defun RET".
>>
>> Go in first line to beginning of "‘byte-run.el’". From "Char: ‘ (8216, #o20030,
>> #x2018,"
>> "M-x forward-sexp RET". Cursor stops at dot, but should go until Char: ’ (8217,
>> #o20031, #x2019)
>>
>> Later on in buffer from start of "‘interactive’" it works as expected.
>> Also on ‘defun-declarations-alist’.
>>
>> Thanks developing Emacs,
>> Andreas
> hi Andreas,
>
> even for ‘interactive’ it does not work as expected - the cursor moves
> right on top of the second curly singlequote instead of the character to
> its right (the dot in that output). But this happens in the help
> text. In scratch buffer for the below string:
>
> boing ‘forward-sexp.test’ boing
>
> forward-sexp acknowledges 3 sexps. But adding an extra space
>
> boing ‘forward-sexp.test ’ boing
>
> will break it and the second sexp ends on the newly added space.
> Looks like that these specific quotation marks are not considered as equivalents
> of parentheses.
>
> do you see this as a general problem or just for the help-mode text?
>
> thank you in advance,
> Michalis

Hi Michalis,

don't have any idea WRT importance of this.

Just thought it may work  - modifying the help-mode syntax-table should 
do it.

Best,

Andreas

bug#39134: 28.0.50; forward-sexp fails on curved singlequotes

[Michalis V.]

[-- Attachment #1: Type: text/plain, Size: 991 bytes --]

Andreas Röhler <andreas.roehler@easy-emacs.de> writes:

> Andreas Röhler <andreas.roehler@easy-emacs.de> writes:
>
> Hi Michalis,
>
> don't have any idea WRT importance of this.
>
> Just thought it may work  - modifying the help-mode syntax-table should do it.
>
> Best,
>
> Andreas

hi Andreas,

Looking at the help-mode.el it wasn't very clear to me how the help-mode
syntax-table variable was actually created - there wasn't any defvar
defined but rather the syntax-table was applied inside two
defuns. Despite that "C-h v help-mode-syntax-table" would redirect 
to this line in help-mode.el

(define-derived-mode help-mode special-mode "Help"

for some cryptic reason. Anyway attached is a patch that introduces
properly the help-mode-syntax-table var which derives from the elisp one
and sets ‘’ as parens. Now forward-sexp jumps over the definitions in
the *Help* buffer as it should despite any punctuation they may contain. 

cheers,
Michalis


[-- Attachment #2: help-mode.diff --]
[-- Type: text/x-patch, Size: 1527 bytes --]

diff --git a/lisp/help-mode.el b/lisp/help-mode.el
index 2f82d83ceb..6280fad1c1 100644
--- a/lisp/help-mode.el
+++ b/lisp/help-mode.el
@@ -111,6 +111,15 @@ help-xref-stack-forward-item
 (setq-default help-xref-stack nil help-xref-stack-item nil)
 (setq-default help-xref-forward-stack nil help-xref-forward-stack-item nil)
 
+(defvar help-mode-syntax-table
+  (let ((table (make-syntax-table emacs-lisp-mode-syntax-table)))
+    ;; treat single quotes as parens so that forward-sexp does not
+    ;; break when quoted string contains punctuation
+    (modify-syntax-entry ?‘ "(’  " table)
+    (modify-syntax-entry ?’ ")‘  " table)
+    table)
+  "Syntax table used in `help-mode'.")
+
 (defcustom help-mode-hook nil
   "Hook run by `help-mode'."
   :type 'hook
@@ -490,7 +499,7 @@ help-make-xrefs
         (let ((stab (syntax-table))
               (case-fold-search t)
               (inhibit-read-only t))
-          (set-syntax-table emacs-lisp-mode-syntax-table)
+          (set-syntax-table help-mode-syntax-table)
           ;; The following should probably be abstracted out.
           (unwind-protect
               (progn
@@ -643,7 +652,7 @@ help-insert-xref-button
 (defun help-xref-on-pp (from to)
   "Add xrefs for symbols in `pp's output between FROM and TO."
   (if (> (- to from) 5000) nil
-    (with-syntax-table emacs-lisp-mode-syntax-table
+    (with-syntax-table help-mode-syntax-table
       (save-excursion
 	(save-restriction
 	  (narrow-to-region from to)

bug#39134: 28.0.50; forward-sexp fails on curved singlequotes

[Lars Ingebrigtsen]

"Michalis V." <mvar.40k@gmail.com> writes:

> for some cryptic reason. Anyway attached is a patch that introduces
> properly the help-mode-syntax-table var which derives from the elisp one
> and sets ‘’ as parens. Now forward-sexp jumps over the definitions in
> the *Help* buffer as it should despite any punctuation they may contain. 

Thanks; I've now pushed your patch to Emacs 28.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no