bug#70435: 30.0.50; cc-mode: <> are sometimes not reconized as parentheses

bug#70435: 30.0.50; cc-mode: <> are sometimes not reconized as parentheses

[Herman, Géza]

This is a subtle bug.  In some cases, <> template delimiters are
not recognized as delimiters, but punctuation characters.

Repro:
- put the yasnippet file (included below) into
<emacs-config-dir>/snippets/c++-mode/something
- install yasnippet
- start emacs
- M-x c++-mode
- M-x yas-minor-mode
- load snippets with "M-x yas-reload-all"
- write "ig", then press TAB to "yas-expand" the snippet
- move the cursor on the opening "<", and execute "M-x describe-char"
- notice that it will say "syntax: . which means: punctuation"
- if you edit the buffer (like add a space somewhere), and execute
describe-char again, Emacs will say "syntax: > which means: open,
matches >", so the syntax class becomes correct.

A possible explanation for this is that yasnippet edits the buffer in a
way that cc-mode doesn't notices the edit, so it has no chance to put
the correct syntax info on the inserted characters.

This is the snippet file (a simple template declaration):

--8<---------------cut here---------------start------------->8---
# -*- mode: snippet -*-
# name: something
# key: ig
# --
template <${1:typename AAA}> struct Foo;
--8<---------------cut here---------------end--------------->8---



In GNU Emacs 30.0.50 (build 1, x86_64-pc-linux-gnu, cairo version
 1.18.0) of 2024-04-12 built on okoska
Repository revision: b83d0d07bb316cd851517897a9d688d639441f90
Repository branch: my-modifications
Windowing system distributor 'The X.Org Foundation', version 11.0.12101008
System Description: Debian GNU/Linux trixie/sid

Configured using:
 'configure --with-native-compilation --without-compress-install
 --without-gconf --without-gsettings --without-dbus --with-small-ja-dic
 --with-json --with-xinput2 --with-x-toolkit=no --with-tree-sitter
 --with-cairo --with-cairo-xcb --disable-silent-rules
 'CFLAGS=-mtune=native -march=native -g3 -O3''

Configured features:
ACL CAIRO FREETYPE GIF GLIB GMP GNUTLS HARFBUZZ JPEG LCMS2 LIBOTF
LIBSELINUX LIBSYSTEMD LIBXML2 M17N_FLT MODULES NATIVE_COMP NOTIFY
INOTIFY OLDXMENU PDUMPER PNG RSVG SECCOMP SOUND SQLITE3 THREADS TIFF
TREE_SITTER WEBP X11 XDBE XIM XINPUT2 XPM ZLIB

Important settings:
  value of $LC_ALL: C.UTF-8
  value of $LANG: en_US.UTF-8
  value of $XMODIFIERS: @im=none
  locale-coding-system: utf-8-unix

Major mode: Fundamental

Minor modes in effect:
  vertico-multiform-mode: t
  recentf-mode: t
  which-key-mode: t
  global-auto-revert-mode: t
  savehist-mode: t
  ws-butler-global-mode: t
  ws-butler-mode: t
  diff-hl-flydiff-mode: t
  global-diff-hl-mode: t
  clean-aindent-mode: t
  global-whitespace-mode: t
  marginalia-mode: t
  vertico-mode: t
  global-anzu-mode: t
  anzu-mode: t
  global-evil-matchit-mode: t
  evil-matchit-mode: t
  evil-snipe-override-mode: t
  evil-snipe-mode: t
  evil-snipe-override-local-mode: t
  evil-snipe-local-mode: t
  global-evil-surround-mode: t
  evil-surround-mode: t
  global-evil-visualstar-mode: t
  evil-visualstar-mode: t
  better-jumper-mode: t
  better-jumper-local-mode: t
  evil-leader-mode: t
  global-evil-leader-mode: t
  global-hl-todo-mode: t
  winum-mode: t
  hes-mode: t
  gcmh-mode: t
  global-page-break-lines-mode: t
  evil-mode: t
  evil-local-mode: t
  save-place-mode: t
  override-global-mode: t
  minibuffer-depth-indicate-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tab-bar-history-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  minibuffer-regexp-mode: t
  column-number-mode: t
  line-number-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  abbrev-mode: t

Load-path shadows:
/home/geza/.emacs.d/elpa/transient-20240226.2332/transient hides /usr/local/share/emacs/30.0.50/lisp/transient
~/.emacs.d/lisp/emacs-gdb/gdb-mi hides /usr/local/share/emacs/30.0.50/lisp/progmodes/gdb-mi

Features:
(shadow sort project mail-extr emacsbug message mailcap yank-media puny
evil-collection-dired dired-git-info peep-dired dired-narrow delsel
dired-filter f s dired-aux dired-x dired-subtree dired-hacks-utils
evil-collection-wdired wdired ls-lisp dired dired-loaddefs rfc822 mml
mml-sec password-cache epa derived epg rfc6068 epg-config gnus-util
time-date mm-decode mm-bodies mm-encode mail-parse rfc2231 mailabbrev
gmm-utils mailheader sendmail rfc2047 rfc2045 ietf-drums mm-util
mail-prsvr mail-utils help-fns radix-tree mule-util cursor-sensor
evil-collection-consult consult-dir vertico-multiform consult-compile
compile evil-collection-comint comint ansi-osc ansi-color recentf
tree-widget wid-edit shut-up consult bookmark text-property-search pp
face-remap drag-stuff which-key autorevert filenotify savehist bm
evil-collection-info info ws-butler diff-hl-flydiff diff diff-hl
log-view pcvs-util vc-dir ewoc vc vc-dispatcher diff-mode
clean-aindent-mode column-enforce-mode whitespace orderless marginalia
vertico anzu evil-matchit evil-matchit-evil-setup evil-matchit-sdk
semantic/lex semantic/fw eieio eieio-core mode-local find-func
evil-exchange evil-args evil-indent-plus evil-textobj-line
evil-textobj-entire evil-textobj-column evil-textobj-anyblock evil-snipe
evil-surround evil-mc evil-mc-command-execute evil-mc-command-record
evil-mc-cursor-make evil-mc-region evil-mc-cursor-state evil-mc-undo
evil-mc-vars evil-mc-known-commands evil-mc-common avy evil-visualstar
evil-collection-simple evil-collection-replace evil-collection annalist
better-jumper pcase cl-macs evil-leader hl-todo compat hl-line
transpose-frame winum dash ov highlight-escape-sequences gcmh
page-break-lines evil evil-integration evil-maps evil-commands reveal
evil-jumps evil-command-window evil-types evil-search evil-ex
evil-macros evil-repeat evil-states evil-core evil-common thingatpt rect
evil-vars ring edmacro kmacro byte-opt saveplace bind-key easy-mmode
advice mb-depth comp cl-seq comp-cstr cl-extra help-mode warnings icons
subr-x gv cl-loaddefs cl-lib comp-run bytecomp byte-compile comp-common
rx rmc iso-transl tooltip cconv eldoc paren electric uniquify ediff-hook
vc-hooks lisp-float-type elisp-mode mwheel term/x-win x-win
term/common-win x-dnd touch-screen tool-bar dnd fontset image regexp-opt
fringe tabulated-list replace newcomment text-mode lisp-mode prog-mode
register page tab-bar menu-bar rfn-eshadow isearch easymenu timer select
scroll-bar mouse jit-lock font-lock syntax font-core term/tty-colors
frame minibuffer nadvice seq simple cl-generic indonesian philippine
cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech
european ethiopic indian cyrillic chinese composite emoji-zwj charscript
charprop case-table epa-hook jka-cmpr-hook help abbrev obarray oclosure
cl-preloaded button loaddefs theme-loaddefs faces cus-face macroexp
files window text-properties overlay sha1 md5 base64 format env
code-pages mule custom widget keymap hashtable-print-readable backquote
threads inotify lcms2 dynamic-setting font-render-setting cairo xinput2
x multi-tty move-toolbar make-network-process native-compile emacs)

Memory information:
((conses 16 269016 316498) (symbols 48 23000 189) (strings 32 74049 29305) (string-bytes 1 3233199) (vectors 16 36374)
 (vector-slots 8 426217 134450) (floats 8 230 192) (intervals 56 2422 251) (buffers 992 12))

bug#70435: 30.0.50; cc-mode: <> are sometimes not reconized as parentheses

[Eli Zaretskii]

> From: Herman@debbugs.gnu.org, Géza <geza.herman@gmail.com>
> Date: Wed, 17 Apr 2024 12:47:25 +0200
> 
> 
> This is a subtle bug.  In some cases, <> template delimiters are
> not recognized as delimiters, but punctuation characters.
> 
> Repro:
> - put the yasnippet file (included below) into
> <emacs-config-dir>/snippets/c++-mode/something
> - install yasnippet
> - start emacs
> - M-x c++-mode
> - M-x yas-minor-mode
> - load snippets with "M-x yas-reload-all"
> - write "ig", then press TAB to "yas-expand" the snippet
> - move the cursor on the opening "<", and execute "M-x describe-char"
> - notice that it will say "syntax: . which means: punctuation"
> - if you edit the buffer (like add a space somewhere), and execute
> describe-char again, Emacs will say "syntax: > which means: open,
> matches >", so the syntax class becomes correct.
> 
> A possible explanation for this is that yasnippet edits the buffer in a
> way that cc-mode doesn't notices the edit, so it has no chance to put
> the correct syntax info on the inserted characters.
> 
> This is the snippet file (a simple template declaration):
> 
> --8<---------------cut here---------------start------------->8---
> # -*- mode: snippet -*-
> # name: something
> # key: ig
> # --
> template <${1:typename AAA}> struct Foo;
> --8<---------------cut here---------------end--------------->8---

Alan, could you please look into this?

bug#70435: 30.0.50; cc-mode: <> are sometimes not reconized as parentheses

[Alan Mackenzie]

Hello, Eli.

On Sat, Apr 27, 2024 at 11:33:38 +0300, Eli Zaretskii wrote:
> > From: Herman@debbugs.gnu.org, Géza <geza.herman@gmail.com>
> > Date: Wed, 17 Apr 2024 12:47:25 +0200


> > This is a subtle bug.  In some cases, <> template delimiters are
> > not recognized as delimiters, but punctuation characters.

> > Repro:
> > - put the yasnippet file (included below) into
> > <emacs-config-dir>/snippets/c++-mode/something
> > - install yasnippet
> > - start emacs
> > - M-x c++-mode
> > - M-x yas-minor-mode
> > - load snippets with "M-x yas-reload-all"
> > - write "ig", then press TAB to "yas-expand" the snippet
> > - move the cursor on the opening "<", and execute "M-x describe-char"
> > - notice that it will say "syntax: . which means: punctuation"
> > - if you edit the buffer (like add a space somewhere), and execute
> > describe-char again, Emacs will say "syntax: > which means: open,
> > matches >", so the syntax class becomes correct.

> > A possible explanation for this is that yasnippet edits the buffer in a
> > way that cc-mode doesn't notices the edit, so it has no chance to put
> > the correct syntax info on the inserted characters.

> > This is the snippet file (a simple template declaration):

> > --8<---------------cut here---------------start------------->8---
> > # -*- mode: snippet -*-
> > # name: something
> > # key: ig
> > # --
> > template <${1:typename AAA}> struct Foo;
> > --8<---------------cut here---------------end--------------->8---

> Alan, could you please look into this?

Yes, I will.  Thanks!

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#70435: 30.0.50; cc-mode: <> are sometimes not reconized as parentheses

[Alan Mackenzie]

Hello, Géza.

On Wed, Apr 17, 2024 at 12:47:25 +0200, Herman wrote:

> This is a subtle bug.  In some cases, <> template delimiters are
> not recognized as delimiters, but punctuation characters.

> Repro:
> - put the yasnippet file (included below) into
> <emacs-config-dir>/snippets/c++-mode/something
> - install yasnippet
> - start emacs
> - M-x c++-mode
> - M-x yas-minor-mode
> - load snippets with "M-x yas-reload-all"
> - write "ig", then press TAB to "yas-expand" the snippet
> - move the cursor on the opening "<", and execute "M-x describe-char"
> - notice that it will say "syntax: . which means: punctuation"
> - if you edit the buffer (like add a space somewhere), and execute
> describe-char again, Emacs will say "syntax: > which means: open,
> matches >", so the syntax class becomes correct.

You've been a little less than fully explicit, but I think you're
executing these commands in the *scratch* buffer.  The first two lines,
which are commented out in emacs-lisp-mode, are no longer commented out
in C++ Mode.  There is a whole line of garbage after the last end of
statement marker, the (double) semicolon on line 2.

On using ig<TAB> to insert the snippet, it is hardly surprising that CC
Mode's syntactic analysis gets confused.  If you first comment out those
first two lines (put the region around them and do C-c C-c), then the
inserted snippet appears to get the correct syntax on its template
markers.

I don't think there's a bug here.  If you could show ig<TAB> producing
the effect when typed inside a syntactically correct context, things
might be different.  Can you reproduce the effect in correct C++ code?

> A possible explanation for this is that yasnippet edits the buffer in a
> way that cc-mode doesn't notices the edit, so it has no chance to put
> the correct syntax info on the inserted characters.

> This is the snippet file (a simple template declaration):

> --8<---------------cut here---------------start------------->8---
> # -*- mode: snippet -*-
> # name: something
> # key: ig
> # --
> template <${1:typename AAA}> struct Foo;
> --8<---------------cut here---------------end--------------->8---



> In GNU Emacs 30.0.50 (build 1, x86_64-pc-linux-gnu, cairo version
>  1.18.0) of 2024-04-12 built on okoska
> Repository revision: b83d0d07bb316cd851517897a9d688d639441f90
> Repository branch: my-modifications
> Windowing system distributor 'The X.Org Foundation', version 11.0.12101008
> System Description: Debian GNU/Linux trixie/sid

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#70435: 30.0.50; cc-mode: <> are sometimes not reconized as parentheses

[Herman, Géza]

Hello Alan,

Alan Mackenzie <acm@muc.de> writes:

> You've been a little less than fully explicit, but I think 
> you're
> executing these commands in the *scratch* buffer.  The first two 
> lines,
> which are commented out in emacs-lisp-mode, are no longer 
> commented out
> in C++ Mode.  There is a whole line of garbage after the last 
> end of
> statement marker, the (double) semicolon on line 2.
>
> On using ig<TAB> to insert the snippet, it is hardly surprising 
> that CC
> Mode's syntactic analysis gets confused.  If you first comment 
> out those
> first two lines (put the region around them and do C-c C-c), 
> then the
> inserted snippet appears to get the correct syntax on its 
> template
> markers.
>
> I don't think there's a bug here.  If you could show ig<TAB> 
> producing
> the effect when typed inside a syntactically correct context, 
> things
> might be different.  Can you reproduce the effect in correct C++ 
> code?

You're right, it seems that the example I provided wasn't the best 
(this issue happens with me in real code, I tried to create a 
minimal reproducible example).

If you delete the garbage from the scratch buffer, the bug doesn't 
reproduce indeed.  But, if you run (setq 
font-lock-maximum-decoration 2) before switching to c++-mode, the 
issue reproduces with an empty scratch buffer.  I use this setting 
because font-lock runs much faster this way, and I rely on the LSP 
server to do the "full" highlighting.

Sorry about the bad example, here are the fixed repro steps:

Repro:
- put the yasnippet file (included below) into
<emacs-config-dir>/snippets/c++-mode/something
- install yasnippet
- start emacs, scratch buffer appears
- delete the contents of the scratch buffer
- M-: (setq font-lock-maximum-decoration 2)
- M-x c++-mode
- M-x yas-minor-mode
- load snippets with "M-x yas-reload-all"
- write "ig", then press TAB to "yas-expand" the snippet
- move the cursor on the opening "<", and execute "M-x 
  describe-char"
- notice that it will say "syntax: . which means: punctuation"
- if you edit the buffer (like add a space somewhere), and execute
describe-char again, Emacs will say "syntax: > which means: open,
matches >", so the syntax class becomes correct.

Geza

bug#70435: 30.0.50; cc-mode: <> are sometimes not reconized as parentheses

[Alan Mackenzie]

Hello again, Géza.

On Sun, Apr 28, 2024 at 18:47:47 +0200, Herman, Géza wrote:
> Hello Alan,

> Alan Mackenzie <acm@muc.de> writes:

> > You've been a little less than fully explicit, but I think you're
> > executing these commands in the *scratch* buffer.  The first two
> > lines, which are commented out in emacs-lisp-mode, are no longer
> > commented out in C++ Mode.  There is a whole line of garbage after
> > the last end of statement marker, the (double) semicolon on line 2.

> > On using ig<TAB> to insert the snippet, it is hardly surprising that
> > CC Mode's syntactic analysis gets confused.  If you first comment
> > out those first two lines (put the region around them and do C-c
> > C-c), then the inserted snippet appears to get the correct syntax on
> > its template markers.

> > I don't think there's a bug here.  If you could show ig<TAB>
> > producing the effect when typed inside a syntactically correct
> > context, things might be different.  Can you reproduce the effect in
> > correct C++ code?

> You're right, it seems that the example I provided wasn't the best 
> (this issue happens with me in real code, I tried to create a 
> minimal reproducible example).

That's always appreciated.

> If you delete the garbage from the scratch buffer, the bug doesn't 
> reproduce indeed.  But, if you run (setq 
> font-lock-maximum-decoration 2) before switching to c++-mode, the 
> issue reproduces with an empty scratch buffer.  I use this setting 
> because font-lock runs much faster this way, and I rely on the LSP 
> server to do the "full" highlighting.

I confirm I can reproduce the bug with font-lock-maximum-decoration set
to 2.  I'll look into it.

> Sorry about the bad example, here are the fixed repro steps:

> Repro:
> - put the yasnippet file (included below) into
> <emacs-config-dir>/snippets/c++-mode/something
> - install yasnippet
> - start emacs, scratch buffer appears
> - delete the contents of the scratch buffer
> - M-: (setq font-lock-maximum-decoration 2)
> - M-x c++-mode
> - M-x yas-minor-mode
> - load snippets with "M-x yas-reload-all"
> - write "ig", then press TAB to "yas-expand" the snippet
> - move the cursor on the opening "<", and execute "M-x 
>   describe-char"
> - notice that it will say "syntax: . which means: punctuation"
> - if you edit the buffer (like add a space somewhere), and execute
> describe-char again, Emacs will say "syntax: > which means: open,
> matches >", so the syntax class becomes correct.

> Geza

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#70435: 30.0.50; cc-mode: <> are sometimes not reconized as parentheses

[Alan Mackenzie]

Hello, Géza.

On Sun, Apr 28, 2024 at 18:47:47 +0200, Herman, Géza wrote:
> Hello Alan,

> Alan Mackenzie <acm@muc.de> writes:

> > You've been a little less than fully explicit, but I think you're
> > executing these commands in the *scratch* buffer.  The first two
> > lines, which are commented out in emacs-lisp-mode, are no longer
> > commented out in C++ Mode.  There is a whole line of garbage after
> > the last end of statement marker, the (double) semicolon on line 2.

> > On using ig<TAB> to insert the snippet, it is hardly surprising that
> > CC Mode's syntactic analysis gets confused.  If you first comment
> > out those first two lines (put the region around them and do C-c
> > C-c), then the inserted snippet appears to get the correct syntax on
> > its template markers.

> > I don't think there's a bug here.  If you could show ig<TAB>
> > producing the effect when typed inside a syntactically correct
> > context, things might be different.  Can you reproduce the effect in
> > correct C++ code?

> You're right, it seems that the example I provided wasn't the best
> (this issue happens with me in real code, I tried to create a minimal
> reproducible example).

> If you delete the garbage from the scratch buffer, the bug doesn't
> reproduce indeed.  But, if you run (setq font-lock-maximum-decoration
> 2) before switching to c++-mode, the issue reproduces with an empty
> scratch buffer.  I use this setting because font-lock runs much faster
> this way, and I rely on the LSP server to do the "full" highlighting.

OK, as already said, I can reproduce the bug this way.  Thanks!

> Sorry about the bad example, here are the fixed repro steps:

> Repro:
> - put the yasnippet file (included below) into
> <emacs-config-dir>/snippets/c++-mode/something
> - install yasnippet
> - start emacs, scratch buffer appears
> - delete the contents of the scratch buffer
> - M-: (setq font-lock-maximum-decoration 2)
> - M-x c++-mode
> - M-x yas-minor-mode
> - load snippets with "M-x yas-reload-all"
> - write "ig", then press TAB to "yas-expand" the snippet
> - move the cursor on the opening "<", and execute "M-x 
>   describe-char"
> - notice that it will say "syntax: . which means: punctuation"
> - if you edit the buffer (like add a space somewhere), and execute
> describe-char again, Emacs will say "syntax: > which means: open,
> matches >", so the syntax class becomes correct.

I have a fix, I think.  It is actually a two line fix, removing a test
from the top of a function, but it involves reindenting the entire rest
of the function.

Please apply the patch below, recompile cc-engine.el, then load the
resulting CC Mode into a running Emacs.  Please test it on your real C++
code, and let me know if the bug is actually fixed.  Thanks!


diff -r 072940aaeb40 cc-engine.el
--- a/cc-engine.el	Sun Apr 14 07:59:01 2024 +0000
+++ b/cc-engine.el	Mon Apr 29 15:42:05 2024 +0000
@@ -7172,153 +7172,152 @@
   ;; FIXME!!!  This routine ignores the possibility of macros entirely.
   ;; 2010-01-29.
 
-  (when (> end beg)
-    ;; Extend the region (BEG END) to deal with any complicating literals.
-    (let* ((lit-search-beg (if (memq (char-before beg) '(?/ ?*))
-			       (1- beg) beg))
-	   (lit-search-end (if (memq (char-after end) '(?/ ?*))
-			       (1+ end) end))
-	   ;; Note we can't use c-full-pp-to-literal here, since we haven't
-	   ;; yet applied syntax-table properties to ends of lines, etc.
-	   (lit-search-beg-s (c-semi-pp-to-literal lit-search-beg))
-	   (beg-literal-beg (car (cddr lit-search-beg-s)))
-	   (lit-search-end-s (c-semi-pp-to-literal lit-search-end))
-	   (end-literal-beg (car (cddr lit-search-end-s)))
-	   (beg-literal-end (c-end-of-literal lit-search-beg-s lit-search-beg))
-	   (end-literal-end (c-end-of-literal lit-search-end-s lit-search-end))
-	   new-beg new-end search-region)
-
-      ;; Determine any new end of literal resulting from the insertion/deletion.
-      (setq search-region
-	    (if (and (eq beg-literal-beg end-literal-beg)
-		     (eq beg-literal-end end-literal-end))
-		(if beg-literal-beg
-		    nil
-		  (cons beg
-			(max end
-			     (or beg-literal-end (point-min))
-			     (or end-literal-end (point-min)))))
-	      (cons (or beg-literal-beg beg)
-		    (max end
-			 (or beg-literal-end (point-min))
-			 (or end-literal-end (point-min))))))
-
-      (when search-region
-	;; If we've just inserted text, mask its syntaxes temporarily so that
-	;; they won't interfere with the undoing of the properties on the <s
-	;; and >s.
-	(c-save-buffer-state (syn-tab-settings syn-tab-value
-					       swap-open-string-ends)
-	  (unwind-protect
-	      (progn
-		(when old-len
-		  ;; Special case: If a \ has just been inserted into a
-		  ;; string, escaping or unescaping a LF, temporarily swap
-		  ;; the LF's syntax-table text property with that of the
-		  ;; former end of the open string.
-		  (goto-char end)
-		  (when (and (eq (cadr lit-search-beg-s) 'string)
-			     (not (eq beg-literal-end end-literal-end))
-			     (skip-chars-forward "\\\\")
-			     (eq (char-after) ?\n)
-			     (not (zerop (skip-chars-backward "\\\\"))))
-		    (setq swap-open-string-ends t)
-		    (if (c-get-char-property (1- beg-literal-end)
-					     'syntax-table)
-			(progn
-			  (c-clear-char-property (1- beg-literal-end)
-						 'syntax-table)
-			  (c-put-string-fence (1- end-literal-end)))
-		      (c-put-string-fence (1- beg-literal-end))
-		      (c-clear-char-property (1- end-literal-end)
-					     'syntax-table)))
-
-		  ;; Save current settings of the 'syntax-table property in
-		  ;; (BEG END), then splat these with the punctuation value.
-		  (goto-char beg)
-		  (while (setq syn-tab-value
-			       (c-search-forward-non-nil-char-property
-				'syntax-table end))
-		    (when (not (c-get-char-property (1- (point)) 'category))
-		      (push (cons (1- (point)) syn-tab-value)
-			    syn-tab-settings)))
-
-		  (c-put-char-properties beg end 'syntax-table '(1))
-		  ;; If an open string's opener has just been neutralized,
-		  ;; do the same to the terminating LF.
-		  (when (and end-literal-end
-			     (eq (char-before end-literal-end) ?\n)
-			     (equal (c-get-char-property
-				     (1- end-literal-end) 'syntax-table)
-				    '(15)))
-		    (push (cons (1- end-literal-end) '(15)) syn-tab-settings)
-		    (c-put-char-property (1- end-literal-end) 'syntax-table
-					 '(1))))
-
-		(let
-		    ((beg-lit-start (progn (goto-char beg) (c-literal-start)))
-		     beg-limit end-limit <>-pos)
-		  ;; Locate the earliest < after the barrier before the
-		  ;; changed region, which isn't already marked as a paren.
-		  (goto-char (or beg-lit-start beg))
-		  (setq beg-limit (c-determine-limit 5000))
-
-		  ;; Remove the syntax-table/category properties from each pertinent <...>
-		  ;; pair.  Firstly, the ones with the < before beg and > after beg....
-		  (goto-char (cdr search-region))
-		  (while (progn (c-syntactic-skip-backward "^;{}<" beg-limit)
-				(eq (char-before) ?<))
-		    (c-backward-token-2)
-		    (when (eq (char-after) ?<)
-		      (when (setq <>-pos (c-clear-<-pair-props-if-match-after
-					  (car search-region)))
-			(setq new-end <>-pos))
-		      (setq new-beg (point))))
-
-		  ;; ...Then the ones with < before end and > after end.
-		  (goto-char (car search-region))
-		  (setq end-limit (c-determine-+ve-limit 5000))
-		  (while (and (c-syntactic-re-search-forward "[;{}>]" end-limit 'end)
-			      (eq (char-before) ?>))
-		    (when (eq (char-before) ?>)
-		      (if (and (looking-at c->-op-cont-regexp)
-			       (not (eq (char-after) ?>)))
-			  (goto-char (match-end 0))
-			(when
-			    (and (setq <>-pos
-				       (c-clear->-pair-props-if-match-before
-					(cdr search-region)
-					(1- (point))))
-				 (or (not new-beg)
-				     (< <>-pos new-beg)))
-			  (setq new-beg <>-pos))
-			(when (or (not new-end) (> (point) new-end))
-			  (setq new-end (point))))))))
-
-	    (when old-len
-	      (c-clear-char-properties beg end 'syntax-table)
-	      (dolist (elt syn-tab-settings)
-		(if (cdr elt)
-		    (c-put-char-property (car elt) 'syntax-table (cdr elt)))))
-	    ;; Swap the '(15) syntax-table property on open string LFs back
-	    ;; again.
-	    (when swap-open-string-ends
-	      (if (c-get-char-property (1- beg-literal-end)
-				       'syntax-table)
-		  (progn
-		    (c-clear-char-property (1- beg-literal-end)
+  ;; Extend the region (BEG END) to deal with any complicating literals.
+  (let* ((lit-search-beg (if (memq (char-before beg) '(?/ ?*))
+			     (1- beg) beg))
+	 (lit-search-end (if (memq (char-after end) '(?/ ?*))
+			     (1+ end) end))
+	 ;; Note we can't use c-full-pp-to-literal here, since we haven't
+	 ;; yet applied syntax-table properties to ends of lines, etc.
+	 (lit-search-beg-s (c-semi-pp-to-literal lit-search-beg))
+	 (beg-literal-beg (car (cddr lit-search-beg-s)))
+	 (lit-search-end-s (c-semi-pp-to-literal lit-search-end))
+	 (end-literal-beg (car (cddr lit-search-end-s)))
+	 (beg-literal-end (c-end-of-literal lit-search-beg-s lit-search-beg))
+	 (end-literal-end (c-end-of-literal lit-search-end-s lit-search-end))
+	 new-beg new-end search-region)
+
+    ;; Determine any new end of literal resulting from the insertion/deletion.
+    (setq search-region
+	  (if (and (eq beg-literal-beg end-literal-beg)
+		   (eq beg-literal-end end-literal-end))
+	      (if beg-literal-beg
+		  nil
+		(cons beg
+		      (max end
+			   (or beg-literal-end (point-min))
+			   (or end-literal-end (point-min)))))
+	    (cons (or beg-literal-beg beg)
+		  (max end
+		       (or beg-literal-end (point-min))
+		       (or end-literal-end (point-min))))))
+
+    (when search-region
+      ;; If we've just inserted text, mask its syntaxes temporarily so that
+      ;; they won't interfere with the undoing of the properties on the <s
+      ;; and >s.
+      (c-save-buffer-state (syn-tab-settings syn-tab-value
+					     swap-open-string-ends)
+	(unwind-protect
+	    (progn
+	      (when old-len
+		;; Special case: If a \ has just been inserted into a
+		;; string, escaping or unescaping a LF, temporarily swap
+		;; the LF's syntax-table text property with that of the
+		;; former end of the open string.
+		(goto-char end)
+		(when (and (eq (cadr lit-search-beg-s) 'string)
+			   (not (eq beg-literal-end end-literal-end))
+			   (skip-chars-forward "\\\\")
+			   (eq (char-after) ?\n)
+			   (not (zerop (skip-chars-backward "\\\\"))))
+		  (setq swap-open-string-ends t)
+		  (if (c-get-char-property (1- beg-literal-end)
 					   'syntax-table)
-		    (c-put-string-fence (1- end-literal-end)))
-		(c-put-string-fence (1- beg-literal-end))
-		(c-clear-char-property (1- end-literal-end)
-				       'syntax-table)))))
-	  ;; Extend the fontification region, if needed.
-	  (and new-beg
-	       (< new-beg c-new-BEG)
-	       (setq c-new-BEG new-beg))
-	  (and new-end
-	       (> new-end c-new-END)
-	       (setq c-new-END new-end))))))
+		      (progn
+			(c-clear-char-property (1- beg-literal-end)
+					       'syntax-table)
+			(c-put-string-fence (1- end-literal-end)))
+		    (c-put-string-fence (1- beg-literal-end))
+		    (c-clear-char-property (1- end-literal-end)
+					   'syntax-table)))
+
+		;; Save current settings of the 'syntax-table property in
+		;; (BEG END), then splat these with the punctuation value.
+		(goto-char beg)
+		(while (setq syn-tab-value
+			     (c-search-forward-non-nil-char-property
+			      'syntax-table end))
+		  (when (not (c-get-char-property (1- (point)) 'category))
+		    (push (cons (1- (point)) syn-tab-value)
+			  syn-tab-settings)))
+
+		(c-put-char-properties beg end 'syntax-table '(1))
+		;; If an open string's opener has just been neutralized,
+		;; do the same to the terminating LF.
+		(when (and end-literal-end
+			   (eq (char-before end-literal-end) ?\n)
+			   (equal (c-get-char-property
+				   (1- end-literal-end) 'syntax-table)
+				  '(15)))
+		  (push (cons (1- end-literal-end) '(15)) syn-tab-settings)
+		  (c-put-char-property (1- end-literal-end) 'syntax-table
+				       '(1))))
+
+	      (let
+		  ((beg-lit-start (progn (goto-char beg) (c-literal-start)))
+		   beg-limit end-limit <>-pos)
+		;; Locate the earliest < after the barrier before the
+		;; changed region, which isn't already marked as a paren.
+		(goto-char (or beg-lit-start beg))
+		(setq beg-limit (c-determine-limit 5000))
+
+		;; Remove the syntax-table/category properties from each pertinent <...>
+		;; pair.  Firstly, the ones with the < before beg and > after beg....
+		(goto-char (cdr search-region))
+		(while (progn (c-syntactic-skip-backward "^;{}<" beg-limit)
+			      (eq (char-before) ?<))
+		  (c-backward-token-2)
+		  (when (eq (char-after) ?<)
+		    (when (setq <>-pos (c-clear-<-pair-props-if-match-after
+					(car search-region)))
+		      (setq new-end <>-pos))
+		    (setq new-beg (point))))
+
+		;; ...Then the ones with < before end and > after end.
+		(goto-char (car search-region))
+		(setq end-limit (c-determine-+ve-limit 5000))
+		(while (and (c-syntactic-re-search-forward "[;{}>]" end-limit 'end)
+			    (eq (char-before) ?>))
+		  (when (eq (char-before) ?>)
+		    (if (and (looking-at c->-op-cont-regexp)
+			     (not (eq (char-after) ?>)))
+			(goto-char (match-end 0))
+		      (when
+			  (and (setq <>-pos
+				     (c-clear->-pair-props-if-match-before
+				      (cdr search-region)
+				      (1- (point))))
+			       (or (not new-beg)
+				   (< <>-pos new-beg)))
+			(setq new-beg <>-pos))
+		      (when (or (not new-end) (> (point) new-end))
+			(setq new-end (point))))))))
+
+	  (when old-len
+	    (c-clear-char-properties beg end 'syntax-table)
+	    (dolist (elt syn-tab-settings)
+	      (if (cdr elt)
+		  (c-put-char-property (car elt) 'syntax-table (cdr elt)))))
+	  ;; Swap the '(15) syntax-table property on open string LFs back
+	  ;; again.
+	  (when swap-open-string-ends
+	    (if (c-get-char-property (1- beg-literal-end)
+				     'syntax-table)
+		(progn
+		  (c-clear-char-property (1- beg-literal-end)
+					 'syntax-table)
+		  (c-put-string-fence (1- end-literal-end)))
+	      (c-put-string-fence (1- beg-literal-end))
+	      (c-clear-char-property (1- end-literal-end)
+				     'syntax-table)))))
+      ;; Extend the fontification region, if needed.
+      (and new-beg
+	   (< new-beg c-new-BEG)
+	   (setq c-new-BEG new-beg))
+      (and new-end
+	   (> new-end c-new-END)
+	   (setq c-new-END new-end)))))
 
 (defun c-before-change-check-<>-operators (beg end)
   ;; When we're deleting text, unmark certain pairs of "< .... >" which are


> Geza

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#70435: 30.0.50; cc-mode: <> are sometimes not reconized as parentheses

[Herman, Géza]

Hello Alan,

Alan Mackenzie <acm@muc.de> writes:

> I have a fix, I think.  It is actually a two line fix, removing 
> a test
> from the top of a function, but it involves reindenting the 
> entire rest
> of the function.
>
> Please apply the patch below, recompile cc-engine.el, then load 
> the
> resulting CC Mode into a running Emacs.  Please test it on your 
> real C++
> code, and let me know if the bug is actually fixed.  Thanks!

I can confirm that the patch fixes the issue, I tested it in real 
circumstances.

Thanks for fixing the problem so quickly!

Géza

bug#70435: 30.0.50; cc-mode: <> are sometimes not reconized as parentheses

[Eli Zaretskii]

> From: Herman, Géza <geza.herman@gmail.com>
> Cc: Herman Géza <geza.herman@gmail.com>,
>  70435@debbugs.gnu.org, Eli
>  Zaretskii <eliz@gnu.org>
> Date: Mon, 29 Apr 2024 19:21:01 +0200
> 
> 
> Hello Alan,
> 
> Alan Mackenzie <acm@muc.de> writes:
> 
> > I have a fix, I think.  It is actually a two line fix, removing 
> > a test
> > from the top of a function, but it involves reindenting the 
> > entire rest
> > of the function.
> >
> > Please apply the patch below, recompile cc-engine.el, then load 
> > the
> > resulting CC Mode into a running Emacs.  Please test it on your 
> > real C++
> > code, and let me know if the bug is actually fixed.  Thanks!
> 
> I can confirm that the patch fixes the issue, I tested it in real 
> circumstances.
> 
> Thanks for fixing the problem so quickly!

Alan, should this bug be closed now?

bug#70435: 30.0.50; cc-mode: <> are sometimes not reconized as parentheses

[Alan Mackenzie]

Hello, Eli and Géza.

On Thu, May 02, 2024 at 12:30:32 +0300, Eli Zaretskii wrote:
> > From: Herman, Géza <geza.herman@gmail.com>
> > Cc: Herman Géza <geza.herman@gmail.com>,
> >  70435@debbugs.gnu.org, Eli
> >  Zaretskii <eliz@gnu.org>
> > Date: Mon, 29 Apr 2024 19:21:01 +0200


> > Hello Alan,

> > Alan Mackenzie <acm@muc.de> writes:

> > > I have a fix, I think.  It is actually a two line fix, removing 
> > > a test
> > > from the top of a function, but it involves reindenting the 
> > > entire rest
> > > of the function.

> > > Please apply the patch below, recompile cc-engine.el, then load 
> > > the
> > > resulting CC Mode into a running Emacs.  Please test it on your 
> > > real C++
> > > code, and let me know if the bug is actually fixed.  Thanks!

> > I can confirm that the patch fixes the issue, I tested it in real 
> > circumstances.

> > Thanks for fixing the problem so quickly!

> Alan, should this bug be closed now?

Apologies for my dithering.  I've been having second thoughts about the
patch in the last few days, and wondering whether I should refine it.

It's all to do with a function c-unmark-<>-around-region, which was run
in before-change-functions for a deletion, and after-change-functions for
an insertion.  The patch I sent to Géza made that function run in before-
and after-c-f both for insertions and deletions.

I've been wondering whether that is strictly necessary, since it will
slow down CC Mode a little.  I'm thinking that in the
insertion/before-c-f case, I might not need the call.

Géza, would you be prepared to do a little more testing if I modified the
patch?

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#70435: 30.0.50; cc-mode: <> are sometimes not reconized as parentheses

[Herman, Géza]

Alan Mackenzie <acm@muc.de> writes:

> Géza, would you be prepared to do a little more testing if I 
> modified the
> patch?

Sure, I'm happy to test whether a modified patch still fixes the 
issue.

bug#70435: 30.0.50; cc-mode: <> are sometimes not recognized as parentheses

[Alan Mackenzie]

Hello, Géza.

On Thu, May 02, 2024 at 14:49:57 +0200, Herman, Géza wrote:

> Alan Mackenzie <acm@muc.de> writes:

> > Géza, would you be prepared to do a little more testing if I modified
> > the patch?

> Sure, I'm happy to test whether a modified patch still fixes the issue.

Thanks, that's appreciated.

The modified patch is quite a bit shorter than the first one, but it does
a little more.  There was a subtle detail about deleting an unterminated
string, too.

Please remove the first patch from your cc-engine.el, then apply the
patch below.  Byte compile cc-engine.el, then load the resulting CC Mode
into a running Emacs (or restart Emacs).  The test case should (still) be
fixed.

Any testing you can do on real code would also be appreciated.  Please
confirm, again, that the bug is truly fixed, or tell me what still needs
to be fixed.  Thanks!

Here's the patch.  It should apply cleanly to the Emacs master:



diff -r 072940aaeb40 cc-engine.el
--- a/cc-engine.el	Sun Apr 14 07:59:01 2024 +0000
+++ b/cc-engine.el	Thu May 02 13:05:48 2024 +0000
@@ -7172,7 +7172,7 @@
   ;; FIXME!!!  This routine ignores the possibility of macros entirely.
   ;; 2010-01-29.
 
-  (when (> end beg)
+  (when (or old-len (> end beg))
     ;; Extend the region (BEG END) to deal with any complicating literals.
     (let* ((lit-search-beg (if (memq (char-before beg) '(?/ ?*))
 			       (1- beg) beg))
@@ -7246,7 +7246,8 @@
 		  (c-put-char-properties beg end 'syntax-table '(1))
 		  ;; If an open string's opener has just been neutralized,
 		  ;; do the same to the terminating LF.
-		  (when (and end-literal-end
+		  (when (and (> end beg)
+			     end-literal-end
 			     (eq (char-before end-literal-end) ?\n)
 			     (equal (c-get-char-property
 				     (1- end-literal-end) 'syntax-table)


-- 
Alan Mackenzie (Nuremberg, Germany).

bug#70435: 30.0.50; cc-mode: <> are sometimes not recognized as parentheses

[Herman, Géza]

Hi Alan,

Alan Mackenzie <acm@muc.de> writes:

> Any testing you can do on real code would also be appreciated. 
> Please
> confirm, again, that the bug is truly fixed, or tell me what 
> still needs
> to be fixed.  Thanks!

I checked the new patch, and just like the previous patch, this 
one works correctly as well.  Also, I used cc-mode for an hour for 
editing C++ code, I haven't noticed anything strange so far. I'll 
to keep using it and let you know if I find any bugs.

Thanks,
Géza

bug#70435: 30.0.50; cc-mode: <> are sometimes not recognized as parentheses

[Alan Mackenzie]

Hello, Géza.

On Thu, May 02, 2024 at 21:58:33 +0200, Herman, Géza wrote:

> Hi Alan,

> Alan Mackenzie <acm@muc.de> writes:

> > Any testing you can do on real code would also be appreciated. 
> > Please
> > confirm, again, that the bug is truly fixed, or tell me what 
> > still needs
> > to be fixed.  Thanks!

> I checked the new patch, and just like the previous patch, this 
> one works correctly as well.  Also, I used cc-mode for an hour for 
> editing C++ code, I haven't noticed anything strange so far. I'll 
> to keep using it and let you know if I find any bugs.

Many thanks again!  I've now committed the patch to the Emacs master
branch (and a few other places), and am closing the bug with this post.

> Thanks,
> Géza

-- 
Alan Mackenzie (Nuremberg, Germany).