bug#70435: 30.0.50; cc-mode: <> are sometimes not reconized as parentheses

[Alan Mackenzie <acm@muc.de>]

Hello, Géza.

On Sun, Apr 28, 2024 at 18:47:47 +0200, Herman, Géza wrote:
> Hello Alan,

> Alan Mackenzie <acm@muc.de> writes:

> > You've been a little less than fully explicit, but I think you're
> > executing these commands in the *scratch* buffer.  The first two
> > lines, which are commented out in emacs-lisp-mode, are no longer
> > commented out in C++ Mode.  There is a whole line of garbage after
> > the last end of statement marker, the (double) semicolon on line 2.

> > On using ig<TAB> to insert the snippet, it is hardly surprising that
> > CC Mode's syntactic analysis gets confused.  If you first comment
> > out those first two lines (put the region around them and do C-c
> > C-c), then the inserted snippet appears to get the correct syntax on
> > its template markers.

> > I don't think there's a bug here.  If you could show ig<TAB>
> > producing the effect when typed inside a syntactically correct
> > context, things might be different.  Can you reproduce the effect in
> > correct C++ code?

> You're right, it seems that the example I provided wasn't the best
> (this issue happens with me in real code, I tried to create a minimal
> reproducible example).

> If you delete the garbage from the scratch buffer, the bug doesn't
> reproduce indeed.  But, if you run (setq font-lock-maximum-decoration
> 2) before switching to c++-mode, the issue reproduces with an empty
> scratch buffer.  I use this setting because font-lock runs much faster
> this way, and I rely on the LSP server to do the "full" highlighting.

OK, as already said, I can reproduce the bug this way.  Thanks!

> Sorry about the bad example, here are the fixed repro steps:

> Repro:
> - put the yasnippet file (included below) into
> <emacs-config-dir>/snippets/c++-mode/something
> - install yasnippet
> - start emacs, scratch buffer appears
> - delete the contents of the scratch buffer
> - M-: (setq font-lock-maximum-decoration 2)
> - M-x c++-mode
> - M-x yas-minor-mode
> - load snippets with "M-x yas-reload-all"
> - write "ig", then press TAB to "yas-expand" the snippet
> - move the cursor on the opening "<", and execute "M-x 
>   describe-char"
> - notice that it will say "syntax: . which means: punctuation"
> - if you edit the buffer (like add a space somewhere), and execute
> describe-char again, Emacs will say "syntax: > which means: open,
> matches >", so the syntax class becomes correct.

I have a fix, I think.  It is actually a two line fix, removing a test
from the top of a function, but it involves reindenting the entire rest
of the function.

Please apply the patch below, recompile cc-engine.el, then load the
resulting CC Mode into a running Emacs.  Please test it on your real C++
code, and let me know if the bug is actually fixed.  Thanks!


diff -r 072940aaeb40 cc-engine.el
--- a/cc-engine.el	Sun Apr 14 07:59:01 2024 +0000
+++ b/cc-engine.el	Mon Apr 29 15:42:05 2024 +0000
@@ -7172,153 +7172,152 @@
   ;; FIXME!!!  This routine ignores the possibility of macros entirely.
   ;; 2010-01-29.
 
-  (when (> end beg)
-    ;; Extend the region (BEG END) to deal with any complicating literals.
-    (let* ((lit-search-beg (if (memq (char-before beg) '(?/ ?*))
-			       (1- beg) beg))
-	   (lit-search-end (if (memq (char-after end) '(?/ ?*))
-			       (1+ end) end))
-	   ;; Note we can't use c-full-pp-to-literal here, since we haven't
-	   ;; yet applied syntax-table properties to ends of lines, etc.
-	   (lit-search-beg-s (c-semi-pp-to-literal lit-search-beg))
-	   (beg-literal-beg (car (cddr lit-search-beg-s)))
-	   (lit-search-end-s (c-semi-pp-to-literal lit-search-end))
-	   (end-literal-beg (car (cddr lit-search-end-s)))
-	   (beg-literal-end (c-end-of-literal lit-search-beg-s lit-search-beg))
-	   (end-literal-end (c-end-of-literal lit-search-end-s lit-search-end))
-	   new-beg new-end search-region)
-
-      ;; Determine any new end of literal resulting from the insertion/deletion.
-      (setq search-region
-	    (if (and (eq beg-literal-beg end-literal-beg)
-		     (eq beg-literal-end end-literal-end))
-		(if beg-literal-beg
-		    nil
-		  (cons beg
-			(max end
-			     (or beg-literal-end (point-min))
-			     (or end-literal-end (point-min)))))
-	      (cons (or beg-literal-beg beg)
-		    (max end
-			 (or beg-literal-end (point-min))
-			 (or end-literal-end (point-min))))))
-
-      (when search-region
-	;; If we've just inserted text, mask its syntaxes temporarily so that
-	;; they won't interfere with the undoing of the properties on the <s
-	;; and >s.
-	(c-save-buffer-state (syn-tab-settings syn-tab-value
-					       swap-open-string-ends)
-	  (unwind-protect
-	      (progn
-		(when old-len
-		  ;; Special case: If a \ has just been inserted into a
-		  ;; string, escaping or unescaping a LF, temporarily swap
-		  ;; the LF's syntax-table text property with that of the
-		  ;; former end of the open string.
-		  (goto-char end)
-		  (when (and (eq (cadr lit-search-beg-s) 'string)
-			     (not (eq beg-literal-end end-literal-end))
-			     (skip-chars-forward "\\\\")
-			     (eq (char-after) ?\n)
-			     (not (zerop (skip-chars-backward "\\\\"))))
-		    (setq swap-open-string-ends t)
-		    (if (c-get-char-property (1- beg-literal-end)
-					     'syntax-table)
-			(progn
-			  (c-clear-char-property (1- beg-literal-end)
-						 'syntax-table)
-			  (c-put-string-fence (1- end-literal-end)))
-		      (c-put-string-fence (1- beg-literal-end))
-		      (c-clear-char-property (1- end-literal-end)
-					     'syntax-table)))
-
-		  ;; Save current settings of the 'syntax-table property in
-		  ;; (BEG END), then splat these with the punctuation value.
-		  (goto-char beg)
-		  (while (setq syn-tab-value
-			       (c-search-forward-non-nil-char-property
-				'syntax-table end))
-		    (when (not (c-get-char-property (1- (point)) 'category))
-		      (push (cons (1- (point)) syn-tab-value)
-			    syn-tab-settings)))
-
-		  (c-put-char-properties beg end 'syntax-table '(1))
-		  ;; If an open string's opener has just been neutralized,
-		  ;; do the same to the terminating LF.
-		  (when (and end-literal-end
-			     (eq (char-before end-literal-end) ?\n)
-			     (equal (c-get-char-property
-				     (1- end-literal-end) 'syntax-table)
-				    '(15)))
-		    (push (cons (1- end-literal-end) '(15)) syn-tab-settings)
-		    (c-put-char-property (1- end-literal-end) 'syntax-table
-					 '(1))))
-
-		(let
-		    ((beg-lit-start (progn (goto-char beg) (c-literal-start)))
-		     beg-limit end-limit <>-pos)
-		  ;; Locate the earliest < after the barrier before the
-		  ;; changed region, which isn't already marked as a paren.
-		  (goto-char (or beg-lit-start beg))
-		  (setq beg-limit (c-determine-limit 5000))
-
-		  ;; Remove the syntax-table/category properties from each pertinent <...>
-		  ;; pair.  Firstly, the ones with the < before beg and > after beg....
-		  (goto-char (cdr search-region))
-		  (while (progn (c-syntactic-skip-backward "^;{}<" beg-limit)
-				(eq (char-before) ?<))
-		    (c-backward-token-2)
-		    (when (eq (char-after) ?<)
-		      (when (setq <>-pos (c-clear-<-pair-props-if-match-after
-					  (car search-region)))
-			(setq new-end <>-pos))
-		      (setq new-beg (point))))
-
-		  ;; ...Then the ones with < before end and > after end.
-		  (goto-char (car search-region))
-		  (setq end-limit (c-determine-+ve-limit 5000))
-		  (while (and (c-syntactic-re-search-forward "[;{}>]" end-limit 'end)
-			      (eq (char-before) ?>))
-		    (when (eq (char-before) ?>)
-		      (if (and (looking-at c->-op-cont-regexp)
-			       (not (eq (char-after) ?>)))
-			  (goto-char (match-end 0))
-			(when
-			    (and (setq <>-pos
-				       (c-clear->-pair-props-if-match-before
-					(cdr search-region)
-					(1- (point))))
-				 (or (not new-beg)
-				     (< <>-pos new-beg)))
-			  (setq new-beg <>-pos))
-			(when (or (not new-end) (> (point) new-end))
-			  (setq new-end (point))))))))
-
-	    (when old-len
-	      (c-clear-char-properties beg end 'syntax-table)
-	      (dolist (elt syn-tab-settings)
-		(if (cdr elt)
-		    (c-put-char-property (car elt) 'syntax-table (cdr elt)))))
-	    ;; Swap the '(15) syntax-table property on open string LFs back
-	    ;; again.
-	    (when swap-open-string-ends
-	      (if (c-get-char-property (1- beg-literal-end)
-				       'syntax-table)
-		  (progn
-		    (c-clear-char-property (1- beg-literal-end)
+  ;; Extend the region (BEG END) to deal with any complicating literals.
+  (let* ((lit-search-beg (if (memq (char-before beg) '(?/ ?*))
+			     (1- beg) beg))
+	 (lit-search-end (if (memq (char-after end) '(?/ ?*))
+			     (1+ end) end))
+	 ;; Note we can't use c-full-pp-to-literal here, since we haven't
+	 ;; yet applied syntax-table properties to ends of lines, etc.
+	 (lit-search-beg-s (c-semi-pp-to-literal lit-search-beg))
+	 (beg-literal-beg (car (cddr lit-search-beg-s)))
+	 (lit-search-end-s (c-semi-pp-to-literal lit-search-end))
+	 (end-literal-beg (car (cddr lit-search-end-s)))
+	 (beg-literal-end (c-end-of-literal lit-search-beg-s lit-search-beg))
+	 (end-literal-end (c-end-of-literal lit-search-end-s lit-search-end))
+	 new-beg new-end search-region)
+
+    ;; Determine any new end of literal resulting from the insertion/deletion.
+    (setq search-region
+	  (if (and (eq beg-literal-beg end-literal-beg)
+		   (eq beg-literal-end end-literal-end))
+	      (if beg-literal-beg
+		  nil
+		(cons beg
+		      (max end
+			   (or beg-literal-end (point-min))
+			   (or end-literal-end (point-min)))))
+	    (cons (or beg-literal-beg beg)
+		  (max end
+		       (or beg-literal-end (point-min))
+		       (or end-literal-end (point-min))))))
+
+    (when search-region
+      ;; If we've just inserted text, mask its syntaxes temporarily so that
+      ;; they won't interfere with the undoing of the properties on the <s
+      ;; and >s.
+      (c-save-buffer-state (syn-tab-settings syn-tab-value
+					     swap-open-string-ends)
+	(unwind-protect
+	    (progn
+	      (when old-len
+		;; Special case: If a \ has just been inserted into a
+		;; string, escaping or unescaping a LF, temporarily swap
+		;; the LF's syntax-table text property with that of the
+		;; former end of the open string.
+		(goto-char end)
+		(when (and (eq (cadr lit-search-beg-s) 'string)
+			   (not (eq beg-literal-end end-literal-end))
+			   (skip-chars-forward "\\\\")
+			   (eq (char-after) ?\n)
+			   (not (zerop (skip-chars-backward "\\\\"))))
+		  (setq swap-open-string-ends t)
+		  (if (c-get-char-property (1- beg-literal-end)
 					   'syntax-table)
-		    (c-put-string-fence (1- end-literal-end)))
-		(c-put-string-fence (1- beg-literal-end))
-		(c-clear-char-property (1- end-literal-end)
-				       'syntax-table)))))
-	  ;; Extend the fontification region, if needed.
-	  (and new-beg
-	       (< new-beg c-new-BEG)
-	       (setq c-new-BEG new-beg))
-	  (and new-end
-	       (> new-end c-new-END)
-	       (setq c-new-END new-end))))))
+		      (progn
+			(c-clear-char-property (1- beg-literal-end)
+					       'syntax-table)
+			(c-put-string-fence (1- end-literal-end)))
+		    (c-put-string-fence (1- beg-literal-end))
+		    (c-clear-char-property (1- end-literal-end)
+					   'syntax-table)))
+
+		;; Save current settings of the 'syntax-table property in
+		;; (BEG END), then splat these with the punctuation value.
+		(goto-char beg)
+		(while (setq syn-tab-value
+			     (c-search-forward-non-nil-char-property
+			      'syntax-table end))
+		  (when (not (c-get-char-property (1- (point)) 'category))
+		    (push (cons (1- (point)) syn-tab-value)
+			  syn-tab-settings)))
+
+		(c-put-char-properties beg end 'syntax-table '(1))
+		;; If an open string's opener has just been neutralized,
+		;; do the same to the terminating LF.
+		(when (and end-literal-end
+			   (eq (char-before end-literal-end) ?\n)
+			   (equal (c-get-char-property
+				   (1- end-literal-end) 'syntax-table)
+				  '(15)))
+		  (push (cons (1- end-literal-end) '(15)) syn-tab-settings)
+		  (c-put-char-property (1- end-literal-end) 'syntax-table
+				       '(1))))
+
+	      (let
+		  ((beg-lit-start (progn (goto-char beg) (c-literal-start)))
+		   beg-limit end-limit <>-pos)
+		;; Locate the earliest < after the barrier before the
+		;; changed region, which isn't already marked as a paren.
+		(goto-char (or beg-lit-start beg))
+		(setq beg-limit (c-determine-limit 5000))
+
+		;; Remove the syntax-table/category properties from each pertinent <...>
+		;; pair.  Firstly, the ones with the < before beg and > after beg....
+		(goto-char (cdr search-region))
+		(while (progn (c-syntactic-skip-backward "^;{}<" beg-limit)
+			      (eq (char-before) ?<))
+		  (c-backward-token-2)
+		  (when (eq (char-after) ?<)
+		    (when (setq <>-pos (c-clear-<-pair-props-if-match-after
+					(car search-region)))
+		      (setq new-end <>-pos))
+		    (setq new-beg (point))))
+
+		;; ...Then the ones with < before end and > after end.
+		(goto-char (car search-region))
+		(setq end-limit (c-determine-+ve-limit 5000))
+		(while (and (c-syntactic-re-search-forward "[;{}>]" end-limit 'end)
+			    (eq (char-before) ?>))
+		  (when (eq (char-before) ?>)
+		    (if (and (looking-at c->-op-cont-regexp)
+			     (not (eq (char-after) ?>)))
+			(goto-char (match-end 0))
+		      (when
+			  (and (setq <>-pos
+				     (c-clear->-pair-props-if-match-before
+				      (cdr search-region)
+				      (1- (point))))
+			       (or (not new-beg)
+				   (< <>-pos new-beg)))
+			(setq new-beg <>-pos))
+		      (when (or (not new-end) (> (point) new-end))
+			(setq new-end (point))))))))
+
+	  (when old-len
+	    (c-clear-char-properties beg end 'syntax-table)
+	    (dolist (elt syn-tab-settings)
+	      (if (cdr elt)
+		  (c-put-char-property (car elt) 'syntax-table (cdr elt)))))
+	  ;; Swap the '(15) syntax-table property on open string LFs back
+	  ;; again.
+	  (when swap-open-string-ends
+	    (if (c-get-char-property (1- beg-literal-end)
+				     'syntax-table)
+		(progn
+		  (c-clear-char-property (1- beg-literal-end)
+					 'syntax-table)
+		  (c-put-string-fence (1- end-literal-end)))
+	      (c-put-string-fence (1- beg-literal-end))
+	      (c-clear-char-property (1- end-literal-end)
+				     'syntax-table)))))
+      ;; Extend the fontification region, if needed.
+      (and new-beg
+	   (< new-beg c-new-BEG)
+	   (setq c-new-BEG new-beg))
+      (and new-end
+	   (> new-end c-new-END)
+	   (setq c-new-END new-end)))))
 
 (defun c-before-change-check-<>-operators (beg end)
   ;; When we're deleting text, unmark certain pairs of "< .... >" which are


> Geza

-- 
Alan Mackenzie (Nuremberg, Germany).