Two more CC Mode bugs fixed. For Emacs 26?

Two more CC Mode bugs fixed. For Emacs 26?

[Alan Mackenzie]

Hello, Emacs.

Thanks to John Ciolfi for reporting the following two bugs, which are
now fixed in standalone CC Mode:

1/-

In

    #include "b.hpp"//Comment

, the fontification goes haywire.  This is triggered by the lack of a
space between the second " and //.

The mechanism is an attempted optimisation in the "state cache" whereby
'category text properties are placed around the CPP construct not
including the comment.  Since there's no character between the CPP
construct and the comment, there's nowhere to put this text property, so
things go wrong.

The solution is to remove this "optimisation" entirely.  It didn't
really make much difference to CC Mode's run time anyway.

2/-

In a section of comment where doc comments should start, the doc comment
sometimes doesn't get fontified as such, e.g., when CC Mode is set up
for doxygen comments:

    // Function: foo
    /**
     *    Comment line 1 foo foo foo foo foo foo foo foo foo @b one
     *    Comment line 2 foo foo foo foo foo foo foo foo foo foo foo foo
     *    foo foo foo foo foo foo foo

, the lines from that starting with "/**" ought to get c-doc-face, but
don't.

The mechanism here is again a lack of a space.  c-font-lock-doc-comments
is trying to check whether the "/**" is itself within a comment or
string, and does so by testing the face of the character immediately
before.  This is the newline after "foo", which has comment-face, so the
function assumes that "/**" is inside a comment and fails to do the
right thing.

The solution is to check rigorously whether the "/**" is inside a
comment.  This is no problem now that processor cycles are a deal more
plentiful than when this code was originally written.

#########################################################################

I'd like to commit these two fixes to the release branch.  1/- because
it is relatively important, and 2/- because it is cheap and very low
risk.

What do people, Eli and others, say?

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: Two more CC Mode bugs fixed. For Emacs 26?

[Eli Zaretskii]

> Date: Tue, 19 Dec 2017 19:52:05 +0000
> From: Alan Mackenzie <acm@muc.de>
> Cc: John Ciolfi <John.Ciolfi@mathworks.com>
> 
> I'd like to commit these two fixes to the release branch.  1/- because
> it is relatively important, and 2/- because it is cheap and very low
> risk.
> 
> What do people, Eli and others, say?

Sounds good, but please show the patches first.

Thanks.

Re: Two more CC Mode bugs fixed. For Emacs 26?

[Alan Mackenzie]

Hello, Eli.

On Wed, Dec 20, 2017 at 17:52:08 +0200, Eli Zaretskii wrote:
> > Date: Tue, 19 Dec 2017 19:52:05 +0000
> > From: Alan Mackenzie <acm@muc.de>
> > Cc: John Ciolfi <John.Ciolfi@mathworks.com>

> > I'd like to commit these two fixes to the release branch.  1/- because
> > it is relatively important, and 2/- because it is cheap and very low
> > risk.

> > What do people, Eli and others, say?

> Sounds good, but please show the patches first.

OK, here goes.  Please don't be put off by the length of the first
patch - about half of it is just the removal of now redundant macros,
and renaming c-neutralize-syntax-in-and-mark-CPP to remove the
"and-mark-" bit.  Also, that first patch doesn't apply cleanly to the
emacs-26 branch.  Sorry about that.

#########################################################################

1/- [Problem: #include "b.hpp"//Comment misfontifies because of lack of
a space before the comment.]


Fontify a CPP construct correctly when a comment follows without spaces

Do this by removing a broken optimization in the state cache which put
category text properties on a character between the end of the CPP construct
and the beginning of the comment.  This can't work when there's no such
character.

* cc-defs.el (c-cpp-delimiter, c-set-cpp-delimiters,
* c-clear-cpp-delimiters)
(c-comment-out-cpps, c-with-cpps-commented-out)
(c-with-all-but-one-cpps-commented-out): Remove.

* cc-engine.el (c-no-comment-end-of-macro): Return the comment start position
rather than one character before it.
(c-invalidate-state-cache, c-parse-state): Remove the invocations of
c-with-all-but-one-cpps-commented-out and c-with-cpps-commented-out.

* cc-mode.el (c-neutralize-syntax-in-and-mark-CPP): Rename to
c-neutralize-syntax-in-CPP and remove the bits which applied category
properties.

* cc-langs.el (c-before-font-lock-functions): Incorporate the new name of the
function c-neutralize-syntax-in-CPP.



diff -r 0ca472c82800 cc-defs.el
--- a/cc-defs.el	Thu Dec 14 20:55:20 2017 +0000
+++ b/cc-defs.el	Sat Dec 16 18:43:22 2017 +0000
@@ -1417,59 +1417,6 @@
 
 ;;;;;;;;;;;;;;;
 
-(defconst c-cpp-delimiter '(14)) ; generic comment syntax
-;; This is the value of the `category' text property placed on every #
-;; which introduces a CPP construct and every EOL (or EOB, or character
-;; preceding //, etc.) which terminates it.  We can instantly "comment
-;; out" all CPP constructs by giving `c-cpp-delimiter' a syntax-table
-;; property '(14) (generic comment delimiter).
-(defmacro c-set-cpp-delimiters (beg end)
-  ;; This macro does a hidden buffer change.
-  `(progn
-     (c-put-char-property ,beg 'category 'c-cpp-delimiter)
-     (if (< ,end (point-max))
-	 (c-put-char-property ,end 'category 'c-cpp-delimiter))))
-(defmacro c-clear-cpp-delimiters (beg end)
-  ;; This macro does a hidden buffer change.
-  `(progn
-     (c-clear-char-property ,beg 'category)
-     (if (< ,end (point-max))
-	 (c-clear-char-property ,end 'category))))
-
-(defsubst c-comment-out-cpps ()
-  ;; Render all preprocessor constructs syntactically commented out.
-  (put 'c-cpp-delimiter 'syntax-table c-cpp-delimiter))
-(defsubst c-uncomment-out-cpps ()
-  ;; Restore the syntactic visibility of preprocessor constructs.
-  (put 'c-cpp-delimiter 'syntax-table nil))
-
-(defmacro c-with-cpps-commented-out (&rest forms)
-  ;; Execute FORMS... whilst the syntactic effect of all characters in
-  ;; all CPP regions is suppressed.  In particular, this is to suppress
-  ;; the syntactic significance of parens/braces/brackets to functions
-  ;; such as `scan-lists' and `parse-partial-sexp'.
-  `(unwind-protect
-       (c-save-buffer-state ()
-	   (c-comment-out-cpps)
-	   ,@forms)
-     (c-save-buffer-state ()
-       (c-uncomment-out-cpps))))
-
-(defmacro c-with-all-but-one-cpps-commented-out (beg end &rest forms)
-  ;; Execute FORMS... whilst the syntactic effect of all characters in
-  ;; every CPP region APART FROM THE ONE BETWEEN BEG and END is
-  ;; suppressed.
-  `(unwind-protect
-       (c-save-buffer-state ()
-	 (save-restriction
-	   (widen)
-	   (c-clear-cpp-delimiters ,beg ,end))
-	 ,`(c-with-cpps-commented-out ,@forms))
-     (c-save-buffer-state ()
-       (save-restriction
-	 (widen)
-	 (c-set-cpp-delimiters ,beg ,end)))))
-
 (defmacro c-self-bind-state-cache (&rest forms)
   ;; Bind the state cache to itself and execute the FORMS.  Return the result
   ;; of the last FORM executed.  It is assumed that no buffer changes will
diff -r 0ca472c82800 cc-engine.el
--- a/cc-engine.el	Thu Dec 14 20:55:20 2017 +0000
+++ b/cc-engine.el	Sat Dec 16 18:43:22 2017 +0000
@@ -243,8 +243,8 @@
 ;; `c-macro-cache'.
 (defvar c-macro-cache-no-comment nil)
 (make-variable-buffer-local 'c-macro-cache-no-comment)
-;; Either nil, or the last character of the macro currently represented by
-;; `c-macro-cache' which isn't in a comment. */
+;; Either nil, or the position of a comment which is open at the end of the
+;; macro represented by `c-macro-cache'.
 
 (defun c-invalidate-macro-cache (beg end)
   ;; Called from a before-change function.  If the change region is before or
@@ -387,8 +387,9 @@
     (point)))
 
 (defun c-no-comment-end-of-macro ()
-  ;; Go to the end of a CPP directive, or a pos just before which isn't in a
-  ;; comment.  For this purpose, open strings are ignored.
+  ;; Go to the start of the comment which is open at the end of the current
+  ;; CPP directive, or to the end of that directive.  For this purpose, open
+  ;; strings are ignored.
   ;;
   ;; This function must only be called from the beginning of a CPP construct.
   ;;
@@ -406,7 +407,7 @@
 	(setq s (parse-partial-sexp here there)))
       (when (and (nth 4 s)
 		 (not (eq (nth 7 s) 'syntax-table))) ; no pseudo comments.
-	(goto-char (1- (nth 8 s))))
+	(goto-char (nth 8 s)))
       (setq c-macro-cache-no-comment (point)))
     (point)))
 
@@ -3864,14 +3865,7 @@
   (if (memq 'category-properties c-emacs-features)
       ;; Emacs
       (c-with-<->-as-parens-suppressed
-       (if (and c-state-old-cpp-beg
-		(< c-state-old-cpp-beg here))
-	   (c-with-all-but-one-cpps-commented-out
-	    c-state-old-cpp-beg
-	    c-state-old-cpp-end
-	    (c-invalidate-state-cache-1 here))
-	 (c-with-cpps-commented-out
-	  (c-invalidate-state-cache-1 here))))
+       (c-invalidate-state-cache-1 here))
     ;; XEmacs
     (c-invalidate-state-cache-1 here)))
 
@@ -3905,12 +3899,7 @@
 	(if (memq 'category-properties c-emacs-features)
 	    ;; Emacs
 	    (c-with-<->-as-parens-suppressed
-	     (if (and here-cpp-beg (> here-cpp-end here-cpp-beg))
-		 (c-with-all-but-one-cpps-commented-out
-		  here-cpp-beg here-cpp-end
-		  (c-parse-state-1))
-	       (c-with-cpps-commented-out
-		(c-parse-state-1))))
+	     (c-parse-state-1))
 	  ;; XEmacs
 	  (c-parse-state-1))
       (setq c-state-old-cpp-beg
diff -r 0ca472c82800 cc-langs.el
--- a/cc-langs.el	Thu Dec 14 20:55:20 2017 +0000
+++ b/cc-langs.el	Sat Dec 16 18:43:22 2017 +0000
@@ -502,13 +502,13 @@
   (c objc) '(c-depropertize-new-text
 	     c-parse-quotes-after-change
 	     c-extend-font-lock-region-for-macros
-	     c-neutralize-syntax-in-and-mark-CPP
+	     c-neutralize-syntax-in-CPP
 	     c-change-expand-fl-region)
   c++ '(c-depropertize-new-text
 	c-parse-quotes-after-change
 	c-extend-font-lock-region-for-macros
 	c-after-change-re-mark-raw-strings
-	c-neutralize-syntax-in-and-mark-CPP
+	c-neutralize-syntax-in-CPP
 	c-restore-<>-properties
 	c-change-expand-fl-region)
   java '(c-depropertize-new-text
diff -r 0ca472c82800 cc-mode.el
--- a/cc-mode.el	Thu Dec 14 20:55:20 2017 +0000
+++ b/cc-mode.el	Sat Dec 16 18:43:22 2017 +0000
@@ -1002,15 +1002,10 @@
 	      t)
 	     (t nil)))))))
 
-(defun c-neutralize-syntax-in-and-mark-CPP (begg endd old-len)
-  ;; (i) "Neutralize" every preprocessor line wholly or partially in the
-  ;; changed region.  "Restore" lines which were CPP lines before the change
-  ;; and are no longer so.
-  ;;
-  ;; (ii) Mark each CPP construct by placing a `category' property value
-  ;; `c-cpp-delimiter' at its start and end.  The marked characters are the
-  ;; opening # and usually the terminating EOL, but sometimes the character
-  ;; before a comment delimiter.
+(defun c-neutralize-syntax-in-CPP (begg endd old-len)
+  ;; "Neutralize" every preprocessor line wholly or partially in the changed
+  ;; region.  "Restore" lines which were CPP lines before the change and are
+  ;; no longer so.
   ;;
   ;; That is, set syntax-table properties on characters that would otherwise
   ;; interact syntactically with those outside the CPP line(s).
@@ -1030,11 +1025,7 @@
   (c-save-buffer-state (limits)
     ;; Clear 'syntax-table properties "punctuation":
     ;; (c-clear-char-property-with-value c-new-BEG c-new-END 'syntax-table '(1))
-
-    ;; CPP "comment" markers:
-    (if (memq 'category-properties c-emacs-features) ; GNU Emacs.
-	(c-clear-char-property-with-value
-	 c-new-BEG c-new-END 'category 'c-cpp-delimiter))
+    ;; The above is now done in `c-depropertize-CPP'.
 
     ;; Add needed properties to each CPP construct in the region.
     (goto-char c-new-BEG)
@@ -1061,10 +1052,7 @@
 	  (goto-char (match-beginning 1))
 	  (setq mbeg (point))
 	  (if (> (c-no-comment-end-of-macro) mbeg)
-	      (progn
-		(c-neutralize-CPP-line mbeg (point)) ; "punctuation" properties
-		(if (memq 'category-properties c-emacs-features) ; GNU Emacs.
-		    (c-set-cpp-delimiters mbeg (point)))) ; "comment" markers
+	      (c-neutralize-CPP-line mbeg (point)) ; "punctuation" properties
 	    (forward-line))	      ; no infinite loop with, e.g., "#//"
 	  )))))
  
#########################################################################

2/- [Problem: doc comments don't get fontified when the doc comment
opener immediately follows a literal without intervening whitespace.]

Fix loss of documentation face in certain CC Mode doc comment situations

* cc-fonts.el (c-font-lock-doc-comments): Take into account the possibility of
font-lock-comment-delimiter-face.  Test rigorously for "/**" (etc.) being
itself inside a literal, rather than just depending on the face of the
previous character.



diff -r 0ca472c82800 cc-fonts.el
--- a/cc-fonts.el	Thu Dec 14 20:55:20 2017 +0000
+++ b/cc-fonts.el	Sun Dec 17 20:31:08 2017 +0000
@@ -2666,8 +2666,8 @@
   ;; This function might do hidden buffer changes.
 
   (let (comment-beg region-beg)
-    (if (eq (get-text-property (point) 'face)
-	    'font-lock-comment-face)
+    (if (memq (get-text-property (point) 'face)
+	      '(font-lock-comment-face font-lock-comment-delimiter-face))
 	;; Handle the case when the fontified region starts inside a
 	;; comment.
 	(let ((start (c-literal-start)))
@@ -2687,8 +2687,15 @@
 		     (or (not (c-got-face-at comment-beg
 					     c-literal-faces))
 			 (and (/= comment-beg (point-min))
+			      ;; Cheap check which is unreliable (the previous
+			      ;; character could be the end of a previous
+			      ;; comment).
 			      (c-got-face-at (1- comment-beg)
-					     c-literal-faces))))
+					     c-literal-faces)
+			      ;; Expensive reliable check.
+			      (save-excursion
+				(goto-char comment-beg)
+				(c-in-literal)))))
 	      (setq comment-beg nil))
 	    (setq region-beg comment-beg))
  
#########################################################################


> Thanks.

Thank you!

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: Two more CC Mode bugs fixed. For Emacs 26?

[Eli Zaretskii]

> Date: Wed, 20 Dec 2017 17:56:50 +0000
> Cc: emacs-devel@gnu.org, John.Ciolfi@mathworks.com
> From: Alan Mackenzie <acm@muc.de>
> 
> > Sounds good, but please show the patches first.
> 
> OK, here goes.  Please don't be put off by the length of the first
> patch - about half of it is just the removal of now redundant macros,
> and renaming c-neutralize-syntax-in-and-mark-CPP to remove the
> "and-mark-" bit.  Also, that first patch doesn't apply cleanly to the
> emacs-26 branch.  Sorry about that.

OK, please push to emacs-26, and thanks.

Re: Two more CC Mode bugs fixed. For Emacs 26?

[Alan Mackenzie]

Hello, Eli.

On Thu, Dec 21, 2017 at 18:21:23 +0200, Eli Zaretskii wrote:
> > Date: Wed, 20 Dec 2017 17:56:50 +0000
> > Cc: emacs-devel@gnu.org, John.Ciolfi@mathworks.com
> > From: Alan Mackenzie <acm@muc.de>

> > > Sounds good, but please show the patches first.

> > OK, here goes.  Please don't be put off by the length of the first
> > patch - about half of it is just the removal of now redundant macros,
> > and renaming c-neutralize-syntax-in-and-mark-CPP to remove the
> > "and-mark-" bit.  Also, that first patch doesn't apply cleanly to the
> > emacs-26 branch.  Sorry about that.

> OK, please push to emacs-26, and thanks.

Thanks.  DONE (both of them).

-- 
Alan Mackenzie (Nuremberg, Germany).