bug#36650: 27.0.50; CC Mode: Support C++ attributes

bug#36650: 27.0.50; CC Mode: Support C++ attributes

[Óscar Fuentes]

CC Mode does not recognize C++ attributes. Just as an example, the
presence of an attribute causes wrong indentation on this example:

struct C {
  C([[maybe_unused]] int x, int y)
  : a(y)
  {}
  int a;
};

The line that begins with the colon should get another indentation
level.

Attributes are described here:

https://en.cppreference.com/w/cpp/language/attributes

In general, considering them part of the following token for indentation
purposes and fontifying with some existing or new face should be enough.


In GNU Emacs 27.0.50 (build 1, x86_64-pc-linux-gnu, X toolkit)
 of 2019-06-29 built on sky
Repository revision: 0b2841f9fb0ffd8514b0fb99c5673adad6b07abb
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.12004000
System Description: Debian GNU/Linux 10 (buster)

bug#36650: 27.0.50; CC Mode: Support C++ attributes

[Alan Mackenzie]

Hello, Óscar.

In article <mailman.1275.1563115806.2688.bug-gnu-emacs@gnu.org> you wrote:

> CC Mode does not recognize C++ attributes. Just as an example, the
> presence of an attribute causes wrong indentation on this example:

> struct C {
>   C([[maybe_unused]] int x, int y)
>   : a(y)
>   {}
>   int a;
> };

> The line that begins with the colon should get another indentation
> level.

Yes.  I'll have a look at this.

> Attributes are described here:

> https://en.cppreference.com/w/cpp/language/attributes

Thanks.

> In general, considering them part of the following token for indentation
> purposes and fontifying with some existing or new face should be enough.

I will probably end up treating attributes as syntactic whitespace.
They have no syntactic connection with the code they are embedded in,
any more than macros do.

> In GNU Emacs 27.0.50 (build 1, x86_64-pc-linux-gnu, X toolkit)
>  of 2019-06-29 built on sky
> Repository revision: 0b2841f9fb0ffd8514b0fb99c5673adad6b07abb
> Repository branch: master
> Windowing system distributor 'The X.Org Foundation', version 11.0.12004000
> System Description: Debian GNU/Linux 10 (buster)

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#36650: 27.0.50; CC Mode: Support C++ attributes

[Óscar Fuentes]

Alan Mackenzie <acm@muc.de> writes:

> Yes.  I'll have a look at this.

Thanks.

>> In general, considering them part of the following token for indentation
>> purposes and fontifying with some existing or new face should be enough.
>
> I will probably end up treating attributes as syntactic whitespace.
> They have no syntactic connection with the code they are embedded in,
> any more than macros do.

I don't know how syntactic whitespace works on CC Mode, so just in case
I'll mention that this code

int foo([[maybe_unused]] int a,
        int b);
        
should not be formatted as

int foo([[maybe_unused]] int a,
                         int b);

bug#36650: 27.0.50; CC Mode: Support C++ attributes

[Alan Mackenzie]

Hello, Óscar.

On Mon, Jul 15, 2019 at 17:17:27 +0200, Óscar Fuentes wrote:
> Alan Mackenzie <acm@muc.de> writes:

> > Yes.  I'll have a look at this.

> Thanks.

> >> In general, considering them part of the following token for indentation
> >> purposes and fontifying with some existing or new face should be enough.

> > I will probably end up treating attributes as syntactic whitespace.
> > They have no syntactic connection with the code they are embedded in,
> > any more than macros do.

The patch below (which should apply cleanly to the master branch) is a
first attempt at handling C++ attributes.

> I don't know how syntactic whitespace works on CC Mode, so just in case
> I'll mention that this code

> int foo([[maybe_unused]] int a,
>         int b);
        
> should not be formatted as

> int foo([[maybe_unused]] int a,
>                          int b);

Understood.  Happily, the code appears to do the right thing anyway,
without needing tedious fixing.  :-)

Could you please try out this patch, and let me know how well it works.

Thanks!



diff --git a/lisp/progmodes/cc-engine.el b/lisp/progmodes/cc-engine.el
index cb88fc3e58..7963ab0d43 100644
--- a/lisp/progmodes/cc-engine.el
+++ b/lisp/progmodes/cc-engine.el
@@ -1827,45 +1827,122 @@ c-remove-is-and-in-sws
 (def-edebug-spec c-remove-is-and-in-sws t)
 
 ;; The type of literal position `end' is in a `before-change-functions'
-;; function - one of `c', `c++', `pound', or nil (but NOT `string').
+;; function - one of `c', `c++', `pound', `attribute', or nil (but NOT
+;; `string').
 (defvar c-sws-lit-type nil)
-;; A cons (START . STOP) of the bounds of the comment or CPP construct
+;; A cons (START . STOP) of the bounds of the comment or CPP construct, etc.,
 ;; enclosing END, if any, else nil.
 (defvar c-sws-lit-limits nil)
 
+(defmacro c-looking-at-c++-attribute ()
+  ;; If we're in C++ Mode, and point is at the [[ introducing an attribute,
+  ;; return the position of the end of the attribute, otherwise return nil.
+  ;; The match data are NOT preserved over this macro.
+  `(and
+    (c-major-mode-is 'c++-mode)
+    (looking-at "\\[\\[")
+    (save-excursion
+      (and
+       (c-go-list-forward)
+       (eq (char-before) ?\])
+       (eq (char-before (1- (point))) ?\])
+       (point)))))
+
+;; (defmacro c-enclosing-c++-attribute ()
+(defun c-enclosing-c++-attribute ()
+  ;; If we're in C++ Mode, and point is within a correctly balanced [[ ... ]]
+  ;; attribute structure, return a cons of its starting and ending positions.
+  ;; Otherwise, return nil.  We use the c-{in,is}-sws-face text properties for
+  ;; this determination, this macro being intended only for use in the *-sws-*
+  ;; functions and macros.  The match data are NOT preserved over this macro.
+  (let (attr-end pos-is-sws)
+     (and
+      (c-major-mode-is 'c++-mode)
+      (> (point) (point-min))
+      (setq pos-is-sws
+	    (if (get-text-property (1- (point)) 'c-is-sws)
+		(1- (point))
+	      (1- (previous-single-property-change
+		   (point) 'c-is-sws nil (point-min)))))
+      (save-excursion
+	(goto-char pos-is-sws)
+	(setq attr-end (c-looking-at-c++-attribute)))
+      (> attr-end (point))
+      (cons pos-is-sws attr-end))))
+
+(defun c-slow-enclosing-c++-attribute ()
+  ;; Like `c-enclosing-c++-attribute', but does not depend on the c-i[ns]-sws
+  ;; properties being set.
+  (and
+   (c-major-mode-is 'c++-mode)
+   (save-excursion
+     (let ((paren-state (c-parse-state))
+	   cand)
+       (while
+	   (progn
+	     (setq cand
+		   (catch 'found-cand
+		     (while (cdr paren-state)
+		       (when (and (numberp (car paren-state))
+				  (numberp (cadr paren-state))
+				  (eq (car paren-state)
+				      (1+ (cadr paren-state)))
+				  (eq (char-after (car paren-state)) ?\[)
+				  (eq (char-after (cadr paren-state)) ?\[))
+			 (throw 'found-cand (cadr paren-state)))
+		       (setq paren-state (cdr paren-state)))))
+	     (and cand
+		  (not
+		   (and (c-go-list-forward cand)
+			(eq (char-before) ?\])
+			(eq (char-before (1- (point))) ?\])))))
+	 (setq paren-state (cdr paren-state)))
+       (and cand (cons cand (point)))))))
+
 (defun c-invalidate-sws-region-before (beg end)
   ;; Called from c-before-change.  BEG and END are the bounds of the change
   ;; region, the standard parameters given to all before-change-functions.
   ;;
-  ;; Note whether END is inside a comment, CPP construct, or noise macro, and
-  ;; if so note its bounds in `c-sws-lit-limits' and type in `c-sws-lit-type'.
+  ;; Note whether END is inside a comment, CPP construct, C++ attribute, or
+  ;; noise macro, and if so note its bounds in `c-sws-lit-limits' and type in
+  ;; `c-sws-lit-type'.
   (setq c-sws-lit-type nil
 	c-sws-lit-limits nil)
-  (save-excursion
-    (goto-char end)
-    (let* ((limits (c-literal-limits))
-	   (lit-type (c-literal-type limits)))
-      (cond
-       ((memq lit-type '(c c++))
-	(setq c-sws-lit-type lit-type
-	      c-sws-lit-limits limits))
-       ((c-beginning-of-macro)
-	(setq c-sws-lit-type 'pound
-	      c-sws-lit-limits (cons (point)
-				     (progn (c-end-of-macro) (point)))))
-       ((progn (skip-syntax-backward "w_")
-	       (looking-at c-noise-macro-name-re))
-	(setq c-sws-lit-type 'noise
-	      c-sws-lit-limits (cons (match-beginning 1) (match-end 1))))
-       (t))))
-  (save-excursion
-    (goto-char beg)
-    (skip-syntax-backward "w_")
-    (when (looking-at c-noise-macro-name-re)
-      (setq c-sws-lit-type 'noise)
-      (if (consp c-sws-lit-limits)
-	  (setcar c-sws-lit-limits (match-beginning 1))
-	(setq c-sws-lit-limits (cons (match-beginning 1) (match-end 1)))))))
+  (save-match-data
+    (save-excursion
+      (goto-char end)
+      (let* ((limits (c-literal-limits))
+	     (lit-type (c-literal-type limits)))
+	(cond
+	 ((memq lit-type '(c c++))
+	  (setq c-sws-lit-type lit-type
+		c-sws-lit-limits limits))
+	 ((c-beginning-of-macro)
+	  (setq c-sws-lit-type 'pound
+		c-sws-lit-limits (cons (point)
+				       (progn (c-end-of-macro) (point)))))
+	 ((eq lit-type 'string))
+	 ((setq c-sws-lit-limits (c-enclosing-c++-attribute))
+	  (setq c-sws-lit-type 'attribute))
+	 ((progn (skip-syntax-backward "w_")
+		 (looking-at c-noise-macro-name-re))
+	  (setq c-sws-lit-type 'noise
+		c-sws-lit-limits (cons (match-beginning 1) (match-end 1))))
+	 (t))))
+    (save-excursion
+      (goto-char beg)
+      (let ((attr-limits (c-enclosing-c++-attribute)))
+	(if attr-limits
+	    (if (consp c-sws-lit-limits)
+		(setcar c-sws-lit-limits (car attr-limits))
+	      (setq c-sws-lit-limits attr-limits))
+	  (skip-syntax-backward "w_")
+	  (when (looking-at c-noise-macro-name-re)
+	    (setq c-sws-lit-type 'noise)
+	    (if (consp c-sws-lit-limits)
+		(setcar c-sws-lit-limits (match-beginning 1))
+	      (setq c-sws-lit-limits (cons (match-beginning 1)
+					   (match-end 1))))))))))
 
 (defun c-invalidate-sws-region-after-del (beg end _old-len)
   ;; Text has been deleted, OLD-LEN characters of it starting from position
@@ -1940,7 +2017,7 @@ c-invalidate-sws-region-after
       (when (and (eolp) (not (eobp)))
 	(setq end (1+ (point)))))
 
-    (when (eq c-sws-lit-type 'noise)
+    (when (memq c-sws-lit-type '(noise attribute))
       (setq beg (car c-sws-lit-limits)
 	    end (cdr c-sws-lit-limits))) ; This last setting may be redundant.
 
@@ -1990,6 +2067,8 @@ c-forward-sws
     (when (or (looking-at c-syntactic-ws-start)
 	      (and c-opt-cpp-prefix
 		   (looking-at c-noise-macro-name-re))
+	      (and (c-major-mode-is 'c++-mode)
+		   (looking-at "\\[\\["))
 	      (looking-at c-doc-line-join-re))
 
       (setq rung-end-pos (min (1+ (point)) (point-max)))
@@ -2126,6 +2205,16 @@ c-forward-sws
 	      (goto-char (match-end 1))
 	      (not (eobp)))
 
+	     ((and (c-major-mode-is 'c++-mode)
+		   (looking-at "\\[\\[")
+		   (save-excursion
+		     (and (c-go-list-forward)
+			  (eq (char-before) ?\])
+			  (eq (char-before (1- (point))) ?\])
+			  (setq next-rung-pos (point))))
+		   (goto-char next-rung-pos))
+	      (not (eobp)))
+
 	     ((looking-at c-doc-line-join-re)
 	      ;; Skip over a line join in (e.g.) Pike autodoc.
 	      (goto-char (match-end 0))
@@ -2240,6 +2329,13 @@ c-backward-sws
 		      (memq (char-before) c-doc-line-join-end-ch) ; For speed.
 		      (re-search-backward doc-line-join-here
 					  (c-point 'bopl) t))
+		     (and
+		      (c-major-mode-is 'c++-mode)
+		      (eq (char-before) ?\])
+		      (eq (char-before (1- (point))) ?\])
+		      (save-excursion
+			(and (c-go-list-backward)
+			     (looking-at "\\(\\)\\[\\["))))
 		     (progn
 		       (backward-char)
 		       (or (looking-at c-syntactic-ws-end)
@@ -2250,7 +2346,7 @@ c-backward-sws
       ;; Try to find a rung position in the simple ws preceding point, so that
       ;; we can get a cache hit even if the last bit of the simple ws has
       ;; changed recently.
-      (setq simple-ws-beg (or (match-end 1) ; Noise macro
+      (setq simple-ws-beg (or (match-end 1) ; Noise macro, etc.
 			      (match-end 0))) ; c-syntactic-ws-end
       (skip-chars-backward " \t\n\r\f\v")
       (if (setq rung-is-marked (text-property-any
@@ -2388,6 +2484,16 @@ c-backward-sws
 	      (goto-char next-rung-pos)
 	      t)
 
+	     ((and (c-major-mode-is 'c++-mode)
+		   (eq (char-before) ?\])
+		   (eq (char-before (1- (point))) ?\])
+		   (save-excursion
+		     (and (c-go-list-backward)
+			  (setq next-rung-pos (point))
+			  (looking-at "\\[\\["))))
+	      (goto-char next-rung-pos)
+	      t)
+
 	     ((and
 	       (memq (char-before) c-doc-line-join-end-ch) ; For speed.
 	       (re-search-backward doc-line-join-here (c-point 'bopl) t)))))
diff --git a/lisp/progmodes/cc-mode.el b/lisp/progmodes/cc-mode.el
index 5ae7e0f09d..a5e158933b 100644
--- a/lisp/progmodes/cc-mode.el
+++ b/lisp/progmodes/cc-mode.el
@@ -2158,9 +2158,11 @@ c-fl-decl-end
   ;; count as being in a declarator (on pragmatic grounds).
   (goto-char pos)
   (let ((lit-start (c-literal-start))
-	pos1)
+	enclosing-attribute pos1)
     (unless lit-start
       (c-backward-syntactic-ws)
+      (when (setq enclosing-attribute (c-slow-enclosing-c++-attribute))
+	(goto-char (car enclosing-attribute))) ; Only happens in C++ Mode.
       (when (setq pos1 (c-on-identifier))
 	(goto-char pos1)
 	(let ((lim (save-excursion


-- 
Alan Mackenzie (Nuremberg, Germany).

bug#36650: 27.0.50; CC Mode: Support C++ attributes

[Óscar Fuentes]

Alan Mackenzie <acm@muc.de> writes:

> The patch below (which should apply cleanly to the master branch) is a
> first attempt at handling C++ attributes.

Thank you Alan. Wow, it required more changes than I would expect.

Seems that it mostly works. The only issue I've found so far is

struct S {
  S[[using : const]]()
  : data(0)
  {}
  int data;
};

Note how ": data(0)" is not indented. If the attribute is moved before the
name:

struct S {
  [[using : const]] S()
    : data(0)
  {}
  int data;
};

or after the parameter list:

struct S {
  S() [[using: const]]
    : data(0)
  {}
  int data;
};


then the indentation is correct. This only happens if the attribute is
of the form "using:" (it is irrelevant if there is space between "using"
and the colon).

bug#36650: 27.0.50; CC Mode: Support C++ attributes

[Alan Mackenzie]

Hello, Óscar.

Thanks for the rapid response!

On Sun, Jul 21, 2019 at 00:21:43 +0200, Óscar Fuentes wrote:
> Alan Mackenzie <acm@muc.de> writes:

> > The patch below (which should apply cleanly to the master branch) is
> > a first attempt at handling C++ attributes.

> Thank you Alan. Wow, it required more changes than I would expect.

These attributes can appear anywhere.  That's why.

> Seems that it mostly works. The only issue I've found so far is

> struct S {
>   S[[using : const]]()
>   : data(0)
>   {}
>   int data;
> };

> Note how ": data(0)" is not indented. If the attribute is moved before the
> name:

> struct S {
>   [[using : const]] S()
>     : data(0)
>   {}
>   int data;
> };

> or after the parameter list:

> struct S {
>   S() [[using: const]]
>     : data(0)
>   {}
>   int data;
> };


> then the indentation is correct. This only happens if the attribute is
> of the form "using:" (it is irrelevant if there is space between "using"
> and the colon).

Yes.  More precisely, it was the presence of the colon inside the
attribute which confused a low level CC Mode function
(c-crosses-statement-barrier-p).   This function scans forward for a
character which might be relevant for statement boundaries, and one of
these characters is :.  I've added handling for [[...]] into this
function, and it now seems to be working.

Would you please apply the following supplementary patch to your current
master state, try it out again, and let me know how it goes.

Thanks!



--- cc-engine.el~	2019-07-20 17:45:13.944753490 +0000
+++ cc-engine.el	2019-07-21 10:43:31.470078502 +0000
@@ -697,6 +697,21 @@
       (overlay-put (make-overlay end ol-end) 'face face))))
 
 \f
+(defmacro c-looking-at-c++-attribute ()
+  ;; If we're in C++ Mode, and point is at the [[ introducing an attribute,
+  ;; return the position of the end of the attribute, otherwise return nil.
+  ;; The match data are NOT preserved over this macro.
+  `(and
+    (c-major-mode-is 'c++-mode)
+    (looking-at "\\[\\[")
+    (save-excursion
+      (and
+       (c-go-list-forward)
+       (eq (char-before) ?\])
+       (eq (char-before (1- (point))) ?\])
+       (point)))))
+
+\f
 ;; `c-beginning-of-statement-1' and accompanying stuff.
 
 ;; KLUDGE ALERT: c-maybe-labelp is used to pass information between
@@ -1420,9 +1435,15 @@
 		      c-opt-cpp-symbol			 ; usually "#"
 		      (substring c-stmt-delim-chars 1))	 ; e.g. ";{}?:"
 	    c-stmt-delim-chars))
+	 (skip-chars
+	  (if (c-major-mode-is 'c++-mode)
+	      (concat (substring skip-chars 0 1) ; "^"
+		      "["			 ; to catch C++ attributes
+		      (substring skip-chars 1))	; e.g. "#;{}?:"
+	    skip-chars))
 	 (non-skip-list
 	  (append (substring skip-chars 1) nil)) ; e.g. (?# ?\; ?{ ?} ?? ?:)
-	 lit-range lit-start vsemi-pos)
+	 lit-range lit-start vsemi-pos attr-end)
     (save-restriction
       (widen)
       (save-excursion
@@ -1446,6 +1467,11 @@
 	     ;; In a string/comment?
 	     ((setq lit-range (c-literal-limits from))
 	      (goto-char (cdr lit-range)))
+	     ;; Skip over a C++ attribute?
+	     ((eq (char-after) ?\[)
+	      (if (setq attr-end (c-looking-at-c++-attribute))
+		  (goto-char attr-end)
+		(forward-char)))
 	     ((eq (char-after) ?:)
 	      (forward-char)
 	      (if (and (eq (char-after) ?:)
@@ -1839,21 +1865,6 @@
 ;; enclosing END, if any, else nil.
 (defvar c-sws-lit-limits nil)
 
-(defmacro c-looking-at-c++-attribute ()
-  ;; If we're in C++ Mode, and point is at the [[ introducing an attribute,
-  ;; return the position of the end of the attribute, otherwise return nil.
-  ;; The match data are NOT preserved over this macro.
-  `(and
-    (c-major-mode-is 'c++-mode)
-    (looking-at "\\[\\[")
-    (save-excursion
-      (and
-       (c-go-list-forward)
-       (eq (char-before) ?\])
-       (eq (char-before (1- (point))) ?\])
-       (point)))))
-
-;; (defmacro c-enclosing-c++-attribute ()
 (defun c-enclosing-c++-attribute ()
   ;; If we're in C++ Mode, and point is within a correctly balanced [[ ... ]]
   ;; attribute structure, return a cons of its starting and ending positions.


-- 
Alan Mackenzie (Nuremberg, Germany).

bug#36650: 27.0.50; CC Mode: Support C++ attributes

[Óscar Fuentes]

Alan Mackenzie <acm@muc.de> writes:

> Would you please apply the following supplementary patch to your current
> master state, try it out again, and let me know how it goes.

I repeated my attempts of breaking the new feature and now I was
defeated :-)

Thanks again Alan.

bug#36650: 27.0.50; CC Mode: Support C++ attributes

[Alan Mackenzie]

Hello again, Óscar.

On Sun, Jul 21, 2019 at 16:56:53 +0200, Óscar Fuentes wrote:
> Alan Mackenzie <acm@muc.de> writes:

> > Would you please apply the following supplementary patch to your current
> > master state, try it out again, and let me know how it goes.

> I repeated my attempts of breaking the new feature and now I was
> defeated :-)

Excellent!

> Thanks again Alan.

I've committed the enhancement to savannah master (warning: it's not
quite identical to the patches I sent you), and I'm closing the bug with
this post.

-- 
Alan Mackenzie (Nuremberg, Germany).