POC: customizable cc-mode keywords

POC: customizable cc-mode keywords

[Daniel Colascione]

[-- Attachment #1: Type: text/plain, Size: 18192 bytes --]

cc-mode has trouble with parsing dialects of C that use the preprocessor
heavily. Consider this example from the Linux kernel:

  static int perf_event_period(struct perf_event *event, u64 __user *arg)

__user is defined to some GCC static analysis nonsense, but since
cc-mode doesn't know that, we see __user fontified in
font-lock-variable-name-face and *arg untouched. This example is fairly
benign (if ugly), but there are other cases where variations in
pre-processor C dialect confuse cc-mode in larger regions, leading to
odd fontification and indentation.

The patch below adds customizable options for additional C-family
language "keywords".

To add this feature, we have to change how cc-mode evaluates its
language variables. Today, we use clever macros to hard-code the values
of all cc-mode language variables into the mode functions of each
cc-mode major mode function or into c-init-language-vars-for, but in
order to allow users to customize cc-mode syntax, we have to be able to
recompute language constants and variables at runtime. The new code
simply evaluates cc-mode language setter forms at mode initialization
instead. This approach is slower, but not by much: it takes 0.9ms to set
up cc-mode's ~130 language variables using the precompiled function
approach, while it takes 1.6ms to do the same work using dynamic
evaluation. I can live with this performance regression.

As implemented, the keyword list can only be customized globally, but
it'd be nice to be able to do something buffer-local too.

=== modified file 'lisp/progmodes/cc-defs.el'
--- lisp/progmodes/cc-defs.el	2014-02-09 12:34:25 +0000
+++ lisp/progmodes/cc-defs.el	2014-05-02 04:47:35 +0000
@@ -89,7 +89,7 @@
 \f
 ;;; Variables also used at compile time.

-(defconst c-version "5.32.5"
+(defconst c-version "5.32.5.1"
   "CC Mode version number.")

 (defconst c-version-sym (intern c-version))
@@ -1812,8 +1812,6 @@
 ;; and other miscellaneous data.  The obarray might also contain
 ;; various other symbols, but those don't have any variable bindings.

-(defvar c-lang-const-expansion nil)
-
 (defsubst c-get-current-file ()
   ;; Return the base name of the current file.
   (let ((file (cond
@@ -1880,19 +1878,6 @@
 constant.  A file is identified by its base name."

   (let* ((sym (intern (symbol-name name) c-lang-constants))
-	 ;; Make `c-lang-const' expand to a straightforward call to
-	 ;; `c-get-lang-constant' in `cl-macroexpand-all' below.
-	 ;;
-	 ;; (The default behavior, i.e. to expand to a call inside
-	 ;; `eval-when-compile' should be equivalent, since that macro
-	 ;; should only expand to its content if it's used inside a
-	 ;; form that's already evaluated at compile time.  It's
-	 ;; however necessary to use our cover macro
-	 ;; `cc-eval-when-compile' due to bugs in `eval-when-compile',
-	 ;; and it expands to a bulkier form that in this case only is
-	 ;; unnecessary garbage that we don't want to store in the
-	 ;; language constant source definitions.)
-	 (c-lang-const-expansion 'call)
 	 (c-langs-are-parametric t)
 	 bindings
 	 pre-files)
@@ -2037,53 +2022,28 @@
 	 "Unknown language %S since it got no `c-mode-prefix' property"
 	 (symbol-name lang))))

-    (if (eq c-lang-const-expansion 'immediate)
-	;; No need to find out the source file(s) when we evaluate
-	;; immediately since all the info is already there in the
-	;; `source' property.
-	`',(c-get-lang-constant name nil mode)
-
-      (let ((file (c-get-current-file)))
-	(if file (setq file (intern file)))
-	;; Get the source file(s) that must be loaded to get the value
-	;; of the constant.  If the symbol isn't defined yet we assume
-	;; that its definition will come later in this file, and thus
-	;; are no file dependencies needed.
-	(setq source-files (nreverse
-			    ;; Reverse to get the right load order.
-			    (apply 'nconc
-				   (mapcar (lambda (elem)
-					     (if (eq file (car elem))
-						 nil ; Exclude our own file.
-					       (list (car elem))))
-					   (get sym 'source))))))
-
-      ;; Make some effort to do a compact call to
-      ;; `c-get-lang-constant' since it will be compiled in.
-      (setq args (and mode `(',mode)))
-      (if (or source-files args)
-	  (setq args (cons (and source-files `',source-files)
-			   args)))
-
-      (if (or (eq c-lang-const-expansion 'call)
-	      (and (not c-lang-const-expansion)
-		   (not mode))
-	      load-in-progress
-	      (not (boundp 'byte-compile-dest-file))
-	      (not (stringp byte-compile-dest-file)))
-	  ;; Either a straight call is requested in the context, or
-	  ;; we're in an "uncontrolled" context and got no language,
-	  ;; or we're not being byte compiled so the compile time
-	  ;; stuff below is unnecessary.
-	  `(c-get-lang-constant ',name ,@args)
-
-	;; Being compiled.  If the loading and compiling version is
-	;; the same we use a value that is evaluated at compile time,
-	;; otherwise it's evaluated at runtime.
-	`(if (eq c-version-sym ',c-version-sym)
-	     (cc-eval-when-compile
-	       (c-get-lang-constant ',name ,@args))
-	   (c-get-lang-constant ',name ,@args))))))
+    (let ((file (c-get-current-file)))
+      (if file (setq file (intern file)))
+      ;; Get the source file(s) that must be loaded to get the value
+      ;; of the constant.  If the symbol isn't defined yet we assume
+      ;; that its definition will come later in this file, and thus
+      ;; are no file dependencies needed.
+      (setq source-files (nreverse
+                          ;; Reverse to get the right load order.
+                          (apply 'nconc
+                                 (mapcar (lambda (elem)
+                                           (if (eq file (car elem))
+                                               nil ; Exclude our own file.
+                                             (list (car elem))))
+                                         (get sym 'source))))))
+
+    ;; Make some effort to do a compact call to `c-get-lang-constant'
+    ;; and omit unneeded arguments since this code will be compiled.
+    (setq args (and mode `(',mode)))
+    (if (or source-files args)
+        (setq args (cons (and source-files `',source-files)
+                         args)))
+    `(c-get-lang-constant ',name ,@args)))

 (defvar c-lang-constants-under-evaluation nil)

@@ -2262,6 +2222,18 @@
 	     (setq buf-mode (get buf-mode 'c-fallback-mode))))
     match))

+(defun c-clear-value-cache ()
+  "Forget already-computed `c-lang-defvar' values.
+Call this function to make changes to cc-mode language
+variables take effect at the next mode initialization."
+  ;; Clear cached constant values
+  (mapatoms (lambda (sym)
+              (set sym nil))
+            c-lang-constants)
+  ;; Recompute our font lock keyword constants
+  (when (featurep 'cc-fonts)
+    (load "cc-fonts" nil t)))
+
 \f
 (cc-provide 'cc-defs)


=== modified file 'lisp/progmodes/cc-langs.el'
--- lisp/progmodes/cc-langs.el	2014-01-01 07:43:34 +0000
+++ lisp/progmodes/cc-langs.el	2014-05-02 05:19:24 +0000
@@ -1921,15 +1921,13 @@
   ;; declaration.  Specifically, they aren't recognized in the middle
   ;; of multi-token types, inside declarators, and between the
   ;; identifier and the arglist paren of a function declaration.
-  ;;
-  ;; FIXME: This ought to be user customizable since compiler stuff
-  ;; like this usually is wrapped in project specific macros.  (It'd
-  ;; of course be even better if we could cope without knowing this.)
-  t nil
-  (c c++) '(;; GCC extension.
-	    "__attribute__"
-	    ;; MSVC extension.
-	    "__declspec"))
+  t (when (boundp (c-mode-symbol "extra-keywords"))
+      (mapcar #'car (c-mode-var "extra-keywords")))
+  (c c++) (append (c-lang-const c-decl-hangon-kwds)
+                  '( ;; GCC extension.
+                    "__attribute__"
+                    ;; MSVC extension.
+                    "__declspec")))

 (c-lang-defconst c-decl-hangon-key
   ;; Adorned regexp matching `c-decl-hangon-kwds'.
@@ -2120,11 +2118,18 @@
 (c-lang-defconst c-paren-nontype-kwds
   "Keywords that may be followed by a parenthesis expression that doesn't
 contain type identifiers."
-  t       nil
-  (c c++) '(;; GCC extension.
-	    "__attribute__"
-	    ;; MSVC extension.
-	    "__declspec"))
+  t       (when (boundp (c-mode-symbol "extra-keywords"))
+            (apply 'nconc
+                   (mapcar (lambda (kw)
+                             (when (cdr kw)
+                               (list (car kw))))
+                           (c-mode-var "extra-keywords"))))
+  (c c++) (append
+           (c-lang-const c-paren-nontype-kwds)
+           '( ;; GCC extension.
+             "__attribute__"
+             ;; MSVC extension.
+             "__declspec")))

 (c-lang-defconst c-paren-type-kwds
   "Keywords that may be followed by a parenthesis expression containing
@@ -3155,115 +3160,38 @@

 ;; Make the `c-lang-setvar' variables buffer local in the current buffer.
 ;; These are typically standard emacs variables such as `comment-start'.
-(defmacro c-make-emacs-variables-local ()
-  `(progn
-     ,@(mapcar (lambda (init)
-		 `(make-local-variable ',(car init)))
-	       (cdr c-emacs-variable-inits))))
-
-(defun c-make-init-lang-vars-fun (mode)
-  "Create a function that initializes all the language dependent variables
-for the given mode.
-
-This function should be evaluated at compile time, so that the
-function it returns is byte compiled with all the evaluated results
-from the language constants.  Use the `c-init-language-vars' macro to
-accomplish that conveniently."
-
-  (if (and (not load-in-progress)
-	   (boundp 'byte-compile-dest-file)
-	   (stringp byte-compile-dest-file))
-
-      ;; No need to byte compile this lambda since the byte compiler is
-      ;; smart enough to detect the `funcall' construct in the
-      ;; `c-init-language-vars' macro below and compile it all straight
-      ;; into the function that contains `c-init-language-vars'.
-      `(lambda ()
-
-	 ;; This let sets up the context for `c-mode-var' and similar
-	 ;; that could be in the result from `cl-macroexpand-all'.
-	 (let ((c-buffer-is-cc-mode ',mode)
-	       current-var source-eval)
-	   (c-make-emacs-variables-local)
-	   (condition-case err
-
-	       (if (eq c-version-sym ',c-version-sym)
-		   (setq ,@(let ((c-buffer-is-cc-mode mode)
-				 (c-lang-const-expansion 'immediate))
-			     ;; `c-lang-const' will expand to the evaluated
-			     ;; constant immediately in `cl-macroexpand-all'
-			     ;; below.
-			      (mapcan
-			       (lambda (init)
-				 `(current-var ',(car init)
-				   ,(car init) ,(cl-macroexpand-all
-						 (elt init 1))))
-			       ;; Note: The following `append' copies the
-			       ;; first argument.  That list is small, so
-			       ;; this doesn't matter too much.
-			      (append (cdr c-emacs-variable-inits)
-				      (cdr c-lang-variable-inits)))))
-
-		 ;; This diagnostic message isn't useful for end
-		 ;; users, so it's disabled.
-		 ;;(unless (get ',mode 'c-has-warned-lang-consts)
-		 ;;  (message ,(concat "%s compiled with CC Mode %s "
-		 ;;		       "but loaded with %s - evaluating "
-		 ;;		       "language constants from source")
-		 ;;	      ',mode ,c-version c-version)
-		 ;;  (put ',mode 'c-has-warned-lang-consts t))
-
-		 (setq source-eval t)
-		 (let ((init ',(append (cdr c-emacs-variable-inits)
-				       (cdr c-lang-variable-inits))))
-		   (while init
-		     (setq current-var (caar init))
-		     (set (caar init) (eval (cadar init)))
-		     (setq init (cdr init)))))
-
-	     (error
-	      (if current-var
-		  (message "Eval error in the `c-lang-defvar' or `c-lang-setvar' for
`%s'%s: %S"
-			   current-var
-			   (if source-eval
-			       (format "\
- (fallback source eval - %s compiled with CC Mode %s but loaded with %s)"
-				       ',mode ,c-version c-version)
-			     "")
-			   err)
-		(signal (car err) (cdr err)))))))
-
-    ;; Being evaluated from source.  Always use the dynamic method to
-    ;; work well when `c-lang-defvar's in this file are reevaluated
-    ;; interactively.
-    `(lambda ()
-       (require 'cc-langs)
-       (let ((c-buffer-is-cc-mode ',mode)
-	     (init (append (cdr c-emacs-variable-inits)
-			   (cdr c-lang-variable-inits)))
-	     current-var)
-	 (c-make-emacs-variables-local)
-	 (condition-case err
-
-	     (while init
-	       (setq current-var (caar init))
-	       (set (caar init) (eval (cadar init)))
-	       (setq init (cdr init)))
-
-	   (error
-	    (if current-var
-		(message
-		 "Eval error in the `c-lang-defvar' or `c-lang-setver' for `%s'
(source eval): %S"
-		 current-var err)
-	      (signal (car err) (cdr err)))))))
-    ))
+(defun c-make-emacs-variables-local ()
+  (mapcar (lambda (init)
+            (make-local-variable (car init)))
+          (cdr c-emacs-variable-inits)))
+
+(defun c-init-language-vars-for (mode)
+  "Initialize the cc-mode language variables for MODE.
+MODE is a symbol naming the mode to initialize."
+  (let ((c-buffer-is-cc-mode mode)
+        (init (append (cdr c-emacs-variable-inits)
+                      (cdr c-lang-variable-inits)))
+        current-var)
+    (c-make-emacs-variables-local)
+    (condition-case err
+        (while init
+          (setq current-var (caar init))
+          (set (caar init) (eval (cadar init) nil))
+          (setq init (cdr init)))
+      (error
+       (if current-var
+           (message
+            "Eval error in the `c-lang-defvar' or `c-lang-setver' for
`%s' (source eval): %S"
+            current-var err)
+         (signal (car err) (cdr err)))))))

 (defmacro c-init-language-vars (mode)
   "Initialize all the language dependent variables for the given mode.
-This macro is expanded at compile time to a form tailored for the mode
-in question, so MODE must be a constant.  Therefore MODE is not
-evaluated and should not be quoted."
-  `(funcall ,(c-make-init-lang-vars-fun mode)))
+MODE is not evaluated and should not be quoted.  This macro used
+to produce an optimized initialization tailored to MODE, but that
+optimization is no longer worth it.  Use
+`c-init-language-vars-for' instead."
+  `(c-init-language-vars-for ',mode))

 \f
 (cc-provide 'cc-langs)

=== modified file 'lisp/progmodes/cc-mode.el'
--- lisp/progmodes/cc-mode.el	2014-03-04 04:03:34 +0000
+++ lisp/progmodes/cc-mode.el	2014-05-02 01:20:44 +0000
@@ -149,21 +149,6 @@
 (defun c-leave-cc-mode-mode ()
   (setq c-buffer-is-cc-mode nil))

-(defun c-init-language-vars-for (mode)
-  "Initialize the language variables for one of the language modes
-directly supported by CC Mode.  This can be used instead of the
-`c-init-language-vars' macro if the language you want to use is one of
-those, rather than a derived language defined through the language
-variable system (see \"cc-langs.el\")."
-  (cond ((eq mode 'c-mode)    (c-init-language-vars c-mode))
-	((eq mode 'c++-mode)  (c-init-language-vars c++-mode))
-	((eq mode 'objc-mode) (c-init-language-vars objc-mode))
-	((eq mode 'java-mode) (c-init-language-vars java-mode))
-	((eq mode 'idl-mode)  (c-init-language-vars idl-mode))
-	((eq mode 'pike-mode) (c-init-language-vars pike-mode))
-	((eq mode 'awk-mode)  (c-init-language-vars awk-mode))
-	(t (error "Unsupported mode %s" mode))))
-
 ;;;###autoload
 (defun c-initialize-cc-mode (&optional new-style-init)
   "Initialize CC Mode for use in the current buffer.

=== modified file 'lisp/progmodes/cc-vars.el'
--- lisp/progmodes/cc-vars.el	2014-01-01 07:43:34 +0000
+++ lisp/progmodes/cc-vars.el	2014-05-02 05:24:28 +0000
@@ -1614,6 +1614,75 @@
   :group 'c)

 \f
+
+(define-widget 'c-extra-keywords-widget 'lazy
+  "Internal CC Mode widget for the `*-extra-keywords' variables."
+  :type '(repeat
+          (cons
+           (string :tag "Keyword")
+           (boolean :tag "Parenthesized expression follows"))))
+
+(defun c-make-extra-keywords-blurb (mode1 mode2)
+  (concat "\
+*List of extra keywords to recognize in "
+          mode1 " mode.
+Each list item should be a cons (KW . PAREN).
+KW should be a string naming a single identifier.
+PAREN should be nil or t.  If t, expect the a parenthesized expression
+after KW and skip over it.
+
+Note that this variable is only consulted when the major mode is
+initialized.  If you change it later you have to reinitialize CC
+Mode by doing \\[" mode2 "].  Additionally, if you change this
+variable outside of customize, you need to call
+`c-clear-value-cache' to make your changes take effect."))
+
+(defun c-extra-keywords-setter (sym val)
+  (set-default sym val)
+  (c-clear-value-cache))
+
+(defcustom c-extra-keywords
+  nil
+  (c-make-extra-keywords-blurb "C" "c-mode")
+  :type 'c-extra-keywords-widget
+  :set 'c-extra-keywords-setter
+  :group 'c)
+
+(defcustom c++-extra-keywords
+  nil
+  (c-make-extra-keywords-blurb "C++" "c++-mode")
+  :type 'c-extra-keywords-widget
+  :set 'c-extra-keywords-setter
+  :group 'c)
+
+(defcustom objc-extra-keywords
+  nil
+  (c-make-extra-keywords-blurb "ObjC" "objc-mode")
+  :type 'c-extra-keywords-widget
+  :set 'c-extra-keywords-setter
+  :group 'c)
+
+(defcustom java-extra-keywords
+  nil
+  (c-make-extra-keywords-blurb "Java" "java-mode")
+  :type 'c-extra-keywords-widget
+  :set 'c-extra-keywords-setter
+  :group 'c)
+
+(defcustom idl-extra-keywords nil
+  nil
+  :type 'c-extra-keywords-widget
+  :set 'c-extra-keywords-setter
+  :group 'c)
+
+(defcustom pike-extra-keywords
+  nil
+  (c-make-extra-keywords-blurb "Pike" "pike-mode")
+  :type 'c-extra-keywords-widget
+  :set 'c-extra-keywords-setter
+  :group 'c)
+
+\f
 ;; Non-customizable variables, still part of the interface to CC Mode
 (defvar c-macro-with-semi-re nil
   ;; Regular expression which matches a (#define'd) symbol whose expansion



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 884 bytes --]

Re: POC: customizable cc-mode keywords

[Daniel Colascione]

[-- Attachment #1: Type: text/plain, Size: 764 bytes --]

On 05/01/2014 10:26 PM, Daniel Colascione wrote:
> cc-mode has trouble with parsing dialects of C that use the preprocessor
> heavily. Consider this example from the Linux kernel:
> 
>   static int perf_event_period(struct perf_event *event, u64 __user *arg)
> 
> __user is defined to some GCC static analysis nonsense, but since
> cc-mode doesn't know that, we see __user fontified in
> font-lock-variable-name-face and *arg untouched. This example is fairly
> benign (if ugly), but there are other cases where variations in
> pre-processor C dialect confuse cc-mode in larger regions, leading to
> odd fontification and indentation.
> 
> The patch below adds customizable options for additional C-family
> language "keywords".

Ping. CC Alan.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 884 bytes --]

Re: POC: customizable cc-mode keywords

[Alan Mackenzie]

Hi, Daniel.

On Thu, May 01, 2014 at 10:26:07PM -0700, Daniel Colascione wrote:
> cc-mode has trouble with parsing dialects of C that use the preprocessor
> heavily.

This has been true since 4004 BC, since C hackers are able to write
monstrosities using macros.  But we do our best to cope with the less
outlandish variants.

> Consider this example from the Linux kernel:

>   static int perf_event_period(struct perf_event *event, u64 __user *arg)

> __user is defined to some GCC static analysis nonsense, but since
> cc-mode doesn't know that, we see __user fontified in
> font-lock-variable-name-face and *arg untouched. This example is fairly
> benign (if ugly), but there are other cases where variations in
> pre-processor C dialect confuse cc-mode in larger regions, leading to
> odd fontification and indentation.

> The patch below adds customizable options for additional C-family
> language "keywords".

> To add this feature, we have to change how cc-mode evaluates its
> language variables.

:-)

> Today, we use clever macros to hard-code the values of all cc-mode
> language variables into the mode functions of each cc-mode major mode
> function or into c-init-language-vars-for, but in order to allow users
> to customize cc-mode syntax, we have to be able to recompute language
> constants and variables at runtime.

Do we, now?  You can imagine I've one or two reservations about this
idea.  It's far from clear that this level of customisation is a good
thing.  The current idea is that the language variables are for creating
new CC Mode languages, not as a means of customisation.

> The new code simply evaluates cc-mode language setter forms at mode
> initialization instead. This approach is slower, but not by much: it
> takes 0.9ms to set up cc-mode's ~130 language variables using the
> precompiled function approach, while it takes 1.6ms to do the same work
> using dynamic evaluation. I can live with this performance regression.

Have you considered turning the pertinent language variables into
customisable variables in cc-vars.el, along the lines of
*-font-lock-extra-types?

> As implemented, the keyword list can only be customized globally, but
> it'd be nice to be able to do something buffer-local too.

[ snipped patch for now. ]

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: POC: customizable cc-mode keywords

[Daniel Colascione]

[-- Attachment #1: Type: text/plain, Size: 3004 bytes --]

On 05/11/2014 02:13 PM, Alan Mackenzie wrote:
> Hi, Daniel.
> 
> On Thu, May 01, 2014 at 10:26:07PM -0700, Daniel Colascione wrote:
>> cc-mode has trouble with parsing dialects of C that use the preprocessor
>> heavily.
> 
> This has been true since 4004 BC, since C hackers are able to write
> monstrosities using macros.  But we do our best to cope with the less
> outlandish variants.

We have to address the syntactic confusion these macros cause somehow,
and I don't think automatic heuristics will be sufficient. I'm really
sick of seeing a lot of code I look at in the real world misfontified.

>> Consider this example from the Linux kernel:
> 
>>   static int perf_event_period(struct perf_event *event, u64 __user *arg)
> 
>> __user is defined to some GCC static analysis nonsense, but since
>> cc-mode doesn't know that, we see __user fontified in
>> font-lock-variable-name-face and *arg untouched. This example is fairly
>> benign (if ugly), but there are other cases where variations in
>> pre-processor C dialect confuse cc-mode in larger regions, leading to
>> odd fontification and indentation.
> 
>> The patch below adds customizable options for additional C-family
>> language "keywords".
> 
>> To add this feature, we have to change how cc-mode evaluates its
>> language variables.
> 
> :-)
> 
>> Today, we use clever macros to hard-code the values of all cc-mode
>> language variables into the mode functions of each cc-mode major mode
>> function or into c-init-language-vars-for, but in order to allow users
>> to customize cc-mode syntax, we have to be able to recompute language
>> constants and variables at runtime.
> 
> Do we, now?  You can imagine I've one or two reservations about this
> idea. 

What's your alternative?

> It's far from clear that this level of customisation is a good
> thing.  The current idea is that the language variables are for creating
> new CC Mode languages, not as a means of customisation.

They're public either way.

>> The new code simply evaluates cc-mode language setter forms at mode
>> initialization instead. This approach is slower, but not by much: it
>> takes 0.9ms to set up cc-mode's ~130 language variables using the
>> precompiled function approach, while it takes 1.6ms to do the same work
>> using dynamic evaluation. I can live with this performance regression.
> 
> Have you considered turning the pertinent language variables into
> customisable variables in cc-vars.el, along the lines of
> *-font-lock-extra-types?

The patch includes user-customizable variables. The actual cc-lang
constants aren't advertised as user customization points. The existing
type customization is simple because it's done in isolation from the
rest of cc-mode's syntactic analysis. Actually understanding keywords
requires deeper, cascading changes to core language constants. The only
practical way of making that happen is to allow these constants to
change at runtime.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 884 bytes --]

Re: POC: customizable cc-mode keywords

[Alan Mackenzie]

Hi, Daniel.

On Sun, May 11, 2014 at 02:23:15PM -0700, Daniel Colascione wrote:
> On 05/11/2014 02:13 PM, Alan Mackenzie wrote:
> > Hi, Daniel.

> > On Thu, May 01, 2014 at 10:26:07PM -0700, Daniel Colascione wrote:
> >> cc-mode has trouble with parsing dialects of C that use the preprocessor
> >> heavily.

> > This has been true since 4004 BC, since C hackers are able to write
> > monstrosities using macros.  But we do our best to cope with the less
> > outlandish variants.

> We have to address the syntactic confusion these macros cause somehow,
> and I don't think automatic heuristics will be sufficient. I'm really
> sick of seeing a lot of code I look at in the real world misfontified.

OK, so we need to have configurable "noise macros", just as we already
have configurable "macros ending with a semicolon" and "extra types".

[ .... ]

> >> The patch below adds customizable options for additional C-family
> >> language "keywords".

> >> To add this feature, we have to change how cc-mode evaluates its
> >> language variables.

> > :-)

> >> Today, we use clever macros to hard-code the values of all cc-mode
> >> language variables into the mode functions of each cc-mode major mode
> >> function or into c-init-language-vars-for, but in order to allow users
> >> to customize cc-mode syntax, we have to be able to recompute language
> >> constants and variables at runtime.

> > Do we, now?  You can imagine I've one or two reservations about this
> > idea. 

> What's your alternative?

Turning the pertinent c-lang-defvars, and only these, into configurable
variables in cc-vars.el.

[ .... ]

> >> The new code simply evaluates cc-mode language setter forms at mode
> >> initialization instead. This approach is slower, but not by much: it
> >> takes 0.9ms to set up cc-mode's ~130 language variables using the
> >> precompiled function approach, while it takes 1.6ms to do the same work
> >> using dynamic evaluation. I can live with this performance regression.

> > Have you considered turning the pertinent language variables into
> > customisable variables in cc-vars.el, along the lines of
> > *-font-lock-extra-types?

> The patch includes user-customizable variables. The actual cc-lang
> constants aren't advertised as user customization points. The existing
> type customization is simple because it's done in isolation from the
> rest of cc-mode's syntactic analysis. Actually understanding keywords
> requires deeper, cascading changes to core language constants. The only
> practical way of making that happen is to allow these constants to
> change at runtime.

I'm sorry, but I can't make much of that paragraph; it doesn't seem
coherent to me.  What I still don't understand is why it should be
necessary to make _all_ c-lang-defvars configurable variables, rather
than just those one or two directly involved in parsing these awkward C
(etc.) declarations.

As I said, I'm not at all happy at making such a massive change to CC
Mode's architecture.  There would surely be unforeseen consequences, some
of which might well be negative.

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: POC: customizable cc-mode keywords

[Daniel Colascione]

[-- Attachment #1: Type: text/plain, Size: 1851 bytes --]

On 05/16/2014 10:52 AM, Alan Mackenzie wrote:
>>>> Today, we use clever macros to hard-code the values of all cc-mode
>>>> language variables into the mode functions of each cc-mode major mode
>>>> function or into c-init-language-vars-for, but in order to allow users
>>>> to customize cc-mode syntax, we have to be able to recompute language
>>>> constants and variables at runtime.
> 
>>> Do we, now?  You can imagine I've one or two reservations about this
>>> idea. 
> 
>> What's your alternative?
> 
> Turning the pertinent c-lang-defvars, and only these, into configurable
> variables in cc-vars.el.

Have you looked into what task actually involves? You need to modify
c-decl-hangon-kwds, and as a result, c-prefix-spec-kwds,
c-postfix-spec-kwds, c-keywords, c-keywords-regexp, c-keywords-obarray,
c-nonlabel-token-key, and c-regular-keywords-regexp. It's easier and
more flexible to simply allow the entire set of c-lang-defconst values
to change. You also have to recompute the font-lock matchers, which is
similarly involved.

You claim that there is a great risk of negative side effects resulting
from this change. I don't agree, and I don't see anyone else proposing,
much less implementing, a solution to this problem, and cc-mode's being
developed outside the tree makes it frustratingly slow and difficult to
get much-needed fixes into core where users can see them.

> As I said, I'm not at all happy at making such a massive change to CC
> Mode's architecture.  There would surely be unforeseen consequences, some
> of which might well be negative.

It's not a massive change in architecture. cc-mode can already evaluate
these variables at runtime in the case of a version mismatch. I propose
simply doing so all the time. My timing indicates that it's not a
performance problem in practice.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 884 bytes --]

Re: POC: customizable cc-mode keywords

[Alan Mackenzie]

Hello, Daniel.

On Fri, May 16, 2014 at 11:06:24AM -0700, Daniel Colascione wrote:
> On 05/16/2014 10:52 AM, Alan Mackenzie wrote:
> >>>> Today, we use clever macros to hard-code the values of all cc-mode
> >>>> language variables into the mode functions of each cc-mode major mode
> >>>> function or into c-init-language-vars-for, but in order to allow users
> >>>> to customize cc-mode syntax, we have to be able to recompute language
> >>>> constants and variables at runtime.

> >>> Do we, now?  You can imagine I've one or two reservations about this
> >>> idea. 

> >> What's your alternative?

> > Turning the pertinent c-lang-defvars, and only these, into configurable
> > variables in cc-vars.el.

> Have you looked into what task actually involves?

No I hadn't.  

> You need to modify c-decl-hangon-kwds, and as a result,
> c-prefix-spec-kwds, c-postfix-spec-kwds, c-keywords,
> c-keywords-regexp, c-keywords-obarray, c-nonlabel-token-key, and
> c-regular-keywords-regexp.

OK.  Then another alternative is to use `or' forms in the appropriate
places, of which there are several rather than many, the extra arm of
the `or' looking something like "(looking-at c-noise-macro-re)".  Did you
consider this at all?  If so, why do you prefer the solution you now
propose?

> It's easier and more flexible to simply allow the entire set of
> c-lang-defconst values to change.

It's the flexibility that worries me.  Flexibility means complexity (of
which there's already too much in CC Mode), and might open up a larger
"attack surface" for future bugs.

I've spent a lot of time trying to pin down exactly what your proposed
change does and how it does it, but amn't there yet.  I only have the
before and after versions, not a detailed description of the change.

> You also have to recompute the font-lock matchers, which is similarly
> involved.

Yes.

> You claim that there is a great risk of negative side effects resulting
> from this change.

No.  I've said there is a risk, without quantifying it.

One question which you haven't addressed yet is does the changed code
work properly on modes derived from a CC Mode mode?  How easy is it to
create a `foo-extra-keywords' for such a mode?

Another is does the change work properly under XEmacs?

What is the purpose of {java,idl,pike}-extra-keywords, given that these
three languages don't have #defines?

> I don't agree, and I don't see anyone else proposing, much less
> implementing, a solution to this problem, .....

:-)  This problem was first highlighted as such, by you, this month, not
several years ago.  Your patch is complicated, it's disruptive, yet you
seem put out that I want to discuss it and understand it, rather than
just blindly applying it as it is.  It seems your use of "You need to"
and "we have to" weren't mere hyperbole as I first thought.  Forgive me,
but I'm always suspicious when somebody tells me "there's no
alternative".  There are always alternatives.  The question is why is
your patch, which is not a straightforward or obvious change, the best
solution to the problem.  What are the tradeoffs?

> .... and cc-mode's being developed outside the tree makes it
> frustratingly slow and difficult to get much-needed fixes into core
> where users can see them.

Would it really be much faster if CC Mode were integrated?  Not really.
What makes things slow is the small number of people working on CC Mode,
i.e. one, with occasional help from people like yourself.

> > As I said, I'm not at all happy at making such a massive change to CC
> > Mode's architecture.  There would surely be unforeseen consequences, some
> > of which might well be negative.

> It's not a massive change in architecture.

As I said, I don't fully understand the change, yet.  But moving
calculations from compile time to run time _is_ a change of
architecture, and it seems to me not to be small.  It seems to me
that the rigid separation between language (e.g. C) and user
configuration is being broken down.  How much easier will it be for a
user mistakenly to attempt to configure her CC Mode by messing around
with the c-lang-consts?

> cc-mode can already evaluate these variables at runtime in the case of
> a version mismatch.

That's a feature intended purely for CC Mode developers.

> I propose simply doing so all the time. My timing indicates that it's
> not a performance problem in practice.

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: POC: customizable cc-mode keywords

[Daniel Colascione]

[-- Attachment #1: Type: text/plain, Size: 7711 bytes --]

On 05/18/2014 02:33 PM, Alan Mackenzie wrote:
> On Fri, May 16, 2014 at 11:06:24AM -0700, Daniel Colascione wrote:
>> On 05/16/2014 10:52 AM, Alan Mackenzie wrote:
>>>>>> Today, we use clever macros to hard-code the values of all cc-mode
>>>>>> language variables into the mode functions of each cc-mode major mode
>>>>>> function or into c-init-language-vars-for, but in order to allow users
>>>>>> to customize cc-mode syntax, we have to be able to recompute language
>>>>>> constants and variables at runtime.
> 
>>>>> Do we, now?  You can imagine I've one or two reservations about this
>>>>> idea. 
> 
>>>> What's your alternative?
> 
>>> Turning the pertinent c-lang-defvars, and only these, into configurable
>>> variables in cc-vars.el.
> 
>> Have you looked into what task actually involves?
> 
> No I hadn't.  
> 
>> You need to modify c-decl-hangon-kwds, and as a result,
>> c-prefix-spec-kwds, c-postfix-spec-kwds, c-keywords,
>> c-keywords-regexp, c-keywords-obarray, c-nonlabel-token-key, and
>> c-regular-keywords-regexp.
> 
> OK.  Then another alternative is to use `or' forms in the appropriate
> places, of which there are several rather than many, the extra arm of
> the `or' looking something like "(looking-at c-noise-macro-re)".  Did you
> consider this at all?  If so, why do you prefer the solution you now
> propose?

Doing it that way further increases the amount of work needed at parse
time, and cc-mode isn't a speed demon already. Your proposal also
introduces much-increased for complexity in the core and creates a risk
of "missing a spot" and introducing a bug where we check the primary
keywords but not the user-customized ones. (And what about the checking
we do after calling intern-soft on the symbol obarray?) cc-engine is
very difficult to follow as it is. I'd rather gracefully extend the
existing functionality than add yet another set of special cases.

Another approach I was considering was to change cc-mode to always skip
over regions marked with a certain text properly. Then a minor mode
could independently parse and fontify the buffer, marking regions it
wanted cc-mode to ignore with this property. This approach is more
flexible, but I don't think the complexity is worth it when a simple
extension to cc-mode is within reach.

>> It's easier and more flexible to simply allow the entire set of
>> c-lang-defconst values to change.
> 
> It's the flexibility that worries me.  Flexibility means complexity (of
> which there's already too much in CC Mode), and might open up a larger
> "attack surface" for future bugs.
> 
> I've spent a lot of time trying to pin down exactly what your proposed
> change does and how it does it, but amn't there yet.  I only have the
> before and after versions, not a detailed description of the change.

I'd hoped my initial message clearly explained the overall approach.
Briefly, we want to allow users to tell cc-mode to ignore "noise macros"
when parsing. IMHO, the best way to do that is to let user
customizations augment the existing set of "noise macro" keywords
cc-mode defines for each language. Today, cc-mode builds various data
structures from these keyword sets at compile time. In order for user
customizations to have any effect, we have to build these data
structures at runtime instead. Doing that has no perceivable effect on
mode initialization performance and lets user customizations have an effect.

>> You claim that there is a great risk of negative side effects resulting
>> from this change.
> 
> No.  I've said there is a risk, without quantifying it.
> 
> One question which you haven't addressed yet is does the changed code
> work properly on modes derived from a CC Mode mode?  How easy is it to
> create a `foo-extra-keywords' for such a mode?

Trivial --- just include the variable in the c-lang-defconst calculation
for the derived mode.

> Another is does the change work properly under XEmacs?

I don't see why it wouldn't.

> What is the purpose of {java,idl,pike}-extra-keywords, given that these
> three languages don't have #defines?

Consistency with the type defcustoms? Accommodations for weird custom
preprocessors? Why not?

>> I don't agree, and I don't see anyone else proposing, much less
>> implementing, a solution to this problem, .....
> 
> :-)  This problem was first highlighted as such, by you, this month, not
> several years ago.

There's been a TODO in the code to this effect since 2005 in the
definition of c-decl-hangon-kwds; my patch removes this TODO.

> Your patch is complicated, it's disruptive, yet you
> seem put out that I want to discuss it and understand it, rather than
> just blindly applying it as it is.  It seems your use of "You need to"
> and "we have to" weren't mere hyperbole as I first thought.  Forgive me,
> but I'm always suspicious when somebody tells me "there's no
> alternative".  There are always alternatives.  The question is why is
> your patch, which is not a straightforward or obvious change, the best
> solution to the problem.  What are the tradeoffs?

I discussed some of them above. The chief risk is that users might
customize cc-mode by altering its language constants. I strongly
discount this risk, especially since users can *already* do that by
creating derived modes.

>> .... and cc-mode's being developed outside the tree makes it
>> frustratingly slow and difficult to get much-needed fixes into core
>> where users can see them.
> 
> Would it really be much faster if CC Mode were integrated?  Not really.
> What makes things slow is the small number of people working on CC Mode,
> i.e. one, with occasional help from people like yourself.
> 
>>> As I said, I'm not at all happy at making such a massive change to CC
>>> Mode's architecture.  There would surely be unforeseen consequences, some
>>> of which might well be negative.
> 
>> It's not a massive change in architecture.
> 
> As I said, I don't fully understand the change, yet.  But moving
> calculations from compile time to run time _is_ a change of
> architecture, and it seems to me not to be small.

All the cc-defvar variables get the same values they did before, by
default. cc-engine parses text the same way it always has. All the
cc-mode customization remains. My patch merely lets users customize some
existing cc-mode constants, then makes these customizations have runtime
effect by removing some unnecessary old optimizations in cc-mode
initialization. If you call that a massive change in architecture, then
what I call architecture might as well be plate tectonics.

> It seems to me
> that the rigid separation between language (e.g. C) and user
> configuration is being broken down.  How much easier will it be for a
> user mistakenly to attempt to configure her CC Mode by messing around
> with the c-lang-consts?

Much easier --- but it's never been a priority in Emacs to prevent users
from hanging themselves. Users should be drawn to the customization
settings we provide, and if they choose to bypass them, they probably
know well enough what they're doing. As I mentioned, determined users
can already derived from cc-mode built-in modes.

>> cc-mode can already evaluate these variables at runtime in the case of
>> a version mismatch.
> 
> That's a feature intended purely for CC Mode developers.

Or for people who upgrade without recompiling derived modes. But it
doesn't matter --- the dynamic-computation code is there and works fine.
My change just uses it all the time instead of going down a different
path involving dubious compile-time optimizations.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 884 bytes --]

Re: POC: customizable cc-mode keywords

[Stefan Monnier]

[ Commenting from the side lines.  ]

Daniel, you sell your patch very convincingly.

I don't know the technical details well enough to judge, but I find
CC-mode's compilation fiendishly complex (E.g. a lot of the
cc-bytecomp.el tricks seem to be there so as to allow manual calls to
the byte-compiler access to some data/functions which would otherwise
only be defined later during runtime, IIUC), so I'm wondering if your
change might rather help or hurt in this respect.


        Stefan

Re: POC: customizable cc-mode keywords

[Alan Mackenzie]

Hi, Daniel.

Thanks for the detailed response.  In general, you're gradually
persuading me of your new approach.

On Sun, May 18, 2014 at 03:28:50PM -0700, Daniel Colascione wrote:
> On 05/18/2014 02:33 PM, Alan Mackenzie wrote:
> > On Fri, May 16, 2014 at 11:06:24AM -0700, Daniel Colascione wrote:
> >> On 05/16/2014 10:52 AM, Alan Mackenzie wrote:
> >>>>>> Today, we use clever macros to hard-code the values of all cc-mode
> >>>>>> language variables into the mode functions of each cc-mode major mode
> >>>>>> function or into c-init-language-vars-for, but in order to allow users
> >>>>>> to customize cc-mode syntax, we have to be able to recompute language
> >>>>>> constants and variables at runtime.

> >>>>> Do we, now?  You can imagine I've one or two reservations about this
> >>>>> idea. 

> >>>> What's your alternative?

> >>> Turning the pertinent c-lang-defvars, and only these, into configurable
> >>> variables in cc-vars.el.

> >> Have you looked into what task actually involves?

> > No I hadn't.  

> >> You need to modify c-decl-hangon-kwds, and as a result,
> >> c-prefix-spec-kwds, c-postfix-spec-kwds, c-keywords,
> >> c-keywords-regexp, c-keywords-obarray, c-nonlabel-token-key, and
> >> c-regular-keywords-regexp.

Thinking about it, changing the minimum number of c-lang-consts/vars to
the new scheme would also be a feasible strategy.  It would reduce the
opportunity of semi-experienced users to shoot themselves in the foot (as
we're discussing lower down).

> > OK.  Then another alternative is to use `or' forms in the appropriate
> > places, of which there are several rather than many, the extra arm of
> > the `or' looking something like "(looking-at c-noise-macro-re)".  Did you
> > consider this at all?  If so, why do you prefer the solution you now
> > propose?

> Doing it that way further increases the amount of work needed at parse
> time, and cc-mode isn't a speed demon already. Your proposal also
> introduces much-increased for complexity in the core and creates a risk
> of "missing a spot" and introducing a bug where we check the primary
> keywords but not the user-customized ones. (And what about the checking
> we do after calling intern-soft on the symbol obarray?) cc-engine is
> very difficult to follow as it is. I'd rather gracefully extend the
> existing functionality than add yet another set of special cases.

Yes, some slight loss of processing speed, and a little extra complexity.

> Another approach I was considering was to change cc-mode to always skip
> over regions marked with a certain text properly. Then a minor mode
> could independently parse and fontify the buffer, marking regions it
> wanted cc-mode to ignore with this property. This approach is more
> flexible, but I don't think the complexity is worth it when a simple
> extension to cc-mode is within reach.

I agree with you here.

[ .... ]

> > I've spent a lot of time trying to pin down exactly what your proposed
> > change does and how it does it, but amn't there yet.  I only have the
> > before and after versions, not a detailed description of the change.

> I'd hoped my initial message clearly explained the overall approach.
> Briefly, we want to allow users to tell cc-mode to ignore "noise macros"
> when parsing. IMHO, the best way to do that is to let user
> customizations augment the existing set of "noise macro" keywords
> cc-mode defines for each language. Today, cc-mode builds various data
> structures from these keyword sets at compile time. In order for user
> customizations to have any effect, we have to build these data
> structures at runtime instead. Doing that has no perceivable effect on
> mode initialization performance and lets user customizations have an effect.

I've got the overall strategy and motivation.  What I don't have is a
detailed description of the change - what defuns/defvars have been
changed, the nitty-gritty purpose of each change, the detailed
functionalities which have been removed/added, things like that.  In
short, what would go in a very detailed ChangeLog entry.  Diff files tend
to approach the "write-only" nature, and it's difficult to reconstruct
somebody else's thought patterns from only this diff file.

[ .... ]

> > One question which you haven't addressed yet is does the changed code
> > work properly on modes derived from a CC Mode mode?

Meaning, "Somebody" (tm) will need to test this.

> > How easy is it to create a `foo-extra-keywords' for such a mode?

> Trivial --- just include the variable in the c-lang-defconst calculation
> for the derived mode.

I don't follow the semantics here.  Hopefully maintainers of derived
modes won't feel the need to add extra c-lang-consts, since c-lang-consts
are meant to be defined for every language.  But answering my own
question, it should be easy enough to define `foo-extra-keywords' using
the same mechanism as for c-extra-keywords, since it will then be picked
up by the (c-mode-symbol "extra-keywords") in cc-langs.el.

> > Another is does the change work properly under XEmacs?

> I don't see why it wouldn't.

Well, it might be using, say, customize-... functions which don't (yet)
exist in XEmacs.  "Somebody" (tm) will need to test this too.

> > What is the purpose of {java,idl,pike}-extra-keywords, given that these
> > three languages don't have #defines?

> Consistency with the type defcustoms? Accommodations for weird custom
> preprocessors? Why not?

Why not, indeed?  Just there isn't one for AWK.  ;-)

> >> I don't agree, and I don't see anyone else proposing, much less
> >> implementing, a solution to this problem, .....

> > :-)  This problem was first highlighted as such, by you, this month, not
> > several years ago.

> There's been a TODO in the code to this effect since 2005 in the
> definition of c-decl-hangon-kwds; my patch removes this TODO.

Yes, OK.  But nobody's raised a fuss about it up till now.

> > Your patch is complicated, it's disruptive, yet you
> > seem put out that I want to discuss it and understand it, rather than
> > just blindly applying it as it is.  It seems your use of "You need to"
> > and "we have to" weren't mere hyperbole as I first thought.  Forgive me,
> > but I'm always suspicious when somebody tells me "there's no
> > alternative".  There are always alternatives.  The question is why is
> > your patch, which is not a straightforward or obvious change, the best
> > solution to the problem.  What are the tradeoffs?

> I discussed some of them above. The chief risk is that users might
> customize cc-mode by altering its language constants. I strongly
> discount this risk, especially since users can *already* do that by
> creating derived modes.

No, these activities are conceptually different.  Creating a derived mode
is clearly hacking, as contrasted with configuring.  The CC Mode language
constants are like defuns of typical major modes, not their configuration
variables.  It should somehow be clear to people changing the
c-lang-consts that they are actually hacking and not configuring.

[ .... ]

> >>> As I said, I'm not at all happy at making such a massive change to
> >>> CC Mode's architecture.  There would surely be unforeseen
> >>> consequences, some of which might well be negative.

> >> It's not a massive change in architecture.

> > As I said, I don't fully understand the change, yet.  But moving
> > calculations from compile time to run time _is_ a change of
> > architecture, and it seems to me not to be small.

> All the cc-defvar variables get the same values they did before, by
> default. cc-engine parses text the same way it always has. All the
> cc-mode customization remains. My patch merely lets users customize some
> existing cc-mode constants, then makes these customizations have runtime
> effect by removing some unnecessary old optimizations in cc-mode
> initialization. If you call that a massive change in architecture, then
> what I call architecture might as well be plate tectonics.

We've now reached the stage of arguing about words.  Let's stop!

> > It seems to me that the rigid separation between language (e.g. C)
> > and user configuration is being broken down.  How much easier will it
> > be for a user mistakenly to attempt to configure her CC Mode by
> > messing around with the c-lang-consts?

> Much easier --- but it's never been a priority in Emacs to prevent users
> from hanging themselves.

Yes, but who's going to have to bring the corpse down from the scaffold
and bury it?  By what mechanism will the said user come to understand
that c-lang-consts are not configuration variables?

> Users should be drawn to the customization settings we provide, and if
> they choose to bypass them, they probably know well enough what they're
> doing.

I've no problem with that if they're making an informed choice.  But even
in the current CC Mode mechanisms, I've occasionally been asked about
configuring CC Mode via c-lang-consts.  It shouldn't be too easy.

> As I mentioned, determined users can already derive from cc-mode
> built-in modes.

Yes.  But this is an intended facility, and isn't something that anyone
can stumble into by accident.

> >> cc-mode can already evaluate these variables at runtime in the case of
> >> a version mismatch.

> > That's a feature intended purely for CC Mode developers.

> Or for people who upgrade without recompiling derived modes. But it
> doesn't matter --- the dynamic-computation code is there and works fine.
> My change just uses it all the time instead of going down a different
> path involving dubious compile-time optimizations.

One other point.  Would it be possible to call foo-extra-keywords
something else?  What we're dealing with _aren't_ keywords like "if" or
"class", they're macros which expand to nothing.  I know that things like
"__attribute__" are currently called keywords, but extending the metaphor
to things like "__user" seems to be pushing things too far.  How about
"foo-noise-macros"?

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: POC: customizable cc-mode keywords

[Stefan Monnier]

> :-)  This problem was first highlighted as such, by you, this month, not
> several years ago.  Your patch is complicated, it's disruptive, yet you
> seem put out that I want to discuss it and understand it, rather than
> just blindly applying it as it is.

FWIW, his patch is not that complicated.  It has 2 parts:
- get rid of some of the compile-time precomputation of settings.
- add a new (set of) custom vars and use them where appropriate.

The second part is very straightforward (quoted below, for reference).
The first part mostly removes code, and more specifically removes
complex code, so it actually makes the code simpler.

I think the performance impact is higher than what Daniel claims
(his machine is clearly faster than mine), but I suspect that this can
be fixed.

>> It's not a massive change in architecture.
> As I said, I don't fully understand the change, yet.

Take another look at it, below.

> But moving calculations from compile time to run time _is_ a change of
> architecture, and it seems to me not to be small.

Right, this one is larger, but if it can be made without significant
performance degradation, then it's a good change.

> It seems to me that the rigid separation between language (e.g. C) and
> user configuration is being broken down.  How much easier will it be
> for a user mistakenly to attempt to configure her CC Mode by messing
> around with the c-lang-consts?

I'm not too worried ;-)


        Stefan


=== modified file 'lisp/progmodes/cc-langs.el'
--- lisp/progmodes/cc-langs.el	2014-01-01 07:43:34 +0000
+++ lisp/progmodes/cc-langs.el	2014-05-02 05:19:24 +0000
@@ -1921,15 +1921,13 @@
   ;; declaration.  Specifically, they aren't recognized in the middle
   ;; of multi-token types, inside declarators, and between the
   ;; identifier and the arglist paren of a function declaration.
-  ;;
-  ;; FIXME: This ought to be user customizable since compiler stuff
-  ;; like this usually is wrapped in project specific macros.  (It'd
-  ;; of course be even better if we could cope without knowing this.)
-  t nil
-  (c c++) '(;; GCC extension.
-	    "__attribute__"
-	    ;; MSVC extension.
-	    "__declspec"))
+  t (when (boundp (c-mode-symbol "extra-keywords"))
+      (mapcar #'car (c-mode-var "extra-keywords")))
+  (c c++) (append (c-lang-const c-decl-hangon-kwds)
+                  '( ;; GCC extension.
+                    "__attribute__"
+                    ;; MSVC extension.
+                    "__declspec")))

 (c-lang-defconst c-decl-hangon-key
   ;; Adorned regexp matching `c-decl-hangon-kwds'.
@@ -2120,11 +2118,18 @@
 (c-lang-defconst c-paren-nontype-kwds
   "Keywords that may be followed by a parenthesis expression that doesn't
 contain type identifiers."
-  t       nil
-  (c c++) '(;; GCC extension.
-	    "__attribute__"
-	    ;; MSVC extension.
-	    "__declspec"))
+  t       (when (boundp (c-mode-symbol "extra-keywords"))
+            (apply 'nconc
+                   (mapcar (lambda (kw)
+                             (when (cdr kw)
+                               (list (car kw))))
+                           (c-mode-var "extra-keywords"))))
+  (c c++) (append
+           (c-lang-const c-paren-nontype-kwds)
+           '( ;; GCC extension.
+             "__attribute__"
+             ;; MSVC extension.
+             "__declspec")))

 (c-lang-defconst c-paren-type-kwds
   "Keywords that may be followed by a parenthesis expression containing
=== modified file 'lisp/progmodes/cc-vars.el'
--- lisp/progmodes/cc-vars.el	2014-01-01 07:43:34 +0000
+++ lisp/progmodes/cc-vars.el	2014-05-02 05:24:28 +0000
@@ -1614,6 +1614,75 @@
   :group 'c)

 \f
+
+(define-widget 'c-extra-keywords-widget 'lazy
+  "Internal CC Mode widget for the `*-extra-keywords' variables."
+  :type '(repeat
+          (cons
+           (string :tag "Keyword")
+           (boolean :tag "Parenthesized expression follows"))))
+
+(defun c-make-extra-keywords-blurb (mode1 mode2)
+  (concat "\
+*List of extra keywords to recognize in "
+          mode1 " mode.
+Each list item should be a cons (KW . PAREN).
+KW should be a string naming a single identifier.
+PAREN should be nil or t.  If t, expect the a parenthesized expression
+after KW and skip over it.
+
+Note that this variable is only consulted when the major mode is
+initialized.  If you change it later you have to reinitialize CC
+Mode by doing \\[" mode2 "].  Additionally, if you change this
+variable outside of customize, you need to call
+`c-clear-value-cache' to make your changes take effect."))
+
+(defun c-extra-keywords-setter (sym val)
+  (set-default sym val)
+  (c-clear-value-cache))
+
+(defcustom c-extra-keywords
+  nil
+  (c-make-extra-keywords-blurb "C" "c-mode")
+  :type 'c-extra-keywords-widget
+  :set 'c-extra-keywords-setter
+  :group 'c)
+
+(defcustom c++-extra-keywords
+  nil
+  (c-make-extra-keywords-blurb "C++" "c++-mode")
+  :type 'c-extra-keywords-widget
+  :set 'c-extra-keywords-setter
+  :group 'c)
+
+(defcustom objc-extra-keywords
+  nil
+  (c-make-extra-keywords-blurb "ObjC" "objc-mode")
+  :type 'c-extra-keywords-widget
+  :set 'c-extra-keywords-setter
+  :group 'c)
+
+(defcustom java-extra-keywords
+  nil
+  (c-make-extra-keywords-blurb "Java" "java-mode")
+  :type 'c-extra-keywords-widget
+  :set 'c-extra-keywords-setter
+  :group 'c)
+
+(defcustom idl-extra-keywords nil
+  nil
+  :type 'c-extra-keywords-widget
+  :set 'c-extra-keywords-setter
+  :group 'c)
+
+(defcustom pike-extra-keywords
+  nil
+  (c-make-extra-keywords-blurb "Pike" "pike-mode")
+  :type 'c-extra-keywords-widget
+  :set 'c-extra-keywords-setter
+  :group 'c)
+
+\f
 ;; Non-customizable variables, still part of the interface to CC Mode
 (defvar c-macro-with-semi-re nil
   ;; Regular expression which matches a (#define'd) symbol whose expansion


[[End of PGP Signed Part]]

Further CC-mode changes

[Stefan Monnier]

While waiting for your guidance as to which parts of my patches need to
be reverted and why, here's some candidate patch which (contrary to the
one you reverted) actually changes the code's behavior.

It's to be applied on top of the ones you removed (tho since the ones
you remove are mostly cosmetic, it shouldn't make much of a difference).

Questions:
- should C-c C-n and friends be moved to c-derivative-mode-map
  (AFAIK neither Java nor IDL nor Pike have CPP conditionals)?
- I notice that all cc-mode enable abbrev-mode except for idl-mode.
  Was that on purpose or is it just accidental?
- BTW, I think it would be better not to enable abbrev-mode (let it be
  a user's choice).  IIUC CC-mode relies on abbrev-mode internally
  for some of its electric-indent functionality, so we'd have to use
  some other approach (e.g. a post-self-insert-hook).

This is not really tested, especially not on XEmacs, but AFAIK it
doesn't depend on any feature specific to a particular Emacs version.


        Stefan


- Move awk-mode's actual definition to cc-awk so as to get rid of the
  (require 'cc-awk) and the ":syntax-table nil" hack.
- make awk-mode use syntax-propertize when available.
- Use define-derived-mode's built-in keymap inheritance rather than
  c-make-inherited-keymap.
- initialize c-mode-base-map in its defvar.
- fix c-after-font-lock-init so it doesn't accidentally add
  a c-after-change which wasn't there to begin with.
- Run c-update-modeline from after-change-major-mode-hook and
  hack-local-variables-hook rather than from the major mode function.
  This lets us fix the bug where c-mode-hook is run twice and saves us
  from having to define c-run-mode-hooks.
- make c-mode-common into an actual parent mode.  There is a naming problem
  since we have c-mode-base-map for the "parent map" but
  c-mode-common-hook for the "parent hook", and also because this "mode"
  doesn't end in "-mode", but I think we can live with it.
- add a new parent mode c-derivative-mode which can hold those things
  common to c, c++, and objc (i.e. languages which include C as a subset).
  I was tempted to simply use c-mode as the parent for c++-mode and
  objc-mode, but this would be a lot more intrusive and it would force
  the presence of a "C" menu in both C++ and ObjC modes (we could fix
  that for Emacs, but I'm not sure how to do it for XEmacs, and it
  doesn't seem worth the trouble).


=== modified file 'lisp/progmodes/cc-awk.el'
--- lisp/progmodes/cc-awk.el	2014-09-09 15:08:08 +0000
+++ lisp/progmodes/cc-awk.el	2014-09-11 13:48:50 +0000
@@ -42,6 +42,7 @@
 
 (require 'cc-defs)
 (require 'cc-engine)
+(require 'cc-mode)
 
 (defvar awk-mode-syntax-table
   (let ((st (make-syntax-table)))
@@ -52,7 +53,7 @@
     (modify-syntax-entry ?\# "<   " st)
     ;; / can delimit regexes or be a division operator.  By default we assume
     ;; that it is a division sign, and fix the regexp operator cases with
-    ;; `font-lock-syntactic-keywords'.
+    ;; `syntax-table' text-properties.
     (modify-syntax-entry ?/ "." st)     ; ACM 2002/4/27.
     (modify-syntax-entry ?* "." st)
     (modify-syntax-entry ?+ "." st)
@@ -848,6 +849,17 @@
   (max (+ (- c-awk-old-ByLL old-len) (- end beg))
        (c-awk-beyond-logical-line end)))
 
+(defun c-awk-syntax-propertize (beg end)
+  (c-awk-clear-NL-props beg (point-max))
+  (goto-char beg)
+  (c-awk-set-syntax-table-properties end))
+
+(defun c-awk-syntax-propertize-extend-region (beg end)
+  (let ((newend (c-awk-beyond-logical-line end))
+        (newbeg (c-awk-beginning-of-logical-line beg)))
+    (if (or (< newbeg beg) (> newend end))
+        (cons newbeg newend))))
+
 ;; ACM 2002/5/25.  When font-locking is invoked by a buffer change, the region
 ;; specified by the font-lock after-change function must be expanded to
 ;; include ALL of any string or regexp within the region.  The simplest way to
@@ -1129,5 +1141,80 @@
          (goto-char (min start-point end-point)))))))
 
 \f
+;; Support for AWK
+
+;;;###autoload (add-to-list 'auto-mode-alist '("\\.awk\\'" . awk-mode))
+;;;###autoload (add-to-list 'interpreter-mode-alist '("awk" . awk-mode))
+;;;###autoload (add-to-list 'interpreter-mode-alist '("mawk" . awk-mode))
+;;;###autoload (add-to-list 'interpreter-mode-alist '("nawk" . awk-mode))
+;;;###autoload (add-to-list 'interpreter-mode-alist '("gawk" . awk-mode))
+
+(c-define-abbrev-table 'awk-mode-abbrev-table
+  '(("else" "else" c-electric-continued-statement 0)
+    ("while" "while" c-electric-continued-statement 0))
+  "Abbreviation table used in awk-mode buffers.")
+
+(defvar awk-mode-map
+  (let ((map (make-sparse-keymap))
+        (sic (if (featurep 'xemacs) 'self-insert-command)))
+    ;; Add bindings which are only useful for awk.
+    (define-key map "#" sic) ; Override electric parent binding.
+    (define-key map "/" sic) ; Override electric parent binding.
+    (define-key map "*" sic) ; Override electric parent binding.
+    (define-key map "\C-c\C-n" 'undefined) ; #if doesn't exist in awk.
+    (define-key map "\C-c\C-p" 'undefined)
+    (define-key map "\C-c\C-u" 'undefined)
+    (define-key map "\M-a" 'c-beginning-of-statement) ; 2003/10/7
+    (define-key map "\M-e" 'c-end-of-statement)       ; 2003/10/7
+    (define-key map "\C-\M-a" 'c-awk-beginning-of-defun)
+    (define-key map "\C-\M-e" 'c-awk-end-of-defun)
+    map)
+  "Keymap used in awk-mode buffers.")
+
+(easy-menu-define c-awk-menu awk-mode-map "AWK Mode Commands"
+		  (cons "AWK" (c-lang-const c-mode-menu awk)))
+
+;;;###autoload
+(define-derived-mode awk-mode c-mode-common "AWK"
+  "Major mode for editing AWK code.
+To submit a problem report, enter `\\[c-submit-bug-report]' from an
+awk-mode buffer.  This automatically sets up a mail buffer with version
+information already added.  You just need to add a description of the
+problem, including a reproducible test case, and send the message.
+
+To see what version of CC Mode you are running, enter `\\[c-version]'.
+
+The hook `c-mode-common-hook' is run with no args at mode
+initialization, then `awk-mode-hook'.
+
+Key bindings:
+\\{awk-mode-map}"
+  (c-init-language-vars-for 'awk-mode)
+  (c-common-init 'awk-mode)
+  (c-awk-unstick-NL-prop)
+  (when (fboundp 'syntax-propertize)
+    ;; `c-basic-common-init' unconditionally adds `c-before-change' but this
+    ;; function does lots of irrelevant stuff for us (to do with maintaining
+    ;; `c-found-types' and `c-state-cache').  The only useful thing it does is
+    ;; run  `c-get-state-before-change-functions' where we normally add
+    ;; `c-awk-record-region-clear-NL'.
+    (remove-hook 'before-change-functions #'c-before-change t)
+    ;; Same thing for `after-change-functions' with `c-after-change' vs
+    ;; `c-awk-extend-and-syntax-tablify-region' (via
+    ;; `c-before-font-lock-functions').
+    (remove-hook 'after-change-functions #'c-after-change t)
+    ;; CC-mode setq c-new-BEG/END in `c-after-change' and then uses them in
+    ;; `font-lock-extend-after-change-region-function'.  We don't need that and
+    ;; additionally we don't maintain c-new-BEG/END any more, so we really need
+    ;; it out of the way.
+    (kill-local-variable 'font-lock-extend-after-change-region-function)
+    ;; Now, setup syntax-propertize.
+    (add-hook 'syntax-propertize-extend-region-functions
+              #'c-awk-syntax-propertize-extend-region
+              nil t)
+    (set (make-local-variable 'syntax-propertize-function)
+         #'c-awk-syntax-propertize)))
+
+\f
 (provide 'cc-awk)
 ;;; cc-awk.el ends here

=== modified file 'lisp/progmodes/cc-mode.el'
--- lisp/progmodes/cc-mode.el	2014-09-10 16:32:36 +0000
+++ lisp/progmodes/cc-mode.el	2014-09-11 13:33:16 +0000
@@ -198,24 +183,7 @@
 \f
 ;;; Common routines.
 
-(defvar c-mode-base-map ()
-  "Keymap shared by all CC Mode related modes.")
-
-(defun c-make-inherited-keymap ()
-  (let ((map (make-sparse-keymap)))
-    ;; Necessary to use `cc-bytecomp-fboundp' below since this
-    ;; function is called from top-level forms that are evaluated
-    ;; while cc-bytecomp is active when one does M-x eval-buffer.
-    (cond
-     ;; Emacs
-     ((cc-bytecomp-fboundp 'set-keymap-parent)
-      (set-keymap-parent map c-mode-base-map))
-     ;; XEmacs
-     ((fboundp 'set-keymap-parents)
-      (set-keymap-parents map c-mode-base-map))
-     ;; incompatible
-     (t (error "CC Mode is incompatible with this version of Emacs")))
-    map))
+(defvar c-mode-base-map)
 
 (defun c-define-abbrev-table (name defs &optional doc)
   ;; Compatibility wrapper for `define-abbrev' which passes a non-nil
@@ -236,59 +204,58 @@
       (setq defs (cdr defs)))))
 (put 'c-define-abbrev-table 'lisp-indent-function 1)
 
-(defun c-bind-special-erase-keys ()
+(defun c-bind-special-erase-keys (&optional map)
   ;; Only used in Emacs to bind C-c C-<delete> and C-c C-<backspace>
   ;; to the proper keys depending on `normal-erase-is-backspace'.
+  (unless map (setq map c-mode-base-map))
   (if normal-erase-is-backspace
       (progn
-	(define-key c-mode-base-map (kbd "C-c C-<delete>")
+	(define-key map (kbd "C-c C-<delete>")
 	  'c-hungry-delete-forward)
-	(define-key c-mode-base-map (kbd "C-c C-<backspace>")
+	(define-key map (kbd "C-c C-<backspace>")
 	  'c-hungry-delete-backwards))
-    (define-key c-mode-base-map (kbd "C-c C-<delete>")
+    (define-key map (kbd "C-c C-<delete>")
       'c-hungry-delete-backwards)
-    (define-key c-mode-base-map (kbd "C-c C-<backspace>")
+    (define-key map (kbd "C-c C-<backspace>")
       'c-hungry-delete-forward)))
 
-(if c-mode-base-map
-    nil
-
-  (setq c-mode-base-map (make-sparse-keymap))
+(defvar c-mode-base-map
+  (let ((map (make-sparse-keymap)))
 
   ;; Separate M-BS from C-M-h.  The former should remain
   ;; backward-kill-word.
-  (define-key c-mode-base-map [(control meta h)] 'c-mark-function)
-  (define-key c-mode-base-map "\e\C-q"    'c-indent-exp)
+  (define-key map [(control meta h)] 'c-mark-function)
+  (define-key map "\e\C-q"    'c-indent-exp)
   (substitute-key-definition 'backward-sentence
 			     'c-beginning-of-statement
-			     c-mode-base-map global-map)
+			     map global-map)
   (substitute-key-definition 'forward-sentence
 			     'c-end-of-statement
-			     c-mode-base-map global-map)
+			     map global-map)
   (substitute-key-definition 'indent-new-comment-line
 			     'c-indent-new-comment-line
-			     c-mode-base-map global-map)
+			     map global-map)
   (substitute-key-definition 'indent-for-tab-command
 			     ;; XXX Is this the right thing to do
 			     ;; here?
 			     'c-indent-line-or-region
-			     c-mode-base-map global-map)
+			     map global-map)
   (when (fboundp 'comment-indent-new-line)
     ;; indent-new-comment-line has changed name to
     ;; comment-indent-new-line in Emacs 21.
     (substitute-key-definition 'comment-indent-new-line
 			       'c-indent-new-comment-line
-			       c-mode-base-map global-map))
+			       map global-map))
 
   ;; RMS says don't make these the default.
   ;; (April 2006): RMS has now approved these commands as defaults.
   (unless (memq 'argumentative-bod-function c-emacs-features)
-    (define-key c-mode-base-map "\e\C-a"    'c-beginning-of-defun)
-    (define-key c-mode-base-map "\e\C-e"    'c-end-of-defun))
+    (define-key map "\e\C-a"    'c-beginning-of-defun)
+    (define-key map "\e\C-e"    'c-end-of-defun))
 
-  (define-key c-mode-base-map "\C-c\C-n"  'c-forward-conditional)
-  (define-key c-mode-base-map "\C-c\C-p"  'c-backward-conditional)
-  (define-key c-mode-base-map "\C-c\C-u"  'c-up-conditional)
+  (define-key map "\C-c\C-n"  'c-forward-conditional)
+  (define-key map "\C-c\C-p"  'c-backward-conditional)
+  (define-key map "\C-c\C-u"  'c-up-conditional)
 
   ;; It doesn't suffice to put `c-fill-paragraph' on
   ;; `fill-paragraph-function' since `c-fill-paragraph' must be called
@@ -299,11 +266,11 @@
   ;; `fill-paragraph' and the value on `fill-paragraph-function' to
   ;; do the actual filling work.
   (substitute-key-definition 'fill-paragraph 'c-fill-paragraph
-			     c-mode-base-map global-map)
+			     map global-map)
   ;; In XEmacs the default fill function is called
   ;; fill-paragraph-or-region.
   (substitute-key-definition 'fill-paragraph-or-region 'c-fill-paragraph
-			     c-mode-base-map global-map)
+			     map global-map)
 
   ;; We bind the forward deletion key and (implicitly) C-d to
   ;; `c-electric-delete-forward', and the backward deletion key to
@@ -320,21 +287,21 @@
   ;; automatically maps the [delete] and [backspace] keys to these two
   ;; depending on window system and user preferences.  (In earlier
   ;; versions it's possible to do the same by using `function-key-map'.)
-  (define-key c-mode-base-map "\C-d" 'c-electric-delete-forward)
-  (define-key c-mode-base-map "\177" 'c-electric-backspace)
-  (define-key c-mode-base-map "\C-c\C-d"     'c-hungry-delete-forward)
-  (define-key c-mode-base-map [?\C-c ?\d]    'c-hungry-delete-backwards)
-  (define-key c-mode-base-map [?\C-c ?\C-\d] 'c-hungry-delete-backwards)
-  (define-key c-mode-base-map [?\C-c deletechar] 'c-hungry-delete-forward) ; C-c <delete> on a tty.
-  (define-key c-mode-base-map [?\C-c (control deletechar)] ; C-c C-<delete> on a tty.
+  (define-key map "\C-d" 'c-electric-delete-forward)
+  (define-key map "\177" 'c-electric-backspace)
+  (define-key map "\C-c\C-d"     'c-hungry-delete-forward)
+  (define-key map [?\C-c ?\d]    'c-hungry-delete-backwards)
+  (define-key map [?\C-c ?\C-\d] 'c-hungry-delete-backwards)
+  (define-key map [?\C-c deletechar] 'c-hungry-delete-forward) ; C-c <delete> on a tty.
+  (define-key map [?\C-c (control deletechar)] ; C-c C-<delete> on a tty.
     'c-hungry-delete-forward)
   (when (boundp 'normal-erase-is-backspace)
     ;; The automatic C-d and DEL mapping functionality doesn't extend
     ;; to special combinations like C-c C-<delete>, so we have to hook
     ;; into the `normal-erase-is-backspace' system to bind it directly
     ;; as appropriate.
-    (add-hook 'normal-erase-is-backspace-hook 'c-bind-special-erase-keys)
-    (c-bind-special-erase-keys))
+    (add-hook 'normal-erase-is-backspace-hook #'c-bind-special-erase-keys)
+    (c-bind-special-erase-keys map))
 
   (when (fboundp 'delete-forward-p)
     ;; In XEmacs we fix the forward and backward deletion behavior by
@@ -342,41 +309,42 @@
     ;; directly, and use `delete-forward-p' to decide what [delete]
     ;; should do.  That's done in the XEmacs specific
     ;; `c-electric-delete' and `c-hungry-delete' functions.
-    (define-key c-mode-base-map [delete]    'c-electric-delete)
-    (define-key c-mode-base-map [backspace] 'c-electric-backspace)
-    (define-key c-mode-base-map (kbd "C-c <delete>") 'c-hungry-delete)
-    (define-key c-mode-base-map (kbd "C-c C-<delete>") 'c-hungry-delete)
-    (define-key c-mode-base-map (kbd "C-c <backspace>")
+    (define-key map [delete]    'c-electric-delete)
+    (define-key map [backspace] 'c-electric-backspace)
+    (define-key map (kbd "C-c <delete>") 'c-hungry-delete)
+    (define-key map (kbd "C-c C-<delete>") 'c-hungry-delete)
+    (define-key map (kbd "C-c <backspace>")
       'c-hungry-delete-backwards)
-    (define-key c-mode-base-map (kbd "C-c C-<backspace>")
+    (define-key map (kbd "C-c C-<backspace>")
       'c-hungry-delete-backwards))
 
-  (define-key c-mode-base-map "#"         'c-electric-pound)
-  (define-key c-mode-base-map "{"         'c-electric-brace)
-  (define-key c-mode-base-map "}"         'c-electric-brace)
-  (define-key c-mode-base-map "/"         'c-electric-slash)
-  (define-key c-mode-base-map "*"         'c-electric-star)
-  (define-key c-mode-base-map ";"         'c-electric-semi&comma)
-  (define-key c-mode-base-map ","         'c-electric-semi&comma)
-  (define-key c-mode-base-map ":"         'c-electric-colon)
-  (define-key c-mode-base-map "("         'c-electric-paren)
-  (define-key c-mode-base-map ")"         'c-electric-paren)
-
-  (define-key c-mode-base-map "\C-c\C-\\" 'c-backslash-region)
-  (define-key c-mode-base-map "\C-c\C-a"  'c-toggle-auto-newline)
-  (define-key c-mode-base-map "\C-c\C-b"  'c-submit-bug-report)
-  (define-key c-mode-base-map "\C-c\C-c"  'comment-region)
-  (define-key c-mode-base-map "\C-c\C-l"  'c-toggle-electric-state)
-  (define-key c-mode-base-map "\C-c\C-o"  'c-set-offset)
-  (define-key c-mode-base-map "\C-c\C-q"  'c-indent-defun)
-  (define-key c-mode-base-map "\C-c\C-s"  'c-show-syntactic-information)
-  ;; (define-key c-mode-base-map "\C-c\C-t"  'c-toggle-auto-hungry-state)  Commented out by ACM, 2005-03-05.
-  (define-key c-mode-base-map "\C-c."     'c-set-style)
+  (define-key map "#"         'c-electric-pound)
+  (define-key map "{"         'c-electric-brace)
+  (define-key map "}"         'c-electric-brace)
+  (define-key map "/"         'c-electric-slash)
+  (define-key map "*"         'c-electric-star)
+  (define-key map ";"         'c-electric-semi&comma)
+  (define-key map ","         'c-electric-semi&comma)
+  (define-key map ":"         'c-electric-colon)
+  (define-key map "("         'c-electric-paren)
+  (define-key map ")"         'c-electric-paren)
+
+  (define-key map "\C-c\C-\\" 'c-backslash-region)
+  (define-key map "\C-c\C-a"  'c-toggle-auto-newline)
+  (define-key map "\C-c\C-b"  'c-submit-bug-report)
+  (define-key map "\C-c\C-c"  'comment-region)
+  (define-key map "\C-c\C-l"  'c-toggle-electric-state)
+  (define-key map "\C-c\C-o"  'c-set-offset)
+  (define-key map "\C-c\C-q"  'c-indent-defun)
+  (define-key map "\C-c\C-s"  'c-show-syntactic-information)
+  ;; (define-key map "\C-c\C-t"  'c-toggle-auto-hungry-state)  Commented out by ACM, 2005-03-05.
+  (define-key map "\C-c."     'c-set-style)
   ;; conflicts with OOBR
-  ;;(define-key c-mode-base-map "\C-c\C-v"  'c-version)
-  ;; (define-key c-mode-base-map "\C-c\C-y"  'c-toggle-hungry-state)  Commented out by ACM, 2005-11-22.
-  (define-key c-mode-base-map "\C-c\C-w" 'subword-mode)
-  )
+  ;;(define-key map "\C-c\C-v"  'c-version)
+  ;; (define-key map "\C-c\C-y"  'c-toggle-hungry-state)  Commented out by ACM, 2005-11-22.
+  (define-key map "\C-c\C-w" 'subword-mode)
+  map)
+  "Keymap shared by all CC Mode related modes.")
 
 ;; We don't require the outline package, but we configure it a bit anyway.
 (cc-bytecomp-defvar outline-level)
@@ -805,13 +773,6 @@
     (add-hook 'before-hack-local-variables-hook 'c-before-hack-hook)
   (add-hook 'hack-local-variables-hook 'c-postprocess-file-styles))
 
-(defmacro c-run-mode-hooks (&rest hooks)
-  ;; Emacs 21.1 has introduced a system with delayed mode hooks that
-  ;; requires the use of the new function `run-mode-hooks'.
-  (if (fboundp 'run-mode-hooks)
-      `(run-mode-hooks ,@hooks)
-    `(progn ,@(mapcar (lambda (hook) `(run-hooks ,hook)) hooks))))
-
 \f
 ;;; Change hooks, linking with Font Lock and electric-indent-mode.
 
@@ -1202,8 +1163,9 @@
 (defun c-after-font-lock-init ()
   ;; Put on `font-lock-mode-hook'.  This function ensures our after-change
   ;; function will get executed before the font-lock one.
+  (when (memq #'c-after-change after-change-functions)
   (remove-hook 'after-change-functions 'c-after-change t)
-  (add-hook 'after-change-functions 'c-after-change nil t))
+    (add-hook 'after-change-functions 'c-after-change nil t)))
 
 (defun c-font-lock-init ()
   "Set up the font-lock variables for using the font-lock support in CC Mode.
@@ -1264,6 +1226,30 @@
     (c-update-modeline)))
 
 \f
+(defvar c-mode-common-map c-mode-base-map)
+
+(define-derived-mode c-mode-common prog-mode "CC-generic"
+  "Pseudo major mode, parent of all modes using the CC engine."
+  (c-initialize-cc-mode t)
+  (setq abbrev-mode t)                  ;FIXME: Why?
+  ;; Refresh the modeline flags after running the major mode hooks.
+  (add-hook 'after-change-major-mode-hook #'c-update-modeline 'append t)
+  (add-hook 'hack-local-variables-hook #'c-update-modeline 'append t))
+
+(defvar c-derivative-mode-map
+  ;; FIXME: We can't have the menu on this keymap, because the menus for C,
+  ;; C++, and ObjC can't be shared: the only difference between them is their
+  ;; title, but easy-menu offers no way to compute the title dynamically.
+  (let ((map (make-sparse-keymap)))
+    ;; Add bindings which are useful for any C derivative.
+    (define-key map "\C-c\C-e"  'c-macro-expand)
+    map)
+  "Keymap used in c-derivative-mode buffers.")
+
+(define-derived-mode c-derivative-mode c-mode-common "C-derivative"
+  "Pseudo major mode, parent of all modes for C derivatives.
+A C derivative is a language which is a superset of C (or is C itself).")
+
 ;; Support for C
 
 (defvar c-mode-syntax-table
@@ -1276,9 +1262,8 @@
   "Abbreviation table used in c-mode buffers.")
 
 (defvar c-mode-map
-  (let ((map (c-make-inherited-keymap)))
+  (let ((map (make-sparse-keymap)))
     ;; Add bindings which are only useful for C.
-    (define-key map "\C-c\C-e"  'c-macro-expand)
     map)
   "Keymap used in c-mode buffers.")
 
@@ -1314,9 +1299,8 @@
 ;;;###autoload (add-to-list 'auto-mode-alist '("\\.i\\'" . c-mode))
 ;;;###autoload (add-to-list 'auto-mode-alist '("\\.ii\\'" . c++-mode))
 
-
 ;;;###autoload
-(define-derived-mode c-mode prog-mode "C"
+(define-derived-mode c-mode c-derivative-mode "C"
   "Major mode for editing K&R and ANSI C code.
 To submit a problem report, enter `\\[c-submit-bug-report]' from a
 c-mode buffer.  This automatically sets up a mail buffer with version
@@ -1330,18 +1314,11 @@
 
 Key bindings:
 \\{c-mode-map}"
-  (c-initialize-cc-mode t)
-  (set-syntax-table c-mode-syntax-table)
-  (setq local-abbrev-table c-mode-abbrev-table
-	abbrev-mode t)
-  (use-local-map c-mode-map)
   (c-init-language-vars-for 'c-mode)
   (c-make-macro-with-semi-re) ; matches macro names whose expansion ends with ;
   (c-common-init 'c-mode)
   (easy-menu-add c-c-menu)
-  (cc-imenu-init cc-imenu-c-generic-expression)
-  (c-run-mode-hooks 'c-mode-common-hook 'c-mode-hook)
-  (c-update-modeline))
+  (cc-imenu-init cc-imenu-c-generic-expression))
 
 \f
 ;; Support for C++
@@ -1357,9 +1334,8 @@
   "Abbreviation table used in c++-mode buffers.")
 
 (defvar c++-mode-map
-  (let ((map (c-make-inherited-keymap)))
+  (let ((map (make-sparse-keymap)))
     ;; Add bindings which are only useful for C++.
-    (define-key map "\C-c\C-e" 'c-macro-expand)
     (define-key map "\C-c:"    'c-scope-operator)
     (define-key map "<"        'c-electric-lt-gt)
     (define-key map ">"        'c-electric-lt-gt)
@@ -1370,7 +1346,7 @@
 		  (cons "C++" (c-lang-const c-mode-menu c++)))
 
 ;;;###autoload
-(define-derived-mode c++-mode prog-mode "C++"
+(define-derived-mode c++-mode c-derivative-mode "C++"
   "Major mode for editing C++ code.
 To submit a problem report, enter `\\[c-submit-bug-report]' from a
 c++-mode buffer.  This automatically sets up a mail buffer with
@@ -1385,18 +1361,11 @@
 
 Key bindings:
 \\{c++-mode-map}"
-  (c-initialize-cc-mode t)
-  (set-syntax-table c++-mode-syntax-table)
-  (setq local-abbrev-table c++-mode-abbrev-table
-	abbrev-mode t)
-  (use-local-map c++-mode-map)
   (c-init-language-vars-for 'c++-mode)
   (c-make-macro-with-semi-re) ; matches macro names whose expansion ends with ;
   (c-common-init 'c++-mode)
   (easy-menu-add c-c++-menu)
-  (cc-imenu-init cc-imenu-c++-generic-expression)
-  (c-run-mode-hooks 'c-mode-common-hook 'c++-mode-hook)
-  (c-update-modeline))
+  (cc-imenu-init cc-imenu-c++-generic-expression))
 
 \f
 ;; Support for Objective-C
@@ -1411,9 +1380,8 @@
   "Abbreviation table used in objc-mode buffers.")
 
 (defvar objc-mode-map
-  (let ((map (c-make-inherited-keymap)))
+  (let ((map (make-sparse-keymap)))
     ;; Add bindings which are only useful for Objective-C.
-    (define-key map "\C-c\C-e" 'c-macro-expand)
     map)
   "Keymap used in objc-mode buffers.")
 
@@ -1423,7 +1391,7 @@
 ;;;###autoload (add-to-list 'auto-mode-alist '("\\.m\\'" . objc-mode))
 
 ;;;###autoload
-(define-derived-mode objc-mode prog-mode "ObjC"
+(define-derived-mode objc-mode c-derivative-mode "ObjC"
   "Major mode for editing Objective C code.
 To submit a problem report, enter `\\[c-submit-bug-report]' from an
 objc-mode buffer.  This automatically sets up a mail buffer with
@@ -1438,18 +1406,11 @@
 
 Key bindings:
 \\{objc-mode-map}"
-  (c-initialize-cc-mode t)
-  (set-syntax-table objc-mode-syntax-table)
-  (setq local-abbrev-table objc-mode-abbrev-table
-	abbrev-mode t)
-  (use-local-map objc-mode-map)
   (c-init-language-vars-for 'objc-mode)
   (c-make-macro-with-semi-re) ; matches macro names whose expansion ends with ;
   (c-common-init 'objc-mode)
   (easy-menu-add c-objc-menu)
-  (cc-imenu-init nil 'cc-imenu-objc-function)
-  (c-run-mode-hooks 'c-mode-common-hook 'objc-mode-hook)
-  (c-update-modeline))
+  (cc-imenu-init nil 'cc-imenu-objc-function))
 
 \f
 ;; Support for Java
@@ -1466,7 +1427,7 @@
   "Abbreviation table used in java-mode buffers.")
 
 (defvar java-mode-map
-  (let ((map (c-make-inherited-keymap)))
+  (let ((map (make-sparse-keymap)))
     ;; Add bindings which are only useful for Java.
     map)
   "Keymap used in java-mode buffers.")
@@ -1484,7 +1445,7 @@
 ;;;###autoload (add-to-list 'auto-mode-alist '("\\.java\\'" . java-mode))
 
 ;;;###autoload
-(define-derived-mode java-mode prog-mode "Java"
+(define-derived-mode java-mode c-mode-common "Java"
   "Major mode for editing Java code.
 To submit a problem report, enter `\\[c-submit-bug-report]' from a
 java-mode buffer.  This automatically sets up a mail buffer with
@@ -1499,17 +1460,10 @@
 
 Key bindings:
 \\{java-mode-map}"
-  (c-initialize-cc-mode t)
-  (set-syntax-table java-mode-syntax-table)
-  (setq local-abbrev-table java-mode-abbrev-table
-	abbrev-mode t)
-  (use-local-map java-mode-map)
   (c-init-language-vars-for 'java-mode)
   (c-common-init 'java-mode)
   (easy-menu-add c-java-menu)
-  (cc-imenu-init cc-imenu-java-generic-expression)
-  (c-run-mode-hooks 'c-mode-common-hook 'java-mode-hook)
-  (c-update-modeline))
+  (cc-imenu-init cc-imenu-java-generic-expression))
 
 \f
 ;; Support for CORBA's IDL language
@@ -1522,7 +1476,7 @@
   "Abbreviation table used in idl-mode buffers.")
 
 (defvar idl-mode-map
-  (let ((map (c-make-inherited-keymap)))
+  (let ((map (make-sparse-keymap)))
     ;; Add bindings which are only useful for IDL.
     map)
   "Keymap used in idl-mode buffers.")
@@ -1533,7 +1487,7 @@
 ;;;###autoload (add-to-list 'auto-mode-alist '("\\.idl\\'" . idl-mode))
 
 ;;;###autoload
-(define-derived-mode idl-mode prog-mode "IDL"
+(define-derived-mode idl-mode c-mode-common "IDL"
   "Major mode for editing CORBA's IDL, PSDL and CIDL code.
 To submit a problem report, enter `\\[c-submit-bug-report]' from an
 idl-mode buffer.  This automatically sets up a mail buffer with
@@ -1548,16 +1502,12 @@
 
 Key bindings:
 \\{idl-mode-map}"
-  (c-initialize-cc-mode t)
-  (set-syntax-table idl-mode-syntax-table)
-  (setq local-abbrev-table idl-mode-abbrev-table)
-  (use-local-map idl-mode-map)
+  (kill-local-variable 'abbrev-mode) ;FIXME: Only mode that doesn't enable it!?
   (c-init-language-vars-for 'idl-mode)
   (c-common-init 'idl-mode)
   (easy-menu-add c-idl-menu)
   ;;(cc-imenu-init cc-imenu-idl-generic-expression) ;TODO
-  (c-run-mode-hooks 'c-mode-common-hook 'idl-mode-hook)
-  (c-update-modeline))
+  )
 
 \f
 ;; Support for Pike
@@ -1572,7 +1522,7 @@
   "Abbreviation table used in pike-mode buffers.")
 
 (defvar pike-mode-map
-  (let ((map (c-make-inherited-keymap)))
+  (let ((map (make-sparse-keymap)))
     ;; Additional bindings.
     (define-key map "\C-c\C-e" 'c-macro-expand)
     map)
@@ -1585,7 +1535,7 @@
 ;;;###autoload (add-to-list 'interpreter-mode-alist '("pike" . pike-mode))
 
 ;;;###autoload
-(define-derived-mode pike-mode prog-mode "Pike"
+(define-derived-mode pike-mode c-mode-common "Pike"
   "Major mode for editing Pike code.
 To submit a problem report, enter `\\[c-submit-bug-report]' from a
 pike-mode buffer.  This automatically sets up a mail buffer with
@@ -1600,86 +1550,11 @@
 
 Key bindings:
 \\{pike-mode-map}"
-  (c-initialize-cc-mode t)
-  (set-syntax-table pike-mode-syntax-table)
-  (setq local-abbrev-table pike-mode-abbrev-table
-	abbrev-mode t)
-  (use-local-map pike-mode-map)
   (c-init-language-vars-for 'pike-mode)
   (c-common-init 'pike-mode)
   (easy-menu-add c-pike-menu)
   ;;(cc-imenu-init cc-imenu-pike-generic-expression) ;TODO
-  (c-run-mode-hooks 'c-mode-common-hook 'pike-mode-hook)
-  (c-update-modeline))
-
-\f
-;; Support for AWK
-
-;;;###autoload (add-to-list 'auto-mode-alist '("\\.awk\\'" . awk-mode))
-;;;###autoload (add-to-list 'interpreter-mode-alist '("awk" . awk-mode))
-;;;###autoload (add-to-list 'interpreter-mode-alist '("mawk" . awk-mode))
-;;;###autoload (add-to-list 'interpreter-mode-alist '("nawk" . awk-mode))
-;;;###autoload (add-to-list 'interpreter-mode-alist '("gawk" . awk-mode))
-
-(c-define-abbrev-table 'awk-mode-abbrev-table
-  '(("else" "else" c-electric-continued-statement 0)
-    ("while" "while" c-electric-continued-statement 0))
-  "Abbreviation table used in awk-mode buffers.")
-
-(defvar awk-mode-map
-  (let ((map (c-make-inherited-keymap)))
-    ;; Add bindings which are only useful for awk.
-    (define-key map "#" 'self-insert-command)
-    (define-key map "/" 'self-insert-command)
-    (define-key map "*" 'self-insert-command)
-    (define-key map "\C-c\C-n" 'undefined) ; #if doesn't exist in awk.
-    (define-key map "\C-c\C-p" 'undefined)
-    (define-key map "\C-c\C-u" 'undefined)
-    (define-key map "\M-a" 'c-beginning-of-statement) ; 2003/10/7
-    (define-key map "\M-e" 'c-end-of-statement)       ; 2003/10/7
-    (define-key map "\C-\M-a" 'c-awk-beginning-of-defun)
-    (define-key map "\C-\M-e" 'c-awk-end-of-defun)
-    map)
-  "Keymap used in awk-mode buffers.")
-
-(easy-menu-define c-awk-menu awk-mode-map "AWK Mode Commands"
-		  (cons "AWK" (c-lang-const c-mode-menu awk)))
-
-;; (require 'cc-awk) brings these in.
-(defvar awk-mode-syntax-table)
-(declare-function c-awk-unstick-NL-prop "cc-awk" ())
-
-;;;###autoload
-(define-derived-mode awk-mode prog-mode "AWK"
-  "Major mode for editing AWK code.
-To submit a problem report, enter `\\[c-submit-bug-report]' from an
-awk-mode buffer.  This automatically sets up a mail buffer with version
-information already added.  You just need to add a description of the
-problem, including a reproducible test case, and send the message.
-
-To see what version of CC Mode you are running, enter `\\[c-version]'.
-
-The hook `c-mode-common-hook' is run with no args at mode
-initialization, then `awk-mode-hook'.
-
-Key bindings:
-\\{awk-mode-map}"
-  ;; We need the next line to stop the macro defining
-  ;; `awk-mode-syntax-table'.  This would mask the real table which is
-  ;; declared in cc-awk.el and hasn't yet been loaded.
-  :syntax-table nil
-  (require 'cc-awk)			; Added 2003/6/10.
-  (c-initialize-cc-mode t)
-  (set-syntax-table awk-mode-syntax-table)
-  (setq local-abbrev-table awk-mode-abbrev-table
-	abbrev-mode t)
-  (use-local-map awk-mode-map)
-  (c-init-language-vars-for 'awk-mode)
-  (c-common-init 'awk-mode)
-  (c-awk-unstick-NL-prop)
-
-  (c-run-mode-hooks 'c-mode-common-hook 'awk-mode-hook)
-  (c-update-modeline))
+  )
 
 \f
 ;; bug reporting

Re: Further CC-mode changes

[Alan Mackenzie]

Hello, Stefan.

On Thu, Sep 11, 2014 at 09:55:29AM -0400, Stefan Monnier wrote:
> While waiting for your guidance as to which parts of my patches need to
> be reverted and why, here's some candidate patch which (contrary to the
> one you reverted) actually changes the code's behavior.

> It's to be applied on top of the ones you removed (tho since the ones
> you remove are mostly cosmetic, it shouldn't make much of a difference).

> Questions:
> - should C-c C-n and friends be moved to c-derivative-mode-map
>   (AFAIK neither Java nor IDL nor Pike have CPP conditionals)?

No.  Less disruptive is either to remove these bindings from
c-mode-base-map and put them in c-mode-map etc., or to splat them out
from java-mode-map, etc., as is done in awk-mode-map.  This is a bug, I
suppose, but a tiny one - attempting C-c C-n in Java Mode throws an error
immediately.

> - I notice that all cc-mode enable abbrev-mode except for idl-mode.
>   Was that on purpose or is it just accidental?

Probably deliberate.  IDL Mode doesn't have keywords like "else", as far
as I'm aware.  I'm also not aware of anyone using IDL Mode, there having
been no bug reports for it for many years, if ever.

> - BTW, I think it would be better not to enable abbrev-mode (let it be
>   a user's choice).  IIUC CC-mode relies on abbrev-mode internally
>   for some of its electric-indent functionality, so we'd have to use
>   some other approach (e.g. a post-self-insert-hook).

I tend to agree, having been a bit disgusted myself when I discovered how
"else" and friends were being electrically indented.
after-change-functions might be better here, since it's more general, and
also better supported.

> This is not really tested, especially not on XEmacs, but AFAIK it
> doesn't depend on any feature specific to a particular Emacs version.

"This"??

>         Stefan


> - Move awk-mode's actual definition to cc-awk so as to get rid of the
>   (require 'cc-awk) and the ":syntax-table nil" hack.

Would leave traps for maintainers, and break uniformity between the
modes.  Putting all the mode defuns together was a deliberate decision,
though I can't remember whether it was Martin or me that made it.  This
change wouldn't enhance functionality or maintainability.

> - make awk-mode use syntax-propertize when available.

Why?  It's a lot of work to change, syntax-propertize is only available
in relatively recent versions of GNU Emacs, and the documentation
suggests that syntax-propertize is not called after buffer changes in
general, the documentation not being entirely clear on this point.  This
change wouldn't enhance functionality or maintainability.

> - Use define-derived-mode's built-in keymap inheritance rather than
>   c-make-inherited-keymap.

c-make-inherited-keymap is simpler.  This change wouldn't enhance
functionality or maintainability.

> - initialize c-mode-base-map in its defvar.

An inconsequential change that wouldn't enhance functionality or
maintainability.  It's work to make, though.

> - fix c-after-font-lock-init so it doesn't accidentally add
>   a c-after-change which wasn't there to begin with.

???  There's nothing broken here, surely?

> - Run c-update-modeline from after-change-major-mode-hook and
>   hack-local-variables-hook rather than from the major mode function.
>   This lets us fix the bug where c-mode-hook is run twice and saves us
>   from having to define c-run-mode-hooks.

after-change-major-mode-hook is a GNU Emacs speciality.  The major mode
functions (and several commands, of course) are the natural places to
call c-update-modeline from.  How about enhancing define-derived-mode so
that modes that have things to do after running their hooks can do them?
That would surely be a simple change?

> - make c-mode-common into an actual parent mode.  There is a naming problem
>   since we have c-mode-base-map for the "parent map" but
>   c-mode-common-hook for the "parent hook", and also because this "mode"
>   doesn't end in "-mode", but I think we can live with it.

Why?  There is no such mode c-mode-common; nobody's ever going to have a
buffer in this mode; it would merely be a software-engineeringy way of
achieving... what?  It would add complexity where none is warranted.

> - add a new parent mode c-derivative-mode which can hold those things
>   common to c, c++, and objc (i.e. languages which include C as a subset).
>   I was tempted to simply use c-mode as the parent for c++-mode and
>   objc-mode, but this would be a lot more intrusive and it would force
>   the presence of a "C" menu in both C++ and ObjC modes (we could fix
>   that for Emacs, but I'm not sure how to do it for XEmacs, and it
>   doesn't seem worth the trouble).

Why?  What complixity would c-derivative-mode save that would justify its
existence?  I think your last clause is true for the entire paragraph.

[ Snipped ~750 lines of patch. ]

In that patch, your assumptions about what AWK Mode needs in
c-{before,after}-change were incorrect.  There are likely other bugs in
the patch too.  Remember, a previous, "simple, cosmetic" change, using
define-derived-mode to derive CC Mode's modes from prog-mode, introduced
two bugs, each of which took several hours to track down.  One of these
is still unfixed.

It has taken me several hours to study your email and answer it.  Your
patch is all about rearranging the deckchairs around the cruise ship's
pool when the engines need looking at.  It's trying to fix what's not
broken.  It would not contribute one single thing to fixing the many
things in CC Mode which need fixing.  On the contrary, it would act as a
distraction, merely consuming lots more of my time reconciling these
changes with the stand-alone CC Mode.

In the mean time, there are several reported bugs not being fixed, and
the enhancements needed for, e.g., C++11 are not progressing.  The time I
can spend on Emacs and CC Mode is much less than both of us would like.

What needs attending to in CC Mode is NOT its embedding within X?Emacs
and it's definitely NOT the lang-var system, which works just fine.  It's
the complexity in c-guess-basic-syntax, in c-font-lock-declarations,
c-forward-type, c-forward-decl-or-cast-1, and many other such functions,
and the difficulties this causes.

Exasperated,

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: Further CC-mode changes

[Ivan Andrus]

On Sep 12, 2014, at 5:59 PM, Alan Mackenzie <acm@muc.de> wrote:

> Hello, Stefan.
> 
> On Thu, Sep 11, 2014 at 09:55:29AM -0400, Stefan Monnier wrote:
>> While waiting for your guidance as to which parts of my patches need to
>> be reverted and why, here's some candidate patch which (contrary to the
>> one you reverted) actually changes the code's behavior.
> 
>> It's to be applied on top of the ones you removed (tho since the ones
>> you remove are mostly cosmetic, it shouldn't make much of a difference).
> 
>> Questions:
>> - should C-c C-n and friends be moved to c-derivative-mode-map
>>  (AFAIK neither Java nor IDL nor Pike have CPP conditionals)?
> 
> No.  Less disruptive is either to remove these bindings from
> c-mode-base-map and put them in c-mode-map etc., or to splat them out
> from java-mode-map, etc., as is done in awk-mode-map.  This is a bug, I
> suppose, but a tiny one - attempting C-c C-n in Java Mode throws an error
> immediately.
> 
>> - I notice that all cc-mode enable abbrev-mode except for idl-mode.
>>  Was that on purpose or is it just accidental?
> 
> Probably deliberate.  IDL Mode doesn't have keywords like "else", as far
> as I'm aware.  I'm also not aware of anyone using IDL Mode, there having
> been no bug reports for it for many years, if ever.

FWIW I created slice-mode (for the ZeroC Slice language [1]) derived from IDL mode.  It’s only in my .emacs right now since it’s so trivial.  It’s not perfect, but good enough for the few times I actually have to edit the files (they define interfaces and are therefore quite stable).

-Ivan

[1] http://doc.zeroc.com/display/Ice/The+Slice+Language

(define-derived-mode slice-mode idl-mode "Slice"
  "This is a mode for editing slice (Ice definition) files.
It is based on idl-mode because of the comment at
http://www.zeroc.com/forums/help-center/818-any-editor-slice-syntax-highlight.html
I haven't been able to determine the differences between slice
and idl, so I don't know how good it is."

  (font-lock-add-keywords
   'slice-mode
   `(
     (,(concat "\\<\\("
               (regexp-opt
                '("Object" "LocalObject" "exception" "interface" "idempotent"))
               "\\)\\> ")

      1 font-lock-keyword-face t)

     ("^! .*"   0 font-lock-warning-face t)

     ;; Reserved names
     ("\\(Ice\\w*\\)" 1 font-lock-builtin-face)
     ,(concat "\\<\\(\\w*"
              (regexp-opt
               '("Helper" "Holder" "Prx" "Ptr"))
              "\\)\\> ")

     ;; Built-in types
     ,(concat "\\<\\(\\w*"
              (regexp-opt
               '("bool" "byte" "short" "int" "long" "float" "double" "string"))
              "\\)\\> ")
     )))

Re: Further CC-mode changes

[Alan Mackenzie]

Hi, Ivan.

On Fri, Sep 12, 2014 at 07:09:18PM -0600, Ivan Andrus wrote:
> On Sep 12, 2014, at 5:59 PM, Alan Mackenzie <acm@muc.de> wrote:

> > IDL Mode doesn't have keywords like "else", as far as I'm aware.  I'm
> > also not aware of anyone using IDL Mode, there having been no bug
> > reports for it for many years, if ever.

> FWIW I created slice-mode (for the ZeroC Slice language [1]) derived
> from IDL mode.  It’s only in my .emacs right now since it’s so trivial.
> It’s not perfect, but good enough for the few times I actually have to
> edit the files (they define interfaces and are therefore quite stable).

Hey, that's great!  That removes any temptation I might have had just to
expunge IDL from CC Mode.

> -Ivan

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: Further CC-mode changes

[Stefan Monnier]

>> - BTW, I think it would be better not to enable abbrev-mode (let it be
>> a user's choice).  IIUC CC-mode relies on abbrev-mode internally
>> for some of its electric-indent functionality, so we'd have to use
>> some other approach (e.g. a post-self-insert-hook).
> I tend to agree, having been a bit disgusted myself when I discovered how
> "else" and friends were being electrically indented.
> after-change-functions might be better here, since it's more general, and
> also better supported.

I think "more general" is a disadvantage in this case (I'd rather not
have auto-indentation after I paste a space).
As for better supported, I don't care too much: I'd just keep the old
abbrev code for the Emacsen that don't have post-self-insert-hook yet.

>> This is not really tested, especially not on XEmacs, but AFAIK it
>> doesn't depend on any feature specific to a particular Emacs version.
> "This"??

The patch.

>> - Move awk-mode's actual definition to cc-awk so as to get rid of the
>> (require 'cc-awk) and the ":syntax-table nil" hack.
> Would leave traps for maintainers,

Such as?

> and break uniformity between the modes.

Actually, I think the right move would be to move all awk-related code
into cc-awk.el, thus making it more uniform with the various externally
maintained cc-modes.

> This change wouldn't enhance functionality or maintainability.

The code is cleaner and simpler with the change than without.

>> - make awk-mode use syntax-propertize when available.
> Why?

You know why: `syntax-propertize' is used by all major modes except
cc-mode derived ones.  And it's more efficient, because it's more lazy.
Also its correctness argument is much simpler since it doesn't rely on
interactions between before-change-functions, after-change-functions
(and font-lock), contrary to your code.

> It's a lot of work to change,

It's work I did, pro bono.  Just enjoy it.

> syntax-propertize is only available in relatively recent versions of
> GNU Emacs, and the documentation suggests that syntax-propertize is
> not called after buffer changes in general, the documentation not
> being entirely clear on this point.

Indeed it's not.  If you want to be sure that syntax properties have
been applied upto POS, you need to call (syntax-propertize POS).

> This change wouldn't enhance functionality or maintainability.

Yes, it enhances maintainability of awk-mode by making it work more like
all other modes.  It also stops cc-mode from doing all kinds of weird
unrelated crap having to do with maintaining c-found-types and
c-state-cache, which are not used in cc-awk.

>> - Use define-derived-mode's built-in keymap inheritance rather than
>> c-make-inherited-keymap.
> c-make-inherited-keymap is simpler.

Huh?  Have you even looked at the patch?  There is nothing to do to use
define-derived-mode's built-in keymap inheritance, so how can your extra
code be simpler?

> This change wouldn't enhance functionality or maintainability.

It removes unneeded code.  That improves maintainability.

>> - initialize c-mode-base-map in its defvar.
> An inconsequential change that wouldn't enhance functionality or
> maintainability.  It's work to make, though.

It just makes code more standard, with no known downside.

>> - fix c-after-font-lock-init so it doesn't accidentally add
>> a c-after-change which wasn't there to begin with.
> ???  There's nothing broken here, surely?

The current code of c-after-font-lock-init has a bug, but no the current
cc-mode does not trigger this bug.

The bug is that c-after-font-lock-init is supposed to *move*
c-after-change to the front of after-change-functions, but in case
c-after-change is not in after-change-functions, it ends up *adding*
it instead.

Your approach to syntax-table properties requires you to be extra
careful about ordering between c-after-change and font-lock's own
after-change-functions, hence the c-after-font-lock-inithack.  But if
for some reason some cc-mode (such as cc-awk when it uses
syntax-propertize) does not need/want c-after-change, the current code
in c-after-font-lock-init will end up making it needlessly difficult to
remove c-after-change from after-change-functions.

>> - Run c-update-modeline from after-change-major-mode-hook and
>> hack-local-variables-hook rather than from the major mode function.
>> This lets us fix the bug where c-mode-hook is run twice and saves us
>> from having to define c-run-mode-hooks.
> after-change-major-mode-hook is a GNU Emacs speciality.

That's why I also do it from hack-local-variables-hook.

> How about enhancing define-derived-mode so that modes that have things
> to do after running their hooks can do them?

That's not late enough, since you need to do it after setting the
file-local variables.

In this particular case, the better fix would be to introduce a new
`mode-name-sidekick' in the mode-line-format, so we can get rid of
c-update-modeline altogether.

>> - make c-mode-common into an actual parent mode.  There is a naming problem
>> since we have c-mode-base-map for the "parent map" but
>> c-mode-common-hook for the "parent hook", and also because this "mode"
>> doesn't end in "-mode", but I think we can live with it.
> Why?

Why not?

> There is no such mode c-mode-common; nobody's ever going to have a
> buffer in this mode; it would merely be a software-engineeringy way of
> achieving... what?  It would add complexity where none is warranted.

What complexity?  It reduces code duplication, if anything.

> In that patch, your assumptions about what AWK Mode needs in
> c-{before,after}-change were incorrect.

Could you help me understand what I missed?

> There are likely other bugs in the patch too.  Remember, a previous,
> "simple, cosmetic" change, using define-derived-mode to derive CC
> Mode's modes from prog-mode, introduced two bugs, each of which took
> several hours to track down.  One of these is still unfixed.

I admit to not remembering, especially not the remaining bug due to
using prog-mode.  Can you remind us which bug this is?

> It has taken me several hours to study your email and answer it.  Your
> patch is all about rearranging the deckchairs around the cruise ship's
> pool when the engines need looking at.

Any package always has several problems that need fixing.  Just because
a change doesn't fix what you see as the main problem doesn't mean it
should be rejected.  My overall aim is to make cc-modes more "standard"
so as to lower the entry barrier for other people.  I also want to make
it more modular, which I thought I'd start by moving all the awk
functionality out of the generic part of the code and into cc-awk.

> It's trying to fix what's not broken.

I'm not sure what other people on this list feel like w.r.t cc-mode, but
personally so far my attitude has been to close my eyes and pinch my
nose while forwarding the bugs to you because the code is so
horrendously outlandish, messy, and outdated.

> It would not contribute one single thing to fixing the many things in
> CC Mode which need fixing.  On the contrary, it would act as
> a distraction, merely consuming lots more of my time reconciling these
> changes with the stand-alone CC Mode.

The problem here is not my changes, it's having those two separate
code bases.  There is no good reason to have 2 different code bases.
So let's fix this problem once and for all by merging the two and then
make sure they can't diverge any more (i.e. by using the Emacs code as
the place where development happens).

> In the mean time, there are several reported bugs not being fixed, and
> the enhancements needed for, e.g., C++11 are not progressing.  The time I
> can spend on Emacs and CC Mode is much less than both of us would like.

On the one hand you say that you don't have enough time to deal with
CC-mode, and on the other you reject the rare patches sent your way.

If you prefer that I keep just forwarding bug-reports to you and never
ever again look at cc-mode, I can do that.  But if you want help, you'll
need to be a bit more flexible.

> What needs attending to in CC Mode is NOT its embedding within X?Emacs
> and it's definitely NOT the lang-var system, which works just fine.  It's
> the complexity in c-guess-basic-syntax, in c-font-lock-declarations,
> c-forward-type, c-forward-decl-or-cast-1, and many other such functions,
> and the difficulties this causes.

One thing at a time.


        Stefan

Re: Further CC-mode changes

[Alan Mackenzie]

Hello, Stefan.

Sorry I was a bit vague about one or two things last night.  I'll try
and clear them up.

On Fri, Sep 12, 2014 at 11:04:26PM -0400, Stefan Monnier wrote:
> >> - BTW, I think it would be better not to enable abbrev-mode (let it be
> >> a user's choice).  IIUC CC-mode relies on abbrev-mode internally
> >> for some of its electric-indent functionality, so we'd have to use
> >> some other approach (e.g. a post-self-insert-hook).
> > I tend to agree, having been a bit disgusted myself when I discovered how
> > "else" and friends were being electrically indented.
> > after-change-functions might be better here, since it's more general, and
> > also better supported.

> I think "more general" is a disadvantage in this case (I'd rather not
> have auto-indentation after I paste a space).
> As for better supported, I don't care too much: I'd just keep the old
> abbrev code for the Emacsen that don't have post-self-insert-hook yet.

Yes.  That's a bit messy, though.  But I think one could legitimately
categorise the mandatory enablement of abbreviation mode as a bug.

[ ... ]

> >> - Move awk-mode's actual definition to cc-awk so as to get rid of the
> >> (require 'cc-awk) and the ":syntax-table nil" hack.
> > Would leave traps for maintainers,

> Such as?

A change to all of c-mode, c++-mode, ....., pike-mode.  There's a danger
that the maintainer will forget awk-mode, since it's not there any more.

> > and break uniformity between the modes.

> Actually, I think the right move would be to move all awk-related code
> into cc-awk.el, thus making it more uniform with the various externally
> maintained cc-modes.

Another simplification would be simply to load cc-awk.elc along with the
rest of CC Mode, on the grounds that the 19k file is no longer big enough
to worry about on modern machines with gigabytes of RAM.

> > This change wouldn't enhance functionality or maintainability.

> The code is cleaner and simpler with the change than without.

Yes, maybe, but....

> >> - make awk-mode use syntax-propertize when available.
> > Why?

> You know why: `syntax-propertize' is used by all major modes except
> cc-mode derived ones.  And it's more efficient, because it's more lazy.

Lazy?  Does C-M-f, for example, trigger syntax-propertize in any way?

> Also its correctness argument is much simpler since it doesn't rely on
> interactions between before-change-functions, after-change-functions
> (and font-lock), contrary to your code.

The argument could run that because syntax-propertize doesn't take
account of these subleties, its actions aren't always correct.  But ...

syntax-propertize isn't documented in the Elisp manual.  It's unclear
when it gets called (in relation to after-change-functions, for example)
and even _if_ it always gets called (e.g. when font-lock isn't enabled).
Adapting code to use it is thus a _massive_ amount of work - it involves
reading the source code for syntax-propertize and friends to work out
what it does.  I suspect that for a mode which needs syntax-table text
properties early in its after-change-functions, syntax-properties
wouldn't work.  I don't think CC Mode comes into that category (though
I'm not sure).

> > It's a lot of work to change,

> It's work I did, pro bono.  Just enjoy it.

No, you haven't.  What's the equivalent of `syntax-propertize' in XEmacs,
or an old GNU Emacs?  "Somebody" (tm) has to write a compatibility macro
which will look something like this:

(defmacro c-set-syntax-propertize (f)
  (if (boundp 'syntax-propertize-function)
      `(setq syntax-propertize-function ,f)
    `(????
    )))

What exactly goes in "????"?  So many changes you want to make are of
this nature - using new facilities of Emacs to replace something which
works fine anyhow, and then leaving somebody else, me, to take care of
backwards and XEmacs compatibility.

> > syntax-propertize is only available in relatively recent versions of
> > GNU Emacs, and the documentation suggests that syntax-propertize is
> > not called after buffer changes in general, the documentation not
> > being entirely clear on this point.

What I meant here was it is not clear when (or even whether)
syntax-propertize gets called when font-lock is not enabled.  If s-p is
not called with disabled font-lock, I think you should state this
explicitly somewhere in the Elisp manual and/or doc string (or fix the
problem and then document the fixed function).

> Indeed it's not.  If you want to be sure that syntax properties have
> been applied upto POS, you need to call (syntax-propertize POS).

Right.  So I'm now going to have three alternative code flows in place of
the one I currently have: (i) One flow for when font lock's enabled; (ii)
one for when FL is disabled, which explicitly calls syntax-propertize
from after-change-functions; (iii) one for Emacsen lacking
syntax-propertize.  I've already got (iii).  I think you're asking me to
add (i) and (ii).  This is an increase in complexity and a lot of work.

> > This change wouldn't enhance functionality or maintainability.

> Yes, it enhances maintainability of awk-mode by making it work more like
> all other modes.  It also stops cc-mode from doing all kinds of weird
> unrelated crap having to do with maintaining c-found-types and
> c-state-cache, which are not used in cc-awk.

c-state-cache IS used in AWK; it's used in all CC Mode modes and is
fundamental to parsing braces etc.  c-found-types isn't, since there are
no types in AWK.  Probably the neatest solution here would be to move the
c-found-types stuff into its own function, called via the
`c-get-state-before-change-functions' mechanism, or similar.

> >> - Use define-derived-mode's built-in keymap inheritance rather than
> >> c-make-inherited-keymap.
> > c-make-inherited-keymap is simpler.

> Huh?  Have you even looked at the patch?  There is nothing to do to use
> define-derived-mode's built-in keymap inheritance, so how can your extra
> code be simpler?

OK, maybe it's not that bad.  define-derived-mode does exist in (at
least) newer versions of XEmacs.  But it seems to mean having at least
one "artificial" mode which will add to complexity.  define-derived-mode
is a complicated macro, with so many implicit things, that often the only
way to see what it does is with `macroexpand'.

> > This change wouldn't enhance functionality or maintainability.

> It removes unneeded code.  That improves maintainability.

> >> - initialize c-mode-base-map in its defvar.
> > An inconsequential change that wouldn't enhance functionality or
> > maintainability.  It's work to make, though.

> It just makes code more standard, with no known downside.

But with no upside either.  The downside, as already said, is the work
involved in doing it and propagating the change to stand alone CC Mode.

> >> - fix c-after-font-lock-init so it doesn't accidentally add
> >> a c-after-change which wasn't there to begin with.
> > ???  There's nothing broken here, surely?

> The current code of c-after-font-lock-init has a bug, but no the current
> cc-mode does not trigger this bug.

> The bug is that c-after-font-lock-init is supposed to *move*
> c-after-change to the front of after-change-functions, but in case
> c-after-change is not in after-change-functions, it ends up *adding*
> it instead.

In CC Mode, c-after-change IS on after-change-functions.  This is
fundamental.  There is no situation in which this can't be the case.

> Your approach to syntax-table properties requires you to be extra
> careful about ordering between c-after-change and font-lock's own
> after-change-functions, hence the c-after-font-lock-inithack.  But if
> for some reason some cc-mode (such as cc-awk when it uses
> syntax-propertize) does not need/want c-after-change, the current code
> in c-after-font-lock-init will end up making it needlessly difficult to
> remove c-after-change from after-change-functions.

See above.  I really can't foresee any circumstances where a CC Mode mode
won't need before and after change functions.  Both c-before-change and
c-after-change are equipped with a mechanism to have mode dependent
functions called.  Maybe a little more modularisation using these
facilities wouldn't go amiss.

> >> - Run c-update-modeline from after-change-major-mode-hook and
> >> hack-local-variables-hook rather than from the major mode function.
> >> This lets us fix the bug where c-mode-hook is run twice and saves us
> >> from having to define c-run-mode-hooks.
> > after-change-major-mode-hook is a GNU Emacs speciality.

> That's why I also do it from hack-local-variables-hook.

> > How about enhancing define-derived-mode so that modes that have things
> > to do after running their hooks can do them?

> That's not late enough, since you need to do it after setting the
> file-local variables.

How about doing it anyway?  A major mode incorporating user changes made
in its mode hook into its current state is a reasonable thing to expect
to do.

> In this particular case, the better fix would be to introduce a new
> `mode-name-sidekick' in the mode-line-format, so we can get rid of
> c-update-modeline altogether.

I don't think XEmacs has the capability of running functions from mode
line format elements.  I looked, once.

> >> - make c-mode-common into an actual parent mode.  There is a naming problem
> >> since we have c-mode-base-map for the "parent map" but
> >> c-mode-common-hook for the "parent hook", and also because this "mode"
> >> doesn't end in "-mode", but I think we can live with it.
> > Why?

> Why not?

From a user point of view there is no "mode" here - artificially
introducing it would merely create confusion.

> > There is no such mode c-mode-common; nobody's ever going to have a
> > buffer in this mode; it would merely be a software-engineeringy way of
> > achieving... what?  It would add complexity where none is warranted.

> What complexity?  It reduces code duplication, if anything.

The complexity is in having a deep inheritance tree of modes and "modes".
The number of duplicated code lines is of the order of 2 or 3, or maybe 5
or 6.

Originally C Mode was simply a defun.  Now it's got prog-mode as a
super-mode.  You're advocating inserting another one or two (I'm a bit
confused as to which) modes into that tree.  This might be OK if those
modes had a coherent meaning (as prog-mode does), but it seems that they
do not.  _That_ is complexity.

What makes code difficult to read is always "having to look somewhere
else".  Your approach will split (the function) c-mode into two or three
fragments, making readers and maintainers have to deal with these two or
three bits.

> > In that patch, your assumptions about what AWK Mode needs in
> > c-{before,after}-change were incorrect.

> Could you help me understand what I missed?

Already dealt with.  At the very least, AWK Mode uses c-parse-state and
thus needs its before-change function.

> > There are likely other bugs in the patch too.  Remember, a previous,
> > "simple, cosmetic" change, using define-derived-mode to derive CC
> > Mode's modes from prog-mode, introduced two bugs, each of which took
> > several hours to track down.  One of these is still unfixed.

> I admit to not remembering, especially not the remaining bug due to
> using prog-mode.  Can you remind us which bug this is?

The two bugs were the double calling of the mode hooks, and the splatting
of AWK Mode syntax table by define-derived-mode defvar'ing
awk-mode-syntax-table itself.

> > It has taken me several hours to study your email and answer it.  Your
> > patch is all about rearranging the deckchairs around the cruise ship's
> > pool when the engines need looking at.

> Any package always has several problems that need fixing.  Just because
> a change doesn't fix what you see as the main problem doesn't mean it
> should be rejected.  My overall aim is to make cc-modes more "standard"
> so as to lower the entry barrier for other people.  I also want to make
> it more modular, which I thought I'd start by moving all the awk
> functionality out of the generic part of the code and into cc-awk.

As discussed above, you're swapping one problem for another.  All the
entry points to CC Mode are in cc-mode.el.  You'd lose that unity.  As
already suggested, maybe simply loading cc-awk.elc all the time would be
best.

> > It's trying to fix what's not broken.

> I'm not sure what other people on this list feel like w.r.t cc-mode, but
> personally so far my attitude has been to close my eyes and pinch my
> nose while forwarding the bugs to you because the code is so
> horrendously outlandish, messy, and outdated.

It's outlandish and messy, yes, because it's 30 years old and handles a
multiplicity of complicated languages.  Are there any other Emacs modes
which handle many variants on a theme, like C, C++, Java, ...?  Your
solution to the messiness and outdatedness is to use Emacs's newest
facilities, rendering the code base in need of heavy adaptation to run
elsewhere.  That's not a solution.

> > It would not contribute one single thing to fixing the many things in
> > CC Mode which need fixing.  On the contrary, it would act as
> > a distraction, merely consuming lots more of my time reconciling these
> > changes with the stand-alone CC Mode.

> The problem here is not my changes, ....

But the problem _is_ your changes.  You would quite happily have
committed your patch despite not realising it breaks AWK Mode.  It's not
reasonable to expect that you, the maintainer of Emacs as a whole, should
be familiar with the last nuance of every bit of code in the system.
What you want to change in CC Mode needs to be checked (just as something
I might want to change in syntax.c or font lock needs checking).

> ... it's having those two separate code bases.  There is no good reason
> to have 2 different code bases.

In times past, the CC Mode code base was the primary one, and the number
of changes to CC Mode made directly in savannah was small.  The latter
has changed in recent years.

> So let's fix this problem once and for all by merging the two and then
> make sure they can't diverge any more (i.e. by using the Emacs code as
> the place where development happens).

Then XEmacs and backward compatibility would inevitably disappear,
destroyed by the inexorable creeping use of GNU Emacs only facilities.
I'm not sure how you'd feel about the CC Mode version in savannah being a
portable one and remaining so.  You seem very unhappy about a lot of the
portable coding in CC Mode.

> > In the mean time, there are several reported bugs not being fixed, and
> > the enhancements needed for, e.g., C++11 are not progressing.  The time I
> > can spend on Emacs and CC Mode is much less than both of us would like.

> On the one hand you say that you don't have enough time to deal with
> CC-mode, and on the other you reject the rare patches sent your way.

These rare patches tend to be buggy and take a lot of time to check, and
they tend to address "non-functional" issues.  If the patch came in an
email with Subject: "Proposed fix for bug #23456", as one did within the
last fortnight (and also the one from Daniel Colascione in May), I'd be
very much happier.

> If you prefer that I keep just forwarding bug-reports to you and never
> ever again look at cc-mode, I can do that.  But if you want help, you'll
> need to be a bit more flexible.

> > What needs attending to in CC Mode is NOT its embedding within X?Emacs
> > and it's definitely NOT the lang-var system, which works just fine.  It's
> > the complexity in c-guess-basic-syntax, in c-font-lock-declarations,
> > c-forward-type, c-forward-decl-or-cast-1, and many other such functions,
> > and the difficulties this causes.

> One thing at a time.

:-)

>         Stefan

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: Further CC-mode changes

[Stefan Monnier]

> A change to all of c-mode, c++-mode, ....., pike-mode.  There's a danger
> that the maintainer will forget awk-mode, since it's not there any more.

The same already applies to the dozen externally maintained modes that
use the cc-mode engine.  Making cc-awk more like those modes would
actually help make sure the modularization is done right (or at least,
"right enough for AWK").

> Another simplification would be simply to load cc-awk.elc along with the
> rest of CC Mode, on the grounds that the 19k file is no longer big enough
> to worry about on modern machines with gigabytes of RAM.

I don't see why that would help or what it would simplify.

>> You know why: `syntax-propertize' is used by all major modes except
>> cc-mode derived ones.  And it's more efficient, because it's more lazy.
> Lazy?  Does C-M-f, for example, trigger syntax-propertize in any way?

Not yet, no.

>> Also its correctness argument is much simpler since it doesn't rely on
>> interactions between before-change-functions, after-change-functions
>> (and font-lock), contrary to your code.
> The argument could run that because syntax-propertize doesn't take
> account of these subleties, its actions aren't always correct.  But ...
> syntax-propertize isn't documented in the Elisp manual.  It's unclear
> when it gets called (in relation to after-change-functions, for example)
> and even _if_ it always gets called (e.g. when font-lock isn't enabled).

That's because it's irrelevant: if *you* need the properties to be
applied, then call (syntax-propertize POS).  If they've already been
applied, then this will return immediately.

Since font-lock's syntactic fontification needs those properties to be
applied, it does call syntax-propertize.  Similarly,
indent-according-to-mode calls it since it's usually a good idea.
And yes, C-M-f should too, tho noone has been sufficiently bothered by
it to implement it yet.

> Adapting code to use it is thus a _massive_ amount of work - it involves
> reading the source code for syntax-propertize and friends to work out
> what it does.

No you don't: the contract is just what I stated above and you don't
need to know more than that.

> I suspect that for a mode which needs syntax-table text properties
> early in its after-change-functions, syntax-properties wouldn't work.

Suspect all you want.  But I assure you it works just fine.

>> > It's a lot of work to change,
>> It's work I did, pro bono.  Just enjoy it.
> No, you haven't.  What's the equivalent of `syntax-propertize' in XEmacs,
> or an old GNU Emacs?

Did you look at my patch?  It only uses syntax-propertize
when available.  When not, it keeps using your old code.  And the bulk of
the code is shared between the two cases.

> Right.  So I'm now going to have three alternative code flows in place of
> the one I currently have: (i) One flow for when font lock's enabled; (ii)
> one for when FL is disabled, which explicitly calls syntax-propertize
> from after-change-functions; (iii) one for Emacsen lacking
> syntax-propertize.  I've already got (iii).  I think you're asking me to
> add (i) and (ii).  This is an increase in complexity and a lot of work.

There are many different ways to handle backward compatibility.  You can
do it as above (where my position would be to simply keep the current
code for the old Emacsen that don't have syntax-propertize, so while
there are more different ways to work than you'd like the old ways have
been tested and there's no reason they should stop working, so you can
largely ignore them).
Or you can provide you own implementation of syntax-propertize.
It's actually simple.  If you prefer this route, I can do
that as well.

>> > This change wouldn't enhance functionality or maintainability.
>> Yes, it enhances maintainability of awk-mode by making it work more like
>> all other modes.  It also stops cc-mode from doing all kinds of weird
>> unrelated crap having to do with maintaining c-found-types and
>> c-state-cache, which are not used in cc-awk.
> c-state-cache IS used in AWK; it's used in all CC Mode modes and is
> fundamental to parsing braces etc.

Hmmm... I didn't notice it in my tests.  I guess it only shows up if you
really go around editing the test such that the cache becomes stale?
OK, I'll rework my syntax-propertize patch accordingly.

> OK, maybe it's not that bad.  define-derived-mode does exist in (at
> least) newer versions of XEmacs.  But it seems to mean having at least
> one "artificial" mode which will add to complexity.  define-derived-mode
> is a complicated macro, with so many implicit things, that often the only
> way to see what it does is with `macroexpand'.

The behavior I rely upon existed in define-derived-mode long before
I started I mess with it for Emacs-21.  You already have an "artificial"
parent mode.  Its hook is called c-mode-common-hook its keymap is called
c-mode-base-map, its name for c-lang-defconst is `t', its name for
c-fallback-mode' is nil.

Its standard practice in OO to have a parent which is not a "real" class
(because it doesn't have instances), but is only present so that various
classes can share a common parent.  This is just the same.

> See above.  I really can't foresee any circumstances where a CC Mode mode
> won't need before and after change functions.

syntax-propertize uses a before-change-function to "flush the cache", of
course.  After that you just call it when you need it.  E.g. If
font-lock itself doesn't call it, then we'll need to arrange to run it
before font-lock does its thing.

> How about doing it anyway?  A major mode incorporating user changes made
> in its mode hook into its current state is a reasonable thing to expect
> to do.

Yes, but usually those modes, just like CC-mode, need to do that after
setting up file-local variables.  So using hack-local-variables-hook is
really not a bad option.

>> In this particular case, the better fix would be to introduce a new
>> `mode-name-sidekick' in the mode-line-format, so we can get rid of
>> c-update-modeline altogether.
> I don't think XEmacs has the capability of running functions from mode
> line format elements.  I looked, once.

Your mode-line elements are sufficiently simple that there's no need to
run functions.  An element like:

      (c-electric-flag ("/l"
                        (c-auto-newline "a")
                        (c-hungry-delete-key "h")
                        (subword-mode "w"))
       (c-hungry-delete-key ("/h" (subword-mode "w")))
       (subword-mode "/w"))

should work just fine in XEmacs as well.

> The complexity is in having a deep inheritance tree of modes and "modes".

That lets users configure their CC-mode more easily: they can add
a single keybinding (or hook function) that'll appear in C, C++, and ObjC.

>> > There are likely other bugs in the patch too.  Remember, a previous,
>> > "simple, cosmetic" change, using define-derived-mode to derive CC
>> > Mode's modes from prog-mode, introduced two bugs, each of which took
>> > several hours to track down.  One of these is still unfixed.
>> I admit to not remembering, especially not the remaining bug due to
>> using prog-mode.  Can you remind us which bug this is?
> The two bugs were the double calling of the mode hooks, and the splatting
> of AWK Mode syntax table by define-derived-mode defvar'ing
> awk-mode-syntax-table itself.

Ah, right, then: both are fixed by my patch.  That's the advantage of
using "standard coding practices" which have been tried and tested in
many other modes already.

> As discussed above, you're swapping one problem for another.  All the
> entry points to CC Mode are in cc-mode.el.

I don't see d-mode's entry point in cc-mode.  Neither do I see
cuda-mode, dart-mode, php-mode, ...

> You'd lose that unity.

This unity is an illusion.

> As already suggested, maybe simply loading cc-awk.elc all the time
> would be best.

No, we do not want one big monolithic monster.

> multiplicity of complicated languages.  Are there any other Emacs modes
> which handle many variants on a theme, like C, C++, Java, ...?

There's SMIE, of course.  It's used for a fair number of programming
languages now (Coq, OCaml, SML, Prolog, RC, SH, Modula-2, Octave, Ruby,
CSS, Swift, GAP, Elixir).

> Your solution to the messiness and outdatedness is to use Emacs's
> newest facilities, rendering the code base in need of heavy adaptation
> to run elsewhere.  That's not a solution.

While I have spent most of the last decade focusing on code that only
runs on the latest Emacs, I have a fair bit of experience with backward
compatibility.  There are always various different ways to solve the problem.

BTW, I'd be interested to know more about the standalone package:
- what kind of emacsen is it mostly used on (e.g. is it mostly used by XEmacs
  users, or mostly Emacs users)?
- is it mostly used so as to get support for newer language features or
  is it mostly used to get new major-mode features?
- how many users are we talking about?

This said, I admit that my general position is that backward
compatibility is a burden that's not very valuable: after all people who
use still use Emacs-22 can use the CC-mode that came with Emacs-22.
So it's OK to make some effort at backward compatibility, but you
quickly get to a point of diminishing returns.

In the case of CC-mode, I think the outdated state of the code that
results from the way backward compatibility has been handled is slowing
down CC-mode's progress, so while a few users of XEmacs and Emacs-22 may
benefit a bit, it is to the detriment of most users of Emacs-24.

> But the problem _is_ your changes.  You would quite happily have
> committed your patch despite not realising it breaks AWK Mode.

FWIW, I sent it for review, in case you did not notice.

BTW, you still haven't told me what my previous (cosmetic) patches break
(although you did revert them).

>> So let's fix this problem once and for all by merging the two and then
>> make sure they can't diverge any more (i.e. by using the Emacs code as
>> the place where development happens).
> Then XEmacs and backward compatibility would inevitably disappear,
> destroyed by the inexorable creeping use of GNU Emacs only facilities.

Don't be ridiculous.  There are many packages included in Emacs where we
need to preserve backward compatibility (because we know that the code
is also distributed separately), so we know we have to be careful to
preserve backward compatibility.  Of course, we do occasionally break it
by accident, but its far from common.
Again, we have a lot of experience preserving backward compatibility
(even if reluctantly).

> I'm not sure how you'd feel about the CC Mode version in savannah being a
> portable one and remaining so.  You seem very unhappy about a lot of the
> portable coding in CC Mode.

I'm not happy about you refusing to use newer facilities.
That's not the same as being unhappy about preserving compatibility.

> These rare patches tend to be buggy and take a lot of time to check, and
> they tend to address "non-functional" issues.  If the patch came in an
> email with Subject: "Proposed fix for bug #23456", as one did within the
> last fortnight (and also the one from Daniel Colascione in May), I'd be
> very much happier.

I don't intend to fix bugs, no.  I intend to clean up the code so that
it is more accessible.  Hopefully that will also make it easier to
improve the modes and fix bugs.

What are my options: ask for your extensive test suite (which we did
already a long time ago, and haven't seen it yet), then test my changes
against it, then install my patch.  And then see it reverted because you
don't like my cosmetic changes?


        Stefan

Syntax-propertize and CC-mode [Was: Further CC-mode changes]

[Alan Mackenzie]

Hello again, Stefan.

I've separated the discussion on syntax-propertize from the rest,
because things are threatening to get out of hand.

On Sat, Sep 13, 2014 at 03:24:03PM -0400, Stefan Monnier wrote:

> >> You know why: `syntax-propertize' is used by all major modes except
> >> cc-mode derived ones.  And it's more efficient, because it's more lazy.
> > Lazy?  Does C-M-f, for example, trigger syntax-propertize in any way?

> Not yet, no.

OK.  I'm not yet convinced that syntax-propertize works well in languages
like C++ and Java, where template/generic delimiters < and > are marked
with category properties.  (These have symbols with syntax-table
properties.)

Because applying these properties is expensive at runtime, they are tended
to with loving care in CC Mode.  When a buffer change is made, a
before-change function c-before-change-check-<>-operators removes the
category properties from exactly those pairs of <>s which need it, in
particular, the removal region is LIMITED TO THE CURRENT "STATEMENT".  (A
template/generic construct cannot include braces or semicolons.)  The
properties get applied, when necessary, as a side effect of the scanning
which happens during font-locking.  It is a long-standing bug that they
aren't applied when font-locking is disabled.

> >> Also its correctness argument is much simpler since it doesn't rely on
> >> interactions between before-change-functions, after-change-functions
> >> (and font-lock), contrary to your code.
> > The argument could run that because syntax-propertize doesn't take
> > account of these subleties, its actions aren't always correct.  But ...
> > syntax-propertize isn't documented in the Elisp manual.  It's unclear
> > when it gets called (in relation to after-change-functions, for example)
> > and even _if_ it always gets called (e.g. when font-lock isn't enabled).

> That's because it's irrelevant: if *you* need the properties to be
> applied, then call (syntax-propertize POS).  If they've already been
> applied, then this will return immediately.

That's useful to know.  I didn't know that (and couldn't have known it)
before, since it is undocumented.  How is the lower limit determined
(matching the upper limit POS)?  At a guess, this is going to be anywhere
down to BOB.  This isn't very lazy.  Why doesn't syntax-propertize take
TWO parameters, BEG and END?

> Since font-lock's syntactic fontification needs those properties to be
> applied, it does call syntax-propertize.

I didn't know this either.  How will this be interacting with C++/Java
Modes' use of category properties to set the syntax?  I don't have good
feelings about this.  

Looking at the source, syntax-propertize is unoptimised.  It will always
erase the syntax-table properties from the last buffer change onwards, in
contrast to C++ Mode, which only erases them locally as needed.  There is
surely an opportunity for better customisation here.

> Similarly, indent-according-to-mode calls it since it's usually a good
> idea.  And yes, C-M-f should too, tho noone has been sufficiently
> bothered by it to implement it yet.

> > Adapting code to use it is thus a _massive_ amount of work - it involves
> > reading the source code for syntax-propertize and friends to work out
> > what it does.

> No you don't: the contract is just what I stated above and you don't
> need to know more than that.

That is not true.  I need to know that syntax-propertize would nullify
C++/Java Modes' careful optimisations.  I need to know whether or not
syntax-propertize properly handles category properties.  I suspect it
doesn't.  That involves reading the source code, for lack of
documentation.

> > I suspect that for a mode which needs syntax-table text properties
> > early in its after-change-functions, syntax-properties wouldn't work.

> Suspect all you want.  But I assure you it works just fine.

> >> > It's a lot of work to change,
> >> It's work I did, pro bono.  Just enjoy it.
> > No, you haven't.  What's the equivalent of `syntax-propertize' in XEmacs,
> > or an old GNU Emacs?

> Did you look at my patch?  It only uses syntax-propertize
> when available.  When not, it keeps using your old code.  And the bulk of
> the code is shared between the two cases.

I looked at the patch, yes, but didn't catch every last detail.  There's
the philosophical question why add this extra code when it's not
replacing old code, but causing bloat.  I now think syntactic-propertize
contains a severe pessimisation: At a buffer change, all syntax-table
properties are effectively erased between the change point and EOB.  This
wasn't done in the existing AWK Mode code.  Perhaps it doesn't matter too
much, since AWK buffers tend not to be very big.

> > Right.  So I'm now going to have three alternative code flows in place of
> > the one I currently have: (i) One flow for when font lock's enabled; (ii)
> > one for when FL is disabled, which explicitly calls syntax-propertize
> > from after-change-functions; (iii) one for Emacsen lacking
> > syntax-propertize.  I've already got (iii).  I think you're asking me to
> > add (i) and (ii).  This is an increase in complexity and a lot of work.

> There are many different ways to handle backward compatibility.  You can
> do it as above (where my position would be to simply keep the current
> code for the old Emacsen that don't have syntax-propertize, so while
> there are more different ways to work than you'd like the old ways have
> been tested and there's no reason they should stop working, so you can
> largely ignore them).
> Or you can provide you own implementation of syntax-propertize.
> It's actually simple.  If you prefer this route, I can do
> that as well.

syntax-propertize needs a way of customizing its before-change function
to avoid removing ST properties needlessly from large chunks of buffer.
How about `syntax-depropertize-function', which if non-nil would be run
in place of a bit of syntax-ppss-flush-cache?  Or something like that?

[ .... ]

> syntax-propertize uses a before-change-function to "flush the cache", of
> course.  After that you just call it when you need it.  E.g. If
> font-lock itself doesn't call it, then we'll need to arrange to run it
> before font-lock does its thing.

[ .... ]

>         Stefan

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: Syntax-propertize and CC-mode [Was: Further CC-mode changes]

[Stefan Monnier]

> I've separated the discussion on syntax-propertize from the rest,
> because things are threatening to get out of hand.

Important note: I only provided a patch to use syntax-propertize for
awk-mode.  This case is much simpler to handle than the C/C++/Java cases
(at least, IIUC, the way you apply syntax-table properties in awk-mode
is easy to map to what syntax-propertize needs).

Using syntax-propertize for C/C++/Java is a whole other project, which
I hope to get to at some point, but I'll need to dig much deeper into
the belly of the beast before I can hope to do that.

> How is the lower limit determined (matching the upper limit POS)?

Syntax-propertize internally keeps track of "up-to-where have properties
been applied", so that it can provide the lower limit.

> At a guess, this is going to be anywhere down to BOB.

That can be, tho usually it's just "the beginning of the last buffer
modification".

> This isn't very lazy.  Why doesn't syntax-propertize take TWO
> parameters, BEG and END?

How would you know what is BEG without knowing what other packages
(indent.el, font-lock.el, ...) have already `syntax-propertize'd?

> Looking at the source, syntax-propertize is unoptimised.  It will always
> erase the syntax-table properties from the last buffer change onwards, in
> contrast to C++ Mode, which only erases them locally as needed.

syntax-propertize is lazy, but indeed it always assumes pessimistically
that any change to the buffer at POS might cause everything after POS to
be interpreted completely differently and hence require re-propertization.
So far, this has not proved to be problematic in any of the 20 or so
major modes that use syntax-propertize.

> There is surely an opportunity for better customisation here.

Yes, we could spice up syntax.el so that you can set some properties and
explain when modifications in previous text don't invalidate them.
We haven't seen a need for it so far.

> That is not true.  I need to know that syntax-propertize would nullify
> C++/Java Modes' careful optimisations.

syntax-propertize (as it stands) is probably not compatible with some of
the optimizations you use, indeed.  It probably gives you other
optimization opportunities, of course.  How well it would work is hard
to tell without a lot more investigation.

> I need to know whether or not syntax-propertize properly handles
> category properties.

By and large, `syntax-propertize' doesn't handle anything at all.

It does is keep track of the BEG argument for you, and it does erase the
`syntax-table' properties over BEG..END before calling
syntax-propertize-function.  But that's about all it does.

> I suspect it doesn't.  That involves reading the
> source code, for lack of documentation.

The source is all in a single function (called, you guessed,
syntax-propertize) which weighs in at 39 lines of Elisp code, a good
half of which has to do with calling
syntax-propertize-extend-region-functions.

> syntax-propertize needs a way of customizing its before-change function
> to avoid removing ST properties needlessly from large chunks of buffer.

Note that in 99% of the cases, there aren't any properties applied
further in the buffer anyway, so we don't actually "needlessly"
throwaway data on large chunks of buffer.

That's why so far, this optimization you're suggesting has not been
requested by anyone, even though modes such as (C)perl do have rather
complex/costly syntax-propertize-functions and are sometimes applied to
very large buffers as well.  IOW, I wouldn't worry about this problem
for now.  We'll cross this bridge when we get there.


        Stefan

Sync'ing cc-mode

[Stefan Monnier]

> What are my options: ask for your extensive test suite (which we did
> already a long time ago, and haven't seen it yet),

OK I finally noticed that it's included in the Hg repository of the
standalone CC-mode.  Talking about that standalone CC-mode: when were
the two last merged (ideally, what is the exact revision IDs (both int the Hg
repository and in Emacs's Bzr) corresponding to this sync)?


        Stefan

Re: Sync'ing cc-mode

[Stefan Monnier]

Hi Alan,

Find below a patch against Emacs's trunk which should bring Emacs's
trunk up-to-date w.r.t your standalone cc-mode Hg repository.

IOW I think that most of the remaining differences should either be
applied to your Hg repository, or kept as "differences between the two
versions".  Note that many/most of the differences are in whitespace, so
use "diff -bw" or something like that if you want to see the "real"
remaining changes.  I decided to keep the whitespace as it is in Emacs
rather than in your branch, except for some cases where our branch had
clearly some mis-indentations (which can be seen in big hunks in
cc-engine.el below, which merit checking with diff-refine-hunk to see
that they indeed only change whitespace).

Actually, there are also a few grey areas where I couldn't decide how to
merge the two: e.g. one side uses `eval-after-load' and the other
commented it out and I'm not sure exactly how they should be merged.

Guaranteed 99% untested (I can open a file in C, C++, Java, AWK modes,
but I didn't get to test much more than that).


        Stefan


PS: Most changes are just straightforward inclusion of new code from
your branch, but there are a few other details of note:
- rather than merge your cc-bytecomp-defmacro and cc-bytecomp-push
  thingies, I just removed all their uses, since there pretty much weren't
  any left anyway.
- I define c-subword-mode as an obsolete alias for subword-mode when
  subword-mode is available so as to try and share some code between the
  two cases.
- I use c--macroexpand-all to handle the fact that some emacsen
  don't have cl-macroexpand-all and some don't have macroexpand-all.
- I use c--make-local-hook to handle the fact that some emacsen require
  the use of make-local-hook whereas others have obsoleted it many years
  ago and removed it a few years ago.
- I reverted the Emacs code to use `cl' instead of `cl-lib' in cc-langs
  now that it's not loaded at runtime any more.
- When introducing c-unmark-<->-as-paren, you changed
  a (c-clear-char-property pos 'category) to (c-unmark-<->-as-paren
  (point)) and I assumed this was a copy&paste error.



=== modified file 'lisp/progmodes/cc-align.el'
--- lisp/progmodes/cc-align.el	2014-01-01 07:43:34 +0000
+++ lisp/progmodes/cc-align.el	2014-09-25 17:09:45 +0000
@@ -1229,6 +1229,18 @@
     (back-to-indentation)
     (vector (current-column))))
 
+(defun c-lineup-respect-col-0 (langelem)
+  "If the current line starts at column 0, return [0].	Otherwise return nil.
+
+This can be used for comments (in conjunction with, say,
+`c-lineup-comment'), to keep comments already at column 0
+anchored there, but reindent other comments."
+  (save-excursion
+    (back-to-indentation)
+    (if (eq (current-column) 0)
+	[0]
+      nil)))
+
 \f
 (defun c-snug-do-while (syntax pos)
   "Dynamically calculate brace hanginess for do-while statements.

=== modified file 'lisp/progmodes/cc-awk.el'
--- lisp/progmodes/cc-awk.el	2014-09-10 21:38:11 +0000
+++ lisp/progmodes/cc-awk.el	2014-09-25 17:09:33 +0000
@@ -61,7 +61,6 @@
 (cc-bytecomp-defun c-backward-token-1)
 (cc-bytecomp-defun c-beginning-of-statement-1)
 (cc-bytecomp-defun c-backward-sws)
-(cc-bytecomp-defun c-forward-sws)
 
 (defvar awk-mode-syntax-table
   (let ((st (make-syntax-table)))
@@ -174,6 +173,7 @@
   (concat "\"\\(" c-awk-string-ch-re "\\|" c-awk-esc-pair-re "\\)*"
 	  "\\(\"\\|$\\|\\'\\)"))
 
+
 ;; REGEXPS FOR AWK REGEXPS.
 (defconst c-awk-regexp-normal-re "[^[/\\\n\r]")
 ;;   Matches any AWK regexp character which doesn't require special analysis.
@@ -201,6 +201,7 @@
   (concat "/" c-awk-regexp-innards-re))
 ;; Matches an AWK regexp up to, but not including, any terminating /.
 
+
 ;; REGEXPS used for scanning an AWK buffer in order to decide IF A '/' IS A
 ;; REGEXP OPENER OR A DIVISION SIGN.  By "state" in the following is meant
 ;; whether a '/' at the current position would by a regexp opener or a

=== modified file 'lisp/progmodes/cc-bytecomp.el'
--- lisp/progmodes/cc-bytecomp.el	2014-01-01 07:43:34 +0000
+++ lisp/progmodes/cc-bytecomp.el	2014-09-25 21:30:55 +0000
@@ -65,8 +65,7 @@
 ;; elsewhere in the load path.
 ;;
 ;; To suppress byte compiler warnings, use the macros
-;; `cc-bytecomp-defun', `cc-bytecomp-defvar',
-;; `cc-bytecomp-obsolete-fun', and `cc-bytecomp-obsolete-var'.
+;; `cc-bytecomp-defun' and `cc-bytecomp-defvar'.
 ;;
 ;; This file is not used at all after the package has been byte
 ;; compiled.  It is however necessary when running uncompiled.
@@ -78,6 +77,12 @@
 (defvar cc-bytecomp-original-functions nil)
 (defvar cc-bytecomp-original-properties nil)
 (defvar cc-bytecomp-loaded-files nil)
+
+(setq cc-bytecomp-unbound-variables nil)
+(setq cc-bytecomp-original-functions nil)
+(setq cc-bytecomp-original-properties nil)
+(setq cc-bytecomp-loaded-files nil)
+
 (defvar cc-bytecomp-environment-set nil)
 
 (defmacro cc-bytecomp-debug-msg (&rest args)
@@ -370,33 +375,6 @@
       "cc-bytecomp-put: Bound property %s for %s to %s"
       ,propname ,symbol ,value)))
 
-(defmacro cc-bytecomp-obsolete-var (symbol)
-  "Suppress warnings that the given symbol is an obsolete variable.
-Don't use within `eval-when-compile'."
-  `(eval-when-compile
-     (if (get ',symbol 'byte-obsolete-variable)
-	 (cc-bytecomp-put ',symbol 'byte-obsolete-variable nil)
-       ;; This avoids a superfluous compiler warning
-       ;; about calling `get' for effect.
-       t)))
-
-(defun cc-bytecomp-ignore-obsolete (form)
-  ;; Wraps a call to `byte-compile-obsolete' that suppresses the warning.
-  (let ((byte-compile-warnings byte-compile-warnings))
-    (byte-compile-disable-warning 'obsolete)
-    (byte-compile-obsolete form)))
-
-(defmacro cc-bytecomp-obsolete-fun (symbol)
-  "Suppress warnings that the given symbol is an obsolete function.
-Don't use within `eval-when-compile'."
-  `(eval-when-compile
-     (if (eq (get ',symbol 'byte-compile) 'byte-compile-obsolete)
-	 (cc-bytecomp-put ',symbol 'byte-compile
-			  'cc-bytecomp-ignore-obsolete)
-       ;; This avoids a superfluous compiler warning
-       ;; about calling `get' for effect.
-       t)))
-
 (defmacro cc-bytecomp-boundp (symbol)
   "Return non-nil if the given symbol is bound as a variable outside
 the compilation.  This is the same as using `boundp' but additionally

=== modified file 'lisp/progmodes/cc-cmds.el'
--- lisp/progmodes/cc-cmds.el	2014-02-17 18:16:32 +0000
+++ lisp/progmodes/cc-cmds.el	2014-09-26 15:25:52 +0000
@@ -45,8 +45,11 @@
 (cc-require 'cc-engine)
 
 ;; Silence the compiler.
+(cc-bytecomp-defun delete-forward-p)	; XEmacs
 (cc-bytecomp-defvar filladapt-mode)	; c-fill-paragraph contains a kludge
 					; which looks at this.
+(cc-bytecomp-defun c-forward-subword)
+(cc-bytecomp-defun c-backward-subword)
 \f
 ;; Indentation / Display syntax functions
 (defvar c-fix-backslashes t)
@@ -258,9 +261,11 @@
 			 "a" "")
 		     (if c-hungry-delete-key "h" "")
 		     (if (and
-			  ;; subword might not be loaded.
-			  (boundp 'subword-mode)
-			  (symbol-value 'subword-mode))
+			  ;; (cc-)subword might not be loaded.
+			  (boundp 'c-subword-mode)
+			  (symbol-value 'c-subword-mode))
+                         ;; FIXME: subword-mode already comes with its
+                         ;; own lighter!
 			 "w"
 		       "")))
         ;; FIXME: Derived modes might want to use something else
@@ -936,8 +941,15 @@
 	;; do all cleanups and newline insertions if c-auto-newline is on.
 	(if (or (not c-auto-newline)
 		(not (looking-at "[ \t]*\\\\?$")))
-	    (if c-syntactic-indentation
-		(c-indent-line))
+	    (when c-syntactic-indentation
+		(let ((syntax (c-guess-basic-syntax)))
+		  (c-indent-line syntax)
+		  ;; Guard against the C hacker inserting a statement before a
+		  ;; non-compound statement in an if/while/for.
+		  (if (eq (caar syntax) 'substatement)
+		      (save-excursion
+			(if (eq 0 (forward-line))
+			    (c-indent-line))))))
 	  ;; clean ups: list-close-comma or defun-close-semi
 	  (let ((pos (- (point-max) (point))))
 	    (if (c-save-buffer-state ()
@@ -1090,7 +1102,7 @@
 
   (interactive "*P")
   (let ((c-echo-syntactic-information-p nil)
-	final-pos close-paren-inserted found-delim case-fold-search)
+	final-pos found-delim case-fold-search)
 
     (self-insert-command (prefix-numeric-value arg))
     (setq final-pos (point))
@@ -1303,20 +1315,43 @@
 (declare-function subword-forward "subword" (&optional arg))
 (declare-function subword-backward "subword" (&optional arg))
 
+(cond
+ ((and (fboundp 'subword-mode) (not (fboundp 'c-subword-mode)))
+  ;; Recent Emacsen come with their own subword support.  Use that.
+  (define-obsolete-function-alias 'c-subword-mode 'subword-mode "24.3")
+  (define-obsolete-variable-alias 'c-subword-mode 'subword-mode "24.3"))
+ (t
+  ;; Autoload directive for emacsen that doesn't have an older CC Mode
+  ;; version in the dist.
+  (autoload 'c-subword-mode "cc-subword"
+    "Mode enabling subword movement and editing keys." t)))
+
 ;; "nomenclature" functions + c-scope-operator.
 (defun c-forward-into-nomenclature (&optional arg)
   "Compatibility alias for `c-forward-subword'."
   (interactive "p")
-  (require 'subword)
-  (subword-forward arg))
-(make-obsolete 'c-forward-into-nomenclature 'subword-forward "23.2")
+  (if (fboundp 'subword-mode)
+      (progn
+        (require 'subword)
+        (subword-forward arg))
+    (require 'cc-subword)
+    (c-forward-subword arg)))
+(make-obsolete 'c-forward-into-nomenclature
+               (if (fboundp 'subword-mode) 'subword-forward 'c-forward-subword)
+               "23.2")
 
 (defun c-backward-into-nomenclature (&optional arg)
   "Compatibility alias for `c-backward-subword'."
   (interactive "p")
-  (require 'subword)
-  (subword-backward arg))
-(make-obsolete 'c-backward-into-nomenclature 'subword-backward "23.2")
+  (if (fboundp 'subword-mode)
+      (progn
+        (require 'subword)
+        (subword-backward arg))
+    (require 'cc-subword)
+    (c-forward-backword arg)))
+(make-obsolete
+ 'c-backward-into-nomenclature
+ (if (fboundp 'subword-mode) 'subword-backward 'c-backward-subword) "23.2")
 
 (defun c-scope-operator ()
   "Insert a double colon scope operator at point.
@@ -1408,18 +1443,20 @@
 	;; Kluge so that c-beginning-of-decl-1 won't go back if we're already
 	;; at a declaration.
 	(if (or (and (eolp) (not (eobp))) ; EOL is matched by "\\s>"
-		(not (looking-at
-"\\([;#]\\|\\'\\|\\s(\\|\\s)\\|\\s\"\\|\\s\\\\|\\s$\\|\\s<\\|\\s>\\|\\s!\\)")))
+		(not (c-looking-at-non-alphnumspace)))
 	    (forward-char))
 	(setq kluge-start (point))
 	(setq decl-result
 	      (car (c-beginning-of-decl-1
 		    ;; NOTE: If we're in a K&R region, this might be the start
 		    ;; of a parameter declaration, not the actual function.
+		    ;; It might also leave us at a label or "label" like
+		    ;; "private:".
 		    (and least-enclosing ; LIMIT for c-b-of-decl-1
 			 (c-safe-position least-enclosing paren-state)))))
 
 	;; Has the declaration we've gone back to got braces?
+	(or (eq decl-result 'label)
 	(setq brace-decl-p
 	      (save-excursion
 		    (and (c-syntactic-re-search-forward "[;{]" nil t t)
@@ -1429,10 +1466,11 @@
 				  ;; ';' in a K&R argdecl.  In
 				  ;; that case the declaration
 				  ;; should contain a block.
-				  (c-in-knr-argdecl))))))
+				  (c-in-knr-argdecl)))))))
 
 	(cond
-	 ((= (point) kluge-start)	; might be BOB or unbalanced parens.
+	 ((or (eq decl-result 'label)	; e.g. "private:" or invalid syntax.
+	      (= (point) kluge-start))	; might be BOB or unbalanced parens.
 	  'outwith-function)
 	 ((eq decl-result 'same)
 	  (if brace-decl-p
@@ -1808,7 +1846,7 @@
 	      (looking-at c-symbol-key))
 	    (match-string-no-properties 0))
 
-	   ((looking-at "DEFUN\\_>")
+	   ((looking-at "DEFUN\\s-*(") ;"DEFUN\\_>") think of XEmacs!
 	    ;; DEFUN ("file-name-directory", Ffile_name_directory, Sfile_name_directory, ...) ==> Ffile_name_directory
 	    ;; DEFUN(POSIX::STREAM-LOCK, stream lockp &key BLOCK SHARED START LENGTH) ==> POSIX::STREAM-LOCK
 	    (down-list 1)
@@ -2001,7 +2039,7 @@
 		   (eq last-command 'c-mark-function)))
 	     (push-mark-p (and (eq this-command 'c-mark-function)
 			       (not extend-region-p)
-			       (not (and transient-mark-mode mark-active)))))
+			       (not (c-region-is-active-p)))))
 	(if push-mark-p (push-mark (point)))
 	(if extend-region-p
 	    (progn
@@ -3338,7 +3376,7 @@
 starting on the current line.
 Otherwise reindent just the current line."
   (interactive
-   (list current-prefix-arg (use-region-p)))
+   (list current-prefix-arg (c-region-is-active-p)))
   (if region
       (c-indent-region (region-beginning) (region-end))
     (c-indent-command arg)))

=== modified file 'lisp/progmodes/cc-defs.el'
--- lisp/progmodes/cc-defs.el	2014-09-10 21:38:11 +0000
+++ lisp/progmodes/cc-defs.el	2014-09-26 15:12:06 +0000
@@ -48,12 +48,16 @@
 
 ;; Silence the compiler.
 (cc-bytecomp-defvar c-enable-xemacs-performance-kludge-p) ; In cc-vars.el
+(cc-bytecomp-defun buffer-syntactic-context-depth) ; XEmacs
 (cc-bytecomp-defun region-active-p)	; XEmacs
+(cc-bytecomp-defvar zmacs-region-stays)	; XEmacs
+(cc-bytecomp-defvar zmacs-regions)	; XEmacs
 (cc-bytecomp-defvar mark-active)	; Emacs
 (cc-bytecomp-defvar deactivate-mark)	; Emacs
 (cc-bytecomp-defvar inhibit-point-motion-hooks) ; Emacs
 (cc-bytecomp-defvar parse-sexp-lookup-properties) ; Emacs
 (cc-bytecomp-defvar text-property-default-nonsticky) ; Emacs 21
+(cc-bytecomp-defvar lookup-syntax-properties) ; XEmacs
 (cc-bytecomp-defun string-to-syntax)	; Emacs 21
 
 \f
@@ -64,15 +68,14 @@
 	  (not (fboundp 'push)))
       (cc-load "cc-fix")))
 
-; (eval-after-load "font-lock"  ; 2006-07-09.  font-lock is now preloaded
-;   '
-(if (and (featurep 'xemacs)	; There is now (2005/12) code in GNU Emacs CVS
+(when (featurep 'xemacs)  ; There is now (2005/12) code in GNU Emacs CVS
 				; to make the call to f-l-c-k throw an error.
-	 (not (featurep 'cc-fix)) ; only load the file once.
+  (eval-after-load "font-lock"
+    '(if (and (not (featurep 'cc-fix)) ; only load the file once.
 	 (let (font-lock-keywords)
 	   (font-lock-compile-keywords '("\\<\\>"))
 	   font-lock-keywords))     ; did the previous call foul this up?
-    (load "cc-fix")) ;)
+         (load "cc-fix"))))
 
 ;; The above takes care of the delayed loading, but this is necessary
 ;; to ensure correct byte compilation.
@@ -86,10 +89,15 @@
 	       font-lock-keywords)))
       (cc-load "cc-fix")))
 
+;; XEmacs 21.4 doesn't have `delete-dups'.
+(eval-and-compile
+  (if (and (not (fboundp 'delete-dups))
+	   (not (featurep 'cc-fix)))
+      (cc-load "cc-fix")))
 \f
 ;;; Variables also used at compile time.
 
-(defconst c-version "5.32.5"
+(defconst c-version "5.33"
   "CC Mode version number.")
 
 (defconst c-version-sym (intern c-version))
@@ -169,6 +177,10 @@
 
   (put 'cc-eval-when-compile 'lisp-indent-hook 0))
 
+(eval-and-compile
+  (defalias 'c--macroexpand-all
+    (if (fboundp 'macroexpand-all)
+        'macroexpand-all 'cl-macroexpand-all)))
 \f
 ;;; Macros.
 
@@ -327,21 +339,28 @@
 	  (t (error "Unknown buffer position requested: %s" position))))
        (point))))
 
+(defmacro c-next-single-property-change (position prop &optional object limit)
+  ;; See the doc string for either of the defuns expanded to.
+  (if (and (featurep 'xemacs)
+	   (fboundp 'next-single-char-property-change))
+      ;; XEmacs >= 2005-01-25
+      `(next-single-char-property-change ,position ,prop ,object ,limit)
+    ;; Emacs and earlier XEmacs
+    `(next-single-property-change ,position ,prop ,object ,limit)))
+
 (defmacro c-region-is-active-p ()
   ;; Return t when the region is active.  The determination of region
   ;; activeness is different in both Emacs and XEmacs.
-  ;; FIXME? Emacs has region-active-p since 23.1, so maybe this test
-  ;; should be updated.
-  (if (cc-bytecomp-boundp 'mark-active)
-      ;; Emacs.
-      'mark-active
+  (if (cc-bytecomp-fboundp 'region-active-p)
     ;; XEmacs.
-    '(region-active-p)))
+      '(region-active-p)
+    ;; Old Emacs.
+    'mark-active))
 
 (defmacro c-set-region-active (activate)
   ;; Activate the region if ACTIVE is non-nil, deactivate it
   ;; otherwise.  Covers the differences between Emacs and XEmacs.
-  (if (fboundp 'zmacs-activate-region)
+  (if (cc-bytecomp-fboundp 'zmacs-activate-region)
       ;; XEmacs.
       `(if ,activate
 	   (zmacs-activate-region)
@@ -438,6 +457,7 @@
 (put 'c-save-buffer-state 'lisp-indent-function 1)
 
 (defmacro c-tentative-buffer-changes (&rest body)
+  ;; FIXME: Use atomic-change-group!
   "Eval BODY and optionally restore the buffer contents to the state it
 was in before BODY.  Any changes are kept if the last form in BODY
 returns non-nil.  Otherwise it's undone using the undo facility, and
@@ -705,9 +725,9 @@
   ;; `c-parse-state'.
 
   `(progn
-     (if (and ,(fboundp 'buffer-syntactic-context-depth)
+     (if (and ,(cc-bytecomp-fboundp 'buffer-syntactic-context-depth)
 	      c-enable-xemacs-performance-kludge-p)
-	 ,(when (fboundp 'buffer-syntactic-context-depth)
+	 ,(when (cc-bytecomp-fboundp 'buffer-syntactic-context-depth)
 	    ;; XEmacs only.  This can improve the performance of
 	    ;; c-parse-state to between 3 and 60 times faster when
 	    ;; braces are hung.  It can also degrade performance by
@@ -913,6 +933,12 @@
 			       (cc-bytecomp-fboundp 'delete-extent)
 			       (cc-bytecomp-fboundp 'map-extents))))
 
+(defconst c-<-as-paren-syntax '(4 . ?>))
+(put 'c-<-as-paren-syntax 'syntax-table c-<-as-paren-syntax)
+
+(defconst c->-as-paren-syntax '(5 . ?<))
+(put 'c->-as-paren-syntax 'syntax-table c->-as-paren-syntax)
+
 ;; `c-put-char-property' is complex enough in XEmacs and Emacs < 21 to
 ;; make it a function.
 (defalias 'c-put-char-property-fun
@@ -966,6 +992,21 @@
     `(let ((-pos- ,pos))
        (put-text-property -pos- (1+ -pos-) ',property ,value))))
 
+(eval-and-compile
+  ;; Constant to decide at compilation time whether to use category
+  ;; properties.  Currently (2010-03) they're available only on GNU Emacs.
+  (defconst c-use-category
+    (with-temp-buffer
+      (let ((parse-sexp-lookup-properties t)
+	    (lookup-syntax-properties t))
+        (set-syntax-table (make-syntax-table))
+        (insert "<()>")
+        (c-put-char-property (point-min) 'category 'c-<-as-paren-syntax)
+        (c-put-char-property (+ 3 (point-min)) 'category 'c->-as-paren-syntax)
+        (goto-char (point-min))
+        (forward-sexp)
+        (= (point) (+ 4 (point-min)))))))
+
 (defmacro c-get-char-property (pos property)
   ;; Get the value of the given property on the character at POS if
   ;; it's been put there by `c-put-char-property'.  PROPERTY is
@@ -1046,8 +1087,8 @@
      (while
 	 (and
 	  (< place ,(or limit '(point-max)))
-	  (not (equal (get-text-property place ,property) ,value)))
-       (setq place (next-single-property-change
+	  (not (equal (c-get-char-property place ,property) ,value)))
+       (setq place (c-next-single-property-change
 		    place ,property nil ,(or limit '(point-max)))))
      (when (< place ,(or limit '(point-max)))
        (goto-char place)
@@ -1065,10 +1106,15 @@
      (while
 	 (and
 	  (> place ,(or limit '(point-min)))
-	  (not (equal (get-text-property (1- place) ,property) ,value)))
-       (setq place (previous-single-property-change
+	  (not (equal (c-get-char-property (1- place) ,property) ,value)))
+       (setq place (,(if (and (featurep 'xemacs)
+			      (fboundp 'previous-single-char-property-change))
+			 ;; XEmacs > 2005-01-25.
+			 'previous-single-char-property-change
+		       ;; Emacs and earlier XEmacs.
+		       'previous-single-property-change)
 		    place ,property nil ,(or limit '(point-min)))))
-     (when (> place ,(or limit '(point-max)))
+     (when (> place ,(or limit '(point-min)))
        (goto-char place)
        (search-backward-regexp ".")	; to set the match-data.
        (point))))
@@ -1085,9 +1131,9 @@
 	      (and
 	       (< place to)
 	       (not (equal (get-text-property place property) value)))
-	    (setq place (next-single-property-change place property nil to)))
+	    (setq place (c-next-single-property-change place property nil to)))
 	  (< place to))
-      (setq end-place (next-single-property-change place property nil to))
+      (setq end-place (c-next-single-property-change place property nil to))
       (remove-text-properties place end-place (cons property nil))
       ;; Do we have to do anything with stickiness here?
       (setq place end-place))))
@@ -1104,7 +1150,7 @@
 			(if (equal (extent-property ext -property-) val)
 			    (delete-extent ext)))
 		      nil ,from ,to ,value nil -property-))
-  ;; Gnu Emacs
+    ;; GNU Emacs
     `(c-clear-char-property-with-value-function ,from ,to ,property ,value)))
 \f
 ;; Macros to put overlays (Emacs) or extents (XEmacs) on buffer text.
@@ -1188,42 +1234,43 @@
     (if (< (point) start)
 	(goto-char (point-max)))))
 
-(defconst c-<-as-paren-syntax '(4 . ?>))
-(put 'c-<-as-paren-syntax 'syntax-table c-<-as-paren-syntax)
-
-(defsubst c-mark-<-as-paren (pos)
+(defmacro c-mark-<-as-paren (pos)
   ;; Mark the "<" character at POS as a template opener using the
-  ;; `syntax-table' property via the `category' property.
+  ;; `syntax-table' property either directly (XEmacs) or via a `category'
+  ;; property (GNU Emacs).
   ;;
   ;; This function does a hidden buffer change.  Note that we use
   ;; indirection through the `category' text property.  This allows us to
   ;; toggle the property in all template brackets simultaneously and
   ;; cheaply.  We use this, for instance, in `c-parse-state'.
-  (c-put-char-property pos 'category 'c-<-as-paren-syntax))
-
-(defconst c->-as-paren-syntax '(5 . ?<))
-(put 'c->-as-paren-syntax 'syntax-table c->-as-paren-syntax)
-
-(defsubst c-mark->-as-paren (pos)
+  (if c-use-category
+      `(c-put-char-property ,pos 'category 'c-<-as-paren-syntax)
+    `(c-put-char-property ,pos 'syntax-table c-<-as-paren-syntax)))
+
+
+(defmacro c-mark->-as-paren (pos)
   ;; Mark the ">" character at POS as an sexp list closer using the
-  ;; syntax-table property.
+  ;; `syntax-table' property either directly (XEmacs) or via a `category'
+  ;; property (GNU Emacs).
   ;;
   ;; This function does a hidden buffer change.  Note that we use
   ;; indirection through the `category' text property.  This allows us to
   ;; toggle the property in all template brackets simultaneously and
   ;; cheaply.  We use this, for instance, in `c-parse-state'.
-  (c-put-char-property pos 'category 'c->-as-paren-syntax))
+  (if c-use-category
+      `(c-put-char-property ,pos 'category 'c->-as-paren-syntax)
+    `(c-put-char-property ,pos 'syntax-table c->-as-paren-syntax)))
 
-(defsubst c-unmark-<->-as-paren (pos)
-  ;; Unmark the "<" or "<" character at POS as an sexp list opener using
-  ;; the syntax-table property indirectly through the `category' text
-  ;; property.
+(defmacro c-unmark-<->-as-paren (pos)
+  ;; Unmark the "<" or "<" character at POS as an sexp list opener using the
+  ;; `syntax-table' property either directly or indirectly through a
+  ;; `category' text property.
   ;;
-  ;; This function does a hidden buffer change.  Note that we use
+  ;; This function does a hidden buffer change.  Note that we try to use
   ;; indirection through the `category' text property.  This allows us to
   ;; toggle the property in all template brackets simultaneously and
   ;; cheaply.  We use this, for instance, in `c-parse-state'.
-  (c-clear-char-property pos 'category))
+  `(c-clear-char-property ,pos ,(if c-use-category ''category ''syntax-table)))
 
 (defsubst c-suppress-<->-as-parens ()
   ;; Suppress the syntactic effect of all marked < and > as parens.  Note
@@ -1304,6 +1351,122 @@
 	 (widen)
 	 (c-set-cpp-delimiters ,beg ,end)))))
 \f
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+;; The following macros are to be used only in `c-parse-state' and its
+;; subroutines.  Their main purpose is to simplify the handling of C++/Java
+;; template delimiters and CPP macros.  In GNU Emacs, this is done slickly by
+;; the judicious use of 'category properties.  These don't exist in XEmacs.
+;;
+;; Note: in the following macros, there is no special handling for parentheses
+;; inside CPP constructs.  That is because CPPs are always syntactically
+;; balanced, thanks to `c-neutralize-CPP-line' in cc-mode.el.
+(defmacro c-sc-scan-lists-no-category+1+1 (from)
+  ;; Do a (scan-lists FROM 1 1).  Any finishing position which either (i) is
+  ;; determined by and angle bracket; or (ii) is inside a macro whose start
+  ;; isn't POINT-MACRO-START doesn't count as a finishing position.
+  `(let ((here (point))
+	 (pos (scan-lists ,from 1 1)))
+     (while (eq (char-before pos) ?>)
+       (setq pos (scan-lists pos 1 1)))
+     pos))
+
+(defmacro c-sc-scan-lists-no-category+1-1 (from)
+  ;; Do a (scan-lists FROM 1 -1).  Any finishing position which either (i) is
+  ;; determined by an angle bracket; or (ii) is inside a macro whose start
+  ;; isn't POINT-MACRO-START doesn't count as a finishing position.
+  `(let ((here (point))
+	 (pos (scan-lists ,from 1 -1)))
+     (while (eq (char-before pos) ?<)
+       (setq pos (scan-lists pos 1 1))
+       (setq pos (scan-lists pos 1 -1)))
+     pos))
+
+(defmacro c-sc-scan-lists-no-category-1+1 (from)
+  ;; Do a (scan-lists FROM -1 1).  Any finishing position which either (i) is
+  ;; determined by and angle bracket; or (ii) is inside a macro whose start
+  ;; isn't POINT-MACRO-START doesn't count as a finishing position.
+  `(let ((here (point))
+	 (pos (scan-lists ,from -1 1)))
+     (while (eq (char-after pos) ?<)
+       (setq pos (scan-lists pos -1 1)))
+     pos))
+
+(defmacro c-sc-scan-lists-no-category-1-1 (from)
+  ;; Do a (scan-lists FROM -1 -1).  Any finishing position which either (i) is
+  ;; determined by and angle bracket; or (ii) is inside a macro whose start
+  ;; isn't POINT-MACRO-START doesn't count as a finishing position.
+  `(let ((here (point))
+	 (pos (scan-lists ,from -1 -1)))
+     (while (eq (char-after pos) ?>)
+       (setq pos (scan-lists pos -1 1))
+       (setq pos (scan-lists pos -1 -1)))
+     pos))
+
+(defmacro c-sc-scan-lists (from count depth)
+  (if c-use-category
+      `(scan-lists ,from ,count ,depth)
+    (cond
+     ((and (eq count 1) (eq depth 1))
+      `(c-sc-scan-lists-no-category+1+1 ,from))
+     ((and (eq count 1) (eq depth -1))
+      `(c-sc-scan-lists-no-category+1-1 ,from))
+     ((and (eq count -1) (eq depth 1))
+      `(c-sc-scan-lists-no-category-1+1 ,from))
+     ((and (eq count -1) (eq depth -1))
+      `(c-sc-scan-lists-no-category-1-1 ,from))
+     (t (error "Invalid parameter(s) to c-sc-scan-lists")))))
+
+
+(defun c-sc-parse-partial-sexp-no-category (from to targetdepth stopbefore
+						 oldstate)
+  ;; Do a parse-partial-sexp using the supplied arguments, disregarding
+  ;; template/generic delimiters < > and disregarding macros other than the
+  ;; one at POINT-MACRO-START.
+  ;;
+  ;; NOTE that STOPBEFORE must be nil.  TARGETDEPTH should be one less than
+  ;; the depth in OLDSTATE.  This function is thus a SPECIAL PURPOSE variation
+  ;; on parse-partial-sexp, designed for calling from
+  ;; `c-remove-stale-state-cache'.
+  ;;
+  ;; Any finishing position which is determined by an angle bracket delimiter
+  ;; doesn't count as a finishing position.
+  ;;
+  ;; Note there is no special handling of CPP constructs here, since these are
+  ;; always syntactically balanced (thanks to `c-neutralize-CPP-line').
+  (let ((state
+	 (parse-partial-sexp from to targetdepth stopbefore oldstate)))
+    (while
+	(and (< (point) to)
+	     ;; We must have hit targetdepth.
+	     (or (eq (char-before) ?<)
+		 (eq (char-before) ?>)))
+      (setcar state
+	      (if (memq (char-before) '(?> ?\) ?\} ?\]))
+		  (1+ (car state))
+		(1- (car state))))
+      (setq state
+	    (parse-partial-sexp (point) to targetdepth stopbefore oldstate)))
+    state))
+
+(defmacro c-sc-parse-partial-sexp (from to &optional targetdepth stopbefore
+					oldstate)
+  (if c-use-category
+      `(parse-partial-sexp ,from ,to ,targetdepth ,stopbefore ,oldstate)
+    `(c-sc-parse-partial-sexp-no-category ,from ,to ,targetdepth ,stopbefore
+					  ,oldstate)))
+
+\f
+(defmacro c-looking-at-non-alphnumspace ()
+  "Are we looking at a character which isn't alphanumeric or space?"
+  (if (memq 'gen-comment-delim c-emacs-features)
+      `(looking-at
+"\\([;#]\\|\\'\\|\\s(\\|\\s)\\|\\s\"\\|\\s\\\\|\\s$\\|\\s<\\|\\s>\\|\\s!\\)")
+    `(or (looking-at
+"\\([;#]\\|\\'\\|\\s(\\|\\s)\\|\\s\"\\|\\s\\\\|\\s$\\|\\s<\\|\\s>\\)"
+         (let ((prop (c-get-char-property (point) 'syntax-table)))
+	   (eq prop '(14)))))))		; '(14) is generic comment delimiter.
+
+\f
 (defsubst c-intersect-lists (list alist)
   ;; return the element of ALIST that matches the first element found
   ;; in LIST.  Uses assq.
@@ -1419,8 +1582,8 @@
 
 (defun c-make-keywords-re (adorn list &optional mode)
   "Make a regexp that matches all the strings the list.
-Duplicates and nil elements in the list are removed.  The resulting
-regexp may contain zero or more submatch expressions.
+Duplicates and nil elements in the list are removed.  The
+resulting regexp may contain zero or more submatch expressions.
 
 If ADORN is t there will be at least one submatch and the first
 surrounds the matched alternative, and the regexp will also not match
@@ -1438,11 +1601,7 @@
 when it's needed.  The default is the current language taken from
 `c-buffer-is-cc-mode'."
 
-  (let (unique)
-    (dolist (elt list)
-      (unless (member elt unique)
-	(push elt unique)))
-    (setq list (delete nil unique)))
+  (setq list (delete nil (delete-dups list)))
   (if list
       (let (re)
 
@@ -1556,7 +1715,11 @@
 
 (cc-bytecomp-defvar open-paren-in-column-0-is-defun-start)
 
-(defvar lookup-syntax-properties)       ;XEmacs.
+(defalias 'c--make-local-hook
+  (let ((tmphook (make-symbol "tmphook")))
+    ;; Check if `add-hook' makes the hook buffer-local.
+    (add-hook tmphook 'foo nil t)
+    (if (local-variable-p tmphook) 'ignore 'make-local-hook)))
 
 (defconst c-emacs-features
   (let (list)
@@ -1609,6 +1772,9 @@
 			       (not (end-of-defun))))
 	  (setq list (cons 'argumentative-bod-function list))))
 
+    ;; Record whether the `category' text property works.
+    (if c-use-category (setq list (cons 'category-properties list)))
+
     (let ((buf (generate-new-buffer " test"))
 	  parse-sexp-lookup-properties
 	  parse-sexp-ignore-comments
@@ -1638,13 +1804,13 @@
 		  "support for the `syntax-table' text property "
 		  "is required.")))
 
-	;; Find out if generic comment delimiters work.
+	;; Find out if "\\s!" (generic comment delimiters) work.
 	(c-safe
 	  (modify-syntax-entry ?x "!")
 	  (if (string-match "\\s!" "x")
 	      (setq list (cons 'gen-comment-delim list))))
 
-	;; Find out if generic string delimiters work.
+	;; Find out if "\\s|" (generic string delimiters) work.
 	(c-safe
 	  (modify-syntax-entry ?x "|")
 	  (if (string-match "\\s|" "x")
@@ -1691,7 +1857,8 @@
       (kill-buffer buf))
 
     ;; See if `parse-partial-sexp' returns the eighth element.
-    (if (c-safe (>= (length (save-excursion (parse-partial-sexp (point) (point))))
+    (if (c-safe (>= (length (save-excursion
+			      (parse-partial-sexp (point) (point))))
 		    10))
 	(setq list (cons 'pps-extended-state list))
       (error (concat
@@ -1707,13 +1874,14 @@
 
 '8-bit              8 bit syntax entry flags (XEmacs style).
 '1-bit              1 bit syntax entry flags (Emacs style).
-'argumentative-bod-function         beginning-of-defun passes ARG through
-                    to a non-null beginning-of-defun-function.  It is assumed
-		    the end-of-defun does the same thing.
+'argumentative-bod-function	    beginning-of-defun and end-of-defun pass
+		    ARG through to beginning/end-of-defun-function.
 'syntax-properties  It works to override the syntax for specific characters
 		    in the buffer with the 'syntax-table property.  It's
 		    always set - CC Mode no longer works in emacsen without
 		    this feature.
+'category-properties Syntax routines can add a level of indirection to text
+		    properties using the 'category property.
 'gen-comment-delim  Generic comment delimiters work
 		    (i.e. the syntax class `!').
 'gen-string-delim   Generic string delimiters work
@@ -1803,18 +1971,18 @@
     (error "Unknown base mode `%s'" base-mode))
   (put mode 'c-fallback-mode base-mode))
 
-(defvar c-lang-constants (make-vector 151 0)
-  "Obarray used as a cache to keep track of the language constants.
-The constants stored are those defined by `c-lang-defconst' and the values
-computed by `c-lang-const'.  It's mostly used at compile time but it's not
-stored in compiled files.
+(defvar c-lang-constants (make-vector 151 0))
+;;   Obarray used as a cache to keep track of the language constants.
+;; The constants stored are those defined by `c-lang-defconst' and the values
+;; computed by `c-lang-const'.  It's mostly used at compile time but it's not
+;; stored in compiled files.
 
-The obarray contains all the language constants as symbols.  The
-value cells hold the evaluated values as alists where each car is
-the mode name symbol and the corresponding cdr is the evaluated
-value in that mode.  The property lists hold the source definitions
-and other miscellaneous data.  The obarray might also contain
-various other symbols, but those don't have any variable bindings.")
+;; The obarray contains all the language constants as symbols.  The
+;; value cells hold the evaluated values as alists where each car is
+;; the mode name symbol and the corresponding cdr is the evaluated
+;; value in that mode.  The property lists hold the source definitions
+;; and other miscellaneous data.  The obarray might also contain
+;; various other symbols, but those don't have any variable bindings.
 
 (defvar c-lang-const-expansion nil)
 
@@ -1831,7 +1999,9 @@
 	       (t
 		;; Being evaluated interactively.
 		(buffer-file-name)))))
-    (and file (file-name-base file))))
+    (and file
+	 (file-name-sans-extension
+	  (file-name-nondirectory file)))))
 
 (defmacro c-lang-defconst-eval-immediately (form)
   "Can be used inside a VAL in `c-lang-defconst' to evaluate FORM
@@ -1945,7 +2115,7 @@
 	;; reason, but we also use this expansion handle
 	;; `c-lang-defconst-eval-immediately' and to register
 	;; dependencies on the `c-lang-const's in VAL.)
-	(setq val (macroexpand-all val))
+	(setq val (c--macroexpand-all val))
 
 	(setq bindings `(cons (cons ',assigned-mode (lambda () ,val)) ,bindings)
 	      args (cdr args))))
@@ -1963,10 +2133,9 @@
 			       ,@(and pre-files `(',pre-files))))))
 
 (put 'c-lang-defconst 'lisp-indent-function 1)
-;(eval-after-load "edebug" ; 2006-07-09: def-edebug-spec is now in subr.el.
-;  '
-(def-edebug-spec c-lang-defconst
-  (&define name [&optional stringp] [&rest sexp def-form]))
+(eval-after-load "edebug"
+  '(def-edebug-spec c-lang-defconst
+     (&define name [&optional stringp] [&rest sexp def-form])))
 
 (defun c-define-lang-constant (name bindings &optional pre-files)
   ;; Used by `c-lang-defconst'.

=== modified file 'lisp/progmodes/cc-engine.el'
--- lisp/progmodes/cc-engine.el	2014-09-11 19:44:25 +0000
+++ lisp/progmodes/cc-engine.el	2014-09-25 20:11:21 +0000
@@ -147,18 +147,24 @@
 (cc-require-when-compile 'cc-langs)
 (cc-require 'cc-vars)
 
+(eval-when-compile (require 'cl))
+
+;; Silence the compiler.
+(cc-bytecomp-defun buffer-syntactic-context) ; XEmacs
+(cc-bytecomp-defun c-fontify-recorded-types-and-refs)
+(cc-bytecomp-defvar c-maybe-stale-found-type)
+
 \f
 ;; Make declarations for all the `c-lang-defvar' variables in cc-langs.
 
 (defmacro c-declare-lang-variables ()
   `(progn
-     ,@(apply 'nconc
-	      (mapcar (lambda (init)
-			`(,(if (elt init 2)
-			       `(defvar ,(car init) nil ,(elt init 2))
-			     `(defvar ,(car init) nil))
-			  (make-variable-buffer-local ',(car init))))
-		      (cdr c-lang-variable-inits)))))
+     ,@(mapcan (lambda (init)
+		 `(,(if (elt init 2)
+			`(defvar ,(car init) nil ,(elt init 2))
+		      `(defvar ,(car init) nil))
+		   (make-variable-buffer-local ',(car init))))
+	       (cdr c-lang-variable-inits))))
 (c-declare-lang-variables)
 
 \f
@@ -514,7 +520,7 @@
     (while (progn
 	     (when (eq (get-text-property (point) 'c-type) value)
 	       (c-clear-char-property (point) 'c-type))
-	     (goto-char (next-single-property-change (point) 'c-type nil to))
+	     (goto-char (c-next-single-property-change (point) 'c-type nil to))
 	     (< (point) to)))))
 
 \f
@@ -826,7 +832,6 @@
 	    ;; Record this as the first token if not starting inside it.
 	    (setq tok start))
 
-
 	;; The following while loop goes back one sexp (balanced parens,
 	;; etc. with contents, or symbol or suchlike) each iteration.  This
 	;; movement is accomplished with a call to c-backward-sexp approx 170
@@ -1370,8 +1375,9 @@
 	;; Emacs includes the ending newline in a b-style (c++)
 	;; comment, but XEmacs doesn't.  We depend on the Emacs
 	;; behavior (which also is symmetric).
-	(if (and (eolp) (elt (parse-partial-sexp start (point)) 7))
-	    (condition-case nil (forward-char 1)))
+        (when (featurep 'xemacs)
+          (if (and (eolp) (elt (parse-partial-sexp start (point)) 7))
+              (condition-case nil (forward-char 1))))
 
 	t))))
 
@@ -1704,7 +1710,7 @@
 	  ;; the cases when the marked rung is complete.
 	  ;; (`next-single-property-change' is certain to move at least one
 	  ;; step forward.)
-	  (setq rung-pos (1- (next-single-property-change
+	  (setq rung-pos (1- (c-next-single-property-change
 			      rung-is-marked 'c-is-sws nil rung-end-pos)))
 	;; Got no marked rung here.  Since the simple ws might have started
 	;; inside a line comment or cpp directive we must set `rung-pos' as
@@ -1720,7 +1726,7 @@
 
 		  ;; The following search is the main reason that `c-in-sws'
 		  ;; and `c-is-sws' aren't combined to one property.
-		  (goto-char (next-single-property-change
+		  (goto-char (c-next-single-property-change
 			      (point) 'c-in-sws nil (point-max)))
 		  (unless (get-text-property (point) 'c-is-sws)
 		    ;; If the `c-in-sws' region extended past the last
@@ -1842,7 +1848,7 @@
 	  ;; possible since we can't be in the ending ws of a line comment or
 	  ;; cpp directive now.
 	  (if (setq rung-is-marked next-rung-is-marked)
-	      (setq rung-pos (1- (next-single-property-change
+	      (setq rung-pos (1- (c-next-single-property-change
 				  rung-is-marked 'c-is-sws nil rung-end-pos)))
 	    (setq rung-pos next-rung-pos))
 	  (setq safe-start t)))
@@ -1920,7 +1926,7 @@
 		  (unless (get-text-property (point) 'c-is-sws)
 		    ;; If the `c-in-sws' region extended past the first
 		    ;; `c-is-sws' char we have to go forward a bit.
-		    (goto-char (next-single-property-change
+		    (goto-char (c-next-single-property-change
 				(point) 'c-is-sws)))
 
 		  (c-debug-sws-msg
@@ -2159,7 +2165,6 @@
 ;; the middle of the desert, as long as it is not within a brace pair
 ;; recorded in `c-state-cache' or a paren/bracket pair.
 
-
 ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
 ;; We maintain a simple cache of positions which aren't in a literal, so as to
 ;; speed up testing for non-literality.
@@ -2530,7 +2535,7 @@
 	(setq pos here+)
 	(c-safe
 	  (while
-	      (setq ren+1 (scan-lists pos 1 1)) ; might signal
+	      (setq ren+1 (c-sc-scan-lists pos 1 1)) ; might signal
 	    (setq lonely-rens (cons ren+1 lonely-rens)
 		  pos ren+1)))))
 
@@ -2542,7 +2547,7 @@
       (c-safe
 	(while
 	    (and lonely-rens		; actual values aren't used.
-		 (setq pa (scan-lists pos -1 1)))
+		 (setq pa (c-sc-scan-lists pos -1 1)))
 	  (setq pos pa)
 	  (setq lonely-rens (cdr lonely-rens)))))
     pos))
@@ -2698,8 +2703,8 @@
 	      (progn
 		(c-safe
 		  (while
-		      (and (setq ce (scan-lists bra -1 -1)) ; back past )/]/}; might signal
-			   (setq bra (scan-lists ce -1 1)) ; back past (/[/{; might signal
+		      (and (setq ce (c-sc-scan-lists bra -1 -1)) ; back past )/]/}; might signal
+			   (setq bra (c-sc-scan-lists ce -1 1)) ; back past (/[/{; might signal
 			   (or (> bra here) ;(> ce here)
 			       (and
 				(< ce here)
@@ -2751,7 +2756,7 @@
 		     (not (c-beginning-of-macro))))
 	  (setq c-state-cache
 		(cons (cons (1- bra+1)
-			    (scan-lists bra+1 1 1))
+			    (c-sc-scan-lists bra+1 1 1))
 		      (if (consp (car c-state-cache))
 			  (cdr c-state-cache)
 			c-state-cache)))
@@ -2780,7 +2785,7 @@
 	paren+1		      ; Pos after some opening or closing paren.
 	paren+1s	      ; A list of `paren+1's; used to determine a
 			      ; good-pos.
-	bra+1 ce+1	      ; just after L/R bra-ces.
+	bra+1		      ; just after L bra-ce.
 	bra+1s		      ; list of OLD values of bra+1.
 	mstart)		      ; start of a macro.
 
@@ -2801,9 +2806,9 @@
 	      ;; are no more b/b/p's to scan.
 	      (c-safe
 		(while t
-		  (setq pa+1 (scan-lists ren+1 1 -1) ; Into (/{/[; might signal
+		  (setq pa+1 (c-sc-scan-lists ren+1 1 -1) ; Into (/{/[; might signal
 			paren+1s (cons pa+1 paren+1s))
-		  (setq ren+1 (scan-lists pa+1 1 1)) ; Out of )/}/]; might signal
+		  (setq ren+1 (c-sc-scan-lists pa+1 1 1)) ; Out of )/}/]; might signal
 		  (if (and (eq (char-before pa+1) ?{)) ; Check for a macro later.
 		      (setq bra+1 pa+1))
 		  (setcar paren+1s ren+1)))
@@ -2827,7 +2832,7 @@
 		;; finished - we just need to check for having found an
 		;; unmatched )/}/], which we ignore.  Such a )/}/] can't be in a
 		;; macro, due the action of `c-neutralize-syntax-in-CPP'.
-		(c-safe (setq ren+1 (scan-lists ren+1 1 1)))))) ; acts as loop control.
+		(c-safe (setq ren+1 (c-sc-scan-lists ren+1 1 1)))))) ; acts as loop control.
 
 	;; Record the final, innermost, brace-pair if there is one.
 	(c-state-push-any-brace-pair bra+1 macro-start-or-here)
@@ -2954,7 +2959,7 @@
 
 	  ;; Scan!
 	  (setq pps-state
-		(parse-partial-sexp
+		(c-sc-parse-partial-sexp
 		 (point) (if (< (point) pps-point) pps-point here)
 		 target-depth
 		 nil pps-state))
@@ -2985,9 +2990,10 @@
 	     )))
 
 	(if (< (point) pps-point)
-	    (setq pps-state (parse-partial-sexp (point) pps-point
-						nil nil ; TARGETDEPTH, STOPBEFORE
-						pps-state)))
+	    (setq pps-state (c-sc-parse-partial-sexp
+			     (point) pps-point
+			     nil nil ; TARGETDEPTH, STOPBEFORE
+			     pps-state)))
 
 	;; If the last paren pair we moved out of was actually a brace pair,
 	;; insert it into `c-state-cache'.
@@ -3108,12 +3114,15 @@
 	(save-restriction
 	  (narrow-to-region here-bol (point-max))
 	  (setq pos here-lit-start)
-	  (c-safe (while (setq pa (scan-lists pos -1 1))
+	  (c-safe (while (setq pa (c-sc-scan-lists pos -1 1))
 		    (setq pos pa))))	; might signal
 	nil))				; for the cond
 
-     ((setq ren (c-safe-scan-lists pos -1 -1 too-far-back))
-       ;; CASE 3: After a }/)/] before `here''s BOL.
+     ((save-restriction
+        (narrow-to-region too-far-back (point-max))
+        (setq ren (c-safe (c-sc-scan-lists pos -1 -1))))
+
+      ;; CASE 3: After a }/)/] before `here''s BOL.
       (list (1+ ren) (and dropped-cons pos) nil)) ; Return value
 
      (t
@@ -3335,15 +3344,19 @@
   ;; of all parens in preprocessor constructs, except for any such construct
   ;; containing point.  We can then call `c-invalidate-state-cache-1' without
   ;; worrying further about macros and template delimiters.
-  (c-with-<->-as-parens-suppressed
-   (if (and c-state-old-cpp-beg
-	    (< c-state-old-cpp-beg here))
-       (c-with-all-but-one-cpps-commented-out
-	c-state-old-cpp-beg
-	(min c-state-old-cpp-end here)
-	(c-invalidate-state-cache-1 here))
-     (c-with-cpps-commented-out
-      (c-invalidate-state-cache-1 here)))))
+  (if (eval-when-compile (memq 'category-properties c-emacs-features))
+      ;; Emacs
+      (c-with-<->-as-parens-suppressed
+       (if (and c-state-old-cpp-beg
+		(< c-state-old-cpp-beg here))
+	   (c-with-all-but-one-cpps-commented-out
+	    c-state-old-cpp-beg
+	    (min c-state-old-cpp-end here)
+	    (c-invalidate-state-cache-1 here))
+	 (c-with-cpps-commented-out
+	  (c-invalidate-state-cache-1 here))))
+    ;; XEmacs
+    (c-invalidate-state-cache-1 here)))
 
 (defmacro c-state-maybe-marker (place marker)
   ;; If PLACE is non-nil, return a marker marking it, otherwise nil.
@@ -3371,13 +3384,17 @@
     ;; FIXME!!! Put in a `condition-case' here to protect the integrity of the
     ;; subsystem.
     (prog1
-	(c-with-<->-as-parens-suppressed
-	 (if (and here-cpp-beg (> here-cpp-end here-cpp-beg))
-	     (c-with-all-but-one-cpps-commented-out
-	      here-cpp-beg here-cpp-end
-	      (c-parse-state-1))
-	   (c-with-cpps-commented-out
-	    (c-parse-state-1))))
+	(if (eval-when-compile (memq 'category-properties c-emacs-features))
+	    ;; Emacs
+	    (c-with-<->-as-parens-suppressed
+	     (if (and here-cpp-beg (> here-cpp-end here-cpp-beg))
+		 (c-with-all-but-one-cpps-commented-out
+		  here-cpp-beg here-cpp-end
+		  (c-parse-state-1))
+	       (c-with-cpps-commented-out
+		(c-parse-state-1))))
+	  ;; XEmacs
+	  (c-parse-state-1))
       (setq c-state-old-cpp-beg
 	    (c-state-maybe-marker here-cpp-beg c-state-old-cpp-beg-marker)
 	    c-state-old-cpp-end
@@ -3392,6 +3409,7 @@
 
 (defvar c-parse-state-point nil)
 (defvar c-parse-state-state nil)
+(make-variable-buffer-local 'c-parse-state-state)
 (defun c-record-parse-state-state ()
   (setq c-parse-state-point (point))
   (setq c-parse-state-state
@@ -3399,9 +3417,9 @@
 	 (lambda (arg)
 	   (let ((val (symbol-value arg)))
 	     (cons arg
-		   (if (consp val)
-		       (copy-tree val)
-		     val))))
+		   (cond ((consp val) (copy-tree val))
+			 ((markerp val) (copy-marker val))
+			 (t val)))))
 	 '(c-state-cache
 	   c-state-cache-good-pos
 	   c-state-nonlit-pos-cache
@@ -3421,7 +3439,11 @@
    (concat "(setq "
     (mapconcat
      (lambda (arg)
-       (format "%s %s%s" (car arg) (if (atom (cdr arg)) "" "'") (cdr arg)))
+       (format "%s %s%s" (car arg)
+	       (if (atom (cdr arg)) "" "'")
+	       (if (markerp (cdr arg))
+		   (format "(copy-marker %s)" (marker-position (cdr arg)))
+		 (cdr arg))))
      c-parse-state-state "  ")
     ")")))
 
@@ -4156,8 +4178,7 @@
 	(goto-char bound))
       nil)))
 
-(defvar safe-pos-list)		  ; bound in c-syntactic-skip-backward
-
+(cc-bytecomp-defvar safe-pos-list)      ; bound in c-syntactic-skip-backward
 (defsubst c-ssb-lit-begin ()
   ;; Return the start of the literal point is in, or nil.
   ;; We read and write the variables `safe-pos', `safe-pos-list', `state'
@@ -4166,7 +4187,7 @@
   ;; Use `parse-partial-sexp' from a safe position down to the point to check
   ;; if it's outside comments and strings.
   (save-excursion
-    (let ((pos (point)) safe-pos state pps-end-pos)
+    (let ((pos (point)) safe-pos state)
       ;; Pick a safe position as close to the point as possible.
       ;;
       ;; FIXME: Consult `syntax-ppss' here if our cache doesn't give a good
@@ -4765,7 +4786,7 @@
      (unless cfd-prop-match
        (save-excursion
 	 (while (progn
-		  (goto-char (next-single-property-change
+		  (goto-char (c-next-single-property-change
 			      (point) 'c-type nil cfd-limit))
 		  (and (< (point) cfd-limit)
 		       (not (eq (c-get-char-property (1- (point)) 'c-type)
@@ -4805,7 +4826,7 @@
 	       ;; Pseudo match inside a comment or string literal.  Skip out
 	       ;; of comments and string literals.
 	       (while (progn
-			(goto-char (next-single-property-change
+			(goto-char (c-next-single-property-change
 				    (point) 'face nil cfd-limit))
 			(and (< (point) cfd-limit)
 			     (c-got-face-at (point) c-literal-faces))))
@@ -4860,14 +4881,17 @@
   ;; it should return non-nil to ensure that the next search will find them.
   ;;
   ;; Such a spot is:
-  ;; o  The first token after bob.
-  ;; o  The first token after the end of submatch 1 in
-  ;;    `c-decl-prefix-or-start-re' when that submatch matches.
-  ;; o  The start of each `c-decl-prefix-or-start-re' match when
-  ;;    submatch 1 doesn't match.
-  ;; o  The first token after the end of each occurrence of the
-  ;;    `c-type' text property with the value `c-decl-end', provided
-  ;;    `c-type-decl-end-used' is set.
+  ;; o	The first token after bob.
+  ;; o	The first token after the end of submatch 1 in
+  ;;	`c-decl-prefix-or-start-re' when that submatch matches.	 This
+  ;;	submatch is typically a (L or R) brace or paren, a ;, or a ,.
+  ;; o	The start of each `c-decl-prefix-or-start-re' match when
+  ;;	submatch 1 doesn't match.  This is, for example, the keyword
+  ;;	"class" in Pike.
+  ;; o	The start of a previously recognized declaration; "recognized"
+  ;;	means that the last char of the previous token has a `c-type'
+  ;;	text property with the value `c-decl-end'; this only holds
+  ;;	when `c-type-decl-end-used' is set.
   ;;
   ;; Only a spot that match CFD-DECL-RE and whose face is in the
   ;; CFD-FACE-CHECKLIST list causes CFD-FUN to be called.  The face
@@ -4899,7 +4923,7 @@
   ;;
   ;; This function might do hidden buffer changes.
 
-  (let ((cfd-start-pos (point))
+  (let ((cfd-start-pos (point))		; never changed
 	(cfd-buffer-end (point-max))
 	;; The end of the token preceding the decl spot last found
 	;; with `c-decl-prefix-or-start-re'.  `cfd-limit' if there's
@@ -4913,7 +4937,16 @@
 	;; `c-find-decl-prefix-search'.  `cfd-limit' if there's no
 	;; match.  If searching for the property isn't needed then we
 	;; disable it by setting it to `cfd-limit' directly.
-	(cfd-prop-match (unless c-type-decl-end-used cfd-limit))
+	(cfd-prop-match
+	 (if c-type-decl-end-used
+	     (save-excursion
+	       (c-backward-syntactic-ws)
+	       (and
+		(not (bobp))
+		(eq (c-get-char-property (1- (point)) 'c-type)
+		    'c-decl-end)
+		(point)))
+	   cfd-limit))
 	;; The end of the token preceding the decl spot last found by
 	;; `c-find-decl-prefix-search'.  0 for the implicit match at
 	;; bob.  `cfd-limit' if there's no match.  In other words,
@@ -4938,10 +4971,20 @@
       ;; statement or declaration, which is earlier than the first
       ;; returned match.
 
+      ;; This `cond' moves back over any literals or macros.  It has special
+      ;; handling for when the region being searched is entirely within a
+      ;; macro.  It sets `cfd-continue-pos' (unless we've reached
+      ;; `cfd-limit').
       (cond
        ;; First we need to move to a syntactically relevant position.
        ;; Begin by backing out of comment or string literals.
+       ;;
+       ;; This arm of the cond actually triggers if we're in a literal,
+       ;; and cfd-limit is at most at BONL.
        ((and
+	 ;; This arm of the `and' moves backwards out of a literal when
+	 ;; the face at point is a literal face.  In this case, its value
+	 ;; is always non-nil.
 	 (when (c-got-face-at (point) c-literal-faces)
 	   ;; Try to use the faces to back up to the start of the
 	   ;; literal.  FIXME: What if the point is on a declaration
@@ -4970,7 +5013,7 @@
 	     (let ((range (c-literal-limits)))
 	       (if range (goto-char (car range)))))
 
-	   (setq start-in-literal (point)))
+	   (setq start-in-literal (point))) ; end of `and' arm.
 
 	 ;; The start is in a literal.  If the limit is in the same
 	 ;; one we don't have to find a syntactic position etc.  We
@@ -4981,22 +5024,22 @@
 	 (save-excursion
 	   (goto-char cfd-start-pos)
 	   (while (progn
-		    (goto-char (next-single-property-change
+		    (goto-char (c-next-single-property-change
 				(point) 'face nil cfd-limit))
 		    (and (< (point) cfd-limit)
 			 (c-got-face-at (point) c-literal-faces))))
-	   (= (point) cfd-limit)))
+	   (= (point) cfd-limit)))	; end of `cond' arm condition
 
 	;; Completely inside a literal.  Set up variables to trig the
 	;; (< cfd-continue-pos cfd-start-pos) case below and it'll
 	;; find a suitable start position.
-	(setq cfd-continue-pos start-in-literal))
+	(setq cfd-continue-pos start-in-literal)) ; end of `cond' arm
 
        ;; Check if the region might be completely inside a macro, to
        ;; optimize that like the completely-inside-literal above.
        ((save-excursion
 	  (and (= (forward-line 1) 0)
-	       (bolp)			; forward-line has funny behavior at eob.
+	       (bolp)                 ; forward-line has funny behavior at eob.
 	       (>= (point) cfd-limit)
 	       (progn (backward-char)
 		      (eq (char-before) ?\\))))
@@ -5006,6 +5049,8 @@
 	(setq cfd-continue-pos (1- cfd-start-pos)
 	      start-in-macro t))
 
+       ;; The default arm of the `cond' moves back over any macro we're in
+       ;; and over any syntactic WS.  It sets `c-find-decl-syntactic-pos'.
        (t
 	;; Back out of any macro so we don't miss any declaration
 	;; that could follow after it.
@@ -5052,10 +5097,10 @@
 		  (< (point) cfd-limit))
 	    ;; Do an initial search now.  In the bob case above it's
 	    ;; only done to search for a `c-decl-end' spot.
-	    (c-find-decl-prefix-search))
+	    (c-find-decl-prefix-search)) ; sets cfd-continue-pos
 
 	  (setq c-find-decl-match-pos (and (< cfd-match-pos cfd-start-pos)
-					   cfd-match-pos)))))
+					   cfd-match-pos))))) ; end of `cond'
 
       ;; Advance `cfd-continue-pos' if it's before the start position.
       ;; The closest continue position that might have effect at or
@@ -5114,7 +5159,7 @@
 	  ;; `cfd-match-pos' so we can continue at the start position.
 	  ;; (Note that we don't get here if the first match is below
 	  ;; it.)
-	  (goto-char cfd-start-pos)))
+	  (goto-char cfd-start-pos)))	; end of `cond'
 
 	;; Delete found matches if they are before our new continue
 	;; position, so that `c-find-decl-prefix-search' won't back up
@@ -5123,7 +5168,7 @@
 	(when (and cfd-re-match (< cfd-re-match cfd-continue-pos))
 	  (setq cfd-re-match nil))
 	(when (and cfd-prop-match (< cfd-prop-match cfd-continue-pos))
-	  (setq cfd-prop-match nil)))
+	  (setq cfd-prop-match nil)))	; end of `when'
 
       (if syntactic-pos
 	  ;; This is the normal case and we got a proper syntactic
@@ -5144,9 +5189,10 @@
 	;; good start position for the search, so do it.
 	(c-find-decl-prefix-search)))
 
-    ;; Now loop.  Round what?  (ACM, 2006/7/5).  We already got the first match.
-
+    ;; Now loop, one decl spot per iteration.  We already have the first
+    ;; match in `cfd-match-pos'.
     (while (progn
+	     ;; Go foward over "false matches", one per iteration.
 	     (while (and
 		     (< cfd-match-pos cfd-limit)
 
@@ -5187,10 +5233,10 @@
 			  (goto-char cfd-continue-pos)
 			  t)))
 
-		     (< (point) cfd-limit))
-	       (c-find-decl-prefix-search))
+		     (< (point) cfd-limit)) ; end of "false matches" condition
+	       (c-find-decl-prefix-search)) ; end of "false matches" loop
 
-	     (< (point) cfd-limit))
+	     (< (point) cfd-limit))   ; end of condition for "decl-spot" while
 
       (when (and
 	     (>= (point) cfd-start-pos)
@@ -5218,7 +5264,7 @@
 		   ;; The matched token was the last thing in the macro,
 		   ;; so the whole match is bogus.
 		   (setq cfd-macro-end 0)
-		   nil))))
+		   nil))))		; end of when condition
 
 	(c-debug-put-decl-spot-faces cfd-match-pos (point))
 	(if (funcall cfd-fun cfd-match-pos (/= cfd-macro-end 0))
@@ -5398,8 +5444,8 @@
 	(c-go-list-forward))
       (when (equal (c-get-char-property (1- (point)) 'syntax-table)
 		   c->-as-paren-syntax) ; should always be true.
-	(c-clear-char-property (1- (point)) 'category))
-      (c-clear-char-property pos 'category))))
+	(c-unmark-<->-as-paren (1- (point))))
+      (c-unmark-<->-as-paren pos))))
 
 (defun c-clear->-pair-props (&optional pos)
   ;; POS (default point) is at a > character.  If it is marked with
@@ -5415,8 +5461,8 @@
 	(c-go-up-list-backward))
       (when (equal (c-get-char-property (point) 'syntax-table)
 			c-<-as-paren-syntax) ; should always be true.
-	(c-clear-char-property (point) 'category))
-      (c-clear-char-property pos 'category))))
+	(c-unmark-<->-as-paren (point)))
+      (c-unmark-<->-as-paren pos))))
 
 (defun c-clear-<>-pair-props (&optional pos)
   ;; POS (default point) is at a < or > character.  If it has an
@@ -5505,9 +5551,10 @@
       (c-syntactic-skip-backward "^;{}" (c-determine-limit 512))
       (setq new-beg (point))
 
-      ;; Remove the syntax-table properties from each pertinent <...> pair.
-      ;; Firsly, the ones with the < before beg and > after beg.
-      (while (c-search-forward-char-property 'category 'c-<-as-paren-syntax beg)
+      ;; Remove the syntax-table/category properties from each pertinent <...>
+      ;; pair.  Firsly, the ones with the < before beg and > after beg.
+      (while
+	  (c-search-forward-char-property 'syntax-table c-<-as-paren-syntax beg)
 	(if (c-clear-<-pair-props-if-match-after beg (1- (point)))
 	    (setq need-new-beg t)))
 
@@ -5518,7 +5565,7 @@
 
       ;; Remove syntax-table properties from the remaining pertinent <...>
       ;; pairs, those with a > after end and < before end.
-      (while (c-search-backward-char-property 'category 'c->-as-paren-syntax end)
+      (while (c-search-backward-char-property 'syntax-table c->-as-paren-syntax end)
 	(if (c-clear->-pair-props-if-match-before end)
 	    (setq need-new-end t)))
 
@@ -5531,8 +5578,6 @@
       (when need-new-end
 	(and (> new-end c-new-END) (setq c-new-END new-end))))))
 
-
-
 (defun c-after-change-check-<>-operators (beg end)
   ;; This is called from `after-change-functions' when
   ;; c-recognize-<>-arglists' is set.  It ensures that no "<" or ">"
@@ -5867,7 +5912,6 @@
   ;; Recursive part of `c-forward-<>-arglist'.
   ;;
   ;; This function might do hidden buffer changes.
-
   (let ((start (point)) res pos tmp
 	;; Cover this so that any recorded found type ranges are
 	;; automatically lost if it turns out to not be an angle
@@ -5903,32 +5947,31 @@
 	(while (and
 		(progn
 		  (c-forward-syntactic-ws)
-		  (let ((orig-record-found-types c-record-found-types))
-		    (when (or (and c-record-type-identifiers all-types)
-			      (c-major-mode-is 'java-mode))
-		      ;; All encountered identifiers are types, so set the
-		      ;; promote flag and parse the type.
-		      (progn
-			(c-forward-syntactic-ws)
-			(if (looking-at "\\?")
-			    (forward-char)
-			  (when (looking-at c-identifier-start)
-			    (let ((c-promote-possible-types t)
-				  (c-record-found-types t))
-			      (c-forward-type))))
-
-			(c-forward-syntactic-ws)
-
-			(when (or (looking-at "extends")
-				  (looking-at "super"))
-			  (forward-word)
-			  (c-forward-syntactic-ws)
+		  (when (or (and c-record-type-identifiers all-types)
+			    (c-major-mode-is 'java-mode))
+		    ;; All encountered identifiers are types, so set the
+		    ;; promote flag and parse the type.
+		    (progn
+		      (c-forward-syntactic-ws)
+		      (if (looking-at "\\?")
+			  (forward-char)
+			(when (looking-at c-identifier-start)
 			  (let ((c-promote-possible-types t)
 				(c-record-found-types t))
-			    (c-forward-type)
-			    (c-forward-syntactic-ws))))))
-
-		  (setq pos (point))    ; e.g. first token inside the '<'
+			    (c-forward-type))))
+
+		      (c-forward-syntactic-ws)
+
+		      (when (or (looking-at "extends")
+				(looking-at "super"))
+			(forward-word)
+			(c-forward-syntactic-ws)
+			(let ((c-promote-possible-types t)
+			      (c-record-found-types t))
+			  (c-forward-type)
+			  (c-forward-syntactic-ws)))))
+
+		  (setq pos (point))	; e.g. first token inside the '<'
 
 		  ;; Note: These regexps exploit the match order in \| so
 		  ;; that "<>" is matched by "<" rather than "[^>:-]>".
@@ -5993,7 +6036,6 @@
 				       (c-keyword-member
 					(c-keyword-sym (match-string 1))
 					'c-<>-type-kwds)))))))
-
 		      ;; It was an angle bracket arglist.
 		      (setq c-record-found-types subres)
 
@@ -6449,6 +6491,7 @@
 	  (goto-char (match-end 1))
 	  (c-forward-syntactic-ws)
 	  (setq res t)))
+
       ;; Step over any type suffix operator.  Do not let the existence
       ;; of these alter the classification of the found type, since
       ;; these operators typically are allowed in normal expressions
@@ -7100,36 +7143,35 @@
        (catch 'at-decl-or-cast
 
 	 ;; CASE 1
-	(when (> paren-depth 0)
-	  ;; Encountered something inside parens that isn't matched by
-	  ;; the `c-type-decl-*' regexps, so it's not a type decl
-	  ;; expression.  Try to skip out to the same paren depth to
-	  ;; not confuse the cast check below.
-	  (c-safe (goto-char (scan-lists (point) 1 paren-depth)))
-	  ;; If we've found a specifier keyword then it's a
-	  ;; declaration regardless.
-	  (throw 'at-decl-or-cast (eq at-decl-or-cast t)))
-
-	(setq at-decl-end
-	      (looking-at (cond ((eq context '<>) "[,>]")
-				(context "[,\)]")
-				(t "[,;]"))))
-
-	;; Now we've collected info about various characteristics of
-	;; the construct we're looking at.  Below follows a decision
-	;; tree based on that.  It's ordered to check more certain
-	;; signs before less certain ones.
-
-	(if got-identifier
-	    (progn
-
-	      ;; CASE 2
-	      (when (and (or at-type maybe-typeless)
-			 (not (or got-prefix got-parens)))
-		;; Got another identifier directly after the type, so it's a
-		;; declaration.
-		(throw 'at-decl-or-cast t))
-
+	 (when (> paren-depth 0)
+	   ;; Encountered something inside parens that isn't matched by
+	   ;; the `c-type-decl-*' regexps, so it's not a type decl
+	   ;; expression.  Try to skip out to the same paren depth to
+	   ;; not confuse the cast check below.
+	   (c-safe (goto-char (scan-lists (point) 1 paren-depth)))
+	   ;; If we've found a specifier keyword then it's a
+	   ;; declaration regardless.
+	   (throw 'at-decl-or-cast (eq at-decl-or-cast t)))
+
+	 (setq at-decl-end
+	       (looking-at (cond ((eq context '<>) "[,>]")
+				 (context "[,\)]")
+				 (t "[,;]"))))
+
+	 ;; Now we've collected info about various characteristics of
+	 ;; the construct we're looking at.  Below follows a decision
+	 ;; tree based on that.	 It's ordered to check more certain
+	 ;; signs before less certain ones.
+
+	 (if got-identifier
+	     (progn
+
+	       ;; CASE 2
+	       (when (and (or at-type maybe-typeless)
+			  (not (or got-prefix got-parens)))
+		 ;; Got another identifier directly after the type, so it's a
+		 ;; declaration.
+		 (throw 'at-decl-or-cast t))
 
 	       (when (and got-parens
 			  (not got-prefix)
@@ -7151,9 +7193,9 @@
 		 (c-fdoc-shift-type-backward)))
 
 	   ;; Found no identifier.
-	  (if backup-at-type
-	      (progn
 
+	   (if backup-at-type
+	       (progn
 
 		 ;; CASE 3
 		 (when (= (point) start)
@@ -7176,251 +7218,251 @@
 		     (setq backup-if-not-cast t)
 		     (throw 'at-decl-or-cast t)))
 
-		;; CASE 4
-		(when (and got-suffix
-			   (not got-prefix)
-			   (not got-parens))
-		  ;; Got a plain list of identifiers followed by some suffix.
-		  ;; If this isn't a cast then the last identifier probably is
-		  ;; the declared one and we should back up to the previous
-		  ;; type.
-		  (setq backup-if-not-cast t)
-		  (throw 'at-decl-or-cast t)))
-
-	    ;; CASE 5
-	    (when (eq at-type t)
-	      ;; If the type is known we know that there can't be any
-	      ;; identifier somewhere else, and it's only in declarations in
-	      ;; e.g. function prototypes and in casts that the identifier may
-	      ;; be left out.
-	      (throw 'at-decl-or-cast t))
-
-	    (when (= (point) start)
-	      ;; Only got a single identifier (parsed as a type so far).
-	      ;; CASE 6
-	      (if (and
-		   ;; Check that the identifier isn't at the start of an
-		   ;; expression.
-		   at-decl-end
-		   (cond
-		    ((eq context 'decl)
-		     ;; Inside an arglist that contains declarations.  If K&R
-		     ;; style declarations and parenthesis style initializers
-		     ;; aren't allowed then the single identifier must be a
-		     ;; type, else we require that it's known or found
-		     ;; (primitive types are handled above).
-		     (or (and (not c-recognize-knr-p)
-			      (not c-recognize-paren-inits))
-			 (memq at-type '(known found))))
-		    ((eq context '<>)
-		     ;; Inside a template arglist.  Accept known and found
-		     ;; types; other identifiers could just as well be
-		     ;; constants in C++.
-		     (memq at-type '(known found)))))
-		  (throw 'at-decl-or-cast t)
-		;; CASE 7
-		;; Can't be a valid declaration or cast, but if we've found a
-		;; specifier it can't be anything else either, so treat it as
-		;; an invalid/unfinished declaration or cast.
-		(throw 'at-decl-or-cast at-decl-or-cast))))
-
-	  (if (and got-parens
-		   (not got-prefix)
-		   (not context)
-		   (not (eq at-type t))
-		   (or backup-at-type
-		       maybe-typeless
-		       backup-maybe-typeless
-		       (when c-recognize-typeless-decls
-			 (or (not got-suffix)
-			     (not (looking-at
-				   c-after-suffixed-type-maybe-decl-key))))))
-	      ;; Got an empty paren pair and a preceding type that probably
-	      ;; really is the identifier.  Shift the type backwards to make
-	      ;; the last one the identifier.  This is analogous to the
-	      ;; "backtracking" done inside the `c-type-decl-suffix-key' loop
-	      ;; above.
-	      ;;
-	      ;; Exception: In addition to the conditions in that
-	      ;; "backtracking" code, do not shift backward if we're not
-	      ;; looking at either `c-after-suffixed-type-decl-key' or "[;,]".
-	      ;; Since there's no preceding type, the shift would mean that
-	      ;; the declaration is typeless.  But if the regexp doesn't match
-	      ;; then we will simply fall through in the tests below and not
-	      ;; recognize it at all, so it's better to try it as an abstract
-	      ;; declarator instead.
-	      (c-fdoc-shift-type-backward)
-
-	    ;; Still no identifier.
-	    ;; CASE 8
-	    (when (and got-prefix (or got-parens got-suffix))
-	      ;; Require `got-prefix' together with either `got-parens' or
-	      ;; `got-suffix' to recognize it as an abstract declarator:
-	      ;; `got-parens' only is probably an empty function call.
-	      ;; `got-suffix' only can build an ordinary expression together
-	      ;; with the preceding identifier which we've taken as a type.
-	      ;; We could actually accept on `got-prefix' only, but that can
-	      ;; easily occur temporarily while writing an expression so we
-	      ;; avoid that case anyway.  We could do a better job if we knew
-	      ;; the point when the fontification was invoked.
-	      (throw 'at-decl-or-cast t))
-
-	    ;; CASE 9
-	    (when (and at-type
-		       (not got-prefix)
-		       (not got-parens)
-		       got-suffix-after-parens
-		       (eq (char-after got-suffix-after-parens) ?\())
-	      ;; Got a type, no declarator but a paren suffix. I.e. it's a
-	      ;; normal function call after all (or perhaps a C++ style object
-	      ;; instantiation expression).
-	      (throw 'at-decl-or-cast nil))))
-
-	;; CASE 10
-	(when at-decl-or-cast
-	  ;; By now we've located the type in the declaration that we know
-	  ;; we're in.
-	  (throw 'at-decl-or-cast t))
-
-	;; CASE 11
-	(when (and got-identifier
-		   (not context)
-		   (looking-at c-after-suffixed-type-decl-key)
-		   (if (and got-parens
-			    (not got-prefix)
-			    (not got-suffix)
-			    (not (eq at-type t)))
-		       ;; Shift the type backward in the case that there's a
-		       ;; single identifier inside parens.  That can only
-		       ;; occur in K&R style function declarations so it's
-		       ;; more likely that it really is a function call.
-		       ;; Therefore we only do this after
-		       ;; `c-after-suffixed-type-decl-key' has matched.
-		       (progn (c-fdoc-shift-type-backward) t)
-		     got-suffix-after-parens))
-	  ;; A declaration according to `c-after-suffixed-type-decl-key'.
-	  (throw 'at-decl-or-cast t))
-
-	;; CASE 12
-	(when (and (or got-prefix (not got-parens))
-		   (memq at-type '(t known)))
-	  ;; It's a declaration if a known type precedes it and it can't be a
-	  ;; function call.
-	  (throw 'at-decl-or-cast t))
-
-	;; If we get here we can't tell if this is a type decl or a normal
-	;; expression by looking at it alone.  (That's under the assumption
-	;; that normal expressions always can look like type decl expressions,
-	;; which isn't really true but the cases where it doesn't hold are so
-	;; uncommon (e.g. some placements of "const" in C++) it's not worth
-	;; the effort to look for them.)
+		 ;; CASE 4
+		 (when (and got-suffix
+			    (not got-prefix)
+			    (not got-parens))
+		   ;; Got a plain list of identifiers followed by some suffix.
+		   ;; If this isn't a cast then the last identifier probably is
+		   ;; the declared one and we should back up to the previous
+		   ;; type.
+		   (setq backup-if-not-cast t)
+		   (throw 'at-decl-or-cast t)))
+
+	     ;; CASE 5
+	     (when (eq at-type t)
+	       ;; If the type is known we know that there can't be any
+	       ;; identifier somewhere else, and it's only in declarations in
+	       ;; e.g. function prototypes and in casts that the identifier may
+	       ;; be left out.
+	       (throw 'at-decl-or-cast t))
+
+	     (when (= (point) start)
+	       ;; Only got a single identifier (parsed as a type so far).
+	       ;; CASE 6
+	       (if (and
+		    ;; Check that the identifier isn't at the start of an
+		    ;; expression.
+		    at-decl-end
+		    (cond
+		     ((eq context 'decl)
+		      ;; Inside an arglist that contains declarations.	If K&R
+		      ;; style declarations and parenthesis style initializers
+		      ;; aren't allowed then the single identifier must be a
+		      ;; type, else we require that it's known or found
+		      ;; (primitive types are handled above).
+		      (or (and (not c-recognize-knr-p)
+			       (not c-recognize-paren-inits))
+			  (memq at-type '(known found))))
+		     ((eq context '<>)
+		      ;; Inside a template arglist.  Accept known and found
+		      ;; types; other identifiers could just as well be
+		      ;; constants in C++.
+		      (memq at-type '(known found)))))
+		   (throw 'at-decl-or-cast t)
+		 ;; CASE 7
+		 ;; Can't be a valid declaration or cast, but if we've found a
+		 ;; specifier it can't be anything else either, so treat it as
+		 ;; an invalid/unfinished declaration or cast.
+		 (throw 'at-decl-or-cast at-decl-or-cast))))
+
+	   (if (and got-parens
+		    (not got-prefix)
+		    (not context)
+		    (not (eq at-type t))
+		    (or backup-at-type
+			maybe-typeless
+			backup-maybe-typeless
+			(when c-recognize-typeless-decls
+			  (or (not got-suffix)
+			      (not (looking-at
+				    c-after-suffixed-type-maybe-decl-key))))))
+	       ;; Got an empty paren pair and a preceding type that probably
+	       ;; really is the identifier.  Shift the type backwards to make
+	       ;; the last one the identifier.	This is analogous to the
+	       ;; "backtracking" done inside the `c-type-decl-suffix-key' loop
+	       ;; above.
+	       ;;
+	       ;; Exception: In addition to the conditions in that
+	       ;; "backtracking" code, do not shift backward if we're not
+	       ;; looking at either `c-after-suffixed-type-decl-key' or "[;,]".
+	       ;; Since there's no preceding type, the shift would mean that
+	       ;; the declaration is typeless.	But if the regexp doesn't match
+	       ;; then we will simply fall through in the tests below and not
+	       ;; recognize it at all, so it's better to try it as an abstract
+	       ;; declarator instead.
+	       (c-fdoc-shift-type-backward)
+
+	     ;; Still no identifier.
+	     ;; CASE 8
+	     (when (and got-prefix (or got-parens got-suffix))
+	       ;; Require `got-prefix' together with either `got-parens' or
+	       ;; `got-suffix' to recognize it as an abstract declarator:
+	       ;; `got-parens' only is probably an empty function call.
+	       ;; `got-suffix' only can build an ordinary expression together
+	       ;; with the preceding identifier which we've taken as a type.
+	       ;; We could actually accept on `got-prefix' only, but that can
+	       ;; easily occur temporarily while writing an expression so we
+	       ;; avoid that case anyway.  We could do a better job if we knew
+	       ;; the point when the fontification was invoked.
+	       (throw 'at-decl-or-cast t))
+
+	     ;; CASE 9
+	     (when (and at-type
+			(not got-prefix)
+			(not got-parens)
+			got-suffix-after-parens
+			(eq (char-after got-suffix-after-parens) ?\())
+	       ;; Got a type, no declarator but a paren suffix. I.e. it's a
+	       ;; normal function call after all (or perhaps a C++ style object
+	       ;; instantiation expression).
+	       (throw 'at-decl-or-cast nil))))
+
+	 ;; CASE 10
+	 (when at-decl-or-cast
+	   ;; By now we've located the type in the declaration that we know
+	   ;; we're in.
+	   (throw 'at-decl-or-cast t))
+
+	 ;; CASE 11
+	 (when (and got-identifier
+		    (not context)
+		    (looking-at c-after-suffixed-type-decl-key)
+		    (if (and got-parens
+			     (not got-prefix)
+			     (not got-suffix)
+			     (not (eq at-type t)))
+			;; Shift the type backward in the case that there's a
+			;; single identifier inside parens.  That can only
+			;; occur in K&R style function declarations so it's
+			;; more likely that it really is a function call.
+			;; Therefore we only do this after
+			;; `c-after-suffixed-type-decl-key' has matched.
+			(progn (c-fdoc-shift-type-backward) t)
+		      got-suffix-after-parens))
+	   ;; A declaration according to `c-after-suffixed-type-decl-key'.
+	   (throw 'at-decl-or-cast t))
+
+	 ;; CASE 12
+	 (when (and (or got-prefix (not got-parens))
+		    (memq at-type '(t known)))
+	   ;; It's a declaration if a known type precedes it and it can't be a
+	   ;; function call.
+	   (throw 'at-decl-or-cast t))
+
+	 ;; If we get here we can't tell if this is a type decl or a normal
+	 ;; expression by looking at it alone.	(That's under the assumption
+	 ;; that normal expressions always can look like type decl expressions,
+	 ;; which isn't really true but the cases where it doesn't hold are so
+	 ;; uncommon (e.g. some placements of "const" in C++) it's not worth
+	 ;; the effort to look for them.)
 
 ;;; 2008-04-16: commented out the next form, to allow the function to recognize
 ;;; "foo (int bar)" in CC (an implicit type (in class foo) without a semicolon)
 ;;; as a(n almost complete) declaration, enabling it to be fontified.
-	;; CASE 13
-	;; (unless (or at-decl-end (looking-at "=[^=]"))
-	;; If this is a declaration it should end here or its initializer(*)
-	;; should start here, so check for allowed separation tokens.  Note
-	;; that this rule doesn't work e.g. with a K&R arglist after a
-	;; function header.
-	;;
-	;; *) Don't check for C++ style initializers using parens
-	;; since those already have been matched as suffixes.
-	;;
-	;; If `at-decl-or-cast' is then we've found some other sign that
-	;; it's a declaration or cast, so then it's probably an
-	;; invalid/unfinished one.
-	;;  (throw 'at-decl-or-cast at-decl-or-cast))
-
-	;; Below are tests that only should be applied when we're certain to
-	;; not have parsed halfway through an expression.
-
-	;; CASE 14
-	(when (memq at-type '(t known))
-	  ;; The expression starts with a known type so treat it as a
-	  ;; declaration.
-	  (throw 'at-decl-or-cast t))
-
-	;; CASE 15
-	(when (and (c-major-mode-is 'c++-mode)
-		   ;; In C++ we check if the identifier is a known type, since
-		   ;; (con|de)structors use the class name as identifier.
-		   ;; We've always shifted over the identifier as a type and
-		   ;; then backed up again in this case.
-		   identifier-type
-		   (or (memq identifier-type '(found known))
-		       (and (eq (char-after identifier-start) ?~)
-			    ;; `at-type' probably won't be 'found for
-			    ;; destructors since the "~" is then part of the
-			    ;; type name being checked against the list of
-			    ;; known types, so do a check without that
-			    ;; operator.
-			    (or (save-excursion
-				  (goto-char (1+ identifier-start))
-				  (c-forward-syntactic-ws)
-				  (c-with-syntax-table
-				      c-identifier-syntax-table
-				    (looking-at c-known-type-key)))
-				(save-excursion
-				  (goto-char (1+ identifier-start))
-				  ;; We have already parsed the type earlier,
-				  ;; so it'd be possible to cache the end
-				  ;; position instead of redoing it here, but
-				  ;; then we'd need to keep track of another
-				  ;; position everywhere.
-				  (c-check-type (point)
-						(progn (c-forward-type)
-						       (point))))))))
-	  (throw 'at-decl-or-cast t))
-
-	(if got-identifier
-	    (progn
-	      ;; CASE 16
-	      (when (and got-prefix-before-parens
-			 at-type
-			 (or at-decl-end (looking-at "=[^=]"))
-			 (not context)
-			 (not got-suffix))
-		;; Got something like "foo * bar;".  Since we're not inside an
-		;; arglist it would be a meaningless expression because the
-		;; result isn't used.  We therefore choose to recognize it as
-		;; a declaration.  Do not allow a suffix since it could then
-		;; be a function call.
-		(throw 'at-decl-or-cast t))
-
-	      ;; CASE 17
-	      (when (and (or got-suffix-after-parens
-			     (looking-at "=[^=]"))
-			 (eq at-type 'found)
-			 (not (eq context 'arglist)))
-		;; Got something like "a (*b) (c);" or "a (b) = c;".  It could
-		;; be an odd expression or it could be a declaration.  Treat
-		;; it as a declaration if "a" has been used as a type
-		;; somewhere else (if it's a known type we won't get here).
-		(throw 'at-decl-or-cast t)))
-
-	  ;; CASE 18
-	  (when (and context
-		     (or got-prefix
-			 (and (eq context 'decl)
-			      (not c-recognize-paren-inits)
-			      (or got-parens got-suffix))))
-	    ;; Got a type followed by an abstract declarator.  If `got-prefix'
-	    ;; is set it's something like "a *" without anything after it.  If
-	    ;; `got-parens' or `got-suffix' is set it's "a()", "a[]", "a()[]",
-	    ;; or similar, which we accept only if the context rules out
-	    ;; expressions.
-	    (throw 'at-decl-or-cast t)))
-
-	;; If we had a complete symbol table here (which rules out
-	;; `c-found-types') we should return t due to the disambiguation rule
-	;; (in at least C++) that anything that can be parsed as a declaration
-	;; is a declaration.  Now we're being more defensive and prefer to
-	;; highlight things like "foo (bar);" as a declaration only if we're
-	;; inside an arglist that contains declarations.
-	;; CASE 19
-	(eq context 'decl))))
+	 ;; CASE 13
+	 ;;	(unless (or at-decl-end (looking-at "=[^=]"))
+	 ;; If this is a declaration it should end here or its initializer(*)
+	 ;; should start here, so check for allowed separation tokens.	Note
+	 ;; that this rule doesn't work e.g. with a K&R arglist after a
+	 ;; function header.
+	 ;;
+	 ;; *) Don't check for C++ style initializers using parens
+	 ;; since those already have been matched as suffixes.
+	 ;;
+	 ;; If `at-decl-or-cast' is then we've found some other sign that
+	 ;; it's a declaration or cast, so then it's probably an
+	 ;; invalid/unfinished one.
+	 ;;	  (throw 'at-decl-or-cast at-decl-or-cast))
+
+	 ;; Below are tests that only should be applied when we're certain to
+	 ;; not have parsed halfway through an expression.
+
+	 ;; CASE 14
+	 (when (memq at-type '(t known))
+	   ;; The expression starts with a known type so treat it as a
+	   ;; declaration.
+	   (throw 'at-decl-or-cast t))
+
+	 ;; CASE 15
+	 (when (and (c-major-mode-is 'c++-mode)
+		    ;; In C++ we check if the identifier is a known type, since
+		    ;; (con|de)structors use the class name as identifier.
+		    ;; We've always shifted over the identifier as a type and
+		    ;; then backed up again in this case.
+		    identifier-type
+		    (or (memq identifier-type '(found known))
+			(and (eq (char-after identifier-start) ?~)
+			     ;; `at-type' probably won't be 'found for
+			     ;; destructors since the "~" is then part of the
+			     ;; type name being checked against the list of
+			     ;; known types, so do a check without that
+			     ;; operator.
+			     (or (save-excursion
+				   (goto-char (1+ identifier-start))
+				   (c-forward-syntactic-ws)
+				   (c-with-syntax-table
+				       c-identifier-syntax-table
+				     (looking-at c-known-type-key)))
+				 (save-excursion
+				   (goto-char (1+ identifier-start))
+				   ;; We have already parsed the type earlier,
+				   ;; so it'd be possible to cache the end
+				   ;; position instead of redoing it here, but
+				   ;; then we'd need to keep track of another
+				   ;; position everywhere.
+				   (c-check-type (point)
+						 (progn (c-forward-type)
+							(point))))))))
+	   (throw 'at-decl-or-cast t))
+
+	 (if got-identifier
+	     (progn
+	       ;; CASE 16
+	       (when (and got-prefix-before-parens
+			  at-type
+			  (or at-decl-end (looking-at "=[^=]"))
+			  (not context)
+			  (not got-suffix))
+		 ;; Got something like "foo * bar;".  Since we're not inside an
+		 ;; arglist it would be a meaningless expression because the
+		 ;; result isn't used.	We therefore choose to recognize it as
+		 ;; a declaration.  Do not allow a suffix since it could then
+		 ;; be a function call.
+		 (throw 'at-decl-or-cast t))
+
+	       ;; CASE 17
+	       (when (and (or got-suffix-after-parens
+			      (looking-at "=[^=]"))
+			  (eq at-type 'found)
+			  (not (eq context 'arglist)))
+		 ;; Got something like "a (*b) (c);" or "a (b) = c;".  It could
+		 ;; be an odd expression or it could be a declaration.	Treat
+		 ;; it as a declaration if "a" has been used as a type
+		 ;; somewhere else (if it's a known type we won't get here).
+		 (throw 'at-decl-or-cast t)))
+
+	   ;; CASE 18
+	   (when (and context
+		      (or got-prefix
+			  (and (eq context 'decl)
+			       (not c-recognize-paren-inits)
+			       (or got-parens got-suffix))))
+	     ;; Got a type followed by an abstract declarator.	If `got-prefix'
+	     ;; is set it's something like "a *" without anything after it.  If
+	     ;; `got-parens' or `got-suffix' is set it's "a()", "a[]", "a()[]",
+	     ;; or similar, which we accept only if the context rules out
+	     ;; expressions.
+	     (throw 'at-decl-or-cast t)))
+
+	 ;; If we had a complete symbol table here (which rules out
+	 ;; `c-found-types') we should return t due to the disambiguation rule
+	 ;; (in at least C++) that anything that can be parsed as a declaration
+	 ;; is a declaration.  Now we're being more defensive and prefer to
+	 ;; highlight things like "foo (bar);" as a declaration only if we're
+	 ;; inside an arglist that contains declarations.
+         ;; CASE 19
+	 (eq context 'decl))))
 
     ;; The point is now after the type decl expression.
 
@@ -7609,10 +7651,10 @@
 		    (c-put-c-type-property (1- (point)) 'c-decl-end)
 		    t)
 
-	      ;; It's an unfinished label.  We consider the keyword enough
-	      ;; to recognize it as a label, so that it gets fontified.
-	      ;; Leave the point at the end of it, but don't put any
-	      ;; `c-decl-end' marker.
+		;; It's an unfinished label.  We consider the keyword enough
+		;; to recognize it as a label, so that it gets fontified.
+		;; Leave the point at the end of it, but don't put any
+		;; `c-decl-end' marker.
 		(goto-char kwd-end)
 		t))))
 
@@ -7797,69 +7839,69 @@
   ;;
   ;; This function might do hidden buffer changes.
 
-    (let ((start (point))
-	  start-char
-	  (c-promote-possible-types t)
-	  lim
-	  ;; Turn off recognition of angle bracket arglists while parsing
-	  ;; types here since the protocol reference list might then be
-	  ;; considered part of the preceding name or superclass-name.
-	  c-recognize-<>-arglists)
-
-      (if (or
-	   (when (looking-at
-		  (eval-when-compile
-		    (c-make-keywords-re t
-		      (append (c-lang-const c-protection-kwds objc)
-			      '("@end"))
-		      'objc-mode)))
-	     (goto-char (match-end 1))
-	     t)
-
-	   (and
-	    (looking-at
-	     (eval-when-compile
-	       (c-make-keywords-re t
-		 '("@interface" "@implementation" "@protocol")
-		 'objc-mode)))
-
-	    ;; Handle the name of the class itself.
-	    (progn
-;	      (c-forward-token-2) ; 2006/1/13 This doesn't move if the token's
-;	      at EOB.
-	      (goto-char (match-end 0))
-	      (setq lim (point))
-	      (c-skip-ws-forward)
-	      (c-forward-type))
-
-	    (catch 'break
-	      ;; Look for ": superclass-name" or "( category-name )".
-	      (when (looking-at "[:\(]")
-		(setq start-char (char-after))
-		(forward-char)
-		(c-forward-syntactic-ws)
-		(unless (c-forward-type) (throw 'break nil))
-		(when (eq start-char ?\()
-		  (unless (eq (char-after) ?\)) (throw 'break nil))
-		  (forward-char)
-		  (c-forward-syntactic-ws)))
-
-	      ;; Look for a protocol reference list.
-	      (if (eq (char-after) ?<)
-		  (let ((c-recognize-<>-arglists t)
-			(c-parse-and-markup-<>-arglists t)
-			c-restricted-<>-arglists)
-		    (c-forward-<>-arglist t))
-		t))))
-
-	  (progn
-	    (c-backward-syntactic-ws lim)
-	    (c-clear-c-type-property start (1- (point)) 'c-decl-end)
-	    (c-put-c-type-property (1- (point)) 'c-decl-end)
-	    t)
-
-	(c-clear-c-type-property start (point) 'c-decl-end)
-	nil)))
+  (let ((start (point))
+	start-char
+	(c-promote-possible-types t)
+	lim
+	;; Turn off recognition of angle bracket arglists while parsing
+	;; types here since the protocol reference list might then be
+	;; considered part of the preceding name or superclass-name.
+	c-recognize-<>-arglists)
+
+    (if (or
+	 (when (looking-at
+		(eval-when-compile
+		  (c-make-keywords-re t
+		    (append (c-lang-const c-protection-kwds objc)
+			    '("@end"))
+		    'objc-mode)))
+	   (goto-char (match-end 1))
+	   t)
+
+	 (and
+	  (looking-at
+	   (eval-when-compile
+	     (c-make-keywords-re t
+	       '("@interface" "@implementation" "@protocol")
+	       'objc-mode)))
+
+	  ;; Handle the name of the class itself.
+	  (progn
+            ;; (c-forward-token-2) ; 2006/1/13 This doesn't move if the token's
+            ;; at EOB.
+	    (goto-char (match-end 0))
+	    (setq lim (point))
+	    (c-skip-ws-forward)
+	    (c-forward-type))
+
+	  (catch 'break
+	    ;; Look for ": superclass-name" or "( category-name )".
+	    (when (looking-at "[:\(]")
+	      (setq start-char (char-after))
+	      (forward-char)
+	      (c-forward-syntactic-ws)
+	      (unless (c-forward-type) (throw 'break nil))
+	      (when (eq start-char ?\()
+		(unless (eq (char-after) ?\)) (throw 'break nil))
+		(forward-char)
+		(c-forward-syntactic-ws)))
+
+	    ;; Look for a protocol reference list.
+	    (if (eq (char-after) ?<)
+		(let ((c-recognize-<>-arglists t)
+		      (c-parse-and-markup-<>-arglists t)
+		      c-restricted-<>-arglists)
+		  (c-forward-<>-arglist t))
+	      t))))
+
+	(progn
+	  (c-backward-syntactic-ws lim)
+	  (c-clear-c-type-property start (1- (point)) 'c-decl-end)
+	  (c-put-c-type-property (1- (point)) 'c-decl-end)
+	  t)
+
+      (c-clear-c-type-property start (point) 'c-decl-end)
+      nil)))
 
 (defun c-beginning-of-inheritance-list (&optional lim)
   ;; Go to the first non-whitespace after the colon that starts a
@@ -7946,7 +7988,7 @@
   ;;
   ;; This function might do hidden buffer changes.
 
-  (let ((beg (point)) end id-start)
+  (let ((beg (point)) id-start)
     (and
      (eq (c-beginning-of-statement-1 lim) 'same)
 
@@ -8036,54 +8078,54 @@
 		   (throw 'knr nil)))
 
 	    (if after-rparen
-	    ;; We're inside a paren.  Could it be our argument list....?
-	      (if
-		  (and
-		   (progn
-		     (goto-char after-rparen)
-		     (unless (c-go-list-backward) (throw 'knr nil)) ;
-		;; FIXME!!!  What about macros between the parens?  2007/01/20
-		     (setq before-lparen (point)))
-
-		   ;; It can't be the arg list if next token is ; or {
-		   (progn (goto-char after-rparen)
-			  (c-forward-syntactic-ws)
-			  (not (memq (char-after) '(?\; ?\{ ?\=))))
-
-		   ;; Is the thing preceding the list an identifier (the
-		   ;; function name), or a macro expansion?
-		   (progn
-		     (goto-char before-lparen)
-		     (eq (c-backward-token-2) 0)
-		     (or (eq (c-on-identifier) (point))
-			 (and (eq (char-after) ?\))
-			      (c-go-up-list-backward)
-			      (eq (c-backward-token-2) 0)
-			      (eq (c-on-identifier) (point)))))
-
-		   ;; Have we got a non-empty list of comma-separated
-		   ;; identifiers?
-		   (progn
-		     (goto-char before-lparen)
-		     (c-forward-token-2) ; to first token inside parens
-		     (and
-		      (c-on-identifier)
-		      (c-forward-token-2)
-		      (catch 'id-list
-			(while (eq (char-after) ?\,)
-			  (c-forward-token-2)
-			  (unless (c-on-identifier) (throw 'id-list nil))
-			  (c-forward-token-2))
-			(eq (char-after) ?\))))))
-
-		  ;; ...Yes.  We've identified the function's argument list.
-		  (throw 'knr
-		       (progn (goto-char after-rparen)
-			      (c-forward-syntactic-ws)
-			      (point)))
-
-		;; ...No.  The current parens aren't the function's arg list.
-		(goto-char before-lparen))
+		;; We're inside a paren.  Could it be our argument list....?
+		(if
+		    (and
+		     (progn
+		       (goto-char after-rparen)
+		       (unless (c-go-list-backward) (throw 'knr nil)) ;
+		       ;; FIXME!!!  What about macros between the parens?  2007/01/20
+		       (setq before-lparen (point)))
+
+		     ;; It can't be the arg list if next token is ; or {
+		     (progn (goto-char after-rparen)
+			    (c-forward-syntactic-ws)
+			    (not (memq (char-after) '(?\; ?\{ ?\=))))
+
+		     ;; Is the thing preceding the list an identifier (the
+		     ;; function name), or a macro expansion?
+		     (progn
+		       (goto-char before-lparen)
+		       (eq (c-backward-token-2) 0)
+		       (or (eq (c-on-identifier) (point))
+			   (and (eq (char-after) ?\))
+				(c-go-up-list-backward)
+				(eq (c-backward-token-2) 0)
+				(eq (c-on-identifier) (point)))))
+
+		     ;; Have we got a non-empty list of comma-separated
+		     ;; identifiers?
+		     (progn
+		       (goto-char before-lparen)
+		       (c-forward-token-2) ; to first token inside parens
+		       (and
+			(c-on-identifier)
+			(c-forward-token-2)
+			(catch 'id-list
+			  (while (eq (char-after) ?\,)
+			    (c-forward-token-2)
+			    (unless (c-on-identifier) (throw 'id-list nil))
+			    (c-forward-token-2))
+			  (eq (char-after) ?\))))))
+
+		    ;; ...Yes.	We've identified the function's argument list.
+		    (throw 'knr
+			   (progn (goto-char after-rparen)
+				  (c-forward-syntactic-ws)
+				  (point)))
+
+		  ;; ...No.  The current parens aren't the function's arg list.
+		  (goto-char before-lparen))
 
 	      (or (c-go-list-backward)	; backwards over [ .... ]
 		  (throw 'knr nil)))))))))
@@ -8289,7 +8331,7 @@
 		   (and
 		    (progn
 		      (while  ; keep going back to "[;={"s until we either find
-			    ; no more, or get to one which isn't an "operator ="
+                           ; no more, or get to one which isn't an "operator ="
 			  (and (c-syntactic-re-search-forward "[;={]" start t t t)
 			       (eq (char-before) ?=)
 			       c-overloadable-operators-regexp
@@ -8403,10 +8445,7 @@
     (when (and c-recognize-<>-arglists
 	       (eq (char-before) ?>))
       ;; Could be at the end of a template arglist.
-      (let ((c-parse-and-markup-<>-arglists t)
-	    (c-disallow-comma-in-<>-arglists
-	     (and containing-sexp
-		  (not (eq (char-after containing-sexp) ?{)))))
+      (let ((c-parse-and-markup-<>-arglists t))
 	(while (and
 		(c-backward-<>-arglist nil limit)
 		(progn
@@ -8612,86 +8651,86 @@
        (while (and (not bufpos)
 		   containing-sexp)
 	 (when paren-state
-	     (if (consp (car paren-state))
-		 (setq lim (cdr (car paren-state))
-		       paren-state (cdr paren-state))
-	       (setq lim (car paren-state)))
-	     (when paren-state
-	       (setq next-containing (car paren-state)
-		     paren-state (cdr paren-state))))
-	   (goto-char containing-sexp)
-	   (if (c-looking-at-inexpr-block next-containing next-containing)
-	       ;; We're in an in-expression block of some kind.  Do not
-	       ;; check nesting.  We deliberately set the limit to the
-	       ;; containing sexp, so that c-looking-at-inexpr-block
-	       ;; doesn't check for an identifier before it.
-	       (setq containing-sexp nil)
-	     ;; see if the open brace is preceded by = or [...] in
-	     ;; this statement, but watch out for operator=
-	     (setq braceassignp 'dontknow)
-	     (c-backward-token-2 1 t lim)
-	     ;; Checks to do only on the first sexp before the brace.
-	     (when (and c-opt-inexpr-brace-list-key
-			(eq (char-after) ?\[))
-	       ;; In Java, an initialization brace list may follow
-	       ;; directly after "new Foo[]", so check for a "new"
-	       ;; earlier.
-	       (while (eq braceassignp 'dontknow)
-		 (setq braceassignp
-		       (cond ((/= (c-backward-token-2 1 t lim) 0) nil)
-			     ((looking-at c-opt-inexpr-brace-list-key) t)
-			     ((looking-at "\\sw\\|\\s_\\|[.[]")
-			      ;; Carry on looking if this is an
-			      ;; identifier (may contain "." in Java)
-			      ;; or another "[]" sexp.
-			      'dontknow)
-			     (t nil)))))
-	     ;; Checks to do on all sexps before the brace, up to the
-	     ;; beginning of the statement.
-	     (while (eq braceassignp 'dontknow)
-	       (cond ((eq (char-after) ?\;)
-		      (setq braceassignp nil))
-		     ((and class-key
-			   (looking-at class-key))
-		      (setq braceassignp nil))
-		     ((eq (char-after) ?=)
-		      ;; We've seen a =, but must check earlier tokens so
-		      ;; that it isn't something that should be ignored.
-		      (setq braceassignp 'maybe)
-		      (while (and (eq braceassignp 'maybe)
-				  (zerop (c-backward-token-2 1 t lim)))
-			(setq braceassignp
-			      (cond
-			       ;; Check for operator =
-			       ((and c-opt-op-identifier-prefix
-				     (looking-at c-opt-op-identifier-prefix))
-				nil)
-			       ;; Check for `<opchar>= in Pike.
-			       ((and (c-major-mode-is 'pike-mode)
-				     (or (eq (char-after) ?`)
-					 ;; Special case for Pikes
-					 ;; `[]=, since '[' is not in
-					 ;; the punctuation class.
-					 (and (eq (char-after) ?\[)
-					      (eq (char-before) ?`))))
-				nil)
-			       ((looking-at "\\s.") 'maybe)
-			       ;; make sure we're not in a C++ template
-			       ;; argument assignment
-			       ((and
-				 (c-major-mode-is 'c++-mode)
-				 (save-excursion
-				   (let ((here (point))
-					 (pos< (progn
-						 (skip-chars-backward "^<>")
-						 (point))))
-				     (and (eq (char-before) ?<)
-					  (not (c-crosses-statement-barrier-p
-						pos< here))
-					  (not (c-in-literal))
-					  ))))
-				nil)
-			       (t t))))))
+	   (if (consp (car paren-state))
+	       (setq lim (cdr (car paren-state))
+		     paren-state (cdr paren-state))
+	     (setq lim (car paren-state)))
+	   (when paren-state
+	     (setq next-containing (car paren-state)
+		   paren-state (cdr paren-state))))
+	 (goto-char containing-sexp)
+	 (if (c-looking-at-inexpr-block next-containing next-containing)
+	     ;; We're in an in-expression block of some kind.  Do not
+	     ;; check nesting.	We deliberately set the limit to the
+	     ;; containing sexp, so that c-looking-at-inexpr-block
+	     ;; doesn't check for an identifier before it.
+	     (setq containing-sexp nil)
+	   ;; see if the open brace is preceded by = or [...] in
+	   ;; this statement, but watch out for operator=
+	   (setq braceassignp 'dontknow)
+	   (c-backward-token-2 1 t lim)
+	   ;; Checks to do only on the first sexp before the brace.
+	   (when (and c-opt-inexpr-brace-list-key
+		      (eq (char-after) ?\[))
+	     ;; In Java, an initialization brace list may follow
+	     ;; directly after "new Foo[]", so check for a "new"
+	     ;; earlier.
+	     (while (eq braceassignp 'dontknow)
+	       (setq braceassignp
+		     (cond ((/= (c-backward-token-2 1 t lim) 0) nil)
+			   ((looking-at c-opt-inexpr-brace-list-key) t)
+			   ((looking-at "\\sw\\|\\s_\\|[.[]")
+			    ;; Carry on looking if this is an
+			    ;; identifier (may contain "." in Java)
+			    ;; or another "[]" sexp.
+			    'dontknow)
+			   (t nil)))))
+	   ;; Checks to do on all sexps before the brace, up to the
+	   ;; beginning of the statement.
+	   (while (eq braceassignp 'dontknow)
+	     (cond ((eq (char-after) ?\;)
+		    (setq braceassignp nil))
+		   ((and class-key
+			 (looking-at class-key))
+		    (setq braceassignp nil))
+		   ((eq (char-after) ?=)
+		    ;; We've seen a =, but must check earlier tokens so
+		    ;; that it isn't something that should be ignored.
+		    (setq braceassignp 'maybe)
+		    (while (and (eq braceassignp 'maybe)
+				(zerop (c-backward-token-2 1 t lim)))
+		      (setq braceassignp
+			    (cond
+			     ;; Check for operator =
+			     ((and c-opt-op-identifier-prefix
+				   (looking-at c-opt-op-identifier-prefix))
+			      nil)
+			     ;; Check for `<opchar>= in Pike.
+			     ((and (c-major-mode-is 'pike-mode)
+				   (or (eq (char-after) ?`)
+				       ;; Special case for Pikes
+				       ;; `[]=, since '[' is not in
+				       ;; the punctuation class.
+				       (and (eq (char-after) ?\[)
+					    (eq (char-before) ?`))))
+			      nil)
+			     ((looking-at "\\s.") 'maybe)
+			     ;; make sure we're not in a C++ template
+			     ;; argument assignment
+			     ((and
+			       (c-major-mode-is 'c++-mode)
+			       (save-excursion
+				 (let ((here (point))
+				       (pos< (progn
+					       (skip-chars-backward "^<>")
+					       (point))))
+				   (and (eq (char-before) ?<)
+					(not (c-crosses-statement-barrier-p
+					      pos< here))
+					(not (c-in-literal))
+					))))
+			      nil)
+			     (t t))))))
 	     (if (and (eq braceassignp 'dontknow)
 		      (/= (c-backward-token-2 1 t lim) 0))
 		 (setq braceassignp nil)))
@@ -9214,7 +9253,7 @@
 		     (max (c-point 'boi paren-pos) (point))))
 		   (t (c-add-syntax 'defun-block-intro nil))))
 
-		 (c-add-syntax 'statement-block-intro nil)))
+	      (c-add-syntax 'statement-block-intro nil)))
 
 	  (if (= paren-pos boi)
 	      ;; Always done if the open brace was at boi.  The
@@ -10327,7 +10366,6 @@
 	  (c-add-syntax 'topmost-intro-cont (c-point 'boi)))
 	 ))
 
-
        ;; (CASE 6 has been removed.)
 
        ;; CASE 7: line is an expression, not a statement.  Most

=== modified file 'lisp/progmodes/cc-fonts.el'
--- lisp/progmodes/cc-fonts.el	2014-09-10 21:38:11 +0000
+++ lisp/progmodes/cc-fonts.el	2014-09-25 20:19:06 +0000
@@ -176,6 +176,7 @@
       'font-lock-negation-char-face))
 
 (cc-bytecomp-defun face-inverse-video-p) ; Only in Emacs.
+(cc-bytecomp-defun face-property-instance) ; Only in XEmacs.
 
 (defun c-make-inverse-face (oldface newface)
   ;; Emacs and XEmacs have completely different face manipulation
@@ -266,7 +267,7 @@
     ;; This function might do hidden buffer changes.
     (when (c-got-face-at (point) c-literal-faces)
       (while (progn
-	       (goto-char (next-single-property-change
+	       (goto-char (c-next-single-property-change
 			   (point) 'face nil limit))
 	       (and (< (point) limit)
 		    (c-got-face-at (point) c-literal-faces))))
@@ -366,39 +367,7 @@
 	      (parse-sexp-lookup-properties
 	       (cc-eval-when-compile
 		 (boundp 'parse-sexp-lookup-properties))))
-
-	  ;; (while (re-search-forward ,regexp limit t)
-	  ;;   (unless (progn
-	  ;; 	      (goto-char (match-beginning 0))
-	  ;; 	      (c-skip-comments-and-strings limit))
-	  ;;     (goto-char (match-end 0))
-	  ;;     ,@(mapcar
-	  ;; 	 (lambda (highlight)
-	  ;; 	   (if (integerp (car highlight))
-	  ;; 	       (progn
-	  ;; 		 (unless (eq (nth 2 highlight) t)
-	  ;; 		   (error
-	  ;; 		    "The override flag must currently be t in %s"
-	  ;; 		    highlight))
-	  ;; 		 (when (nth 3 highlight)
-	  ;; 		   (error
-	  ;; 		    "The laxmatch flag may currently not be set in %s"
-	  ;; 		    highlight))
-	  ;; 		 `(save-match-data
-	  ;; 		    (c-put-font-lock-face
-	  ;; 		     (match-beginning ,(car highlight))
-	  ;; 		     (match-end ,(car highlight))
-	  ;; 		     ,(elt highlight 1))))
-	  ;; 	     (when (nth 3 highlight)
-	  ;; 	       (error "Match highlights currently not supported in %s"
-	  ;; 		      highlight))
-	  ;; 	     `(progn
-	  ;; 		,(nth 1 highlight)
-	  ;; 		(save-match-data ,(car highlight))
-	  ;; 		,(nth 2 highlight))))
-	  ;; 	 highlights)))
 	  ,(c-make-font-lock-search-form regexp highlights))
-
 	nil)))
 
   (defun c-make-font-lock-BO-decl-search-function (regexp &rest highlights)
@@ -591,8 +560,7 @@
 				   (progn
 				     (c-mark-<-as-paren beg)
 				     (c-mark->-as-paren end))
-				 ;; (c-clear-char-property beg 'syntax-table)
-				 (c-clear-char-property beg 'category)))
+				 (c-unmark-<->-as-paren beg)))
 			     nil)))))))
 
 	      ;; #define.
@@ -716,7 +684,11 @@
   (let ((start (1- (point))))
     (save-excursion
       (and (eq (elt (parse-partial-sexp start (c-point 'eol)) 8) start)
-	   (if (integerp c-multiline-string-start-char)
+	   (if (if (eval-when-compile (integerp ?c))
+		   ;; Emacs
+		   (integerp c-multiline-string-start-char)
+		 ;; XEmacs
+		 (characterp c-multiline-string-start-char))
 	       ;; There's no multiline string start char before the
 	       ;; string, so newlines aren't allowed.
 	       (not (eq (char-before start) c-multiline-string-start-char))
@@ -771,6 +743,13 @@
 	 `(,(concat "\\<" (c-lang-const c-regular-keywords-regexp))
 	   1 font-lock-keyword-face))
 
+      ;; The following must come before c-font-lock-enclosing-decls in
+      ;; c-complex-decl-matchers.  It fontifies java @annotations.
+      ,@(when (c-major-mode-is 'java-mode)
+	  `((eval . (list "\\<\\(@[a-zA-Z0-9]+\\)\\>" 1
+			  c-annotation-face
+			  ))))
+
       ;; Fontify leading identifiers in fully qualified names like
       ;; "foo::bar" in languages that supports such things.
       ,@(when (c-lang-const c-opt-identifier-concat-key)
@@ -1151,7 +1130,6 @@
       (when list
 	;; Jump past any initializer or function prototype to see if
 	;; there's a ',' to continue at.
-
 	(cond ((eq id-face 'font-lock-function-name-face)
 	       ;; Skip a parenthesized initializer (C++) or a function
 	       ;; prototype.
@@ -1219,8 +1197,8 @@
 	  ;; o - nil, if not in an arglist at all.  This includes the
 	  ;;   parenthesized condition which follows "if", "while", etc.
 	  context
-	  ;; The position of the next token after the closing paren of
-	  ;; the last detected cast.
+	  ;; A list of starting positions of possible type declarations, or of
+	  ;; the typedef preceding one, if any.
 	  last-cast-end
 	  ;; The result from `c-forward-decl-or-cast-1'.
 	  decl-or-cast
@@ -1564,9 +1542,7 @@
   ;; Note that this function won't attempt to fontify beyond the end of the
   ;; current enum block, if any.
   (let* ((paren-state (c-parse-state))
-	 (encl-pos (c-most-enclosing-brace paren-state))
-	 (start (point))
-	)
+	 (encl-pos (c-most-enclosing-brace paren-state)))
     (when (and
 	   encl-pos
 	   (eq (char-after encl-pos) ?\{)
@@ -1948,11 +1924,7 @@
 			 "\\)\\>")
 		 '((c-fontify-types-and-refs ((c-promote-possible-types t))
 		     (c-forward-keyword-clause 1)
-		     (if (> (point) limit) (goto-char limit))))))))
-
-	,@(when (c-major-mode-is 'java-mode)
-	    `((eval . (list "\\<\\(@[a-zA-Z0-9]+\\)\\>" 1 c-annotation-face))))
-      ))
+		     (if (> (point) limit) (goto-char limit))))))))))
 
 (c-lang-defconst c-matchers-1
   t (c-lang-const c-cpp-matchers))

=== modified file 'lisp/progmodes/cc-guess.el'
--- lisp/progmodes/cc-guess.el	2014-06-26 07:13:13 +0000
+++ lisp/progmodes/cc-guess.el	2014-09-25 18:27:32 +0000
@@ -75,6 +75,9 @@
 (cc-require 'cc-defs)
 (cc-require 'cc-engine)
 (cc-require 'cc-styles)
+(cc-bytecomp-defvar make-progress-reporter)
+(cc-bytecomp-defvar progress-reporter-update)
+(cc-bytecomp-defvar progress-reporter-done)
 
 \f
 
@@ -519,7 +522,8 @@
       (goto-char (point-min))
       (when (search-forward (concat "("
 				    (symbol-name (car needs-markers))
-				    " ") nil t)
+				    " ")
+                            nil t)
 	(move-end-of-line 1)
 	(comment-dwim nil)
 	(insert " Guessed value"))

=== modified file 'lisp/progmodes/cc-langs.el'
--- lisp/progmodes/cc-langs.el	2014-09-10 21:38:11 +0000
+++ lisp/progmodes/cc-langs.el	2014-09-26 15:07:26 +0000
@@ -130,9 +130,7 @@
 
 
 ;; This file is not always loaded.  See note above.
-;; Except it is always loaded - see bug#17463.
-;;;(cc-external-require 'cl)
-(require 'cl-lib)
+(cc-external-require 'cl)
 
 \f
 ;;; Setup for the `c-lang-defvar' system.
@@ -253,14 +251,14 @@
     (unless xlate
       (setq xlate 'identity))
     (c-with-syntax-table (c-lang-const c-mode-syntax-table)
-      (cl-delete-duplicates
-       (cl-mapcan (lambda (opgroup)
+      (delete-duplicates
+       (mapcan (lambda (opgroup)
 		 (when (if (symbolp (car opgroup))
 			   (when (funcall opgroup-filter (car opgroup))
 			     (setq opgroup (cdr opgroup))
 			     t)
 			 t)
-		   (cl-mapcan (lambda (op)
+		   (mapcan (lambda (op)
 			     (when (funcall op-filter op)
 			       (let ((res (funcall xlate op)))
 				 (if (listp res) res (list res)))))
@@ -301,7 +299,8 @@
        ["Set Style..."                   c-set-style t]
        ["Show Current Style Name"        (message
 					  "Style Name: %s"
-					  c-indentation-style) t]
+					  c-indentation-style)
+                                         t]
        ["Guess Style from this Buffer"   c-guess-buffer-no-install t]
        ["Install the Last Guessed Style..." c-guess-install
 	(and c-guess-guessed-offsets-alist
@@ -319,9 +318,9 @@
 	:style toggle :selected c-auto-newline]
        ["Hungry delete"         c-toggle-hungry-state
 	:style toggle :selected c-hungry-delete-key]
-       ["Subword mode"          subword-mode
-	:style toggle :selected (and (boundp 'subword-mode)
-                                     subword-mode)])))
+       ["Subword mode"          c-subword-mode
+	:style toggle :selected (and (boundp 'c-subword-mode)
+                                     c-subword-mode)])))
 
 \f
 ;;; Syntax tables.
@@ -578,6 +577,15 @@
   (c c++ objc) t)
 (c-lang-defvar c-has-bitfields (c-lang-const c-has-bitfields))
 
+(c-lang-defconst c-modified-constant
+  "Regexp that matches a \"modified\" constant literal such as \"L'a'\",
+a \"long character\".  In particular, this recognizes forms of constant
+which `c-backward-sexp' needs to be called twice to move backwards over."
+  t nil
+  (c c++ objc) "L'\\([^\\'\t\f\n\r]\\|\\\\.\\)'")
+;; FIXME!!!  Extend this to cover strings, if needed.  2008-04-11
+(c-lang-defvar c-modified-constant (c-lang-const c-modified-constant))
+
 (c-lang-defconst c-symbol-start
   "Regexp that matches the start of a symbol, i.e. any identifier or
 keyword.  It's unspecified how far it matches.	Does not contain a \\|
@@ -1144,7 +1152,8 @@
   c++  (append '("&" "<%" "%>" "<:" ":>" "%:" "%:%:")
 	       (c-lang-const c-other-op-syntax-tokens))
   objc (append '("#" "##"		; Used by cpp.
-		 "+" "-") (c-lang-const c-other-op-syntax-tokens))
+		 "+" "-")
+               (c-lang-const c-other-op-syntax-tokens))
   idl  (append '("#" "##")		; Used by cpp.
 	       (c-lang-const c-other-op-syntax-tokens))
   pike (append '("..")
@@ -1155,7 +1164,7 @@
 (c-lang-defconst c-all-op-syntax-tokens
   ;; List of all tokens in the punctuation and parenthesis syntax
   ;; classes.
-  t (cl-delete-duplicates (append (c-lang-const c-other-op-syntax-tokens)
+  t (delete-duplicates (append (c-lang-const c-other-op-syntax-tokens)
 			       (c-lang-const c-operator-list))
 		       :test 'string-equal))
 
@@ -1587,13 +1596,14 @@
 (c-lang-defvar c-syntactic-eol (c-lang-const c-syntactic-eol))
 
 \f
-;;; Defun functions
+;;; Defun handling.
 
-;; The Emacs variables beginning-of-defun-function and
-;; end-of-defun-function will be set so that commands like
-;; `mark-defun' and `narrow-to-defun' work right.  The key sequences
-;; C-M-a and C-M-e are, however, bound directly to the CC Mode
-;; functions, allowing optimization for large n.
+;; The Emacs variables beginning-of-defun-function and end-of-defun-function
+;; will be set so that commands like `mark-defun' and `narrow-to-defun' work
+;; right.  In older Emacsen, the key sequences C-M-a and C-M-e are, however,
+;; bound directly to the CC Mode functions, allowing optimisation for large n.
+;; From Emacs 23, this isn't necessary any more, since n is passed to the two
+;; functions.
 (c-lang-defconst beginning-of-defun-function
   "Function to which beginning-of-defun-function will be set."
   t 'c-beginning-of-defun
@@ -1754,7 +1764,7 @@
 (c-lang-defconst c-type-start-kwds
   ;; All keywords that can start a type (i.e. are either a type prefix
   ;; or a complete type).
-  t (cl-delete-duplicates (append (c-lang-const c-primitive-type-kwds)
+  t (delete-duplicates (append (c-lang-const c-primitive-type-kwds)
 			       (c-lang-const c-type-prefix-kwds)
 			       (c-lang-const c-type-modifier-kwds))
 		       :test 'string-equal))
@@ -1998,7 +2008,7 @@
   ;; something is a type or just some sort of macro in front of the
   ;; declaration.  They might be ambiguous with types or type
   ;; prefixes.
-  t (cl-delete-duplicates (append (c-lang-const c-class-decl-kwds)
+  t (delete-duplicates (append (c-lang-const c-class-decl-kwds)
 			       (c-lang-const c-brace-list-decl-kwds)
 			       (c-lang-const c-other-block-decl-kwds)
 			       (c-lang-const c-typedef-decl-kwds)
@@ -2192,7 +2202,7 @@
   pike '("array" "function" "int" "mapping" "multiset" "object" "program"))
 
 (c-lang-defconst c-paren-any-kwds
-  t (cl-delete-duplicates (append (c-lang-const c-paren-nontype-kwds)
+  t (delete-duplicates (append (c-lang-const c-paren-nontype-kwds)
 			       (c-lang-const c-paren-type-kwds))
 		       :test 'string-equal))
 
@@ -2218,7 +2228,7 @@
 
 (c-lang-defconst c-<>-sexp-kwds
   ;; All keywords that can be followed by an angle bracket sexp.
-  t (cl-delete-duplicates (append (c-lang-const c-<>-type-kwds)
+  t (delete-duplicates (append (c-lang-const c-<>-type-kwds)
 			       (c-lang-const c-<>-arglist-kwds))
 		       :test 'string-equal))
 
@@ -2278,7 +2288,7 @@
 
 (c-lang-defconst c-block-stmt-kwds
   ;; Union of `c-block-stmt-1-kwds' and `c-block-stmt-2-kwds'.
-  t (cl-delete-duplicates (append (c-lang-const c-block-stmt-1-kwds)
+  t (delete-duplicates (append (c-lang-const c-block-stmt-1-kwds)
 			       (c-lang-const c-block-stmt-2-kwds))
 		       :test 'string-equal))
 
@@ -2382,7 +2392,7 @@
 (c-lang-defconst c-expr-kwds
   ;; Keywords that can occur anywhere in expressions.  Built from
   ;; `c-primary-expr-kwds' and all keyword operators in `c-operators'.
-  t (cl-delete-duplicates
+  t (delete-duplicates
      (append (c-lang-const c-primary-expr-kwds)
 	     (c-filter-ops (c-lang-const c-operator-list)
 			   t
@@ -2486,7 +2496,7 @@
 
 (c-lang-defconst c-keywords
   ;; All keywords as a list.
-  t (cl-delete-duplicates
+  t (delete-duplicates
      (c-lang-defconst-eval-immediately
       `(append ,@(mapcar (lambda (kwds-lang-const)
 			   `(c-lang-const ,kwds-lang-const))
@@ -2826,7 +2836,7 @@
 (c-lang-defvar c-block-prefix-charset (c-lang-const c-block-prefix-charset))
 
 (c-lang-defconst c-type-decl-prefix-key
-  "Regexp matching the declarator operators that might precede the
+  "Regexp matching any declarator operator that might precede the
 identifier in a declaration, e.g. the \"*\" in \"char *argv\".  This
 regexp should match \"(\" if parentheses are valid in declarators.
 The end of the first submatch is taken as the end of the operator.
@@ -2971,17 +2981,15 @@
 	  (when (boundp (c-mode-symbol "font-lock-extra-types"))
 	    (c-mode-var "font-lock-extra-types")))
 	 (regexp-strings
-	  (apply 'nconc
-		 (mapcar (lambda (re)
-		    (when (string-match "[][.*+?^$\\]" re)
-		      (list re)))
-		  extra-types)))
+	  (delq nil (mapcar (lambda (re)
+			      (when (string-match "[][.*+?^$\\]" re)
+				re))
+			    extra-types)))
 	 (plain-strings
-	  (apply 'nconc
-		 (mapcar (lambda (re)
-		    (unless (string-match "[][.*+?^$\\]" re)
-		      (list re)))
-		  extra-types))))
+	  (delq nil (mapcar (lambda (re)
+			      (unless (string-match "[][.*+?^$\\]" re)
+				re))
+			    extra-types))))
     (concat "\\<\\("
 	    (c-concat-separated
 	     (append (list (c-make-keywords-re nil
@@ -3025,7 +3033,8 @@
 expression is considered to be a type."
   t (or (consp (c-lang-const c-<>-type-kwds))
 	(consp (c-lang-const c-<>-arglist-kwds)))
-  java t)
+  java t)	    ; 2008-10-19.  This is crude.  The syntax for java
+		    ; generics is not yet coded in CC Mode.
 (c-lang-defvar c-recognize-<>-arglists (c-lang-const c-recognize-<>-arglists))
 
 (c-lang-defconst c-enums-contain-decls
@@ -3249,10 +3258,10 @@
 			     ;; `c-lang-const' will expand to the evaluated
 			     ;; constant immediately in `macroexpand-all'
 			     ;; below.
-			      (cl-mapcan
+			      (mapcan
 			       (lambda (init)
 				 `(current-var ',(car init)
-				   ,(car init) ,(macroexpand-all
+				   ,(car init) ,(c--macroexpand-all
 						 (elt init 1))))
 			       ;; Note: The following `append' copies the
 			       ;; first argument.  That list is small, so

=== modified file 'lisp/progmodes/cc-menus.el'
--- lisp/progmodes/cc-menus.el	2014-01-01 07:43:34 +0000
+++ lisp/progmodes/cc-menus.el	2014-09-25 18:32:58 +0000
@@ -45,6 +45,7 @@
 (cc-bytecomp-defvar imenu-case-fold-search)
 (cc-bytecomp-defvar imenu-generic-expression)
 (cc-bytecomp-defvar imenu-create-index-function)
+(cc-bytecomp-defun imenu-progress-message)
 
 \f
 ;; imenu integration
@@ -361,7 +362,7 @@
 	    p (1+ p))
       (cond
        ;; Is CHAR part of a objc token?
-       ((and (not inargvar)     ; Ignore if CHAR is part of an argument variable.
+       ((and (not inargvar)   ; Ignore if CHAR is part of an argument variable.
 	     (eq 0 betweenparen) ; Ignore if CHAR is in parentheses.
 	     (or (and (<= ?a char) (<= char ?z))
 		 (and (<= ?A char) (<= char ?Z))

=== modified file 'lisp/progmodes/cc-mode.el'
--- lisp/progmodes/cc-mode.el	2014-09-10 21:38:11 +0000
+++ lisp/progmodes/cc-mode.el	2014-09-25 20:30:37 +0000
@@ -97,7 +97,11 @@
 
 ;; Silence the compiler.
 (cc-bytecomp-defvar adaptive-fill-first-line-regexp) ; Emacs
+(cc-bytecomp-defun set-keymap-parents)	; XEmacs
 (cc-bytecomp-defun run-mode-hooks)	; Emacs 21.1
+(cc-bytecomp-defvar normal-erase-is-backspace)
+(cc-bytecomp-defvar file-local-variables-alist)
+(cc-bytecomp-defvar dir-local-variables-alist)
 
 ;; We set these variables during mode init, yet we don't require
 ;; font-lock.
@@ -108,11 +112,6 @@
 ;; with your version of Emacs, you are incompatible!
 (cc-external-require 'easymenu)
 
-;; Autoload directive for emacsen that doesn't have an older CC Mode
-;; version in the dist.
-(autoload 'subword-mode "subword"
-  "Mode enabling subword movement and editing keys." t)
-
 ;; Load cc-fonts first after font-lock is loaded, since it isn't
 ;; necessary until font locking is requested.
 ; (eval-after-load "font-lock" ; 2006-07-09: font-lock is now preloaded.
@@ -185,8 +184,7 @@
 	    (run-hooks 'c-initialization-hook)
 	    ;; Fix obsolete variables.
 	    (if (boundp 'c-comment-continuation-stars)
-		(setq c-block-comment-prefix
-		      (symbol-value 'c-comment-continuation-stars)))
+		(setq c-block-comment-prefix c-comment-continuation-stars))
 	    (add-hook 'change-major-mode-hook 'c-leave-cc-mode-mode)
 	    (setq c-initialization-ok t)
 	    ;; Connect up with Emacs's electric-indent-mode, for >= Emacs 24.4
@@ -380,7 +378,7 @@
   ;; conflicts with OOBR
   ;;(define-key c-mode-base-map "\C-c\C-v"  'c-version)
   ;; (define-key c-mode-base-map "\C-c\C-y"  'c-toggle-hungry-state)  Commented out by ACM, 2005-11-22.
-  (define-key c-mode-base-map "\C-c\C-w" 'subword-mode)
+  (define-key c-mode-base-map "\C-c\C-w" 'c-subword-mode)
   )
 
 ;; We don't require the outline package, but we configure it a bit anyway.
@@ -472,6 +470,14 @@
 (defvar c-maybe-stale-found-type nil)
 (make-variable-buffer-local 'c-maybe-stale-found-type)
 
+(defvar c-just-done-before-change nil)
+(make-variable-buffer-local 'c-just-done-before-change)
+;; This variable is set to t by `c-before-change' and to nil by
+;; `c-after-change'.  It is used to detect a spurious invocation of
+;; `before-change-functions' directly following on from a correct one.  This
+;; happens in some Emacsen, for example when `basic-save-buffer' does (insert
+;; ?\n) when `require-final-newline' is non-nil.
+
 (defun c-basic-common-init (mode default-style)
   "Do the necessary initialization for the syntax handling routines
 and the line breaking/filling code.  Intended to be used by other
@@ -542,10 +548,11 @@
   ;; Use this in Emacs 21+ to avoid meddling with the rear-nonsticky
   ;; property on each character.
   (when (boundp 'text-property-default-nonsticky)
+    (make-local-variable 'text-property-default-nonsticky)
     (mapc (lambda (tprop)
 	    (unless (assq tprop text-property-default-nonsticky)
-	      (set (make-local-variable 'text-property-default-nonsticky)
-                   (cons `(,tprop . t) text-property-default-nonsticky))))
+	      (setq text-property-default-nonsticky
+                    (cons `(,tprop . t) text-property-default-nonsticky))))
 	  '(syntax-table category c-type)))
 
   ;; In Emacs 21 and later it's possible to turn off the ad-hoc
@@ -601,14 +608,15 @@
 
   ;; Install the functions that ensure that various internal caches
   ;; don't become invalid due to buffer changes.
-  (when (featurep 'xemacs)
-    (make-local-hook 'before-change-functions)
-    (make-local-hook 'after-change-functions))
+  (c--make-local-hook 'before-change-functions)
   (add-hook 'before-change-functions 'c-before-change nil t)
+  (setq c-just-done-before-change nil)
+  (c--make-local-hook 'after-change-functions)
   (add-hook 'after-change-functions 'c-after-change nil t)
-  (set (make-local-variable 'font-lock-extend-after-change-region-function)
- 	'c-extend-after-change-region))	; Currently (2009-05) used by all
-			; languages with #define (C, C++,; ObjC), and by AWK.
+  (when (boundp 'font-lock-extend-after-change-region-function)
+    (set (make-local-variable 'font-lock-extend-after-change-region-function)
+         'c-extend-after-change-region))) ; Currently (2009-05) used by all
+                          ; languages with #define (C, C++,; ObjC), and by AWK.
 
 (defun c-setup-doc-comment-style ()
   "Initialize the variables that depend on the value of `c-doc-comment-style'."
@@ -669,9 +677,11 @@
 	 (or (c-cpp-define-name) (c-defun-name))))
   (let ((rfn (assq mode c-require-final-newline)))
     (when rfn
-      (and (cdr rfn)
-	   (set (make-local-variable 'require-final-newline)
-                mode-require-final-newline)))))
+      (if (boundp 'mode-require-final-newline)
+          (and (cdr rfn)
+               (set (make-local-variable 'require-final-newline)
+                    mode-require-final-newline))
+        (set (make-local-variable 'require-final-newline) (cdr rfn))))))
 
 (defun c-count-cfss (lv-alist)
   ;; LV-ALIST is an alist like `file-local-variables-alist'.  Count how many
@@ -948,8 +958,11 @@
 	  c-new-END (min (cdr new-bounds) (c-determine-+ve-limit 500 endd)))
     ;; Clear all old relevant properties.
     (c-clear-char-property-with-value c-new-BEG c-new-END 'syntax-table '(1))
-    (c-clear-char-property-with-value c-new-BEG c-new-END 'category 'c-cpp-delimiter)
-    ;; FIXME!!!  What about the "<" and ">" category properties?  2009-11-16
+
+    ;; CPP "comment" markers:
+    (if (eval-when-compile (memq 'category-properties c-emacs-features));Emacs.
+	(c-clear-char-property-with-value
+	 c-new-BEG c-new-END 'category 'c-cpp-delimiter))
 
     ;; Add needed properties to each CPP construct in the region.
     (goto-char c-new-BEG)
@@ -967,8 +980,10 @@
 	  (setq mbeg (point))
 	  (if (> (c-syntactic-end-of-macro) mbeg)
 	      (progn
-		(c-neutralize-CPP-line mbeg (point))
-		(c-set-cpp-delimiters mbeg (point)))
+		(c-neutralize-CPP-line mbeg (point)) ; "punctuation" properties
+		(if (eval-when-compile
+                      (memq 'category-properties c-emacs-features)) ;Emacs.
+		    (c-set-cpp-delimiters mbeg (point)))) ; "comment" markers
 	    (forward-line))	      ; no infinite loop with, e.g., "#//"
 	  )))))
 
@@ -988,64 +1003,67 @@
   ;; it/them from the cache.  Don't worry about being inside a string
   ;; or a comment - "wrongly" removing a symbol from `c-found-types'
   ;; isn't critical.
-  (setq c-maybe-stale-found-type nil)
-  (save-restriction
-    (save-match-data
-      (widen)
-      (save-excursion
-	;; Are we inserting/deleting stuff in the middle of an identifier?
-	(c-unfind-enclosing-token beg)
-	(c-unfind-enclosing-token end)
-	;; Are we coalescing two tokens together, e.g. "fo o" -> "foo"?
-	(when (< beg end)
-	  (c-unfind-coalesced-tokens beg end))
-	;; Are we (potentially) disrupting the syntactic context which
-	;; makes a type a type?  E.g. by inserting stuff after "foo" in
-	;; "foo bar;", or before "foo" in "typedef foo *bar;"?
-	;;
-	;; We search for appropriate c-type properties "near" the change.
-	;; First, find an appropriate boundary for this property search.
-	(let (lim
-	      type type-pos
-	      marked-id term-pos
-	      (end1
-	       (or (and (eq (get-text-property end 'face) 'font-lock-comment-face)
-			(previous-single-property-change end 'face))
-		   end)))
-	  (when (>= end1 beg) ; Don't hassle about changes entirely in comments.
-	    ;; Find a limit for the search for a `c-type' property
-	    (while
-		(and (/= (skip-chars-backward "^;{}") 0)
-		     (> (point) (point-min))
-		     (memq (c-get-char-property (1- (point)) 'face)
-			   '(font-lock-comment-face font-lock-string-face))))
-	    (setq lim (max (point-min) (1- (point))))
-
-	    ;; Look for the latest `c-type' property before end1
-	    (when (and (> end1 (point-min))
-		       (setq type-pos
-			     (if (get-text-property (1- end1) 'c-type)
-				 end1
-			       (previous-single-property-change end1 'c-type nil lim))))
-	      (setq type (get-text-property (max (1- type-pos) lim) 'c-type))
-
-	      (when (memq type '(c-decl-id-start c-decl-type-start))
-		;; Get the identifier, if any, that the property is on.
-		(goto-char (1- type-pos))
-		(setq marked-id
-		      (when (looking-at "\\(\\sw\\|\\s_\\)")
-			(c-beginning-of-current-token)
-			(buffer-substring-no-properties (point) type-pos)))
-
-		(goto-char end1)
-		(skip-chars-forward "^;{}") ; FIXME!!!  loop for comment, maybe
-		(setq lim (point))
-		(setq term-pos
-		      (or (next-single-property-change end 'c-type nil lim) lim))
-		(setq c-maybe-stale-found-type
-		      (list type marked-id
-			    type-pos term-pos
-			    (buffer-substring-no-properties type-pos term-pos)
+  (unless c-just-done-before-change   ; guard against a spurious second
+					; invocation of before-change-functions.
+    (setq c-just-done-before-change t)
+    (setq c-maybe-stale-found-type nil)
+    (save-restriction
+      (save-match-data
+	(widen)
+	(save-excursion
+	  ;; Are we inserting/deleting stuff in the middle of an identifier?
+	  (c-unfind-enclosing-token beg)
+	  (c-unfind-enclosing-token end)
+	  ;; Are we coalescing two tokens together, e.g. "fo o" -> "foo"?
+	  (when (< beg end)
+	    (c-unfind-coalesced-tokens beg end))
+	  ;; Are we (potentially) disrupting the syntactic context which
+	  ;; makes a type a type?  E.g. by inserting stuff after "foo" in
+	  ;; "foo bar;", or before "foo" in "typedef foo *bar;"?
+	  ;;
+	  ;; We search for appropriate c-type properties "near" the change.
+	  ;; First, find an appropriate boundary for this property search.
+	  (let (lim
+		type type-pos
+		marked-id term-pos
+		(end1
+		 (or (and (eq (get-text-property end 'face) 'font-lock-comment-face)
+			  (previous-single-property-change end 'face))
+		     end)))
+	    (when (>= end1 beg) ; Don't hassle about changes entirely in comments.
+	      ;; Find a limit for the search for a `c-type' property
+	      (while
+		  (and (/= (skip-chars-backward "^;{}") 0)
+		       (> (point) (point-min))
+		       (memq (c-get-char-property (1- (point)) 'face)
+			     '(font-lock-comment-face font-lock-string-face))))
+	      (setq lim (max (point-min) (1- (point))))
+
+	      ;; Look for the latest `c-type' property before end1
+	      (when (and (> end1 (point-min))
+			 (setq type-pos
+			       (if (get-text-property (1- end1) 'c-type)
+				   end1
+				 (previous-single-property-change end1 'c-type nil lim))))
+		(setq type (get-text-property (max (1- type-pos) lim) 'c-type))
+
+		(when (memq type '(c-decl-id-start c-decl-type-start))
+		  ;; Get the identifier, if any, that the property is on.
+		  (goto-char (1- type-pos))
+		  (setq marked-id
+			(when (looking-at "\\(\\sw\\|\\s_\\)")
+			  (c-beginning-of-current-token)
+			  (buffer-substring-no-properties (point) type-pos)))
+
+		  (goto-char end1)
+		  (skip-chars-forward "^;{}") ; FIXME!!!  loop for comment, maybe
+		  (setq lim (point))
+		  (setq term-pos
+			(or (c-next-single-property-change end 'c-type nil lim) lim))
+		  (setq c-maybe-stale-found-type
+			(list type marked-id
+			      type-pos term-pos
+			      (buffer-substring-no-properties type-pos term-pos)
 			      (buffer-substring-no-properties beg end)))))))
 
 	  (if c-get-state-before-change-functions
@@ -1056,7 +1074,7 @@
 	  )))
     ;; The following must be done here rather than in `c-after-change' because
     ;; newly inserted parens would foul up the invalidation algorithm.
-  (c-invalidate-state-cache beg))
+    (c-invalidate-state-cache beg)))
 
 (defvar c-in-after-change-fontification nil)
 (make-variable-buffer-local 'c-in-after-change-fontification)
@@ -1077,6 +1095,7 @@
   ;; This calls the language variable c-before-font-lock-functions, if non nil.
   ;; This typically sets `syntax-table' properties.
 
+  (setq c-just-done-before-change nil)
   (c-save-buffer-state (case-fold-search open-paren-in-column-0-is-defun-start)
     ;; When `combine-after-change-calls' is used we might get calls
     ;; with regions outside the current narrowing.  This has been
@@ -1097,11 +1116,12 @@
 	;; C-y is capable of spuriously converting category properties
 	;; c-</>-as-paren-syntax and c-cpp-delimiter into hard syntax-table
 	;; properties.  Remove these when it happens.
-	(c-clear-char-property-with-value beg end 'syntax-table
-					  c-<-as-paren-syntax)
-	(c-clear-char-property-with-value beg end 'syntax-table
-					  c->-as-paren-syntax)
-	(c-clear-char-property-with-value beg end 'syntax-table nil)
+	(when (eval-when-compile (memq 'category-properties c-emacs-features))
+	  (c-clear-char-property-with-value beg end 'syntax-table
+					    c-<-as-paren-syntax)
+	  (c-clear-char-property-with-value beg end 'syntax-table
+					    c->-as-paren-syntax)
+	  (c-clear-char-property-with-value beg end 'syntax-table nil))
 
 	(c-trim-found-types beg end old-len) ; maybe we don't need all of these.
 	(c-invalidate-sws-region-after beg end)
@@ -1235,10 +1255,10 @@
   (make-local-variable 'font-lock-fontify-region-function)
   (setq font-lock-fontify-region-function 'c-font-lock-fontify-region)
 
-  (if (featurep 'xemacs)
-      (make-local-hook 'font-lock-mode-hook))
+  (c--make-local-hook 'font-lock-mode-hook)
   (add-hook 'font-lock-mode-hook 'c-after-font-lock-init nil t))
 
+;; Emacs 22 and later.
 (defun c-extend-after-change-region (_beg _end _old-len)
   "Extend the region to be fontified, if necessary."
   ;; Note: the parameters are ignored here.  This somewhat indirect
@@ -1252,6 +1272,21 @@
   ;; function.
   (cons c-new-BEG c-new-END))
 
+;; Emacs < 22 and XEmacs
+(defmacro c-advise-fl-for-region (function)
+  `(defadvice ,function (before get-awk-region activate)
+     ;; Make sure that any string/regexp is completely font-locked.
+     (when c-buffer-is-cc-mode
+       (save-excursion
+	 (ad-set-arg 1 c-new-END)   ; end
+	 (ad-set-arg 0 c-new-BEG)))))	; beg
+
+(unless (boundp 'font-lock-extend-after-change-region-function)
+  (c-advise-fl-for-region font-lock-after-change-function)
+  (c-advise-fl-for-region jit-lock-after-change)
+  (c-advise-fl-for-region lazy-lock-defer-rest-after-change)
+  (c-advise-fl-for-region lazy-lock-defer-line-after-change))
+
 ;; Connect up to `electric-indent-mode' (Emacs 24.4 and later).
 (defun c-electric-indent-mode-hook ()
   ;; Emacs has en/disabled `electric-indent-mode'.  Propagate this through to
@@ -1322,6 +1357,7 @@
 ;;;###autoload (add-to-list 'auto-mode-alist '("\\.i\\'" . c-mode))
 ;;;###autoload (add-to-list 'auto-mode-alist '("\\.ii\\'" . c++-mode))
 
+(unless (fboundp 'prog-mode) (defalias 'prog-mode 'fundamental-mode))
 
 ;;;###autoload
 (define-derived-mode c-mode prog-mode "C"

=== modified file 'lisp/progmodes/cc-styles.el'
--- lisp/progmodes/cc-styles.el	2014-01-01 07:43:34 +0000
+++ lisp/progmodes/cc-styles.el	2014-09-25 18:44:14 +0000
@@ -645,7 +645,7 @@
     (mapc func varsyms)
     ;; Hooks must be handled specially
     (if this-buf-only-p
-	(if (featurep 'xemacs) (make-local-hook 'c-special-indent-hook))
+        (c--make-local-hook 'c-special-indent-hook)
       (with-no-warnings (make-variable-buffer-local 'c-special-indent-hook))
       (setq c-style-variables-are-local-p t))
     ))

=== modified file 'lisp/progmodes/cc-vars.el'
--- lisp/progmodes/cc-vars.el	2014-01-01 07:43:34 +0000
+++ lisp/progmodes/cc-vars.el	2014-09-25 21:16:12 +0000
@@ -42,6 +42,9 @@
 
 (cc-require 'cc-defs)
 
+;; Silence the compiler.
+(cc-bytecomp-defun get-char-table)	; XEmacs
+
 (cc-eval-when-compile
   (require 'custom)
   (require 'widget))
@@ -185,14 +188,14 @@
 `c-style-alist') when a CC Mode buffer is initialized.  Otherwise,
 the value set here overrides the style system (there is a variable
 `c-old-style-variable-behavior' that changes this, though)."))
-         (typ (eval (plist-get args :type)))
+	 (typ (eval (c-safe (plist-get args :type))))
          (type (if (consp typ) typ (list typ)))
          (head (car type))
          (tail (cdr type))
-         (newt (append (unless (plist-get tail :tag)
+	 (newt (append (unless (c-safe (plist-get tail :tag))
                          '(:tag "Override style settings"))
-                       (unless (plist-get tail :value)
-                         `(:value ,(eval val)))
+		       (unless (c-safe (plist-get tail :value))
+			 `(:value ,val))
                        tail))
          (aggregate `'(radio
                        (const :tag "Use style settings" set-from-style)
@@ -275,13 +278,18 @@
   "*Controls the operation of the TAB key.
 If t, hitting TAB always just indents the current line.  If nil, hitting
 TAB indents the current line if point is at the left margin or in the
-line's indentation, otherwise it calls `c-insert-tab-function' to
-insert a `real' tab character.  If some other value (neither nil nor t),
-then inserts a tab only within literals (comments and strings), but
-always reindents the line.
-
-Note: the variable `c-comment-only-line-offset' also controls the
-indentation of lines containing only comments."
+line's indentation, otherwise it inserts a `real' tab character \(see
+note\).	 If some other value (not nil or t), then tab is inserted only
+within literals \(comments and strings), but the line is always
+reindented.
+
+Note: The value of `indent-tabs-mode' will determine whether a real
+tab character will be inserted, or the equivalent number of spaces.
+When inserting a tab, actually the function stored in the variable
+`c-insert-tab-function' is called.
+
+Note: indentation of lines containing only comments is also controlled
+by the `c-comment-only-line-offset' variable."
   :type '(radio
 	  (const :tag "TAB key always indents, never inserts TAB" t)
 	  (const :tag "TAB key indents in left margin, otherwise inserts TAB" nil)
@@ -456,7 +463,8 @@
   :group 'c)
 
 (make-obsolete-variable 'c-comment-continuation-stars
-			'c-block-comment-prefix "21.1")
+			'c-block-comment-prefix
+			nil)
 
 ;; Although c-comment-continuation-stars is obsolete, we look at it in
 ;; some places in CC Mode anyway, so make the compiler ignore it
@@ -920,7 +928,7 @@
 (defcustom c-special-indent-hook nil
   "*Hook for user defined special indentation adjustments.
 This hook gets called after each line is indented by the mode.  It is only
-called if `c-syntactic-indentation' is non-nil."
+called when `c-syntactic-indentation' is non-nil."
   :type 'hook
   :group 'c)
 
@@ -1689,7 +1697,8 @@
 ;; It isn't possible to specify a doc-string without specifying an
 ;; initial value with `defvar', so the following two variables have been
 ;; given doc-strings by setting the property `variable-documentation'
-;; directly.  It's really good not to have an initial value for
+;; directly.  C-h v will read this documentation only for versions of GNU
+;; Emacs from 22.1.  It's really good not to have an initial value for
 ;; variables like these that always should be dynamically bound, so it's
 ;; worth the inconvenience.
 

Re: Further CC-mode changes

[Glenn Morris]

Alan Mackenzie wrote:

>> It's work I did, pro bono.  Just enjoy it.
>
> No, you haven't.  What's the equivalent of `syntax-propertize' in XEmacs,
> or an old GNU Emacs?  "Somebody" (tm) has to write a compatibility macro
> which will look something like this:

No, nobody has to do that.

You've got limited time to work on cc-mode, and more issues than you can
solve. If you choose to spend some of that time supporting old Emacs
versions (and XEmacs by now is a very old Emacs version), then that is
reducing the amount of time you can spend on eg improving C++11 support
(to pick a non-random example). IMO this is a clear loss, so I hope you
will reconsider your position on that.

I imagine Stefan is one of the few people capable of
modernizing/simplifying/standardizing cc-mode, so FWIW I strongly
encourage you to accept the offer.

Re: Further CC-mode changes

[Stephen J. Turnbull]

Glenn Morris writes:

 > solve. If you choose to spend some of that time supporting old Emacs
 > versions (and XEmacs by now is a very old Emacs version), then that is
 > reducing the amount of time you can spend on eg improving C++11 support
 > (to pick a non-random example). IMO this is a clear loss, so I hope you
 > will reconsider your position on that.

-1

Re: Further CC-mode changes

[Stefan Monnier]

>> solve.  If you choose to spend some of that time supporting old Emacs
>> versions (and XEmacs by now is a very old Emacs version), then that is
>> reducing the amount of time you can spend on eg improving C++11 support
>> (to pick a non-random example).  IMO this is a clear loss, so I hope you
>> will reconsider your position on that.
> -1

IOW, the issue is not "should CC-mode drop support for XEmacs?", but
"should CC-mode's maintainer spend his time preserving XEmacs
compatibility?".  While the indentation code is rather involved (and
hence difficult for someone other than Alan to hack on), the code need
for compatibility with a particular emacs is not particularly deep
or tricky, so many more people can contribute to that.


        Stefan

Re: Further CC-mode changes

[David Kastrup]

"Stephen J. Turnbull" <stephen@xemacs.org> writes:

> Glenn Morris writes:
>
>  > solve. If you choose to spend some of that time supporting old Emacs
>  > versions (and XEmacs by now is a very old Emacs version), then that is
>  > reducing the amount of time you can spend on eg improving C++11 support
>  > (to pick a non-random example). IMO this is a clear loss, so I hope you
>  > will reconsider your position on that.
>
> -1

I think it depends.  "Gratuitous incompatibility" is one thing, but
"refrain from relying on fixes, useful APIs etc that exist for years
already" is another.  The responsibility for keeping the core of XEmacs
in good working shape, or rather the cost for _not_ keeping the core of
XEmacs in good working shape should not be loaded off on Emacs
developers.  Why should we factually ignore years of work on making
Emacs more useful?

-- 
David Kastrup

Re: Further CC-mode changes

[Stephen J. Turnbull]

David Kastrup writes:

 > I think it depends.  "Gratuitous incompatibility" is one thing, but
 > "refrain from relying on fixes, useful APIs etc that exist for years
 > already" is another.

*You* do understand the irony of that claim, don't you?

But no, what it depends on is whether you use those versions or not.
Two separate surveys of large (>100) corporate Emacsen populations
taken in 2003 and 2007 showed that around 10% of users were using
10-year-old or older Emacsen.  Even Microsoft has been taught that
lesson.  The trend in forward-looking projects is to "long term
support" of various kinds, invariably involving multiple versions and
even competing implementations, and even longer-term support is one of
the things that keeps free software businesses profitable.

I think it's ironic that when it comes to the XEmacs support Alan
wants to provide, Glenn is all for "aggressive modernization" and
devil take the hindmost, but when it comes to the same policy toward
the git repo Eric wants to provide, he's the loudest of footdraggers.
I think that is pretty paradigmatic of Emacs, and one of the reasons
why Emacs is itself considered "unprogressive" by most programmers.

Re: Further CC-mode changes

[Stefan Monnier]

> But no, what it depends on is whether you use those versions or not.
> Two separate surveys of large (>100) corporate Emacsen populations
> taken in 2003 and 2007 showed that around 10% of users were using
> 10-year-old or older Emacsen.

I have no illusion about that (although the general trend has changed
slightly due to the constant security update needs).

The issue is not "backward compatibility" but *how* to get it.


        Stefan

Re: Further CC-mode changes

[Stephen J. Turnbull]

Stefan Monnier writes:

 > The issue is not "backward compatibility" but *how* to get it.

Evidently not for Glenn, David, and IIRC RMS, though.  I think in
fact you are in a minority in the Emacs leadership[1] when it comes
to backwards compatibility.

Footnotes: 
[1]  leadership ~= vocal on emacs-devel, ie, in the sense of voicing
opinions about which sensible people disagree, but should be heard.

Re: Further CC-mode changes

[David Kastrup]

"Stephen J. Turnbull" <stephen@xemacs.org> writes:

> Stefan Monnier writes:
>
>  > The issue is not "backward compatibility" but *how* to get it.
>
> Evidently not for Glenn, David, and IIRC RMS, though.  I think in
> fact you are in a minority in the Emacs leadership[1] when it comes
> to backwards compatibility.

You are mixing up two entirely different topics under the title of
"backward" compatibility.

One thing is how long user-provided code should be expected to continue
working in new Emacs versions.  How long should old interfaces leading
to more cumbersome code continue to be provided?

But that's not what we are talking about here _at_ _all_.  Quite the
contrary.  The answer to that has more or less been resolved to "pretty
long" and that makes the actual question even possible:

And that question is: how long should new code intended for Emacs
refrain from relying on new interfaces and new features which have been
created for a reason?  And the answer "15 years because XEmacs won't
upgrade" is not satisfactory.  At some point of time, people who are
happy with a 15 year old Emacs variant will have to face updating either
XEmacs, or backporting software they care for, or just using older
versions of software on XEmacs.

-- 
David Kastrup

Re: Further CC-mode changes

[Stefan Monnier]

> And that question is: how long should new code intended for Emacs
> refrain from relying on new interfaces and new features which have been
> created for a reason?

Actually, that's not really the question either.
I think the questions Alan is facing are more along the lines of:

- should I just keep using my old code which is compatible by a mix of
  "do it all by hand" plus "coerce the few packages on which I need to rely".
- or should I start using the new interface, which integrates better
  with the rest of the system, but comes with some downsides:
  - the new system doesn't work quite like my old one, and I prefer my
    system (after all, *I* designed it, so it does exactly what I want).
  - since it doesn't work quite like my old one, I might introduce bugs.
  - the new one is not supported everywhere, so I have to add new
    compatibility code.

In the short run, the first option (the one usually favored by Alan,
AFAICT) is clearly better.


        Stefan

Re: Further CC-mode changes

[Glenn Morris]

>> Two separate surveys of large (>100) corporate Emacsen populations
>> taken in 2003 and 2007 showed that around 10% of users were using
>> 10-year-old or older Emacsen.

I knew this by now very tired old statement would get trotted out. :)

Obviously it doesn't matter what I think, and I don't expect it to
change anyone's opinion, but based on Stephen's response I'll stop
beating around the bush.

The point was made 6 years ago, and things certainly have not improved
since then:

http://steve-yegge.blogspot.com/2008/04/xemacs-is-dead-long-live-xemacs.html

   But at this point, if you're using XEmacs you're actively damaging
   not only your long-term productivity, but mine as well.

Maintaining compat code for ancient Emacs versions is a waste of
effort, and in fact makes things worse for people using modern Emacs.

https://www.openhub.net/p/emacs  4500 commits in the last year
https://www.openhub.net/p/xemacs   48 commits in the last year

Extend the sample to 5 years if you like, same result.

Throw in SXEmacs too if you like, that had 35 commits last year.

Yes, counting commits is a silly metric, but the result is the same by
any metric. Bug reports closed in the last year? ~ 0 (?) versus 1000s.
New features have been added in the last year (5 years)?
I-don't-know versus see the 4000-line NEWS file.

Re: Further CC-mode changes

[David Kastrup]

Glenn Morris <rgm@gnu.org> writes:

>>> Two separate surveys of large (>100) corporate Emacsen populations
>>> taken in 2003 and 2007 showed that around 10% of users were using
>>> 10-year-old or older Emacsen.
>
> I knew this by now very tired old statement would get trotted out. :)

[...]

I think you are missing one point in your repartee:

Why would it make a difference to those people not updating their
systems what requirements a current version of CC-mode has?

-- 
David Kastrup

Re: Further CC-mode changes

[Eli Zaretskii]

> From: "Stephen J. Turnbull" <stephen@xemacs.org>
> Date: Wed, 17 Sep 2014 08:40:25 +0900
> Cc: emacs-devel@gnu.org
> 
> I think it's ironic that when it comes to the XEmacs support Alan
> wants to provide, Glenn is all for "aggressive modernization" and
> devil take the hindmost, but when it comes to the same policy toward
> the git repo Eric wants to provide, he's the loudest of footdraggers.

That's unfair to Glenn, and is evidently based on just 2 data points
out of many more.

So let me add 2 more from my failing memory:

  . Glenn also lobbied hard to remove the old MS-Windows configury
    (which I objected, and therefore it didn't happen yet)

  . Glenn suggested that the trunk relies on GNU Make, which did
    happen, and made many useful changes which that enabled

I encourage you to browse the logs, because I'm sure you will find an
overwhelming evidence that your conclusion is based on incorrect
interpretation of a single incident.  If you consider the inordinate
amount of work Glenn personally invested in making bzr usage
convenient in Emacs, you might come up with an alternative explanation
to his alleged "footdragging".

Re: Further CC-mode changes

[Stephen J. Turnbull]

Eli Zaretskii writes:
 > > From: "Stephen J. Turnbull" <stephen@xemacs.org>
 > > Date: Wed, 17 Sep 2014 08:40:25 +0900
 > > Cc: emacs-devel@gnu.org
 > > 
 > > I think it's ironic that when it comes to the XEmacs support Alan
 > > wants to provide, Glenn is all for "aggressive modernization" and
 > > devil take the hindmost, but when it comes to the same policy toward
 > > the git repo Eric wants to provide, he's the loudest of footdraggers.
 > 
 > That's unfair to Glenn, and is evidently based on just 2 data points
 > out of many more.
 > 
 > So let me add 2 more from my failing memory:
 > 
 >   . Glenn also lobbied hard to remove the old MS-Windows configury
 >     (which I objected, and therefore it didn't happen yet)
 > 
 >   . Glenn suggested that the trunk relies on GNU Make, which did
 >     happen, and made many useful changes which that enabled

I consider those both to support my claim, which is that if it reduces
the amount of work he needs to do to achieve a unit of Emacs
development in the future, he's in favor of investing both his time,
and users' time and inconvenience, in the change.  If somebody else
does the same, but it inconveniences him, he complains.  That's
*ironic*, no matter how dedicated Glenn may be to Emacs development.

Of course I'm -1 on anything that makes it even harder for XEmacs to
catch up to current reality.  That shouldn't bother core Emacs people
too much, although I'm glad it's a consideration for Alan and many
other package developers.  And you shouldn't be surprised that this is
the point that piqued me enough to speak up.  But ...

My real point is that bzr has held Emacs back for long enough.  Both
Eric's obsession with the 36-24-36 conversion and Glenn's insistance
that everything that is part of the bzr workflow be done for git
*before* the switchover is executed are unfortunate.

 > If you consider the inordinate amount of work Glenn personally
 > invested in making bzr usage convenient in Emacs, you might come up
 > with an alternative explanation to his alleged "footdragging".

Sure.  The labor theory of value: if I worked so hard on it, it must
be worth keeping.  Thing is, bzr was a mistake from day 1, and it went
mostly downhill from there.  The consequent duplication of effort was
not Eric's fault!

Re: Further CC-mode changes

[Eli Zaretskii]

> From: "Stephen J. Turnbull" <stephen@xemacs.org>
> Date: Wed, 17 Sep 2014 15:54:32 +0900
> Cc: dak@gnu.org, emacs-devel@gnu.org
> 
> I consider those both to support my claim

Why am I not surprised?

> which is that if it reduces the amount of work he needs to do to
> achieve a unit of Emacs development in the future, he's in favor of
> investing both his time, and users' time and inconvenience, in the
> change.  If somebody else does the same, but it inconveniences him,
> he complains.  That's *ironic*, no matter how dedicated Glenn may be
> to Emacs development.

This is gross misrepresentation of Glenn's work and attitude, mixed
with mislabeling a natural behavior of a project developer as
"ironically" wrong.

Re: Further CC-mode changes

[David Kastrup]

"Stephen J. Turnbull" <stephen@xemacs.org> writes:

> Of course I'm -1 on anything that makes it even harder for XEmacs to
> catch up to current reality.

That's where our views of what we are talking about differ.  In my book,
we are talking about changes that make it even more pressing (but not
harder) for XEmacs to catch up to current reality.  The whole point of
extended backward compatibility upstream is to avoid the consequences of
not catching up to reality downstream.

> My real point is that bzr has held Emacs back for long enough.  Both
> Eric's obsession with the 36-24-36 conversion and Glenn's insistance
> that everything that is part of the bzr workflow be done for git
> *before* the switchover is executed are unfortunate.
>
>  > If you consider the inordinate amount of work Glenn personally
>  > invested in making bzr usage convenient in Emacs, you might come up
>  > with an alternative explanation to his alleged "footdragging".
>
> Sure.  The labor theory of value: if I worked so hard on it, it must
> be worth keeping.  Thing is, bzr was a mistake from day 1, and it went
> mostly downhill from there.

It didn't for Bazaar: it was the reason for a lot of scalability
problems in Bazaar getting ironed out.  Of course, doscovery of the
wrinkles impacted Emacs development.  That was expected.  Supporting
Bazaar was a political choice in order to keep with a version control
system which was looking better suited for the political goals of the
GNU projects.  A lot of work was expected to make things work well.

The technical side of Bazaar progressed to the state of "workable".  The
political side degressed to "barely workable" after a few years.

Which makes it a lost investment.  Which does not mean that all
investments of labor in the view of political goals must be lost
investments, or we would not be having a GNU project.

> The consequent duplication of effort was not Eric's fault!

The country of Oz has pronounced a travelling warning for strawmen on
the Emacs developer list as they are in serious danger of being rounded
up and beaten to death.

-- 
David Kastrup

Re: Further CC-mode changes

[Stefan Monnier]

> My real point is that bzr has held Emacs back for long enough.  Both
> Eric's obsession with the 36-24-36 conversion and Glenn's insistance
> that everything that is part of the bzr workflow be done for git
> *before* the switchover is executed are unfortunate.

Huh?  Why are they unfortunate?  It's not like they're delaying the
switchover (which is waiting for the 24.4 release).


        Stefan

Re: Further CC-mode changes

[Glenn Morris]

[Nice job shifting the discussion Stephen! :)]

>> My real point is that bzr has held Emacs back for long enough.  Both
>> Eric's obsession with the 36-24-36 conversion and Glenn's insistance
>> that everything that is part of the bzr workflow be done for git
>> *before* the switchover is executed are unfortunate.

I thought that the people who actually develop Emacs came up with a list
of things to be taken care of before the switch. I'd like to make sure
they all get taken care of, as we were promised, is all.

Re: Further CC-mode changes

[Stephen J. Turnbull]

Stefan Monnier writes:

 > > My real point is that bzr has held Emacs back for long enough.  Both
 > > Eric's obsession with the 36-24-36 conversion and Glenn's insistance
 > > that everything that is part of the bzr workflow be done for git
 > > *before* the switchover is executed are unfortunate.
 > 
 > Huh?  Why are they unfortunate?  It's not like they're delaying the
 > switchover (which is waiting for the 24.4 release).

That is, they're not on the critical path now.  Later?  If you say
not, I guess they're not, and I'm happy for Emacs.

Re: Further CC-mode changes

[Emilio Lopes]

2014-09-17 1:40 GMT+02:00 Stephen J. Turnbull <stephen@xemacs.org>:
> But no, what it depends on is whether you use those versions or not.
> Two separate surveys of large (>100) corporate Emacsen populations
> taken in 2003 and 2007 showed that around 10% of users were using
> 10-year-old or older Emacsen.  Even Microsoft has been taught that
> lesson.  […]

And from those "around 10% of users who were using 10-year-old or older
Emacsen", how many of them update their CC-mode?

 Emílio