Re: A possible way for CC Mode to resolve its sluggishness

[Stefan Monnier <monnier@iro.umontreal.ca>]

> The problem is that CC Mode's before/after-change-functions are very
> general, and scan the buffer looking for situations which only arise
> sporadically.  Things like an open string getting closed, or a > being
> inserted which needs to be checked for a template delimiter.  However,
> these expensive checks are performed for _every_ buffer change.  Even
> doing something like inserting a letter or a digit causes the full range
> of tests to be performed.  This is not good.

Part of the problem is that CC-mode is very eager in its management of
syntax information: the `syntax-table` text-properties are always kept
up-to-date over the whole buffer right after every single change.

Modes using syntax-propertize work more lazily: before-change-functions
only marks that some change occurred at position POS and the syntax-table
properties after that position are only updated afterward on-demand.

CC-mode tries to make up for it by being more clever about which parts of
the buffer after position POS actually need to be updated, but when
there are several consecutive changes, the extra work performed between
each one of those changes add up quickly.

[ Of course, there are cases where the approach used in
  syntax-propertize loses big time.  E.g. if you have a loop that first
  modifies a char near point-min, then asks for the syntax-table
  properties near point-max, and then repeats... performance will suck.
  But luckily I haven't yet seen a real-world use case where
  this occurs.  ]

Maybe another part of the problem is that CC-mode tries to do more than
most other major modes: e.g. the highlighting of unclosed strings.
For plain single-line strings this can be fairly cheap, but for
multiline strings, keeping this information constantly up-to-date over
the whole buffer can be costly.

Most other major modes just let the font-lock-string-face bleeds further
than the user intended, which requires much less work and works well
enough for all other syntactic elements (CC-mode doesn't highlight
unclosed parens, or mismatched parens, or `do` with missing `while`,
...).  When needed these many different kinds of errors are detected and
shown to the user via things like flymake or LSP instead, which work
much more lazily w.r.t buffer changes, so they don't need to same kind
of engineering efforts to make them fast enough.

> Thoughts?

Not sure whether you intend this to be just a change to CC-mode (it does
sound like it can all be implemented in Elisp) or you intend for some
change at the C level.  My gut feeling is that the checks you suggest in
(iii) could be implemented in Elisp without losing too much performance
(they should spend most of their time within a few C primitives), tho it
depends on the specifics of the cases you'll want to catch.  Also if you
want to implement it in C those same specifics will need to be spelled
out to figure out how a major mode will communicate them to the C code
(for this to be useful beyond CC-mode, it would need to be very general,
so it could be tricky to design).

But to tell you the truth, other than CC-mode, I'm having a hard time
imagining which other major mode will want to use such a thing.
Performance of syntax-propertize is not stellar but doesn't seem
problematic, and it is not too hard to use (its functioning is not
exactly the same as what a real lexer would do, but you can make use of
the language spec more or less straightforwardly), whereas I get the
impression that your suggestion relies on properties of the language
which are not often used, so are less familiar to the average
mode implementor (and a language spec is unlikely to help you figure out
what to do).

Maybe if we want to speed things up, we should consider a new parsing
engine (instead of parse-partial-sexp and syntax-tables) based maybe on
a DFA for the tokenizer and GLR parser on top.  That might arguably be
more generally useful and easier to use (in the sense that one can more
or less follow the language spec when implementing the major mode).


        Stefan