Re: cc-mode fontification feels random

[Daniel Colascione <dancol@dancol.org>]

On 6/6/21 11:18 AM, Stefan Monnier wrote:
>> So if the first windowful of a file that's displayed is at EOB,
>> fontification must go all the way back to BOB and start scanning
>> there, until it comes to the end?
> Yup.  The way to make it bearable is to make that scan be as simple and
> fast as possible.
>
> Note that `syntax-propertize` and `syntax-ppss` also work this way, so
> it's already the case that when we start by displaying EOB we first have
> to apply `syntax-propertize` over the whole buffer :-(
>
> In theory, there are various cases (which depend on the specific
> programming language under consideration) where we could avoid such
> a scan, but it would introduce a lot of complexity so we don't bother.

I've been thinking of a new core facility for helping modes implement 
this kind of incremental buffer analysis. Basically, it works like this: 
fontification logically proceeds from bob to eob in fixed-size chunks. 
After each chunk, we checkpoint the state of the fontification engine in 
a text property. Whenever we modify the buffer, we invalidate chunks 
that the modification might have affected and proceed from the last 
known-valid checkpoint.

It's more subtle than it sounds though.

First, we need to support lookahead. Fontification of region [A, B) 
might do lookahead and depend on text in region [B, C). If it does, a 
modification occurs somewhere between B and C, we need to invalidate the 
[A, B) chunk. If we put the fontification-by-chunking code in core, we 
can track (via core magic) a high-water-mark of accessed buffer position 
for fontification of each chunk. This way, invalidation becomes 
automatically correct.

Second, writing fontification as some kind of callback with explicit 
checkpoint and restore support is annoying, and nobody's going to do 
that. If it were possible to write fontification programs as coroutines, 
we would keep mode fontification routines simply and declarative and 
automatically do both the chunking and the checkpointing.