bug#20146: font-lock-extend-jit-lock-region-after-change: results are discarded instead of being returned.

[Stefan Monnier <monnier@IRO.UMontreal.CA>]

>> > Perhaps we could implement the convention that when a major mode has
>> > positively set the font-lock region's start and end points, these should
>> > be accepted by F/J-lock, but when not, F/J-lock should be free to alter
>> > them (as it typically does now).
>> No the core of the API is font-lock-fontify-region and it should work
>> with *any* bounds (i.e. if these need to be extended, it should be done
>> by font-lock-extend-region-function).
> However, when the bounds are set by the major mode, those bounds should
> be respected by Font Lock.

The major mode sets font-lock-extend-region-function and this functions'
result should be (and is) respected by the rest of font-lock.

But callers of font-lock-fontify-region (such as
font-lock-after-change-function, or jit-lock) can choose *any* bounds
they feel like and font-lock-fontify-region should behave correctly.

> As I pointed out to Daniel, the major mode is the expert in where
> font-locking must start

Indeed, and it instructs font-lock on how to extend the region by
setting font-lock-extend-region-function.

>> Jit-lock is implemented on top of that API and is hence free to use any
>> bounds it sees fit.
> But surely not to pass bounds to the major mode that the major mode must,
> for a second time, adjust.  This is crazy.

No, it is just good design to keep complexity under check.

>> If you rely on more specific bounds being passed to
>> font-lock-fontify-region, that means you have a problem on your side.
> There is a problem, yes, but on which side it is is a matter for
> discussion.  CC Mode has always relied on the specific bounds that it
> provides.

AFAIK CC-mode does not provide any bounds.  Instead it uses
font-lock-extend-after-change-region-function which changes the part of
the buffer that is invalidated, which is something different.

The relationship between the two is brittle and subtle, so relying on it
is crazy and a good way to get impenetrable code with special cases
tacked on top of special cases with lots of unspecified assumptions that
only hold in common cases.

> You yourself pointed out that CC Mode is the sole user of the
> functionality for expanding FL regions.

Indeed.  Others use font-lock-extend-region-function.

font-lock-extend-after-change-region-function can be useful when
a *change* on line N can cause a highlighting change on a line <N, or
when it causes a highlighting change on a line >N and you don't like the
jit-lock-context-time delay (or you want it to work correctly even if
jit-lock(-context) is not in use).


        Stefan