bug#20146: font-lock-extend-jit-lock-region-after-change: results are discarded instead of being returned.

[Stefan Monnier <monnier@IRO.UMontreal.CA>]

>> The major mode sets font-lock-extend-region-function and this functions'
>> result should be (and is) respected by the rest of font-lock.
> It is not.  font-lock-extend-region-functions (note the "s") is plural,
> and all functions on it are run repeatedly until none makes a change.  So
> when the major mode sets the region, this is instantly violated by the
> other functions in f-l-extend-r-f.  This is what caused bug #19669, and
> I'm still struggling to find a way round it.

The whole set of functions is under the control of the major-mode.
So if you don't like the other two functions, you can remove them just
fine (as you've done).

But as you've now seen in this bug#20146, removing
font-lock-extend-region-wholelines is probably not a good idea because
your own font-lock rules rely on it.

And font-lock-extend-region-multiline has no effect if you don't use
set the `font-lock-multiline' property, so removing it would only affect
performance, not behavior.

> Is there any use case where it is helpful for one of these functions to
> make a second (or subsequent) change to the font-lock region?

Of course: most font-lock-keywords will misbehave if the region is not
made up of whole lines.  So if font-lock-extend-region-multiline extends
the region to something that's not made of whole lines we have
a problem.
Similarly, if font-lock-extend-region-wholelines extends the region to
start or end in the middle of a font-lock-multiline property we have
a problem.
So they need to be cycled.

>> But callers of font-lock-fontify-region (such as
>> font-lock-after-change-function, or jit-lock) can choose *any* bounds
>> they feel like and font-lock-fontify-region should behave correctly.
> If the major mode is going to get *any* bounds rather than the ones it
> has already specified by its function on f-l-extend-region-functions,

f-l-extend-region-functions is run *after* font-lock-fontify-region is
called, so I don't understand what you mean by "already".
And those bounds aren't changed afterwards.

>> No, it is just good design to keep complexity under check.
> ???

For example, it means, that if the highlighting is incorrect, it *can't*
be because of a bug in jit-lock.  A highlighting problem can only come
from jit-lock in case the highlighting has simply not been (re)applied.

>> AFAIK CC-mode does not provide any bounds.  Instead it uses
>> font-lock-extend-after-change-region-function which changes the part of
>> the buffer that is invalidated, which is something different.
> No it's not different.  The bounds CC Mode provides are those around the
> region which is to be invalidated, and later refontified.  I think you're
> picking nits here.

I'm definitely not picking nits.  Those two concepts are similar yet
different and independent.

> What is the alternative?  CC Mode knows exactly what portion of the
> buffer needs refontifying, Font Lock doesn't, and can't.  Any chance of a
> robust way of communicating those region bounds to Font Lock?

Yes: font-lock-extend-region-functions.


        Stefan