bug#20146: font-lock-extend-jit-lock-region-after-change: results are discarded instead of being returned.

[Stefan Monnier <monnier@iro.umontreal.ca>]

>> >> The major mode sets font-lock-extend-region-function and this functions'
>> >> result should be (and is) respected by the rest of font-lock.
>> > If only, but this is simply not the case at the moment.
>> > jit-lock-fontify-now has a hard-coded extension to whole lines,
>> > regardless of the contents of font-lock-extend-region-functions.
>> There's no relationship between the two.  The bounds that
>> font-lock-fontify-region [gets] are not under the major mode's control.
> Yes they are.  They can be set by
> font-lock-extend-after-change-region-function, as described in the elisp
> manual.

That's where you're confused:
font-lock-extend-after-change-region-function affects the region that's
declared as "in need of refresh because of a change".  But that doesn't
mean that this region will then be passed to font-lock-fontify-region.
For example, if that region is not currently visible in any window, then
jit-lock will typically do nothing.  And if only some part of that
region is visible in a window, then maybe only that part will be then
passed to font-lock-fontify-region so as to actually refresh it.
And maybe that part will be combined with some other parts that were
also in need of refresh and are now newly visible.  Or maybe that part
will actually be fontified in several steps (i.e. several calls to
font-lock-fontify-region).

> Yes.  I think you've said, indeed quite often, that font-lock has the
> same behaviour regardless of whether jit-lock is enabled or not.

That's not exactly what I said.  What I said is that any highlighting bug (other
than a "lack of highlighting") you see with jit-lock can also be
reproduced without jit-lock (in the worst case: with manual calls to
font-lock-fontify-region).

> I need to care a great deal about what jit-lock does, for the reasons in
> this thread.

If you need to care about it, it can only be because of incorrect
assumptions in your font-lock code.

> Looking more critically at jit-lock-fontify-now, there is simply no need
> for it to expand the region to whole lines.

I agree with this statement (I actually have such a change in my local
tree), but that won't help you.
Looking at jit-lock as the source of your pain is a waste of time.

> As a bonus, not extending (start next) would completely prevent the
> condition that triggers (via a timer) jit-lock-force-redisplay, so we
> could remove that part of the code and j-l-f-redisplay itself.  (It is
> true that buffer positions before start could have been refontified, but
> that is also true of the existing code).

> Likewise, there is then no longer any purpose in extending to whole
> lines in font-lock-extend-jit-lock-region-after-change, so we can remove
> that bit, too.

Obviously you still haven't understood the comment:
If we don't extend to whole lines in
font-lock-extend-jit-lock-region-after-change, then any change on a line
which causes a previous char's highlighting to be changed will require
a second redisplay before this change is visible.
The current code could handle it right (i.e. use
jit-lock-force-redisplay to make sure this second redisplay happens
right away), but if you additionally get rid of
jit-lock-force-redisplay, then the incorrect display would stay visible
until something else causes a redisplay.

IOW extending to whole lines in
font-lock-extend-jit-lock-region-after-change is an optimization, but is
definitely not indispensable.

> So, as an opener, I propose the following improvement:

Non-starter, especially given that you have shown repeatedly that you
don't understand the very fundamental design choice that it's perfectly
correct to call font-lock-fontify-region with any bounds whatsoever.


        Stefan