Re: Last steps for pretesting (font-lock-extend-region-function)

[Stefan Monnier <monnier@iro.umontreal.ca>]

> Good morning, Stefan!

Good morning Alan,

> The font locking is triggered by the Change.  Therefore it needs details
> of that change  to determine what region of text needs refontifying.  It

No, it only needs to know what the current text looks like (obviously) and
how the text was font-locked last time (to properly remove highlighting
where it doesn't apply any more).  This last part canbe obtained via the
font-lock-multiline property.

> Perhaps I have truly misunderstood you.  I thought you were telling me
> that some code contained within the major mode would do the
> (put-text-property ... 'font-lock-multiline ...).  Are you really saying
> that the major mode merely has to set up its font-lock-keywords so that
> font-lock.el finds the right places to apply the text property?

I've generally been assuming that your code would explicitly do
a put-text-property, but indeed font-lock.el includes some code which tries
to do that for you.

>> IIUC, something like the following should do:

>> (defun c-awk-font-lock-extend-region (beg end)
>> (cons (c-awk-beginning-of-logical-line beg)
>> (c-awk-end-of-logical-line end))) <==================================
>> (setq font-lock-extend-region-function 'c-awk-font-lock-extend-region)

> That, quite demonstrably, WON'T do, since that will fail to extend the
> region to what was the end of the logical line before the change.

Of course it's not enough: it only takes care of making sure current atomic
elements are properly fontified, but not that previously atomic elements are
properly refontified.  That's what the font-lock-multiline property is for.

>>>> Here is the problems I see with your proposal:
>>>> - an `extend-region' hook in font-lock-fontify-region is needed

>>> It is indeed.  It is a simple patch, and I am ready and willing to write
>>> it, both for font-core.el and modes.texi.  I do first ask that it's
>>> accepted in principle, though.

>> It's completely accepted.  I just hope we can call it
>> "font-lock-extend-region-function".

> It has been accepted (by Richard, no less), as part of the
> font-lock-after-change-function/jit-lock-after-change.  I'm proposing to
> enhance it to do the necessary region extension from within
> f-l-default-fontify-region/j-l-fontify-now too.

I consider the two as separate: the first is already installed (but I want
it removed) and the second is not installed yet, but we agree that it should
be added.  I.e. I want it moved from a-c-f to f-l-d-f-r (it should not be
added to j-l-fontify-now).

>>>> - the font-lock-multiline property should be enough in all cases to make
>>>> it unnecessary to use an after-change-function hook.

>>> f-l-extend-region-function also makes a (separate) after-change-f hook
>>> unnecessary.

>> Which f-l-extend-region-function?  The one called from after-change-f?

> Yes.

>> Then it's what I meant by "an after-change-function hook".  The fact that
>> it's not called directly from a-c-f but via f-t-a-c-f is not very
>> important for this discussion.

> I think it's important.  It saves the major mode maintainer from having
> to install his code as a separate a-c-function, and juggling them around
> to make sure they get called in the correct order.

Yes it is important in general.  But this discussion is about the need to
have *any* a-c-f hook.  If we need one, its place is in f-l-a-c-f, indeed,
but I argue that we don't need one, so the precise place is not really
relevant to the discussion.

>> The code will have probably the same cost whether it's called from a-c-f or
>> from font-lock, but in one case it'll be called (much) more often.
> Yes.  But that calling from a-c-f is essential to correct font locking.

That's what I claim is not true.

> #########################################################################
> In AWK Mode:

> Point is at EOL 3, and we delete the backslash there.

> 1. "string \
> 2. over \
> 3. several \       <========= point is at EOL 3, about to delete the \
> 4. #lines."

OK: if the text had never been font-locked before, an a hook in
f-l-fontify-region is all we need, right?  So the interesting case is when
the text had already been fontified.
In this case, the whole multiline-line has had a font-lock-multiline added,
so at this point in your example, the multiline-line is 100% covered by
a font-lock-multiline property.

> 1. In c-awk-before-change (see above for the code), we calculate
> c-awk-old-EOLL ("end of old logical line").  This give 36, (EOL 4).

Here let's say instead we don't do anything.

> 2. The \ gets deleted, and the text looks like this:

> 1. "string \
> 2. over \
> 3. several 
> 4. #lines."

OK, so the whole text from line 1 to line 4 still has a font-lock-multiline
property.

> 3. jit-lock-after-change gets called as (jit-lock-after-change 26 26 1),
> which in its turn calls (c-awk-font-lock-extend-region 26 26 1).  This
> "1" is the OLD-LEN, of course.

Here let's also say that instead we don't do anything special.

> 4. c-awk-f-l-e-r calls (c-awk-end-of-change-region 26 26 1).  This does
> the following:
> (i) It determines the current position of the PREVIOUS end of logical
> line: (+ (- c-awk-old-EOLL old-len) (- end beg)),
>       => (+ (- 36 1) (- 26 26))
>       => 35
> NOTE THE USE OF old-len.

> (ii) It determines the current end of logical line:
>     (c-awk-end-of-logical 26)
>      => 26.  This is EOL 3

> (iii) It selects the greater of these two positions:
>     (max (+ (- c-awk-old-EOLL old-len) (- end beg))
>          (c-awk-end-of-logical-line end))
>     => (max 35 26)
>     => 35.
> This 35 is the current EOL 4.

> (iv) 35 is returned to jit-lock-after-change in the cons (1 . 35), the
> extended fontification region.
> #########################################################################

Here let's also say that instead we don't do anything special.

But when we later get to f-l-fontify-region, the code looks at the
font-lock-multiline property at the boundary (i.e. at the beginning and end
of line 3) and noticies that it's set, so it extends the region to be
fontified to span all 4 lines.  Just what you wanted all along.

Look ma!  No old-len!  Tadaaa!

If you insert text (instead of removing it), it gets a bit more interesting
(because the inserted text doesn't have a font-lock-multiline property)
but it basically works along the same lines.


        Stefan