What's the best (i.e. least bad) way to re-redisplay?

What's the best (i.e. least bad) way to re-redisplay?

[Alan Mackenzie]

Hello, Emacs.

I'm currently thinking about issues raised in the thread "cc-mode
fontification feels random" in June.

I envisage a situation where an identifier is identified as a type during
jit-lock fontification, and thus font-lock-type-face needs to be applied
to all occurrences of this identifier.

The problem is, there will be occurences in the foreground window before
the one which triggered the fontification.  Redisplay will already have
passed this earlier buffer position.  So I need to trigger another
redisplay immediately after the current one.

What is the least bad way of doing this?

Currently, I'm thinking of something like the CC Mode fontification
functions setting a flag, and a repeating timer function testing this
flag every 0.025s, say, and triggering a redisplay if it's set.

One problem is knowing whether or not redisplay is currently active.  Is
there a simple way to do this in Lisp?  (I know there is in C.)

I surely can't be the first person needing to do this sort of thing.
Suggestions and comments, please, from the wise.  :-)

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: What's the best (i.e. least bad) way to re-redisplay?

[Eli Zaretskii]

> Date: Mon, 23 Aug 2021 18:41:59 +0000
> From: Alan Mackenzie <acm@muc.de>
> 
> I envisage a situation where an identifier is identified as a type during
> jit-lock fontification, and thus font-lock-type-face needs to be applied
> to all occurrences of this identifier.
> 
> The problem is, there will be occurences in the foreground window before
> the one which triggered the fontification.  Redisplay will already have
> passed this earlier buffer position.  So I need to trigger another
> redisplay immediately after the current one.
> 
> What is the least bad way of doing this?

You mean, the same identifier is somewhere in the same window-ful, but
before the position where you suddenly decided your previous
fontification was incorrect?  (Why is this a reasonable scenario,
btw?)

> Currently, I'm thinking of something like the CC Mode fontification
> functions setting a flag, and a repeating timer function testing this
> flag every 0.025s, say, and triggering a redisplay if it's set.

Why not simply call a one-time timer function that runs
jit-lock-force-redisplay, when you decide that you need to refontify?
jit-lock.el already does that in some cases.  I see no reason to have
a periodic timer and a flag, because the same code that sets the flag
could instead arm a one-time timer.

> One problem is knowing whether or not redisplay is currently active.  Is
> there a simple way to do this in Lisp?  (I know there is in C.)

I don't understand your problem here.  Timers cannot run while
redisplay works, because timers are only run by the main loop.  So
when a timer runs, redisplay by definition isn't.

Re: What's the best (i.e. least bad) way to re-redisplay?

[Eli Zaretskii]

> Date: Mon, 23 Aug 2021 22:35:22 +0300
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: emacs-devel@gnu.org
> 
> Why not simply call a one-time timer function that runs
> jit-lock-force-redisplay, when you decide that you need to refontify?
> jit-lock.el already does that in some cases.

Btw, you could try making it so your fontification function arrange
for jit-lock to call jit-lock-force-redisplay via its return value and
the jit-lock-bounds thingy.

Re: What's the best (i.e. least bad) way to re-redisplay?

[Alan Mackenzie]

Hello, Eli.

On Mon, Aug 23, 2021 at 22:35:22 +0300, Eli Zaretskii wrote:
> > Date: Mon, 23 Aug 2021 18:41:59 +0000
> > From: Alan Mackenzie <acm@muc.de>

> > I envisage a situation where an identifier is identified as a type during
> > jit-lock fontification, and thus font-lock-type-face needs to be applied
> > to all occurrences of this identifier.

> > The problem is, there will be occurences in the foreground window before
> > the one which triggered the fontification.  Redisplay will already have
> > passed this earlier buffer position.  So I need to trigger another
> > redisplay immediately after the current one.

> > What is the least bad way of doing this?

> You mean, the same identifier is somewhere in the same window-ful, but
> before the position where you suddenly decided your previous
> fontification was incorrect?  (Why is this a reasonable scenario,
> btw?)

For example, in a C file we have a function prototype without parameter
names:

     fooMain (foo);

, and later on this gets fleshed out:

     fooMain (foo bar)
     {
     ....
     }

The first occurrence will get fontified as a function call, I think.
Only when we pass the second occurrence do we see foo must be a type.

> > Currently, I'm thinking of something like the CC Mode fontification
> > functions setting a flag, and a repeating timer function testing this
> > flag every 0.025s, say, and triggering a redisplay if it's set.

> Why not simply call a one-time timer function that runs
> jit-lock-force-redisplay, when you decide that you need to refontify?
> jit-lock.el already does that in some cases.  I see no reason to have
> a periodic timer and a flag, because the same code that sets the flag
> could instead arm a one-time timer.

Thanks, that's exactly the answer I was asking for.  :-)  I didn't know
about jit-lock-force-display.

> > One problem is knowing whether or not redisplay is currently active.  Is
> > there a simple way to do this in Lisp?  (I know there is in C.)

> I don't understand your problem here.  Timers cannot run while
> redisplay works, because timers are only run by the main loop.  So
> when a timer runs, redisplay by definition isn't.

That was going off on a bit of a tangent.  If the above situation occurs
when we know we're not already in redisplay, we could simply call
redisplay without using a timer.  I suppose it's not really that important.

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: What's the best (i.e. least bad) way to re-redisplay?

[Stefan Monnier]

> The problem is, there will be occurences in the foreground window before
> the one which triggered the fontification.  Redisplay will already have
> passed this earlier buffer position.  So I need to trigger another
> redisplay immediately after the current one.
>
> What is the least bad way of doing this?

I think something along the lines of

    (run-with-timer 0 nil
                    (lambda (buf)
                      (with-current-buffer buf (font-lock-flush)))
                    (current-buffer))

is the cleanest that comes to mind.


        Stefan
                         

Re: What's the best (i.e. least bad) way to re-redisplay?

[Alan Mackenzie]

Hello, Stefan.

On Tue, Aug 24, 2021 at 18:43:04 -0400, Stefan Monnier wrote:
> > The problem is, there will be occurences in the foreground window before
> > the one which triggered the fontification.  Redisplay will already have
> > passed this earlier buffer position.  So I need to trigger another
> > redisplay immediately after the current one.

> > What is the least bad way of doing this?

> I think something along the lines of

>     (run-with-timer 0 nil
>                     (lambda (buf)
>                       (with-current-buffer buf (font-lock-flush)))
>                     (current-buffer))

> is the cleanest that comes to mind.

Thanks.  That's probably cleaner than a direct use of
jit-lock-force-redisplay.  I'll just need to add BEG and END into that,
somehow, but font-lock-flush has these as &optional parameters, so that
won't be a problem.

But, looking at the code, font-lock-flush works by clearing the
'fontified property, nothing more.  This won't always trigger a
redisplay, will it?  Doesn't it need after-change-functions to be active
at the time?  Maybe after-change-functions will always be active during
a redisplay (and hence immediately after redisplay completes), but my
head's beginning to hurt.

>         Stefan
                         
-- 
Alan Mackenzie (Nuremberg, Germany).

Re: What's the best (i.e. least bad) way to re-redisplay?

[Stefan Monnier]

> But, looking at the code, font-lock-flush works by clearing the
> 'fontified property, nothing more.  This won't always trigger a
> redisplay, will it?  Doesn't it need after-change-functions to be active
> at the time?

Yes, that's why it's run from a timer, to avoid the let-binding of
`inhibit-modification-hooks` that's active during font-lock.


        Stefan

Re: What's the best (i.e. least bad) way to re-redisplay?

[Eli Zaretskii]

> Date: Wed, 25 Aug 2021 20:09:24 +0000
> From: Alan Mackenzie <acm@muc.de>
> Cc: emacs-devel@gnu.org
> 
> > I think something along the lines of
> 
> >     (run-with-timer 0 nil
> >                     (lambda (buf)
> >                       (with-current-buffer buf (font-lock-flush)))
> >                     (current-buffer))
> 
> > is the cleanest that comes to mind.
> 
> Thanks.  That's probably cleaner than a direct use of
> jit-lock-force-redisplay.

Why is it cleaner?  jit-lock and font-lock include provision for the
fontification function to tell jit-lock what was the region actually
fontified, and that will trigger a call to jit-lock-force-redisplay in
a timer.  Why not use an existing mechanism?

Re: What's the best (i.e. least bad) way to re-redisplay?

[Stefan Monnier]

>> > I think something along the lines of
>> 
>> >     (run-with-timer 0 nil
>> >                     (lambda (buf)
>> >                       (with-current-buffer buf (font-lock-flush)))
>> >                     (current-buffer))
>> 
>> > is the cleanest that comes to mind.
>> 
>> Thanks.  That's probably cleaner than a direct use of
>> jit-lock-force-redisplay.
>
> Why is it cleaner?  jit-lock and font-lock include provision for the
> fontification function to tell jit-lock what was the region actually
> fontified, and that will trigger a call to jit-lock-force-redisplay in
> a timer.  Why not use an existing mechanism?

That's indeed another option.  I assumed that the situation is such that
while fontifying region POS1..POS2 we discover a new type that forces
the whole buffer to be refontified (or at least some region POS3..POS4),
in which case it's best to use `font-lock-flush` and let this
refontification happen lazily.

But if indeed we can cheaply adjust the few places in the buffer that
are affected without performing a full "refontify" it might indeed be
a good idea to do that and adjust the `jit-lock-bounds` return value,
but in that case we have to be sure that the new bounds indeed describe
an area of the buffer that is now fully fontified, which is likely to be
a problem because there is no reason to assume that the buffer outside
of POS1..POS2 had already been fontified, so it may force us to eagerly
fontify additional parts of POS3..POS4.

Another option if we can cheaply adjust the few places in the buffer that
are affected without performing a full "refontify", is to do those
adjustments and then only force a redisplay but not a refontification.
I think this can be done with something like:

    (run-with-timer 0 nil
                    (lambda (buf)
                      (with-current-buffer buf
                        (put-text-property (point-min) (point-max) 'cc-dummy t)
                        (remove-text-properties (point-min) (point-max) '(cc-dummy))))
                    (current-buffer))


-- Stefan

Re: What's the best (i.e. least bad) way to re-redisplay?

[Alan Mackenzie]

Hello, Stefan.

On Thu, Aug 26, 2021 at 09:49:33 -0400, Stefan Monnier wrote:
> >> > I think something along the lines of

> >> >     (run-with-timer 0 nil
> >> >                     (lambda (buf)
> >> >                       (with-current-buffer buf (font-lock-flush)))
> >> >                     (current-buffer))

> >> > is the cleanest that comes to mind.

> >> Thanks.  That's probably cleaner than a direct use of
> >> jit-lock-force-redisplay.

> > Why is it cleaner?  jit-lock and font-lock include provision for the
> > fontification function to tell jit-lock what was the region actually
> > fontified, and that will trigger a call to jit-lock-force-redisplay in
> > a timer.  Why not use an existing mechanism?

> That's indeed another option.  I assumed that the situation is such that
> while fontifying region POS1..POS2 we discover a new type that forces
> the whole buffer to be refontified (or at least some region POS3..POS4),
> in which case it's best to use `font-lock-flush` and let this
> refontification happen lazily.

My idea is that, say, a new type "foo" is found by stealth fontification.
CC Mode will then do a regexp search for "\\_<foo\\_>" from BOB to EOB
removing 'fontified text properties from each match it finds.  This is
fast (around 10ms per found type in xdisp.c).  It should be enough to
force (re)fontification of (the statements containing) "foo" whenever
these occurrences are scrolled onto.  Then there's the currently
displayed window, as we've been discussing.

That, of course, involves somehow persuading the user to enable stealth
fontification.

> But if indeed we can cheaply adjust the few places in the buffer that
> are affected without performing a full "refontify" it might indeed be
> a good idea to do that and adjust the `jit-lock-bounds` return value,
> but in that case we have to be sure that the new bounds indeed describe
> an area of the buffer that is now fully fontified, which is likely to be
> a problem because there is no reason to assume that the buffer outside
> of POS1..POS2 had already been fontified, so it may force us to eagerly
> fontify additional parts of POS3..POS4.

> Another option if we can cheaply adjust the few places in the buffer that
> are affected without performing a full "refontify", is to do those
> adjustments and then only force a redisplay but not a refontification.

This is exactly my plan.

> I think this can be done with something like:

>     (run-with-timer 0 nil
>                     (lambda (buf)
>                       (with-current-buffer buf
>                         (put-text-property (point-min) (point-max) 'cc-dummy t)
>                         (remove-text-properties (point-min) (point-max) '(cc-dummy))))
>                     (current-buffer))

I don't understand what that bit of code's doing.

> -- Stefan

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: What's the best (i.e. least bad) way to re-redisplay?

[Stefan Monnier]

>>     (run-with-timer 0 nil
>>                     (lambda (buf)
>>                       (with-current-buffer buf
>>                         (put-text-property (point-min) (point-max) 'cc-dummy t)
>>                         (remove-text-properties (point-min) (point-max) '(cc-dummy))))
>>                     (current-buffer))
>
> I don't understand what that bit of code's doing.

The intention is to perform a dummy change that forces the redisplay
to rerender that chunk of text.  Of course, I see I forgot
`with-silent-modifications` and I might have missed other things.


        Stefan

Re: What's the best (i.e. least bad) way to re-redisplay?

[Alan Mackenzie]

Hello, Eli and Stefan.

On Thu, Aug 26, 2021 at 09:37:14 +0300, Eli Zaretskii wrote:
> > Date: Wed, 25 Aug 2021 20:09:24 +0000
> > From: Alan Mackenzie <acm@muc.de>
> > Cc: emacs-devel@gnu.org

> > > I think something along the lines of

> > >     (run-with-timer 0 nil
> > >                     (lambda (buf)
> > >                       (with-current-buffer buf (font-lock-flush)))
> > >                     (current-buffer))

> > > is the cleanest that comes to mind.

> > Thanks.  That's probably cleaner than a direct use of
> > jit-lock-force-redisplay.

> Why is it cleaner?  jit-lock and font-lock include provision for the
> fontification function to tell jit-lock what was the region actually
> fontified, and that will trigger a call to jit-lock-force-redisplay in
> a timer.  Why not use an existing mechanism?

In the end, I coded up a direct use of jit-lock-force-redisplay in a
one-time timer.  The reason I preferred this over font-lock-flush is that
j-l-f-r says in its doc string that it forces a redisplay, whereas f-l-f
merely says it declares (BEG END) as being out of date.

I hope to post my proposed new code on emacs-devel soon.

-- 
Alan Mackenzie (Nuremberg, Germany).