Recipe:

emacs -Q
C-x C-f src/xdisp.c RET
M-g c 28 RET
;; take note of the word there: "window"
M-: (get-char-property 28 'fontified) RET
;; observe that this returns t
M-g g 800 RET
C-v
M-: (get-char-property 28 'fontified) RET
;; observe that this returns nil, because "struct window" is now visible

This is a BUG.

When font locking has put a fontified property and one of the 
font-lock-*-faces on characters in the buffer, a mode should not undo that 
unless it has a very good reason to do so.  Otherwise scrolling again 
through an already fontified buffer calls fontification functions again 
without reason.

Patch attached.

With the patch, scrolling again through an already fontified buffer is 
~25% faster.