Re: bug-reference-prog-mode slows down CC Mode's scrolling by ~7%

[Alan Mackenzie <acm@muc.de>]

Hello, Eli.

On Mon, Sep 06, 2021 at 14:10:54 +0300, Eli Zaretskii wrote:
> > Date: Mon, 6 Sep 2021 10:46:48 +0000
> > Cc: monnier@iro.umontreal.ca, emacs-devel@gnu.org
> > From: Alan Mackenzie <acm@muc.de>

> > I think the optimal size for jit-lock-chunk-size is a little over how
> > much text fits in a window.  That way, an entire window can be fontified
> > in a single chunk, minimising overhead.  However, much more than that,
> > and the fontification is less JIT, more like fontifying large chunks of
> > a buffer just in case.

> > So, I propose a default value of 2000.  Approximately 1700 characters of
> > xdisp.c fit into a default sized GUI window.

> Not here.  I only get 1300.

Sorry.  Slip between brain and keyboard.

> In any case, can we please perform the single-windowful scroll
> benchmark before making the decision?  Doing this stuff from intuition
> is known to be a bad idea.

From another post, you wrote:
>>> To see the "waste" in single-windowful scrolls, we need a suitable
>>> benchmark, which measures the time of each scroll separately.  It
>>> would probably make sense to make it generate a random value of
>>> point, then measure the time it takes to go there and display.

So I hacked together a quick benchmark chunk-size-benchmark, which takes
a single argument, the jit-lock-chunk-size, and then in the current
window and buffer moves successively to 10%, 20%, 30%, ...., 90% through
the buffer and waits for fontification.  The average of those ten values
is output to the *Messages* buffer.

Then I wrote a variation, which jumps to a 10x% position in the buffer,
waits for fontification, then scrolls forward a screen and waits again
for fontification.  The average value is, again, printed out.

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
(defmacro time-it (&rest forms)
  "Time the running of a sequence of forms using `float-time'.
Call like this: \"M-: (time-it (foo ...) (bar ...) ...)\"."
  `(let ((start (float-time)))
     ,@forms
     (- (float-time) start)))

(defun chunk-size-benchmark (chunk-size)
  (interactive "p")
  (if (< chunk-size 100) (error "chunk-size, %s, is less than 100" chunk-size))
  (widen)
  (let ((jit-lock-chunk-size chunk-size)
	(buf-size (point-max))
	this-time (total 0)
	)
    (dolist (i '(0 1 2 3 4 5 6 7 8 9))
      (font-lock-flush)
      (setq this-time
	    (time-it (goto-char (1+ (/ (* i buf-size) 10)))
		     (sit-for 0)))
      (setq total (+ total this-time)))
    (message "Average time for chunk-size %s is %s" chunk-size (/ total 10))))
	
	
(defun chunk-size-benchmark-2 (chunk-size)
  (interactive "p")
  (if (< chunk-size 100) (error "chunk-size, %s, is less than 100" chunk-size))
  (widen)
  (let ((jit-lock-chunk-size chunk-size)
	(buf-size (point-max))
	this-time (total 0)
	)
    (dolist (i '(0 1 2 3 4 5 6 7 8 9))
      (font-lock-flush)
      (setq this-time
	    (time-it (goto-char (1+ (/ (* i buf-size) 10)))
		     (sit-for 0)
		     (scroll-up)
		     (sit-for 0)))
      (setq total (+ total this-time)))
    (message "Average time for two chunk-size %s is %s" chunk-size (/ total 10))))
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

I got these timings, using xdisp.c each time:

(i) On a 65 line window on a Linux console tty:
Average time for chunk-size 500 is 0.09474895000457764
Average time for chunk-size 1000 is 0.08993968963623047
Average time for chunk-size 1500 is 0.07838799953460693
Average time for chunk-size 2000 is 0.0901902437210083
Average time for chunk-size 2500 is 0.09380950927734374
Average time for chunk-size 3000 is 0.09139835834503174
Average time for chunk-size 3500 is 0.09265313148498536
Average time for chunk-size 4000 is 0.09945495128631592

Average time for two chunk-size 500 is 0.14148397445678712
Average time for two chunk-size 1000 is 0.11814579963684083
Average time for two chunk-size 1500 is 0.11486186981201171
Average time for two chunk-size 2000 is 0.12313232421875
Average time for two chunk-size 2500 is 0.12045748233795166
Average time for two chunk-size 3000 is 0.1140979528427124
Average time for two chunk-size 3500 is 0.10788719654083252
Average time for two chunk-size 4000 is 0.11099658012390137

(ii) On a 34 line window (default size) in X Windows:
Average time for chunk-size 500 is 0.06480436325073242
Average time for chunk-size 1000 is 0.06025857925415039
Average time for chunk-size 1500 is 0.06389858722686767
Average time for chunk-size 2000 is 0.06813716888427734
Average time for chunk-size 2500 is 0.07740669250488282
Average time for chunk-size 3000 is 0.0859900712966919
Average time for chunk-size 3500 is 0.09434397220611572
Average time for chunk-size 4000 is 0.10622837543487548

Average time for two chunk-size 500 is 0.09408855438232422
Average time for two chunk-size 1000 is 0.08264987468719483
Average time for two chunk-size 1500 is 0.08883922100067139
Average time for two chunk-size 2000 is 0.07863264083862305
Average time for two chunk-size 2500 is 0.08350887298583984
Average time for two chunk-size 3000 is 0.08896465301513672
Average time for two chunk-size 3500 is 0.09870367050170899
Average time for two chunk-size 4000 is 0.10917885303497314

(iii) On a 54 line window in X Windows:
Average time for chunk-size 500 is 0.11301305294036865
Average time for chunk-size 1000 is 0.10326454639434815
Average time for chunk-size 1500 is 0.10011684894561768
Average time for chunk-size 2000 is 0.10956480503082275
Average time for chunk-size 2500 is 0.11465866565704345
Average time for chunk-size 3000 is 0.12143938541412354
Average time for chunk-size 3500 is 0.13569414615631104
Average time for chunk-size 4000 is 0.15041179656982423

Average time for two chunk-size 500 is 0.16088194847106935
Average time for two chunk-size 1000 is 0.14736626148223878
Average time for two chunk-size 1500 is 0.13186230659484863
Average time for two chunk-size 2000 is 0.14865143299102784
Average time for two chunk-size 2500 is 0.13905446529388427
Average time for two chunk-size 3000 is 0.13570570945739746
Average time for two chunk-size 3500 is 0.14243290424346924
Average time for two chunk-size 4000 is 0.15591373443603515

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

So, I would reason as follows:
(i) 500 bytes is too small.
(ii) We should be more concerned about the timings on larger windows, if
we are concerned about redisplay times.  0.1s (for a 54 line window) is
appreciably greater than 0.06s (a 34 line window), which is on the point
of seeming instantaneous.
(iii) The two chunk-size time on 54 lines suggests an optimum of 1500,
which is supported by the corresponding times on the tty.
(iv) 1500 is not far from optimal on 34 lines single chunk, though not
so for the two chunk time.

(It is notable that redisplay is quite a bit slower in X at 54 lines
than on a Linux console at 65 lines.)

So, based on these timings, I would suggest a new jit-lock-chunk-size of
1500.

-- 
Alan Mackenzie (Nuremberg, Germany).