bug#74281: 30.0.91; font-lock mode hangs on scrolling large Scheme file

[Stefan Monnier via "Bug reports for GNU Emacs, the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>]

> Ping!  Divya, could you please try Stefan's suggestions and report
> back?

FWIW, I think the change is an improvement in any case, so we could
install it.  Since it's in a "core" file, it might be too risky for
`emacs-30`, so I'm thinking of installing it in `master`, WDYT?


        Stefan


>> Cc: Divya Ranjan <divya@subvertising.org>, 74281@debbugs.gnu.org
>> From: Stefan Monnier <monnier@iro.umontreal.ca>
>> Date: Thu, 14 Nov 2024 11:56:37 -0500
>> 
>> > Stefan, what tools do we have to investigate slowness related to
>> > parse-partial-sexp?  Or maybe you have suggestions for how to speed up
>> > font-lock in this case?
>> 
>> Hmm... `parse-partial-sexp` is normally expected to be fast, unless it
>> has to scan a lot of text.
>> 
>> > Here's the profile I get while moving with C-p through the above file:
>> 
>> A stab in the dark, but maybe the relevant call is the one in:
>> 
>>           (state (if (or syntax-ppss-table
>>                          (not font-lock--syntax-table-affects-ppss))
>>                      (syntax-ppss start)
>>                    ;; If `syntax-ppss' doesn't have its own syntax-table and
>>                    ;; we have installed our own syntax-table which
>>                    ;; differs from the standard one in ways which affects PPSS,
>>                    ;; then we can't use `syntax-ppss' since that would pollute
>>                    ;; and be polluted by its cache.
>>                    (parse-partial-sexp (point-min) start)))
>> 
>> so the origin of the slowdown would be the (?#. "w 14") in the setting
>> below in `scheme.el`:
>> 
>>   (setq font-lock-defaults
>>         '((scheme-font-lock-keywords
>>            scheme-font-lock-keywords-1 scheme-font-lock-keywords-2)
>>           nil t (("+-*/.<>=!?$%_&~^:" . "w") (?#. "w 14"))
>>           beginning-of-defun
>>           (font-lock-mark-block-function . mark-defun)))
>> 
>> in which case, setting a `syntax-ppss-table` should fix the problem, tho
>> we could also fix it by being more careful: AFAICT the purpose of this
>> (?#. "w 14") is only to change the syntax of `#` from "prefix" to "word"
>> without changing the comment-related flags, so it shouldn't cause
>> `font-lock--syntax-table-affects-ppss` to be set.
>> So, we could solve it by improving the code that sets
>> `font-lock--syntax-table-affects-ppss`, as in the patch below.
>> 
>> 
>>         Stefan
>> 
>> 
>> diff --git a/lisp/font-lock.el b/lisp/font-lock.el
>> index 203131bfd5a..f6299920c0a 100644
>> --- a/lisp/font-lock.el
>> +++ b/lisp/font-lock.el
>> @@ -1955,14 +1955,15 @@ font-lock-set-defaults
>>  	    (dolist (char (if (numberp (car selem))
>>  			      (list (car selem))
>>  			    (mapcar #'identity (car selem))))
>> -	      (unless (memq (car (aref font-lock-syntax-table char))
>> -	                    '(1 2 3))    ;"." "w" "_"
>> -	        (setq font-lock--syntax-table-affects-ppss t))
>> -	      (modify-syntax-entry char syntax font-lock-syntax-table)
>> -	      (unless (memq (car (aref font-lock-syntax-table char))
>> -	                    '(1 2 3))    ;"." "w" "_"
>> -	        (setq font-lock--syntax-table-affects-ppss t))
>> -	      ))))
>> +	      (let ((old-syntax (aref font-lock-syntax-table char)))
>> +	        (modify-syntax-entry char syntax font-lock-syntax-table)
>> +	        (let ((new-syntax (aref font-lock-syntax-table char)))
>> +	          (unless (and (equal (cdr old-syntax) (cdr new-syntax))
>> +	                       (memq (logand (car old-syntax) 255) '(1 2 3 6))
>> +	                       (memq (logand (car new-syntax) 255) '(1 2 3 6))
>> +	                       (equal (ash (car old-syntax) -8)
>> +	                              (ash (car new-syntax) -8)))
>> +	            (setq font-lock--syntax-table-affects-ppss t))))))))
>>        ;; (nth 4 defaults) used to hold `font-lock-beginning-of-syntax-function',
>>        ;; but that was removed in 25.1, so if it's a cons cell, we assume that
>>        ;; it's part of the variable alist.
>> 
>> 
>> 
>> 
>>