Re: Major modes using `widen' is a good, even essential, programming practice.

[Dmitry Gutov <dgutov@yandex.ru>]

On 09.08.2022 20:05, Eli Zaretskii wrote:
>> Date: Tue, 9 Aug 2022 19:52:46 +0300
>> Cc: monnier@iro.umontreal.ca, acm@muc.de, gregory@heytings.org,
>>   emacs-devel@gnu.org
>> From: Dmitry Gutov <dgutov@yandex.ru>
>>
>>> Then I don't really understand what is it that you are arguing about.
>>>
>>> My proposal to Stefan was to make syntax-ppss and friends less of a
>>> burden _instead_ of the currently implemented "arbitrary restrictions"
>>> that he doesn't like.  You seemed to have contradicted my proposal by
>>> saying that the file is already displayed quickly enough, but that
>>> only happens _with_ those "arbitrary restrictions".
>>
>> No, it doesn't.
>>
>> You might recall the patch I suggested recently that doesn't change
>> either of those vars but disables narrowing in handle_fontified_prop.
> 
> Why is that of importance?  More importantly, how is that proposal
> related to what I was discussing with Stefan?

We've been talking about "unconstrained" font-lock and the performance 
problems it causes or can cause. The patch is the way to actually try it 
without bringing back unrelated performance problems.

>> BTW, you can try js-json-mode in the latest master, I have fixed another
>> source of slow font-locking there (coming from js-mode).
> 
> I already did.  This trime I got impatient more quickly, and killed
> the session only after 5 minutes that it was unable to show me
> dictionary.json (after disabling the narrowing).

Am I correct to assume that you tried it while setting 
long-line-threshold to nil?

That kind of experiment tells me nothing, as explained before.

>> Just remove the expression that starts with 'if
>> (current_buffer->long_line_optimizations_p)' from handle_fontified_prop,
>> recompile, and visit dictionary.json.
> 
> Sorry, I cannot afford trying half-baked solutions.  I asked you to
> push a feature branch or an optional feature on master precisely so
> that I won't need to hack my development branch.  When such a feature
> is available, I'll surely test it in a variety of scenarios,

It's a tiny patch. I can push it to a branch, but it will still be a 
tiny patch that just removes 14 lines.

I'm also working on a bigger change that will push the 
narrowing/limiting mechanics down to font-lock, but I'm yet to find the 
best place to put that logic.

And the problem with working on a feature like that is that it will be 
fixing performance problems I don't really have. And, as such, cannot 
evaluate different tradeoffs. And neither you nor Gregory want to give 
me feedback by actually trying that tiny patch.

>>>> If you concerned with the speed of font-lock itself (and not with the
>>>> speed of syntax-ppss cache maintenance which we've talked about before),
>>>
>>> I'm concerned with both, because font-lock typically calls syntax-ppss
>>> in many modes.
>>
>> "Stealth" syntax-ppss, to have any visible impact, is likely to have the
>> problem I described: lots of work, the results of which are regularly
>> discarded. Meaning, lost of wasting CPU energy.
> 
> Well, my many years of using jit-lock-stealth clearly prove otherwise.
> By the time I get to revisit the buffers after some break, they are
> already fully fontified.

Then you are not in the same usage scenario that we described before 
(lots of jumping around *and* lots of editing). Because said edits 
invalidate the syntax highlighting, forcing Emacs to do it all over again.

>> What might work better instead (and would benefit specifically the
>> scenario with a lot of jumping around and editing in different parts of
>> a large file) is to try to avoid dumping the whole spss cache when the
>> use edits near BOB, and instead record the fact of such edits but later,
>> but later try to "revalidate" the cache entries by calling
>> parse-partial-sexp on the interval where the edits occurred in the
>> meantime, and keep them if the result shows that the edits should have
>> no effect on the later values. That's something tree-sitter does, AFAIU,
>> but for much complex parse tree.
>>
>> Anyway, that approach would require some work and subsequent testing,
>> and it would improve performance for a particular class of operations.
>> It's not a given that the performance issues you see in CC Mode fit that
>> profile.
> 
> Well, I hope someone will actually try to make that happen.

Definitely. But so far I'm not convinced that your impression of CC 
Mode's "sluggishness" comes from syntax-ppss at all. Or, at least, most 
of it.

And not from CC Mode's approach to maintaining syntax information 
eagerly, through buffer change hooks (after-change-functions, etc).

>> And I hope that somebody could look into improving that 10x difference
>> between yours and mine performance of parse-partial-sexp first, so then
>> we could see where the remaining bottlenecks are.
> 
> That too.

Yep.