Re: emacs rendering comparisson between emacs23 and emacs26.3

[Alan Mackenzie <acm@muc.de>]

Hello, Dmitry.

On Thu, Apr 16, 2020 at 04:43:23 +0300, Dmitry Gutov wrote:
> Hi Alan,

> On 12.04.2020 18:34, Alan Mackenzie wrote:
> >>> But this would merely transfer the start up time to the
> >>> time taken in early scrolls forward.

> >> Not really. The start up scans the whole buffer, doesn't it? The early
> >> scrolls forward would still scan only a fraction of it.

> > I'm thinking more about "scrolling" to the function in the file that one
> > wants to work on or look at.  On average, this will be a little more
> > than half way through the file (there is often a large comment block at
> > BOB).  So you'd only be saving about half of CC Mode's start-up scan.

> Yes, on average it's only 2x benefit, but then again, if the buffer 
> opens at its beginning first, the extra initialization will be spread 
> across user commands. Which should have some additional positive effect 
> on apparent responsiveness.

Will it?  A slightly faster appearance of the buffer at line 1 will be
offset by a slightly more sluggish response when doing M-x imenu foo.  And
"slightly" here means a fraction of a second on a modern machine,
presumably somewhat longer on an older one.

> >> Inserting characters can alter the syntax state of the whole buffer. At
> >> least that's true for some of them. Full buffer scan sounds inevitable
> >> in those cases.

> > Full buffer scans are very unusual.  Inserting a " where every
> > subsequent line ended with a backslash might do that.  Inserting a C++ <
> > wouldn't - its effect is limited to up to the next brace or semicolon.

> > Inserting a C++ raw string opener does typically necessitate a full
> > scan (a search for a matching closer), but that would also be the case
> > using syntax-propertize.

> Not really. It would just mark the opener as a string opener (maybe with 
> some extra text property), and that's that.

You don't know whether it's an unterminated raw string (the usual case)
until you've scanned for a potential closing delimiter.  This affects the
font locking.  (An unterminated opening delimiter gets
font-lock-warning-face, a terminated one doesn't.)

This is the sort of feature which I'm not willing to sacrifice.

> Then font-lock would fontify the following text as string contents
> (until the end of the window or a little bit farther). Then you type
> the closer, it only has to scan a little far back (it'll call
> syntax-ppss to find the string opener), the closer is propertized as
> appropriate, and that's that. No full buffer scans at any step.

> I recall that fontifying the rest of the buffer as text after a simple 
> string opener could be a sore topic for you, but raw strings should be 
> rare enough (aren't they?), or if they are not, fontification logic 
> could opt to do something different, while syntax-table properties will 
> be applied the "correct" way.

I'm not sure what you mean by "as text".  I've no reason to think raw
strings are at all rare.  I've had several bug reports for them.  I'm not
sure what you mean by "fontification logic ... something different" - do
you mean in the raw string case?  Or by "while s-t properties will be
applied in the "correct" way" - these properties are currently correctly
applied (modulo any remaining bugs).

> > So, we would merely be moving functions from
> > c-get-state-before-change-functions and c-before-font-lock-functions
> > (effectively lists of before-/after-change functions) to
> > s-p-extend-region-f, together with adaptation. Would you agree that
> > such a change to CC Mode would be largely pointless if some of these
> > functions had to remain on c-get-state-b-c-f and c-before-f-l-f?

> Yes, I think before-change-functions should become empty. Or much emptier.

It can't become empty.  after-change-functions is fine for dealing with
insertions, but can't do much after a deletion.  Consider the case where
you're in a string and all you know is that 5 characters have been
deleted.  Those characters might have been )foo", so after checking the
beginning of the string starts off with R"bar(, you've then got to scan a
long way forward looking for )bar".  Effectively every deletion within a
string would involve scanning to the end of that string.

By contrast, in a before-change-function, a quick check will show that
)foo", which is about to be deleted is here not a raw string delimiter,
so no special action is needed.

> > But the way s-p-extend-region-f functions are called is to keep calling
> > them repeatedly until they've all said "no change" together.  This would
> > dramatically slow down CC Mode, where currently these functions are each
> > called exactly once.

> Here's a simple solution: create one function special for CC Mode that 
> would do that, and add it to s-p-extend-region-f.

> But there might be even better way to do that. I'm not the best person 
> to discuss that with.

Sorry for not thinking it through, that's not a problem at all, really.

> > Also, the syntax-propertize mechanism is weaker than CC Mode's: When it
> > is run, there is no way of knowing whether it's being called as a change
> > function, and if it is, OLD-LEN is discarded.  How can it have access to
> > variables set in before-change-functions?  (An example of such is
> > c-raw-string-end-delim-disrupted.  In before change, it is set when the
> > existing raw string end delimiter is about to cease to be.  In after
> > change, the fact of this flag being nil means we don't need to search
> > for an alternative closing delimiter, etc.  This change can obviously
> > not be detected in an after-change function.)

> As we seem to agree, before-change-functions should not be needed. 

I don't agree at all.  before-change-functions is needed (see above).

> Neither should be the code that tracks the details of the edits that the 
> user makes. That alone can simplify some things.

At the cost of complicating other things, and slowing things down.  If
you have no b-c-f, you have no information about a deletion beyond its
OLD-LEN.  So you have to do a lot more checking and scanning in an a-c-f
to recover that lost information.

> >> I'm pretty sure I have thought of that example because it's an instance
> >> of a syntax problem that's easy enough to solve within
> >> syntax-propertize-function framework.

> > Having actually gone through all the issues and implemented raw strings,
> > I can't agree with you there.  There are all sorts of subtleties which
> > necessitate intimate cooperation between the before-change-functions and
> > after-change-functions.  Such cooperation seems to be excluded by the
> > syntax-propertize mechanism.

> It encourages a different approach. Again: there are examples of raw 
> strings support in other major modes.

Do any of them have the syntactical complexities of C++'s raw strings?

> >>> Consider(2) a C++ template: excusing my C++ syntax knowledge, type in

> >>>      template class foo < bar, baz >= bar>

> >>> , perhaps typing in the odd newline inside the template (a common
> >>> occurrence), or nesting further templates inside it (also a common
> >>> occurrence).  Note how the parenthesis text properties are added and
> >>> removed as you type.  All these modification are necessary, and they are
> >>> largely _before_ the point of insertion, not after it.

> >> The current implementation of applying these properties can probably be
> >> transferred into a syntax-propertize-function with only modest changes.

> > Maybe, but with a slowdown.  More of these properties will get erased
> > than needed (with nested template forms), and they will all need to get
> > put back again.

> We won't really know until we can measure the result.

What's the point in investing all the effort to make the change, when
there's not even a prediction of a speed up?

> >>>> Some scenarios can become slower, that's for sure. But the more common
> >>>> ones can get faster. We won't know until we try.

> > Other than starting up a buffer, we still haven't identified any
> > specific scenarios where speed up might happen.

> When before-changes-functions only contains (t syntax-ppss-flush-cache), 
> that can visibly change the performance tradeoffs.

It can indeed, for the worse.  See above.

> > I don't think the syntax-propertize mechanism is all that brilliant.
> > It's too constrained, and places too many restrictions on what can be
> > done with the syntax-table text property.  For example, (from
> > syntax.el):

> I wouldn't say it's perfect either. But it's proven helpful over the 
> years, and provided a base design for a lot of major mode implementations.

Your first sentence is the pertinent one, here.

And I'm not sure where the proof of the syntax-propertize mechanism being
helpful is.  Has anybody but its originator positively chosen to use it,
whilst being aware of the alternatives?

And the mechanism has caused difficult to debug bugs, such as bug #40661.

To become usable for CC Mode, it would need to provide something on
before-change-functions to complement what's on a-c-f, and it would need
to provide some control to the major mode over which syntax-table
properties get erased.

> > (defvar syntax-propertize-function nil
> >    ;; Rather than a -functions hook, this is a -function because it's easier
> >    ;; to do a single scan than several scans: with multiple scans, one cannot
> >    ;; assume that the text before point has been propertized, so syntax-ppss
> >    ;; gives unreliable results (and stores them in its cache to boot, so we'd
> >    ;; have to flush that cache between each function, and we couldn't use
> >    ;; syntax-ppss-flush-cache since that would not only flush the cache but also
> >    ;; reset syntax-propertize--done which should not be done in this case).

> >  From my point of view, "multiple scans" are _much_ easier.  They are
> > prohibited here only because syntax-ppss and syntax-propertize-function
> > have got themselves tied up in a tight knot.  One answer would be not to
> > use syntax-ppss inside a s-p-function.  (CC Mode doesn't use syntax-ppss
> > at all).  Another answer would be to give the responsibility of removing
> > the s-t text properties to the s-p-function.

> I think we could extend the customizability in that direction. But first 
> we'd have to see clear evidence that the current design is not good 
> enough (e.g. CC Mode has been reimplemented on top of it, and the result 
> is decidedly not fast enough).

Before we even start reimplementing CC Mode, we'd want some evidence that
the current design of syntax-propertize is adequate.  I see quite clearly
that it is not.

> >> Simply collaborating with one other developer on an overhaul project
> >> (whether it succeeds or not; perhaps partially) can improve on that.

> > But take a massive amount of time.

> You could consider it an investment.

As yet, I don't honestly see how.

-- 
Alan Mackenzie (Nuremberg, Germany).