33% speed up in CC Mode scrolling.

33% speed up in CC Mode scrolling.

[Alan Mackenzie]

Hello Martin, Eli, John, and Emacs.

I've just committed b76385c05076f0adaf7cf89d0ed95dfe5e8570e0 to the
master branch.  This takes some code out of CC Mode's font-locking's
hottest loop and puts it back in again in a place where is gets called
less often.

This speeds up CC Mode's scrolling (as measured on our favourite file,
xdisp.c) by about one third.  There may be scope for further optimisation
of this code.

This is a substantial and worthwhile, although not spectacular, speed up
of CC Mode.

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: 33% speed up in CC Mode scrolling.

[John Wiegley]

>>>>> Alan Mackenzie <acm@muc.de> writes:

> This is a substantial and worthwhile, although not spectacular, speed up of
> CC Mode.

That's great!  I remember heavily relying on CC mode a few years ago, and that
scrolling was quite slow in large Boost-related files.

-- 
John Wiegley                  GPG fingerprint = 4710 CF98 AF9B 327B B80F
http://newartisans.com                          60E1 46C4 BD1A 7AC1 4BA2

Re: 33% speed up in CC Mode scrolling.

[Clément Pit--Claudel]

[-- Attachment #1.1: Type: text/plain, Size: 177 bytes --]

On 2016-06-15 18:20, Alan Mackenzie wrote:
> This is a substantial and worthwhile, although not spectacular, speed up
> of CC Mode.

C-x 8 RET CLAPPING HANDS SIGN RET :)


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: 33% speed up in CC Mode scrolling.

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

I have found CC mode font lock such a nuisance that I sometimes
turn it off.

It is a good thing to have sped it up, but it might still be far too
slow.  Perhaps we should go back to an older version that is fast enough
even if it is more often wrong.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: 33% speed up in CC Mode scrolling.

[Alan Mackenzie]

Hello, Richard.

On Thu, Jun 16, 2016 at 07:15:44PM -0400, Richard Stallman wrote:
> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]

> I have found CC mode font lock such a nuisance that I sometimes
> turn it off.

> It is a good thing to have sped it up, but it might still be far too
> slow.  Perhaps we should go back to an older version that is fast enough
> even if it is more often wrong.

Or, why not just set `font-lock-maximum-decoration' to 2?  This runs at
~2½ times the speed of the default fontification (measured by scrolling
all the way through .../src/xdisp.c), and is barely slower than
`font-lock-maximum-decoration' being set to 1 (which merely fontifies
comments and strings).

Fair warning: setting `f-l-m-d', and making the setting work, is
difficult (see bug #23783).  One way is to do

    M-: (font-lock-refresh-defaults)

in a buffer after changing `f-l-m-d'.

> -- 
> Dr Richard Stallman
> President, Free Software Foundation (gnu.org, fsf.org)
> Internet Hall-of-Famer (internethalloffame.org)
> Skype: No way! See stallman.org/skype.html.

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: 33% speed up in CC Mode scrolling.

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Or, why not just set `font-lock-maximum-decoration' to 2?

Indeed, that makes it much better for me.

But is something painfully slow for many users really the right choice
of default?  I don't think so.

I think the default for `font-lock-maximum-decoration' should
be such that the really slow kinds of fontification are above it
and thus disabled by default.


-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: 33% speed up in CC Mode scrolling.

[Stefan Monnier]

> But is something painfully slow for many users really the right choice
> of default?  I don't think so.

While I agree, I generally use old/slow machines and the "painfully
slow" behavior is rare and should simply be considered as a bug to
report and fix, rather than an unavoidable issue that should be
circumvented by lowering our expectations.

IOW I'm a firm believer that we can have our cake and eat it too,


        Stefan

Re: 33% speed up in CC Mode scrolling.

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > While I agree, I generally use old/slow machines and the "painfully
  > slow" behavior is rare and should simply be considered as a bug to
  > report and fix, rather than an unavoidable issue that should be
  > circumvented by lowering our expectations.

I got painfully slow behavior for search.c.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: 33% speed up in CC Mode scrolling.

[Stefan Monnier]

> I got painfully slow behavior for search.c.

It would be great if you could make a repeatable recipe for it.
Alan is good at fixing those things, but without a repeatable test-case
it's tremendously difficult to make any progress.


        Stefan

Re: 33% speed up in CC Mode scrolling.

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

Typing C-M-e at the start of regex_compile (in regex.c) took two
seconds.  That is the worst case I can find right now, but I recall
seeing worse.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: 33% speed up in CC Mode scrolling.

[martin rudalics]

 > I've just committed b76385c05076f0adaf7cf89d0ed95dfe5e8570e0 to the
 > master branch.  This takes some code out of CC Mode's font-locking's
 > hottest loop and puts it back in again in a place where is gets called
 > less often.
 >
 > This speeds up CC Mode's scrolling (as measured on our favourite file,
 > xdisp.c) by about one third.  There may be scope for further optimisation
 > of this code.
 >
 > This is a substantial and worthwhile, although not spectacular, speed up
 > of CC Mode.

This change has hardly any effect here.  Scrolling xdisp.c with a '-O0
-g2 -gdwarf-2' build run via emacs -Q is still virtually impossible.

martin

Re: 33% speed up in CC Mode scrolling.

[Alan Mackenzie]

Hello, Martin.

On Mon, Jun 27, 2016 at 08:31:42AM +0200, martin rudalics wrote:
>  > I've just committed b76385c05076f0adaf7cf89d0ed95dfe5e8570e0 to the
>  > master branch.  This takes some code out of CC Mode's font-locking's
>  > hottest loop and puts it back in again in a place where is gets called
>  > less often.

>  > This speeds up CC Mode's scrolling (as measured on our favourite file,
>  > xdisp.c) by about one third.  There may be scope for further optimisation
>  > of this code.

>  > This is a substantial and worthwhile, although not spectacular, speed up
>  > of CC Mode.

> This change has hardly any effect here.  Scrolling xdisp.c with a '-O0
> -g2 -gdwarf-2' build run via emacs -Q is still virtually impossible.

Could you, perhaps, try running with font-lock-maximum-decoration set to
2 for C Mode?  This gives a speed advantage of around a factor of 3 over
decoration level 3, at a cost of some inaccuracies.

I have not finished squeezing cycles out of level 3 font locking.  I can
foresee a total speed up of around 70% (from before the recent 33%).

> martin

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: 33% speed up in CC Mode scrolling.

[martin rudalics]

[-- Attachment #1: Type: text/plain, Size: 1088 bytes --]

 > Could you, perhaps, try running with font-lock-maximum-decoration set to
 > 2 for C Mode?  This gives a speed advantage of around a factor of 3 over
 > decoration level 3, at a cost of some inaccuracies.

Hardly any better.  Setting it to 1 doesn't improve things much either.
The optimized build with decorations at level 3 outperforms the debug
build with decorations at level 1 by an order of magnitude.

I used the attached file test2.el.  If you want to try it set
‘font-lock-maximum-decoration’ to the according value and adapt the path
to frame.c to the one of your emacs src directory.  With emacs -Q
evaluate the test2.el file and do M-x foofoo.  Here I get the following
results for my debug build (the call count is 169):

font-lock-maximum-decoration  elapsed  average
3                             1.999    0.012
2                             1.765    0.010
1                             1.466    0.009

In comparison the optimized build with ‘font-lock-maximum-decoration’ 3
gets me:

3                             0.409    0.002

martin

[-- Attachment #2: test2.el --]
[-- Type: application/emacs-lisp, Size: 628 bytes --]

Re: 33% speed up in CC Mode scrolling.

[martin rudalics]

[-- Attachment #1: Type: text/plain, Size: 1250 bytes --]

 > font-lock-maximum-decoration  elapsed  average
 > 3                             1.999    0.012
 > 2                             1.765    0.010
 > 1                             1.466    0.009
 >
 > In comparison the optimized build with ‘font-lock-maximum-decoration’ 3
 > gets me:
 >
 > 3                             0.409    0.002

These tests actually time scrolling only so they are misleading.  Using
the modified attached file the results below are more accurate: R stands
for the release branch, M for master, and O for an optimized build of
master.  The following number is the ‘font-lock-maximum-decoration’
value - 1, 2 or 3.  So R3 stands for the release branch with decoration
level 3.  The second column is the elapsed, the third the average time.

R3     109.61000000  0.6485798816
M3      79.62599999  0.4711597633
O3      11.516       0.0681420118
R2      23.71799999  0.1403431952
M2      25.35900000  0.1500532544
R1      19.32799999  0.1143668639
M1      20.406       0.1207455621

So there is a significant speed up from R3 to M3 and the speed up of M3
to M2 is by a factor of 3 approximately.  With decoration levels 2 and 1
master is slightly slower than the release branch here.

martin

[-- Attachment #2: test2.el --]
[-- Type: application/emacs-lisp, Size: 660 bytes --]

Re: 33% speed up in CC Mode scrolling.

[Alan Mackenzie]

Hello, Martin.

On Mon, Jun 27, 2016 at 07:28:55PM +0200, martin rudalics wrote:
>  > font-lock-maximum-decoration  elapsed  average
>  > 3                             1.999    0.012
>  > 2                             1.765    0.010
>  > 1                             1.466    0.009

>  > In comparison the optimized build with ‘font-lock-maximum-decoration’ 3
>  > gets me:

>  > 3                             0.409    0.002

> These tests actually time scrolling only so they are misleading.  Using
> the modified attached file the results below are more accurate: R stands
> for the release branch, M for master, and O for an optimized build of
> master.  The following number is the ‘font-lock-maximum-decoration’
> value - 1, 2 or 3.  So R3 stands for the release branch with decoration
> level 3.  The second column is the elapsed, the third the average time.

> R3     109.61000000  0.6485798816
> M3      79.62599999  0.4711597633
> O3      11.516       0.0681420118
> R2      23.71799999  0.1403431952
> M2      25.35900000  0.1500532544
> R1      19.32799999  0.1143668639
> M1      20.406       0.1207455621

> So there is a significant speed up from R3 to M3 .....

This is the 33% speedup of the Subject: line.

> ....., and the speed up of M3 to M2 is by a factor of 3 approximately.

I see this, too.  The point is that fontification can hardly go faster
than level 1, which is mainly just syntactic fontification (i.e. strings
and comments).  The difference between levels 2 and 1 is probably small
enough that one might as well use level 2.

> With decoration levels 2 and 1 master is slightly slower than the
> release branch here.

I should keep an eye on this to make sure "slightly" is as far as it
goes.  It is interesting that the optimised compilation wins as much as
a factor of ~7.

My timing on this is (M-x foofoo on .../src/frame.c):

O3        7.1140785890  0.0817710182
Q3        8.939460839   0.1027524234

, where "Q3" is an optimised build from the release branch.

> martin

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: 33% speed up in CC Mode scrolling.

[martin rudalics]

 >> O3      11.516       0.0681420118
...
 > My timing on this is (M-x foofoo on .../src/frame.c):
 >
 > O3        7.1140785890  0.0817710182
 > Q3        8.939460839   0.1027524234
 >
 > , where "Q3" is an optimised build from the release branch.

Your frames seem to have twice as many lines.

martin

Re: 33% speed up in CC Mode scrolling.

[Alan Mackenzie]

Hello, Martin.

On Tue, Jun 28, 2016 at 10:23:49AM +0200, martin rudalics wrote:
>  >> O3      11.516       0.0681420118
> ...
>  > My timing on this is (M-x foofoo on .../src/frame.c):

>  > O3        7.1140785890  0.0817710182
>  > Q3        8.939460839   0.1027524234

>  > , where "Q3" is an optimised build from the release branch.

> Your frames seem to have twice as many lines.

65 lines + mode line + echo area = 67.  That's how many 16-pixel lines fit on my
Linux tty on a monitor 1080 pixels high.  (It should actually be 67½
:-).

On X-windows, (window-height) returns 35.  So it's about twice as many
lines, yes, but not quite.

> martin

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: 33% speed up in CC Mode scrolling.

[Eli Zaretskii]

> Date: Mon, 27 Jun 2016 08:31:42 +0200
> From: martin rudalics <rudalics@gmx.at>
> 
> This change has hardly any effect here.  Scrolling xdisp.c with a '-O0
> -g2 -gdwarf-2' build run via emacs -Q is still virtually impossible.

What exactly does "scrolling xdisp.c [...] is still virtually
impossible" mean?  How do you scroll?

Re: 33% speed up in CC Mode scrolling.

[martin rudalics]

 >> This change has hardly any effect here.  Scrolling xdisp.c with a '-O0
 >> -g2 -gdwarf-2' build run via emacs -Q is still virtually impossible.
 >
 > What exactly does "scrolling xdisp.c [...] is still virtually
 > impossible" mean?  How do you scroll?

By leaning on the <next> key which runs ‘scroll-up-command’.

martin

Re: 33% speed up in CC Mode scrolling.

[Eli Zaretskii]

> Date: Mon, 27 Jun 2016 18:57:09 +0200
> From: martin rudalics <rudalics@gmx.at>
> CC: acm@muc.de, johnw@gnu.org, emacs-devel@gnu.org
> 
>  >> This change has hardly any effect here.  Scrolling xdisp.c with a '-O0
>  >> -g2 -gdwarf-2' build run via emacs -Q is still virtually impossible.
>  >
>  > What exactly does "scrolling xdisp.c [...] is still virtually
>  > impossible" mean?  How do you scroll?
> 
> By leaning on the <next> key which runs ‘scroll-up-command’.

Then don't lean.

Once you get past the initial large comment, the scrolling becomes
more responsive.  And I definitely do see a visible speedup when I set
font-lock-maximum-decoration to 2.  An unoptimized 32-build with
wide-ints here.

Re: 33% speed up in CC Mode scrolling.

[martin rudalics]

 >> By leaning on the <next> key which runs ‘scroll-up-command’.
 >
 > Then don't lean.

Old habits die hard.  And it's only C buffers causing problems here.

 > Once you get past the initial large comment, the scrolling becomes
 > more responsive.  And I definitely do see a visible speedup when I set
 > font-lock-maximum-decoration to 2.

I can't because the display still freezes.

martin

Re: 33% speed up in CC Mode scrolling.

[Alan Mackenzie]

Hello, Martin.

On Tue, Jun 28, 2016 at 10:23:20AM +0200, martin rudalics wrote:
>  >> By leaning on the <next> key which runs ‘scroll-up-command’.

>  > Then don't lean.

> Old habits die hard.  And it's only C buffers causing problems here.

>  > Once you get past the initial large comment, the scrolling becomes
>  > more responsive.  And I definitely do see a visible speedup when I set
>  > font-lock-maximum-decoration to 2.

> I can't because the display still freezes.

Please set `fast-but-imprecise-scrolling' to non-nil.  This is intended
precisely to prevent the display freezing when font-locking a screen
takes longer than the repeat period of an auto-repeat keyboard.

> martin

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#23861: 25.1.50; fast-but-imprecise-scrolling: What does "somewhat inaccurate" mean?

[Phil Sainty]

It's unclear to me what the following excerpt from the docstring
for the `fast-but-imprecise-scrolling' variable actually means:

 > Note that this optimization can cause the portion of the buffer
 > displayed after a scrolling operation to be somewhat inaccurate.

"somewhat inaccurate" is incredibly vague if you don't already
know what it means.

Does it mean that the position scrolled to might be further (or
less far) than it would have been otherwise?

Does it mean that the font-locking may be incomplete or incorrect?

Does it mean that the buffer text may appear corrupted?!

I certainly can't tell what it means. It would be good if that
text was clarified.

bug#23861: 25.1.50; fast-but-imprecise-scrolling: What does "somewhat inaccurate" mean?

[Eli Zaretskii]

> From: Phil Sainty <psainty@orcon.net.nz>
> Date: Wed, 29 Jun 2016 01:03:00 +1200
> 
> It's unclear to me what the following excerpt from the docstring
> for the `fast-but-imprecise-scrolling' variable actually means:
> 
>  > Note that this optimization can cause the portion of the buffer
>  > displayed after a scrolling operation to be somewhat inaccurate.
> 
> "somewhat inaccurate" is incredibly vague if you don't already
> know what it means.

It means that what's on the screen might not reflect what's in the
buffer.

It's hard to tell anything more specific, since which redisplay
optimizations will be taken in each case depends on many factors on
which the user has no control.

> Does it mean that the position scrolled to might be further (or
> less far) than it would have been otherwise?

It could.

> Does it mean that the font-locking may be incomplete or incorrect?

It could.

> Does it mean that the buffer text may appear corrupted?!

It could, if faces change the font size.

bug#23861: 25.1.50; fast-but-imprecise-scrolling: What does "somewhat inaccurate" mean?

[Stefan Kangas]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Phil Sainty <psainty@orcon.net.nz>
>> Date: Wed, 29 Jun 2016 01:03:00 +1200
>>
>> It's unclear to me what the following excerpt from the docstring
>> for the `fast-but-imprecise-scrolling' variable actually means:
>>
>>  > Note that this optimization can cause the portion of the buffer
>>  > displayed after a scrolling operation to be somewhat inaccurate.
>>
>> "somewhat inaccurate" is incredibly vague if you don't already
>> know what it means.
>
> It means that what's on the screen might not reflect what's in the
> buffer.
>
> It's hard to tell anything more specific, since which redisplay
> optimizations will be taken in each case depends on many factors on
> which the user has no control.
>
>> Does it mean that the position scrolled to might be further (or
>> less far) than it would have been otherwise?
>
> It could.
>
>> Does it mean that the font-locking may be incomplete or incorrect?
>
> It could.
>
>> Does it mean that the buffer text may appear corrupted?!
>
> It could, if faces change the font size.

That was 3 years ago.  It doesn't look like there is anything more to do
here, so I'm closing this bug report.

Please reopen the bug if that's incorrect.

Best regards,
Stefan Kangas