Why emacs rendering is slow when encountering long lines?

Why emacs rendering is slow when encountering long lines?

[Jiaxin Cao]

When I open a file containing long lines, it is not smooth to scroll up and down, even if the file is not large. 

Does anyone know how to configure emacs to make it smoother?

Re: Why emacs rendering is slow when encountering long lines?

[Jiaxin Cao]

On Sunday, October 20, 2013 1:00:45 PM UTC+8, Jiaxin Cao wrote:
> When I open a file containing long lines, it is not smooth to scroll up and down, even if the file is not large. 
> 
> 
> 
> Does anyone know how to configure emacs to make it smoother?

Specifically, scroll-down is way worse than scroll-up. 

I did a little test for it. A single scroll up takes about 15ms to finish, while a scroll down takes about 50ms. 

So if you press down pageup to trigger consecutive scroll-downs, the screen will basically not respond to anything. 

Anyone can explain why?

Re: Why emacs rendering is slow when encountering long lines?

[Eli Zaretskii]

> Date: Sun, 20 Oct 2013 03:25:30 -0700 (PDT)
> From: Jiaxin Cao <jiaxin.cao@gmail.com>
> 
> On Sunday, October 20, 2013 1:00:45 PM UTC+8, Jiaxin Cao wrote:
> > When I open a file containing long lines, it is not smooth to scroll up and down, even if the file is not large. 
> > 
> > 
> > 
> > Does anyone know how to configure emacs to make it smoother?

Hard to tell without knowing what was in the file.  The performance is
somewhat sensitive to the contents of the file.

But in general, the Emacs display engine is known to have trouble with
very long lines, see bug #13675.

> Specifically, scroll-down is way worse than scroll-up. 
> 
> I did a little test for it. A single scroll up takes about 15ms to finish, while a scroll down takes about 50ms. 
> 
> So if you press down pageup to trigger consecutive scroll-downs, the screen will basically not respond to anything. 
> 
> Anyone can explain why?

The reasons for the asymmetry are complex and have to do with the
display engine internals.  But first let me ask you: when you say
scroll-up, do you mean go forward in the buffer, or do you mean going
back?

Also, what was the major mode of the buffer where you tried this?

Re: Why emacs rendering is slow when encountering long lines?

[Dan Espen]

Jiaxin Cao <jiaxin.cao@gmail.com> writes:

> On Sunday, October 20, 2013 1:00:45 PM UTC+8, Jiaxin Cao wrote:
>> When I open a file containing long lines, it is not smooth to scroll up and down, even if the file is not large. 
>> 
>> 
>> 
>> Does anyone know how to configure emacs to make it smoother?
>
> Specifically, scroll-down is way worse than scroll-up. 
> I did a little test for it. A single scroll up takes about 15ms to finish, while a scroll down takes about 50ms. 
> So if you press down pageup to trigger consecutive scroll-downs, the screen will basically not respond to anything. 
> Anyone can explain why?

I just opened /usr/bin/ls.  It has pretty long lines.
It also scrolls pretty fast in both directions.

I think you need to provide more information, including emacs version
and the mode that emacs goes into.
Also see if the problem is still there when you start with emacs -Q.

-- 
Dan Espen

Re: Why emacs rendering is slow when encountering long lines?

[Eli Zaretskii]

> From: Dan Espen <despen@verizon.net>
> Date: Sun, 20 Oct 2013 11:50:07 -0400
> 
> I just opened /usr/bin/ls.  It has pretty long lines.
> It also scrolls pretty fast in both directions.

That's because /usr/bin/ls is a binary file, so Emacs automatically
visits it literally (i.e. with "no-conversions").  Many display
features are turned off in buffers that visit files literally.

> I think you need to provide more information, including emacs version
> and the mode that emacs goes into.
> Also see if the problem is still there when you start with emacs -Q.

Indeed.

Re: Why emacs rendering is slow when encountering long lines?

[Allan Streib]

Jiaxin Cao <jiaxin.cao@gmail.com> writes:

> When I open a file containing long lines, it is not smooth to scroll up and down, even if the file is not large. 

Have you tried enabling truncate-lines? I realize this is a matter of
subjective preference, but I generally prefer this for any programming
modes (I only wrap long lines for reading text e.g. email)

Allan

Re: Why emacs rendering is slow when encountering long lines?

[Jiaxin Cao]

On Monday, October 21, 2013 12:09:26 AM UTC+8, Allan Streib wrote:
> Jiaxin Cao <jiaxin.cao@gmail.com> writes:
> 
> 
> 
> > When I open a file containing long lines, it is not smooth to scroll up and down, even if the file is not large. 
> 
> 
> 
> Have you tried enabling truncate-lines? I realize this is a matter of
> 
> subjective preference, but I generally prefer this for any programming
> 
> modes (I only wrap long lines for reading text e.g. email)
> 
> 
> 
> Allan

Yes, I tried both cases with truncate-lines enabled and disabled. Scroll-down is bad in both cases. Scroll-up behaves a little better. 

Let me provide more information about this. To reproduce the scenario, just create a line containing about 450 chars, and copy the line 10000 times. I use the text-mode to test this buffer. Scroll-down means the cursor goes up. My OS is windows 8. Emacs version is 24.3.1.

I think this is not the problem of the rendering engine. You can do some tests like this. 
(defun test-redraw-display ()
  (interactive)
  (message (format-time-string "%Y-%m-%dT%T:%6N"))
  (redraw-display)
  (message (format-time-string "%Y-%m-%dT%T:%6N")))

(defun test-scroll-down ()
  (interactive)
  (message (format-time-string "%Y-%m-%dT%T:%6N"))
  (scroll-down)
  (message (format-time-string "%Y-%m-%dT%T:%6N")))

(defun test-scroll-up ()
  (interactive)
  (message (format-time-string "%Y-%m-%dT%T:%6N"))
  (scroll-up)
  (message (format-time-string "%Y-%m-%dT%T:%6N")))

Here is the test results on the buffer mentioned above.
|-------------------------+----------------+-------------+-----------|
|                         | redraw-display | scroll-down | scroll-up |
|-------------------------+----------------+-------------+-----------|
| Truncate-lines enabled  | < 1ms          | 82ms        | 47ms      |
| Truncate-lines disabled | < 1ms          | 76ms        | 54ms      |
|-------------------------+----------------+-------------+-----------|

Based on the results, I guess the scroll-down is the bottleneck of the scrolling performance. I know scroll-down is a little complicated to implement, but if you  do the same thing in notepad, you'll find notepad is way smoother than emacs in term of scrolling.

Re: Why emacs rendering is slow when encountering long lines?

[Eli Zaretskii]

> Date: Sun, 20 Oct 2013 19:08:51 -0700 (PDT)
> From: Jiaxin Cao <jiaxin.cao@gmail.com>
> Injection-Date: Mon, 21 Oct 2013 02:08:52 +0000
> 
> Let me provide more information about this. To reproduce the scenario, just create a line containing about 450 chars, and copy the line 10000 times.

What characters?  It's important.

Also, if you add an empty line here and there, do things improve?

> I use the text-mode to test this buffer. Scroll-down means the cursor goes up. My OS is windows 8. Emacs version is 24.3.1.

In that case, indeed scroll-down is expected to be slower than
scroll-up.  The technical reasons, in a nutshell, is that Emacs needs
to find where to start the display of the next screenful, and that is
harder when you go backwards in the buffer.

> I think this is not the problem of the rendering engine. You can do some tests like this. 
> (defun test-redraw-display ()
>   (interactive)
>   (message (format-time-string "%Y-%m-%dT%T:%6N"))
>   (redraw-display)
>   (message (format-time-string "%Y-%m-%dT%T:%6N")))
> 
> (defun test-scroll-down ()
>   (interactive)
>   (message (format-time-string "%Y-%m-%dT%T:%6N"))
>   (scroll-down)
>   (message (format-time-string "%Y-%m-%dT%T:%6N")))
> 
> (defun test-scroll-up ()
>   (interactive)
>   (message (format-time-string "%Y-%m-%dT%T:%6N"))
>   (scroll-up)
>   (message (format-time-string "%Y-%m-%dT%T:%6N")))
> 
> Here is the test results on the buffer mentioned above.
> |-------------------------+----------------+-------------+-----------|
> |                         | redraw-display | scroll-down | scroll-up |
> |-------------------------+----------------+-------------+-----------|
> | Truncate-lines enabled  | < 1ms          | 82ms        | 47ms      |
> | Truncate-lines disabled | < 1ms          | 76ms        | 54ms      |
> |-------------------------+----------------+-------------+-----------|
> 
> Based on the results, I guess the scroll-down is the bottleneck of the scrolling performance.

Your conclusion is wrong.  You assume that scroll-down is just that,
but it isn't.  It invokes the display engine to know where to start
the next/previous screenful.  Try to think of an algorithm to
determine that, when the buffer might have different fonts, and I
think you will see why.

> I know scroll-down is a little complicated to implement, but if you  do the same thing in notepad, you'll find notepad is way smoother than emacs in term of scrolling.

Emacs gives you a plethora of display features that Notepad can only
dream of.  Start with the fact that Emacs needs to support
variable-size fonts in the same buffer and even on the same line.
That complicates matters quite a lot.

Re: Why emacs rendering is slow when encountering long lines?

[Jules Colding]

On Oct 21, 2013, at 5:00 AM, Eli Zaretskii <eliz@gnu.org> wrote:

>> Date: Sun, 20 Oct 2013 19:08:51 -0700 (PDT)
>> From: Jiaxin Cao <jiaxin.cao@gmail.com>
>> Injection-Date: Mon, 21 Oct 2013 02:08:52 +0000
>> 
>> Let me provide more information about this. To reproduce the scenario, just create a line containing about 450 chars, and copy the line 10000 times.
> 
>> 
>> Based on the results, I guess the scroll-down is the bottleneck of the scrolling performance.
> 
> Your conclusion is wrong.  You assume that scroll-down is just that,
> but it isn't.  It invokes the display engine to know where to start
> the next/previous screenful.  Try to think of an algorithm to
> determine that, when the buffer might have different fonts, and I
> think you will see why.

I have a file here:

https://github.com/colding/MercuryFIX/blob/master/applib/fixio_tests/check_fixio.cpp

which has an extremely long word (one font, all caps) at line 213. As soon as scrolling down comes within 2-3 lines distance, everything stops for a second or two.

Just a datapoint...


-- 
  jules

Re: Why emacs rendering is slow when encountering long lines?

[Eli Zaretskii]

> Date: Mon, 21 Oct 2013 06:00:52 +0300
> From: Eli Zaretskii <eliz@gnu.org>
> 
> > Date: Sun, 20 Oct 2013 19:08:51 -0700 (PDT)
> > From: Jiaxin Cao <jiaxin.cao@gmail.com>
> > Injection-Date: Mon, 21 Oct 2013 02:08:52 +0000
> > 
> > Let me provide more information about this. To reproduce the scenario, just create a line containing about 450 chars, and copy the line 10000 times.
> 
> What characters?  It's important.
> 
> Also, if you add an empty line here and there, do things improve?
> 
> > I use the text-mode to test this buffer. Scroll-down means the cursor goes up. My OS is windows 8. Emacs version is 24.3.1.

Btw, just for comparison, I made a file like you described with each
line made of 450 'a' characters, and tried your timing functions.  My
times on a 3-year old machine running Windows 7 are much faster: about
50 msec for scroll-down and less than 20 for scroll-up.  It's short
enough to not annoy.

Re: Why emacs rendering is slow when encountering long lines?

[Eli Zaretskii]

> From: Jules Colding <colding@venalicium.dk>
> Date: Mon, 21 Oct 2013 16:39:39 +0200
> Cc: help-gnu-emacs@gnu.org
> 
> I have a file here:
> 
> https://github.com/colding/MercuryFIX/blob/master/applib/fixio_tests/check_fixio.cpp
> 
> which has an extremely long word (one font, all caps) at line 213. As soon as scrolling down comes within 2-3 lines distance, everything stops for a second or two.
> 
> Just a datapoint...

That's an entirely different issue.  To see why, type

  M-x global-font-lock-mode RET

and then visit that file and scroll through the problematic spot.
Puff! the problem goes away.

So not really a data point wrt the issue at hand.