Emacs 23, so slow to refresh on Windows

Emacs 23, so slow to refresh on Windows

[bogossian]

Hi,

with the release of emacs 23.1 last year, I've noticed a huge 
performance
regression with the Windows build. I hoped this would be fixed in the 
next
release, but the version 23.2 is just as slow.
Emacs is especially slow when scrolling through files containing very 
long
lines. With some files, it can hang for several minutes hogging the CPU.

To illustrate the slow down, I built an 8MB file containing a single 
line.
I opened the file in emacs, and then I hit "ESC >" to reach the end of 
the
buffer. I measured the time it takes for emacs to refresh from the 
moment
I typed the macro. Here are the results:

Emacs 21.3.1:    8s
Emacs 22.3.1:   14s
Emacs 23.2.1:   63s

(Test setup: Athlon XP 2GHz, Windows XP SP3)

In the tests above, emacs was started using the command line "emacs -q".
Now, if I repeat the same test using the command "emacs -q -fn 6x13", to
use my usual font, things are getting even worse:

Emacs 21.3.1:    8s
Emacs 22.3.1:   14s
Emacs 23.2.1:  515s

I searched the mailing list and found a guy complaining about similar
problems back in 2008:

http://lists.gnu.org/archive/html/emacs-devel/2008-06/msg01136.html

The replies he received seem to indicate that the performance problem 
was
well known (and related to the new font backend I guess), though it's 
not
very clear whether or not someone was working on addressing it.

So, is there hope to see improvements in the future ?

Regards,

Pierre

Re: Emacs 23, so slow to refresh on Windows

[Juanma Barranquero]

On Mon, May 17, 2010 at 15:19,  <bogossian@mail.com> wrote:

> To illustrate the slow down, I built an 8MB file containing a single line.

Emacs is not optimized for ultra-long lines. Do you find it also slow
editing more normal-looking files?

> The replies he received seem to indicate that the performance problem was
> well known (and related to the new font backend I guess), though it's not
> very clear whether or not someone was working on addressing it.

Could you try

  emacs -q -xrm Emacs.FontBackend:gdi

and see whether it makes a difference?

> So, is there hope to see improvements in the future ?

For a while, during the development of 23.1, the new font backend was
really slow on Windows, but currently it is not so (or, at least, we
don't receive many complaints about its speed); though it's true that
it is slower than 22.X; it's the price to pay for much better font
support.

If you find it unbearably slow I'd suggest filing a bug in the Emacs
bug tracker, giving as much relevant information as possible (for
example, fonts used, etc.).

    Juanma

Re: Emacs 23, so slow to refresh on Windows

[bogossian]

>Emacs is not optimized for ultra-long lines. Do you find it also slow
>editing more normal-looking files?

No, I've done more testing and so far I haven't noticed any slowdown 
with
normal files (ie. files without long lines).
So, I think we can assume the performance issue is specific to files 
with
long lines.

>Could you try
>
>  emacs -q -xrm Emacs.FontBackend:gdi
>
>and see whether it makes a difference?

Yes, it brings an improvement with the default font:

Emacs 21.3.1:              8s
Emacs 22.3.1:             14s
Emacs 23.2.1 (uniscribe): 63s
Emacs 23.2.1 (gdi):       38s

With the 6x13 font, it doesn't make a difference (6x13 being a bitmap
font, I suppose it's always rendered using the gdi backend).

>If you find it unbearably slow I'd suggest filing a bug in the Emacs
>bug tracker, giving as much relevant information as possible (for
>example, fonts used, etc.).

I might do that indeed.
Though I can understand that Emacs is not optimized for very long lines,
it's supposed to be a versatile tool, so it should at least perform
decently with any kind of files and fonts. And I'm probably not the only
one who has to deal with computer generated files (dumps, logs, export
files, ...) containing very long lines.

Thank you for your reply Juanma,

Pierre

Re: Emacs 23, so slow to refresh on Windows

[Juanma Barranquero]

On Tue, May 18, 2010 at 01:52,  <bogossian@mail.com> wrote:

> No, I've done more testing and so far I haven't noticed any slowdown with
> normal files (ie. files without long lines).
> So, I think we can assume the performance issue is specific to files with
> long lines.

Assuming you use continuation lines, and not truncation, does the
speed improve setting `cache-long-line-scans' to t? (Note that the
variable is automatically buffer-local, so be sure to set it in the
appropriate buffer, or use setq-default.)

> With the 6x13 font, it doesn't make a difference (6x13 being a bitmap
> font, I suppose it's always rendered using the gdi backend).

Yes, I think so. You can see the backend used for a font by moving the
cursor over a character in that font and doing "C-u C-x =". The font
information will start with gdi: or uniscribe:

> And I'm probably not the only
> one who has to deal with computer generated files (dumps, logs, export
> files, ...) containing very long lines.

No, that is indeed very common.

    Juanma

Re: Emacs 23, so slow to refresh on Windows

[Eli Zaretskii]

> From: Juanma Barranquero <lekktu@gmail.com>
> Date: Tue, 18 May 2010 09:25:28 +0200
> Cc: emacs-devel@gnu.org
> 
> On Tue, May 18, 2010 at 01:52,  <bogossian@mail.com> wrote:
> 
> > No, I've done more testing and so far I haven't noticed any slowdown with
> > normal files (ie. files without long lines).
> > So, I think we can assume the performance issue is specific to files with
> > long lines.
> 
> Assuming you use continuation lines, and not truncation, does the
> speed improve setting `cache-long-line-scans' to t?

Note that in a test file with only one line, this could give skewed
results.

Re: Emacs 23, so slow to refresh on Windows

[Juanma Barranquero]

On Tue, May 18, 2010 at 11:13, Eli Zaretskii <eliz@gnu.org> wrote:

> Note that in a test file with only one line, this could give skewed
> results.

AFAICS, the OP is interested in computer-generated files (dumps, logs,
etc.), which presumably have lots of lines. The
just-one-extremely-long-line file is a contrived example (not that
these kind of files do not exist, but I suppose they aren't that
common).

    Juanma

Re: Emacs 23, so slow to refresh on Windows

[Eli Zaretskii]

> From: Juanma Barranquero <lekktu@gmail.com>
> Date: Tue, 18 May 2010 11:32:56 +0200
> Cc: bogossian@mail.com, emacs-devel@gnu.org
> 
> On Tue, May 18, 2010 at 11:13, Eli Zaretskii <eliz@gnu.org> wrote:
> 
> > Note that in a test file with only one line, this could give skewed
> > results.
> 
> AFAICS, the OP is interested in computer-generated files (dumps, logs,
> etc.), which presumably have lots of lines. The
> just-one-extremely-long-line file is a contrived example

Exactly.  So what I was saying is that measuring the performance on a
file with a single long line will not give a good estimate of the
effect of this variable on real-life files.

Re: Emacs 23, so slow to refresh on Windows

[bogossian]

>AFAICS, the OP is interested in computer-generated files (dumps, logs,
>etc.), which presumably have lots of lines. The
>just-one-extremely-long-line file is a contrived example (not that
>these kind of files do not exist, but I suppose they aren't that
>common).

Exactly, the 8MB file was just a simple and easy to reproduce example,
that I could use in a small benchmark in order to get some figures.

Given that you're using the right font (such as 6x13 ou the bitmap 
version
of courier), even scrolling though a file whose lines are 1000 
characters long
will feel painfully slow with Emacs 23, while it was perfectly smooth 
with
previous versions.

But I have sometimes to open log files containing lines that can be 
tens of
thousands characters long. For such files, Emacs is totally unusable, 
every
move in the file resulting in a long hang.

Re: Emacs 23, so slow to refresh on Windows

[bogossian]

>Assuming you use continuation lines, and not truncation, does the
>speed improve setting `cache-long-line-scans' to t? (Note that the
>variable is automatically buffer-local, so be sure to set it in the
>appropriate buffer, or use setq-default.)

Setting this variable doesn't make a noticeable difference.
For instance, scrolling through a 200k csv file made of 200 lines of 
1000
characters each, feels just as choppy whether the cache is enabled or 
not.

>Yes, I think so. You can see the backend used for a font by moving the
>cursor over a character in that font and doing "C-u C-x =". The font
>information will start with gdi: or uniscribe:

As expected the gdi backend is used:

        character: D (68, #o104, #x44)
preferred charset: ascii (ASCII (ISO646 IRV))
       code point: 0x44
           syntax: w    which means: word
         category: .:Base, a:ASCII, l:Latin, r:Roman
      buffer code: #x44
        file code: #x44 (encoded by coding system undecided-dos)
          display: by this font (glyph code)
     gdi:-raster-6X13-normal-normal-normal-*-13-*-*-*-c-*-iso8859-1 
(#x44)

Character code properties: customize what to show
  name: LATIN CAPITAL LETTER D
  general-category: Lu (Letter, Uppercase)

BTW, 6x13 is not the only font for which I noticed these extremely bad
performances. The bitmap version of the courier font (which I think is
installed by default with Windows XP) exhibits the same slowdown.
So it should be easy for you to reproduce the problem using this font.

        character: D (68, #o104, #x44)
preferred charset: ascii (ASCII (ISO646 IRV))
       code point: 0x44
           syntax: w    which means: word
         category: .:Base, a:ASCII, l:Latin, r:Roman
      buffer code: #x44
        file code: #x44 (encoded by coding system undecided-dos)
          display: by this font (glyph code)
     
gdi:-raster-Courier-normal-normal-normal-mono-13-*-*-*-c-*-iso8859-1 
(#x44)

Character code properties: customize what to show
  name: LATIN CAPITAL LETTER D
   general-category: Lu (Letter, Uppercase)

Re: Emacs 23, so slow to refresh on Windows

[Eli Zaretskii]

> Date: Mon, 17 May 2010 19:52:44 -0400
> From: bogossian@mail.com
> Cc: emacs-devel@gnu.org
> 
> Though I can understand that Emacs is not optimized for very long lines,
> it's supposed to be a versatile tool, so it should at least perform
> decently with any kind of files and fonts. And I'm probably not the only
> one who has to deal with computer generated files (dumps, logs, export
> files, ...) containing very long lines.

Can you make your file with the 8MB-long line available somewhere?

Also, while a single 8MB-long line might be good for making a point,
what was the real-life use case that made you aware of the slow-down?

Re: Emacs 23, so slow to refresh on Windows

[David Kastrup]

Eli Zaretskii <eliz@gnu.org> writes:

>> Date: Mon, 17 May 2010 19:52:44 -0400
>> From: bogossian@mail.com
>> Cc: emacs-devel@gnu.org
>> 
>> Though I can understand that Emacs is not optimized for very long lines,
>> it's supposed to be a versatile tool, so it should at least perform
>> decently with any kind of files and fonts. And I'm probably not the only
>> one who has to deal with computer generated files (dumps, logs, export
>> files, ...) containing very long lines.
>
> Can you make your file with the 8MB-long line available somewhere?

C-u 8000000 x C-p

in the *scratch* buffer should be instructive, I think.

-- 
David Kastrup

Re: Emacs 23, so slow to refresh on Windows

[bogossian]

>Can you make your file with the 8MB-long line available somewhere?

I generated my file like that:

      seq 1 1300000 | tr '\n' '*' | head -c 8m > foo.txt

But you can use any file, as long as you removed all newlines (tr -d 
'\n' should do the job).

Re: Emacs 23, so slow to refresh on Windows

[Uday S Reddy]

On 5/17/2010 2:19 PM, bogossian@mail.com wrote:

> To illustrate the slow down, I built an 8MB file containing a single line.
> I opened the file in emacs, and then I hit "ESC >" to reach the end of the
> buffer. I measured the time it takes for emacs to refresh from the moment
> I typed the macro. Here are the results:
>
> Emacs 21.3.1: 8s
> Emacs 22.3.1: 14s
> Emacs 23.2.1: 63s

Hi, does it make a difference if you turn on the visual-line-mode in the buffer?

Cheers,
Uday

Re: Emacs 23, so slow to refresh on Windows

[bogossian]

>>Emacs 21.3.1: 8s
>>Emacs 22.3.1: 14s
>>Emacs 23.2.1: 63s
>
>Hi, does it make a difference if you turn on the visual-line-mode in 
the
>buffer?

If my super-long-line doesn't contain any space, it doesn't make a 
difference.
If I use a line with many spaces, it's a little slower: 67s

Pierre