bug#6364: Windows: Emacs 23 slow with long lines and raster fonts

bug#6364: Windows: Emacs 23 slow with long lines and raster fonts

[bogossian]

Hi,

the Windows build of Emacs 23 is extremely slow when scrolling in a 
buffer
containing long lines if a bitmap font is used.

For example, scrolling though a file made of 200 lines of 1000 
characters
each feels choppy.
Doing the same with a file containing very long lines (like tens of 
thousand
characters) can make Emacs hang for seconds or minutes, making it 
practically
unusable.

Editing files with such long lines is not absurd, various types of 
computer
generated files (log files, export files, dumps, ...) can have very very
long lines.

These bad performances are a regression, Emacs 21 and Emcas 22 were much
faster in the same conditions.

To illustrate this regression, and to show it's related to the use of 
bitmap
fonts, I've made this very simple and easy to reproduce benchmark:

I built an 8MB file containing a single line (the content of the file
doesn't matter). I opened the file in emacs, and then I hit "ESC >" to 
reach
the end of the buffer. I measured the time it takes for emacs to refresh
from the moment I typed the macro.

(Test setup: Athlon XP 2GHz, Windows XP SP3)

If Emacs is started with the command line "emacs -q", here are the 
results:

Emacs 21.3.1:    8s
Emacs 22.3.1:   14s
Emacs 23.2.1:   63s  (uniscribe backend)

Emacs 23 can be made a little faster by forcing the gdi rendering 
backend
with the command line "emacs -q -xrm Emacs.FontBackend:gdi":

Emacs 23.2.1:   38s  (gdi backend)

Now, if I repeat the same test using the command "emacs -q -fn 
Courier", to
use a bitmap font, things are getting really bad for Emacs 23:

Emacs 21.3.1:    8s  (raster font)
Emacs 22.3.1:   14s  (raster font)
Emacs 23.2.1:  515s  (raster font, gdi backend)

Regards,

Pierre

PS: this issue has been discussed on the emacs-devel mailing list:
http://lists.gnu.org/archive/html/emacs-devel/2010-05/msg00478.html

bug#6364: [PATCH] Use GetCharABCWidthsFloatW if GetGlyphOutlineW fails.

[Tom Seddon]

Please find below a patch to improve the poor scrolling performance when using bitmap fonts. #14721 (http://debbugs.gnu.org/cgi/bugreport.cgi?bug=14721) and and 14307 (http://debbugs.gnu.org/cgi/bugreport.cgi?bug=14307) may also be affected. The patch has been tested against emacs 24.3. It applied cleanly to git head (22687e54e0e4d7c73c098417478574a55393fe2c) but I haven't built it.

Performance with particularly long lines is still rather poor, but general responsiveness is much improved. (Once the buffer is fontified, emacs can now usually keep up if I hold down PgUp, PgDn, C-s, etc.)

(I settled on GetCharABCWidthsFloatW because it works for bitmap fonts and TrueType fonts alike. But the key thing is simply not to create a DC each time w32font_text_extents is called, so there are various other functions that could be called instead if preferred.)

--Tom

From ccedd16f6bd2027145b9e172346d2c3b31c811df Mon Sep 17 00:00:00 2001
From: Tom Seddon <tom@tmbp-w7>
Date: Mon, 25 Nov 2013 22:19:47 +0000
Subject: [PATCH] Use GetCharABCWidthsFloatW if GetGlyphOutlineW fails.

The previous fallback - which will still be used if
GetCharABCWidthsFloatW fails - was to call GetTextExtentPoint32W. This
can be rather slow due to having to create a DC for it each time.
---
 src/w32font.c | 32 ++++++++++++++++++++++++++++++++
 1 file changed, 32 insertions(+)

diff --git a/src/w32font.c b/src/w32font.c
index 5c5a15c..3577dfa 100644
--- a/src/w32font.c
+++ b/src/w32font.c
@@ -149,6 +149,7 @@ static BOOL g_b_init_get_outline_metrics_w;
 static BOOL g_b_init_get_text_metrics_w;
 static BOOL g_b_init_get_glyph_outline_w;
 static BOOL g_b_init_get_glyph_outline_w;
+static BOOL g_b_init_get_char_abc_widths_float_w;
 
 typedef UINT (WINAPI * GetOutlineTextMetricsW_Proc) (
    HDC hdc,
@@ -165,6 +166,11 @@ typedef DWORD (WINAPI * GetGlyphOutlineW_Proc) (
    DWORD cbBuffer,
    LPVOID lpvBuffer,
    const MAT2 *lpmat2);
+typedef BOOL (WINAPI * GetCharABCWidthsFloatW_Proc) (
+   HDC hdc,
+   UINT uFirstChar,
+   UINT uLastChar,
+   LPABCFLOAT lpabc);
 
 /* Several "wide" functions we use to support the font backends are
    unavailable on Windows 9X, unless UNICOWS.DLL is installed (their
@@ -274,6 +280,23 @@ get_glyph_outline_w (HDC hdc, UINT uChar, UINT uFormat, LPGLYPHMETRICS lpgm,
 				   lpvBuffer, lpmat2);
 }
 
+static DWORD WINAPI get_char_abc_widths_float_w (HDC hdc, UINT uFirstChar,
+						 UINT uLastChar, LPABCFLOAT lpabc)
+{
+  static GetCharABCWidthsFloatW_Proc s_pfn_Get_Char_ABC_Widths_FloatW = NULL;
+  HMODULE hm_unicows = NULL;
+  if (g_b_init_get_char_abc_widths_float_w == 0)
+    {
+      g_b_init_get_char_abc_widths_float_w = 1;
+      hm_unicows = w32_load_unicows_or_gdi32 ();
+      if (hm_unicows)
+	s_pfn_Get_Char_ABC_Widths_FloatW = (GetCharABCWidthsFloatW_Proc)
+	  GetProcAddress (hm_unicows, "GetCharABCWidthsFloatW");
+    }
+  eassert (s_pfn_Get_Char_ABC_Widths_FloatW != NULL);
+  return s_pfn_Get_Char_ABC_Widths_FloatW (hdc, uFirstChar, uLastChar, lpabc);
+}
+
 static int
 memq_no_quit (Lisp_Object elt, Lisp_Object list)
 {
@@ -2438,6 +2461,7 @@ compute_metrics (HDC dc, struct w32font_info *w32_font, unsigned int code,
   GLYPHMETRICS gm;
   MAT2 transform;
   unsigned int options = GGO_METRICS;
+  ABCFLOAT abc;
 
   if (w32_font->glyph_idx)
     options |= GGO_GLYPH_INDEX;
@@ -2454,6 +2478,14 @@ compute_metrics (HDC dc, struct w32font_info *w32_font, unsigned int code,
       metrics->width = gm.gmCellIncX;
       metrics->status = W32METRIC_SUCCESS;
     }
+  else if (get_char_abc_widths_float_w (dc, code, code, &abc) != 0)
+    {
+      int width = (int) (abc.abcfA + abc.abcfB + abc.abcfC);
+      metrics->lbearing = 0;
+      metrics->rbearing = width;
+      metrics->width = width;
+      metrics->status = W32METRIC_SUCCESS;
+    }
   else
     metrics->status = W32METRIC_FAIL;
 }
-- 
1.8.1.msysgit.1

bug#6364: [PATCH] Use GetCharABCWidthsFloatW if GetGlyphOutlineW fails.

[Eli Zaretskii]

> From: Tom Seddon <emacs@tomseddon.plus.com>
> Date: Tue, 26 Nov 2013 00:35:05 +0000
> 
> Please find below a patch to improve the poor scrolling performance when using bitmap fonts. #14721 (http://debbugs.gnu.org/cgi/bugreport.cgi?bug=14721) and and 14307 (http://debbugs.gnu.org/cgi/bugreport.cgi?bug=14307) may also be affected. The patch has been tested against emacs 24.3. It applied cleanly to git head (22687e54e0e4d7c73c098417478574a55393fe2c) but I haven't built it.
> 
> Performance with particularly long lines is still rather poor, but general responsiveness is much improved. (Once the buffer is fontified, emacs can now usually keep up if I hold down PgUp, PgDn, C-s, etc.)

Thanks, but can you please provide a reproducible test case, including
the font where the slow scrolling happens?  I don't see any fonts
named in these bug reports.

> (I settled on GetCharABCWidthsFloatW because it works for bitmap fonts and TrueType fonts alike. But the key thing is simply not to create a DC each time w32font_text_extents is called, so there are various other functions that could be called instead if preferred.)

Are there other possibilities to fix this, without using
GetCharABCWidthsFloatW?  The problem with that function is that it is
not available on Windows 9X (in the unicows.dll library), so this
problem will be left unsolved on those systems.

bug#6364: [PATCH] Use GetCharABCWidthsFloatW if GetGlyphOutlineW fails.

[Tom Seddon]

On 26 Nov 2013, at 17:52, Eli Zaretskii <eliz@gnu.org> wrote:

>> From: Tom Seddon <emacs@tomseddon.plus.com>
>> Date: Tue, 26 Nov 2013 00:35:05 +0000
>> 
>> Please find below a patch to improve the poor scrolling performance when using bitmap fonts. #14721 (http://debbugs.gnu.org/cgi/bugreport.cgi?bug=14721) and and 14307 (http://debbugs.gnu.org/cgi/bugreport.cgi?bug=14307) may also be affected. The patch has been tested against emacs 24.3. It applied cleanly to git head (22687e54e0e4d7c73c098417478574a55393fe2c) but I haven't built it.
>> 
>> Performance with particularly long lines is still rather poor, but general responsiveness is much improved. (Once the buffer is fontified, emacs can now usually keep up if I hold down PgUp, PgDn, C-s, etc.)
> 
> Thanks, but can you please provide a reproducible test case, including
> the font where the slow scrolling happens?  I don't see any fonts
> named in these bug reports.

Certainly. I've uploaded some test cases to http://www.tomseddon.plus.com/.emacs/bug6364.tar.gz.

To reproduce:

0. visit Control Panel, Keyboard, Speed tab, and move the Repeat rate slider all the way to "Fast". Click OK.

1. unpack bug6364.tar.gz somewhere

2. install fixed.fon (its name in the fonts list will be "fixed")

3. start shell and change to folder containing contents of bug6364.tar.gz

4. start emacs: emacs -Q -l start-ok.el. You should see emacs load a file, set Courier New (a truetype font) as the display font, and put point at the end of the file. Once it's loaded, hold PgUp. Note adequate scrolling responsiveness.

5. start emacs: emacs -Q -l start-slow.el. You should see emacs load again, but this time with fixed.fon (a bitmap font). Once again, once it's loaded, hold PgUp. On my PC, once it gets to about 94% of the way through the file, the screen just stops updating. Once you stop holding the key, after a brief pause, emacs catches up again.

With my patch, both cases are pretty much the same.

Try also the other included file, 16384. This doesn't have any .el files associated with it I'm afraid as it was just a quick thing I made to see what performance with longer lines was like. It's 16,384 lines of 700-odd chars each. It's supposed to be viewed with fixed.fon, and truncate-lines off. Start emacs with emacs -Q 16384, then shift+click on the buffer to select the font.

Without my patch, at the top of the file, moving the cursor downwards from screen line to screen line is responsive, but moving it upwards is terribly slow; with my patch, at the top of the file, moving the cursor in either direction is responsive.

With or without my patch, cursor responsiveness moving upwards is poor towards the end of the file.

> 
>> (I settled on GetCharABCWidthsFloatW because it works for bitmap fonts and TrueType fonts alike. But the key thing is simply not to create a DC each time w32font_text_extents is called, so there are various other functions that could be called instead if preferred.)
> 
> Are there other possibilities to fix this, without using
> GetCharABCWidthsFloatW?  The problem with that function is that it is
> not available on Windows 9X (in the unicows.dll library), so this
> problem will be left unsolved on those systems.

I couldn't find a good list when I was writing the code so I just went by DUMPBIN /EXPORTS and spotted it as export #158. (Don't ask me why I've got any copies at all on my Windows 7 PC! - I expect some program installed it just in case.)

I've found an official (if outdated-looking) list now. So if GetCharABCWidthsFloatW ends up your only objection, I'll update the code to use a function from the official list. 

Thanks,

--Tom

bug#6364: [PATCH] Use GetCharABCWidthsFloatW if GetGlyphOutlineW fails.

[Eli Zaretskii]

> From: Tom Seddon <emacs@tomseddon.plus.com>
> Date: Tue, 26 Nov 2013 19:39:37 +0000
> Cc: 6364@debbugs.gnu.org
> 
> > Thanks, but can you please provide a reproducible test case, including
> > the font where the slow scrolling happens?  I don't see any fonts
> > named in these bug reports.
> 
> Certainly. I've uploaded some test cases to http://www.tomseddon.plus.com/.emacs/bug6364.tar.gz.

Thanks, I will use that and see what I get.

> > Are there other possibilities to fix this, without using
> > GetCharABCWidthsFloatW?  The problem with that function is that it is
> > not available on Windows 9X (in the unicows.dll library), so this
> > problem will be left unsolved on those systems.
> 
> I couldn't find a good list when I was writing the code so I just went by DUMPBIN /EXPORTS and spotted it as export #158. (Don't ask me why I've got any copies at all on my Windows 7 PC! - I expect some program installed it just in case.)

The export exists, but the function is a stub, not a real
implementation, AFAIK.

> I've found an official (if outdated-looking) list now. So if GetCharABCWidthsFloatW ends up your only objection, I'll update the code to use a function from the official list. 

Sorry, I don't understand what you mean.  Which official list did you
find, and what does it say about this?

bug#6364: [PATCH] Use GetCharABCWidthsFloatW if GetGlyphOutlineW fails.

[Tom Seddon]

On 26 Nov 2013, at 20:20, Eli Zaretskii <eliz@gnu.org> wrote:

>> I couldn't find a good list when I was writing the code so I just went by DUMPBIN /EXPORTS and spotted it as export #158. (Don't ask me why I've got any copies at all on my Windows 7 PC! - I expect some program installed it just in case.)
> 
> The export exists, but the function is a stub, not a real
> implementation, AFAIK.

>> I've found an official (if outdated-looking) list now. So if GetCharABCWidthsFloatW ends up your only objection, I'll update the code to use a function from the official list. 
> 
> Sorry, I don't understand what you mean.  Which official list did you
> find, and what does it say about this?

This list: http://msdn.microsoft.com/en-us/library/ms812845.aspx - it looks to be an outdated MSDN contents page, or something very much like, so I assume it's at lesat fairly authoritative. (I got the link here, which I found via google: http://microsoft.public.platformsdk.mslayerforunicode.narkive.com/QbsrNDDh/list-of-functions-handled-by-mslu)

And under G, GetCharABCWidthsFloat is indeed missing.

I suspect the page is a bit outdated, because some of the links are broken. (And also, of course, because it deal with Windows '9x...)

Thanks,

--Tom

bug#6364: [PATCH] Use GetCharABCWidthsFloatW if GetGlyphOutlineW fails.

[Eli Zaretskii]

> From: Tom Seddon <emacs@tomseddon.plus.com>
> Date: Tue, 26 Nov 2013 20:30:38 +0000
> Cc: 6364@debbugs.gnu.org
> 
> This list: http://msdn.microsoft.com/en-us/library/ms812845.aspx - it looks to be an outdated MSDN contents page, or something very much like, so I assume it's at lesat fairly authoritative. (I got the link here, which I found via google: http://microsoft.public.platformsdk.mslayerforunicode.narkive.com/QbsrNDDh/list-of-functions-handled-by-mslu)
> 
> And under G, GetCharABCWidthsFloat is indeed missing.
> 
> I suspect the page is a bit outdated, because some of the links are broken. (And also, of course, because it deal with Windows '9x...)

OK.  So what other function(s) can be used for this purpose?

If there are no good alternatives, I guess we will go with
GetCharABCWidthsFloatW anyway, since the situation cannot become worse
than it is already.

Btw, I used your recipe, but didn't see any significant slowdown with
fixed.fon (also, the file bigline.txt is missing, I just used the
16384 thingy instead.

bug#6364: [PATCH] Use GetCharABCWidthsFloatW if GetGlyphOutlineW fails.

[Tom Seddon]

On 26 Nov 2013, at 20:48, Eli Zaretskii <eliz@gnu.org> wrote:

> OK.  So what other function(s) can be used for this purpose?
> 
> If there are no good alternatives, I guess we will go with
> GetCharABCWidthsFloatW anyway, since the situation cannot become worse
> than it is already.

I've changed it to GetCharWidth32, which is in the list on that MSDN page - see patch below. I've checked this against all bitmap fonts on my system and it produces the same results (and emacs looks to behave the same, including in terms of performance).

This function only works for bitmap fonts, but hopefully if GetGlyphOutlineW ever fails, the reason is that the font is a bitmap one! (And if not, and GetCharWidth32 fails too, there's the same fallback as before in the form of GetTextExtentPoint32.)

> Btw, I used your recipe, but didn't see any significant slowdown with
> fixed.fon (also, the file bigline.txt is missing, I just used the
> 16384 thingy instead.

Agh, my mistake - I should have included start-slow.el, not start-bigline.el. Sorry. start-slow.el looks like this:

(set-face-attribute 'default nil :font "fixed")
(switch-to-buffer (find-file "usb.ids"))
(goto-char (point-max))

Maybe that will show up the problem? But it sounds rather like your computer just doesn't suffer from this issue, for whatever reason...

Thanks,

--Tom

P.S. my two PCs are as follows, and unpatched emacs performs poorly on both of them when using bitmap fonts:

- 3.4GHz Core i7 desktop, 16GB RAM, GeForce 560Ti, Windows 7
- 2.53GHz Core 2 Duo laptop, 8GB RAM, GeForce 9400M, Windows 7 (this is a Macbook Pro - the performance is also bad running in a VM)

So this could be something to do with NVidia display drivers on Windows 7, I suppose? Though I'd expect the VM to have its own drivers, suggesting that it might not actually be the graphics card vendor...

-----------------------------------

From 09fe3b0c0eb07c5ada87aee0864fa1d9478fdcf1 Mon Sep 17 00:00:00 2001
From: Tom Seddon <tom@tmbp-w7>
Date: Mon, 25 Nov 2013 22:19:47 +0000
Subject: [PATCH] Use GetCharWidth32W if GetGlyphOutlineW fails.

The previous fallback - which will still be used if GetCharWidth32W
fails - was to call GetTextExtentPoint32W. This can be rather slow,
mainly due to having to create a DC for it each time - but not to say
that using the cache isn't cheaper than GetTextExtentPoint32W anyway.
---
 src/w32font.c | 31 +++++++++++++++++++++++++++++++
 1 file changed, 31 insertions(+)

diff --git a/src/w32font.c b/src/w32font.c
index 5c5a15c..08316ec 100644
--- a/src/w32font.c
+++ b/src/w32font.c
@@ -149,6 +149,7 @@ static BOOL g_b_init_get_outline_metrics_w;
 static BOOL g_b_init_get_text_metrics_w;
 static BOOL g_b_init_get_glyph_outline_w;
 static BOOL g_b_init_get_glyph_outline_w;
+static BOOL g_b_init_get_char_width_32_w;
 
 typedef UINT (WINAPI * GetOutlineTextMetricsW_Proc) (
    HDC hdc,
@@ -165,6 +166,11 @@ typedef DWORD (WINAPI * GetGlyphOutlineW_Proc) (
    DWORD cbBuffer,
    LPVOID lpvBuffer,
    const MAT2 *lpmat2);
+typedef BOOL (WINAPI * GetCharWidth32W_Proc) (
+   HDC hdc,
+   UINT uFirstChar,
+   UINT uLastChar,
+   LPINT lpBuffer);
 
 /* Several "wide" functions we use to support the font backends are
    unavailable on Windows 9X, unless UNICOWS.DLL is installed (their
@@ -274,6 +280,23 @@ get_glyph_outline_w (HDC hdc, UINT uChar, UINT uFormat, LPGLYPHMETRICS lpgm,
 				   lpvBuffer, lpmat2);
 }
 
+static DWORD WINAPI get_char_width_32_w (HDC hdc, UINT uFirstChar,
+					 UINT uLastChar, LPINT lpBuffer)
+{
+  static GetCharWidth32W_Proc s_pfn_Get_Char_Width_32W = NULL;
+  HMODULE hm_unicows = NULL;
+  if (g_b_init_get_char_width_32_w == 0)
+    {
+      g_b_init_get_char_width_32_w = 1;
+      hm_unicows = w32_load_unicows_or_gdi32 ();
+      if (hm_unicows)
+	s_pfn_Get_Char_Width_32W = (GetCharWidth32W_Proc)
+	  GetProcAddress (hm_unicows, "GetCharWidth32W");
+    }
+  eassert (s_pfn_Get_Char_Width_32W != NULL);
+  return s_pfn_Get_Char_Width_32W (hdc, uFirstChar, uLastChar, lpBuffer);
+}
+
 static int
 memq_no_quit (Lisp_Object elt, Lisp_Object list)
 {
@@ -2438,6 +2461,7 @@ compute_metrics (HDC dc, struct w32font_info *w32_font, unsigned int code,
   GLYPHMETRICS gm;
   MAT2 transform;
   unsigned int options = GGO_METRICS;
+  INT width;
 
   if (w32_font->glyph_idx)
     options |= GGO_GLYPH_INDEX;
@@ -2454,6 +2478,13 @@ compute_metrics (HDC dc, struct w32font_info *w32_font, unsigned int code,
       metrics->width = gm.gmCellIncX;
       metrics->status = W32METRIC_SUCCESS;
     }
+  else if (get_char_width_32_w (dc, code, code, &width) != 0)
+    {
+      metrics->lbearing = 0;
+      metrics->rbearing = width;
+      metrics->width = width;
+      metrics->status = W32METRIC_SUCCESS;
+    }
   else
     metrics->status = W32METRIC_FAIL;
 }
-- 
1.8.1.msysgit.1

bug#6364: [PATCH] Use GetCharABCWidthsFloatW if GetGlyphOutlineW fails.

[Tom Seddon]

On 26 Nov 2013, at 21:50, Tom Seddon <emacs@tomseddon.plus.com> wrote:

> On 26 Nov 2013, at 20:48, Eli Zaretskii <eliz@gnu.org> wrote:
> 
>> OK.  So what other function(s) can be used for this purpose?
>> 
>> If there are no good alternatives, I guess we will go with
>> GetCharABCWidthsFloatW anyway, since the situation cannot become worse
>> than it is already.
> 
> I've changed it to GetCharWidth32, which is in the list on that MSDN page - see patch below. I've checked this against all bitmap fonts on my system and it produces the same results (and emacs looks to behave the same, including in terms of performance).

Just to be clear - this patch is against unchanged emacs-24.3, not against an emacs patched with my previous patch!

So please revert my last patch before applying it.

Thanks,

--Tom

bug#6364: [PATCH] Use GetCharABCWidthsFloatW if GetGlyphOutlineW fails.

[Eli Zaretskii]

> From: Tom Seddon <emacs@tomseddon.plus.com>
> Date: Tue, 26 Nov 2013 21:50:02 +0000
> Cc: 6364@debbugs.gnu.org
> 
> On 26 Nov 2013, at 20:48, Eli Zaretskii <eliz@gnu.org> wrote:
> 
> > OK.  So what other function(s) can be used for this purpose?
> > 
> > If there are no good alternatives, I guess we will go with
> > GetCharABCWidthsFloatW anyway, since the situation cannot become worse
> > than it is already.
> 
> I've changed it to GetCharWidth32, which is in the list on that MSDN page - see patch below. I've checked this against all bitmap fonts on my system and it produces the same results (and emacs looks to behave the same, including in terms of performance).

Thanks, I committed this in your name, with one change: you forgot to
initialize g_b_init_get_char_width_32_w in globals_of_w32font.

> > Btw, I used your recipe, but didn't see any significant slowdown with
> > fixed.fon (also, the file bigline.txt is missing, I just used the
> > 16384 thingy instead.
> 
> Agh, my mistake - I should have included start-slow.el, not start-bigline.el. Sorry. start-slow.el looks like this:
> 
> (set-face-attribute 'default nil :font "fixed")
> (switch-to-buffer (find-file "usb.ids"))
> (goto-char (point-max))
> 
> Maybe that will show up the problem? But it sounds rather like your computer just doesn't suffer from this issue, for whatever reason...

I see the CPU usage decrease by half if I use Emacs with your patch
with a bitmap font, so I guess the effect is visible on my system as
well.

Thanks.  I'm closing this bug.