Display slowness that is painful

Display slowness that is painful

[Richard M. Stallman]

I just displayed a large binary buffer which had many chars in the range
0200 to 0377.  And the region was large, too.  This led to a very long loop
in which I saw the following on the stack

    try_scrolling, Fwindow_end,
    start_display...get_next_display_element...handle_stop...handle_face_prop,
    face_at_buffer_position, merge_named_face (on Qregion)

... means I omitted some intervening function calls.
I did not have a way to actually save the backtrace, so I typed
in enough that I think people can see what the issue is.

It seems that this case badly needs to be optimized.
Can someone please work on it, then ack?

I can make the test case available if necessary.
It is a 1.3 meg GIF file.

Display slowness that is painful

[Richard Stallman]

[I sent this message a week ago but did not get a response.
Would someone please look at this problem?]

I just displayed a large binary buffer which had many chars in the range
0200 to 0377.  And the region was large, too.  This led to a very long loop
in which I saw the following on the stack

    try_scrolling, Fwindow_end,
    start_display...get_next_display_element...handle_stop...handle_face_prop,
    face_at_buffer_position, merge_named_face (on Qregion)

... means I omitted some intervening function calls.
I did not have a way to actually save the backtrace, so I typed
in enough that I think people can see what the issue is.

It seems that this case badly needs to be optimized.
Can someone please work on it, then ack?

I can make the test case available if necessary.
It is a 1.3 meg GIF file.

Display slowness that is painful

[Richard Stallman]

[I sent this message twice but did not get a response.
Would someone please look at this problem?]

I just displayed a large binary buffer which had many chars in the range
0200 to 0377.  And the region was large, too.  This led to a very long loop
in which I saw the following on the stack

    try_scrolling, Fwindow_end,
    start_display...get_next_display_element...handle_stop...handle_face_prop,
    face_at_buffer_position, merge_named_face (on Qregion)

... means I omitted some intervening function calls.
I did not have a way to actually save the backtrace, so I typed
in enough that I think people can see what the issue is.

It seems that this case badly needs to be optimized.
Can someone please work on it, then ack?

I can make the test case available if necessary.
It is a 1.3 meg GIF file.

Re: Display slowness that is painful

[Evil Boris]

Richard Stallman <rms@gnu.org> writes:

> I just displayed a large binary buffer which had many chars in the range
> 0200 to 0377.  And the region was large, too.  This led to a very long loop
> in which I saw the following on the stack
>
>     try_scrolling, Fwindow_end,
>     start_display...get_next_display_element...handle_stop...handle_face_prop,
>     face_at_buffer_position, merge_named_face (on Qregion)
>
[...]
> I can make the test case available if necessary.
> It is a 1.3 meg GIF file.

I seem to have run into a similar problem.  In my case, it's an .EPS
file produced by Adobe Photoshop (about 200kb), full of characters in
that range too (some sort of binary data, plus maybe a preview).
Turning font-lock off does not help.  Switching to fundamental-mode
does not help.  Symptoms: The file takes a long time to load.
Executing M-> took 10-5 mins (!!!) on my machine.  Cursor motions w/o
scrolling when none of the funny characters are visible is near
instantaneous.  When some of the characters are visible on the top of
the screen (top one line), moving the cursor up and down takes a
second or two.  Doing large motions with much of the screen occupied
by funny characters sends Emacs into CPU-time eating look. 100% CPU
utilization for minutes at a time.  First time I ran into this I
thought Emacs hung altogether.

Not sure if it helps.  What else can I check?  I would rather not send
the file, though.

    --Boris

Re: Display slowness that is painful

[Chong Yidong]

How about this fix?

*** emacs/lisp/image-mode.el.~1.8.~	2006-01-30 22:50:00.000000000 -0500
--- emacs/lisp/image-mode.el	2006-01-31 00:08:56.000000000 -0500
***************
*** 58,63 ****
--- 58,65 ----
    (kill-all-local-variables)
    (setq mode-name "Image")
    (setq major-mode 'image-mode)
+   (set (make-local-variable 'unibyte-display-via-language-environment)
+        nil)
    (use-local-map image-mode-map)
    (add-hook 'change-major-mode-hook 'image-toggle-display-text nil t)
    (if (not (get-text-property (point-min) 'display))

Re: Display slowness that is painful

[Richard M. Stallman]

A change in image-mode can't fix this problem.  I did not visit that
file; I produced it in a buffer with base64-decode.  It was not in
Image mode.

The change in Image mode might avoid the problem in many cases, and it
might be correct, but it doesn't fix this bug.  The bug is at the
level of redisplay, and it has to be fixed there.

Re: Display slowness that is painful

[Chong Yidong]

"Richard M. Stallman" <rms@gnu.org> writes:

> A change in image-mode can't fix this problem.  I did not visit that
> file; I produced it in a buffer with base64-decode.  It was not in
> Image mode.
>
> The change in Image mode might avoid the problem in many cases, and it
> might be correct, but it doesn't fix this bug.  The bug is at the
> level of redisplay, and it has to be fixed there.

This is *really* not the time to make changes in redisplay.  I think
the problem here is simply that if you want to display characters as
\XXX, the redisplay iterator has to go through four times as many
glyphs.

How about simply making unibyte_display_via_language_environment
default to t?  That's basically the situation in Emacs 21, and it
makes display responsive again.

Re: Display slowness that is painful

[Stefan Monnier]

> This is *really* not the time to make changes in redisplay.  I think
> the problem here is simply that if you want to display characters as
> \XXX, the redisplay iterator has to go through four times as many
> glyphs.

Huh?  Four times more glyphs per char/byte, yes.
But four times fewer chars/bytes get displayed, so it should be a wash.

In utf-8 environments, unibyte_display_via_language_environment doesn't make
much sense, and around here utf-8 environments are becoming the norm.


        Stefan

Re: Display slowness that is painful

[Chong Yidong]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> This is *really* not the time to make changes in redisplay.  I think
>> the problem here is simply that if you want to display characters as
>> \XXX, the redisplay iterator has to go through four times as many
>> glyphs.
>
> Huh?  Four times more glyphs per char/byte, yes.
> But four times fewer chars/bytes get displayed, so it should be a wash.
>
> In utf-8 environments, unibyte_display_via_language_environment doesn't make
> much sense, and around here utf-8 environments are becoming the norm.

I'm confused.  How does it make sense to display characters in \XXX
format instead?  Both methods produce gibberish (as one would expect
of a binary file).  So we might as well choose the gibberish that
redisplays faster.

Re: Display slowness that is painful

[Miles Bader]

Chong Yidong <cyd@stupidchicken.com> writes:
> I'm confused.  How does it make sense to display characters in \XXX
> format instead?  Both methods produce gibberish (as one would expect
> of a binary file).  So we might as well choose the gibberish that
> redisplays faster.

Because gibberish with a big "Gibberish" label (which is sort of what
the octal syntax is) is a lot more presentable, and looks less like
Emacs just fucked something up.

-miles
-- 
"Suppose we've chosen the wrong god. Every time we go to church we're
just making him madder and madder." -- Homer Simpson

Re: Display slowness that is painful

[Andreas Schwab]

Chong Yidong <cyd@stupidchicken.com> writes:

> How about simply making unibyte_display_via_language_environment
> default to t?

It completely messes up display in termcap frames.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Display slowness that is painful

[Andreas Schwab]

Richard Stallman <rms@gnu.org> writes:

> I can make the test case available if necessary.
> It is a 1.3 meg GIF file.

When I visit a 13MB binary file (eg. the Emacs binary) I can hardly notice
a slowdown here.  Only scrolling backwards when inside a very long line is
rather slow, but that's probably due to requiring to rerender the whole
line each time to be able to find the correct window start.  There is no
such problem when scrolling forward.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Display slowness that is painful

[Chong Yidong]

Andreas Schwab <schwab@suse.de> writes:

> Richard Stallman <rms@gnu.org> writes:
>
>> I can make the test case available if necessary.
>> It is a 1.3 meg GIF file.
>
> When I visit a 13MB binary file (eg. the Emacs binary) I can hardly notice
> a slowdown here.  Only scrolling backwards when inside a very long line is
> rather slow, but that's probably due to requiring to rerender the whole
> line each time to be able to find the correct window start.  There is no
> such problem when scrolling forward.

I've put Richard's "gif" file (it's actually a bmp file) on

http://web.mit.edu/cyd/www/bug.bmp

There is indeed a slowdown when visiting this file.

Re: Display slowness that is painful

[Richard M. Stallman]

    I'm confused.  How does it make sense to display characters in \XXX
    format instead?  Both methods produce gibberish (as one would expect
    of a binary file).  So we might as well choose the gibberish that
    redisplays faster.

Making it 4 times as fast is not enough.  It will still be
intolerable.  This ought to be hundreds of times as fast, and I am
sure it can be, with a fairly small change--once someone sees what
is taking the time.

This is a bug.  Please help fix it.

Re: Display slowness that is painful

[Richard M. Stallman]

    When I visit a 13MB binary file (eg. the Emacs binary) I can hardly notice
    a slowdown here.

I felicitate you on having found a case in which the problem
does not occur.  Unfortunately, there are also cases where it does.

Re: Display slowness that is painful

[Richard M. Stallman]

    This is *really* not the time to make changes in redisplay.

Yes it is.  Even if we had started the pretest already,
we would fix this bug.  Or what are pretests for?

Re: Display slowness that is painful

[Chong Yidong]

"Richard M. Stallman" <rms@gnu.org> writes:

>     I'm confused.  How does it make sense to display characters in \XXX
>     format instead?  Both methods produce gibberish (as one would expect
>     of a binary file).  So we might as well choose the gibberish that
>     redisplays faster.
>
> Making it 4 times as fast is not enough.  It will still be
> intolerable.  This ought to be hundreds of times as fast, and I am
> sure it can be, with a fairly small change--once someone sees what
> is taking the time.

Actually, displaying unibyte text according to the language
environment is faster than displaying it in octal format -- in the
case of the bmp file you provided, the delay is negligible.

Re: Display slowness that is painful

[Chong Yidong]

"Richard M. Stallman" <rms@gnu.org> writes:

>     This is *really* not the time to make changes in redisplay.
>
> Yes it is.  Even if we had started the pretest already,
> we would fix this bug.  Or what are pretests for?

Managers of successful software projects often make the conscious
choice to postphone resolving bugs whose "fixes" can introduce even
more serious bugs while conferring limited benefits.  (And they seem
to be able to actually make releases...)  If a workaround can be found
that avoids this problem without changing redisplay code, I think it
is preferable.

In this particular case, having looked through the relevant redisplay
code, I don't see any apparent code stupidity going on.  It's simply
the case that, in this file, the first "newline" takes place 340,000
characters in.  The normally negligible delay from displaying glyphs
in octal format (v.s. unibyte-display-via-language-environment) is
magnified by this amount.  The redisplay engine has to scroll through
the entire "line" to check for overlays, text properties, etc., and
this seems to be unavoidable.

Of course, if you can point out a specific redisplay optimization that
we're missing, I'm all ears.

Re: Display slowness that is painful

[Chong Yidong]

Miles Bader <miles.bader@necel.com> writes:

> Chong Yidong <cyd@stupidchicken.com> writes:
>> I'm confused.  How does it make sense to display characters in \XXX
>> format instead?  Both methods produce gibberish (as one would expect
>> of a binary file).  So we might as well choose the gibberish that
>> redisplays faster.
>
> Because gibberish with a big "Gibberish" label (which is sort of what
> the octal syntax is) is a lot more presentable, and looks less like
> Emacs just fucked something up.

Maybe we should look for something that displays faster than the octal
representation, but is "clearly" gibberish.  Like a single "empty box"
glyph, for example.

Apart from binary files, are unibyte buffers used for anything?  Is
there any situation in which characters above 127, occuring in unibyte
buffers, have any user-discernable meaning?

Re: Display slowness that is painful

[Chong Yidong]

Chong Yidong <cyd@stupidchicken.com> writes:

> "Richard M. Stallman" <rms@gnu.org> writes:
>
>>     This is *really* not the time to make changes in redisplay.
>>
>> Yes it is.  Even if we had started the pretest already,
>> we would fix this bug.  Or what are pretests for?
>
> Managers of successful software projects often make the conscious
> choice to postphone resolving bugs whose "fixes" can introduce even
> more serious bugs while conferring limited benefits.  (And they seem
> to be able to actually make releases...)  If a workaround can be found
> that avoids this problem without changing redisplay code, I think it
> is preferable.

Lest there be any misunderstanding, I appreciate the job you're doing
as the Emacs maintainer, and the fact that you're trying to maintain a
high level of quality for Emacs.  But it's extremely frustrating to
participate in a perpetual-motion-machine-like release process, with a
leaky to non-existent feature-freeze, that never seems to get any
closer to the actual release (or even the pretest!)

Re: Display slowness that is painful

[David Kastrup]

Chong Yidong <cyd@stupidchicken.com> writes:

> Lest there be any misunderstanding, I appreciate the job you're
> doing as the Emacs maintainer, and the fact that you're trying to
> maintain a high level of quality for Emacs.  But it's extremely
> frustrating to participate in a perpetual-motion-machine-like
> release process, with a leaky to non-existent feature-freeze, that
> never seems to get any closer to the actual release (or even the
> pretest!)

Not fixing bugs in core functionality is not the way to get release
quality.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: Display slowness that is painful

[Andreas Schwab]

"Richard M. Stallman" <rms@gnu.org> writes:

>     When I visit a 13MB binary file (eg. the Emacs binary) I can hardly notice
>     a slowdown here.
>
> I felicitate you on having found a case in which the problem
> does not occur.  Unfortunately, there are also cases where it does.

I was able to reproduce it with your test case, the key difference appears
to be the extraordinary long lines.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Display slowness that is painful

[Kim F. Storm]

Chong Yidong <cyd@stupidchicken.com> writes:

> "Richard M. Stallman" <rms@gnu.org> writes:
>
>>     This is *really* not the time to make changes in redisplay.
>>
>> Yes it is.  Even if we had started the pretest already,
>> we would fix this bug.  Or what are pretests for?

I don't know -- it's been so long since I participated in one that I
have forgot :-)

> Managers of successful software projects often make the conscious
> choice to postphone resolving bugs whose "fixes" can introduce even
> more serious bugs while conferring limited benefits.  (And they seem
> to be able to actually make releases...)

Yes, that is what "management" is all about -- making decisions when
"enough is enough".

> In this particular case, having looked through the relevant redisplay
> code, I don't see any apparent code stupidity going on.  It's simply
> the case that, in this file, the first "newline" takes place 340,000
> characters in.

Good catch!!

That can definitely cause problems for redisplay as it does search for
line beginnings/ends quite a lot, but usually the penalty is neglible.

>                 The normally negligible delay from displaying glyphs
> in octal format (v.s. unibyte-display-via-language-environment) is
> magnified by this amount.  The redisplay engine has to scroll through
> the entire "line" to check for overlays, text properties, etc., and
> this seems to be unavoidable.

I tried to create two files like this:

emacs -q

C-x C-f file1.txt RET RET C-p ESC 3 4 0 0 0 0 x

C-x C-f file2.txt RET RET C-p ESC 3 4 0 0 0 0 C-q 2 1 2 RET

and indeed doing C-p C-n (or C-a C-e) in the second buffer is
significantly slower than in the first buffer.

Ok, one might expect it to be 4 times slower, but it is (feels) much
slower than that (perhaps 8 times slower).

It seems to be because it needs to merge_face with the escape glyph
for each character -- and that takes extra time.  Below is a patch to
speed up the processing for this specific part of the problem.  With
the patch, redisplay time for file2 seems to be approx 4 x redisplay time
for file1.



*** xdisp.c	24 Jan 2006 09:30:15 +0100	1.1073
--- xdisp.c	02 Feb 2006 11:33:04 +0100	
***************
*** 5329,5334 ****
--- 5329,5338 ----
     display element from the current position of IT.  Value is zero if
     end of buffer (or C string) is reached.  */
  
+ static struct frame *last_escape_glyph_frame = NULL;
+ static unsigned last_escape_glyph_face_id = (1 << FACE_ID_BITS);
+ static int last_escape_glyph_merged_face_id = 0;
+ 
  int
  get_next_display_element (it)
       struct it *it;
***************
*** 5445,5455 ****
--- 5449,5467 ----
  		       face_id = merge_faces (it->f, Qt, lface_id,
  					      it->face_id);
  		    }
+ 		  else if (it->f == last_escape_glyph_frame
+ 			   && it->face_id == last_escape_glyph_face_id)
+ 		    {
+ 		      face_id = last_escape_glyph_merged_face_id;
+ 		    }
  		  else
  		    {
  		      /* Merge the escape-glyph face into the current face.  */
  		      face_id = merge_faces (it->f, Qescape_glyph, 0,
  					     it->face_id);
+ 		      last_escape_glyph_frame = it->f;
+ 		      last_escape_glyph_face_id = it->face_id;
+ 		      last_escape_glyph_merged_face_id = face_id;
  		    }
  
  		  XSETINT (it->ctl_chars[0], g);
***************
*** 5496,5506 ****
--- 5508,5526 ----
  		  face_id = merge_faces (it->f, Qt, lface_id,
  					 it->face_id);
  		}
+ 	      else if (it->f == last_escape_glyph_frame
+ 		       && it->face_id == last_escape_glyph_face_id)
+ 		{
+ 		  face_id = last_escape_glyph_merged_face_id;
+ 		}
  	      else
  		{
  		  /* Merge the escape-glyph face into the current face.  */
  		  face_id = merge_faces (it->f, Qescape_glyph, 0,
  					 it->face_id);
+ 		  last_escape_glyph_frame = it->f;
+ 		  last_escape_glyph_face_id = it->face_id;
+ 		  last_escape_glyph_merged_face_id = face_id;
  		}
  
  	      /* Handle soft hyphens in the mode where they only get
***************
*** 10545,10550 ****
--- 10565,10572 ----
   retry:
    pause = 0;
    reconsider_clip_changes (w, current_buffer);
+   last_escape_glyph_frame = NULL;
+   last_escape_glyph_face_id = (1 << FACE_ID_BITS);
  
    /* If new fonts have been loaded that make a glyph matrix adjustment
       necessary, do it.  */



--
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: Display slowness that is painful

[Stefan Monnier]

> In this particular case, having looked through the relevant redisplay
> code, I don't see any apparent code stupidity going on.  It's simply
> the case that, in this file, the first "newline" takes place 340,000
> characters in.  The normally negligible delay from displaying glyphs
> in octal format (v.s. unibyte-display-via-language-environment) is
> magnified by this amount.

Oh, I see, so the "4 times more work per char" is actually not compensated
by the "4 times fewer chars" since these are not displayed anyway.

So it seems to be a plain and simple algorithmic "bug" (linear rather than
constant time complexity), which has been around for a long time already.

Given this, I agree that it doesn't have to be fixed before the release
(tho of course if someone can come up with a quick fix, that's great).


        Stefan

Re: Display slowness that is painful

[Stefan Monnier]

> Not fixing bugs in core functionality is not the way to get release
> quality.

Insisting on fixing algorithmic performance bugs, even knowing that there
are many more of the same kind, will make sure that you don't even get any
release at all (if you want, I'll construct a useful piece of code that uses
enough overlays to bring the redisplay to its knees, and then I'll let you
try and come up with a better implementation of overlays).


        Stefan

Re: Display slowness that is painful

[Kenichi Handa]

In article <m3d5i6m1vd.fsf@kfs-l.imdomain.dk>, storm@cua.dk (Kim F. Storm) writes:

> It seems to be because it needs to merge_face with the escape glyph
> for each character -- and that takes extra time.  Below is a patch to
> speed up the processing for this specific part of the problem.  With
> the patch, redisplay time for file2 seems to be approx 4 x redisplay time
> for file1.

I think another reason of the slowness is this part (in
set_iterator_to_next):

    case GET_FROM_DISPLAY_VECTOR:
[...]
      /* Restore face of the iterator to what they were before the
         display vector entry (these entries may contain faces).  */
      it->face_id = it->saved_face_id;

      if (it->dpvec + it->current.dpvec_index == it->dpend)
	{
[...]
	  /* Recheck faces after display vector */
	  it->stop_charpos = IT_CHARPOS (*it);
	}
      break;

Because of this, it seems that handle_stop is called for
every character after an eight bit character.  Isn't it
possible to tune it?  First of all, why should we recheck
faces after display vector?

---
Kenichi Handa
handa@m17n.org

Re: Display slowness that is painful

[Kenichi Handa]

In article <jek6cfmka8.fsf@sykes.suse.de>, Andreas Schwab <schwab@suse.de> writes:

> Chong Yidong <cyd@stupidchicken.com> writes:
>> How about simply making unibyte_display_via_language_environment
>> default to t?

> It completely messes up display in termcap frames.

Really?  If unibyte_display_via_language_environment is t,
each eight-bit character is converted to a multibyte
character according to your current locale, and then is
writen out after being encoded again according to your
current locale.  If some multibyte character can't be
encoded by your locale coding system, just SPC is produced.
So, as far as your terminal is capable of your locale, the
display should not be messed up.

---
Kenichi Handa
handa@m17n.org

Re: Display slowness that is painful

[Richard M. Stallman]

Thanks for writing that optimization.  I figured some such simple fix
would be possible.  Please install your fix.

What my experience has taught me is that cases of anomalous gross
slowness often have a simple fix.  Thus, before taking up the
difficult question of whether major surgery is justified, it's a good
idea to look for a fix like this one which does the job with a band-aid.

(Yi-dong found a different way to apply a band-aid, but it assumes
that the file was visited in Image mode, so it doesn't solve the whole
problem.  This change does, I think.)

Re: Display slowness that is painful

[Kim F. Storm]

Kenichi Handa <handa@m17n.org> writes:

> In article <m3d5i6m1vd.fsf@kfs-l.imdomain.dk>, storm@cua.dk (Kim F. Storm) writes:
>
> I think another reason of the slowness is this part (in
> set_iterator_to_next):
>
>     case GET_FROM_DISPLAY_VECTOR:
> [...]
>       /* Restore face of the iterator to what they were before the
>          display vector entry (these entries may contain faces).  */
>       it->face_id = it->saved_face_id;
>
>       if (it->dpvec + it->current.dpvec_index == it->dpend)
> 	{
> [...]
> 	  /* Recheck faces after display vector */
> 	  it->stop_charpos = IT_CHARPOS (*it);
> 	}
>       break;
>
> Because of this, it seems that handle_stop is called for
> every character after an eight bit character.  Isn't it
> possible to tune it?  First of all, why should we recheck
> faces after display vector?

2004-12-13  Kim F. Storm  <storm@cua.dk>

	* xdisp.c (set_iterator_to_next): Reset stop_charpos after display
	vector.

It was made to fix the following bug:

> From: 	Stefan Monnier
> Subject: 	Incorrect faces on display
> Date: 	Thu, 09 Dec 2004 17:15:31 -0500
> User-agent: 	Gnus/5.11 (Gnus v5.11) Emacs/21.3.50 (gnu/linux)
>
> I started seeing recently problems where elisp code gets displayed with the
> font-lock-comment-face even though it's not a comment.
>
> This always shows up right after an invisible piece of text (I use
> outline-minor-mode in elisp).
>
> I first thought it was some funny bug in font-lock or jit-lock, but a quick
> C-u C-x = shows that the text-properties are correct: the problem is in
> the redisplay.
>
> I finally came up with a recipe to reproduce it:
>
>    emacs -Q -f global-font-lock-mode -f outline-minor-mode
>
>    (; RET a RET (foobar C-u 1000 M-x hide-sublevels
>
> If all goes well, you'll see that the second line displays "(foobar" using
> font-lock-comment-face.


The following addition to my previous patch changes this to only
recheck after ellipsis.  It does indeed speed up redisplay even
more.

I cannot say whether this will break other redisplay cases though:



*** xdisp.c	24 Jan 2006 09:30:15 +0100	1.1073
--- xdisp.c	03 Feb 2006 10:28:21 +0100	
***************
*** 5694,5699 ****
--- 5714,5721 ----
  
        if (it->dpvec + it->current.dpvec_index == it->dpend)
  	{
+ 	  int recheck_faces = it->ellipsis_p;
+ 
  	  if (it->s)
  	    it->method = GET_FROM_C_STRING;
  	  else if (STRINGP (it->string))
***************
*** 5716,5723 ****
  	      set_iterator_to_next (it, reseat_p);
  	    }
  
! 	  /* Recheck faces after display vector */
! 	  it->stop_charpos = IT_CHARPOS (*it);
  	}
        break;
  
--- 5738,5746 ----
  	      set_iterator_to_next (it, reseat_p);
  	    }
  
! 	  /* Maybe recheck faces after display vector */
! 	  if (recheck_faces)
! 	    it->stop_charpos = IT_CHARPOS (*it);
  	}
        break;
  


--
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: Display slowness that is painful

[Kim F. Storm]

"Richard M. Stallman" <rms@gnu.org> writes:

> Thanks for writing that optimization.  I figured some such simple fix
> would be possible.  Please install your fix.
>
> What my experience has taught me is that cases of anomalous gross
> slowness often have a simple fix.  Thus, before taking up the
> difficult question of whether major surgery is justified, it's a good
> idea to look for a fix like this one which does the job with a band-aid.
>
> (Yi-dong found a different way to apply a band-aid, but it assumes
> that the file was visited in Image mode, so it doesn't solve the whole
> problem.  This change does, I think.)


While debugging this, I noticed that sometimes redisplay was (quite) fast,
while at other times, it was VERY slow.

I traced this to emacs starting to process SELECTION_REQUEST_EVENT
events after each redisplay round -- and consequently doing another
redisplay...  So in effect it never seemed to reach a point where it
would actually update the display.

Doing M-: (setq redisplay-dont-pause t) RET made redisplay much
snappier in this specific case.

Perhaps we should have some form of timer associated with the
(default) redisplay-dont-pause == nil case, so that even if
input_pending, we will still update the display if the last completed
redisplay happened more than, say 2 seconds ago.

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: Display slowness that is painful

[Andreas Schwab]

Kenichi Handa <handa@m17n.org> writes:

> In article <jek6cfmka8.fsf@sykes.suse.de>, Andreas Schwab <schwab@suse.de> writes:
>
>> Chong Yidong <cyd@stupidchicken.com> writes:
>>> How about simply making unibyte_display_via_language_environment
>>> default to t?
>
>> It completely messes up display in termcap frames.
>
> Really?

Yes.  All unibyte characters (except control characters) are written out
verbatim.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Display slowness that is painful

[Kenichi Handa]

In article <jeoe1ozr1l.fsf@sykes.suse.de>, Andreas Schwab <schwab@suse.de> writes:

> Yes.  All unibyte characters (except control characters) are written out
> verbatim.

What locale are you in?

---
Kenichi Handa
handa@m17n.org

Re: Display slowness that is painful

[Andreas Schwab]

Kenichi Handa <handa@m17n.org> writes:

> In article <jeoe1ozr1l.fsf@sykes.suse.de>, Andreas Schwab <schwab@suse.de> writes:
>
>> Yes.  All unibyte characters (except control characters) are written out
>> verbatim.
>
> What locale are you in?

$ locale
LANG=de_DE.UTF-8
LC_CTYPE="de_DE.UTF-8"
LC_NUMERIC=en_US.UTF-8
LC_TIME="de_DE.UTF-8"
LC_COLLATE=POSIX
LC_MONETARY="de_DE.UTF-8"
LC_MESSAGES=en_US.UTF-8
LC_PAPER="de_DE.UTF-8"
LC_NAME="de_DE.UTF-8"
LC_ADDRESS="de_DE.UTF-8"
LC_TELEPHONE="de_DE.UTF-8"
LC_MEASUREMENT="de_DE.UTF-8"
LC_IDENTIFICATION="de_DE.UTF-8"
LC_ALL=

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Display slowness that is painful

[Eli Zaretskii]

> From: Andreas Schwab <schwab@suse.de>
> Date: Fri, 03 Feb 2006 11:15:50 +0100
> Cc: cyd@stupidchicken.com, rms@gnu.org, emacs-devel@gnu.org
> 
> Kenichi Handa <handa@m17n.org> writes:
> 
> > In article <jek6cfmka8.fsf@sykes.suse.de>, Andreas Schwab <schwab@suse.de> writes:
> >
> >> Chong Yidong <cyd@stupidchicken.com> writes:
> >>> How about simply making unibyte_display_via_language_environment
> >>> default to t?
> >
> >> It completely messes up display in termcap frames.
> >
> > Really?
> 
> Yes.  All unibyte characters (except control characters) are written out
> verbatim.

Then what is it that you think unibyte-display-via-language-environment
actually does?  Unibyte characters are written verbatim when this
option is set to nil, not to t.  When it's non-nil, unibyte characters
are converted to multibyte and then written as if they were multibyte.
Or at least that's the theory was last time I looked.

The doc string still says:

    unibyte-display-via-language-environment's value is nil

    *Non-nil means display unibyte text according to language environment.
    Specifically this means that unibyte non-ASCII characters
    are displayed by converting them to the equivalent multibyte characters
    according to the current language environment.  As a result, they are
    displayed according to the current fontset.

Is that a lie?

Re: Display slowness that is painful

[Stefan Monnier]

> Perhaps we should have some form of timer associated with the
> (default) redisplay-dont-pause == nil case, so that even if
> input_pending, we will still update the display if the last completed
> redisplay happened more than, say 2 seconds ago.

We should probably do the usual "competitive analysis": keep track of the
amount of time thrown away because redisplay got interrupted, and when the
sum gets higher than the amount of time it would take to do a full
redisplay, then force a full redisplay.

This way we can guarantee we're within a factor 2 of the optimal setting for
redisplay-dont-pause.


        Stefan


PS: Of course, we maybe don't know how much time a full redisplay would take
and we probably don't want to keep track of actual time anyway, only some
estimate of "amount of work".

Re: Display slowness that is painful

[Andreas Schwab]

Eli Zaretskii <eliz@gnu.org> writes:

> The doc string still says:
>
>     unibyte-display-via-language-environment's value is nil
>
>     *Non-nil means display unibyte text according to language environment.
>     Specifically this means that unibyte non-ASCII characters
>     are displayed by converting them to the equivalent multibyte characters
>     according to the current language environment.  As a result, they are
>     displayed according to the current fontset.
>
> Is that a lie?

Apparently the conversion to multibyte is broken in some way.  In the
normal case, when unibyte-display-via-language-environment is nil, no
non-ascii character are ever used for display, so it works as expected.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Display slowness that is painful

[Richard M. Stallman]

    Insisting on fixing algorithmic performance bugs, even knowing that there
    are many more of the same kind, will make sure that you don't even get any
    release at all (if you want, I'll construct a useful piece of code that uses
    enough overlays to bring the redisplay to its knees, and then I'll let you
    try and come up with a better implementation of overlays).

I did not "construct" code to produce this bug.  It happened to me
during my normal mail-reading procedure.  We don't have to look
for trouble, but we need to fix the problems that really happen.

Meanwhile, wasn't this one easy?

Re: Display slowness that is painful

[Richard M. Stallman]

    I think another reason of the slowness is this part (in
    set_iterator_to_next):

    ...

    Because of this, it seems that handle_stop is called for
    every character after an eight bit character.  Isn't it
    possible to tune it?  First of all, why should we recheck
    faces after display vector?

Kim, did your changes address this issue?

I am sure there was a good reason to recheck the faces, in general.
But there may be specific cases--perhaps this case--where it can
be skipped.

Re: Display slowness that is painful

[Richard M. Stallman]

    I traced this to emacs starting to process SELECTION_REQUEST_EVENT
    events after each redisplay round -- and consequently doing another
    redisplay...

Why did Emacs get a SELECTION_REQUEST_EVENT at all?
Normally that should happen only if you try to paste
into another app some text that you selected in Emacs.
Did you do that?

When people do that, normally Emacs is idle, not redisplaying
anything.  Of course, it could be running some sort of program
that would keep on redisplaying.  Was that the situation?

One other question is, why does Emacs redisplay after handling a
SELECTION_REQUEST_EVENT?  Is it because this arrives during redisplay
and pre-empts it? 

Are you saying that Emacs receives a series of SELECTION_REQUEST_EVENTs,
pre-empting redisplay over and over?  If so, why do they arrive?

Re: Display slowness that is painful

[Kim F. Storm]

"Richard M. Stallman" <rms@gnu.org> writes:

>     I traced this to emacs starting to process SELECTION_REQUEST_EVENT
>     events after each redisplay round -- and consequently doing another
>     redisplay...
>
> Why did Emacs get a SELECTION_REQUEST_EVENT at all?
> Normally that should happen only if you try to paste
> into another app some text that you selected in Emacs.
> Did you do that?

No.  When emacs has the selection, it is polled with
SELECTION_REQUEST_EVENTs every seconds by the version of klipper (X
clipboard) that I'm using (this is with KDE on Redhat 9.0).

This version of klipper is buggy according to PROBLEMS:

*** KDE: Emacs hangs on KDE when a large portion of text is killed.

This is caused by a bug in the KDE applet `klipper' which periodically
requests the X clipboard contents from applications.  Early versions
of klipper don't implement the ICCCM protocol for large selections,
which leads to Emacs being flooded with selection requests.  After a
while, Emacs may print a message:

  Timed out waiting for property-notify event

A workaround is to not use `klipper'.  An upgrade to the `klipper' that
comes with KDE 3.3 or later also solves the problem.

>
> When people do that, normally Emacs is idle, not redisplaying
> anything.  Of course, it could be running some sort of program
> that would keep on redisplaying.  Was that the situation?

No.

>
> One other question is, why does Emacs redisplay after handling a
> SELECTION_REQUEST_EVENT?  Is it because this arrives during redisplay
> and pre-empts it? 

It seems so.

>
> Are you saying that Emacs receives a series of SELECTION_REQUEST_EVENTs,
> pre-empting redisplay over and over?  

Yes.  So since redisplay in this case takes longer than 1 second,
redisplay seems to be stalled for a long time -- usually it completes
when I change focus (or move selection) to some other application.

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: Display slowness that is painful

[Kim F. Storm]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> Perhaps we should have some form of timer associated with the
>> (default) redisplay-dont-pause == nil case, so that even if
>> input_pending, we will still update the display if the last completed
>> redisplay happened more than, say 2 seconds ago.
>
> We should probably do the usual "competitive analysis": keep track of the
> amount of time thrown away because redisplay got interrupted, and when the
> sum gets higher than the amount of time it would take to do a full
> redisplay, then force a full redisplay.

This sounds way too complex for my taste, and it doesn't give
predictable results, i.e. you still don't really know when 
redisplay will/should actually complete.

This is a matter of user perception ... if update takes too long to
complete, emacs feels sloppy/slow.  If we guarantee that redisplay
will complete at least every 2 or 5 seconds, that is predictable
behaviour.
>
> This way we can guarantee we're within a factor 2 of the optimal setting for
> redisplay-dont-pause.
>
>
>         Stefan
>
>
> PS: Of course, we maybe don't know how much time a full redisplay would take
> and we probably don't want to keep track of actual time anyway, only some
> estimate of "amount of work".

I don't know how to do that.

OTOH, I _do_ know how to compare time(&start_of_redisplay) and time(&now)
when testing redisplay_dont_pause.
 
-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: Display slowness that is painful

[Miles Bader]

2006/2/5, Richard M. Stallman <rms@gnu.org>:
> Why did Emacs get a SELECTION_REQUEST_EVENT at all?
> Normally that should happen only if you try to paste
> into another app some text that you selected in Emacs.
> Did you do that?

Wasn't there previously a discussion on this list about about a
mis-behaving KDE cut-ring-manager type program (klipper?) that went
nuts requesting the selection over and over again?  Maybe that's
related some how.

-Miles
--
Do not taunt Happy Fun Ball.

Re: Display slowness that is painful

[Kim F. Storm]

Miles Bader <miles@gnu.org> writes:

> 2006/2/5, Richard M. Stallman <rms@gnu.org>:
>> Why did Emacs get a SELECTION_REQUEST_EVENT at all?
>> Normally that should happen only if you try to paste
>> into another app some text that you selected in Emacs.
>> Did you do that?
>
> Wasn't there previously a discussion on this list about about a
> mis-behaving KDE cut-ring-manager type program (klipper?) that went
> nuts requesting the selection over and over again?  Maybe that's
> related some how.

Exactly (as I wrote in a separate response which seems to be
delayed by the gnu.org mail server which has been really slow 
for the last few days)...
-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: Display slowness that is painful

[Stefan Monnier]

>> Are you saying that Emacs receives a series of SELECTION_REQUEST_EVENTs,
>> pre-empting redisplay over and over?  

> Yes.  So since redisplay in this case takes longer than 1 second,
> redisplay seems to be stalled for a long time -- usually it completes
> when I change focus (or move selection) to some other application.

Of course, SELECTION_REQUEST_EVENTs shouldn't interrupt redisplay (contrary
to user input).


        Stefan

Re: Display slowness that is painful

[Richard M. Stallman]

    This version of klipper is buggy according to PROBLEMS:

Maybe we don't need to do anything about this beyond telling people
not to run that old version of Klipper.  Which we already do.

If we want to do something more, we could put something in Emacs
to detect receipt of several SELECTION_REQUEST_EVENTs within a few
seconds, and display a warning (just once per session) telling the
user to kill the Klipper window.

Re: Display slowness that is painful

[Richard M. Stallman]

    Of course, SELECTION_REQUEST_EVENTs shouldn't interrupt redisplay (contrary
    to user input).

If redisplay is reasonably fast, then I would tend to agree.  However,
this problem occurs because redisplay is pretty slow in a certain
case.  And if redisplay is that slow, maybe SELECTION_REQUEST_EVENT should
pre-empt it.  After all, SELECTION_REQUEST_EVENT is only supposed
to be sent when the user clicks in another window, and the user ought
to get fast service.

We shouldn't try to optimize the support for SELECTION_REQUEST_EVENT
for the case of a broken Klipper program.

What I think is suspicious is why redisplay spends so much time
thinking before it starts really outputting anything.  That could
be a flaw in Emacs.

Re: Display slowness that is painful

[Kim F. Storm]

"Richard M. Stallman" <rms@gnu.org> writes:

>     This version of klipper is buggy according to PROBLEMS:
>
> Maybe we don't need to do anything about this beyond telling people
> not to run that old version of Klipper.  Which we already do.
>
> If we want to do something more, we could put something in Emacs
> to detect receipt of several SELECTION_REQUEST_EVENTs within a few
> seconds, and display a warning (just once per session) telling the
> user to kill the Klipper window.

That would be bad!

Firstly, There is no klipper window to kill....   

Secondly, It is the basic X clipboard handler which has this bug, so
if you kill it, you cannot copy/paste between applications...


-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: Display slowness that is painful

[Kim F. Storm]

"Richard M. Stallman" <rms@gnu.org> writes:

> We shouldn't try to optimize the support for SELECTION_REQUEST_EVENT
> for the case of a broken Klipper program.

Ok.

> What I think is suspicious is why redisplay spends so much time
> thinking before it starts really outputting anything.  That could
> be a flaw in Emacs.

Well, it generates the desired_matrix for all windows/frames
before it decides what parts of the display should be updated
by comparing to the current_matrix.  

If generating the desired_matrix takes a long time, emacs
doesn't update the display for a long time...

Hasn't it always been like that?

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: Display slowness that is painful

[Miles Bader]

2006/2/6, Kim F. Storm <storm@cua.dk>:
> Firstly, There is no klipper window to kill....
>
> Secondly, It is the basic X clipboard handler which has this bug, so
> if you kill it, you cannot copy/paste between applications...

I don't run klipper (or kde at all) and I can copy/paste between
applications just fine...   Is this something specific to kde apps,
that they depend on klipper to function?

-miles
--
Do not taunt Happy Fun Ball.

Re: Display slowness that is painful

[Kim F. Storm]

Miles Bader <miles@gnu.org> writes:

> 2006/2/6, Kim F. Storm <storm@cua.dk>:
>> Firstly, There is no klipper window to kill....
>>
>> Secondly, It is the basic X clipboard handler which has this bug, so
>> if you kill it, you cannot copy/paste between applications...
>
> I don't run klipper (or kde at all) and I can copy/paste between
> applications just fine...   Is this something specific to kde apps,
> that they depend on klipper to function?

Of course, you are right!  

Without klipper you lose the "clipboard history", but you can
still paste the "last kill/copy".

Sorry for the mis-information!

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: Display slowness that is painful

[Kenichi Handa]

In article <je7j8ct3ss.fsf@sykes.suse.de>, Andreas Schwab <schwab@suse.de> writes:

> Apparently the conversion to multibyte is broken in some way.  In the
> normal case, when unibyte-display-via-language-environment is nil, no
> non-ascii character are ever used for display, so it works as expected.

I found what is wrong.  There are two problems related to
this bug.

At first, when running on a terminal, produce_glyphs is
called instead of x_produce_glyphs, but
unibyte-display-via-language-environment is handled only in
x_produce_glyphs.  So, I've just installed a fix of this
problem.  It at least fixes messing up of display as far as
your terminal and Emacs agree with terminal coding system.

Next, even in utf-8 environment, on displaying, emacs
converts unibyte characters of the range 0xA0..0xFF to some
multibyte characters if
unibyte-display-via-language-environment is non-nil.
Theoretically, as utf-8 doesn't use single byte for
non-ASCII characters, such a conversion should not be done.
But, that conversion has been done for many (7 or more)
years, and now is not a good time to change that code.

---
Kenichi Handa
handa@m17n.org

Re: Display slowness that is painful

[Richard M. Stallman]

Does the latest version of Klipper still have the bug?

Re: Display slowness that is painful

[Richard M. Stallman]

    Well, it generates the desired_matrix for all windows/frames
    before it decides what parts of the display should be updated
    by comparing to the current_matrix.  

    If generating the desired_matrix takes a long time, emacs
    doesn't update the display for a long time...

It seems surprising that generating the desired_matrix would take
a whole second on a modern computer.  Is that often the case
in your usage?

Re: Display slowness that is painful

[Kim F. Storm]

"Richard M. Stallman" <rms@gnu.org> writes:

>     Well, it generates the desired_matrix for all windows/frames
>     before it decides what parts of the display should be updated
>     by comparing to the current_matrix.  
>
>     If generating the desired_matrix takes a long time, emacs
>     doesn't update the display for a long time...
>
> It seems surprising that generating the desired_matrix would take
> a whole second on a modern computer. 

Redisplay time is [at least] O(n) when n is the length of
the longest line in the current window.  So just use a sufficiently
long line to make redisplay take more than 1 second.

In the case you raised, redisplay takes 4 seconds on my 1.4 GHz laptop.

> Is that often the case
> in your usage?

Huh? No.

YOU said YOU have this problem and wanted us to work on a fix.  
Now you say it is not an issue anymore.  So I'm confused!

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: Display slowness that is painful

[Richard M. Stallman]

    YOU said YOU have this problem and wanted us to work on a fix.  
    Now you say it is not an issue anymore.  So I'm confused!

We're talking about two different issues.  There was the large file
(named .gif, though it wasn't really one) which took minutes to
redisplay.  You installed a fix for that problem, right?

Then there's the problem you reported, where redisplay took a few
seconds, and that was long enough to be pre-empted by selection
requests from Klipper.

That is also a serious issue, but it is due to a Klipper bug and we
can address it sufficiently by telling users "Update your KDE or don't
run Klipper".  Isn't that so?

Re: Display slowness that is painful

[Kim F. Storm]

"Richard M. Stallman" <rms@gnu.org> writes:

>     YOU said YOU have this problem and wanted us to work on a fix.  
>     Now you say it is not an issue anymore.  So I'm confused!
>
> We're talking about two different issues.  There was the large file
> (named .gif, though it wasn't really one) which took minutes to
> redisplay.  You installed a fix for that problem, right?

I installed a fix to make it faster -- but I can still create
a file which takes minutes to display -- it just requires a line
which is veeeeeeerrrrrryyyyy long.

In my case, that specific .gif file still takes 4-5 seconds to display
even with my fix.

So the fundamental problem of redisplay being O(n) to the length
of lines in the buffer is not fixed, and I don't know how to
solve it (and we certainly shouldn't do this BTR).

> Then there's the problem you reported, where redisplay took a few
> seconds, and that was long enough to be pre-empted by selection
> requests from Klipper.

Yes, that is a different problem as such -- but I guess we can
find other cases where some activity is stalling redisplay when
redisplay-dont-pause is nil.  So I would still like to put some
upper limit on the time where we force a redisplay to actually
update the screen.

> That is also a serious issue, but it is due to a Klipper bug and we
> can address it sufficiently by telling users "Update your KDE or don't
> run Klipper".  Isn't that so?

That is a work-around for one manifestation of the problem...

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: Display slowness that is painful

[Eli Zaretskii]

> From: storm@cua.dk (Kim F. Storm)
> Date: Thu, 09 Feb 2006 10:20:47 +0100
> Cc: cyd@stupidchicken.com, monnier@iro.umontreal.ca, emacs-devel@gnu.org
> 
> I installed a fix to make it faster -- but I can still create
> a file which takes minutes to display -- it just requires a line
> which is veeeeeeerrrrrryyyyy long.

I don't think we should worry about this too much at this time: long
lines are a known and old problem in the Emacs display engine.  I'm
sorry my faulty memory didn't bring this up before you rediscovered
that in this thread; I should have remembered better what Gerd once
told me about this.

Re: Display slowness that is painful

[Richard M. Stallman]

    > Then there's the problem you reported, where redisplay took a few
    > seconds, and that was long enough to be pre-empted by selection
    > requests from Klipper.

    Yes, that is a different problem as such -- but I guess we can
    find other cases where some activity is stalling redisplay when
    redisplay-dont-pause is nil.

This is not exactly a bug in Emacs, it is a bad interaction.  I will
not claim it necessarily indicates a bug in some other program, but I
have a feeling that that is so.  Anyway, we don't know of a case where
it occurs except due to a bug in some other program.

So I think we need not worry about this now.  If and when we find
a case where this happens, and we don't think the other program
is at fault, we can think about what change is needed in Emacs.

    > That is also a serious issue, but it is due to a Klipper bug and we
    > can address it sufficiently by telling users "Update your KDE or don't
    > run Klipper".  Isn't that so?

    That is a work-around for one manifestation of the problem...

Is any other manifestation known?

Re: Display slowness that is painful

[Richard M. Stallman]

    I installed a fix to make it faster -- but I can still create
    a file which takes minutes to display -- it just requires a line
    which is veeeeeeerrrrrryyyyy long.

    In my case, that specific .gif file still takes 4-5 seconds to display
    even with my fix.

My case took a minute or so to redisplay.  If currently it takes just
5 seconds, you've made a big improvement, and I think we don't need
to do any more just now.