Display performance degradation

Display performance degradation

[YAMAMOTO Mitsuharu]

I can observe significant display performance degradation on Emacs
23.1.90 compared with 23.1, especially when scrolling TUTORIAL.ja on a
frame that uses the xft font backend.  I also observe the increase of
the total number of xftfont_draw calls, and a string in a single font
and color, which was originally displayed by one call, is now
unnecessarily divided into smaller units.

As an experiment, I tried restoring the following change, and then the
performance became comparable to 23.1.

2009-11-17  Jan Djärv  <jan.h.d@swipnet.se>

	* font.c (font_open_entity): Do not use cache, it does not pick up new
	fontconfig settings like hinting.

*** 2975,2985 ****
--- 2987,3001 ----
    else if (CONSP (Vface_font_rescale_alist))
      scaled_pixel_size = pixel_size * font_rescale_ratio (entity);
  
+ #if 0
+   /* This doesn't work if you have changed hinting or any other parameter.
+      We need to make a new object in every case to be sure. */
    for (objlist = AREF (entity, FONT_OBJLIST_INDEX); CONSP (objlist);
         objlist = XCDR (objlist))
      if (! NILP (AREF (XCAR (objlist), FONT_TYPE_INDEX))
  	&& XFONT_OBJECT (XCAR (objlist))->pixel_size == pixel_size)
        return  XCAR (objlist);
+ #endif
  
    val = AREF (entity, FONT_TYPE_INDEX);
    for (driver_list = f->font_driver_list;

The added comment implies that the simple removal of #if 0 causes
another problem for some cases, but I think creating a new font object
for each call is too much for the usual cases.  Perhaps this part
needs some improvement.

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

Re: Display performance degradation

[Jan D.]

On 2009-12-17 03:49, YAMAMOTO Mitsuharu wrote:
> I can observe significant display performance degradation on Emacs
> 23.1.90 compared with 23.1, especially when scrolling TUTORIAL.ja on a
> frame that uses the xft font backend.  I also observe the increase of
> the total number of xftfont_draw calls, and a string in a single font
> and color, which was originally displayed by one call, is now
> unnecessarily divided into smaller units.
>
> As an experiment, I tried restoring the following change, and then the
> performance became comparable to 23.1.

The basic question is why is this called so much?  It is a fundamental 
flaw in Emacs that makes it hard to add new stuff without degrading 
performance.  The core problem is that Emacs internally doesn't track 
what is changed.  So it re-evaluates faces, fonts, menus, toolbars and 
so on, all the time before redisplay, and 99.99% of those times, nothing 
has changed.  So to speed things up, we have various caches to check if 
things are the same as before.  But these caches cause problems 
elsewhere.  For example, if a new font is added or if /etc/font.conf is 
changed, Emacs must be restarted because caches prevents Emacs from 
noticing the change.

In particular, this cache prevents Emacs from picking up changes in 
hintstyle or dpi or other font-related redering parameters.

Emacs should to things all the way to get rid of this problem.  Now, 
when a menu for exampls is changed in lisp, the actual widgets on the 
screen are not changed until later.  So there is no connection between 
the change and the update on the screen.  So Emacs re-evaluates menus a 
lot of times just to be sure, in case something changed at the lisp 
level.  Instead the lisp change should lead to a screen change at once. 
  That way we know that the menu is up to date and we don't have to 
re-evaluate it.  The same is true for faces, which I assume causes this 
problem.

I don't know if this particular problem can be worked around, probably 
there is a way with different caching and/or status flags, but it will 
be just another band-aid.  The real problem is harder to fix.

	Jan D.


>
> 2009-11-17  Jan Djärv<jan.h.d@swipnet.se>
>
> 	* font.c (font_open_entity): Do not use cache, it does not pick up new
> 	fontconfig settings like hinting.
>
> *** 2975,2985 ****
> --- 2987,3001 ----
>      else if (CONSP (Vface_font_rescale_alist))
>        scaled_pixel_size = pixel_size * font_rescale_ratio (entity);
>
> + #if 0
> +   /* This doesn't work if you have changed hinting or any other parameter.
> +      We need to make a new object in every case to be sure. */
>      for (objlist = AREF (entity, FONT_OBJLIST_INDEX); CONSP (objlist);
>           objlist = XCDR (objlist))
>        if (! NILP (AREF (XCAR (objlist), FONT_TYPE_INDEX))
>    	&&  XFONT_OBJECT (XCAR (objlist))->pixel_size == pixel_size)
>          return  XCAR (objlist);
> + #endif
>
>      val = AREF (entity, FONT_TYPE_INDEX);
>      for (driver_list = f->font_driver_list;
>
> The added comment implies that the simple removal of #if 0 causes
> another problem for some cases, but I think creating a new font object
> for each call is too much for the usual cases.  Perhaps this part
> needs some improvement.
>
> 				     YAMAMOTO Mitsuharu
> 				mituharu@math.s.chiba-u.ac.jp
>

Re: Display performance degradation

[Eli Zaretskii]

> Date: Thu, 17 Dec 2009 08:39:50 +0100
> From: "Jan D." <jan.h.d@swipnet.se>
> Cc: emacs-devel@gnu.org
> 
> The core problem is that Emacs internally doesn't track 
> what is changed.  So it re-evaluates faces, fonts, menus, toolbars and 
> so on, all the time before redisplay, and 99.99% of those times, nothing 
> has changed.

Unless you are talking specifically about fonts (where I'm almost
totally ignorant), the above is not really true.  The redisplay code
includes quite a few shortcuts that try to reuse large parts of
existing display, instead of redrawing it.

> Emacs should to things all the way to get rid of this problem.  Now, 
> when a menu for exampls is changed in lisp, the actual widgets on the 
> screen are not changed until later.  So there is no connection between 
> the change and the update on the screen.  So Emacs re-evaluates menus a 
> lot of times just to be sure, in case something changed at the lisp 
> level.  Instead the lisp change should lead to a screen change at once. 

I'm far from being an expert on this, but AFAIU, what you want is
impossible without a total redesign of one of the main features of
Emacs architecture: that Lisp code changes only the buffer text and
various attributes of buffer text, while redisplay happens
independently of that, when Emacs decides that ``it's a good time'' to
do that.

>   That way we know that the menu is up to date and we don't have to 
> re-evaluate it.

I think re-evaluation of menus is unrelated to redisplay.

> The same is true for faces, which I assume causes this problem.

Faces are not recomputed unless necessary.  Large portions of the
display engine try very hard to avoid recomputing faces as much as
possible, to the degree that sometimes you need to work very hard even
to see the code which recomputes faces in action, because the various
optimizations completely short-circuit it.

Re: Display performance degradation

[Jan Djärv]

Eli Zaretskii skrev:
>> Date: Thu, 17 Dec 2009 08:39:50 +0100
>> From: "Jan D." <jan.h.d@swipnet.se>
>> Cc: emacs-devel@gnu.org
>>
>> The core problem is that Emacs internally doesn't track 
>> what is changed.  So it re-evaluates faces, fonts, menus, toolbars and 
>> so on, all the time before redisplay, and 99.99% of those times, nothing 
>> has changed.
> 
> Unless you are talking specifically about fonts (where I'm almost
> totally ignorant), the above is not really true.  The redisplay code
> includes quite a few shortcuts that try to reuse large parts of
> existing display, instead of redrawing it.
>

Sure, but for each redisplay there is a lot of updating of other stuff, that 
really doesn't need updating, like toll bars, menus and faces.


>> Emacs should to things all the way to get rid of this problem.  Now, 
>> when a menu for exampls is changed in lisp, the actual widgets on the 
>> screen are not changed until later.  So there is no connection between 
>> the change and the update on the screen.  So Emacs re-evaluates menus a 
>> lot of times just to be sure, in case something changed at the lisp 
>> level.  Instead the lisp change should lead to a screen change at once. 
> 
> I'm far from being an expert on this, but AFAIU, what you want is
> impossible without a total redesign of one of the main features of
> Emacs architecture: that Lisp code changes only the buffer text and
> various attributes of buffer text, while redisplay happens
> independently of that, when Emacs decides that ``it's a good time'' to
> do that.
> 

Redisplay is one thing, I'm not talking about that.  Rader the realization and 
merging of faces that happens when redisplay is to be done.

>>   That way we know that the menu is up to date and we don't have to 
>> re-evaluate it.
> 
> I think re-evaluation of menus is unrelated to redisplay.

Sure, but not to preformance.

> 
>> The same is true for faces, which I assume causes this problem.
> 
> Faces are not recomputed unless necessary.  Large portions of the
> display engine try very hard to avoid recomputing faces as much as
> possible, to the degree that sometimes you need to work very hard even
> to see the code which recomputes faces in action, because the various
> optimizations completely short-circuit it.

Hmm, faces are recomputed all the time.  If I start emacs like so:
% ./emacs -Q ../etc/tutorials/TUTORIAL.ja

Emacs does 102 calls to merge_face_vectors.  This may be needed, I haven't 
looked at this in detail.

What is strange though, is that if I scroll to the bottom of the buffer, 
another 62 calls to merge_face_vectors are done even if no face has changed.

	Jan D.

Re: Display performance degradation

[Eli Zaretskii]

> Date: Fri, 18 Dec 2009 17:47:22 +0100
> From: Jan_Djärv <jan.h.d@swipnet.se>
> Cc: mituharu@math.s.chiba-u.ac.jp, emacs-devel@gnu.org
> 
> Sure, but for each redisplay there is a lot of updating of other stuff, that 
> really doesn't need updating, like toll bars, menus and faces.

If you know that they don't need to be updated, sure.  But how do you
know that scroll bars and menus don't need to be updated?  E.g., for
menus, we have menu items whose sensitivity is dynamically
computed--how would you know in advance whether the conditions for
that changed or not?

> Redisplay is one thing, I'm not talking about that.  Rader the realization and 
> merging of faces that happens when redisplay is to be done.

Any new text that is being displayed, i.e. text that was not on the
screen during the previous redisplay, needs its faces to be computed
and merged.  If the text on the screen did not change, Emacs avoids
redisplay almost entirely.  If most of the text didn't change, only
the new or modified text is rendered, at least in the most frequent
cases.

> Hmm, faces are recomputed all the time.  If I start emacs like so:
> % ./emacs -Q ../etc/tutorials/TUTORIAL.ja
> 
> Emacs does 102 calls to merge_face_vectors.  This may be needed, I haven't 
> looked at this in detail.

Remember: each charset can potentially use a different font, i.e. a
different face.  But Emacs looks up any face it needs in the face
cache, before it decides to realize a new face.

> What is strange though, is that if I scroll to the bottom of the buffer, 
> another 62 calls to merge_face_vectors are done even if no face has changed.

Did you try to see where are those 62 calls done?

Re: Display performance degradation

[Jan Djärv]

Eli Zaretskii skrev 2009-12-18 22.20:
>> Date: Fri, 18 Dec 2009 17:47:22 +0100
>> From: Jan_Djärv<jan.h.d@swipnet.se>
>
>> What is strange though, is that if I scroll to the bottom of the buffer,
>> another 62 calls to merge_face_vectors are done even if no face has changed.
>
> Did you try to see where are those 62 calls done?

Most of them are from lookup_named_face.  It seems to me that 
lookup_named_face always merges the face with the default face.


	Jan D.

Re: Display performance degradation

[Eli Zaretskii]

> Date: Sat, 02 Jan 2010 12:05:12 +0100
> From: Jan_Djärv <jan.h.d@swipnet.se>
> Cc: mituharu@math.s.chiba-u.ac.jp, emacs-devel@gnu.org
> 
> It seems to me that lookup_named_face always merges the face with
> the default face.

Well, it must, mustn't it?  A face specification does not need to be
complete; it is expected that the unspecified attributes are inherited
from the default face.

The initial issue was why lookup_named_face is at all called, if no
faces were encountered while scanning buffer text to be displayed.
But I think I know the answer: Emacs needs a fully specified face to
look it up in the face cache.  Technically, this is because the cache
lookup is by face attributes, but there's probably a deeper reason
for this design.

Anyway, is the call to lookup_named_face expensive, for all the calls
but the first one which use the same face symbol?  If it is cheap,
then perhaps this is a non-issue.

Re: Display performance degradation

[Jan Djärv]

Eli Zaretskii skrev 2010-01-02 12.21:
>
> Anyway, is the call to lookup_named_face expensive, for all the calls
> but the first one which use the same face symbol?  If it is cheap,
> then perhaps this is a non-issue.
>

All calls have the same cost.  What they do is to clear properties in the 
font.  These properties later on gets restored again.  But the distinction 
between user specified font properties and matched font properties gets lost, 
so what the user specified initially is lost.  This makes re-applying the font 
when some parameter (dpi, hinting etc.) has changed impossible.

I'm trying to address this by another mechanism.

	Jan D.

Re: Display performance degradation

[Kenichi Handa]

In article <wlmy1iuxgi.wl%mituharu@math.s.chiba-u.ac.jp>, YAMAMOTO Mitsuharu <mituharu@math.s.chiba-u.ac.jp> writes:

> I can observe significant display performance degradation on Emacs
> 23.1.90 compared with 23.1, especially when scrolling TUTORIAL.ja on a
> frame that uses the xft font backend.  I also observe the increase of
> the total number of xftfont_draw calls, and a string in a single font
> and color, which was originally displayed by one call, is now
> unnecessarily divided into smaller units.
[...]
> + #if 0
> +   /* This doesn't work if you have changed hinting or any other parameter.
> +      We need to make a new object in every case to be sure. */
>     for (objlist = AREF (entity, FONT_OBJLIST_INDEX); CONSP (objlist);
>          objlist = XCDR (objlist))
>       if (! NILP (AREF (XCAR (objlist), FONT_TYPE_INDEX))
>   	&& XFONT_OBJECT (XCAR (objlist))->pixel_size == pixel_size)
>         return  XCAR (objlist);
> + #endif
  
I didn't notice this change because I'm using X fonts
mainly.

In article <4B29E046.8050107@swipnet.se>, "Jan D." <jan.h.d@swipnet.se> writes:

> The basic question is why is this called so much?

If it is not known, the above code should not be commented out.

> It is a fundamental 
> flaw in Emacs that makes it hard to add new stuff without degrading 
> performance.  The core problem is that Emacs internally doesn't track 
> what is changed.  So it re-evaluates faces, fonts, menus, toolbars and 
> so on, all the time before redisplay, and 99.99% of those times, nothing 
> has changed.  So to speed things up, we have various caches to check if 
> things are the same as before.  But these caches cause problems 
> elsewhere.

Those caches (including face-cache) are there because of the
current redisplay engine (and face handling).  So, without
fixing it (in your word, a fundamental flaw), those caches
should not be disabled.

> For example, if a new font is added or if /etc/font.conf is 
> changed, Emacs must be restarted because caches prevents Emacs from 
> noticing the change.

The problem of extremely slow display is the bigger problem
than it.

> In particular, this cache prevents Emacs from picking up changes in 
> hintstyle or dpi or other font-related redering parameters.

> Emacs should to things all the way to get rid of this problem.  Now, 
> when a menu for exampls is changed in lisp, the actual widgets on the 
> screen are not changed until later.  So there is no connection between 
> the change and the update on the screen.  So Emacs re-evaluates menus a 
> lot of times just to be sure, in case something changed at the lisp 
> level.  Instead the lisp change should lead to a screen change at once. 
>   That way we know that the menu is up to date and we don't have to 
> re-evaluate it.  The same is true for faces, which I assume causes this 
> problem.

> I don't know if this particular problem can be worked around, probably 
> there is a way with different caching and/or status flags, but it will 
> be just another band-aid.  The real problem is harder to fix.

Then please enable the feature of making Emacs react to the
change of dpi, etc only after the workaroubnd is found or
the real problem is fixed.

---
Kenichi Handa
handa@m17n.org

Re: Display performance degradation

[Chong Yidong]

Kenichi Handa <handa@m17n.org> writes:

>> I don't know if this particular problem can be worked around, probably
>> there is a way with different caching and/or status flags, but it will
>> be just another band-aid.  The real problem is harder to fix.
>
> Then please enable the feature of making Emacs react to the
> change of dpi, etc only after the workaroubnd is found or
> the real problem is fixed.

Yes, this is the only sane approach.

Re: Display performance degradation

[Miles Bader]

Chong Yidong <cyd@stupidchicken.com> writes:
>>> I don't know if this particular problem can be worked around, probably
>>> there is a way with different caching and/or status flags, but it will
>>> be just another band-aid.  The real problem is harder to fix.
>>
>> Then please enable the feature of making Emacs react to the
>> change of dpi, etc only after the workaroubnd is found or
>> the real problem is fixed.
>
> Yes, this is the only sane approach.

For "unlikely changes," it would seem fine to have a sort of out-of-band
checker that checks only occasionally and invalidates relevant caches
(and forces redisplay) when a change is found...

-miles

-- 
Faith, n. Belief without evidence in what is told by one who speaks without
knowledge, of things without parallel.

Re: Display performance degradation

[Jan Djärv]

Kenichi Handa skrev:
> In article <wlmy1iuxgi.wl%mituharu@math.s.chiba-u.ac.jp>, YAMAMOTO Mitsuharu <mituharu@math.s.chiba-u.ac.jp> writes:
> 
>> I can observe significant display performance degradation on Emacs
>> 23.1.90 compared with 23.1, especially when scrolling TUTORIAL.ja on a
>> frame that uses the xft font backend.  I also observe the increase of
>> the total number of xftfont_draw calls, and a string in a single font
>> and color, which was originally displayed by one call, is now
>> unnecessarily divided into smaller units.
> [...]
>> + #if 0
>> +   /* This doesn't work if you have changed hinting or any other parameter.
>> +      We need to make a new object in every case to be sure. */
>>     for (objlist = AREF (entity, FONT_OBJLIST_INDEX); CONSP (objlist);
>>          objlist = XCDR (objlist))
>>       if (! NILP (AREF (XCAR (objlist), FONT_TYPE_INDEX))
>>   	&& XFONT_OBJECT (XCAR (objlist))->pixel_size == pixel_size)
>>         return  XCAR (objlist);
>> + #endif
>   
> I didn't notice this change because I'm using X fonts
> mainly.
> 
> In article <4B29E046.8050107@swipnet.se>, "Jan D." <jan.h.d@swipnet.se> writes:
> 
>> The basic question is why is this called so much?
> 
> If it is not known, the above code should not be commented out.
> 
>> It is a fundamental 
>> flaw in Emacs that makes it hard to add new stuff without degrading 
>> performance.  The core problem is that Emacs internally doesn't track 
>> what is changed.  So it re-evaluates faces, fonts, menus, toolbars and 
>> so on, all the time before redisplay, and 99.99% of those times, nothing 
>> has changed.  So to speed things up, we have various caches to check if 
>> things are the same as before.  But these caches cause problems 
>> elsewhere.
> 
> Those caches (including face-cache) are there because of the
> current redisplay engine (and face handling).  So, without
> fixing it (in your word, a fundamental flaw), those caches
> should not be disabled.
> 
>> For example, if a new font is added or if /etc/font.conf is 
>> changed, Emacs must be restarted because caches prevents Emacs from 
>> noticing the change.
> 
> The problem of extremely slow display is the bigger problem
> than it.
> 

How much is extremely, do you have figures?  I haven't noticed any change, 
maybe it only manifests itself for certain locales.

	Jan D.

Re: Display performance degradation

[YAMAMOTO Mitsuharu]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset=ISO-2022-JP-2, Size: 3431 bytes --]

>>>>> On Thu, 17 Dec 2009 17:42:58 +0100, Jan Dj^[.A^[Ndrv <jan.h.d@swipnet.se> said:

>>> For example, if a new font is added or if /etc/font.conf is
>>> changed, Emacs must be restarted because caches prevents Emacs
>>> from noticing the change.
>> 
>> The problem of extremely slow display is the bigger problem than
>> it.
>> 

> How much is extremely, do you have figures?  I haven't noticed any change, 
> maybe it only manifests itself for certain locales.

I can notice a clear difference on Ubuntu 9.10 with some additions for
Japanese support, running on MacBook Air (the bug reporter output is
at the end).

Steps to reproduce:

  1. emacs -Q
  2. C-u C-h t j RET
  3. Keep pressing C-v

Then scrolling looks like "waving".  The font used for displaying
Japanese characters is

  xft:-unknown-VL ^[$B%4%7%C%/^[(B-normal-normal-normal-*-13-*-*-*-*-0-iso10646-1
  (The above Japanese word means "Gothic")

I think it's not noticeable with the tutorial in English (C-u C-h t a
RET in Step 2).  And on some other system such as X11 on Mac OS X
10.5, I found the difference minor.

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

In GNU Emacs 23.1.90.1 (i686-pc-linux-gnu, GTK+ Version 2.18.3)
 of 2009-12-18 on mituharu-laptop
Windowing system distributor `The X.Org Foundation', version 11.0.10604000
Important settings:
  value of $LC_ALL: nil
  value of $LC_COLLATE: nil
  value of $LC_CTYPE: nil
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: nil
  value of $LANG: ja_JP.UTF-8
  value of $XMODIFIERS: @im=ibus
  locale-coding-system: utf-8-unix
  default enable-multibyte-characters: t

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  global-auto-composition-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent input:
<help-echo> <help-echo> <help-echo> <help-echo> <help-echo> 
<help-echo> <help-echo> <help-echo> <help-echo> <menu-bar> 
<help-menu> <send-emacs-bug-report>

Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.

Load-path shadows:
None found.

Features:
(shadow sort mail-extr message sendmail regexp-opt ecomplete rfc822 mml
easymenu mml-sec password-cache mm-decode mm-bodies mm-encode mailcap
mail-parse rfc2231 rfc2047 rfc2045 qp ietf-drums mailabbrev nnheader
gnus-util netrc time-date mm-util mail-prsvr gmm-utils wid-edit
mailheader canlock sha1 hex-util hashcash mail-utils emacsbug japan-util
tooltip ediff-hook vc-hooks lisp-float-type mwheel x-win x-dnd
font-setting tool-bar dnd fontset image fringe lisp-mode register page
menu-bar rfn-eshadow timer select scroll-bar mldrag mouse jit-lock
font-lock syntax facemenu font-core frame cham georgian utf-8-lang
misc-lang vietnamese tibetan thai tai-viet lao korean japanese hebrew
greek romanian slovak czech european ethiopic indian cyrillic chinese
case-table epa-hook jka-cmpr-hook help simple abbrev loaddefs button
minibuffer faces cus-face files text-properties overlay md5 base64
format env code-pages mule custom widget hashtable-print-readable
backquote make-network-process dbusbind system-font-setting
font-render-setting gtk x-toolkit x multi-tty emacs)

Re: Display performance degradation

[Kenichi Handa]

In article <4B2A5F92.9090008@swipnet.se>, Jan Djärv <jan.h.d@swipnet.se> writes:

> How much is extremely, do you have figures?  I haven't noticed any change, 
> maybe it only manifests itself for certain locales.

Actually, "extremely" was a too strong word.  The following test:

(save-window-excursion
  (delete-other-windows)
  (switch-to-buffer "TUTORIAL.ja")
  (goto-char (point-min))
  (benchmark-run 1
      (sit-for 0)
    (while (condition-case nil
	       (progn (scroll-up) (sit-for 0))
	     (error nil)))))

shows that the latest Emacs is about 1.45 times slower (I
turned off all of tool-bar-mode, menu-bar-mode,
scrool-bar-mode, line-number-mode).

But, the perceived difference is much more.

With the previous code, when I type C-v in TUTORIAL.ja
buffer, the upper and middle part of the window (40 lines)
is updated almost instantly, and I notice the redrawing
process only at the bottom few lines.  So, I feel no
frustration.  But, with the current code, with C-v, I see
that the middle part is still under construction.  It's a
big difference in human's perception.

---
Kenichi Handa
handa@m17n.org

Re: Display performance degradation

[YAMAMOTO Mitsuharu]

While I was looking at font caching code, I found a part that seems to
be wrong, though it is not directly related to the original
performance issue.

Fclear_font_cache (in src/font.c):

  4484		    val = XCDR (cache);
  4485		    while (! NILP (val)
  4486			   && ! EQ (XCAR (XCAR (val)), driver_list->driver->type))
  4487		      val = XCDR (val);
  4488		    font_assert (! NILP (val));
  4489		    val = XCDR (XCAR (val));
  4490		    if (XINT (XCAR (val)) == 0)
  4491		      {
  4492			font_clear_cache (f, XCAR (val), driver_list->driver);
  4493			XSETCDR (cache, XCDR (val));
  4494		      }
  4495		  }

At line 4490, XCAR (val) is expected to be of type Lisp integer.  But
it is also passed to font_clear_cache at line 4492, which seems to
expect the value of `val' as of line 4488 rather than the one assigned
at line 4489.

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

Re: Display performance degradation

[Andreas Schwab]

YAMAMOTO Mitsuharu <mituharu@math.s.chiba-u.ac.jp> writes:

>   4490		    if (XINT (XCAR (val)) == 0)
>   4491		      {
>   4492			font_clear_cache (f, XCAR (val), driver_list->driver);
>   4493			XSETCDR (cache, XCDR (val));
>   4494		      }
>   4495		  }
>
> At line 4490, XCAR (val) is expected to be of type Lisp integer.  But
> it is also passed to font_clear_cache at line 4492, which seems to
> expect the value of `val' as of line 4488 rather than the one assigned
> at line 4489.

If the condition is ever true then Emacs crashes.  I've checked in a
fix.  But then, I don't think this function is ever more than a no-op
anyway, since as soon as NUM-FRAMES becomes zero the cache is cleared
automatically anyway.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Display performance degradation

[YAMAMOTO Mitsuharu]

>>>>> On Sat, 19 Dec 2009 11:00:56 +0100, Andreas Schwab <schwab@linux-m68k.org> said:

> YAMAMOTO Mitsuharu <mituharu@math.s.chiba-u.ac.jp> writes:
>> 4490		    if (XINT (XCAR (val)) == 0)
>> 4491		      {
>> 4492			font_clear_cache (f, XCAR (val), driver_list->driver);
>> 4493			XSETCDR (cache, XCDR (val));
>> 4494		      }
>> 4495		  }
>> 
>> At line 4490, XCAR (val) is expected to be of type Lisp integer.  But
>> it is also passed to font_clear_cache at line 4492, which seems to
>> expect the value of `val' as of line 4488 rather than the one assigned
>> at line 4489.

> If the condition is ever true then Emacs crashes.  I've checked in a
> fix.  But then, I don't think this function is ever more than a no-op
> anyway, since as soon as NUM-FRAMES becomes zero the cache is cleared
> automatically anyway.

That's another reason I posted it rather than installing a trivial fix
myself silently.  If the code has never be executed, it might contain
more hidden problems than those can easily be found with simple type
checking by hand.

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

Re: Display performance degradation

[Kenichi Handa]

In article <wlljgzwu1c.wl%mituharu@math.s.chiba-u.ac.jp>, YAMAMOTO Mitsuharu <mituharu@math.s.chiba-u.ac.jp> writes:

> While I was looking at font caching code, I found a part that seems to
> be wrong, though it is not directly related to the original
> performance issue.

> Fclear_font_cache (in src/font.c):

>   4484		    val = XCDR (cache);
>   4485		    while (! NILP (val)
>   4486			   && ! EQ (XCAR (XCAR (val)), driver_list->driver->type))
>   4487		      val = XCDR (val);
>   4488		    font_assert (! NILP (val));
>   4489		    val = XCDR (XCAR (val));
>   4490		    if (XINT (XCAR (val)) == 0)
>   4491		      {
>   4492			font_clear_cache (f, XCAR (val), driver_list->driver);
>   4493			XSETCDR (cache, XCDR (val));
>   4494		      }
>   4495		  }

> At line 4490, XCAR (val) is expected to be of type Lisp integer.  But
> it is also passed to font_clear_cache at line 4492, which seems to
> expect the value of `val' as of line 4488 rather than the one assigned
> at line 4489.

Oops, Fclear_font_cache is a funciton I made for debugging,
but I have forgotten to put it within #ifdef FONT_DEBUG
... #endif, and also forgotten to adjsut it for the other
changes.

In article <m2eimrjnaf.fsf@igel.home>, Andreas Schwab <schwab@linux-m68k.org> writes:
> If the condition is ever true then Emacs crashes.  I've checked in a
> fix.

Thank you!

> But then, I don't think this function is ever more than a no-op
> anyway, since as soon as NUM-FRAMES becomes zero the cache is cleared
> automatically anyway.

Right, so it exists just for debugging.  But, I've just
found that the new file font-setting.el calls it now.  Ummm,
I'm not sure it works while a frame is alive.  I'll check
the code again.

---
Kenichi Handa
handa@m17n.org

Re: Display performance degradation

[Miles Bader]

Kenichi Handa <handa@m17n.org> writes:
> Oops, Fclear_font_cache is a funciton I made for debugging,
> but I have forgotten to put it within #ifdef FONT_DEBUG
> ... #endif, and also forgotten to adjsut it for the other
> changes.

It would be handy to have an official way to get Emacs to see external
font changes without restarting... (e.g., when I change ~/.fonts.conf or
install a new font, I have to restart emacs... :( )

-miles

-- 
Learning, n. The kind of ignorance distinguishing the studious.

Re: Display performance degradation

[Lennart Borgman]

On Mon, Dec 21, 2009 at 3:11 AM, Miles Bader <miles@gnu.org> wrote:
> Kenichi Handa <handa@m17n.org> writes:
>> Oops, Fclear_font_cache is a funciton I made for debugging,
>> but I have forgotten to put it within #ifdef FONT_DEBUG
>> ... #endif, and also forgotten to adjsut it for the other
>> changes.
>
> It would be handy to have an official way to get Emacs to see external
> font changes without restarting... (e.g., when I change ~/.fonts.conf or
> install a new font, I have to restart emacs... :( )


I do not deal with fonts but I have to restart quite often. So finally
I wrote a command, restart-emacs. With desktop-save-mode (and some
additions to desktop) + winsave-save-mode it works quite well for me.
(The command is part of my swizz-style package nXhtml.)

Re: Display performance degradation

[Miles Bader]

Lennart Borgman <lennart.borgman@gmail.com> writes:
> I do not deal with fonts but I have to restart quite often. So finally
> I wrote a command, restart-emacs. With desktop-save-mode (and some
> additions to desktop) + winsave-save-mode it works quite well for me.
> (The command is part of my swizz-style package nXhtml.)

There's always something tho... (e.g., irc chat history, shell history
and output, long-running commands actually still executing in some
subshell, ...)

[There's usually a way you _could_ restore it all, given enough work and
planning, but it's definitely annoying]

-Miles

-- 
Immortality, n.  A toy which people cry for, And on their knees apply for,
      Dispute, contend and lie for, And if allowed Would be right proud
      Eternally to die for.

Re: Display performance degradation

[Lennart Borgman]

On Mon, Dec 21, 2009 at 4:05 AM, Miles Bader <miles@gnu.org> wrote:
> Lennart Borgman <lennart.borgman@gmail.com> writes:
>> I do not deal with fonts but I have to restart quite often. So finally
>> I wrote a command, restart-emacs. With desktop-save-mode (and some
>> additions to desktop) + winsave-save-mode it works quite well for me.
>> (The command is part of my swizz-style package nXhtml.)
>
> There's always something tho... (e.g., irc chat history, shell history
> and output, long-running commands actually still executing in some
> subshell, ...)
>
> [There's usually a way you _could_ restore it all, given enough work and
> planning, but it's definitely annoying]


Ah, yes. But I tend to use that frustration to for example add some
capability to desktop or similar. A kind of "ok, I take this way
then".

Re: Display performance degradation

[Miles Bader]

Lennart Borgman <lennart.borgman@gmail.com> writes:
> Ah, yes. But I tend to use that frustration to for example add some
> capability to desktop or similar. A kind of "ok, I take this way
> then".

Is there a general api for doing that... that modes can use?

e.g.:

    (add-desktop-save-restore-functions 'major-mode 'save-fun 'restore-fun "file-to-load-for-restore-fun.el")

or something...?

-Miles

-- 
Back, n. That part of your friend which it is your privilege to contemplate in
your adversity.

Re: Display performance degradation

[Lennart Borgman]

On Mon, Dec 21, 2009 at 4:44 AM, Miles Bader <miles@gnu.org> wrote:
> Lennart Borgman <lennart.borgman@gmail.com> writes:
>> Ah, yes. But I tend to use that frustration to for example add some
>> capability to desktop or similar. A kind of "ok, I take this way
>> then".
>
> Is there a general api for doing that... that modes can use?
>
> e.g.:
>
>    (add-desktop-save-restore-functions 'major-mode 'save-fun 'restore-fun "file-to-load-for-restore-fun.el")
>
> or something...?


There are hooks, desktop-save-hook (which should be named
desktop-before-save-hook) and desktop-after-read-hook. (But I have not
used these now.)

Re: Display performance degradation

[Miles Bader]

Lennart Borgman <lennart.borgman@gmail.com> writes:
> There are hooks, desktop-save-hook (which should be named
> desktop-before-save-hook) and desktop-after-read-hook. (But I have not
> used these now.)

does that mean all special modes supported by desktop are hardwired into
desktop.el...?  (ick?)

-miles

-- 
The automobile has not merely taken over the street, it has dissolved the
living tissue of the city.  Its appetite for space is absolutely insatiable;
moving and parked, it devours urban land, leaving the buildings as mere
islands of habitable space in a sea of dangerous and ugly traffic.
[James Marston Fitch, New York Times, 1 May 1960]

Re: Display performance degradation

[Lennart Borgman]

On Mon, Dec 21, 2009 at 5:16 AM, Miles Bader <miles@gnu.org> wrote:
> Lennart Borgman <lennart.borgman@gmail.com> writes:
>> There are hooks, desktop-save-hook (which should be named
>> desktop-before-save-hook) and desktop-after-read-hook. (But I have not
>> used these now.)
>
> does that mean all special modes supported by desktop are hardwired into
> desktop.el...?  (ick?)

No. I just happened to hardwire some small things there (in my patched
version of Emacs in Emacs+EmacsW32) before I discovered the hooks.

But some things are hardwired, yes. It might be better to move these
to the hooks.

Re: Display performance degradation

[Jan Djärv]

YAMAMOTO Mitsuharu skrev:
> I can observe significant display performance degradation on Emacs
> 23.1.90 compared with 23.1, especially when scrolling TUTORIAL.ja on a
> frame that uses the xft font backend.  I also observe the increase of
> the total number of xftfont_draw calls, and a string in a single font
> and color, which was originally displayed by one call, is now
> unnecessarily divided into smaller units.
> 

Can you try again?  I've reinstored the part below, but with some changes.

Thanks,

	Jan D.

> As an experiment, I tried restoring the following change, and then the
> performance became comparable to 23.1.
> 
> 2009-11-17  Jan Djärv  <jan.h.d@swipnet.se>
> 
> 	* font.c (font_open_entity): Do not use cache, it does not pick up new
> 	fontconfig settings like hinting.
> 
> *** 2975,2985 ****
> --- 2987,3001 ----
>     else if (CONSP (Vface_font_rescale_alist))
>       scaled_pixel_size = pixel_size * font_rescale_ratio (entity);
>   
> + #if 0
> +   /* This doesn't work if you have changed hinting or any other parameter.
> +      We need to make a new object in every case to be sure. */
>     for (objlist = AREF (entity, FONT_OBJLIST_INDEX); CONSP (objlist);
>          objlist = XCDR (objlist))
>       if (! NILP (AREF (XCAR (objlist), FONT_TYPE_INDEX))
>   	&& XFONT_OBJECT (XCAR (objlist))->pixel_size == pixel_size)
>         return  XCAR (objlist);
> + #endif
>   
>     val = AREF (entity, FONT_TYPE_INDEX);
>     for (driver_list = f->font_driver_list;
> 
> The added comment implies that the simple removal of #if 0 causes
> another problem for some cases, but I think creating a new font object
> for each call is too much for the usual cases.  Perhaps this part
> needs some improvement.
> 
> 				     YAMAMOTO Mitsuharu
> 				mituharu@math.s.chiba-u.ac.jp
> 

Re: Display performance degradation

[YAMAMOTO Mitsuharu]

>>>>> On Wed, 06 Jan 2010 20:46:37 +0100, Jan Djärv <jan.h.d@swipnet.se> said:

> YAMAMOTO Mitsuharu skrev:
>> I can observe significant display performance degradation on Emacs
>> 23.1.90 compared with 23.1, especially when scrolling TUTORIAL.ja
>> on a frame that uses the xft font backend.  I also observe the
>> increase of the total number of xftfont_draw calls, and a string in
>> a single font and color, which was originally displayed by one
>> call, is now unnecessarily divided into smaller units.
>> 

> Can you try again?  I've reinstored the part below, but with some changes.

Yes, scroll in the TUTORIAL.ja window is much faster now.  Thanks.

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp