bug#15876: 24.3.50; Highly degraded performance between rev 114715 and 115006

bug#15876: 24.3.50; Highly degraded performance between rev 114715 and 115006

[Sebastien Vauban]

Hello,

Having switched from Emacs 24.3.50.1 rev:114715 to rev:115006, I've
noticed a huge degradation of performance -- with the same .emacs file.

See a comparative demo on http://screencast.com/t/grH48ZtIbvI2.

What's become slow:
- movements (<down>, <C-home>, etc.) in Org files
- movements (<down>, <C-home>, etc.) in mails
- maybe other stuff (not yet tested)

What's become VERY, VERY slow:
- cycling in an Org file with <S-TAB>: from more or less 1 to 20
  seconds... Almost impossible to use.

Flyspell is not the culprit (at least, in the Org file): I did disable
it before making the video.

I'm willing to use elp, would I know which packages to instrument.

Best regards,
Seb

In GNU Emacs 24.3.50.1 (i686-pc-mingw32)
 of 2013-11-06 on LEG570
Bzr revision: 115006 rgm-x6JLEaVhGFms3UPBXdcZRKrZZz3iRgCd1dLETq3s2gLlzGs343WqgA@public.gmane.org
Windowing system distributor `Microsoft Corp.', version 6.2.9200
Configured using:
 `configure --enable-checking 'CFLAGS=-O0 -g3' CPPFLAGS=-DGLYPH_DEBUG=1'

Important settings:
  value of $LANG: en_US.utf8
  locale-coding-system: cp1252
  default enable-multibyte-characters: t

Major mode: Org

Minor modes in effect:
  global-auto-complete-mode: t
  auto-image-file-mode: t
  recentf-mode: t
  whitespace-mode: t
  shell-dirtrack-mode: t
  helm-match-plugin-mode: t
  helm-occur-match-plugin-mode: t
  yas-global-mode: t
  yas-minor-mode: t
  tooltip-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  column-number-mode: t
  line-number-mode: t
  auto-fill-function: org-auto-fill-function
  transient-mark-mode: t

Recent messages:
Unable to load color "unspecified-fg"     <<<  I don't know where it comes from!
Flyspell mode disabled
Mark set [3 times]
Beginning of buffer [2 times]
Mark set [4 times]

Load-path shadows:
d:/Users/sva/.emacs.d/elpa/graphviz-dot-mode-20120821.1835/graphviz-dot-mode hides ~/.emacs.d/site-lisp/graphviz-dot-mode
~/Public/Repositories/org-mode/lisp/org hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/org
~/Public/Repositories/org-mode/contrib/lisp/org-wl hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/org-wl
~/Public/Repositories/org-mode/lisp/org-w3m hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/org-w3m
~/Public/Repositories/org-mode/contrib/lisp/org-vm hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/org-vm
~/Public/Repositories/org-mode/lisp/org-version hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/org-version
~/Public/Repositories/org-mode/lisp/org-timer hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/org-timer
~/Public/Repositories/org-mode/lisp/org-table hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/org-table
~/Public/Repositories/org-mode/lisp/org-src hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/org-src
~/Public/Repositories/org-mode/lisp/org-rmail hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/org-rmail
~/Public/Repositories/org-mode/lisp/org-protocol hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/org-protocol
~/Public/Repositories/org-mode/lisp/org-plot hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/org-plot
~/Public/Repositories/org-mode/lisp/org-pcomplete hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/org-pcomplete
~/Public/Repositories/org-mode/lisp/org-mouse hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/org-mouse
~/Public/Repositories/org-mode/lisp/org-mobile hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/org-mobile
~/Public/Repositories/org-mode/lisp/org-mhe hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/org-mhe
~/Public/Repositories/org-mode/contrib/lisp/org-mew hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/org-mew
~/Public/Repositories/org-mode/lisp/org-macs hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/org-macs
~/Public/Repositories/org-mode/lisp/org-loaddefs hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/org-loaddefs
~/Public/Repositories/org-mode/lisp/org-list hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/org-list
~/Public/Repositories/org-mode/lisp/org-irc hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/org-irc
~/Public/Repositories/org-mode/lisp/org-install hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/org-install
~/Public/Repositories/org-mode/lisp/org-inlinetask hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/org-inlinetask
~/Public/Repositories/org-mode/lisp/org-info hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/org-info
~/Public/Repositories/org-mode/lisp/org-indent hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/org-indent
~/Public/Repositories/org-mode/lisp/org-id hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/org-id
~/Public/Repositories/org-mode/lisp/org-habit hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/org-habit
~/Public/Repositories/org-mode/lisp/org-gnus hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/org-gnus
~/Public/Repositories/org-mode/lisp/org-footnote hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/org-footnote
~/Public/Repositories/org-mode/lisp/org-feed hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/org-feed
~/Public/Repositories/org-mode/lisp/org-faces hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/org-faces
~/Public/Repositories/org-mode/lisp/org-eshell hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/org-eshell
~/Public/Repositories/org-mode/lisp/org-entities hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/org-entities
~/Public/Repositories/org-mode/lisp/org-element hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/org-element
~/Public/Repositories/org-mode/lisp/org-docview hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/org-docview
~/Public/Repositories/org-mode/lisp/org-datetree hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/org-datetree
~/Public/Repositories/org-mode/lisp/org-ctags hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/org-ctags
~/Public/Repositories/org-mode/lisp/org-crypt hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/org-crypt
~/Public/Repositories/org-mode/lisp/org-compat hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/org-compat
~/Public/Repositories/org-mode/lisp/org-colview hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/org-colview
~/Public/Repositories/org-mode/lisp/org-clock hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/org-clock
~/Public/Repositories/org-mode/lisp/org-capture hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/org-capture
~/Public/Repositories/org-mode/lisp/org-bibtex hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/org-bibtex
~/Public/Repositories/org-mode/lisp/org-bbdb hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/org-bbdb
~/Public/Repositories/org-mode/lisp/org-attach hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/org-attach
~/Public/Repositories/org-mode/lisp/org-archive hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/org-archive
~/Public/Repositories/org-mode/lisp/org-agenda hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/org-agenda
~/Public/Repositories/org-mode/lisp/ob hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/ob
~/Public/Repositories/org-mode/lisp/ob-tangle hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/ob-tangle
~/Public/Repositories/org-mode/lisp/ob-table hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/ob-table
~/Public/Repositories/org-mode/lisp/ob-sqlite hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/ob-sqlite
~/Public/Repositories/org-mode/lisp/ob-sql hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/ob-sql
~/Public/Repositories/org-mode/lisp/ob-shen hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/ob-shen
~/Public/Repositories/org-mode/lisp/ob-sh hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/ob-sh
~/Public/Repositories/org-mode/lisp/ob-screen hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/ob-screen
~/Public/Repositories/org-mode/lisp/ob-scheme hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/ob-scheme
~/Public/Repositories/org-mode/lisp/ob-scala hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/ob-scala
~/Public/Repositories/org-mode/lisp/ob-sass hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/ob-sass
~/Public/Repositories/org-mode/lisp/ob-ruby hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/ob-ruby
~/Public/Repositories/org-mode/lisp/ob-ref hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/ob-ref
~/Public/Repositories/org-mode/lisp/ob-R hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/ob-R
~/Public/Repositories/org-mode/lisp/ob-python hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/ob-python
~/Public/Repositories/org-mode/lisp/ob-plantuml hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/ob-plantuml
~/Public/Repositories/org-mode/lisp/ob-picolisp hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/ob-picolisp
~/Public/Repositories/org-mode/lisp/ob-perl hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/ob-perl
~/Public/Repositories/org-mode/lisp/ob-org hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/ob-org
~/Public/Repositories/org-mode/lisp/ob-octave hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/ob-octave
~/Public/Repositories/org-mode/lisp/ob-ocaml hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/ob-ocaml
~/Public/Repositories/org-mode/lisp/ob-mscgen hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/ob-mscgen
~/Public/Repositories/org-mode/lisp/ob-maxima hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/ob-maxima
~/Public/Repositories/org-mode/lisp/ob-matlab hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/ob-matlab
~/Public/Repositories/org-mode/lisp/ob-lob hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/ob-lob
~/Public/Repositories/org-mode/lisp/ob-lisp hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/ob-lisp
~/Public/Repositories/org-mode/lisp/ob-lilypond hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/ob-lilypond
~/Public/Repositories/org-mode/lisp/ob-ledger hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/ob-ledger
~/Public/Repositories/org-mode/lisp/ob-latex hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/ob-latex
~/Public/Repositories/org-mode/lisp/ob-keys hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/ob-keys
~/Public/Repositories/org-mode/lisp/ob-js hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/ob-js
~/Public/Repositories/org-mode/lisp/ob-java hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/ob-java
~/Public/Repositories/org-mode/lisp/ob-io hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/ob-io
~/Public/Repositories/org-mode/lisp/ob-haskell hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/ob-haskell
~/Public/Repositories/org-mode/lisp/ob-gnuplot hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/ob-gnuplot
~/Public/Repositories/org-mode/lisp/ob-fortran hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/ob-fortran
~/Public/Repositories/org-mode/lisp/ob-exp hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/ob-exp
~/Public/Repositories/org-mode/lisp/ob-eval hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/ob-eval
~/Public/Repositories/org-mode/lisp/ob-emacs-lisp hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/ob-emacs-lisp
~/Public/Repositories/org-mode/lisp/ob-dot hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/ob-dot
~/Public/Repositories/org-mode/lisp/ob-ditaa hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/ob-ditaa
~/Public/Repositories/org-mode/lisp/ob-css hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/ob-css
~/Public/Repositories/org-mode/lisp/ob-comint hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/ob-comint
~/Public/Repositories/org-mode/lisp/ob-clojure hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/ob-clojure
~/Public/Repositories/org-mode/lisp/ob-calc hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/ob-calc
~/Public/Repositories/org-mode/lisp/ob-C hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/ob-C
~/Public/Repositories/org-mode/lisp/ob-awk hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/ob-awk
~/Public/Repositories/org-mode/lisp/ob-asymptote hides c:/Program Files (x86)/emacs-trunk/share/emacs/24.3.50/lisp/org/ob-asymptote

Features:
(shadow sort mail-extr gnus-msg gnus-art mm-uu mml2015 epg-config
mm-view mml-smime smime dig emacsbug sendmail helm-command helm-elisp
helm-eval helm-mode eldoc edebug redshank skeleton paredit hideshow
saveplace server auto-complete-config auto-complete popup vc-dispatcher
vc-svn org-table git-commit vc-git org-id org-gnus gnus-sum nnoo
gnus-group gnus-undo nnmail mail-source gnus-start gnus-spec gnus-int
gnus-range message rfc822 mml mml-sec mm-decode mm-bodies mm-encode
mail-parse rfc2231 rfc2047 rfc2045 ietf-drums mailabbrev gmm-utils
mailheader gnus-win gnus gnus-ems nnheader mail-utils org-habit
org-agenda org-info org-element org-mime org-crypt ob-sql ob-sh
ob-python ob-org ob-ledger ob-latex ob-gnuplot ob-dot ob-ditaa ob-calc
calc-store calc-trail calc-ext calc calc-loaddefs calc-macs ob-awk ob-R
appt diary-lib diary-loaddefs org-inlinetask mule-util org org-macro
org-footnote org-pcomplete org-list org-faces org-entities org-version
ob-emacs-lisp ob ob-tangle org-src ob-ref ob-lob ob-table ob-keys ob-exp
ob-comint ob-core ob-eval org-compat org-macs cal-menu calendar
cal-loaddefs filecache image-file bookmark pp recentf tree-widget
wid-edit ido helm-files image-dired whitespace flyspell ispell noutline
outline tramp tramp-compat tramp-loaddefs trampver shell pcomplete
format-spec ffap thingatpt helm-buffers helm-elscreen helm-tags
helm-bookmark helm-adaptative helm-info helm-net browse-url xml url
url-proxy url-privacy url-expand url-methods url-history url-cookie
url-domsuf url-util url-parse auth-source eieio byte-opt bytecomp
byte-compile cconv eieio-core gnus-util mm-util mail-prsvr
password-cache url-vars mailcap helm-plugin helm-locate helm-help
helm-match-plugin helm-grep helm-regexp grep helm-external helm-utils
dired-sort-map dired-single dired+ dired-x dired-aux dired compile
comint ansi-color ring helm emacs-leuven leuven-theme yasnippet
help-mode find-func paren mic-paren hl-tags-mode derived org-loaddefs
uniquify helm-config helm-aliases diff-mode- easy-mmode edmacro kmacro
idle-require finder-inf tex-site auto-complete-autoloads bbdb-autoloads
gnuplot-mode-autoloads idle-require-autoloads info easymenu
lcs-autoloads pager-autoloads rainbow-mode-autoloads tidy-autoloads
tracking-autoloads shorten-autoloads package cl-macs gv advice help-fns
cl cl-loaddefs cl-lib time-date tooltip ediff-hook vc-hooks
lisp-float-type mwheel dos-w32 ls-lisp w32-common-fns disp-table w32-win
w32-vars tool-bar dnd fontset image regexp-opt fringe tabulated-list
newcomment lisp-mode prog-mode register page menu-bar rfn-eshadow timer
select scroll-bar mouse jit-lock font-lock syntax facemenu font-core
frame cham georgian utf-8-lang misc-lang vietnamese tibetan thai
tai-viet lao korean japanese hebrew greek romanian slovak czech european
ethiopic indian cyrillic chinese case-table epa-hook jka-cmpr-hook help
simple abbrev minibuffer nadvice loaddefs button faces cus-face macroexp
files text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget hashtable-print-readable backquote make-network-process
w32notify w32 multi-tty emacs)

-- 
Sebastien Vauban

bug#15876: 24.3.50; Highly degraded performance between rev 114715 and 115006

[Glenn Morris]

"Sebastien Vauban" wrote:

> Having switched from Emacs 24.3.50.1 rev:114715 to rev:115006, I've
[...]
> Recent messages:
> Unable to load color "unspecified-fg"     <<<  I don't know where it comes from!

From something that's already been fixed, IIRC, so probably best to
update to the very latest trunk (you are 70+ revisions behind at time of
writing) and do a clean build before spending much time on this.

bug#15876: 24.3.50; Highly degraded performance between rev 114715 and 115006

[Sebastien Vauban]

Glenn Morris wrote:
> "Sebastien Vauban" wrote:
>
>> Having switched from Emacs 24.3.50.1 rev:114715 to rev:115006, I've
> [...]
>> Recent messages:
>> Unable to load color "unspecified-fg"     <<<  I don't know where it comes from!
>
> From something that's already been fixed, IIRC, so probably best to
> update to the very latest trunk (you are 70+ revisions behind at time of
> writing) and do a clean build before spending much time on this.

Is that have something to do with the performance degradation?

Anyway, I'm using Dani's binaries. See
http://sourceforge.net/projects/emacs-bin/?source=directory.

Currently, there is no newer one.

Best regards,
  Seb

-- 
Sebastien Vauban

bug#15876: 24.3.50; Highly degraded performance between rev 114715 and 115006

[Dani Moncayo]

> Anyway, I'm using Dani's binaries. See
> http://sourceforge.net/projects/emacs-bin/?source=directory.
>
> Currently, there is no newer one.

Yesterday I uploaded a new binary.  However, I forgot to manually edit
the file 'lisp/loadup.el' to set the value of 'emacs-bzr-version'.  So
in this binary that variable returns nothing.

Perhaps it would be better to stop relying on that variable, and just
use the timestamp, e.g.:

  (setq frame-title-format
        (format
         "Emacs %s of %s    PID:%d"
         emacs-version
         (format-time-string "%Y-%m-%d" emacs-build-time)
         (emacs-pid)))


-- 
Dani Moncayo

Re: bug#15876: 24.3.50; Highly degraded performance between rev 114715 and 115006

[Sebastien Vauban]

Hello Dani,

Dani Moncayo wrote:
>> Anyway, I'm using Dani's binaries. See
>> http://sourceforge.net/projects/emacs-bin/?source=directory.
>>
>> Currently, there is no newer one.
>
> Yesterday I uploaded a new binary.

Thanks!

> However, I forgot to manually edit the file 'lisp/loadup.el' to set the value
> of 'emacs-bzr-version'. So in this binary that variable returns nothing.
>
> Perhaps it would be better to stop relying on that variable, and just
> use the timestamp, e.g.:
>
>   (setq frame-title-format
>         (format
>          "Emacs %s of %s    PID:%d"
>          emacs-version
>          (format-time-string "%Y-%m-%d" emacs-build-time)
>          (emacs-pid)))

I now use both the rev number (when available) and the date:

--8<---------------cut here---------------start------------->8---
  ;; title bar display of visible frames
  (setq frame-title-format
        (format "Emacs %s rev:%s of %s    PID:%d"
                emacs-version
                (ignore-errors
                  (replace-regexp-in-string " .*" "" emacs-bzr-version))
                (format-time-string "%Y-%m-%d" emacs-build-time)
                (emacs-pid)))
--8<---------------cut here---------------end--------------->8---

Thanks for your input.

Best regards,
  Seb

-- 
Sebastien Vauban

bug#15876: 24.3.50; Highly degraded performance between rev 114715 and 115006

[Sebastien Vauban]

Glenn Moris wrote:
>> Best to update to the very latest trunk (you are 70+ revisions behind at
>> time of writing) and do a clean build before spending much time on this.
>
> Is that have something to do with the performance degradation?

I'm now using rev 115085 (of 2013-11-12), thanks to Dani.

As for 115006, that version is almost unusable as some operations are so slow.

See a comparative demo of rev 114715 and 115085 on
http://screencast.com/t/ITwSR0Wrz.

Problems:

- Moving with C-up/down takes more time with latest Emacs, and uses up to 33%
  of my CPU (vs about 1% with rev 114715).

- S-TAB'ing in an Org mode file takes 20s (vs 1s with rev 114715) and eats up
  to 33% of my CPU as well.

Best regards,
  Seb

-- 
Sebastien Vauban

bug#15876: 24.3.50; Highly degraded performance between rev 114715 and 115006

[Eli Zaretskii]

> From: "Sebastien Vauban" <sva-news@mygooglest.com>
> Date: Wed, 13 Nov 2013 15:04:06 +0100
> Cc: 15876@debbugs.gnu.org
> 
> - Moving with C-up/down takes more time with latest Emacs, and uses up to 33%
>   of my CPU (vs about 1% with rev 114715).
> 
> - S-TAB'ing in an Org mode file takes 20s (vs 1s with rev 114715) and eats up
>   to 33% of my CPU as well.

Try setting cache-long-scans to nil in your Org buffers, and see if
that makes any change.  If it does, then please report as a bug with
some minimal reproduction recipe starting with "emacs -Q".

bug#15876: 24.3.50; Highly degraded performance between rev 114715 and 115006

[Stefan Monnier]

> - Moving with C-up/down takes more time with latest Emacs, and uses up to 33%
>   of my CPU (vs about 1% with rev 114715).

> - S-TAB'ing in an Org mode file takes 20s (vs 1s with rev 114715) and eats up
>   to 33% of my CPU as well.

If you (setq cache-long-scans nil), does the performance problem disappear?


        Stefan

bug#15876: 24.3.50; Highly degraded performance between rev 114715 and 115006

[Sebastien Vauban]

Eli Zaretskii wrote:
>> From: "Sebastien Vauban" <sva-news-D0wtAvR13HarG/iDocfnWg@public.gmane.org>
>> Date: Wed, 13 Nov 2013 15:04:06 +0100
>> Cc: 15876-ubl+/3LiMTaZdePnXv/OxA@public.gmane.org
>> 
>> - Moving with C-up/down takes more time with latest Emacs, and uses up to 33%
>>   of my CPU (vs about 1% with rev 114715).
>> 
>> - S-TAB'ing in an Org mode file takes 20s (vs 1s with rev 114715) and eats up
>>   to 33% of my CPU as well.
>
> Try setting cache-long-scans to nil in your Org buffers, and see if
> that makes any change.  If it does, then please report as a bug with
> some minimal reproduction recipe starting with "emacs -Q".

It does not, as you can see in http://screencast.com/t/nBZ9VqyyW.

There, I first try with the default setting, then eval your expression, and
continue to move the cursor or S-TAB. Still the same long delays, and
over-usage of the CPU.

BTW, I was expecting it to be so, as I'm using a Git copy of Org [1] -- this
was not said in my mail. Hence, if a change in Org was responsible for the
problem, I *should* have it in both Emacs versions or not at all, no?

Of course, would I use the Org built-in with Emacs, changing of Emacs version
would mean also changing of Org version. But that's not my case.

Best regards,
  Seb

[1] Currently: Org-mode version 8.2.1 (release_8.2.1-202-gea797e.dirty @
    ~/Public/Repositories/org-mode/lisp/).

-- 
Sebastien Vauban

bug#15876: 24.3.50; Highly degraded performance between rev 114715 and 115006

[Eli Zaretskii]

> From: "Sebastien Vauban" <sva-news@mygooglest.com>
> Cc: rgm@gnu.org,  15876@debbugs.gnu.org
> Date: Wed, 13 Nov 2013 21:23:14 +0100
> 
> Eli Zaretskii wrote:
> >> From: "Sebastien Vauban" <sva-news@mygooglest.com>
> >> Date: Wed, 13 Nov 2013 15:04:06 +0100
> >> Cc: 15876@debbugs.gnu.org
> >> 
> >> - Moving with C-up/down takes more time with latest Emacs, and uses up to 33%
> >>   of my CPU (vs about 1% with rev 114715).
> >> 
> >> - S-TAB'ing in an Org mode file takes 20s (vs 1s with rev 114715) and eats up
> >>   to 33% of my CPU as well.
> >
> > Try setting cache-long-scans to nil in your Org buffers, and see if
> > that makes any change.  If it does, then please report as a bug with
> > some minimal reproduction recipe starting with "emacs -Q".
> 
> It does not, as you can see in http://screencast.com/t/nBZ9VqyyW.

Thanks.  I expected that, but wanted to be sure.

> BTW, I was expecting it to be so, as I'm using a Git copy of Org [1] -- this
> was not said in my mail. Hence, if a change in Org was responsible for the
> problem, I *should* have it in both Emacs versions or not at all, no?

I was not talking about changes in Org.

bug#15876: 24.3.50; Highly degraded performance between rev 114715 and 115006

[Glenn Morris]

"Sebastien Vauban" wrote:

> BTW, I was expecting it to be so, as I'm using a Git copy of Org [1] -- this
> was not said in my mail. Hence, if a change in Org was responsible for the
> problem, I *should* have it in both Emacs versions or not at all, no?

What this for sure would mean is that it is pointless to report it here,
as opposed to the Org list.

Although you originally said:

>    What's become slow:
>    - movements (<down>, <C-home>, etc.) in Org files
>    - movements (<down>, <C-home>, etc.) in mails
>    - maybe other stuff (not yet tested)

I don't see how point 2 can be Org-related. But then I have no idea what
setup you are using.

If you think Emacs is generally slow, then please post a recipe
starting from emacs -Q that shows the problem, preferably not involving
Org (since it has many moving parts).

If you think your git copy of Org has become slower, then talk to the Org
mailing list.

bug#15876: 24.3.50; Highly degraded performance between rev 114715 and 115006

[Sebastien Vauban]

Glenn Morris wrote:
> "Sebastien Vauban" wrote:
>
>> BTW, I was expecting it to be so, as I'm using a Git copy of Org [1] -- this
>> was not said in my mail. Hence, if a change in Org was responsible for the
>> problem, I *should* have it in both Emacs versions or not at all, no?
>
> What this for sure would mean is that it is pointless to report it here,
> as opposed to the Org list.

I know, I just wanted to write that explicitly. The problem is not Org related,
even if it's more visible within Org.

> Although you originally said:
>
>>    What's become slow:
>>    - movements (<down>, <C-home>, etc.) in Org files
>>    - movements (<down>, <C-home>, etc.) in mails
>>    - maybe other stuff (not yet tested)
>
> I don't see how point 2 can be Org-related.

It's not.

See a comparative demo when moving point in a *Gnus summary buffer* on
http://screencast.com/t/Qr5KxNgCYL. Every movement is much slower there.

Though I don't use rev 115085 much because of those problems, I tried to bite
the bullet and worked a bit more with it. I can tell you the problem is NOT
general. For example, moving point inside a Java buffer is quick, as expected.

Currently, I constantly observe the performance problem inside:

- Org files,
- Gnus summary buffers.

I don't observe it in:

- Java files,
- Gnus group buffer,
- Org files (when major mode is changed to `text').

> But then I have no idea what setup you are using.
>
> If you think Emacs is generally slow, then please post a recipe
> starting from emacs -Q that shows the problem, preferably not involving
> Org (since it has many moving parts).

I'll try to do that later today. But it seems it won't be obvious, as the only
spots where I observed them are Org and Gnus, and I guess you'd like not having
any of these two enabled in the recipe.

Best regards,
  Seb

-- 
Sebastien Vauban

Re: bug#15876: 24.3.50; Highly degraded performance between rev 114715 and 115006

[Sebastien Vauban]

Glenn,

"Sebastien Vauban" wrote:
>>>    What's become slow:
>>>    - movements (<down>, <C-home>, etc.) in Org files
>>>    - movements (<down>, <C-home>, etc.) in mails
>>>    - maybe other stuff (not yet tested)
>>
>> I don't see how point 2 can be Org-related.
>
> It's not.

Even if the problem is not related to Org (as it is also present in Gnus
summary buffers), point 2 could be Org-related, as many Org users (me included)
do have orgtbl, orgstruct-mode or orgstruct++ enabled when composing mails:

--8<---------------cut here---------------start------------->8---
      (defun my-message-mode-hook ()

        ;; tab completion for alias in `.mailrc'
        (local-set-key (kbd "<M-tab>") 'mail-abbrev-complete-alias)

        ;; enable automatic word-wrap when composing messages
        (setq fill-column 79)
        (turn-on-auto-fill)

        ;; turn on the `org-mode' table editor
        (turn-on-orgtbl)

        ;; turn on (the enhanced version of) orgstruct-mode
        (turn-on-orgstruct++))

      (add-hook 'message-mode-hook 'my-message-mode-hook)
--8<---------------cut here---------------end--------------->8---

Best regards,
  Seb

-- 
Sebastien Vauban

bug#15876: 24.3.50; Highly degraded performance between rev 114715 and 115006

[Jarek Czekalski]

W dniu 2013-11-12 16:32, Sebastien Vauban pisze:
> Configured using:
>   `configure --enable-checking 'CFLAGS=-O0 -g3' CPPFLAGS=-DGLYPH_DEBUG=1'

I guess these flags are not present in the production versions of Emacs. 
But maybe this is not so important.

Jarek

bug#15876: 24.3.50; Highly degraded performance between rev 114715 and 115006

[Eli Zaretskii]

> Date: Thu, 14 Nov 2013 12:03:53 +0100
> From: Jarek Czekalski <jarekczek@poczta.onet.pl>
> 
> W dniu 2013-11-12 16:32, Sebastien Vauban pisze:
> > Configured using:
> >   `configure --enable-checking 'CFLAGS=-O0 -g3' CPPFLAGS=-DGLYPH_DEBUG=1'
> 
> I guess these flags are not present in the production versions of Emacs. 
> But maybe this is not so important.

These flags cannot cause such a painful degradation.  They might
explain something like twofold slowdown, but not what is being
described here.

(FWIW, I always compile the trunk with the above flags, so I would
sense the slowdown immediately.)

bug#15876: 24.3.50; Highly degraded performance between rev 114715 and 115006

[Glenn Morris]

"Sebastien Vauban" wrote:

> Even if the problem is not related to Org (as it is also present in Gnus
> summary buffers), point 2 could be Org-related, as many Org users (me included)
> do have orgtbl, orgstruct-mode or orgstruct++ enabled when composing mails:

You can't expect us to know these things unless you tell us.

Also, please don't post via the newsgroup, since it bypasses the bug
tracker. (I think I've said this before.) Use Gmane if you really want a
news interface to this list.

bug#15876: 24.3.50; Highly degraded performance between rev 114715 and 115006

[Sebastien Vauban]

"Sebastien Vauban" wrote:
> Glenn Morris wrote:
>> "Sebastien Vauban" wrote:
>
> The problem is not Org related, even if it's more visible within Org.
>
>>> What's become slow:
>>> - movements (<down>, <C-home>, etc.) in Org files
>>> - movements (<down>, <C-home>, etc.) in mails
>>> - maybe other stuff (not yet tested)
>
> Though I don't use rev 115085 much because of those problems, I tried to bite
> the bullet and worked a bit more with it.
>
>> If you think Emacs is generally slow, then please post a recipe starting
>> from emacs -Q that shows the problem, preferably not involving Org (since it
>> has many moving parts).
>
> I'll try to do that later today. But it seems it won't be obvious, as the
> only spots where I observed them are Org and Gnus, and I guess you'd like not
> having any of these two enabled in the recipe.

I don't have yet a minimized Emacs configuration file, but I already can tell
that, in my full .emacs file, disabling the following line does solve the
problem!

--8<---------------cut here---------------start------------->8---
;; highlight trailing whitespaces in all modes
(setq-default show-trailing-whitespace t)
--8<---------------cut here---------------end--------------->8---

So, it seems to conflict with something else from my .emacs -- still have to
find what.

Does the above ring a bell?

Best regards,
  Seb

-- 
Sebastien Vauban

bug#15876: 24.3.50; Highly degraded performance between rev 114715 and 115006

[Stefan Monnier]

> --8<---------------cut here---------------start------------->8---
> ;; highlight trailing whitespaces in all modes
> (setq-default show-trailing-whitespace t)
> --8<---------------cut here---------------end--------------->8---

Can you try to compare the performance of revision 114849 and
revision 114848?  The difference between the two is that the region
highlighting was moved from C to Elisp.  This usually shouldn't affect
performance, but maybe there's some interaction with redisplay
that ends up disabling most optimizations?


        Stefan

bug#15876: 24.3.50; Highly degraded performance between rev 114715 and 115006

[Eli Zaretskii]

> From: "Sebastien Vauban" <sva-news@mygooglest.com>
> Date: Tue, 19 Nov 2013 23:52:18 +0100
> Cc: 15876@debbugs.gnu.org
> 
> I don't have yet a minimized Emacs configuration file, but I already can tell
> that, in my full .emacs file, disabling the following line does solve the
> problem!
> 
> --8<---------------cut here---------------start------------->8---
> ;; highlight trailing whitespaces in all modes
> (setq-default show-trailing-whitespace t)
> --8<---------------cut here---------------end--------------->8---
> 
> So, it seems to conflict with something else from my .emacs -- still have to
> find what.
> 
> Does the above ring a bell?

If you enable show-trailing-whitespace only in a few modes (via a mode
hook), does the performance problem remain only in those modes and
disappear in others?

bug#15876: 24.3.50; Highly degraded performance between rev 114715 and 115006

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Date: Tue, 19 Nov 2013 20:47:52 -0500
> Cc: 15876@debbugs.gnu.org
> 
> > --8<---------------cut here---------------start------------->8---
> > ;; highlight trailing whitespaces in all modes
> > (setq-default show-trailing-whitespace t)
> > --8<---------------cut here---------------end--------------->8---
> 
> Can you try to compare the performance of revision 114849 and
> revision 114848?  The difference between the two is that the region
> highlighting was moved from C to Elisp.  This usually shouldn't affect
> performance, but maybe there's some interaction with redisplay
> that ends up disabling most optimizations?

The old region highlighting code was also an optimization killer.
Besides, no disabled optimizations should ever have such a terrible
effect.  It's most probably a bug.

bug#15876: 24.3.50; Highly degraded performance between rev 114715 and 115006

[Sebastien Vauban]

Stefan Monnier wrote:
>> --8<---------------cut here---------------start------------->8---
>> ;; highlight trailing whitespaces in all modes
>> (setq-default show-trailing-whitespace t)
>> --8<---------------cut here---------------end--------------->8---
>
> Can you try to compare the performance of revision 114849 and
> revision 114848?  The difference between the two is that the region
> highlighting was moved from C to Elisp.  This usually shouldn't affect
> performance, but maybe there's some interaction with redisplay
> that ends up disabling most optimizations?

I can compare such versions, if I can find the Windows binaries somewhere. I
don't have the ability to construct it by myself -- yet.

Best regards,
  Seb

-- 
Sebastien Vauban

bug#15876: 24.3.50; Highly degraded performance between rev 114715 and 115006

[Sebastien Vauban]

"Sebastien Vauban" wrote:
> "Sebastien Vauban" wrote:
>> Glenn Morris wrote:
>>> "Sebastien Vauban" wrote:
>>
>> Though I don't use rev 115085 much because of those problems, I tried to bite
>> the bullet and worked a bit more with it.
>>
>>> If you think Emacs is generally slow, then please post a recipe starting
>>> from emacs -Q that shows the problem, preferably not involving Org (since it
>>> has many moving parts).
>>
>> I'll try to do that later today. But it seems it won't be obvious, as the
>> only spots where I observed them are Org and Gnus, and I guess you'd like not
>> having any of these two enabled in the recipe.
>
> I don't have yet a minimized Emacs configuration file, but I already can tell
> that, in my full .emacs file, disabling the following line does solve the
> problem!
>
> ;; highlight trailing whitespaces in all modes
> (setq-default show-trailing-whitespace t)
>
> So, it seems to conflict with something else from my .emacs -- still have to
> find what.

The following item that makes the problem happen (or not, depending whether
commented out) is:

--8<---------------cut here---------------start------------->8---
  ;; ellipsis to use in the Org mode outline
  (setq org-ellipsis " \u25B7") ; white triangle
--8<---------------cut here---------------end--------------->8---

Best regards,
  Seb

-- 
Sebastien Vauban

bug#15876: 24.3.50; Highly degraded performance between rev 114715 and 115006

[Eli Zaretskii]

> From: "Sebastien Vauban" <sva-news@mygooglest.com>
> Date: Wed, 20 Nov 2013 23:32:44 +0100
> 
> "Sebastien Vauban" wrote:
> > "Sebastien Vauban" wrote:
> >> Glenn Morris wrote:
> >>> "Sebastien Vauban" wrote:
> >>
> >> Though I don't use rev 115085 much because of those problems, I tried to bite
> >> the bullet and worked a bit more with it.
> >>
> >>> If you think Emacs is generally slow, then please post a recipe starting
> >>> from emacs -Q that shows the problem, preferably not involving Org (since it
> >>> has many moving parts).
> >>
> >> I'll try to do that later today. But it seems it won't be obvious, as the
> >> only spots where I observed them are Org and Gnus, and I guess you'd like not
> >> having any of these two enabled in the recipe.
> >
> > I don't have yet a minimized Emacs configuration file, but I already can tell
> > that, in my full .emacs file, disabling the following line does solve the
> > problem!
> >
> > ;; highlight trailing whitespaces in all modes
> > (setq-default show-trailing-whitespace t)
> >
> > So, it seems to conflict with something else from my .emacs -- still have to
> > find what.
> 
> The following item that makes the problem happen (or not, depending whether
> commented out) is:
> 
> --8<---------------cut here---------------start------------->8---
>   ;; ellipsis to use in the Org mode outline
>   (setq org-ellipsis " \u25B7") ; white triangle
> --8<---------------cut here---------------end--------------->8---

So what is the minimum .emacs that will cause the problem?

bug#15876: 24.3.50; Highly degraded performance between rev 114715 and 115006

[Sebastien Vauban]

Eli Zaretskii wrote:
>> From: "Sebastien Vauban" <sva-news-D0wtAvR13HarG/iDocfnWg@public.gmane.org>
>> Date: Wed, 20 Nov 2013 23:32:44 +0100
>> 
>> "Sebastien Vauban" wrote:
>> > "Sebastien Vauban" wrote:
>> >> Glenn Morris wrote:
>> >>> "Sebastien Vauban" wrote:
>> >>
>> >> Though I don't use rev 115085 much because of those problems, I tried to bite
>> >> the bullet and worked a bit more with it.
>> >>
>> >>> If you think Emacs is generally slow, then please post a recipe starting
>> >>> from emacs -Q that shows the problem, preferably not involving Org (since it
>> >>> has many moving parts).
>> >>
>> >> I'll try to do that later today. But it seems it won't be obvious, as the
>> >> only spots where I observed them are Org and Gnus, and I guess you'd like not
>> >> having any of these two enabled in the recipe.
>> >
>> > I don't have yet a minimized Emacs configuration file, but I already can tell
>> > that, in my full .emacs file, disabling the following line does solve the
>> > problem!
>> >
>> > ;; highlight trailing whitespaces in all modes
>> > (setq-default show-trailing-whitespace t)
>> >
>> > So, it seems to conflict with something else from my .emacs -- still have to
>> > find what.
>> 
>> The following item that makes the problem happen (or not, depending whether
>> commented out) is:
>> 
>> --8<---------------cut here---------------start------------->8---
>>   ;; ellipsis to use in the Org mode outline
>>   (setq org-ellipsis " \u25B7") ; white triangle
>> --8<---------------cut here---------------end--------------->8---
>
> So what is the minimum .emacs that will cause the problem?

That does it:

--8<---------------cut here---------------start------------->8---
;; highlight trailing whitespaces in all modes
(setq-default show-trailing-whitespace t)

(setq org-ellipsis " \u25B7")           ; string
--8<---------------cut here---------------end--------------->8---

Best regards,
  Seb

-- 
Sebastien Vauban

bug#15876: 24.3.50; Highly degraded performance between rev 114715 and 115006

[Eli Zaretskii]

> From: "Sebastien Vauban" <sva-news@mygooglest.com>
> Cc: 15876@debbugs.gnu.org
> Date: Tue, 26 Nov 2013 21:52:05 +0100
> 
> > So what is the minimum .emacs that will cause the problem?
> 
> That does it:
> 
> --8<---------------cut here---------------start------------->8---
> ;; highlight trailing whitespaces in all modes
> (setq-default show-trailing-whitespace t)
> 
> (setq org-ellipsis " \u25B7")           ; string
> --8<---------------cut here---------------end--------------->8---

And this makes Emacs slow in scrolling for any file you visit?  Or
just in Org buffers?

If the latter, perhaps your Org files are special in some way, because
the ones I tried don't show any slowdown I could sense.

bug#15876: 24.3.50; Highly degraded performance between rev 114715 and 115006

[Sebastien Vauban]

Eli Zaretskii wrote:
>> From: "Sebastien Vauban" <sva-news-D0wtAvR13HarG/iDocfnWg@public.gmane.org>
>> Cc: 15876-ubl+/3LiMTaZdePnXv/OxA@public.gmane.org
>> Date: Tue, 26 Nov 2013 21:52:05 +0100
>> 
>> > So what is the minimum .emacs that will cause the problem?
>> 
>> That does it:
>> 
>> --8<---------------cut here---------------start------------->8---
>> ;; highlight trailing whitespaces in all modes
>> (setq-default show-trailing-whitespace t)
>> 
>> (setq org-ellipsis " \u25B7")           ; string
>> --8<---------------cut here---------------end--------------->8---
>
> And this makes Emacs slow in scrolling for any file you visit?  Or
> just in Org buffers?

No problem, with the same files, if I put them in `text-mode'. So, it becomes
apparent with Org.

Remember I told there were similar speed problems with some Gnus buffer, but I
haven't tested them again.

> If the latter, perhaps your Org files are special in some way, because
> the ones I tried don't show any slowdown I could sense.

They aren't special, for two reasons:

- They're not mine; they're public Org files taken from the Worg project ("Wiki
  Org").

- They work perfectly in Emacs rev 114715: no performance degradation
  whatsoever.

For the sake of clarity, I've redone a comparison of the two different Emacs
(r114715 and the recent r115235) on the same two Org files:

- ChangeLog.org (1,156 bytes)
  http://code.ohloh.net/file?fid=s6NzuuiKK0Nlga9EF-Vh8KzVXiw&cid=SNfK_pTfQmo&s=&fp=19410&mp=&projSelected=true#L0

- org-faq.org (164,174 bytes)
  http://code.ohloh.net/file?fid=Jk4U4mwQtJT_5LfYpl3sBLhj42s&cid=SNfK_pTfQmo&s=&fp=19410&mp=&projSelected=true#L0

Just look at the CPU usage, and see how the cursor moves slowly -- while I
press on the arrows at the same speed in both configs.

See http://screencast.com/t/oHIPTP0eAW.

For the records, my .emacs is really tiny, as you can see:

--8<---------------cut here---------------start------------->8---
;; highlight trailing whitespaces in all modes
(setq-default show-trailing-whitespace t)

(setq org-ellipsis " \u25B7")           ; string

;; title bar display of visible frames
(setq frame-title-format
      (format "Emacs %s rev:%s of %s    PID:%d"
              ;; (capitalize (symbol-name system-type))
              emacs-version
              (ignore-errors
                (replace-regexp-in-string " .*" "" emacs-bzr-version))
              (format-time-string "%Y-%m-%d" emacs-build-time)
              (emacs-pid)))
--8<---------------cut here---------------end--------------->8---

Please don't tell me nobody can reproduce this problem!?

Best regards,
  Seb

-- 
Sebastien Vauban

bug#15876: 24.3.50; Highly degraded performance between rev 114715 and 115006

[Eli Zaretskii]

> From: "Sebastien Vauban" <sva-news@mygooglest.com>
> Cc: 15876@debbugs.gnu.org
> Date: Fri, 29 Nov 2013 22:01:10 +0100
> 
> Eli Zaretskii wrote:
> >> From: "Sebastien Vauban" <sva-news@mygooglest.com>
> >> Cc: 15876@debbugs.gnu.org
> >> Date: Tue, 26 Nov 2013 21:52:05 +0100
> >> 
> >> > So what is the minimum .emacs that will cause the problem?
> >> 
> >> That does it:
> >> 
> >> --8<---------------cut here---------------start------------->8---
> >> ;; highlight trailing whitespaces in all modes
> >> (setq-default show-trailing-whitespace t)
> >> 
> >> (setq org-ellipsis " \u25B7")           ; string
> >> --8<---------------cut here---------------end--------------->8---
> >
> > And this makes Emacs slow in scrolling for any file you visit?  Or
> > just in Org buffers?
> 
> No problem, with the same files, if I put them in `text-mode'. So, it becomes
> apparent with Org.
> 
> Remember I told there were similar speed problems with some Gnus buffer, but I
> haven't tested them again.
> 
> > If the latter, perhaps your Org files are special in some way, because
> > the ones I tried don't show any slowdown I could sense.
> 
> They aren't special, for two reasons:
> 
> - They're not mine; they're public Org files taken from the Worg project ("Wiki
>   Org").
> 
> - They work perfectly in Emacs rev 114715: no performance degradation
>   whatsoever.
> 
> For the sake of clarity, I've redone a comparison of the two different Emacs
> (r114715 and the recent r115235) on the same two Org files:
> 
> - ChangeLog.org (1,156 bytes)
>   http://code.ohloh.net/file?fid=s6NzuuiKK0Nlga9EF-Vh8KzVXiw&cid=SNfK_pTfQmo&s=&fp=19410&mp=&projSelected=true#L0
> 
> - org-faq.org (164,174 bytes)
>   http://code.ohloh.net/file?fid=Jk4U4mwQtJT_5LfYpl3sBLhj42s&cid=SNfK_pTfQmo&s=&fp=19410&mp=&projSelected=true#L0

I looked at this issue.

It has nothing to do with org-ellipsis, or the Org mode in general.
To reproduce it, it's enough to do just this:

  emacs -Q
  C-x 8 RET 25b7 RET

Now try moving the cursor across the line that displays the triangle
character u+25B7.  If you are on Windows 7, this will use the
BatangChe font for the triangle, and every redisplay operation, even
cursor motion, will be extremely slow.  (On XP, this font is not
installed, and even if I install it, I'm unable to trigger the same
problem, for some reason.)

This slowdown seems to be caused by this commit:

  revno: 114735
  committer: Dmitry Antipov <dmantipov@yandex.ru>
  branch nick: trunk
  timestamp: Mon 2013-10-21 14:11:25 +0000
  message:
    Do not allow font caches to grow too large.
    * alloc.c (compact_font_cache_entry, compact_font_caches):
    New functions or stub if not HAVE_WINDOW_SYSTEM.
    (compact_undo_list): Factor out from Fgarbage_collect.
    Add comment.
    (mark_face_cache): Mark face font.  Move down to avoid
    extra prototypes.
    (mark_terminals): Do not mark font cache here.
    (Fgarbage_collect): Call compaction functions described
    above.  Adjust comment.

What happens is that the above-mentioned font causes a lot of consing,
when loaded (perhaps because it supports many different Unicode
blocks).  That triggers GC immediately after redisplay; GC calls
compact_font_caches, which for some reason decides that the font-specs
in the font cache can be removed.  Then the next redisplay needs the
deleted font again, so it again loads it, which causes consing, which
triggers GC, etc. etc.

If I disable the compacting, like this:

  static void
  compact_font_caches (void)
  {
    struct terminal *t;

    for (t = terminal_list; t; t = t->next_terminal)
      {
	Lisp_Object cache = TERMINAL_FONT_CACHE (t);

  #if 0
	if (CONSP (cache))
	  {
	    Lisp_Object entry;

	    for (entry = XCDR (cache); CONSP (entry); entry = XCDR (entry))
	      XSETCAR (entry, compact_font_cache_entry (XCAR (entry)));
	  }
  #endif
	mark_object (cache);
      }
  }

then the problem goes away.

Dmitry, can you please look into this?  I'm not familiar enough with
the font stuff, so I don't see how was the font-spec and its
font-entities stored in the font cache supposed to be referenced from
some other Lisp object, to make sure they are marked and not GC'ed.
In any case, it seems like they are never marked on Windows, because
if I set a breakpoint in the line marked below:

	  /* If font-spec is not marked, most likely all font-entities
	     are not marked too.  But we must be sure that nothing is
	     marked within OBJ before we really drop it.  */
	  for (i = 0; i < size; i++)
	    if (VECTOR_MARKED_P (XFONT_ENTITY (AREF (XCDR (obj), i))))
	      break;

	  if (i == size)  <<<<<<<<<<<<<<<<<<<<<
	    drop = 1;
	}

with a condition i != size, that breakpoint never breaks.  If you see
something different on X, please tell which code is responsible for
marking those font-entities and/or the font-spec's in the font cache.
(I guess you meant mark_face_cache to do that, but it marks fonts, not
font-entities or font-spec's.)

bug#15876: 24.3.50; Highly degraded performance between rev 114715 and 115006

[Sebastien Vauban]

Eli Zaretskii wrote:
>> From: "Sebastien Vauban" <sva-news-D0wtAvR13HarG/iDocfnWg@public.gmane.org>
>> Cc: 15876-ubl+/3LiMTaZdePnXv/OxA@public.gmane.org
>> Date: Fri, 29 Nov 2013 22:01:10 +0100
>> 
>> Eli Zaretskii wrote:
>> >> From: "Sebastien Vauban" <sva-news-D0wtAvR13HarG/iDocfnWg@public.gmane.org>
>> >> Cc: 15876-ubl+/3LiMTaZdePnXv/OxA@public.gmane.org
>> >> Date: Tue, 26 Nov 2013 21:52:05 +0100
>> >> 
>> >> > So what is the minimum .emacs that will cause the problem?
>> >> 
>> >> That does it:
>> >> 
>> >> --8<---------------cut here---------------start------------->8---
>> >> ;; highlight trailing whitespaces in all modes
>> >> (setq-default show-trailing-whitespace t)
>> >> 
>> >> (setq org-ellipsis " \u25B7")           ; string
>> >> --8<---------------cut here---------------end--------------->8---
>> >
>> > And this makes Emacs slow in scrolling for any file you visit?  Or
>> > just in Org buffers?
>> 
>> No problem, with the same files, if I put them in `text-mode'. So, it becomes
>> apparent with Org.
>> 
>> Remember I told there were similar speed problems with some Gnus buffer, but I
>> haven't tested them again.
>> 
>> > If the latter, perhaps your Org files are special in some way, because
>> > the ones I tried don't show any slowdown I could sense.
>> 
>> They aren't special, for two reasons:
>> 
>> - They're not mine; they're public Org files taken from the Worg project ("Wiki
>>   Org").
>> 
>> - They work perfectly in Emacs rev 114715: no performance degradation
>>   whatsoever.
>> 
>> For the sake of clarity, I've redone a comparison of the two different Emacs
>> (r114715 and the recent r115235) on the same two Org files:
>> 
>> - ChangeLog.org (1,156 bytes)
>>   http://code.ohloh.net/file?fid=s6NzuuiKK0Nlga9EF-Vh8KzVXiw&cid=SNfK_pTfQmo&s=&fp=19410&mp=&projSelected=true#L0
>> 
>> - org-faq.org (164,174 bytes)
>>   http://code.ohloh.net/file?fid=Jk4U4mwQtJT_5LfYpl3sBLhj42s&cid=SNfK_pTfQmo&s=&fp=19410&mp=&projSelected=true#L0
>
> I looked at this issue.
>
> It has nothing to do with org-ellipsis, or the Org mode in general.
> To reproduce it, it's enough to do just this:
>
>   emacs -Q
>   C-x 8 RET 25b7 RET
>
> Now try moving the cursor across the line that displays the triangle
> character u+25B7.

Glad you could finally reproduce and observe my problem. I'm not a fool
anymore ;-)

> If you are on Windows 7, this will use the
> BatangChe font for the triangle, and every redisplay operation, even
> cursor motion, will be extremely slow.  (On XP, this font is not
> installed, and even if I install it, I'm unable to trigger the same
> problem, for some reason.)

FWIW, I don't know which font is used by default in the very minimal example I
gave, and used. But, in my real .emacs conf file, I'm using Consolas (from
Microsoft).

Best regards,
  Seb

-- 
Sebastien Vauban

bug#15876: 24.3.50; Highly degraded performance between rev 114715 and 115006

[Eli Zaretskii]

> From: "Sebastien Vauban" <sva-news@mygooglest.com>
> Cc: Dmitry Antipov <dmantipov@yandex.ru>,  15876@debbugs.gnu.org
> Date: Sun, 01 Dec 2013 21:20:31 +0100
> 
> > If you are on Windows 7, this will use the
> > BatangChe font for the triangle, and every redisplay operation, even
> > cursor motion, will be extremely slow.  (On XP, this font is not
> > installed, and even if I install it, I'm unable to trigger the same
> > problem, for some reason.)
> 
> FWIW, I don't know which font is used by default in the very minimal example I
> gave, and used. But, in my real .emacs conf file, I'm using Consolas (from
> Microsoft).

For which character(s)?  You might be using Consolas for the text, but
something else for the triangle.  "C-u C-x =" should tell you.

bug#15876: 24.3.50; Highly degraded performance between rev 114715 and 115006

[Sebastien Vauban]

Eli Zaretskii wrote:
>> From: "Sebastien Vauban" <sva-news-D0wtAvR13HarG/iDocfnWg@public.gmane.org>
>> Cc: Dmitry Antipov <dmantipov-o+MxOtu4lMCHXe+LvDLADg@public.gmane.org>,  15876-ubl+/3LiMTaZdePnXv/OxA@public.gmane.org
>> Date: Sun, 01 Dec 2013 21:20:31 +0100
>> 
>> > If you are on Windows 7, this will use the
>> > BatangChe font for the triangle, and every redisplay operation, even
>> > cursor motion, will be extremely slow.  (On XP, this font is not
>> > installed, and even if I install it, I'm unable to trigger the same
>> > problem, for some reason.)
>> 
>> FWIW, I don't know which font is used by default in the very minimal example I
>> gave, and used. But, in my real .emacs conf file, I'm using Consolas (from
>> Microsoft).
>
> For which character(s)?  You might be using Consolas for the text, but
> something else for the triangle.  "C-u C-x =" should tell you.

I don't succeed putting point on it. See http://screencast.com/t/jiC9rO3bj9T.

I can move to the end of the line, and be before it, or after it, but never on
it!?

Anyway, it must be Consolas, as Consolas is the font used everywhere else, and
as I have the following in my .emacs:

--8<---------------cut here---------------start------------->8---
    (setq org-ellipsis
          (if (char-displayable-p ?\u25B7) ; white right-pointing triangle
              " \u25B7"                 ; string
            'org-ellipsis)))            ; face
--8<---------------cut here---------------end--------------->8---

As you see, the triangle is used only if the char is displayable in the current
font, that is in Consolas in my case.

Best regards,
  Seb

-- 
Sebastien Vauban

bug#15876: 24.3.50; Highly degraded performance between rev 114715 and 115006

[Eli Zaretskii]

> From: "Sebastien Vauban" <sva-news@mygooglest.com>
> Cc: 15876@debbugs.gnu.org,  dmantipov@yandex.ru
> Date: Sun, 01 Dec 2013 22:51:32 +0100
> 
> I don't succeed putting point on it. See http://screencast.com/t/jiC9rO3bj9T.

Use my recipe, not yours.  The problem happens as soon as the u+25b7
character is displayed in the buffer, no matter where it is displayed
and by which mechanism.

> I can move to the end of the line, and be before it, or after it, but never on
> it!?

Of course, this is how ellipsis behaves -- it doesn't come from buffer
text, so you cannot put point there.

> Anyway, it must be Consolas, as Consolas is the font used everywhere else, and
> as I have the following in my .emacs:
> 
> --8<---------------cut here---------------start------------->8---
>     (setq org-ellipsis
>           (if (char-displayable-p ?\u25B7) ; white right-pointing triangle
>               " \u25B7"                 ; string
>             'org-ellipsis)))            ; face
> --8<---------------cut here---------------end--------------->8---
> 
> As you see, the triangle is used only if the char is displayable in the current
> font, that is in Consolas in my case.

Where does it say "in the current font"?  It doesn't.  Emacs will use
any font available to it.  So your conclusion is incorrect.

Only "C-u C-x =" can tell what font is used.

bug#15876: 24.3.50; Highly degraded performance between rev 114715 and 115006

[Sebastien Vauban]

Eli Zaretskii wrote:
>> From: "Sebastien Vauban" <sva-news-D0wtAvR13HarG/iDocfnWg@public.gmane.org>
>> Cc: 15876-ubl+/3LiMTaZdePnXv/OxA@public.gmane.org,  dmantipov-o+MxOtu4lMCHXe+LvDLADg@public.gmane.org
>> Date: Sun, 01 Dec 2013 22:51:32 +0100
>> 
>> I don't succeed putting point on it. See http://screencast.com/t/jiC9rO3bj9T.
>
> Use my recipe, not yours.  The problem happens as soon as the u+25b7
> character is displayed in the buffer, no matter where it is displayed
> and by which mechanism.
>
>> Anyway, it must be Consolas, as Consolas is the font used everywhere else, and
>> as I have the following in my .emacs:
>> 
>> --8<---------------cut here---------------start------------->8---
>>     (setq org-ellipsis
>>           (if (char-displayable-p ?\u25B7) ; white right-pointing triangle
>>               " \u25B7"                 ; string
>>             'org-ellipsis)))            ; face
>> --8<---------------cut here---------------end--------------->8---
>> 
>> As you see, the triangle is used only if the char is displayable in the current
>> font, that is in Consolas in my case.
>
> Where does it say "in the current font"?  It doesn't.  Emacs will use
> any font available to it.  So your conclusion is incorrect.

I had the impression that, when I wrote that code, I tested it with different
fonts, and that I sometimes had the triangle, sometimes "...".

That made me think that my code does correctly check in the current font if the
triangle is displayable (or not), and then chooses the right character whether
it is (not).

I must be mistaken because...

> Only "C-u C-x =" can tell what font is used.

... you're right: on my Windows 8, Emacs uses the BatangChe font for the
triangle.

Best regards,
  Seb

-- 
Sebastien Vauban

bug#15876: 24.3.50; Highly degraded performance between rev 114715 and 115006

[Dmitry Antipov]

On 12/01/2013 08:31 PM, Eli Zaretskii wrote:

> Dmitry, can you please look into this?  I'm not familiar enough with
> the font stuff, so I don't see how was the font-spec and its
> font-entities stored in the font cache supposed to be referenced from
> some other Lisp object, to make sure they are marked and not GC'ed.

It should be easy - just use this:

=== modified file 'src/alloc.c'
--- src/alloc.c	2013-12-01 22:33:13 +0000
+++ src/alloc.c	2013-12-02 09:32:55 +0000
@@ -5731,7 +5731,18 @@
    eassert (!VECTOR_MARKED_P (ptr));
    VECTOR_MARK (ptr);		/* Else mark it.  */
    if (size & PSEUDOVECTOR_FLAG)
-    size &= PSEUDOVECTOR_SIZE_MASK;
+    {
+      size &= PSEUDOVECTOR_SIZE_MASK;
+      if (PSEUDOVECTOR_TYPEP (&ptr->header, PVEC_FONT))
+	{
+	  Lisp_Object tmp;
+	  XSETFONT (tmp, ptr);
+	  if (FONT_SPEC_P (tmp))
+	    fprintf (stderr, "mark font-spec\n");
+	  else if (FONT_ENTITY_P (tmp))
+	    fprintf (stderr, "mark font-entity\n");
+	}
+    }

and set breakpoints to fprintf(). I tried this and see that font-spec
objects can be referenced from face caches:

(gdb) bt 3 full
#0  mark_vectorlike (ptr=0x13178b8) at ../../trunk/src/alloc.c:5741
         tmp = {i = 20019389}
         size = 13
         i = 20019389
#1  0x00000000005ec050 in mark_object (arg=...) at ../../trunk/src/alloc.c:6084
         ptr = 0x13178b8
         pvectype = 15
         obj = {i = 20019384}
         cdr_count = 0
#2  0x00000000005eb92b in mark_face_cache (c=0x15049b0) at ../../trunk/src/alloc.c:5839
         face = 0x18d2050
         i = 2
         j = 15
(More stack frames follow...)

I.e. (struct face *)0x18d2050 has font-spec object in lface[15].

But the most of font-spec and font-entity objects are referenced via
staticpro'ed globals Vfontset_table and ft_face_cache. For example:

(gdb) bt 16
#0  mark_vectorlike (ptr=0x129a288) at ../../trunk/src/alloc.c:5741
#1  0x00000000005ec050 in mark_object (arg=...) at ../../trunk/src/alloc.c:6084
#2  0x00000000005eb5f0 in mark_vectorlike (ptr=0x1296428) at ../../trunk/src/alloc.c:5752
#3  0x00000000005ec050 in mark_object (arg=...) at ../../trunk/src/alloc.c:6084
#4  0x00000000005eb5f0 in mark_vectorlike (ptr=0x129a4d0) at ../../trunk/src/alloc.c:5752
#5  0x00000000005ec050 in mark_object (arg=...) at ../../trunk/src/alloc.c:6084
#6  0x00000000005eb725 in mark_char_table (ptr=0x1788060) at ../../trunk/src/alloc.c:5779
#7  0x00000000005eb717 in mark_char_table (ptr=0x12094d0) at ../../trunk/src/alloc.c:5776
#8  0x00000000005eb717 in mark_char_table (ptr=0x13e2578) at ../../trunk/src/alloc.c:5776
#9  0x00000000005eb717 in mark_char_table (ptr=0xd85188) at ../../trunk/src/alloc.c:5776
#10 0x00000000005ec02c in mark_object (arg=...) at ../../trunk/src/alloc.c:6069
#11 0x00000000005eb5f0 in mark_vectorlike (ptr=0xd85080) at ../../trunk/src/alloc.c:5752
#12 0x00000000005ec050 in mark_object (arg=...) at ../../trunk/src/alloc.c:6084
#13 0x00000000005eac4b in Fgarbage_collect () at ../../trunk/src/alloc.c:5489
#14 0x0000000000563500 in maybe_gc () at ../../trunk/src/lisp.h:4462
#15 0x000000000060ec3e in Ffuncall (nargs=2, args=0x7fffffff0790) at ../../trunk/src/eval.c:2756
(gdb) bt 16 full
...
#13 0x00000000005eac4b in Fgarbage_collect () at ../../trunk/src/alloc.c:5489
         nextb = 0x0
         stack_top_variable = 0 '\000'
         i = 1491
         message_p = false
         count = 140
         start = {tv_sec = 1385977872, tv_nsec = 656653658}
         retval = {i = 13914818}
         tot_before = 0
...
(gdb) p staticvec[1491]
$8 = (Lisp_Object *) 0xd25cb0 <Vfontset_table>

> In any case, it seems like they are never marked on Windows, because
> if I set a breakpoint in the line marked below:
>
> 	  /* If font-spec is not marked, most likely all font-entities
> 	     are not marked too.  But we must be sure that nothing is
> 	     marked within OBJ before we really drop it.  */
> 	  for (i = 0; i < size; i++)
> 	    if (VECTOR_MARKED_P (XFONT_ENTITY (AREF (XCDR (obj), i))))
> 	      break;
>
> 	  if (i == size)  <<<<<<<<<<<<<<<<<<<<<
> 	    drop = 1;
> 	}
>
> with a condition i != size, that breakpoint never breaks.

The best way to hit this breakpoint is to run something like:

(defun bloat-font ()
   (interactive)
   (let ((fonts (x-list-fonts "*")))
     (while fonts
       (condition-case nil (set-frame-font (car fonts)) (error nil))
       (setq fonts (cdr fonts))
       (redisplay))))

(This is X-specific; I believe there should be a similar MS-Windows method).

Subtle "triangle example" works just fine for me (with default font used by
GTK3 build and 'emacs -Q' at least).

Also I'm curious how to reproduce this issue with X. In particular, how to
find a font so "heavy" that an attempt to load it creates a lot of Lisp data
(hundreds Kbytes, enough to reach gc_cons_threshold each time)?

Dmitry

bug#15876: 24.3.50; Highly degraded performance between rev 114715 and 115006

[Dmitry Antipov]

[-- Attachment #1: Type: text/plain, Size: 275 bytes --]

BTW, I have grabbed BatangChe font from MS-Windows machine and tried:

./src/emacs -Q -font '-hanyang system-batangche-medium-r-normal--0-0-0-0-m-0-iso10646-1'

This looks extremely ugly, but basic editing around "magic triangle" works without any
visible slowdown.

Dmitry


[-- Attachment #2: emacs-batang.png --]
[-- Type: image/png, Size: 28628 bytes --]

bug#15876: 24.3.50; Highly degraded performance between rev 114715 and 115006

[Eli Zaretskii]

> Date: Mon, 02 Dec 2013 14:45:25 +0400
> From: Dmitry Antipov <dmantipov@yandex.ru>
> CC: 15876@debbugs.gnu.org
> 
> On 12/01/2013 08:31 PM, Eli Zaretskii wrote:
> 
> > Dmitry, can you please look into this?  I'm not familiar enough with
> > the font stuff, so I don't see how was the font-spec and its
> > font-entities stored in the font cache supposed to be referenced from
> > some other Lisp object, to make sure they are marked and not GC'ed.
> 
> It should be easy - just use this:
> 
> === modified file 'src/alloc.c'
> --- src/alloc.c	2013-12-01 22:33:13 +0000
> +++ src/alloc.c	2013-12-02 09:32:55 +0000
> @@ -5731,7 +5731,18 @@
>     eassert (!VECTOR_MARKED_P (ptr));
>     VECTOR_MARK (ptr);		/* Else mark it.  */
>     if (size & PSEUDOVECTOR_FLAG)
> -    size &= PSEUDOVECTOR_SIZE_MASK;
> +    {
> +      size &= PSEUDOVECTOR_SIZE_MASK;
> +      if (PSEUDOVECTOR_TYPEP (&ptr->header, PVEC_FONT))
> +	{
> +	  Lisp_Object tmp;
> +	  XSETFONT (tmp, ptr);
> +	  if (FONT_SPEC_P (tmp))
> +	    fprintf (stderr, "mark font-spec\n");
> +	  else if (FONT_ENTITY_P (tmp))
> +	    fprintf (stderr, "mark font-entity\n");
> +	}
> +    }
> 
> and set breakpoints to fprintf(). I tried this and see that font-spec
> objects can be referenced from face caches:

Thanks.  But I think I didn't make myself clear: the issue is not just
to see ANY font-spec objects being marked.  The issue is with those
font-spec objects that are recorded in the font caches that are
compacted by compact_font_caches.  I don't see any code that makes
sure some Lisp object references those caches.  Without that, they
cannot be possibly marked, and will be GC'ed, right?

> But the most of font-spec and font-entity objects are referenced via
> staticpro'ed globals Vfontset_table and ft_face_cache.

Those staticpro'ed objects might just be the reason why you don't see
the problem.  Your build uses the ftfont driver, doesn't it?  Because
ftfont.c has this implementation of the get_cache method:

  static Lisp_Object
  ftfont_get_cache (struct frame *f)
  {
    return freetype_font_cache;
  }

and freetype_font_cache is a staticpro'ed variable.  By contrast,
w32font.c does this:

  Lisp_Object
  w32font_get_cache (struct frame *f)
  {
    struct w32_display_info *dpyinfo = FRAME_DISPLAY_INFO (f);

    return (dpyinfo->name_list_element);
  }

and dpyinfo->name_list_element is what compact_font_caches examines.

(Note that xfont.c and xftfont.c also use a cache stored in
dpyinfo->name_list_element, so this issue is not entirely Windows
specific.)

> > In any case, it seems like they are never marked on Windows, because
> > if I set a breakpoint in the line marked below:
> >
> > 	  /* If font-spec is not marked, most likely all font-entities
> > 	     are not marked too.  But we must be sure that nothing is
> > 	     marked within OBJ before we really drop it.  */
> > 	  for (i = 0; i < size; i++)
> > 	    if (VECTOR_MARKED_P (XFONT_ENTITY (AREF (XCDR (obj), i))))
> > 	      break;
> >
> > 	  if (i == size)  <<<<<<<<<<<<<<<<<<<<<
> > 	    drop = 1;
> > 	}
> >
> > with a condition i != size, that breakpoint never breaks.
> 
> The best way to hit this breakpoint is to run something like:
> 
> (defun bloat-font ()
>    (interactive)
>    (let ((fonts (x-list-fonts "*")))
>      (while fonts
>        (condition-case nil (set-frame-font (car fonts)) (error nil))
>        (setq fonts (cdr fonts))
>        (redisplay))))
> 
> (This is X-specific; I believe there should be a similar MS-Windows method).

What is X-specific here?  If you mean x-list-fonts, then it exists on
Windows and does the same.

Anyway, I think we are mis-communicating again: the above Lisp code
will cause the breakpoint to fire with i == size, i.e. we will remove
the font-entities from the cache.  I wanted the opposite: to see at
least 1 font-entity that is NOT removed.  So my breakpoint condition
was "i != size", which means we will NOT drop the font-entity.  What I
saw is that this condition is never true, which means we remove ALL of
the font-entities from the cache.

> Subtle "triangle example" works just fine for me (with default font used by
> GTK3 build and 'emacs -Q' at least).

If the font cache is not cleared, you will not see any problem at all.

> Also I'm curious how to reproduce this issue with X. In particular, how to
> find a font so "heavy" that an attempt to load it creates a lot of Lisp data
> (hundreds Kbytes, enough to reach gc_cons_threshold each time)?

Well, you could lower gc_cons_threshold instead, couldn't you?

bug#15876: 24.3.50; Highly degraded performance between rev 114715 and 115006

[Eli Zaretskii]

> Date: Mon, 02 Dec 2013 15:43:21 +0400
> From: Dmitry Antipov <dmantipov@yandex.ru>
> CC: 15876@debbugs.gnu.org
> 
> BTW, I have grabbed BatangChe font from MS-Windows machine and tried:
> 
> ./src/emacs -Q -font '-hanyang system-batangche-medium-r-normal--0-0-0-0-m-0-iso10646-1'
> 
> This looks extremely ugly

Relax, that font is only used for the triangle, not for the text.

> but basic editing around "magic triangle" works without any visible
> slowdown.

Again, if you don't see the font caches being GC'ed, you will not
notice any slowdown.

Can you describe the "life cycle" of the font cache?  All I see is
that it is consed in font.c, which also maintains a reference counter
for each driver-type, and that's it.  All the rest is left to the font
driver, but I see no code that records the font caches anywhere,
except in dpyinfo->name_list_element, which does not seem to be part
of any Lisp object visible to GC.

If the above is correct, then how can we expect the font caches that
are still in use to be marked?  What am I missing?

bug#15876: 24.3.50; Highly degraded performance between rev 114715 and 115006

[Dmitry Antipov]

On 12/02/2013 09:52 PM, Eli Zaretskii wrote:

 > Thanks.  But I think I didn't make myself clear: the issue is not just
 > to see ANY font-spec objects being marked.  The issue is with those
 > font-spec objects that are recorded in the font caches that are
 > compacted by compact_font_caches.  I don't see any code that makes
 > sure some Lisp object references those caches.  Without that, they
 > cannot be possibly marked, and will be GC'ed, right?

Yes, but this is somewhat similar to markers.  All markers are chained
via 'next' pointers into per-buffer lists, but there is no guarantee that
_each_ marker is referenced in some other way.  Instead, the marker _may_
be referenced from some other object.  If this is so, the marker is live.
Otherwise it's dead.

 >> But the most of font-spec and font-entity objects are referenced via
 >> staticpro'ed globals Vfontset_table and ft_face_cache.
 >
 > Those staticpro'ed objects might just be the reason why you don't see
 > the problem.  Your build uses the ftfont driver, doesn't it?  Because
 > ftfont.c has this implementation of the get_cache method:
 >
 >    static Lisp_Object
 >    ftfont_get_cache (struct frame *f)
 >    {
 >      return freetype_font_cache;
 >    }
 >
 > and freetype_font_cache is a staticpro'ed variable.  By contrast,
 > w32font.c does this:
 >
 >    Lisp_Object
 >    w32font_get_cache (struct frame *f)
 >    {
 >      struct w32_display_info *dpyinfo = FRAME_DISPLAY_INFO (f);
 >
 >      return (dpyinfo->name_list_element);
 >    }
 >
 > and dpyinfo->name_list_element is what compact_font_caches examines.

But Vfontset_table should present on MS-Windows too, isn't it?

 > Anyway, I think we are mis-communicating again: the above Lisp code
 > will cause the breakpoint to fire with i == size, i.e. we will remove
 > the font-entities from the cache.  I wanted the opposite: to see at
 > least 1 font-entity that is NOT removed.  So my breakpoint condition
 > was "i != size", which means we will NOT drop the font-entity.  What I
 > saw is that this condition is never true, which means we remove ALL of
 > the font-entities from the cache.

I understand the problem, but (as of r115362) I'm able to hit the breakpoint
on the 'break' statement before 'if (i == size)' and found marked font-entity:

...
Breakpoint 1, compact_font_cache_entry (entry=...) at ../../trunk/src/alloc.c:5327
5327		      break;
(gdb) bt 4
#0  compact_font_cache_entry (entry=...) at ../../trunk/src/alloc.c:5327
#1  0x00000000005ea772 in compact_font_caches () at ../../trunk/src/alloc.c:5357
#2  0x00000000005ead99 in Fgarbage_collect () at ../../trunk/src/alloc.c:5531
#3  0x00000000005635d8 in maybe_gc () at ../../trunk/src/lisp.h:4462
(More stack frames follow...)
(gdb) p i
$1 = 0
(gdb) p size
$2 = 4
(gdb) p XFONT_ENTITY (AREF (XCDR (obj), i))
$3 = (struct font_entity *) 0x12fd578
(gdb) p /x XFONT_ENTITY (AREF (XCDR (obj), i))->header.size & ARRAY_MARK_FLAG
$4 = 0x8000000000000000	      ;;; non-zero ==> mark bit is set
...

Since this font-entity is marked, the whole cache entry is not removed.

I'll try to trace marking and find an object which references this font-entity.

Dmitry

bug#15876: 24.3.50; Highly degraded performance between rev 114715 and 115006

[Eli Zaretskii]

> Date: Tue, 03 Dec 2013 13:57:01 +0400
> From: Dmitry Antipov <dmantipov@yandex.ru>
> CC: sva-news@mygooglest.com, 15876@debbugs.gnu.org
> 
> On 12/02/2013 09:52 PM, Eli Zaretskii wrote:
> 
>  > Thanks.  But I think I didn't make myself clear: the issue is not just
>  > to see ANY font-spec objects being marked.  The issue is with those
>  > font-spec objects that are recorded in the font caches that are
>  > compacted by compact_font_caches.  I don't see any code that makes
>  > sure some Lisp object references those caches.  Without that, they
>  > cannot be possibly marked, and will be GC'ed, right?
> 
> Yes, but this is somewhat similar to markers.  All markers are chained
> via 'next' pointers into per-buffer lists, but there is no guarantee that
> _each_ marker is referenced in some other way.  Instead, the marker _may_
> be referenced from some other object.  If this is so, the marker is live.
> Otherwise it's dead.

OK, but the difference is that a buffer is a Lisp object that is
examined by GC, and so its marker list is examined.  Where's something
similar for the font caches?

> But Vfontset_table should present on MS-Windows too, isn't it?

It is present, but how is it related to the issue at hand?

>  > Anyway, I think we are mis-communicating again: the above Lisp code
>  > will cause the breakpoint to fire with i == size, i.e. we will remove
>  > the font-entities from the cache.  I wanted the opposite: to see at
>  > least 1 font-entity that is NOT removed.  So my breakpoint condition
>  > was "i != size", which means we will NOT drop the font-entity.  What I
>  > saw is that this condition is never true, which means we remove ALL of
>  > the font-entities from the cache.
> 
> I understand the problem, but (as of r115362) I'm able to hit the breakpoint
> on the 'break' statement before 'if (i == size)' and found marked font-entity:
> 
> ...
> Breakpoint 1, compact_font_cache_entry (entry=...) at ../../trunk/src/alloc.c:5327
> 5327		      break;
> (gdb) bt 4
> #0  compact_font_cache_entry (entry=...) at ../../trunk/src/alloc.c:5327
> #1  0x00000000005ea772 in compact_font_caches () at ../../trunk/src/alloc.c:5357
> #2  0x00000000005ead99 in Fgarbage_collect () at ../../trunk/src/alloc.c:5531
> #3  0x00000000005635d8 in maybe_gc () at ../../trunk/src/lisp.h:4462
> (More stack frames follow...)
> (gdb) p i
> $1 = 0
> (gdb) p size
> $2 = 4
> (gdb) p XFONT_ENTITY (AREF (XCDR (obj), i))
> $3 = (struct font_entity *) 0x12fd578
> (gdb) p /x XFONT_ENTITY (AREF (XCDR (obj), i))->header.size & ARRAY_MARK_FLAG
> $4 = 0x8000000000000000	      ;;; non-zero ==> mark bit is set
> ...
> 
> Since this font-entity is marked, the whole cache entry is not removed.

This never happened to me on Windows.

bug#15876: 24.3.50; Highly degraded performance between rev 114715 and 115006

[Dmitry Antipov]

Oops.

According to comments at the beginning of font.h, FONT_ENTITY_INDEX slot
of font object should reference font-entity from which this font object
is opened.  But now I'm seeing that this slot is always Qnil.  I.e. the
following patch should help:

=== modified file 'src/font.c'
--- src/font.c	2013-10-29 16:11:50 +0000
+++ src/font.c	2013-12-03 12:10:28 +0000
@@ -196,6 +196,7 @@
        if (! NILP (AREF (entity, FONT_EXTRA_INDEX)))
  	font->props[FONT_EXTRA_INDEX]
  	  = Fcopy_alist (AREF (entity, FONT_EXTRA_INDEX));
+      font->props[FONT_ENTITY_INDEX] = entity;
      }
    if (size > 0)
      font->props[FONT_SIZE_INDEX] = make_number (pixelsize);

With this patch applied, I can hit 'break' at alloc.c:5327 even if
configured with --without-xft; this should help MS-Windows build too.

(Just for fun: font entity recording was introduced in r100788 but
reverted in r100814 due to OTF handling changes.  How dumb.)

Dmitry

bug#15876: 24.3.50; Highly degraded performance between rev 114715 and 115006

[Eli Zaretskii]

> Date: Tue, 03 Dec 2013 19:09:10 +0400
> From: Dmitry Antipov <dmantipov@yandex.ru>
> CC: sva-news@mygooglest.com, 15876@debbugs.gnu.org
> 
> According to comments at the beginning of font.h, FONT_ENTITY_INDEX slot
> of font object should reference font-entity from which this font object
> is opened.  But now I'm seeing that this slot is always Qnil.  I.e. the
> following patch should help:
> 
> === modified file 'src/font.c'
> --- src/font.c	2013-10-29 16:11:50 +0000
> +++ src/font.c	2013-12-03 12:10:28 +0000
> @@ -196,6 +196,7 @@
>         if (! NILP (AREF (entity, FONT_EXTRA_INDEX)))
>   	font->props[FONT_EXTRA_INDEX]
>   	  = Fcopy_alist (AREF (entity, FONT_EXTRA_INDEX));
> +      font->props[FONT_ENTITY_INDEX] = entity;
>       }
>     if (size > 0)
>       font->props[FONT_SIZE_INDEX] = make_number (pixelsize);
> 
> With this patch applied, I can hit 'break' at alloc.c:5327 even if
> configured with --without-xft; this should help MS-Windows build too.

Thanks.

Indeed, with this patch, I now see several font-entities spared from
being GC'ed.  Unfortunately, this doesn't solve the slow-down with
that character and font, while ifdef'ing away this fragment from
compact_font_caches:

      if (CONSP (cache))
	{
	  Lisp_Object entry;

	  for (entry = XCDR (cache); CONSP (entry); entry = XCDR (entry))
	    XSETCAR (entry, compact_font_cache_entry (XCAR (entry)));
	}

does solve it.

So there's some other factor at work here that causes us to drop
portions of the font cache that we shouldn't have.

What kind of information do I need to collect on Windows to make it
clear where the problem is?  Can you suggest some code that would
collect the necessary information, or ideas for writing such code?
(It's inconvenient to collect the data in GDB, since many objects are
marked at this point, and printing them from GDB crashes Emacs.)

bug#15876: 24.3.50; Highly degraded performance between rev 114715 and 115006

[Dmitry Antipov]

[-- Attachment #1: Type: text/plain, Size: 1889 bytes --]

On 12/04/2013 09:53 PM, Eli Zaretskii wrote:

 > What kind of information do I need to collect on Windows to make it
 > clear where the problem is?  Can you suggest some code that would
 > collect the necessary information, or ideas for writing such code?
 > (It's inconvenient to collect the data in GDB, since many objects are
 > marked at this point, and printing them from GDB crashes Emacs.)

OK, let's try the following steps.

1. Apply attached patch and rebuild.
2. Run emacs -Q, then M-x insert-char 25b7 and M-x describe-char.
    Now you should know the C pointer to font object which is used
    to display right-pointing triangle (underlined red on screenshot).
3. Insert GDB breakpoints to fprintf, move cursor, do some basic editing
    around the triangle and run M-x garbage-collect few times.  An
    interesting font object should be either marked or swept; if it's
    markerd, you should hit the breakpoint "GCX: mark interesting font Y".
    For my --without-xft build, M-x garbage-collect always shows that an
    interesting font object is marked in expected way - via face cache. E.g.:

(gdb) bt 8
#0  __fprintf (stream=0x3869dbb1e0 <_IO_2_1_stderr_>, format=0x6cf620 "GC%ld: mark interesting font %p\n") at fprintf.c:27
#1  0x00000000005c7269 in mark_vectorlike (ptr=0xd7c8b8) at ../../trunk/src/alloc.c:5745
#2  0x00000000005c75a4 in mark_face_cache (c=0x11456b0) at ../../trunk/src/alloc.c:5838
#3  0x00000000005c7b43 in mark_object (arg=...) at ../../trunk/src/alloc.c:6014
#4  0x00000000005c7288 in mark_vectorlike (ptr=0x127d4f8) at ../../trunk/src/alloc.c:5754
#5  0x00000000005c7baa in mark_object (arg=...) at ../../trunk/src/alloc.c:6031
#6  0x00000000005c7288 in mark_vectorlike (ptr=0xcf3230) at ../../trunk/src/alloc.c:5754
#7  0x00000000005c7457 in mark_buffer (buffer=0xcf3230) at ../../trunk/src/alloc.c:5805
(More stack frames follow...)

Dmitry


[-- Attachment #2: font-pointer.png --]
[-- Type: image/png, Size: 24250 bytes --]

[-- Attachment #3: font_debug_bug15876.patch --]
[-- Type: text/x-patch, Size: 2672 bytes --]

=== modified file 'src/alloc.c'
--- src/alloc.c	2013-12-01 22:33:13 +0000
+++ src/alloc.c	2013-12-05 06:10:49 +0000
@@ -2877,7 +2877,12 @@
   if (PSEUDOVECTOR_TYPEP (&vector->header, PVEC_FONT)
       && ((vector->header.size & PSEUDOVECTOR_SIZE_MASK)
 	  == FONT_OBJECT_MAX))
-    ((struct font *) vector)->driver->close ((struct font *) vector);
+    {
+      if (((struct font *) vector)->debug)
+	fprintf (stderr, "GC%ld: free interesting font %p\n",
+		 gcs_done, vector);
+      ((struct font *) vector)->driver->close ((struct font *) vector);
+    }
 }
 
 /* Reclaim space used by unmarked vectors.  */
@@ -5733,6 +5738,14 @@
   if (size & PSEUDOVECTOR_FLAG)
     size &= PSEUDOVECTOR_SIZE_MASK;
 
+  if (FONT_OBJECT_P (make_lisp_ptr (ptr, Lisp_Vectorlike)))
+    {
+      struct font *f = (struct font *) ptr;
+      if (f->debug)
+	fprintf (stderr, "GC%ld: mark interesting font %p\n",
+		 gcs_done, f);
+    }
+
   /* Note that this size is not the memory-footprint size, but only
      the number of Lisp_Object fields that we should trace.
      The distinction is used e.g. by Lisp_Process which places extra

=== modified file 'src/font.c'
--- src/font.c	2013-12-04 13:08:30 +0000
+++ src/font.c	2013-12-05 06:00:56 +0000
@@ -188,6 +188,7 @@
   int i;
 
   XSETFONT (font_object, font);
+  font->debug = 0;
 
   if (! NILP (entity))
     {
@@ -196,6 +197,7 @@
       if (! NILP (AREF (entity, FONT_EXTRA_INDEX)))
 	font->props[FONT_EXTRA_INDEX]
 	  = Fcopy_alist (AREF (entity, FONT_EXTRA_INDEX));
+      font->props[FONT_ENTITY_INDEX] = entity;
     }
   if (size > 0)
     font->props[FONT_SIZE_INDEX] = make_number (pixelsize);
@@ -4189,7 +4191,7 @@
 the consecutive wildcards are folded into one.  */)
   (Lisp_Object font, Lisp_Object fold_wildcards)
 {
-  char name[256];
+  char name[256 + 32];
   int namelen, pixel_size = 0;
 
   CHECK_FONT (font);
@@ -4202,7 +4204,11 @@
 	  && SDATA (font_name)[0] == '-')
 	{
 	  if (NILP (fold_wildcards))
-	    return font_name;
+	    {
+	      XFONT_OBJECT (font)->debug = 1;
+	      return make_formatted_string
+		(name, "%s (%p)", SSDATA (font_name), XFONT_OBJECT (font));
+	    }
 	  strcpy (name, SSDATA (font_name));
 	  namelen = SBYTES (font_name);
 	  goto done;

=== modified file 'src/font.h'
--- src/font.h	2013-12-04 13:35:41 +0000
+++ src/font.h	2013-12-05 05:59:31 +0000
@@ -285,6 +285,8 @@
 
   /* Beyond here, there should be no more Lisp_Object components.  */
 
+  int debug;
+
   /* Minimum and maximum glyph widths, in pixels.  Some font backends,
      such as xft, lack the information to easily compute minimum and
      maximum widths over all characters; in that case, these values

bug#15876: 24.3.50; Highly degraded performance between rev 114715 and 115006

[Eli Zaretskii]

> Date: Thu, 05 Dec 2013 10:29:45 +0400
> From: Dmitry Antipov <dmantipov@yandex.ru>
> CC: sva-news@mygooglest.com, 15876@debbugs.gnu.org
> 
> 1. Apply attached patch and rebuild.
> 2. Run emacs -Q, then M-x insert-char 25b7 and M-x describe-char.
>     Now you should know the C pointer to font object which is used
>     to display right-pointing triangle (underlined red on screenshot).
> 3. Insert GDB breakpoints to fprintf, move cursor, do some basic editing
>     around the triangle and run M-x garbage-collect few times.  An
>     interesting font object should be either marked or swept; if it's
>     markerd, you should hit the breakpoint "GCX: mark interesting font Y".
>     For my --without-xft build, M-x garbage-collect always shows that an
>     interesting font object is marked in expected way - via face cache. E.g.:
> 
> (gdb) bt 8
> #0  __fprintf (stream=0x3869dbb1e0 <_IO_2_1_stderr_>, format=0x6cf620 "GC%ld: mark interesting font %p\n") at fprintf.c:27
> #1  0x00000000005c7269 in mark_vectorlike (ptr=0xd7c8b8) at ../../trunk/src/alloc.c:5745
> #2  0x00000000005c75a4 in mark_face_cache (c=0x11456b0) at ../../trunk/src/alloc.c:5838
> #3  0x00000000005c7b43 in mark_object (arg=...) at ../../trunk/src/alloc.c:6014
> #4  0x00000000005c7288 in mark_vectorlike (ptr=0x127d4f8) at ../../trunk/src/alloc.c:5754
> #5  0x00000000005c7baa in mark_object (arg=...) at ../../trunk/src/alloc.c:6031
> #6  0x00000000005c7288 in mark_vectorlike (ptr=0xcf3230) at ../../trunk/src/alloc.c:5754
> #7  0x00000000005c7457 in mark_buffer (buffer=0xcf3230) at ../../trunk/src/alloc.c:5805
> (More stack frames follow...)

Thanks.

Yes, I see the same on Windows.  However, I'm not sure we are looking
at objects we should be looking at.  The ones your code traces are
font objects.  By contrast, the entries in the font caches are cons
cells like this:

   (font-spec . [font-entity1 font-entity2 ...])

IOW, I think we should be tracing font-specs and font-entities, not
font objects.

bug#15876: 24.3.50; Highly degraded performance between rev 114715 and 115006

[Dmitry Antipov]

[-- Attachment #1: Type: text/plain, Size: 3756 bytes --]

On 12/05/2013 09:36 PM, Eli Zaretskii wrote:

> Yes, I see the same on Windows.  However, I'm not sure we are looking
> at objects we should be looking at.  The ones your code traces are
> font objects.  By contrast, the entries in the font caches are cons
> cells like this:
>
>     (font-spec . [font-entity1 font-entity2 ...])
>
> IOW, I think we should be tracing font-specs and font-entities, not
> font objects.

IMO the real problem is that redisplay sometimes isn't clever enough about requesting
font for the particular character (0x25b7 in our case):

#0  font_make_entity () at ../../trunk/src/font.c:169
#1  0x00000000005a8a3f in xfont_list_pattern (display=display@entry=0x1379260,
     pattern=pattern@entry=0x7fffffff7730 "-adobe-courier-*-*-*--*-*-*-*-*-*-iso10646-1", registry=registry@entry=12260082,
     script=script@entry=12084946) at ../../trunk/src/xfont.c:403
#2  0x00000000005a9059 in xfont_list (f=<optimized out>, spec=12183525) at ../../trunk/src/xfont.c:515
#3  0x0000000000563193 in font_list_entities (f=f@entry=0x111fd58, spec=spec@entry=15986301) at ../../trunk/src/font.c:2735
#4  0x0000000000563b42 in font_find_for_lface (f=f@entry=0x111fd58, attrs=attrs@entry=0x16b9510, spec=<optimized out>,
     c=c@entry=9655) at ../../trunk/src/font.c:3206
#5  0x00000000005a9d44 in fontset_find_font (fontset=18338269, c=c@entry=9655, face=face@entry=0x16b9510, id=id@entry=-1,
     fallback=fallback@entry=false) at ../../trunk/src/fontset.c:681
#6  0x00000000005aa210 in fontset_font (fontset=fontset@entry=17988749, c=c@entry=9655, face=face@entry=0x16b9510, id=-1)
     at ../../trunk/src/fontset.c:754
#7  0x00000000005aacb2 in face_for_char (f=0x111fd58, face=face@entry=0x16b9510, c=9655, pos=<optimized out>, object=<optimized out>)
     at ../../trunk/src/fontset.c:978
#8  0x000000000043cb45 in get_next_display_element (it=it@entry=0x7fffffff97f0) at ../../trunk/src/xdisp.c:6997
#9  0x0000000000441bc3 in display_line (it=it@entry=0x7fffffff97f0) at ../../trunk/src/xdisp.c:19593
#10 0x000000000044506a in try_window (window=window@entry=17964077, pos=..., flags=flags@entry=1) at ../../trunk/src/xdisp.c:16505
#11 0x000000000045ab57 in redisplay_window (window=17964077, just_this_one_p=just_this_one_p@entry=false)
     at ../../trunk/src/xdisp.c:16022
#12 0x000000000045cbf3 in redisplay_window_0 (window=window@entry=17964077) at ../../trunk/src/xdisp.c:14023
#13 0x000000000054d216 in internal_condition_case_1 (bfun=bfun@entry=0x45cbc0 <redisplay_window_0>, arg=17964077,
     handlers=<optimized out>, hfun=hfun@entry=0x4290a0 <redisplay_window_error>) at ../../trunk/src/eval.c:1368
#14 0x000000000042d8ce in redisplay_windows (window=17964077) at ../../trunk/src/xdisp.c:14003
#15 0x000000000044a121 in redisplay_internal () at ../../trunk/src/xdisp.c:13602

Such a request produces a lot of font-entity objects, which are not needed (and becomes
reachable only via font cache) after the "best match" font is selected for the particular
character. Next, if font loading is too memory-intensive and gc_cons_threshold was hit, GC
clears font cache. Next cursor movement triggers redisplay, which in turn asks for the "best
match" for 0x25b7 again. If font cache is never cleared, this is acceptable because all possible
matches (represented as a vector of font-entities) are cached. But, if font cache is cleared,
font_list_entities calls to driver->list function, which creates a lot of font-entities again,
etc., etc.

Until we have somewhat smarter redisplay, possible solution is to clear font cache when it
becomes larger than some specified size rather than at each GC. Now I have the virtual machine
with Windows 7 installed, and this fix looks reasonable for me. Could you also please try it?

Dmitry


[-- Attachment #2: font_cache_size_bug15876.patch --]
[-- Type: text/x-patch, Size: 5247 bytes --]

=== modified file 'src/alloc.c'
--- src/alloc.c	2013-12-09 08:23:01 +0000
+++ src/alloc.c	2013-12-11 05:38:41 +0000
@@ -5299,17 +5299,22 @@
 
 #ifdef HAVE_WINDOW_SYSTEM
 
+/* Rather arbitrary but willing to fix Bug#15876.  */
+
+#define FONT_CACHE_THRESHOLD 4096
+
 /* Remove unmarked font-spec and font-entity objects from ENTRY, which is
    (DRIVER-TYPE NUM-FRAMES FONT-CACHE-DATA ...), and return changed entry.  */
 
 static Lisp_Object
-compact_font_cache_entry (Lisp_Object entry)
+compact_font_cache_entry (struct terminal *t, Lisp_Object entry)
 {
   Lisp_Object tail, *prev = &entry;
 
   for (tail = entry; CONSP (tail); tail = XCDR (tail))
     {
       bool drop = 0;
+      ptrdiff_t size = 0;
       Lisp_Object obj = XCAR (tail);
 
       /* Consider OBJ if it is (font-spec . [font-entity font-entity ...]).  */
@@ -5317,8 +5322,9 @@
 	  && !VECTOR_MARKED_P (XFONT_SPEC (XCAR (obj)))
 	  && VECTORP (XCDR (obj)))
 	{
-	  ptrdiff_t i, size = ASIZE (XCDR (obj)) & ~ARRAY_MARK_FLAG;
+	  ptrdiff_t i;
 
+	  size = ASIZE (XCDR (obj)) & ~ARRAY_MARK_FLAG;
 	  /* If font-spec is not marked, most likely all font-entities
 	     are not marked too.  But we must be sure that nothing is
 	     marked within OBJ before we really drop it.  */
@@ -5330,7 +5336,11 @@
 	    drop = 1;
 	}
       if (drop)
-	*prev = XCDR (tail);
+	{
+	  *prev = XCDR (tail);
+	  /* Count font-spec and vector of font-entities.  */
+	  FONT_CACHE_SIZE (t) -= (size + 1);
+	}
       else
 	prev = xcdr_addr (tail);
     }
@@ -5351,10 +5361,20 @@
 
       if (CONSP (cache))
 	{
-	  Lisp_Object entry;
+	  eassert (FONT_CACHE_SIZE (t) >= 0);
+	  if (FONT_CACHE_SIZE (t) > FONT_CACHE_THRESHOLD)
+	    {
+	      Lisp_Object entry;
 
-	  for (entry = XCDR (cache); CONSP (entry); entry = XCDR (entry))
-	    XSETCAR (entry, compact_font_cache_entry (XCAR (entry)));
+	      fprintf (stderr, "GC%ld: font cache compaction %ld -> ",
+		       gcs_done, FONT_CACHE_SIZE (t));
+	      for (entry = XCDR (cache); CONSP (entry); entry = XCDR (entry))
+		XSETCAR (entry, compact_font_cache_entry (t, XCAR (entry)));
+	      fprintf (stderr, "%ld\n", FONT_CACHE_SIZE (t));
+	    }
+	  else
+	    fprintf (stderr, "GC%ld: font cache too small (%d <= %d)\n",
+		     gcs_done, FONT_CACHE_SIZE (t), FONT_CACHE_THRESHOLD);
 	}
       mark_object (cache);
     }

=== modified file 'src/font.c'
--- src/font.c	2013-12-10 03:36:36 +0000
+++ src/font.c	2013-12-11 05:35:25 +0000
@@ -2740,6 +2740,8 @@
 	    copy = copy_font_spec (scratch_font_spec);
 	    ASET (copy, FONT_TYPE_INDEX, driver_list->driver->type);
 	    XSETCDR (cache, Fcons (Fcons (copy, val), XCDR (cache)));
+	    /* Count font-spec and a vector of font-entities.  */
+	    FONT_CACHE_SIZE (f->terminal) += ASIZE (val) + 1;
 	  }
 	if (ASIZE (val) > 0
 	    && (need_filtering
@@ -2793,6 +2795,8 @@
 	    copy = copy_font_spec (work);
 	    ASET (copy, FONT_TYPE_INDEX, driver_list->driver->type);
 	    XSETCDR (cache, Fcons (Fcons (copy, entity), XCDR (cache)));
+	    /* Count font-spec and entity.  */
+	    FONT_CACHE_SIZE (f->terminal) += 2;
 	  }
 	if (! NILP (entity))
 	  break;

=== modified file 'src/nsterm.h'
--- src/nsterm.h	2013-12-07 16:48:12 +0000
+++ src/nsterm.h	2013-12-11 05:35:25 +0000
@@ -560,6 +560,9 @@
   /* This is a cons cell of the form (NAME . FONT-LIST-CACHE).  */
   Lisp_Object name_list_element;
 
+  /* Amount of font-entities and font-specs objects in the cache above.  */
+  ptrdiff_t font_cache_size;
+
   /* The number of fonts loaded. */
   int n_fonts;
 

=== modified file 'src/termhooks.h'
--- src/termhooks.h	2013-10-18 12:57:44 +0000
+++ src/termhooks.h	2013-12-11 05:35:25 +0000
@@ -628,12 +628,15 @@
 #if defined (HAVE_X_WINDOWS)
 #define TERMINAL_FONT_CACHE(t)						\
   (t->type == output_x_window ? t->display_info.x->name_list_element : Qnil)
+#define FONT_CACHE_SIZE(t) (t)->display_info.x->font_cache_size
 #elif defined (HAVE_NTGUI)
 #define TERMINAL_FONT_CACHE(t)						\
   (t->type == output_w32 ? t->display_info.w32->name_list_element : Qnil)
+#define FONT_CACHE_SIZE(t) (t)->display_info.w32->font_cache_size
 #elif defined (HAVE_NS)
 #define TERMINAL_FONT_CACHE(t)						\
   (t->type == output_ns ? t->display_info.ns->name_list_element : Qnil)
+#define FONT_CACHE_SIZE(t) (t)->display_info.ns->font_cache_size
 #endif
 
 extern struct terminal *get_terminal (Lisp_Object terminal, bool);

=== modified file 'src/w32term.h'
--- src/w32term.h	2013-12-02 13:35:53 +0000
+++ src/w32term.h	2013-12-11 05:35:25 +0000
@@ -74,6 +74,9 @@
   /* This is a cons cell of the form (NAME . FONT-LIST-CACHE).  */
   Lisp_Object name_list_element;
 
+  /* Amount of font-entities and font-specs objects in the cache above.  */
+  ptrdiff_t font_cache_size;
+
   /* Number of frames that are on this display.  */
   int reference_count;
 

=== modified file 'src/xterm.h'
--- src/xterm.h	2013-12-07 23:04:10 +0000
+++ src/xterm.h	2013-12-11 05:35:25 +0000
@@ -140,6 +140,9 @@
   /* This is a cons cell of the form (NAME . FONT-LIST-CACHE).  */
   Lisp_Object name_list_element;
 
+  /* Amount of font-entities and font-specs objects in the cache above.  */
+  ptrdiff_t font_cache_size;
+
   /* Number of frames that are on this display.  */
   int reference_count;
 

bug#15876: [SPAM] bug#15876: 24.3.50; Highly degraded performance between rev 114715 and 115006

[Jarek Czekalski]

W dniu 2013-12-11 07:52, Dmitry Antipov pisze:
> On 12/05/2013 09:36 PM, Eli Zaretskii wrote:
> Next cursor movement triggers redisplay, which in turn asks for the "best
> match" for 0x25b7 again.
>

It would be nice if the results of the "best match" calls were 
remembered. Assuming that the fonts don't change during Emacs run time.

Jarek

bug#15876: 24.3.50; Highly degraded performance between rev 114715 and 115006

[Dmitry Antipov]

On 12/11/2013 11:16 AM, Jarek Czekalski wrote:

> It would be nice if the results of the "best match" calls were remembered.

Sure. But currently font cache remembers not the "best match" X but
the vector [X0 X1 ... Xn] which was used to select X for the last time.
Under some circumstances (in particular, when a lot of fonts are loaded),
font cache tends to grow too much (see
http://lists.gnu.org/archive/html/emacs-devel/2013-10/msg00740.html).

> Assuming that the fonts don't change during Emacs run time.

This assumption is wrong. Although explicit font change is rarely done by the
most of users, new fonts may be loaded quite often - to display an "unusual"
character which has no glyph in current font, to display something under
M-x customize, etc.

Dmitry

bug#15876: 24.3.50; Highly degraded performance between rev 114715 and 115006

[Eli Zaretskii]

> Date: Wed, 11 Dec 2013 10:52:24 +0400
> From: Dmitry Antipov <dmantipov@yandex.ru>
> CC: sva-news@mygooglest.com
> 
> IMO the real problem is that redisplay sometimes isn't clever enough about requesting
> font for the particular character (0x25b7 in our case):

Not sure if this is for redisplay to solve.  Redisplay just needs a
face for displaying a character, it knows (almost) nothing about fonts
and fontsets.  It's the font selection machinery that needs to become
smarter, I think.

> Such a request produces a lot of font-entity objects, which are not needed (and becomes
> reachable only via font cache) after the "best match" font is selected for the particular
> character. Next, if font loading is too memory-intensive and gc_cons_threshold was hit, GC
> clears font cache. Next cursor movement triggers redisplay, which in turn asks for the "best
> match" for 0x25b7 again. If font cache is never cleared, this is acceptable because all possible
> matches (represented as a vector of font-entities) are cached. But, if font cache is cleared,
> font_list_entities calls to driver->list function, which creates a lot of font-entities again,
> etc., etc.

Right, this matches my observations.  Did you succeed in finding which
function in this call sequence is the performance bottleneck?

> Until we have somewhat smarter redisplay, possible solution is to clear font cache when it
> becomes larger than some specified size rather than at each GC. Now I have the virtual machine
> with Windows 7 installed, and this fix looks reasonable for me. Could you also please try it?

It solves the immediate problem with this single character, but if I
add a few others from other Unicode blocks, the problem reappears.
This is because the threshold of 4096 seems to be too low: I've seen
the number go above 10K a couple of times.

Anyway, what about the patch below?  With it, the problem disappears
even without your "threshold" based GC.

We could be even more aggressive, and search also the face caches on
all frames (but then, if found, the face needs to be cached on the
current frame as well, before using it) -- this is left as an
exercise ;-)

The problem with this approach is that there's no guarantee we will
find a "best-match" font for a character.  However, I displayed
etc/HELLO with and without the patch, and didn't see any difference.
Maybe some font selection expert could tell if we lose anything
important by going this way.  Note that NS was already reusing the
argument FACE (but it alone) in this way.


--- src/fontset.c~	2013-11-05 06:42:21.000000000 +0200
+++ src/fontset.c	2013-12-11 11:30:03.909213300 +0200
@@ -937,7 +937,6 @@
   if (ASCII_CHAR_P (c) || face->fontset < 0)
     return face->ascii_face->id;
 
-#ifdef HAVE_NS
   if (face->font)
     {
       /* Fonts often have characters in other scripts, like symbol, even if they
@@ -946,9 +945,22 @@
          perhaps be general?  */
       Lisp_Object font_object;
       XSETFONT (font_object, face->font);
-      if (font_has_char (f, font_object, c)) return face->id;
+      if (font_has_char (f, font_object, c))
+	return face->id;
+    }
+
+  for (face_id = 0; face_id < FRAME_FACE_CACHE (f)->used; face_id++)
+    {
+      struct face *this_face = FACE_FROM_ID (f, face_id);
+
+      if (this_face != face && this_face->font)
+	{
+	  Lisp_Object font_object;
+	  XSETFONT (font_object, this_face->font);
+	  if (font_has_char (f, font_object, c))
+	    return this_face->id;
+	}
     }
-#endif
 
   eassert (fontset_id_valid_p (face->fontset));
   fontset = FONTSET_FROM_ID (face->fontset);

bug#15876: 24.3.50; Highly degraded performance between rev 114715 and 115006

[Dmitry Antipov]

On 12/11/2013 08:28 PM, Eli Zaretskii wrote:

> Right, this matches my observations.  Did you succeed in finding which
> function in this call sequence is the performance bottleneck?

Usually this is font_driver->list (xfont_list, w32font_list, etc), which
can create a lot of font-entity objects.

> Anyway, what about the patch below?  With it, the problem disappears
> even without your "threshold" based GC.

Looks good. But can we assume that FACE_FROM_ID (...) is always non-NULL?
Some important routines, like mark_face_cache, do not rely on this.

Dmitry

bug#15876: 24.3.50; Highly degraded performance between rev 114715 and 115006

[Eli Zaretskii]

> Date: Wed, 11 Dec 2013 22:00:48 +0400
> From: Dmitry Antipov <dmantipov@yandex.ru>
> CC: 15876@debbugs.gnu.org, sva-news@mygooglest.com
> 
> > Anyway, what about the patch below?  With it, the problem disappears
> > even without your "threshold" based GC.
> 
> Looks good. But can we assume that FACE_FROM_ID (...) is always non-NULL?
> Some important routines, like mark_face_cache, do not rely on this.

I think you are right, and we should make sure it's non-NULL before
dereferencing.

Jan, Ken'ichi, would you please comment on this?  Are we losing
something by reusing already loaded fonts that support a given
character, as opposed to looking for the "best-match" font?

bug#15876: 24.3.50; Highly degraded performance between rev 114715 and 115006

[Jan Djärv]

Hello.

11 dec 2013 kl. 19:12 skrev Eli Zaretskii <eliz@gnu.org>:

>> Date: Wed, 11 Dec 2013 22:00:48 +0400
>> From: Dmitry Antipov <dmantipov@yandex.ru>
>> CC: 15876@debbugs.gnu.org, sva-news@mygooglest.com
>> 
>>> Anyway, what about the patch below?  With it, the problem disappears
>>> even without your "threshold" based GC.
>> 
>> Looks good. But can we assume that FACE_FROM_ID (...) is always non-NULL?
>> Some important routines, like mark_face_cache, do not rely on this.
> 
> I think you are right, and we should make sure it's non-NULL before
> dereferencing.
> 
> Jan, Ken'ichi, would you please comment on this?  Are we losing
> something by reusing already loaded fonts that support a given
> character, as opposed to looking for the "best-match" font?

See discussion starting here http://debbugs.gnu.org/cgi/bugreport.cgi?bug=15138#29:

Kenichi Handa wrote:

I agree that this change improves font selection for
symbols, but it's not good for many scripts for which just
having a glyph is not enough.  For instance, if the default
font has Hindi glyphs but doesn't have the OTF features for
Hindi script, we must find another proper font for Hindi.

How about modifying the current fontset mechanism as this?

(1) Allow t for FONT-SPEC of set-fontset-font to tell that
    the default font should be tried.
(2) Modiyf the default fontset to include `t' as the
    font-spec for scripts/characters for which the default
    font is ok.

	Jan D.

bug#15876: 24.3.50; Highly degraded performance between rev 114715 and 115006

[Eli Zaretskii]

> From: Jan Djärv <jan.h.d@swipnet.se>
> Date: Wed, 11 Dec 2013 20:50:12 +0100
> Cc: Dmitry Antipov <dmantipov@yandex.ru>,
>  Kenichi Handa <handa@gnu.org>,
>  15876@debbugs.gnu.org,
>  sva-news@mygooglest.com
> 
> > Jan, Ken'ichi, would you please comment on this?  Are we losing
> > something by reusing already loaded fonts that support a given
> > character, as opposed to looking for the "best-match" font?
> 
> See discussion starting here http://debbugs.gnu.org/cgi/bugreport.cgi?bug=15138#29:
> 
> Kenichi Handa wrote:
> 
> I agree that this change improves font selection for
> symbols, but it's not good for many scripts for which just
> having a glyph is not enough.  For instance, if the default
> font has Hindi glyphs but doesn't have the OTF features for
> Hindi script, we must find another proper font for Hindi.
> 
> How about modifying the current fontset mechanism as this?
> 
> (1) Allow t for FONT-SPEC of set-fontset-font to tell that
>     the default font should be tried.
> (2) Modiyf the default fontset to include `t' as the
>     font-spec for scripts/characters for which the default
>     font is ok.

I see.  But then why does the NS build still use a variant of that
approach?  How is it exempt from what Handa-san wrote above?

Anyway, the more I look into this problem, the more I'm beginning to
think that we should simply revert this font-cache compacting.  Here's
my reasoning:

 . The latest approach suggested by Dmitry (before my suggestion) was
   to use some more or less arbitrary threshold to decide whether or
   not to compact the font caches.  I tried playing with that
   approach, and found that I need to bump the threshold to something
   like 30K, before I get reasonable performance in a buffer full of
   unusual characters.

 . But raising the threshold higher simply means we defer the
   compaction more and more, to the point where we have no compaction
   at all.  IOW, doing this is equivalent to deleting that code
   altogether.

 . I also tried a different approach: compact only those font-entities
   that don't match any font that is still used by some face on some
   frame.  (I used font_match_p for that.)  This seems to work, except
   that font_match_p is evidently not safe enough to be used in the
   middle of GC -- I got crashes a few times.  When it did work, it
   again prevented the compaction, as if the compacting code were not
   there at all.

So let me turn the table and ask what do we gain by this compaction?
The original motivation was here:

  http://lists.gnu.org/archive/html/emacs-devel/2013-10/msg00740.html

but it was mainly about not releasing the fonts.  With the current
trunk, if I run that bloat-font function, after disabling the
compaction code, I see only a small increase in the memory footprint,
something like 30MB, at least on Windows.  Do you see something
different on X?

If the growth of the memory footprint is small even in such an extreme
situation, then perhaps we don't need this compaction?

bug#15876: 24.3.50; Highly degraded performance between rev 114715 and 115006

[Dmitry Antipov]

On 12/13/2013 07:22 PM, Eli Zaretskii wrote:

> So let me turn the table and ask what do we gain by this compaction?
> The original motivation was here:
>
>    http://lists.gnu.org/archive/html/emacs-devel/2013-10/msg00740.html
>
> but it was mainly about not releasing the fonts.  With the current
> trunk, if I run that bloat-font function, after disabling the
> compaction code, I see only a small increase in the memory footprint,
> something like 30MB, at least on Windows.  Do you see something
> different on X?

In that e-mail, I reported about ~360M RSS usage reduced to ~150M.
In http://lists.gnu.org/archive/html/emacs-devel/2013-10/msg00750.html,
Stefan has confirmed ~300M RSS usage.  Although things like bloat-font
aren't typical use cases, holding ~200M which we can't reuse is worth
trying to fix, IMHO.

Dmitry

bug#15876: 24.3.50; Highly degraded performance between rev 114715 and 115006

[Stefan Monnier]

> aren't typical use cases, holding ~200M which we can't reuse is worth
> trying to fix, IMHO.

Actually, we *can* reuse that memory, but only by selecting those
fonts again.  I agree with Eli: the benefit is very small.


        Stefan

bug#15876: 24.3.50; Highly degraded performance between rev 114715 and 115006

[Stefan Monnier]

>  . I also tried a different approach: compact only those font-entities
>    that don't match any font that is still used by some face on some
>    frame.  (I used font_match_p for that.)  This seems to work, except
>    that font_match_p is evidently not safe enough to be used in the
>    middle of GC -- I got crashes a few times.  When it did work, it
>    again prevented the compaction, as if the compacting code were not
>    there at all.

Maybe a better way goes along the lines of:
- every time we use a font, we set an "in_use" bit.
- every now and then (ha ha!), we let the GC collect fonts that don't
  have their "in_use" bit set.
The now-and-then could be something as "at least N seconds passed and
we've done at least M garbage collections".

Still, the effort doesn't seem worth the trouble, since the benefit is
too small (only appears in pathological cases).


        Stefan

bug#15876: 24.3.50; Highly degraded performance between rev 114715 and 115006

[Eli Zaretskii]

> Date: Fri, 13 Dec 2013 20:12:59 +0400
> From: Dmitry Antipov <dmantipov@yandex.ru>
> CC: sva-news@mygooglest.com, 15876@debbugs.gnu.org
> 
> >    http://lists.gnu.org/archive/html/emacs-devel/2013-10/msg00740.html
> >
> > but it was mainly about not releasing the fonts.  With the current
> > trunk, if I run that bloat-font function, after disabling the
> > compaction code, I see only a small increase in the memory footprint,
> > something like 30MB, at least on Windows.  Do you see something
> > different on X?
> 
> In that e-mail, I reported about ~360M RSS usage reduced to ~150M.
> In http://lists.gnu.org/archive/html/emacs-devel/2013-10/msg00750.html,
> Stefan has confirmed ~300M RSS usage.  Although things like bloat-font
> aren't typical use cases, holding ~200M which we can't reuse is worth
> trying to fix, IMHO.

Yes, but to fix that you installed more than just the compacting code.

In any case, if without compact_font_caches you still have such a
large increase in the memory footprint, it seems like that problem is
specific to X, while on Windows the cure is worse than the disease.
So if no other solution presents itself, I'm inclined to ifdef away
that code for Windows, if you don't mind.

bug#15876: 24.3.50; Highly degraded performance between rev 114715 and 115006

[Eli Zaretskii]

> From: Stefan Monnier <monnier@IRO.UMontreal.CA>
> Cc: Eli Zaretskii <eliz@gnu.org>,
>         Jan Djärv
>  <jan.h.d@swipnet.se>,
>         sva-news@mygooglest.com, 15876@debbugs.gnu.org
> Date: Fri, 13 Dec 2013 11:45:15 -0500
> 
> > aren't typical use cases, holding ~200M which we can't reuse is worth
> > trying to fix, IMHO.
> 
> Actually, we *can* reuse that memory, but only by selecting those
> fonts again.  I agree with Eli: the benefit is very small.

The problem is not with fonts, it is with font-entities stored in a
font cache.  For some reason that I cannot understand, Emacs needs to
consult the font cache when it looks for a font suitable for
displaying a character, even when the character was already displayed,
and the face (including the font) used for its display was already
realized and cached in the frame's face cache.  I don't understand the
font selection machinery enough to see why is that necessary.  But the
upshot of all this is that the face and the font do exist, but the
face-entities for that font are GC'ed, and that triggers the whole
process of discovering the available fonts in the available fontsets
again, each time Emacs displays the character.  So if that discovery
process conses a lot, it is likely to trigger GC, which will discard
the font-entities, etc. etc.

bug#15876: 24.3.50; Highly degraded performance between rev 114715 and 115006

[Eli Zaretskii]

> From: Stefan Monnier <monnier@IRO.UMontreal.CA>
> Cc: Jan Djärv <jan.h.d@swipnet.se>,
>         sva-news@mygooglest.com, 15876@debbugs.gnu.org, dmantipov@yandex.ru
> Date: Fri, 13 Dec 2013 11:50:49 -0500
> 
> >  . I also tried a different approach: compact only those font-entities
> >    that don't match any font that is still used by some face on some
> >    frame.  (I used font_match_p for that.)  This seems to work, except
> >    that font_match_p is evidently not safe enough to be used in the
> >    middle of GC -- I got crashes a few times.  When it did work, it
> >    again prevented the compaction, as if the compacting code were not
> >    there at all.
> 
> Maybe a better way goes along the lines of:
> - every time we use a font, we set an "in_use" bit.

The problem is not with fonts, it is with font-entities (a different
type of object) stored within the font cache.

I couldn't find any code that marks them "in use" or records them in
some other Lisp object (which would have prevented them from being
GC'ed).

> - every now and then (ha ha!), we let the GC collect fonts that don't
>   have their "in_use" bit set.

You didn't say who will reset that bit.

bug#15876: 24.3.50; Highly degraded performance between rev 114715 and 115006

[Stefan Monnier]

>> - every time we use a font, we set an "in_use" bit.
                         ^^^^
                      font-entity

>> - every now and then (ha ha!), we let the GC collect fonts that don't
>> have their "in_use" bit set.
                            ^^^
                            and reset that bit to false.

-- Stefan

bug#15876: 24.3.50; Highly degraded performance between rev 114715 and 115006

[Jan Djärv]

Hello.

13 dec 2013 kl. 16:22 skrev Eli Zaretskii <eliz@gnu.org>:

>> From: Jan Djärv <jan.h.d@swipnet.se>
>> Date: Wed, 11 Dec 2013 20:50:12 +0100
>> Cc: Dmitry Antipov <dmantipov@yandex.ru>,
>> Kenichi Handa <handa@gnu.org>,
>> 15876@debbugs.gnu.org,
>> sva-news@mygooglest.com
>> 
>>> Jan, Ken'ichi, would you please comment on this?  Are we losing
>>> something by reusing already loaded fonts that support a given
>>> character, as opposed to looking for the "best-match" font?
>> 
>> See discussion starting here http://debbugs.gnu.org/cgi/bugreport.cgi?bug=15138#29:
>> 
>> Kenichi Handa wrote:
>> 
>> I agree that this change improves font selection for
>> symbols, but it's not good for many scripts for which just
>> having a glyph is not enough.  For instance, if the default
>> font has Hindi glyphs but doesn't have the OTF features for
>> Hindi script, we must find another proper font for Hindi.
>> 
>> How about modifying the current fontset mechanism as this?
>> 
>> (1) Allow t for FONT-SPEC of set-fontset-font to tell that
>>    the default font should be tried.
>> (2) Modiyf the default fontset to include `t' as the
>>    font-spec for scripts/characters for which the default
>>    font is ok.
> 
> I see.  But then why does the NS build still use a variant of that
> approach?  How is it exempt from what Handa-san wrote above?

When this was done, NS only had nsfont.m where OTF features are ignored (i.e. not implemented)
except for script.  Since then we have macfont.m but it does the same.

There might be a case where the fontset contains two fonts that contains the same glyph and that we by this code selects the non-preferred one.  But this is highly theoretical, I would like to see a real example where this choice actually matter. 

BTW it was said that the font backend should reject fonts that request OTF features the backend does not support, but if the backend does that it would end up with no font at all in many cases.  There is not first a font with OTF-features and then one fallback without OTF-feature in the fontests.  There is just the one with OTF-features.  So Emacs has made a second bad choice w.r.t. fonts.  The first was using the X11 specific logical font description as its internal format (still does even if X11 says it is deprecated).  Now we are tied to features specified by Open Type.

	Jan D.

bug#15876: 24.3.50; Highly degraded performance between rev 114715 and 115006

[Dmitry Antipov]

On 12/13/2013 10:44 PM, Eli Zaretskii wrote:

> In any case, if without compact_font_caches you still have such a
> large increase in the memory footprint, it seems like that problem is
> specific to X, while on Windows the cure is worse than the disease.
> So if no other solution presents itself, I'm inclined to ifdef away
> that code for Windows, if you don't mind.

Since Stefan has announced a feature freeze in the week, OK for now;
but I still hope to find a somewhat better solution for all of that mess.

Dmitry