bug#36032: 27.0.50; Massive GC when shift-selecting text in HELLO

bug#36032: 27.0.50; Massive GC when shift-selecting text in HELLO

[Eli Zaretskii]

To reproduce:

  emacs -Q
  C-h h
  C-SPC
  C-f C-f C-f ....

You will see a ~0.5 sec delay after each C-f, before its effect of
extending the region is visible.  The reason is GC after every
keystroke, and it seems to be caused by the changes in commit 479f51a.
That commit caused mode-line-default-help-echo to be called each time
Emacs needs to redisplay the mode line, and that seems to produce lots
of garbage, at least with the HELLO file.

If this function cannot be optimized to produce less garbage, could we
perhaps somehow cache the value of the help-echo and not recalculate
it unless something really changed?

In GNU Emacs 27.0.50 (build 1021, i686-pc-mingw32)
 of 2019-05-31 built on HOME-C4E4A596F7
Repository revision: 40bc7ddb7529e516e380cc2f2d904feb3faadced
Repository branch: master
Windowing system distributor 'Microsoft Corp.', version 5.1.2600
System Description: Microsoft Windows XP Service Pack 3 (v5.1.0.2600)

Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.

Configured using:
 'configure -C --prefix=/d/usr --with-wide-int --with-modules
 --enable-checking=yes,glyphs 'CFLAGS=-O0 -gdwarf-4 -g3''

Configured features:
XPM JPEG TIFF GIF PNG RSVG SOUND NOTIFY W32NOTIFY ACL GNUTLS LIBXML2
ZLIB TOOLKIT_SCROLL_BARS MODULES THREADS JSON PDUMPER LCMS2 GMP

Important settings:
  value of $LANG: ENU
  locale-coding-system: cp1255

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message rmc puny dired dired-loaddefs
format-spec rfc822 mml easymenu mml-sec password-cache epa derived epg
epg-config gnus-util rmail rmail-loaddefs text-property-search time-date
seq byte-opt gv bytecomp byte-compile cconv mm-decode mm-bodies
mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader cl-loaddefs
cl-lib sendmail rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils
elec-pair mule-util tooltip eldoc electric uniquify ediff-hook vc-hooks
lisp-float-type mwheel dos-w32 ls-lisp disp-table term/w32-win w32-win
w32-vars term/common-win tool-bar dnd fontset image regexp-opt fringe
tabulated-list replace newcomment text-mode elisp-mode lisp-mode
prog-mode register page menu-bar rfn-eshadow isearch timer select
scroll-bar mouse jit-lock font-lock syntax facemenu font-core
term/tty-colors frame cl-generic cham georgian utf-8-lang misc-lang
vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms cp51932
hebrew greek romanian slovak czech european ethiopic indian cyrillic
chinese composite charscript charprop case-table epa-hook jka-cmpr-hook
help simple abbrev obarray minibuffer cl-preloaded nadvice loaddefs
button faces cus-face macroexp files text-properties overlay sha1 md5
base64 format env code-pages mule custom widget hashtable-print-readable
backquote threads w32notify w32 lcms2 multi-tty make-network-process
emacs)

Memory information:
((conses 16 50602 9597)
 (symbols 48 7062 1)
 (strings 16 18354 2121)
 (string-bytes 1 521183)
 (vectors 16 9578)
 (vector-slots 8 119896 9488)
 (floats 8 19 220)
 (intervals 40 257 97)
 (buffers 880 11))

bug#36032: 27.0.50; Massive GC when shift-selecting text in HELLO

[Eli Zaretskii]

> Date: Fri, 31 May 2019 22:47:05 +0300
> From: Eli Zaretskii <eliz@gnu.org>
> 
> To reproduce:
> 
>   emacs -Q
>   C-h h
>   C-SPC
>   C-f C-f C-f ....
> 
> You will see a ~0.5 sec delay after each C-f, before its effect of
> extending the region is visible.  The reason is GC after every
> keystroke, and it seems to be caused by the changes in commit 479f51a.
> That commit caused mode-line-default-help-echo to be called each time
> Emacs needs to redisplay the mode line, and that seems to produce lots
> of garbage, at least with the HELLO file.

Actually, it looks like GC is not itself the culprit, it's the fact
that by default we also compact the font caches as part of GC.  The
following patch fixes this in HELLO; any objections to push to master?
(I'm beginning to seriously think we should turn off compacting the
font caches by default, as it causes too many problems.)

diff --git a/etc/HELLO b/etc/HELLO
index a56a73b..33e664e 100644
--- a/etc/HELLO
+++ b/etc/HELLO
@@ -124,4 +124,5 @@ along with GNU Emacs.  If not, see <<https://www.gnu.org/licenses/>.
 ;;; tab-width: 32
 ;;; bidi-display-reordering: t
 ;;; coding: utf-8
+;;; inhibit-compacting-font-caches: t
 ;;; End:</x-charset>
diff --git a/lisp/files.el b/lisp/files.el
index 287ad14..35497b5 100644
--- a/lisp/files.el
+++ b/lisp/files.el
@@ -3297,6 +3297,7 @@ safe-local-eval-forms
 	   (default-directory       . stringp)	;; C source code
 	   (fill-column             . integerp)	;; C source code
 	   (indent-tabs-mode        . booleanp)	;; C source code
+	   (inhibit-compacting-font-caches . booleanp) ;; C source code
 	   (left-margin             . integerp)	;; C source code
 	   (no-update-autoloads     . booleanp)
 	   (lexical-binding	 . booleanp)	  ;; C source code

bug#36032: 27.0.50; Massive GC when shift-selecting text in HELLO

[martin rudalics]

[-- Attachment #1: Type: text/plain, Size: 1695 bytes --]

 > To reproduce:
 >
 >    emacs -Q
 >    C-h h
 >    C-SPC
 >    C-f C-f C-f ....
 >
 > You will see a ~0.5 sec delay after each C-f, before its effect of
 > extending the region is visible.  The reason is GC after every
 > keystroke, and it seems to be caused by the changes in commit 479f51a.
 > That commit caused mode-line-default-help-echo to be called each time
 > Emacs needs to redisplay the mode line, and that seems to produce lots
 > of garbage, at least with the HELLO file.

Here the delays are considerably larger (one second at least) and
there is at least one GC after each keystroke, sometimes two.  But I
can't confirm the remainder of what you say.  Attached find two small
files.

Loading the first one (show-gc.el) with emacs -Q gives approximately
the same behaviors with Emacs 26.2.50 (which does not have the
'mode-line-default-help-echo' function) and Emacs 27.0.50 - at least
one GC per keystroke here.

Loading the second one (show-gc-he.el) with emacs -Q (Emacs 27.0.50
only) shows that there is one call of 'mode-line-default-help-echo'
per keystroke and I doubt that one such call can produce that much
garbage.

Also, customizing the option 'mode-line-default-help-echo' to nil does
not make any performance difference here.

All tested with 64-bit MSYS2 builds on Windows 10.  If you get
different results please tell me.  Also I have not installed the
Harfbuzz branch so OMMV.

 > If this function cannot be optimized to produce less garbage, could we
 > perhaps somehow cache the value of the help-echo and not recalculate
 > it unless something really changed?

This is certainly a good idea.  But I doubt it would have any impact
in the case at hand.

martin

[-- Attachment #2: show-gc.el --]
[-- Type: text/plain, Size: 153 bytes --]

(defvar gc 0)
(add-hook 'post-gc-hook (lambda () (setq gc (1+ gc))))
(defun show-gc () (message "gc: %s" gc))
(add-hook 'post-command-hook 'show-gc)

[-- Attachment #3: show-gc-he.el --]
[-- Type: text/plain, Size: 1936 bytes --]

(defvar gc 0)
(defvar he 0)
(add-hook 'post-gc-hook (lambda () (setq gc (1+ gc))))
(defun show-gc-he () (message "gc: %s ... he: %s" gc he))
(add-hook 'post-command-hook 'show-gc-he)

(defun mode-line-default-help-echo (window)
  "Return default help echo text for WINDOW's mode line."
  (let* ((frame (window-frame window))
         (line-1a
          ;; Show text to select window only if the window is not
          ;; selected.
          (not (eq window (frame-selected-window frame))))
         (line-1b
          ;; Show text to drag mode line if either the window is not
          ;; at the bottom of its frame or the minibuffer window of
          ;; this frame can be resized.  This matches a corresponding
          ;; check in `mouse-drag-mode-line'.
          (or (not (window-at-side-p window 'bottom))
              (let ((mini-window (minibuffer-window frame)))
                (and (eq frame (window-frame mini-window))
                     (or (minibuffer-window-active-p mini-window)
                         (not resize-mini-windows))))))
         (line-2
          ;; Show text make window occupy the whole frame
          ;; only if it doesn't already do that.
          (not (eq window (frame-root-window frame))))
         (line-3
          ;; Show text to delete window only if that's possible.
          (not (eq window (frame-root-window frame)))))
    (setq he (1+ he))
    (when (or line-1a line-1b line-2 line-3)
      (concat
       (when (or line-1a line-1b)
         (concat
          "mouse-1: "
          (when line-1a "Select window")
          (when line-1b
            (if line-1a " (drag to resize)" "Drag to resize"))
          (when (or line-2 line-3) "\n")))
       (when line-2
         (concat
          "mouse-2: Make window occupy whole frame"
          (when line-3 "\n")))
       (when line-3
         "mouse-3: Remove window from frame")))))

bug#36032: 27.0.50; Massive GC when shift-selecting text in HELLO

[martin rudalics]

 > Actually, it looks like GC is not itself the culprit, it's the fact
 > that by default we also compact the font caches as part of GC.  The
 > following patch fixes this in HELLO; any objections to push to master?

Yes.  It should go to Emacs 26.0.50 and be the default anywhere.

martin

bug#36032: 27.0.50; Massive GC when shift-selecting text in HELLO

[martin rudalics]

 > Yes.  It should go to Emacs 26.0.50 and be the default anywhere.

I'm afraid it's already too late for Emacs 26.0.50.  Emacs 26.2.50 is
what I meant.

martin

bug#36032: 27.0.50; Massive GC when shift-selecting text in HELLO

[Eli Zaretskii]

> From: martin rudalics <rudalics@gmx.at>
> Date: Sat, 1 Jun 2019 09:49:56 +0200
> 
>  > Actually, it looks like GC is not itself the culprit, it's the fact
>  > that by default we also compact the font caches as part of GC.  The
>  > following patch fixes this in HELLO; any objections to push to master?
> 
> Yes.  It should go to Emacs 26.0.50 and be the default anywhere.

Do you see a similar problem in the emacs-26 branch?  I don't.

bug#36032: 27.0.50; Massive GC when shift-selecting text in HELLO

[martin rudalics]

 >> Yes.  It should go to Emacs 26.0.50 and be the default anywhere.
 >
 > Do you see a similar problem in the emacs-26 branch?  I don't.

When with emacs -Q --load "~/show-gc.el" I'm at the beginning of

Dutch (Nederlands)

doing C-SPC and then C-f until I'm at the end of that text I get 19
additional collection cycles.

GNU Emacs 26.2.50 (build 1, x86_64-w64-mingw32)
  of 2019-05-26

How many do you get?

martin

bug#36032: 27.0.50; Massive GC when shift-selecting text in HELLO

[Eli Zaretskii]

> Cc: 36032@debbugs.gnu.org
> From: martin rudalics <rudalics@gmx.at>
> Date: Sat, 1 Jun 2019 14:10:58 +0200
> 
>  >> Yes.  It should go to Emacs 26.0.50 and be the default anywhere.
>  >
>  > Do you see a similar problem in the emacs-26 branch?  I don't.
> 
> When with emacs -Q --load "~/show-gc.el" I'm at the beginning of
> 
> Dutch (Nederlands)
> 
> doing C-SPC and then C-f until I'm at the end of that text I get 19
> additional collection cycles.

I get 17 for the first C-f at the beginning of the buffer, and then
just one more for the Dutch line.

But neither the 17 first GCs nor the 18th one cause any tangible
delays in response to C-f.

bug#36032: 27.0.50; Massive GC when shift-selecting text in HELLO

[martin rudalics]

 >> When with emacs -Q --load "~/show-gc.el" I'm at the beginning of
 >>
 >> Dutch (Nederlands)
 >>
 >> doing C-SPC and then C-f until I'm at the end of that text I get 19
 >> additional collection cycles.
 >
 > I get 17 for the first C-f at the beginning of the buffer, and then
 > just one more for the Dutch line.
 >
 > But neither the 17 first GCs nor the 18th one cause any tangible
 > delays in response to C-f.

Interesting.  Here the release branch is just as slow as master.
Visiting HELLO takes about 50 seconds and one shifted C-f about one
second (or one GC cycle) regardless of where in the buffer I am.
Moreover, -O0 and -O3 builds incur about the same delays.

martin

bug#36032: 27.0.50; Massive GC when shift-selecting text in HELLO

[Eli Zaretskii]

> Cc: 36032@debbugs.gnu.org
> From: martin rudalics <rudalics@gmx.at>
> Date: Sat, 1 Jun 2019 16:01:42 +0200
> 
>  > But neither the 17 first GCs nor the 18th one cause any tangible
>  > delays in response to C-f.
> 
> Interesting.  Here the release branch is just as slow as master.
> Visiting HELLO takes about 50 seconds and one shifted C-f about one
> second (or one GC cycle) regardless of where in the buffer I am.
> Moreover, -O0 and -O3 builds incur about the same delays.

OK, I installed the fix on the release branch.  Thanks.