Re: Debugging emacs memory management

[Eli Zaretskii <eliz@gnu.org>]

> From: Dima Kogan <lists@dima.secretsauce.net>
> Date: Sun, 20 Sep 2015 15:01:01 -0700
> Cc: emacs-devel@gnu.org
> 
> Thank you Paul. I'm continuint to dig. I found out that repeatedly
> creating/destroying the first frame is far more leaky than a non-first
> frame:
> 
>   http://debbugs.gnu.org/cgi/bugreport.cgi?bug=21509
> 
> This should be fixed, but in my use I always have multiple frames open,
> so this isn't biting me. Even with a frame open, each new frame
> create/destroy cycle leaks about 15kB. Digging into this, the leaky
> malloc() path is:
> 
>   emacs-snapshot- 28044 [000] 587944.079120: probe_libc:malloc: (7fa38bffc020) bytes=0x1000
>                      7c020 malloc (/lib/x86_64-linux-gnu/libc-2.19.so)
>                     14518f allocate_vectorlike.part.19 (/usr/bin/emacs-snapshot-lucid)
>                     145ead allocate_vector (/usr/bin/emacs-snapshot-lucid)
>                      a427c make_sub_char_table (/usr/bin/emacs-snapshot-lucid)
>                      a4f1a sub_char_table_set_range (/usr/bin/emacs-snapshot-lucid)
>                      a4f34 sub_char_table_set_range (/usr/bin/emacs-snapshot-lucid)
>                      a69a8 char_table_set_range (/usr/bin/emacs-snapshot-lucid)
>                      a6b40 Fset_char_table_range (/usr/bin/emacs-snapshot-lucid)
>                     1c838b Fset_fontset_font (/usr/bin/emacs-snapshot-lucid)
>                     1c9dfc fontset_from_font (/usr/bin/emacs-snapshot-lucid)
>                      c9e50 x_new_font (/usr/bin/emacs-snapshot-lucid)
>                      2534e x_set_font (/usr/bin/emacs-snapshot-lucid)
>                      23e66 x_set_frame_parameters (/usr/bin/emacs-snapshot-lucid)
>                      26ab5 x_default_parameter (/usr/bin/emacs-snapshot-lucid)
>                      d03ba x_default_font_parameter (/usr/bin/emacs-snapshot-lucid)
>                      d7f98 Fx_create_frame (/usr/bin/emacs-snapshot-lucid)

This doesn't seem to be a leak in itself.  What happens here is that
Emacs is creating a new frame, one of whose parameters is a font spec.
You didn't show your font/fontset customizations (I'm assuming this is
not from "emacs -Q"), so I can only guess the details of what follows
from this incomplete backtrace: the font definitions cause Emacs to
modify the existing fontset, and that requires an addition of a
sub-char-table to the existing char-table (fontsets are implemented as
char-tables internally), because your font customizations affect only
some subset of the Unicode codespace.  See the definition of
FONTSET_ADD in fontset.c and the call to it in Fset_fontset_font.

Since the fontset is a global structure in Emacs, it is okay to add
information to it that never gets GC'ed until the session ends.  What
would be NOT okay is if the information about the same font and/or the
same modification of the fontset repeatedly created more and more of
these additional sub-char-tables, instead of reusing the one created
the first time.

So what we need to investigate this particular issue is:

  . a full backtrace from GDB, including parameters of each function
    calls, preferably from an unoptimized build
  . details about your font/fontset related customizations, if any
  . evidence that this very backtrace, including the call to
    make_sub_char_table and the associated memory consumption, is
    repeated each time you create a new frame with the same
    font/fontset customizations, when you create several such frames
    one after another

> I'll do a writeup about how exactly to get that later. In the meantime,
> does this speak to anybody? It looks like we're creating a new Lisp
> object

We are creating a sub-char-table.  This is needed when some call sets
a range of characters in a char-table to some value, where previously
a much larger range was set to another (usually, the default) value.
Char-tables are memory-efficient, so they only allocate additional
storage when needed, in this fashion.

> so I guess the gc is supposed to clean it out.

GC should clean it up only if the char-table that is being modified is
not needed afterwards.  If I'm right, and the char-table in question
is the (global) fontset, then it will not be GC'ed any time soon.

In any case, to make sure that it isn't GC'ed, you need to call
garbage-collect by hand.

> Is the recursive call to sub_char_table_set_range() causing issues?

No, it shouldn't.