bug#42013: 26.3; hash tables are not garbage collected

bug#42013: 26.3; hash tables are not garbage collected

[Hendrik Tews]

Hi,

I have the impression hash tables and their content are not
garbage collected when the hash table is not referenced any more.
Consider the following piece of elisp code:

    (let (gc-start gc-state hash symbol result)
      ;; record gc statistics
      (setq gc-start (garbage-collect))
      ;; create a hash and put an uninterned symbol into it
      (setq hash (make-hash-table))
      (setq symbol (make-symbol "x"))
      (puthash symbol t hash)
      ;; take gc stats and record the differences of used symbols and used
      ;; vectors in result
      (setq gc-state (garbage-collect))
      (push (list 'sym1 (- (caddr (assoc 'symbols gc-state))
                           (caddr (assoc 'symbols gc-start))))
            result)
      (push (list 'vec1 (- (caddr (assoc 'vectors gc-state))
                           (caddr (assoc 'vectors gc-start))))
            result)
      ;; make the symbol and the hash inaccessible 
      (setq symbol nil)
      (setq hash nil)
      ;; take gc stats again and record the differences of used symbols
      ;; and used vectors in result
      (setq gc-state (garbage-collect))
      (push (list 'sym2 (- (caddr (assoc 'symbols gc-state))
                           (caddr (assoc 'symbols gc-start))))
            result)
      (push (list 'vec2 (- (caddr (assoc 'vectors gc-state))
                           (caddr (assoc 'vectors gc-start))))
            result)
      result)

With emacs -Q it returns ((vec2 5) (sym2 1) (vec1 5) (sym1 1)),
while I was expecting ((vec2 0) (sym2 0) ...).

What am I doing wrong that prevents the third call to
garbage-collect to garbage collect the hash and the uninterned
symbol?

Note that with inserting a clrhash before setting hash to nil,
the symbol gets collected, but the vectors won't. What is needed
to garbage collect a hash table?

Thanks,

Hendrik


text generated by report-emacs-bug:

In GNU Emacs 26.3 (build 2, x86_64-pc-linux-gnu, GTK+ Version 3.24.20)
 of 2020-05-17, modified by Debian built on x86-csail-01
Windowing system distributor 'The X.Org Foundation', version 11.0.12008000
System Description:	Debian GNU/Linux bullseye/sid

Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.

Configured using:
 'configure --build x86_64-linux-gnu --prefix=/usr
 --sharedstatedir=/var/lib --libexecdir=/usr/lib
 --localstatedir=/var/lib --infodir=/usr/share/info
 --mandir=/usr/share/man --enable-libsystemd --with-pop=yes
 --enable-locallisppath=/etc/emacs:/usr/local/share/emacs/26.3/site-lisp:/usr/local/share/emacs/site-lisp:/usr/share/emacs/26.3/site-lisp:/usr/share/emacs/site-lisp
 --with-sound=alsa --without-gconf --with-mailutils --build
 x86_64-linux-gnu --prefix=/usr --sharedstatedir=/var/lib
 --libexecdir=/usr/lib --localstatedir=/var/lib
 --infodir=/usr/share/info --mandir=/usr/share/man --enable-libsystemd
 --with-pop=yes
 --enable-locallisppath=/etc/emacs:/usr/local/share/emacs/26.3/site-lisp:/usr/local/share/emacs/site-lisp:/usr/share/emacs/26.3/site-lisp:/usr/share/emacs/site-lisp
 --with-sound=alsa --without-gconf --with-mailutils --with-x=yes
 --with-x-toolkit=gtk3 --with-toolkit-scroll-bars 'CFLAGS=-g -O2
 -fdebug-prefix-map=/build/emacs-mHAik2/emacs-26.3+1=. -fstack-protector-strong
 -Wformat -Werror=format-security -Wall' 'CPPFLAGS=-Wdate-time
 -D_FORTIFY_SOURCE=2' LDFLAGS=-Wl,-z,relro'

Configured features:
XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND GPM DBUS GSETTINGS GLIB
NOTIFY ACL LIBSELINUX GNUTLS LIBXML2 FREETYPE M17N_FLT LIBOTF XFT ZLIB
TOOLKIT_SCROLL_BARS GTK3 X11 XDBE XIM THREADS LIBSYSTEMD LCMS2

Important settings:
  value of $LANG: en_US.UTF-8
  value of $XMODIFIERS: @im=ibus
  locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message rmc puny seq byte-opt gv
bytecomp byte-compile cconv cl-loaddefs cl-lib dired dired-loaddefs
format-spec rfc822 mml easymenu mml-sec password-cache epa derived epg
epg-config gnus-util rmail rmail-loaddefs mm-decode mm-bodies mm-encode
mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail rfc2047
rfc2045 ietf-drums mm-util mail-prsvr mail-utils elec-pair time-date
mule-util tooltip eldoc electric uniquify ediff-hook vc-hooks
lisp-float-type mwheel term/x-win x-win term/common-win x-dnd tool-bar
dnd fontset image regexp-opt fringe tabulated-list replace newcomment
text-mode elisp-mode lisp-mode prog-mode register page menu-bar
rfn-eshadow isearch timer select scroll-bar mouse jit-lock font-lock
syntax facemenu font-core term/tty-colors frame cl-generic cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese eucjp-ms cp51932 hebrew greek romanian slovak czech european
ethiopic indian cyrillic chinese composite charscript charprop
case-table epa-hook jka-cmpr-hook help simple abbrev obarray minibuffer
cl-preloaded nadvice loaddefs button faces cus-face macroexp files
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget hashtable-print-readable backquote threads dbusbind
inotify lcms2 dynamic-setting system-font-setting font-render-setting
move-toolbar gtk x-toolkit x multi-tty make-network-process emacs)

Memory information:
((conses 16 96516 5845)
 (symbols 48 20785 1)
 (miscs 40 45 96)
 (strings 32 29163 1930)
 (string-bytes 1 763910)
 (vectors 16 14012)
 (vector-slots 8 507754 6556)
 (floats 8 49 68)
 (intervals 56 268 0)
 (buffers 992 12))

bug#42013: 26.3; hash tables are not garbage collected

[Lars Ingebrigtsen]

Hendrik Tews <hendrik.tews@kernkonzept.com> writes:

> What am I doing wrong that prevents the third call to
> garbage-collect to garbage collect the hash and the uninterned
> symbol?

(let (gc-start gc-state hash result)
  ;; record gc statistics
  (setq gc-start (garbage-collect))
  (setq hash (make-hash-table))
  (setq gc-state (garbage-collect))
  (push (list 'vec1 (- (caddr (assoc 'vectors gc-state))
                       (caddr (assoc 'vectors gc-start))))
        result)
  ;; make the hash inaccessible 
  (setq hash nil)
  (setq gc-state (garbage-collect))
  (push (list 'vec2 (- (caddr (assoc 'vectors gc-state))
                       (caddr (assoc 'vectors gc-start))))
        result)
  result)

I've simplified the test case.  As the bug submitter says, there's more
vectors after this than before:

((vec2 5) (vec1 5))

But I'm not sure you can use this data in this fine-grained way.  For
instance:

(let (gc-start gc-state hash result)
  ;; record gc statistics
  (setq gc-start (garbage-collect))
  ;;(setq hash (make-hash-table))
  (setq gc-state (garbage-collect))
  (push (list 'vec1 (- (caddr (assoc 'vectors gc-state))
                       (caddr (assoc 'vectors gc-start))))
        result)
  ;; make the hash inaccessible 
  (setq hash nil)
  (setq gc-state (garbage-collect))
  (push (list 'vec2 (- (caddr (assoc 'vectors gc-state))
                       (caddr (assoc 'vectors gc-start))))
        result)
  result)

=> ((vec2 0) (vec1 0))

Sounds good.


(let (gc-start gc-state hash result)
  ;; record gc statistics
  (setq gc-start (garbage-collect))
  (make-hash-table)
  (setq gc-state (garbage-collect))
  (push (list 'vec1 (- (caddr (assoc 'vectors gc-state))
                       (caddr (assoc 'vectors gc-start))))
        result)
  ;; make the hash inaccessible 
  (setq hash nil)
  (setq gc-state (garbage-collect))
  (push (list 'vec2 (- (caddr (assoc 'vectors gc-state))
                       (caddr (assoc 'vectors gc-start))))
        result)
  result)

=> ((vec2 -1) (vec1 4))

Uhmn...

So I'm not sure there's anything to fix here, except perhaps noting in
the doc string that you can't do precision math on the numbers?  Anybody?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#42013: 26.3; hash tables are not garbage collected

[Hendrik Tews]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> => ((vec2 -1) (vec1 4))
>
> Uhmn...

It is not really clear what you executed to obtain the negative
number. I am assuming you executed all the let expressions in
your reply in sequence.

To me, the most obvious explanation for the negative number would
be that garbage-collect does not always perform a full garbage
collection, therefore the last garbage-collect by chance finally
performs a full collection and collects one vector that has
survived the other garbage collections...

> So I'm not sure there's anything to fix here, except perhaps noting in
> the doc string that you can't do precision math on the numbers?  Anybody?

To me the observed behavior suggests that the Elisp garbage
collector is more complicated than the simple mark and sweep
described in the manual. Can you really confirm, that this is not
the case? Because otherwise I would expect a documentation update
for garbage-collect and in the Garbage Collection appendix.

Hendrik

bug#42013: 26.3; hash tables are not garbage collected

[Lars Ingebrigtsen]

Hendrik Tews <hendrik.tews@kernkonzept.com> writes:

> It is not really clear what you executed to obtain the negative
> number. I am assuming you executed all the let expressions in
> your reply in sequence.

Yes.  Sometimes it's -1 and sometimes it's 0, apparently.

> To me, the most obvious explanation for the negative number would
> be that garbage-collect does not always perform a full garbage
> collection, therefore the last garbage-collect by chance finally
> performs a full collection and collects one vector that has
> survived the other garbage collections...

Well, or that gc itself allocates some vectors sometimes.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#42013: 26.3; hash tables are not garbage collected

[Andreas Schwab]

On Jun 23 2020, Hendrik Tews wrote:

> With emacs -Q it returns ((vec2 5) (sym2 1) (vec1 5) (sym1 1)),
> while I was expecting ((vec2 0) (sym2 0) ...).

The garbage collector is conservative, so objects may not be collected
immediately, even if they appear unreferenced.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

bug#42013: 26.3; hash tables are not garbage collected

[Hendrik Tews]

> The garbage collector is conservative, so objects may not be collected
> immediately, even if they appear unreferenced.

OK, would it be possible to update the documentation?

Is there a way to trigger a full collection of all unreachable
objects? I would like to use this and the numbers that
garbage-collect returns to check that a certain piece of code
does not leak memory. Or is there another way to check that all
allocations done within a form are unreachable after leaving that
form?

Thanks

Hendrik

bug#42013: 26.3; hash tables are not garbage collected

[Lars Ingebrigtsen]

Hendrik Tews <hendrik.tews@kernkonzept.com> writes:

>> The garbage collector is conservative, so objects may not be collected
>> immediately, even if they appear unreferenced.
>
> OK, would it be possible to update the documentation?

I've now updated the manual on this point in Emacs 28.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no