Re: MPS: weak hash tables

[Helmut Eller <eller.helmut@gmail.com>]

On Sat, Jul 06 2024, Gerd Möllmann wrote:

>> ert-tests.el worries me the most.  There, MPS looks pretty bad compared
>> to the old GC.  (Who would have thought that printing backtraces
>> involves copying and rebalancing interval trees with thousands of nodes?
>> And why does it perform so poorly with a generational GC?)
>
> Hm, sounds almost like a bug to me. The bug being the copying. Why does
> it do that, do you know?

One reason may be that something tries to remove invisible text with
buffer-substring.  That needs to copy part of the interval tree.  But
the copying happens for both, the MPS and the non-MPS version.

When I run

 perf record ../src/emacs -Q --batch -l lisp/emacs-lisp/ert-tests.el
 --eval '(ert-run-tests-batch-and-exit
 "ert-test-\\(parse\\|plist\\|record\\|run-tests-batch-exp\\)")'
 
then the top entries in the perf output for the MPS version looks like:

   9.55%  emacs    emacs                       [.] balance_an_interval
   5.46%  emacs    emacs                       [.] amcSegFix
   5.20%  emacs    emacs                       [.] exec_byte_code
   3.71%  emacs    emacs                       [.] _mps_fix2
   3.10%  emacs    emacs                       [.] fix_lisp_obj
   2.81%  emacs    emacs                       [.] SegSetGrey
   2.37%  emacs    emacs                       [.] shieldQueue
   2.29%  emacs    emacs                       [.] SegFix
   2.24%  emacs    emacs                       [.] is_dflt_fwd
   2.07%  emacs    emacs                       [.] alloc_impl
   2.02%  emacs    emacs                       [.] dflt_scan_obj.constprop.0

and the non-MPS version:

  19.63%  emacs    emacs                 [.] balance_an_interval
  10.22%  emacs    emacs                 [.] exec_byte_code
   9.73%  emacs    emacs                 [.] process_mark_stack
   3.04%  emacs    emacs                 [.] next_interval
   2.42%  emacs    emacs                 [.] sweep_intervals
   2.26%  emacs    emacs                 [.] Fmemq
   2.16%  emacs    emacs                 [.] find_interval
   2.05%  emacs    emacs                 [.] Fcons

So balance_an_interval is pretty high in both, but it seems that the GC
related part is quite a bit bigger for the MPS version.  Maybe it has
something to do with memory barriers.

>> It would be nice to have a way to set a "memory limit".  Preferably with
>> some "out of memory error".  Even aborting would be better than starting
>> to swap.
>
> There is
>
>   MPS_KEY_COMMIT_LIMIT (type size_t) is the maximum amount of main memory,
>   in bytes (1), that the MPS will obtain from the operating system. See
>   mps_arena_commit_limit() for details. The default commit limit is the
>   maximum value of the size_t type.
>
> Never tried that. There are also functions to read/set the limit, so
> maybe we could introduce a DEFVAR with variable watcher for playing with
> it, or a function.

When I add

  MPS_ARGS_ADD (args, MPS_KEY_COMMIT_LIMIT,  1 << 30);

to make_arena then

  make lisp/emacs-lisp/ert-tests

aborts with

  trace.c:1088: Emacs fatal error: assertion failed: band != RankWEAK || rank == band

This looks like a bug.

>> I also think that something needs to be done about the pidigits
>> benchmark.  It's not about the bignums (which I couldn't care less
>> about) but the problem that finalization messages are only processed
>> when Emacs is idle.
>
> Hm, so this one didn't help.
>
>   void
>   garbage_collect (void)
>   {
>   #ifdef HAVE_MPS
>     igc_process_messages ();
>   #endif
>
> Looks like it needs to be put somewhere else.

I think finalize_bignum is ever called for something like this:

 ./src/emacs -Q -batch -eval '
 (dotimes (_ 10000000) (set (quote foo) (ash most-positive-fixnum 10000)))'