Re: Garbage collector: is 800kb a good default?

[Dmitrii Korobeinikov <dim1212k@gmail.com>]

> Running GC frequently is done largely to try and keep total heap size
> and memory fragmentation under control.  So completely refraining from
> running GC while non-idle is probably not a great solution, but I do
> agree that adding a "GC when idle" would not only be great in itself but
> would also let us increase the default thresholds somewhat.  Tho I don't
> think we can increase the threshold as much as you suggest without
> having significant detrimental effects.

So, are there any stats on how much worse the fragmentation gets w/
higher thresholds and how much it effects emacs as a result? (By the
way, I am not really suggesting 80MB, just testing it out at the
moment : ) )

> 2- 38% is pretty high

Just for the reference, I've seen the value of 25 in one of the
threads nearby [1], not sure why mine is so high.

Best,
DK

[1] https://lists.gnu.org/archive/html/emacs-devel/2020-04/msg00406.html

чт, 9 апр. 2020 г. в 19:47, Stefan Monnier <monnier@iro.umontreal.ca>:
>
> > Of course, raising the threshold significantly higher on its own is
> > not a very good idea. But if paired with an idle timer like suggested
> > here [2], then it all starts looking like a decent combination:
>
> I agree that it's worth investigating improvements based on dynamically
> changing the GC threshold.  What you're pointing out is that from the
> user's point of view, a GC during idle time is free (so it can occur
> frequently) and also that a GC during non-idle time can delay redisplay,
> so (ignoring all other impacts of GC) we should refrain from running GC
> while non-idle and trigger GC everything we're idle.
>
> Running GC frequently is done largely to try and keep total heap size
> and memory fragmentation under control.  So completely refraining from
> running GC while non-idle is probably not a great solution, but I do
> agree that adding a "GC when idle" would not only be great in itself but
> would also let us increase the default thresholds somewhat.  Tho I don't
> think we can increase the threshold as much as you suggest without
> having significant detrimental effects.
>
> There 2 additional ways to attack the problem, BTW:
>
> 1- replace our GC with a concurrent GC.  This would let us move those
>    38% GC overheard to one of the other CPU cores sitting idle while the
>    only CPU core running Emacs is frantically trying to scroll through
>    your buffer (it would also slow down both the GC and the main thread
>    a bit, but in the current context of plentiful CPU cores it would
>    still be very worthwhile).
>    [ Along the same lines, making our GC parallel could cut those 38%
>      down to some extent, as would a generational GC.  ]
>
> 2- 38% is pretty high.  Usually this indicates that the GC is not
>    efficient: it works hard yet reclaims very little memory likely
>    because the application is allocating a lot of objects which are
>    *not* temporary.  This is also the typical situation during Emacs's
>    startup where we're loading packages and initializing big
>    data-structures, so after allocating our GC-threshold of data the GC
>    is run but doesn't collect much garbage because all that data is
>    there to stay.
>    We can't really know beforehand if a GC will reclaim a lot of memory,
>    but we could "use the past to predict the future": we could set the
>    threshold higher when the last GC reclaimed too little garbage.
>
> I had sent a tentative patch for the "GC when idle" feature but it had
> some rough edges, and some details of the GC code have changed since.
> I'd welcome a patch to do that.  It shouldn't require many changes.
> Similarly, point (2) above should be fairly simple to add.
>
>
>         Stefan
>