Garbage collector: is 800kb a good default?

Garbage collector: is 800kb a good default?

[Dmitrii Korobeinikov]

Hi all,

So, I was debugging scrolling, it felt evidently clunky for large
files for me. Font lock is the elephant in the room, but I decided to
profile my custom half-screen scroll functions anyway and was quite
surprised to find that GC was gobbling 38% of the CPU time. That
didn't have anything to do with my functions, the default full page
scroll has it the same.

That lead to some experimenation with gc-cons-threshold. Higher values
of this variable resulted in much less frequent freezes/pauses while
scrolling [1]. Setting garbage-collection-messages to t and
gs-cons-threshold to 80MB, I figured I can scroll a voluminous sheet
of 7k lines before gc has to do its job. With the default 0.8 it's
every 2-3 pagescrolls. That's a big difference.

Of course, raising the threshold significantly higher on its own is
not a very good idea. But if paired with an idle timer like suggested
here [2], then it all starts looking like a decent combination:

(setq gc-cons-threshold mem)
(run-with-idle-timer 2 t 'garbage-collect)

It would be good to find a threshold which the user barely has a
chance of reaching unless running a long uninterrupted task. And, by
the way, in this latter case, it is better to have a higher threshold
too due to the increase in speed, like with startup [3]. Setting the
threshold just a bit higher than the current default wouldn't help
much but may actually be worse as the pauses would be longer, but
about as frequent. Too high a value would create very long freezes for
uninterrupted tasks. But there should be a good middle ground.

So, as far as I can see, defaulting to a higher threshold with an idle
timer could yield better user experience with practically no
significant tradeoffs. Is there something I might be missing here?

Best,
DK

[1] This is evident in vanilla emacs if you pay attention, but it's
much more so for me w/ my the bloated config. I haven't yet found the
culprit.
[2] https://emacs.stackexchange.com/questions/34342/is-there-any-downside-to-setting-gc-cons-threshold-very-high-and-collecting-ga
[3]
| threshold (MB) | startup time (s) |
|----------------+------------------|
|            0.8 |             9.14 |
|              2 |             8.77 |
|              5 |             7.54 |
|             20 |             6.88 |
|             50 |             6.65 |
|             80 |             6.73 |
|            100 |             6.17 |

Re: Garbage collector: is 800kb a good default?

[Eli Zaretskii]

> From: Dmitrii Korobeinikov <dim1212k@gmail.com>
> Date: Thu, 9 Apr 2020 17:59:04 +0600
> 
> So, as far as I can see, defaulting to a higher threshold with an idle
> timer could yield better user experience with practically no
> significant tradeoffs. Is there something I might be missing here?

Two things: (1) the timer will not run as long as Emacs is running
some prolonged calculation, which could produce a lot of garbage; and
(2) you seem to ignore the increased memory pressure on the rest of
the system from the growing memory footprint of Emacs.  On GNU/Linux,
Emacs doesn't really return malloc'ed memory to the system, so once
the memory footprint grows, it more or less stays that way even after
GC.

Now, I'm not saying that we cannot or shouldn't increase the default
GC threshold, but we must keep these two aspects in mind while
discussing this.

Re: Garbage collector: is 800kb a good default?

[Dmitrii Korobeinikov]

> Two things: (1) the timer will not run as long as Emacs is running
> some prolonged calculation, which could produce a lot of garbage; and

True, that's a reason for not making it too high. Emacs would also
freeze very noticably in those cases. But I think the threshold
doesn't have to be very high to reap most of the the benefits. For
example, I am testing 80MB at the moment. Pretty sure I would be
satisfied with half that. By satisfied I mean 3.5k lines of scroll
action would still be plenty (before a ~1/3-1/2 second gc freeze
happens) and the startup time would be shaved off quite close to the
apparent limit anyway.

> (2) you seem to ignore the increased memory pressure on the rest of
> the system from the growing memory footprint of Emacs.  On GNU/Linux,
> Emacs doesn't really return malloc'ed memory to the system, so once
> the memory footprint grows, it more or less stays that way even after
> GC.

Didn't know about that. A ground of reasoning here could be what the
average user would see as acceptable. Of course, there are also people
who optimize their system for memory consumption. But they are
probably tech-savvy enough to find the gc options anyway.

чт, 9 апр. 2020 г. в 18:33, Eli Zaretskii <eliz@gnu.org>:
>
> > From: Dmitrii Korobeinikov <dim1212k@gmail.com>
> > Date: Thu, 9 Apr 2020 17:59:04 +0600
> >
> > So, as far as I can see, defaulting to a higher threshold with an idle
> > timer could yield better user experience with practically no
> > significant tradeoffs. Is there something I might be missing here?
>
> Two things: (1) the timer will not run as long as Emacs is running
> some prolonged calculation, which could produce a lot of garbage; and
> (2) you seem to ignore the increased memory pressure on the rest of
> the system from the growing memory footprint of Emacs.  On GNU/Linux,
> Emacs doesn't really return malloc'ed memory to the system, so once
> the memory footprint grows, it more or less stays that way even after
> GC.
>
> Now, I'm not saying that we cannot or shouldn't increase the default
> GC threshold, but we must keep these two aspects in mind while
> discussing this.

Re: Garbage collector: is 800kb a good default?

[Eli Zaretskii]

> From: Dmitrii Korobeinikov <dim1212k@gmail.com>
> Date: Thu, 9 Apr 2020 19:44:40 +0600
> Cc: emacs-devel <emacs-devel@gnu.org>
> 
> > Two things: (1) the timer will not run as long as Emacs is running
> > some prolonged calculation, which could produce a lot of garbage; and
> 
> True, that's a reason for not making it too high. Emacs would also
> freeze very noticably in those cases. But I think the threshold
> doesn't have to be very high to reap most of the the benefits. For
> example, I am testing 80MB at the moment. Pretty sure I would be
> satisfied with half that. By satisfied I mean 3.5k lines of scroll
> action would still be plenty (before a ~1/3-1/2 second gc freeze
> happens) and the startup time would be shaved off quite close to the
> apparent limit anyway.

The "right" value depends on your usage patterns, among other
factors.  How to account for that in the default value?

> > (2) you seem to ignore the increased memory pressure on the rest of
> > the system from the growing memory footprint of Emacs.  On GNU/Linux,
> > Emacs doesn't really return malloc'ed memory to the system, so once
> > the memory footprint grows, it more or less stays that way even after
> > GC.
> 
> Didn't know about that. A ground of reasoning here could be what the
> average user would see as acceptable. Of course, there are also people
> who optimize their system for memory consumption. But they are
> probably tech-savvy enough to find the gc options anyway.

The effect also depends on how much VM does the user have on his/her
system, and what other applications routinely run there.  Since we are
talking about defaults, we should find a setting that is safe,
i.e. works for everyone.

Re: Garbage collector: is 800kb a good default?

[Dmitrii Korobeinikov]

> The "right" value depends on your usage patterns, among other
> factors.  How to account for that in the default value?

I am not sure what you mean exactly by usage patterns.

> Let's not forget that running GC while Emacs is idle will make Emacs
> less responsive if the user starts typing while GC is in progress,
> specially if gc-cons-threshold is high.  So it isn't entirely "free".

Maybe it would be possible to garbage collect in chunks and check
after each chunk for input?

Best,
DK

чт, 9 апр. 2020 г. в 20:02, Eli Zaretskii <eliz@gnu.org>:
>
> > From: Dmitrii Korobeinikov <dim1212k@gmail.com>
> > Date: Thu, 9 Apr 2020 19:44:40 +0600
> > Cc: emacs-devel <emacs-devel@gnu.org>
> >
> > > Two things: (1) the timer will not run as long as Emacs is running
> > > some prolonged calculation, which could produce a lot of garbage; and
> >
> > True, that's a reason for not making it too high. Emacs would also
> > freeze very noticably in those cases. But I think the threshold
> > doesn't have to be very high to reap most of the the benefits. For
> > example, I am testing 80MB at the moment. Pretty sure I would be
> > satisfied with half that. By satisfied I mean 3.5k lines of scroll
> > action would still be plenty (before a ~1/3-1/2 second gc freeze
> > happens) and the startup time would be shaved off quite close to the
> > apparent limit anyway.
>
> The "right" value depends on your usage patterns, among other
> factors.  How to account for that in the default value?
>
> > > (2) you seem to ignore the increased memory pressure on the rest of
> > > the system from the growing memory footprint of Emacs.  On GNU/Linux,
> > > Emacs doesn't really return malloc'ed memory to the system, so once
> > > the memory footprint grows, it more or less stays that way even after
> > > GC.
> >
> > Didn't know about that. A ground of reasoning here could be what the
> > average user would see as acceptable. Of course, there are also people
> > who optimize their system for memory consumption. But they are
> > probably tech-savvy enough to find the gc options anyway.
>
> The effect also depends on how much VM does the user have on his/her
> system, and what other applications routinely run there.  Since we are
> talking about defaults, we should find a setting that is safe,
> i.e. works for everyone.

Re: Garbage collector: is 800kb a good default?

[Eli Zaretskii]

> From: Dmitrii Korobeinikov <dim1212k@gmail.com>
> Date: Thu, 9 Apr 2020 20:46:53 +0600
> Cc: emacs-devel <emacs-devel@gnu.org>
> 
> > The "right" value depends on your usage patterns, among other
> > factors.  How to account for that in the default value?
> 
> I am not sure what you mean exactly by usage patterns.

Depending on what features and packages you use frequently, your
sessions could produce more or less garbage.  The optimal threshold
should depend on that.

> > Let's not forget that running GC while Emacs is idle will make Emacs
> > less responsive if the user starts typing while GC is in progress,
> > specially if gc-cons-threshold is high.  So it isn't entirely "free".
> 
> Maybe it would be possible to garbage collect in chunks and check
> after each chunk for input?

AFAIU, this will be hard, since Lisp data structures are very
recursive.  But maybe I'm missing something.

Re: Garbage collector: is 800kb a good default?

[Dmitrii Korobeinikov]

> Depending on what features and packages you use frequently, your
> sessions could produce more or less garbage.  The optimal threshold
> should depend on that.

It would be hard to make general improvements unless one employs
general metrics. History of garbage for the past n milliseconds,
highest memory consumption for the session and such come to mind.
However I am not sure how exactly gc could use these facts while
guaranteeing that the benefits would outweigh the downsides (even on
average).

By the way, the point Stefan is making, I believe, is this: for
continuous actions, if both strategies use the same amount of memory,
the user is guaranteed to experience 100% of gc freeze if it's
non-idle-time. Meanwhile, 100% is just the worst case for the
idle-time strategy. The only tradeoff I see for the idle-time strategy
is the overhead of more frequent calls to the garbage collector. About
the user expecting an immediate response during the idle time: I think
the user always expects an immediate response. Delay is bad in any
case.

Now, what from what I gather in this thread so far, the two problems
with elevating the threshold are:

1. Large heap may slow down the gc. Tests are needed.

2. If a lot of garbage is generated after some process, the user will
experience a lag if he interacts w/ emacs right after (well, after the
idle time). This kind of scenario seems like a rarity to me. And in
case it happens, the lag probably wouldn't have to be that long
anyway. Plus, if it comes to that, sequential or concurrent gc solves
this problem in its entirety.

It would also help to know what's the point at which threshold
elevation starts giving only negligible benefits. Usage patterns /
scenarios are needed to test this.

Best,
DK

чт, 9 апр. 2020 г. в 22:42, Eli Zaretskii <eliz@gnu.org>:
>
> > From: Dmitrii Korobeinikov <dim1212k@gmail.com>
> > Date: Thu, 9 Apr 2020 20:46:53 +0600
> > Cc: emacs-devel <emacs-devel@gnu.org>
> >
> > > The "right" value depends on your usage patterns, among other
> > > factors.  How to account for that in the default value?
> >
> > I am not sure what you mean exactly by usage patterns.
>
> Depending on what features and packages you use frequently, your
> sessions could produce more or less garbage.  The optimal threshold
> should depend on that.
>
> > > Let's not forget that running GC while Emacs is idle will make Emacs
> > > less responsive if the user starts typing while GC is in progress,
> > > specially if gc-cons-threshold is high.  So it isn't entirely "free".
> >
> > Maybe it would be possible to garbage collect in chunks and check
> > after each chunk for input?
>
> AFAIU, this will be hard, since Lisp data structures are very
> recursive.  But maybe I'm missing something.

Re: Garbage collector: is 800kb a good default?

[Stefan Monnier]

> Maybe it would be possible to garbage collect in chunks and check
> after each chunk for input?

It's called "incremental GC".  XEmacs does it this way.  It'd be nice
to have, indeed.  It's often not too hard to go from "incremental" to
"concurrent", tho, so I think if we want to go there we should aim for
a concurrent GC.


        Stefan

Re: Garbage collector: is 800kb a good default?

[Dmitrii Korobeinikov]

> It's called "incremental GC".  XEmacs does it this way.

That's cool, didn't know that!

> to have, indeed.  It's often not too hard to go from "incremental" to
> "concurrent", tho, so I think if we want to go there we should aim for
> a concurrent GC.

I see.

> I'm personally not too worried about that (I don't run much more than
> Emacs on my machines), but a high GC threshold tends to lead to a large
> heap indeed and the problem with it (for me) is that it tends to make
> the GC yet slower (to which the naive reaction would be to increase the
> threshold yet further to make GC even less frequent) and it can also
> slow down the non-GC execution by increasing pressure on the memory
> hierarchy (using more VM pages and more cache lines, reducing
> effectiveness of the CPU's prefetcher) because the heap is more
> sparsely populated.
>
> This is not documented with precise measurements, sadly.
> So it's far from clear how much is too much.

Thanks for expanding on these, I didn't know the details.

For a start, I guess, just timing garbage-collect would work? Then
what's left is to pick a few various ways to generate the garbage.

Best,
DK

пт, 10 апр. 2020 г. в 01:05, Stefan Monnier <monnier@iro.umontreal.ca>:
>
> > Maybe it would be possible to garbage collect in chunks and check
> > after each chunk for input?
>
> It's called "incremental GC".  XEmacs does it this way.  It'd be nice
> to have, indeed.  It's often not too hard to go from "incremental" to
> "concurrent", tho, so I think if we want to go there we should aim for
> a concurrent GC.
>
>
>         Stefan
>

Re: Garbage collector: is 800kb a good default?

[Stefan Monnier]

>> (2) you seem to ignore the increased memory pressure on the rest of
>> the system from the growing memory footprint of Emacs.  On GNU/Linux,
>> Emacs doesn't really return malloc'ed memory to the system, so once
>> the memory footprint grows, it more or less stays that way even after GC.

I'm personally not too worried about that (I don't run much more than
Emacs on my machines), but a high GC threshold tends to lead to a large
heap indeed and the problem with it (for me) is that it tends to make
the GC yet slower (to which the naive reaction would be to increase the
threshold yet further to make GC even less frequent) and it can also
slow down the non-GC execution by increasing pressure on the memory
hierarchy (using more VM pages and more cache lines, reducing
effectiveness of the CPU's prefetcher) because the heap is more
sparsely populated.

This is not documented with precise measurements, sadly.
So it's far from clear how much is too much.


        Stefan

Re: Garbage collector: is 800kb a good default?

[Bruno Félix Rezende Ribeiro]

[-- Attachment #1: Type: text/plain, Size: 327 bytes --]

Eli Zaretskii <eliz@gnu.org> writes:

> On GNU/Linux, Emacs doesn't really return malloc'ed memory to the
> system, so once the memory footprint grows, it more or less stays that
> way even after GC.

Really?  Why is that?

-- 
Bruno Félix Rezende Ribeiro (oitofelix) [0x28D618AF]
<http://oitofelix.freeshell.org/>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: Garbage collector: is 800kb a good default?

[Eli Zaretskii]

> From: Bruno Félix Rezende Ribeiro <oitofelix@gnu.org>
> Cc: Dmitrii Korobeinikov <dim1212k@gmail.com>,  emacs-devel@gnu.org
> Date: Fri, 10 Apr 2020 11:26:36 -0300
> 
> > On GNU/Linux, Emacs doesn't really return malloc'ed memory to the
> > system, so once the memory footprint grows, it more or less stays that
> > way even after GC.
> 
> Really?  Why is that?

Because that's how malloc/free are implemented in glibc.

Re: Garbage collector: is 800kb a good default?

[Bruno Félix Rezende Ribeiro]

[-- Attachment #1: Type: text/plain, Size: 1434 bytes --]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Bruno Félix Rezende Ribeiro <oitofelix@gnu.org>
>> Cc: Dmitrii Korobeinikov <dim1212k@gmail.com>,  emacs-devel@gnu.org
>> Date: Fri, 10 Apr 2020 11:26:36 -0300
>> > On GNU/Linux, Emacs doesn't really return malloc'ed memory to the
>> > system, so once the memory footprint grows, it more or less stays that
>> > way even after GC.
>> Really?  Why is that?
> Because that's how malloc/free are implemented in glibc.

I’m surprised to find this out and I’m surprised that the GNU system
overall behaves like that (because its C library does).

How is that not a practical problem?  Sometimes I deal with very large
buffers, and it’s not uncommon to have an Emacs uptime of more than a
month.  This explains what I thought were memory leaks: Emacs
progressively eating large chunks of memory and eventually leading my
machine to die out of starvation.[1]

Can’t we use an alternative allocator that do return freed memory?[2]


Footnotes: 

[1] Not good for my dreams of using a perpetual Lisp machine.

[2] IIRC, I read somewhere that the OpenBSD one does.  I wonder if this
  and the GC behavior in general has something to do with Emacs being
  very slow and unresponsive there.  It’s a Lemote YeeLoong, but Emacs
  worked fine there on GNU/Linux.

-- 
Bruno Félix Rezende Ribeiro (oitofelix) [0x28D618AF]
<http://oitofelix.freeshell.org/>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: Garbage collector: is 800kb a good default?

[Paul Eggert]

On 4/10/20 11:26 AM, Bruno Félix Rezende Ribeiro wrote:
> Can’t we use an alternative allocator that do return freed memory?[2]

glibc malloc/free does return freed memory on occasion. See:

https://sourceware.org/glibc/wiki/MallocInternals

and look for "Free Algorithm".

Re: Garbage collector: is 800kb a good default?

[Stefan Monnier]

> Of course, raising the threshold significantly higher on its own is
> not a very good idea. But if paired with an idle timer like suggested
> here [2], then it all starts looking like a decent combination:

I agree that it's worth investigating improvements based on dynamically
changing the GC threshold.  What you're pointing out is that from the
user's point of view, a GC during idle time is free (so it can occur
frequently) and also that a GC during non-idle time can delay redisplay,
so (ignoring all other impacts of GC) we should refrain from running GC
while non-idle and trigger GC everything we're idle.

Running GC frequently is done largely to try and keep total heap size
and memory fragmentation under control.  So completely refraining from
running GC while non-idle is probably not a great solution, but I do
agree that adding a "GC when idle" would not only be great in itself but
would also let us increase the default thresholds somewhat.  Tho I don't
think we can increase the threshold as much as you suggest without
having significant detrimental effects.

There 2 additional ways to attack the problem, BTW:

1- replace our GC with a concurrent GC.  This would let us move those
   38% GC overheard to one of the other CPU cores sitting idle while the
   only CPU core running Emacs is frantically trying to scroll through
   your buffer (it would also slow down both the GC and the main thread
   a bit, but in the current context of plentiful CPU cores it would
   still be very worthwhile).
   [ Along the same lines, making our GC parallel could cut those 38%
     down to some extent, as would a generational GC.  ]

2- 38% is pretty high.  Usually this indicates that the GC is not
   efficient: it works hard yet reclaims very little memory likely
   because the application is allocating a lot of objects which are
   *not* temporary.  This is also the typical situation during Emacs's
   startup where we're loading packages and initializing big
   data-structures, so after allocating our GC-threshold of data the GC
   is run but doesn't collect much garbage because all that data is
   there to stay.
   We can't really know beforehand if a GC will reclaim a lot of memory,
   but we could "use the past to predict the future": we could set the
   threshold higher when the last GC reclaimed too little garbage.

I had sent a tentative patch for the "GC when idle" feature but it had
some rough edges, and some details of the GC code have changed since.
I'd welcome a patch to do that.  It shouldn't require many changes.
Similarly, point (2) above should be fairly simple to add.


        Stefan

Re: Garbage collector: is 800kb a good default?

[Dmitrii Korobeinikov]

> Running GC frequently is done largely to try and keep total heap size
> and memory fragmentation under control.  So completely refraining from
> running GC while non-idle is probably not a great solution, but I do
> agree that adding a "GC when idle" would not only be great in itself but
> would also let us increase the default thresholds somewhat.  Tho I don't
> think we can increase the threshold as much as you suggest without
> having significant detrimental effects.

So, are there any stats on how much worse the fragmentation gets w/
higher thresholds and how much it effects emacs as a result? (By the
way, I am not really suggesting 80MB, just testing it out at the
moment : ) )

> 2- 38% is pretty high

Just for the reference, I've seen the value of 25 in one of the
threads nearby [1], not sure why mine is so high.

Best,
DK

[1] https://lists.gnu.org/archive/html/emacs-devel/2020-04/msg00406.html

чт, 9 апр. 2020 г. в 19:47, Stefan Monnier <monnier@iro.umontreal.ca>:
>
> > Of course, raising the threshold significantly higher on its own is
> > not a very good idea. But if paired with an idle timer like suggested
> > here [2], then it all starts looking like a decent combination:
>
> I agree that it's worth investigating improvements based on dynamically
> changing the GC threshold.  What you're pointing out is that from the
> user's point of view, a GC during idle time is free (so it can occur
> frequently) and also that a GC during non-idle time can delay redisplay,
> so (ignoring all other impacts of GC) we should refrain from running GC
> while non-idle and trigger GC everything we're idle.
>
> Running GC frequently is done largely to try and keep total heap size
> and memory fragmentation under control.  So completely refraining from
> running GC while non-idle is probably not a great solution, but I do
> agree that adding a "GC when idle" would not only be great in itself but
> would also let us increase the default thresholds somewhat.  Tho I don't
> think we can increase the threshold as much as you suggest without
> having significant detrimental effects.
>
> There 2 additional ways to attack the problem, BTW:
>
> 1- replace our GC with a concurrent GC.  This would let us move those
>    38% GC overheard to one of the other CPU cores sitting idle while the
>    only CPU core running Emacs is frantically trying to scroll through
>    your buffer (it would also slow down both the GC and the main thread
>    a bit, but in the current context of plentiful CPU cores it would
>    still be very worthwhile).
>    [ Along the same lines, making our GC parallel could cut those 38%
>      down to some extent, as would a generational GC.  ]
>
> 2- 38% is pretty high.  Usually this indicates that the GC is not
>    efficient: it works hard yet reclaims very little memory likely
>    because the application is allocating a lot of objects which are
>    *not* temporary.  This is also the typical situation during Emacs's
>    startup where we're loading packages and initializing big
>    data-structures, so after allocating our GC-threshold of data the GC
>    is run but doesn't collect much garbage because all that data is
>    there to stay.
>    We can't really know beforehand if a GC will reclaim a lot of memory,
>    but we could "use the past to predict the future": we could set the
>    threshold higher when the last GC reclaimed too little garbage.
>
> I had sent a tentative patch for the "GC when idle" feature but it had
> some rough edges, and some details of the GC code have changed since.
> I'd welcome a patch to do that.  It shouldn't require many changes.
> Similarly, point (2) above should be fairly simple to add.
>
>
>         Stefan
>

Re: Garbage collector: is 800kb a good default?

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Date: Thu, 09 Apr 2020 09:47:38 -0400
> Cc: emacs-devel <emacs-devel@gnu.org>
> 
> What you're pointing out is that from the user's point of view, a GC
> during idle time is free (so it can occur frequently) and also that
> a GC during non-idle time can delay redisplay, so (ignoring all
> other impacts of GC) we should refrain from running GC while
> non-idle and trigger GC everything we're idle.

Let's not forget that running GC while Emacs is idle will make Emacs
less responsive if the user starts typing while GC is in progress,
specially if gc-cons-threshold is high.  So it isn't entirely "free".

Also, we already try running GC each time Emacs becomes idle.

Re: Garbage collector: is 800kb a good default?

[Stefan Monnier]

>> What you're pointing out is that from the user's point of view, a GC
>> during idle time is free (so it can occur frequently) and also that
>> a GC during non-idle time can delay redisplay, so (ignoring all
>> other impacts of GC) we should refrain from running GC while
>> non-idle and trigger GC everything we're idle.
> Let's not forget that running GC while Emacs is idle will make Emacs
> less responsive if the user starts typing while GC is in progress,

The argument for GC-when-idle goes that if this happens it's not really
worse than what we have now, because if the GC hadn't started "before
the user starts typing", it would have run as part of the command
launched by the user, so from the user's point of view there's no
noticeable difference.

> specially if gc-cons-threshold is high.  So it isn't entirely "free".

In my sample patch I set the "idle GC threshold" to half of the normal
threshold, which means that we'll GC about twice as often.  In such
a case, as long as the probability that the "user starts typing" in the
middle of a GC is less than 50%, the user's commands will not be delayed
by GC more often than with the current strategy.

The actual probability that the user starts typing in the middle of an
idle GC depends on how much idle we wait before starting the GC, how
much a GC takes, and the distribution of time between user events.
But it should be pretty easy to keep it below 50%.

> Also, we already try running GC each time Emacs becomes idle.

Do we?  I thought so as well, but I couldn't see any evidence of it in
the code.


        Stefan

Re: Garbage collector: is 800kb a good default?

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: dim1212k@gmail.com,  emacs-devel@gnu.org
> Date: Thu, 09 Apr 2020 14:42:56 -0400
> 
> > Let's not forget that running GC while Emacs is idle will make Emacs
> > less responsive if the user starts typing while GC is in progress,
> 
> The argument for GC-when-idle goes that if this happens it's not really
> worse than what we have now, because if the GC hadn't started "before
> the user starts typing", it would have run as part of the command
> launched by the user, so from the user's point of view there's no
> noticeable difference.

The difference is that when an idle timer runs a long uninterruptible
calculation, the user thinks Emacs is idle, so having a character
appear, or Emacs respond to a command with a prompt, after a
significant delay makes a very bad UX.  By contrast, when a command is
running, the user will normally know that Emacs isn't idle, so a
delayed response will be less of a surprise.

> > Also, we already try running GC each time Emacs becomes idle.
> 
> Do we?  I thought so as well, but I couldn't see any evidence of it in
> the code.

It's in keyboard.c, right after we auto-save when enough idle time has
passed:

      /* If there is still no input available, ask for GC.  */
      if (!detect_input_pending_run_timers (0))
	maybe_gc ();

Re: Garbage collector: is 800kb a good default?

[Stefan Monnier]

> The difference is that when an idle timer runs a long uninterruptible
> calculation, the user thinks Emacs is idle, so having a character
> appear, or Emacs respond to a command with a prompt, after a
> significant delay makes a very bad UX.  By contrast, when a command is
> running, the user will normally know that Emacs isn't idle, so a
> delayed response will be less of a surprise.

No, that's not what I was saying.  What I was saying is that in the"idle
GC case" the events are as follows:

1- 100%-N% of the GC runs while idle.
2- the user hits a key
3- the remaining N% of the GC runs.
4- the command runs

whereas in the non-idle GC what happens is:

1- the user hits a key
2- the command starts
3- the GC gets run
4- the command continues and finiches

To the user, the result is pretty much the same: in both cases Emacs
took "time to run the command + time to GC" before responding.  In the
idle-GC the time should be arguably a bit *shorter* because only N% of
the GC is counted towards to total delay before sending the response.

>> > Also, we already try running GC each time Emacs becomes idle.
>> Do we?  I thought so as well, but I couldn't see any evidence of it in
>> the code.
> It's in keyboard.c, right after we auto-save when enough idle time has
> passed:
>
>       /* If there is still no input available, ask for GC.  */
>       if (!detect_input_pending_run_timers (0))
> 	maybe_gc ();

But that's not the same: this "maybe_gc" rarely triggers because it uses
the same thresholds as normal code evaluation, so if the GC was not
triggered while running the last command it usually won't trigger
here either.

The fundamental idea of idle-GC is to use a *lower* threshold while
idle, so as to increase the probability that the GC happens on that
occasion rather than one of the other "maybe_gc" checks.


        Stefan

Re: Garbage collector: is 800kb a good default?

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: dim1212k@gmail.com,  emacs-devel@gnu.org
> Date: Thu, 09 Apr 2020 15:31:43 -0400
> 
> No, that's not what I was saying.  What I was saying is that in the"idle
> GC case" the events are as follows:
> 
> 1- 100%-N% of the GC runs while idle.
> 2- the user hits a key
> 3- the remaining N% of the GC runs.
> 4- the command runs
> 
> whereas in the non-idle GC what happens is:
> 
> 1- the user hits a key
> 2- the command starts
> 3- the GC gets run
> 4- the command continues and finiches
> 
> To the user, the result is pretty much the same: in both cases Emacs
> took "time to run the command + time to GC" before responding.

We are miscommunicating.  My complaint was about the _response_ time,
which is the time that passes between me pressing a key and Emacs
reacting to that.  In your description above it is the time between
"the user hits a key" and "the command starts".  That GC happens
during the time the command runs is less annoying, because the user
sees a responsive Emacs.

You can try it yourself: set up an idle timer that would call a
long-running function, then try typing something while it runs.

> >       /* If there is still no input available, ask for GC.  */
> >       if (!detect_input_pending_run_timers (0))
> > 	maybe_gc ();
> 
> But that's not the same: this "maybe_gc" rarely triggers because it uses
> the same thresholds as normal code evaluation, so if the GC was not
> triggered while running the last command it usually won't trigger
> here either.

We have no guarantee that GC has an opportunity to run every command,
do we?  It very much depends on what that command did.  Thus, this
call is not really a no-op.

> The fundamental idea of idle-GC is to use a *lower* threshold while
> idle, so as to increase the probability that the GC happens on that
> occasion rather than one of the other "maybe_gc" checks.

We can easily do that, I think.  Most of the code is already there.

Re: Garbage collector: is 800kb a good default?

[Stefan Monnier]

>> No, that's not what I was saying.  What I was saying is that in the"idle
>> GC case" the events are as follows:
>> 
>> 1- 100%-N% of the GC runs while idle.
>> 2- the user hits a key
>> 3- the remaining N% of the GC runs.
>> 4- the command runs
>> 
>> whereas in the non-idle GC what happens is:
>> 
>> 1- the user hits a key
>> 2- the command starts
>> 3- the GC gets run
>> 4- the command continues and finiches
>> 
>> To the user, the result is pretty much the same: in both cases Emacs
>> took "time to run the command + time to GC" before responding.
>
> We are miscommunicating.  My complaint was about the _response_ time,
> which is the time that passes between me pressing a key and Emacs
> reacting to that.  In your description above it is the time between
> "the user hits a key" and "the command starts".

The user can't know when Emacs reacts to the key: in both cases there's
no sign of life until point 4 finishes (that's assuming the command is
something like self-insert-command where there's no user interaction
*during* the command).


        Stefan

Re: Garbage collector: is 800kb a good default?

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: dim1212k@gmail.com,  emacs-devel@gnu.org
> Date: Fri, 10 Apr 2020 09:03:05 -0400
> 
> > We are miscommunicating.  My complaint was about the _response_ time,
> > which is the time that passes between me pressing a key and Emacs
> > reacting to that.  In your description above it is the time between
> > "the user hits a key" and "the command starts".
> 
> The user can't know when Emacs reacts to the key: in both cases there's
> no sign of life until point 4 finishes (that's assuming the command is
> something like self-insert-command where there's no user interaction
> *during* the command).

Even for self-insert-command, I can easily know whether the command
started or not, because I have garbage-collection-messages turned on.

And many commands do include user interaction.  "C-x C-f" and "C-x b"
come to mind.  They are very frequent in my use patterns, and
typically one of them happens after some amount of idle time, so idle
timers kick in.  As result, I needed to tune several long-running idle
timers.  I'd hate to see GC becoming one of them.

Re: Garbage collector: is 800kb a good default?

[Stefan Monnier]

>> > We are miscommunicating.  My complaint was about the _response_ time,
>> > which is the time that passes between me pressing a key and Emacs
>> > reacting to that.  In your description above it is the time between
>> > "the user hits a key" and "the command starts".
>> 
>> The user can't know when Emacs reacts to the key: in both cases there's
>> no sign of life until point 4 finishes (that's assuming the command is
>> something like self-insert-command where there's no user interaction
>> *during* the command).
>
> Even for self-insert-command, I can easily know whether the command
> started or not, because I have garbage-collection-messages turned on.

Forget it, you don't seem to be able to understand my point, which is
that idle-GC just moves the moment the GC happens.  Without it, GC is
pretty much guaranteed to slow down execution of the user's command.

So if occasionally the user hits a key during the idle-GC, he just gets
to experience the delay that he would otherwise *always* experience.

> timers kick in.  As result, I needed to tune several long-running idle
> timers.  I'd hate to see GC becoming one of them.

The purpose of the idle-GC is not to *add* more background activity, but
to move activity from active time to idle time.  So if it works as
intended it is not comparable to those other long-running idle timers.


        Stefan

Re: Garbage collector: is 800kb a good default?

[Stefan Monnier]

> Forget it, you don't seem to be able to understand my point,

Oh boy!  I really should refrain from posting when I'm stressed.
I'm really sorry about that,


        Stefan

Re: Garbage collector: is 800kb a good default?

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: dim1212k@gmail.com,  emacs-devel@gnu.org
> Date: Fri, 10 Apr 2020 10:34:02 -0400
> 
> Forget it

Done.

Re: Garbage collector: is 800kb a good default?

[Andrea Corallo]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> So if occasionally the user hits a key during the idle-GC, he just gets
> to experience the delay that he would otherwise *always* experience.
>
> The purpose of the idle-GC is not to *add* more background activity, but
> to move activity from active time to idle time.  So if it works as
> intended it is not comparable to those other long-running idle timers.

Following the discussion seems to me that the strategy you are
discussing for the idle GC is exactly what I've implemented in the GCMH
[1].

This into MELPA since about one year (I was without copyright assignment
at the time) but I'd be happy to have it in ELPA or to have it reused
somehow if it comes handy.

Andrea

[1] https://melpa.org/#/gcmh

-- 
akrl@sdf.org

Re: Garbage collector: is 800kb a good default?

[Stefan Monnier]

> Following the discussion seems to me that the strategy you are
> discussing for the idle GC is exactly what I've implemented in the
> GCMH [1].

Indeed.

> This into MELPA since about one year (I was without copyright assignment
> at the time) but I'd be happy to have it in ELPA or to have it reused
> somehow if it comes handy.

I'd like to have it activated by default (which might require extra
polishing ;-)


        Stefan

Re: Garbage collector: is 800kb a good default?

[Andrea Corallo]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> I'd like to have it activated by default (which might require extra
> polishing ;-)

Be my guest in case.

-- 
akrl@sdf.org

Re: Garbage collector: is 800kb a good default?

[Andrea Corallo]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> Of course, raising the threshold significantly higher on its own is
>> not a very good idea. But if paired with an idle timer like suggested
>> here [2], then it all starts looking like a decent combination:
>
> I agree that it's worth investigating improvements based on dynamically
> changing the GC threshold.

This what I run when I'm on X86 machines or SoC where for various
reasons memory latency is a pain.  In this conditions I find it helps
considerably for me.

https://gitlab.com/koral/gcmh/-/blob/master/gcmh.el

On normal conditions I often don't bother.

Andrea

--
akrl@sdf.org