Re: Larger GC thresholds for non-interactive Emacs

[Stefan Monnier <monnier@iro.umontreal.ca>]

>> Maybe it's time for some benchmarks with various values for these knobs
>> to see if the values we're using now are "good enough" or might need to
>> be changed.
>
> Note that I did not initially ask to change the GC defaults for normal
> Emacs session.

You're absolutely right, but we've had various reports of late that
indicate that we're probably spending too much time in the GC.
So I think we're due for some re-evaluation.

> What I am trying to suggest is:
> 1. Have some conservative GC settings in interactive Emacs session (as
>    we do now; I do _not_ suggest changing it)
> 2. Increase GC thresholds in non-interactive --batch sessions, which
>    tend to not run for a long time.

Agreed.  Then again, we occasionally also use long-running commands
interactively so it might be worthwhile to tune the settings for
interactive use as well.

> I am a bit afraid to touch the topic of changing the interactive GC
> defaults.  People tend to have contradicting opinions on this.

:-)

The issue is that it's easy to see "oh god, we're spending so much time
in the GC, let's disable it" but much harder to see the "long term"
impact of increasing the thresholds (which tend to increase the
fragmentation and the total heap size both of which tend to slow down
execution but the second additionally makes every GC cycle take longer,
again increasing the pressure from naive onlookers to disable the GC).

> I'd like to ask one thing from the very beginning here: Please spin a
> separate thread if you have objections on actual GC defaults applicable
> to normal Emacs sessions.

No, I think focusing on batch use is a good idea.  Also because it makes
it much easier to actually measure the impact.

> A possible useful thing Emacs could help with would be a macro dedicated
> to let-binds like the above. Something like:
>
> (with-reduced-gc
>  ;; Do staff)

Sounds about right, tho I think the name of the macro should be related
to what the code does rather than to what the author thinks the GC should
do about it.  Something like `this-code-does-not-generate-garbage`.

>  with-reduced-gc could take care about determining the inner specifics
>  on what alternative gc-cons-threshold value should be used (possibly
>  depending on the system memory information).

Sounds good.

>> It's hard to know beforehand whether a GC will be useful or not, tho.
>> But maybe we can find good heuristics.  E.g. have something like
>> a `gc-cons-percentage` which depends on how much garbage we collected in
>> the last GC: if a GC doesn't collect any garbage (or very little of it),
>> it's a sign that we're in a phase where running the GC is not very
>> useful so we should wait a bit more until the next GC, whereas if the GC
>> collected a fair bit of garbage, it's a sign that we're in a phase where
>> running the GC is beneficial and we can run it a bit more often.
>
> FYI, I have played with this approach making use of
> https://gitlab.com/koral/gcmh

But this one focuses on interactive use.  The kind of heuristic I'm
proposing above would only make sense within a single (long-running)
command, or in batch mode, I think.


        Stefan