Re: Increase gc-cons-threshold during package-init?

[Stefan Monnier <monnier@iro.umontreal.ca>]

> With Emacs now calling package-initialize on its own,

Not sure what you mean by "now": it's been the case since Emacs-24.1
(recent changes on `master` have only moved the time at which it takes
place).

> that emacs startup was now  entailing 3 times as many gc calls during
> startup as before.  One guess is that my init file was lexically
> binding gc-cons-threshold to 64MB ---

Lexically-binding it would have no effect on the GC, and since that var
is declared as dynamically scoped, it was probably dynamically bound.

Have you checked how much extra time is spent in the GC because of those
extra calls?

> the default gc-cons-threshold of 800K is likely from the past and
> could be increased.

This is a misunderstanding: the real gc-cons-threshold is computed
dynamically based on the heap size (see gc-cons-percentage).

Maybe those values need to be tweaked because tradeoffs have shifted
somewhat, but it doesn't seem very likely.

The problem with the above GC calls is much more fundamental and
unrelated to the actual values we choose: during periods of
initialization, programs typically allocate a lot of memory without
generating much garbage.  So during those times running a GC tends to be
a waste of effort (we scan the whole heap only to find a few crumbs here
and there).

People often look at situations like yours and conclude that
gc-cons-threshold is too small.  Yes, it's too small for those cases,
but bumping it once and forall makes things worse for the "usual" case,
leading to increased heap size, fragmentation, and longer pauses during
normal operation (tho this "usual case" is much more difficult to
measure).

There's no sadly way the GC can magically know that the last N megabytes
of allocation generated very little garbage, tho.

Maybe should bump those gc-knobs around the startup.el code (or maybe
more generally we should bump those knobs while loading a file).

> Even if we leave the default as is, it might make sense to temporarily
> increase gc-cons-threshold within package-initialize

Right.  Tho doing this will likely cause a GC to happen right after
package-initialize, i.e. just before loading your ~/.emacs.
Better would be to bump it around both, I guess.


        Stefan