reducing emacs size by more frequent garbage-collect in loadup.el

reducing emacs size by more frequent garbage-collect in loadup.el

[Dan Nicolaescu]

As the Subject says, replacing each `load' line in loadup.el with 
`load' + `garbage-collect' will reduce the size of the stripped emacs
binary: (.7 is before, .8 is after the change)

$ ls -l emacs-23.1.50.8 emacs-23.1.50.7
-rwx------ 1 dann dann 6722788 Jul 24 14:20 emacs-23.1.50.8*
-rwx------ 1 dann dann 6857956 Jul 24 14:20 emacs-23.1.50.7*

$ size  emacs-23.1.50.8 emacs-23.1.50.7
   text    data     bss     dec     hex filename
1883659 4833256       0 6716915  667df3 emacs-23.1.50.8
1883659 4968424       0 6852083  688df3 emacs-23.1.50.7

so we get about 2% reduction by doing something very simple and safe... 

[This happens because loading multiple files generate more garbage that
can be collected, but it is not returned to the OS, so it appears in
the dumped image].

Should we make this change?

Re: reducing emacs size by more frequent garbage-collect in loadup.el

[Stefan Monnier]

> As the Subject says, replacing each `load' line in loadup.el with 
> `load' + `garbage-collect' will reduce the size of the stripped emacs
> binary: (.7 is before, .8 is after the change)

> $ ls -l emacs-23.1.50.8 emacs-23.1.50.7
> -rwx------ 1 dann dann 6722788 Jul 24 14:20 emacs-23.1.50.8*
> -rwx------ 1 dann dann 6857956 Jul 24 14:20 emacs-23.1.50.7*

> $ size  emacs-23.1.50.8 emacs-23.1.50.7
>    text    data     bss     dec     hex filename
> 1883659 4833256       0 6716915  667df3 emacs-23.1.50.8
> 1883659 4968424       0 6852083  688df3 emacs-23.1.50.7

> so we get about 2% reduction by doing something very simple and safe... 

> [This happens because loading multiple files generate more garbage that
> can be collected, but it is not returned to the OS, so it appears in
> the dumped image].

> Should we make this change?

I'd rather not uglify loadup.el with tons of calls to garbage-collect,
but maybe it's OK to replace all calls to load by calls to load&gc.
BTW, how does it affect the time to dump Emacs?

Another way to win similar improvements might be to lower the value of
gc-cons-threshold and friends during loadup.el.


        Stefan

Re: reducing emacs size by more frequent garbage-collect in loadup.el

[Dan Nicolaescu]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

  > > As the Subject says, replacing each `load' line in loadup.el with 
  > > `load' + `garbage-collect' will reduce the size of the stripped emacs
  > > binary: (.7 is before, .8 is after the change)
  > 
  > > $ ls -l emacs-23.1.50.8 emacs-23.1.50.7
  > > -rwx------ 1 dann dann 6722788 Jul 24 14:20 emacs-23.1.50.8*
  > > -rwx------ 1 dann dann 6857956 Jul 24 14:20 emacs-23.1.50.7*
  > 
  > > $ size  emacs-23.1.50.8 emacs-23.1.50.7
  > >    text    data     bss     dec     hex filename
  > > 1883659 4833256       0 6716915  667df3 emacs-23.1.50.8
  > > 1883659 4968424       0 6852083  688df3 emacs-23.1.50.7
  > 
  > > so we get about 2% reduction by doing something very simple and safe... 
  > 
  > > [This happens because loading multiple files generate more garbage that
  > > can be collected, but it is not returned to the OS, so it appears in
  > > the dumped image].
  > 
  > > Should we make this change?
  > 
  > I'd rather not uglify loadup.el with tons of calls to garbage-collect,
  > but maybe it's OK to replace all calls to load by calls to load&gc.
  > BTW, how does it affect the time to dump Emacs?

It's in the noise for a 3 year old machine that I tried it on.

  > Another way to win similar improvements might be to lower the value of
  > gc-cons-threshold and friends during loadup.el.

This sounds like it's a bit more complicated...

Re: reducing emacs size by more frequent garbage-collect in loadup.el

[Ken Raeburn]

On Jul 26, 2009, at 14:57, Dan Nicolaescu wrote:
>> Another way to win similar improvements might be to lower the value  
>> of
>> gc-cons-threshold and friends during loadup.el.
>
> This sounds like it's a bit more complicated...

It should just take a 'let' binding around most of the interesting  
parts of loadup.el.  (Or setq, if you don't mind not recovering the  
default value set in C code.)  So it means reindenting, but not  
dropping in lots of explicit GC calls.  It also means GC will run more  
frequently during loads, instead of in between.  There's also a lower  
limit, below which gc-cons-threshold will be reset during GC.  So the  
impact in time and space may be quite different.

I'd also be surprised if there didn't turn out to be one or two  
strategic places in loadup.el where you could insert GC calls and get  
back quite a portion of that 2%.  In the early parts of the process,  
where most of the garbage slots generated are likely to be filled by  
files loaded later, doing extra GC passes just seems like wasted work.

On the other hand, I would expect that 2% savings in file size is  
mainly just saving disk space; the extra memory in the larger image  
should be reused fairly soon after starting up the dumped Emacs as it  
allocates and GCs, shouldn't it, where the smaller Emacs will just  
have to allocate the space from the OS a bit sooner?  Is there any  
impact on run-time memory use?  At a difference of 132K, maybe it  
would take a little while for the smaller one to catch up, but how  
much does it take -- a few editing or programming modes to be loaded?   
Running font-lock over a large C file?

I just did a quick experiment on my Mac; let-binding gc-cons-threshold  
to its minimum value of 10000 for the duration of the loads and other  
work cut 80K of the data-section size in the dumped image, which is  
less than 1%; by the time the processes started up, created X11  
windows, and loaded my .emacs, process memory size differed by only  
32K, which suggests to me that most of that space saved in the dumped  
image was needed soon after startup anyways, though not all of it.

Ken

Re: reducing emacs size by more frequent garbage-collect in loadup.el

[Dan Nicolaescu]

Ken Raeburn <raeburn@raeburn.org> writes:

  > On Jul 26, 2009, at 14:57, Dan Nicolaescu wrote:
  > >> Another way to win similar improvements might be to lower the value
  > >> of
  > >> gc-cons-threshold and friends during loadup.el.
  > >
  > > This sounds like it's a bit more complicated...
  > 
  > It should just take a 'let' binding around most of the interesting
  > parts of loadup.el.  (Or setq, if you don't mind not recovering the
  > default value set in C code.)  So it means reindenting, but not
  > dropping in lots of explicit GC calls.  It also means GC will run more
  > frequently during loads, instead of in between.  There's also a lower
  > limit, below which gc-cons-threshold will be reset during GC.  So the
  > impact in time and space may be quite different.
  > 
  > I'd also be surprised if there didn't turn out to be one or two
  > strategic places in loadup.el where you could insert GC calls and get
  > back quite a portion of that 2%.  In the early parts of the process,
  > where most of the garbage slots generated are likely to be filled by
  > files loaded later, doing extra GC passes just seems like wasted work.

Given that the time it takes is in the noise, it doesn't really matter.
I'm a bit uneasy about that approach because setting the gc thresholds
might to be adjusted from time to time.  And also we have never run with
different thresholds during dumping and during normal usage, so that
might be a small risk too.

Whereas just adding a gc call after each load will not need any extra
maintenance.  When new files are added, it's obvious to the person that
does the addition that there's a need to add a gc call too.

  > On the other hand, I would expect that 2% savings in file size is
  > mainly just saving disk space; 

Yes, this is about saving disk space, and it can be done with minimum
work and no extra future maintenance.

Re: reducing emacs size by more frequent garbage-collect in loadup.el

[Dan Nicolaescu]

Dan Nicolaescu <dann@ics.uci.edu> writes:

  > Stefan Monnier <monnier@iro.umontreal.ca> writes:
  > 
  >   > > As the Subject says, replacing each `load' line in loadup.el with 
  >   > > `load' + `garbage-collect' will reduce the size of the stripped emacs
  >   > > binary: (.7 is before, .8 is after the change)
  >   > 
  >   > > $ ls -l emacs-23.1.50.8 emacs-23.1.50.7
  >   > > -rwx------ 1 dann dann 6722788 Jul 24 14:20 emacs-23.1.50.8*
  >   > > -rwx------ 1 dann dann 6857956 Jul 24 14:20 emacs-23.1.50.7*
  >   > 
  >   > > $ size  emacs-23.1.50.8 emacs-23.1.50.7
  >   > >    text    data     bss     dec     hex filename
  >   > > 1883659 4833256       0 6716915  667df3 emacs-23.1.50.8
  >   > > 1883659 4968424       0 6852083  688df3 emacs-23.1.50.7
  >   > 
  >   > > so we get about 2% reduction by doing something very simple and safe... 
  >   > 
  >   > > [This happens because loading multiple files generate more garbage that
  >   > > can be collected, but it is not returned to the OS, so it appears in
  >   > > the dumped image].
  >   > 
  >   > > Should we make this change?
  >   > 
  >   > I'd rather not uglify loadup.el with tons of calls to garbage-collect,
  >   > but maybe it's OK to replace all calls to load by calls to load&gc.
  >   > BTW, how does it affect the time to dump Emacs?
  > 
  > It's in the noise for a 3 year old machine that I tried it on.

Ping.  Any decision on this?

Re: reducing emacs size by more frequent garbage-collect in loadup.el

[Stephen J. Turnbull]

Stefan Monnier writes:

 > I'd rather not uglify loadup.el with tons of calls to garbage-collect,
 > but maybe it's OK to replace all calls to load by calls to load&gc.
 > BTW, how does it affect the time to dump Emacs?

I don't recall the mechanism offhand, but XEmacs has always (for
values of "always" == as long as I've known anything about the dump
process) gc'd after every load during dump.  This has caused zero
complaints that I am aware of.

For developers, there's an optional --quickbuild configure option
which disables GC entirely during the default make (as well as not
generating info or DOC or finder.el, etc).  Obviously not something
you want for a production Emacs, but may be just the ticket for people
doing delicate work on the C code or Lisp primitives.