BASE_PURESIZE

BASE_PURESIZE

[Nick Roberts]

I get the warning message "Building Emacs overflowed pure space".  Assuming
this is related to decreasing BASE_PURESIZE, the argument for keeping it to a
minimum for computers with little memory against the inconvenience of
repeatedly tripping over it, is completely lost on me.  Especially with a
development version (as we could. presumably, just pare it back prior to
release).


-- 
Nick                                           http://www.inet.net.nz/~nickrob

Re: BASE_PURESIZE

[Eli Zaretskii]

> From: Nick Roberts <nickrob@snap.net.nz>
> Date: Sat, 5 Aug 2006 14:16:54 +1200
> 
> I get the warning message "Building Emacs overflowed pure space".  Assuming
> this is related to decreasing BASE_PURESIZE, the argument for keeping it to a
> minimum for computers with little memory against the inconvenience of
> repeatedly tripping over it, is completely lost on me.

No, it was decreased because we made a change lately that lowers the
required pure size by several tens of KB, IIRC.

Re: BASE_PURESIZE

[Reiner Steib]

On Sat, Aug 05 2006, Eli Zaretskii wrote:

>> From: Nick Roberts <nickrob@snap.net.nz>
>> Date: Sat, 5 Aug 2006 14:16:54 +1200
>> 
>> I get the warning message "Building Emacs overflowed pure space".
>> Assuming this is related to decreasing BASE_PURESIZE, the argument
>> for keeping it to a minimum for computers with little memory
>> against the inconvenience of repeatedly tripping over it, is
>> completely lost on me.
>
> No, it was decreased because we made a change lately that lowers the
> required pure size by several tens of KB, IIRC.

FWIW, starting with this change, I had to set SITELOAD_PURESIZE_EXTRA
also[1] on the machine (SUSE 10.0 [2]) on which it wasn't necessary
until some weeks before this change.

Unless we find (and fix) the reason for the different pure space
requirements, it would be good to increase BASE_PURESIZE to an amount
that allows building Emacs without an overflow on most machines, I
think.

Bye, Reiner.

[1] I still need to increase it on the machines as reported
    previously[3].  On i686 (2x Intel Xeon (2.40GHz)) and x86_64 (AMD
    Athlon 64 3500+):

$ gcc --version
gcc (GCC) 3.3.4 (pre 3.3.5 20040809)
[...]
$ cat /etc/SuSE-release 
SuSE Linux 9.2 (i586)         [... or x86_64 resp.]
VERSION = 9.2

[2] i686 (Intel(R) Pentium(R) M processor 1500MHz):
$ cat /etc/SuSE-release 
SUSE LINUX 10.0 (i586) OSS
VERSION = 10.0
$ gcc --version
gcc (GCC) 4.0.2 20050901 (prerelease) (SUSE Linux)

[3] See the thread around
    <http://thread.gmane.org/v9d5diibsr.fsf%40marauder.physik.uni-ulm.de>.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

Re: BASE_PURESIZE

[Richard Stallman]

    Unless we find (and fix) the reason for the different pure space
    requirements, it would be good to increase BASE_PURESIZE to an amount
    that allows building Emacs without an overflow on most machines, I
    think.

How much increase do you propose?

Re: BASE_PURESIZE

[Reiner Steib]

On Sat, Aug 05 2006, Richard Stallman wrote:

>     Unless we find (and fix) the reason for the different pure space
>     requirements, it would be good to increase BASE_PURESIZE to an amount
>     that allows building Emacs without an overflow on most machines, I
>     think.
>
> How much increase do you propose?

The last time I tried to minimize the extra space in 10k steps, I had
to set SITELOAD_PURESIZE_EXTRA to values between 100000 and 200000
(different for i686 and x86_64) to avoid the overflow.

But I didn't try it recently with the current sources (and I cannot do
this at the moment on the other machines).

For the 2006-07-29 sources, I need 150000 extra bytes on my SUSE 10.0
installation:
1102000 is the default BASE_PURESIZE.  During dumping, I get:
1243908 pure bytes used

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

Re: BASE_PURESIZE

[Richard Stallman]

    For the 2006-07-29 sources, I need 150000 extra bytes on my SUSE 10.0
    installation:
    1102000 is the default BASE_PURESIZE.  During dumping, I get:
    1243908 pure bytes used

That's more or less what it used before the optimizations.  It sounds
like something is wrong, and those optimizations are not working in
your case.

Re: BASE_PURESIZE

[Reiner Steib]

On Sun, Aug 06 2006, Richard Stallman wrote:

>     For the 2006-07-29 sources, I need 150000 extra bytes on my SUSE 10.0
>     installation:
>     1102000 is the default BASE_PURESIZE.  During dumping, I get:
>     1243908 pure bytes used
>
> That's more or less what it used before the optimizations.  It sounds
> like something is wrong, and those optimizations are not working in
> your case.

That's possible.  Until mid of June (some weeks before the
optimizations), the default value was sufficient on this machine
(Pentium M 1500MHz; gcc (GCC) 4.0.2 20050901 (prerelease); SUSE 10.0).
I didn't update the sources often on this host, so I'm not sure when
got the overflow the first time.

On the other machines, I had to add some extra space at least since
April.  See the thread "PURESIZE increased (again)" (April-June)
<URL:http://thread.gmane.org/v9d5diibsr.fsf%40marauder.physik.uni-ulm.de>.

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

Re: BASE_PURESIZE

[Reiner Steib]

On Sat, Aug 05 2006, Reiner Steib wrote:

> On Sat, Aug 05 2006, Richard Stallman wrote:
>
>>     Unless we find (and fix) the reason for the different pure space
>>     requirements, it would be good to increase BASE_PURESIZE to an amount
>>     that allows building Emacs without an overflow on most machines, I
>>     think.
>>
>> How much increase do you propose?
>
> The last time I tried to minimize the extra space in 10k steps, I had
> to set SITELOAD_PURESIZE_EXTRA to values between 100000 and 200000
> (different for i686 and x86_64) to avoid the overflow.
>
> But I didn't try it recently with the current sources (and I cannot do
> this at the moment on the other machines).

On i686 (2x Intel Xeon 2.40GHz) I get "1245323 pure bytes used".
On x86_64 (AMD Athlon 64 3500+) I get "2064768 pure bytes used".

Both have the same gcc (GCC) 3.3.4 (pre 3.3.5 20040809) and both are
running SuSE 9.2.  I updated the sources from CVS just before testing
and sending this mail.

> For the 2006-07-29 sources, I need 150000 extra bytes on my SUSE 10.0
> installation:
> 1102000 is the default BASE_PURESIZE.  During dumping, I get:
> 1243908 pure bytes used

With the current sources on this machine: 1245620 pure bytes used

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

Re: BASE_PURESIZE

[Eli Zaretskii]

> From: Andreas Schwab <schwab@suse.de>
> Cc: eliz@gnu.org, nickrob@snap.net.nz, emacs-devel@gnu.org
> Date: Sun, 06 Aug 2006 22:25:23 +0200
> 
> Reiner Steib <reinersteib+gmane@imap.cc> writes:
> 
> > On i686 (2x Intel Xeon 2.40GHz) I get "1245323 pure bytes used".
> > On x86_64 (AMD Athlon 64 3500+) I get "2064768 pure bytes used".
> >
> > Both have the same gcc (GCC) 3.3.4 (pre 3.3.5 20040809) and both are
> > running SuSE 9.2.
> 
> On the same systems I only get 1100794/1788523 pure bytes with my usual
> configuration (--with-x-toolkit=lucid).

I still think we _must_ investigate this strange disparity, it's just
that I don't have enough time right now to continue the investigation
I started with Reiner's help.  The last thing we've determined was
that the difference happens only when loadup.el loads .elc files, not
.el files.  Perhaps someone else could pick up where I left off;
Reiner mentioned that past thread in one of his last messages here.

BASE_PURESIZE

[Eli Zaretskii]

Isn't the current definition of BASE_PURESIZE too large?

  #define BASE_PURESIZE (1430000 + SYSTEM_PURESIZE_EXTRA + SITELOAD_PURESIZE_EXTRA)

I looked at the values of pure_size vs pure_bytes_used in several
builds on several platforms, and I see that we are wasting at least
130KB:

    MS-Windows:
    (gdb) p pure_size
    $1 = 1480000
    (gdb) p pure_bytes_used
    $2 = 1357888

    64-bit GNU/Linux (--without-x):
    (gdb) p pure_size
    $1 = 2383333
    (gdb) p pure_bytes_used
    $2 = 2015813

    64-bit GNU/Linux (with X):
    (gdb) p pure_size
    $1 = 2383333
    (gdb) p pure_bytes_used
    $2 = 2193049

    MS-DOS:
    (gdb) p pure_size
    $1 = 1440000
    (gdb) p pure_bytes_used
    $2 = 1275442

GNU/Linux without-X is the extreme example: it wastes 370KB.

How about reducing the 1430000 number above?

Re: BASE_PURESIZE

[Andreas Schwab]

Eli Zaretskii <eliz@gnu.org> writes:

> Isn't the current definition of BASE_PURESIZE too large?

Fits quite well here (pure_size - pure_bytes_used == 79770).

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: BASE_PURESIZE

[Dan Nicolaescu]

Eli Zaretskii <eliz@gnu.org> writes:

  > Isn't the current definition of BASE_PURESIZE too large?
  > 
  >   #define BASE_PURESIZE (1430000 + SYSTEM_PURESIZE_EXTRA + SITELOAD_PURESIZE_EXTRA)
  > 
  > I looked at the values of pure_size vs pure_bytes_used in several
  > builds on several platforms, and I see that we are wasting at least
  > 130KB:
  > 
  >     MS-Windows:
  >     (gdb) p pure_size
  >     $1 = 1480000
  >     (gdb) p pure_bytes_used
  >     $2 = 1357888
  > 
  >     64-bit GNU/Linux (--without-x):
  >     (gdb) p pure_size
  >     $1 = 2383333
  >     (gdb) p pure_bytes_used
  >     $2 = 2015813
  > 
  >     64-bit GNU/Linux (with X):
  >     (gdb) p pure_size
  >     $1 = 2383333
  >     (gdb) p pure_bytes_used
  >     $2 = 2193049
  > 
  >     MS-DOS:
  >     (gdb) p pure_size
  >     $1 = 1440000
  >     (gdb) p pure_bytes_used
  >     $2 = 1275442
  > 
  > GNU/Linux without-X is the extreme example: it wastes 370KB.
  > 
  > How about reducing the 1430000 number above?

I have a few pending changes that will make the sizes needed just under
that, no need to fiddle with it all the time.

Re: BASE_PURESIZE

[Eli Zaretskii]

> From: Andreas Schwab <schwab@linux-m68k.org>
> Date: Fri, 23 Oct 2009 13:39:37 +0200
> Cc: emacs-devel@gnu.org
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Isn't the current definition of BASE_PURESIZE too large?
> 
> Fits quite well here (pure_size - pure_bytes_used == 79770).

What configuration is that?

Anyway, the numerical constant is not supposed to be tuned to the
largest user of pure[], that's what SYSTEM_PURESIZE_EXTRA and friends
are for.

But since Dan says he has changes in the pipe to use that up, I guess
that's okay.

Re: BASE_PURESIZE

[Juanma Barranquero]

On Fri, Oct 23, 2009 at 13:00, Eli Zaretskii <eliz@gnu.org> wrote:

> I looked at the values of pure_size vs pure_bytes_used in several
> builds on several platforms, and I see that we are wasting at least
> 130KB:
>
>    MS-Windows:
>    (gdb) p pure_size
>    $1 = 1480000
>    (gdb) p pure_bytes_used
>    $2 = 1357888

With this system-configuration-options

  --with-gcc (4.4) --cflags -DENABLE_CHECKING=1 -DXASSERTS=1
-IC:/emacs/build/include -fno-crossjumping

on Windows I get

  (gdb) p pure_size
  $1 = 1776000
  (gdb) p pure_bytes_used
  $2 = 1518217

or about 252 KiB wasted.

    Juanma

Re: BASE_PURESIZE

[Andreas Schwab]

Eli Zaretskii <eliz@gnu.org> writes:

>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> > Isn't the current definition of BASE_PURESIZE too large?
>> 
>> Fits quite well here (pure_size - pure_bytes_used == 79770).
>
> What configuration is that?

powerpc-suse-linux

> Anyway, the numerical constant is not supposed to be tuned to the
> largest user of pure[], that's what SYSTEM_PURESIZE_EXTRA and friends
> are for.

There are no system specific additions.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: BASE_PURESIZE

[Eli Zaretskii]

> From: Andreas Schwab <schwab@linux-m68k.org>
> Cc: emacs-devel@gnu.org
> Date: Fri, 23 Oct 2009 16:24:26 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> Eli Zaretskii <eliz@gnu.org> writes:
> >> 
> >> > Isn't the current definition of BASE_PURESIZE too large?
> >> 
> >> Fits quite well here (pure_size - pure_bytes_used == 79770).
> >
> > What configuration is that?
> 
> powerpc-suse-linux

That's strange.  Is that a 64-bit system?  If so, do you have any
ideas why two different GNU/Linux systems, one on x86_64, the other
yours, have such significantly different pure space usage sizes?

Re: BASE_PURESIZE

[Andreas Schwab]

Eli Zaretskii <eliz@gnu.org> writes:

>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> >> Eli Zaretskii <eliz@gnu.org> writes:
>> >> 
>> >> > Isn't the current definition of BASE_PURESIZE too large?
>> >> 
>> >> Fits quite well here (pure_size - pure_bytes_used == 79770).
>> >
>> > What configuration is that?
>> 
>> powerpc-suse-linux
>
> That's strange.  Is that a 64-bit system?

No.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: BASE_PURESIZE

[Stephen J. Turnbull]

Juanma Barranquero writes:
 > On Fri, Oct 23, 2009 at 13:00, Eli Zaretskii <eliz@gnu.org> wrote:
 > 
 > > I looked at the values of pure_size vs pure_bytes_used in several
 > > builds on several platforms, and I see that we are wasting at least
 > > 130KB:
[...]
 > on Windows I get
[...]
 > or about 252 KiB wasted.

Three comments:

1.  XEmacs abandoned pure space years ago on the assumption that (bugs
    aside) copy-on-write means that dumped text will be shared anyway.
    Is that incorrect?

2.  252 KiB is not negligible, I suppose, but these days the systems
    Emacs runs on typically sport >1GB of memory, and since that's
    pure space even with multiple instances of Emacs running that is
    all that will be wasted ever.

3.  To save the space, dump twice, the second time using the precise
    number you can measure from the first try.  Unlike the Lisp
    compilation stage, this takes less than an extra minute IIRC.  If
    you still care about the extra time, make the second dump part of
    the install target.

Re: BASE_PURESIZE

[Dan Nicolaescu]

"Stephen J. Turnbull" <stephen@xemacs.org> writes:

  > Juanma Barranquero writes:
  >  > On Fri, Oct 23, 2009 at 13:00, Eli Zaretskii <eliz@gnu.org> wrote:
  >  > 
  >  > > I looked at the values of pure_size vs pure_bytes_used in several
  >  > > builds on several platforms, and I see that we are wasting at least
  >  > > 130KB:
  > [...]
  >  > on Windows I get
  > [...]
  >  > or about 252 KiB wasted.
  > 
  > Three comments:
  > 
  > 1.  XEmacs abandoned pure space years ago on the assumption that (bugs
  >     aside) copy-on-write means that dumped text will be shared anyway.
  >     Is that incorrect?

Do you have generational GC? If not, all the data in the dumped image
dumped image will be GCed every time, and that means lots of pages will
get written to, so won't be shareable.

Re: BASE_PURESIZE

[Stephen J. Turnbull]

Dan Nicolaescu writes:

 >   > 1.  XEmacs abandoned pure space years ago on the assumption that (bugs
 >   >     aside) copy-on-write means that dumped text will be shared anyway.
 >   >     Is that incorrect?
 > 
 > Do you have generational GC? If not, all the data in the dumped image
 > dumped image will be GCed every time, and that means lots of pages will
 > get written to, so won't be shareable.

I don't think that's necessarily true.  I'm not sure what the
mechanism for avoiding writes to dumped objects is in XEmacs; it might
be something like setting a "permanently marked object" bit and
putting them all in the set of GC roots at dumptime.  (It's been a
while and I wasn't involved in that work at all.)  And doesn't Emacs
keep its mark bits separately, being able to do that because objects
are allocated in arrays of same-sized blocks?

Re: BASE_PURESIZE

[Eli Zaretskii]

> Date: Fri, 23 Oct 2009 16:10:47 +0200
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: emacs-devel@gnu.org
> 
> > From: Andreas Schwab <schwab@linux-m68k.org>
> > Date: Fri, 23 Oct 2009 13:39:37 +0200
> > Cc: emacs-devel@gnu.org
> > 
> > Eli Zaretskii <eliz@gnu.org> writes:
> > 
> > > Isn't the current definition of BASE_PURESIZE too large?
> > 
> > Fits quite well here (pure_size - pure_bytes_used == 79770).
> 
> What configuration is that?
> 
> Anyway, the numerical constant is not supposed to be tuned to the
> largest user of pure[], that's what SYSTEM_PURESIZE_EXTRA and friends
> are for.
> 
> But since Dan says he has changes in the pipe to use that up, I guess
> that's okay.

After Dan committed his changes that use more purecopy, and after this
change:

  2009-10-23  Andreas Schwab  <schwab@linux-m68k.org>

	  * puresize.h (PURESIZE_RATIO): Decrease to 11/7.

pure space overflows on a 64-bit GNU/Linux host, and I need to enlarge
the 1430000 constant to at least 1460000, i.e. by 30KB, to fix that.
On a 32-bit Windows, the old constant of 1430000 still works (there's
70KB of spare pure space in the dumped Emacs).  So I'm not sure if the
problem is with the ratio or with something else.

For the record, the extra use of purecopy caused the pure_bytes_used
value to go up by 52KB on 32-bit Windows, and by 92KB on 64-bit
GNU/Linux.  So it looks like the ratio is actually closer to 9/5 than
to either the old 10/6 or the new 11/7.  Or maybe I'm missing
something.

Re: BASE_PURESIZE

[Eli Zaretskii]

> From: "Stephen J. Turnbull" <stephen@xemacs.org>
> Date: Sat, 24 Oct 2009 13:41:50 +0900
> 
> 2.  252 KiB is not negligible, I suppose, but these days the systems
>     Emacs runs on typically sport >1GB of memory, and since that's
>     pure space even with multiple instances of Emacs running that is
>     all that will be wasted ever.

First, we had changes committed lately that save much less than that
(FWIW, I support them).  The memory size is large nowadays, but it's
all taken by programs that have even larger memory footprint.

More importantly, I think the issue here is that the numbers I
presented indicate that there's something wrong with the way we
compute PURESIZE, because on some hosts it seems to be exactly right,
while on others it wastes a lot of memory.

> 3.  To save the space, dump twice, the second time using the precise
>     number you can measure from the first try.

PURESIZE is a compile-time C constant, it cannot be changed at dump
time.  You need to recompile.

Re: BASE_PURESIZE

[Andreas Schwab]

Eli Zaretskii <eliz@gnu.org> writes:

> For the record, the extra use of purecopy caused the pure_bytes_used
> value to go up by 52KB on 32-bit Windows, and by 92KB on 64-bit
> GNU/Linux.  So it looks like the ratio is actually closer to 9/5 than
> to either the old 10/6 or the new 11/7.  Or maybe I'm missing
> something.

It all depends on the ratio of string data vs. lisp object pure storage.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: BASE_PURESIZE

[Stephen J. Turnbull]

Eli Zaretskii writes:

 > > 3.  To save the space, dump twice, the second time using the precise
 > >     number you can measure from the first try.

 > PURESIZE is a compile-time C constant, it cannot be changed at dump
 > time.  You need to recompile.

OK, so recompile the relevant module and relink.  You may actually be
pushing 60 extra seconds on a very slow machine at this point.

If you're more worried about errors in the computation of PURESIZE or
of the amount actually used when dumping, that's another issue, of
course, and this doesn't help with that.

Re: BASE_PURESIZE

[Eli Zaretskii]

> From: Andreas Schwab <schwab@linux-m68k.org>
> Cc: emacs-devel@gnu.org
> Date: Sat, 24 Oct 2009 12:37:19 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > For the record, the extra use of purecopy caused the pure_bytes_used
> > value to go up by 52KB on 32-bit Windows, and by 92KB on 64-bit
> > GNU/Linux.  So it looks like the ratio is actually closer to 9/5 than
> > to either the old 10/6 or the new 11/7.  Or maybe I'm missing
> > something.
> 
> It all depends on the ratio of string data vs. lisp object pure storage.

I made some measurements.  The ratio of 11/7 seems to work pretty
well, but there are two additional problems:

 . The default value of SYSTEM_PURESIZE_EXTRA is zero, and is not
   increased for GUI builds.  This causes a --without-x build to waste
   some 100KB.  If we want to handle this, the basic constant in
   BASE_PURESIZE can be as low as 1290000 and SYSTEM_PURESIZE_EXTRA
   should have its default at 140000 for GUI builds, zero otherwise.

 . The amount of pure storage used by load-history depends on the
   length of the filename of the directory where Emacs is dumped.  In
   my case, I have 32 characters before the "emacs/lisp/" part, so I'm
   guessing that's the main reason the value of 1430000 was too small
   for me.

We could decide that we don't care too much about the --without-x
case, but what about the second problem?  If we want to handle it
without wasting storage on systems with shorter file names, we would
need some code in src/Makefile.in that would measure the length of the
directory name and enlarge PURESIZE accordingly.

Re: BASE_PURESIZE

[Dan Nicolaescu]

Eli Zaretskii <eliz@gnu.org> writes:

  > > From: Andreas Schwab <schwab@linux-m68k.org>
  > > Cc: emacs-devel@gnu.org
  > > Date: Sat, 24 Oct 2009 12:37:19 +0200
  > > 
  > > Eli Zaretskii <eliz@gnu.org> writes:
  > > 
  > > > For the record, the extra use of purecopy caused the pure_bytes_used
  > > > value to go up by 52KB on 32-bit Windows, and by 92KB on 64-bit
  > > > GNU/Linux.  So it looks like the ratio is actually closer to 9/5 than
  > > > to either the old 10/6 or the new 11/7.  Or maybe I'm missing
  > > > something.
  > > 
  > > It all depends on the ratio of string data vs. lisp object pure storage.
  > 
  > I made some measurements.  The ratio of 11/7 seems to work pretty
  > well, but there are two additional problems:
  > 
  >  . The default value of SYSTEM_PURESIZE_EXTRA is zero, and is not
  >    increased for GUI builds.  This causes a --without-x build to waste
  >    some 100KB.  If we want to handle this, the basic constant in
  >    BASE_PURESIZE can be as low as 1290000 and SYSTEM_PURESIZE_EXTRA
  >    should have its default at 140000 for GUI builds, zero otherwise.
  > 
  >  . The amount of pure storage used by load-history depends on the
  >    length of the filename of the directory where Emacs is dumped.  In
  >    my case, I have 32 characters before the "emacs/lisp/" part, so I'm
  >    guessing that's the main reason the value of 1430000 was too small
  >    for me.

We have 2 more problems with load-history: although in loadup.el is
purecopied, something still seems to maintain references to the file
name strings, they are still present as non-pure strings in the dumped
image both as absolute file names and as the arguments passed to load
(see the simple patch I posted yesterday to dump strings).  So we are
still wasting memory on those.

It would be great if load-history would be constructed in pure memory
from the beginning when dumping (instead of purecopying later).
Maybe someone that understands that code could do that...

  > We could decide that we don't care too much about the --without-x

IMHO --without-x is completely unimportant.

Re: BASE_PURESIZE

[Stefan Monnier]

>> . The default value of SYSTEM_PURESIZE_EXTRA is zero, and is not
>> increased for GUI builds.  This causes a --without-x build to waste
>> some 100KB.  If we want to handle this, the basic constant in
>> BASE_PURESIZE can be as low as 1290000 and SYSTEM_PURESIZE_EXTRA
>> should have its default at 140000 for GUI builds, zero otherwise.

I'm not terribly concerned about wasting 100KB in pure space (or even
more than that): it's space we don't ever touch or look at, so it's
never brought into RAM.  IOW it's very cheap.
OTOH checking what really goes into purespace and why something doesn't
is good, because moving data into purespace is good.


        Stefan


PS: Regarding the recent purecopy of autoload's file names: why not just
purecopy the whole autload list?


--- src/eval.c	2009-10-24 12:32:03 +0000
+++ src/eval.c	2009-10-24 18:32:23 +0000
@@ -2112,10 +2148,7 @@
      (function, file, docstring, interactive, type)
      Lisp_Object function, file, docstring, interactive, type;
 {
-  Lisp_Object args[4];
-
   CHECK_SYMBOL (function);
-  CHECK_STRING (file);
 
   /* If function is defined and not as an autoload, don't override */
   if (!EQ (XSYMBOL (function)->function, Qunbound)
@@ -2128,15 +2161,13 @@
        not useful and else we get loads of them from the loaddefs.el.  */
     LOADHIST_ATTACH (Fcons (Qautoload, function));
 
-  if (NILP (Vpurify_flag))
-    args[0] = file;
-  else
-    args[0] = Fpurecopy (file);
-  args[1] = docstring;
-  args[2] = interactive;
-  args[3] = type;
-
-  return Ffset (function, Fcons (Qautoload, Flist (4, &args[0])));
+  if (!NILP (Vpurify_flag))
+    /* We don't want the docstring in purespace (instead,
+       Snarf-documentation should (hopefully) overwrite it).  */
+    docstring = make_number (0);
+  return Ffset (function,
+		Fpurecopy (list5 (Qautoload, file, docstring,
+				  interactive, type)));
 }
 
 Lisp_Object

Re: BASE_PURESIZE

[Chong Yidong]

Dan Nicolaescu <dann@ics.uci.edu> writes:

>   > We could decide that we don't care too much about the --without-x
>
> IMHO --without-x is completely unimportant.

Not sure what you mean by this statement.  If you mean that wasting a
few extra kB of memory on --without-x builds isn't important, I can
agree with that.  But I don't even care about a few extra kB in X
builds, personally.

Re: BASE_PURESIZE

[Dan Nicolaescu]

Chong Yidong <cyd@stupidchicken.com> writes:

  > Dan Nicolaescu <dann@ics.uci.edu> writes:
  > 
  > >   > We could decide that we don't care too much about the --without-x
  > >
  > > IMHO --without-x is completely unimportant.
  > 
  > Not sure what you mean by this statement.  If you mean that wasting a
  > few extra kB of memory on --without-x builds isn't important,

That's exactly what I mean.

Re: BASE_PURESIZE

[Dan Nicolaescu]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

  > PS: Regarding the recent purecopy of autoload's file names: why not just
  > purecopy the whole autload list?

If it works it should have an important impact...
So please install it.  I bet you'll have to bump BASE_PURESIZE again 
after doing that :-)

Another issue that I hit:

Changing (in files.el)

From:
(defcustom directory-free-space-program "df"
to:
(defcustom directory-free-space-program (purecopy "df")

does not get rid of the "df" string in GC memory.  The reason is

ELISP> (get 'directory-free-space-program 'standard-value)
((purecopy "df"))

BTW, this is not a problem for autoloaded defcustoms, they get
transformed into defvars in loaddefs.el.

defcustom standard-value (was: Re: BASE_PURESIZE)

[Dan Nicolaescu]

Dan Nicolaescu <dann@ics.uci.edu> writes:

  > Another issue that I hit:
  > 
  > Changing (in files.el)
  > 
  > From:
  > (defcustom directory-free-space-program "df"
  > to:
  > (defcustom directory-free-space-program (purecopy "df")
  > 
  > does not get rid of the "df" string in GC memory.  The reason is
  > 
  > ELISP> (get 'directory-free-space-program 'standard-value)
  > ((purecopy "df"))
  > 
  > BTW, this is not a problem for autoloaded defcustoms, they get
  > transformed into defvars in loaddefs.el.

Is this patch TRTD?

Index: lisp/custom.el
===================================================================
RCS file: /cvsroot/emacs/emacs/lisp/custom.el,v
retrieving revision 1.149
diff -u -3 -p -r1.149 custom.el
--- lisp/custom.el    15 Sep 2009 04:08:04 -0000        1.149
+++ lisp/custom.el    29 Oct 2009 07:50:00 -0000
@@ -131,7 +131,7 @@ not the default value itself.
 DEFAULT is stored as SYMBOL's standard value, in SYMBOL's property
 `standard-value'.  At the same time, SYMBOL's property `force-value' is
 set to nil, as the value is no longer rogue."
-  (put symbol 'standard-value (list default))
+  (put symbol 'standard-value (purecopy (list default)))
   ;; Maybe this option was rogue in an earlier version.  It no longer is.
   (when (get symbol 'force-value)
     (put symbol 'force-value nil))