all messages for Emacs-related lists mirrored at yhetil.org
 help / color / mirror / code / Atom feed
* BASE_PURESIZE
@ 2006-08-05  2:16 Nick Roberts
  2006-08-05  9:41 ` BASE_PURESIZE Eli Zaretskii
  0 siblings, 1 reply; 31+ messages in thread
From: Nick Roberts @ 2006-08-05  2:16 UTC (permalink / raw)



I get the warning message "Building Emacs overflowed pure space".  Assuming
this is related to decreasing BASE_PURESIZE, the argument for keeping it to a
minimum for computers with little memory against the inconvenience of
repeatedly tripping over it, is completely lost on me.  Especially with a
development version (as we could. presumably, just pare it back prior to
release).


-- 
Nick                                           http://www.inet.net.nz/~nickrob

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: BASE_PURESIZE
  2006-08-05  2:16 BASE_PURESIZE Nick Roberts
@ 2006-08-05  9:41 ` Eli Zaretskii
  2006-08-05 16:23   ` BASE_PURESIZE Reiner Steib
  0 siblings, 1 reply; 31+ messages in thread
From: Eli Zaretskii @ 2006-08-05  9:41 UTC (permalink / raw)
  Cc: emacs-devel

> From: Nick Roberts <nickrob@snap.net.nz>
> Date: Sat, 5 Aug 2006 14:16:54 +1200
> 
> I get the warning message "Building Emacs overflowed pure space".  Assuming
> this is related to decreasing BASE_PURESIZE, the argument for keeping it to a
> minimum for computers with little memory against the inconvenience of
> repeatedly tripping over it, is completely lost on me.

No, it was decreased because we made a change lately that lowers the
required pure size by several tens of KB, IIRC.

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: BASE_PURESIZE
  2006-08-05  9:41 ` BASE_PURESIZE Eli Zaretskii
@ 2006-08-05 16:23   ` Reiner Steib
  2006-08-05 18:51     ` BASE_PURESIZE Richard Stallman
  0 siblings, 1 reply; 31+ messages in thread
From: Reiner Steib @ 2006-08-05 16:23 UTC (permalink / raw)
  Cc: Nick Roberts, emacs-devel

On Sat, Aug 05 2006, Eli Zaretskii wrote:

>> From: Nick Roberts <nickrob@snap.net.nz>
>> Date: Sat, 5 Aug 2006 14:16:54 +1200
>> 
>> I get the warning message "Building Emacs overflowed pure space".
>> Assuming this is related to decreasing BASE_PURESIZE, the argument
>> for keeping it to a minimum for computers with little memory
>> against the inconvenience of repeatedly tripping over it, is
>> completely lost on me.
>
> No, it was decreased because we made a change lately that lowers the
> required pure size by several tens of KB, IIRC.

FWIW, starting with this change, I had to set SITELOAD_PURESIZE_EXTRA
also[1] on the machine (SUSE 10.0 [2]) on which it wasn't necessary
until some weeks before this change.

Unless we find (and fix) the reason for the different pure space
requirements, it would be good to increase BASE_PURESIZE to an amount
that allows building Emacs without an overflow on most machines, I
think.

Bye, Reiner.

[1] I still need to increase it on the machines as reported
    previously[3].  On i686 (2x Intel Xeon (2.40GHz)) and x86_64 (AMD
    Athlon 64 3500+):

$ gcc --version
gcc (GCC) 3.3.4 (pre 3.3.5 20040809)
[...]
$ cat /etc/SuSE-release 
SuSE Linux 9.2 (i586)         [... or x86_64 resp.]
VERSION = 9.2

[2] i686 (Intel(R) Pentium(R) M processor 1500MHz):
$ cat /etc/SuSE-release 
SUSE LINUX 10.0 (i586) OSS
VERSION = 10.0
$ gcc --version
gcc (GCC) 4.0.2 20050901 (prerelease) (SUSE Linux)

[3] See the thread around
    <http://thread.gmane.org/v9d5diibsr.fsf%40marauder.physik.uni-ulm.de>.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: BASE_PURESIZE
  2006-08-05 16:23   ` BASE_PURESIZE Reiner Steib
@ 2006-08-05 18:51     ` Richard Stallman
  2006-08-05 20:07       ` BASE_PURESIZE Reiner Steib
  0 siblings, 1 reply; 31+ messages in thread
From: Richard Stallman @ 2006-08-05 18:51 UTC (permalink / raw)
  Cc: eliz, nickrob, emacs-devel

    Unless we find (and fix) the reason for the different pure space
    requirements, it would be good to increase BASE_PURESIZE to an amount
    that allows building Emacs without an overflow on most machines, I
    think.

How much increase do you propose?

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: BASE_PURESIZE
  2006-08-05 18:51     ` BASE_PURESIZE Richard Stallman
@ 2006-08-05 20:07       ` Reiner Steib
  2006-08-06  4:43         ` BASE_PURESIZE Richard Stallman
  2006-08-06 19:40         ` BASE_PURESIZE Reiner Steib
  0 siblings, 2 replies; 31+ messages in thread
From: Reiner Steib @ 2006-08-05 20:07 UTC (permalink / raw)
  Cc: eliz, nickrob, emacs-devel

On Sat, Aug 05 2006, Richard Stallman wrote:

>     Unless we find (and fix) the reason for the different pure space
>     requirements, it would be good to increase BASE_PURESIZE to an amount
>     that allows building Emacs without an overflow on most machines, I
>     think.
>
> How much increase do you propose?

The last time I tried to minimize the extra space in 10k steps, I had
to set SITELOAD_PURESIZE_EXTRA to values between 100000 and 200000
(different for i686 and x86_64) to avoid the overflow.

But I didn't try it recently with the current sources (and I cannot do
this at the moment on the other machines).

For the 2006-07-29 sources, I need 150000 extra bytes on my SUSE 10.0
installation:
1102000 is the default BASE_PURESIZE.  During dumping, I get:
1243908 pure bytes used

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: BASE_PURESIZE
  2006-08-05 20:07       ` BASE_PURESIZE Reiner Steib
@ 2006-08-06  4:43         ` Richard Stallman
  2006-08-06  9:19           ` BASE_PURESIZE Reiner Steib
  2006-08-06 19:40         ` BASE_PURESIZE Reiner Steib
  1 sibling, 1 reply; 31+ messages in thread
From: Richard Stallman @ 2006-08-06  4:43 UTC (permalink / raw)
  Cc: eliz, nickrob, emacs-devel

    For the 2006-07-29 sources, I need 150000 extra bytes on my SUSE 10.0
    installation:
    1102000 is the default BASE_PURESIZE.  During dumping, I get:
    1243908 pure bytes used

That's more or less what it used before the optimizations.  It sounds
like something is wrong, and those optimizations are not working in
your case.

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: BASE_PURESIZE
  2006-08-06  4:43         ` BASE_PURESIZE Richard Stallman
@ 2006-08-06  9:19           ` Reiner Steib
  0 siblings, 0 replies; 31+ messages in thread
From: Reiner Steib @ 2006-08-06  9:19 UTC (permalink / raw)
  Cc: eliz, nickrob, emacs-devel

On Sun, Aug 06 2006, Richard Stallman wrote:

>     For the 2006-07-29 sources, I need 150000 extra bytes on my SUSE 10.0
>     installation:
>     1102000 is the default BASE_PURESIZE.  During dumping, I get:
>     1243908 pure bytes used
>
> That's more or less what it used before the optimizations.  It sounds
> like something is wrong, and those optimizations are not working in
> your case.

That's possible.  Until mid of June (some weeks before the
optimizations), the default value was sufficient on this machine
(Pentium M 1500MHz; gcc (GCC) 4.0.2 20050901 (prerelease); SUSE 10.0).
I didn't update the sources often on this host, so I'm not sure when
got the overflow the first time.

On the other machines, I had to add some extra space at least since
April.  See the thread "PURESIZE increased (again)" (April-June)
<URL:http://thread.gmane.org/v9d5diibsr.fsf%40marauder.physik.uni-ulm.de>.

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: BASE_PURESIZE
  2006-08-05 20:07       ` BASE_PURESIZE Reiner Steib
  2006-08-06  4:43         ` BASE_PURESIZE Richard Stallman
@ 2006-08-06 19:40         ` Reiner Steib
       [not found]           ` <jebqqx393g.fsf@sykes.suse.de>
  1 sibling, 1 reply; 31+ messages in thread
From: Reiner Steib @ 2006-08-06 19:40 UTC (permalink / raw)
  Cc: eliz, nickrob, emacs-devel

On Sat, Aug 05 2006, Reiner Steib wrote:

> On Sat, Aug 05 2006, Richard Stallman wrote:
>
>>     Unless we find (and fix) the reason for the different pure space
>>     requirements, it would be good to increase BASE_PURESIZE to an amount
>>     that allows building Emacs without an overflow on most machines, I
>>     think.
>>
>> How much increase do you propose?
>
> The last time I tried to minimize the extra space in 10k steps, I had
> to set SITELOAD_PURESIZE_EXTRA to values between 100000 and 200000
> (different for i686 and x86_64) to avoid the overflow.
>
> But I didn't try it recently with the current sources (and I cannot do
> this at the moment on the other machines).

On i686 (2x Intel Xeon 2.40GHz) I get "1245323 pure bytes used".
On x86_64 (AMD Athlon 64 3500+) I get "2064768 pure bytes used".

Both have the same gcc (GCC) 3.3.4 (pre 3.3.5 20040809) and both are
running SuSE 9.2.  I updated the sources from CVS just before testing
and sending this mail.

> For the 2006-07-29 sources, I need 150000 extra bytes on my SUSE 10.0
> installation:
> 1102000 is the default BASE_PURESIZE.  During dumping, I get:
> 1243908 pure bytes used

With the current sources on this machine: 1245620 pure bytes used

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: BASE_PURESIZE
       [not found]           ` <jebqqx393g.fsf@sykes.suse.de>
@ 2006-08-07  3:53             ` Eli Zaretskii
  0 siblings, 0 replies; 31+ messages in thread
From: Eli Zaretskii @ 2006-08-07  3:53 UTC (permalink / raw)
  Cc: nickrob, rms, emacs-devel

> From: Andreas Schwab <schwab@suse.de>
> Cc: eliz@gnu.org, nickrob@snap.net.nz, emacs-devel@gnu.org
> Date: Sun, 06 Aug 2006 22:25:23 +0200
> 
> Reiner Steib <reinersteib+gmane@imap.cc> writes:
> 
> > On i686 (2x Intel Xeon 2.40GHz) I get "1245323 pure bytes used".
> > On x86_64 (AMD Athlon 64 3500+) I get "2064768 pure bytes used".
> >
> > Both have the same gcc (GCC) 3.3.4 (pre 3.3.5 20040809) and both are
> > running SuSE 9.2.
> 
> On the same systems I only get 1100794/1788523 pure bytes with my usual
> configuration (--with-x-toolkit=lucid).

I still think we _must_ investigate this strange disparity, it's just
that I don't have enough time right now to continue the investigation
I started with Reiner's help.  The last thing we've determined was
that the difference happens only when loadup.el loads .elc files, not
.el files.  Perhaps someone else could pick up where I left off;
Reiner mentioned that past thread in one of his last messages here.

^ permalink raw reply	[flat|nested] 31+ messages in thread

* BASE_PURESIZE
@ 2009-10-23 11:00 Eli Zaretskii
  2009-10-23 11:39 ` BASE_PURESIZE Andreas Schwab
                   ` (2 more replies)
  0 siblings, 3 replies; 31+ messages in thread
From: Eli Zaretskii @ 2009-10-23 11:00 UTC (permalink / raw)
  To: emacs-devel

Isn't the current definition of BASE_PURESIZE too large?

  #define BASE_PURESIZE (1430000 + SYSTEM_PURESIZE_EXTRA + SITELOAD_PURESIZE_EXTRA)

I looked at the values of pure_size vs pure_bytes_used in several
builds on several platforms, and I see that we are wasting at least
130KB:

    MS-Windows:
    (gdb) p pure_size
    $1 = 1480000
    (gdb) p pure_bytes_used
    $2 = 1357888

    64-bit GNU/Linux (--without-x):
    (gdb) p pure_size
    $1 = 2383333
    (gdb) p pure_bytes_used
    $2 = 2015813

    64-bit GNU/Linux (with X):
    (gdb) p pure_size
    $1 = 2383333
    (gdb) p pure_bytes_used
    $2 = 2193049

    MS-DOS:
    (gdb) p pure_size
    $1 = 1440000
    (gdb) p pure_bytes_used
    $2 = 1275442

GNU/Linux without-X is the extreme example: it wastes 370KB.

How about reducing the 1430000 number above?




^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: BASE_PURESIZE
  2009-10-23 11:00 BASE_PURESIZE Eli Zaretskii
@ 2009-10-23 11:39 ` Andreas Schwab
  2009-10-23 14:10   ` BASE_PURESIZE Eli Zaretskii
  2009-10-23 11:58 ` BASE_PURESIZE Dan Nicolaescu
  2009-10-23 14:24 ` BASE_PURESIZE Juanma Barranquero
  2 siblings, 1 reply; 31+ messages in thread
From: Andreas Schwab @ 2009-10-23 11:39 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: emacs-devel

Eli Zaretskii <eliz@gnu.org> writes:

> Isn't the current definition of BASE_PURESIZE too large?

Fits quite well here (pure_size - pure_bytes_used == 79770).

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."




^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: BASE_PURESIZE
  2009-10-23 11:00 BASE_PURESIZE Eli Zaretskii
  2009-10-23 11:39 ` BASE_PURESIZE Andreas Schwab
@ 2009-10-23 11:58 ` Dan Nicolaescu
  2009-10-23 14:24 ` BASE_PURESIZE Juanma Barranquero
  2 siblings, 0 replies; 31+ messages in thread
From: Dan Nicolaescu @ 2009-10-23 11:58 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: emacs-devel

Eli Zaretskii <eliz@gnu.org> writes:

  > Isn't the current definition of BASE_PURESIZE too large?
  > 
  >   #define BASE_PURESIZE (1430000 + SYSTEM_PURESIZE_EXTRA + SITELOAD_PURESIZE_EXTRA)
  > 
  > I looked at the values of pure_size vs pure_bytes_used in several
  > builds on several platforms, and I see that we are wasting at least
  > 130KB:
  > 
  >     MS-Windows:
  >     (gdb) p pure_size
  >     $1 = 1480000
  >     (gdb) p pure_bytes_used
  >     $2 = 1357888
  > 
  >     64-bit GNU/Linux (--without-x):
  >     (gdb) p pure_size
  >     $1 = 2383333
  >     (gdb) p pure_bytes_used
  >     $2 = 2015813
  > 
  >     64-bit GNU/Linux (with X):
  >     (gdb) p pure_size
  >     $1 = 2383333
  >     (gdb) p pure_bytes_used
  >     $2 = 2193049
  > 
  >     MS-DOS:
  >     (gdb) p pure_size
  >     $1 = 1440000
  >     (gdb) p pure_bytes_used
  >     $2 = 1275442
  > 
  > GNU/Linux without-X is the extreme example: it wastes 370KB.
  > 
  > How about reducing the 1430000 number above?

I have a few pending changes that will make the sizes needed just under
that, no need to fiddle with it all the time.




^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: BASE_PURESIZE
  2009-10-23 11:39 ` BASE_PURESIZE Andreas Schwab
@ 2009-10-23 14:10   ` Eli Zaretskii
  2009-10-23 14:24     ` BASE_PURESIZE Andreas Schwab
  2009-10-24 10:05     ` BASE_PURESIZE Eli Zaretskii
  0 siblings, 2 replies; 31+ messages in thread
From: Eli Zaretskii @ 2009-10-23 14:10 UTC (permalink / raw)
  To: Andreas Schwab; +Cc: emacs-devel

> From: Andreas Schwab <schwab@linux-m68k.org>
> Date: Fri, 23 Oct 2009 13:39:37 +0200
> Cc: emacs-devel@gnu.org
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Isn't the current definition of BASE_PURESIZE too large?
> 
> Fits quite well here (pure_size - pure_bytes_used == 79770).

What configuration is that?

Anyway, the numerical constant is not supposed to be tuned to the
largest user of pure[], that's what SYSTEM_PURESIZE_EXTRA and friends
are for.

But since Dan says he has changes in the pipe to use that up, I guess
that's okay.




^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: BASE_PURESIZE
  2009-10-23 11:00 BASE_PURESIZE Eli Zaretskii
  2009-10-23 11:39 ` BASE_PURESIZE Andreas Schwab
  2009-10-23 11:58 ` BASE_PURESIZE Dan Nicolaescu
@ 2009-10-23 14:24 ` Juanma Barranquero
  2009-10-24  4:41   ` BASE_PURESIZE Stephen J. Turnbull
  2 siblings, 1 reply; 31+ messages in thread
From: Juanma Barranquero @ 2009-10-23 14:24 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: emacs-devel

On Fri, Oct 23, 2009 at 13:00, Eli Zaretskii <eliz@gnu.org> wrote:

> I looked at the values of pure_size vs pure_bytes_used in several
> builds on several platforms, and I see that we are wasting at least
> 130KB:
>
>    MS-Windows:
>    (gdb) p pure_size
>    $1 = 1480000
>    (gdb) p pure_bytes_used
>    $2 = 1357888

With this system-configuration-options

  --with-gcc (4.4) --cflags -DENABLE_CHECKING=1 -DXASSERTS=1
-IC:/emacs/build/include -fno-crossjumping

on Windows I get

  (gdb) p pure_size
  $1 = 1776000
  (gdb) p pure_bytes_used
  $2 = 1518217

or about 252 KiB wasted.

    Juanma




^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: BASE_PURESIZE
  2009-10-23 14:10   ` BASE_PURESIZE Eli Zaretskii
@ 2009-10-23 14:24     ` Andreas Schwab
  2009-10-23 14:35       ` BASE_PURESIZE Eli Zaretskii
  2009-10-24 10:05     ` BASE_PURESIZE Eli Zaretskii
  1 sibling, 1 reply; 31+ messages in thread
From: Andreas Schwab @ 2009-10-23 14:24 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: emacs-devel

Eli Zaretskii <eliz@gnu.org> writes:

>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> > Isn't the current definition of BASE_PURESIZE too large?
>> 
>> Fits quite well here (pure_size - pure_bytes_used == 79770).
>
> What configuration is that?

powerpc-suse-linux

> Anyway, the numerical constant is not supposed to be tuned to the
> largest user of pure[], that's what SYSTEM_PURESIZE_EXTRA and friends
> are for.

There are no system specific additions.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."




^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: BASE_PURESIZE
  2009-10-23 14:24     ` BASE_PURESIZE Andreas Schwab
@ 2009-10-23 14:35       ` Eli Zaretskii
  2009-10-23 14:50         ` BASE_PURESIZE Andreas Schwab
  0 siblings, 1 reply; 31+ messages in thread
From: Eli Zaretskii @ 2009-10-23 14:35 UTC (permalink / raw)
  To: Andreas Schwab; +Cc: emacs-devel

> From: Andreas Schwab <schwab@linux-m68k.org>
> Cc: emacs-devel@gnu.org
> Date: Fri, 23 Oct 2009 16:24:26 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> Eli Zaretskii <eliz@gnu.org> writes:
> >> 
> >> > Isn't the current definition of BASE_PURESIZE too large?
> >> 
> >> Fits quite well here (pure_size - pure_bytes_used == 79770).
> >
> > What configuration is that?
> 
> powerpc-suse-linux

That's strange.  Is that a 64-bit system?  If so, do you have any
ideas why two different GNU/Linux systems, one on x86_64, the other
yours, have such significantly different pure space usage sizes?




^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: BASE_PURESIZE
  2009-10-23 14:35       ` BASE_PURESIZE Eli Zaretskii
@ 2009-10-23 14:50         ` Andreas Schwab
  0 siblings, 0 replies; 31+ messages in thread
From: Andreas Schwab @ 2009-10-23 14:50 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: emacs-devel

Eli Zaretskii <eliz@gnu.org> writes:

>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> >> Eli Zaretskii <eliz@gnu.org> writes:
>> >> 
>> >> > Isn't the current definition of BASE_PURESIZE too large?
>> >> 
>> >> Fits quite well here (pure_size - pure_bytes_used == 79770).
>> >
>> > What configuration is that?
>> 
>> powerpc-suse-linux
>
> That's strange.  Is that a 64-bit system?

No.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."




^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: BASE_PURESIZE
  2009-10-23 14:24 ` BASE_PURESIZE Juanma Barranquero
@ 2009-10-24  4:41   ` Stephen J. Turnbull
  2009-10-24  6:47     ` BASE_PURESIZE Dan Nicolaescu
  2009-10-24 10:22     ` BASE_PURESIZE Eli Zaretskii
  0 siblings, 2 replies; 31+ messages in thread
From: Stephen J. Turnbull @ 2009-10-24  4:41 UTC (permalink / raw)
  To: emacs-devel

Juanma Barranquero writes:
 > On Fri, Oct 23, 2009 at 13:00, Eli Zaretskii <eliz@gnu.org> wrote:
 > 
 > > I looked at the values of pure_size vs pure_bytes_used in several
 > > builds on several platforms, and I see that we are wasting at least
 > > 130KB:
[...]
 > on Windows I get
[...]
 > or about 252 KiB wasted.

Three comments:

1.  XEmacs abandoned pure space years ago on the assumption that (bugs
    aside) copy-on-write means that dumped text will be shared anyway.
    Is that incorrect?

2.  252 KiB is not negligible, I suppose, but these days the systems
    Emacs runs on typically sport >1GB of memory, and since that's
    pure space even with multiple instances of Emacs running that is
    all that will be wasted ever.

3.  To save the space, dump twice, the second time using the precise
    number you can measure from the first try.  Unlike the Lisp
    compilation stage, this takes less than an extra minute IIRC.  If
    you still care about the extra time, make the second dump part of
    the install target.





^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: BASE_PURESIZE
  2009-10-24  4:41   ` BASE_PURESIZE Stephen J. Turnbull
@ 2009-10-24  6:47     ` Dan Nicolaescu
  2009-10-24  8:24       ` BASE_PURESIZE Stephen J. Turnbull
  2009-10-24 10:22     ` BASE_PURESIZE Eli Zaretskii
  1 sibling, 1 reply; 31+ messages in thread
From: Dan Nicolaescu @ 2009-10-24  6:47 UTC (permalink / raw)
  To: Stephen J. Turnbull; +Cc: emacs-devel

"Stephen J. Turnbull" <stephen@xemacs.org> writes:

  > Juanma Barranquero writes:
  >  > On Fri, Oct 23, 2009 at 13:00, Eli Zaretskii <eliz@gnu.org> wrote:
  >  > 
  >  > > I looked at the values of pure_size vs pure_bytes_used in several
  >  > > builds on several platforms, and I see that we are wasting at least
  >  > > 130KB:
  > [...]
  >  > on Windows I get
  > [...]
  >  > or about 252 KiB wasted.
  > 
  > Three comments:
  > 
  > 1.  XEmacs abandoned pure space years ago on the assumption that (bugs
  >     aside) copy-on-write means that dumped text will be shared anyway.
  >     Is that incorrect?

Do you have generational GC? If not, all the data in the dumped image
dumped image will be GCed every time, and that means lots of pages will
get written to, so won't be shareable.




^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: BASE_PURESIZE
  2009-10-24  6:47     ` BASE_PURESIZE Dan Nicolaescu
@ 2009-10-24  8:24       ` Stephen J. Turnbull
  0 siblings, 0 replies; 31+ messages in thread
From: Stephen J. Turnbull @ 2009-10-24  8:24 UTC (permalink / raw)
  To: Dan Nicolaescu; +Cc: emacs-devel

Dan Nicolaescu writes:

 >   > 1.  XEmacs abandoned pure space years ago on the assumption that (bugs
 >   >     aside) copy-on-write means that dumped text will be shared anyway.
 >   >     Is that incorrect?
 > 
 > Do you have generational GC? If not, all the data in the dumped image
 > dumped image will be GCed every time, and that means lots of pages will
 > get written to, so won't be shareable.

I don't think that's necessarily true.  I'm not sure what the
mechanism for avoiding writes to dumped objects is in XEmacs; it might
be something like setting a "permanently marked object" bit and
putting them all in the set of GC roots at dumptime.  (It's been a
while and I wasn't involved in that work at all.)  And doesn't Emacs
keep its mark bits separately, being able to do that because objects
are allocated in arrays of same-sized blocks?





^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: BASE_PURESIZE
  2009-10-23 14:10   ` BASE_PURESIZE Eli Zaretskii
  2009-10-23 14:24     ` BASE_PURESIZE Andreas Schwab
@ 2009-10-24 10:05     ` Eli Zaretskii
  2009-10-24 10:37       ` BASE_PURESIZE Andreas Schwab
  1 sibling, 1 reply; 31+ messages in thread
From: Eli Zaretskii @ 2009-10-24 10:05 UTC (permalink / raw)
  To: schwab, emacs-devel

> Date: Fri, 23 Oct 2009 16:10:47 +0200
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: emacs-devel@gnu.org
> 
> > From: Andreas Schwab <schwab@linux-m68k.org>
> > Date: Fri, 23 Oct 2009 13:39:37 +0200
> > Cc: emacs-devel@gnu.org
> > 
> > Eli Zaretskii <eliz@gnu.org> writes:
> > 
> > > Isn't the current definition of BASE_PURESIZE too large?
> > 
> > Fits quite well here (pure_size - pure_bytes_used == 79770).
> 
> What configuration is that?
> 
> Anyway, the numerical constant is not supposed to be tuned to the
> largest user of pure[], that's what SYSTEM_PURESIZE_EXTRA and friends
> are for.
> 
> But since Dan says he has changes in the pipe to use that up, I guess
> that's okay.

After Dan committed his changes that use more purecopy, and after this
change:

  2009-10-23  Andreas Schwab  <schwab@linux-m68k.org>

	  * puresize.h (PURESIZE_RATIO): Decrease to 11/7.

pure space overflows on a 64-bit GNU/Linux host, and I need to enlarge
the 1430000 constant to at least 1460000, i.e. by 30KB, to fix that.
On a 32-bit Windows, the old constant of 1430000 still works (there's
70KB of spare pure space in the dumped Emacs).  So I'm not sure if the
problem is with the ratio or with something else.

For the record, the extra use of purecopy caused the pure_bytes_used
value to go up by 52KB on 32-bit Windows, and by 92KB on 64-bit
GNU/Linux.  So it looks like the ratio is actually closer to 9/5 than
to either the old 10/6 or the new 11/7.  Or maybe I'm missing
something.




^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: BASE_PURESIZE
  2009-10-24  4:41   ` BASE_PURESIZE Stephen J. Turnbull
  2009-10-24  6:47     ` BASE_PURESIZE Dan Nicolaescu
@ 2009-10-24 10:22     ` Eli Zaretskii
  2009-10-24 11:14       ` BASE_PURESIZE Stephen J. Turnbull
  1 sibling, 1 reply; 31+ messages in thread
From: Eli Zaretskii @ 2009-10-24 10:22 UTC (permalink / raw)
  To: Stephen J. Turnbull; +Cc: emacs-devel

> From: "Stephen J. Turnbull" <stephen@xemacs.org>
> Date: Sat, 24 Oct 2009 13:41:50 +0900
> 
> 2.  252 KiB is not negligible, I suppose, but these days the systems
>     Emacs runs on typically sport >1GB of memory, and since that's
>     pure space even with multiple instances of Emacs running that is
>     all that will be wasted ever.

First, we had changes committed lately that save much less than that
(FWIW, I support them).  The memory size is large nowadays, but it's
all taken by programs that have even larger memory footprint.

More importantly, I think the issue here is that the numbers I
presented indicate that there's something wrong with the way we
compute PURESIZE, because on some hosts it seems to be exactly right,
while on others it wastes a lot of memory.

> 3.  To save the space, dump twice, the second time using the precise
>     number you can measure from the first try.

PURESIZE is a compile-time C constant, it cannot be changed at dump
time.  You need to recompile.




^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: BASE_PURESIZE
  2009-10-24 10:05     ` BASE_PURESIZE Eli Zaretskii
@ 2009-10-24 10:37       ` Andreas Schwab
  2009-10-24 13:18         ` BASE_PURESIZE Eli Zaretskii
  0 siblings, 1 reply; 31+ messages in thread
From: Andreas Schwab @ 2009-10-24 10:37 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: emacs-devel

Eli Zaretskii <eliz@gnu.org> writes:

> For the record, the extra use of purecopy caused the pure_bytes_used
> value to go up by 52KB on 32-bit Windows, and by 92KB on 64-bit
> GNU/Linux.  So it looks like the ratio is actually closer to 9/5 than
> to either the old 10/6 or the new 11/7.  Or maybe I'm missing
> something.

It all depends on the ratio of string data vs. lisp object pure storage.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."




^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: BASE_PURESIZE
  2009-10-24 10:22     ` BASE_PURESIZE Eli Zaretskii
@ 2009-10-24 11:14       ` Stephen J. Turnbull
  0 siblings, 0 replies; 31+ messages in thread
From: Stephen J. Turnbull @ 2009-10-24 11:14 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: emacs-devel

Eli Zaretskii writes:

 > > 3.  To save the space, dump twice, the second time using the precise
 > >     number you can measure from the first try.

 > PURESIZE is a compile-time C constant, it cannot be changed at dump
 > time.  You need to recompile.

OK, so recompile the relevant module and relink.  You may actually be
pushing 60 extra seconds on a very slow machine at this point.

If you're more worried about errors in the computation of PURESIZE or
of the amount actually used when dumping, that's another issue, of
course, and this doesn't help with that.




^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: BASE_PURESIZE
  2009-10-24 10:37       ` BASE_PURESIZE Andreas Schwab
@ 2009-10-24 13:18         ` Eli Zaretskii
  2009-10-24 17:27           ` BASE_PURESIZE Dan Nicolaescu
  0 siblings, 1 reply; 31+ messages in thread
From: Eli Zaretskii @ 2009-10-24 13:18 UTC (permalink / raw)
  To: Andreas Schwab; +Cc: emacs-devel

> From: Andreas Schwab <schwab@linux-m68k.org>
> Cc: emacs-devel@gnu.org
> Date: Sat, 24 Oct 2009 12:37:19 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > For the record, the extra use of purecopy caused the pure_bytes_used
> > value to go up by 52KB on 32-bit Windows, and by 92KB on 64-bit
> > GNU/Linux.  So it looks like the ratio is actually closer to 9/5 than
> > to either the old 10/6 or the new 11/7.  Or maybe I'm missing
> > something.
> 
> It all depends on the ratio of string data vs. lisp object pure storage.

I made some measurements.  The ratio of 11/7 seems to work pretty
well, but there are two additional problems:

 . The default value of SYSTEM_PURESIZE_EXTRA is zero, and is not
   increased for GUI builds.  This causes a --without-x build to waste
   some 100KB.  If we want to handle this, the basic constant in
   BASE_PURESIZE can be as low as 1290000 and SYSTEM_PURESIZE_EXTRA
   should have its default at 140000 for GUI builds, zero otherwise.

 . The amount of pure storage used by load-history depends on the
   length of the filename of the directory where Emacs is dumped.  In
   my case, I have 32 characters before the "emacs/lisp/" part, so I'm
   guessing that's the main reason the value of 1430000 was too small
   for me.

We could decide that we don't care too much about the --without-x
case, but what about the second problem?  If we want to handle it
without wasting storage on systems with shorter file names, we would
need some code in src/Makefile.in that would measure the length of the
directory name and enlarge PURESIZE accordingly.




^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: BASE_PURESIZE
  2009-10-24 13:18         ` BASE_PURESIZE Eli Zaretskii
@ 2009-10-24 17:27           ` Dan Nicolaescu
  2009-10-24 19:01             ` BASE_PURESIZE Stefan Monnier
  2009-10-24 19:04             ` BASE_PURESIZE Chong Yidong
  0 siblings, 2 replies; 31+ messages in thread
From: Dan Nicolaescu @ 2009-10-24 17:27 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: Andreas Schwab, emacs-devel

Eli Zaretskii <eliz@gnu.org> writes:

  > > From: Andreas Schwab <schwab@linux-m68k.org>
  > > Cc: emacs-devel@gnu.org
  > > Date: Sat, 24 Oct 2009 12:37:19 +0200
  > > 
  > > Eli Zaretskii <eliz@gnu.org> writes:
  > > 
  > > > For the record, the extra use of purecopy caused the pure_bytes_used
  > > > value to go up by 52KB on 32-bit Windows, and by 92KB on 64-bit
  > > > GNU/Linux.  So it looks like the ratio is actually closer to 9/5 than
  > > > to either the old 10/6 or the new 11/7.  Or maybe I'm missing
  > > > something.
  > > 
  > > It all depends on the ratio of string data vs. lisp object pure storage.
  > 
  > I made some measurements.  The ratio of 11/7 seems to work pretty
  > well, but there are two additional problems:
  > 
  >  . The default value of SYSTEM_PURESIZE_EXTRA is zero, and is not
  >    increased for GUI builds.  This causes a --without-x build to waste
  >    some 100KB.  If we want to handle this, the basic constant in
  >    BASE_PURESIZE can be as low as 1290000 and SYSTEM_PURESIZE_EXTRA
  >    should have its default at 140000 for GUI builds, zero otherwise.
  > 
  >  . The amount of pure storage used by load-history depends on the
  >    length of the filename of the directory where Emacs is dumped.  In
  >    my case, I have 32 characters before the "emacs/lisp/" part, so I'm
  >    guessing that's the main reason the value of 1430000 was too small
  >    for me.

We have 2 more problems with load-history: although in loadup.el is
purecopied, something still seems to maintain references to the file
name strings, they are still present as non-pure strings in the dumped
image both as absolute file names and as the arguments passed to load
(see the simple patch I posted yesterday to dump strings).  So we are
still wasting memory on those.

It would be great if load-history would be constructed in pure memory
from the beginning when dumping (instead of purecopying later).
Maybe someone that understands that code could do that...

  > We could decide that we don't care too much about the --without-x

IMHO --without-x is completely unimportant.




^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: BASE_PURESIZE
  2009-10-24 17:27           ` BASE_PURESIZE Dan Nicolaescu
@ 2009-10-24 19:01             ` Stefan Monnier
  2009-10-25  8:24               ` BASE_PURESIZE Dan Nicolaescu
  2009-10-24 19:04             ` BASE_PURESIZE Chong Yidong
  1 sibling, 1 reply; 31+ messages in thread
From: Stefan Monnier @ 2009-10-24 19:01 UTC (permalink / raw)
  To: Dan Nicolaescu; +Cc: Eli Zaretskii, Andreas Schwab, emacs-devel

>> . The default value of SYSTEM_PURESIZE_EXTRA is zero, and is not
>> increased for GUI builds.  This causes a --without-x build to waste
>> some 100KB.  If we want to handle this, the basic constant in
>> BASE_PURESIZE can be as low as 1290000 and SYSTEM_PURESIZE_EXTRA
>> should have its default at 140000 for GUI builds, zero otherwise.

I'm not terribly concerned about wasting 100KB in pure space (or even
more than that): it's space we don't ever touch or look at, so it's
never brought into RAM.  IOW it's very cheap.
OTOH checking what really goes into purespace and why something doesn't
is good, because moving data into purespace is good.


        Stefan


PS: Regarding the recent purecopy of autoload's file names: why not just
purecopy the whole autload list?


--- src/eval.c	2009-10-24 12:32:03 +0000
+++ src/eval.c	2009-10-24 18:32:23 +0000
@@ -2112,10 +2148,7 @@
      (function, file, docstring, interactive, type)
      Lisp_Object function, file, docstring, interactive, type;
 {
-  Lisp_Object args[4];
-
   CHECK_SYMBOL (function);
-  CHECK_STRING (file);
 
   /* If function is defined and not as an autoload, don't override */
   if (!EQ (XSYMBOL (function)->function, Qunbound)
@@ -2128,15 +2161,13 @@
        not useful and else we get loads of them from the loaddefs.el.  */
     LOADHIST_ATTACH (Fcons (Qautoload, function));
 
-  if (NILP (Vpurify_flag))
-    args[0] = file;
-  else
-    args[0] = Fpurecopy (file);
-  args[1] = docstring;
-  args[2] = interactive;
-  args[3] = type;
-
-  return Ffset (function, Fcons (Qautoload, Flist (4, &args[0])));
+  if (!NILP (Vpurify_flag))
+    /* We don't want the docstring in purespace (instead,
+       Snarf-documentation should (hopefully) overwrite it).  */
+    docstring = make_number (0);
+  return Ffset (function,
+		Fpurecopy (list5 (Qautoload, file, docstring,
+				  interactive, type)));
 }
 
 Lisp_Object





^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: BASE_PURESIZE
  2009-10-24 17:27           ` BASE_PURESIZE Dan Nicolaescu
  2009-10-24 19:01             ` BASE_PURESIZE Stefan Monnier
@ 2009-10-24 19:04             ` Chong Yidong
  2009-10-24 19:16               ` BASE_PURESIZE Dan Nicolaescu
  1 sibling, 1 reply; 31+ messages in thread
From: Chong Yidong @ 2009-10-24 19:04 UTC (permalink / raw)
  To: Dan Nicolaescu; +Cc: Eli Zaretskii, Andreas Schwab, emacs-devel

Dan Nicolaescu <dann@ics.uci.edu> writes:

>   > We could decide that we don't care too much about the --without-x
>
> IMHO --without-x is completely unimportant.

Not sure what you mean by this statement.  If you mean that wasting a
few extra kB of memory on --without-x builds isn't important, I can
agree with that.  But I don't even care about a few extra kB in X
builds, personally.




^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: BASE_PURESIZE
  2009-10-24 19:04             ` BASE_PURESIZE Chong Yidong
@ 2009-10-24 19:16               ` Dan Nicolaescu
  0 siblings, 0 replies; 31+ messages in thread
From: Dan Nicolaescu @ 2009-10-24 19:16 UTC (permalink / raw)
  To: Chong Yidong; +Cc: Eli Zaretskii, Andreas Schwab, emacs-devel

Chong Yidong <cyd@stupidchicken.com> writes:

  > Dan Nicolaescu <dann@ics.uci.edu> writes:
  > 
  > >   > We could decide that we don't care too much about the --without-x
  > >
  > > IMHO --without-x is completely unimportant.
  > 
  > Not sure what you mean by this statement.  If you mean that wasting a
  > few extra kB of memory on --without-x builds isn't important,

That's exactly what I mean.




^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: BASE_PURESIZE
  2009-10-24 19:01             ` BASE_PURESIZE Stefan Monnier
@ 2009-10-25  8:24               ` Dan Nicolaescu
  2009-10-30  1:17                 ` defcustom standard-value (was: Re: BASE_PURESIZE) Dan Nicolaescu
  0 siblings, 1 reply; 31+ messages in thread
From: Dan Nicolaescu @ 2009-10-25  8:24 UTC (permalink / raw)
  To: Stefan Monnier; +Cc: Eli Zaretskii, Andreas Schwab, emacs-devel

Stefan Monnier <monnier@iro.umontreal.ca> writes:

  > PS: Regarding the recent purecopy of autoload's file names: why not just
  > purecopy the whole autload list?

If it works it should have an important impact...
So please install it.  I bet you'll have to bump BASE_PURESIZE again 
after doing that :-)

Another issue that I hit:

Changing (in files.el)

From:
(defcustom directory-free-space-program "df"
to:
(defcustom directory-free-space-program (purecopy "df")

does not get rid of the "df" string in GC memory.  The reason is

ELISP> (get 'directory-free-space-program 'standard-value)
((purecopy "df"))

BTW, this is not a problem for autoloaded defcustoms, they get
transformed into defvars in loaddefs.el.




^ permalink raw reply	[flat|nested] 31+ messages in thread

* defcustom standard-value (was: Re: BASE_PURESIZE)
  2009-10-25  8:24               ` BASE_PURESIZE Dan Nicolaescu
@ 2009-10-30  1:17                 ` Dan Nicolaescu
  0 siblings, 0 replies; 31+ messages in thread
From: Dan Nicolaescu @ 2009-10-30  1:17 UTC (permalink / raw)
  To: Stefan Monnier; +Cc: Eli Zaretskii, Andreas Schwab, emacs-devel

Dan Nicolaescu <dann@ics.uci.edu> writes:

  > Another issue that I hit:
  > 
  > Changing (in files.el)
  > 
  > From:
  > (defcustom directory-free-space-program "df"
  > to:
  > (defcustom directory-free-space-program (purecopy "df")
  > 
  > does not get rid of the "df" string in GC memory.  The reason is
  > 
  > ELISP> (get 'directory-free-space-program 'standard-value)
  > ((purecopy "df"))
  > 
  > BTW, this is not a problem for autoloaded defcustoms, they get
  > transformed into defvars in loaddefs.el.

Is this patch TRTD?

Index: lisp/custom.el
===================================================================
RCS file: /cvsroot/emacs/emacs/lisp/custom.el,v
retrieving revision 1.149
diff -u -3 -p -r1.149 custom.el
--- lisp/custom.el    15 Sep 2009 04:08:04 -0000        1.149
+++ lisp/custom.el    29 Oct 2009 07:50:00 -0000
@@ -131,7 +131,7 @@ not the default value itself.
 DEFAULT is stored as SYMBOL's standard value, in SYMBOL's property
 `standard-value'.  At the same time, SYMBOL's property `force-value' is
 set to nil, as the value is no longer rogue."
-  (put symbol 'standard-value (list default))
+  (put symbol 'standard-value (purecopy (list default)))
   ;; Maybe this option was rogue in an earlier version.  It no longer is.
   (when (get symbol 'force-value)
     (put symbol 'force-value nil))




^ permalink raw reply	[flat|nested] 31+ messages in thread

end of thread, other threads:[~2009-10-30  1:17 UTC | newest]

Thread overview: 31+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2009-10-23 11:00 BASE_PURESIZE Eli Zaretskii
2009-10-23 11:39 ` BASE_PURESIZE Andreas Schwab
2009-10-23 14:10   ` BASE_PURESIZE Eli Zaretskii
2009-10-23 14:24     ` BASE_PURESIZE Andreas Schwab
2009-10-23 14:35       ` BASE_PURESIZE Eli Zaretskii
2009-10-23 14:50         ` BASE_PURESIZE Andreas Schwab
2009-10-24 10:05     ` BASE_PURESIZE Eli Zaretskii
2009-10-24 10:37       ` BASE_PURESIZE Andreas Schwab
2009-10-24 13:18         ` BASE_PURESIZE Eli Zaretskii
2009-10-24 17:27           ` BASE_PURESIZE Dan Nicolaescu
2009-10-24 19:01             ` BASE_PURESIZE Stefan Monnier
2009-10-25  8:24               ` BASE_PURESIZE Dan Nicolaescu
2009-10-30  1:17                 ` defcustom standard-value (was: Re: BASE_PURESIZE) Dan Nicolaescu
2009-10-24 19:04             ` BASE_PURESIZE Chong Yidong
2009-10-24 19:16               ` BASE_PURESIZE Dan Nicolaescu
2009-10-23 11:58 ` BASE_PURESIZE Dan Nicolaescu
2009-10-23 14:24 ` BASE_PURESIZE Juanma Barranquero
2009-10-24  4:41   ` BASE_PURESIZE Stephen J. Turnbull
2009-10-24  6:47     ` BASE_PURESIZE Dan Nicolaescu
2009-10-24  8:24       ` BASE_PURESIZE Stephen J. Turnbull
2009-10-24 10:22     ` BASE_PURESIZE Eli Zaretskii
2009-10-24 11:14       ` BASE_PURESIZE Stephen J. Turnbull
  -- strict thread matches above, loose matches on Subject: below --
2006-08-05  2:16 BASE_PURESIZE Nick Roberts
2006-08-05  9:41 ` BASE_PURESIZE Eli Zaretskii
2006-08-05 16:23   ` BASE_PURESIZE Reiner Steib
2006-08-05 18:51     ` BASE_PURESIZE Richard Stallman
2006-08-05 20:07       ` BASE_PURESIZE Reiner Steib
2006-08-06  4:43         ` BASE_PURESIZE Richard Stallman
2006-08-06  9:19           ` BASE_PURESIZE Reiner Steib
2006-08-06 19:40         ` BASE_PURESIZE Reiner Steib
     [not found]           ` <jebqqx393g.fsf@sykes.suse.de>
2006-08-07  3:53             ` BASE_PURESIZE Eli Zaretskii

Code repositories for project(s) associated with this external index

	https://git.savannah.gnu.org/cgit/emacs.git
	https://git.savannah.gnu.org/cgit/emacs/org-mode.git

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.