Re: mem leak

Re: mem leak

[LENNART BORGMAN]

From: Miles Bader <miles.bader@necel.com>
> David Kastrup <dak@gnu.org> writes:
> > (garbage-collect)
> >
> > Reclaim storage for Lisp objects no longer needed.
> > Garbage collection happens automatically if you cons more than
> > `gc-cons-threshold' bytes of Lisp data since previous garbage 
> collection....
> > However, if there was overflow in pure space, `garbage-collect'
> > returns nil, because real GC can't be done.
> 
> That seems to answer my question -- when I do (garbage-collect), it
> returns nil!!!!!  If it's not GCing... :-O
> 
> I guess pure space overflowed when I built emacs and I didn't notice
> (even with all the recent discussion :-).

What message should one look for? Does not the build fail?

Re: mem leak

[Miles Bader]

LENNART BORGMAN <lennart.borgman.073@student.lu.se> writes:
>> I guess pure space overflowed when I built emacs and I didn't notice
>> (even with all the recent discussion :-).
>
> What message should one look for? Does not the build fail?

It prints a warning message, but it tends to get lost among the large
amount of other noise that emacs spits out during compilation.

[I don't have the exact message handy, but I believe it was quoted in
the recent discussion about purespace.]

-Miles
-- 
The car has become... an article of dress without which we feel uncertain,
unclad, and incomplete.  [Marshall McLuhan, Understanding Media, 1964]

Re: mem leak

[Romain Francoise]

Miles Bader <miles.bader@necel.com> writes:

> It prints a warning message, but it tends to get lost among the large
> amount of other noise that emacs spits out during compilation.

The splash screen shows a warning message when pure space overflowed
during the build.  It's hard to miss.

-- 
Romain Francoise <romain@orebokech.com> | The sea! the sea! the open
it's a miracle -- http://orebokech.com/ | sea! The blue, the fresh, the
                                        | ever free! --Bryan W. Procter

Re: mem leak

[Miles Bader]

Romain Francoise <romain@orebokech.com> writes:
>> It prints a warning message, but it tends to get lost among the large
>> amount of other noise that emacs spits out during compilation.
>
> The splash screen shows a warning message when pure space overflowed
> during the build.  It's hard to miss.

... unless you have the splash screen disabled (as I do) ... :-/

-miles

-- 
The car has become... an article of dress without which we feel uncertain,
unclad, and incomplete.  [Marshall McLuhan, Understanding Media, 1964]

Re: mem leak

[Miles Bader]

Miles Bader <miles.bader@necel.com> writes:
>>> It prints a warning message, but it tends to get lost among the large
>>> amount of other noise that emacs spits out during compilation.
>>
>> The splash screen shows a warning message when pure space overflowed
>> during the build.  It's hard to miss.
>
> ... unless you have the splash screen disabled (as I do) ... :-/

Is there any reason _not_ to make the build fail in this case?

As I've found an Emacs with an overflowed pure space is not very usable
anyway... so it's something that really should be fixed immediately by
the builder if it happens.

-miles
-- 
It wasn't the Exxon Valdez captain's driving that caused the Alaskan oil spill.
It was yours.  [Greenpeace advertisement, New York Times, 25 February 1990]

Re: mem leak

[Stefan Monnier]

> As I've found an Emacs with an overflowed pure space is not very usable
> anyway... so it's something that really should be fixed immediately by
> the builder if it happens.

Overflow can happen because the .elc file is missing, and the
not-completely-functional Emacs that results is quite functional enough to
then byte-compile the file, so it makes bootstrapping easier.


        Stefan

Re: mem leak

[Miles Bader]

Stefan Monnier <monnier@iro.umontreal.ca> writes:
> Overflow can happen because the .elc file is missing, and the
> not-completely-functional Emacs that results is quite functional enough to
> then byte-compile the file, so it makes bootstrapping easier.

Hmmm, how hard would it be to have it runnable, but not _installable_ in
such a case (for instance, if it retained the executable, but gave it
another name, "emacs-non-pure" or something)?

[I don't know whether it's easy to get the results of the failure in a
form that make can easily use...]

-Miles

-- 
Somebody has to do something, and it's just incredibly pathetic that it
has to be us.  -- Jerry Garcia

Re: mem leak

[Richard Stallman]

    Is there any reason _not_ to make the build fail in this case?

Yes.  Sometimes pure space overflow is due to a bug.  The best way to
investigate it is to look inside the dumped Emacs.  If overflow made
the build fail, you'd have to edit puresize.h before you could debug.

memory consumption with pure-space-overflow (was: mem leak)

[Reiner Steib]

On Mon, Apr 24 2006, Richard Stallman wrote:

>     Is there any reason _not_ to make the build fail in this case?
>
> Yes.  Sometimes pure space overflow is due to a bug.  The best way to
> investigate it is to look inside the dumped Emacs.  If overflow made
> the build fail, you'd have to edit puresize.h before you could debug.

I noticed an enormous memory consumption of emacs, but I didn't notice
the warning during compilation for a long time and I have
`inhibit-splash-screen' set to t.

Shouldn't we add an appropriate `display-warning' in `startup.el' if
`pure-space-overflow' is t even if `inhibit-splash-screen' it t?
And/or add a "sleep ..." in the Makefile in this case.

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

Re: memory consumption with pure-space-overflow (was: mem leak)

[Richard Stallman]

    Shouldn't we add an appropriate `display-warning' in `startup.el' if
    `pure-space-overflow' is t even if `inhibit-splash-screen' it t?

That sounds good.  Please do.

Re: memory consumption with pure-space-overflow

[Reiner Steib]

On Thu, Apr 27 2006, Richard Stallman wrote:

>     Shouldn't we add an appropriate `display-warning' in `startup.el' if
>     `pure-space-overflow' is t even if `inhibit-splash-screen' it t?
>
> That sounds good.  Please do.

Done.

I think we should tell the user what kind of problems are possible
because of `pure-space-overflow' and how to fix the overflow.  But I
don't know what we should suggest.

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

Re: memory consumption with pure-space-overflow

[Richard Stallman]

    I think we should tell the user what kind of problems are possible
    because of `pure-space-overflow' and how to fix the overflow.

A reference to the node Building Emacs in the Lisp manual
should be enough.

Re: memory consumption with pure-space-overflow

[Reiner Steib]

On Thu, Apr 27 2006, Richard Stallman wrote:

>     I think we should tell the user what kind of problems are possible
>     because of `pure-space-overflow' and how to fix the overflow.
>
> A reference to the node Building Emacs in the Lisp manual
> should be enough.

I've added a reference.

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

Re: memory consumption with pure-space-overflow

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> Date: Thu, 27 Apr 2006 17:27:04 -0400
> Cc: miles@gnu.org, emacs-devel@gnu.org
> 
>     I think we should tell the user what kind of problems are possible
>     because of `pure-space-overflow' and how to fix the overflow.
> 
> A reference to the node Building Emacs in the Lisp manual
> should be enough.

Done.

Re: memory consumption with pure-space-overflow

[Reiner Steib]

On Fri, May 05 2006, Eli Zaretskii wrote:

>> From: Richard Stallman <rms@gnu.org>
[...]
>> A reference to the node Building Emacs in the Lisp manual
>> should be enough.
>
> Done.

I had added a similar reference in `command-line-1' which I updated to
refer to "Pure Storage" now.  Maybe we should put the common text into
a defconst and/or use `display-warning' also in `normal-splash-screen'
and `fancy-splash-screens-1'?

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

Re: memory consumption with pure-space-overflow

[Richard Stallman]

    I had added a similar reference in `command-line-1' which I updated to
    refer to "Pure Storage" now.  Maybe we should put the common text into
    a defconst 

How big is this common text?

	       and/or use `display-warning' also in `normal-splash-screen'
    and `fancy-splash-screens-1'?

Use them for what?  What problem are you trying to solve?

Re: memory consumption with pure-space-overflow

[Reiner Steib]

On Sat, May 06 2006, Richard Stallman wrote:

>     I had added a similar reference in `command-line-1' which I updated to
>     refer to "Pure Storage" now.  Maybe we should put the common text into
>     a defconst 
>
> How big is this common text?

This text is used twice (once in `normal-splash-screen' and once
`fancy-splash-screens-1):

  Warning Warning!!!  Pure space overflow    !!!Warning Warning
  \(See the node Building Emacs in the Lisp manual for details.)\n

In `command-line-1' we have a similar text:

     (display-warning
      'initialization
      "Building Emacs overflowed pure space.  (See the node Pure Storage in the Lisp manual for details.)"
      :warning)

> 	       and/or use `display-warning' also in `normal-splash-screen'
>     and `fancy-splash-screens-1'?
>
> Use them for what?  What problem are you trying to solve?

(It's more a minor issue than a problem, but ...)

When updating the text, we might forget one of the three occurrences.
(See above.)

In `normal-splash-screen' and `fancy-splash-screens-1' (one of these
is called from `display-splash-screen'), we simply insert the waring
into the splash screen.  Maybe using `display-warning' would make the
warning stand out more clear.  E.g. when the user already types
something during initialization, the splash screen isn't displayed:

    ;; If user typed input during all that work,
    ;; abort the startup screen.  Otherwise, display it now.
    (unless (input-pending-p)
      (display-splash-screen))))

Maybe the place in `command-line-1' where I added `display-warning' is
not the best choice or we can simplify displaying the pure space
warning in some other way without having it several times in
`startup.el', e.g. using `display-warning' once if `(and
pure-space-overflow (not noninteractive))' is t.  I will look into
this in a few days (I don't have the sources here, ATM), unless .

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

Re: memory consumption with pure-space-overflow

[Richard Stallman]

    This text is used twice (once in `normal-splash-screen' and once
    `fancy-splash-screens-1):

      Warning Warning!!!  Pure space overflow    !!!Warning Warning
      \(See the node Building Emacs in the Lisp manual for details.)\n

    In `command-line-1' we have a similar text:

	 (display-warning
	  'initialization
	  "Building Emacs overflowed pure space.  (See the node Pure Storage in the Lisp manual for details.)"
	  :warning)

Please just leave it alone.  It's not worth the trouble of a subroutine
just because there are three similar 4-line pieces of code.

Let's not spend more time on this now.

Re: memory consumption with pure-space-overflow

[Juri Linkov]

> Maybe the place in `command-line-1' where I added `display-warning' is
> not the best choice or we can simplify displaying the pure space
> warning in some other way

Another way of displaying the pure space warning is using the %e specifier
in the mode line.  Currently, it shows the message "!MEM FULL! " in the
mode line when the variable Vmemory_full is non-nil.  It could display
a message "!PURE SPACE OVERFLOW! " in the mode line when the variable
pure_bytes_used_before_overflow is non-nil.

Regardless whether this is good idea or not, I want to turn attention
to the lack of documentation for some %-constructs in the mode line.
Below is a list of options currently documented in Info and docstring,
and really implemented options (thus spaces denote undocumented options):

In the node (info "(elisp)%-Constructs"): bc fFiIlmMnpPst  *+&[]-%
In the docstring of `mode-line-format':   bc fFiIlm npPs zZ*+&[]-%
Really implemented in `decode_mode_spec': bcefFiIlm npPstzZ*+&[]-%

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: memory consumption with pure-space-overflow

[Richard Stallman]

    Below is a list of options currently documented in Info and docstring,
    and really implemented options (thus spaces denote undocumented options):

    In the node (info "(elisp)%-Constructs"): bc fFiIlmMnpPst  *+&[]-%
    In the docstring of `mode-line-format':   bc fFiIlm npPs zZ*+&[]-%
    Really implemented in `decode_mode_spec': bcefFiIlm npPstzZ*+&[]-%

Would someone please write the missing documentation for options
e, t, z, Z, then ack?

Undocumented %-constructs (was: memory consumption with pure-space-overflow)

[Juri Linkov]

>     Below is a list of options currently documented in Info and docstring,
>     and really implemented options (thus spaces denote undocumented options):
>
>     In the node (info "(elisp)%-Constructs"): bc fFiIlmMnpPst  *+&[]-%
>     In the docstring of `mode-line-format':   bc fFiIlm npPs zZ*+&[]-%
>     Really implemented in `decode_mode_spec': bcefFiIlm npPstzZ*+&[]-%
>
> Would someone please write the missing documentation for options
> e, t, z, Z, then ack?

Below is a patch with the missing documentation based on the source code.
Please correct me if I misinterpreted the meaning of these constructs.
I removed %M from Info because it is not supported, while %m is supported,
albeit obsolete.

Index: lispref/modes.texi
===================================================================
RCS file: /sources/emacs/emacs/lispref/modes.texi,v
retrieving revision 1.139
diff -c -r1.139 modes.texi
*** lispref/modes.texi	30 Apr 2006 02:35:35 -0000	1.139
--- lispref/modes.texi	11 May 2006 23:03:59 -0000
***************
*** 1927,1932 ****
--- 1931,1939 ----
  @item %c
  The current column number of point.
  
+ @item %e
+ The error message if Emacs is close to out of memory for Lisp objects.
+ 
  @item %f
  The visited file name, obtained with the @code{buffer-file-name}
  function.  @xref{Buffer File Name}.
***************
*** 1972,1977 ****
--- 1979,1990 ----
  meaningful distinction only on certain operating systems (@pxref{MS-DOS
  File Types}).
  
+ @item %z
+ The mnemonics of buffer, terminal, and keyboard coding systems.
+ 
+ @item %Z
+ Like @samp{%z}, but including the end-of-line format.
+ 
  @item %*
  @samp{%} if the buffer is read only (see @code{buffer-read-only}); @*
  @samp{*} if the buffer is modified (see @code{buffer-modified-p}); @*
***************
*** 2003,2018 ****
  string in which @code{%}-constructs are allowed.
  @end table
  
! The following two @code{%}-constructs are still supported, but they are
! obsolete, since you can get the same results with the variables
! @code{mode-name} and @code{global-mode-string}.
  
  @table @code
  @item %m
  The value of @code{mode-name}.
- 
- @item %M
- The value of @code{global-mode-string}.
  @end table
  
  @node Properties in Mode
--- 2016,2028 ----
  string in which @code{%}-constructs are allowed.
  @end table
  
! The following @code{%}-construct is still supported, but it is
! obsolete, since you can get the same result with the variable
! @code{mode-name}.
  
  @table @code
  @item %m
  The value of @code{mode-name}.
  @end table
  
  @node Properties in Mode

Index: src/buffer.c
===================================================================
RCS file: /sources/emacs/emacs/src/buffer.c,v
retrieving revision 1.501
diff -c -r1.501 buffer.c
*** src/buffer.c	25 Feb 2006 23:33:57 -0000	1.501
--- src/buffer.c	11 May 2006 23:06:16 -0000
***************
*** 5440,5447 ****
--- 5440,5449 ----
          or print Bottom or All.
    %m -- print the mode name.
    %n -- print Narrow if appropriate.
+   %t -- visited file is text or binary (if OS supports this distinction).
    %z -- print mnemonics of buffer, terminal, and keyboard coding systems.
    %Z -- like %z, but including the end-of-line format.
+   %e -- print error message about full memory.
    %[ -- print one [ for each recursive editing level.  %] similar.
    %% -- print %.   %- -- print infinitely many dashes.
  Decimal digits after the % specify field width to which to pad.  */);

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: Undocumented %-constructs

[Johan Bockgård]

Juri Linkov <juri@jurta.org> writes:

> I removed %M from Info because it is not supported

But it is supported (see `display_mode_element').

-- 
Johan Bockgård

Re: Undocumented %-constructs

[Juri Linkov]

>> I removed %M from Info because it is not supported
>
> But it is supported (see `display_mode_element').

Yep, I missed this unusual place.  Please consider my latest patch
modulo removing %M.

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: Undocumented %-constructs (was: memory consumption with pure-space-overflow)

[Richard Stallman]

    + The error message if Emacs is close to out of memory for Lisp objects.

It would be clearer to say,

    When Emacs is nearly out of memory for Lisp objects,
    a brief message saying so.  Otherwise, this is empty.

Please install your patch, with that change.

mem leak

[Miles Bader]

Does anybody have advice for finding a memory leak in emacs?

Recent builds seems to have some leak:  even with only a few small
buffers open, Emacs rapidly consumes more and more memory (to the point
where the system eventually becomes unusable).

-Miles
-- 
Ich bin ein Virus. Mach' mit und kopiere mich in Deine .signature.

Re: mem leak

[David Kastrup]

Miles Bader <miles.bader@necel.com> writes:

> Does anybody have advice for finding a memory leak in emacs?
>
> Recent builds seems to have some leak:  even with only a few small
> buffers open, Emacs rapidly consumes more and more memory (to the point
> where the system eventually becomes unusable).

garbage-collect is an interactive built-in function in `C source code'.
(garbage-collect)

Reclaim storage for Lisp objects no longer needed.
Garbage collection happens automatically if you cons more than
`gc-cons-threshold' bytes of Lisp data since previous garbage collection.
`garbage-collect' normally returns a list with info on amount of space in use:
 ((USED-CONSES . FREE-CONSES) (USED-SYMS . FREE-SYMS)
  (USED-MARKERS . FREE-MARKERS) USED-STRING-CHARS USED-VECTOR-SLOTS
  (USED-FLOATS . FREE-FLOATS) (USED-INTERVALS . FREE-INTERVALS)
  (USED-STRINGS . FREE-STRINGS))
However, if there was overflow in pure space, `garbage-collect'
returns nil, because real GC can't be done.

[back]

This might help tracking down what kind of memory is consumed, and
whether it is Lisp-visible.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: mem leak

[Miles Bader]

David Kastrup <dak@gnu.org> writes:
> (garbage-collect)
>
> Reclaim storage for Lisp objects no longer needed.
> Garbage collection happens automatically if you cons more than
> `gc-cons-threshold' bytes of Lisp data since previous garbage collection.
...
> However, if there was overflow in pure space, `garbage-collect'
> returns nil, because real GC can't be done.

That seems to answer my question -- when I do (garbage-collect), it
returns nil!!!!!  If it's not GCing... :-O

I guess pure space overflowed when I built emacs and I didn't notice
(even with all the recent discussion :-).

Thanks,

-Miles
-- 
"Most attacks seem to take place at night, during a rainstorm, uphill,
 where four map sheets join."   -- Anon. British Officer in WW I