unofficial mirror of emacs-devel@gnu.org 
 help / color / mirror / code / Atom feed
* how do you track down emacs memory leaks?
@ 2008-11-02  4:05 Ami Fischman
  2008-11-03 15:13 ` Chong Yidong
  0 siblings, 1 reply; 7+ messages in thread
From: Ami Fischman @ 2008-11-02  4:05 UTC (permalink / raw)
  To: emacs-devel

[-- Attachment #1: Type: text/plain, Size: 1600 bytes --]

Hi emacs-devel,

I'm looking for advice on how to track down a memory leak in emacs.  CVS
HEAD builds I did on 20080702, 20080810, 20080915, and 20081025 all steadily
grow in process VmRSS and VmSize to the tune of ~70-100MB per day when I use
gnus and let it automatically check for new mail.  This quickly makes emacs
sluggish for all operations, presumably because of heap-walking slowing way
down (for GC? for consing?  I don't know).  Stefan Monnier's memory-usage.el
shows growth across the board over time, but none of the numbers come close
to the RSS.  For example right now my emacs is at 75MB of RSS (and 175M
total size) but memory-usage says:

Garbage collection stats:
((749993 . 70859) (52116 . 152) (7501 . 2362) 3678192 1020951 (676 . 756)
(19003 . 1331) (136481 . 18922))

 =>    6566816 bytes in cons cells
    1254432 bytes in symbols
    197260 bytes in markers
    3678192 bytes of string chars
    4083804 bytes of vector slots
    17184 bytes in floats
    569352 bytes in intervals

Total bytes in lisp objects (not counting string and vector headers):
13304187

Buffer ralloc memory usage:
67 buffers
949032 bytes total (240671 in gaps)

(for a total of ~22.8MB).

I've tried using valgrind and tcmalloc but neither works with a dumped emacs
binary, and I couldn't get enough work done with a temacs binary to extract
useful information.

So, anyone else see memory leaks in the last 6 months?  Anyone have any
clever tips on how to snapshot the lisp-space heap and different points in
time and then extracting the delta, preferably summarized by allocation
stacktrace?

-a

[-- Attachment #2: Type: text/html, Size: 1875 bytes --]

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: how do you track down emacs memory leaks?
  2008-11-02  4:05 how do you track down emacs memory leaks? Ami Fischman
@ 2008-11-03 15:13 ` Chong Yidong
  2008-11-03 15:24   ` Ami Fischman
  0 siblings, 1 reply; 7+ messages in thread
From: Chong Yidong @ 2008-11-03 15:13 UTC (permalink / raw)
  To: Ami Fischman; +Cc: emacs-devel

> CVS HEAD builds I did on 20080702, 20080810, 20080915, and 20081025
> all steadily grow in process VmRSS and VmSize to the tune of ~70-100MB
> per day when I use gnus and let it automatically check for new mail.

Does this problem only show up when you use gnus?  Could you try keeping
another Emacs session around for other non-gnus usage, and see if it
leaks memory?




^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: how do you track down emacs memory leaks?
  2008-11-03 15:13 ` Chong Yidong
@ 2008-11-03 15:24   ` Ami Fischman
  2008-11-04 16:59     ` Chong Yidong
  0 siblings, 1 reply; 7+ messages in thread
From: Ami Fischman @ 2008-11-03 15:24 UTC (permalink / raw)
  To: Chong Yidong; +Cc: emacs-devel

> Does this problem only show up when you use gnus?

Yes.

> Could you try keeping
> another Emacs session around for other non-gnus usage, and see if it
> leaks memory?

I have and it doesn't.  At least not nearly at the rate that the
gnus-using session does.

I updated my version of gnus from ngnus-0.10 to CVS head yesterday and
the leak seems to have slowed down significantly - only about 8MB
overnight.  So I suspect some trigger has been coincidentally
suppressed, but it seems that no elisp code should be able to cause
emacs to grow in memory usage disproportional to the numbers reported
by memory-usage, so there is still probably a lurking leak in emacs.

Do you have any tools/techniques you use to track down C-level leaks?
Do you have any tools/techniques to attribute the numbers in
memory-usage?  I.e. if you see "bytes in cons cells" go up
dramatically, any way to take snapshots of this data and then analyze
the difference visually or programmatically?

-a




^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: how do you track down emacs memory leaks?
  2008-11-03 15:24   ` Ami Fischman
@ 2008-11-04 16:59     ` Chong Yidong
  2008-12-20 16:27       ` Ami Fischman
  0 siblings, 1 reply; 7+ messages in thread
From: Chong Yidong @ 2008-11-04 16:59 UTC (permalink / raw)
  To: Ami Fischman; +Cc: emacs-devel

"Ami Fischman" <ami@fischman.org> writes:

>> Does this problem only show up when you use gnus?
>
> Yes.
>
>> Could you try keeping
>> another Emacs session around for other non-gnus usage, and see if it
>> leaks memory?
>
> I have and it doesn't.  At least not nearly at the rate that the
> gnus-using session does.
>
> I updated my version of gnus from ngnus-0.10 to CVS head yesterday and
> the leak seems to have slowed down significantly - only about 8MB
> overnight.  So I suspect some trigger has been coincidentally
> suppressed, but it seems that no elisp code should be able to cause
> emacs to grow in memory usage disproportional to the numbers reported
> by memory-usage, so there is still probably a lurking leak in emacs.
>
> Do you have any tools/techniques you use to track down C-level leaks?

I'm afraid not.  Maybe someone else on this list can suggest something.

One possibility is to try and write a simple test case that demonstrates
the leak.  For instance, a short Elisp program that keeps creating and
killing network processes.  If such a program causes memory to increase,
that would demonstrate that the memory leak is occurring in the Emacs
network process code.




^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: how do you track down emacs memory leaks?
  2008-11-04 16:59     ` Chong Yidong
@ 2008-12-20 16:27       ` Ami Fischman
  2008-12-20 17:38         ` Leo
  2008-12-20 20:39         ` Chong Yidong
  0 siblings, 2 replies; 7+ messages in thread
From: Ami Fischman @ 2008-12-20 16:27 UTC (permalink / raw)
  To: Chong Yidong; +Cc: emacs-devel

This leak ended up being caused by the bug I reported last night
(mode-line-eol-desc-cache being grown indefinitely).  Gnus was not the
cause but rather the trigger, since it visits so many files (using
nnml backend) and visiting files was a prime way to tickle the
mode-line-eol-desc-cache bug.

Since this thread was about debugging methods, I figured I'd add to it
the way that I finally found the culprit:

(defun ami-big-var-p (sym)
  (when (and (boundp sym)
             (> (length (prin1-to-string (symbol-value sym))) (* 1024 1024)))
    t))
(mapatoms '(lambda (sym)
             (when (ami-big-var-p sym)
               (message (symbol-name sym)))))

This led me to the insanely large variable and from there it was just
a matter of tracking down its construction.

Cheers,
-a

On Tue, Nov 4, 2008 at 8:59 AM, Chong Yidong <cyd@stupidchicken.com> wrote:
> "Ami Fischman" <ami@fischman.org> writes:
>
>>> Does this problem only show up when you use gnus?
>>
>> Yes.
>>
>>> Could you try keeping
>>> another Emacs session around for other non-gnus usage, and see if it
>>> leaks memory?
>>
>> I have and it doesn't.  At least not nearly at the rate that the
>> gnus-using session does.
>>
>> I updated my version of gnus from ngnus-0.10 to CVS head yesterday and
>> the leak seems to have slowed down significantly - only about 8MB
>> overnight.  So I suspect some trigger has been coincidentally
>> suppressed, but it seems that no elisp code should be able to cause
>> emacs to grow in memory usage disproportional to the numbers reported
>> by memory-usage, so there is still probably a lurking leak in emacs.
>>
>> Do you have any tools/techniques you use to track down C-level leaks?
>
> I'm afraid not.  Maybe someone else on this list can suggest something.
>
> One possibility is to try and write a simple test case that demonstrates
> the leak.  For instance, a short Elisp program that keeps creating and
> killing network processes.  If such a program causes memory to increase,
> that would demonstrate that the memory leak is occurring in the Emacs
> network process code.
>




^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: how do you track down emacs memory leaks?
  2008-12-20 16:27       ` Ami Fischman
@ 2008-12-20 17:38         ` Leo
  2008-12-20 20:39         ` Chong Yidong
  1 sibling, 0 replies; 7+ messages in thread
From: Leo @ 2008-12-20 17:38 UTC (permalink / raw)
  To: emacs-devel

On 2008-12-20 16:27 +0000, Ami Fischman wrote:
> This leak ended up being caused by the bug I reported last night
> (mode-line-eol-desc-cache being grown indefinitely).  Gnus was not the
> cause but rather the trigger, since it visits so many files (using
> nnml backend) and visiting files was a prime way to tickle the
> mode-line-eol-desc-cache bug.

Thank you for finding the bug. It helped my long running Emacs too.

-- 
.:  Leo  :.  [ sdl.web AT gmail.com ]  .: I use Emacs :.





^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: how do you track down emacs memory leaks?
  2008-12-20 16:27       ` Ami Fischman
  2008-12-20 17:38         ` Leo
@ 2008-12-20 20:39         ` Chong Yidong
  1 sibling, 0 replies; 7+ messages in thread
From: Chong Yidong @ 2008-12-20 20:39 UTC (permalink / raw)
  To: Ami Fischman; +Cc: emacs-devel

"Ami Fischman" <ami@fischman.org> writes:

> Since this thread was about debugging methods, I figured I'd add to it
> the way that I finally found the culprit:
>
> (defun ami-big-var-p (sym)
>   (when (and (boundp sym)
>              (> (length (prin1-to-string (symbol-value sym))) (* 1024 1024)))
>     t))
> (mapatoms '(lambda (sym)
>              (when (ami-big-var-p sym)
>                (message (symbol-name sym)))))
>
> This led me to the insanely large variable and from there it was just
> a matter of tracking down its construction.

Thanks.  Could you write this up into something that can go into
etc/DEBUG?




^ permalink raw reply	[flat|nested] 7+ messages in thread

end of thread, other threads:[~2008-12-20 20:39 UTC | newest]

Thread overview: 7+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2008-11-02  4:05 how do you track down emacs memory leaks? Ami Fischman
2008-11-03 15:13 ` Chong Yidong
2008-11-03 15:24   ` Ami Fischman
2008-11-04 16:59     ` Chong Yidong
2008-12-20 16:27       ` Ami Fischman
2008-12-20 17:38         ` Leo
2008-12-20 20:39         ` Chong Yidong

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/emacs.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).