Memory Leak was: Re: Update on the Emacs release schedule? 8 messages

Memory Leak was: Re: Update on the Emacs release schedule? 8 messages

[T. V. Raman]

I'm running Emacs 24 on three machines, and am seeing leaks on
one of them -- Details below.

Home: Ubuntu Jaunty x86_32 quad-core -- No Leak
Laptop: Intel x86_64 dual core Ubuntu Lucid -- No Leak
Work: amd_64 16-core HP  workstation Ubuntu Lucid -- Leak!

I know other Emacs hackers at work who are running the identical
Lucid  build on 64-bit machines -- but not on the 16-core HP
workstation who are not seeing the leak.

I've tried building with and without gnutls -- no difference.

Symptoms of the leak:

The office workstation has 12gb of RAM and 36GB swap.

Before starting Emacs: `free' shows 2gb in use.

Start Emacs 24 -- with just emacspeak loaded -- no immediate
signs of a leak -- `free' shows abut 6gb in use -- which does
seem a lot.

M-x shell in that emacs
and wait for a couple of minutes.

Running `free' on a separate terminal shows that all 12GB  of
memory in use -- emacs RSS is   at 10GB.

If you dont kill the runnning emacs-24 at that point, it brings
the workstation to its knees and the machine stops responding and
needs to be rebooted.

--Raman 

-- 
Best Regards,
--raman

Re: Memory Leak was: Re: Update on the Emacs release schedule? 8 messages

[Stefan Monnier]

> Before starting Emacs: `free' shows 2gb in use.

FWIW, "free" is not a very good tool to track the memory use of
a specific process.  Better check the VSZ and RSS of the process itself
(and note that RSS can stay stable even in the presence of a leak,
so VSZ is important).

> Start Emacs 24 -- with just emacspeak loaded -- no immediate
> signs of a leak -- `free' shows abut 6gb in use -- which does
> seem a lot.

What is Emacs's RSS and/or VSZ at startup?

> M-x shell in that emacs
> and wait for a couple of minutes.

> Running `free' on a separate terminal shows that all 12GB  of
> memory in use -- emacs RSS is   at 10GB.

> If you dont kill the runnning emacs-24 at that point, it brings
> the workstation to its knees and the machine stops responding and
> needs to be rebooted.

So you're saying that "emacs -Q" plus emacspeak plus "M-x shell" results
on this machine in a process hat keeps growing even if you leave
it alone?
Could you run it under GDB (from the `emacs/src' directory) and
interrupt the process (with C-z) every once in a while to try and see
what it's doing?


        Stefan

Memory Leak was: Re: Update on the Emacs release schedule? 8 messages

[T.V. Raman]

Stephane,

Unfortunately, Emacspeak is what talks to me, so it's well-nigh
impossible for me to debug emacs with GDB.

But here are a few additional bits of info that might help:

RSS etc that I reported were from proced in emacs ---
also, it's hard to get any useful info out snce the workstation
crashes or becomes unusable fairly rapidly.

I'm attaching two other bits of information that may be useful:

The binary src/emacs is approx 13mb big on my laptop and home
machines where I dont see the leak -- 7mb after
stripping. Constrast this with the numbers I see on the machine
where I do see the leak -- 19mb before stripping, 12mb+ after
stripping. I'll also attach the output of ldd on the offending
machine in case that sheds any light.

ls -l src/emacs
-rwxr-x--- 3 raman eng 19612869 2012-01-06 15:37 src/emacs*
16:34:49 retriever emacs $ strip src/emacs
16:34:56 retriever emacs $ !ls
ls -l src/emacs
-rwxr-x--- 3 raman eng 12707840 2012-01-17 16:34 src/emacs*
16:35:01 retriever emacs $ ldd src/emacs
	linux-vdso.so.1 =>  (0x00007fff4bf8b000)
	libgtk-x11-2.0.so.0 => /usr/lib/libgtk-x11-2.0.so.0 (0x00007f2ef3ae6000)
	libgdk-x11-2.0.so.0 => /usr/lib/libgdk-x11-2.0.so.0 (0x00007f2ef3839000)
	libatk-1.0.so.0 => /usr/lib/libatk-1.0.so.0 (0x00007f2ef3617000)
	libgio-2.0.so.0 => /usr/lib/libgio-2.0.so.0 (0x00007f2ef3363000)
	libpangoft2-1.0.so.0 => /usr/lib/libpangoft2-1.0.so.0 (0x00007f2ef3139000)
	libgdk_pixbuf-2.0.so.0 => /usr/lib/libgdk_pixbuf-2.0.so.0 (0x00007f2ef2f1c000)
	libm.so.6 => /lib/libm.so.6 (0x00007f2ef2c99000)
	libpangocairo-1.0.so.0 => /usr/lib/libpangocairo-1.0.so.0 (0x00007f2ef2a8c000)
	libcairo.so.2 => /usr/lib/libcairo.so.2 (0x00007f2ef2808000)
	libpango-1.0.so.0 => /usr/lib/libpango-1.0.so.0 (0x00007f2ef25be000)
	libfreetype.so.6 => /usr/lib/libfreetype.so.6 (0x00007f2ef2338000)
	libfontconfig.so.1 => /usr/lib/libfontconfig.so.1 (0x00007f2ef2102000)
	libgobject-2.0.so.0 => /usr/lib/libgobject-2.0.so.0 (0x00007f2ef1eba000)
	libgmodule-2.0.so.0 => /usr/lib/libgmodule-2.0.so.0 (0x00007f2ef1cb6000)
	libgthread-2.0.so.0 => /usr/lib/libgthread-2.0.so.0 (0x00007f2ef1ab0000)
	librt.so.1 => /lib/librt.so.1 (0x00007f2ef18a8000)
	libglib-2.0.so.0 => /lib/libglib-2.0.so.0 (0x00007f2ef15ca000)
	libSM.so.6 => /usr/lib/libSM.so.6 (0x00007f2ef13c0000)
	libICE.so.6 => /usr/lib/libICE.so.6 (0x00007f2ef11a5000)
	libtiff.so.4 => /usr/lib/libtiff.so.4 (0x00007f2ef0f43000)
	libjpeg.so.62 => /usr/lib/libjpeg.so.62 (0x00007f2ef0d1e000)
	libpng12.so.0 => /lib/libpng12.so.0 (0x00007f2ef0af7000)
	libz.so.1 => /lib/libz.so.1 (0x00007f2ef08e0000)
	libgif.so.4 => /usr/lib/libgif.so.4 (0x00007f2ef06d6000)
	libXpm.so.4 => /usr/lib/libXpm.so.4 (0x00007f2ef04c5000)
	libX11.so.6 => /usr/lib/libX11.so.6 (0x00007f2ef018f000)
	libXrender.so.1 => /usr/lib/libXrender.so.1 (0x00007f2eeff84000)
	libXft.so.2 => /usr/lib/libXft.so.2 (0x00007f2eefd6f000)
	libasound.so.2 => /usr/lib/libasound.so.2 (0x00007f2eefa8e000)
	librsvg-2.so.2 => /usr/lib/librsvg-2.so.2 (0x00007f2eef856000)
	libdbus-1.so.3 => /lib/libdbus-1.so.3 (0x00007f2eef617000)
	libpthread.so.0 => /lib/libpthread.so.0 (0x00007f2eef3fa000)
	libxml2.so.2 => /usr/lib/libxml2.so.2 (0x00007f2eef0a9000)
	libgpm.so.2 => /usr/lib/libgpm.so.2 (0x00007f2eeeea2000)
	libncurses.so.5 => /lib/libncurses.so.5 (0x00007f2eeec5f000)
	libgconf-2.so.4 => /usr/lib/libgconf-2.so.4 (0x00007f2eeea21000)
	libselinux.so.1 => /lib/libselinux.so.1 (0x00007f2eee803000)
	libotf.so.0 => /usr/lib/libotf.so.0 (0x00007f2eee5ef000)
	libm17n-flt.so.0 => /usr/lib/libm17n-flt.so.0 (0x00007f2eee3e4000)
	libm17n-core.so.0 => /usr/lib/libm17n-core.so.0 (0x00007f2eee1b7000)
	libgnutls.so.26 => /usr/local/lib/libgnutls.so.26 (0x00007f2eedf0b000)
	libc.so.6 => /lib/libc.so.6 (0x00007f2eedb87000)
	libXext.so.6 => /usr/lib/libXext.so.6 (0x00007f2eed975000)
	libXinerama.so.1 => /usr/lib/libXinerama.so.1 (0x00007f2eed772000)
	libXi.so.6 => /usr/lib/libXi.so.6 (0x00007f2eed561000)
	libXrandr.so.2 => /usr/lib/libXrandr.so.2 (0x00007f2eed358000)
	libXcursor.so.1 => /usr/lib/libXcursor.so.1 (0x00007f2eed14e000)
	libXcomposite.so.1 => /usr/lib/libXcomposite.so.1 (0x00007f2eecf4a000)
	libXdamage.so.1 => /usr/lib/libXdamage.so.1 (0x00007f2eecd47000)
	libXfixes.so.3 => /usr/lib/libXfixes.so.3 (0x00007f2eecb41000)
	libdl.so.2 => /lib/libdl.so.2 (0x00007f2eec93c000)
	libpcre.so.3 => /lib/libpcre.so.3 (0x00007f2eec70e000)
	libresolv.so.2 => /lib/libresolv.so.2 (0x00007f2eec4f4000)
	libpixman-1.so.0 => /usr/lib/libpixman-1.so.0 (0x00007f2eec29b000)
	libdirectfb-1.2.so.0 => /usr/lib/libdirectfb-1.2.so.0 (0x00007f2eec017000)
	libfusion-1.2.so.0 => /usr/lib/libfusion-1.2.so.0 (0x00007f2eebe0d000)
	libdirect-1.2.so.0 => /usr/lib/libdirect-1.2.so.0 (0x00007f2eebbf4000)
	libxcb-render-util.so.0 => /usr/lib/libxcb-render-util.so.0
(0x00007f2eeb9ef000)
	libxcb-render.so.0 => /usr/lib/libxcb-render.so.0 (0x00007f2eeb7e6000)
	libxcb.so.1 => /usr/lib/libxcb.so.1 (0x00007f2eeb5ca000)
	libexpat.so.1 => /lib/libexpat.so.1 (0x00007f2eeb3a0000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f2ef412a000)
	libuuid.so.1 => /lib/libuuid.so.1 (0x00007f2eeb19b000)
	libgsf-1.so.114 => /usr/lib/libgsf-1.so.114 (0x00007f2eeaf5b000)
	libcroco-0.6.so.3 => /usr/lib/libcroco-0.6.so.3 (0x00007f2eead22000)
	libORBit-2.so.0 => /usr/lib/libORBit-2.so.0 (0x00007f2eeaab3000)
	libdbus-glib-1.so.2 => /usr/lib/libdbus-glib-1.so.2 (0x00007f2eea891000)
	libthai.so.0 => /usr/lib/libthai.so.0 (0x00007f2eea688000)
	libtasn1.so.3 => /usr/lib/libtasn1.so.3 (0x00007f2eea476000)
	libgcrypt.so.11 => /lib/libgcrypt.so.11 (0x00007f2eea1fe000)
	libgpg-error.so.0 => /lib/libgpg-error.so.0 (0x00007f2ee9ffa000)
	libXau.so.6 => /usr/lib/libXau.so.6 (0x00007f2ee9df5000)
	libXdmcp.so.6 => /usr/lib/libXdmcp.so.6 (0x00007f2ee9bef000)
	libbz2.so.1.0 => /lib/libbz2.so.1.0 (0x00007f2ee99dd000)
	libdatrie.so.1 => /usr/lib/libdatrie.so.1 (0x00007f2ee97d7000)
16:35:09 retriever emacs $

On 1/8/12, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
>> Before starting Emacs: `free' shows 2gb in use.
>
> FWIW, "free" is not a very good tool to track the memory use of
> a specific process.  Better check the VSZ and RSS of the process itself
> (and note that RSS can stay stable even in the presence of a leak,
> so VSZ is important).
>
>> Start Emacs 24 -- with just emacspeak loaded -- no immediate
>> signs of a leak -- `free' shows abut 6gb in use -- which does
>> seem a lot.
>
> What is Emacs's RSS and/or VSZ at startup?
>
>> M-x shell in that emacs
>> and wait for a couple of minutes.
>
>> Running `free' on a separate terminal shows that all 12GB  of
>> memory in use -- emacs RSS is   at 10GB.
>
>> If you dont kill the runnning emacs-24 at that point, it brings
>> the workstation to its knees and the machine stops responding and
>> needs to be rebooted.
>
> So you're saying that "emacs -Q" plus emacspeak plus "M-x shell" results
> on this machine in a process hat keeps growing even if you leave
> it alone?
> Could you run it under GDB (from the `emacs/src' directory) and
> interrupt the process (with C-z) every once in a while to try and see
> what it's doing?
>
>
>         Stefan
>

Memory Leak was: Re: Update on the Emacs release schedule? 8 messages

[T.V. Raman]

Hi Stephane,

I just installed the emacs 23.4 prebuild on the workstation that
shows the memory leak with emacs-24 from git, and am happy to
report there is no leak.

On 1/8/12, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
>> Before starting Emacs: `free' shows 2gb in use.
>
> FWIW, "free" is not a very good tool to track the memory use of
> a specific process.  Better check the VSZ and RSS of the process itself
> (and note that RSS can stay stable even in the presence of a leak,
> so VSZ is important).
>
>> Start Emacs 24 -- with just emacspeak loaded -- no immediate
>> signs of a leak -- `free' shows abut 6gb in use -- which does
>> seem a lot.
>
> What is Emacs's RSS and/or VSZ at startup?
>
>> M-x shell in that emacs
>> and wait for a couple of minutes.
>
>> Running `free' on a separate terminal shows that all 12GB  of
>> memory in use -- emacs RSS is   at 10GB.
>
>> If you dont kill the runnning emacs-24 at that point, it brings
>> the workstation to its knees and the machine stops responding and
>> needs to be rebooted.
>
> So you're saying that "emacs -Q" plus emacspeak plus "M-x shell" results
> on this machine in a process hat keeps growing even if you leave
> it alone?
> Could you run it under GDB (from the `emacs/src' directory) and
> interrupt the process (with C-z) every once in a while to try and see
> what it's doing?
>
>
>         Stefan
>

Re: Memory Leak was: Re: Update on the Emacs release schedule? 8 messages

[Nix]

On 8 Jan 2012, T. V. Raman outgrape:
> Before starting Emacs: `free' shows 2gb in use.

An aside: the memory-in-use numbers from free are essentially
meaningless.

ps -o rss,vsz -C emacs

from a shell buffer is what you probably want.

(er, assuming that you can *get* a shell buffer without bringing Emacs
to its knees).

-- 
NULL && (void)