* Memory leak
@ 2008-05-15 22:50 Stefan Monnier
2008-05-15 23:06 ` David Kastrup
` (2 more replies)
0 siblings, 3 replies; 39+ messages in thread
From: Stefan Monnier @ 2008-05-15 22:50 UTC (permalink / raw
To: emacs-devel
We have a pretty serious memory leak.
My Emacs session recently died with a memory full situation (the
process was around 500MB VSZ). So I took a look around and it seem that
every time I create a new frame, my Emacs session grows by around 10MB
and deleting the frame does not recover this memory.
This memory is not accounted for by the counts returned by
`garbage-collect'.
I'm not sure when this problem got introduced, tho the new font code is
a likely culprit.
Stefan
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Memory leak
2008-05-15 22:50 Memory leak Stefan Monnier
@ 2008-05-15 23:06 ` David Kastrup
2008-05-16 0:25 ` Lennart Borgman (gmail)
2008-05-16 2:19 ` Memory leak Stefan Monnier
2008-05-16 0:38 ` YAMAMOTO Mitsuharu
2008-05-17 15:16 ` Evil Boris
2 siblings, 2 replies; 39+ messages in thread
From: David Kastrup @ 2008-05-15 23:06 UTC (permalink / raw
To: Stefan Monnier; +Cc: emacs-devel
Stefan Monnier <monnier@IRO.UMontreal.CA> writes:
> We have a pretty serious memory leak.
> My Emacs session recently died with a memory full situation (the
> process was around 500MB VSZ). So I took a look around and it seem that
> every time I create a new frame, my Emacs session grows by around 10MB
> and deleting the frame does not recover this memory.
> This memory is not accounted for by the counts returned by
> `garbage-collect'.
>
> I'm not sure when this problem got introduced, tho the new font code is
> a likely culprit.
Just to rule out something obvious (pure space overflow):
M-: (garbage-collect) RET
does not return nil, does it?
--
David Kastrup, Kriemhildstr. 15, 44793 Bochum
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Memory leak
2008-05-15 23:06 ` David Kastrup
@ 2008-05-16 0:25 ` Lennart Borgman (gmail)
2008-05-16 8:35 ` Eli Zaretskii
2008-05-16 9:43 ` Eli Zaretskii
2008-05-16 2:19 ` Memory leak Stefan Monnier
1 sibling, 2 replies; 39+ messages in thread
From: Lennart Borgman (gmail) @ 2008-05-16 0:25 UTC (permalink / raw
To: David Kastrup; +Cc: Stefan Monnier, emacs-devel
David Kastrup wrote:
> Stefan Monnier <monnier@IRO.UMontreal.CA> writes:
>
>> We have a pretty serious memory leak.
>> My Emacs session recently died with a memory full situation (the
>> process was around 500MB VSZ). So I took a look around and it seem that
>> every time I create a new frame, my Emacs session grows by around 10MB
>> and deleting the frame does not recover this memory.
>> This memory is not accounted for by the counts returned by
>> `garbage-collect'.
>>
>> I'm not sure when this problem got introduced, tho the new font code is
>> a likely culprit.
>
> Just to rule out something obvious (pure space overflow):
>
> M-: (garbage-collect) RET
>
> does not return nil, does it?
I can see a memory leak on w32 of around 240 K in the above operation
with a build from 2008-03-01, but not with a build from 2008-01-21 (no
build in between starts ok).
(garbage-collect) does not return nil with any of the builds above.
PS: Is there any command line utility that shows memory usage on w32?
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Memory leak
2008-05-15 22:50 Memory leak Stefan Monnier
2008-05-15 23:06 ` David Kastrup
@ 2008-05-16 0:38 ` YAMAMOTO Mitsuharu
2008-05-17 15:16 ` Evil Boris
2 siblings, 0 replies; 39+ messages in thread
From: YAMAMOTO Mitsuharu @ 2008-05-16 0:38 UTC (permalink / raw
To: Stefan Monnier; +Cc: emacs-devel
>>>>> On Thu, 15 May 2008 18:50:29 -0400, Stefan Monnier <monnier@IRO.UMontreal.CA> said:
> We have a pretty serious memory leak. My Emacs session recently
> died with a memory full situation (the process was around 500MB
> VSZ). So I took a look around and it seem that every time I create
> a new frame, my Emacs session grows by around 10MB and deleting the
> frame does not recover this memory. This memory is not accounted
> for by the counts returned by `garbage-collect'.
One thing I'm concerned about is the removal of `ftfont_free_entity'.
Actually, the `leaks' command on Mac OS X shows that several resources
allocated at ftfont_pattern_entity are leaking.
And some minor nitpicking: In xftfont_open, font->pixel_size and
font->driver are set twice (and to different values for
font->pixel_size).
YAMAMOTO Mitsuharu
mituharu@math.s.chiba-u.ac.jp
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Memory leak
2008-05-15 23:06 ` David Kastrup
2008-05-16 0:25 ` Lennart Borgman (gmail)
@ 2008-05-16 2:19 ` Stefan Monnier
1 sibling, 0 replies; 39+ messages in thread
From: Stefan Monnier @ 2008-05-16 2:19 UTC (permalink / raw
To: David Kastrup; +Cc: emacs-devel
> Just to rule out something obvious (pure space overflow):
> M-: (garbage-collect) RET
> does not return nil, does it?
No.
Stefan
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Memory leak
2008-05-16 0:25 ` Lennart Borgman (gmail)
@ 2008-05-16 8:35 ` Eli Zaretskii
2008-05-16 9:03 ` Juanma Barranquero
2008-05-16 9:43 ` Eli Zaretskii
1 sibling, 1 reply; 39+ messages in thread
From: Eli Zaretskii @ 2008-05-16 8:35 UTC (permalink / raw
To: Lennart Borgman (gmail); +Cc: emacs-devel
> Date: Fri, 16 May 2008 02:25:50 +0200
> From: "Lennart Borgman (gmail)" <lennart.borgman@gmail.com>
> Cc: Stefan Monnier <monnier@IRO.UMontreal.CA>, emacs-devel@gnu.org
>
> PS: Is there any command line utility that shows memory usage on w32?
Download PsTools, and then say "pslist -m emacs".
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Memory leak
2008-05-16 8:35 ` Eli Zaretskii
@ 2008-05-16 9:03 ` Juanma Barranquero
2008-05-16 9:25 ` Eli Zaretskii
0 siblings, 1 reply; 39+ messages in thread
From: Juanma Barranquero @ 2008-05-16 9:03 UTC (permalink / raw
To: Eli Zaretskii; +Cc: Lennart Borgman (gmail), emacs-devel
On Fri, May 16, 2008 at 10:35 AM, Eli Zaretskii <eliz@gnu.org> wrote:
> Download PsTools, and then say "pslist -m emacs".
Or, if you're using Windows XP Pro or a Server edition, you can simply do
tasklist /fi "imagename eq emacs*"
Juanma
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Memory leak
2008-05-16 9:03 ` Juanma Barranquero
@ 2008-05-16 9:25 ` Eli Zaretskii
2008-05-16 9:29 ` Juanma Barranquero
2008-05-16 20:41 ` David Robinow
0 siblings, 2 replies; 39+ messages in thread
From: Eli Zaretskii @ 2008-05-16 9:25 UTC (permalink / raw
To: Juanma Barranquero; +Cc: lennart.borgman, emacs-devel
> Date: Fri, 16 May 2008 11:03:30 +0200
> From: "Juanma Barranquero" <lekktu@gmail.com>
> Cc: "Lennart Borgman (gmail)" <lennart.borgman@gmail.com>, emacs-devel@gnu.org
>
> tasklist /fi "imagename eq emacs*"
You need to use emacs.exe as the IMAGENAME, otherwise it will not
work.
(Btw, I think this command is present on XP Home Edition as well.)
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Memory leak
2008-05-16 9:25 ` Eli Zaretskii
@ 2008-05-16 9:29 ` Juanma Barranquero
2008-05-16 9:46 ` Eli Zaretskii
2008-05-16 9:49 ` Juanma Barranquero
2008-05-16 20:41 ` David Robinow
1 sibling, 2 replies; 39+ messages in thread
From: Juanma Barranquero @ 2008-05-16 9:29 UTC (permalink / raw
To: Eli Zaretskii; +Cc: lennart.borgman, emacs-devel
On Fri, May 16, 2008 at 11:25 AM, Eli Zaretskii <eliz@gnu.org> wrote:
> You need to use emacs.exe as the IMAGENAME, otherwise it will not
> work.
The wildcard I used in my example works:
C:\> tasklist /fi "imagename eq emacs*"
Image Name PID Session Name Session# Mem Usage
========================= ====== ================ ======== ============
emacs.exe 2448 Console 0 23.528 K
> (Btw, I think this command is present on XP Home Edition as well.)
Ah. I read somewhere that I didn't, but I haven't checked it. In any
case, it can be downloaded off several websites.
Juanma
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Memory leak
2008-05-16 0:25 ` Lennart Borgman (gmail)
2008-05-16 8:35 ` Eli Zaretskii
@ 2008-05-16 9:43 ` Eli Zaretskii
2008-05-16 12:50 ` Jason Rumney
1 sibling, 1 reply; 39+ messages in thread
From: Eli Zaretskii @ 2008-05-16 9:43 UTC (permalink / raw
To: Lennart Borgman (gmail); +Cc: monnier, emacs-devel
> Date: Fri, 16 May 2008 02:25:50 +0200
> From: "Lennart Borgman (gmail)" <lennart.borgman@gmail.com>
> Cc: Stefan Monnier <monnier@IRO.UMontreal.CA>, emacs-devel@gnu.org
>
> I can see a memory leak on w32 of around 240 K in the above operation
> with a build from 2008-03-01
With today's CVS on w32, the memory grows by ~900KB with each new
frame I create, and that memory is not released when the frame and its
buffer are deleted.
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Memory leak
2008-05-16 9:29 ` Juanma Barranquero
@ 2008-05-16 9:46 ` Eli Zaretskii
2008-05-16 9:49 ` Juanma Barranquero
1 sibling, 0 replies; 39+ messages in thread
From: Eli Zaretskii @ 2008-05-16 9:46 UTC (permalink / raw
To: Juanma Barranquero; +Cc: lennart.borgman, emacs-devel
> Date: Fri, 16 May 2008 11:29:45 +0200
> From: "Juanma Barranquero" <lekktu@gmail.com>
> Cc: lennart.borgman@gmail.com, emacs-devel@gnu.org
>
> On Fri, May 16, 2008 at 11:25 AM, Eli Zaretskii <eliz@gnu.org> wrote:
>
> > You need to use emacs.exe as the IMAGENAME, otherwise it will not
> > work.
>
> The wildcard I used in my example works:
>
> C:\> tasklist /fi "imagename eq emacs*"
Sorry, I missed that little star.
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Memory leak
2008-05-16 9:29 ` Juanma Barranquero
2008-05-16 9:46 ` Eli Zaretskii
@ 2008-05-16 9:49 ` Juanma Barranquero
2008-05-16 20:13 ` Stephen J. Turnbull
1 sibling, 1 reply; 39+ messages in thread
From: Juanma Barranquero @ 2008-05-16 9:49 UTC (permalink / raw
To: Eli Zaretskii; +Cc: lennart.borgman, emacs-devel
On Fri, May 16, 2008 at 11:29 AM, Juanma Barranquero <lekktu@gmail.com> wrote:
> Ah. I read somewhere that I didn't, but I haven't checked it. In any
Funny. s/I didn't/it didn't/, of course.
Juanma
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Memory leak
2008-05-16 9:43 ` Eli Zaretskii
@ 2008-05-16 12:50 ` Jason Rumney
2008-05-19 8:05 ` Kenichi Handa
2008-05-24 2:30 ` calculation of text extents [Re: Memory leak] Kenichi Handa
0 siblings, 2 replies; 39+ messages in thread
From: Jason Rumney @ 2008-05-16 12:50 UTC (permalink / raw
To: Eli Zaretskii; +Cc: Lennart Borgman (gmail), monnier, emacs-devel
Eli Zaretskii wrote:
> With today's CVS on w32, the memory grows by ~900KB with each new
> frame I create, and that memory is not released when the frame and its
> buffer are deleted.
>
If the leak is coming from the font cache, then I wouldn't expect it to
be released when the frame is deleted, as the font cache is shared
between all frames on w32 (and I think between all frames on the same
display on X). Stefan is seeing 10x the increase as you are on Windows,
which could be explained by xft loading fonts into Emacs's process
space, while on Windows the font itself stays in system owned memory,
and only Emacs's internal info that it associates with the font
contributes to the growth. That also points to fonts being responsible
(though circumstantial).
A while ago I noticed w32font_open being called multiple times for the
same font, so I suspect that we may be overlooking the font cache in
places and reopening fonts when we could be reusing an existing font.
There is another similar problem when we get text extents - each redraw
cycle can call w32font_text_extents a number of times for the same
glyph_string (from compute_glyph_string_overhangs, left_overwriting and
right_overwriting in quick succession in draw_glyphs, with nested
compute_overhangs_and_x within some of the if statements that cause
extra calls for preceding and following glyph_strings if there is
overhang, which there often is with antialising enabled). Perhaps this
operation is cheap with the X font backends, but on Windows it is
expensive (even with caching of metrics), and I suspect this is a major
cause of the slowdown reported by Windows users.
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Memory leak
2008-05-16 9:49 ` Juanma Barranquero
@ 2008-05-16 20:13 ` Stephen J. Turnbull
2008-05-16 21:15 ` [OT, silly] " Thomas Lord
0 siblings, 1 reply; 39+ messages in thread
From: Stephen J. Turnbull @ 2008-05-16 20:13 UTC (permalink / raw
To: Juanma Barranquero; +Cc: Eli Zaretskii, lennart.borgman, emacs-devel
Juanma Barranquero writes:
> On Fri, May 16, 2008 at 11:29 AM, Juanma Barranquero <lekktu@gmail.com> wrote:
>
> > Ah. I read somewhere that I didn't, but I haven't checked it. In any
>
> Funny. s/I didn't/it didn't/, of course.
I think that deserves "Typo of the Decade" award!
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Memory leak
2008-05-16 9:25 ` Eli Zaretskii
2008-05-16 9:29 ` Juanma Barranquero
@ 2008-05-16 20:41 ` David Robinow
2008-05-16 21:36 ` Lennart Borgman (gmail)
2008-05-17 7:10 ` Eli Zaretskii
1 sibling, 2 replies; 39+ messages in thread
From: David Robinow @ 2008-05-16 20:41 UTC (permalink / raw
To: Eli Zaretskii; +Cc: Juanma Barranquero, lennart.borgman, emacs-devel
On Fri, May 16, 2008 at 5:25 AM, Eli Zaretskii <eliz@gnu.org> wrote:
>> Date: Fri, 16 May 2008 11:03:30 +0200
>> From: "Juanma Barranquero" <lekktu@gmail.com>
>> Cc: "Lennart Borgman (gmail)" <lennart.borgman@gmail.com>, emacs-devel@gnu.org
>>
>> tasklist /fi "imagename eq emacs*"
> You need to use emacs.exe as the IMAGENAME, otherwise it will not
> work.
> (Btw, I think this command is present on XP Home Edition as well.)
Apparently not. I have one of each. Works on XP Pro. Not on XP Home.
But thanks, Juanma. I didn't know about that command.
^ permalink raw reply [flat|nested] 39+ messages in thread
* [OT, silly] Re: Memory leak
2008-05-16 20:13 ` Stephen J. Turnbull
@ 2008-05-16 21:15 ` Thomas Lord
0 siblings, 0 replies; 39+ messages in thread
From: Thomas Lord @ 2008-05-16 21:15 UTC (permalink / raw
To: Stephen J. Turnbull
Cc: Juanma Barranquero, Eli Zaretskii, lennart.borgman, emacs-devel
Stephen J. Turnbull wrote:
> Juanma Barranquero writes:
> > On Fri, May 16, 2008 at 11:29 AM, Juanma Barranquero <lekktu@gmail.com> wrote:
> >
> > > Ah. I read somewhere that I didn't, but I haven't checked it. In any
> >
> > Funny. s/I didn't/it didn't/, of course.
>
> I think that deserves "Typo of the Decade" award!
>
>
>
>
Typo? Nah, it's the animist "I". :-)
-t
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Memory leak
2008-05-16 20:41 ` David Robinow
@ 2008-05-16 21:36 ` Lennart Borgman (gmail)
2008-05-17 7:10 ` Eli Zaretskii
1 sibling, 0 replies; 39+ messages in thread
From: Lennart Borgman (gmail) @ 2008-05-16 21:36 UTC (permalink / raw
To: David Robinow; +Cc: Juanma Barranquero, Eli Zaretskii, emacs-devel
David Robinow wrote:
> On Fri, May 16, 2008 at 5:25 AM, Eli Zaretskii <eliz@gnu.org> wrote:
>>> Date: Fri, 16 May 2008 11:03:30 +0200
>>> From: "Juanma Barranquero" <lekktu@gmail.com>
>>> Cc: "Lennart Borgman (gmail)" <lennart.borgman@gmail.com>, emacs-devel@gnu.org
>>>
>>> tasklist /fi "imagename eq emacs*"
>> You need to use emacs.exe as the IMAGENAME, otherwise it will not
>> work.
>> (Btw, I think this command is present on XP Home Edition as well.)
>
> Apparently not. I have one of each. Works on XP Pro. Not on XP Home.
> But thanks, Juanma. I didn't know about that command.
Thanks Juanma and Eli.
Though while doing things like this I find it more convenient to just do
tasklist | findstr emacs >> memory-leak.tmp
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Memory leak
2008-05-16 20:41 ` David Robinow
2008-05-16 21:36 ` Lennart Borgman (gmail)
@ 2008-05-17 7:10 ` Eli Zaretskii
1 sibling, 0 replies; 39+ messages in thread
From: Eli Zaretskii @ 2008-05-17 7:10 UTC (permalink / raw
To: David Robinow; +Cc: lekktu, lennart.borgman, emacs-devel
> Date: Fri, 16 May 2008 16:41:39 -0400
> From: "David Robinow" <drobinow@gmail.com>
> Cc: "Juanma Barranquero" <lekktu@gmail.com>, lennart.borgman@gmail.com,
> emacs-devel@gnu.org
>
> > (Btw, I think this command is present on XP Home Edition as well.)
>
> Apparently not. I have one of each. Works on XP Pro. Not on XP Home.
It does on mine, but maybe it came with one of the optional toolkits I
downloaded from the MS site.
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Memory leak
2008-05-15 22:50 Memory leak Stefan Monnier
2008-05-15 23:06 ` David Kastrup
2008-05-16 0:38 ` YAMAMOTO Mitsuharu
@ 2008-05-17 15:16 ` Evil Boris
2008-05-17 19:11 ` Stephen J. Turnbull
2008-05-23 13:54 ` Evil Boris
2 siblings, 2 replies; 39+ messages in thread
From: Evil Boris @ 2008-05-17 15:16 UTC (permalink / raw
To: emacs-devel
Stefan Monnier <monnier@IRO.UMontreal.CA> writes:
> We have a pretty serious memory leak.
> My Emacs session recently died with a memory full situation (the
> process was around 500MB VSZ).
I do not know if this is related, but sometime after January (around
Unicode merge) I noticed that, after running for a while (have not
been able to notice when precisely this happens, but seems to require
having at least one X frame, sometimes happens within a few minutes of
starting), Emacs starts occupying 190-250Mb of memory. Since I
usually edit smallish files (<50-100K) and not too many of them (<10,
generally), the only candidate for "eating space" is my RMAIL file,
which is around 30Mb.
Any ideas? Any obvious place to check what memory is being used
(ancient Solaris 2.7, GCC 3.1)?
Tried on many different CVS versions, including the most current one.
--Boris
PS. Was about to contradict myself---the currently running copy is
179M and has no X frame, but I now recall creating an X frame
yesterday, and then destroying it.
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Memory leak
2008-05-17 15:16 ` Evil Boris
@ 2008-05-17 19:11 ` Stephen J. Turnbull
2008-05-17 19:17 ` Eric Hanchrow
2008-05-23 13:54 ` Evil Boris
1 sibling, 1 reply; 39+ messages in thread
From: Stephen J. Turnbull @ 2008-05-17 19:11 UTC (permalink / raw
To: Evil Boris; +Cc: emacs-devel
Evil Boris writes:
> I do not know if this is related, but sometime after January (around
> Unicode merge) I noticed that, after running for a while (have not
> been able to notice when precisely this happens, but seems to require
> having at least one X frame, sometimes happens within a few minutes of
> starting), Emacs starts occupying 190-250Mb of memory.
FWIW, XEmacs has some hard-to-pin-down memory leaks, too, one or more
of which seems to be related to Xft/fontconfig.
These packages are coded in the classic X11 style, meaning that the
burden is entirely on the client code to clean things up. There *is*
refcounting and things like that, but for internal use of the Xft
code.
I have no idea what the memory use characteristics of underlying
rendering engines such as FreeType might be, so this has been really
hard to track down.
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Memory leak
2008-05-17 19:11 ` Stephen J. Turnbull
@ 2008-05-17 19:17 ` Eric Hanchrow
2008-05-18 11:26 ` Jan Djärv
0 siblings, 1 reply; 39+ messages in thread
From: Eric Hanchrow @ 2008-05-17 19:17 UTC (permalink / raw
To: emacs-devel
Well, here's a dumb idea: has anyone tried running Emacs under valgrind?
I dimly recall that tool being remarkably useful for finding memory
leaks.
--
Governing the U.S. is like playing 200 simultaneous chess
matches (while whiny columnists second-guess every move on every
board).
-- Nicholas Kristof, New York Times columnist
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Memory leak
2008-05-17 19:17 ` Eric Hanchrow
@ 2008-05-18 11:26 ` Jan Djärv
2008-05-18 11:29 ` David Kastrup
` (2 more replies)
0 siblings, 3 replies; 39+ messages in thread
From: Jan Djärv @ 2008-05-18 11:26 UTC (permalink / raw
To: Eric Hanchrow; +Cc: emacs-devel
Eric Hanchrow skrev:
> Well, here's a dumb idea: has anyone tried running Emacs under valgrind?
> I dimly recall that tool being remarkably useful for finding memory
> leaks.
Valgrind just craches when run on Emacs. Emacs memory handling is too much
for valgrind. It is the undump think that it doesn't understand.
Jan D.
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Memory leak
2008-05-18 11:26 ` Jan Djärv
@ 2008-05-18 11:29 ` David Kastrup
2008-05-20 6:46 ` Jan Djärv
2008-05-18 13:34 ` Werner LEMBERG
2008-05-19 2:44 ` Stephen J. Turnbull
2 siblings, 1 reply; 39+ messages in thread
From: David Kastrup @ 2008-05-18 11:29 UTC (permalink / raw
To: Jan Djärv; +Cc: Eric Hanchrow, emacs-devel
Jan Djärv <jan.h.d@swipnet.se> writes:
> Eric Hanchrow skrev:
>> Well, here's a dumb idea: has anyone tried running Emacs under valgrind?
>> I dimly recall that tool being remarkably useful for finding memory
>> leaks.
>
> Valgrind just craches when run on Emacs. Emacs memory handling is too
> much for valgrind. It is the undump think that it doesn't understand.
What about running valgrind on temacs then?
--
David Kastrup, Kriemhildstr. 15, 44793 Bochum
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Memory leak
2008-05-18 11:26 ` Jan Djärv
2008-05-18 11:29 ` David Kastrup
@ 2008-05-18 13:34 ` Werner LEMBERG
2008-05-20 6:48 ` Jan Djärv
2008-05-19 2:44 ` Stephen J. Turnbull
2 siblings, 1 reply; 39+ messages in thread
From: Werner LEMBERG @ 2008-05-18 13:34 UTC (permalink / raw
To: jan.h.d; +Cc: offby1, emacs-devel
> Valgrind just craches when run on Emacs. Emacs memory handling is too
> much for valgrind. It is the undump think that it doesn't understand.
What about contacting the valgrind guys directly for help? I think
that Emacs would get *huge* benefits if checked with it.
Werner
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Memory leak
2008-05-18 11:26 ` Jan Djärv
2008-05-18 11:29 ` David Kastrup
2008-05-18 13:34 ` Werner LEMBERG
@ 2008-05-19 2:44 ` Stephen J. Turnbull
2008-05-20 6:50 ` Jan Djärv
2 siblings, 1 reply; 39+ messages in thread
From: Stephen J. Turnbull @ 2008-05-19 2:44 UTC (permalink / raw
To: Jan Djärv; +Cc: Eric Hanchrow, emacs-devel
Jan Djärv writes:
> Valgrind just craches when run on Emacs.
Portable dumper wins here: XEmacs at least and maybe SXEmacs valgrind
perfectly well.
You should also be able to valgrind temacs.
For more info, you could post to xemacs-beta@xemacs.org (nonmember
posts moderated) and the relevant sxemacs.org list (www.sxemacs.org
for references, dunno about mod policy).
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Memory leak
2008-05-16 12:50 ` Jason Rumney
@ 2008-05-19 8:05 ` Kenichi Handa
2008-05-19 8:45 ` YAMAMOTO Mitsuharu
2008-05-19 13:34 ` David Hansen
2008-05-24 2:30 ` calculation of text extents [Re: Memory leak] Kenichi Handa
1 sibling, 2 replies; 39+ messages in thread
From: Kenichi Handa @ 2008-05-19 8:05 UTC (permalink / raw
To: Jason Rumney; +Cc: eliz, lennart.borgman, monnier, emacs-devel
In article <482D8332.5030908@gnu.org>, Jason Rumney <jasonr@gnu.org> writes:
> If the leak is coming from the font cache, then I wouldn't expect it to
> be released when the frame is deleted, as the font cache is shared
> between all frames on w32 (and I think between all frames on the same
> display on X). Stefan is seeing 10x the increase as you are on Windows,
> which could be explained by xft loading fonts into Emacs's process
> space, while on Windows the font itself stays in system owned memory,
> and only Emacs's internal info that it associates with the font
> contributes to the growth. That also points to fonts being responsible
> (though circumstantial).
I've just installed a fix in font-cache handling. Now, even
when I create many new frames, the memory usage doesn't
increase that much. Though it still increases several
handreds K-bytes each time a new frame is created, and it is
not reduced when I delete the frames. So, I think there
still be a problem somewhere.
FYI, when I compile Emacs without gtk, the amount of
(non-freed) memory increase becomes smaller.
> There is another similar problem when we get text extents - each redraw
> cycle can call w32font_text_extents a number of times for the same
> glyph_string (from compute_glyph_string_overhangs, left_overwriting and
> right_overwriting in quick succession in draw_glyphs, with nested
> compute_overhangs_and_x within some of the if statements that cause
> extra calls for preceding and following glyph_strings if there is
> overhang, which there often is with antialising enabled). Perhaps this
> operation is cheap with the X font backends, but on Windows it is
> expensive (even with caching of metrics), and I suspect this is a major
> cause of the slowdown reported by Windows users.
I remember that I modified x_draw_glyph_string for such a
font-backend as xft that can't do foreground-only drawing
because of antialiasing, which increases the count of
overhang-checking. Perhaps we should study the code of
draw_glyphs and x_draw_glyph_string and find a solution.
---
Kenichi Handa
handa@ni.aist.go.jp
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Memory leak
2008-05-19 8:05 ` Kenichi Handa
@ 2008-05-19 8:45 ` YAMAMOTO Mitsuharu
2008-05-19 11:13 ` Kenichi Handa
2008-05-19 13:34 ` David Hansen
1 sibling, 1 reply; 39+ messages in thread
From: YAMAMOTO Mitsuharu @ 2008-05-19 8:45 UTC (permalink / raw
To: Kenichi Handa; +Cc: eliz, emacs-devel, lennart.borgman, monnier, Jason Rumney
>>>>> On Mon, 19 May 2008 17:05:42 +0900, Kenichi Handa <handa@m17n.org> said:
> I've just installed a fix in font-cache handling.
It seems like the `free_entity' entry in `struct font_driver' no
longer makes sense because you tried to remove the memory leaks by
avoiding Lisp_Save_Value in font entities instead of calling
free_entity at some proper timings. Is that right?
YAMAMOTO Mitsuharu
mituharu@math.s.chiba-u.ac.jp
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Memory leak
2008-05-19 8:45 ` YAMAMOTO Mitsuharu
@ 2008-05-19 11:13 ` Kenichi Handa
2008-05-19 11:59 ` YAMAMOTO Mitsuharu
0 siblings, 1 reply; 39+ messages in thread
From: Kenichi Handa @ 2008-05-19 11:13 UTC (permalink / raw
To: YAMAMOTO Mitsuharu; +Cc: eliz, emacs-devel, lennart.borgman, monnier, jasonr
In article <wlzlqmu1gw.wl%mituharu@math.s.chiba-u.ac.jp>, YAMAMOTO Mitsuharu <mituharu@math.s.chiba-u.ac.jp> writes:
>>>>>> On Mon, 19 May 2008 17:05:42 +0900, Kenichi Handa <handa@m17n.org> said:
> > I've just installed a fix in font-cache handling.
> It seems like the `free_entity' entry in `struct font_driver' no
> longer makes sense because you tried to remove the memory leaks by
> avoiding Lisp_Save_Value in font entities instead of calling
> free_entity at some proper timings. Is that right?
Not exactly. When I deleted ftfont_free_entity, it was in
my mind to change also ftfont_list to avoid using
Lisp_Save_Value, but I just forgot to implement that change.
The free_entity callback is still called in font_clear_cache
if a font-driver provides that function.
---
Kenichi Handa
handa@ni.aist.go.jp
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Memory leak
2008-05-19 11:13 ` Kenichi Handa
@ 2008-05-19 11:59 ` YAMAMOTO Mitsuharu
2008-05-23 2:24 ` Kenichi Handa
0 siblings, 1 reply; 39+ messages in thread
From: YAMAMOTO Mitsuharu @ 2008-05-19 11:59 UTC (permalink / raw
To: Kenichi Handa; +Cc: eliz, emacs-devel, lennart.borgman, monnier, jasonr
>>>>> On Mon, 19 May 2008 20:13:15 +0900, Kenichi Handa <handa@m17n.org> said:
>> It seems like the `free_entity' entry in `struct font_driver' no
>> longer makes sense because you tried to remove the memory leaks by
>> avoiding Lisp_Save_Value in font entities instead of calling
>> free_entity at some proper timings. Is that right?
> Not exactly. When I deleted ftfont_free_entity, it was in my mind
> to change also ftfont_list to avoid using Lisp_Save_Value, but I
> just forgot to implement that change.
> The free_entity callback is still called in font_clear_cache if a
> font-driver provides that function.
So, `free_entity' is supposed to be still valid. Then I think
the entities deleted in `font_delete_unmatched' need to be freed
as they are not kept in the cache.
Also, there are some suspected leaks in font-related code:
* `names' in xfont_list_pattern.
* `charset' in ftfont_spec_pattern when it is created from
Vscript_representative_chars.
YAMAMOTO Mitsuharu
mituharu@math.s.chiba-u.ac.jp
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Memory leak
2008-05-19 8:05 ` Kenichi Handa
2008-05-19 8:45 ` YAMAMOTO Mitsuharu
@ 2008-05-19 13:34 ` David Hansen
2008-05-19 19:42 ` David Hansen
1 sibling, 1 reply; 39+ messages in thread
From: David Hansen @ 2008-05-19 13:34 UTC (permalink / raw
To: emacs-devel
On Mon, 19 May 2008 17:05:42 +0900 Kenichi Handa wrote:
> I've just installed a fix in font-cache handling. Now, even
> when I create many new frames, the memory usage doesn't
> increase that much. Though it still increases several
> handreds K-bytes each time a new frame is created, and it is
> not reduced when I delete the frames. So, I think there
> still be a problem somewhere.
I used to have two frames open in my emacs session, never create or
destroy any other frames. Memory usage climbs to > 500M here after some
hours of usage. I doubt it has much to do with frame creation.
Will update right now and report back if your changes help.
David
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Memory leak
2008-05-19 13:34 ` David Hansen
@ 2008-05-19 19:42 ` David Hansen
0 siblings, 0 replies; 39+ messages in thread
From: David Hansen @ 2008-05-19 19:42 UTC (permalink / raw
To: emacs-devel
On Mon, 19 May 2008 15:34:20 +0200 David Hansen wrote:
> Will update right now and report back if your changes help.
At least not a that excessive one. Usage is now at about 70M which is
pretty normal for my emacs usage.
But: - Italics isn't working anymore
- The Gnus *summary* buffer flickers …
- … as does looking at some parts of the UTF-8-demo.txt[1] (seems
to correlate with the display of line drawing characters) …
- … which is also incredible slooooow (scrolling and cursor
movements on the timescale of several seconds on my 1.8GHz AMD
box)
- Combining characters is broken (at least the example in above
demo file).
David
Footnotes:
[1] http://www.cl.cam.ac.uk/~mgk25/ucs/examples/UTF-8-demo.txt
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Memory leak
2008-05-18 11:29 ` David Kastrup
@ 2008-05-20 6:46 ` Jan Djärv
0 siblings, 0 replies; 39+ messages in thread
From: Jan Djärv @ 2008-05-20 6:46 UTC (permalink / raw
To: David Kastrup; +Cc: Eric Hanchrow, emacs-devel
David Kastrup skrev:
> Jan Djärv <jan.h.d@swipnet.se> writes:
>
>> Eric Hanchrow skrev:
>>> Well, here's a dumb idea: has anyone tried running Emacs under valgrind?
>>> I dimly recall that tool being remarkably useful for finding memory
>>> leaks.
>> Valgrind just craches when run on Emacs. Emacs memory handling is too
>> much for valgrind. It is the undump think that it doesn't understand.
>
> What about running valgrind on temacs then?
>
That should work, I haven't tried it.
Jan D.
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Memory leak
2008-05-18 13:34 ` Werner LEMBERG
@ 2008-05-20 6:48 ` Jan Djärv
0 siblings, 0 replies; 39+ messages in thread
From: Jan Djärv @ 2008-05-20 6:48 UTC (permalink / raw
To: Werner LEMBERG; +Cc: offby1, emacs-devel
Werner LEMBERG skrev:
>> Valgrind just craches when run on Emacs. Emacs memory handling is too
>> much for valgrind. It is the undump think that it doesn't understand.
>
> What about contacting the valgrind guys directly for help? I think
> that Emacs would get *huge* benefits if checked with it.
>
>
If my memory is correct, someone already did that. I must search the mail
archives. I think the Emacs way of doing things was too special for the
valgrind guys to devote time to. I actually agree with that. From what I
recall, it was the saving and restoring of malloc state during dump and undump
that was the main problem. Valgrind has no way of correctly capture this
state and begin tracking from there.
Jan D.
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Memory leak
2008-05-19 2:44 ` Stephen J. Turnbull
@ 2008-05-20 6:50 ` Jan Djärv
2008-05-21 6:10 ` Stephen J. Turnbull
0 siblings, 1 reply; 39+ messages in thread
From: Jan Djärv @ 2008-05-20 6:50 UTC (permalink / raw
To: Stephen J. Turnbull; +Cc: Eric Hanchrow, emacs-devel
Stephen J. Turnbull skrev:
> Jan Djärv writes:
>
> > Valgrind just craches when run on Emacs.
>
> Portable dumper wins here: XEmacs at least and maybe SXEmacs valgrind
> perfectly well.
Yes. But it is not a small project.
>
> You should also be able to valgrind temacs.
>
That is a small project :-)
Jan D.
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Memory leak
2008-05-20 6:50 ` Jan Djärv
@ 2008-05-21 6:10 ` Stephen J. Turnbull
2008-05-21 17:32 ` Stefan Monnier
0 siblings, 1 reply; 39+ messages in thread
From: Stephen J. Turnbull @ 2008-05-21 6:10 UTC (permalink / raw
To: Jan Djärv; +Cc: Eric Hanchrow, emacs-devel
Jan Djärv writes:
>
>
> Stephen J. Turnbull skrev:
> > Jan Djärv writes:
> >
> > > Valgrind just craches when run on Emacs.
> >
> > Portable dumper wins here: XEmacs at least and maybe SXEmacs valgrind
> > perfectly well.
>
> Yes. But it is not a small project.
Um, IIRC it's a done project, though. Somebody's got a patch in the
queue, no?
I don't know if valgrind is a big enough win to push that into the
mainline now, though.
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Memory leak
2008-05-21 6:10 ` Stephen J. Turnbull
@ 2008-05-21 17:32 ` Stefan Monnier
0 siblings, 0 replies; 39+ messages in thread
From: Stefan Monnier @ 2008-05-21 17:32 UTC (permalink / raw
To: Stephen J. Turnbull; +Cc: Eric Hanchrow, Jan Djärv, emacs-devel
> Um, IIRC it's a done project, though. Somebody's got a patch in the
> queue, no?
I've heard rumors that someone has worked on it at some point, but
that's a far cry from a "done project", sadly. For one, I still haven't
seen any patch for it. And if there's such a patch, I wouldn't be
surprised if it bitrotted significantly.
Stefan
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Memory leak
2008-05-19 11:59 ` YAMAMOTO Mitsuharu
@ 2008-05-23 2:24 ` Kenichi Handa
0 siblings, 0 replies; 39+ messages in thread
From: Kenichi Handa @ 2008-05-23 2:24 UTC (permalink / raw
To: YAMAMOTO Mitsuharu; +Cc: eliz, emacs-devel, lennart.borgman, monnier, jasonr
In article <wl4p8ua4ip.wl%mituharu@math.s.chiba-u.ac.jp>, YAMAMOTO Mitsuharu <mituharu@math.s.chiba-u.ac.jp> writes:
> So, `free_entity' is supposed to be still valid. Then I think
> the entities deleted in `font_delete_unmatched' need to be freed
> as they are not kept in the cache.
> Also, there are some suspected leaks in font-related code:
> * `names' in xfont_list_pattern.
> * `charset' in ftfont_spec_pattern when it is created from
> Vscript_representative_chars.
Thank you for finding these bugs. I fixed them.
---
Kenichi Handa
handa@ni.aist.go.jp
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Memory leak
2008-05-17 15:16 ` Evil Boris
2008-05-17 19:11 ` Stephen J. Turnbull
@ 2008-05-23 13:54 ` Evil Boris
1 sibling, 0 replies; 39+ messages in thread
From: Evil Boris @ 2008-05-23 13:54 UTC (permalink / raw
To: emacs-devel
"Evil Boris" <evilborisnet@netscape.net> writes:
> I do not know if this is related, but sometime after January (around
> Unicode merge) I noticed that, after running for a while (have not
> been able to notice when precisely this happens, but seems to require
> having at least one X frame, sometimes happens within a few minutes of
> starting), Emacs starts occupying 190-250Mb of memory. Since I
> usually edit smallish files (<50-100K) and not too many of them (<10,
> generally), the only candidate for "eating space" is my RMAIL file,
> which is around 30Mb.
I just checked, with a recent version from CVS [GNU Emacs 23.0.60.9
(sparc-sun-solaris2.7, X toolkit) of 2008-05-21 on ziggy], with no X
frames ever opened, playing almost exclusively with a few small files,
with the sole exception of my 30Mb mailbox, top shows:
PID USERNAME THR PRI NICE SIZE RES STATE TIME CPU COMMAND
1754 ... 1 58 0 174M 172M sleep 4:59 0.00% emacs
174Mb again? Any ideas? Any code I can run to try to see what the
memory is being being used for?
--Boris
^ permalink raw reply [flat|nested] 39+ messages in thread
* calculation of text extents [Re: Memory leak]
2008-05-16 12:50 ` Jason Rumney
2008-05-19 8:05 ` Kenichi Handa
@ 2008-05-24 2:30 ` Kenichi Handa
1 sibling, 0 replies; 39+ messages in thread
From: Kenichi Handa @ 2008-05-24 2:30 UTC (permalink / raw
To: Jason Rumney; +Cc: eliz, lennart.borgman, monnier, emacs-devel
In article <482D8332.5030908@gnu.org>, Jason Rumney <jasonr@gnu.org> writes:
> A while ago I noticed w32font_open being called multiple times for the
> same font, so I suspect that we may be overlooking the font cache in
> places and reopening fonts when we could be reusing an existing font.
With the recent fix of font-cache handling, I think the
number of calling `open' callback of a font-backend for the
same font is reduced. Can you confirm that?
> There is another similar problem when we get text extents - each redraw
> cycle can call w32font_text_extents a number of times for the same
> glyph_string (from compute_glyph_string_overhangs, left_overwriting and
> right_overwriting in quick succession in draw_glyphs, with nested
> compute_overhangs_and_x within some of the if statements that cause
> extra calls for preceding and following glyph_strings if there is
> overhang, which there often is with antialising enabled). Perhaps this
> operation is cheap with the X font backends, but on Windows it is
> expensive (even with caching of metrics), and I suspect this is a major
> cause of the slowdown reported by Windows users.
It seems that we can reduce the number of calling
`text_extents' callback by recording lbearing and rbearing
in struct glyph; which grows the size of that structure from
the current 32-byte to 36-byte. I'm not sure how much that
change impacts the memory usage of Emacs, but isn't it worth
trying?
---
Kenichi Handa
handa@ni.aist.go.jp
^ permalink raw reply [flat|nested] 39+ messages in thread
end of thread, other threads:[~2008-05-24 2:30 UTC | newest]
Thread overview: 39+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2008-05-15 22:50 Memory leak Stefan Monnier
2008-05-15 23:06 ` David Kastrup
2008-05-16 0:25 ` Lennart Borgman (gmail)
2008-05-16 8:35 ` Eli Zaretskii
2008-05-16 9:03 ` Juanma Barranquero
2008-05-16 9:25 ` Eli Zaretskii
2008-05-16 9:29 ` Juanma Barranquero
2008-05-16 9:46 ` Eli Zaretskii
2008-05-16 9:49 ` Juanma Barranquero
2008-05-16 20:13 ` Stephen J. Turnbull
2008-05-16 21:15 ` [OT, silly] " Thomas Lord
2008-05-16 20:41 ` David Robinow
2008-05-16 21:36 ` Lennart Borgman (gmail)
2008-05-17 7:10 ` Eli Zaretskii
2008-05-16 9:43 ` Eli Zaretskii
2008-05-16 12:50 ` Jason Rumney
2008-05-19 8:05 ` Kenichi Handa
2008-05-19 8:45 ` YAMAMOTO Mitsuharu
2008-05-19 11:13 ` Kenichi Handa
2008-05-19 11:59 ` YAMAMOTO Mitsuharu
2008-05-23 2:24 ` Kenichi Handa
2008-05-19 13:34 ` David Hansen
2008-05-19 19:42 ` David Hansen
2008-05-24 2:30 ` calculation of text extents [Re: Memory leak] Kenichi Handa
2008-05-16 2:19 ` Memory leak Stefan Monnier
2008-05-16 0:38 ` YAMAMOTO Mitsuharu
2008-05-17 15:16 ` Evil Boris
2008-05-17 19:11 ` Stephen J. Turnbull
2008-05-17 19:17 ` Eric Hanchrow
2008-05-18 11:26 ` Jan Djärv
2008-05-18 11:29 ` David Kastrup
2008-05-20 6:46 ` Jan Djärv
2008-05-18 13:34 ` Werner LEMBERG
2008-05-20 6:48 ` Jan Djärv
2008-05-19 2:44 ` Stephen J. Turnbull
2008-05-20 6:50 ` Jan Djärv
2008-05-21 6:10 ` Stephen J. Turnbull
2008-05-21 17:32 ` Stefan Monnier
2008-05-23 13:54 ` Evil Boris
Code repositories for project(s) associated with this external index
https://git.savannah.gnu.org/cgit/emacs.git
https://git.savannah.gnu.org/cgit/emacs/org-mode.git
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.