* Memory leak @ 2008-05-15 22:50 Stefan Monnier 2008-05-15 23:06 ` David Kastrup ` (2 more replies) 0 siblings, 3 replies; 39+ messages in thread From: Stefan Monnier @ 2008-05-15 22:50 UTC (permalink / raw To: emacs-devel We have a pretty serious memory leak. My Emacs session recently died with a memory full situation (the process was around 500MB VSZ). So I took a look around and it seem that every time I create a new frame, my Emacs session grows by around 10MB and deleting the frame does not recover this memory. This memory is not accounted for by the counts returned by `garbage-collect'. I'm not sure when this problem got introduced, tho the new font code is a likely culprit. Stefan ^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Memory leak 2008-05-15 22:50 Memory leak Stefan Monnier @ 2008-05-15 23:06 ` David Kastrup 2008-05-16 0:25 ` Lennart Borgman (gmail) 2008-05-16 2:19 ` Memory leak Stefan Monnier 2008-05-16 0:38 ` YAMAMOTO Mitsuharu 2008-05-17 15:16 ` Evil Boris 2 siblings, 2 replies; 39+ messages in thread From: David Kastrup @ 2008-05-15 23:06 UTC (permalink / raw To: Stefan Monnier; +Cc: emacs-devel Stefan Monnier <monnier@IRO.UMontreal.CA> writes: > We have a pretty serious memory leak. > My Emacs session recently died with a memory full situation (the > process was around 500MB VSZ). So I took a look around and it seem that > every time I create a new frame, my Emacs session grows by around 10MB > and deleting the frame does not recover this memory. > This memory is not accounted for by the counts returned by > `garbage-collect'. > > I'm not sure when this problem got introduced, tho the new font code is > a likely culprit. Just to rule out something obvious (pure space overflow): M-: (garbage-collect) RET does not return nil, does it? -- David Kastrup, Kriemhildstr. 15, 44793 Bochum ^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Memory leak 2008-05-15 23:06 ` David Kastrup @ 2008-05-16 0:25 ` Lennart Borgman (gmail) 2008-05-16 8:35 ` Eli Zaretskii 2008-05-16 9:43 ` Eli Zaretskii 2008-05-16 2:19 ` Memory leak Stefan Monnier 1 sibling, 2 replies; 39+ messages in thread From: Lennart Borgman (gmail) @ 2008-05-16 0:25 UTC (permalink / raw To: David Kastrup; +Cc: Stefan Monnier, emacs-devel David Kastrup wrote: > Stefan Monnier <monnier@IRO.UMontreal.CA> writes: > >> We have a pretty serious memory leak. >> My Emacs session recently died with a memory full situation (the >> process was around 500MB VSZ). So I took a look around and it seem that >> every time I create a new frame, my Emacs session grows by around 10MB >> and deleting the frame does not recover this memory. >> This memory is not accounted for by the counts returned by >> `garbage-collect'. >> >> I'm not sure when this problem got introduced, tho the new font code is >> a likely culprit. > > Just to rule out something obvious (pure space overflow): > > M-: (garbage-collect) RET > > does not return nil, does it? I can see a memory leak on w32 of around 240 K in the above operation with a build from 2008-03-01, but not with a build from 2008-01-21 (no build in between starts ok). (garbage-collect) does not return nil with any of the builds above. PS: Is there any command line utility that shows memory usage on w32? ^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Memory leak 2008-05-16 0:25 ` Lennart Borgman (gmail) @ 2008-05-16 8:35 ` Eli Zaretskii 2008-05-16 9:03 ` Juanma Barranquero 2008-05-16 9:43 ` Eli Zaretskii 1 sibling, 1 reply; 39+ messages in thread From: Eli Zaretskii @ 2008-05-16 8:35 UTC (permalink / raw To: Lennart Borgman (gmail); +Cc: emacs-devel > Date: Fri, 16 May 2008 02:25:50 +0200 > From: "Lennart Borgman (gmail)" <lennart.borgman@gmail.com> > Cc: Stefan Monnier <monnier@IRO.UMontreal.CA>, emacs-devel@gnu.org > > PS: Is there any command line utility that shows memory usage on w32? Download PsTools, and then say "pslist -m emacs". ^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Memory leak 2008-05-16 8:35 ` Eli Zaretskii @ 2008-05-16 9:03 ` Juanma Barranquero 2008-05-16 9:25 ` Eli Zaretskii 0 siblings, 1 reply; 39+ messages in thread From: Juanma Barranquero @ 2008-05-16 9:03 UTC (permalink / raw To: Eli Zaretskii; +Cc: Lennart Borgman (gmail), emacs-devel On Fri, May 16, 2008 at 10:35 AM, Eli Zaretskii <eliz@gnu.org> wrote: > Download PsTools, and then say "pslist -m emacs". Or, if you're using Windows XP Pro or a Server edition, you can simply do tasklist /fi "imagename eq emacs*" Juanma ^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Memory leak 2008-05-16 9:03 ` Juanma Barranquero @ 2008-05-16 9:25 ` Eli Zaretskii 2008-05-16 9:29 ` Juanma Barranquero 2008-05-16 20:41 ` David Robinow 0 siblings, 2 replies; 39+ messages in thread From: Eli Zaretskii @ 2008-05-16 9:25 UTC (permalink / raw To: Juanma Barranquero; +Cc: lennart.borgman, emacs-devel > Date: Fri, 16 May 2008 11:03:30 +0200 > From: "Juanma Barranquero" <lekktu@gmail.com> > Cc: "Lennart Borgman (gmail)" <lennart.borgman@gmail.com>, emacs-devel@gnu.org > > tasklist /fi "imagename eq emacs*" You need to use emacs.exe as the IMAGENAME, otherwise it will not work. (Btw, I think this command is present on XP Home Edition as well.) ^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Memory leak 2008-05-16 9:25 ` Eli Zaretskii @ 2008-05-16 9:29 ` Juanma Barranquero 2008-05-16 9:46 ` Eli Zaretskii 2008-05-16 9:49 ` Juanma Barranquero 2008-05-16 20:41 ` David Robinow 1 sibling, 2 replies; 39+ messages in thread From: Juanma Barranquero @ 2008-05-16 9:29 UTC (permalink / raw To: Eli Zaretskii; +Cc: lennart.borgman, emacs-devel On Fri, May 16, 2008 at 11:25 AM, Eli Zaretskii <eliz@gnu.org> wrote: > You need to use emacs.exe as the IMAGENAME, otherwise it will not > work. The wildcard I used in my example works: C:\> tasklist /fi "imagename eq emacs*" Image Name PID Session Name Session# Mem Usage ========================= ====== ================ ======== ============ emacs.exe 2448 Console 0 23.528 K > (Btw, I think this command is present on XP Home Edition as well.) Ah. I read somewhere that I didn't, but I haven't checked it. In any case, it can be downloaded off several websites. Juanma ^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Memory leak 2008-05-16 9:29 ` Juanma Barranquero @ 2008-05-16 9:46 ` Eli Zaretskii 2008-05-16 9:49 ` Juanma Barranquero 1 sibling, 0 replies; 39+ messages in thread From: Eli Zaretskii @ 2008-05-16 9:46 UTC (permalink / raw To: Juanma Barranquero; +Cc: lennart.borgman, emacs-devel > Date: Fri, 16 May 2008 11:29:45 +0200 > From: "Juanma Barranquero" <lekktu@gmail.com> > Cc: lennart.borgman@gmail.com, emacs-devel@gnu.org > > On Fri, May 16, 2008 at 11:25 AM, Eli Zaretskii <eliz@gnu.org> wrote: > > > You need to use emacs.exe as the IMAGENAME, otherwise it will not > > work. > > The wildcard I used in my example works: > > C:\> tasklist /fi "imagename eq emacs*" Sorry, I missed that little star. ^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Memory leak 2008-05-16 9:29 ` Juanma Barranquero 2008-05-16 9:46 ` Eli Zaretskii @ 2008-05-16 9:49 ` Juanma Barranquero 2008-05-16 20:13 ` Stephen J. Turnbull 1 sibling, 1 reply; 39+ messages in thread From: Juanma Barranquero @ 2008-05-16 9:49 UTC (permalink / raw To: Eli Zaretskii; +Cc: lennart.borgman, emacs-devel On Fri, May 16, 2008 at 11:29 AM, Juanma Barranquero <lekktu@gmail.com> wrote: > Ah. I read somewhere that I didn't, but I haven't checked it. In any Funny. s/I didn't/it didn't/, of course. Juanma ^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Memory leak 2008-05-16 9:49 ` Juanma Barranquero @ 2008-05-16 20:13 ` Stephen J. Turnbull 2008-05-16 21:15 ` [OT, silly] " Thomas Lord 0 siblings, 1 reply; 39+ messages in thread From: Stephen J. Turnbull @ 2008-05-16 20:13 UTC (permalink / raw To: Juanma Barranquero; +Cc: Eli Zaretskii, lennart.borgman, emacs-devel Juanma Barranquero writes: > On Fri, May 16, 2008 at 11:29 AM, Juanma Barranquero <lekktu@gmail.com> wrote: > > > Ah. I read somewhere that I didn't, but I haven't checked it. In any > > Funny. s/I didn't/it didn't/, of course. I think that deserves "Typo of the Decade" award! ^ permalink raw reply [flat|nested] 39+ messages in thread
* [OT, silly] Re: Memory leak 2008-05-16 20:13 ` Stephen J. Turnbull @ 2008-05-16 21:15 ` Thomas Lord 0 siblings, 0 replies; 39+ messages in thread From: Thomas Lord @ 2008-05-16 21:15 UTC (permalink / raw To: Stephen J. Turnbull Cc: Juanma Barranquero, Eli Zaretskii, lennart.borgman, emacs-devel Stephen J. Turnbull wrote: > Juanma Barranquero writes: > > On Fri, May 16, 2008 at 11:29 AM, Juanma Barranquero <lekktu@gmail.com> wrote: > > > > > Ah. I read somewhere that I didn't, but I haven't checked it. In any > > > > Funny. s/I didn't/it didn't/, of course. > > I think that deserves "Typo of the Decade" award! > > > > Typo? Nah, it's the animist "I". :-) -t ^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Memory leak 2008-05-16 9:25 ` Eli Zaretskii 2008-05-16 9:29 ` Juanma Barranquero @ 2008-05-16 20:41 ` David Robinow 2008-05-16 21:36 ` Lennart Borgman (gmail) 2008-05-17 7:10 ` Eli Zaretskii 1 sibling, 2 replies; 39+ messages in thread From: David Robinow @ 2008-05-16 20:41 UTC (permalink / raw To: Eli Zaretskii; +Cc: Juanma Barranquero, lennart.borgman, emacs-devel On Fri, May 16, 2008 at 5:25 AM, Eli Zaretskii <eliz@gnu.org> wrote: >> Date: Fri, 16 May 2008 11:03:30 +0200 >> From: "Juanma Barranquero" <lekktu@gmail.com> >> Cc: "Lennart Borgman (gmail)" <lennart.borgman@gmail.com>, emacs-devel@gnu.org >> >> tasklist /fi "imagename eq emacs*" > You need to use emacs.exe as the IMAGENAME, otherwise it will not > work. > (Btw, I think this command is present on XP Home Edition as well.) Apparently not. I have one of each. Works on XP Pro. Not on XP Home. But thanks, Juanma. I didn't know about that command. ^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Memory leak 2008-05-16 20:41 ` David Robinow @ 2008-05-16 21:36 ` Lennart Borgman (gmail) 2008-05-17 7:10 ` Eli Zaretskii 1 sibling, 0 replies; 39+ messages in thread From: Lennart Borgman (gmail) @ 2008-05-16 21:36 UTC (permalink / raw To: David Robinow; +Cc: Juanma Barranquero, Eli Zaretskii, emacs-devel David Robinow wrote: > On Fri, May 16, 2008 at 5:25 AM, Eli Zaretskii <eliz@gnu.org> wrote: >>> Date: Fri, 16 May 2008 11:03:30 +0200 >>> From: "Juanma Barranquero" <lekktu@gmail.com> >>> Cc: "Lennart Borgman (gmail)" <lennart.borgman@gmail.com>, emacs-devel@gnu.org >>> >>> tasklist /fi "imagename eq emacs*" >> You need to use emacs.exe as the IMAGENAME, otherwise it will not >> work. >> (Btw, I think this command is present on XP Home Edition as well.) > > Apparently not. I have one of each. Works on XP Pro. Not on XP Home. > But thanks, Juanma. I didn't know about that command. Thanks Juanma and Eli. Though while doing things like this I find it more convenient to just do tasklist | findstr emacs >> memory-leak.tmp ^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Memory leak 2008-05-16 20:41 ` David Robinow 2008-05-16 21:36 ` Lennart Borgman (gmail) @ 2008-05-17 7:10 ` Eli Zaretskii 1 sibling, 0 replies; 39+ messages in thread From: Eli Zaretskii @ 2008-05-17 7:10 UTC (permalink / raw To: David Robinow; +Cc: lekktu, lennart.borgman, emacs-devel > Date: Fri, 16 May 2008 16:41:39 -0400 > From: "David Robinow" <drobinow@gmail.com> > Cc: "Juanma Barranquero" <lekktu@gmail.com>, lennart.borgman@gmail.com, > emacs-devel@gnu.org > > > (Btw, I think this command is present on XP Home Edition as well.) > > Apparently not. I have one of each. Works on XP Pro. Not on XP Home. It does on mine, but maybe it came with one of the optional toolkits I downloaded from the MS site. ^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Memory leak 2008-05-16 0:25 ` Lennart Borgman (gmail) 2008-05-16 8:35 ` Eli Zaretskii @ 2008-05-16 9:43 ` Eli Zaretskii 2008-05-16 12:50 ` Jason Rumney 1 sibling, 1 reply; 39+ messages in thread From: Eli Zaretskii @ 2008-05-16 9:43 UTC (permalink / raw To: Lennart Borgman (gmail); +Cc: monnier, emacs-devel > Date: Fri, 16 May 2008 02:25:50 +0200 > From: "Lennart Borgman (gmail)" <lennart.borgman@gmail.com> > Cc: Stefan Monnier <monnier@IRO.UMontreal.CA>, emacs-devel@gnu.org > > I can see a memory leak on w32 of around 240 K in the above operation > with a build from 2008-03-01 With today's CVS on w32, the memory grows by ~900KB with each new frame I create, and that memory is not released when the frame and its buffer are deleted. ^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Memory leak 2008-05-16 9:43 ` Eli Zaretskii @ 2008-05-16 12:50 ` Jason Rumney 2008-05-19 8:05 ` Kenichi Handa 2008-05-24 2:30 ` calculation of text extents [Re: Memory leak] Kenichi Handa 0 siblings, 2 replies; 39+ messages in thread From: Jason Rumney @ 2008-05-16 12:50 UTC (permalink / raw To: Eli Zaretskii; +Cc: Lennart Borgman (gmail), monnier, emacs-devel Eli Zaretskii wrote: > With today's CVS on w32, the memory grows by ~900KB with each new > frame I create, and that memory is not released when the frame and its > buffer are deleted. > If the leak is coming from the font cache, then I wouldn't expect it to be released when the frame is deleted, as the font cache is shared between all frames on w32 (and I think between all frames on the same display on X). Stefan is seeing 10x the increase as you are on Windows, which could be explained by xft loading fonts into Emacs's process space, while on Windows the font itself stays in system owned memory, and only Emacs's internal info that it associates with the font contributes to the growth. That also points to fonts being responsible (though circumstantial). A while ago I noticed w32font_open being called multiple times for the same font, so I suspect that we may be overlooking the font cache in places and reopening fonts when we could be reusing an existing font. There is another similar problem when we get text extents - each redraw cycle can call w32font_text_extents a number of times for the same glyph_string (from compute_glyph_string_overhangs, left_overwriting and right_overwriting in quick succession in draw_glyphs, with nested compute_overhangs_and_x within some of the if statements that cause extra calls for preceding and following glyph_strings if there is overhang, which there often is with antialising enabled). Perhaps this operation is cheap with the X font backends, but on Windows it is expensive (even with caching of metrics), and I suspect this is a major cause of the slowdown reported by Windows users. ^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Memory leak 2008-05-16 12:50 ` Jason Rumney @ 2008-05-19 8:05 ` Kenichi Handa 2008-05-19 8:45 ` YAMAMOTO Mitsuharu 2008-05-19 13:34 ` David Hansen 2008-05-24 2:30 ` calculation of text extents [Re: Memory leak] Kenichi Handa 1 sibling, 2 replies; 39+ messages in thread From: Kenichi Handa @ 2008-05-19 8:05 UTC (permalink / raw To: Jason Rumney; +Cc: eliz, lennart.borgman, monnier, emacs-devel In article <482D8332.5030908@gnu.org>, Jason Rumney <jasonr@gnu.org> writes: > If the leak is coming from the font cache, then I wouldn't expect it to > be released when the frame is deleted, as the font cache is shared > between all frames on w32 (and I think between all frames on the same > display on X). Stefan is seeing 10x the increase as you are on Windows, > which could be explained by xft loading fonts into Emacs's process > space, while on Windows the font itself stays in system owned memory, > and only Emacs's internal info that it associates with the font > contributes to the growth. That also points to fonts being responsible > (though circumstantial). I've just installed a fix in font-cache handling. Now, even when I create many new frames, the memory usage doesn't increase that much. Though it still increases several handreds K-bytes each time a new frame is created, and it is not reduced when I delete the frames. So, I think there still be a problem somewhere. FYI, when I compile Emacs without gtk, the amount of (non-freed) memory increase becomes smaller. > There is another similar problem when we get text extents - each redraw > cycle can call w32font_text_extents a number of times for the same > glyph_string (from compute_glyph_string_overhangs, left_overwriting and > right_overwriting in quick succession in draw_glyphs, with nested > compute_overhangs_and_x within some of the if statements that cause > extra calls for preceding and following glyph_strings if there is > overhang, which there often is with antialising enabled). Perhaps this > operation is cheap with the X font backends, but on Windows it is > expensive (even with caching of metrics), and I suspect this is a major > cause of the slowdown reported by Windows users. I remember that I modified x_draw_glyph_string for such a font-backend as xft that can't do foreground-only drawing because of antialiasing, which increases the count of overhang-checking. Perhaps we should study the code of draw_glyphs and x_draw_glyph_string and find a solution. --- Kenichi Handa handa@ni.aist.go.jp ^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Memory leak 2008-05-19 8:05 ` Kenichi Handa @ 2008-05-19 8:45 ` YAMAMOTO Mitsuharu 2008-05-19 11:13 ` Kenichi Handa 2008-05-19 13:34 ` David Hansen 1 sibling, 1 reply; 39+ messages in thread From: YAMAMOTO Mitsuharu @ 2008-05-19 8:45 UTC (permalink / raw To: Kenichi Handa; +Cc: eliz, emacs-devel, lennart.borgman, monnier, Jason Rumney >>>>> On Mon, 19 May 2008 17:05:42 +0900, Kenichi Handa <handa@m17n.org> said: > I've just installed a fix in font-cache handling. It seems like the `free_entity' entry in `struct font_driver' no longer makes sense because you tried to remove the memory leaks by avoiding Lisp_Save_Value in font entities instead of calling free_entity at some proper timings. Is that right? YAMAMOTO Mitsuharu mituharu@math.s.chiba-u.ac.jp ^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Memory leak 2008-05-19 8:45 ` YAMAMOTO Mitsuharu @ 2008-05-19 11:13 ` Kenichi Handa 2008-05-19 11:59 ` YAMAMOTO Mitsuharu 0 siblings, 1 reply; 39+ messages in thread From: Kenichi Handa @ 2008-05-19 11:13 UTC (permalink / raw To: YAMAMOTO Mitsuharu; +Cc: eliz, emacs-devel, lennart.borgman, monnier, jasonr In article <wlzlqmu1gw.wl%mituharu@math.s.chiba-u.ac.jp>, YAMAMOTO Mitsuharu <mituharu@math.s.chiba-u.ac.jp> writes: >>>>>> On Mon, 19 May 2008 17:05:42 +0900, Kenichi Handa <handa@m17n.org> said: > > I've just installed a fix in font-cache handling. > It seems like the `free_entity' entry in `struct font_driver' no > longer makes sense because you tried to remove the memory leaks by > avoiding Lisp_Save_Value in font entities instead of calling > free_entity at some proper timings. Is that right? Not exactly. When I deleted ftfont_free_entity, it was in my mind to change also ftfont_list to avoid using Lisp_Save_Value, but I just forgot to implement that change. The free_entity callback is still called in font_clear_cache if a font-driver provides that function. --- Kenichi Handa handa@ni.aist.go.jp ^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Memory leak 2008-05-19 11:13 ` Kenichi Handa @ 2008-05-19 11:59 ` YAMAMOTO Mitsuharu 2008-05-23 2:24 ` Kenichi Handa 0 siblings, 1 reply; 39+ messages in thread From: YAMAMOTO Mitsuharu @ 2008-05-19 11:59 UTC (permalink / raw To: Kenichi Handa; +Cc: eliz, emacs-devel, lennart.borgman, monnier, jasonr >>>>> On Mon, 19 May 2008 20:13:15 +0900, Kenichi Handa <handa@m17n.org> said: >> It seems like the `free_entity' entry in `struct font_driver' no >> longer makes sense because you tried to remove the memory leaks by >> avoiding Lisp_Save_Value in font entities instead of calling >> free_entity at some proper timings. Is that right? > Not exactly. When I deleted ftfont_free_entity, it was in my mind > to change also ftfont_list to avoid using Lisp_Save_Value, but I > just forgot to implement that change. > The free_entity callback is still called in font_clear_cache if a > font-driver provides that function. So, `free_entity' is supposed to be still valid. Then I think the entities deleted in `font_delete_unmatched' need to be freed as they are not kept in the cache. Also, there are some suspected leaks in font-related code: * `names' in xfont_list_pattern. * `charset' in ftfont_spec_pattern when it is created from Vscript_representative_chars. YAMAMOTO Mitsuharu mituharu@math.s.chiba-u.ac.jp ^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Memory leak 2008-05-19 11:59 ` YAMAMOTO Mitsuharu @ 2008-05-23 2:24 ` Kenichi Handa 0 siblings, 0 replies; 39+ messages in thread From: Kenichi Handa @ 2008-05-23 2:24 UTC (permalink / raw To: YAMAMOTO Mitsuharu; +Cc: eliz, emacs-devel, lennart.borgman, monnier, jasonr In article <wl4p8ua4ip.wl%mituharu@math.s.chiba-u.ac.jp>, YAMAMOTO Mitsuharu <mituharu@math.s.chiba-u.ac.jp> writes: > So, `free_entity' is supposed to be still valid. Then I think > the entities deleted in `font_delete_unmatched' need to be freed > as they are not kept in the cache. > Also, there are some suspected leaks in font-related code: > * `names' in xfont_list_pattern. > * `charset' in ftfont_spec_pattern when it is created from > Vscript_representative_chars. Thank you for finding these bugs. I fixed them. --- Kenichi Handa handa@ni.aist.go.jp ^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Memory leak 2008-05-19 8:05 ` Kenichi Handa 2008-05-19 8:45 ` YAMAMOTO Mitsuharu @ 2008-05-19 13:34 ` David Hansen 2008-05-19 19:42 ` David Hansen 1 sibling, 1 reply; 39+ messages in thread From: David Hansen @ 2008-05-19 13:34 UTC (permalink / raw To: emacs-devel On Mon, 19 May 2008 17:05:42 +0900 Kenichi Handa wrote: > I've just installed a fix in font-cache handling. Now, even > when I create many new frames, the memory usage doesn't > increase that much. Though it still increases several > handreds K-bytes each time a new frame is created, and it is > not reduced when I delete the frames. So, I think there > still be a problem somewhere. I used to have two frames open in my emacs session, never create or destroy any other frames. Memory usage climbs to > 500M here after some hours of usage. I doubt it has much to do with frame creation. Will update right now and report back if your changes help. David ^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Memory leak 2008-05-19 13:34 ` David Hansen @ 2008-05-19 19:42 ` David Hansen 0 siblings, 0 replies; 39+ messages in thread From: David Hansen @ 2008-05-19 19:42 UTC (permalink / raw To: emacs-devel On Mon, 19 May 2008 15:34:20 +0200 David Hansen wrote: > Will update right now and report back if your changes help. At least not a that excessive one. Usage is now at about 70M which is pretty normal for my emacs usage. But: - Italics isn't working anymore - The Gnus *summary* buffer flickers … - … as does looking at some parts of the UTF-8-demo.txt[1] (seems to correlate with the display of line drawing characters) … - … which is also incredible slooooow (scrolling and cursor movements on the timescale of several seconds on my 1.8GHz AMD box) - Combining characters is broken (at least the example in above demo file). David Footnotes: [1] http://www.cl.cam.ac.uk/~mgk25/ucs/examples/UTF-8-demo.txt ^ permalink raw reply [flat|nested] 39+ messages in thread
* calculation of text extents [Re: Memory leak] 2008-05-16 12:50 ` Jason Rumney 2008-05-19 8:05 ` Kenichi Handa @ 2008-05-24 2:30 ` Kenichi Handa 1 sibling, 0 replies; 39+ messages in thread From: Kenichi Handa @ 2008-05-24 2:30 UTC (permalink / raw To: Jason Rumney; +Cc: eliz, lennart.borgman, monnier, emacs-devel In article <482D8332.5030908@gnu.org>, Jason Rumney <jasonr@gnu.org> writes: > A while ago I noticed w32font_open being called multiple times for the > same font, so I suspect that we may be overlooking the font cache in > places and reopening fonts when we could be reusing an existing font. With the recent fix of font-cache handling, I think the number of calling `open' callback of a font-backend for the same font is reduced. Can you confirm that? > There is another similar problem when we get text extents - each redraw > cycle can call w32font_text_extents a number of times for the same > glyph_string (from compute_glyph_string_overhangs, left_overwriting and > right_overwriting in quick succession in draw_glyphs, with nested > compute_overhangs_and_x within some of the if statements that cause > extra calls for preceding and following glyph_strings if there is > overhang, which there often is with antialising enabled). Perhaps this > operation is cheap with the X font backends, but on Windows it is > expensive (even with caching of metrics), and I suspect this is a major > cause of the slowdown reported by Windows users. It seems that we can reduce the number of calling `text_extents' callback by recording lbearing and rbearing in struct glyph; which grows the size of that structure from the current 32-byte to 36-byte. I'm not sure how much that change impacts the memory usage of Emacs, but isn't it worth trying? --- Kenichi Handa handa@ni.aist.go.jp ^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Memory leak 2008-05-15 23:06 ` David Kastrup 2008-05-16 0:25 ` Lennart Borgman (gmail) @ 2008-05-16 2:19 ` Stefan Monnier 1 sibling, 0 replies; 39+ messages in thread From: Stefan Monnier @ 2008-05-16 2:19 UTC (permalink / raw To: David Kastrup; +Cc: emacs-devel > Just to rule out something obvious (pure space overflow): > M-: (garbage-collect) RET > does not return nil, does it? No. Stefan ^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Memory leak 2008-05-15 22:50 Memory leak Stefan Monnier 2008-05-15 23:06 ` David Kastrup @ 2008-05-16 0:38 ` YAMAMOTO Mitsuharu 2008-05-17 15:16 ` Evil Boris 2 siblings, 0 replies; 39+ messages in thread From: YAMAMOTO Mitsuharu @ 2008-05-16 0:38 UTC (permalink / raw To: Stefan Monnier; +Cc: emacs-devel >>>>> On Thu, 15 May 2008 18:50:29 -0400, Stefan Monnier <monnier@IRO.UMontreal.CA> said: > We have a pretty serious memory leak. My Emacs session recently > died with a memory full situation (the process was around 500MB > VSZ). So I took a look around and it seem that every time I create > a new frame, my Emacs session grows by around 10MB and deleting the > frame does not recover this memory. This memory is not accounted > for by the counts returned by `garbage-collect'. One thing I'm concerned about is the removal of `ftfont_free_entity'. Actually, the `leaks' command on Mac OS X shows that several resources allocated at ftfont_pattern_entity are leaking. And some minor nitpicking: In xftfont_open, font->pixel_size and font->driver are set twice (and to different values for font->pixel_size). YAMAMOTO Mitsuharu mituharu@math.s.chiba-u.ac.jp ^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Memory leak 2008-05-15 22:50 Memory leak Stefan Monnier 2008-05-15 23:06 ` David Kastrup 2008-05-16 0:38 ` YAMAMOTO Mitsuharu @ 2008-05-17 15:16 ` Evil Boris 2008-05-17 19:11 ` Stephen J. Turnbull 2008-05-23 13:54 ` Evil Boris 2 siblings, 2 replies; 39+ messages in thread From: Evil Boris @ 2008-05-17 15:16 UTC (permalink / raw To: emacs-devel Stefan Monnier <monnier@IRO.UMontreal.CA> writes: > We have a pretty serious memory leak. > My Emacs session recently died with a memory full situation (the > process was around 500MB VSZ). I do not know if this is related, but sometime after January (around Unicode merge) I noticed that, after running for a while (have not been able to notice when precisely this happens, but seems to require having at least one X frame, sometimes happens within a few minutes of starting), Emacs starts occupying 190-250Mb of memory. Since I usually edit smallish files (<50-100K) and not too many of them (<10, generally), the only candidate for "eating space" is my RMAIL file, which is around 30Mb. Any ideas? Any obvious place to check what memory is being used (ancient Solaris 2.7, GCC 3.1)? Tried on many different CVS versions, including the most current one. --Boris PS. Was about to contradict myself---the currently running copy is 179M and has no X frame, but I now recall creating an X frame yesterday, and then destroying it. ^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Memory leak 2008-05-17 15:16 ` Evil Boris @ 2008-05-17 19:11 ` Stephen J. Turnbull 2008-05-17 19:17 ` Eric Hanchrow 2008-05-23 13:54 ` Evil Boris 1 sibling, 1 reply; 39+ messages in thread From: Stephen J. Turnbull @ 2008-05-17 19:11 UTC (permalink / raw To: Evil Boris; +Cc: emacs-devel Evil Boris writes: > I do not know if this is related, but sometime after January (around > Unicode merge) I noticed that, after running for a while (have not > been able to notice when precisely this happens, but seems to require > having at least one X frame, sometimes happens within a few minutes of > starting), Emacs starts occupying 190-250Mb of memory. FWIW, XEmacs has some hard-to-pin-down memory leaks, too, one or more of which seems to be related to Xft/fontconfig. These packages are coded in the classic X11 style, meaning that the burden is entirely on the client code to clean things up. There *is* refcounting and things like that, but for internal use of the Xft code. I have no idea what the memory use characteristics of underlying rendering engines such as FreeType might be, so this has been really hard to track down. ^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Memory leak 2008-05-17 19:11 ` Stephen J. Turnbull @ 2008-05-17 19:17 ` Eric Hanchrow 2008-05-18 11:26 ` Jan Djärv 0 siblings, 1 reply; 39+ messages in thread From: Eric Hanchrow @ 2008-05-17 19:17 UTC (permalink / raw To: emacs-devel Well, here's a dumb idea: has anyone tried running Emacs under valgrind? I dimly recall that tool being remarkably useful for finding memory leaks. -- Governing the U.S. is like playing 200 simultaneous chess matches (while whiny columnists second-guess every move on every board). -- Nicholas Kristof, New York Times columnist ^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Memory leak 2008-05-17 19:17 ` Eric Hanchrow @ 2008-05-18 11:26 ` Jan Djärv 2008-05-18 11:29 ` David Kastrup ` (2 more replies) 0 siblings, 3 replies; 39+ messages in thread From: Jan Djärv @ 2008-05-18 11:26 UTC (permalink / raw To: Eric Hanchrow; +Cc: emacs-devel Eric Hanchrow skrev: > Well, here's a dumb idea: has anyone tried running Emacs under valgrind? > I dimly recall that tool being remarkably useful for finding memory > leaks. Valgrind just craches when run on Emacs. Emacs memory handling is too much for valgrind. It is the undump think that it doesn't understand. Jan D. ^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Memory leak 2008-05-18 11:26 ` Jan Djärv @ 2008-05-18 11:29 ` David Kastrup 2008-05-20 6:46 ` Jan Djärv 2008-05-18 13:34 ` Werner LEMBERG 2008-05-19 2:44 ` Stephen J. Turnbull 2 siblings, 1 reply; 39+ messages in thread From: David Kastrup @ 2008-05-18 11:29 UTC (permalink / raw To: Jan Djärv; +Cc: Eric Hanchrow, emacs-devel Jan Djärv <jan.h.d@swipnet.se> writes: > Eric Hanchrow skrev: >> Well, here's a dumb idea: has anyone tried running Emacs under valgrind? >> I dimly recall that tool being remarkably useful for finding memory >> leaks. > > Valgrind just craches when run on Emacs. Emacs memory handling is too > much for valgrind. It is the undump think that it doesn't understand. What about running valgrind on temacs then? -- David Kastrup, Kriemhildstr. 15, 44793 Bochum ^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Memory leak 2008-05-18 11:29 ` David Kastrup @ 2008-05-20 6:46 ` Jan Djärv 0 siblings, 0 replies; 39+ messages in thread From: Jan Djärv @ 2008-05-20 6:46 UTC (permalink / raw To: David Kastrup; +Cc: Eric Hanchrow, emacs-devel David Kastrup skrev: > Jan Djärv <jan.h.d@swipnet.se> writes: > >> Eric Hanchrow skrev: >>> Well, here's a dumb idea: has anyone tried running Emacs under valgrind? >>> I dimly recall that tool being remarkably useful for finding memory >>> leaks. >> Valgrind just craches when run on Emacs. Emacs memory handling is too >> much for valgrind. It is the undump think that it doesn't understand. > > What about running valgrind on temacs then? > That should work, I haven't tried it. Jan D. ^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Memory leak 2008-05-18 11:26 ` Jan Djärv 2008-05-18 11:29 ` David Kastrup @ 2008-05-18 13:34 ` Werner LEMBERG 2008-05-20 6:48 ` Jan Djärv 2008-05-19 2:44 ` Stephen J. Turnbull 2 siblings, 1 reply; 39+ messages in thread From: Werner LEMBERG @ 2008-05-18 13:34 UTC (permalink / raw To: jan.h.d; +Cc: offby1, emacs-devel > Valgrind just craches when run on Emacs. Emacs memory handling is too > much for valgrind. It is the undump think that it doesn't understand. What about contacting the valgrind guys directly for help? I think that Emacs would get *huge* benefits if checked with it. Werner ^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Memory leak 2008-05-18 13:34 ` Werner LEMBERG @ 2008-05-20 6:48 ` Jan Djärv 0 siblings, 0 replies; 39+ messages in thread From: Jan Djärv @ 2008-05-20 6:48 UTC (permalink / raw To: Werner LEMBERG; +Cc: offby1, emacs-devel Werner LEMBERG skrev: >> Valgrind just craches when run on Emacs. Emacs memory handling is too >> much for valgrind. It is the undump think that it doesn't understand. > > What about contacting the valgrind guys directly for help? I think > that Emacs would get *huge* benefits if checked with it. > > If my memory is correct, someone already did that. I must search the mail archives. I think the Emacs way of doing things was too special for the valgrind guys to devote time to. I actually agree with that. From what I recall, it was the saving and restoring of malloc state during dump and undump that was the main problem. Valgrind has no way of correctly capture this state and begin tracking from there. Jan D. ^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Memory leak 2008-05-18 11:26 ` Jan Djärv 2008-05-18 11:29 ` David Kastrup 2008-05-18 13:34 ` Werner LEMBERG @ 2008-05-19 2:44 ` Stephen J. Turnbull 2008-05-20 6:50 ` Jan Djärv 2 siblings, 1 reply; 39+ messages in thread From: Stephen J. Turnbull @ 2008-05-19 2:44 UTC (permalink / raw To: Jan Djärv; +Cc: Eric Hanchrow, emacs-devel Jan Djärv writes: > Valgrind just craches when run on Emacs. Portable dumper wins here: XEmacs at least and maybe SXEmacs valgrind perfectly well. You should also be able to valgrind temacs. For more info, you could post to xemacs-beta@xemacs.org (nonmember posts moderated) and the relevant sxemacs.org list (www.sxemacs.org for references, dunno about mod policy). ^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Memory leak 2008-05-19 2:44 ` Stephen J. Turnbull @ 2008-05-20 6:50 ` Jan Djärv 2008-05-21 6:10 ` Stephen J. Turnbull 0 siblings, 1 reply; 39+ messages in thread From: Jan Djärv @ 2008-05-20 6:50 UTC (permalink / raw To: Stephen J. Turnbull; +Cc: Eric Hanchrow, emacs-devel Stephen J. Turnbull skrev: > Jan Djärv writes: > > > Valgrind just craches when run on Emacs. > > Portable dumper wins here: XEmacs at least and maybe SXEmacs valgrind > perfectly well. Yes. But it is not a small project. > > You should also be able to valgrind temacs. > That is a small project :-) Jan D. ^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Memory leak 2008-05-20 6:50 ` Jan Djärv @ 2008-05-21 6:10 ` Stephen J. Turnbull 2008-05-21 17:32 ` Stefan Monnier 0 siblings, 1 reply; 39+ messages in thread From: Stephen J. Turnbull @ 2008-05-21 6:10 UTC (permalink / raw To: Jan Djärv; +Cc: Eric Hanchrow, emacs-devel Jan Djärv writes: > > > Stephen J. Turnbull skrev: > > Jan Djärv writes: > > > > > Valgrind just craches when run on Emacs. > > > > Portable dumper wins here: XEmacs at least and maybe SXEmacs valgrind > > perfectly well. > > Yes. But it is not a small project. Um, IIRC it's a done project, though. Somebody's got a patch in the queue, no? I don't know if valgrind is a big enough win to push that into the mainline now, though. ^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Memory leak 2008-05-21 6:10 ` Stephen J. Turnbull @ 2008-05-21 17:32 ` Stefan Monnier 0 siblings, 0 replies; 39+ messages in thread From: Stefan Monnier @ 2008-05-21 17:32 UTC (permalink / raw To: Stephen J. Turnbull; +Cc: Eric Hanchrow, Jan Djärv, emacs-devel > Um, IIRC it's a done project, though. Somebody's got a patch in the > queue, no? I've heard rumors that someone has worked on it at some point, but that's a far cry from a "done project", sadly. For one, I still haven't seen any patch for it. And if there's such a patch, I wouldn't be surprised if it bitrotted significantly. Stefan ^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Memory leak 2008-05-17 15:16 ` Evil Boris 2008-05-17 19:11 ` Stephen J. Turnbull @ 2008-05-23 13:54 ` Evil Boris 1 sibling, 0 replies; 39+ messages in thread From: Evil Boris @ 2008-05-23 13:54 UTC (permalink / raw To: emacs-devel "Evil Boris" <evilborisnet@netscape.net> writes: > I do not know if this is related, but sometime after January (around > Unicode merge) I noticed that, after running for a while (have not > been able to notice when precisely this happens, but seems to require > having at least one X frame, sometimes happens within a few minutes of > starting), Emacs starts occupying 190-250Mb of memory. Since I > usually edit smallish files (<50-100K) and not too many of them (<10, > generally), the only candidate for "eating space" is my RMAIL file, > which is around 30Mb. I just checked, with a recent version from CVS [GNU Emacs 23.0.60.9 (sparc-sun-solaris2.7, X toolkit) of 2008-05-21 on ziggy], with no X frames ever opened, playing almost exclusively with a few small files, with the sole exception of my 30Mb mailbox, top shows: PID USERNAME THR PRI NICE SIZE RES STATE TIME CPU COMMAND 1754 ... 1 58 0 174M 172M sleep 4:59 0.00% emacs 174Mb again? Any ideas? Any code I can run to try to see what the memory is being being used for? --Boris ^ permalink raw reply [flat|nested] 39+ messages in thread
end of thread, other threads:[~2008-05-24 2:30 UTC | newest] Thread overview: 39+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2008-05-15 22:50 Memory leak Stefan Monnier 2008-05-15 23:06 ` David Kastrup 2008-05-16 0:25 ` Lennart Borgman (gmail) 2008-05-16 8:35 ` Eli Zaretskii 2008-05-16 9:03 ` Juanma Barranquero 2008-05-16 9:25 ` Eli Zaretskii 2008-05-16 9:29 ` Juanma Barranquero 2008-05-16 9:46 ` Eli Zaretskii 2008-05-16 9:49 ` Juanma Barranquero 2008-05-16 20:13 ` Stephen J. Turnbull 2008-05-16 21:15 ` [OT, silly] " Thomas Lord 2008-05-16 20:41 ` David Robinow 2008-05-16 21:36 ` Lennart Borgman (gmail) 2008-05-17 7:10 ` Eli Zaretskii 2008-05-16 9:43 ` Eli Zaretskii 2008-05-16 12:50 ` Jason Rumney 2008-05-19 8:05 ` Kenichi Handa 2008-05-19 8:45 ` YAMAMOTO Mitsuharu 2008-05-19 11:13 ` Kenichi Handa 2008-05-19 11:59 ` YAMAMOTO Mitsuharu 2008-05-23 2:24 ` Kenichi Handa 2008-05-19 13:34 ` David Hansen 2008-05-19 19:42 ` David Hansen 2008-05-24 2:30 ` calculation of text extents [Re: Memory leak] Kenichi Handa 2008-05-16 2:19 ` Memory leak Stefan Monnier 2008-05-16 0:38 ` YAMAMOTO Mitsuharu 2008-05-17 15:16 ` Evil Boris 2008-05-17 19:11 ` Stephen J. Turnbull 2008-05-17 19:17 ` Eric Hanchrow 2008-05-18 11:26 ` Jan Djärv 2008-05-18 11:29 ` David Kastrup 2008-05-20 6:46 ` Jan Djärv 2008-05-18 13:34 ` Werner LEMBERG 2008-05-20 6:48 ` Jan Djärv 2008-05-19 2:44 ` Stephen J. Turnbull 2008-05-20 6:50 ` Jan Djärv 2008-05-21 6:10 ` Stephen J. Turnbull 2008-05-21 17:32 ` Stefan Monnier 2008-05-23 13:54 ` Evil Boris
Code repositories for project(s) associated with this external index https://git.savannah.gnu.org/cgit/emacs.git https://git.savannah.gnu.org/cgit/emacs/org-mode.git This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.