From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Stefan Monnier Newsgroups: gmane.emacs.devel Subject: Re: Emacs bzr memory footprint Date: Fri, 21 Oct 2011 13:52:45 -0400 Message-ID: References: <83fwix2osa.fsf@gnu.org> <0B3EE7A4-D0D6-4D1E-ADC4-0BEE68F179B2@mit.edu> <87fwivwp37.fsf@turtle.gmx.de> <87sjmvpmd2.fsf@lifelogs.com> <87aa93wmc4.fsf@turtle.gmx.de> <87sjmnrdjw.fsf@spindle.srvr.nix> <87ty73mc0m.fsf@spindle.srvr.nix> <87wrbyl75y.fsf@spindle.srvr.nix> NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 Content-Type: text/plain X-Trace: dough.gmane.org 1319219577 14437 80.91.229.12 (21 Oct 2011 17:52:57 GMT) X-Complaints-To: usenet@dough.gmane.org NNTP-Posting-Date: Fri, 21 Oct 2011 17:52:57 +0000 (UTC) Cc: John Wiegley , emacs-devel@gnu.org To: Nix Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Fri Oct 21 19:52:52 2011 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([140.186.70.17]) by lo.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1RHJH2-0007i6-0z for ged-emacs-devel@m.gmane.org; Fri, 21 Oct 2011 19:52:52 +0200 Original-Received: from localhost ([::1]:45857 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1RHJH1-0001pF-K0 for ged-emacs-devel@m.gmane.org; Fri, 21 Oct 2011 13:52:51 -0400 Original-Received: from eggs.gnu.org ([140.186.70.92]:55470) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1RHJGy-0001nv-LY for emacs-devel@gnu.org; Fri, 21 Oct 2011 13:52:49 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1RHJGx-0001kC-Dq for emacs-devel@gnu.org; Fri, 21 Oct 2011 13:52:48 -0400 Original-Received: from ironport2-out.teksavvy.com ([206.248.154.183]:36766 helo=ironport2-out.pppoe.ca) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1RHJGx-0001k0-9u for emacs-devel@gnu.org; Fri, 21 Oct 2011 13:52:47 -0400 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Av0EAF2woU5MCqLO/2dsb2JhbABDqR6BBoFuAQEEAVYjBQsLNBIUGA0kiBO0DYhABKEyhEU X-IronPort-AV: E=Sophos;i="4.69,387,1315195200"; d="scan'208";a="143624004" Original-Received: from 76-10-162-206.dsl.teksavvy.com (HELO pastel.home) ([76.10.162.206]) by ironport2-out.pppoe.ca with ESMTP/TLS/ADH-AES256-SHA; 21 Oct 2011 13:52:45 -0400 Original-Received: by pastel.home (Postfix, from userid 20848) id 585E9591FD; Fri, 21 Oct 2011 13:52:45 -0400 (EDT) In-Reply-To: <87wrbyl75y.fsf@spindle.srvr.nix> (nix@esperi.org.uk's message of "Fri, 21 Oct 2011 16:02:17 +0100") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.0.90 (gnu/linux) X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 206.248.154.183 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.devel:145403 Archived-At: >> If this output does not explain the process's size you're seeing, then >> we have a leak in the C code somewhere. If it does, then we have either >> a leak in Elisp, or at least an excessive memory use by some package, >> and hopefully we can at least figure out which category of object >> is involved. > ... or the same thing XEmacs apparently had, which was excessive memory > consumption by one or more other still-unidentified C libraries. Yes, that's what I meant by "in the C code somewhere". Another thread recently seemed to suggest that maybe the new C-level support for gnutls may cause such problems. > True! So it wouldn't be as hellish to write our own allocator as it > would be for most projects, because we've done 90% of the work already > (including the hard part: allocating things of variable size). Actually, no, that doesn't include the hard part: we only do our own allocation for fixed-size objects (cons, floats, markers&overlays, symbols, string headers) plus relocatable variable size objects (string text). For the non-relocatable, non-fixed size objects (i.e. vectors) we defer to malloc. > Yeah. As an aside: most of the inside of Emacs is quite nice, but I am > seriously unimpressed by overlay storage. Linear scans for > 'overlays-at'? Ew. One of these years I might try to implement something > better, like what XEmacs did for extents, so you can find extents over a > given buffer position more efficiently... (not because this is visibly a > problem in profiles that I've seen, merely because the inefficiency > offends me :) ) Yes, overlays (and to some extent markers) are in serious needs of a redesign. We get bitten by this performance problem, which is why a lot of code goes through a fair bit of trouble to avoid overlays and use text-properties instead (and the code that doesn't bother doing that is usually only usable on small buffers). Stefan