From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Stefan Monnier Newsgroups: gmane.emacs.devel Subject: Re: Memory again Date: Fri, 16 Dec 2011 16:55:59 -0500 Message-ID: References: <4ED0F945.5090805@yandex.ru> <83pqge7syw.fsf@gnu.org> <87mxb6tkji.fsf@wanadoo.es> <87borlu0kc.fsf@wanadoo.es> NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 Content-Type: text/plain X-Trace: dough.gmane.org 1324072573 9341 80.91.229.12 (16 Dec 2011 21:56:13 GMT) X-Complaints-To: usenet@dough.gmane.org NNTP-Posting-Date: Fri, 16 Dec 2011 21:56:13 +0000 (UTC) Cc: =?iso-8859-1?Q?=D3scar?= Fuentes , emacs-devel@gnu.org To: Tim Connors Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Fri Dec 16 22:56:08 2011 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([140.186.70.17]) by lo.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1Rbfl8-0001z5-Ip for ged-emacs-devel@m.gmane.org; Fri, 16 Dec 2011 22:56:06 +0100 Original-Received: from localhost ([::1]:53047 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Rbfl7-0007Bc-Vd for ged-emacs-devel@m.gmane.org; Fri, 16 Dec 2011 16:56:05 -0500 Original-Received: from eggs.gnu.org ([140.186.70.92]:33733) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Rbfl4-0007BI-Q1 for emacs-devel@gnu.org; Fri, 16 Dec 2011 16:56:03 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Rbfl3-0007ry-TP for emacs-devel@gnu.org; Fri, 16 Dec 2011 16:56:02 -0500 Original-Received: from smtp-03.vtx.ch ([194.38.175.92]:38075) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Rbfl3-0007rb-KO for emacs-devel@gnu.org; Fri, 16 Dec 2011 16:56:01 -0500 Original-Received: from ceviche.home (dyn.83-228-219-171.dsl.vtx.ch [83.228.219.171]) by smtp-03.vtx.ch (VTX Services SA) with ESMTP id BD5D9297192; Fri, 16 Dec 2011 22:55:59 +0100 (CET) Original-Received: by ceviche.home (Postfix, from userid 20848) id 6F1E0662EE; Fri, 16 Dec 2011 16:55:59 -0500 (EST) In-Reply-To: (Tim Connors's message of "Thu, 15 Dec 2011 14:52:23 +1100 (EST)") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.0.92 (gnu/linux) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6, seldom 2.4 (older, 4) X-Received-From: 194.38.175.92 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.devel:146757 Archived-At: > But right now, having had emacs up for a few days, and only opening 10 > small files with the aid of emacsclient, emacs's RSS is 130M. It had > climbed up to 400M before I most recently killed it. 400M definitely sounds like a bug. > What's the differences between Emacs's and XEmacs's allocators? I don't think the source of your problem/difference is in the allocator. > By the way, the dismissal of this being a real problem because Emacs can > always reuse the fragmented memory either I don't really dismiss it. My point is rather the following: - Many memory problems are misdiagnosed as fragmentation problems, where the real cause is either inefficient memory use, or memory leaks. - Real fragmentation problems exist but are fairly rare. - In some "fragmentation" cases, the Emacs process holds on to memory without any good reason, i.e. it's been "free"d but the malloc library does not want or can't return it to the OS because it did not use mmap to allocate it. This can be fixed, but this memory would really be unused; it can still appear in RSS but only because the memory pressure hasn't yet bumped it out of RAM. IOW these cases may show high memory use in terms of VSZ or RSS but fixing them is low priority because their only direct cost is use of of swap space. - Fixing the remaining real fragmentation problems probably requires a different allocator that can move objects to compact the memory space. Maybe such an allocator can be retrofitted into Emacs (e.g. a mostly-copying allocator), but noone has tried it yet (probably because the outcome is hard to predict, the problem it attempts to fix only occurs rather rarely, and it can be difficult to ensure that it doesn't affect negatively the more common cases). So, I know such problems exist in real life, and I would be happy to accept patches that fix them, but it's not very high on my list of priorities. Stefan