From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Eli Zaretskii Newsgroups: gmane.emacs.bugs Subject: bug#20220: severe memory leak on emacs 24.4.1 Date: Sun, 29 Mar 2015 18:35:42 +0300 Message-ID: <83mw2vzui9.fsf@gnu.org> References: Reply-To: Eli Zaretskii NNTP-Posting-Host: plane.gmane.org X-Trace: ger.gmane.org 1427643447 7453 80.91.229.3 (29 Mar 2015 15:37:27 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Sun, 29 Mar 2015 15:37:27 +0000 (UTC) Cc: 20220@debbugs.gnu.org To: Mario Valencia Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Sun Mar 29 17:37:16 2015 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1YcFGy-0003D7-2g for geb-bug-gnu-emacs@m.gmane.org; Sun, 29 Mar 2015 17:37:12 +0200 Original-Received: from localhost ([::1]:57354 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YcFGw-0005ri-QU for geb-bug-gnu-emacs@m.gmane.org; Sun, 29 Mar 2015 11:37:10 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:35336) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YcFGs-0005nz-2W for bug-gnu-emacs@gnu.org; Sun, 29 Mar 2015 11:37:07 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1YcFGo-0002d9-Sy for bug-gnu-emacs@gnu.org; Sun, 29 Mar 2015 11:37:06 -0400 Original-Received: from debbugs.gnu.org ([140.186.70.43]:49967) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YcFGo-0002d3-Pl for bug-gnu-emacs@gnu.org; Sun, 29 Mar 2015 11:37:02 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.80) (envelope-from ) id 1YcFGo-0008Pp-Cm for bug-gnu-emacs@gnu.org; Sun, 29 Mar 2015 11:37:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Eli Zaretskii Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sun, 29 Mar 2015 15:37:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 20220 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: Original-Received: via spool by 20220-submit@debbugs.gnu.org id=B20220.142764336832285 (code B ref 20220); Sun, 29 Mar 2015 15:37:02 +0000 Original-Received: (at 20220) by debbugs.gnu.org; 29 Mar 2015 15:36:08 +0000 Original-Received: from localhost ([127.0.0.1]:39743 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YcFFv-0008Of-Ri for submit@debbugs.gnu.org; Sun, 29 Mar 2015 11:36:08 -0400 Original-Received: from mtaout24.012.net.il ([80.179.55.180]:35350) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1YcFFs-0008O8-UR for 20220@debbugs.gnu.org; Sun, 29 Mar 2015 11:36:06 -0400 Original-Received: from conversion-daemon.mtaout24.012.net.il by mtaout24.012.net.il (HyperSendmail v2007.08) id <0NLZ00M00C9RYE00@mtaout24.012.net.il> for 20220@debbugs.gnu.org; Sun, 29 Mar 2015 18:27:38 +0300 (IDT) Original-Received: from HOME-C4E4A596F7 ([87.69.4.28]) by mtaout24.012.net.il (HyperSendmail v2007.08) with ESMTPA id <0NLZ00KOBCA22B30@mtaout24.012.net.il>; Sun, 29 Mar 2015 18:27:38 +0300 (IDT) In-reply-to: X-012-Sender: halo1@inter.net.il X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 140.186.70.43 X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.bugs:101022 Archived-At: > Date: Sat, 28 Mar 2015 17:39:38 -0600 > From: Mario Valencia > > to reproduce it, i create a small emtpy html document. then i evaluate the > following expression: > > (dotimes (i 100) (browse-url-of-file)) > > This causes emacs to start opening the file using google chrome. In the task > manager, i can see emacs' memory usage go up by about 5 megabytes each time a > tab is opened. When it opens about 30 tabs, emacs is using 138 megabytes of > memory, and it gives the error below. > the translation is something like this: "ShellExecute failed: Storage space > insufficient to process this command" > My harddrive has enough storage space btw. (The error is not about disk storage, it's about reserving the address space in virtual memory.) FWIW, I don't see such a large memory increase when I reproduce this, I see something around 1MB, sometimes 1.5MB. But I didn't try on Windows 8. Anyway, Emacs doesn't allocate any memory when it calls the ShellExecute function, so I see no way we could leak something here. My guess would be that invoking ShellExecute causes Windows to start a thread in the context of the Emacs process, and reserve 8MB of stack space for that thread. (On one of 3 systems I tried your recipe, I actually saw a thread per invocation, each thread was running some function inside shlwapi.dll, the shared library which implements the ShellExecute API.) The memory actually used by that thread for its stack is much smaller than 8MB, of course, but Windows attempts to reserve 8MB of address space for its stack. The 8MB figure comes from the way we link Emacs: we need such a large stack due to regular expressions, stack-based Lisp objects, and GC which is deeply recursive. By default, each thread reserves the same stack space as the program to which it belongs. For threads we launch in Emacs, we override the default 8MB stack space by a much smaller value, but we have no such control on threads that Windows itself starts on behalf of the Emacs process. The error message and the failure to launch too many browser tabs are caused by the fact that Emacs itself reserves about 1.7GB of the address space for its memory management, leaving only a small portion of the 2GB address space that 32-bit programs can use on Windows. Start enough threads with 8MB stack reservation, and you will run out of address space. Emacs 25 uses a different memory allocation scheme for buffers and strings, which allows to avoid the large memory reservation, so at least the out-of-memory error should happen there only after many more ShellExecute invocations (because threads started by Windows on behalf of ShellExecute will still reserve 8MB of memory for their stack). If someone knows how to force threads started by Windows to reserve less memory, without also lowering the stack space available to Emacs itself (i.e. its main thread), please tell. Otherwise, I guess we will have to live with this limitation on Windows.