bug#20220: severe memory leak on emacs 24.4.1

[Mario Valencia <mariovalspi@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 3414 bytes --]

Then the shell execute function is worthless. I had used it for opening the
browser and also for opening files with an external program, or to open
them in the explorer, i guess i will have to remove all use of that
function from my scripts.

2015-03-29 9:35 GMT-06:00 Eli Zaretskii <eliz@gnu.org>:

> > Date: Sat, 28 Mar 2015 17:39:38 -0600
> > From: Mario Valencia <mariovalspi@gmail.com>
> >
> > to reproduce it, i create a small emtpy html document. then i evaluate
> the
> > following expression:
> >
> > (dotimes (i 100) (browse-url-of-file))
> >
> > This causes emacs to start opening the file using google chrome. In the
> task
> > manager, i can see emacs' memory usage go up by about 5 megabytes each
> time a
> > tab is opened. When it opens about 30 tabs, emacs is using 138 megabytes
> of
> > memory, and it gives the error below.
> > the translation is something like this: "ShellExecute failed: Storage
> space
> > insufficient to process this command"
> > My harddrive has enough storage space btw.
>
> (The error is not about disk storage, it's about reserving the address
> space in virtual memory.)
>
> FWIW, I don't see such a large memory increase when I reproduce this,
> I see something around 1MB, sometimes 1.5MB.  But I didn't try on
> Windows 8.
>
> Anyway, Emacs doesn't allocate any memory when it calls the
> ShellExecute function, so I see no way we could leak something here.
>
> My guess would be that invoking ShellExecute causes Windows to start a
> thread in the context of the Emacs process, and reserve 8MB of stack
> space for that thread.  (On one of 3 systems I tried your recipe, I
> actually saw a thread per invocation, each thread was running some
> function inside shlwapi.dll, the shared library which implements the
> ShellExecute API.)  The memory actually used by that thread for its
> stack is much smaller than 8MB, of course, but Windows attempts to
> reserve 8MB of address space for its stack.
>
> The 8MB figure comes from the way we link Emacs: we need such a large
> stack due to regular expressions, stack-based Lisp objects, and GC
> which is deeply recursive.  By default, each thread reserves the same
> stack space as the program to which it belongs.  For threads we launch
> in Emacs, we override the default 8MB stack space by a much smaller
> value, but we have no such control on threads that Windows itself
> starts on behalf of the Emacs process.
>
> The error message and the failure to launch too many browser tabs are
> caused by the fact that Emacs itself reserves about 1.7GB of the
> address space for its memory management, leaving only a small portion
> of the 2GB address space that 32-bit programs can use on Windows.
> Start enough threads with 8MB stack reservation, and you will run out
> of address space.
>
> Emacs 25 uses a different memory allocation scheme for buffers and
> strings, which allows to avoid the large memory reservation, so at
> least the out-of-memory error should happen there only after many more
> ShellExecute invocations (because threads started by Windows on behalf
> of ShellExecute will still reserve 8MB of memory for their stack).
>
> If someone knows how to force threads started by Windows to reserve
> less memory, without also lowering the stack space available to Emacs
> itself (i.e. its main thread), please tell.  Otherwise, I guess we
> will have to live with this limitation on Windows.
>
>

[-- Attachment #2: Type: text/html, Size: 4033 bytes --]