From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Eli Zaretskii Newsgroups: gmane.emacs.devel Subject: Re: "resource temporarily unavailable" errors on windows 7 Date: Tue, 13 Mar 2012 21:34:14 +0200 Message-ID: <83obs0wb0p.fsf@gnu.org> References: <83aa3lybos.fsf@gnu.org> <83vcm9w3th.fsf@gnu.org> Reply-To: Eli Zaretskii NNTP-Posting-Host: plane.gmane.org X-Trace: dough.gmane.org 1331667265 24139 80.91.229.3 (13 Mar 2012 19:34:25 GMT) X-Complaints-To: usenet@dough.gmane.org NNTP-Posting-Date: Tue, 13 Mar 2012 19:34:25 +0000 (UTC) Cc: emacs-devel@gnu.org To: Alex Harsanyi Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Tue Mar 13 20:34:24 2012 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1S7XUB-0000IO-Lz for ged-emacs-devel@m.gmane.org; Tue, 13 Mar 2012 20:34:19 +0100 Original-Received: from localhost ([::1]:40372 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1S7XUA-00070w-OE for ged-emacs-devel@m.gmane.org; Tue, 13 Mar 2012 15:34:18 -0400 Original-Received: from eggs.gnu.org ([208.118.235.92]:50094) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1S7XU7-00070E-7d for emacs-devel@gnu.org; Tue, 13 Mar 2012 15:34:16 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1S7XTz-0001rW-ML for emacs-devel@gnu.org; Tue, 13 Mar 2012 15:34:14 -0400 Original-Received: from mtaout22.012.net.il ([80.179.55.172]:46560) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1S7XTz-0001r4-8f for emacs-devel@gnu.org; Tue, 13 Mar 2012 15:34:07 -0400 Original-Received: from conversion-daemon.a-mtaout22.012.net.il by a-mtaout22.012.net.il (HyperSendmail v2007.08) id <0M0U00E007RATR00@a-mtaout22.012.net.il> for emacs-devel@gnu.org; Tue, 13 Mar 2012 21:34:05 +0200 (IST) Original-Received: from HOME-C4E4A596F7 ([77.124.179.236]) by a-mtaout22.012.net.il (HyperSendmail v2007.08) with ESMTPA id <0M0U00DKI90QSAO0@a-mtaout22.012.net.il>; Tue, 13 Mar 2012 21:34:04 +0200 (IST) In-reply-to: X-012-Sender: halo1@inter.net.il X-detected-operating-system: by eggs.gnu.org: Solaris 10 (beta) X-Received-From: 80.179.55.172 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.devel:149012 Archived-At: > Date: Tue, 13 Mar 2012 12:50:09 +0800 > From: Alex Harsanyi > Cc: emacs-devel@gnu.org > > 2012/3/13 Eli Zaretskii : > > [Please keep the list on the CC.] > > > >> Date: Tue, 13 Mar 2012 09:31:18 +0800 > >> From: Alex Harsanyi > >> > >> Just a quick update: I modified allocate_heap() to start asking for > >> 1.5GB initially and that also seemed to fix the subprocess creation > >> problem. > > > > What value exactly did you use? > > unsigned long size = 0x60000000; /* start by asking for 1.5GB */ Is this the largest value that gives you a fully functional Emacs? Or can you enlarge it some more, before subprocesses start failing? I'm not happy reducing the maximum memory possible by 0.5GB, especially since no one else complained about this yet, including people who run Emacs on Windows 7. (But I still hope that we can find a better solution than just reducing the memory request; see below. So this is for now a fallback.) > Since I have more memory available than Emacs requests, I don't think > enlarging the amount of virtual memory would help. Starting a new > Emacs session does not seem to affect the amount of free memory > available significantly: my Emacs session uses 25Mb of physical memory > after start-up. My main Emacs session (where I do my work) currently > uses 63Mb, not a very large value. The 1.5GB "reservation" by > allocate_heap() does not decrease the amount of available virtual > memory. OK, good. This seems to indicate that the problem is with the memory of the Emacs process itself, not with the memory for other processes, such as the subprocess you are trying to launch. Most probably, something we do during spawning the subprocess tries to reserve memory, similarly to what allocate_heap does, and fails because the sum of what's already reserved and this new reservation exceeds 2GB. > Perhaps I was a bit imprecise with the terminology, but I did realise > that allocate_heap() only reserves the memory and that the 2GB address > space limit is per process, not per system. Sorry, I couldn't be certain that you understand all the details about that function. > There seems to be some documentation on the net that ANSI > CreateProcess will allocate memory to convert the command line to > UNICODE before calling the UNICODE version of CreateProcess, this is > what I meant by "CreateProcess fails to allocate memory". Could this > memory allocation fail if all the memory is already reserved for > something else? It could in theory, but I very much doubt that conversion of any reasonable command line to UTF-16 could ever need so much memory (0.2GB, the difference between the 1.8GB allocation you reported and the 2GB limit). Anyway, we have no evidence that the error comes from the CreateProcess API. On the contrary (see below), I think we have evidence that it comes from elsewhere. I thought about this some more, read some stuff on MSDN and elsewhere, and I have the following questions and observations. First, the error message. It is triggered by this code in w32proc.c:sys_spawnve: cp = new_child (); if (cp == NULL) { errno = EAGAIN; return -1; } Now, in new_child we see this: cp->char_avail = CreateEvent (NULL, TRUE, FALSE, NULL); if (cp->char_avail) { cp->char_consumed = CreateEvent (NULL, FALSE, FALSE, NULL); if (cp->char_consumed) { cp->thrd = CreateThread (NULL, 1024, reader_thread, cp, 0, &id); if (cp->thrd) return cp; } } delete_child (cp); return NULL; } So this can happen due to error either in one of the two calls to CreateEvent or in the call to CreateThread. My crystal ball bets on the latter, but could you please verify that this is the problem? If it is, please add a call to GetLastError there and see which error code is returned. Assuming it's CreateThread that fails, the obvious suspicion would be that it fails because creating a thread reserves some memory for its stack. The call above is careful to override the amount of stack memory that is actually _committed_ and sets it at a mere 1KB, but the memory _reserved_ for the stack is still 1MB, according to MSDN. Question: does the problem happen right when you invoke the first subprocess, or only after several subprocesses are already running, and you try to invoke another one? Also, can you use the VMMap utility (from Sysinternals) and tell what it shows in the "Private Data", "Thread Stack", and "Free" categories, when you run Emacs with the original code in allocate_heap, after you get the error message? Can you spot, in particular, how much memory is reserved and committed for Emacs and for the additional thread it starts in new_child? TIA