From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Ken Brown Newsgroups: gmane.emacs.bugs Subject: bug#9273: 23.3; malloc initialization should (sometimes) happen at runtime Date: Wed, 10 Aug 2011 14:49:51 -0400 Message-ID: <4E42D2CF.1080200@cornell.edu> References: <4E41940C.2010605@cornell.edu> <83zkjhxnty.fsf@gnu.org> <4E42C567.2000303@cornell.edu> <83k4alxhlv.fsf@gnu.org> NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Trace: dough.gmane.org 1313002265 32507 80.91.229.12 (10 Aug 2011 18:51:05 GMT) X-Complaints-To: usenet@dough.gmane.org NNTP-Posting-Date: Wed, 10 Aug 2011 18:51:05 +0000 (UTC) Cc: "9273@debbugs.gnu.org" <9273@debbugs.gnu.org> To: Eli Zaretskii Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Wed Aug 10 20:50:58 2011 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([140.186.70.17]) by lo.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1QrDrm-0002bW-4A for geb-bug-gnu-emacs@m.gmane.org; Wed, 10 Aug 2011 20:50:58 +0200 Original-Received: from localhost ([::1]:49076 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1QrDrl-0007Bn-HB for geb-bug-gnu-emacs@m.gmane.org; Wed, 10 Aug 2011 14:50:57 -0400 Original-Received: from eggs.gnu.org ([140.186.70.92]:48189) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1QrDrf-0007B4-On for bug-gnu-emacs@gnu.org; Wed, 10 Aug 2011 14:50:55 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1QrDrb-00041U-FK for bug-gnu-emacs@gnu.org; Wed, 10 Aug 2011 14:50:51 -0400 Original-Received: from debbugs.gnu.org ([140.186.70.43]:35137) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1QrDrb-00041Q-BJ for bug-gnu-emacs@gnu.org; Wed, 10 Aug 2011 14:50:47 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.69) (envelope-from ) id 1QrDsn-000380-Mv; Wed, 10 Aug 2011 14:52:01 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Ken Brown Original-Sender: debbugs-submit-bounces@debbugs.gnu.org Resent-To: owner@debbugs.gnu.org Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Wed, 10 Aug 2011 18:52:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 9273 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: Original-Received: via spool by 9273-submit@debbugs.gnu.org id=B9273.131300227111961 (code B ref 9273); Wed, 10 Aug 2011 18:52:01 +0000 Original-Received: (at 9273) by debbugs.gnu.org; 10 Aug 2011 18:51:11 +0000 Original-Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1QrDry-00036r-Nd for submit@debbugs.gnu.org; Wed, 10 Aug 2011 14:51:11 -0400 Original-Received: from granite1.mail.cornell.edu ([128.253.83.141] helo=authusersmtp.mail.cornell.edu) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1QrDru-00036h-CU for 9273@debbugs.gnu.org; Wed, 10 Aug 2011 14:51:08 -0400 Original-Received: from [128.84.234.236] (dhcp236.math.cornell.edu [128.84.234.236]) (authenticated bits=0) by authusersmtp.mail.cornell.edu (8.14.4/8.12.10) with ESMTP id p7AInowj019512 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=NOT); Wed, 10 Aug 2011 14:49:50 -0400 (EDT) User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.18) Gecko/20110616 Thunderbird/3.1.11 In-Reply-To: <83k4alxhlv.fsf@gnu.org> X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list Resent-Date: Wed, 10 Aug 2011 14:52:01 -0400 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 1) X-Received-From: 140.186.70.43 X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.bugs:49992 Archived-At: On 8/10/2011 2:10 PM, Eli Zaretskii wrote: >> Date: Wed, 10 Aug 2011 13:52:39 -0400 >> From: Ken Brown >> CC: "9273@debbugs.gnu.org"<9273@debbugs.gnu.org> >> >> On 8/10/2011 11:56 AM, Eli Zaretskii wrote: >>>> Date: Tue, 09 Aug 2011 16:09:48 -0400 >>>> From: Ken Brown >>> >>>> But when the dumped emacs is run, it uses Cygwin's sbrk, which >>>> allocates memory on a heap that won't (as of Cygwin 1.7.10) be >>>> contiguous with the static heap. The saved value of _heapbase, >>>> which points into the static heap, is never changed, but it will >>>> mess up later calculations as soon as sbrk is called for the first >>>> time. >>> >>> Are you sure this is all that's at work here? AFAIR, gmalloc does >>> have code to cope with non-contiguous memory regions returned by sbrk. >> >> The issue isn't that sbrk returns non-contiguous regions. The issue is >> that two different of sbrk are used. One is used when temacs is >> running, and a different one is used when the dumped emacs.exe is >> running. > > I still don't see the problem: the memory sbrk'ed before dumping is > frozen in the dumped Emacs, so I don't see how that could matter. > Perhaps I'm missing something. The memory sbrk'ed before dumping is in the static heap, which is somewhere in relatively low memory. All the variables that malloc uses for keeping track of this involve these low addresses. But when the dumped emacs is run, Cygwin's sbrk is called, and it returns addresses starting at wherever Cygwin decides to put the heap (which will be either 0x20000000 or 0x80000000 in Cygwin 1.7.10, depending on whether or not large address awareness is enabled for emacs.exe and is supported by the underlying Windows system). The calculations done in gmalloc.c are based on the assumption that the heap starts in the same place in the dumped executable as it did before dumping. See especially the BLOCK and ADDRESS macros, which use the _heapbase variable. But _heapbase was set before dumping, and it points somewhere in the static heap; this is now much lower than the beginning of the runtime heap. The specific problem that led me to notice this was that under some circumstances emacs went into an infinite loop when executing the following (from morecore_nolock in gmalloc.c): newsize = heapsize; do newsize *= 2; while ((__malloc_size_t) BLOCK ((char *) result + size) > newsize); Here `result' is very large, and BLOCK returns a large number because it's using a small _heapbase. So the test is always true, newsize becomes 0 because of overflow, and the loop never terminates. Aside from the infinite loop, however, BLOCK and ADDRESS simply yield results that don't make sense when the heap starts in high memory but _heapbase points to low memory. Surprisingly, I haven't yet run into any problems when Cygwin's heap starts at 0x20000000. (It was 0x80000000 in the situation above.) I don't know if there's a good reason for this or if it's just luck. I think what happens is that malloc behaves as if it's allowed to allocate memory ranging all the way from the static heap to 0x20000000 and beyond. If for some reason it really is legal for malloc to use the memory between the static heap and 0x20000000, then I guess there's no harm done as long as the large addresses don't lead to overflow. Ken