From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Ken Brown Newsgroups: gmane.emacs.bugs Subject: bug#9273: 23.3; malloc initialization should (sometimes) happen at runtime Date: Sat, 13 Aug 2011 23:13:39 -0400 Message-ID: <4E473D63.7040703@cornell.edu> References: <4E41940C.2010605@cornell.edu> <83zkjhxnty.fsf@gnu.org> <4E444D85.4090907@cornell.edu> <838vqzxgq1.fsf@gnu.org> <4E44FC1B.9010207@cornell.edu> <831uwqyidi.fsf@gnu.org> <4E451A1F.4060409@cornell.edu> <4E458BF4.9080108@cornell.edu> <83fwl5wxbz.fsf@gnu.org> <4E4680C4.6040605@cornell.edu> <83d3g9weza.fsf@gnu.org> <4E468FE5.100@cornell.edu> <4E469967.40201@cornell.edu> NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Trace: dough.gmane.org 1313291677 11270 80.91.229.12 (14 Aug 2011 03:14:37 GMT) X-Complaints-To: usenet@dough.gmane.org NNTP-Posting-Date: Sun, 14 Aug 2011 03:14:37 +0000 (UTC) Cc: "9273@debbugs.gnu.org" <9273@debbugs.gnu.org> To: Stefan Monnier Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Sun Aug 14 05:14:33 2011 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([140.186.70.17]) by lo.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1QsR9k-0000Jp-7G for geb-bug-gnu-emacs@m.gmane.org; Sun, 14 Aug 2011 05:14:32 +0200 Original-Received: from localhost ([::1]:44560 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1QsR9j-0004Fj-ND for geb-bug-gnu-emacs@m.gmane.org; Sat, 13 Aug 2011 23:14:31 -0400 Original-Received: from eggs.gnu.org ([140.186.70.92]:49270) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1QsR9h-0004Fd-KY for bug-gnu-emacs@gnu.org; Sat, 13 Aug 2011 23:14:30 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1QsR9g-0003yV-GJ for bug-gnu-emacs@gnu.org; Sat, 13 Aug 2011 23:14:29 -0400 Original-Received: from debbugs.gnu.org ([140.186.70.43]:39733) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1QsR9g-0003yR-Ek for bug-gnu-emacs@gnu.org; Sat, 13 Aug 2011 23:14:28 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.69) (envelope-from ) id 1QsRBB-0000DY-OS; Sat, 13 Aug 2011 23:16:01 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Ken Brown Original-Sender: debbugs-submit-bounces@debbugs.gnu.org Resent-To: owner@debbugs.gnu.org Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sun, 14 Aug 2011 03:16:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 9273 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: Original-Received: via spool by 9273-submit@debbugs.gnu.org id=B9273.1313291737805 (code B ref 9273); Sun, 14 Aug 2011 03:16:01 +0000 Original-Received: (at 9273) by debbugs.gnu.org; 14 Aug 2011 03:15:37 +0000 Original-Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1QsRAm-0000Cv-VE for submit@debbugs.gnu.org; Sat, 13 Aug 2011 23:15:37 -0400 Original-Received: from granite1.mail.cornell.edu ([128.253.83.141] helo=authusersmtp.mail.cornell.edu) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1QsRAj-0000Cn-E5 for 9273@debbugs.gnu.org; Sat, 13 Aug 2011 23:15:35 -0400 Original-Received: from [172.160.101.188] (fl-184-7-236-177.sta.embarqhsd.net [184.7.236.177]) (authenticated bits=0) by authusersmtp.mail.cornell.edu (8.14.4/8.12.10) with ESMTP id p7E3Dt2Y028850 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Sat, 13 Aug 2011 23:13:57 -0400 (EDT) User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:5.0) Gecko/20110624 Thunderbird/5.0 In-Reply-To: X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list Resent-Date: Sat, 13 Aug 2011 23:16:01 -0400 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 1) X-Received-From: 140.186.70.43 X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.bugs:50056 Archived-At: On 8/13/2011 3:19 PM, Stefan Monnier wrote: >> Probably. That would keep me from having to get involved in the details of >> how _heapinfo works, and it would probably be less error prone. So I would >> do something like the following: > >> 1. Temporarily restore the pre-dump malloc state. >> 2. Call xrealloc, getting storage of the new size in the old heap. >> 3. Copy the contents into a temporary buffer, and then free the recently >> allocated storage in the old heap. >> 4. Restore the malloc state, call malloc, and then copy the contents of the >> temporary buffer into the just-acquired storage in the new heap. > >> Is that what you had in mind? > > No, I meant, during initialization of Emacs, make sure we get the old > _heapinfo state, so that malloc/realloc/... just works. No, that wouldn't be good because the old _heapinfo state uses the static heap, which is very small. It's big enough for temacs, but not for a usable emacs. In any case, Eli's suggestion turned out to be very easy to implement. My revised patch is appended below, and it seems to solve all the problems I encountered. I still want to test it further, but I think it's OK. Eli, thanks for your help. Ken === modified file 'src/gmalloc.c' --- src/gmalloc.c 2011-08-04 17:04:39 +0000 +++ src/gmalloc.c 2011-08-13 22:32:45 +0000 @@ -351,11 +351,19 @@ #endif #include -/* How to really get more memory. */ -#if defined(CYGWIN) +/* On Cygwin there are two heaps. temacs uses the static heap + (defined in sheap.c and managed with bss_sbrk), and the dumped + emacs uses the Cygwin heap (managed with sbrk). When emacs starts + on Cygwin, it reinitializes malloc, and we save the old info for + use by free and realloc if they're called with a pointer into the + static heap. */ +#ifdef CYGWIN extern __ptr_t bss_sbrk PP ((ptrdiff_t __size)); extern int bss_sbrk_did_unexec; +char *bss_sbrk_heapbase; +malloc_info *bss_sbrk_heapinfo; #endif + __ptr_t (*__morecore) PP ((__malloc_ptrdiff_t __size)) = __default_morecore; /* Debugging hook for `malloc'. */ @@ -584,6 +592,16 @@ mcheck (NULL); #endif +#ifdef CYGWIN + if (bss_sbrk_did_unexec) + /* we're reinitializing the dumped emacs */ + { + bss_sbrk_heapbase = _heapbase; + bss_sbrk_heapinfo = _heapinfo; + memset (_fraghead, 0, BLOCKLOG * sizeof (struct list)); + } +#endif + if (__malloc_initialize_hook) (*__malloc_initialize_hook) (); @@ -1054,6 +1072,12 @@ if (ptr == NULL) return; +#ifdef CYGWIN + if (ptr < _heapbase) + /* We're being asked to free something in the static heap. */ + return; +#endif + PROTECT_MALLOC_STATE (0); LOCK_ALIGNED_BLOCKS (); @@ -1346,9 +1370,33 @@ #include #endif - #define min(A, B) ((A) < (B) ? (A) : (B)) +/* On Cygwin the dumped emacs may try to realloc storage allocated in + the static heap. We just malloc space in the new heap and copy the + data. */ +#ifdef CYGWIN +__ptr_t +special_realloc (ptr, size) + __ptr_t ptr; + __malloc_size_t size; +{ + __ptr_t result; + int type; + __malloc_size_t block, oldsize; + + block = ((char *) ptr - bss_sbrk_heapbase) / BLOCKSIZE + 1; + type = bss_sbrk_heapinfo[block].busy.type; + oldsize = + type == 0 ? bss_sbrk_heapinfo[block].busy.info.size * BLOCKSIZE + : (__malloc_size_t) 1 << type; + result = _malloc_internal_nolock (size); + if (result != NULL) + memcpy (result, ptr, min (oldsize, size)); + return result; +} +#endif + /* Debugging hook for realloc. */ __ptr_t (*__realloc_hook) PP ((__ptr_t __ptr, __malloc_size_t __size)); @@ -1375,6 +1423,12 @@ else if (ptr == NULL) return _malloc_internal_nolock (size); +#ifdef CYGWIN + if (ptr < _heapbase) + /* ptr points into the static heap */ + return special_realloc (ptr, size); +#endif + block = BLOCK (ptr); PROTECT_MALLOC_STATE (0); === modified file 'src/unexcw.c' --- src/unexcw.c 2011-03-17 20:18:59 +0000 +++ src/unexcw.c 2011-08-12 20:10:03 +0000 @@ -33,6 +33,8 @@ extern int bss_sbrk_did_unexec; +extern int __malloc_initialized; + /* emacs symbols that indicate where bss and data end for emacs internals */ extern char my_endbss[]; extern char my_edata[]; @@ -210,9 +212,12 @@ lseek (fd, (long) (exe_header->section_header[i].s_scnptr), SEEK_SET); assert (ret != -1); + /* force the dumped emacs to reinitialize malloc */ + __malloc_initialized = 0; ret = write (fd, (char *) start_address, my_endbss - (char *) start_address); + __malloc_initialized = 1; assert (ret == (my_endbss - (char *) start_address)); if (debug_unexcw) printf (" .bss, mem start 0x%08x mem length %d\n",