bug#9273: 23.3; malloc initialization should (sometimes) happen at runtime

[Ken Brown <kbrown@cornell.edu>]

On 8/10/2011 2:10 PM, Eli Zaretskii wrote:
>> Date: Wed, 10 Aug 2011 13:52:39 -0400
>> From: Ken Brown<kbrown@cornell.edu>
>> CC: "9273@debbugs.gnu.org"<9273@debbugs.gnu.org>
>>
>> On 8/10/2011 11:56 AM, Eli Zaretskii wrote:
>>>> Date: Tue, 09 Aug 2011 16:09:48 -0400
>>>> From: Ken Brown<kbrown@cornell.edu>
>>>
>>>> But when the dumped emacs is run, it uses Cygwin's sbrk, which
>>>> allocates memory on a heap that won't (as of Cygwin 1.7.10) be
>>>> contiguous with the static heap.  The saved value of _heapbase,
>>>> which points into the static heap, is never changed, but it will
>>>> mess up later calculations as soon as sbrk is called for the first
>>>> time.
>>>
>>> Are you sure this is all that's at work here?  AFAIR, gmalloc does
>>> have code to cope with non-contiguous memory regions returned by sbrk.
>>
>> The issue isn't that sbrk returns non-contiguous regions.  The issue is
>> that two different of sbrk are used.  One is used when temacs is
>> running, and a different one is used when the dumped emacs.exe is
>> running.
>
> I still don't see the problem: the memory sbrk'ed before dumping is
> frozen in the dumped Emacs, so I don't see how that could matter.
> Perhaps I'm missing something.

The memory sbrk'ed before dumping is in the static heap, which is 
somewhere in relatively low memory.  All the variables that malloc uses 
for keeping track of this involve these low addresses.  But when the 
dumped emacs is run, Cygwin's sbrk is called, and it returns addresses 
starting at wherever Cygwin decides to put the heap (which will be 
either 0x20000000 or 0x80000000 in Cygwin 1.7.10, depending on whether 
or not large address awareness is enabled for emacs.exe and is supported 
by the underlying Windows system).

The calculations done in gmalloc.c are based on the assumption that the 
heap starts in the same place in the dumped executable as it did before 
dumping.  See especially the BLOCK and ADDRESS macros, which use the 
_heapbase variable.  But _heapbase was set before dumping, and it points 
somewhere in the static heap; this is now much lower than the beginning 
of the runtime heap.

The specific problem that led me to notice this was that under some 
circumstances emacs went into an infinite loop when executing the 
following (from morecore_nolock in gmalloc.c):

newsize = heapsize;
do
   newsize *= 2;
while ((__malloc_size_t) BLOCK ((char *) result + size) > newsize);

Here `result' is very large, and BLOCK returns a large number because 
it's using a small _heapbase.  So the test is always true, newsize 
becomes 0 because of overflow, and the loop never terminates.

Aside from the infinite loop, however, BLOCK and ADDRESS simply yield 
results that don't make sense when the heap starts in high memory but 
_heapbase points to low memory.

Surprisingly, I haven't yet run into any problems when Cygwin's heap 
starts at 0x20000000.  (It was 0x80000000 in the situation above.)  I 
don't know if there's a good reason for this or if it's just luck.  I 
think what happens is that malloc behaves as if it's allowed to allocate 
memory ranging all the way from the static heap to 0x20000000 and 
beyond.  If for some reason it really is legal for malloc to use the 
memory between the static heap and 0x20000000, then I guess there's no 
harm done as long as the large addresses don't lead to overflow.

Ken