bug#18505: 24.3.93; intermittent unexec failures when building on Mac OS X 10.10 beta, Xcode 6.0

[David Caldwell <david@porkrind.org>]

[-- Attachment #1: Type: text/plain, Size: 2924 bytes --]

On 9/21/14 2:15 AM, Jan Djärv wrote:
> Hello.
> 
> 20 sep 2014 kl. 20:31 skrev David Caldwell <david@porkrind.org>:
> 
>> On 9/20/14 8:31 AM, Jan Djärv wrote:
>>> Hello.
>>>
>>> 19 sep 2014 kl. 06:13 skrev David Caldwell <david@porkrind.org>:
>>>
>>>> Hello,
>>>>
>>>> I tried to build the latest pretest on Mac OS X Yosemite Beta with the
>>>> new Xcode 6.0 (GM) tools and ran into this error during the unexec step:
>>>>
>>>> unexec: not enough room for load commands for new __DATA segments
>>>
>>> Does it happen all the time or just some times?
>>
>> It depends on 2 variables: the number of load commands that need to be
>> added (num_unexec_regions) and text_seg_lowest_offset.
>>
>> num_unexec_regions jumps around a lot, doing "make clean && make" over
>> and over it'll be different every time. Somewhere between 12 and 34.
> 
> What makes it do that?  Some address randomization?  Some other unknown bug?
> I would expect num_unexec_regions to be the same for every make.

I don't know. I would also expect num_unexec_regions to be the same. If
it changes, it seems to mean the malloc behavior is different on every
run. But yes, perhaps address space randomization could cause that to
happen. I don't understand that part of the code well enough to
speculate too much.

> text_seg_lowest_offset could be address randomization, but if it stays somewhat constant, that can't be it.

I just figured that out (and smacked my head because it was obvious).
That changed when I changed headerpad. I got curious and did a binary
search to figure out exactly how -headerpad affects
text_seg_lowest_offset in my setup (all number hex):

 -headerpad      text_seg_lowest_offset
   0 ->  740     17a0
 741 -> 1740     27a0
1741 -> ???      37a0

So, text_seg_lowest_offset directly correlate with -headerpad and ld is
doing some sort of alignment.

> I've seen this failure before, but usually a new make works.
> I'm trying to decide if this is emacs 24 or trunk material.

I think it should go in both. It's really quite a low-risk change: the
-headerpad option is well documented in ld, and the amount my patch adds
gives an extra 1.5K of headroom on a 6M binary (.02%).

I did a bunch of 'bzr log' searches to understand the nature of the
-headerpad setting and it appears to not have been touched since 2006
(in the 32 bit era). I believe that is why the comment in configure is
incorrect: load commands may have been 56 bytes on 32 bit archs, but
they are 78 bytes on my 64 bit computer (which is all current Macs going
forward).

> Is there a way to dynamically react to these changes and adjust headerpad_extra dynamically at dump time?

Unfortunately the -headerpad is specified during link time so to change
it dynamically would require re-linking after unexec-ing. That's a large
Makefile change to get that all working correctly.

-David


[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4219 bytes --]