* bug#18505: 24.3.93; intermittent unexec failures when building on Mac OS X 10.10 beta, Xcode 6.0
@ 2014-09-19 4:13 David Caldwell
2014-09-20 15:31 ` Jan Djärv
0 siblings, 1 reply; 6+ messages in thread
From: David Caldwell @ 2014-09-19 4:13 UTC (permalink / raw)
To: 18505
[-- Attachment #1.1: Type: text/plain, Size: 1545 bytes --]
Hello,
I tried to build the latest pretest on Mac OS X Yosemite Beta with the
new Xcode 6.0 (GM) tools and ran into this error during the unexec step:
unexec: not enough room for load commands for new __DATA segments
(Full unexec log is attached).
Poking around I discovered a couple discrepancies. The "headerpad_extra"
setting in configure.ac says entries are 56 bytes, but they are actually
78 bytes (probably because the comment was originally written in the 32
bit days). Also the number of load commands needed is variable now--it
depends on how many malloc segments were found and how many it could
merge (see find_emacs_zone_regions() and unexec_regions_merge() in
unexmacosx.c).
I printed num_unexec_regions in unexmacosx.c and observed anywhere
from about 14 to 34 regions needed. But given that it seems to be
different on every run and it's super annoying to debug when you run out
of space, I suggest bumping headerpad_extra up to 0x1000. This gives
about 1.5KB of headroom (or wasted space, depending on how you look at
it) and allows for 52 load commands, which should be future proof for
quite a while. I have attached a patch for configure.ac that does this
and changes the comment to reflect my understanding.
I've also provided a patch that makes the unexec error more helpful: it
computes the size needed for headerpad_extra given the number of load
commands that it needed for that particular unexec run.
I assume this affects the trunk code, too, though I didn't look at it.
-David
[-- Attachment #1.2: increase_headerpad_extra.patch --]
[-- Type: text/plain, Size: 1143 bytes --]
--- configure.ac.orig 2014-09-18 18:00:52.000000000 -0700
+++ configure.ac 2014-09-18 20:51:55.000000000 -0700
@@ -4856,17 +4856,19 @@
darwin)
## The -headerpad option tells ld (see man page) to leave room at the
## end of the header for adding load commands. Needed for dumping.
- ## 0x690 is the total size of 30 segment load commands (at 56
- ## each); under Cocoa 31 commands are required.
+ ## 0x1000 is enough for roughly 52 load commands on the x86_64
+ ## architecture (where they are 78 bytes each). The actual number of
+ ## load commands added is not consistent but normally ranges from
+ ## about 14 to about 34. Setting it high gets us plenty of slop and
+ ## only costs about 1.5K of wasted binary space.
+ headerpad_extra=1000
if test "$HAVE_NS" = "yes"; then
libs_nsgui="-framework AppKit"
if test "$NS_IMPL_COCOA" = "yes"; then
libs_nsgui="$libs_nsgui -framework IOKit"
fi
- headerpad_extra=6C8
else
libs_nsgui=
- headerpad_extra=690
fi
LD_SWITCH_SYSTEM_TEMACS="-fno-pie -prebind $libs_nsgui -Xlinker -headerpad -Xlinker $headerpad_extra"
[-- Attachment #1.3: helpful_unexec_error.patch --]
[-- Type: text/plain, Size: 598 bytes --]
--- src/unexmacosx.c.orig 2014-09-18 20:53:21.000000000 -0700
+++ src/unexmacosx.c 2014-09-18 21:10:42.000000000 -0700
@@ -1302,7 +1306,9 @@
}
if (curr_header_offset > text_seg_lowest_offset)
- unexec_error ("not enough room for load commands for new __DATA segments");
+ unexec_error ("not enough room for load commands for new __DATA segments"
+ " (increase headerpad_extra in configure.in to at least %lX)",
+ num_unexec_regions * sizeof (struct segment_command));
printf ("%ld unused bytes follow Mach-O header\n",
text_seg_lowest_offset - curr_header_offset);
[-- Attachment #1.4: unexec-log.txt --]
[-- Type: text/plain, Size: 13359 bytes --]
for file in `find . -type d -print`; do case $file in .*/cedet* | .*/leim* ) ;; *) wins="$wins${wins:+ }$file" ;; esac; done; \
for file in $wins; do \
./../build-aux/update-subdirs $file; \
done;
if test "no" = "yes"; then \
rm -f bootstrap-emacs; \
ln temacs bootstrap-emacs; \
else \
./temacs --batch --load loadup bootstrap || exit 1; \
test "X" = X || -zex emacs; \
mv -f emacs bootstrap-emacs; \
fi
Loading loadup.el (source)...
Using load-path (/var/tmp/emacs-24.3.93/lisp /var/tmp/emacs-24.3.93/lisp/emacs-lisp /var/tmp/emacs-24.3.93/lisp/language /var/tmp/emacs-24.3.93/lisp/international /var/tmp/emacs-24.3.93/lisp/textmodes /var/tmp/emacs-24.3.93/lisp/vc)
Loading emacs-lisp/byte-run...
Loading emacs-lisp/backquote...
Loading subr...
Loading version...
Loading widget...
Loading custom...
Loading emacs-lisp/map-ynp...
Loading international/mule...
Loading international/mule-conf...
Loading env...
Loading format...
Loading bindings...
Loading cus-start...
Loading window...
Loading files...
Loading emacs-lisp/macroexp...
Loading cus-face...
Loading faces...
Loading button...
Loading startup...
Loading /var/tmp/emacs-24.3.93/lisp/loaddefs.el (source)...
Loading emacs-lisp/nadvice...
Loading minibuffer...
Loading abbrev...
Loading simple...
Loading help...
Loading jka-cmpr-hook...
Loading epa-hook...
Loading international/mule-cmds...
Loading case-table...
Loading /var/tmp/emacs-24.3.93/lisp/international/charprop.el (source)...
Loading international/characters...
Loading composite...
Loading language/chinese...
Loading language/cyrillic...
Loading language/indian...
Loading language/sinhala...
Loading language/english...
Loading language/ethiopic...
Loading language/european...
Loading language/czech...
Loading language/slovak...
Loading language/romanian...
Loading language/greek...
Loading language/hebrew...
Loading language/japanese...
Loading /var/tmp/emacs-24.3.93/lisp/international/cp51932.el (source)...
Loading /var/tmp/emacs-24.3.93/lisp/international/eucjp-ms.el (source)...
Loading language/korean...
Loading language/lao...
Loading language/tai-viet...
Loading language/thai...
Loading language/tibetan...
Loading language/vietnamese...
Loading language/misc-lang...
Loading language/utf-8-lang...
Loading language/georgian...
Loading language/khmer...
Loading language/burmese...
Loading language/cham...
Loading indent...
Loading frame...
Loading term/tty-colors...
Loading font-core...
Loading facemenu...
Loading emacs-lisp/syntax...
Loading font-lock...
Loading jit-lock...
Loading mouse...
Loading scroll-bar...
Loading select...
Loading emacs-lisp/timer...
Loading isearch...
Loading rfn-eshadow...
Loading menu-bar...
Loading emacs-lisp/lisp...
Loading textmodes/page...
Loading register...
Loading textmodes/paragraphs...
Loading progmodes/prog-mode...
Loading emacs-lisp/lisp-mode...
Loading textmodes/text-mode...
Loading textmodes/fill...
Loading newcomment...
Loading replace...
Loading emacs-lisp/tabulated-list...
Loading buff-menu...
Loading fringe...
Loading emacs-lisp/regexp-opt...
Loading image...
Loading international/fontset...
Loading dnd...
Loading tool-bar...
Loading term/common-win...
Loading term/ns-win...
Loading mwheel...
Loading emacs-lisp/float-sup...
Loading vc/vc-hooks...
Loading vc/ediff-hook...
Loading uniquify...
Loading electric...
Loading tooltip...
Loading /var/tmp/emacs-24.3.93/lisp/leim/leim-list.el (source)...
Finding pointers to doc strings...
Finding pointers to doc strings...done
Dumping under the name emacs
--- List of All Regions ---
address size prot maxp
--- List of Regions to be Dumped ---
address size prot maxp
--- Header Information ---
Magic = 0xfeedfacf
CPUType = 16777223
CPUSubType = -2147483645
FileType = 0x2
NCmds = 30
SizeOfCmds = 4160
Flags = 0x00000085
Highest address of load commands in input file: 0x100682000
Lowest offset of all sections in __TEXT segment: 0x17c0
--- List of Load Commands in Input File ---
# cmd cmdsize name address size
0 LC_SEGMENT_64 72 __PAGEZERO 0 0x100000000
1 LC_SEGMENT_64 952 __TEXT 0x100000000 0x20f000
__text 0x1000017c0 0x1db19b
__stubs 0x1001dc95c 0xb4c
__stub_helper 0x1001dd4a8 0x12e4
__const 0x1001de790 0x13334
__cstring 0x1001f1ad0 0x1507a
__objc_methname 0x100206b4a 0x386f
__objc_classname 0x10020a3b9 0x12b
__objc_methtype 0x10020a4e4 0x1579
__gcc_except_tab 0x10020ba60 0x4c
__unwind_info 0x10020baac 0x2b24
__eh_frame 0x10020e5d0 0xa28
2 LC_SEGMENT_64 1512 __DATA 0x10020f000 0x3a7000
__nl_symbol_ptr 0x10020f000 0x10
__got 0x10020f010 0x2c8
__la_symbol_ptr 0x10020f2d8 0xf10
__const 0x1002101f0 0x3b30
__cfstring 0x100213d20 0x8e0
__objc_classlist 0x100214600 0x78
__objc_catlist 0x100214678 0x8
__objc_protolist 0x100214680 0x30
__objc_imageinfo 0x1002146b0 0x8
__objc_const 0x1002146b8 0x3c68
__objc_selrefs 0x100218320 0x1170
__objc_classrefs 0x100219490 0x220
__objc_superrefs 0x1002196b0 0x60
__objc_ivar 0x100219710 0x220
__objc_data 0x100219930 0x4b0
__data 0x100219de0 0x3205bc
__common 0x10053a3a0 0x4179
__bss 0x10053e520 0x76e48
3 LC_SEGMENT_64 72 __LINKEDIT 0x1005b6000 0xcc000
4 LC_DYLD_INFO_ONLY 48
5 LC_SYMTAB 24
6 LC_DYSYMTAB 80
7 LC_LOAD_DYLINKER 32
8 LC_UUID 24
9 LC_VERSION_MIN_MACOSX 16
10 LC_SOURCE_VERSION 16
11 LC_MAIN 24
12 LC_LOAD_DYLIB 88
13 LC_LOAD_DYLIB 88
14 LC_LOAD_DYLIB 96
15 LC_LOAD_DYLIB 96
16 LC_LOAD_DYLIB 56
17 LC_LOAD_DYLIB 56
18 LC_LOAD_DYLIB 88
19 LC_LOAD_DYLIB 56
20 LC_LOAD_DYLIB 48
21 LC_LOAD_DYLIB 112
22 LC_LOAD_DYLIB 104
23 LC_LOAD_DYLIB 104
24 LC_LOAD_DYLIB 96
25 LC_LOAD_DYLIB 96
26 LC_LOAD_DYLIB 56
27 LC_FUNCTION_STARTS 16
28 LC_DATA_IN_CODE 16
29 LC_DYLIB_CODE_SIGN_DRS 16
0x107cfc080 (sz: 0x3f24/ 0x3f28)
0x107c00000 (sz: 0x3515/ 0xfc080)
0x100efc080 (sz: 0x3f24/ 0x3f28)
0x100e00000 (sz: 0x99505/ 0xfc080)
0x103dfc080 (sz: 0x3f24/ 0x3f28)
0x103d00000 (sz: 0x5fc65/ 0xfc080)
0x103ffc080 (sz: 0x3f24/ 0x3f28)
0x103f00000 (sz: 0x44205/ 0xfc080)
0x107bfc080 (sz: 0x3f24/ 0x3f28)
0x107b00000 (sz: 0x69df/ 0xfc080)
0x1078fc080 (sz: 0x3f24/ 0x3f28)
0x107800000 (sz: 0x20875/ 0xfc080)
0x103efc080 (sz: 0x3f24/ 0x3f28)
0x103e00000 (sz: 0x4ae5/ 0xfc080)
0x10e0fc080 (sz: 0x3f24/ 0x3f28)
0x10e000000 (sz: 0x1e5/ 0xfc080)
0x101ff8000 (sz: 0x3658/ 0x7fa0)
0x101800000 (sz: 0x3633fd/0x7f8000)
0x109ff8000 (sz: 0xc4/ 0x7fa0)
0x109800000 (sz: 0x9feb/0x7f8000)
0x1097f8000 (sz: 0x75a/ 0x7fa0)
0x109000000 (sz: 0x737ff/0x7f8000)
0x108ff8000 (sz: 0x1018/ 0x7fa0)
0x108800000 (sz: 0xff405/0x7f8000)
0x1077f8000 (sz: 0x2700/ 0x7fa0)
0x107000000 (sz: 0x26bc58/0x7f8000)
0x105ff8000 (sz: 0xba/ 0x7fa0)
0x105800000 (sz: 0x7a8f/0x7f8000)
0x1057f8000 (sz: 0x1a3e/ 0x7fa0)
0x105000000 (sz: 0x1a17fd/0x7f8000)
0x10dff8000 (sz: 0x6e/ 0x7fa0)
0x10d800000 (sz: 0x49f1/0x7f8000)
0x100dfc000 (sz: 0/ 0x1000)
--- Load Commands written to Output File ---
Writing segment __PAGEZERO @ 0 ( 0/0x100000000 @ 0)
Writing segment __TEXT @ 0 (0x20f000/0x20f000 @ 0x100000000)
Writing segment __DATA @ 0x20f000 (0x3a7000/0x3a7000 @ 0x10020f000)
section __nl_symbol_ptr at 0x20f000 - 0x20f010 (sz: 0x10)
section __got at 0x20f010 - 0x20f2d8 (sz: 0x2c8)
section __la_symbol_ptr at 0x20f2d8 - 0x2101e8 (sz: 0xf10)
section __const at 0x2101f0 - 0x213d20 (sz: 0x3b30)
section __cfstring at 0x213d20 - 0x214600 (sz: 0x8e0)
section __objc_classlist at 0x214600 - 0x214678 (sz: 0x78)
section __objc_catlist at 0x214678 - 0x214680 (sz: 0x8)
section __objc_protolist at 0x214680 - 0x2146b0 (sz: 0x30)
section __objc_imageinfo at 0x2146b0 - 0x2146b8 (sz: 0x8)
section __objc_const at 0x2146b8 - 0x218320 (sz: 0x3c68)
section __objc_selrefs at 0x218320 - 0x219490 (sz: 0x1170)
section __objc_classrefs at 0x219490 - 0x2196b0 (sz: 0x220)
section __objc_superrefs at 0x2196b0 - 0x219710 (sz: 0x60)
section __objc_ivar at 0x219710 - 0x219930 (sz: 0x220)
section __objc_data at 0x219930 - 0x219de0 (sz: 0x4b0)
section __data at 0x219de0 - 0x53a39c (sz: 0x3205bc)
section __common at 0x53a3a0 - 0x53e519 (sz: 0x4179)
section __bss at 0x53e520 - 0x5b5368 (sz: 0x76e48)
Writing segment __DATA @ 0x5b6000 ( 0/ 0x1000 @ 0x100dfc000)
Writing segment __DATA @ 0x5b6000 ( 0x99505/ 0xfc000 @ 0x100e00000)
Writing segment __DATA @ 0x650000 ( 0x3fa4/ 0x3fa8 @ 0x100efc000)
Writing segment __DATA @ 0x654000 (0x3633fd/0x7f8000 @ 0x101800000)
Writing segment __DATA @ 0x9b8000 ( 0x3658/ 0x7fa0 @ 0x101ff8000)
Writing segment __DATA @ 0x9bc000 ( 0x5fc65/ 0xfc000 @ 0x103d00000)
Writing segment __DATA @ 0xa1c000 ( 0x3fa4/ 0x3fa8 @ 0x103dfc000)
Writing segment __DATA @ 0xa20000 ( 0x4ae5/ 0xfc000 @ 0x103e00000)
Writing segment __DATA @ 0xa25000 ( 0x3fa4/ 0x3fa8 @ 0x103efc000)
Writing segment __DATA @ 0xa29000 ( 0x44205/ 0xfc000 @ 0x103f00000)
Writing segment __DATA @ 0xa6e000 ( 0x3fa4/ 0x3fa8 @ 0x103ffc000)
Writing segment __DATA @ 0xa72000 (0x1a17fd/0x7f8000 @ 0x105000000)
Writing segment __DATA @ 0xc14000 ( 0x1a3e/ 0x7fa0 @ 0x1057f8000)
Writing segment __DATA @ 0xc16000 ( 0x7a8f/0x7f8000 @ 0x105800000)
Writing segment __DATA @ 0xc1e000 ( 0xba/ 0x7fa0 @ 0x105ff8000)
Writing segment __DATA @ 0xc1f000 (0x26bc58/0x7f8000 @ 0x107000000)
Writing segment __DATA @ 0xe8b000 ( 0x2700/ 0x7fa0 @ 0x1077f8000)
Writing segment __DATA @ 0xe8e000 ( 0x20875/ 0xfc000 @ 0x107800000)
Writing segment __DATA @ 0xeaf000 ( 0x3fa4/ 0x3fa8 @ 0x1078fc000)
Writing segment __DATA @ 0xeb3000 ( 0x69df/ 0xfc000 @ 0x107b00000)
Writing segment __DATA @ 0xeba000 ( 0x3fa4/ 0x3fa8 @ 0x107bfc000)
Writing segment __DATA @ 0xebe000 ( 0x3515/ 0xfc000 @ 0x107c00000)
Writing segment __DATA @ 0xec2000 ( 0x3fa4/ 0x3fa8 @ 0x107cfc000)
Writing segment __DATA @ 0xec6000 ( 0xff405/0x7f8000 @ 0x108800000)
Writing segment __DATA @ 0xfc6000 ( 0x1018/ 0x7fa0 @ 0x108ff8000)
Writing segment __DATA @ 0xfc8000 ( 0x737ff/0x7f8000 @ 0x109000000)
Writing segment __DATA @ 0x103c000 ( 0x75a/ 0x7fa0 @ 0x1097f8000)
Writing segment __DATA @ 0x103d000 ( 0x9feb/0x7f8000 @ 0x109800000)
Writing segment __DATA @ 0x1047000 ( 0xc4/ 0x7fa0 @ 0x109ff8000)
Writing segment __DATA @ 0x1048000 ( 0x49f1/0x7f8000 @ 0x10d800000)
Writing segment __DATA @ 0x104d000 ( 0x6e/ 0x7fa0 @ 0x10dff8000)
Writing segment __DATA @ 0x104e000 ( 0x1e5/ 0xfc000 @ 0x10e000000)
Writing segment __DATA @ 0x104f000 ( 0x3fa4/ 0x3fa8 @ 0x10e0fc000)
Writing segment __LINKEDIT @ 0x1053000 ( 0xcb444/ 0xcc000 @ 0x1005b6000)
Writing LC_DYLD_INFO_ONLY command
Writing LC_SYMTAB command
Writing LC_DYSYMTAB command
Writing LC_LOAD_DYLINKER command
Writing LC_UUID command
Writing LC_VERSION_MIN_MACOSX command
Writing LC_SOURCE_VERSION command
Writing LC_MAIN command
Writing LC_LOAD_DYLIB command
Writing LC_LOAD_DYLIB command
Writing LC_LOAD_DYLIB command
Writing LC_LOAD_DYLIB command
Writing LC_LOAD_DYLIB command
Writing LC_LOAD_DYLIB command
Writing LC_LOAD_DYLIB command
Writing LC_LOAD_DYLIB command
Writing LC_LOAD_DYLIB command
Writing LC_LOAD_DYLIB command
Writing LC_LOAD_DYLIB command
Writing LC_LOAD_DYLIB command
Writing LC_LOAD_DYLIB command
Writing LC_LOAD_DYLIB command
Writing LC_LOAD_DYLIB command
Writing LC_FUNCTION_STARTS command
Writing LC_DATA_IN_CODE command
Writing LC_DYLIB_CODE_SIGN_DRS command
unexec: not enough room for load commands for new __DATA segments
make[1]: *** [bootstrap-emacs] Error 1
make: *** [src] Error 2
[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4219 bytes --]
^ permalink raw reply [flat|nested] 6+ messages in thread
* bug#18505: 24.3.93; intermittent unexec failures when building on Mac OS X 10.10 beta, Xcode 6.0
2014-09-19 4:13 bug#18505: 24.3.93; intermittent unexec failures when building on Mac OS X 10.10 beta, Xcode 6.0 David Caldwell
@ 2014-09-20 15:31 ` Jan Djärv
2014-09-20 18:31 ` David Caldwell
0 siblings, 1 reply; 6+ messages in thread
From: Jan Djärv @ 2014-09-20 15:31 UTC (permalink / raw)
To: David Caldwell; +Cc: 18505
Hello.
19 sep 2014 kl. 06:13 skrev David Caldwell <david@porkrind.org>:
> Hello,
>
> I tried to build the latest pretest on Mac OS X Yosemite Beta with the
> new Xcode 6.0 (GM) tools and ran into this error during the unexec step:
>
> unexec: not enough room for load commands for new __DATA segments
Does it happen all the time or just some times?
Jan D.
>
> (Full unexec log is attached).
>
> Poking around I discovered a couple discrepancies. The "headerpad_extra"
> setting in configure.ac says entries are 56 bytes, but they are actually
> 78 bytes (probably because the comment was originally written in the 32
> bit days). Also the number of load commands needed is variable now--it
> depends on how many malloc segments were found and how many it could
> merge (see find_emacs_zone_regions() and unexec_regions_merge() in
> unexmacosx.c).
>
> I printed num_unexec_regions in unexmacosx.c and observed anywhere
> from about 14 to 34 regions needed. But given that it seems to be
> different on every run and it's super annoying to debug when you run out
> of space, I suggest bumping headerpad_extra up to 0x1000. This gives
> about 1.5KB of headroom (or wasted space, depending on how you look at
> it) and allows for 52 load commands, which should be future proof for
> quite a while. I have attached a patch for configure.ac that does this
> and changes the comment to reflect my understanding.
>
> I've also provided a patch that makes the unexec error more helpful: it
> computes the size needed for headerpad_extra given the number of load
> commands that it needed for that particular unexec run.
>
> I assume this affects the trunk code, too, though I didn't look at it.
>
> -David
> <increase_headerpad_extra.patch><helpful_unexec_error.patch><unexec-log.txt>
^ permalink raw reply [flat|nested] 6+ messages in thread
* bug#18505: 24.3.93; intermittent unexec failures when building on Mac OS X 10.10 beta, Xcode 6.0
2014-09-20 15:31 ` Jan Djärv
@ 2014-09-20 18:31 ` David Caldwell
2014-09-21 9:15 ` Jan Djärv
0 siblings, 1 reply; 6+ messages in thread
From: David Caldwell @ 2014-09-20 18:31 UTC (permalink / raw)
To: Jan Djärv; +Cc: 18505
[-- Attachment #1: Type: text/plain, Size: 1757 bytes --]
On 9/20/14 8:31 AM, Jan Djärv wrote:
> Hello.
>
> 19 sep 2014 kl. 06:13 skrev David Caldwell <david@porkrind.org>:
>
>> Hello,
>>
>> I tried to build the latest pretest on Mac OS X Yosemite Beta with the
>> new Xcode 6.0 (GM) tools and ran into this error during the unexec step:
>>
>> unexec: not enough room for load commands for new __DATA segments
>
> Does it happen all the time or just some times?
It depends on 2 variables: the number of load commands that need to be
added (num_unexec_regions) and text_seg_lowest_offset.
num_unexec_regions jumps around a lot, doing "make clean && make" over
and over it'll be different every time. Somewhere between 12 and 34.
text_seg_lowest_offset seems more stable, but it still changes. When I
first unpacked the tarball and compiled, text_seg_lowest_offset was very
low: 0x17c0. This stayed constant for about an hour while I was
debugging this (adding debug prints) and then mysteriously jumped up to
0x24f0 at which point the unexec started succeeding without the change
to headerpad_extra.
I just unpacked a clean pretest source directory and
text_seg_lowest_offset was 0x17a0 and unexec failed.
Doing
(make clean && make) 2>&1 | grep "Lowest offset of all sections"
over and over gives me:
Lowest offset of all sections in __TEXT segment: 0x17a0
Lowest offset of all sections in __TEXT segment: 0x17a0
Lowest offset of all sections in __TEXT segment: 0x17a0
Lowest offset of all sections in __TEXT segment: 0x17a0
Lowest offset of all sections in __TEXT segment: 0x17a0
Lowest offset of all sections in __TEXT segment: 0x17a0
Though one of those times it didn't fail (presumably because
num_unexec_regions was low enough).
-David
[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4219 bytes --]
^ permalink raw reply [flat|nested] 6+ messages in thread
* bug#18505: 24.3.93; intermittent unexec failures when building on Mac OS X 10.10 beta, Xcode 6.0
2014-09-20 18:31 ` David Caldwell
@ 2014-09-21 9:15 ` Jan Djärv
2014-09-21 18:07 ` David Caldwell
0 siblings, 1 reply; 6+ messages in thread
From: Jan Djärv @ 2014-09-21 9:15 UTC (permalink / raw)
To: David Caldwell; +Cc: 18505
Hello.
20 sep 2014 kl. 20:31 skrev David Caldwell <david@porkrind.org>:
> On 9/20/14 8:31 AM, Jan Djärv wrote:
>> Hello.
>>
>> 19 sep 2014 kl. 06:13 skrev David Caldwell <david@porkrind.org>:
>>
>>> Hello,
>>>
>>> I tried to build the latest pretest on Mac OS X Yosemite Beta with the
>>> new Xcode 6.0 (GM) tools and ran into this error during the unexec step:
>>>
>>> unexec: not enough room for load commands for new __DATA segments
>>
>> Does it happen all the time or just some times?
>
> It depends on 2 variables: the number of load commands that need to be
> added (num_unexec_regions) and text_seg_lowest_offset.
>
> num_unexec_regions jumps around a lot, doing "make clean && make" over
> and over it'll be different every time. Somewhere between 12 and 34.
What makes it do that? Some address randomization? Some other unknown bug?
I would expect num_unexec_regions to be the same for every make. text_seg_lowest_offset could be address randomization, but if it stays somewhat constant, that can't be it.
I've seen this failure before, but usually a new make works.
I'm trying to decide if this is emacs 24 or trunk material.
Is there a way to dynamically react to these changes and adjust headerpad_extra dynamically at dump time?
>
> text_seg_lowest_offset seems more stable, but it still changes. When I
> first unpacked the tarball and compiled, text_seg_lowest_offset was very
> low: 0x17c0. This stayed constant for about an hour while I was
> debugging this (adding debug prints) and then mysteriously jumped up to
> 0x24f0 at which point the unexec started succeeding without the change
> to headerpad_extra.
>
> I just unpacked a clean pretest source directory and
> text_seg_lowest_offset was 0x17a0 and unexec failed.
>
> Doing
> (make clean && make) 2>&1 | grep "Lowest offset of all sections"
> over and over gives me:
>
> Lowest offset of all sections in __TEXT segment: 0x17a0
> Lowest offset of all sections in __TEXT segment: 0x17a0
> Lowest offset of all sections in __TEXT segment: 0x17a0
> Lowest offset of all sections in __TEXT segment: 0x17a0
> Lowest offset of all sections in __TEXT segment: 0x17a0
> Lowest offset of all sections in __TEXT segment: 0x17a0
>
> Though one of those times it didn't fail (presumably because
> num_unexec_regions was low enough).
Jan D.
^ permalink raw reply [flat|nested] 6+ messages in thread
* bug#18505: 24.3.93; intermittent unexec failures when building on Mac OS X 10.10 beta, Xcode 6.0
2014-09-21 9:15 ` Jan Djärv
@ 2014-09-21 18:07 ` David Caldwell
2014-09-21 20:37 ` Jan Djärv
0 siblings, 1 reply; 6+ messages in thread
From: David Caldwell @ 2014-09-21 18:07 UTC (permalink / raw)
To: Jan Djärv; +Cc: 18505
[-- Attachment #1: Type: text/plain, Size: 2924 bytes --]
On 9/21/14 2:15 AM, Jan Djärv wrote:
> Hello.
>
> 20 sep 2014 kl. 20:31 skrev David Caldwell <david@porkrind.org>:
>
>> On 9/20/14 8:31 AM, Jan Djärv wrote:
>>> Hello.
>>>
>>> 19 sep 2014 kl. 06:13 skrev David Caldwell <david@porkrind.org>:
>>>
>>>> Hello,
>>>>
>>>> I tried to build the latest pretest on Mac OS X Yosemite Beta with the
>>>> new Xcode 6.0 (GM) tools and ran into this error during the unexec step:
>>>>
>>>> unexec: not enough room for load commands for new __DATA segments
>>>
>>> Does it happen all the time or just some times?
>>
>> It depends on 2 variables: the number of load commands that need to be
>> added (num_unexec_regions) and text_seg_lowest_offset.
>>
>> num_unexec_regions jumps around a lot, doing "make clean && make" over
>> and over it'll be different every time. Somewhere between 12 and 34.
>
> What makes it do that? Some address randomization? Some other unknown bug?
> I would expect num_unexec_regions to be the same for every make.
I don't know. I would also expect num_unexec_regions to be the same. If
it changes, it seems to mean the malloc behavior is different on every
run. But yes, perhaps address space randomization could cause that to
happen. I don't understand that part of the code well enough to
speculate too much.
> text_seg_lowest_offset could be address randomization, but if it stays somewhat constant, that can't be it.
I just figured that out (and smacked my head because it was obvious).
That changed when I changed headerpad. I got curious and did a binary
search to figure out exactly how -headerpad affects
text_seg_lowest_offset in my setup (all number hex):
-headerpad text_seg_lowest_offset
0 -> 740 17a0
741 -> 1740 27a0
1741 -> ??? 37a0
So, text_seg_lowest_offset directly correlate with -headerpad and ld is
doing some sort of alignment.
> I've seen this failure before, but usually a new make works.
> I'm trying to decide if this is emacs 24 or trunk material.
I think it should go in both. It's really quite a low-risk change: the
-headerpad option is well documented in ld, and the amount my patch adds
gives an extra 1.5K of headroom on a 6M binary (.02%).
I did a bunch of 'bzr log' searches to understand the nature of the
-headerpad setting and it appears to not have been touched since 2006
(in the 32 bit era). I believe that is why the comment in configure is
incorrect: load commands may have been 56 bytes on 32 bit archs, but
they are 78 bytes on my 64 bit computer (which is all current Macs going
forward).
> Is there a way to dynamically react to these changes and adjust headerpad_extra dynamically at dump time?
Unfortunately the -headerpad is specified during link time so to change
it dynamically would require re-linking after unexec-ing. That's a large
Makefile change to get that all working correctly.
-David
[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4219 bytes --]
^ permalink raw reply [flat|nested] 6+ messages in thread
* bug#18505: 24.3.93; intermittent unexec failures when building on Mac OS X 10.10 beta, Xcode 6.0
2014-09-21 18:07 ` David Caldwell
@ 2014-09-21 20:37 ` Jan Djärv
0 siblings, 0 replies; 6+ messages in thread
From: Jan Djärv @ 2014-09-21 20:37 UTC (permalink / raw)
To: David Caldwell; +Cc: 18505-done
Hi.
21 sep 2014 kl. 20:07 skrev David Caldwell <david@porkrind.org>:
> On 9/21/14 2:15 AM, Jan Djärv wrote:
>> Hello.
>>
>> 20 sep 2014 kl. 20:31 skrev David Caldwell <david@porkrind.org>:
>>
>>> On 9/20/14 8:31 AM, Jan Djärv wrote:
>>>> Hello.
>>>>
>>>> 19 sep 2014 kl. 06:13 skrev David Caldwell <david@porkrind.org>:
>>>>
>>>>> Hello,
>>>>>
>>>>> I tried to build the latest pretest on Mac OS X Yosemite Beta with the
>>>>> new Xcode 6.0 (GM) tools and ran into this error during the unexec step:
>>>>>
>>>>> unexec: not enough room for load commands for new __DATA segments
>>>>
>>>> Does it happen all the time or just some times?
>>>
>>> It depends on 2 variables: the number of load commands that need to be
>>> added (num_unexec_regions) and text_seg_lowest_offset.
>>>
>>> num_unexec_regions jumps around a lot, doing "make clean && make" over
>>> and over it'll be different every time. Somewhere between 12 and 34.
>>
>> What makes it do that? Some address randomization? Some other unknown bug?
>> I would expect num_unexec_regions to be the same for every make.
>
> I don't know. I would also expect num_unexec_regions to be the same. If
> it changes, it seems to mean the malloc behavior is different on every
> run. But yes, perhaps address space randomization could cause that to
> happen. I don't understand that part of the code well enough to
> speculate too much.
Me neither.
>
>> text_seg_lowest_offset could be address randomization, but if it stays somewhat constant, that can't be it.
>
> I just figured that out (and smacked my head because it was obvious).
> That changed when I changed headerpad. I got curious and did a binary
> search to figure out exactly how -headerpad affects
> text_seg_lowest_offset in my setup (all number hex):
>
> -headerpad text_seg_lowest_offset
> 0 -> 740 17a0
> 741 -> 1740 27a0
> 1741 -> ??? 37a0
>
> So, text_seg_lowest_offset directly correlate with -headerpad and ld is
> doing some sort of alignment.
Okay.
>
>> I've seen this failure before, but usually a new make works.
>> I'm trying to decide if this is emacs 24 or trunk material.
>
> I think it should go in both. It's really quite a low-risk change: the
> -headerpad option is well documented in ld, and the amount my patch adds
> gives an extra 1.5K of headroom on a 6M binary (.02%).
>
> I did a bunch of 'bzr log' searches to understand the nature of the
> -headerpad setting and it appears to not have been touched since 2006
> (in the 32 bit era). I believe that is why the comment in configure is
> incorrect: load commands may have been 56 bytes on 32 bit archs, but
> they are 78 bytes on my 64 bit computer (which is all current Macs going
> forward).
I checked it in in the 24-branch.
>
>> Is there a way to dynamically react to these changes and adjust headerpad_extra dynamically at dump time?
>
> Unfortunately the -headerpad is specified during link time so to change
> it dynamically would require re-linking after unexec-ing. That's a large
> Makefile change to get that all working correctly.
And possibly overkill.
Thanks,
Jan D.
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2014-09-21 20:37 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2014-09-19 4:13 bug#18505: 24.3.93; intermittent unexec failures when building on Mac OS X 10.10 beta, Xcode 6.0 David Caldwell
2014-09-20 15:31 ` Jan Djärv
2014-09-20 18:31 ` David Caldwell
2014-09-21 9:15 ` Jan Djärv
2014-09-21 18:07 ` David Caldwell
2014-09-21 20:37 ` Jan Djärv
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/emacs.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).