bug#22522: Commit b88e9cded7ae3756e3a2ec4a23e8df352a0239f9 breaks emacs dumping for me

bug#22522: Commit b88e9cded7ae3756e3a2ec4a23e8df352a0239f9 breaks emacs dumping for me

[Elric Milon]

Trying to build current master failed while dumping emacs, so I ran a
git bisect with the following script to find the culprit:

8<------------8<------------8<------------8<------------8<------------
set -ex

./autogen.sh

./configure --prefix $(readlink -fe $PWD/../inst) \
    --enable-link-time-optimization \
    --without-pop \
    --with-x-toolkit=gtk3 \
    --without-xaw3d \
    --without-selinux \
    --with-file-notification=yes \
    --with-modules \
    --with-cairo \
    --without-pop \
    --with-x

make -j$(grep processor /proc/cpuinfo |wc -l ) bootstrap
make -j$(grep processor /proc/cpuinfo |wc -l )

make install

8<------------8<------------8<------------8<------------8<------------

And this is the failure message:

8<------------8<------------8<------------8<------------8<------------

[...]
Loading /var/data/src/emacs/emacs/lisp/tooltip.el (source)...
Finding pointers to doc strings...
Finding pointers to doc strings...done
Dumping under the name emacs
92361 pure bytes used
: paxctl -zex emacs
mv -f emacs bootstrap-emacs
make -C ../lisp compile-first EMACS="../src/bootstrap-emacs"
make[3]: Entering directory '/var/data/src/emacs/emacs/lisp'
  ELC      emacs-lisp/byte-opt.elc
  ELC      emacs-lisp/autoload.elc
  ELC      emacs-lisp/cconv.elc
  ELC      emacs-lisp/macroexp.elc
  ELC      emacs-lisp/bytecomp.elc
*** Error in `../src/bootstrap-emacs': corrupted double-linked list: 0x0000000001b7c860 ***

Backtrace:
../src/bootstrap-emacs[0x4da592]
../src/bootstrap-emacs[0x511f79]
../src/bootstrap-emacs[0x4cfb9e]
../src/bootstrap-emacs[0x498263]
/lib/x86_64-linux-gnu/libpthread.so.0[0x3dae810660]
/lib/x86_64-linux-gnu/libc.so.6(gsignal+0x37)[0x3dae033507]
/lib/x86_64-linux-gnu/libc.so.6(abort+0x16a)[0x3dae0348da]
/lib/x86_64-linux-gnu/libc.so.6[0x3dae071a63]
/lib/x86_64-linux-gnu/libc.so.6[0x3dae076ebe]
/lib/x86_64-linux-gnu/libc.so.6[0x3dae079f62]
/lib/x86_64-linux-gnu/libc.so.6(realloc+0xf0)[0x3dae07b0e0]
../src/bootstrap-emacs[0x4ab8c2]
../src/bootstrap-emacs[0x4b33b7]
../src/bootstrap-emacs[0x566b92]
../src/bootstrap-emacs[0x47eb4b]
../src/bootstrap-emacs[0x48b4f6]
../src/bootstrap-emacs[0x470f23]
../src/bootstrap-emacs[0x47b3dd]
../src/bootstrap-emacs[0x470ebe]
../src/bootstrap-emacs[0x47b12d]
../src/bootstrap-emacs[0x470ebe]
../src/bootstrap-emacs[0x47b3dd]
../src/bootstrap-emacs[0x470ebe]
../src/bootstrap-emacs[0x47bbbd]
../src/bootstrap-emacs[0x470ebe]
../src/bootstrap-emacs[0x47149d]
../src/bootstrap-emacs[0x479298]
../src/bootstrap-emacs[0x470cdf]
../src/bootstrap-emacs[0x4793b1]
../src/bootstrap-emacs[0x4707a2]
../src/bootstrap-emacs[0x507e24]
../src/bootstrap-emacs[0x47084b]
../src/bootstrap-emacs[0x511c48]
../src/bootstrap-emacs[0x511cdb]
../src/bootstrap-emacs[0x511e08]
../src/bootstrap-emacs[0x41941f]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)[0x3dae020870]
../src/bootstrap-emacs[0x41b6a9]
*** Error in `../src/bootstrap-emacs': corrupted double-linked list: 0x0000000001b7c860 ***
Fatal error 6: Aborted
Backtrace:
../src/bootstrap-emacs[0x4da592]
*** Error in `../src/bootstrap-emacs[0x511f79]
../src/bootstrap-emacs[0x4cfb9e]
../src/bootstrap-emacs[0x498263]
/lib/x86_64-linux-gnu/libpthread.so.0[0x3dae810660]
/lib/x86_64-linux-gnu/libc.so.6(gsignal+0x37)[0x3dae033507]
/lib/x86_64-linux-gnu/libc.so.6(abort+0x
Backtrace:
16a../src/bootstrap-emacs)[0x[0x4da592]
../src/bootstrap-emacs3dae0348da]
[0x511f79]
../src/bootstrap-emacs[0x4cfb9e]
../src/bootstrap-emacs[0x/lib/x86_64-linux-gnu/libc.so.6498263]
[0x3dae071a63]
/lib/x86_64-linux-gnu/libpthread.so.0[0x3dae810660]
/lib/x86_64-linux-gnu/libc.so.6[0x3dae076ebe]
/lib/x86_64-linux-gnu/libc.so.6(gsignal+0x37)[0x3dae033507]
/lib/x86_64-linux-gnu/libc.so.6[0x3dae079f62]
/lib/x86_64-linux-gnu/libc.so.6(abort+0x16a)[0x3dae0348da/lib/x86_64-linux-gnu/libc.so.6]
(realloc+0xf0)[0x3dae07b0e0]
../src/bootstrap-emacs[0x4ab8c2/lib/x86_64-linux-gnu/libc.so.6[0x]
../src/bootstrap-emacs3dae071a63]
[0x4b33b7]
../src/bootstrap-emacs[0x566b92]
../src/bootstrap-emacs[0x/lib/x86_64-linux-gnu/libc.so.647eb4b]
[0x../src/bootstrap-emacs[0x48b4f63dae076ebe]
]
../src/bootstrap-emacs[0x470f23]
../src/bootstrap-emacs[0x47b3dd]
/lib/x86_64-linux-gnu/libc.so.6[0x3dae079f62../src/bootstrap-emacs[0x]
470ebe]
../src/bootstrap-emacs[0x47b12d]
../src/bootstrap-emacs[0x470ebe]
../src/bootstrap-emacs/lib/x86_64-linux-gnu/libc.so.6[0x47b3dd]
../src/bootstrap-emacs(realloc+0xf0[0x470ebe]
../src/bootstrap-emacs)[0x[0x47bbbd]
../src/bootstrap-emacs3dae07b0e0]
[0x../src/bootstrap-emacs[0x470ebe]
../src/bootstrap-emacs4ab8c2]
../src/bootstrap-emacs[0x47149d]
../src/bootstrap-emacs[0x[0x479298]
../src/bootstrap-emacs4b33b7]
[0x470cdf]
../src/bootstrap-emacs../src/bootstrap-emacs[0x[0x566b92]
4793b1]
../src/bootstrap-emacs../src/bootstrap-emacs[0x47eb4b[0x4707a2]
]
../src/bootstrap-emacs[0x507e24../src/bootstrap-emacs]
../src/bootstrap-emacs[0x48b4f6]
[0x../src/bootstrap-emacs[0x47084b]
470f23../src/bootstrap-emacs[0x511c48]
../src/bootstrap-emacs[0x47b3dd]
]
../src/bootstrap-emacs../src/bootstrap-emacs[0x470ebe]
../src/bootstrap-emacs[0x511cdb[0x47b12d]
]
../src/bootstrap-emacs../src/bootstrap-emacs[0x[0x470ebe]
511e08]
../src/bootstrap-emacs../src/bootstrap-emacs[0x[0x41941f]
47b3dd]
../src/bootstrap-emacs[0x470ebe]
../src/bootstrap-emacs/lib/x86_64-linux-gnu/libc.so.6([0x47bbbd]
../src/bootstrap-emacs__libc_start_main+0xf0[0x470ebe)[0x3dae020870]
../src/bootstrap-emacs]
[0x../src/bootstrap-emacs[0x47149d]
../src/bootstrap-emacs41b6a9]
[0x479298]
../src/bootstrap-emacs[0x470cdf]
../src/bootstrap-emacs[0x4793b1]
../src/bootstrap-emacs[0x4707a2]
../src/bootstrap-emacs[0x507e24]
../src/bootstrap-emacs[0x47084b]
../src/bootstrap-emacs[0x511c48]
../src/bootstrap-emacs[0x511cdb]
../src/bootstrap-emacs[0x511e08]
../src/bootstrap-emacs[0x41941f]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)[0x3dae020870]
../src/bootstrap-emacs[0x41b6a9]
/bin/bash: line 1: 10631 Aborted                 EMACSLOADPATH= '../src/bootstrap-emacs' -batch --no-site-file --no-site-lisp --eval "(setq max-lisp-eval-depth 2200)" --eval '(setq load-prefer-newer t)' -f batch-byte-compile emacs-lisp/autoload.el
Makefile:268: recipe for target 'emacs-lisp/autoload.elc' failed
make[3]: *** [emacs-lisp/autoload.elc] Error 134
make[3]: *** Waiting for unfinished jobs....
/bin/bash: line 1: 10633 Aborted                 EMACSLOADPATH= '../src/bootstrap-emacs' -batch --no-site-file --no-site-lisp --eval "(setq max-lisp-eval-depth 2200)" --eval '(setq load-prefer-newer t)' -f batch-byte-compile emacs-lisp/macroexp.el
Makefile:268: recipe for target 'emacs-lisp/macroexp.elc' failed
make[3]: *** [emacs-lisp/macroexp.elc] Error 134
/bin/bash: line 1: 10632 Aborted                 EMACSLOADPATH= '../src/bootstrap-emacs' -batch --no-site-file --no-site-lisp --eval "(setq max-lisp-eval-depth 2200)" --eval '(setq load-prefer-newer t)' -f batch-byte-compile emacs-lisp/cconv.el
Makefile:268: recipe for target 'emacs-lisp/cconv.elc' failed
make[3]: *** [emacs-lisp/cconv.elc] Error 134
*** Error in `../src/bootstrap-emacs': corrupted double-linked list: 0x0000000001b7c860 ***

Backtrace:

Backtrace:
../src/bootstrap-emacs[0x4da592]
../src/bootstrap-emacs[0x4da592../src/bootstrap-emacs]
[0x../src/bootstrap-emacs511f79]
../src/bootstrap-emacs[0x511f79[0x4cfb9e]
]
../src/bootstrap-emacs[0x498263]
../src/bootstrap-emacs/lib/x86_64-linux-gnu/libpthread.so.0[0x4cfb9e[0x3dae810660]
]
../src/bootstrap-emacs[0x498263]
/lib/x86_64-linux-gnu/libc.so.6(gsignal+0x37)[0x/lib/x86_64-linux-gnu/libpthread.so.03dae033507]
[0x3dae810660]
/lib/x86_64-linux-gnu/libc.so.6(abort+0x16a)[0x3dae0348da]
/lib/x86_64-linux-gnu/libc.so.6(gsignal+0x37)[0x3dae033507]
/lib/x86_64-linux-gnu/libc.so.6[0x3dae071a63]
/lib/x86_64-linux-gnu/libc.so.6(abort+0x/lib/x86_64-linux-gnu/libc.so.6[0x16a)3dae076ebe]
[0x3dae0348da]
/lib/x86_64-linux-gnu/libc.so.6[0x3dae079f62]
/lib/x86_64-linux-gnu/libc.so.6[0x3dae071a63]
/lib/x86_64-linux-gnu/libc.so.6(realloc+0xf0)[0x3dae07b0e0]
../src/bootstrap-emacs[0x4ab8c2]
../src/bootstrap-emacs[0x/lib/x86_64-linux-gnu/libc.so.64b33b7]
../src/bootstrap-emacs[0x3dae076ebe[0x566b92]
../src/bootstrap-emacs]
[0x47eb4b]
../src/bootstrap-emacs[0x48b4f6]
../src/bootstrap-emacs[0x470f23]
../src/bootstrap-emacs[0x47b3dd]
/lib/x86_64-linux-gnu/libc.so.6../src/bootstrap-emacs[0x470ebe]
[0x../src/bootstrap-emacs[0x47b12d]
../src/bootstrap-emacs3dae079f62]
[0x470ebe]
../src/bootstrap-emacs[0x47b3dd]
../src/bootstrap-emacs[0x470ebe]
../src/bootstrap-emacs[0x47bbbd]
../src/bootstrap-emacs[0x470ebe/lib/x86_64-linux-gnu/libc.so.6(]
../src/bootstrap-emacsrealloc[0x47149d]
../src/bootstrap-emacs+0xf0[0x479298]
)../src/bootstrap-emacs[0x470cdf]
[0x../src/bootstrap-emacs[0x4793b1]
3dae07b0e0../src/bootstrap-emacs[0x4707a2]
../src/bootstrap-emacs]
[0x../src/bootstrap-emacs507e24]
../src/bootstrap-emacs[0x4ab8c2[0x47084b]
../src/bootstrap-emacs]
[0x../src/bootstrap-emacs[0x511c48]
../src/bootstrap-emacs4b33b7]
[0x../src/bootstrap-emacs511cdb]
../src/bootstrap-emacs[0x566b92[0x511e08]
../src/bootstrap-emacs]
[0x../src/bootstrap-emacs[0x41941f]
47eb4b]
../src/bootstrap-emacs[0x48b4f6/lib/x86_64-linux-gnu/libc.so.6]
(../src/bootstrap-emacs[0x__libc_start_main+0xf0)[0x470f23]
3dae020870../src/bootstrap-emacs]
../src/bootstrap-emacs[0x47b3dd[0x41b6a9]
]
../src/bootstrap-emacs[0x470ebe]
../src/bootstrap-emacs[0x47b12d]
../src/bootstrap-emacs[0x470ebe]
../src/bootstrap-emacs[0x47b3dd]
../src/bootstrap-emacs[0x470ebe]
../src/bootstrap-emacs[0x47bbbd]
../src/bootstrap-emacs[0x470ebe]
../src/bootstrap-emacs[0x47149d]
../src/bootstrap-emacs[0x479298]
../src/bootstrap-emacs[0x470cdf]
../src/bootstrap-emacs[0x4793b1]
../src/bootstrap-emacs[0x4707a2]
../src/bootstrap-emacs[0x507e24]
../src/bootstrap-emacs[0x47084b]
../src/bootstrap-emacs[0x511c48]
../src/bootstrap-emacs[0x511cdb]
../src/bootstrap-emacs[0x511e08]
../src/bootstrap-emacs[0x41941f]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)[0x3dae020870]
../src/bootstrap-emacs[0x41b6a9]
/bin/bash: line 1: 10630 Aborted                 EMACSLOADPATH= '../src/bootstrap-emacs' -batch --no-site-file --no-site-lisp --eval "(setq max-lisp-eval-depth 2200)" --eval '(setq load-prefer-newer t)' -f batch-byte-compile emacs-lisp/byte-opt.el
Makefile:268: recipe for target 'emacs-lisp/byte-opt.elc' failed
make[3]: *** [emacs-lisp/byte-opt.elc] Error 134
/bin/bash: line 1: 10634 Aborted                 EMACSLOADPATH= '../src/bootstrap-emacs' -batch --no-site-file --no-site-lisp --eval "(setq max-lisp-eval-depth 2200)" --eval '(setq load-prefer-newer t)' -f batch-byte-compile emacs-lisp/bytecomp.el
Makefile:268: recipe for target 'emacs-lisp/bytecomp.elc' failed
make[3]: *** [emacs-lisp/bytecomp.elc] Error 134
make[3]: Leaving directory '/var/data/src/emacs/emacs/lisp'
Makefile:727: recipe for target 'bootstrap-emacs' failed
make[2]: *** [bootstrap-emacs] Error 2
make[2]: Leaving directory '/var/data/src/emacs/emacs/src'
Makefile:394: recipe for target 'src' failed
make[1]: *** [src] Error 2
make[1]: Leaving directory '/var/data/src/emacs/emacs'
Makefile:1087: recipe for target 'bootstrap' failed
make: *** [bootstrap] Error 2
b88e9cded7ae3756e3a2ec4a23e8df352a0239f9 is the first bad commit
commit b88e9cded7ae3756e3a2ec4a23e8df352a0239f9
Author: Paul Eggert <eggert@cs.ucla.edu>
Date:   Tue Jan 26 23:00:10 2016 -0800

    malloc.h hygiene

    This attempts to future-proof Emacs a bit against possible glibc
    changes, by having Emacs use <malloc.h> declarations rather than
    coding them up by hand.  Problem noted by Florian Weimer in:
    https://sourceware.org/ml/libc-alpha/2016-01/msg00777.html
    Implement this mainly by moving malloc.h-related functions from
    emacs.c (which does not include <malloc.h>) to alloc.c (which does).
    * src/alloc.c (my_heap_start) [DOUG_LEA_MALLOC || GNU_LINUX]:
    New function.
    The remaining changes to this file apply only if DOUG_LEA_MALLOC.
    (alloc_unexec_pre, alloc_unexec_post): New functions.
    (malloc_initialize_hook): Use my_heap_start and alloc_unexec_post.
    (__MALLOC_HOOK_VOLATILE): New macro, if not already defined.
    (__malloc_initialize_hook): Use it.
    (malloc_state_ptr, malloc_initialize_hook, __malloc_initialize_hook):
    Move here from ...
    * src/emacs.c: ... here.
    (malloc_get_state, malloc_set_state): Remove extern decls.
    (my_heap_start) [DOUG_LEA_MALLOC || GNU_LINUX]: Remove static var.
    All uses changed to similarly-named new function.
    (Fdump_emacs): Use new functions alloc_unexec_pre, alloc_unexec_post.
    * src/lisp.h (my_heap_start, alloc_unexec_pre, alloc_unexec_post):
    New decls.

:040000 040000 e46f469e02031e990af4af272806980f066ef53e d36369c1e71f9630b0d9eeca83f4e04f3f411376 M	src
bisect run success
git bisect run ../build.sh  7533,93s user 264,26s system 369% cpu 35:12,15 total

8<------------8<------------8<------------8<------------8<------------

I've tried building master on two different up to date Debian SID
machines with similar results.

Note that I tried disabling `randomize_va_space` and running it with
`setarch x86_64 -R` and it failed too.

Thanks!

--
Elric Milon
PGP: 3939C2B494084E2F | http://whirm.eu

bug#22522: Commit b88e9cded7ae3756e3a2ec4a23e8df352a0239f9 breaks emacs dumping for me

[Andy Moreton]

On Mon 01 Feb 2016, Elric Milon wrote:

> Trying to build current master failed while dumping emacs, so I ran a
> git bisect with the following script to find the culprit:

[build script and bisect results snipped]

I see a similar problem with the 64bit cygwin w32 build:

make[2]: Leaving directory '/cygdrive/c/emacs/git/emacs/master/obj-cygwin64-w32/lisp'
./temacs --batch --load loadup bootstrap
Makefile:735: recipe for target 'bootstrap-emacs.exe' failed
make[1]: *** [bootstrap-emacs.exe] Segmentation fault (core dumped)
make[1]: Leaving directory '/cygdrive/c/emacs/git/emacs/master/obj-cygwin64-w32/src'
Makefile:394: recipe for target 'src' failed

A full bootstrap from commit b88e9cded7ae^ completes normally.

    AndyM

bug#22522: Commit b88e9cded7ae3756e3a2ec4a23e8df352a0239f9 breaks emacs dumping for me

[Ken Brown]

On 2/1/2016 5:27 PM, Andy Moreton wrote:
> On Mon 01 Feb 2016, Elric Milon wrote:
>
>> Trying to build current master failed while dumping emacs, so I ran a
>> git bisect with the following script to find the culprit:
>
> [build script and bisect results snipped]
>
> I see a similar problem with the 64bit cygwin w32 build:

I see the following warnings during the build:

../../master/src/alloc.c: In function ‘lisp_align_malloc’:
../../master/src/alloc.c:1247:7: warning: implicit declaration of 
function ‘hybrid_aligned_alloc’ [-Wimplicit-function-declaration]
        abase = base = aligned_alloc (BLOCK_ALIGN, ABLOCKS_BYTES);
        ^
../../master/src/alloc.c:1247:20: warning: assignment makes pointer from 
integer without a cast
        abase = base = aligned_alloc (BLOCK_ALIGN, ABLOCKS_BYTES);
                     ^

This could be the reason for the build problem on (some) 64-bit 
platforms, but I don't have time to look into it further tonight.

Ken

bug#22522: Commit b88e9cded7ae3756e3a2ec4a23e8df352a0239f9 breaks emacs dumping for me

[Wolfgang Jenkner]

On Mon, Feb 01 2016, Ken Brown wrote:

> ../../master/src/alloc.c: In function ‘lisp_align_malloc’:
> ../../master/src/alloc.c:1247:7: warning: implicit declaration of
> function ‘hybrid_aligned_alloc’ [-Wimplicit-function-declaration]

Before Paul's 7fdc3cf, src/alloc.c used to contain a declaration for
aligned_alloc(), which a preprocessor definition turned into
a declaration for hybrid_aligned_alloc().  The preprocessor definition
was redundant as it is contained in src/conf_post.h as well, but the
declaration has to be supplied by some other include file.

(For FreeBSD, stdlib.h, which alloc.c includes, supplies the
declaration, guarded by #if __ISO_C_VISIBLE >= 2011 || __cplusplus >=
201103L, which is true by default, at least on FreeBSD 10).

bug#22522: Commit b88e9cded7ae3756e3a2ec4a23e8df352a0239f9 breaks emacs dumping for me

[Wolfgang Jenkner]

On Tue, Feb 02 2016, Wolfgang Jenkner wrote:

> (For FreeBSD, stdlib.h, which alloc.c includes,
Rather, conf_post.h includes stdlib.h.

bug#22522: Commit b88e9cded7ae3756e3a2ec4a23e8df352a0239f9 breaks emacs dumping for me

[Paul Eggert]

[-- Attachment #1: Type: text/plain, Size: 765 bytes --]

Thanks for reporting this. I reproduced the problem on Fedora 23 x86-64. 
It appears to be a bug in link-time optimization. The symbol 
__malloc_initialize_hook is marked external in alloc.o, but merely 
static (private) in temacs:

$ nm -o alloc.o temacs | grep __malloc_init
alloc.o:00000000002e0a40 D __malloc_initialize_hook
temacs:0000000000b25340 d __malloc_initialize_hook

We used to define this variable in emacs.o, and we now do it in alloc.o. 
Possibly we were lucky that the code ever worked, as I guess the LTO bug 
strikes depending on link time order.

I installed the attached patch, which works around the bug for me. 
Please give it a try. Are any of you connected to the folks who 
implement LTO? It'd be nice to report this bug to them somehow.

[-- Attachment #2: 0001-Port-malloc.h-hygiene-fix-to-LTO.patch --]
[-- Type: application/x-patch, Size: 919 bytes --]

bug#22522: Commit b88e9cded7ae3756e3a2ec4a23e8df352a0239f9 breaks emacs dumping for me

[Paul Eggert]

[-- Attachment #1: Type: text/plain, Size: 453 bytes --]

On 02/01/2016 06:59 PM, Ken Brown wrote:
> ../../master/src/alloc.c: In function ‘lisp_align_malloc’:
> ../../master/src/alloc.c:1247:7: warning: implicit declaration of 
> function ‘hybrid_aligned_alloc’ [-Wimplicit-function-declaration]
>        abase = base = aligned_alloc (BLOCK_ALIGN, ABLOCKS_BYTES);

Thanks, I think this problem is independent but it should be fixed too. 
I installed the attached patch; please give it a try.


[-- Attachment #2: 0001-Port-better-to-platforms-lacking-aligned_alloc.patch --]
[-- Type: application/x-patch, Size: 1162 bytes --]

bug#22522: Commit b88e9cded7ae3756e3a2ec4a23e8df352a0239f9 breaks emacs dumping for me

[Ken Brown]

On 2/2/2016 12:34 PM, Paul Eggert wrote:
> On 02/01/2016 06:59 PM, Ken Brown wrote:
>> ../../master/src/alloc.c: In function ‘lisp_align_malloc’:
>> ../../master/src/alloc.c:1247:7: warning: implicit declaration of
>> function ‘hybrid_aligned_alloc’ [-Wimplicit-function-declaration]
>>        abase = base = aligned_alloc (BLOCK_ALIGN, ABLOCKS_BYTES);
>
> Thanks, I think this problem is independent but it should be fixed too.
> I installed the attached patch; please give it a try.

That doesn't help on Cygwin, because Cygwin *does* have aligned_alloc. 
So we still don't get a declaration of hybrid_aligned_alloc in alloc.c.

Ken

bug#22522: Commit b88e9cded7ae3756e3a2ec4a23e8df352a0239f9 breaks emacs dumping for me

[Ken Brown]

On 2/2/2016 9:20 AM, Wolfgang Jenkner wrote:
> On Mon, Feb 01 2016, Ken Brown wrote:
>
>> ../../master/src/alloc.c: In function ‘lisp_align_malloc’:
>> ../../master/src/alloc.c:1247:7: warning: implicit declaration of
>> function ‘hybrid_aligned_alloc’ [-Wimplicit-function-declaration]
>
> Before Paul's 7fdc3cf, src/alloc.c used to contain a declaration for
> aligned_alloc(), which a preprocessor definition turned into
> a declaration for hybrid_aligned_alloc().  The preprocessor definition
> was redundant as it is contained in src/conf_post.h as well, but the
> declaration has to be supplied by some other include file.
>
> (For FreeBSD, stdlib.h, which alloc.c includes, supplies the
> declaration, guarded by #if __ISO_C_VISIBLE >= 2011 || __cplusplus >=
> 201103L, which is true by default, at least on FreeBSD 10).

Cygwin's stdlib.h also has the declaration with the same guard.  But for 
some reason, alloc.c is still not getting the declaration.  I'll have to 
figure out what's going on.

Ken

bug#22522: Commit b88e9cded7ae3756e3a2ec4a23e8df352a0239f9 breaks emacs dumping for me

[Elric Milon]

Paul Eggert <eggert@cs.ucla.edu> writes:

> Thanks for reporting this. I reproduced the problem on Fedora 23 x86-64.
> It appears to be a bug in link-time optimization. The symbol
> __malloc_initialize_hook is marked external in alloc.o, but merely
> static (private) in temacs:
>
> $ nm -o alloc.o temacs | grep __malloc_init
> alloc.o:00000000002e0a40 D __malloc_initialize_hook
> temacs:0000000000b25340 d __malloc_initialize_hook
>
> We used to define this variable in emacs.o, and we now do it in alloc.o.
> Possibly we were lucky that the code ever worked, as I guess the LTO bug
> strikes depending on link time order.
>
> I installed the attached patch, which works around the bug for me.
> Please give it a try. Are any of you connected to the folks who
> implement LTO? It'd be nice to report this bug to them somehow.


I checked out latest master which appears to contain this patch and it's
building again.

Thanks!


--
Elric Milon
PGP: 3939C2B494084E2F | https://whirm.eu

bug#22522: Commit b88e9cded7ae3756e3a2ec4a23e8df352a0239f9 breaks emacs dumping for me

[Andy Moreton]

On Tue 02 Feb 2016, Ken Brown wrote:

> On 2/2/2016 9:20 AM, Wolfgang Jenkner wrote:
>> On Mon, Feb 01 2016, Ken Brown wrote:
>>
>>> ../../master/src/alloc.c: In function ‘lisp_align_malloc’:
>>> ../../master/src/alloc.c:1247:7: warning: implicit declaration of
>>> function ‘hybrid_aligned_alloc’ [-Wimplicit-function-declaration]
>>
>> Before Paul's 7fdc3cf, src/alloc.c used to contain a declaration for
>> aligned_alloc(), which a preprocessor definition turned into
>> a declaration for hybrid_aligned_alloc().  The preprocessor definition
>> was redundant as it is contained in src/conf_post.h as well, but the
>> declaration has to be supplied by some other include file.
>>
>> (For FreeBSD, stdlib.h, which alloc.c includes, supplies the
>> declaration, guarded by #if __ISO_C_VISIBLE >= 2011 || __cplusplus >=
>> 201103L, which is true by default, at least on FreeBSD 10).
>
> Cygwin's stdlib.h also has the declaration with the same guard.  But for some
> reason, alloc.c is still not getting the declaration.  I'll have to figure out
> what's going on.

Building with V=1 shows the cygwin build uses "gcc -std=gnu99", and
aligned_alloc() is from C11, so I would expect that the declaration in
Cygwin's stdlib.h is not seen.

    AndyM

bug#22522: Commit b88e9cded7ae3756e3a2ec4a23e8df352a0239f9 breaks emacs dumping for me

[Ken Brown]

On 2/2/2016 3:25 PM, Ken Brown wrote:
> On 2/2/2016 9:20 AM, Wolfgang Jenkner wrote:
>> On Mon, Feb 01 2016, Ken Brown wrote:
>>
>>> ../../master/src/alloc.c: In function ‘lisp_align_malloc’:
>>> ../../master/src/alloc.c:1247:7: warning: implicit declaration of
>>> function ‘hybrid_aligned_alloc’ [-Wimplicit-function-declaration]
>>
>> Before Paul's 7fdc3cf, src/alloc.c used to contain a declaration for
>> aligned_alloc(), which a preprocessor definition turned into
>> a declaration for hybrid_aligned_alloc().  The preprocessor definition
>> was redundant as it is contained in src/conf_post.h as well, but the
>> declaration has to be supplied by some other include file.
>>
>> (For FreeBSD, stdlib.h, which alloc.c includes, supplies the
>> declaration, guarded by #if __ISO_C_VISIBLE >= 2011 || __cplusplus >=
>> 201103L, which is true by default, at least on FreeBSD 10).
>
> Cygwin's stdlib.h also has the declaration with the same guard.  But for
> some reason, alloc.c is still not getting the declaration.  I'll have to
> figure out what's going on.

OK, here's what's happening.  config.h defines _GNU_SOURCE.  The 
following excerpt from /usr/include/sys/cdefs.h then causes 
__ISO_C_VISIBLE to be defined as 1999, thereby hiding the declaration of 
aligned_alloc:

#ifdef _GNU_SOURCE
[...]
#define	_XOPEN_SOURCE		700
[...]
#endif
[...]
#if _XOPEN_SOURCE - 0 >= 700
[...]
#define	_POSIX_C_SOURCE		200809
[...]
#endif
[...]
#if _POSIX_C_SOURCE >= 200809
[...]
#define	__ISO_C_VISIBLE		1999
[...]
#endif /* _POSIX_C_SOURCE */

Paul, I can ask on the Cygwin list whether this should be changed to be 
more in line with other platforms.  In the meantime, what's the best way 
to deal with this?

Ken

bug#22522: Commit b88e9cded7ae3756e3a2ec4a23e8df352a0239f9 breaks emacs dumping for me

[Ken Brown]

On 2/2/2016 5:02 PM, Andy Moreton wrote:
> On Tue 02 Feb 2016, Ken Brown wrote:
>
>> On 2/2/2016 9:20 AM, Wolfgang Jenkner wrote:
>>> On Mon, Feb 01 2016, Ken Brown wrote:
>>>
>>>> ../../master/src/alloc.c: In function ‘lisp_align_malloc’:
>>>> ../../master/src/alloc.c:1247:7: warning: implicit declaration of
>>>> function ‘hybrid_aligned_alloc’ [-Wimplicit-function-declaration]
>>>
>>> Before Paul's 7fdc3cf, src/alloc.c used to contain a declaration for
>>> aligned_alloc(), which a preprocessor definition turned into
>>> a declaration for hybrid_aligned_alloc().  The preprocessor definition
>>> was redundant as it is contained in src/conf_post.h as well, but the
>>> declaration has to be supplied by some other include file.
>>>
>>> (For FreeBSD, stdlib.h, which alloc.c includes, supplies the
>>> declaration, guarded by #if __ISO_C_VISIBLE >= 2011 || __cplusplus >=
>>> 201103L, which is true by default, at least on FreeBSD 10).
>>
>> Cygwin's stdlib.h also has the declaration with the same guard.  But for some
>> reason, alloc.c is still not getting the declaration.  I'll have to figure out
>> what's going on.
>
> Building with V=1 shows the cygwin build uses "gcc -std=gnu99", and
> aligned_alloc() is from C11, so I would expect that the declaration in
> Cygwin's stdlib.h is not seen.

That was my first thought also, but it turns out not to be the issue. 
See my last message.

Ken

bug#22522: Commit b88e9cded7ae3756e3a2ec4a23e8df352a0239f9 breaks emacs dumping for me

[Paul Eggert]

[-- Attachment #1: Type: text/plain, Size: 707 bytes --]

On 02/02/2016 02:08 PM, Ken Brown wrote:
> Paul, I can ask on the Cygwin list whether this should be changed to 
> be more in line with other platforms.

Yes it should.  Defining _GNU_SOURCE should make aligned_alloc visible 
regardless of whether -std=c99 is specified. This is because defining 
_GNU_SOURCE means, "Make GNU symbols visible even when compiling 
pedantically." This is OK, since the C standard says the behavior is 
undefined whenever the user defines a reserved symbol like _GNU_SOURCE.

> In the meantime, what's the best way to deal with this?

I installed the attached patch into the Emacs master, to try to work 
around this problem by putting GCC into C11 mode. Please give it a try.

[-- Attachment #2: 0001-Build-with-C11-if-available.patch --]
[-- Type: application/x-patch, Size: 29836 bytes --]

bug#22522: Commit b88e9cded7ae3756e3a2ec4a23e8df352a0239f9 breaks emacs dumping for me

[Paul Eggert]

On 02/02/2016 01:14 PM, Elric Milon wrote:
> I checked out latest master which appears to contain this patch and it's
> building again.
Thanks for checking; closing the bug, as the original problem is fixed. 
We still may have a problem with hybrid_aligned_alloc but if so we can 
open a new bug report for that.

bug#22522: Commit b88e9cded7ae3756e3a2ec4a23e8df352a0239f9 breaks emacs dumping for me

[Ken Brown]

On 2/2/2016 5:54 PM, Paul Eggert wrote:
> On 02/02/2016 02:08 PM, Ken Brown wrote:
>> Paul, I can ask on the Cygwin list whether this should be changed to
>> be more in line with other platforms.
>
> Yes it should.  Defining _GNU_SOURCE should make aligned_alloc visible
> regardless of whether -std=c99 is specified. This is because defining
> _GNU_SOURCE means, "Make GNU symbols visible even when compiling
> pedantically." This is OK, since the C standard says the behavior is
> undefined whenever the user defines a reserved symbol like _GNU_SOURCE.
>
>> In the meantime, what's the best way to deal with this?
>
> I installed the attached patch into the Emacs master, to try to work
> around this problem by putting GCC into C11 mode. Please give it a try.

No, that didn't fix it.  The snippet I posted from <sys/cdefs.h> still 
causes __ISO_C_VISIBLE to be defined as 1999.

What about an ad hoc temporary solution that simply defines 
__ISO_C_VISIBLE to be 2011 on Cygwin prior to the inclusion of 
<stdlib.h>?  Or do you have a better idea?

Ken

P.S. I've started a thread about this on the Cygwin mailing list.

bug#22522: Commit b88e9cded7ae3756e3a2ec4a23e8df352a0239f9 breaks emacs dumping for me

[Ken Brown]

On 2/2/2016 7:09 PM, Ken Brown wrote:
> What about an ad hoc temporary solution that simply defines
> __ISO_C_VISIBLE to be 2011 on Cygwin prior to the inclusion of
> <stdlib.h>?  Or do you have a better idea?

Something like this?

diff --git a/src/conf_post.h b/src/conf_post.h
index c5eec5a..1cb4953 100644
--- a/src/conf_post.h
+++ b/src/conf_post.h
@@ -221,7 +221,17 @@ extern char *emacs_getenv_TZ (void);
  extern int emacs_setenv_TZ (char const *);

  #include <string.h>
+
+/* On Cygwin as of 2016-02-02 __ISO_C_VISIBLE is defined to be 1999 if
+   _GNU_SOURCE is defined.  This hides the declaration of
+   aligned_alloc in <stdlib.h> */
+#ifdef CYGWIN
+#pragma push_macro("__ISO_C_VISIBLE")
+#undef __ISO_C_VISIBLE
+#define __ISO_C_VISIBLE 2011
  #include <stdlib.h>
+#pragma pop_macro("__ISO_C_VISIBLE")
+#endif

  #if __GNUC__ >= 3  /* On GCC 3.0 we might get a warning.  */
  #define NO_INLINE __attribute__((noinline))

Ken

bug#22522: Commit b88e9cded7ae3756e3a2ec4a23e8df352a0239f9 breaks emacs dumping for me

[Paul Eggert]

[-- Attachment #1: Type: text/plain, Size: 791 bytes --]

Ken Brown wrote:
>> I installed the attached patch into the Emacs master, to try to work
>> around this problem by putting GCC into C11 mode. Please give it a try.
>
> No, that didn't fix it.

Can you investigate why not?  The patch should cause 'configure' to put 
-stdc=gnu11 into CFLAGS. If you look at config.log, it should have something 
like this:

configure:5924: checking for gcc option to enable C11 features
...
configure:6146: result: -std=gnu11

If this isn't the result, what is the result and why?

> What about an ad hoc temporary solution that simply defines __ISO_C_VISIBLE to be 2011 on Cygwin prior to the inclusion of <stdlib.h>?

I'd rather not go that route, as __ISO_C_VISIBLE is supposed to be private. I 
installed the attached patch instead; please give it a try.

[-- Attachment #2: 0001-Port-aligned_alloc-decl-to-Cygwin.patch --]
[-- Type: text/x-diff, Size: 1600 bytes --]

From 0fed204ff1adab4636cad9c6492b3a96fabf1010 Mon Sep 17 00:00:00 2001
From: Paul Eggert <eggert@cs.ucla.edu>
Date: Wed, 3 Feb 2016 00:37:44 -0800
Subject: [PATCH] Port aligned_alloc decl to Cygwin.

Problem reported by Ken Brown (Bug#22522#38).
* configure.ac (aligned_alloc): Check for decl too.
* src/lisp.h (aligned_alloc): Declare if not already declared.
---
 configure.ac | 1 +
 src/lisp.h   | 5 ++---
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/configure.ac b/configure.ac
index d97d9e5..1e076c7 100644
--- a/configure.ac
+++ b/configure.ac
@@ -3824,6 +3824,7 @@ if (test -z "$GMALLOC_OBJ" || test "$hybrid_malloc" = yes) \
   && test "$opsys" != darwin; then
   AC_CHECK_FUNCS([aligned_alloc posix_memalign], [break])
 fi
+AC_CHECK_DECLS([aligned_alloc], [], [], [[#include <stdlib.h>]])
 
 dnl Cannot use AC_CHECK_FUNCS
 AC_CACHE_CHECK([for __builtin_unwind_init],
diff --git a/src/lisp.h b/src/lisp.h
index 54bce0f..a99002b 100644
--- a/src/lisp.h
+++ b/src/lisp.h
@@ -3774,10 +3774,9 @@ INLINE void (check_cons_list) (void) { lisp_h_check_cons_list (); }
 /* Defined in gmalloc.c.  */
 #if !defined DOUG_LEA_MALLOC && !defined HYBRID_MALLOC && !defined SYSTEM_MALLOC
 extern size_t __malloc_extra_blocks;
-extern void *aligned_alloc (size_t, size_t);
 #endif
-#if defined HYBRID_MALLOC && !defined HAVE_ALIGNED_ALLOC
-extern void *hybrid_aligned_alloc (size_t, size_t) ATTRIBUTE_MALLOC_SIZE ((2));
+#ifndef HAVE_DECL_ALIGNED_ALLOC
+extern void *aligned_alloc (size_t, size_t) ATTRIBUTE_MALLOC_SIZE ((2));
 #endif
 extern void malloc_enable_thread (void);
 
-- 
2.5.0

bug#22522: Commit b88e9cded7ae3756e3a2ec4a23e8df352a0239f9 breaks emacs dumping for me

[Ken Brown]

On 2/3/2016 3:41 AM, Paul Eggert wrote:
> Ken Brown wrote:
>>> I installed the attached patch into the Emacs master, to try to work
>>> around this problem by putting GCC into C11 mode. Please give it a try.
>>
>> No, that didn't fix it.
> 
> Can you investigate why not?  The patch should cause 'configure' to put 
> -stdc=gnu11 into CFLAGS. If you look at config.log, it should have 
> something like this:
> 
> configure:5924: checking for gcc option to enable C11 features
> ...
> configure:6146: result: -std=gnu11
> 
> If this isn't the result, what is the result and why?

Yes, that's the result.  Compiling with that option caused __STDC_VERSION__ to be defined as 201112L.  But it didn't affect __ISO_C_VISIBLE, which is what guards the declaration of aligned_alloc.

> I'd rather not go that route, as __ISO_C_VISIBLE is supposed to be private. I installed the attached patch instead; please give it a try.

We're almost there.  After your change, I get the following in config.h:

/* Define to 1 if you have the declaration of `aligned_alloc', and to 0 if you
   don't. */
#define HAVE_DECL_ALIGNED_ALLOC 0

So we need the following additional change:

diff --git a/src/lisp.h b/src/lisp.h
index a99002b..c8363be 100644
--- a/src/lisp.h
+++ b/src/lisp.h
@@ -3775,7 +3775,7 @@ INLINE void (check_cons_list) (void) { lisp_h_check_cons_list (); }
 #if !defined DOUG_LEA_MALLOC && !defined HYBRID_MALLOC && !defined SYSTEM_MALLOC
 extern size_t __malloc_extra_blocks;
 #endif
-#ifndef HAVE_DECL_ALIGNED_ALLOC
+#if !HAVE_DECL_ALIGNED_ALLOC
 extern void *aligned_alloc (size_t, size_t) ATTRIBUTE_MALLOC_SIZE ((2));
 #endif
 extern void malloc_enable_thread (void);

Thanks.

Ken

bug#22522: Commit b88e9cded7ae3756e3a2ec4a23e8df352a0239f9 breaks emacs dumping for me

[Andy Moreton]

On Wed 03 Feb 2016, Ken Brown wrote:

> On 2/3/2016 3:41 AM, Paul Eggert wrote:
>> Ken Brown wrote:
>>>> I installed the attached patch into the Emacs master, to try to work
>>>> around this problem by putting GCC into C11 mode. Please give it a try.
>>>
>>> No, that didn't fix it.
>> 
>> Can you investigate why not?  The patch should cause 'configure' to put 
>> -stdc=gnu11 into CFLAGS. If you look at config.log, it should have 
>> something like this:
>> 
>> configure:5924: checking for gcc option to enable C11 features
>> ...
>> configure:6146: result: -std=gnu11
>> 
>> If this isn't the result, what is the result and why?
>
> Yes, that's the result. Compiling with that option caused __STDC_VERSION__ to
> be defined as 201112L. But it didn't affect __ISO_C_VISIBLE, which is what
> guards the declaration of aligned_alloc.
>
>> I'd rather not go that route, as __ISO_C_VISIBLE is supposed to be private.
>> I installed the attached patch instead; please give it a try.
>
> We're almost there.  After your change, I get the following in config.h:
>
> /* Define to 1 if you have the declaration of `aligned_alloc', and to 0 if you
>    don't. */
> #define HAVE_DECL_ALIGNED_ALLOC 0
>
> So we need the following additional change:
>
> diff --git a/src/lisp.h b/src/lisp.h
> index a99002b..c8363be 100644
> --- a/src/lisp.h
> +++ b/src/lisp.h
> @@ -3775,7 +3775,7 @@ INLINE void (check_cons_list) (void) { lisp_h_check_cons_list (); }
>  #if !defined DOUG_LEA_MALLOC && !defined HYBRID_MALLOC && !defined SYSTEM_MALLOC
>  extern size_t __malloc_extra_blocks;
>  #endif
> -#ifndef HAVE_DECL_ALIGNED_ALLOC
> +#if !HAVE_DECL_ALIGNED_ALLOC
>  extern void *aligned_alloc (size_t, size_t) ATTRIBUTE_MALLOC_SIZE ((2));
>  #endif
>  extern void malloc_enable_thread (void);
>

With this change applied to changeset 5fcd89f52ec6, I have successfully built:
  64bit cygwin w32
  64bit mingw64
  32bit mingw32
  32bit mingw32 --with-wide-int

Thanks Ken (and Paul) for sorting this out.

   AndyM

bug#22522: Commit b88e9cded7ae3756e3a2ec4a23e8df352a0239f9 breaks emacs dumping for me

[Ken Brown]

On 2/3/2016 9:01 AM, Andy Moreton wrote:
> With this change applied to changeset 5fcd89f52ec6, I have successfully built:
>    64bit cygwin w32
>    64bit mingw64
>    32bit mingw32
>    32bit mingw32 --with-wide-int
>
> Thanks Ken (and Paul) for sorting this out.

Thanks for testing.  I've pushed the change.

Ken