bug#22086: 25.1.50; [PATCH] Integrate the musl hybrid malloc patch for elf systems

[Daniel Colascione <dancol@dancol.org>]

On December 21, 2015 12:49:13 PM PST, Rich Felker <dalias@aerifal.cx> wrote:
>On Mon, Dec 21, 2015 at 12:08:46PM -0800, Daniel Colascione wrote:
>> On 12/20/2015 08:06 PM, Rich Felker wrote:
>> > On Sun, Dec 20, 2015 at 10:37:24PM -0500, Ken Brown wrote:
>> >> On 12/20/2015 5:33 PM, Paul Eggert wrote:
>> >>> While thinking over this patch I'd like to propose what should be
>a
>> >>> simpler approach. This new proposal is more radical, and so
>should not
>> >>> be applied to the emacs-25 branch, but it should make the port to
>musl
>> >>> etc. automatic.
>> >>>
>> >>> The simpler approach is to remove gmalloc.c, and to use the
>system
>> >>> memory allocator, i.e., to behave as if SYSTEM_MALLOC is defined
>on all
>> >>> platforms.
>> >>>
>> >>> We can still support hybrid malloc for Cygwin, if SYSTEM_MALLOC
>wouldn't
>> >>> work on Cygwin for some reason; and we can support the similar
>hybrid on
>> >>> Darwin, if it's still needed.
>> >>
>> >> SYSTEM_MALLOC doesn't work on Cygwin, largely because Cygwin's
>> >> malloc doesn't support malloc_set_state and malloc_get_state. 
>There
>> >> may be other problems too.  (It's been a while since I tried it.)
>> > 
>> > I don't see how this is possible; malloc_[gs]et_state do not exist
>on
>> > other systems either. Presumably this is some hack needed for the
>> > dumper, which wouldn't be needed if malloc weren't used
>pre-dumping.
>> 
>> We really shouldn't be dumping the native heap at all, really.
>> Eventually, Emacs should be a position-independent executable (as
>should
>> every other program), and to unexec a PIE, we need to emit
>relocations,
>> and to emit relocations, we need to know where all the pointers are.
>We
>> can't do that if the internal heap structure is opaque to us.
>
>Actually you can, because the internal heap structure is not what you
>need to dump. The "lisp heap" is what you need to dump, and it's
>walkable in the same way you walk it now for garbage collection
>purposes. It should be possible to adapt the GC code into a dumper
>that dumps only the lisp data (with relocations in a form emacs can
>internally process, i.e. not relying on writing a new executable
>binary with ELF-level or other system-specific relocs) and no unwanted
>additional state; then, even static linking should work correctly.
>
>Rich

Sure. That's the XEmacs portable dumper approach, and it works all right. I'm just worried that we might have implicit dependencies on the C heap being preserved across unexec.

The same code we use to relocate when Emacs is a PIE would also help us do GC compaction. (We'd have to pin conservatively reachable roots.)