Re: Dumper problems and a possible solutions

[Rich Felker <dalias@libc.org>]

On Wed, Jun 25, 2014 at 04:53:58PM -0400, Samuel Bronson wrote:
> On 6/25/14, Rich Felker <dalias@libc.org> wrote:
> 
> > In musl's malloc, we use brk if it's available (note: with PIE, most
> > kernels give you almost no available brk space due to the way the
> > mappings are laid out) [...]
> 
> Yeah, that tiny gap has bitten in other ways, too:
> <http://www.postgresql.org/message-id/20140519115318.GB7296@msgid.df7cb.de>
> talks about a stack overflow with the same cause; I really think the
> kernel should stop doing that.

Agreed. But it was a big enough issue (basically a show-stopper for
using PIE) that I just added a simple way to use mmap without having
to care about discontiguity because the waste is asymptotically zero.

> > Also, musl does not provide a working sbrk at all, since synchronizing
> > with an application's use of sbrk would introduce performance costs
> > into all correct applications that don't poke at the brk.
> 
> Looking at the manpage, I can't really follow how having sbrk() would
> involve a slowdown in everything? Do you mean that musl's malloc gets
> a speed bonus out of assuming it's the sole user of brk()/sbrk(), and
> thus the whole region from the initial brk()point to the current
> brk()point is belongs to it?

Yes, basically. malloc simply assumes the brk is where it last left
it, rather than querying it again, which would double the syscall
overhead. This could be avoided via a cooperating lock between sbrk
and malloc, but basically after sbrk touches the brk, malloc could not
safely use it anymore; if it did, the application might later clobber
malloc's heap.

brk/sbrk are documented as not being usable if malloc is used (see
http://pubs.opengroup.org/onlinepubs/7908799/xsh/brk.html) and they
were later removed from the standards because that essentially makes
it impossible to use them at all.

There's an ongoing discussion on whether it's desirable to provide an
emulated sbrk for legacy applications (note: these applications would
not work with real sbrk anyway if compiled as PIE!) but since there
are basically no users of brk/sbrk left except malloc implementations,
nobody has really cared much one way or the other whether it gets
added.

> (Yeah, all (potentially) 125 MiB -
> stacksize of it!)

I don't follow. brk has nothing to do with the stack, and in practice
it can obtain nearly the full 3GB (for 32-bit) of available virtual
address space in non-PIE programs.

Anyway, discussion of musl's malloc implementation is largely
off-topic to this discussion except insomuch as it reflects the
diversity of implementation possibilities that it would be nice for
emacs to support without ugly hacks. I'd rather this thread not be
"please change emacs because __________ in musl" but rather "let's
address a long-standing portability issue in emacs in such a way that
it won't come up again, and possibly gives other benefits like support
for PIE and fixing ugly corner-cases that are going to arise now that
modern emacs is linking so many libraries".

Rich