Re: Dumper problems and a possible solutions

[Eli Zaretskii <eliz@gnu.org>]

> Date: Wed, 25 Jun 2014 15:57:30 -0400
> From: Rich Felker <dalias@libc.org>
> Cc: dmantipov@yandex.ru, emacs-devel@gnu.org
> 
> > But I still don't understand how you get to 400MB.  It's not that we
> > allocate hundreds of those 700K tables for charsets.  Do you have an
> > explanation for this?
> 
> Not hundreds at a time, but if the malloc operation is just positive
> (fake-)sbrk and the free operation is a nop, hundreds of such charset
> load operations will quickly add up.

Free operation shouldn't be a no-op, not in malloc.

And still, there are only a few (maybe 10) times we allocate these
700K tables, so 400MB sound very strange to me.

> > Sorry, I don't see the difficulty.  Just make malloc/realloc/free be
> > pointers that point to gmalloc's implementation before dumping, and to
> > the libc implementation after it.  You may need some #define to rename
> > malloc to some other symbol, to avoid name clashes.  Am I missing
> > something?
> 
> Yeah, what happens if, after dumping, the real emacs at runtime ends
> up calling free() on one of the pre-dump pointers?

You intercept the call and do nothing.

> > > No, it's less reliable. See my other posts in the thread about what
> > > happens if you have other libraries linked and they do nontrivial
> > > things prior to dumping (e.g. from static ctors).
> > 
> > But in those other posts I thought we agreed that whatever those ctors
> > do is irrelevant, as the dumped Emacs cannot possibly use what they
> > allocate, and those ctors will be invoked again in the dumped Emacs.
> 
> Those ctors are free to inspect global data. For example one might
> contain (this sort of idiom is necessary if you can't control the
> relative order of ctors): if (!init) { do_something(); init=1; }. In
> that case, the dump would save the value of init, and do_something()
> would fail to happen at runtime.

That's the same problem as with your clock_gettime, and it must be
fixed anyway, because any ctor run at dump time is almost certainly
picking up data that is irrelevant to the run time.

> > > As an example (I think I mentioned this earlier), if you static
> > > link, musl libc is remembering the clock_gettime vdso pointer from
> > > the pre-dump state and attempting to use it later (which is not
> > > valid because the kernel maps it at a random address).
> > 
> > You need to fix your libc to detect that it is re-exec'ed, and
> > re-initialize this stuff.
> 
> I don't see any easy way to do that that's not a big violation of
> encapsulation (startup code has no business knowing about
> clock_gettime internals) and the whole efficiency of static linking
> (startup code does not pull in any vdso stuff; that only gets pulled
> in by using a function that needs a vdso lookup).

One way is to have a counter that gets incremented each re-exec.  Then
the initialization code of clock_gettime could examine the counter and
re-initialize when its value changes since last time.