Re: Dumper problems and a possible solutions

[Eli Zaretskii <eliz@gnu.org>]

> Date: Thu, 26 Jun 2014 00:28:39 -0400
> From: Rich Felker <dalias@libc.org>
> Cc: dmantipov@yandex.ru, emacs-devel@gnu.org
> 
> > > In my log, I see 768k allocations occuring roughly 94 times.
> > 
> > 768K times 94 doesn't get anywhere close to 400MB.
> 
> Yes, there were also a number of ~200k and ~400k allocations though,
> which I did not get around to identifying the source of.

If your 'free' is a no-op, I guess it could explain any number,
including 400MB.

Once again, I suggest to use gmalloc, or some existing less simplistic
implementation, instead.

Or, put it another way: if MS-Windows provides a way to use the libc
malloc with a custom 'sbrk' function, then perhaps the free software
platforms should have a similar feature as well (if they don't
already).  Once such a feature is provided, we could supply such a
custom 'sbrk' for the dumping phase, which will put all the data in a
static array, which will cleanly solve this issue for the observable
future.

> > > > > Those ctors are free to inspect global data. For example one might
> > > > > contain (this sort of idiom is necessary if you can't control the
> > > > > relative order of ctors): if (!init) { do_something(); init=1; }. In
> > > > > that case, the dump would save the value of init, and do_something()
> > > > > would fail to happen at runtime.
> > > > 
> > > > That's the same problem as with your clock_gettime, and it must be
> > > > fixed anyway, because any ctor run at dump time is almost certainly
> > > > picking up data that is irrelevant to the run time.
> > > 
> > > Libc could _possibly_ work around it by virtue of having full control
> > > over the init code. For other libraries, the issue is not fixable (see
> > > my above example with code that has to control dependency order of
> > > ctors), and shouldn't have to be fixed.
> > 
> > But the problem likely doesn't exist, because otherwise we will have
> > known about it by now.  Emacs cannot use such libraries.
> 
> I wouldn't be so sure. How much testing is even done with static
> linking?

In the past, quite a lot (the Emacs dumping method didn't change
significantly since about day one).  Nowadays, I don't think it is
tested much, except in the DJGPP (a.k.a. "MS-DOS") build of Emacs,
which uses static linking exclusively, because DJGPP doesn't support
shared libraries.  DJGPP's libc implements the startup code trick I
mentioned that forces reinitialization of state by functions which
need that, when they are first called in the dumped Emacs.  What's
left can be seen in unexcoff.c, where some of the global state is
explicitly reverted to its initial state before writing the image to
disk, see there in 'copy_text_and_data'.  Btw, one of these is the
'atexit' chain, which is probably relevant to other systems, if and
when they are statically linked.

> With dynamic linking, the library's state will all be lost
> across dump. Lots of the libraries emacs can optionally use have
> sketchy global state, and I wouldn't be surprised at all if at least
> one of them were failing to properly initialize in the post-dump
> emacs. The symptoms might not even be immediately visible if the state
> saved when dumping were "close enough" to correct to be used
> post-dump.

But if static linking is not used anymore on supported platforms
(putting MS-DOS aside), then this is not an issue, right?