From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Eli Zaretskii Newsgroups: gmane.emacs.devel Subject: Re: Dumper problems and a possible solutions Date: Thu, 26 Jun 2014 18:02:18 +0300 Message-ID: <83ionnpvd1.fsf@gnu.org> References: <53AB0EF8.4090608@yandex.ru> <831tucrguf.fsf@gnu.org> <20140625183241.GW179@brightrain.aerifal.cx> <83wqc4q0xl.fsf@gnu.org> <20140625190333.GZ179@brightrain.aerifal.cx> <83vbropzlz.fsf@gnu.org> <20140625195730.GA179@brightrain.aerifal.cx> <83tx78pwzd.fsf@gnu.org> <20140625203403.GC179@brightrain.aerifal.cx> <83simspexv.fsf@gnu.org> <20140626042839.GI179@brightrain.aerifal.cx> Reply-To: Eli Zaretskii NNTP-Posting-Host: plane.gmane.org X-Trace: ger.gmane.org 1403794982 29873 80.91.229.3 (26 Jun 2014 15:03:02 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Thu, 26 Jun 2014 15:03:02 +0000 (UTC) Cc: dmantipov@yandex.ru, emacs-devel@gnu.org To: Rich Felker Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Thu Jun 26 17:02:52 2014 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1X0BCN-0002bI-Ls for ged-emacs-devel@m.gmane.org; Thu, 26 Jun 2014 17:02:51 +0200 Original-Received: from localhost ([::1]:45168 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1X0BCN-0004aK-BB for ged-emacs-devel@m.gmane.org; Thu, 26 Jun 2014 11:02:51 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:60632) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1X0BCD-0004Ys-6c for emacs-devel@gnu.org; Thu, 26 Jun 2014 11:02:49 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1X0BC7-0006IS-05 for emacs-devel@gnu.org; Thu, 26 Jun 2014 11:02:41 -0400 Original-Received: from mtaout28.012.net.il ([80.179.55.184]:47016) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1X0BC6-0006I6-FI for emacs-devel@gnu.org; Thu, 26 Jun 2014 11:02:34 -0400 Original-Received: from conversion-daemon.mtaout28.012.net.il by mtaout28.012.net.il (HyperSendmail v2007.08) id <0N7S006006PMXH00@mtaout28.012.net.il> for emacs-devel@gnu.org; Thu, 26 Jun 2014 18:01:23 +0300 (IDT) Original-Received: from HOME-C4E4A596F7 ([87.69.4.28]) by mtaout28.012.net.il (HyperSendmail v2007.08) with ESMTPA id <0N7S002B472BLG60@mtaout28.012.net.il>; Thu, 26 Jun 2014 18:01:23 +0300 (IDT) In-reply-to: <20140626042839.GI179@brightrain.aerifal.cx> X-012-Sender: halo1@inter.net.il X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 80.179.55.184 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.devel:172753 Archived-At: > Date: Thu, 26 Jun 2014 00:28:39 -0400 > From: Rich Felker > Cc: dmantipov@yandex.ru, emacs-devel@gnu.org > > > > In my log, I see 768k allocations occuring roughly 94 times. > > > > 768K times 94 doesn't get anywhere close to 400MB. > > Yes, there were also a number of ~200k and ~400k allocations though, > which I did not get around to identifying the source of. If your 'free' is a no-op, I guess it could explain any number, including 400MB. Once again, I suggest to use gmalloc, or some existing less simplistic implementation, instead. Or, put it another way: if MS-Windows provides a way to use the libc malloc with a custom 'sbrk' function, then perhaps the free software platforms should have a similar feature as well (if they don't already). Once such a feature is provided, we could supply such a custom 'sbrk' for the dumping phase, which will put all the data in a static array, which will cleanly solve this issue for the observable future. > > > > > Those ctors are free to inspect global data. For example one might > > > > > contain (this sort of idiom is necessary if you can't control the > > > > > relative order of ctors): if (!init) { do_something(); init=1; }. In > > > > > that case, the dump would save the value of init, and do_something() > > > > > would fail to happen at runtime. > > > > > > > > That's the same problem as with your clock_gettime, and it must be > > > > fixed anyway, because any ctor run at dump time is almost certainly > > > > picking up data that is irrelevant to the run time. > > > > > > Libc could _possibly_ work around it by virtue of having full control > > > over the init code. For other libraries, the issue is not fixable (see > > > my above example with code that has to control dependency order of > > > ctors), and shouldn't have to be fixed. > > > > But the problem likely doesn't exist, because otherwise we will have > > known about it by now. Emacs cannot use such libraries. > > I wouldn't be so sure. How much testing is even done with static > linking? In the past, quite a lot (the Emacs dumping method didn't change significantly since about day one). Nowadays, I don't think it is tested much, except in the DJGPP (a.k.a. "MS-DOS") build of Emacs, which uses static linking exclusively, because DJGPP doesn't support shared libraries. DJGPP's libc implements the startup code trick I mentioned that forces reinitialization of state by functions which need that, when they are first called in the dumped Emacs. What's left can be seen in unexcoff.c, where some of the global state is explicitly reverted to its initial state before writing the image to disk, see there in 'copy_text_and_data'. Btw, one of these is the 'atexit' chain, which is probably relevant to other systems, if and when they are statically linked. > With dynamic linking, the library's state will all be lost > across dump. Lots of the libraries emacs can optionally use have > sketchy global state, and I wouldn't be surprised at all if at least > one of them were failing to properly initialize in the post-dump > emacs. The symptoms might not even be immediately visible if the state > saved when dumping were "close enough" to correct to be used > post-dump. But if static linking is not used anymore on supported platforms (putting MS-DOS aside), then this is not an issue, right?