From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Eli Zaretskii Newsgroups: gmane.emacs.devel Subject: Re: Dumper problems and a possible solutions Date: Thu, 26 Jun 2014 05:44:44 +0300 Message-ID: <83simspexv.fsf@gnu.org> References: <20140624171955.GS179@brightrain.aerifal.cx> <53AB0EF8.4090608@yandex.ru> <831tucrguf.fsf@gnu.org> <20140625183241.GW179@brightrain.aerifal.cx> <83wqc4q0xl.fsf@gnu.org> <20140625190333.GZ179@brightrain.aerifal.cx> <83vbropzlz.fsf@gnu.org> <20140625195730.GA179@brightrain.aerifal.cx> <83tx78pwzd.fsf@gnu.org> <20140625203403.GC179@brightrain.aerifal.cx> Reply-To: Eli Zaretskii NNTP-Posting-Host: plane.gmane.org X-Trace: ger.gmane.org 1403750721 29987 80.91.229.3 (26 Jun 2014 02:45:21 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Thu, 26 Jun 2014 02:45:21 +0000 (UTC) Cc: dmantipov@yandex.ru, emacs-devel@gnu.org To: Rich Felker Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Thu Jun 26 04:45:13 2014 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1WzzgW-0004GG-SS for ged-emacs-devel@m.gmane.org; Thu, 26 Jun 2014 04:45:12 +0200 Original-Received: from localhost ([::1]:41966 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WzzgW-0005tg-G7 for ged-emacs-devel@m.gmane.org; Wed, 25 Jun 2014 22:45:12 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:34310) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WzzgP-0005q2-Vq for emacs-devel@gnu.org; Wed, 25 Jun 2014 22:45:10 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1WzzgL-0005iB-7j for emacs-devel@gnu.org; Wed, 25 Jun 2014 22:45:05 -0400 Original-Received: from mtaout21.012.net.il ([80.179.55.169]:37338) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WzzgK-0005i7-Vl for emacs-devel@gnu.org; Wed, 25 Jun 2014 22:45:01 -0400 Original-Received: from conversion-daemon.a-mtaout21.012.net.il by a-mtaout21.012.net.il (HyperSendmail v2007.08) id <0N7R00B0083BWZ00@a-mtaout21.012.net.il> for emacs-devel@gnu.org; Thu, 26 Jun 2014 05:44:59 +0300 (IDT) Original-Received: from HOME-C4E4A596F7 ([87.69.4.28]) by a-mtaout21.012.net.il (HyperSendmail v2007.08) with ESMTPA id <0N7R00BIC8YZWS40@a-mtaout21.012.net.il>; Thu, 26 Jun 2014 05:44:59 +0300 (IDT) In-reply-to: <20140625203403.GC179@brightrain.aerifal.cx> X-012-Sender: halo1@inter.net.il X-detected-operating-system: by eggs.gnu.org: Solaris 10 X-Received-From: 80.179.55.169 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.devel:172741 Archived-At: > Date: Wed, 25 Jun 2014 16:34:03 -0400 > From: Rich Felker > Cc: dmantipov@yandex.ru, emacs-devel@gnu.org > > > And still, there are only a few (maybe 10) times we allocate these > > 700K tables, so 400MB sound very strange to me. > > In my log, I see 768k allocations occuring roughly 94 times. 768K times 94 doesn't get anywhere close to 400MB. > Is it possible that the temacs --batch commands I'm testing (IIRC > taken from commands that were failing in leim/Makefile, but perhaps > I changed it in some way I didn't notice?) are pulling in my .emacs > file, which might be causing more charsets to be loaded? No, it's more probable that I was mistaken about the count. But still, 400MB sounds way too much. > > > > Sorry, I don't see the difficulty. Just make malloc/realloc/free be > > > > pointers that point to gmalloc's implementation before dumping, and to > > > > the libc implementation after it. You may need some #define to rename > > > > malloc to some other symbol, to avoid name clashes. Am I missing > > > > something? > > > > > > Yeah, what happens if, after dumping, the real emacs at runtime ends > > > up calling free() on one of the pre-dump pointers? > > > > You intercept the call and do nothing. > > Right, but the free pointer can't directly point to the real (libc) > free. It has to point to the wrapper that does this range-check. Right. > > > Those ctors are free to inspect global data. For example one might > > > contain (this sort of idiom is necessary if you can't control the > > > relative order of ctors): if (!init) { do_something(); init=1; }. In > > > that case, the dump would save the value of init, and do_something() > > > would fail to happen at runtime. > > > > That's the same problem as with your clock_gettime, and it must be > > fixed anyway, because any ctor run at dump time is almost certainly > > picking up data that is irrelevant to the run time. > > Libc could _possibly_ work around it by virtue of having full control > over the init code. For other libraries, the issue is not fixable (see > my above example with code that has to control dependency order of > ctors), and shouldn't have to be fixed. But the problem likely doesn't exist, because otherwise we will have known about it by now. Emacs cannot use such libraries.