From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Eli Zaretskii Newsgroups: gmane.emacs.devel Subject: Re: Dumper problems and a possible solutions Date: Wed, 25 Jun 2014 23:15:02 +0300 Message-ID: <83tx78pwzd.fsf@gnu.org> References: <20140624171955.GS179@brightrain.aerifal.cx> <53AB0EF8.4090608@yandex.ru> <831tucrguf.fsf@gnu.org> <20140625183241.GW179@brightrain.aerifal.cx> <83wqc4q0xl.fsf@gnu.org> <20140625190333.GZ179@brightrain.aerifal.cx> <83vbropzlz.fsf@gnu.org> <20140625195730.GA179@brightrain.aerifal.cx> Reply-To: Eli Zaretskii NNTP-Posting-Host: plane.gmane.org X-Trace: ger.gmane.org 1403727333 5107 80.91.229.3 (25 Jun 2014 20:15:33 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Wed, 25 Jun 2014 20:15:33 +0000 (UTC) Cc: dmantipov@yandex.ru, emacs-devel@gnu.org To: Rich Felker Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Wed Jun 25 22:15:25 2014 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1WztbH-0002uO-E6 for ged-emacs-devel@m.gmane.org; Wed, 25 Jun 2014 22:15:23 +0200 Original-Received: from localhost ([::1]:40840 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WztbH-0007aT-03 for ged-emacs-devel@m.gmane.org; Wed, 25 Jun 2014 16:15:23 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:47290) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WztbA-0007a3-BW for emacs-devel@gnu.org; Wed, 25 Jun 2014 16:15:21 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Wztb5-0005ga-Gk for emacs-devel@gnu.org; Wed, 25 Jun 2014 16:15:16 -0400 Original-Received: from mtaout20.012.net.il ([80.179.55.166]:53881) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Wztb5-0005gA-9h for emacs-devel@gnu.org; Wed, 25 Jun 2014 16:15:11 -0400 Original-Received: from conversion-daemon.a-mtaout20.012.net.il by a-mtaout20.012.net.il (HyperSendmail v2007.08) id <0N7Q00200QV7XD00@a-mtaout20.012.net.il> for emacs-devel@gnu.org; Wed, 25 Jun 2014 23:15:10 +0300 (IDT) Original-Received: from HOME-C4E4A596F7 ([87.69.4.28]) by a-mtaout20.012.net.il (HyperSendmail v2007.08) with ESMTPA id <0N7Q0024WQX9H3D0@a-mtaout20.012.net.il>; Wed, 25 Jun 2014 23:15:10 +0300 (IDT) In-reply-to: <20140625195730.GA179@brightrain.aerifal.cx> X-012-Sender: halo1@inter.net.il X-detected-operating-system: by eggs.gnu.org: Solaris 10 X-Received-From: 80.179.55.166 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.devel:172726 Archived-At: > Date: Wed, 25 Jun 2014 15:57:30 -0400 > From: Rich Felker > Cc: dmantipov@yandex.ru, emacs-devel@gnu.org > > > But I still don't understand how you get to 400MB. It's not that we > > allocate hundreds of those 700K tables for charsets. Do you have an > > explanation for this? > > Not hundreds at a time, but if the malloc operation is just positive > (fake-)sbrk and the free operation is a nop, hundreds of such charset > load operations will quickly add up. Free operation shouldn't be a no-op, not in malloc. And still, there are only a few (maybe 10) times we allocate these 700K tables, so 400MB sound very strange to me. > > Sorry, I don't see the difficulty. Just make malloc/realloc/free be > > pointers that point to gmalloc's implementation before dumping, and to > > the libc implementation after it. You may need some #define to rename > > malloc to some other symbol, to avoid name clashes. Am I missing > > something? > > Yeah, what happens if, after dumping, the real emacs at runtime ends > up calling free() on one of the pre-dump pointers? You intercept the call and do nothing. > > > No, it's less reliable. See my other posts in the thread about what > > > happens if you have other libraries linked and they do nontrivial > > > things prior to dumping (e.g. from static ctors). > > > > But in those other posts I thought we agreed that whatever those ctors > > do is irrelevant, as the dumped Emacs cannot possibly use what they > > allocate, and those ctors will be invoked again in the dumped Emacs. > > Those ctors are free to inspect global data. For example one might > contain (this sort of idiom is necessary if you can't control the > relative order of ctors): if (!init) { do_something(); init=1; }. In > that case, the dump would save the value of init, and do_something() > would fail to happen at runtime. That's the same problem as with your clock_gettime, and it must be fixed anyway, because any ctor run at dump time is almost certainly picking up data that is irrelevant to the run time. > > > As an example (I think I mentioned this earlier), if you static > > > link, musl libc is remembering the clock_gettime vdso pointer from > > > the pre-dump state and attempting to use it later (which is not > > > valid because the kernel maps it at a random address). > > > > You need to fix your libc to detect that it is re-exec'ed, and > > re-initialize this stuff. > > I don't see any easy way to do that that's not a big violation of > encapsulation (startup code has no business knowing about > clock_gettime internals) and the whole efficiency of static linking > (startup code does not pull in any vdso stuff; that only gets pulled > in by using a function that needs a vdso lookup). One way is to have a counter that gets incremented each re-exec. Then the initialization code of clock_gettime could examine the counter and re-initialize when its value changes since last time.