From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Rich Felker Newsgroups: gmane.emacs.devel Subject: Re: Dumper problems and a possible solutions Date: Wed, 25 Jun 2014 16:34:03 -0400 Message-ID: <20140625203403.GC179@brightrain.aerifal.cx> References: <20140624171955.GS179@brightrain.aerifal.cx> <53AB0EF8.4090608@yandex.ru> <831tucrguf.fsf@gnu.org> <20140625183241.GW179@brightrain.aerifal.cx> <83wqc4q0xl.fsf@gnu.org> <20140625190333.GZ179@brightrain.aerifal.cx> <83vbropzlz.fsf@gnu.org> <20140625195730.GA179@brightrain.aerifal.cx> <83tx78pwzd.fsf@gnu.org> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: ger.gmane.org 1403728471 18823 80.91.229.3 (25 Jun 2014 20:34:31 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Wed, 25 Jun 2014 20:34:31 +0000 (UTC) Cc: dmantipov@yandex.ru, emacs-devel@gnu.org To: Eli Zaretskii Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Wed Jun 25 22:34:23 2014 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1Wzttc-0001Ye-Ty for ged-emacs-devel@m.gmane.org; Wed, 25 Jun 2014 22:34:21 +0200 Original-Received: from localhost ([::1]:40900 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Wzttc-0007wd-3P for ged-emacs-devel@m.gmane.org; Wed, 25 Jun 2014 16:34:20 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:52251) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WzttV-0007vR-9H for emacs-devel@gnu.org; Wed, 25 Jun 2014 16:34:17 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1WzttR-00045n-LA for emacs-devel@gnu.org; Wed, 25 Jun 2014 16:34:13 -0400 Original-Received: from 216-12-86-13.cv.mvl.ntelos.net ([216.12.86.13]:44438 helo=brightrain.aerifal.cx) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WzttM-00044J-PI; Wed, 25 Jun 2014 16:34:04 -0400 Original-Received: from dalias by brightrain.aerifal.cx with local (Exim 3.15 #2) id 1WzttL-0005C2-00; Wed, 25 Jun 2014 20:34:03 +0000 Content-Disposition: inline In-Reply-To: <83tx78pwzd.fsf@gnu.org> User-Agent: Mutt/1.5.21 (2010-09-15) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 216.12.86.13 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.devel:172728 Archived-At: On Wed, Jun 25, 2014 at 11:15:02PM +0300, Eli Zaretskii wrote: > > Date: Wed, 25 Jun 2014 15:57:30 -0400 > > From: Rich Felker > > Cc: dmantipov@yandex.ru, emacs-devel@gnu.org > > > > > But I still don't understand how you get to 400MB. It's not that we > > > allocate hundreds of those 700K tables for charsets. Do you have an > > > explanation for this? > > > > Not hundreds at a time, but if the malloc operation is just positive > > (fake-)sbrk and the free operation is a nop, hundreds of such charset > > load operations will quickly add up. > > Free operation shouldn't be a no-op, not in malloc. Agreed. But the question was about why my quick hack took 400MB, and the answer is that it was using a static fake-brk with malloc=sbrk and free=nop. > And still, there are only a few (maybe 10) times we allocate these > 700K tables, so 400MB sound very strange to me. In my log, I see 768k allocations occuring roughly 94 times. Is it possible that the temacs --batch commands I'm testing (IIRC taken from commands that were failing in leim/Makefile, but perhaps I changed it in some way I didn't notice?) are pulling in my .emacs file, which might be causing more charsets to be loaded? > > > Sorry, I don't see the difficulty. Just make malloc/realloc/free be > > > pointers that point to gmalloc's implementation before dumping, and to > > > the libc implementation after it. You may need some #define to rename > > > malloc to some other symbol, to avoid name clashes. Am I missing > > > something? > > > > Yeah, what happens if, after dumping, the real emacs at runtime ends > > up calling free() on one of the pre-dump pointers? > > You intercept the call and do nothing. Right, but the free pointer can't directly point to the real (libc) free. It has to point to the wrapper that does this range-check. > > > > No, it's less reliable. See my other posts in the thread about what > > > > happens if you have other libraries linked and they do nontrivial > > > > things prior to dumping (e.g. from static ctors). > > > > > > But in those other posts I thought we agreed that whatever those ctors > > > do is irrelevant, as the dumped Emacs cannot possibly use what they > > > allocate, and those ctors will be invoked again in the dumped Emacs. > > > > Those ctors are free to inspect global data. For example one might > > contain (this sort of idiom is necessary if you can't control the > > relative order of ctors): if (!init) { do_something(); init=1; }. In > > that case, the dump would save the value of init, and do_something() > > would fail to happen at runtime. > > That's the same problem as with your clock_gettime, and it must be > fixed anyway, because any ctor run at dump time is almost certainly > picking up data that is irrelevant to the run time. Libc could _possibly_ work around it by virtue of having full control over the init code. For other libraries, the issue is not fixable (see my above example with code that has to control dependency order of ctors), and shouldn't have to be fixed. If the library is written such that static objects have a particular nominal initial value at the source level, it should be able to rely on that value actually being present at runtime. Failure to provide this guarantee is a bug in the runtime (in this case, in the tool which produced the ELF file, i.e. unexelf.c). Rich