From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Rich Felker Newsgroups: gmane.emacs.devel Subject: Re: Dumper problems and a possible solutions Date: Tue, 24 Jun 2014 17:15:19 -0400 Message-ID: <20140624211519.GU179@brightrain.aerifal.cx> References: <20140624171955.GS179@brightrain.aerifal.cx> <20140624194026.GT179@brightrain.aerifal.cx> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: ger.gmane.org 1403664018 24441 80.91.229.3 (25 Jun 2014 02:40:18 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Wed, 25 Jun 2014 02:40:18 +0000 (UTC) Cc: emacs-devel@gnu.org To: Stefan Monnier Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Wed Jun 25 04:40:11 2014 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1Wzd86-0002hs-Df for ged-emacs-devel@m.gmane.org; Wed, 25 Jun 2014 04:40:10 +0200 Original-Received: from localhost ([::1]:34766 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Wzd85-0005kY-NO for ged-emacs-devel@m.gmane.org; Tue, 24 Jun 2014 22:40:09 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:47782) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WzY3t-0002Mf-P4 for emacs-devel@gnu.org; Tue, 24 Jun 2014 17:15:37 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1WzY3m-0001RX-2D for emacs-devel@gnu.org; Tue, 24 Jun 2014 17:15:29 -0400 Original-Received: from 216-12-86-13.cv.mvl.ntelos.net ([216.12.86.13]:44414 helo=brightrain.aerifal.cx) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WzY3l-0001RM-Rx for emacs-devel@gnu.org; Tue, 24 Jun 2014 17:15:21 -0400 Original-Received: from dalias by brightrain.aerifal.cx with local (Exim 3.15 #2) id 1WzY3j-0008QP-00; Tue, 24 Jun 2014 21:15:19 +0000 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 216.12.86.13 X-Mailman-Approved-At: Tue, 24 Jun 2014 22:40:07 -0400 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.devel:172697 Archived-At: On Tue, Jun 24, 2014 at 04:24:47PM -0400, Stefan Monnier wrote: > > Thanks for the feedback. Can you elaborate on how/why the hash > > changes, and where it's stored that would need to be updated? > > When placing an object in a hash-table, the hashing function often just > uses the address as "the hash value". So any hash-table that uses such > a hash-function will need to be rehashed after relocation. I see. Is this hashing all at the C level, or is it happening in lisp code? Can the lisp code even see the address value for lisp objects? If it's purely at the C level I doubt it would be hard to re-do the hashes but I obviously haven't read the relevant code. > > As far as the relocation, my impression is that it would just need to > > be able to identify pointers in lisp objects (this is already possible > > since the GC needs to do it, right?), and rewrite them to > > (essentially) "static_lisp_heap + offset_of_pointed_to_object" when > > writing the dump out as a C array. > > Yes, the GC already knows how to find the references that are inside > Lisp objects, but there can also be references coming from global > variables (for sure) or non-Lisp data-structures or maybe from the stack > (not sure about those last two). How does the GC avoid freeing objects that have these kinds of references? BTW, at the point of dumping, my impression is that there should not be relevant references from the stack; the whole call frame at the time of dumping is thrown away and replaced by the new invocation of main. Conceptually, the dumper can be thought of as a longjmp back to the start of main. > > This is the xemacs "portable dumper" approach, and I believe it's > > inferrior because it depends on being able to map back at the same > > location. > > We could support relocation at mmap-time to solve this. Yes, but that's conceptually just as difficult as dumping to a C array: you have to patch up all the addresses and the hash values will change. > Neither solution is clearly superior to the other, it just depends on > you priority. For me, either way would be an improvement over what we > have (tho, again, it might still depend on the constraints imposed by > the need to perform relocation). I agree completely. The current situation makes it nearly impossible to port emacs to a system that's not making strong guarantees about its implementation internals, and (at least from my understanding reading list archives) it's imposing ugly constraints on existing implementations (glibc) not to change internals in ways that would break emacs' dumper. I would really like to see fixing this issue treated as a priority in the future direction of emacs. Rich