From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Stefan Monnier Newsgroups: gmane.emacs.devel Subject: Re: Dumper problems and a possible solutions Date: Tue, 24 Jun 2014 15:27:39 -0400 Message-ID: References: <20140624171955.GS179@brightrain.aerifal.cx> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain X-Trace: ger.gmane.org 1403638093 22957 80.91.229.3 (24 Jun 2014 19:28:13 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Tue, 24 Jun 2014 19:28:13 +0000 (UTC) Cc: emacs-devel@gnu.org To: Rich Felker Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Tue Jun 24 21:28:06 2014 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1WzWNv-00077W-ND for ged-emacs-devel@m.gmane.org; Tue, 24 Jun 2014 21:28:03 +0200 Original-Received: from localhost ([::1]:33523 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WzWNv-0001Mi-9d for ged-emacs-devel@m.gmane.org; Tue, 24 Jun 2014 15:28:03 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:54736) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WzWNi-0001Ek-UA for emacs-devel@gnu.org; Tue, 24 Jun 2014 15:27:58 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1WzWNZ-0000ec-09 for emacs-devel@gnu.org; Tue, 24 Jun 2014 15:27:50 -0400 Original-Received: from ironport2-out.teksavvy.com ([206.248.154.181]:43231) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WzWNY-0000eQ-Sz for emacs-devel@gnu.org; Tue, 24 Jun 2014 15:27:40 -0400 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: ArUGAIDvNVNLd+D9/2dsb2JhbABZgwaDSsA9gRcXdIIlAQEBAQIBViMFCwsOJhIUGA0kLodWCNIZF456B4Q4BKkZgWqDTCE X-IPAS-Result: ArUGAIDvNVNLd+D9/2dsb2JhbABZgwaDSsA9gRcXdIIlAQEBAQIBViMFCwsOJhIUGA0kLodWCNIZF456B4Q4BKkZgWqDTCE X-IronPort-AV: E=Sophos;i="4.97,753,1389762000"; d="scan'208";a="69426619" Original-Received: from 75-119-224-253.dsl.teksavvy.com (HELO ceviche.home) ([75.119.224.253]) by ironport2-out.teksavvy.com with ESMTP/TLS/ADH-AES256-SHA; 24 Jun 2014 15:27:39 -0400 Original-Received: by ceviche.home (Postfix, from userid 20848) id 75D666638E; Tue, 24 Jun 2014 15:27:39 -0400 (EDT) In-Reply-To: <20140624171955.GS179@brightrain.aerifal.cx> (Rich Felker's message of "Tue, 24 Jun 2014 13:19:55 -0400") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.4.50 (gnu/linux) X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 206.248.154.181 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.devel:172689 Archived-At: > To solve ALL of the problems with the dumper (which seems to be a > recurring theme), I have a proposed design to make it fully portable > -- even moreso than xemacs "portable dumper" which is still an ugly > hack. The idea is simple: after loading all of the lisp objects that > need dumping, walk the lisp heap and output a representation for each > object as a giant static array in C source format, then compile and > link this new translation unit with the rest of the emacs .o files to > produce a final emacs binary. No hacks with binary formats would be > involved; everything would happen at the C source level. As part of > the lisp heap dumping, address references to other objects would have > to be relocated to refer to the object's position in the static array > rather than the original address at which the object resided when > created in temacs. That's some non-trivial work, but definitely no > prohibitive, and as a bonus, the generated address-constant references > in the static array would transform to load-address-relative > relocations for the linker, allowing emacs to be built as a > position-indepdendent executable (PIE) if desired. Generating a big static C array against which to link sounds fine and very portable, indeed. I'm not sure how hard/easy the relocation could turn out to be. There's the problem of finding *all* the references, and there's the problem that moving an object means that its "hash" value changes. > If not, or if that's going to be a very long-term project, would a > cleaned-up version of my current solution be acceptable upstream? Making the "dump" portable would be very welcome. Generating a big static C array sounds OK. So whether the result is acceptable or not will depend on what's needed to solve the problems linked to relocation. Another option is to "dump" the heap into a binary file that we would later on "mmap". Stefan