From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: Daniel Colascione Newsgroups: gmane.emacs.devel Subject: Re: Preview: portable dumper Date: Fri, 16 Feb 2018 07:10:42 -0800 Message-ID: <0577d3be-cd99-60d6-d520-9e10cc7fc90e@dancol.org> References: <83o9kpe402.fsf@gnu.org> NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-Trace: blaine.gmane.org 1518793812 7864 195.159.176.226 (16 Feb 2018 15:10:12 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Fri, 16 Feb 2018 15:10:12 +0000 (UTC) User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.5.0 Cc: alan@idiocy.org, angelo.g0@libero.it, emacs-devel@gnu.org To: Eli Zaretskii Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Fri Feb 16 16:10:07 2018 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1emhdn-0008Cn-Ip for ged-emacs-devel@m.gmane.org; Fri, 16 Feb 2018 16:09:35 +0100 Original-Received: from localhost ([::1]:58716 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1emhfp-0002YO-Lu for ged-emacs-devel@m.gmane.org; Fri, 16 Feb 2018 10:11:41 -0500 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:52229) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1emhf2-0002WP-94 for emacs-devel@gnu.org; Fri, 16 Feb 2018 10:10:53 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1emhf1-0003qx-0g for emacs-devel@gnu.org; Fri, 16 Feb 2018 10:10:52 -0500 Original-Received: from dancol.org ([2600:3c01::f03c:91ff:fedf:adf3]:43040) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1emhf0-0003qN-Lq; Fri, 16 Feb 2018 10:10:50 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=dancol.org; s=x; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:MIME-Version:Date:Message-ID:From:References:Cc:To:Subject; bh=3x7lJ33kcm08DsEHqrr6+Jsb7OlxdlsycFMdc4l3FKo=; b=GQE6mIQg5HTnSxAEW3GfoON/4GB1sHe6mea6l5uMTqVpG6qRMXbiDcotx1Ay+ZzUkFlP9lhlBqhX8H8TJnohqs86vKi9aE3Ho5EgK68abuHYMzBnGDw5vBOp/IZ1+xpXAisiUZt7+IOx+6KL4fz6Wm+iV8vJLZfTHFn7DP7aYwd8oVuEH+s8GPnZKT8Qw7jxISYZQj0bo5N1b5iZslwlAjevkDkaiM4xgPBDMcHn+dy0xy8tu+WXsm2hula+dN2woeHxyW8L93wCDff7P2w4aAn6ydcvnLNBSo5OA2C9CiHa/qESpSQQ4E9YPjvk0gOgpkoXtC2X3eY6wTlgvZgV+Q==; Original-Received: from [2604:4080:1321:8ab0:717a:f892:f8e7:f978] by dancol.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.84_2) (envelope-from ) id 1emhez-00073T-Bo; Fri, 16 Feb 2018 07:10:49 -0800 In-Reply-To: <83o9kpe402.fsf@gnu.org> Content-Language: en-US X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2600:3c01::f03c:91ff:fedf:adf3 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.org gmane.emacs.devel:222814 Archived-At: On 02/16/2018 12:24 AM, Eli Zaretskii wrote: >> Date: Thu, 15 Feb 2018 15:34:13 -0800 >> From: Daniel Colascione >> Cc: Eli Zaretskii , Angelo Graziosi , >> emacs-devel@gnu.org >> >> I do wonder whether it makes sense to try to copy the dump into the Emacs executable itself instead of >> leaving it as a separate file. We could do it independently of executable format by defining a data array in static >> storage that's initially full of, say, 15MB of zeroes prefixed by a long random header (like a MIME boundary), >> then, after we generate emacs.pdmp, copying the dump file into the executable at the place where we see >> that random header. If the dump turns out to be bigger than that 15MB, we can fail the build and ask the user >> to enlarge the array. >> >> I don't know of any executable format for which this scheme would fail. > > Wouldn't that make the dumper stuff less portable, in the sense that > it would need to be compatible with low-level details of executable > file formats on various systems? No: I'm thinking about only options that work independently of particular executable formats. > At least on non-ELF systems, AFAIK the flexibility of putting > arbitrary sections into an executable is lower than desired. For > example, before Emacs 25 the MS-Windows build would create a special > section for the initialized Emacs data, which had the annoying effect > of running afoul of 'strip', because Binutils don't know about this > section, and therefore stripping would produce a dysfunctional > executable. That does sound annoying. The original proposal I had in mind was to make a normal data array: uint8_t dump[15*1024*1024] = { embedded_dump_signature }; And then refer to this array on code. No part of the toolchain is allowed to remove this array. The "dump insertion" procedure we're considering would just amount to changing the contents of this array in an already-linked executable using dumb find-and-replace anchored on embedded_dump_signature. > It also prevented re-dumping Emacs, something we had in > the past and I'd like us to have again in the future. I'd like to have it too. The data-section-array approach above wouldn't permit redumping unless the new dump fit into the old array. The concatenation approach would allow arbitrary redumping. And of course keeping the dump as a separate file would too. > Wouldn't copying the dump into the executable hit the same problems, > at least in principle? > >> This way, we could get rid of the current complicated dump-file location logic early in Emacs init. > > Can you elaborate why is that logic so complicated? Two reasons: 1) The code to generate the dump file is complicated, especially because we want to support windows, and 2) with a separate dump file, it's easier to accidentally break the correspondence between the dump and the emacs binary itself. If we somehow stapled the dump to the binary, it'd be a lot harder to break this way. I like the idea of being able to mv emacs emacs.old, make emacs, and then have emacs.old and emacs still both run and work normally. I don't think the numbered build mechanism you fixed completely solves the problem, since it only works in one specific context.