From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: Daniel Colascione Newsgroups: gmane.emacs.devel Subject: Re: Preview: portable dumper Date: Fri, 16 Feb 2018 07:15:09 -0800 Message-ID: References: <83o9kpe402.fsf@gnu.org> NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-Trace: blaine.gmane.org 1518794198 31938 195.159.176.226 (16 Feb 2018 15:16:38 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Fri, 16 Feb 2018 15:16:38 +0000 (UTC) User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.5.0 To: Andy Moreton , emacs-devel@gnu.org Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Fri Feb 16 16:16:33 2018 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1emhk2-0006Jv-PE for ged-emacs-devel@m.gmane.org; Fri, 16 Feb 2018 16:16:03 +0100 Original-Received: from localhost ([::1]:59219 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1emhm4-0007IL-Nz for ged-emacs-devel@m.gmane.org; Fri, 16 Feb 2018 10:18:08 -0500 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:53175) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1emhjK-00069T-IH for emacs-devel@gnu.org; Fri, 16 Feb 2018 10:15:19 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1emhjJ-00072X-IL for emacs-devel@gnu.org; Fri, 16 Feb 2018 10:15:18 -0500 Original-Received: from dancol.org ([2600:3c01::f03c:91ff:fedf:adf3]:43136) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1emhjJ-00070l-9t for emacs-devel@gnu.org; Fri, 16 Feb 2018 10:15:17 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=dancol.org; s=x; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:MIME-Version:Date:Message-ID:From:References:To:Subject; bh=VpN3Ome66eAoC1JZ0VBIthxQNHAeWWe+sZVPUsHivDI=; b=ekye1PUsKG6izK4Fkb5I6cE0vYIK/0uSPXQZoZmxPdSbLEE77rACadXTFvekR51rE6Jh2uiY25Quj04RRFhr54cof+0RHo5tzDbEadwXLW2TBJhGxAFtLAVP49pSYaIuocj5Vl7lmlro89alRCBQbym4jR0B/JKdTmWSMbCLUrge/ApbVYjDu84t9IpYcDsrDs3D4T58KOzHMH/J7ChXfJtnXBvxG0oXbZFknyQrUI9FBB42654YbppJ3PmcaXnY853It522poVTVIbLwylnNWM+H5Lw4EiZ0ow0ueCkZz6+qAGxfHCz+wqlSkax6Jq7RHyzVyCP+cJa4R6rcf8G/Q==; Original-Received: from [2604:4080:1321:8ab0:90c2:f4df:1b81:aa2] by dancol.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.84_2) (envelope-from ) id 1emhjI-00074a-1F; Fri, 16 Feb 2018 07:15:16 -0800 In-Reply-To: Content-Language: en-US X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2600:3c01::f03c:91ff:fedf:adf3 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.org gmane.emacs.devel:222815 Archived-At: On 02/16/2018 03:30 AM, Andy Moreton wrote: > On Fri 16 Feb 2018, Eli Zaretskii wrote: > >>> Date: Thu, 15 Feb 2018 15:34:13 -0800 >>> From: Daniel Colascione >>> Cc: Eli Zaretskii , Angelo Graziosi , >>> emacs-devel@gnu.org >>> >>> I do wonder whether it makes sense to try to copy the dump into the Emacs executable itself instead of >>> leaving it as a separate file. We could do it independently of executable format by defining a data array in static >>> storage that's initially full of, say, 15MB of zeroes prefixed by a long random header (like a MIME boundary), >>> then, after we generate emacs.pdmp, copying the dump file into the executable at the place where we see >>> that random header. If the dump turns out to be bigger than that 15MB, we can fail the build and ask the user >>> to enlarge the array. >>> >>> I don't know of any executable format for which this scheme would fail. >> >> Wouldn't that make the dumper stuff less portable, in the sense that >> it would need to be compatible with low-level details of executable >> file formats on various systems? >> >> At least on non-ELF systems, AFAIK the flexibility of putting >> arbitrary sections into an executable is lower than desired. For >> example, before Emacs 25 the MS-Windows build would create a special >> section for the initialized Emacs data, which had the annoying effect >> of running afoul of 'strip', because Binutils don't know about this >> section, and therefore stripping would produce a dysfunctional >> executable. It also prevented re-dumping Emacs, something we had in >> the past and I'd like us to have again in the future. >> >> Wouldn't copying the dump into the executable hit the same problems, >> at least in principle? > > I don't understand the desire to put the dump within the eamcs > executable, as I thought the whole point of this exercise was to avoid > dodgy manipulation of executable file formats. It's just a matter of packaging. As I mentioned in the other message I sent, keeping the dump separate from the executable breaks some perfectly-natural things you might want to do with the Emacs executable, like copying and renaming it. If possible, I'd like to keep these things working. If we can't, that's a shame, but it's not the end of the world. We do need to avoid platform-specific executable file manipulation and weird runtime hacks. There are three options on the table: 1) Keeping the dump as a separate file, 2) Appending the dump to the end of the executable, and 3) Overwriting a data-section array with the dump contents. All of these options avoid dodgy platform-specific executable modifications of the sort you see in, say, unexw32.c, and they're all compatible with emacs as a position-independent executable. > Anything that fiddles with the executable prevents signing the > executable, which in many environments is desireable. Ideally, we'd be able to re-sign after modification.