From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: Daniel Colascione Newsgroups: gmane.emacs.devel Subject: Re: Skipping unexec via a big .elc file Date: Mon, 24 Oct 2016 12:47:56 -0700 Message-ID: References: <838ttyhhzu.fsf@gnu.org> <871szqwu51.fsf@users.sourceforge.net> <831szqhbc2.fsf@gnu.org> <87d1itt79z.fsf_-_@users.sourceforge.net> <7baa18d4-2b09-caa8-005e-29008a383ad1@cs.ucla.edu> <83mvhwrgd5.fsf@gnu.org> <8539f38f-9a11-44c3-4de7-bb974c96206c@cs.ucla.edu> <8360ojpndr.fsf@gnu.org> <83shrnm0k1.fsf@gnu.org> <83insi5jy9.fsf@gnu.org> <83mvht50qb.fsf@gnu.org> <8c085c3e-361d-7d10-6f34-07c387eb3b43@dancol.org> <83a8dt4u3a.fsf@gnu.org> NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain X-Trace: blaine.gmane.org 1477338523 8214 195.159.176.226 (24 Oct 2016 19:48:43 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Mon, 24 Oct 2016 19:48:43 +0000 (UTC) User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.0.50 (gnu/linux) Cc: Eli Zaretskii , monnier@iro.umontreal.ca, emacs-devel@gnu.org To: Philipp Stephani Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Mon Oct 24 21:48:39 2016 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bylEG-0006xo-1e for ged-emacs-devel@m.gmane.org; Mon, 24 Oct 2016 21:48:16 +0200 Original-Received: from localhost ([::1]:49500 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bylEI-0002rK-9k for ged-emacs-devel@m.gmane.org; Mon, 24 Oct 2016 15:48:18 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:36306) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bylE6-0002pL-Dh for emacs-devel@gnu.org; Mon, 24 Oct 2016 15:48:07 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bylE5-0008Kx-BA for emacs-devel@gnu.org; Mon, 24 Oct 2016 15:48:06 -0400 Original-Received: from dancol.org ([2600:3c01::f03c:91ff:fedf:adf3]:40694) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1bylE5-0008K8-2U; Mon, 24 Oct 2016 15:48:05 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=dancol.org; s=x; h=Content-Type:MIME-Version:Message-ID:In-Reply-To:Date:References:Subject:Cc:To:From; bh=5rQBmQRWTck5ipHwkJ96nkqWE1EUct4W6tENAhQXxxA=; b=a5/u7ONGraXVsbnvOhinsb8mclatKmC3L+X7eh2YNV8ZJHCLLO5XPYTE5t5w+7IhGj767Qes36V/NyuAPN5RSBVc9r0wRS7tqgKt131kWIABcInWXTVmI740GYZOxxXspmkLYmiDbRYCo7iB4dchf/VurrKXdPIMjt0AOHW4UZoi0MtPQSJPb97qbIr+hto+/f+o9lvSI8i1tEWviMREcMBNtCrQ4ijj69tmYz6iieBV2nS9SfEIprY8l6DOTyz6JP9GsHtexAydk4TvMJVEfGl4eOdqPTsYqBltfypjxtm26KMnL15jrPFwM3dpuF9oomApYDsEFVuyenvw3S3z0w==; Original-Received: from [2620:0:1008:100b:d140:de1d:b389:155] (helo=dancol-glaptop0) by dancol.org with esmtpsa (TLS1.2:DHE_RSA_AES_128_CBC_SHA1:128) (Exim 4.84_2) (envelope-from ) id 1bylE2-0005tc-K5; Mon, 24 Oct 2016 12:48:02 -0700 In-Reply-To: (Philipp Stephani's message of "Mon, 24 Oct 2016 16:51:20 +0000") X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2600:3c01::f03c:91ff:fedf:adf3 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.org gmane.emacs.devel:208741 Archived-At: Philipp Stephani writes: > Daniel Colascione schrieb am Mo., 24. Okt. 2016 um 18:35 Uhr: > > That is, we *could* get into a situation where "no people on board [] > know enough about unexec to solve the next problem" > > I'd argue that we are already in this situation. For example, nobody > knows how to make unexec work with ASLR or PIE; when I tried fuzzing > Emacs with AFL, the dumped binary would simply crash; the dumped > binary is not reproducible (i.e. bit-by-bit identical after every > build); and I think dumping also doesn't work with ASan. The fraction > of situation where unexec doesn't work any more gets larger and > larger. If we had people who could solve these problems, it should get > smaller instead. It's not a matter of "not knowing" how to make unexec work with PIE and PIC code generally --- the problem is that the naive approach currently used for serializing program state depends on the process address state being reproducible: we don't specially mark pointers in the saved image, so we can't relocate them. There have been numerous discussions on emacs-devel about relocation schemes, with proposals ranging from just making elc faster to translating elisp to C. Everyone who's seriously thought about the unexec problem _understands_ the issue. unexec isn't black magic. Getting rid of the current scheme is a matter of finding the right relocation scheme (which for all I know might as well be "make elc better") and finding the time to implement it. My preferred approach is the portable dumper one: basically what we're doing today, except that instead of just blindly copying the data segment and heap to a new emacs binary, we'll write this information to a separate file, stored in a portable format, a file that we'll keep alongside the Emacs binary. We'll store in this file metadata about where the pointers are. (There are two kinds of pointers in this file: pointers to other parts of the file and pointers to the Emacs binary.) At startup, we'll load the dump file and walk the relocations, fixing up all the embedded addresses to account for the new process's different address space. There's no binary other than the one that the compiler generates; this data file is just data, so ASLR, ASAN, and other clever things should work fine. (Some people have proposed asking the system dynamic linker to do the relocating, but I'd prefer to do it ourselves, in a portable way.) We can't save all of the Emacs data segment this way, but we can relocate and restore anything that's marked with staticpro. The overall experience should be very similar to what we have today. Additionally, the purespace concept remains useful: if we take pure storage and put it in its own region of the dump file, we don't need to take copy-on-write faults for data that cannot contain pointers. Speaking of COW faults: a refinement of this scheme is to do the relocations lazily, in a SIGSEGV handler. (Map the dump file PROT_NONE so any access traps.) In the SIGSEGV handler, we can relocate just the page we faulted, then continue. This way, we don't need to slurp in the entire dump file from disk just to start emacs -Q -batch: we can demand-page! Whether this refinement is worth the trouble is something only experimentation can tell, but it's an option if we need it. With this refinement, the portable dumping approach should be safe, semantically familiar to unexec, ASLR-compatible, _and_ very nearly as fast as what we have today.