Re: Skipping unexec via a big .elc file

[Eli Zaretskii <eliz@gnu.org>]

> From: Daniel Colascione <dancol@dancol.org>
> Cc: Eli Zaretskii <eliz@gnu.org>,  monnier@iro.umontreal.ca,  emacs-devel@gnu.org
> Date: Mon, 24 Oct 2016 12:47:56 -0700
> 
> > I'd argue that we are already in this situation.  For example, nobody
> > knows how to make unexec work with ASLR or PIE; when I tried fuzzing
> > Emacs with AFL, the dumped binary would simply crash; the dumped
> > binary is not reproducible (i.e. bit-by-bit identical after every
> > build); and I think dumping also doesn't work with ASan. The fraction
> > of situation where unexec doesn't work any more gets larger and
> > larger. If we had people who could solve these problems, it should get
> > smaller instead.
> 
> Everyone who's seriously thought about the unexec problem _understands_
> the issue.

The important point is that the number of people here who can claim
such understanding, enough so to fix the issues, is diminishingly
small, and gets smaller every year.

> My preferred approach is the portable dumper one: basically what we're
> doing today, except that instead of just blindly copying the data
> segment and heap to a new emacs binary, we'll write this information to
> a separate file, stored in a portable format, a file that we'll keep
> alongside the Emacs binary.  We'll store in this file metadata about
> where the pointers are. (There are two kinds of pointers in this file:
> pointers to other parts of the file and pointers to the Emacs binary.)
> 
> At startup, we'll load the dump file and walk the relocations, fixing up
> all the embedded addresses to account for the new process's different
> address space.

Why do you think this will have better performance that reading a
single .elc file at startup?  It's still mainly file I/O and
processing of the file's contents, just like with byte-compiled files.

If we have no reason to believe this portable dumper will be
significantly faster, we should IMO investigate the .elc method first,
because it's so much simpler, both in its implementation and in future
maintenance.  E.g., adding a new kind of Lisp object to Emacs would
require corresponding changes in the dumper.

> We can't save all of the Emacs data segment this way, but we can
> relocate and restore anything that's marked with staticpro. The overall
> experience should be very similar to what we have today.
> [...]
> Speaking of COW faults: a refinement of this scheme is to do the
> relocations lazily, in a SIGSEGV handler.  (Map the dump file PROT_NONE
> so any access traps.)  In the SIGSEGV handler, we can relocate just the
> page we faulted, then continue. This way, we don't need to slurp in the
> entire dump file from disk just to start emacs -Q -batch: we can
> demand-page!

Demand paging in an application, and an application such as Emacs on
top of that, makes little sense to me.  This is the OS business, not
ours.  Using mmap as a fast way to read a file, yes, that's done in
many applications.  But please lets leave demand paging out of our
scope.

IMO the less we mess with low-level techniques that no other
applications use the better, both because we have very few people who
can do that and because doing so runs higher risk of becoming broken
by future developments in the platforms we deem important.  The
long-term tendency in Emacs development should be to move away from
such techniques, not to acquire more of them.