Re: Native-compilation build process (was Re: Loading tramp for dump goes into infinite regress)

[Eli Zaretskii <eliz@gnu.org>]

> From: Lynn Winebarger <owinebar@gmail.com>
> Date: Wed, 3 Aug 2022 10:53:12 -0400
> Cc: Andrea Corallo <akrl@sdf.org>, Michael Albinus <michael.albinus@gmx.de>, 
> 	emacs-devel <emacs-devel@gnu.org>, Stefan Monnier <monnier@iro.umontreal.ca>
> 
> > Why are you running "make" only in 'src'?  Some necessary parts of the
> > build must also run "make" in 'lisp' and in other places.
> 
> I'm replicating the 3 stage build process Stefan Monnier (added to the
> cc list) described by invoking a shell script from site-load, when
> dump-mode is set to pdump.

Well, Stefan definitely didn't mean for you to shoot yourself in the
foot.  Isn't it possible to run these 2 stages from the top-level
Makefile?

Alternatively, say "make BINDESTDIR=..." etc.

Or, as I suggested, copy the default values of these two variables
from the top-level Makefile.in to src/Makefile.in.  Then it should
probably work from 'src' as well, provided that the other variables
these two reference are also defined there.

>                           The shell script produces the second stage
> build - the one with a dump containing just the files loaded directly
> from loadup - then uses it to compile the files loaded by site-load.
> It explicitly builds the emacs.pdmp target in src, moves emacs and
> emacs.pdmp to emacs-1 and emacs-1.pdmp, then invokes:
> make -C ../lisp EMACS=../src/emacs-1 compile-target
> TARGETS="${SL_TARGETS}" BYTE_COMPILE_EXTRA_FLAGS="--eval '(setq
> comp-file-preloaded-p t)'"
> where SL_TARGETS has been extracted from site-load.el using the same
> procedure used in constructing lisp.mk

You could add BIN_DESTDIR= and ELN_DESTDIR= settings to those
commands.

> > The final installation _must_ be specified, yes.  Using the build
> > directory won't help because Emacs already does that internally.  IOW,
> > it only needs help in knowing where the stuff _will_ be installed
> 
> That is true for the dump that is going to be installed, yes, but for
> intermediate stages in the build that knowledge is not required for
> the resulting dump to be useful.

The Emacs build always produces binaries that can be invoked either
way.  So if you use our Makefiles, you need to play by their rules.

> In any case, once loadup determines there are native compile units
> loaded while preparing for the dump, either they all should have the
> names fixed up, or an error should be signaled.

The names cannot be fixed up, because the build doesn't know how.
These two variables get their values from the configure script, where
the user specifies where the package will be installed.  Without that,
the build cannot know their values.

> It's not clear to me, though, why the pdmp loading process should
> refuse to work if the compilation units recorded in the dump file are
> just strings.

Because it needs to record in the binary where to look for those *.eln
files.  Emacs can be installed in a variety of ways, using all kinds
of "tricks" like symlinks and such, so the names need to include
directories to allow Emacs to find these files quickly at startup.  If
startup is delayed because Emacs looks hight and low for its files,
users will be annoyed.

> It seems to me the "dual location" is more of a "nice
> to have" feature than a real requirement.

No, it's a requirement.  If nothing else, the just-built Emacs binary
is run as part of the build, so it needs to be able to do that.

> There could be a flag to
> dump an image with only the (relative) final installation paths
> recorded, or it could just be a build that will always be run in place
> (e.g. an ordinary user build).

Many users never install Emacs, or at least never install some of its
versions.

Anyway, this is a non-starter: we won't give up this useful
functionality.

> For user dumps, recording the absolute paths would presumably work
> fine, no?

No, because then you cannot relocate the Emacs tree to a different
top-level directory, and have it still work.

There are good reasons behind all these features, they aren't
arbitrary nor redundant.