Re: Suppressing native compilation (short and long term)

[Eli Zaretskii <eliz@gnu.org>]

> From: Rob Browning <rlb@defaultvalue.org>
> Cc: david@tethera.net, emacs-devel@gnu.org, akrl@sdf.org
> Date: Sat, 01 Oct 2022 11:38:54 -0500
> 
> I've finally had a chance to catch up, and this thread to date appears
> to cover a good bit of the relevant ground.  As requested, I'll try to
> add some additional context that hopefully better explains how and why
> we arrived at the current situation, asking the questions we're asking.

Thanks.

> Long ago, we (speaking from the Debian side) came up with the Debian
> Emacs policy.  This was both a policy, and the infrastructure to
> implement some of it (in the form of the emacsen-common package, and
> now, additionally, the dh-elpa infrastructure).
> 
> At the time, XEmacs was still active, and Emacs itself was about to
> transition from 19 to 20, and my recollection is that there was a
> nontrivial amount of backward incompatibility with the latter, or at
> least I recall hearing from people who really didn't want to be forced
> to upgrade immediately.
> 
> I'm not completely familiar with those concerns because this was just
> about the time I started working on Debian's Emacs support.  In any
> case, one of the key goals of that support was to allow Debian to
> package, and people to install, one or more major versions of Emacs,
> and/or XEmacs simultaneously.  As a result, we added support for
> "emacsen flavors": emacs20, emacs21, emacs22, xemacs19, etc.  (Worth
> noting that a few years back we did "unversion" the Emacs packages, so
> that we no longer have emacs27, emacs28, etc., just emacs.)
> 
> We also wanted to support the creation of "add-on" packages like org,
> newer versions of gnus, etc., and we wanted each of those packages to be
> able to work with as many flavors (Emacs or XEmacs) as the add-on
> package maintainer decided was feasible.
> 
> We wanted to have only one system-level .elc file for each flavor for
> each .el file.  We did not want to have to build and maintain separate
> packages for each flavor in the archive, i.e. org-emacs21, org-emacs22,
> etc., each with a full set of the relevant .elc files.  (I'm also not
> sure that would have worked (easily), given the versioning and
> dependency concerns described next.)
> 
> We didn't want to risk breakage from backward-incompatible changes to
> macros in dependencies, i.e. if the magit package depended on the foo
> (library) package, we assumed we'd need to rebuild (during package
> upgrade) all the majit .elc files whenever the foo package changed
> because foo might have changed a macro backward incompatibly.
> 
> Since we didn't have any (reasonable?) way to detect the inter-package
> dependency graph, we just insisted that those be correctly described by
> the Debian package dependencies.  Then at install time, we use that
> graph to determine what packages to "rebuild" (mostly just the .elc
> files), and the order in which to rebuild them.
> 
> We also decided to rebuild *all* the add-on packages (.elc files) for a
> given flavor whenever the version of the flavor's package changes
> (e.g. when there's a new Emacs or XEmacs version) because we didn't know
> whether or not any given release might make that necessary/desirable.
> 
> Uninstalling an add-on, of course, cleans up all the relevant bits
> (usually, mostly just the automatically generated .elc files).
> 
> So when you "apt install elpa-magit", Debian builds all the .elc files
> and puts them in the right place in /usr for every flavor you have
> installed.  When you "apt install emacs" and it's a new install, Debian
> builds all of the .elc files for every installed add-on package in
> dependency order for that flavor.  The same thing happens during an
> emacs package upgrade.
> 
> Note that we were also thinking of the case where you have a large
> machine with hundreds of users (as was more often the case when we
> started).  There we didn't want to have N copies of the same file for N
> users (whether .elc or now .eln).

Thanks for the background.

You should know that the problems you needed to address with the *.elc
files don't exist with the *.eln files.  The *.eln files have version
information of the Emacs to which they "belong" hard-coded into their
names.  That's why window.el gets compiled into something like
window-0d1b8b93-7ef4271a.eln, and lives under a directory named
something like 28.2-4fafb146 -- these 3 hashes encode both the Emacs
binary and its version, and the original .el file's absolute file name
and contents.  So there's no way users will have any trouble when
multiple Emacs versions are installed or when different users use
different versions: the *.eln files are effectively automatically
"versioned", and Emacs will only load a .eln file that was compiled
for it.  Moreover, as soon as some user decides to modify a .el file,
the .eln file produced from it will have a different name, and if the
Emacs binary changes as result of rebuilding, the new .eln file will
be written to a different directory.

> Regarding the writes to HOME, etc.  I think David covered that fairly
> well -- Debian runs emacs to "do things" (build .elcs, run tests, etc.)
> at package build time, package install time, and during package testing.
> In all of these cases, we're likely to want to avoid side-effects outside
> the build/test dir like writing .elc or .eln files to the current user's
> HOME, whether that's /root/ or /home/*.  It sounds like we may be able
> to accomplish that by redirecting everything to a temp dir, which is
> likely fine.

Yes, by changing native-comp-eln-load-path you should be able to avoid
writing to the home directory of the user under whose credentials the
installation runs.  If you need that.