Re: Suppressing native compilation (short and long term)

[Eli Zaretskii <eliz@gnu.org>]

> From: Liliana Marie Prikler <liliana.prikler@gmail.com>
> Cc: emacs-devel@gnu.org
> Date: Wed, 12 Oct 2022 22:22:46 +0200
> 
> In GNU Guix, we observe certain packages breaking when trying to
> compile them natively.  While some of that is on us for messing up the
> package descriptions, one could also encounter genuine bugs in native
> compilation that one wants to work around by disabling it.

When you encounter bugs in native compilation, please report them to
us, so we could fix them.  As of now, we are not aware of any such
bugs that were reported and haven't been fixed.  So if you still have
such problem, please report them ASAP.

> (For added
> trouble, whether or not your package breaks could depend on the CPU due
> to being *native* compilation; thus the "no-native-compile" local
> variable is insufficient to address this generally.)

Why isn't it sufficient to use no-native-compile?  It just means that
on some architectures the corresponding file will be loaded as
byte-compiled, and thus will be slightly slower (how much slower
depends on the code, so if you are worried, my recommendation is first
to measure the difference -- you might be surprised).

In any case, if no-native-compile doesn't for some reason solve the
problem, I don't understand how disabling native compilation will: the
latter is a more blunt instrument than the former, e.g., it cannot be
applied on the per-file resolution.

> In GNU Guix, we default to not compiling packages ahead-of-time,
> instead using a minimal emacs that can only do byte compilation for
> most packages.  Users can however pretty easily switch to ahead-of-time
> compilation by swapping out the emacs package (using what Guix calls
> package transformations).  This is also important because apart from
> the current Emacs we typically provide an emacs-next package for
> upcoming versions, as well as some other variants that the user might
> want to use instead.  Again, we assume that users who want to opt in to
> native compilation do so via a transformation.

Sorry, I'm unfamiliar with this terminology.  When you say
"ahead-of-time compilation", do you mean native compilation of all the
Lisp files before they are loaded for the first time?  Or do you mean
something else?

And what is "swapping out the emacs package", and what is "package
transformation"?

> In this context, the default of compiling everything that has hitherto
> only been byte-compiled is an ill-advised choice.  Now, there is a
> chance that the user meant to do this anyway, but I think they rather
> a) use whatever the distro default is without caring either way or b)
> actually didn't want this package natively compiled.  Due to some
> packages breaking, we had a lot of b) in our mailing lists in the past
> few days.

If a package is a single file or a small number of files, those users
can add the no-native-compile cookies in those files.  And again,
disabling native compilation is a method that doesn't allow them
fine-tuning anyway, so I fail to see how it could help them here.  If
the problem is real (and I don't yet see it is), we should perhaps
discuss its details and invent some new method of disabling
compilation with finer granularity.

> As for loading, I think there could be a case made to restrict loading
> to certain directories managed by "trusted entities", but I think
> native-comp-eln-load-path already accomplishes this.

Yes, but see below.

> If one wanted to disable loading altogether, I'm pretty sure one
> could set it to nil and be done (with perhaps the exception that
> deferred compilation breaks if there's no path to store binaries
> in).

I don't think you can set native-comp-eln-load-path to nil.  The last
entry, the one which points to where the preloaded *.eln files live,
must be there, or else Emacs will refuse to start.  And at least one
other directory should be there as well, because if and when some
package advises some primitive, Emacs will need to generate and
compile a trampoline .eln file.  But yes, if users want to prevent
loading from a certain directory, they can remove it from
native-comp-eln-load-path, provided that the two necessary entries are
still left in the list.

> Thus, I think the issue is really only one of inadvertently launching
> deferred compilation when all the packages the user *expects* to be
> natively compiled are already ensured to be compiled ahead-of-time.

This will not happen, so no need to prevent compilation in that case.
If a .eln file already exists and is up-to-date, Emacs will not try to
compile it again as part of loading the package.  If some of the
packages aren't natively compiled, against user's expectations, I
still see no reason for users to want to disallow JIT compilation,
except as a kind of surprise effect due to something they see for the
first time.  How is JIT compilation different from ahead-of-time
compilation?  There's nothing really dangerous or harmful in JIT
compilation, so all it takes is some getting used to it.  Educating
your users to get used to it will go a long way towards eliminating
the fears, because the dangers aren't real, they are largely
imaginary.

> I hope this answers some of the questions raised.  In short, we believe
> that users might want to control not only from where native shared
> libraries will be loaded, but also whether to generate them at runtime,
> and that (distro) package managers should enable them to compile all
> such shared libraries ahead of time – and better yet, in a reproducible
> manner, but this hits other bugs I do not want to discuss in detail at
> the moment.

Thanks for the explanations.  I still think the reasons for disabling
native compilation are rather weak at best, and the users' requests to
allow that based on either bugs that need to be solved or surprise and
fears that have no real basis.  Moreover, disabling native compilation
is a very blunt instrument that cannot be applied better than the
existing ones, like no-native-compile (and a few others that we didn't
mention; see the defcustom's in comp.el).