bug#73303: 30.0.91; Native compiler repeatedly interrupts at random moments

[Eli Zaretskii <eliz@gnu.org>]

> From: "N. Jackson" <njackson@posteo.net>
> Cc: 73303@debbugs.gnu.org, Andrea Corallo <acorallo@gnu.org>
> Date: Tue, 17 Sep 2024 15:22:42 +0000
> 
> Hello Eli, thank you for your response.
> 
> At 21:42 +0300 on Monday 2024-09-16, Eli Zaretskii wrote:
> 
> > I see no bug in what you describe.
> 
> Fair enough.  The concern that prompted me to file a bug report was
> not that native compile is broken (a literal bug) but rather that
> since it's a newish feature you might welcome end-user feedback
> about it's default behaviour (or perceived behaviour) that might
> lead to to beneficial tweaks to either the behaviour or to the
> documentation.

The feedback is always welcome, of course (if my wording somehow made
an impression it wasn't, I apologize).  And what you describe is
already known: a few users posted similar experiences during the
development and testing of Emacs 29.  So these issues are not new, and
I think Emacs 30 is better equipped to deal with them, and gives users
more knobs to deal with them.

> > These warnings usually mean that the offending Lisp file lacks
> > some 'require's.  JIT native compilation runs in a separate Emacs
> > session, which only loads the file it compiles, so any missing
> > 'require's or autoloading cookies trigger these warnings, whereas
> > when the same packages are loaded into your main Emacs session,
> > they can benefit from packages loaded earlier in the session.
> ...
> > JIT native compilation is triggered by loading a .elc file that
> > doesn't yet have a corresponding .eln file
> 
> Thank you for the overview of how native compilation works.  I had
> wrongly guessed that it was deferring the compilation lazily and
> doing it later on an idle timer or something of that sort.  If I
> understand what you are saying correctly, the native compiler, not
> having a crystal ball, cannot possibly know ahead of time what .elc
> files a user might load, so it cannot compile them ahead of time.

Yes, exactly.  JIT compilation compiles only packages that you load,
when you load them for the first time.

> However, as a naïve end-user of Emacs, when I am _using_ Emacs, as a
> tool to get a job done -- writing an essay, say -- I don't want to
> think about the guts of how Emacs works.  I'm happy to save that for
> when I'm building Emacs or maintaining my configuration files and
> that is when I would want the native compilation to happen, if that
> were possible.
> 
> You write
> 
> > You can disable these warnings if you are annoyed by them.  They
> > are just warnings, and will not usually cause any problems when
> > using the compiled code.  See
> > native-comp-async-report-warnings-errors.
> 
> Sweeping warnings under the rug isn't something I'm comfortable with
> even if nine times out of ten (or 999 times in 1000) they're false
> alarms.  (I'm the sort of person that -Werror was made for.)

If you set native-comp-async-report-warnings-errors to the value
'silent', the warnings will be logged in the *Warnings* buffer, where
you can later review them, but will not be shown in a popup buffer,
which might distract you when you don't want that.

> > You can cause these compilations to happen at the beginning of your
> > Emacs session if you change your init files such that Emacs loads all
> > these files.
> ...
> > In any case, Emacs compiles each Lisp file just once, so you should
> > only see these when you start a new Emacs version for the first time.
> 
> I guess, then, that I would want to (somehow) process my init files --
> before using Emacs for the first time after upgrading to a new
> version -- to ensure native compilation of all dependencies.
> 
> Do you have any pointers on how to go about doing that? It seems
> impractical for a casual Emacs user since they don't necessary know
> what files will be loaded by the packages they use.  The warnings
> I've seen so far all refer to files I've never heard of.

If your init file arranges for many packages to load only on demand,
then I don't think there is a way, except summarily compile all the
packages under your ~/.emacs.d/ directory (assuming that's where you
install them).  Maybe we should have a native-compile-directory
function to make that easier; currently we only have
emacs-lisp-native-compile, which compiles a single file.  Andrea,
WDYT? 

> I don't suppose there's a function
> native-compile-eagerly-compile-all-dependencies-of-my-init-files-and-do-it-synchronously-right-now?!

We could do that, but before we do' we'd need to come up with a
find-all-dependecies-of-my-init-files function ;-)

> If I can deal with my init files (ensuring that they won't load any
> files that haven't yet been native compiled), it seems I would be
> left with two potential sources of files being native compiled --
> Emacs itself, and newly installed/updated packages.  Two questions
> about that, then:
> 
> 1. Does the native compiler compile all the .elc files in
> Emacs itself into .eln files ahead of time, or does it also leave
> those until they are loaded?

The preloaded files are natively compiled as part of the build, but
the rest are only natively compiled if the build uses the optional
"ahead-of-time" feature, via a switch to the configure script.

> 2. When the package system installs/updates a package, does it
> natively compile the files at the same time as it byte compiles
> them?

I'm not sure, but I think it doesn't.  Philip, can you answer that?