bug#43532: [feature/native-comp] *.eln file name hashing algorithm doesn't seem to play nice with NATIVE_FULL_AOT and self-contained Emacs.app builds for macOS

[Jim Myhrberg <contact@jimeh.me>]

Hi,

TL;DR:

If I've understood correctly, the filename of the cached *.eln files
includes a hash based on the absolute file path and content of the
source *.el/*.el.gz file. On macOS when producing a self-contained
Emacs.app bundle this means that the cached *.eln files bundled into
the app cannot be used, as their absolute path won't match what they
were during build time. And app bundles on macOS can be placed and
launched from anywhere on disk.

I'm not sure what the best solution here might be. But if the actual
content of the file is used to produce the hash, maybe there's no need
to use the full absolute file path, and instead just the filename
itself, along with the content?

The Long Version:

I'm not sure how familiar people here might be with macOS application
bundles, so here's a brief summary just in case; Application bundles
are self-contained macOS GUI applications which can be launched by for
example double clicking on them, among other ways. On a technical
level, they're simply a folder called "<AppName>.app" which contains
all assets needed to run the application. The main executable is
"<AppName>.app/Contents/MacOS/<AppName>".

In the case of building a self-contained Emacs.app on macOS, it looks
the *.eln caches are generated for the files located in
"<workdir>/nextstep/Emacs.app/Contents/Resources/lisp", and the *.eln
cache folder is
"<workdir>/nextstep/Emacs.app/Contents/MacOS/lib/emacs/28.0.50/native-lisp".
Which means the full set of *.eln caches can only be used when
launched from within the Emacs source directory where you built the
application.

Typical usage on macOS would have the Emacs.app movied/copied to the
"/Applications" folder, but there's no guarantee about that, as
applications are supposed work fine regardless of where it's located
on disk they're located.

I've done some tests while using NATIVE_FULL_AOT=1, which produces
1,572 *.eln files within Emacs.app. But when starting emacs with my
config, it starts by compiling a whole bunch of files from
"Emacs.app/Contents/Resources/lisp/", like "emacs-lisp/cconv.el.gz",
"emacs-lisp/bytecomp.el.gz", "help-mode.el.gz", and others. If I grab
the new *.eln files it produced in my user cache, and move them to
within Emacs.app, it doesn't compile them on launch again. If I then
move Emacs.app to another location on disk, it does compile them
again, as the absolute path has changed.

However there's 83 *.eln files which are loaded from Emacs.app,
because if they're removed, it fails to start at all with a dlopen
error instead. I don't know enough about the internals of Emacs, but
I'd assume they are referred to from the dump file, while the others
are loaded up due to my config requiring them, at which point a cache
miss happens due to the absolute path yielding a different change
filename.


Thanks :)