bug#46256: [feature/native-comp] AOT eln files ignored if run from build tree

[Pip Cet <pipcet@gmail.com>]

On Fri, Mar 5, 2021 at 9:33 AM Andrea Corallo via Bug reports for GNU
Emacs, the Swiss army knife of text editors <bug-gnu-emacs@gnu.org>
wrote:
> Eli Zaretskii <eliz@gnu.org> writes:
> > So currently the only way to fill up a newly created subdirectory of
> > native-lisp/ is to manually delete the *.elc files of all the files in
> > lisp.mk's $shortlisp list, is that sufficient?
>
> Yes I think so.
>
> The trouble of using make for building such a system is that make is not
> aware of the .eln filename, so it should be necessary to ask the Emacs
> binary about that to create dynamically the precise (multiple target)
> rule.  Not very practical IMO...

I do wonder whether the whole filename scheme is really the best option.

IIUC, and that's a big if in this case, the main motivation for using
hashes in the .eln filenames is that dlopen() is broken and may return
the same handle for subsequent dlopen()s of the same name, even if the
underlying file changed in between.

Merely verifying that the ABI is correct could be done at runtime, so
that's no reason to keep a hash in the filename.

So my vague idea is this:

1. implement fixed_dlopen(), which keeps track of filenames that have
been opened and, if necessary, creates a temporary file and loads that
instead of its argument.
2. compile lisp/emacs-lisp/bytecomp.el to lisp/emacs-lisp/bytecomp.elc
and native-lisp/emacs-lisp/bytecomp.eln
3. add extra code in the top level function of each .eln to check that
the ABI is correct.

This would allow us to use standard make rules. It would also make
.eln filenames predictable. It might even draw someone's attention to
the fact that dlopen() is broken and make them fix it.

I'm probably missing other good reasons for the hashed filename scheme.

Pip