bug#46256: [feature/native-comp] AOT eln files ignored if run from build tree

[Andrea Corallo via "Bug reports for GNU Emacs, the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrea Corallo <akrl@sdf.org>
>> Cc: 46256@debbugs.gnu.org
>> Date: Thu, 04 Mar 2021 20:11:27 +0000
>> 
>> > Now I make some change in Emacs that modifies the ABI hash, and
>> > rebuild.  The previous subdirectory of native-lisp/ is no longer
>> > valid; if I modify some of the preloaded Lisp files, a new .eln file
>> > is produced in a new subdirectory of native-lisp/.  But now that new
>> > subdirectory has only the *.eln files for those Lisp files I modified
>> > _after_ the ABI-changing change.  Which means most of the preloaded
>> > files do not have *.eln files in the native-lisp/ subdirectory that
>> > corresponds to the latest ABI.  Does this mean Emacs now falls back to
>> > using *.elc files when it produces the emacs.pdmp file?
>> 
>> Yes, I think so.  ATM if the ABI hash is modified something like 'make
>> bootstrap' is needed to re-build all .eln.
>
> Ouch!  We should fix that, because making ABI-breaking changes in the
> tree is a frequent case during development, and bootstrap removes all
> the previous binaries, which is why I never bootstrap.
>
> So currently the only way to fill up a newly created subdirectory of
> native-lisp/ is to manually delete the *.elc files of all the files in
> lisp.mk's $shortlisp list, is that sufficient?

Yes I think so.

The trouble of using make for building such a system is that make is not
aware of the .eln filename, so it should be necessary to ask the Emacs
binary about that to create dynamically the precise (multiple target)
rule.  Not very practical IMO...

In the past I've experimented with making the elc .FORCE targets and
have the Emacs decide what to do, but the downside there is that for
each file that might need compilation Emacs has to start and often
decide that nothing has to be done because the .eln is already there...
As a consequence a make invocation that was supposed to do nothing
became considerably slower.

Another option would be to invoke Emacs only once passing to it the list
of the .el files to be compiled and the parallelism requested and have
Emacs do the job.  I think this might be easier and we have in the
codebase already the all that's needed for that.  The downside is that
we'd drift away from how the vanilla build is working.

Regards

  Andrea