bug#46617: 28.0.50; nativecomp: native compile cache is not invalidated when file is re-byte compiled and changes

[Andrea Corallo via "Bug reports for GNU Emacs, the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>]

Aaron Jensen <aaronjensen@gmail.com> writes:

> On Sun, Mar 21, 2021 at 12:27 PM Andrea Corallo <akrl@sdf.org> wrote:
>>
>> Hi Aaron,
>>
>> I had to slightly modify your init.el reproducer as we have changed the
>> semantic of `load' [1].
>
> With this change, how do you specify that you want the eln to load if
> it's there? If I understand correctly, this prevents loading the eln
> with load unless it is specified explicitly? Or can you leave the
> extension off when loading and that will prefer the eln, then elc,
> then el?

Exactly load by default tries to add the extentions to the FILE
argument.

>> I think what happened is that in the original reproducer the second load
>> of the .elc file still automatically loaded in place the corresponding
>> .eln as the source file didn't changed.  The issue here is that in the
>> compilation unit repro.elk is capturing some of the environment and this
>> is not accounted in the hash computation of the .eln file.
>
> The original repro simulated a restart without a restart. The real bug
> (in straight.el) is that on start, it loads a byte compiled/native
> compiled file that relies on outside environment. So, that may be
> fixed now because I believe they load the elc explicitly, but that
> means it'll never load the native compiled version. Ideally, they
> would, but I don't know how to force loading byte compiled OR native
> compiled in one command.

ATM to force the byte-code version to be used one can use `load' making
the .elc extension explicit (or without using the extension binding
`load-no-native' to non-nil).

To explicitly force load the native code version one has to do like:

(load (comp-el-to-eln-filename ".../foo.el"))

But this is limited as it works only if the .eln is in the first
directory of `com-eln-load-path', so probably we have to offer a more
complete solution.

>> The new reproducer (well it does not reproduce much :) ) works because
>> now `load' called explicitly on a .elc file does not load automatically
>> the corresponding .eln.  Alternatively to overcome the issue still using
>> native compilation one has to force a new native compilation using
>> `native-compile.  WDYT?
>
> I think it sounds like straight.el will need to add detection for
> native compilation, and any time they byte compile, also native
> compile. I also think that this is extremely fragile and I wonder if
> byte compiling should result in "invalidating" (deleting) the
> previously corresponding eln. We essentially have a cache invalidation
> that is relying purely on a cache key that is incomplete.

Mmhh maybe it's a good idea.  The other option would be to re-introduce
the constrain that, to be used in place, the .eln must be younger than
the .elc.

Thanks

  Andrea