bug#74382: `compile-first` Make rule is no longer using `load-prefer-newer`

[Alan Mackenzie <acm@muc.de>]

Hello, Konstantin.

On Sat, Nov 16, 2024 at 18:11:01 +0300, Konstantin Kharlamov wrote:
> CC: Alan Mackenzie, author of the change in 10083e788f7349fa363d100687dc3d94bea88f57

> I've seen for a long time Emacs master builds fail from time to time in spectacular
> ways after updating the repo, sometimes so badly that `make clean` doesn't help.

> I never dug into that though, but I'm attributing this to the occasional build
> messages similar to:

>     Source file ‘/home/constantine/Projects/emacs/lisp/emacs-lisp/comp.el’ newer than byte-compiled file; using older file
>     Source file ‘/home/constantine/Projects/emacs/lisp/emacs-lisp/bytecomp.el’ newer than byte-compiled file; using older file
>     Source file ‘/home/constantine/Projects/emacs/lisp/emacs-lisp/comp-cstr.el’ newer than byte-compiled file; using older file

> …which makes sense, because if the repo changed `comp.el` API and Emacs during the
> build of newer files is trying to make use of older `.elc` file and hence the older
> API, it may result in failure.

The idea is that the byte compiler is first built from .el files, giving
..elc files.  The .elc byte compiler is then used for all the other
files, since it's much faster.

> Got some spare time today, dug into one of the messages. From what I understand it's
> caused by this line `lisp/Makefile.in`:

>     # ... but we must prefer .elc files for those in the early bootstrap.
>     compile-first: BYTE_COMPILE_FLAGS = $(BYTE_COMPILE_EXTRA_FLAGS)

> >From what I understand, this rewrites BYTE_COMPILE_FLAGS to be an empty variable,
> which results in `(setq load-prefer-newer t)` being stripped off of the build.

Yes, this is to prefer the faster .elc byte compiler, whose file dates
have been set back to the epoch (1970-01-01:00:00:00 UTC).  I think this
was to ensure that when we come to native compilation, the .el source
files of the compiler will be newer than the .elc files, hence
triggering a native compilation of them.

I honestly don't believe that this mechanism for speeding up the early
build is the cause of the spectacular failures in some of your builds,
but I could be wrong.

> The straightforward solution is to remove this line. But since the line's commentary
> opposes to such solution, I'm starting up a discussion what exactly should be the
> behavior here 😊

Perhaps if you could be more specific about the failures you see, we
might be able to be of more help.

-- 
Alan Mackenzie (Nuremberg, Germany).