bug#46502: 28.0.50; [feature/native-comp] (d3a399dd) native-comp bootstrap failure

[Pip Cet <pipcet@gmail.com>]

On Fri, Feb 19, 2021 at 1:47 PM Eli Zaretskii <eliz@gnu.org> wrote:
> > From: Pip Cet <pipcet@gmail.com>
> > Date: Fri, 19 Feb 2021 13:31:49 +0000
> > Cc: akrl@sdf.org, mwd@md5i.com, 46502@debbugs.gnu.org
> >
> > > > One thing I've noticed in my experiments is that many builds that are
> > > > interrupted at the wrong point and then resumed produce different
> > > > results. I.e. you type "make", then hit Ctrl-C at the wrong time, then
> > > > type "make" again and you get a different result.
> > >
> > > What does "different result" mean in this case? is the produced .eln
> > > file different? or something else?
> >
> > There are differences both in the .elc and .eln, and I saw different
> > success/failure behavior but only with local modifications.
>
> Let's talk about *.elc files first, as this is not supposed to happen.

Okay. Here's what I'm seeing (master branch, interrupted after
macroexp.elc is compiled).

--- build-uninterrupted/lisp/emacs-lisp/bytecomp.elc    2021-02-20
01:13:51.706632694 +0000
+++ build-interrupted/lisp/emacs-lisp/bytecomp.elc    2021-02-20
02:26:57.242906775 +0000
-(defalias 'byte-compile-form #[513
"\211 :\204H\09\203 \0\211\305>\206
\0\306!\262\203+\09\203$\0\307!\210\310!\210\202\203A\0
   \203A\09\203<\0\307!\210\311
\202f\312!\210\202f@9\2031@\211\313N\314N\206\\\0\n>\315>\203\236\0
A\211:\203\235\0\211@\211:\203\234\0\211@\211\316=\203\233\0A\211:\203\232\0\211@A\211\204\230\0\211
>\203\227\0\317\320\321 \f
#!\210\210\266\210\210\210\210\322>\203\266\0
A\242\211\242\316=\203\265\0\323A@\311\"\210\210\324\325!\203\320\0\211\305>\206\306\0\306!\262\203\320\0\326\327
\"\210\324\314 \"\203\375\0\211\203\375\0\326\330
;\203\355\0\331\332\333  !\"\202\373\0 \334=\204\372\0\320\335
\"\202\373\0\336#\210 @K\242\337=\203 \317\331\340
@#!\210\203)\341!\203)\342=\204) !\266 \202f\343 !\266
\202f\344@!\203E\f\345>\203E\346!\210\202f@\242\347=\203b\350
!\211\262 =\204b\35\"\210\311 \202f\343!\21\205l\352
)\207" [byte-compile--for-effect byte-compile-delete-errors
byte-compile-interactive-only-functions byte-compile-lexical-variables
byte-optimize (nil t) keywordp byte-compile-set-symbol-position
byte-compile-constant nil byte-compile-variable-ref byte-compile
interactive-only (set symbol-value run-hooks add-hook remove-hook
run-hook-with-args run-hook-with-args-until-success
run-hook-with-args-until-failure) quote byte-compile-report-error
format-message "%s cannot use lexical var `%s'" (add-hook remove-hook)
byte-compile-check-variable byte-compile-warning-enabled-p suspicious
byte-compile-warn "`%s' called as a function" "`%s' is for interactive
use only%s" format "; %s" substitute-command-keys t "; use `%s'
instead." "." macro "`%s' defined after use in %S (missing `require'
of a library file?)" functionp cl-byte-compile-compiler-macro
byte-compile-normal-call byte-code-function-p (t lap)
byte-compile-unfold-bcf lambda macroexp--unfold-lambda
byte-compile-form byte-compile-discard] 17 (#$ . 90529)])
\350>\203\221\351!\210\202\262@\242\352=\203\256\353 !\211\262
=\204\256\35\"\210\306 \202\262\346!\21\205\270\355
)\207" [byte-compile--for-effect byte-compile-const-variables
byte-compile-delete-errors byte-compile-interactive-only-functions
byte-compile-lexical-variables byte-optimize nil (nil t) keywordp
boundp (setting-constant) t byte-compile-set-symbol-position
byte-compile-constant byte-compile-variable-ref byte-compile
interactive-only (set symbol-value run-hooks add-hook remove-hook
run-hook-with-args run-hook-with-args-until-success
run-hook-with-args-until-failure) quote byte-compile-report-error
format-message "%s cannot use lexical var `%s'" (add-hook remove-hook)
byte-compile-check-variable byte-compile-warning-enabled-p suspicious
byte-compile-warn "`%s' called as a function" "`%s' is for interactive
use only%s" format "; %s" substitute-command-keys "; use `%s'
instead." "." macro "`%s' defined after use in %S (missing `require'
of a library file?)" functionp cl-byte-compile-compiler-macro
byte-compile-normal-call byte-code-function-p (t lap)
byte-compile-unfold-bcf lambda macroexp--unfold-lambda
byte-compile-form byte-compile-discard] 17 (#$ . 90718)])

It looks like macroexp--const-symbol-p, a defsubst, is substituted in
both cases, but differently. This makes sense to me given the make
rules, and it's not harmful, but I don't like it.

> AFAIR, we write the bytecode into a temporary file, and then rename it
> atomically only when the compilation finishes successfully.  So
> interrupting should not do any harm, and therefore I'm curious what
> kind of differences in *.elc files do you see in these cases.

I'm not sure I'm seeing harmful differences, just ... differences.

So what happens on the three branches appears to be:

master: harmless differences between interrupted make and uninterrupted make
native-comp: interrupting make at the wrong point will sometimes
result in an unbuildable tree
local: interrupting make at the wrong point and resuming it will
sometimes succeed when the uninterrupted build wouldn't have

We should deal with the first two; the third is likely to be related
to my stack size limit or my local changes.

> > It's possible that this is all harmless, but I have the bad habit of
> > assuming I can just type "make" again and have it resume an
> > interrupted build, and that certainly does not work on the
> > native-comp branch (I'm not sure it works on the master branch).
> I'd suggest to start with master, as that is supposed to be much more
> mature.  If that turns out to work correctly (and (IME it is), then we
> could take a look at the native-comp branch, where there could be
> problems we didn't yet fix.

I'll leave it to you to decide whether the harmless differences I've
seen so far count as incorrect behavior.

> In general, Make itself will delete any target files it knows about
> that were not fully built at the time of SIGINT.  Maybe we don't tell
> Make enough about the files native-comp produces?

So we should delete the early-compilation .elcs if we end up deleting
bootstrap-emacs.pdmp, too?

I'll try getting a usable backtrace for the stack overflow.