From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Konstantin Kharlamov Newsgroups: gmane.emacs.bugs Subject: bug#74382: `compile-first` Make rule is no longer using `load-prefer-newer` Date: Sat, 16 Nov 2024 20:45:47 +0300 Message-ID: References: <6bc3a410f0857c3e3433070ac19deaf7eae88c63.camel@yandex.ru> Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="31593"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Evolution 3.54.1 Cc: 74382@debbugs.gnu.org To: Alan Mackenzie Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Sat Nov 16 18:47:21 2024 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1tCMtF-00082B-Ai for geb-bug-gnu-emacs@m.gmane-mx.org; Sat, 16 Nov 2024 18:47:21 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tCMsy-0008Jw-Bu; Sat, 16 Nov 2024 12:47:05 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tCMsw-0008JV-Rq for bug-gnu-emacs@gnu.org; Sat, 16 Nov 2024 12:47:02 -0500 Original-Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tCMsw-0001qd-D8 for bug-gnu-emacs@gnu.org; Sat, 16 Nov 2024 12:47:02 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=debbugs.gnu.org; s=debbugs-gnu-org; h=MIME-Version:References:In-Reply-To:Date:From:To:Subject; bh=NwMkyhB4EKBd16yTjP6M/qYrSYJ8I23wUdfoPYEa8Kg=; b=KMSsJvq1zZagAQsTkfa7VeyxWD5QPmsndP+85FdKFUSWPFtY3v2zqzUT2CQUYFayz7JCNZunsdb2EHa8zzgntNPhMJxfFKeqBpT/2vxIuHRmeFub4QMni8ssxrgk88RayNJjWI9RDP1/p6xxMBSnlQuCg9Xt+m0I9mBfy4wZZwIX8MQqZ8kEevaKgkOJj2jOr2Mwbsk+7FBgirhjPjOYzEdoVyBE8GTcdD9Ut6DW4ZjNan66b4o9YVgqC4h8vQGoer6z0rIaibCgJVXveXWADZsYfQ1dWLsKCd1m421OzEgsxGozVWLWO+dKoiCoEmuWtHlvZQulu7Ez331lh+gdag==; Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1tCMsw-0006sk-8k for bug-gnu-emacs@gnu.org; Sat, 16 Nov 2024 12:47:02 -0500 X-Loop: help-debbugs@gnu.org Resent-From: Konstantin Kharlamov Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sat, 16 Nov 2024 17:47:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 74382 X-GNU-PR-Package: emacs Original-Received: via spool by 74382-submit@debbugs.gnu.org id=B74382.173177916326352 (code B ref 74382); Sat, 16 Nov 2024 17:47:02 +0000 Original-Received: (at 74382) by debbugs.gnu.org; 16 Nov 2024 17:46:03 +0000 Original-Received: from localhost ([127.0.0.1]:54695 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tCMrz-0006qx-69 for submit@debbugs.gnu.org; Sat, 16 Nov 2024 12:46:03 -0500 Original-Received: from forward502b.mail.yandex.net ([178.154.239.146]:33506) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tCMrv-0006qD-8f for 74382@debbugs.gnu.org; Sat, 16 Nov 2024 12:46:01 -0500 Original-Received: from mail-nwsmtp-smtp-production-main-36.sas.yp-c.yandex.net (mail-nwsmtp-smtp-production-main-36.sas.yp-c.yandex.net [IPv6:2a02:6b8:c24:18aa:0:640:5723:0]) by forward502b.mail.yandex.net (Yandex) with ESMTPS id 74E306116F; Sat, 16 Nov 2024 20:45:50 +0300 (MSK) Original-Received: by mail-nwsmtp-smtp-production-main-36.sas.yp-c.yandex.net (smtp/Yandex) with ESMTPSA id mjSAd2MOdOs0-zeP1oTg8; Sat, 16 Nov 2024 20:45:50 +0300 X-Yandex-Fwd: 1 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1731779150; bh=NwMkyhB4EKBd16yTjP6M/qYrSYJ8I23wUdfoPYEa8Kg=; h=References:Date:In-Reply-To:Cc:To:From:Subject:Message-ID; b=bngocX1gN2jMkz8s4OpEL8uycp+h0LSoPQgBhul0B5tTnp0Ne2kYscFtZsM5pviiL TfnuzO2nmtUICSQWZ4I9D+NC2LmjNWAakNNDKZJnxjHwnBXjf06S4AipuG6gwk4AgT zVo/EoCrQD3ANyu5D5w+b7FOzW8r6Rur+ur4Vlvo= Authentication-Results: mail-nwsmtp-smtp-production-main-36.sas.yp-c.yandex.net; dkim=pass header.i=@yandex.ru In-Reply-To: X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.bugs:295478 Archived-At: On Sat, 2024-11-16 at 16:53 +0000, Alan Mackenzie wrote: > Hello, Konstantin. > > On Sat, Nov 16, 2024 at 18:11:01 +0300, Konstantin Kharlamov wrote: > > CC: Alan Mackenzie, author of the change in > > 10083e788f7349fa363d100687dc3d94bea88f57 > > > I've seen for a long time Emacs master builds fail from time to > > time in spectacular > > ways after updating the repo, sometimes so badly that `make clean` > > doesn't help. > > > I never dug into that though, but I'm attributing this to the > > occasional build > > messages similar to: > > > =C2=A0=C2=A0=C2=A0 Source file =E2=80=98/home/constantine/Projects/emac= s/lisp/emacs- > > lisp/comp.el=E2=80=99 newer than byte-compiled file; using older file > > =C2=A0=C2=A0=C2=A0 Source file =E2=80=98/home/constantine/Projects/emac= s/lisp/emacs- > > lisp/bytecomp.el=E2=80=99 newer than byte-compiled file; using older fi= le > > =C2=A0=C2=A0=C2=A0 Source file =E2=80=98/home/constantine/Projects/emac= s/lisp/emacs- > > lisp/comp-cstr.el=E2=80=99 newer than byte-compiled file; using older f= ile > > > =E2=80=A6which makes sense, because if the repo changed `comp.el` API a= nd > > Emacs during the > > build of newer files is trying to make use of older `.elc` file and > > hence the older > > API, it may result in failure. > > The idea is that the byte compiler is first built from .el files, > giving > ..elc files.=C2=A0 The .elc byte compiler is then used for all the other > files, since it's much faster. > > > Got some spare time today, dug into one of the messages. From what > > I understand it's > > caused by this line `lisp/Makefile.in`: > > > =C2=A0=C2=A0=C2=A0 # ... but we must prefer .elc files for those in the= early > > bootstrap. > > =C2=A0=C2=A0=C2=A0 compile-first: BYTE_COMPILE_FLAGS =3D $(BYTE_COMPILE= _EXTRA_FLAGS) > > > > From what I understand, this rewrites BYTE_COMPILE_FLAGS to be an > > > empty variable, > > which results in `(setq load-prefer-newer t)` being stripped off of > > the build. > > Yes, this is to prefer the faster .elc byte compiler, whose file > dates > have been set back to the epoch (1970-01-01:00:00:00 UTC).=C2=A0 I think > this > was to ensure that when we come to native compilation, the .el source > files of the compiler will be newer than the .elc files, hence > triggering a native compilation of them. > > I honestly don't believe that this mechanism for speeding up the > early > build is the cause of the spectacular failures in some of your > builds, > but I could be wrong. > > > The straightforward solution is to remove this line. But since the > > line's commentary > > opposes to such solution, I'm starting up a discussion what exactly > > should be the > > behavior here =F0=9F=98=8A > > Perhaps if you could be more specific about the failures you see, we > might be able to be of more help. Sure, I just reproduced it after removing all `.elc` files in the repo, here how: 1. `git checkout f2f13fa630b` (a commit from April) 2. `make -j$(nproc)` to compile. Note: you don't need to wait for build to finish, I just waited for all files under `lisp/emacs-lisp` directory to finish compilation, and then ^C'ed it. 3. `git checkout 29098a291f5` (a November commit). 4. `make -j$(nproc)` ## Expected Build finishes ## Actual It fails with stacktrace so huge that it goes beyond Konsole scrollback and with the following message: Loading macroexp.elc (compiled; note, source file is newer)... Wrong type argument: "../../lisp/emacs-lisp/comp.el", hash-table-p, (#s= (byte-to-native-top-level (byte-code "=EF=BF=BD=EF=BF=BD!=EF=BF=BD=EF=BF=BD= =EF=BF=BD!=EF=BF=BD=EF=BF=BD=EF=BF=BD!=EF=BF=BD=EF=BF=BD=EF=BF=BD!=EF=BF=BD= =EF=BF=BD=EF=BF=BD!=EF=BF=BD=EF=BF=BD=EF=BF=BD!=EF=BF=BD=EF=BF=BD=EF=BF=BD!= =EF=BF=BD=EF=BF=BD=EF=BF=BD!=EF=BF=BD=EF=BF=BD=EF=BF=BD=EF=BF=BD=EF=BF=BD= =EF=BF=BD%=EF=BF=BD=EF=BF=BD=EF=BF=BD=EF=BF=BD=EF=BF=BDDD=EF=BF=BD=EF=BF=BD= =EF=BF=BD=EF=BF=BD=EF=BF=BD& =EF=BF=BD=EF=BF=BD=EF=BF=BD=EF=BF=BD=EF= =BF=BDDD=EF=BF=BD=EF=BF=BD=EF=BF=BD=EF=BF=BD=EF=BF=BD& =EF=BF=BD=EF=BF=BD= =EF=BF=BD=EF=BF=BD=EF=BF=BDDD=EF=BF=BD=EF=BF=BD=EF=BF=BD&=EF=BF=BD=EF=BF=BD= =EF=BF=BD=EF=BF=BD=EF=BF=BDDD=EF=BF=BD=EF=BF=BD=EF=BF=BD=EF=BF=BD=EF=BF=BD&= =EF=BF=BD=EF=BF=BD=EF=BF=BD=EF=BF=BD=EF=BF=BD=EF=BF=BDDD=EF=BF=BD=EF=BF=BD= =EF=BF=BD=EF=BF=BD=EF=BF=BD&=EF=BF=BD=EF=BF=BD=EF=BF=BD=EF=BF=BD=EF=BF=BD= =EF=BF=BDDD=EF=BF=BD=EF=BF=BD=EF=BF=BD=EF=BF=BD=EF=BF=BD&=EF=BF=BD=EF=BF=BD= =EF=BF=BD=EF=BF=BD=EF=BF=BD=EF=BF=BDDD=EF=BF=BD=EF=BF=BD=EF=BF=BD=EF=BF=BD= =EF=BF=BD&=EF=BF=BD" [require bytecomp cl-lib gv rx subr-x warnings comp-co= mmon comp-cstr custom-declare-group ...] 10) t) #s(byte-to-native-top-level= (# # nil "Non-nil= to prevent native-compiling of Emacs Lisp code. Note that when `no-byte-compile' is set to non-nil it overrides the val= ue of `no-native-compile'. This is normally set in local file variables at the end of the Emacs Lisp file: You can see it fails on the `comp.el`, and I presume the reason is it simpl= y didn't rebuild the necessary .elc files, and instead loaded outdated byte= code.