From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Alan Mackenzie Newsgroups: gmane.emacs.devel Subject: Re: Speeding up the bootstrap build - a quick hack. Date: Wed, 19 Jan 2022 21:32:30 +0000 Message-ID: References: <83pmopunzl.fsf@gnu.org> <83y23cu9bl.fsf@gnu.org> <835yqgt0bf.fsf@gnu.org> <835yqfsln1.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="31044"; mail-complaints-to="usenet@ciao.gmane.io" Cc: monnier@iro.umontreal.ca, emacs-devel@gnu.org To: Eli Zaretskii Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Wed Jan 19 22:34:02 2022 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1nAIaf-0007rO-6A for ged-emacs-devel@m.gmane-mx.org; Wed, 19 Jan 2022 22:34:01 +0100 Original-Received: from localhost ([::1]:55424 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nAIae-0004nw-Aa for ged-emacs-devel@m.gmane-mx.org; Wed, 19 Jan 2022 16:34:00 -0500 Original-Received: from eggs.gnu.org ([209.51.188.92]:55166) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nAIZW-00041a-HF for emacs-devel@gnu.org; Wed, 19 Jan 2022 16:32:50 -0500 Original-Received: from colin.muc.de ([193.149.48.1]:18281 helo=mail.muc.de) by eggs.gnu.org with smtp (Exim 4.90_1) (envelope-from ) id 1nAIZU-0000ao-9r for emacs-devel@gnu.org; Wed, 19 Jan 2022 16:32:50 -0500 Original-Received: (qmail 96281 invoked by uid 3782); 19 Jan 2022 21:32:31 -0000 Original-Received: from acm.muc.de (p4fe15af9.dip0.t-ipconnect.de [79.225.90.249]) (using STARTTLS) by colin.muc.de (tmda-ofmipd) with ESMTP; Wed, 19 Jan 2022 22:32:31 +0100 Original-Received: (qmail 22137 invoked by uid 1000); 19 Jan 2022 21:32:30 -0000 Content-Disposition: inline In-Reply-To: <835yqfsln1.fsf@gnu.org> X-Submission-Agent: TMDA/1.3.x (Ph3nix) X-Primary-Address: acm@muc.de Received-SPF: pass client-ip=193.149.48.1; envelope-from=acm@muc.de; helo=mail.muc.de X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.io gmane.emacs.devel:285006 Archived-At: Hello, Eli. On Wed, Jan 19, 2022 at 19:03:46 +0200, Eli Zaretskii wrote: > > Date: Wed, 19 Jan 2022 16:50:11 +0000 > > Cc: monnier@iro.umontreal.ca, emacs-devel@gnu.org > > From: Alan Mackenzie > > The idea is to leave lisp/Makefile.in unchanged, and make all the > > alterations in src/Makefile.in. > > The segment of code you cite above builds all of compile-first in one > > invocation, so there's no need to worry about mixtures of interpreted > > source and .elc. > > So, we duplicate that bit of code, setting the Emacs variable > > no-native-compile on the command line of the first occurrence. This > > will cause the byte compilation of all of compile-first. > > After this bit of new code, we use 'touch' to set the date of these new > > *.elc's back to the distant past. This will ensure that these *.el's > > get built again in the next step. > > The second duplicate of the old bit of make code will build the .eln's, > > using the ("very old") .elc's which are still available inside Emacs. > > It should do this reasonably quickly, because it is using *.elc's. > Yes, that's the idea. > > Yes, I agree that this approach is more elegant and surely easier to > > maintain than my original hack, if it can be made to work (which it > > surely can). I will look at this this evening. Thanks for the > > suggestion. > Thank you. OK, things didn't go quite according to plan. The name compile-first only really has meaning in lisp/Makefile.in, so most of the change had to happen there. What gave trouble was the setting of BYTE_COMPILE_FLAGS. It contains (setq load-prefer-newer t), which prevents the new mechanism from working; the newly build seven .elc files have been given a date in 1970, so the source files are newer and thus get loaded in place of the .elc's. This was surprisingly difficult to solve. There appears to be no way in make to set a variable depending on what the target is. The make manual doesn't say this explicitly, it just depends on vagueness. After an hour of searching for such a feature, it gradually dawns on you that there is no such feature, even though one might be expected. I'm glad the Emacs manuals aren't like that. So I "solved" this problem by passing an argument as an environment variable from src/Makefile.in to lisp/Makefile.in. Yuck! If anybody can come up with a better way, it would be appreciated. Anyhow, this new scheme appears to work, and it's almost certainly better than the first approach. Here's the current patch. Comments and criticisms would be welcome. Thanks! diff --git a/lisp/Makefile.in b/lisp/Makefile.in index 3a72034463..8f4001cf24 100644 --- a/lisp/Makefile.in +++ b/lisp/Makefile.in @@ -74,9 +74,13 @@ loaddefs = AUTOGENEL = ${loaddefs} ${srcdir}/cus-load.el ${srcdir}/finder-inf.el \ ${srcdir}/subdirs.el ${srcdir}/eshell/esh-groups.el +ifeq ($(PREFER_ELC),yes) +BYTE_COMPILE_FLAGS = $(BYTE_COMPILE_EXTRA_FLAGS) +else # Set load-prefer-newer for the benefit of the non-bootstrappers. BYTE_COMPILE_FLAGS = \ --eval '(setq load-prefer-newer t)' $(BYTE_COMPILE_EXTRA_FLAGS) +endif # Files to compile before others during a bootstrap. This is done to # speed up the bootstrap process. They're ordered by size, so we use @@ -303,9 +307,16 @@ .SUFFIXES: # An old-fashioned suffix rule, which, according to the GNU Make manual, # cannot have prerequisites. ifeq ($(HAVE_NATIVE_COMP),yes) +ifeq ($(ANCIENT),yes) +.el.elc: + $(AM_V_ELC)$(emacs) $(BYTE_COMPILE_FLAGS) \ + -l comp -f batch-byte-compile $< + touch -t 197001010000 $@ +else .el.elc: $(AM_V_ELC)$(emacs) $(BYTE_COMPILE_FLAGS) \ -l comp -f batch-byte+native-compile $< +endif else .el.elc: $(AM_V_ELC)$(emacs) $(BYTE_COMPILE_FLAGS) -f batch-byte-compile $< diff --git a/src/Makefile.in b/src/Makefile.in index 04fabd5f42..4b28c01f03 100644 --- a/src/Makefile.in +++ b/src/Makefile.in @@ -914,7 +914,11 @@ $(bootstrap_pdmp): $(RUN_TEMACS) --batch $(BUILD_DETAILS) -l loadup --temacs=pbootstrap \ --bin-dest $(BIN_DESTDIR) --eln-dest $(ELN_DESTDIR) @: Compile some files earlier to speed up further compilation. - $(MAKE) -C ../lisp compile-first EMACS="$(bootstrap_exe)" + @: First, byte compile these files, .... + ANCIENT=yes $(MAKE) -C ../lisp compile-first \ + EMACS="$(bootstrap_exe)" + @: ... then use them in native compiling these and other files. + PREFER_ELC=yes $(MAKE) -C ../lisp compile-first EMACS="$(bootstrap_exe)" endif ### Flymake support (for C only) diff --git a/src/verbose.mk.in b/src/verbose.mk.in index e3f5678303..01076df946 100644 --- a/src/verbose.mk.in +++ b/src/verbose.mk.in @@ -40,12 +40,17 @@ AM_V_CXX = @$(info $ CXX $@) AM_V_CCLD = @$(info $ CCLD $@) AM_V_CXXLD = @$(info $ CXXLD $@) ifeq ($(HAVE_NATIVE_COMP),yes) -ifeq ($(NATIVE_DISABLED),1) +ifneq ($(NATIVE_DISABLED),1) +ifneq ($(ANCIENT),yes) +AM_V_ELC = @$(info $ ELC+ELN $@) +AM_V_ELN = @$(info $ ELN $@) +else AM_V_ELC = @$(info $ ELC $@) AM_V_ELN = +endif else -AM_V_ELC = @$(info $ ELC+ELN $@) -AM_V_ELN = @$(info $ ELN $@) +AM_V_ELC = @$(info $ ELC $@) +AM_V_ELN = endif else AM_V_ELC = @$(info $ ELC $@) -- Alan Mackenzie (Nuremberg, Germany).