From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: Mark H Weaver Newsgroups: gmane.lisp.guile.devel Subject: Re: Bootstrap optimization Date: Sat, 27 Oct 2018 21:34:46 -0400 Message-ID: <87y3aiam5q.fsf@netris.org> References: NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain X-Trace: blaine.gmane.org 1540690433 24021 195.159.176.226 (28 Oct 2018 01:33:53 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Sun, 28 Oct 2018 01:33:53 +0000 (UTC) User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux) Cc: guile-devel To: Mikael Djurfeldt Original-X-From: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Sun Oct 28 02:33:48 2018 Return-path: Envelope-to: guile-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gGZxc-00069l-FH for guile-devel@m.gmane.org; Sun, 28 Oct 2018 02:33:48 +0100 Original-Received: from localhost ([::1]:38387 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gGZzi-0001nR-Rz for guile-devel@m.gmane.org; Sat, 27 Oct 2018 21:35:58 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:47260) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gGZzP-0001nL-VT for guile-devel@gnu.org; Sat, 27 Oct 2018 21:35:40 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gGZzM-0002HE-S8 for guile-devel@gnu.org; Sat, 27 Oct 2018 21:35:39 -0400 Original-Received: from world.peace.net ([64.112.178.59]:57460) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gGZzJ-000258-2q for guile-devel@gnu.org; Sat, 27 Oct 2018 21:35:33 -0400 Original-Received: from mhw by world.peace.net with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.89) (envelope-from ) id 1gGZz3-0003gt-IW; Sat, 27 Oct 2018 21:35:17 -0400 In-Reply-To: (Mikael Djurfeldt's message of "Thu, 25 Oct 2018 14:25:29 +0200") X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 64.112.178.59 X-BeenThere: guile-devel@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: "Developers list for Guile, the GNU extensibility library" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Original-Sender: "guile-devel" Xref: news.gmane.org gmane.lisp.guile.devel:19691 Archived-At: Hi Mikael, Mikael Djurfeldt writes: > I find that the attached patch saves a few minutes for parallel builds > on a four core machine. What it does is to build both eval.go and > psyntax.pp.go serially before the rest is built in parallel (when make > is given -j). > > Here's an attempt at explanation why this saves time: > > Let's denote the .go objects by numbers 1, 2, ... where 1 is eval.go > and 2 is psyntax.pp.go. We also prepend a prefix "S" for a slow build > with interpreted psyntax.pp and "F" for a fast build. > > Assuming a four core machine, we previously had something like: > > S1 > S2 S3 S4 S5 > S6 S7 S8 (since S2 builds so slowly) > F9 F10 ... > > Now, instead, we have: > > S1 > S2 > F3 F4 F5 F6 > ... > > On my machine, the patch saved 10 minutes out of 55 minutes without > the patch. Yes, I've also noticed this issue, and long thought that it would be better to serialize building several of the early bootstrap .go objects. However ... > From 0fe521100487411a2c48f6af14796a22964844a1 Mon Sep 17 00:00:00 2001 > From: Mikael Djurfeldt > Date: Thu, 25 Oct 2018 13:53:47 +0200 > Subject: [PATCH] Bootstrap optimization > > * bootstrap/Makefile.am: Build both eval.go and psyntax-pp.go before the rest > of the .go files so that they are handled by a fast macro expander. This > saves time for a parallel build. > --- > bootstrap/Makefile.am | 7 +++++-- > 1 file changed, 5 insertions(+), 2 deletions(-) > > diff --git a/bootstrap/Makefile.am b/bootstrap/Makefile.am > index 57b62eb56..3e92ef1af 100644 > --- a/bootstrap/Makefile.am > +++ b/bootstrap/Makefile.am > @@ -32,5 +32,8 @@ GUILE_OPTIMIZATIONS = -O1 -Oresolve-primitives > include $(top_srcdir)/am/bootstrap.am > > # We must build the evaluator first, so that we can be sure to control > -# the stack. > -$(filter-out ice-9/eval.go, $(GOBJECTS)): ice-9/eval.go > +# the stack. Then, we build the syntax-case macro expander before the > +# rest, in order to speed up parallel builds. > +ice-9/psyntax-pp.go: ice-9/eval.go > + > +$(filter-out ice-9/eval.go ice-9/psyntax-pp.go, $(GOBJECTS)): ice-9/psyntax-pp.go The downside of this approach to serialization is that when we add file X.scm to the list of objects to build serially, we force a full rebuild whenever X.scm is modified. At present, eval.scm is the only file that forces a full rebuild. Your patch would add psyntax-pp.scm to that list. I don't feel strongly about it, and maybe your patch is still a net benefit, but I very much wish we had a better way to optimize the early bootstrap without adding these bogus dependencies. > To which branch should this be applied, stable-2.2 or master? If we decide to adopt this approach, it should probably be committed to stable-2.2, but first I'd like to give other people an opportunity to share their thoughts on this. Thoughts? Mark