From mboxrd@z Thu Jan 1 00:00:00 1970 From: ludo@gnu.org (Ludovic =?UTF-8?Q?Court=C3=A8s?=) Subject: bug#27284: [PATCH 1/8] build: Factorize module compilation in (guix build compile). Date: Sun, 22 Oct 2017 22:10:12 -0700 Message-ID: <874lqqv497.fsf@gnu.org> References: <87poamv2i7.fsf@gnu.org> <20171020160557.27096-1-ludo@gnu.org> <20171020160557.27096-2-ludo@gnu.org> <87efpu5142.fsf@gmail.com> <87a80iws1q.fsf@gnu.org> <20171022164216.5fdf9f55@centurylink.net> <87r2tuvaot.fsf@gnu.org> <20171022175253.2b72da76@centurylink.net> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:51652) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e6V0z-0002UJ-Rv for bug-guix@gnu.org; Mon, 23 Oct 2017 01:11:07 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e6V0w-0006F7-Jl for bug-guix@gnu.org; Mon, 23 Oct 2017 01:11:05 -0400 Received: from debbugs.gnu.org ([208.118.235.43]:47810) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1e6V0w-0006Ev-Ea for bug-guix@gnu.org; Mon, 23 Oct 2017 01:11:02 -0400 Sender: "Debbugs-submit" Resent-Message-ID: In-Reply-To: <20171022175253.2b72da76@centurylink.net> (Eric Bavier's message of "Sun, 22 Oct 2017 17:52:53 -0500") List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-guix-bounces+gcggb-bug-guix=m.gmane.org@gnu.org Sender: "bug-Guix" To: Eric Bavier Cc: 27284@debbugs.gnu.org, Maxim Cournoyer --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Eric Bavier skribis: > On Sun, 22 Oct 2017 19:51:14 -0700 > ludo@gnu.org (Ludovic Court=C3=A8s) wrote: [...] >> > It Would Be Nice if the functionality worked for older versions of make >> > that people might have on their systems.=20=20 >>=20 >> With the patch I posted, Scheme compilation would always use one thread >> per core, which is what it currently does in =E2=80=98master=E2=80=99. Oops, that was not quite true: it would use one thread. I fixed it like this: --=-=-= Content-Type: text/x-patch Content-Disposition: inline diff --git a/build-aux/compile-all.scm b/build-aux/compile-all.scm index 4aa4ac9b9..c7ca5a6f6 100644 --- a/build-aux/compile-all.scm +++ b/build-aux/compile-all.scm @@ -19,6 +19,7 @@ (use-modules (ice-9 match) (ice-9 threads) + (srfi srfi-1) (guix build compile) (guix build utils)) @@ -50,24 +51,32 @@ to 'make'." (match flags (#f (current-processor-count)) (flags - (let loop ((flags (string-tokenize flags))) - (match flags - (() - 1) - (("-j" (= string->number count) _ ...) - (if (integer? count) - count - (current-processor-count))) - ((head tail ...) - (if (string-prefix? "-j" head) - (match (string-drop head 2) - ("" - (current-processor-count)) - ((= string->number count) - (if (integer? count) - count - (current-processor-count)))) - (loop tail)))))))) + (let ((initial-flags (string-tokenize flags))) + (let loop ((flags initial-flags)) + (match flags + (() + ;; Note: GNU make prior to version 4.2 would hide "-j" flags from + ;; $MAKEFLAGS. Thus, check for a "--jobserver" flag here and + ;; assume we're using all cores if specified. + (if (any (lambda (flag) + (string-prefix? "--jobserver" flag)) + initial-flags) + (current-processor-count) ;GNU make < 4.2 + 1)) ;sequential make + (("-j" (= string->number count) _ ...) + (if (integer? count) + count + (current-processor-count))) + ((head tail ...) + (if (string-prefix? "-j" head) + (match (string-drop head 2) + ("" + (current-processor-count)) + ((= string->number count) + (if (integer? count) + count + (current-processor-count)))) + (loop tail))))))))) ;; Install a SIGINT handler to give unwind handlers in 'compile-file' an ;; opportunity to run upon SIGINT and to remove temporary output files. --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable >> > Using the jobserver directly would require quite a bit of work for the >> > current patch set, but I wonder if there is another way to determine >> > the -jN parameter for make<4.2 that we could use. Maybe simply >> > polling the jobserver fds at the start?=20=20 >>=20 >> AIUI the job server does not reveal how many jobs are allowed. It >> merely grants you an execution token. >>=20 >> Or did you have something else in mind? > > The idea (hardely tested) would be to read tokens from the input fd > until it blocks, do the scheme compiles with however many tokens were > read, then write them back out. Crude, I guess, and probably error > prone; compile-all.scm could be invoked when make has job tokens tied > up building the daemon source... > > Anyhow, the current patch works well for me with a recent make. I'm > content leaving further improvements to a future hypothetical hacker. :) Sounds reasonable. Let=E2=80=99s make sure the hypothetical hacker has eno= ugh on their plate. :-) I=E2=80=99ve pushed this and the easy parts of this patch series, with comm= it d298c815e638581d466222f3a883b280f019b368 as the tip. Thanks for the review! Ludo=E2=80=99. --=-=-=--