From mboxrd@z Thu Jan 1 00:00:00 1970 From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) Subject: Re: [PATCH] build: Speed up .go compilation. Date: Fri, 08 Jan 2016 18:06:39 +0100 Message-ID: <87ziwgf1b4.fsf@gnu.org> References: <87lha3rx04.fsf@T420.taylan> <87mvsgxpef.fsf@T420.taylan> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:47189) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aHaUw-0002Lo-OI for guix-devel@gnu.org; Fri, 08 Jan 2016 12:06:47 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1aHaUs-0004T0-E0 for guix-devel@gnu.org; Fri, 08 Jan 2016 12:06:46 -0500 In-Reply-To: <87mvsgxpef.fsf@T420.taylan> ("Taylan Ulrich \=\?utf-8\?Q\?\=5C\=22Bay\=C4\=B1rl\=C4\=B1\=2FKammer\=5C\=22\=22's\?\= message of "Fri, 08 Jan 2016 12:48:56 +0100") List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org To: Taylan Ulrich =?utf-8?Q?=22Bay=C4=B1rl=C4=B1?= =?utf-8?Q?=2FKammer=22?= Cc: guix-devel@gnu.org taylanbayirli@gmail.com (Taylan Ulrich "Bay=C4=B1rl=C4=B1/Kammer") skribis: > Here's an updated version that uses the same strategy as the one we > settled on for 'guix pull'. So, what speedup to you get compared to =E2=80=98make -jN=E2=80=99? > From 220a8caed6da22e349545899d5c51083bb3a8ac5 Mon Sep 17 00:00:00 2001 > From: =3D?UTF-8?q?Taylan=3D20Ulrich=3D20Bay=3DC4=3DB1rl=3DC4=3DB1/Kammer?= =3D > > Date: Thu, 5 Nov 2015 23:42:45 +0100 > Subject: [PATCH] build: Speed up .go compilation. > > * build-aux/compile-all.scm: New file. > * Makefile.am: Call build-aux/compile-all.scm to compile many .scm files > in a single process. Rather: * Makefile.am (%.go, make-go): New rules. Also, the new script must be added to =E2=80=98EXTRA_DIST=E2=80=99. > +++ b/build-aux/compile-all.scm Please add a copyright/license header. > +(define (file->module file) > + (map string->symbol > + (string-split (string-drop-right file 4) #\/))) Does this work with out-of-tree builds? > +(let* ((args (cdr (command-line))) > + (target (car args)) > + (files (cdr args))) Please use =E2=80=98match=E2=80=99 for the win! :-) > + (for-each > + (lambda (file) > + (let ((go (scm->go file))) > + (unless (and (file-exists? go) > + (file-mtime + (let ((module (file->module file))) > + (format #t " LOAD ~s~%" module) > + (resolve-interface module))))) > + files) Make the =E2=80=98lambda=E2=80=99 a named top-level procedure, for clarity.= Also add a reference to the evil bug that makes this hack necessary in the first place. Maybe it would be clearer to have: (define (file-needs-compilation? file) (file-mtimego file) file)) and then to: (let ((files (filter file-needs-compilation? files))) ;; =E2=80=A6 ) > + (with-target target > + (lambda () > + (let ((mutex (make-mutex))) > + (par-for-each > + (lambda (file) > + (let ((go (scm->go file))) > + (unless (and (file-exists? go) > + (file-mtime + (with-mutex mutex > + (format #t " GUILEC ~s~%" file) > + (force-output)) > + (compile-file file #:output-file go #:opts compile-option= s) > + (with-mutex mutex > + (format #t " WROTE ~s~%" go) > + (force-output))))) > + files))))) Ditto: name the lambda. I would use ~a instead of ~s, to match the current output, and remove the =E2=80=9CWROTE=E2=80=9D output. Could you send an updated patch? It would be awesome if you could check that =E2=80=98make distcheck=E2=80= =99 still passes, and also make sure things behave correctly when modifying just one file and running =E2=80=98make=E2=80=99, things like that. Thank you! Ludo=E2=80=99.