From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:470:142:3::10]:53200) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hm3Kn-00045c-Lf for guix-patches@gnu.org; Fri, 12 Jul 2019 17:44:06 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hm3Kl-00069p-DE for guix-patches@gnu.org; Fri, 12 Jul 2019 17:44:05 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:60279) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1hm3Kk-000696-BW for guix-patches@gnu.org; Fri, 12 Jul 2019 17:44:03 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1hm3Kk-0006PO-7I for guix-patches@gnu.org; Fri, 12 Jul 2019 17:44:02 -0400 Subject: [bug#36630] [PATCH] guix: parallelize building the manual-database Resent-Message-ID: Received: from eggs.gnu.org ([2001:470:142:3::10]:52943) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hm3Jz-0003qD-Tr for guix-patches@gnu.org; Fri, 12 Jul 2019 17:43:16 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hm3Jy-0005gA-Sn for guix-patches@gnu.org; Fri, 12 Jul 2019 17:43:15 -0400 Received: from mout.web.de ([217.72.192.78]:33057) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1hm3Jw-0005dr-Po for guix-patches@gnu.org; Fri, 12 Jul 2019 17:43:13 -0400 From: Arne Babenhauserheide Date: Fri, 12 Jul 2019 23:42:45 +0200 Message-Id: <20190712214245.23857-1-arne_bab@web.de> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: arne_bab@web.de Errors-To: guix-patches-bounces+kyle=kyleam.com@gnu.org Sender: "Guix-patches" To: 36630@debbugs.gnu.org Cc: Arne Babenhauserheide * guix/profiles.scm (manual-database): par-map over the entries. This distributes the load roughly equally over all cores and avoids blocking on I/O. The order of the entries stays the same since write-mandb-database s= orts them. =2D-- guix/profiles.scm | 20 ++++++++++++-------- 1 file changed, 12 insertions(+), 8 deletions(-) diff --git a/guix/profiles.scm b/guix/profiles.scm index f5c863945c..374f0f8a90 100644 =2D-- a/guix/profiles.scm +++ b/guix/profiles.scm @@ -1312,15 +1312,11 @@ the entries in MANIFEST." #~(begin (use-modules (guix man-db) (guix build utils) + (ice-9 threads) (srfi srfi-1) (srfi srfi-19)) - (define (compute-entries) - ;; This is the most expensive part (I/O and CPU, due to - ;; decompression), so report progress as we traverse INPUTS= . - (let* ((inputs '#$(manifest-inputs manifest)) - (total (length inputs))) - (append-map (lambda (directory count) + (define (compute-entry directory count total) (format #t "\r[~3d/~3d] building list of \ man-db entries..." count total) @@ -1330,8 +1326,16 @@ man-db entries..." (if (directory-exists? man) (mandb-entries man) '()))) - inputs - (iota total 1)))) + + (define (compute-entries) + ;; This is the most expensive part (I/O and CPU, due to + ;; decompression), so report progress as we traverse INPUTS= . + (let* ((inputs '#$(manifest-inputs manifest)) + (total (length inputs))) + (apply append (par-map compute-entry + inputs + (iota total 1) + (make-list total total))))) (define man-directory (string-append #$output "/share/man")) =2D- 2.22.0