From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52835) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ctiON-0003uB-HL for guix-patches@gnu.org; Thu, 30 Mar 2017 18:18:08 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ctiOK-0002cI-DO for guix-patches@gnu.org; Thu, 30 Mar 2017 18:18:07 -0400 Received: from debbugs.gnu.org ([208.118.235.43]:55070) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1ctiOK-0002c4-8x for guix-patches@gnu.org; Thu, 30 Mar 2017 18:18:04 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1ctiOI-0000Wf-84 for guix-patches@gnu.org; Thu, 30 Mar 2017 18:18:03 -0400 Subject: bug#26316: Enable threaded compression of source tarballs Resent-Message-ID: Received: from eggs.gnu.org ([2001:4830:134:3::10]:52547) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ctiNe-0003r5-KY for guix-patches@gnu.org; Thu, 30 Mar 2017 18:17:23 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ctiNb-0002FE-HA for guix-patches@gnu.org; Thu, 30 Mar 2017 18:17:22 -0400 Received: from out1-smtp.messagingengine.com ([66.111.4.25]:49820) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ctiNb-0002EC-6C for guix-patches@gnu.org; Thu, 30 Mar 2017 18:17:19 -0400 Received: from localhost (c-73-165-108-70.hsd1.pa.comcast.net [73.165.108.70]) by mail.messagingengine.com (Postfix) with ESMTPA id 708AC7E70C for ; Thu, 30 Mar 2017 18:17:16 -0400 (EDT) Date: Thu, 30 Mar 2017 18:17:15 -0400 From: Leo Famulari Message-ID: <20170330221715.GA10357@jasmine> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="i0/AhcQY5QxfSsSZ" Content-Disposition: inline List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+kyle=kyleam.com@gnu.org Sender: "Guix-patches" To: 26316@debbugs.gnu.org --i0/AhcQY5QxfSsSZ Content-Type: multipart/mixed; boundary="NzB8fVQJ5HfG6fxh" Content-Disposition: inline --NzB8fVQJ5HfG6fxh Content-Type: text/plain; charset=us-ascii Content-Disposition: inline This patch enables multi-threaded xz compression in the repacking stage of building source tarballs. With it applied, I get a ~2.2x speedup building the source of linux-libre with 4 threads, compared to 1 thread. Specifically, with the upstream linux-libre tarball in the page cache, `guix build -S linux-libre` took ~366 seconds using a single thread for xz, and ~164 seconds using 4 threads. --NzB8fVQJ5HfG6fxh Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="0001-packages-Enable-threaded-compression-of-source-tarba.patch" Content-Transfer-Encoding: quoted-printable =46rom 78547a09e50440c649c3d28d7691f32fdd47cc25 Mon Sep 17 00:00:00 2001 =46rom: Leo Famulari Date: Thu, 30 Mar 2017 03:10:48 -0400 Subject: [PATCH] packages: Enable threaded compression of source tarballs. This provides a ~2x speedup when using 4 threads. * guix/packages.scm (patch-and-repack)[build]: Invoke xz with '--threads=3D0' when re-packing tarballs. --- guix/packages.scm | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/guix/packages.scm b/guix/packages.scm index 4bc4b017f..30be5bf38 100644 --- a/guix/packages.scm +++ b/guix/packages.scm @@ -573,7 +573,12 @@ specifies modules in scope when evaluating SNIPPET." #:fail-on-error? #t)))= )) (zero? (apply system* (string-append #+tar "/bin/tar") - "cvfa" #$output + "cvf" #$output + ;; The bootstrap xz does not support + ;; threaded compression (introduced = in + ;; 5.2.0), but it ignores the extra = flag. + (string-append "--use-compress-progr= am=3D" + #+xz "/bin/xz --threa= ds=3D0") ;; avoid non-determinism in the arch= ive "--mtime=3D@0" "--owner=3Droot:0" --=20 2.12.2 --NzB8fVQJ5HfG6fxh-- --i0/AhcQY5QxfSsSZ Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAABCAAdFiEEsFFZSPHn08G5gDigJkb6MLrKfwgFAljdg+cACgkQJkb6MLrK fwjeVRAA0icp4AHyeaJKf4U1oovWmJHzY0rF5Qodvm8aA9z2PVWfe/4+L5pFGsCX KxK+sDZtYmW0ZokJZp1PfEUVc+zgzeJbc0LSOzXl0SYFQ/5ExHS1FSLxzlyu0LOg Fo9dxq18eT1HXOScL+FSq8h0+01DFP2ItNnbCRSPlUuvRjUhj/XfNzV/Kmmq9FyM dxFDO1TjDqY5C/1z2++zDUBM2Ajv61770a6LCvc8yWhejjsTmixxAmH6uOPHhoZ0 N5g8i85Cf/+FSCBhSZt9h6Ux57Wb/PO8ETC7rPR3afyyZ/i8p3vL1X8+WTddEras QV/Fi+xPYIMitK/MN3lTfhQEQRpLv9Z3gIlOLu2gU177S0YzIB4fjafncgQ6CS/8 XWXTk6kHGMWDg5U+n4aSLi51GUMOM9ZtNn6LtB3IOTHkm9Z30Bk0XzFwxu80QA2Z Ysgt6s696gYnvwz/M8K4mqQHopwSDdt1UVMQqyoh9Xat1Rirait2Rfz+2hHqpuzh 4V1YJWrT3ewFjXUvdZEtvODZzoVUXceKJpqTBC2FIb4z7mVdB+dgJN4vahp+hhJT gJfIfUNJlG8QaoWIofGoO68imhhEmQW8ho+6aXMTVArTidrF84lXq35xqZoxyKfE SKytwHeRZFjkv9ZmEdqsBrkrDvQkVb+vMoEM8AvVjR9CEn1Hlsk= =QzSR -----END PGP SIGNATURE----- --i0/AhcQY5QxfSsSZ--