unofficial mirror of guix-patches@gnu.org 
 help / color / mirror / code / Atom feed
From: Leo Famulari <leo@famulari.name>
To: 26316@debbugs.gnu.org
Subject: bug#26316: Enable threaded compression of source tarballs
Date: Thu, 30 Mar 2017 18:17:15 -0400	[thread overview]
Message-ID: <20170330221715.GA10357@jasmine> (raw)


[-- Attachment #1.1: Type: text/plain, Size: 396 bytes --]

This patch enables multi-threaded xz compression in the repacking stage
of building source tarballs.

With it applied, I get a ~2.2x speedup building the source of
linux-libre with 4 threads, compared to 1 thread.

Specifically, with the upstream linux-libre tarball in the page cache,
`guix build -S linux-libre` took ~366 seconds using a single thread for
xz, and ~164 seconds using 4 threads.

[-- Attachment #1.2: 0001-packages-Enable-threaded-compression-of-source-tarba.patch --]
[-- Type: text/plain, Size: 1604 bytes --]

From 78547a09e50440c649c3d28d7691f32fdd47cc25 Mon Sep 17 00:00:00 2001
From: Leo Famulari <leo@famulari.name>
Date: Thu, 30 Mar 2017 03:10:48 -0400
Subject: [PATCH] packages: Enable threaded compression of source tarballs.

This provides a ~2x speedup when using 4 threads.

* guix/packages.scm (patch-and-repack)[build]: Invoke xz with
'--threads=0' when re-packing tarballs.
---
 guix/packages.scm | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/guix/packages.scm b/guix/packages.scm
index 4bc4b017f..30be5bf38 100644
--- a/guix/packages.scm
+++ b/guix/packages.scm
@@ -573,7 +573,12 @@ specifies modules in scope when evaluating SNIPPET."
                                                     #:fail-on-error? #t)))))
                         (zero? (apply system*
                                       (string-append #+tar "/bin/tar")
-                                      "cvfa" #$output
+                                      "cvf" #$output
+                                      ;; The bootstrap xz does not support
+                                      ;; threaded compression (introduced in
+                                      ;; 5.2.0), but it ignores the extra flag.
+                                      (string-append "--use-compress-program="
+                                                     #+xz "/bin/xz --threads=0")
                                       ;; avoid non-determinism in the archive
                                       "--mtime=@0"
                                       "--owner=root:0"
-- 
2.12.2


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

             reply	other threads:[~2017-03-30 22:18 UTC|newest]

Thread overview: 6+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-03-30 22:17 Leo Famulari [this message]
2017-03-31 18:15 ` bug#26316: Enable threaded compression of source tarballs Kei Kebreau
2017-03-31 23:04   ` Leo Famulari
2017-03-31 23:56     ` Kei Kebreau
2017-04-01  0:15       ` Leo Famulari
2017-04-15 22:35 ` Leo Famulari

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://guix.gnu.org/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20170330221715.GA10357@jasmine \
    --to=leo@famulari.name \
    --cc=26316@debbugs.gnu.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/guix.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).