From: Christopher Baines <mail@cbaines.net>
To: 33643@debbugs.gnu.org
Cc: Efraim Flashner <efraim@flashner.co.il>,
Leo Famulari <leo@famulari.name>
Subject: [bug#33643] [PATCH] gnu-build-system: Enable xz to decompress in parallel.
Date: Wed, 13 May 2020 19:20:08 +0100 [thread overview]
Message-ID: <87k11fofxj.fsf@cbaines.net> (raw)
In-Reply-To: <20181206075615.4637-1-mail@cbaines.net>
[-- Attachment #1: Type: text/plain, Size: 1991 bytes --]
Christopher Baines <mail@cbaines.net> writes:
> It can take a little while to decompress some packages with large xz
> compressed source tar files. xz includes support for parallelism, so enable
> this using the parallel job count for the overall derivation.
>
> * guix/build/gnu-build-system.scm (unpack): Set XZ_OPT to pass the -T option
> to xz to enable it to work in parallel if appropriate.
> ---
> guix/build/gnu-build-system.scm | 6 +++++-
> 1 file changed, 5 insertions(+), 1 deletion(-)
>
> diff --git a/guix/build/gnu-build-system.scm b/guix/build/gnu-build-system.scm
> index e5f3197b0..9d11e5b1e 100644
> --- a/guix/build/gnu-build-system.scm
> +++ b/guix/build/gnu-build-system.scm
> @@ -147,7 +147,7 @@ chance to be set."
> locale (strerror (system-error-errno args)))
> #t)))
>
> -(define* (unpack #:key source #:allow-other-keys)
> +(define* (unpack #:key source parallel-build? #:allow-other-keys)
> "Unpack SOURCE in the working directory, and change directory within the
> source. When SOURCE is a directory, copy it in a sub-directory of the current
> working directory."
> @@ -161,6 +161,10 @@ working directory."
> (copy-recursively source "."
> #:keep-mtime? #t))
> (begin
> + (when parallel-build?
> + (setenv "XZ_OPT"
> + (format #f "-T~d" (parallel-job-count))))
> +
> (if (string-suffix? ".zip" source)
> (invoke "unzip" source)
> (invoke "tar" "xvf" source))
It's been a long long while, but now that core-updates has recently been
merged, I'd like to try and take a look at this again.
I think the consensus was that this will only help for xz compressed
files where they have been compressed in parallel. I think it's still
worth doing though, as some of the big xz files that need decompressing
have been compressed in parallel, and this will speed up the builds when
multiple cores are available.
Thanks,
Chris
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 962 bytes --]
next prev parent reply other threads:[~2020-05-13 18:24 UTC|newest]
Thread overview: 11+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-12-06 7:56 [bug#33643] [PATCH] gnu-build-system: Enable xz to decompress in parallel Christopher Baines
2018-12-06 8:08 ` Christopher Baines
2018-12-06 8:13 ` Leo Famulari
2018-12-06 19:38 ` Christopher Baines
2018-12-06 21:06 ` Leo Famulari
2018-12-09 14:32 ` Efraim Flashner
2018-12-10 16:24 ` Leo Famulari
2018-12-10 18:48 ` Efraim Flashner
2020-05-13 18:20 ` Christopher Baines [this message]
2020-05-13 19:07 ` Efraim Flashner
2020-05-14 7:37 ` bug#33643: " Christopher Baines
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://guix.gnu.org/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87k11fofxj.fsf@cbaines.net \
--to=mail@cbaines.net \
--cc=33643@debbugs.gnu.org \
--cc=efraim@flashner.co.il \
--cc=leo@famulari.name \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/guix.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).