Efraim Flashner writes: > On Wed, May 13, 2020 at 07:20:08PM +0100, Christopher Baines wrote: >> >> Christopher Baines writes: >> >> > It can take a little while to decompress some packages with large xz >> > compressed source tar files. xz includes support for parallelism, so enable >> > this using the parallel job count for the overall derivation. >> > >> > * guix/build/gnu-build-system.scm (unpack): Set XZ_OPT to pass the -T option >> > to xz to enable it to work in parallel if appropriate. >> > --- >> > guix/build/gnu-build-system.scm | 6 +++++- >> > 1 file changed, 5 insertions(+), 1 deletion(-) >> > >> > diff --git a/guix/build/gnu-build-system.scm b/guix/build/gnu-build-system.scm >> > index e5f3197b0..9d11e5b1e 100644 >> > --- a/guix/build/gnu-build-system.scm >> > +++ b/guix/build/gnu-build-system.scm >> > @@ -147,7 +147,7 @@ chance to be set." >> > locale (strerror (system-error-errno args))) >> > #t))) >> > >> > -(define* (unpack #:key source #:allow-other-keys) >> > +(define* (unpack #:key source parallel-build? #:allow-other-keys) >> > "Unpack SOURCE in the working directory, and change directory within the >> > source. When SOURCE is a directory, copy it in a sub-directory of the current >> > working directory." >> > @@ -161,6 +161,10 @@ working directory." >> > (copy-recursively source "." >> > #:keep-mtime? #t)) >> > (begin >> > + (when parallel-build? >> > + (setenv "XZ_OPT" >> > + (format #f "-T~d" (parallel-job-count)))) >> > + >> > (if (string-suffix? ".zip" source) >> > (invoke "unzip" source) >> > (invoke "tar" "xvf" source)) >> >> It's been a long long while, but now that core-updates has recently been >> merged, I'd like to try and take a look at this again. >> >> I think the consensus was that this will only help for xz compressed >> files where they have been compressed in parallel. I think it's still >> worth doing though, as some of the big xz files that need decompressing >> have been compressed in parallel, and this will speed up the builds when >> multiple cores are available. >> >> Thanks, >> >> Chris > > I thought the last time we looked into this we figured out that there > was a mistake in release notes or something and that parallel > decompression isn't actually supported. Hmm, I had a look to see if I could find some examples of where this would apply, but I couldn't find any xz archives that we use in Guix where it's been compressed in a way that allows multithreaded decompression... I'm pretty sure I had some examples before, but maybe somethings changed in the intervening year. Anyway, if I discover this again, I'll actually make a note of where it's applicable.