[bug#32196] [PATCH core-updates] packages, scripts, utils: Disable threaded xz compression.

[bug#32196] [PATCH core-updates] packages, scripts, utils: Disable threaded xz compression.

[Marius Bakke]

Archives produces by "xz --threads" are not reproducible: they depend on the
number of cores used for compressing.  See <https://bugs.gnu.org/31015>.

* guix/packages.scm (patch-and-repack): Explicitly use 1 thread for compression.
* guix/scripts/pack.scm (%compressors, bootstrap-xz): Likewise.
* guix/utils.scm (decompressed-port, compressed-port, compressed-output-port):
Likewise.
---
 guix/packages.scm     | 2 +-
 guix/scripts/pack.scm | 4 ++--
 guix/utils.scm        | 6 +++---
 3 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/guix/packages.scm b/guix/packages.scm
index c762fa7c3..ef88e0c59 100644
--- a/guix/packages.scm
+++ b/guix/packages.scm
@@ -633,7 +633,7 @@ specifies modules in scope when evaluating SNIPPET."
                      ;; threaded compression (introduced in
                      ;; 5.2.0), but it ignores the extra flag.
                      (string-append "--use-compress-program="
-                                    #+xz "/bin/xz --threads=0")
+                                    #+xz "/bin/xz --threads=1")
                      ;; avoid non-determinism in the archive
                      "--mtime=@0"
                      "--owner=root:0"
diff --git a/guix/scripts/pack.scm b/guix/scripts/pack.scm
index 7f087a3a3..e0d6f1dee 100644
--- a/guix/scripts/pack.scm
+++ b/guix/scripts/pack.scm
@@ -69,7 +69,7 @@
         (compressor "lzip"  ".lz"
                     #~(#+(file-append lzip "/bin/lzip") "-9"))
         (compressor "xz"    ".xz"
-                    #~(#+(file-append xz "/bin/xz") "-e -T0"))
+                    #~(#+(file-append xz "/bin/xz") "-e -T1"))
         (compressor "bzip2" ".bz2"
                     #~(#+(file-append bzip2 "/bin/bzip2") "-9"))
         (compressor "none" "" #f)))
@@ -77,7 +77,7 @@
 ;; This one is only for use in this module, so don't put it in %compressors.
 (define bootstrap-xz
   (compressor "bootstrap-xz" ".xz"
-              #~(#+(file-append %bootstrap-coreutils&co "/bin/xz") "-e -T0")))
+              #~(#+(file-append %bootstrap-coreutils&co "/bin/xz") "-e -T1")))
 
 (define (lookup-compressor name)
   "Return the compressor object called NAME.  Error out if it could not be
diff --git a/guix/utils.scm b/guix/utils.scm
index a5de9605e..84176a22e 100644
--- a/guix/utils.scm
+++ b/guix/utils.scm
@@ -175,7 +175,7 @@ a symbol such as 'xz."
   (match compression
     ((or #f 'none) (values input '()))
     ('bzip2        (filtered-port `(,%bzip2 "-dc") input))
-    ('xz           (filtered-port `(,%xz "-dc" "-T0") input))
+    ('xz           (filtered-port `(,%xz "-dc" "-T1") input))
     ('gzip         (filtered-port `(,%gzip "-dc") input))
     (else          (error "unsupported compression scheme" compression))))
 
@@ -185,7 +185,7 @@ a symbol such as 'xz."
   (match compression
     ((or #f 'none) (values input '()))
     ('bzip2        (filtered-port `(,%bzip2 "-c") input))
-    ('xz           (filtered-port `(,%xz "-c" "-T0") input))
+    ('xz           (filtered-port `(,%xz "-c" "-T1") input))
     ('gzip         (filtered-port `(,%gzip "-c") input))
     (else          (error "unsupported compression scheme" compression))))
 
@@ -242,7 +242,7 @@ program--e.g., '(\"--fast\")."
   (match compression
     ((or #f 'none) (values output '()))
     ('bzip2        (filtered-output-port `(,%bzip2 "-c" ,@options) output))
-    ('xz           (filtered-output-port `(,%xz "-c" "-T0" ,@options) output))
+    ('xz           (filtered-output-port `(,%xz "-c" "-T1" ,@options) output))
     ('gzip         (filtered-output-port `(,%gzip "-c" ,@options) output))
     (else          (error "unsupported compression scheme" compression))))
 
-- 
2.18.0

[bug#32196] [PATCH core-updates] packages, scripts, utils: Disable threaded xz compression.

[Ludovic Courtès]

Hi Marius,

Marius Bakke <mbakke@fastmail.com> skribis:

> Archives produces by "xz --threads" are not reproducible: they depend on the
> number of cores used for compressing.  See <https://bugs.gnu.org/31015>.
>
> * guix/packages.scm (patch-and-repack): Explicitly use 1 thread for compression.
> * guix/scripts/pack.scm (%compressors, bootstrap-xz): Likewise.
> * guix/utils.scm (decompressed-port, compressed-port, compressed-output-port):
> Likewise.

I hadn’t noticed -T0 had made it to ‘master’.

I’d suggest removing 63102406f22412bb922de5549deb89d3594a38c0 on master
(no rebuild needed), with a reference to #31015 in the commit log.  And
then similarly reverting c8a3dea847bb9f87fa1876d0c6c3356d6226f121 on
‘core-updates’.

Sounds good?

If we’re concerned about speed, perhaps we should switch to lzip, which
I think has better behavior.

Thanks,
Ludo’.

bug#32196: [PATCH core-updates] packages, scripts, utils: Disable threaded xz compression.

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 1240 bytes --]

ludo@gnu.org (Ludovic Courtès) writes:

> Hi Marius,
>
> Marius Bakke <mbakke@fastmail.com> skribis:
>
>> Archives produces by "xz --threads" are not reproducible: they depend on the
>> number of cores used for compressing.  See <https://bugs.gnu.org/31015>.
>>
>> * guix/packages.scm (patch-and-repack): Explicitly use 1 thread for compression.
>> * guix/scripts/pack.scm (%compressors, bootstrap-xz): Likewise.
>> * guix/utils.scm (decompressed-port, compressed-port, compressed-output-port):
>> Likewise.
>
> I hadn’t noticed -T0 had made it to ‘master’.
>
> I’d suggest removing 63102406f22412bb922de5549deb89d3594a38c0 on master
> (no rebuild needed), with a reference to #31015 in the commit log.  And
> then similarly reverting c8a3dea847bb9f87fa1876d0c6c3356d6226f121 on
> ‘core-updates’.
>
> Sounds good?

Done in commits e9be2c5409f37173d70b202aa06752e3814ccdc2 and
3e95125e9bd0676d4a9add9105217ad3eaef3ff0.

> If we’re concerned about speed, perhaps we should switch to lzip, which
> I think has better behavior.

That sounds great.  lzip has other benefits too, such as the ability to
recover from bit flips.  Recommended reading:
<https://www.nongnu.org/lzip/xz_inadequate.html>.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]