bug#62394: [PATCH] make more packages tunable

bug#62394: [PATCH] make more packages tunable

[Dr. Arne Babenhauserheide]

[-- Attachment #1.1: Type: text/plain, Size: 345 bytes --]

The attached patch adds the (tunable? . #t) properties to additional
programs for which Clear Linux has an advantage over Ubuntu with the
same scheduler settings in a zen4 benchmark article
https://www.phoronix.com/review/zen4-clear-linux/2

This is not guaranteed to help, but the article indicates possible
gains for these packages of 5-10%.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.2: 0001-gnu-make-more-packages-tunable.patch --]
[-- Type: text/x-patch, Size: 7936 bytes --]

From 61ecf3cbf9a64133a254693c73a0abe4db03358a Mon Sep 17 00:00:00 2001
From: Arne Babenhauserheide <arne_bab@web.de>
Date: Wed, 22 Mar 2023 12:57:15 +0100
Subject: [PATCH] gnu: make more packages tunable

* gnu/packages/compression.scm (zlib): add (tunable? . #t) property
* gnu/packages/geo.scm (gdal): add (tunable? . #t) property
* gnu/packages/java.scm (openjdk19, openjdk14): add (tunable? . #t) property
* gnu/packages/maths.scm (hdf5-1.8, netcdf): add (tunable? . #t) property
* gnu/packages/protobuf.scm (protobuf): add (tunable? . #t) property
---
 gnu/packages/chromium.scm    | 1 +
 gnu/packages/compression.scm | 3 +++
 gnu/packages/geo.scm         | 1 +
 gnu/packages/gnuzilla.scm    | 1 +
 gnu/packages/java.scm        | 2 ++
 gnu/packages/maths.scm       | 2 ++
 gnu/packages/protobuf.scm    | 1 +
 gnu/packages/video.scm       | 6 +++++-
 8 files changed, 16 insertions(+), 1 deletion(-)

diff --git a/gnu/packages/chromium.scm b/gnu/packages/chromium.scm
index 94e22d8c21..cbccfd03b2 100644
--- a/gnu/packages/chromium.scm
+++ b/gnu/packages/chromium.scm
@@ -966,6 +966,7 @@ (define-public ungoogled-chromium
     (properties '((timeout . 144000)    ;40 hours
                   ;; The linking step may take more than an hour on some hardware.
                   (max-silent-time . 7200)
+                  (tunable? . #t) ;; advantages of Clear Linux over Ubuntu seen + perf: https://www.phoronix.com/review/zen4-clear-linux/2
                   (cpe-name . "chrome")))
 
     (home-page "https://github.com/Eloston/ungoogled-chromium")
diff --git a/gnu/packages/compression.scm b/gnu/packages/compression.scm
index e58fd65e49..aeb9f613de 100644
--- a/gnu/packages/compression.scm
+++ b/gnu/packages/compression.scm
@@ -118,6 +118,7 @@ (define-public zlib
         "18dighcs333gsvajvvgqp8l4cx7h1x7yx9gd5xacnk80spyykrf3"))))
     (build-system gnu-build-system)
     (outputs '("out" "static"))
+    (properties '((tunable? . #t)))
     (arguments
      `(#:make-flags
        ,(if (target-mingw?)
@@ -877,6 +878,7 @@ (define-public lz4
        (if (member (%current-system) (package-supported-systems valgrind))
          (list valgrind)
          '())))
+    (properties '((tunable? . #t))) ;; advantages of Clear Linux over Ubuntu seen + perf: https://www.phoronix.com/review/zen4-clear-linux/2
     (arguments
      `(;; Not designed for parallel testing.
        ;; See https://github.com/lz4/lz4/issues/957#issuecomment-737419821
@@ -1647,6 +1649,7 @@ (define-public zstd
     (outputs '("out"                    ;1.2MiB executables and documentation
                "lib"                    ;1.2MiB shared library and headers
                "static"))               ;1.2MiB static library
+    (properties '((tunable? . #t))) ;; advantages of Clear Linux over Ubuntu seen + perf: https://www.phoronix.com/review/zen4-clear-linux/2
     (arguments
      `(#:phases
        (modify-phases %standard-phases
diff --git a/gnu/packages/geo.scm b/gnu/packages/geo.scm
index 753ab558ad..659ac76afe 100644
--- a/gnu/packages/geo.scm
+++ b/gnu/packages/geo.scm
@@ -1068,6 +1068,7 @@ (define-public gdal
                        "frmts/zlib"
                        "ogr/ogrsf_frmts/geojson/libjson"))))))
     (build-system cmake-build-system)
+    (properties '((tunable? . #t)))
     (arguments
      `(#:tests? #f
        #:configure-flags
diff --git a/gnu/packages/gnuzilla.scm b/gnu/packages/gnuzilla.scm
index 253b8c285c..28df9ecd25 100644
--- a/gnu/packages/gnuzilla.scm
+++ b/gnu/packages/gnuzilla.scm
@@ -1104,6 +1104,7 @@ (define (runpaths-of-input label)
     (license license:mpl2.0)     ;and others, see toolkit/content/license.html
     (properties
      `((ftp-directory . "/gnu/gnuzilla")
+       (tunable? . #t) ;; advantages of Clear Linux over Ubuntu seen + perf: https://www.phoronix.com/review/zen4-clear-linux/2
        (cpe-name . "firefox_esr")
        (cpe-version . ,(first (string-split version #\-)))))))
 
diff --git a/gnu/packages/java.scm b/gnu/packages/java.scm
index de3a0003e5..51ee64edfb 100644
--- a/gnu/packages/java.scm
+++ b/gnu/packages/java.scm
@@ -1461,6 +1461,7 @@ (define-public openjdk15
              (snippet #f)
              (patches
               (search-patches "openjdk-15-xcursor-no-dynamic.patch"))))
+   (properties '((tunable? . #t)))
    (inputs
     (modify-inputs (package-inputs base)
       (append libxcursor)))             ;for our patch to work
@@ -1495,6 +1496,7 @@ (define-public openjdk18
 (define-public openjdk19
   (make-openjdk openjdk18 "19.0.1"
                 "0kyalb391znw6idmfn3dsx6c2mal1hl63f0bwa4mlnsxfl380bi1"
+   (properties '((tunable? . #t)))
    (arguments
     (substitute-keyword-arguments (package-arguments openjdk18)
       ((#:phases phases)
diff --git a/gnu/packages/maths.scm b/gnu/packages/maths.scm
index 238352f278..33bf95964f 100644
--- a/gnu/packages/maths.scm
+++ b/gnu/packages/maths.scm
@@ -1307,6 +1307,7 @@ (define-public hdf5-1.8
       (sha256
        (base32 "194ki2s5jrgl4czkvy5nc9nwjyapah0fj72l0gb0aysplp38i6v8"))
       (patches (search-patches "hdf5-config-date.patch"))))
+    (properties '((tunable? . #t)))
     (build-system gnu-build-system)
     (inputs
      (list zlib))
@@ -1886,6 +1887,7 @@ (define-public netcdf
        ("libxml2" ,libxml2)
        ("unzip" ,unzip)
        ("zlib" ,zlib)))
+    (properties '((tunable? . #t)))
     (arguments
      `(#:configure-flags '("--enable-doxygen"
                            "--enable-dot"
diff --git a/gnu/packages/protobuf.scm b/gnu/packages/protobuf.scm
index b580f8b2a7..c7d2ec06a0 100644
--- a/gnu/packages/protobuf.scm
+++ b/gnu/packages/protobuf.scm
@@ -110,6 +110,7 @@ (define-public protobuf
     (outputs (list "out"
                    "static"))           ; ~12 MiB of .a files
     (build-system cmake-build-system)
+    (properties '((tunable? . #t)))
     (arguments
      (list
       ;; TODO: Add the BUILD_SHARED_LIBS flag to cmake-build-system.
diff --git a/gnu/packages/video.scm b/gnu/packages/video.scm
index 73d0c1ca54..c2736be743 100644
--- a/gnu/packages/video.scm
+++ b/gnu/packages/video.scm
@@ -862,6 +862,7 @@ (define-public libaom
     (build-system cmake-build-system)
     (native-inputs
      (list perl pkg-config python)) ; to detect the version
+    (properties '((tunable? . #t))) ;; advantages of Clear Linux over Ubuntu seen + perf: https://www.phoronix.com/review/zen4-clear-linux/2
     (arguments
      `(#:tests? #f                      ; downloads many video clips
        #:configure-flags
@@ -1258,6 +1259,7 @@ (define-public x265
      (if (string-prefix? "i686" (%current-system))
          '()
          `(("nasm" ,nasm))))
+    (properties '((tunable? . #t))) ;; advantages of Clear Linux over Ubuntu seen + perf: https://www.phoronix.com/review/zen4-clear-linux/2
     (arguments
      `(#:tests? #f ; tests are skipped if ENABLE_ASSEMBLY is TRUE.
        #:configure-flags
@@ -1656,7 +1658,8 @@ (define-public ffmpeg-5
            texinfo
            speex
            yasm))
-    (arguments
+    (properties '((tunable? . #t))) ;; advantages of Clear Linux over Ubuntu seen + perf: https://www.phoronix.com/review/zen4-clear-linux/2
+   (arguments
      (list
       #:test-target "fate"
       #:configure-flags
@@ -5020,6 +5023,7 @@ (define-public dav1d
          (base32 "0jkvb5as7danpalzlwd0w1dc9i2vijvmf39z0j6fwqvialsgnnj5"))))
     (build-system meson-build-system)
     (native-inputs (list nasm))
+    (properties '((tunable? . #t))) ;; advantages of Clear Linux over Ubuntu seen + perf: https://www.phoronix.com/review/zen4-clear-linux/2
     (home-page "https://code.videolan.org/videolan/dav1d")
     (synopsis "AV1 decoder")
     (description "dav1d is a new AV1 cross-platform decoder, and focused on
-- 
2.39.2


[-- Attachment #1.3: Type: text/plain, Size: 101 bytes --]


Best wishes,
Arne
-- 
Unpolitisch sein
heißt politisch sein,
ohne es zu merken.
draketo.de

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 1125 bytes --]

bug#62394: [PATCH] make more packages tunable

[Liliana Marie Prikler]

reassign 62394 guix-patches
thanks

> * gnu/packages/compression.scm (zlib): add (tunable? . #t) property
Should probably be ... (zlib)[properties]: Add tunable?.
Likewise for the other packages.

> +    (properties '((tunable? . #t))) ;; advantages of Clear Linux
> over Ubuntu seen + perf:
> https://www.phoronix.com/review/zen4-clear-linux/2
>      (arguments
Don't exceed the line limit even in a comment.
Properties are supposed to be the *last* field of a package.

Cheers

[bug#62394] [PATCH] make more packages tunable

[Dr. Arne Babenhauserheide]

[-- Attachment #1.1: Type: text/plain, Size: 773 bytes --]

Thank you for your review!

Liliana Marie Prikler <liliana.prikler@gmail.com> writes:

> reassign 62394 guix-patches
> thanks
>
>> * gnu/packages/compression.scm (zlib): add (tunable? . #t) property
> Should probably be ... (zlib)[properties]: Add tunable?.
> Likewise for the other packages.

Thank you — fixed.

>> +    (properties '((tunable? . #t))) ;; advantages of Clear Linux
>> over Ubuntu seen + perf:
>> https://www.phoronix.com/review/zen4-clear-linux/2
>>      (arguments
> Don't exceed the line limit even in a comment.
> Properties are supposed to be the *last* field of a package.

Thank you — I moved the properties to the last field and moved the
explanation and description into the commit message.

A new patch is attached.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.2: 0001-gnu-make-more-packages-tunable.patch --]
[-- Type: text/x-patch, Size: 8877 bytes --]

From 4e498d911ee7948bb82bbdb3c58552c67bbc3832 Mon Sep 17 00:00:00 2001
From: Arne Babenhauserheide <arne_bab@web.de>
Date: Wed, 22 Mar 2023 12:57:15 +0100
Subject: [PATCH] gnu: make more packages tunable

* gnu/packages/compression.scm (zlib)[properties]: add tunable?.
* gnu/packages/geo.scm (gdal)[properties]: add tunable?.
* gnu/packages/java.scm (openjdk19, openjdk14)[properties]: add tunable?.
* gnu/packages/maths.scm (hdf5-1.8, netcdf)[properties]: add tunable?.
* gnu/packages/protobuf.scm (protobuf)[properties]: add tunable?.

advantages of Clear Linux over Ubuntu seen + perf:
https://www.phoronix.com/review/zen4-clear-linux/2
---
 gnu/packages/chromium.scm    | 17 ++++++++---------
 gnu/packages/compression.scm |  8 ++++++--
 gnu/packages/geo.scm         |  1 +
 gnu/packages/gnuzilla.scm    |  1 +
 gnu/packages/java.scm        |  2 ++
 gnu/packages/maths.scm       |  2 ++
 gnu/packages/protobuf.scm    |  1 +
 gnu/packages/video.scm       | 12 ++++++++----
 8 files changed, 29 insertions(+), 15 deletions(-)

diff --git a/gnu/packages/chromium.scm b/gnu/packages/chromium.scm
index 94e22d8c21..395834895d 100644
--- a/gnu/packages/chromium.scm
+++ b/gnu/packages/chromium.scm
@@ -960,14 +960,6 @@ (define-public ungoogled-chromium
             (variable "CHROMIUM_EXTENSION_DIRECTORY")
             (separator #f)              ;single entry
             (files '("share/chromium/extensions")))))
-
-    ;; Building Chromium takes ... a very long time.  On a single core, a busy
-    ;; mid-end x86 system may need more than 24 hours to complete the build.
-    (properties '((timeout . 144000)    ;40 hours
-                  ;; The linking step may take more than an hour on some hardware.
-                  (max-silent-time . 7200)
-                  (cpe-name . "chrome")))
-
     (home-page "https://github.com/Eloston/ungoogled-chromium")
     (description
      "Ungoogled-Chromium is the Chromium web browser, with some functionality
@@ -986,7 +978,14 @@ (define-public ungoogled-chromium
                    license:isc
                    (license:non-copyleft "chrome://credits"
                                          "See chrome://credits for more information.")
-                   license:lgpl2.1+))))
+                   license:lgpl2.1+))
+    ;; Building Chromium takes ... a very long time.  On a single core, a busy
+    ;; mid-end x86 system may need more than 24 hours to complete the build.
+    (properties '((timeout . 144000)    ;40 hours
+                  ;; The linking step may take more than an hour on some hardware.
+                  (max-silent-time . 7200)
+                  (tunable? . #t)
+                  (cpe-name . "chrome")))))
 
 (define-public ungoogled-chromium/wayland
   (package
diff --git a/gnu/packages/compression.scm b/gnu/packages/compression.scm
index e58fd65e49..cedc45ddd1 100644
--- a/gnu/packages/compression.scm
+++ b/gnu/packages/compression.scm
@@ -118,6 +118,7 @@ (define-public zlib
         "18dighcs333gsvajvvgqp8l4cx7h1x7yx9gd5xacnk80spyykrf3"))))
     (build-system gnu-build-system)
     (outputs '("out" "static"))
+    (properties '((tunable? . #t)))
     (arguments
      `(#:make-flags
        ,(if (target-mingw?)
@@ -911,7 +912,8 @@ (define-public lz4
 time for compression ratio.")
     ;; The libraries (lz4, lz4hc, and xxhash) are BSD licenced. The command
     ;; line interface programs (lz4, fullbench, fuzzer, datagen) are GPL2+.
-    (license (list license:bsd-2 license:gpl2+))))
+    (license (list license:bsd-2 license:gpl2+))
+    (properties '((tunable? . #t)))))
 
 (define-public squashfs-tools
   (package
@@ -1719,7 +1721,9 @@ (define-public zstd
                    license:gpl3+         ; tests/gzip/*.sh
                    license:expat         ; lib/dictBuilder/divsufsort.[ch]
                    license:public-domain ; zlibWrapper/examples/fitblk*
-                   license:zlib))))      ; zlibWrapper/{gz*.c,gzguts.h}
+                   license:zlib))      ; zlibWrapper/{gz*.c,gzguts.h}
+    (properties '((tunable? . #t)))))
+
 
 (define-public pzstd
   (package/inherit zstd
diff --git a/gnu/packages/geo.scm b/gnu/packages/geo.scm
index 753ab558ad..659ac76afe 100644
--- a/gnu/packages/geo.scm
+++ b/gnu/packages/geo.scm
@@ -1068,6 +1068,7 @@ (define-public gdal
                        "frmts/zlib"
                        "ogr/ogrsf_frmts/geojson/libjson"))))))
     (build-system cmake-build-system)
+    (properties '((tunable? . #t)))
     (arguments
      `(#:tests? #f
        #:configure-flags
diff --git a/gnu/packages/gnuzilla.scm b/gnu/packages/gnuzilla.scm
index 253b8c285c..5ad4ca68d3 100644
--- a/gnu/packages/gnuzilla.scm
+++ b/gnu/packages/gnuzilla.scm
@@ -1104,6 +1104,7 @@ (define (runpaths-of-input label)
     (license license:mpl2.0)     ;and others, see toolkit/content/license.html
     (properties
      `((ftp-directory . "/gnu/gnuzilla")
+       (tunable? . #t)
        (cpe-name . "firefox_esr")
        (cpe-version . ,(first (string-split version #\-)))))))
 
diff --git a/gnu/packages/java.scm b/gnu/packages/java.scm
index de3a0003e5..51ee64edfb 100644
--- a/gnu/packages/java.scm
+++ b/gnu/packages/java.scm
@@ -1461,6 +1461,7 @@ (define-public openjdk15
              (snippet #f)
              (patches
               (search-patches "openjdk-15-xcursor-no-dynamic.patch"))))
+   (properties '((tunable? . #t)))
    (inputs
     (modify-inputs (package-inputs base)
       (append libxcursor)))             ;for our patch to work
@@ -1495,6 +1496,7 @@ (define-public openjdk18
 (define-public openjdk19
   (make-openjdk openjdk18 "19.0.1"
                 "0kyalb391znw6idmfn3dsx6c2mal1hl63f0bwa4mlnsxfl380bi1"
+   (properties '((tunable? . #t)))
    (arguments
     (substitute-keyword-arguments (package-arguments openjdk18)
       ((#:phases phases)
diff --git a/gnu/packages/maths.scm b/gnu/packages/maths.scm
index 238352f278..33bf95964f 100644
--- a/gnu/packages/maths.scm
+++ b/gnu/packages/maths.scm
@@ -1307,6 +1307,7 @@ (define-public hdf5-1.8
       (sha256
        (base32 "194ki2s5jrgl4czkvy5nc9nwjyapah0fj72l0gb0aysplp38i6v8"))
       (patches (search-patches "hdf5-config-date.patch"))))
+    (properties '((tunable? . #t)))
     (build-system gnu-build-system)
     (inputs
      (list zlib))
@@ -1886,6 +1887,7 @@ (define-public netcdf
        ("libxml2" ,libxml2)
        ("unzip" ,unzip)
        ("zlib" ,zlib)))
+    (properties '((tunable? . #t)))
     (arguments
      `(#:configure-flags '("--enable-doxygen"
                            "--enable-dot"
diff --git a/gnu/packages/protobuf.scm b/gnu/packages/protobuf.scm
index b580f8b2a7..c7d2ec06a0 100644
--- a/gnu/packages/protobuf.scm
+++ b/gnu/packages/protobuf.scm
@@ -110,6 +110,7 @@ (define-public protobuf
     (outputs (list "out"
                    "static"))           ; ~12 MiB of .a files
     (build-system cmake-build-system)
+    (properties '((tunable? . #t)))
     (arguments
      (list
       ;; TODO: Add the BUILD_SHARED_LIBS flag to cmake-build-system.
diff --git a/gnu/packages/video.scm b/gnu/packages/video.scm
index 73d0c1ca54..532fdfa65e 100644
--- a/gnu/packages/video.scm
+++ b/gnu/packages/video.scm
@@ -882,7 +882,8 @@ (define-public libaom
     (synopsis "AV1 video codec")
     (description "Libaom is the reference implementation of AV1.  It includes a
 shared library and encoder and decoder command-line executables.")
-    (license license:bsd-2)))
+    (license license:bsd-2)
+    (properties '((tunable? . #t)))))
 
 (define-public libmpeg2
   (package
@@ -1346,7 +1347,8 @@ (define-public x265
     (synopsis "Library for encoding h.265/HEVC video streams")
     (description "x265 is a H.265 / HEVC video encoder application library,
 designed to encode video or images into an H.265 / HEVC encoded bitstream.")
-    (license license:gpl2+)))
+    (license license:gpl2+)
+    (properties '((tunable? . #t)))))
 
 (define-public libass
   (package
@@ -1790,7 +1792,8 @@ (define-public ffmpeg-5
     (description "FFmpeg is a complete, cross-platform solution to record,
 convert and stream audio and video.  It includes the libavcodec
 audio/video codec library.")
-    (license license:gpl2+)))
+    (license license:gpl2+)
+    (properties '((tunable? . #t)))))
 
 (define-public ffmpeg-4
   (package
@@ -5024,7 +5027,8 @@ (define-public dav1d
     (synopsis "AV1 decoder")
     (description "dav1d is a new AV1 cross-platform decoder, and focused on
 speed and correctness.")
-    (license license:bsd-2)))
+    (license license:bsd-2)
+    (properties '((tunable? . #t)))))
 
 (define-public wlstream
   (let ((commit "182076a94562b128c3a97ecc53cc68905ea86838")
-- 
2.39.2


[-- Attachment #1.3: Type: text/plain, Size: 103 bytes --]



Best wishes,
Arne
-- 
Unpolitisch sein
heißt politisch sein,
ohne es zu merken.
draketo.de

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 1125 bytes --]

[bug#62394] [PATCH] make more packages tunable

[Ludovic Courtès]

Hi,

"Dr. Arne Babenhauserheide" <arne_bab@web.de> skribis:

> From 4e498d911ee7948bb82bbdb3c58552c67bbc3832 Mon Sep 17 00:00:00 2001
> From: Arne Babenhauserheide <arne_bab@web.de>
> Date: Wed, 22 Mar 2023 12:57:15 +0100
> Subject: [PATCH] gnu: make more packages tunable
>
> * gnu/packages/compression.scm (zlib)[properties]: add tunable?.
> * gnu/packages/geo.scm (gdal)[properties]: add tunable?.
> * gnu/packages/java.scm (openjdk19, openjdk14)[properties]: add tunable?.
> * gnu/packages/maths.scm (hdf5-1.8, netcdf)[properties]: add tunable?.
> * gnu/packages/protobuf.scm (protobuf)[properties]: add tunable?.
>
> advantages of Clear Linux over Ubuntu seen + perf:
> https://www.phoronix.com/review/zen4-clear-linux/2

I’m reluctant to marking things like Chromium or OpenJDK as tunable
because they take ages to build: it’s more work for ci.guix (which
builds a few tuned variants of each tunable package), and possibly leads
to local recompilations for users when substitutes for their
micro-architecture are unavailable.

It’s also not immediately obvious to me why these two packages as well
as zlib would benefit from micro-architecture-specific optimizations.

Thoughts?

Ludo’.

[bug#62394] [PATCH] make more packages tunable

[Simon Tournier]

Hi,

On sam., 01 avril 2023 at 10:19, Ludovic Courtès <ludo@gnu.org> wrote:

> I’m reluctant to marking things like Chromium or OpenJDK as tunable
> because they take ages to build: it’s more work for ci.guix (which
> builds a few tuned variants of each tunable package), and possibly leads
> to local recompilations for users when substitutes for their
> micro-architecture are unavailable.
>
> It’s also not immediately obvious to me why these two packages as well
> as zlib would benefit from micro-architecture-specific optimizations.

What is the status of this patch?

Well, I also have mixed feelings about the tuning of these packages.


Cheers,
simon