[bug#47251] [core-updates] [patch] gnu: python: Enable optimizations.

[Julien Lepiller <julien@lepiller.eu>]

[-- Attachment #1: Type: text/plain, Size: 1089 bytes --]

Hi Guix!

This is the second patch I created in my exploration of our Python
package. Here, I noticed that our Python was very slow compared to
other distros and I tried to find which options they used. I found some
hints from Fedora. In this patch, I enable a few options and add a C/LD
flag to make python run faster.

I also attached a report of timing running the benchmark from
pyperformance on my Fedora machine. For comparison, I have also added
the results for Fedora's python3, which is somewhere in between.

I noticed that this increases the size of the python package by 20MB,
which is due entirely to --with-lto option, which also accounts for
half of the speedup. Overall, this python is almost twice as fast on
some benchmarks, and always faster on all of them. It's about 1.5 times
faster on average.

Strangely, the python package from Alpine is twice as small as this
package, and they use the --with-lto option. I wonder how they achieve
that.

Note that I modified python-2, as its flags are inherited by python-3
(and also the bootstrap python from commencement.scm).

[-- Attachment #2: 0002-gnu-python-Enable-optimizations.patch --]
[-- Type: text/x-patch, Size: 1767 bytes --]

From c72e34aa0eb0db589557e047c5db0128273d5d12 Mon Sep 17 00:00:00 2001
From: Julien Lepiller <julien@lepiller.eu>
Date: Thu, 18 Mar 2021 23:41:40 +0100
Subject: [PATCH 2/2] gnu: python: Enable optimizations.

As a result, python execution time of scripts is decreased by 6 to 40%.

gnu/packages/python (python-2.7)[arguments]: Add configure and make flags to
optimize the resulting python.
---
 gnu/packages/python.scm | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/gnu/packages/python.scm b/gnu/packages/python.scm
index da9859b11a..8de0ca343c 100644
--- a/gnu/packages/python.scm
+++ b/gnu/packages/python.scm
@@ -183,7 +183,10 @@
              "--with-system-expat"      ;for XML support
              "--with-system-ffi"        ;build ctypes
              "--with-ensurepip=install" ;install pip and setuptools
+             "--with-computed-gotos"    ;main interpreter loop optimization
+             "--with-lto"               ;increase size by 20MB, but 15% speedup
              "--enable-unicode=ucs4"
+             "--enable-optimizations"
 
              ;; Prevent the installed _sysconfigdata.py from retaining a reference
              ;; to coreutils.
@@ -197,8 +200,10 @@
                      "ac_cv_file__dev_ptmx=no"
                      "ac_cv_file__dev_ptc=no")
                    '())
+             "CFLAGS=-fno-semantic-interposition"
              (string-append "LDFLAGS=-Wl,-rpath="
-                            (assoc-ref %outputs "out") "/lib"))
+                            (assoc-ref %outputs "out") "/lib"
+                            " -fno-semantic-interposition"))
        ;; With no -j argument tests use all available cpus, so provide one.
        #:make-flags
        (list (string-append
-- 
2.30.0


[-- Attachment #3: python-report --]
[-- Type: application/octet-stream, Size: 5498 bytes --]

                        | optimized c-u        | current c-u          | fedora (3.8.2)
------------------------+----------------------+----------------------+-------------------
2to3                    | 511 ms +- 25 ms      | 617 ms +- 21 ms      | 578 ms +- 23 ms
chameleon               | 13.8 ms +- 0.3 ms    | 18.0 ms +- 0.5 ms    | 16.1 ms +- 0.6 ms
chaos                   | 162 ms +- 4 ms       | 232 ms +- 3 ms       | 204 ms +- 4 ms
crypto_pyaes            | 162 ms +- 3 ms       | 205 ms +- 5 ms       | 218 ms +- 8 ms
deltablue               | 10.8 ms +- 0.3 ms    | 15.3 ms +- 0.5 ms    | 12.8 ms +- 0.3 ms
django_template         | 75.8 ms +- 1.0 ms    | 104 ms +- 3 ms       | 88.2 ms +- 1.8 ms
dulwich_log             | 111 ms +- 4 ms       | 133 ms +- 2 ms       | 121 ms +- 7 ms
fannkuch                | 695 ms +- 9 ms       | 822 ms +- 8 ms       | 874 ms +- 16 ms
float                   | 171 ms +- 8 ms       | 209 ms +- 4 ms       | 217 ms +- 8 ms
genshi_text             | 43.8 ms +- 0.8 ms    | 55.4 ms +- 0.8 ms    | 53.5 ms +- 2.6 ms
genshi_xml              | 93.8 ms +- 1.5 ms    | 113 ms +- 1 ms       | 115 ms +- 6 ms
go                      | 372 ms +- 7 ms       | 497 ms +- 4 ms       | 453 ms +- 8 ms
hexiom                  | 14.4 ms +- 0.5 ms    | 18.9 ms +- 0.2 ms    | 18.2 ms +- 1.0 ms
json_dumps              | 19.2 ms +- 0.5 ms    | 23.0 ms +- 0.3 ms    | 22.7 ms +- 1.0 ms
json_loads              | 34.9 us +- 0.7 us    | 43.5 us +- 0.7 us    | 48.1 us +- 2.2 us
logging_format          | 14.6 us +- 0.8 us    | 20.0 us +- 0.5 us    | 16.9 us +- 1.0 us
logging_silent          | 290 ns +- 13 ns      | 399 ns +- 10 ns      | 343 ns +- 10 ns
logging_simple          | 13.1 us +- 0.3 us    | 18.4 us +- 0.7 us    | 15.5 us +- 0.6 us
mako                    | 24.0 ms +- 0.4 ms    | 31.2 ms +- 0.7 ms    | 29.6 ms +- 1.2 ms
meteor_contest          | 157 ms +- 4 ms       | 179 ms +- 2 ms       | 189 ms +- 9 ms
nbody                   | 184 ms +- 4 ms       | 251 ms +- 4 ms       | 229 ms +- 4 ms
nqueens                 | 145 ms +- 3 ms       | 180 ms +- 5 ms       | 184 ms +- 5 ms
pathlib                 | 28.4 ms +- 0.7 ms    | 36.0 ms +- 1.3 ms    | 35.2 ms +- 2.4 ms
pickle                  | 14.6 us +- 0.7 us    | 17.3 us +- 0.5 us    | 16.4 us +- 0.6 us
pickle_dict             | 35.1 us +- 0.4 us    | 42.3 us +- 0.6 us    | 42.8 us +- 2.0 us
pickle_list             | 5.42 us +- 0.19 us   | 6.24 us +- 0.10 us   | 6.34 us +- 0.35 us
pickle_pure_python      | 699 us +- 15 us      | 941 us +- 19 us      | 800 us +- 16 us
pidigits                | 252 ms +- 3 ms       | 267 ms +- 3 ms       | 282 ms +- 4 ms
pyflate                 | 1.01 sec +- 0.02 sec | 1.37 sec +- 0.07 sec | 1.23 sec +- 0.02 sec
python_startup          | 16.9 ms +- 0.6 ms    | 19.6 ms +- 0.4 ms    | 17.6 ms +- 0.5 ms
python_startup_no_site  | 11.2 ms +- 0.6 ms    | 12.7 ms +- 0.2 ms    | 11.5 ms +- 0.3 ms
raytrace                | 733 ms +- 11 ms      | 1.22 sec +- 0.01 sec | 886 ms +- 10 ms
regex_compile           | 257 ms +- 2 ms       | 353 ms +- 3 ms       | 300 ms +- 4 ms
regex_dna               | 268 ms +- 4 ms       | 296 ms +- 7 ms       | 293 ms +- 15 ms
regex_effbot            | 4.35 ms +- 0.07 ms   | 5.13 ms +- 0.39 ms   | 4.93 ms +- 0.23 ms
regex_v8                | 35.9 ms +- 0.7 ms    | 43.3 ms +- 2.0 ms    | 37.5 ms +- 0.4 ms
richards                | 100 ms +- 1 ms       | 168 ms +- 9 ms       | 124 ms +- 4 ms
scimark_fft             | 538 ms +- 6 ms       | 773 ms +- 8 ms       | 710 ms +- 33 ms
scimark_lu              | 235 ms +- 5 ms       | 340 ms +- 3 ms       | 288 ms +- 5 ms
scimark_monte_carlo     | 154 ms +- 2 ms       | 231 ms +- 4 ms       | 191 ms +- 4 ms
scimark_sor             | 301 ms +- 4 ms       | 446 ms +- 8 ms       | 363 ms +- 6 ms
scimark_sparse_mat_mult | 6.65 ms +- 0.26 ms   | 10.3 ms +- 0.2 ms    | 8.14 ms +- 0.09 ms
spectral_norm           | 205 ms +- 5 ms       | 296 ms +- 5 ms       | 251 ms +- 2 ms
sqlalchemy_declarative  | 257 ms +- 3 ms       | 313 ms +- 4 ms       | 289 ms +- 3 ms
sqlalchemy_imperative   | 51.4 ms +- 1.3 ms    | 63.7 ms +- 4.0 ms    | 55.5 ms +- 3.3 ms
sqlite_synth            | 4.23 us +- 0.12 us   | 5.77 us +- 0.17 us   | 5.24 us +- 0.51 us
sympy_expand            | 766 ms +- 13 ms      | 1.03 sec +- 0.04 sec | 806 ms +- 13 ms
sympy_integrate         | 38.4 ms +- 0.7 ms    | 48.3 ms +- 0.4 ms    | 42.0 ms +- 0.6 ms
sympy_sum               | 348 ms +- 8 ms       | 433 ms +- 4 ms       | 369 ms +- 6 ms
sympy_str               | 526 ms +- 8 ms       | 676 ms +- 6 ms       | 562 ms +- 6 ms
telco                   | 8.56 ms +- 0.13 ms   | 13.0 ms +- 0.7 ms    | 12.0 ms +- 0.4 ms
tornado_http            | 293 ms +- 11 ms      | 376 ms +- 23 ms      | 330 ms +- 13 ms
unpack_sequence         | 83.8 ns +- 4.0 ns    | 92.8 ns +- 5.3 ns    | 89.8 ns +- 3.2 ns
unpickle                | 19.5 us +- 0.3 us    | 27.0 us +- 0.5 us    | 25.0 us +- 1.1 us
unpickle_list           | 5.94 us +- 0.12 us   | 7.99 us +- 0.38 us   | 6.82 us +- 0.53 us
unpickle_pure_python    | 461 us +- 10 us      | 749 us +- 24 us      | 566 us +- 26 us
xml_etree_parse         | 219 ms +- 3 ms       | 259 ms +- 2 ms       | 261 ms +- 13 ms
xml_etree_iterparse     | 155 ms +- 4 ms       | 192 ms +- 2 ms       | 180 ms +- 3 ms
xml_etree_generate      | 138 ms +- 3 ms       | 185 ms +- 3 ms       | 175 ms +- 9 ms
xml_etree_process       | 109 ms +- 4 ms       | 148 ms +- 2 ms       | 136 ms +- 2 ms