From: Stefan Monnier <monnier@iro.umontreal.ca>
To: emacs-devel@gnu.org
Subject: elisp-benchmarks
Date: Wed, 09 Feb 2022 17:19:28 -0500 [thread overview]
Message-ID: <jwv5ypn7l9z.fsf-monnier+emacs@gnu.org> (raw)
Hi,
I just ran the elisp-benchmarks (with a repeat count of 10) on some
little-used server we have here and figured others might like to see the
result:
Before sympos (i.e. commit f57f28935a1f0c46776976dd497322a07d248f5f)
| test | non-gc avg (s) | gc avg (s) | gcs avg | tot avg (s) | tot avg err (s) |
|--------------------+----------------+------------+---------+-------------+-----------------|
| bubble | 6.75 | 8.75 | 433 | 15.50 | 0.03 |
| bubble-no-cons | 16.26 | 0.06 | 3 | 16.32 | 0.00 |
| bytecomp | 3.24 | 2.53 | 123 | 5.77 | 0.16 |
| dhrystone | 15.54 | 0.00 | 0 | 15.54 | 0.00 |
| eieio | 5.57 | 3.38 | 166 | 8.95 | 0.02 |
| fibn | 5.45 | 0.00 | 0 | 5.45 | 0.00 |
| fibn-named-let | 6.00 | 0.00 | 0 | 6.00 | 0.01 |
| fibn-rec | 11.11 | 0.00 | 0 | 11.11 | 0.01 |
| fibn-tc | 9.24 | 0.00 | 0 | 9.24 | 0.00 |
| flet | 18.50 | 0.00 | 0 | 18.50 | 0.01 |
| inclist | 18.11 | 0.00 | 0 | 18.11 | 0.01 |
| inclist-type-hints | 18.11 | 0.00 | 0 | 18.11 | 0.01 |
| listlen-tc | 9.33 | 0.00 | 0 | 9.33 | 0.02 |
| map-closure | 14.49 | 0.00 | 0 | 14.49 | 0.01 |
| nbody | 5.98 | 16.34 | 791 | 22.32 | 0.06 |
| pack-unpack | 1.07 | 0.87 | 43 | 1.94 | 0.00 |
| pack-unpack-old | 3.29 | 1.76 | 87 | 5.06 | 0.01 |
| pcase | 20.28 | 0.00 | 0 | 20.28 | 0.00 |
| pidigits | 11.96 | 10.17 | 401 | 22.13 | 0.18 |
| scroll | 1.34 | 0.12 | 6 | 1.46 | 0.00 |
| smie | 3.06 | 0.95 | 46 | 4.00 | 0.01 |
|--------------------+----------------+------------+---------+-------------+-----------------|
| total | 204.67 | 44.94 | 2102 | 249.61 | 0.26 |
Right after sympos (i.e. commit df49e3a3ab4cddf1e3c0f5482c7fdd809d8a8884)
| test | non-gc avg (s) | gc avg (s) | gcs avg | tot avg (s) | tot avg err (s) |
|--------------------+----------------+------------+---------+-------------+-----------------|
| bubble | 6.39 | 9.09 | 449 | 15.48 | 0.04 |
| bubble-no-cons | 16.50 | 0.06 | 3 | 16.56 | 0.00 |
| bytecomp | 3.43 | 2.69 | 130 | 6.12 | 0.16 |
| dhrystone | 16.07 | 0.00 | 0 | 16.07 | 0.01 |
| eieio | 5.50 | 3.38 | 166 | 8.88 | 0.02 |
| fibn | 5.42 | 0.00 | 0 | 5.42 | 0.00 |
| fibn-named-let | 5.87 | 0.00 | 0 | 5.87 | 0.00 |
| fibn-rec | 11.27 | 0.00 | 0 | 11.27 | 0.01 |
| fibn-tc | 9.42 | 0.00 | 0 | 9.42 | 0.00 |
| flet | 19.65 | 0.00 | 0 | 19.65 | 0.01 |
| inclist | 19.43 | 0.00 | 0 | 19.43 | 0.00 |
| inclist-type-hints | 19.43 | 0.00 | 0 | 19.43 | 0.00 |
| listlen-tc | 9.35 | 0.00 | 0 | 9.35 | 0.01 |
| map-closure | 14.83 | 0.00 | 0 | 14.83 | 0.05 |
| nbody | 5.78 | 16.38 | 789 | 22.15 | 0.07 |
| pack-unpack | 1.06 | 0.87 | 43 | 1.93 | 0.00 |
| pack-unpack-old | 3.32 | 1.77 | 86 | 5.09 | 0.01 |
| pcase | 17.65 | 0.00 | 0 | 17.65 | 0.00 |
| pidigits | 11.15 | 10.06 | 400 | 21.21 | 0.18 |
| scroll | 1.30 | 0.12 | 6 | 1.42 | 0.00 |
| smie | 3.11 | 0.95 | 46 | 4.06 | 0.00 |
|--------------------+----------------+------------+---------+-------------+-----------------|
| total | 205.92 | 45.37 | 2122 | 251.29 | 0.26 |
From master (i.e. commit d41a5e7e33067eb38b147ee2f8a1615f6faed7a4)
| test | non-gc avg (s) | gc avg (s) | gcs avg | tot avg (s) | tot avg err (s) |
|--------------------+----------------+------------+---------+-------------+-----------------|
| bubble | 5.00 | 9.15 | 433 | 14.15 | 0.07 |
| bubble-no-cons | 13.43 | 0.06 | 3 | 13.49 | 0.00 |
| bytecomp | 3.22 | 2.65 | 123 | 5.87 | 0.16 |
| dhrystone | 12.37 | 0.00 | 0 | 12.37 | 0.03 |
| eieio | 4.65 | 3.40 | 160 | 8.05 | 0.06 |
| fibn | 3.74 | 0.00 | 0 | 3.74 | 0.00 |
| fibn-named-let | 4.10 | 0.00 | 0 | 4.10 | 0.00 |
| fibn-rec | 7.33 | 0.00 | 0 | 7.33 | 0.00 |
| fibn-tc | 5.58 | 0.00 | 0 | 5.58 | 0.00 |
| flet | 12.55 | 0.00 | 0 | 12.55 | 0.01 |
| inclist | 17.67 | 0.00 | 0 | 17.67 | 0.01 |
| inclist-type-hints | 17.67 | 0.00 | 0 | 17.67 | 0.01 |
| listlen-tc | 6.84 | 0.00 | 0 | 6.84 | 0.01 |
| map-closure | 10.06 | 0.00 | 0 | 10.06 | 0.00 |
| nbody | 5.08 | 16.51 | 762 | 21.59 | 0.04 |
| pack-unpack | 0.86 | 0.88 | 41 | 1.74 | 0.01 |
| pack-unpack-old | 2.99 | 1.78 | 83 | 4.77 | 0.02 |
| pcase | 17.36 | 0.00 | 0 | 17.36 | 0.00 |
| pidigits | 10.32 | 9.97 | 386 | 20.28 | 0.34 |
| scroll | 1.39 | 0.13 | 6 | 1.52 | 0.00 |
| smie | 3.10 | 0.95 | 44 | 4.06 | 0.02 |
|--------------------+----------------+------------+---------+-------------+-----------------|
| total | 165.30 | 45.49 | 2044 | 210.78 | 0.39 |
`bytecomp`, `scroll`, and `smie` are benchmarks I added yesterday which
are not microbenchmarks and thus hopefully reflect "real use".
This was all with the byte-compiler and not the native compiler.
These result suggest that the symbol-with-pos changes don't introduce
a significant slowdown, even for the bytecompiler.
And we see that Matthias's recent improvements to the bytecode
interpreter do make a quite significant difference on several of those
microbenchmarks ;-), and also on the bytecompiler benchmark (offsetting
the extra work needed for the symbol-with-positions) tho they don't make
much of a difference when it comes to scrolling(with-jit-lock) or when
it comes to reindenting code with SMIE :-)
The >10% slowdown recently seen on the test suite is still a mystery
waiting for someone to figure out what's going on.
BTW, I think one thing is clear when I look at those benchmarks:
Emacs's GC is not good enough.
Stefan
next reply other threads:[~2022-02-09 22:19 UTC|newest]
Thread overview: 9+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-02-09 22:19 Stefan Monnier [this message]
2022-02-10 6:50 ` elisp-benchmarks Lars Ingebrigtsen
2022-02-10 7:52 ` elisp-benchmarks Eli Zaretskii
2022-02-10 12:12 ` elisp-benchmarks Mattias Engdegård
2022-02-10 14:13 ` elisp-benchmarks Stefan Monnier
2022-02-10 14:18 ` elisp-benchmarks Stefan Monnier
2022-02-10 16:51 ` elisp-benchmarks Stefan Monnier
2022-02-10 21:53 ` elisp-benchmarks Mattias Engdegård
2022-02-10 22:31 ` elisp-benchmarks Stefan Monnier
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=jwv5ypn7l9z.fsf-monnier+emacs@gnu.org \
--to=monnier@iro.umontreal.ca \
--cc=emacs-devel@gnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this external index
https://git.savannah.gnu.org/cgit/emacs.git
https://git.savannah.gnu.org/cgit/emacs/org-mode.git
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.