From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Stefan Monnier Newsgroups: gmane.emacs.devel Subject: elisp-benchmarks Date: Wed, 09 Feb 2022 17:19:28 -0500 Message-ID: Mime-Version: 1.0 Content-Type: text/plain Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="29683"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/29.0.50 (gnu/linux) To: emacs-devel@gnu.org Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Wed Feb 09 23:27:57 2022 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1nHvRM-0007Wn-Rp for ged-emacs-devel@m.gmane-mx.org; Wed, 09 Feb 2022 23:27:57 +0100 Original-Received: from localhost ([::1]:36464 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nHvRL-0006vQ-FM for ged-emacs-devel@m.gmane-mx.org; Wed, 09 Feb 2022 17:27:55 -0500 Original-Received: from eggs.gnu.org ([209.51.188.92]:42606) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nHvJK-0007v8-Ct for emacs-devel@gnu.org; Wed, 09 Feb 2022 17:19:41 -0500 Original-Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:6140) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nHvJG-00087e-DS for emacs-devel@gnu.org; Wed, 09 Feb 2022 17:19:36 -0500 Original-Received: from pmg1.iro.umontreal.ca (localhost.localdomain [127.0.0.1]) by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id F1A56100211; Wed, 9 Feb 2022 17:19:31 -0500 (EST) Original-Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg1.iro.umontreal.ca (Proxmox) with ESMTP id 1FE1F100166; Wed, 9 Feb 2022 17:19:30 -0500 (EST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1644445170; bh=/boBFLijrG3/YvJBWFeVeTSxlgECdMGGHaHn4zDVo3w=; h=From:To:Subject:Date:From; b=W/AsUuq3whLiDwDMP4NEuIyNIUuQGT+iXF8/lqov51bP5sNHLHNg77y0hA8ZmAgeG 0+blNKcuo74VxLIwtPfrr1yJ8/7thT/V0kmuFQmvm92ARDH86iZqn37yVvJ9UVBSv+ gN6/kuWUHrfHWCgnC0db99YOAH4/qZcvY4T3yEmqutH/ie84PECdQLZ2tFPBpPHDNZ J1xMjVuh+JP3xgAB9TZ8Z9zwTmZjYPuqW6EfG69sdjre8jN9gOUD7qoP3OgRoLMyk9 WX1xxKwaSTmRzb7P8Pl+24l7KRH5YDIUJv1XD1TQalTmgvEbLjg0K91d+zCPkesrRt SUvKla3WBjhyA== Original-Received: from alfajor (lechon.iro.umontreal.ca [132.204.27.242]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 13F8E12073E; Wed, 9 Feb 2022 17:19:30 -0500 (EST) Received-SPF: pass client-ip=132.204.25.50; envelope-from=monnier@iro.umontreal.ca; helo=mailscanner.iro.umontreal.ca X-Spam_score_int: -42 X-Spam_score: -4.3 X-Spam_bar: ---- X-Spam_report: (-4.3 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.io gmane.emacs.devel:286116 Archived-At: Hi, I just ran the elisp-benchmarks (with a repeat count of 10) on some little-used server we have here and figured others might like to see the result: Before sympos (i.e. commit f57f28935a1f0c46776976dd497322a07d248f5f) | test | non-gc avg (s) | gc avg (s) | gcs avg | tot avg (s) | tot avg err (s) | |--------------------+----------------+------------+---------+-------------+-----------------| | bubble | 6.75 | 8.75 | 433 | 15.50 | 0.03 | | bubble-no-cons | 16.26 | 0.06 | 3 | 16.32 | 0.00 | | bytecomp | 3.24 | 2.53 | 123 | 5.77 | 0.16 | | dhrystone | 15.54 | 0.00 | 0 | 15.54 | 0.00 | | eieio | 5.57 | 3.38 | 166 | 8.95 | 0.02 | | fibn | 5.45 | 0.00 | 0 | 5.45 | 0.00 | | fibn-named-let | 6.00 | 0.00 | 0 | 6.00 | 0.01 | | fibn-rec | 11.11 | 0.00 | 0 | 11.11 | 0.01 | | fibn-tc | 9.24 | 0.00 | 0 | 9.24 | 0.00 | | flet | 18.50 | 0.00 | 0 | 18.50 | 0.01 | | inclist | 18.11 | 0.00 | 0 | 18.11 | 0.01 | | inclist-type-hints | 18.11 | 0.00 | 0 | 18.11 | 0.01 | | listlen-tc | 9.33 | 0.00 | 0 | 9.33 | 0.02 | | map-closure | 14.49 | 0.00 | 0 | 14.49 | 0.01 | | nbody | 5.98 | 16.34 | 791 | 22.32 | 0.06 | | pack-unpack | 1.07 | 0.87 | 43 | 1.94 | 0.00 | | pack-unpack-old | 3.29 | 1.76 | 87 | 5.06 | 0.01 | | pcase | 20.28 | 0.00 | 0 | 20.28 | 0.00 | | pidigits | 11.96 | 10.17 | 401 | 22.13 | 0.18 | | scroll | 1.34 | 0.12 | 6 | 1.46 | 0.00 | | smie | 3.06 | 0.95 | 46 | 4.00 | 0.01 | |--------------------+----------------+------------+---------+-------------+-----------------| | total | 204.67 | 44.94 | 2102 | 249.61 | 0.26 | Right after sympos (i.e. commit df49e3a3ab4cddf1e3c0f5482c7fdd809d8a8884) | test | non-gc avg (s) | gc avg (s) | gcs avg | tot avg (s) | tot avg err (s) | |--------------------+----------------+------------+---------+-------------+-----------------| | bubble | 6.39 | 9.09 | 449 | 15.48 | 0.04 | | bubble-no-cons | 16.50 | 0.06 | 3 | 16.56 | 0.00 | | bytecomp | 3.43 | 2.69 | 130 | 6.12 | 0.16 | | dhrystone | 16.07 | 0.00 | 0 | 16.07 | 0.01 | | eieio | 5.50 | 3.38 | 166 | 8.88 | 0.02 | | fibn | 5.42 | 0.00 | 0 | 5.42 | 0.00 | | fibn-named-let | 5.87 | 0.00 | 0 | 5.87 | 0.00 | | fibn-rec | 11.27 | 0.00 | 0 | 11.27 | 0.01 | | fibn-tc | 9.42 | 0.00 | 0 | 9.42 | 0.00 | | flet | 19.65 | 0.00 | 0 | 19.65 | 0.01 | | inclist | 19.43 | 0.00 | 0 | 19.43 | 0.00 | | inclist-type-hints | 19.43 | 0.00 | 0 | 19.43 | 0.00 | | listlen-tc | 9.35 | 0.00 | 0 | 9.35 | 0.01 | | map-closure | 14.83 | 0.00 | 0 | 14.83 | 0.05 | | nbody | 5.78 | 16.38 | 789 | 22.15 | 0.07 | | pack-unpack | 1.06 | 0.87 | 43 | 1.93 | 0.00 | | pack-unpack-old | 3.32 | 1.77 | 86 | 5.09 | 0.01 | | pcase | 17.65 | 0.00 | 0 | 17.65 | 0.00 | | pidigits | 11.15 | 10.06 | 400 | 21.21 | 0.18 | | scroll | 1.30 | 0.12 | 6 | 1.42 | 0.00 | | smie | 3.11 | 0.95 | 46 | 4.06 | 0.00 | |--------------------+----------------+------------+---------+-------------+-----------------| | total | 205.92 | 45.37 | 2122 | 251.29 | 0.26 | >From master (i.e. commit d41a5e7e33067eb38b147ee2f8a1615f6faed7a4) | test | non-gc avg (s) | gc avg (s) | gcs avg | tot avg (s) | tot avg err (s) | |--------------------+----------------+------------+---------+-------------+-----------------| | bubble | 5.00 | 9.15 | 433 | 14.15 | 0.07 | | bubble-no-cons | 13.43 | 0.06 | 3 | 13.49 | 0.00 | | bytecomp | 3.22 | 2.65 | 123 | 5.87 | 0.16 | | dhrystone | 12.37 | 0.00 | 0 | 12.37 | 0.03 | | eieio | 4.65 | 3.40 | 160 | 8.05 | 0.06 | | fibn | 3.74 | 0.00 | 0 | 3.74 | 0.00 | | fibn-named-let | 4.10 | 0.00 | 0 | 4.10 | 0.00 | | fibn-rec | 7.33 | 0.00 | 0 | 7.33 | 0.00 | | fibn-tc | 5.58 | 0.00 | 0 | 5.58 | 0.00 | | flet | 12.55 | 0.00 | 0 | 12.55 | 0.01 | | inclist | 17.67 | 0.00 | 0 | 17.67 | 0.01 | | inclist-type-hints | 17.67 | 0.00 | 0 | 17.67 | 0.01 | | listlen-tc | 6.84 | 0.00 | 0 | 6.84 | 0.01 | | map-closure | 10.06 | 0.00 | 0 | 10.06 | 0.00 | | nbody | 5.08 | 16.51 | 762 | 21.59 | 0.04 | | pack-unpack | 0.86 | 0.88 | 41 | 1.74 | 0.01 | | pack-unpack-old | 2.99 | 1.78 | 83 | 4.77 | 0.02 | | pcase | 17.36 | 0.00 | 0 | 17.36 | 0.00 | | pidigits | 10.32 | 9.97 | 386 | 20.28 | 0.34 | | scroll | 1.39 | 0.13 | 6 | 1.52 | 0.00 | | smie | 3.10 | 0.95 | 44 | 4.06 | 0.02 | |--------------------+----------------+------------+---------+-------------+-----------------| | total | 165.30 | 45.49 | 2044 | 210.78 | 0.39 | `bytecomp`, `scroll`, and `smie` are benchmarks I added yesterday which are not microbenchmarks and thus hopefully reflect "real use". This was all with the byte-compiler and not the native compiler. These result suggest that the symbol-with-pos changes don't introduce a significant slowdown, even for the bytecompiler. And we see that Matthias's recent improvements to the bytecode interpreter do make a quite significant difference on several of those microbenchmarks ;-), and also on the bytecompiler benchmark (offsetting the extra work needed for the symbol-with-positions) tho they don't make much of a difference when it comes to scrolling(with-jit-lock) or when it comes to reindenting code with SMIE :-) The >10% slowdown recently seen on the test suite is still a mystery waiting for someone to figure out what's going on. BTW, I think one thing is clear when I look at those benchmarks: Emacs's GC is not good enough. Stefan