From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Stefan Monnier Newsgroups: gmane.emacs.devel Subject: Re: elisp-benchmarks Date: Thu, 10 Feb 2022 17:31:59 -0500 Message-ID: References: Mime-Version: 1.0 Content-Type: text/plain Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="7096"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/29.0.50 (gnu/linux) Cc: emacs-devel@gnu.org To: Mattias =?windows-1252?Q?Engdeg=E5rd?= Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Thu Feb 10 23:33:40 2022 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1nII0R-0001cD-V5 for ged-emacs-devel@m.gmane-mx.org; Thu, 10 Feb 2022 23:33:40 +0100 Original-Received: from localhost ([::1]:54280 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nII0Q-0006Mg-Mg for ged-emacs-devel@m.gmane-mx.org; Thu, 10 Feb 2022 17:33:38 -0500 Original-Received: from eggs.gnu.org ([209.51.188.92]:58956) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nIHyy-0005Ik-B3 for emacs-devel@gnu.org; Thu, 10 Feb 2022 17:32:08 -0500 Original-Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:27735) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nIHyv-0008VT-Gk for emacs-devel@gnu.org; Thu, 10 Feb 2022 17:32:07 -0500 Original-Received: from pmg3.iro.umontreal.ca (localhost [127.0.0.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id 9261A442258; Thu, 10 Feb 2022 17:32:02 -0500 (EST) Original-Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id 04F104421B8; Thu, 10 Feb 2022 17:32:01 -0500 (EST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1644532321; bh=3IiaAVfleLhXcFeYzxlyPDVzcqF8ZmpeSafQOdwkay4=; h=From:To:Cc:Subject:References:Date:In-Reply-To:From; b=NzaHNUEUB2UJKbeivdSAq1OPD7FEjww4aEu6kINPEll6LVWKKqLHh3ZGSawkc5ZUG kLt/5JnbLZ3TJThhetgfoZ0bHTQQjvFHQzMSraXabKB2EFkls9Tc8HMjOtQYHaLwvM vUTcA+yMvPtQcstAEaDI8B08g8lzTkhAVksLe0U/EoIb2AzU7YvkV2QZ53LBCGyzWJ EQNYL5/2JNWTJX8SNSIY45XUtxVt1gZg8Aw331EO46QGGNAPD5PsdlIT3BFKncM5qm zb3EiSQ3d9Tm1c0G4+ygF9+oITqWqLGSVlKnpypDE6fyJDsFCuTyjOoxs0JQ8t3h2y ZuYZgeL4vDhqA== Original-Received: from pastel (76-10-138-212.dsl.teksavvy.com [76.10.138.212]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id C917C120475; Thu, 10 Feb 2022 17:32:00 -0500 (EST) In-Reply-To: ("Mattias =?windows-1252?Q?Engdeg=E5rd=22's?= message of "Thu, 10 Feb 2022 22:53:06 +0100") Received-SPF: pass client-ip=132.204.25.50; envelope-from=monnier@iro.umontreal.ca; helo=mailscanner.iro.umontreal.ca X-Spam_score_int: -42 X-Spam_score: -4.3 X-Spam_bar: ---- X-Spam_report: (-4.3 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.io gmane.emacs.devel:286151 Archived-At: > Thank you! Normalised to the first version where each benchmark result > appears, we have (excluding GC): > > | | 26 | 27 | 28 | pre sympos | sympos | master | > |--------------------+------+------+------+------------+--------+--------| > | bubble | 1.00 | 0.81 | 0.54 | 0.62 | 0.58 | 0.46 | > | bubble-no-cons | 1.00 | 0.87 | 0.75 | 0.77 | 0.78 | 0.63 | > | bytecomp | 1.00 | 0.94 | 1.00 | 1.07 | 1.13 | 1.06 | > | dhrystone | 1.00 | 0.91 | 0.82 | 0.83 | 0.86 | 0.66 | > | eieio | 1.00 | 0.98 | 0.95 | 1.04 | 1.03 | 0.87 | > | fibn | 1.00 | 0.86 | 0.75 | 0.75 | 0.75 | 0.52 | > | fibn-named-let | | | 1.00 | 1.03 | 1.01 | 0.70 | > | fibn-rec | 1.00 | 1.14 | 1.00 | 1.02 | 1.04 | 0.68 | > | fibn-tc | 1.00 | 1.05 | 0.96 | 0.99 | 1.00 | 0.59 | > | flet | 1.00 | 1.23 | 0.88 | 0.91 | 0.96 | 0.62 | > | inclist | 1.00 | 1.36 | 0.79 | 0.78 | 0.84 | 0.76 | > | inclist-type-hints | 1.00 | 1.40 | 0.60 | 0.59 | 0.64 | 0.58 | > | listlen-tc | 1.00 | 0.97 | 0.89 | 0.91 | 0.92 | 0.67 | > | map-closure | 1.00 | 1.14 | 1.04 | 1.04 | 1.06 | 0.72 | > | nbody | 1.00 | 0.95 | 0.82 | 0.84 | 0.81 | 0.71 | > | pack-unpack | | | 1.00 | 1.04 | 1.03 | 0.83 | > | pack-unpack-old | 1.00 | 0.89 | 0.49 | 0.52 | 0.52 | 0.47 | > | pcase | 1.00 | 0.86 | 0.72 | 0.74 | 0.64 | 0.63 | > | pidigits | | 1.00 | 1.06 | 1.15 | 1.07 | 0.99 | > | scroll | | | 1.00 | 1.05 | 1.02 | 1.09 | > | smie | 1.00 | 0.86 | 0.78 | 0.77 | 0.79 | 0.78 | Thanks. > (The pidigits result was omitted from Emacs 26 because it didn't make much sense.) Good catch, yes, `pidigits` needs bignums to make sense, otherwise it computes (quickly) a bogus result. > The bytecomp benchmark is mobile -- it does more work (and generates better > code) now than in earlier versions. Yes, we don't want `bytecomp` to become unbearably so, it's OK if it gets a bit slower if we get better code or better error messages out of it. > Not sure how comparable scroll is, but it's probably a difficult thing to measure. I don't know how to interpret the evolution of that benchmark, indeed. It paradoxically seems to be made faster by sympos and slower by your subsequent bytecode improvements. I'd tend to assume that the "faster with sympos" is true-or-negligible, and but the "slower on master" is a bit more perplexing. Could it be that there have been other changes that happened on master recently that would cause that? My intuition is that neither sympos nor your bytecode work should make very much difference to the `scroll` benchmark since it should spend most of its time in other parts of the C code (e.g. text-property manipulations, regexp matching, redisplay, string extraction from the buffer, ...). Stefan