From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: Eli Zaretskii Newsgroups: gmane.emacs.devel Subject: Re: An Emacs benchmarking suite Date: Mon, 14 May 2018 19:18:48 +0300 Message-ID: <83a7t2ky6v.fsf@gnu.org> References: <8336yypvq0.fsf@gnu.org> <83bmdmnskd.fsf@gnu.org> <0af8a67e-3403-8e81-a666-78b3d8da54b1@disroot.org> <838t8qnld0.fsf@gnu.org> <87a7t21na2.fsf@russet.org.uk> Reply-To: Eli Zaretskii NNTP-Posting-Host: blaine.gmane.org X-Trace: blaine.gmane.org 1526314647 25029 195.159.176.226 (14 May 2018 16:17:27 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Mon, 14 May 2018 16:17:27 +0000 (UTC) Cc: johnw@gnu.org, emacs-devel@gnu.org To: phillip.lord@russet.org.uk (Phillip Lord) Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Mon May 14 18:17:22 2018 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1fIGA6-0006QS-EJ for ged-emacs-devel@m.gmane.org; Mon, 14 May 2018 18:17:22 +0200 Original-Received: from localhost ([::1]:46035 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIGCD-0002ri-AO for ged-emacs-devel@m.gmane.org; Mon, 14 May 2018 12:19:33 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:53040) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIGBY-0002r4-7Z for emacs-devel@gnu.org; Mon, 14 May 2018 12:18:53 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fIGBX-0000k0-9Q for emacs-devel@gnu.org; Mon, 14 May 2018 12:18:52 -0400 Original-Received: from fencepost.gnu.org ([2001:4830:134:3::e]:41942) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIGBR-0000gw-1L; Mon, 14 May 2018 12:18:45 -0400 Original-Received: from [176.228.60.248] (port=4847 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1fIGBQ-0006o2-FP; Mon, 14 May 2018 12:18:44 -0400 In-reply-to: <87a7t21na2.fsf@russet.org.uk> (phillip.lord@russet.org.uk) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.org gmane.emacs.devel:225290 Archived-At: > From: phillip.lord@russet.org.uk (Phillip Lord) > Cc: "Siraphob \(Ben\) Phipathananunth" , johnw@gnu.org, emacs-devel@gnu.org > Date: Mon, 14 May 2018 12:37:09 +0100 > > > Performance should indeed compare several versions n the same system. > > > I wonder if there is a way to work out a base line value. Benchmarks > could then look for multiples of this. But such a base line would also have to be specific to a platform and a given set of build options. > So, something like time to create a big list (for CPU), and time to read > a defined file (for IO). Then you could say "this file should parse in > 1x IO base-line + 10x CPU base-line. You assume a linear scalability, but that is not necessarily so. The ratio between performance indices of different codes could vary depending on the build option and the underlying OS. Btw, IME I/O is mostly negligible in Emacs applications, and generally is not interesting for the issue at hand. Only CPU is important, and maybe also memory usage.