From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: Eli Zaretskii Newsgroups: gmane.emacs.devel Subject: Re: An Emacs benchmarking suite (was: An idea, now that we have dynamic loading) Date: Fri, 11 May 2018 20:38:35 +0300 Message-ID: <838t8qnld0.fsf@gnu.org> References: <8336yypvq0.fsf@gnu.org> <83bmdmnskd.fsf@gnu.org> <0af8a67e-3403-8e81-a666-78b3d8da54b1@disroot.org> Reply-To: Eli Zaretskii NNTP-Posting-Host: blaine.gmane.org X-Trace: blaine.gmane.org 1526060353 29367 195.159.176.226 (11 May 2018 17:39:13 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Fri, 11 May 2018 17:39:13 +0000 (UTC) Cc: johnw@gnu.org, emacs-devel@gnu.org To: "Siraphob \(Ben\) Phipathananunth" Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Fri May 11 19:39:08 2018 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1fHC0X-0007Un-5R for ged-emacs-devel@m.gmane.org; Fri, 11 May 2018 19:39:05 +0200 Original-Received: from localhost ([::1]:60261 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fHC2e-00020W-9R for ged-emacs-devel@m.gmane.org; Fri, 11 May 2018 13:41:16 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:47206) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fHC0B-0004AM-EP for emacs-devel@gnu.org; Fri, 11 May 2018 13:38:44 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fHC0A-0003hm-Pc for emacs-devel@gnu.org; Fri, 11 May 2018 13:38:43 -0400 Original-Received: from fencepost.gnu.org ([2001:4830:134:3::e]:41796) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fHC05-0003gE-PQ; Fri, 11 May 2018 13:38:37 -0400 Original-Received: from [176.228.60.248] (port=1092 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1fHC05-0000uZ-7L; Fri, 11 May 2018 13:38:37 -0400 In-reply-to: <0af8a67e-3403-8e81-a666-78b3d8da54b1@disroot.org> (siraben@disroot.org) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.org gmane.emacs.devel:225246 Archived-At: > Cc: johnw@gnu.org, emacs-devel@gnu.org > From: "Siraphob (Ben) Phipathananunth" > Date: Fri, 11 May 2018 22:22:47 +0700 > > Eli Zaretskii wrote: > > John even asked for creation of such a suite of performance tests, but > > AFAIK no one has picked the gauntlet till now. > > Would it suffice to use Emacs Lisp to run these performance tests > (i.e. using benchmark.el)? I don't see why not. > Even so, how would one account for external factors in the operating > system? Perhaps when the test is performed, the deltas between > commits are given as percentages instead of CPU time. Once we have > that, it would be great to have tests run multiple times and/or on > various devices to refine the data further. Performance should indeed compare several versions n the same system. > An area that would be interesting to look at is memory usage of > functions over commits, but I don't know of a way of measuring memory > usage in Emacs (especially over an extended period, to analyze things > such as maximum memory usage). Memory analysis is tricky on modern systems. but it isn't impossible. One could start using the values reported by process-attributes.