From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: Gemini Lasswell Newsgroups: gmane.emacs.devel Subject: Re: a benchmark suite for Emacs Date: Mon, 26 Nov 2018 17:28:26 -0800 Message-ID: <87y39fgvgl.fsf@runbox.com> References: <87efb8iptb.fsf@runbox.com> <87k1kzo392.fsf@mail.linkov.net> NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain X-Trace: blaine.gmane.org 1543282002 18108 195.159.176.226 (27 Nov 2018 01:26:42 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Tue, 27 Nov 2018 01:26:42 +0000 (UTC) User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1.90 (gnu/linux) Cc: emacs-devel@gnu.org To: Juri Linkov Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Tue Nov 27 02:26:38 2018 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gRS98-0004cD-2v for ged-emacs-devel@m.gmane.org; Tue, 27 Nov 2018 02:26:38 +0100 Original-Received: from localhost ([::1]:39612 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gRSBE-0004MY-M7 for ged-emacs-devel@m.gmane.org; Mon, 26 Nov 2018 20:28:48 -0500 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:40137) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gRSB8-0004MS-It for emacs-devel@gnu.org; Mon, 26 Nov 2018 20:28:43 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gRSB4-0005WR-LK for emacs-devel@gnu.org; Mon, 26 Nov 2018 20:28:42 -0500 Original-Received: from aibo.runbox.com ([91.220.196.211]:43418) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1gRSB4-0005VK-1S for emacs-devel@gnu.org; Mon, 26 Nov 2018 20:28:38 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=runbox.com; s=rbselector1; h=Content-Type:MIME-Version:Message-ID:In-Reply-To:Date: References:Subject:Cc:To:From; bh=oGeNxzMxHmJbSwMPpn3tFJo04M8lNI4NiyrM/Hc4DAQ=; b=a2lxm0YBGNRGV/rAJwS4thoK0/ Iy5xhydUOcQ3mnMd7966UqRvi439vztKgbZAt8u2fme1y1pqkwpIxDIZLDOTztocOkPz/flYJ7zUr Jni3zsctgtUvw5AwXJKVUOwKHLjNH6ZKiHmvQH+TryZf5/ZDZoZaMO7pE3b3HgGEgXEbFzXw+4cM/ yyH1s2A5ASiQce9hABs7GdzGh/eklkV4aEponQg7UsYtyCbpH5PmR00neyA0poxSDdTTCtgRgIKDs K7RoBGUFs5UDfcit1uNVnn4c0vhjL7hf95kzoRMGhFvq/NjaRqtTDikY5JjPOykuY2l74twv84NoP /VYDe18g==; Original-Received: from [10.9.9.211] (helo=mailfront11.runbox.com) by mailtransmit02.runbox with esmtp (Exim 4.86_2) (envelope-from ) id 1gRSB1-000262-G3; Tue, 27 Nov 2018 02:28:35 +0100 Original-Received: by mailfront11.runbox.com with esmtpsa (uid:179284 ) (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) id 1gRSAu-0002vA-Lf; Tue, 27 Nov 2018 02:28:29 +0100 In-Reply-To: <87k1kzo392.fsf@mail.linkov.net> (Juri Linkov's message of "Tue, 27 Nov 2018 01:18:17 +0200") X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 91.220.196.211 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.org gmane.emacs.devel:231425 Archived-At: Juri Linkov writes: >> - a local clone of a git repo >> - a program/shell script that produces an executable given that >> git repo and a commit >> - a directory in the git repo with Lisp files containing tasks to be >> benchmarked >> - a range of commits (as for git-rev-list(1)) >> - a number of commits to select from the range >> >> it can use the shell script to build older versions of Emacs from >> commits selected from the commit range given, run the tasks in those >> older Emacs, record timing data, errors, process output and messages, >> and then produce a report in the form of an org-mode file ready for >> org-babel-gnuplot. > > Is it like Diffbench for Ruby? Something like that was sorely missed > for a long time in Emacs. Like Diffbench, but bigger and better. Diffbench looks like it does a quick comparison of two states of the project. That's something this can do, but my vision for it is to build a database of benchmark results for multiple machines and for commits going back as far as we can figure out how to build and run them, so we can see how performance has changed over time and how it varies on different systems.