From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: Joshua Branson Newsgroups: gmane.emacs.devel Subject: Re: a benchmark suite for Emacs Date: Mon, 26 Nov 2018 09:16:36 -0500 Message-ID: <878t1fapq3.fsf@fastmail.com> References: <87efb8iptb.fsf@runbox.com> NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain X-Trace: blaine.gmane.org 1543241734 32444 195.159.176.226 (26 Nov 2018 14:15:34 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Mon, 26 Nov 2018 14:15:34 +0000 (UTC) User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux) To: emacs-devel@gnu.org Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Mon Nov 26 15:15:30 2018 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gRHfd-0008JS-Tj for ged-emacs-devel@m.gmane.org; Mon, 26 Nov 2018 15:15:30 +0100 Original-Received: from localhost ([::1]:36579 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gRHhj-0006QH-Vl for ged-emacs-devel@m.gmane.org; Mon, 26 Nov 2018 09:17:40 -0500 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:48922) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gRHgS-0006Q4-7c for emacs-devel@gnu.org; Mon, 26 Nov 2018 09:16:20 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gRHgP-0000t1-4Y for emacs-devel@gnu.org; Mon, 26 Nov 2018 09:16:20 -0500 Original-Received: from wout2-smtp.messagingengine.com ([64.147.123.25]:42139) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gRHgO-0000rE-NK for emacs-devel@gnu.org; Mon, 26 Nov 2018 09:16:17 -0500 Original-Received: from compute6.internal (compute6.nyi.internal [10.202.2.46]) by mailout.west.internal (Postfix) with ESMTP id 76300D00 for ; Mon, 26 Nov 2018 09:16:13 -0500 (EST) Original-Received: from mailfrontend1 ([10.202.2.162]) by compute6.internal (MEProxy); Mon, 26 Nov 2018 09:16:13 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastmail.com; h= from:to:subject:references:date:in-reply-to:message-id :mime-version:content-type; s=fm1; bh=TjfU+ByNcMgjLzQNNlcsxpwTEd FbuUtUIjoWS4Fb4Og=; b=GNixbZkO74AF9ET+tTuYbsd1rLN7tK9aFxiqhn6hlz 0VXJy7Gfb+8llnONQoZyI1CEsLCNVIJ+1oee5+3cqpCJLru7JbhCVO5wvX6E6XvG 098WPkFEcZ7uXFX74NlG/nCxpU/exH5kljD3ANyOnINOK5eX7i0wrtxIlA6HlZVh Fb0Aa3Tzor/IAR45lfWEi+pEzQlFz9Wuyyf91rjcPpn8B2W5GlwjyuF3ZcVnNvUz o1pkI+qBI0j36jxx/e98C5WFzWNcTgUR5t6KYNCjPyHCWMZYeLz6eF9C95JrsqxD OKlHvFf/T+r2Tk//vVE049De/snItJi0NqjdjSlD/b1A== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm1; bh=TjfU+B yNcMgjLzQNNlcsxpwTEdFbuUtUIjoWS4Fb4Og=; b=lGKA1JO1oBGh8Q2/T+cIFF ycAV/zB01YYA2QonWwbQd7QjIWDtBWCALa5Ffsu3CE7VzvqGRZj+kglK9fHmEErr Hdas6BRChHKqNtwi+ojj51/ko5BWXX4PSGJpuEouPAkck+dgZaHRqOK1+Lk9vn/m 0aS7VhL/RxWVsRpeL0KzYStaU5E/cmsOxKUWA/ANL1D4Si8Bhv7ES9/RXxpzaeYm ICKlIilZLRhX5JSBOxEUz86VyAnzuaT43ALrze2WCEe2vXKjzqguAna/1BM9SOsl 21b2Xxz4peSgU8rjfUEbk4UPLzTirbsQpkE6Wy7lK7FVn9lkGZzIoMt/6rJpmjMQ == X-ME-Sender: X-ME-Proxy: Original-Received: from dobby (unknown [72.12.220.132]) by mail.messagingengine.com (Postfix) with ESMTPA id 8D7CAE49DF for ; Mon, 26 Nov 2018 09:16:12 -0500 (EST) In-Reply-To: <87efb8iptb.fsf@runbox.com> (Gemini Lasswell's message of "Sun, 25 Nov 2018 17:35:12 -0800") X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 64.147.123.25 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.org gmane.emacs.devel:231395 Archived-At: This is super cool! Gemini Lasswell writes: > Hello all, > > I've just pushed to scratch/benchmarks my WIP benchmarking suite for > Emacs and Emacs packages. > > It's a long ways from done. Some major missing parts are: > > - Better/more complete user interface > - Documentation > - Statistical analysis (report on noisiness of data) > - Tramp integration/run benchmarks remotely > - Use VC rather than calling git directly > - More benchmark tasks > - Tests > > But, given: > > - a local clone of a git repo > - a program/shell script that produces an executable given that > git repo and a commit > - a directory in the git repo with Lisp files containing tasks to be > benchmarked > - a range of commits (as for git-rev-list(1)) > - a number of commits to select from the range > > it can use the shell script to build older versions of Emacs from > commits selected from the commit range given, run the tasks in those > older Emacs, record timing data, errors, process output and messages, > and then produce a report in the form of an org-mode file ready for > org-babel-gnuplot. > > The benchmark data is stored in a directory hierarchy organized so that > benchmark data created on separate machines can be easily git-merged > together. > > There's no documentation yet, but if any of you have time and interest > in running benchmarks, let me know and I'll write up instructions. > > For those of you who generously spend your time reviewing code, the new > benchmark directory, thread.el and erb-task.el are ready for your > attention, but erb.el is not. > > I also welcome ideas for more benchmarks, or anyone who wants to > write some.