From mboxrd@z Thu Jan 1 00:00:00 1970 Path: main.gmane.org!not-for-mail From: Rob Browning Newsgroups: gmane.lisp.guile.devel Subject: Introspective benchmarking. Date: Tue, 19 Mar 2002 21:49:45 -0600 Sender: guile-devel-admin@gnu.org Message-ID: <87pu1zucdi.fsf@raven.i.defaultvalue.org> NNTP-Posting-Host: localhost.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: main.gmane.org 1016667767 19259 127.0.0.1 (20 Mar 2002 23:42:47 GMT) X-Complaints-To: usenet@main.gmane.org NNTP-Posting-Date: Wed, 20 Mar 2002 23:42:47 +0000 (UTC) Original-Received: from fencepost.gnu.org ([199.232.76.164]) by main.gmane.org with esmtp (Exim 3.33 #1 (Debian)) id 16npjC-00050W-00 for ; Thu, 21 Mar 2002 00:42:46 +0100 Original-Received: from localhost ([127.0.0.1] helo=fencepost.gnu.org) by fencepost.gnu.org with esmtp (Exim 3.34 #1 (Debian)) id 16npj3-000419-00; Wed, 20 Mar 2002 18:42:37 -0500 Original-Received: from delysid.gnu.org ([158.121.106.20]) by fencepost.gnu.org with esmtp (Exim 3.34 #1 (Debian)) id 16not0-0008LZ-00 for ; Wed, 20 Mar 2002 17:48:51 -0500 Original-Received: from dsl-209-87-109-2.constant.com ([209.87.109.2] helo=defaultvalue.org) by delysid.gnu.org with esmtp (Exim 3.34 #2) id 16nX9p-0001Yc-00 for ; Tue, 19 Mar 2002 22:53:01 -0500 Original-Received: from raven.i.defaultvalue.org (raven.i.defaultvalue.org [192.168.1.7]) by defaultvalue.org (Postfix) with ESMTP id 0C53341B4 for ; Tue, 19 Mar 2002 21:49:51 -0600 (CST) Original-Received: by raven.i.defaultvalue.org (Postfix, from userid 1000) id 5B20B1DC6; Tue, 19 Mar 2002 21:49:45 -0600 (CST) Original-To: guile-devel@gnu.org Original-Lines: 31 User-Agent: Gnus/5.090006 (Oort Gnus v0.06) Emacs/21.1 (i386-debian-linux-gnu) Errors-To: guile-devel-admin@gnu.org X-BeenThere: guile-devel@gnu.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: Developers list for Guile, the GNU extensibility library List-Unsubscribe: , List-Archive: Xref: main.gmane.org gmane.lisp.guile.devel:133 X-Report-Spam: http://spam.gmane.org/gmane.lisp.guile.devel:133 I feel like it would be extremely helpful to have a set of benchmarks that we can run in order to help evaluate our changes. While I'm well aware of the risks involved in benchmarking, I think even a moderately good set could provide a healthy sanity check of our efforts. What I'm wondering is how the benchmarks should be handled. For example, I feel like they probably shouldn't be part of the main (core) CVS tree, if for no other reason than not having to worry as much about about copyright assignments. There's also the argument that they don't belong with any one tree since we should probably be using them to test across guile versions. I don't know how many people have ever looked, but stalin has a fairly large set of benchmarks in ./benchmarks. I've played around a bit and found so far that many of the files will run just fine as-is, though if we want to be able to run the tests in a reasonable amount of time we'll need to lower a lot of the final looping constants by a couple of orders of magnitude or so :/ Stalin's source tree can actually run the set of benchmarks across a set of scheme implementations: bigloo, stalin, scheme->c, gambit-c, and chez, but it takes about two days on a 450MHz Xeon... Thoughts? -- Rob Browning rlb @defaultvalue.org, @linuxdevel.com, and @debian.org Previously @cs.utexas.edu GPG=1C58 8B2C FB5E 3F64 EA5C 64AE 78FE E5FE F0CB A0AD _______________________________________________ Guile-devel mailing list Guile-devel@gnu.org http://mail.gnu.org/mailman/listinfo/guile-devel