From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: ludo@gnu.org (Ludovic =?iso-8859-1?Q?Court=E8s?=) Newsgroups: gmane.lisp.guile.devel,gmane.lisp.guile.user Subject: Re: "Pace is nothing without guile" Date: Tue, 15 Jul 2008 21:21:56 +0200 Message-ID: <87ej5v9cmj.fsf@gnu.org> References: <49dd78620807131006m4efc2a96v2d9778b6c4a501b4@mail.gmail.com> NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: ger.gmane.org 1216149762 10705 80.91.229.12 (15 Jul 2008 19:22:42 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Tue, 15 Jul 2008 19:22:42 +0000 (UTC) Cc: guile-user@gnu.org To: guile-devel@gnu.org Original-X-From: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Tue Jul 15 21:23:30 2008 Return-path: Envelope-to: guile-devel@m.gmane.org Original-Received: from lists.gnu.org ([199.232.76.165]) by lo.gmane.org with esmtp (Exim 4.50) id 1KIq7P-0005ko-4h for guile-devel@m.gmane.org; Tue, 15 Jul 2008 21:23:23 +0200 Original-Received: from localhost ([127.0.0.1]:52717 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1KIq6W-0002LW-JQ for guile-devel@m.gmane.org; Tue, 15 Jul 2008 15:22:28 -0400 Original-Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1KIq6L-0002JT-WF for guile-devel@gnu.org; Tue, 15 Jul 2008 15:22:18 -0400 Original-Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1KIq6L-0002J7-A9 for guile-devel@gnu.org; Tue, 15 Jul 2008 15:22:17 -0400 Original-Received: from [199.232.76.173] (port=57166 helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1KIq6K-0002J1-Uo for guile-devel@gnu.org; Tue, 15 Jul 2008 15:22:16 -0400 Original-Received: from main.gmane.org ([80.91.229.2]:47494 helo=ciao.gmane.org) by monty-python.gnu.org with esmtps (TLS-1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.60) (envelope-from ) id 1KIq6K-0002FJ-E7 for guile-devel@gnu.org; Tue, 15 Jul 2008 15:22:16 -0400 Original-Received: from list by ciao.gmane.org with local (Exim 4.43) id 1KIq6G-00081U-12 for guile-devel@gnu.org; Tue, 15 Jul 2008 19:22:12 +0000 Original-Received: from reverse-83.fdn.fr ([80.67.176.83]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Tue, 15 Jul 2008 19:22:12 +0000 Original-Received: from ludo by reverse-83.fdn.fr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Tue, 15 Jul 2008 19:22:12 +0000 X-Injected-Via-Gmane: http://gmane.org/ Original-Followup-To: gmane.lisp.guile.devel Original-Lines: 31 Original-X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: reverse-83.fdn.fr X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 28 Messidor an 216 de la =?iso-8859-1?Q?R=E9volution?= X-PGP-Key-ID: 0xEA52ECF4 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 821D 815D 902A 7EAB 5CEE D120 7FBA 3D4F EB1F 5364 X-OS: i686-pc-linux-gnu User-Agent: Gnus/5.11 (Gnus v5.11) Emacs/22.2 (gnu/linux) Cancel-Lock: sha1:U24vZ5X0bVDP9104DMvZ3dBc3eI= X-detected-kernel: by monty-python.gnu.org: Linux 2.6, seldom 2.4 (older, 4) X-BeenThere: guile-devel@gnu.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Developers list for Guile, the GNU extensibility library" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Original-Sender: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Errors-To: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.lisp.guile.devel:7380 gmane.lisp.guile.user:6664 Archived-At: Hi, "Neil Jerram" writes: > So this email is about systematic performance data. I was wondering > what benchmarks we could run to get good coverage of all Guile's > function, and suddenly thought "of course, the test suite!" Like Greg, I'm a bit suspicious about using the test suite as a collection of micro-benchmarks. Usually, micro-benchmarks aim to assess the cost of a specific operation, which must consequently be isolated to avoid interference with unrelated computations. Conversely, unit tests aim to verify that certain invariants hold, regardless of "peripheral" computations required for that verification. For instance, a few SRFI-14 tests use `every', a few SRFI-69 tests use `lset=', various tests use `memq', etc. (OTOH, looking at the test suite, I'm not sure whether these tests are exceptions.) Other than that, my feeling is that it may be harder to analyze timings of tests that were not written as micro-benchmarks in the first place, since one must first determine what the test actually measures. > - anyone running the tests and uploading data, not just Guile core developers Although quite in fashion these days (see CDash), I'm not sure we absolutely need such a tool. Having *some* benchmark suite to run seems more important as a first step. :-) Thanks, Ludo'.