From mboxrd@z Thu Jan 1 00:00:00 1970 Path: main.gmane.org!not-for-mail From: "Robert Anderson" Newsgroups: gmane.emacs.devel Subject: Re: emacs test suite? Date: Thu, 16 Jan 2003 19:25:07 -0500 Sender: emacs-devel-bounces+emacs-devel=quimby.gnus.org@gnu.org Message-ID: NNTP-Posting-Host: main.gmane.org X-Trace: main.gmane.org 1042763113 16585 80.91.224.249 (17 Jan 2003 00:25:13 GMT) X-Complaints-To: usenet@main.gmane.org NNTP-Posting-Date: Fri, 17 Jan 2003 00:25:13 +0000 (UTC) Cc: emacs-devel@gnu.org Return-path: Original-Received: from quimby.gnus.org ([80.91.224.244]) by main.gmane.org with esmtp (Exim 3.35 #1 (Debian)) id 18ZKJp-0004J3-00 for ; Fri, 17 Jan 2003 01:25:09 +0100 Original-Received: from monty-python.gnu.org ([199.232.76.173]) by quimby.gnus.org with esmtp (Exim 3.12 #1 (Debian)) id 18ZKT0-0006hI-00 for ; Fri, 17 Jan 2003 01:34:38 +0100 Original-Received: from localhost ([127.0.0.1] helo=monty-python.gnu.org) by monty-python.gnu.org with esmtp (Exim 4.10.13) id 18ZKKJ-0003zh-02 for emacs-devel@quimby.gnus.org; Thu, 16 Jan 2003 19:25:39 -0500 Original-Received: from list by monty-python.gnu.org with tmda-scanned (Exim 4.10.13) id 18ZKJu-0003r9-00 for emacs-devel@gnu.org; Thu, 16 Jan 2003 19:25:14 -0500 Original-Received: from mail by monty-python.gnu.org with spam-scanned (Exim 4.10.13) id 18ZKJp-0003h3-00 for emacs-devel@gnu.org; Thu, 16 Jan 2003 19:25:11 -0500 Original-Received: from pimout4-ext.prodigy.net ([207.115.63.103]) by monty-python.gnu.org with esmtp (Exim 4.10.13) id 18ZKJo-0003fG-00 for emacs-devel@gnu.org; Thu, 16 Jan 2003 19:25:08 -0500 Original-Received: from www4.prodigy.net (www4-ext.prodigy.net [207.115.61.60]) id h0H0P7BK054346; Thu, 16 Jan 2003 19:25:07 -0500 Original-To: "Thien-Thi Nguyen" X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1b5 Precedence: list List-Id: Emacs development discussions. List-Help: List-Post: List-Subscribe: , List-Archive: List-Unsubscribe: , Errors-To: emacs-devel-bounces+emacs-devel=quimby.gnus.org@gnu.org Xref: main.gmane.org gmane.emacs.devel:10788 X-Report-Spam: http://spam.gmane.org/gmane.emacs.devel:10788 --- Original Message --- From: Thien-Thi Nguyen To: "Robert Anderson" CC: emacs-devel@gnu.org Subject: Re: emacs test suite? >"Robert Anderson" writes: > > When I help identify and fix bugs, I like to be able to provide a > "push-button" test case that demonstrates the problem, and > prevents regression once the problem has been fixed. > >this approach is also useful for new code. > > When I run "make check" for GNU emacs, it tells me there are no > tests for emacs yet. > > Have there been any initiatives to develop a testing system for > emacs? > >that's what the ftp site is for, no? (half-smiley.) > >i'm in favor of such a system. for current-column munging (in the presence of >variable-width fonts), i have a local testing setup but it is not generalized. >this could be because testing interactive programs typically requires a driver >program external to the one under test, and that starts stepping into debugger >territory (where programmers' disparate debugging styles is nonregularizable). > >however, having said that, perhaps someone could build a system on top of >screen(1), for text mode; and something analogous, for X. > >thi [sorry for the screwed up quoting, my mailer is horrible.] It depends on what you are trying to test, I suppose. I think there is a broad class of "textual transformation" tests that would be trivial to implement as batch jobs called from a simple script harness. In fact, I've made a very simple sketch of how such a thing would work, which verifies the cc-mode filling bug in CVS HEAD (and also demonstrates that it works "as expected" for 21.1.1 and 20.7.1.) There is a driver script which runs GNU find on a subtree looking for programs which have the name, by convention, "run-test". Each command "run-test" accepts an argument "--description" which is spits out a one-line description to be displayed while the test is running, as well as an option to specify a full path to the emacs binary to test. Each program "run-test" returns 0 for pass and non-zero for fail. I suspect from perusing the mailing list that this sort of thing might be valuable, if people would use it. Bob