From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Christian Ohler Newsgroups: gmane.emacs.devel Subject: Re: unit test framework Date: Sat, 12 Jun 2010 16:06:45 +1000 Message-ID: <4C1323F5.80301@fastmail.net> References: <20100318.163911.560917315840337376.yamato@redhat.com> NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-15; format=flowed Content-Transfer-Encoding: quoted-printable X-Trace: dough.gmane.org 1276322839 24250 80.91.229.12 (12 Jun 2010 06:07:19 GMT) X-Complaints-To: usenet@dough.gmane.org NNTP-Posting-Date: Sat, 12 Jun 2010 06:07:19 +0000 (UTC) Cc: Nikolaj Schumacher , web@shellarchive.co.uk, Lennart Borgman , Masatake YAMATO , joakim@verona.se, emacs-devel@gnu.org, rubikitch , Phil Hagelberg To: Stefan Monnier Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Sat Jun 12 08:07:16 2010 connect(): No such file or directory Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([199.232.76.165]) by lo.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1ONJsB-0005pB-As for ged-emacs-devel@m.gmane.org; Sat, 12 Jun 2010 08:07:15 +0200 Original-Received: from localhost ([127.0.0.1]:50137 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1ONJsA-0001aT-N6 for ged-emacs-devel@m.gmane.org; Sat, 12 Jun 2010 02:07:14 -0400 Original-Received: from [140.186.70.92] (port=33385 helo=eggs.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1ONJrz-0001ZF-CD for emacs-devel@gnu.org; Sat, 12 Jun 2010 02:07:04 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.69) (envelope-from ) id 1ONJrx-0005T5-Ra for emacs-devel@gnu.org; Sat, 12 Jun 2010 02:07:03 -0400 Original-Received: from out1.smtp.messagingengine.com ([66.111.4.25]:36977) by eggs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1ONJrx-0005Sy-OH for emacs-devel@gnu.org; Sat, 12 Jun 2010 02:07:01 -0400 Original-Received: from compute2.internal (compute2.internal [10.202.2.42]) by gateway1.messagingengine.com (Postfix) with ESMTP id 4A6ACF880A; Sat, 12 Jun 2010 02:06:54 -0400 (EDT) Original-Received: from heartbeat2.messagingengine.com ([10.202.2.161]) by compute2.internal (MEProxy); Sat, 12 Jun 2010 02:06:57 -0400 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=messagingengine.com; h=message-id:date:from:mime-version:to:cc:subject:references:in-reply-to:content-type:content-transfer-encoding; s=smtpout; bh=ARmvFjIQo+DRlSW7ph2MQcLNGyk=; b=q4NIqzbEL+wH+1YVqRfy8Gi79cx4iU+u8x5h3Kf2qd6PBDvMwDQ+MuqSyTcs6GEwLIbmHIIvG6XMlXtAc3SCbyQQ/i2coPDnI6oFDSkDjuuedMUfM1A119lc473x8fUfDTBSfd4Wzw0r8+i4Zkz3qI3RdHuoKzVlJ114AyGmt4Q= X-Sasl-enc: M86JryLR3xu3KmoratyOhZtbza1e1n4Ozg7C1+iVZrJU 1276322813 Original-Received: from kraut.local (17.8.70.115.static.exetel.com.au [115.70.8.17]) by mail.messagingengine.com (Postfix) with ESMTPSA id C07B84C896; Sat, 12 Jun 2010 02:06:50 -0400 (EDT) In-Reply-To: X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.devel:125784 Archived-At: joakim@verona.se, 2010-04-13: > Stefan Monnier writes: >=20 >>> I write try to write unit tests for my elisp, and it would be much >>> easier if there was a canonical unit test framework in Emacs. >> Yes, that's the intention. We should pick one and run with it. >> The choice doesn't have to be "the very best", but it should aim for: >> - simple&clean code, easy to maintain&extend. >> - well maintained with a maintainer interested in taking a lead helpin= g >> us integrate it into Emacs. >> >> >> Stefan >=20 > I've now tested "elunit" and converted to "ert". So far I prefer "ert" > because its available in ELPA, and the report looks like the ones I'm > used to get with JUnit or something similar. >=20 > OTOH none of the packages reported exactly where in my code my errors > where. "ert" was slightly better tough. >=20 > Just my =A40.02, to get the thread going again. Again, to quote Stefan,= we > should just pick something and run with it. I'm sure Emacs will > improve from it. I was very busy and missed this whole thread. Sorry. I assume no=20 conclusion has been reached? It seems that ert has a few supporters already. I had a look at the=20 other frameworks, and while things like mocking support and perhaps=20 elk-test's fringe markers could be nice to add on top, I am fairly=20 confident that ert has the most useful core, by which I mean basic=20 functionality like defining and running tests and presenting failures in = a (somewhat) comprehensible way. For example, it has a `should' macro to express assertions that=20 generalizes the `assert-eq', `assert-equal' etc. macros in elk-test, and = installs a debugger hook rather than catching errors with condition-case = so that it can capture backtraces of errors in tests. Features like=20 these add a little bit of complexity to ert's code, but they simplify=20 testing and debugging other code, so they are worthwhile. Unlike some other frameworks, ert does not dictate any particular style=20 of testing; its deftest is very similar to defun, and tests can be=20 stateful or functional as needed. It was designed for Emacs Lisp and=20 fits nicely into it rather than imitating frameworks from other=20 environments. It does not yet include utilities for testing Emacs state such as window = layouts and buffer contents with text properties, or user input, but=20 such utilities are orthogonal to the basic test execution framework and=20 can be added incrementally. I think Lennart has some of this already in = his fork. I don't know if the code qualifies as well-maintained; I haven't touched = it in a while, since no bugs have been reported, but I haven't been very = good at accepting patches to add features, either. (Sorry Yamato-san,=20 I'll look at your patch today.) On the other hand, the other frameworks = have been dormant for long periods as well. I have some spare time now and can offer to work on integrating ert into = Emacs; any suggestions where to start? Where does a file like ert.el=20 belong? Where should the corresponding tests go? Should I try to=20 change makefiles to let tests run automatically during build? My employer has a blanket copyright assignment. Please contact me=20 off-list if this needs further discussion. Christian.