From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Stefan Merten Newsgroups: gmane.emacs.devel Subject: Re: call for more ert tests Date: Wed, 26 Jun 2013 11:22:24 +0200 Message-ID: <8541.1372238544@theowa.merten-home.homelinux.org> References: NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha1; protocol="application/pgp-signature" X-Trace: ger.gmane.org 1372248695 25778 80.91.229.3 (26 Jun 2013 12:11:35 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Wed, 26 Jun 2013 12:11:35 +0000 (UTC) To: emacs-devel@gnu.org Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Wed Jun 26 14:11:35 2013 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1UroZS-0007VT-G0 for ged-emacs-devel@m.gmane.org; Wed, 26 Jun 2013 14:11:34 +0200 Original-Received: from localhost ([::1]:46752 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UroZS-0002yV-4R for ged-emacs-devel@m.gmane.org; Wed, 26 Jun 2013 08:11:34 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:32996) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UroZM-0002y0-Vd for emacs-devel@gnu.org; Wed, 26 Jun 2013 08:11:31 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1UroZK-0003tR-GM for emacs-devel@gnu.org; Wed, 26 Jun 2013 08:11:28 -0400 Original-Received: from moutng.kundenserver.de ([212.227.17.9]:61951) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UroZK-0003t5-4s for emacs-devel@gnu.org; Wed, 26 Jun 2013 08:11:26 -0400 Original-Received: from theowa.merten-home.homelinux.org (fra4.ip-bone.antilo.net [89.221.5.40]) by mrelayeu.kundenserver.de (node=mrbap2) with ESMTP (Nemesis) id 0MYLUv-1UndgQ4AE5-00UqGs; Wed, 26 Jun 2013 14:11:24 +0200 Original-Received: by theowa.merten-home.homelinux.org (Postfix, from userid 1000) id C88C6400AE; Wed, 26 Jun 2013 11:22:24 +0200 (CEST) Original-Received: from theowa.merten-home.homelinux.org (localhost [127.0.0.1]) by theowa.merten-home.homelinux.org (Postfix) with ESMTP id 950277A007 for ; Wed, 26 Jun 2013 11:22:24 +0200 (CEST) In-reply-to: Comments: In-reply-to Glenn Morris message dated "Mon, 24 Jun 2013 13:31:50 -0400." X-Mailer: MH-E 8.2; nmh 1.3; GNU Emacs 23.3.1 X-Provags-ID: V02:K0:A04sEZaeUlTavb9Ka7jHCzpaLJqvllNkWQZIufHXaLq a2A6tJihxfsqHsvEFx/Y425AlzismYM+HHCD4XCgU1/MlbrZU7 /nEBzptjukINmPF59hqi8iduOooaJ0Gc3w6Cz3UnM1Sonf+aTY LiztFtgZS9e5VlhiPLSXhc//PFoW8IS+Q+vzyudylC20cyoenp 4tQAtxtHbHRtFP+TLuXDYqRK5q09yZi3b104NQ4fRGiuMixaFM hR9tiN0A1AfYtevko/lqm5DgtPsZn3Cv4IqEXuQ1smglmt4k4w fbPx113bI7705LT2MXhB2BW5LA/vjv1OecjfYkvJFancnVwRpi LpaT3qJQTIMk9h3Oz57HBP2LTzRsmHbANqAtoVVSK4NwLxCjpw tzoe7Vj/z0mfDAX5CYGsr1LqPTwtUKyRAs= X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.4.x-2.6.x [generic] X-Received-From: 212.227.17.9 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.devel:161087 Archived-At: --=-=-= Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Hi all! Being a relatively new Emacs developer my impressions might be helpful for this debate. I'm subscribed to all(?) lists but usually check only the subject lines for things interesting me. Also I'm maintaining rst.el and so far did not do anything else in Emacs. 2 days ago Glenn Morris wrote: > There are a lot of bugs in Emacs. [1] > To me, too many of these feel like breakage of things that used to work > before, because of implementation changes, or whatever. I share this impression. And I think it's a shame that this is so. > One thing that could help reduce this is more unit tests. Definitely. Together with a build bot running tests automatically. > If you haven't used it, ERT makes it pretty easy to write tests. Some time ago I started to care about rst.el really. I wanted to do a lot of changes in code I did not wrote myself and today this is nothing to do without good test support. I found ert and it did most of what I needed. It definitely is a very good foundation. > Of course, many aspects of Emacs's behaviour are not easy to test In rst.el I tested a lot of functionality which operated on buffers. Thus I wanted to know that a buffer contained a certain content after running a function. I wrote a little extension based on ert which allowed such tests easily. I even suggested this code here but the ert maintainer did not like it and suggested a different approach I never checked out. I think this can be done in a more generalized way. For instance the next thing I was thinking of was to check for correct fontification - since this is one of the major features in rst.el. I already thought about ways of extending ert for this. IMHO ert is a perfect foundation but needs extensions to test standard cases where you are not (only) interested in the return values of a function call. > (GUI > stuff, etc.), but many are. See test/automated/ for examples. [2] Indeed they are. Often when I browse through the documented API I wonder why there are no unit tests for it. > I know writing tests is maybe not as interesting as writing shiny new > features, but I think it will save work in the long run. Well, I love writing tests. May be there are more like me? Yesterday Stephen J Turnbull wrote: > Does Emacs really need that high quality? Definitely. Emacs is probably the oldest Free Software project and IMHO it's a shame that the traffic on the development list is that high because of bugs. IMHO a good part of this could be prevented by suitable unit tests. Interactive testing is nice but definitely not enough. > That said, I encourage the core developers to lead by example, > creating tests, even trivial ones, whenever possible. One obvious task would be to have unit tests for the API specified in the info documentation. There are also good chances that the documentation can be improved by this. > Today's general > programming culture is very sympathetic to testing, even to the point > of "write tests first". Definitely. Frankly speaking I was shocked when I entered the Emacs community and saw that in the end there are (close to) *no* tests. > Testing is inherently boring > work, exploring the things that shouldn't happen, and the burden is > heavy, because writing good tests requires good specs, which we rarely > have. So designing those specs (and maybe even writing them down) is > a hidden burden of testing. I disagree that testing is boring. To me it's interesting to catch all those corner cases in tests and improve the specification or even the implementation as a side effect. May be this is a tester approach (which I'm not ;-) ). Yesterday Lars Magne Ingebrigtsen wrote: > ert is fine, but, I think, somewhat misguided. It allows us to test the > functions we have Lisp interfaces for, but not the deep internal C > bits. And that's kinda backward. May be I do not understand Lars' intentions here, but what is testing the C code good for at all? I mean there is an API - or rather: lots of APIs - and *this* is what needs testing. The C code to me looks like an implementation detail I'd not write tests for. Or at least separate those tests clearly from the API tests. Gr=FC=DFe Stefan --=-=-= Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) iQCVAwUBUcqyzQnTZgC3zSk5AQIvyQP/f9mH+/8jKmwZgD/VHlolJDPvJNMJrzvU fXAmnQiTPlV0QjrayKfCDC9FVha+9wHRP8fwitviGmHBgm1XEvZWF1iIFFoGM/ii PL6f4Pri3tpuMD9O/mA3P9yULO+6ON2GC02qFTNNOngt/vAebuuQKQ4fzB1v+LU5 7husTMInr0w= =dAcI -----END PGP SIGNATURE----- --=-=-=--