On Wednesday, June 26, 2013 at 12:22 PM, Stefan Merten wrote:
Hi all!Being a relatively new Emacs developer my impressions might behelpful for this debate. I'm subscribed to all(?) lists but usuallycheck only the subject lines for things interesting me. Also I'mmaintaining rst.el and so far did not do anything else in Emacs.2 days ago Glenn Morris wrote:There are a lot of bugs in Emacs. [1]To me, too many of these feel like breakage of things that used to workbefore, because of implementation changes, or whatever.I share this impression. And I think it's a shame that this is so.One thing that could help reduce this is more unit tests.Definitely. Together with a build bot running tests automatically.If you haven't used it, ERT makes it pretty easy to write tests.Some time ago I started to care about rst.el really. I wanted to do alot of changes in code I did not wrote myself and today this isnothing to do without good test support. I found ert and it did mostof what I needed. It definitely is a very good foundation.Of course, many aspects of Emacs's behaviour are not easy to testIn rst.el I tested a lot of functionality which operated on buffers.Thus I wanted to know that a buffer contained a certain content afterrunning a function. I wrote a little extension based on ert whichallowed such tests easily. I even suggested this code here but the ertmaintainer did not like it and suggested a different approach I neverchecked out.I think this can be done in a more generalized way. For instance thenext thing I was thinking of was to check for correct fontification -since this is one of the major features in rst.el. I already thoughtabout ways of extending ert for this.IMHO ert is a perfect foundation but needs extensions to test standardcases where you are not (only) interested in the return values of afunction call.(GUIstuff, etc.), but many are. See test/automated/ for examples. [2]Indeed they are. Often when I browse through the documented API Iwonder why there are no unit tests for it.I know writing tests is maybe not as interesting as writing shiny newfeatures, but I think it will save work in the long run.Well, I love writing tests. May be there are more like me?Yesterday Stephen J Turnbull wrote:Does Emacs really need that high quality?Definitely. Emacs is probably the oldest Free Software project andIMHO it's a shame that the traffic on the development list is thathigh because of bugs. IMHO a good part of this could be prevented bysuitable unit tests. Interactive testing is nice but definitely notenough.That said, I encourage the core developers to lead by example,creating tests, even trivial ones, whenever possible.One obvious task would be to have unit tests for the API specified inthe info documentation. There are also good chances that thedocumentation can be improved by this.Today's generalprogramming culture is very sympathetic to testing, even to the pointof "write tests first".Definitely. Frankly speaking I was shocked when I entered the Emacscommunity and saw that in the end there are (close to) *no* tests.Testing is inherently boringwork, exploring the things that shouldn't happen, and the burden isheavy, because writing good tests requires good specs, which we rarelyhave. So designing those specs (and maybe even writing them down) isa hidden burden of testing.I disagree that testing is boring. To me it's interesting to catch allthose corner cases in tests and improve the specification or even theimplementation as a side effect. May be this is a tester approach(which I'm not ;-) ).Yesterday Lars Magne Ingebrigtsen wrote:ert is fine, but, I think, somewhat misguided. It allows us to test thefunctions we have Lisp interfaces for, but not the deep internal Cbits. And that's kinda backward.May be I do not understand Lars' intentions here, but what is testingthe C code good for at all? I mean there is an API - or rather: lotsof APIs - and *this* is what needs testing. The C code to me lookslike an implementation detail I'd not write tests for. Or at leastseparate those tests clearly from the API tests.GrüßeStefan