* unit test framework
@ 2010-03-18 7:39 Masatake YAMATO
2010-03-18 9:31 ` Lennart Borgman
2010-03-18 14:22 ` Stefan Monnier
0 siblings, 2 replies; 24+ messages in thread
From: Masatake YAMATO @ 2010-03-18 7:39 UTC (permalink / raw
To: emacs-devel
Hi,
Do you have a plan to include an elisp library for unit
testing in Emacs-24?
I've tried some and I'm using ert.el written by Christian M. Ohler.
http://github.com/masatake/ert/blob/master/ert.el
Masatake YAMATO
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: unit test framework
2010-03-18 7:39 unit test framework Masatake YAMATO
@ 2010-03-18 9:31 ` Lennart Borgman
2010-03-18 10:07 ` Leo
2010-03-18 14:22 ` Stefan Monnier
1 sibling, 1 reply; 24+ messages in thread
From: Lennart Borgman @ 2010-03-18 9:31 UTC (permalink / raw
To: Masatake YAMATO; +Cc: emacs-devel
On Thu, Mar 18, 2010 at 8:39 AM, Masatake YAMATO <yamato@redhat.com> wrote:
> Hi,
>
> Do you have a plan to include an elisp library for unit
> testing in Emacs-24?
>
> I've tried some and I'm using ert.el written by Christian M. Ohler.
>
> http://github.com/masatake/ert/blob/master/ert.el
I am using that in nXhtml with some additions to test fontifications
and commands.
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: unit test framework
2010-03-18 9:31 ` Lennart Borgman
@ 2010-03-18 10:07 ` Leo
2010-03-18 10:23 ` Lennart Borgman
2010-03-18 11:25 ` Masatake YAMATO
0 siblings, 2 replies; 24+ messages in thread
From: Leo @ 2010-03-18 10:07 UTC (permalink / raw
To: emacs-devel
On 2010-03-18 09:31 +0000, Lennart Borgman wrote:
> On Thu, Mar 18, 2010 at 8:39 AM, Masatake YAMATO <yamato@redhat.com> wrote:
>> Hi,
>>
>> Do you have a plan to include an elisp library for unit
>> testing in Emacs-24?
>>
>> I've tried some and I'm using ert.el written by Christian M. Ohler.
>>
>> http://github.com/masatake/ert/blob/master/ert.el
>
>
> I am using that in nXhtml with some additions to test fontifications
> and commands.
I have been thinking about using a unit test framework even for my own
.emacs which has grown to a few thousand lines. But there are so many of
them available and I don't know which one is good. Could you share why
you use ert.el? Thanks.
Leo
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: unit test framework
2010-03-18 10:07 ` Leo
@ 2010-03-18 10:23 ` Lennart Borgman
2010-03-18 11:25 ` Masatake YAMATO
1 sibling, 0 replies; 24+ messages in thread
From: Lennart Borgman @ 2010-03-18 10:23 UTC (permalink / raw
To: Leo; +Cc: emacs-devel
On Thu, Mar 18, 2010 at 11:07 AM, Leo <sdl.web@gmail.com> wrote:
> On 2010-03-18 09:31 +0000, Lennart Borgman wrote:
>> On Thu, Mar 18, 2010 at 8:39 AM, Masatake YAMATO <yamato@redhat.com> wrote:
>>> Hi,
>>>
>>> Do you have a plan to include an elisp library for unit
>>> testing in Emacs-24?
>>>
>>> I've tried some and I'm using ert.el written by Christian M. Ohler.
>>>
>>> http://github.com/masatake/ert/blob/master/ert.el
>>
>>
>> I am using that in nXhtml with some additions to test fontifications
>> and commands.
>
> I have been thinking about using a unit test framework even for my own
> .emacs which has grown to a few thousand lines. But there are so many of
> them available and I don't know which one is good. Could you share why
> you use ert.el? Thanks.
At the time I looked at it ert seemed to be the best. I do not
remember why now and maybe it is not so important since there are
alternatives now. I will not be switching now because the
modifications I have made to ert is really what I need. (If however
another unit framework gets included in Emacs I will try to switch.)
I think those who have been writing new frameworks should be those
that answers why they have done so instead of trying to make ert
better fit their needs.
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: unit test framework
2010-03-18 10:07 ` Leo
2010-03-18 10:23 ` Lennart Borgman
@ 2010-03-18 11:25 ` Masatake YAMATO
1 sibling, 0 replies; 24+ messages in thread
From: Masatake YAMATO @ 2010-03-18 11:25 UTC (permalink / raw
To: sdl.web; +Cc: emacs-devel
> On 2010-03-18 09:31 +0000, Lennart Borgman wrote:
>> On Thu, Mar 18, 2010 at 8:39 AM, Masatake YAMATO <yamato@redhat.com> wrote:
>>> Hi,
>>>
>>> Do you have a plan to include an elisp library for unit
>>> testing in Emacs-24?
>>>
>>> I've tried some and I'm using ert.el written by Christian M. Ohler.
>>>
>>> http://github.com/masatake/ert/blob/master/ert.el
>>
>>
>> I am using that in nXhtml with some additions to test fontifications
>> and commands.
>
> I have been thinking about using a unit test framework even for my own
> .emacs which has grown to a few thousand lines. But there are so many of
> them available and I don't know which one is good. Could you share why
> you use ert.el? Thanks.
I'm using it because it supports a use case, testing an elisp with "emacs --batch".
The emacs process run for unit testing can tell the test result via exit status;
0 is for successful, 1 is for failure. With ert.el it is possible to integrate
running unit test to "make check".
If you are interested in, see the new docuemnt for ``ert-run-tests-batch'' in
http://github.com/masatake/ert/commit/5f1bc45c5c54e1f95e9e8ba787f3f66b33ddc583
Masatake YAMATO
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: unit test framework
2010-03-18 7:39 unit test framework Masatake YAMATO
2010-03-18 9:31 ` Lennart Borgman
@ 2010-03-18 14:22 ` Stefan Monnier
2010-03-18 14:56 ` Lennart Borgman
2010-06-27 18:50 ` Philip Jackson
1 sibling, 2 replies; 24+ messages in thread
From: Stefan Monnier @ 2010-03-18 14:22 UTC (permalink / raw
To: Masatake YAMATO; +Cc: emacs-devel
> Do you have a plan to include an elisp library for unit
> testing in Emacs-24?
It's not in the current plan, but I'm in favor of adding such
a thing, yes. I simply don't have the time and energy to delve much
into it.
> I've tried some and I'm using ert.el written by Christian M. Ohler.
> http://github.com/masatake/ert/blob/master/ert.el
If other people can chime in on the various options and form
a consensus, that would be wonderful.
Stefan
PS: One thing for which I'd like to be able to use such a test framework, is
to link tests<->bug numbers<->commits.
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: unit test framework
2010-03-18 14:22 ` Stefan Monnier
@ 2010-03-18 14:56 ` Lennart Borgman
2010-03-19 0:16 ` rubikitch
` (4 more replies)
2010-06-27 18:50 ` Philip Jackson
1 sibling, 5 replies; 24+ messages in thread
From: Lennart Borgman @ 2010-03-18 14:56 UTC (permalink / raw
To: Stefan Monnier
Cc: Nikolaj Schumacher, web, Masatake YAMATO, ohler+emacs,
emacs-devel, rubikitch, Phil Hagelberg
On Thu, Mar 18, 2010 at 3:22 PM, Stefan Monnier
<monnier@iro.umontreal.ca> wrote:
>> Do you have a plan to include an elisp library for unit
>> testing in Emacs-24?
>
> It's not in the current plan, but I'm in favor of adding such
> a thing, yes. I simply don't have the time and energy to delve much
> into it.
>
>> I've tried some and I'm using ert.el written by Christian M. Ohler.
>> http://github.com/masatake/ert/blob/master/ert.el
>
> If other people can chime in on the various options and form
> a consensus, that would be wonderful.
I asked long ago for a comparision on EmacsWiki:
http://www.emacswiki.org/cgi-bin/wiki/UnitTesting
However there have not been any interest in that. If have cc:ed the
unit test framework authors, maybe they want to say something.
I am not sure which of them have signed papers:
ert: Christian Ohler
elunit: Phil Hagelberg
elk-test: Nikolaj Schumacher
etest: Phil Jackson
el-mock: rubikitch
> Stefan
>
>
> PS: One thing for which I'd like to be able to use such a test framework, is
> to link tests<->bug numbers<->commits.
>
>
>
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: unit test framework
2010-03-18 14:56 ` Lennart Borgman
@ 2010-03-19 0:16 ` rubikitch
2010-03-19 19:00 ` Phil Hagelberg
` (3 subsequent siblings)
4 siblings, 0 replies; 24+ messages in thread
From: rubikitch @ 2010-03-19 0:16 UTC (permalink / raw
To: lennart.borgman; +Cc: me, web, yamato, ohler+emacs, emacs-devel, phil, monnier
From: Lennart Borgman <lennart.borgman@gmail.com>
Subject: Re: unit test framework
Date: Thu, 18 Mar 2010 15:56:51 +0100
Hi,
> I asked long ago for a comparision on EmacsWiki:
>
> http://www.emacswiki.org/cgi-bin/wiki/UnitTesting
>
> However there have not been any interest in that. If have cc:ed the
> unit test framework authors, maybe they want to say something.
>
> I am not sure which of them have signed papers:
>
> ert: Christian Ohler
> elunit: Phil Hagelberg
> elk-test: Nikolaj Schumacher
> etest: Phil Jackson
> el-mock: rubikitch
I am the author of el-expectations and el-mock.
el-expectations is focused on simpleness and readability.
No assert-* functions, no setup and teardown functions.
It is the same philosophy as expectations in Ruby.
http://expectations.rubyforge.org/files/README.html
el-mock is a mock framework.
Cheers,
--
rubikitch
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: unit test framework
2010-03-18 14:56 ` Lennart Borgman
2010-03-19 0:16 ` rubikitch
@ 2010-03-19 19:00 ` Phil Hagelberg
2010-03-20 11:01 ` Nikolaj Schumacher
` (2 subsequent siblings)
4 siblings, 0 replies; 24+ messages in thread
From: Phil Hagelberg @ 2010-03-19 19:00 UTC (permalink / raw
To: Lennart Borgman
Cc: Nikolaj Schumacher, web, Masatake YAMATO, ohler+emacs,
emacs-devel, rubikitch, Stefan Monnier
On Thu, Mar 18, 2010 at 7:56 AM, Lennart Borgman
<lennart.borgman@gmail.com> wrote:
> I asked long ago for a comparision on EmacsWiki:
>
> http://www.emacswiki.org/cgi-bin/wiki/UnitTesting
>
> However there have not been any interest in that. If have cc:ed the
> unit test framework authors, maybe they want to say something.
>
> I am not sure which of them have signed papers:
>
> ert: Christian Ohler
> elunit: Phil Hagelberg
> elk-test: Nikolaj Schumacher
> etest: Phil Jackson
> el-mock: rubikitch
elunit is deprecated in favour of ert, so it should not be considered
for inclusion. When I was investigating the options (~2 years ago),
ert was the clear winner, though I don't remember the details clearly.
Mostly it had to do with having the clearest failure reports. Beyond
that there were not a lot of differences; unit testing is a pretty
straightforward task.
Things may have changed since then.
-Phil
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: unit test framework
2010-03-18 14:56 ` Lennart Borgman
2010-03-19 0:16 ` rubikitch
2010-03-19 19:00 ` Phil Hagelberg
@ 2010-03-20 11:01 ` Nikolaj Schumacher
2010-03-23 9:02 ` joakim
2010-03-23 10:15 ` Philip Jackson
4 siblings, 0 replies; 24+ messages in thread
From: Nikolaj Schumacher @ 2010-03-20 11:01 UTC (permalink / raw
To: Lennart Borgman
Cc: web, Masatake YAMATO, ohler+emacs, emacs-devel, rubikitch,
Phil Hagelberg, Stefan Monnier
On 2010-03-18 15:56 , Lennart Borgman wrote:
> elk-test: Nikolaj Schumacher
elk-test might look more complicated than it is. Most of the code is
for the user interface. The testing framework itself is just a thin
wrapper around Emacs' error signaling with better readability.
While
(assert (equal "yy" foo))
causes
(cl-assertion-failed (equal "yy" foo))
the macro
(assert-equal "yy" foo)
will cause the more readable:
(error "assert-equal for <foo> failed: expected <yy>, was <xx>")
`assert-that' can take care of the formatting for custom tests, and
`assert-error' fails when a specific error isn't thrown.
The nice thing about these macros is that they work in any evaluation
context. I'd vote to include such macros at the least.
The
(deftest "test name" ...
macro just provides an name anchor for running individual tests. And
the test runner will just catch errors and display them as text,
highlight where the error occurred, and mark the buffer as "red". This
GUI is more designed for test-driven development than automatic
regression tests, but the macros would work fine for that, I suppose.
kind regards,
Nikolaj Schumacher
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: unit test framework
2010-03-18 14:56 ` Lennart Borgman
` (2 preceding siblings ...)
2010-03-20 11:01 ` Nikolaj Schumacher
@ 2010-03-23 9:02 ` joakim
2010-03-23 13:15 ` Lennart Borgman
2010-03-23 14:22 ` Stefan Monnier
2010-03-23 10:15 ` Philip Jackson
4 siblings, 2 replies; 24+ messages in thread
From: joakim @ 2010-03-23 9:02 UTC (permalink / raw
To: Lennart Borgman
Cc: Nikolaj Schumacher, web, Masatake YAMATO, ohler+emacs,
emacs-devel, rubikitch, Stefan Monnier, Phil Hagelberg
Lennart Borgman <lennart.borgman@gmail.com> writes:
> On Thu, Mar 18, 2010 at 3:22 PM, Stefan Monnier
> <monnier@iro.umontreal.ca> wrote:
>>> Do you have a plan to include an elisp library for unit
>>> testing in Emacs-24?
>>
>> It's not in the current plan, but I'm in favor of adding such
>> a thing, yes. I simply don't have the time and energy to delve much
>> into it.
>>
>>> I've tried some and I'm using ert.el written by Christian M. Ohler.
>>> http://github.com/masatake/ert/blob/master/ert.el>
>> If other people can chime in on the various options and form
>> a consensus, that would be wonderful.
>
>
> I asked long ago for a comparision on EmacsWiki:
>
> http://www.emacswiki.org/cgi-bin/wiki/UnitTesting
>
> However there have not been any interest in that. If have cc:ed the
> unit test framework authors, maybe they want to say something.
>
> I am not sure which of them have signed papers:
>
> ert: Christian Ohler
> elunit: Phil Hagelberg
> elk-test: Nikolaj Schumacher
> etest: Phil Jackson
> el-mock: rubikitch
>
>
>> Stefan
>>
>>
>> PS: One thing for which I'd like to be able to use such a test framework, is
>> to link tests<->bug numbers<->commits.
I write try to write unit tests for my elisp, and it would be much
easier if there was a canonical unit test framework in Emacs.
I have written some tests for CEDET, which doesnt really use a
framework, and should be included in Emacs(I havent looked at the merged
version yet). I have also written some tests in elunit for other
packages.
I'm currently writing some code that would benefit from simulation of
keyboard input, in particular code that uses read, read-number,
completing-read etc. Some of the contenders listed above surely does
this, right?
Anyway, maybe we should make a list of requirements for a unit test
package for inclusion. Heres my take:
- possibility to generate unit tests reports in test runs during "make"
- at least rudimentary user input simulation, to test read, read-number,
completing-read etc.
- some way of isolating test runs from normal operations
- good error reporting, so that you dont have to spend hours in edebug needlessly
- convenient way of defining and running tests, in group or one-by-one
--
Joakim Verona
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: unit test framework
2010-03-18 14:56 ` Lennart Borgman
` (3 preceding siblings ...)
2010-03-23 9:02 ` joakim
@ 2010-03-23 10:15 ` Philip Jackson
4 siblings, 0 replies; 24+ messages in thread
From: Philip Jackson @ 2010-03-23 10:15 UTC (permalink / raw
To: Lennart Borgman
Cc: Nikolaj Schumacher, web, Masatake YAMATO, ohler+emacs,
emacs-devel, rubikitch, Stefan Monnier, Phil Hagelberg
At Thu, 18 Mar 2010 15:56:51 +0100,
Lennart Borgman wrote:
>>> Do you have a plan to include an elisp library for unit
>>> testing in Emacs-24?
> I asked long ago for a comparision on EmacsWiki:
>
> http://www.emacswiki.org/cgi-bin/wiki/UnitTesting
>
> However there have not been any interest in that. If have cc:ed the
> unit test framework authors, maybe they want to say something.
>
> I am not sure which of them have signed papers:
>
> ert: Christian Ohler
> elunit: Phil Hagelberg
> elk-test: Nikolaj Schumacher
> etest: Phil Jackson
> el-mock: rubikitch
I wrote etest:
http://www.shellarchive.co.uk/content/etest.html
http://www.flickr.com/photos/phil-jackson/2702854411/
I've signed the papers, would be happy for its inclusion but have to
admit I've not done much of a comparison with the other tools.
Cheers,
Phil
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: unit test framework
2010-03-23 9:02 ` joakim
@ 2010-03-23 13:15 ` Lennart Borgman
2010-03-24 3:03 ` Masatake YAMATO
2010-03-23 14:22 ` Stefan Monnier
1 sibling, 1 reply; 24+ messages in thread
From: Lennart Borgman @ 2010-03-23 13:15 UTC (permalink / raw
To: joakim
Cc: Nikolaj Schumacher, web, Masatake YAMATO, ohler+emacs,
emacs-devel, rubikitch, Stefan Monnier, Phil Hagelberg
On Tue, Mar 23, 2010 at 10:02 AM, <joakim@verona.se> wrote:
>
> I write try to write unit tests for my elisp, and it would be much
> easier if there was a canonical unit test framework in Emacs.
>
> I have written some tests for CEDET, which doesnt really use a
> framework, and should be included in Emacs(I havent looked at the merged
> version yet). I have also written some tests in elunit for other
> packages.
>
> I'm currently writing some code that would benefit from simulation of
> keyboard input, in particular code that uses read, read-number,
> completing-read etc. Some of the contenders listed above surely does
> this, right?
>
> Anyway, maybe we should make a list of requirements for a unit test
> package for inclusion. Heres my take:
>
> - possibility to generate unit tests reports in test runs during "make"
>
> - at least rudimentary user input simulation, to test read, read-number,
> completing-read etc.
I have written some extensions to ert.el that allows running commands
in the test function (ie calling relevant hooks etc).
> - some way of isolating test runs from normal operations
I have some small things that starts a new Emacs instance for the tests.
> - good error reporting, so that you dont have to spend hours in edebug needlessly
ert.el have good interactive reporting (that needs some usability
enhancements, like menu entries), but not so much for saving to file
(or maybe I have just missed that part).
> - convenient way of defining and running tests, in group or one-by-one
ert.el does it by regexp matching the names of the test functions.
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: unit test framework
2010-03-23 9:02 ` joakim
2010-03-23 13:15 ` Lennart Borgman
@ 2010-03-23 14:22 ` Stefan Monnier
2010-04-13 10:32 ` joakim
1 sibling, 1 reply; 24+ messages in thread
From: Stefan Monnier @ 2010-03-23 14:22 UTC (permalink / raw
To: joakim
Cc: Nikolaj Schumacher, web, Lennart Borgman, Masatake YAMATO,
ohler+emacs, emacs-devel, rubikitch, Phil Hagelberg
> I write try to write unit tests for my elisp, and it would be much
> easier if there was a canonical unit test framework in Emacs.
Yes, that's the intention. We should pick one and run with it.
The choice doesn't have to be "the very best", but it should aim for:
- simple&clean code, easy to maintain&extend.
- well maintained with a maintainer interested in taking a lead helping
us integrate it into Emacs.
Stefan
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: unit test framework
2010-03-23 13:15 ` Lennart Borgman
@ 2010-03-24 3:03 ` Masatake YAMATO
0 siblings, 0 replies; 24+ messages in thread
From: Masatake YAMATO @ 2010-03-24 3:03 UTC (permalink / raw
To: lennart.borgman
Cc: me, web, phil, ohler+emacs, emacs-devel, rubikitch, joakim,
monnier
> On Tue, Mar 23, 2010 at 10:02 AM, <joakim@verona.se> wrote:
>>
>> I write try to write unit tests for my elisp, and it would be much
>> easier if there was a canonical unit test framework in Emacs.
>>
>> I have written some tests for CEDET, which doesnt really use a
>> framework, and should be included in Emacs(I havent looked at the merged
>> version yet). I have also written some tests in elunit for other
>> packages.
>>
>> I'm currently writing some code that would benefit from simulation of
>> keyboard input, in particular code that uses read, read-number,
>> completing-read etc. Some of the contenders listed above surely does
>> this, right?
>>
>> Anyway, maybe we should make a list of requirements for a unit test
>> package for inclusion. Heres my take:
>>
>> - possibility to generate unit tests reports in test runs during "make"
I've extended ert.el for the purpose.
http://github.com/masatake/ert/commit/5f1bc45c5c54e1f95e9e8ba787f3f66b33ddc583
The patch was improved based on the comment from the original author,
Christian M. Ohler. I asked him to merge my patch to his source tree, but I have got
no response yet.
>> - at least rudimentary user input simulation, to test read, read-number,
>> completing-read etc.
>
>
> I have written some extensions to ert.el that allows running commands
> in the test function (ie calling relevant hooks etc).
>
>
>> - some way of isolating test runs from normal operations
>
>
> I have some small things that starts a new Emacs instance for the tests.
>
>
>
>> - good error reporting, so that you dont have to spend hours in edebug needlessly
>
>
> ert.el have good interactive reporting (that needs some usability
> enhancements, like menu entries), but not so much for saving to file
> (or maybe I have just missed that part).
>
>
>> - convenient way of defining and running tests, in group or one-by-one
>
>
> ert.el does it by regexp matching the names of the test functions.
>
>
Masatake YAMATO
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: unit test framework
2010-03-23 14:22 ` Stefan Monnier
@ 2010-04-13 10:32 ` joakim
2010-06-12 6:06 ` Christian Ohler
0 siblings, 1 reply; 24+ messages in thread
From: joakim @ 2010-04-13 10:32 UTC (permalink / raw
To: Stefan Monnier
Cc: Nikolaj Schumacher, web, Lennart Borgman, Masatake YAMATO,
ohler+emacs, emacs-devel, rubikitch, Phil Hagelberg
Stefan Monnier <monnier@IRO.UMontreal.CA> writes:
>> I write try to write unit tests for my elisp, and it would be much
>> easier if there was a canonical unit test framework in Emacs.
>
> Yes, that's the intention. We should pick one and run with it.
> The choice doesn't have to be "the very best", but it should aim for:
> - simple&clean code, easy to maintain&extend.
> - well maintained with a maintainer interested in taking a lead helping
> us integrate it into Emacs.
>
>
> Stefan
I've now tested "elunit" and converted to "ert". So far I prefer "ert"
because its available in ELPA, and the report looks like the ones I'm
used to get with JUnit or something similar.
OTOH none of the packages reported exactly where in my code my errors
where. "ert" was slightly better tough.
Just my €0.02, to get the thread going again. Again, to quote Stefan, we
should just pick something and run with it. I'm sure Emacs will
improve from it.
--
Joakim Verona
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: unit test framework
2010-04-13 10:32 ` joakim
@ 2010-06-12 6:06 ` Christian Ohler
2010-06-12 13:48 ` Lennart Borgman
` (2 more replies)
0 siblings, 3 replies; 24+ messages in thread
From: Christian Ohler @ 2010-06-12 6:06 UTC (permalink / raw
To: Stefan Monnier
Cc: Nikolaj Schumacher, web, Lennart Borgman, Masatake YAMATO, joakim,
emacs-devel, rubikitch, Phil Hagelberg
joakim@verona.se, 2010-04-13:
> Stefan Monnier <monnier@IRO.UMontreal.CA> writes:
>
>>> I write try to write unit tests for my elisp, and it would be much
>>> easier if there was a canonical unit test framework in Emacs.
>> Yes, that's the intention. We should pick one and run with it.
>> The choice doesn't have to be "the very best", but it should aim for:
>> - simple&clean code, easy to maintain&extend.
>> - well maintained with a maintainer interested in taking a lead helping
>> us integrate it into Emacs.
>>
>>
>> Stefan
>
> I've now tested "elunit" and converted to "ert". So far I prefer "ert"
> because its available in ELPA, and the report looks like the ones I'm
> used to get with JUnit or something similar.
>
> OTOH none of the packages reported exactly where in my code my errors
> where. "ert" was slightly better tough.
>
> Just my €0.02, to get the thread going again. Again, to quote Stefan, we
> should just pick something and run with it. I'm sure Emacs will
> improve from it.
I was very busy and missed this whole thread. Sorry. I assume no
conclusion has been reached?
It seems that ert has a few supporters already. I had a look at the
other frameworks, and while things like mocking support and perhaps
elk-test's fringe markers could be nice to add on top, I am fairly
confident that ert has the most useful core, by which I mean basic
functionality like defining and running tests and presenting failures in
a (somewhat) comprehensible way.
For example, it has a `should' macro to express assertions that
generalizes the `assert-eq', `assert-equal' etc. macros in elk-test, and
installs a debugger hook rather than catching errors with condition-case
so that it can capture backtraces of errors in tests. Features like
these add a little bit of complexity to ert's code, but they simplify
testing and debugging other code, so they are worthwhile.
Unlike some other frameworks, ert does not dictate any particular style
of testing; its deftest is very similar to defun, and tests can be
stateful or functional as needed. It was designed for Emacs Lisp and
fits nicely into it rather than imitating frameworks from other
environments.
It does not yet include utilities for testing Emacs state such as window
layouts and buffer contents with text properties, or user input, but
such utilities are orthogonal to the basic test execution framework and
can be added incrementally. I think Lennart has some of this already in
his fork.
I don't know if the code qualifies as well-maintained; I haven't touched
it in a while, since no bugs have been reported, but I haven't been very
good at accepting patches to add features, either. (Sorry Yamato-san,
I'll look at your patch today.) On the other hand, the other frameworks
have been dormant for long periods as well.
I have some spare time now and can offer to work on integrating ert into
Emacs; any suggestions where to start? Where does a file like ert.el
belong? Where should the corresponding tests go? Should I try to
change makefiles to let tests run automatically during build?
My employer has a blanket copyright assignment. Please contact me
off-list if this needs further discussion.
Christian.
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: unit test framework
2010-06-12 6:06 ` Christian Ohler
@ 2010-06-12 13:48 ` Lennart Borgman
2010-06-14 11:24 ` Masatake YAMATO
2010-07-23 21:58 ` Stefan Monnier
2 siblings, 0 replies; 24+ messages in thread
From: Lennart Borgman @ 2010-06-12 13:48 UTC (permalink / raw
To: Christian Ohler
Cc: Nikolaj Schumacher, web, Masatake YAMATO, joakim, emacs-devel,
rubikitch, Stefan Monnier, Phil Hagelberg
On Sat, Jun 12, 2010 at 8:06 AM, Christian Ohler
<ohler+emacs@fastmail.net> wrote:
>
> It does not yet include utilities for testing Emacs state such as window
> layouts and buffer contents with text properties, or user input, but such
> utilities are orthogonal to the basic test execution framework and can be
> added incrementally. I think Lennart has some of this already in his fork.
My additions are for running commands (i.e. running the necessary
hooks in hopefully correct order) and for testing of fontification.
There is also a little bit for adding a test input file for a test.
And a bit for keeping track of the test name.
But I suggest you review my code and tweak it to your liking. Please
contact me off list.
> I have some spare time now and can offer to work on integrating ert into
> Emacs; any suggestions where to start? Where does a file like ert.el
> belong? Where should the corresponding tests go?
There is already a sub-directory for tests, "emacs/test". To my
surprise the elisp files for test also is there. I suggest these
should be somewhere like "emacs/lisp/test" instead and that
"emacs/test" will be for data files etc instead.
An objection to this might be that the tests can't be run without the
data. Yes, but the code in the elisp files can serve as examples for
those writing tests.
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: unit test framework
2010-06-12 6:06 ` Christian Ohler
2010-06-12 13:48 ` Lennart Borgman
@ 2010-06-14 11:24 ` Masatake YAMATO
2010-07-23 21:58 ` Stefan Monnier
2 siblings, 0 replies; 24+ messages in thread
From: Masatake YAMATO @ 2010-06-14 11:24 UTC (permalink / raw
To: ohler+emacs
Cc: me, web, lennart.borgman, joakim, emacs-devel, rubikitch, monnier,
phil
Christian,
> I don't know if the code qualifies as well-maintained; I haven't
> touched it in a while, since no bugs have been reported, but I haven't
> been very good at accepting patches to add features, either. (Sorry
> Yamato-san, I'll look at your patch today.) On the other hand, the
> other frameworks have been dormant for long periods as well.
I've confirmed that my patch is include in you tree. Thanks.
As I wrote repeatedly supporting batch mode is a big advantage of
ert.
Masatake YAMATO
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: unit test framework
2010-03-18 14:22 ` Stefan Monnier
2010-03-18 14:56 ` Lennart Borgman
@ 2010-06-27 18:50 ` Philip Jackson
2010-07-09 13:43 ` Christian Ohler
1 sibling, 1 reply; 24+ messages in thread
From: Philip Jackson @ 2010-06-27 18:50 UTC (permalink / raw
To: Stefan Monnier; +Cc: Masatake YAMATO, emacs-devel
At Thu, 18 Mar 2010 10:22:42 -0400,
Stefan Monnier wrote:
>
> > Do you have a plan to include an elisp library for unit
> > testing in Emacs-24?
>
> It's not in the current plan, but I'm in favor of adding such
> a thing, yes. I simply don't have the time and energy to delve much
> into it.
>
> > I've tried some and I'm using ert.el written by Christian M. Ohler.
> > http://github.com/masatake/ert/blob/master/ert.el
>
> If other people can chime in on the various options and form
> a consensus, that would be wonderful.
It looks like ert has the most support. How about running with it?
Cheers,
Phil
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: unit test framework
2010-06-27 18:50 ` Philip Jackson
@ 2010-07-09 13:43 ` Christian Ohler
2010-07-09 23:01 ` Glenn Morris
0 siblings, 1 reply; 24+ messages in thread
From: Christian Ohler @ 2010-07-09 13:43 UTC (permalink / raw
To: Philip Jackson; +Cc: emacs-devel
Philip Jackson, 2010-06-28:
> At Thu, 18 Mar 2010 10:22:42 -0400,
> Stefan Monnier wrote:
>>> Do you have a plan to include an elisp library for unit
>>> testing in Emacs-24?
>> It's not in the current plan, but I'm in favor of adding such
>> a thing, yes. I simply don't have the time and energy to delve much
>> into it.
>>
>>> I've tried some and I'm using ert.el written by Christian M. Ohler.
>>> http://github.com/masatake/ert/blob/master/ert.el
>> If other people can chime in on the various options and form
>> a consensus, that would be wonderful.
>
> It looks like ert has the most support. How about running with it?
I think that is pretty much the plan. I volunteered to do the necessary
work and have been cleaning up the code.
Unfortunately, I still don't have a copyright assignment; assign at gnu
dot org has yet to respond to the request that I sent and re-sent
several times over the last few weeks. Does anyone have a way to find
out what's going on there?
Christian.
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: unit test framework
2010-07-09 13:43 ` Christian Ohler
@ 2010-07-09 23:01 ` Glenn Morris
2010-07-10 1:12 ` Christian Ohler
0 siblings, 1 reply; 24+ messages in thread
From: Glenn Morris @ 2010-07-09 23:01 UTC (permalink / raw
To: Christian Ohler; +Cc: Philip Jackson, emacs-devel
Christian Ohler wrote:
> Unfortunately, I still don't have a copyright assignment; assign at
> gnu dot org has yet to respond to the request that I sent and re-sent
> several times over the last few weeks. Does anyone have a way to find
> out what's going on there?
What are you trying to do? If you are just trying to get started, I
can send you the first (electronic) form to fill in. If you have
already done that and are waiting to get a paper form, I am afraid I
cannot help. Sometimes assign@gnu does seem to get busy.
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: unit test framework
2010-07-09 23:01 ` Glenn Morris
@ 2010-07-10 1:12 ` Christian Ohler
0 siblings, 0 replies; 24+ messages in thread
From: Christian Ohler @ 2010-07-10 1:12 UTC (permalink / raw
To: Glenn Morris; +Cc: Philip Jackson, emacs-devel
Glenn Morris, 2010-07-10:
> What are you trying to do? If you are just trying to get started, I
> can send you the first (electronic) form to fill in. If you have
> already done that and are waiting to get a paper form, I am afraid I
> cannot help. Sometimes assign@gnu does seem to get busy.
Thanks, I have the electronic form. The paper form is what I'm waiting
for, sorry for not being clear. I guess I'll just keep waiting.
Christian.
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: unit test framework
2010-06-12 6:06 ` Christian Ohler
2010-06-12 13:48 ` Lennart Borgman
2010-06-14 11:24 ` Masatake YAMATO
@ 2010-07-23 21:58 ` Stefan Monnier
2 siblings, 0 replies; 24+ messages in thread
From: Stefan Monnier @ 2010-07-23 21:58 UTC (permalink / raw
To: Christian Ohler
Cc: Nikolaj Schumacher, web, Lennart Borgman, Masatake YAMATO, joakim,
emacs-devel, rubikitch, Phil Hagelberg
> I have some spare time now and can offer to work on integrating ert into
> Emacs; any suggestions where to start?
Try integrating it, provide some example tests in emacs/test.
> Where does a file like ert.el belong?
In emacs/lisp/emacs-lisp, I think.
> Where should the corresponding tests go?
We've been using emacs/test for the (very) few tests we have, so that's
where they should go.
> Should I try to change makefiles to let tests run automatically
> during build?
I don't think so. You can just try and make "cd test; make" do
something useful. We can later on decide whether running tests shojuld
be performed as part of routine compilations.
Stefan
^ permalink raw reply [flat|nested] 24+ messages in thread
end of thread, other threads:[~2010-07-23 21:58 UTC | newest]
Thread overview: 24+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2010-03-18 7:39 unit test framework Masatake YAMATO
2010-03-18 9:31 ` Lennart Borgman
2010-03-18 10:07 ` Leo
2010-03-18 10:23 ` Lennart Borgman
2010-03-18 11:25 ` Masatake YAMATO
2010-03-18 14:22 ` Stefan Monnier
2010-03-18 14:56 ` Lennart Borgman
2010-03-19 0:16 ` rubikitch
2010-03-19 19:00 ` Phil Hagelberg
2010-03-20 11:01 ` Nikolaj Schumacher
2010-03-23 9:02 ` joakim
2010-03-23 13:15 ` Lennart Borgman
2010-03-24 3:03 ` Masatake YAMATO
2010-03-23 14:22 ` Stefan Monnier
2010-04-13 10:32 ` joakim
2010-06-12 6:06 ` Christian Ohler
2010-06-12 13:48 ` Lennart Borgman
2010-06-14 11:24 ` Masatake YAMATO
2010-07-23 21:58 ` Stefan Monnier
2010-03-23 10:15 ` Philip Jackson
2010-06-27 18:50 ` Philip Jackson
2010-07-09 13:43 ` Christian Ohler
2010-07-09 23:01 ` Glenn Morris
2010-07-10 1:12 ` Christian Ohler
Code repositories for project(s) associated with this external index
https://git.savannah.gnu.org/cgit/emacs.git
https://git.savannah.gnu.org/cgit/emacs/org-mode.git
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.