TDD and BDD for emacs-lisp

TDD and BDD for emacs-lisp

[Xavier Maillard]

Hello,

for my next project (an IM tool), I would like to try to embrace TDD/BDD
methodology. I have never done this before and I even do not know the way I
will have to change my coding habits.

I have looked here and there to see if it is something emacs-lisp hackers
do, it seems that it has not spread much.

So, what tool(s) do you use if you do your devs using TDD/BDD ? Do you have
simple examples to share ? Is it recommended to do both ? Where does one
shine more than another ? etc. (I will probably have to read tons of
documentation but that's fine).

With ERT shipped with GNU emacs >= 24, there is no reason not to use it but
for BDD, I can see at least 2 (good) tools: ecukes and buttercup. Is there
any comparison chart somewhere ? 

Any help would be appreciated to start using TDD/BDD.

Thank you
Xavier

Re: TDD and BDD for emacs-lisp

[Phillip Lord]

Xavier Maillard <xavier@maillard.im> writes:

> Hello,
>
> for my next project (an IM tool), I would like to try to embrace TDD/BDD
> methodology. I have never done this before and I even do not know the way I
> will have to change my coding habits.
>
> I have looked here and there to see if it is something emacs-lisp hackers
> do, it seems that it has not spread much.
>
> So, what tool(s) do you use if you do your devs using TDD/BDD ? Do you have
> simple examples to share ? Is it recommended to do both ? Where does one
> shine more than another ? etc. (I will probably have to read tons of
> documentation but that's fine).
>
> With ERT shipped with GNU emacs >= 24, there is no reason not to use it but
> for BDD, I can see at least 2 (good) tools: ecukes and buttercup. Is there
> any comparison chart somewhere ? 
>
> Any help would be appreciated to start using TDD/BDD.


I am a Cask (http://github.com/cask/cask/) junkie which provides really
nice command line invocation of tests, supports multiple versions of
Emacs, and runs in a headless environment. I normally run it inside
M-x compile.

Testing, I use ert.el. The framework is nice, but the reporting can be
poor, and keeping tests independent is hard work (that's Emacs' fault
rather than ert, I feel). So if I may be excused the plug, I've written
assess (https://github.com/phillord/assess) which adds better reporting,
macros for keeping Emacs tests free of side-effects, and adds some nice
predicates (for roundtripping indentation, testing font-lock and so on).
Probably, assess will go into core for Emacs-26 as ert-assess.

ecukes and buttercup, I cannot really vouch for one way or the
other. The whole BDD doesn't entirely make sense to me.

Phil

Re: TDD and BDD for emacs-lisp

[Xavier Maillard]

phillip.lord@russet.org.uk (Phillip Lord) writes:

> Xavier Maillard <xavier@maillard.im> writes:
>
>> Hello,
>>
>> for my next project (an IM tool), I would like to try to embrace TDD/BDD
>> methodology. I have never done this before and I even do not know the way I
>> will have to change my coding habits.
>>
>> I have looked here and there to see if it is something emacs-lisp hackers
>> do, it seems that it has not spread much.
>>
>> So, what tool(s) do you use if you do your devs using TDD/BDD ? Do you have
>> simple examples to share ? Is it recommended to do both ? Where does one
>> shine more than another ? etc. (I will probably have to read tons of
>> documentation but that's fine).
>>
>> With ERT shipped with GNU emacs >= 24, there is no reason not to use it but
>> for BDD, I can see at least 2 (good) tools: ecukes and buttercup. Is there
>> any comparison chart somewhere ? 
>>
>> Any help would be appreciated to start using TDD/BDD.
>
> I am a Cask (http://github.com/cask/cask/) junkie which provides really
> nice command line invocation of tests, supports multiple versions of
> Emacs, and runs in a headless environment. I normally run it inside
> M-x compile.

I am too. I just discovered Cask weeks ago and I am totally convinced and
converted user. I did not think to « M-x compile »-it but that's a great
idea. Thanks for this tip.


> Testing, I use ert.el. The framework is nice, but the reporting can be
> poor, and keeping tests independent is hard work (that's Emacs' fault
> rather than ert, I feel). So if I may be excused the plug, I've written
> assess (https://github.com/phillord/assess) which adds better reporting,
> macros for keeping Emacs tests free of side-effects, and adds some nice
> predicates (for roundtripping indentation, testing font-lock and so on).
> Probably, assess will go into core for Emacs-26 as ert-assess.

That's great.

> ecukes and buttercup, I cannot really vouch for one way or the
> other. The whole BDD doesn't entirely make sense to me.

Can you explain why ? As far as I read it, it seems one is « technical »
centric while the other is more « business » oriented which makes sense to
me. Also, it seems BDD is just a derived TDD more human oriented.

> Phil

Thanks for your time and suport

Xavier

Re: TDD and BDD for emacs-lisp

[Pascal J. Bourguignon]

Xavier Maillard <xavier@maillard.im> writes:

> Hello,
>
> for my next project (an IM tool), I would like to try to embrace TDD/BDD
> methodology. I have never done this before and I even do not know the way I
> will have to change my coding habits.


For unit tests, there are various emacs lisp test frameworks.  It's
rather easy to write your own too.

https://www.emacswiki.org/emacs/UnitTesting

However, if you want to perform integration tests, since you are
implementing a communication tool, you will have to send data from one
end, and receive it on another end, so, in another process, and check
that it has been correctly received, etc.

Once I had to test a MTA (SMTP filter), and I was in that situation,
with messages composed and sent from emacs, and received in another
process.  You may want to filter and test the logs too (check for error
messages or other events).

In this situation, I know of no test framework helping, and therefore I
tend to write my own test modules or tools, specific to each situation.

Sorry, I cannot help you with code, it was a long time ago, wikis, DCVS
and things like elpa didn't exist yet!


Reusable frameworks are nice when you are doing repeatitively the same
kind of work…


> I have looked here and there to see if it is something emacs-lisp hackers
> do, it seems that it has not spread much.
>
> So, what tool(s) do you use if you do your devs using TDD/BDD ? Do you have
> simple examples to share ? Is it recommended to do both ? Where does one
> shine more than another ? etc. (I will probably have to read tons of
> documentation but that's fine).
>
> With ERT shipped with GNU emacs >= 24, there is no reason not to use it but
> for BDD, I can see at least 2 (good) tools: ecukes and buttercup. Is there
> any comparison chart somewhere ? 

ERT is made to unit test emacs lisp code. I doubt it would be convenient
to use when you have asynchronous and communicating processes. So it
will be applicable only partially (on small local unit tests).


> Any help would be appreciated to start using TDD/BDD.


-- 
__Pascal Bourguignon__                 http://www.informatimago.com/
“The factory of the future will have only two employees, a man and a
dog. The man will be there to feed the dog. The dog will be there to
keep the man from touching the equipment.” -- Carl Bass CEO Autodesk

Re: TDD and BDD for emacs-lisp

[Xavier Maillard]

"Pascal J. Bourguignon" <pjb@informatimago.com> writes:

> Xavier Maillard <xavier@maillard.im> writes:
>
>> Hello,
>>
>> for my next project (an IM tool), I would like to try to embrace TDD/BDD
>> methodology. I have never done this before and I even do not know the way I
>> will have to change my coding habits.
>
> For unit tests, there are various emacs lisp test frameworks.  It's
> rather easy to write your own too.

Yes, but the idea is not really to reinvent the wheel for me and I'd rather
stick with existing (good) projects if possible.

> However, if you want to perform integration tests, since you are
> implementing a communication tool, you will have to send data from one
> end, and receive it on another end, so, in another process, and check
> that it has been correctly received, etc.

That's the idea. I thought I could /fake/ some part of the tests with
stubs/mocks (I am not sure I understand the difference between these two
beasts though) so that I could simulate data exchange with the external
service. Am I wrong ? Do I really have to test the whole chain ?

> Once I had to test a MTA (SMTP filter), and I was in that situation,
> with messages composed and sent from emacs, and received in another
> process.  You may want to filter and test the logs too (check for error
> messages or other events).

Yes, good point.

> In this situation, I know of no test framework helping, and therefore I
> tend to write my own test modules or tools, specific to each situation.

Makes sense.

> Sorry, I cannot help you with code, it was a long time ago, wikis, DCVS
> and things like elpa didn't exist yet!

Oh man, are you that ... o . l . d ? (kidding :D)

[ snip ]

>> With ERT shipped with GNU emacs >= 24, there is no reason not to use it but
>> for BDD, I can see at least 2 (good) tools: ecukes and buttercup. Is there
>> any comparison chart somewhere ? 
>
> ERT is made to unit test emacs lisp code. I doubt it would be convenient
> to use when you have asynchronous and communicating processes. So it
> will be applicable only partially (on small local unit tests).

That's the idea. Given I'd rather starting with BDD and then TDD, that's not a
problem. I also want these tests to be the actual documentation/spec for
end-user (not a manual though).

Thank you Pascal.