Re: Proposal: Emtest as tester

[Carsten Dominik <carsten.dominik@gmail.com>]

On May 24, 2010, at 11:26 PM, Tom Breton (Tehom) wrote:

> At Carsten's request, I am proposing emtest as the tester for
> org-mode.  I would like to hear if there are any objections or
> questions.
>
> ****** About Emtest
>
> Emtest is an emacs-based test framework.  It reads tests, runs them on
> command and presents their results.  Test suites can be run by suite,
> by clause, or by library.
>
> It is extensible and modular.  Nearly everything about it can be
> replaced or extended.
>
> One important feature is its testhelp libraries:
>
> * mocks/filebuf - for making mock files and buffers to run tests in.
> * mocks/dirtree - for making mock directory trees.
> * deep-type-checker - for testing that objects, especially
>   structures, are type-correct right down to their leaves.
> * match - for pattern-matching.  When you want to test return values
>   or similar, but some fields or elements don't have stable values
>   (say, a timestamp or a UUID).
> * tagnames - extremely useful for defining test data and iterating
>   over examples.
> * testpoint - useful for:
>   1. testing functionality that is called deep inside something else,
>      where writing a viable test would mean nearly cloning the
>      something else to get the calling conditions right.
>   2. Testing functionality that uses other functionality that can't
>      be easily controlled by passing arguments.
>   3. Testing that under given circumstances a certain point is
>      reached, not reached, or reached the right number of times.
>
> Also, in less than perfect shape right now:
>
> * mocks/keystuffer - work in progress, for capturing canned user input
> * misc and standard - standard testhelp functions.  Works but
>   undergoing reorganization.
> * types - type specifications, extending what cl provides.  Right
>   now, just a few that I needed.
> * persist - useful for tests of inspected output.  Not working right
>   now due to redesign of an underlying package.
>
> ****** Some questions
>
>  * Where to include it:
>
>    * I'm proposing to put it under org-mode/testing/ So the directory
>      structure would look like:
>
>      * org-mode
>
>       	* lisp
>
>       	* (etc)
>
>       	* testing
>
> 	  * emtest
>
> 	    * Many files
>
> 	  * Some support
> 	  * packages emtest
> 	  * uses.
>
> 	  * org-agenda
>
> 	    * tests.el
>
> 	    * (And other test files)
>
> 	  * org-archive
>
> 	    * tests.el
>
> 	  * org-ascii
>
> 	  * etc (the other org files' directories of test files)
>
>       	* (other existing org directories)
>
>  * Should testing of contrib files be in a separate directory?  It's
>    not clear to me that it needs to be or should be.
>
>  * Loading.
>
>    Of course this shouldn't require much extra work to build and
>    install.  Yet there's a case to be made for not building or
>    installing it by default, "them that don't use it doesn't pay a
>    cost".
>
>    So I'm thinking I should add another target to the makefile to
>    install it, as well as (of course) a test target.

Yes, I agree.

>
>  * How to include it, git-wise.
>
>    What git wants to do with included external projects is to make
>    them submodules.  However, I'm told that's a pain to deal with,
>    moreso from the other end than from mine.  And it does seem like it
>    would be.  Basically git treats a submodule as a single thing, but
>    still "signs" it version-wise with a hex ID, and wants it to be the
>    correct version.  So git insulates you just a little bit, at the
>    cost of having to deal with an additional repository.
>
>    So I'm thinking I'd just include it literally and if that proves
>    hard to maintain then we still have the other option.

I agree with you.

- Carsten