testing macros and fixtures

[phillip.lord@russet.org.uk (Phillip Lord)]

I have been thinking about testing since it was raised in one of the
last gargantuan threads.

ERT is quite nice, but one of the things that I have found lacking is a
nice set of predicates, for use within should.

So, when I wrote test for my "lentic" package I needed some functions
like, so that I could do things like:

(should
  (test-eq-after-this
     "blah-before.txt"
     "blah-after.txt"
     (insert "hello")))

which opens "blah-before.txt" runs (insert "hello") then compares the
result with "blah-after.txt". My version of this also does a diff of the
results if the two are not equal.

I've noticed that "puppet-mode" has some thing similar. For instance:

(puppet-test-with-temp-buffer "# class
bar"
    (should (eq (puppet-test-face-at 1) 'font-lock-comment-delimiter-face))
    (should (eq (puppet-test-face-at 3) 'font-lock-comment-face))
    (should (eq (puppet-test-face-at 7) 'font-lock-comment-face))
    (should (eq (puppet-test-face-at 8) 'font-lock-comment-face))
    (should-not (puppet-test-face-at 9))))

And julia-mode has indentation checking tests like so:

   (julia--should-indent
     "
if foo
    bar
else
baz
end"
     "
if foo
    bar
else
    baz
end"))


My own experience is that these are actually quite hard to right. The
ones in lentic have never worked quite right -- that is, when it all
works they are fine, but restoring state after a crash doesn't always
work. Similarly, checking that, for example, test files are not already
open before a test is run interactively.

So, the point of my question is this; are there any good libraries
providing this kind of fixture logic? A lot of this should surely be
re-usable between different packages.

Phil