Re: Running ert tests on buffers in rst.el and elsewhere

[Christian Ohler <ohler@gnu.org>]

Stefan Merten, 2012-06-18:

> I wanted to test functions which operate on buffers - i.e. use buffer
> content as input and possibly modify the buffer. Buffer includes point
> and mark in this case. Since I found no support for this requirement
> in the ert package I wrote some support code for it.

I only glanced at your code so far, but please do take a look at the 
current version of ERT, in particular the utility functions in ert-x.el 
and the way ert-x-tests.el uses them.

Some preliminary notes:

Is it true that your code covers two features, one about turning buffer 
content, point and mark into a data structure and back as well as 
comparing such data structures, the other about providing input to 
interactive functions like completing-read?

The function `ert-run-test-with-buffer' combines both features, and I 
don't think that's a good thing; it would be better to have a more 
primitive function `ert-run-with-interactive-inputs' (or similar) that 
doesn't do any of the buffer management, and let the programmer combine 
that function with `ert-with-test-buffer' and `ert-Buf-to-buffer' as 
appropriate.

Similarly, `ert-compare-test-with-buffer' looks like it checks a bunch 
of things that should probably be left as separate `should' forms on the 
caller's side.

With functions like `ert-equal-buffer', your code introduces a notion of 
equality of buffers, and its definition seems somewhat arbitrary, so I'm 
not sure it's a good one.  For example, it doesn't take buffer-local 
variables or markers into account.  It should be easy to avoid the 
question of what the right notion of buffer equality is by letting the 
programmer extract ert-Buf data structures from buffers and compare 
those in `should' forms with some equality predicate.

However, compared to the existing approach as used in the tests 
`ert-test-run-tests-interactively-2' and `ert-test-describe-test', your 
buffer handling functions add the ability to test the position of point 
and mark, while sacrificing the ability to test text properties (and 
thus font locking).  From a high-level perspective, testing point and 
mark looks like a small feature on top of testing buffer content, so I'm 
not sure it justifies as much additional machinery as your code seems to 
add; we should look for ways to simplify things.

As a first step, could make the two features (providing interactive 
input and handling buffer content) orthogonal and send separate patches, 
perhaps simplifying the buffer content code after looking at how it's 
done in ert-x-tests.el?


> The next step I considered was to support testing font locking - or
> may be text properties in general. However, I didn't start this yet.
> It certainly would be useful.

ert-x.el does have features related to this, see 
`ert-propertized-string' and `ert-equal-including-properties'.  ERT's 
self-tests make use of them.

Christian.