ERT: How to implement expensive initialization and cleanup?

ERT: How to implement expensive initialization and cleanup?

[Jens Schmidt]

Trying to write my first ERT test.  For the tests I need to set up a 
GnuPG home directory from scratch, including directory creation, key 
import, and key trust management.  Plus I need to clean up that mess 
after the tests have finished.  Takes some 2secs on my system with all 
that process communication between Emacs and GnuPG.

The cleanest solution would be to do that for every test, but that would 
get expensive if I go for many small tests, which seems to be better in 
terms of diagnosability.

So a one-time setup at the beginning of "all tests" and a one-time 
cleanup at the end of "all tests" would be what I need, independently of 
where "all tests" begin and end: It might be really all tests when I do 
a batch run or just one test when I execute `ert-run-tests-interactively'.

I thought about some `ert-setup-hook' and `ert-cleanup-hook' executed 
e.g. in `ert-run-tests', but I haven't found anything like that. 
Besides that, this works out only if you run tests from only one "test 
set" which all share the same setup and cleanup.

Lazy initialization would also work for this kind or problem, but how 
would I trigger "lazy cleanup", then?  Just leave (in my case) the 
temporary GnuPG home directory to be cleaned up by the OS?  Or at least 
clean it up at the end of the test file, which would cover the batch 
tests, but not single tests?

Anything else?

Thanks

Re: ERT: How to implement expensive initialization and cleanup?

[Eli Zaretskii]

> Date: Sun, 4 Jun 2023 22:04:23 +0200
> From: Jens Schmidt <jschmidt4gnu@vodafonemail.de>
> 
> Trying to write my first ERT test.  For the tests I need to set up a 
> GnuPG home directory from scratch, including directory creation, key 
> import, and key trust management.  Plus I need to clean up that mess 
> after the tests have finished.  Takes some 2secs on my system with all 
> that process communication between Emacs and GnuPG.

Are these test files constant, i.e. do you need to create the same
files time and again each time the test is run?  If so, perhaps take a
look at ert-resource-directory and its friends in ert-x.el.  You will
see in our test suite several examples of tests that use such constant
data.  Data that is the same in all tests is better made part of the
test suite, so it doesn't need to be generated as part of running the
test.

> The cleanest solution would be to do that for every test, but that would 
> get expensive if I go for many small tests, which seems to be better in 
> terms of diagnosability.

I wouldn't worry about performance too much in this case.  Don't
forget that an important feature of the test suite is to be able to
run each test in the FOO-tests.el file individually.  So you'll need
to allow per-test initialization anyway.