ielm interfering with ert test signalling

ielm interfering with ert test signalling

[Richard H Lee]

I seem to have encountered a problem with running ert from ielm.

When running ert from M-x ert or from the scratch buffer (ert t) , if 
ert encounters an failing test, it will fail that test and carry on with 
the other tests. However when running (ert t) from ielm, when a failing 
test is reached, all testing is aborted and no more tests are carried out.

This can be reproduced by declaring a failing test .e.g. :
(ert-deftest mytest () (should nil))
and running (ert t) from ielm or *scratch* or M-x ert RET RET .

When ert aborts from ielm it gives the error:
*** Eval error ***  Test failed: ((should nil) :form nil :value nil)

The way ert works is that, when running tests it sets up an environment 
with a debugger that catches all signals and errors. This is done in the 
function ert--run-test-internal , where the debugger is set up to call 
ert--run-test-debugger, passing on some extra test data. The various 
"should" macros are then executed in these environments and the result 
of them raises various signals depending on the outcome of the test. 
These signals are trapped and interpreted by the debugger and result is 
recorded against the test.

This is what is supposed to happen and happens when you run (ert t) in 
*scratch* or M-x ert RET RET .

But when running (ert t) in ielm, what happens is that the signal from 
the "should" macro get trapped by ielm rather than ert's debugger. Hence 
ert interprets the initial fallback test result which is "test aborted".

You can see it catches the signal raised by "should" by running (should 
nil) in ielm. It will give the exact same eval error as above. Also by 
setting a flag (print or breakpoint) in ert--run-test-debugger , you can 
see that it gets hit from *scratch* or M-x ert , but not from ielm.

I even dumped the bound value of debugger in all 3 cases and it was 
correctly set to ert's debugger.

What I would like to know is that what is different in ielm's 
environment that causes it to intercept the signals to ert's debugger?