bug#24402: should-error doesn't catch all errors (Was:bug#24402: More Information)

[Alex <agrambot@gmail.com>]

tags + 24402 confirmed
quit

Gemini Lasswell <gazally@runbox.com> writes:

> I’ve done some investigating of why this is happening. testcover-start
> transforms:
>     (should-error (my-error))
> into:
>     (should-error (testcover-after 2 (my-error)))
>
> Then the macro expansion of should-error separates the form which it
> is passed into a function symbol and list of arguments, and applies
> the function to the arguments inside of a condition-case that traps
> errors. The problem is that the arguments are evaluated first, outside
> of the condition-case, so errors in their evaluation do not get
> caught. This problem is not specific to testcover. In this example,
> the first test passes and the second fails:
>
> (defun div-by (n)
>   (/ 100 n))
>
> (defmacro log-div-by (n)
>   `(message "div-by: %d" (div-by ,n)))
>
> (ert-deftest test-div-by ()
>   (should-error (div-by 0)))
>
> (ert-deftest test-log-div-by ()
>   (should-error (log-div-by 0)))
>   

I just ran into this as well. Consider these two forms:

(should-error (cl-fourth "1234") :type 'wrong-type-argument)

(should-error (car (cdr (cdr (cdr "1234")))) :type 'wrong-type-argument)

Only the second raises an error, even though cl-fourth is equivalent to
the car/cdr chain.