ERT: should-error doesn't catch failed assertions

ERT: should-error doesn't catch failed assertions

[Sean McAfee]

I've just begin trying to use ERT to write unit tests for my code.  I
got hung up almost immediately when trying to confirm that a particular
function call fails an assertion.  A simple example:

(should-error (cl-assert nil))

The assertion failure escapes the should-error form and causes a stack
trace.

Is there a way to catch assertion failures like this?

The code I'm testing implements a checksum of sorts that can only work
with strings of length exactly 5, ie:

(def my-checksum (str)
  (cl-assert (= 5 (length str)))
  ; ...
  )

Re: ERT: should-error doesn't catch failed assertions

[Emanuel Berg]

Sean McAfee <eefacm@gmail.com> writes:

> (should-error (cl-assert nil))

What is "should-error"?

> (def my-checksum (str) (cl-assert (= 5 (length
> str))) ; ... )

With `defun' instead of "def", your code works:

    (require 'cl-lib)
    (defun my-checksum (str)
      (cl-assert (= 5 (length str)))
      )

    (my-checksum "abcdfg") ; error
    (my-checksum "abcdf")  ; nil
    (my-checksum "abcd")   ; error

Unit testing and input verification isn't
necessary but can be helpful.

In this case you can have an ordinary `if' form
with an `error' if the string length is
inconsistent with the the purpose of
the function.

Also, `condition-case' may be useful if you
don't know about it already.

Here is an article about testing - it should be
done, but not overemphasized. The best method
of testing is to have a lot of people using
your software. The second best method is for
you to use the system a lot. I guess the
various methods proposed are a distant third...

    http://user.it.uu.se/~embe8573/testing.txt

-- 
underground experts united .... http://user.it.uu.se/~embe8573
Emacs Gnus Blogomatic ......... http://user.it.uu.se/~embe8573/blogomatic
                   - so far: 58 Blogomatic articles -                   

Re: ERT: should-error doesn't catch failed assertions

[Sean McAfee]

Emanuel Berg <embe8573@student.uu.se> writes:
> Sean McAfee <eefacm@gmail.com> writes:
>
>> (should-error (cl-assert nil))
>
> What is "should-error"?

It's part of ERT, Emacs Lisp Regression Testing, mentioned in the
subject.  should-error is documented here:

https://www.gnu.org/software/emacs/manual/html_node/ert/The-should-Macro.html#The-should-Macro

As described on that page, I see (should-error (/ 1 0)) evaluate to
(arith-error).  But assertion failures seem to not count as "errors" as
far as should-error is concerned.

>> (def my-checksum (str) (cl-assert (= 5 (length
>> str))) ; ... )
>
> With `defun' instead of "def", your code works:

Ah yes, I dashed that off a little too quickly.  Apologies.

> In this case you can have an ordinary `if' form
> with an `error' if the string length is
> inconsistent with the the purpose of
> the function.

I could, but I'm a little leery of letting my testing framework dictate
how I write my code.

Re: ERT: should-error doesn't catch failed assertions

[Emanuel Berg]

Sean McAfee <eefacm@gmail.com> writes:

> As described on that page, I see (should-error
> (/ 1 0)) evaluate to (arith-error).

OK, gotcha:

    (require 'ert)
    (should-error (/ 1 0))

> But assertion failures seem to not count as
> "errors" as far as should-error is concerned.

Why do you want to combine `should-error' with
`assert' and what result is it that you expect?

    (should-error (cl-assert (/ 1 0)))

?

-- 
underground experts united .... http://user.it.uu.se/~embe8573
Emacs Gnus Blogomatic ......... http://user.it.uu.se/~embe8573/blogomatic
                   - so far: 58 Blogomatic articles -                   

Re: ERT: should-error doesn't catch failed assertions

[Noam Postavsky]

Sean McAfee <eefacm <at> gmail.com> writes:

> 
> I've just begin trying to use ERT to write unit tests for my code.  I
> got hung up almost immediately when trying to confirm that a particular
> function call fails an assertion.  A simple example:
> 
> (should-error (cl-assert nil))

Seems to work with

  (should-error (let ((debug-on-error nil)) (cl-assert nil)))

When debug-on-error is non-nil, cl-assert doesn't actually throw an error,
it just calls the debugger directly (see cl--assertion-failed).  ert binds
debug-on-error to t, somewhere along the line.