Re: [PATCH] test: reset known_broken status in test_expect_equal and test_expect_equal_file

[Dmitry Kurochkin <dmitry.kurochkin@gmail.com>]

Hi David.

On Sun, 11 Sep 2011 20:11:54 -0300, david@tethera.net wrote:
> From: David Bremner <bremner@debian.org>
> 
> Commit 4cc6727 introduced the library function
> test_subtest_known_broken which sets a variable
> test_subtest_known_broken_ . Unfortunately this variable is not reset
> if test_begin_subtest is not called before the next
> test_expect_success or test_expect_failure.
> 
> This commit remedies that, under the assumption that exactly one
> test_expect_equal or test_expect_equal_file will follow a
> test_begin_subtest
> ---
> 
> Any comments on this? I didn't follow a lot of the original
> discussions on the test API very closely. Mainly I want to know if the 
> assumption at the end of the commit message seems reasonable.
> 

IMHO this is not a good idea, because:

1. It introduces multiple places where the flag is reset.  If new
   test_expect_* functions are added in the future, there would be more
   of these.  So it brings us more complex code, code duplication, more
   chances for bugs, etc.

   This may be solved by code refactoring, but I am not sure.

2. No support for tests with multiple test_expect_* calls.  I do not
   know if it is used or works now, but the patch certainly does not
   help in this respect.

3. I thought that every test must start with a test_begin_subtest call.
   So it is the right place to put all subtest initialization code to.
   Is this not correct?  If it is correct, then I do not understand why
   we should support buggy tests by hiding (some of) their bugs.  Why do
   we need it?

Regards,
  Dmitry

>  test/test-lib.sh |    2 ++
>  1 files changed, 2 insertions(+), 0 deletions(-)
> 
> diff --git a/test/test-lib.sh b/test/test-lib.sh
> index 196ef49..3c2768c 100755
> --- a/test/test-lib.sh
> +++ b/test/test-lib.sh
> @@ -460,6 +460,7 @@ test_expect_equal ()
>  			test_failure_ "$test_subtest_name" "$(diff -u $testname.expected $testname.output)"
>  		fi
>      fi
> +       test_subtest_known_broken_=
>  }
>  
>  test_expect_equal_file ()
> @@ -483,6 +484,7 @@ test_expect_equal_file ()
>  			test_failure_ "$test_subtest_name" "$(diff -u $testname.expected $testname.output)"
>  		fi
>      fi
> +	test_subtest_known_broken_=
>  }
>  
>  NOTMUCH_NEW ()
> -- 
> 1.7.5.4
> 
> _______________________________________________
> notmuch mailing list
> notmuch@notmuchmail.org
> http://notmuchmail.org/mailman/listinfo/notmuch