one more intermittent test failure

one more intermittent test failure

[David Bremner]

Hi All;

Since we've had some luck cleaning up intermittent test failures, I
thought it would be nice if we could fix this one. We've seen it on the
buildbot from time to time, and it's pretty easy to duplicate with 

make test-binaries && count=0 && while ./new; do 
count=$(( count + 1))
echo $count
done


 FAIL   Ignore files and directories specified in new.ignore (multiple occurrences)
	--- new.22.expected	2013-12-23 00:41:18.015910311 +0000
	+++ new.22.output	2013-12-23 00:41:18.015910311 +0000
	@@ -5,9 +5,6 @@
	 (D) add_files_recursive, pass 1: explicitly ignoring /home/bremner/software/upstream/notmuch/test/tmp.new/mail/one/two/ignored_file
	 (D) add_files_recursive, pass 1: explicitly ignoring /home/bremner/software/upstream/notmuch/test/tmp.new/mail/one/two/three/.git
	 (D) add_files_recursive, pass 1: explicitly ignoring /home/bremner/software/upstream/notmuch/test/tmp.new/mail/one/two/three/ignored_file
	-(D) add_files_recursive, pass 2: explicitly ignoring /home/bremner/software/upstream/notmuch/test/tmp.new/mail/.git
	-(D) add_files_recursive, pass 2: explicitly ignoring /home/bremner/software/upstream/notmuch/test/tmp.new/mail/.ignored_hidden_file
	-(D) add_files_recursive, pass 2: explicitly ignoring /home/bremner/software/upstream/notmuch/test/tmp.new/mail/ignored_file
	 (D) add_files_recursive, pass 2: explicitly ignoring /home/bremner/software/upstream/notmuch/test/tmp.new/mail/one/ignored_file
	 (D) add_files_recursive, pass 2: explicitly ignoring /home/bremner/software/upstream/notmuch/test/tmp.new/mail/one/two/ignored_file
	 (D) add_files_recursive, pass 2: explicitly ignoring /home/bremner/software/upstream/notmuch/test/tmp.new/mail/one/two/three/.git

[PATCH] test: Fix transient error in 'new' test

[Austin Clements]

This fixes a non-deterministic failure in "Ignore files and
directories specified in new.ignore (multiple occurrences)".  The test
assumed that all directories would be scanned, even though nothing
updated the mtime of ${MAIL_DIR}.  It *usually* worked nevertheless
because the tests run quickly enough that the directory mtime is
usually the same as the current time, so notmuch new does not update
the mtime in the database (because more changes could occur in the
same second).  However, when it occasionally did update the mtime in
the database, the notmuch new call in this test would (correctly) skip
"pass 2" of scanning ${MAIL_DIR}, causing it to skip the following
expected lines:

  (D) add_files_recursive, pass 2: explicitly ignoring ${MAIL_DIR}/.git
  (D) add_files_recursive, pass 2: explicitly ignoring ${MAIL_DIR}/.ignored_hidden_file
  (D) add_files_recursive, pass 2: explicitly ignoring ${MAIL_DIR}/ignored_file

This patch fixes this problem by touching ${MAIL_DIR} to ensure it
gets scanned and by rearranging the test to ensure the directories are
touched immediately before the main notmuch new call in the test.
---
 test/new | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/test/new b/test/new
index 3eff2fe..f27423d 100755
--- a/test/new
+++ b/test/new
@@ -218,9 +218,10 @@ test_expect_equal "$output" "Added 1 new message to the database."
 
 test_begin_subtest "Ignore files and directories specified in new.ignore (multiple occurrences)"
 notmuch config set new.ignore .git ignored_file .ignored_hidden_file
+notmuch new > /dev/null # ensure that files/folders will be printed in ASCII order.
 touch "${MAIL_DIR}"/.git # change .git's mtime for notmuch new to rescan.
+touch "${MAIL_DIR}"      # likewise for MAIL_DIR
 mkdir -p "${MAIL_DIR}"/one/two/three/.git
-notmuch new > /dev/null # ensure that files/folders will be printed in ASCII order.
 touch "${MAIL_DIR}"/{one,one/two,one/two/three}/ignored_file
 output=$(NOTMUCH_NEW --debug 2>&1 | sort)
 test_expect_equal "$output" \
-- 
1.8.4.rc3

Re: [PATCH] test: Fix transient error in 'new' test

[David Bremner]

Austin Clements <amdragon@MIT.EDU> writes:
> This patch fixes this problem by touching ${MAIL_DIR} to ensure it
> gets scanned and by rearranging the test to ensure the directories are
> touched immediately before the main notmuch new call in the test

This update seems to make the test reliable for me (roughly 20k
iterations).

d

Re: [PATCH] test: Fix transient error in 'new' test

[David Bremner]

>
> This patch fixes this problem by touching ${MAIL_DIR} to ensure it
> gets scanned and by rearranging the test to ensure the directories are
> touched immediately before the main notmuch new call in the test.

pushed as 0.17~rc4

d