Re: Test suite timing issues?

[David Bremner <david@tethera.net>]

Michael J Gruber <git@grubix.eu> writes:

> Hi there,
>
> I'm trying to package notmuch for Redhat's enterprise linux and clones
> (EPEL, extra packages for enterprise linux). 
>
> This looks mostly fine, including the tests, except for intermittent
> failures on epel-8-s390x. They look like the below, or in tests
> following those, and apparantly all have to do with "db not synced yet"
> or such, so that a message one subtest creates only shows up in the db
> for the next subtest.
>
> I've also had completely fine test runs on epel-8-s390x, but I'm
> starting to wonder whether I've just been lucky so far on other
> platforms ... Could it be possible that generate_message(), even though
> adding the message, still returns false and therefore add_message() does
> not call "notmuch new"? One might want to drop the "&&" there in
> test-lib-common, I dunno. This shouldn't be "intermittent".

It's hard to see what can go wrong (maybe perl doesn't work?), but
failing to generate a message should be a fatal error. Maybe try
something like

diff --git a/test/test-lib-common.sh b/test/test-lib-common.sh
index ebbf4cdf..076777ca 100644
--- a/test/test-lib-common.sh
+++ b/test/test-lib-common.sh
@@ -225,7 +225,7 @@ EOF
 # All of the arguments and return values supported by generate_message
 # are also supported here, so see that function for details.
 add_message () {
-    generate_message "$@" &&
+    generate_message "$@" || error "failed to generate message"
     notmuch new > /dev/null
 }


to see if you get more information