Re: Test suite timing issues?

[Tomi Ollila <tomi.ollila@iki.fi>]

On Sat, Feb 12 2022, Michael J. Gruber wrote:

> David Bremner venit, vidit, dixit 2022-02-12 01:03:00:
>> Michael J Gruber <git@grubix.eu> writes:
>> 
>> > Hi there,
>> >
>> > I'm trying to package notmuch for Redhat's enterprise linux and clones
>> > (EPEL, extra packages for enterprise linux). 
>> >
>> > This looks mostly fine, including the tests, except for intermittent
>> > failures on epel-8-s390x. They look like the below, or in tests
>> > following those, and apparantly all have to do with "db not synced yet"
>> > or such, so that a message one subtest creates only shows up in the db
>> > for the next subtest.
>> >
>> > I've also had completely fine test runs on epel-8-s390x, but I'm
>> > starting to wonder whether I've just been lucky so far on other
>> > platforms ... Could it be possible that generate_message(), even though
>> > adding the message, still returns false and therefore add_message() does
>> > not call "notmuch new"? One might want to drop the "&&" there in
>> > test-lib-common, I dunno. This shouldn't be "intermittent".
>> 
>> It's hard to see what can go wrong (maybe perl doesn't work?), but
>> failing to generate a message should be a fatal error. Maybe try
>> something like
>> 
>
> Well, as you can see in both reports, the pattern is as follows:
>
> The first subttest sees 1 less message than expected.
> The second sees 1 more message than expected.

Only thing that came into mind are directory timestamps... if directory
(m)time is same as before notmuch will not scan it for files...

... following that if the granularity of directory timestamp were 1 second,
then it could easily happen than first one new message is not seen, and
next time there is one extra message to be see...

Just wild guesses...

Tomi


>
> So in summary, they are generated but "notmuch new" is not run or does
> not pick them up. Though it's a Heisenbug on a specific arch, I'll try
> your suggestion and hammer our infra with it ...