Re: [PATCH] test: aggregate-results updates

[Tomi Ollila <tomi.ollila@iki.fi>]

On Sat, Jun 05 2021, David Bremner wrote:

> Tomi Ollila <tomi.ollila@iki.fi> writes:
>>
>> testing all changed branches is somewhat hard; I did by adding
>> sleep 20 and file what effective removed created results files;
>> another change was to change 'total' to 'toxtal' in test_done,
>> then I run (at least) the following command lines:
>
> changing test_done results in pretty nonsensical output. I guess that's
> OK?

Yes.

It is very unlikely that if something is written to file not all data were
written (as the amount of data to be written is low).
A bit less unlikely is the case that output file is created but no data
written.

Should such an extremely rare cases happen it should be enough just to
report it so it get noticed (and not add any extra code to handle such
cases any further). One checking the results would probably guess that
if output is as nonsensical as it is something went badly wrong.

Tomi

>
> $  NOTMUCH_TESTS=T310-emacs.sh ./notmuch-test
>
> ...
>
> '/home/bremner/software/upstream/notmuch/test/test-results/T310-emacs' lacks 'total ...'; results may be inconsistent.
> Notmuch test suite complete.
> 72/0 tests passed.
> 1 test failed.
> -73 tests skipped.
> _______________________________________________
> notmuch mailing list -- notmuch@notmuchmail.org
> To unsubscribe send an email to notmuch-leave@notmuchmail.org