On Sun, Apr 3, 2016 at 4:58 PM Eli Zaretskii wrote: > > From: Philipp Stephani > > Date: Sun, 03 Apr 2016 10:30:10 +0000 > > > > In the emacs-25 branch, in the directory test/automated, run: > > > > declare -i failures=0 > > for ((i=0; i<1000; i++)); do > > if ! output="$(make sort-tests 2>&1)"; then > > ((failures++)) > > echo "Iteration $i: $output" > > fi > > done > > echo "Failures: $failures" > > > > There will be typically a small number of failures (< 10), caused by > > e.g. > > > > Test sort-tests--lines condition: > > (ert-test-failed > > ((should > > (string= > > (substring ... 0 -1) > > sorted-words)) > > :form > > (string= "Zi > > zI" "zI > > Zi") > > :value nil)) > > > > This is presumably due to non-stable sorting. Either the sorting should > > be made stable, or the test shouldn't assume stable sorting. > > Do you sometimes see such failures if you just run the test once? > I don't know, what's the difference between running it once and running it 1000 times? Since it fails randomly, it could fail in any iteration, including the first one. -- Google Germany GmbH Erika-Mann-Straße 33 80636 München Registergericht und -nummer: Hamburg, HRB 86891 Sitz der Gesellschaft: Hamburg Geschäftsführer: Matthew Scott Sucherman, Paul Terence Manicle Diese E-Mail ist vertraulich. Wenn Sie nicht der richtige Adressat sind, leiten Sie diese bitte nicht weiter, informieren Sie den Absender und löschen Sie die E-Mail und alle Anhänge. Vielen Dank. This e-mail is confidential. If you are not the right addressee please do not forward it, please inform the sender, and please erase this e-mail including any attachments. Thanks.