From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Eli Zaretskii Newsgroups: gmane.emacs.bugs Subject: bug#23205: 25.0.92; sort-tests is flaky Date: Sun, 03 Apr 2016 17:57:38 +0300 Message-ID: <83a8larb8d.fsf@gnu.org> References: Reply-To: Eli Zaretskii NNTP-Posting-Host: plane.gmane.org X-Trace: ger.gmane.org 1459695561 11033 80.91.229.3 (3 Apr 2016 14:59:21 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Sun, 3 Apr 2016 14:59:21 +0000 (UTC) Cc: 23205@debbugs.gnu.org To: Philipp Stephani Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Sun Apr 03 16:59:12 2016 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1amjUc-0001L6-Vy for geb-bug-gnu-emacs@m.gmane.org; Sun, 03 Apr 2016 16:59:11 +0200 Original-Received: from localhost ([::1]:53785 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1amjUc-0006YD-G0 for geb-bug-gnu-emacs@m.gmane.org; Sun, 03 Apr 2016 10:59:10 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:36792) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1amjUV-0006Xs-7g for bug-gnu-emacs@gnu.org; Sun, 03 Apr 2016 10:59:07 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1amjUU-00021o-Ev for bug-gnu-emacs@gnu.org; Sun, 03 Apr 2016 10:59:03 -0400 Original-Received: from debbugs.gnu.org ([208.118.235.43]:52964) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1amjUU-00021i-BA for bug-gnu-emacs@gnu.org; Sun, 03 Apr 2016 10:59:02 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1amjUU-0006zt-3W for bug-gnu-emacs@gnu.org; Sun, 03 Apr 2016 10:59:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Eli Zaretskii Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sun, 03 Apr 2016 14:59:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 23205 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: Original-Received: via spool by 23205-submit@debbugs.gnu.org id=B23205.145969549426842 (code B ref 23205); Sun, 03 Apr 2016 14:59:02 +0000 Original-Received: (at 23205) by debbugs.gnu.org; 3 Apr 2016 14:58:14 +0000 Original-Received: from localhost ([127.0.0.1]:50091 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1amjTh-0006ys-Nj for submit@debbugs.gnu.org; Sun, 03 Apr 2016 10:58:13 -0400 Original-Received: from eggs.gnu.org ([208.118.235.92]:45350) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1amjTg-0006yf-Ff for 23205@debbugs.gnu.org; Sun, 03 Apr 2016 10:58:12 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1amjTa-0001QZ-KW for 23205@debbugs.gnu.org; Sun, 03 Apr 2016 10:58:07 -0400 Original-Received: from fencepost.gnu.org ([2001:4830:134:3::e]:54691) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1amjTa-0001QU-Ia; Sun, 03 Apr 2016 10:58:06 -0400 Original-Received: from 84.94.185.246.cable.012.net.il ([84.94.185.246]:3079 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.82) (envelope-from ) id 1amjTZ-0007M7-MT; Sun, 03 Apr 2016 10:58:06 -0400 In-reply-to: (message from Philipp Stephani on Sun, 03 Apr 2016 10:30:10 +0000) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 208.118.235.43 X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.bugs:115932 Archived-At: > From: Philipp Stephani > Date: Sun, 03 Apr 2016 10:30:10 +0000 > > In the emacs-25 branch, in the directory test/automated, run: > > declare -i failures=0 > for ((i=0; i<1000; i++)); do > if ! output="$(make sort-tests 2>&1)"; then > ((failures++)) > echo "Iteration $i: $output" > fi > done > echo "Failures: $failures" > > There will be typically a small number of failures (< 10), caused by > e.g. > > Test sort-tests--lines condition: > (ert-test-failed > ((should > (string= > (substring ... 0 -1) > sorted-words)) > :form > (string= "Zi > zI" "zI > Zi") > :value nil)) > > This is presumably due to non-stable sorting. Either the sorting should > be made stable, or the test shouldn't assume stable sorting. Do you sometimes see such failures if you just run the test once?