From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Philipp Stephani Newsgroups: gmane.emacs.bugs Subject: bug#23205: 25.0.92; sort-tests is flaky Date: Sun, 03 Apr 2016 21:00:50 +0000 Message-ID: References: <83a8larb8d.fsf@gnu.org> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: multipart/alternative; boundary=001a114026d6dd9286052f9ae89f X-Trace: ger.gmane.org 1459717341 2551 80.91.229.3 (3 Apr 2016 21:02:21 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Sun, 3 Apr 2016 21:02:21 +0000 (UTC) Cc: 23205@debbugs.gnu.org To: Eli Zaretskii Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Sun Apr 03 23:02:12 2016 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1amp9w-0001UQ-0y for geb-bug-gnu-emacs@m.gmane.org; Sun, 03 Apr 2016 23:02:12 +0200 Original-Received: from localhost ([::1]:55236 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1amp9v-0007qb-Ex for geb-bug-gnu-emacs@m.gmane.org; Sun, 03 Apr 2016 17:02:11 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:43953) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1amp9r-0007qU-H8 for bug-gnu-emacs@gnu.org; Sun, 03 Apr 2016 17:02:08 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1amp9m-0008Vd-LO for bug-gnu-emacs@gnu.org; Sun, 03 Apr 2016 17:02:07 -0400 Original-Received: from debbugs.gnu.org ([208.118.235.43]:53244) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1amp9m-0008VZ-IC for bug-gnu-emacs@gnu.org; Sun, 03 Apr 2016 17:02:02 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1amp9m-0001q7-CE for bug-gnu-emacs@gnu.org; Sun, 03 Apr 2016 17:02:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Philipp Stephani Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sun, 03 Apr 2016 21:02:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 23205 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: Original-Received: via spool by 23205-submit@debbugs.gnu.org id=B23205.14597172697007 (code B ref 23205); Sun, 03 Apr 2016 21:02:02 +0000 Original-Received: (at 23205) by debbugs.gnu.org; 3 Apr 2016 21:01:09 +0000 Original-Received: from localhost ([127.0.0.1]:50371 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1amp8v-0001ox-HO for submit@debbugs.gnu.org; Sun, 03 Apr 2016 17:01:09 -0400 Original-Received: from mail-lf0-f41.google.com ([209.85.215.41]:33194) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1amp8t-0001oa-M9 for 23205@debbugs.gnu.org; Sun, 03 Apr 2016 17:01:08 -0400 Original-Received: by mail-lf0-f41.google.com with SMTP id p188so115186132lfd.0 for <23205@debbugs.gnu.org>; Sun, 03 Apr 2016 14:01:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=7Wh2kqjvPvcHvd0bCngoP6WwdTtjTwL/dPYj0QcgBek=; b=OkOkKyTGMVRxcGZSnPOuerctyvGH/rQKuV5esSINIvPMfDwbxR8AiUmtj8/4F1+QU4 GLZ/UTJe2PoghV7pCpiT2pkS+Ob4i9xBcte/b6b3UdnkX+aw+9Xiy39mDr30Ouse1SOd Co8b9ZZSc0VkjMQ7uajVFGD1an/WQ4DMbZ/FKeBoimKKi/A2d44i2PdyBlSbJyY6MpRr yzO82xzw3Jvxu4UKtn9WapcUFNxBIGfVfNmRHfdwvhUmvuL4uGj2TdoxwewtZbWLv+7q d5KOkb8Bng0EVgMK9RuxShjOrw/47P5/SAmbBIJwXTib+FKH9VelpRa6isLw1YFOFAXj uMoA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=7Wh2kqjvPvcHvd0bCngoP6WwdTtjTwL/dPYj0QcgBek=; b=RVKyhENf8JZyJws06ECsMdTQhih4K85X59069L4ZdlCDZHAT4j33ACCpK1aLtR1ZvY 78ghhcd/lX+mfWlTrSPrZLbdDRH9D1JYDZkMp3WLeVRmVtMKkrdvohaWhSc8HPJUZL60 cpp4b/TQ9UXRwjshRZwAfGsGb1ytSawIPbOoOEGfAUCwovHDFmXzVihTp3vtsyTzJAn9 cJbKsTmWd6wQrxrDf0ncA6Af+SvOh6lUZ2CGEL8+Crqu43k+lMc61DEf1NmGrq4yGZdm AHeD/aWa0lghiOauD7xuVD1Q6uW9l/J+njgtbZZ1jQz0z1GWuhLCo9oCfq/Spf5tEMVM 3m5g== X-Gm-Message-State: AD7BkJIihepBWI8v7ItDD07yLsIJ3xYb/S751APmoxRtKDXaw0ddByQEyQjpoJs6YtRdH6SVrJb+rRoK9YGEM1cK X-Received: by 10.25.145.136 with SMTP id t130mr5996781lfd.4.1459717261831; Sun, 03 Apr 2016 14:01:01 -0700 (PDT) In-Reply-To: <83a8larb8d.fsf@gnu.org> X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 208.118.235.43 X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.bugs:115986 Archived-At: --001a114026d6dd9286052f9ae89f Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On Sun, Apr 3, 2016 at 4:58 PM Eli Zaretskii wrote: > > From: Philipp Stephani > > Date: Sun, 03 Apr 2016 10:30:10 +0000 > > > > In the emacs-25 branch, in the directory test/automated, run: > > > > declare -i failures=3D0 > > for ((i=3D0; i<1000; i++)); do > > if ! output=3D"$(make sort-tests 2>&1)"; then > > ((failures++)) > > echo "Iteration $i: $output" > > fi > > done > > echo "Failures: $failures" > > > > There will be typically a small number of failures (< 10), caused by > > e.g. > > > > Test sort-tests--lines condition: > > (ert-test-failed > > ((should > > (string=3D > > (substring ... 0 -1) > > sorted-words)) > > :form > > (string=3D "Zi > > zI" "zI > > Zi") > > :value nil)) > > > > This is presumably due to non-stable sorting. Either the sorting should > > be made stable, or the test shouldn't assume stable sorting. > > Do you sometimes see such failures if you just run the test once? > I don't know, what's the difference between running it once and running it 1000 times? Since it fails randomly, it could fail in any iteration, including the first one. --=20 Google Germany GmbH Erika-Mann-Stra=C3=9Fe 33 80636 M=C3=BCnchen Registergericht und -nummer: Hamburg, HRB 86891 Sitz der Gesellschaft: Hamburg Gesch=C3=A4ftsf=C3=BChrer: Matthew Scott Sucherman, Paul Terence Manicle Diese E-Mail ist vertraulich. Wenn Sie nicht der richtige Adressat sind, leiten Sie diese bitte nicht weiter, informieren Sie den Absender und l=C3=B6schen Sie die E-Mail und alle Anh=C3=A4nge. Vielen Dank. This e-mail is confidential. If you are not the right addressee please do not forward it, please inform the sender, and please erase this e-mail including any attachments. Thanks. --001a114026d6dd9286052f9ae89f Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable


On Sun= , Apr 3, 2016 at 4:58 PM Eli Zaretskii <= eliz@gnu.org> wrote:
> Fr= om: Philipp Stephani <phst@google.com>
> Date: Sun, 03 Apr 2016 10:30:10 +0000
>
> In the emacs-25 branch, in the directory test/automated, run:
>
> declare -i failures=3D0
> for ((i=3D0; i<1000; i++)); do
> if ! output=3D"$(make sort-tests 2>&1)"; then
> ((failures++))
> echo "Iteration $i: $output"
> fi
> done
> echo "Failures: $failures"
>
> There will be typically a small number of failures (< 10), caused b= y
> e.g.
>
> Test sort-tests--lines condition:
> (ert-test-failed
> ((should
> (string=3D
> (substring ... 0 -1)
> sorted-words))
> :form
> (string=3D "Zi
> zI" "zI
> Zi")
> :value nil))
>
> This is presumably due to non-stable sorting. Either the sorting shoul= d
> be made stable, or the test shouldn't assume stable sorting.

Do you sometimes see such failures if you just run the test once?

I don't know, what's the difference betw= een running it once and running it 1000 times? Since it fails randomly, it = could fail in any iteration, including the first one.=C2=A0
--

Google Germ= any GmbH
Erika-Mann-Stra=C3=9Fe 33
80636 M=C3=BCnchen

Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
Gesch=C3=A4ftsf=C3=BChrer: Matthew Scott Sucherman, Paul Terence Manicle

Diese E-Mail ist vertraulich. Wenn Sie nicht der richtige Ad= ressat sind, leiten Sie diese bitte nicht weiter, informieren Sie den Absen= der und l=C3=B6schen Sie die E-Mail und alle Anh=C3=A4nge. Vielen Dank.

This e-mail is confidential. If you are not the right addres= see please do not forward it, please inform the sender, and please erase th= is e-mail including any attachments. Thanks.

--001a114026d6dd9286052f9ae89f--