From: Eli Zaretskii <eliz@gnu.org>
To: "Daniel Martín" <mardani29@yahoo.es>
Cc: michael.albinus@gmx.de, stefan@marxist.se, emacs-devel@gnu.org
Subject: Re: master 262d0c6: Mark some tests as expensive
Date: Sat, 12 Sep 2020 17:14:01 +0300 [thread overview]
Message-ID: <83tuw3ccba.fsf@gnu.org> (raw)
In-Reply-To: <m11rj7hz8f.fsf@yahoo.es> (message from Daniel Martín on Sat, 12 Sep 2020 16:00:00 +0200)
> From: Daniel Martín <mardani29@yahoo.es>
> Cc: stefan@marxist.se, michael.albinus@gmx.de, emacs-devel@gnu.org
> Date: Sat, 12 Sep 2020 16:00:00 +0200
>
> Eli Zaretskii <eliz@gnu.org> writes:
> >
> > Tramp tests need more time because they involve a remote system.
> > Moreover, the time taken by each Tramp test depends on the speed of
> > the connection, which you cannot know in advance.
>
> If they depend on the connection speed, or if a host is online or
> offline, they also introduce non-determinism that perhaps is not
> apparent now, but can cause problems when the Tramp codebase (and its
> number of tests) scale. For example, if a Tramp test has a 0.1% chance
> of failure because of an unrelated network problem, then if the Tramp
> test suite reaches a point where 10000 tests are run per day, people
> would be investigating 10 test flakes per day. That's a good reason for
> people to lose confidence in the Tramp test suite and ignore failures.
I don't understand what you are arguing here. AFAIK, Michael already
either marked "unstable" or rewrote any tests that are known to suffer
from non-determinism. Slow speed doesn't necessarily add
non-determinism, not if Emacs waits for the operation to complete.
So I don't see how this is relevant to the issue at hand, which is the
criteria for marking tests "expensive" and not running them by
default. What am I missing?
next prev parent reply other threads:[~2020-09-12 14:14 UTC|newest]
Thread overview: 36+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <20200910182904.20559.25935@vcs0.savannah.gnu.org>
[not found] ` <20200910182905.F0E4520A2E@vcs0.savannah.gnu.org>
2020-09-11 9:25 ` master 262d0c6: Mark some tests as expensive Michael Albinus
2020-09-11 18:06 ` Stefan Kangas
2020-09-12 10:25 ` Daniel Martín
2020-09-12 10:38 ` Eli Zaretskii
2020-09-12 11:15 ` Lars Ingebrigtsen
2020-09-12 11:24 ` Eli Zaretskii
2020-09-12 12:11 ` Lars Ingebrigtsen
2020-09-12 12:29 ` Eli Zaretskii
2020-09-13 12:30 ` Lars Ingebrigtsen
2020-09-13 15:21 ` Stefan Monnier
2020-09-13 15:30 ` Lars Ingebrigtsen
2020-09-13 17:22 ` Michael Albinus
2020-09-12 16:47 ` Michael Albinus
2020-09-13 12:33 ` Lars Ingebrigtsen
2020-09-13 14:37 ` Eli Zaretskii
2020-09-12 11:27 ` Michael Albinus
2020-09-12 12:15 ` Lars Ingebrigtsen
2020-09-12 12:30 ` Michael Albinus
2020-09-12 12:36 ` Lars Ingebrigtsen
2020-09-12 13:04 ` Dmitry Gutov
2020-09-12 14:23 ` Daniel Martín
2020-09-12 14:49 ` Michael Albinus
2020-09-12 16:47 ` Dmitry Gutov
2020-09-12 14:53 ` Lars Ingebrigtsen
2020-09-12 14:00 ` Daniel Martín
2020-09-12 14:14 ` Eli Zaretskii [this message]
2020-09-12 15:16 ` Daniel Martín
2020-09-12 14:43 ` Michael Albinus
2020-09-12 15:02 ` Daniel Martín
2020-09-12 10:52 ` Michael Albinus
2020-09-18 10:22 ` Stefan Kangas
2020-09-18 10:31 ` Michael Albinus
2020-10-18 18:15 ` Stefan Kangas
2020-10-19 12:40 ` Michael Albinus
2020-10-19 15:34 ` Stefan Kangas
2020-10-19 16:42 ` Michael Albinus
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://www.gnu.org/software/emacs/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=83tuw3ccba.fsf@gnu.org \
--to=eliz@gnu.org \
--cc=emacs-devel@gnu.org \
--cc=mardani29@yahoo.es \
--cc=michael.albinus@gmx.de \
--cc=stefan@marxist.se \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/emacs.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).