From: Michael Albinus <michael.albinus@gmx.de>
To: Stefan Kangas <stefan@marxist.se>
Cc: emacs-devel@gnu.org
Subject: Re: master 262d0c6: Mark some tests as expensive
Date: Sat, 12 Sep 2020 12:52:26 +0200 [thread overview]
Message-ID: <87pn6rgtcl.fsf@gmx.de> (raw)
In-Reply-To: <CADwFkm=+g7rg-r9EgdFaLvhxwkKY3B0GE6vmFnqMiv=i-_QL-g@mail.gmail.com> (Stefan Kangas's message of "Fri, 11 Sep 2020 11:06:48 -0700")
Stefan Kangas <stefan@marxist.se> writes:
> Hi Michael,
Hi Stefan,
>> Could you please give a reasoning? Declaring tests as expansive decreases
>> heavily their application, giving us less chances to detect errors.
>
> These tests all took 5-60 seconds to run, most in the lower end
> admittedly.
With your change, you have tagged *all* autorevert tests as
:expensive-test. In practice this means, they won't run by default ever,
because people call "make check" only. Any error there, which is not
related to GNU/Linux, will be hidden (hydra and emba run "make
check-expensive", but only for GNU/Linux).
I don't think this is your intention.
> The idea is to avoid that it will take a very long time to run the unit
> tests as the test suite grows. Arguably a unit test should never take
> longer than a second, and even that is in the slow end. If we have
> 10.000 unit tests taking a second each, just do the math of how long it
> will take to run (even with parallelization). We should not postpone
> working on this until we are in that situation, IMHO, because by then it
> will be a major pain.
There have been discussions about how long tests shall run. But there
hasn't been ever a conclusion to limit them to 1 second. Please discuss
this first, before making such changes.
> Best regards,
> Stefan Kangas
Best regards, Michael.
next prev parent reply other threads:[~2020-09-12 10:52 UTC|newest]
Thread overview: 36+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <20200910182904.20559.25935@vcs0.savannah.gnu.org>
[not found] ` <20200910182905.F0E4520A2E@vcs0.savannah.gnu.org>
2020-09-11 9:25 ` master 262d0c6: Mark some tests as expensive Michael Albinus
2020-09-11 18:06 ` Stefan Kangas
2020-09-12 10:25 ` Daniel Martín
2020-09-12 10:38 ` Eli Zaretskii
2020-09-12 11:15 ` Lars Ingebrigtsen
2020-09-12 11:24 ` Eli Zaretskii
2020-09-12 12:11 ` Lars Ingebrigtsen
2020-09-12 12:29 ` Eli Zaretskii
2020-09-13 12:30 ` Lars Ingebrigtsen
2020-09-13 15:21 ` Stefan Monnier
2020-09-13 15:30 ` Lars Ingebrigtsen
2020-09-13 17:22 ` Michael Albinus
2020-09-12 16:47 ` Michael Albinus
2020-09-13 12:33 ` Lars Ingebrigtsen
2020-09-13 14:37 ` Eli Zaretskii
2020-09-12 11:27 ` Michael Albinus
2020-09-12 12:15 ` Lars Ingebrigtsen
2020-09-12 12:30 ` Michael Albinus
2020-09-12 12:36 ` Lars Ingebrigtsen
2020-09-12 13:04 ` Dmitry Gutov
2020-09-12 14:23 ` Daniel Martín
2020-09-12 14:49 ` Michael Albinus
2020-09-12 16:47 ` Dmitry Gutov
2020-09-12 14:53 ` Lars Ingebrigtsen
2020-09-12 14:00 ` Daniel Martín
2020-09-12 14:14 ` Eli Zaretskii
2020-09-12 15:16 ` Daniel Martín
2020-09-12 14:43 ` Michael Albinus
2020-09-12 15:02 ` Daniel Martín
2020-09-12 10:52 ` Michael Albinus [this message]
2020-09-18 10:22 ` Stefan Kangas
2020-09-18 10:31 ` Michael Albinus
2020-10-18 18:15 ` Stefan Kangas
2020-10-19 12:40 ` Michael Albinus
2020-10-19 15:34 ` Stefan Kangas
2020-10-19 16:42 ` Michael Albinus
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://www.gnu.org/software/emacs/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87pn6rgtcl.fsf@gmx.de \
--to=michael.albinus@gmx.de \
--cc=emacs-devel@gnu.org \
--cc=stefan@marxist.se \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/emacs.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).