unofficial mirror of bug-gnu-emacs@gnu.org 
 help / color / mirror / code / Atom feed
* bug#25618: 25.1.91; More precise limit for tagging expensive tests
@ 2017-02-04 13:42 Tino Calancha
  2017-02-04 14:01 ` Noam Postavsky
                   ` (2 more replies)
  0 siblings, 3 replies; 10+ messages in thread
From: Tino Calancha @ 2017-02-04 13:42 UTC (permalink / raw)
  To: 25618


Prefer set a numeric limit of seconds than using the word 'few'.

For instance, in my box the test `cl-seq-test-bug24264' in
test/lisp/emacs-lisp/cl-seq-tests.el
takes 7 s.
With an explicit numeric threshold is easier to decide whether this test
must be tagged as expensive or not.

In following patch i define a few as 2-3 seconds.  Other people might
prefer 4-5 or even higher.
Please, suggest a proper value to consider a test as expensive.

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
From 5671609ff5120ac7572b8a103a9aeb899542d47f Mon Sep 17 00:00:00 2001
From: Tino Calancha <tino.calancha@gmail.com>
Date: Sat, 4 Feb 2017 22:29:24 +0900
Subject: [PATCH] More precise limit for tagging expensive tests

* CONTRIBUTE (Testing your changes): Tag a test as expensive
when lasts longer than 2-3 seconds.
---
 CONTRIBUTE | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/CONTRIBUTE b/CONTRIBUTE
index e9a0341980..155b7ef762 100644
--- a/CONTRIBUTE
+++ b/CONTRIBUTE
@@ -273,7 +273,7 @@ http://www.gnu.org/software/emacs/manual/html_node/ert/
 or run 'info "(ert)"' for for more information on writing and running
 tests.
 
-If your test lasts longer than some few seconds, mark it in its
+If your test lasts longer than 2-3 seconds, mark it in its
 'ert-deftest' definition with ":tags '(:expensive-test)".
 
 To run tests on the entire Emacs tree, run "make check" from the
-- 
2.11.0

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
In GNU Emacs 25.1.91.1 (x86_64-pc-linux-gnu, GTK+ Version 3.22.7)
 of 2017-02-04
Repository revision: 5e921112680f96d3ed94faf7dcf257e07f92d74b





^ permalink raw reply related	[flat|nested] 10+ messages in thread

* bug#25618: 25.1.91; More precise limit for tagging expensive tests
  2017-02-04 13:42 bug#25618: 25.1.91; More precise limit for tagging expensive tests Tino Calancha
@ 2017-02-04 14:01 ` Noam Postavsky
  2017-02-04 14:13   ` Tino Calancha
  2017-02-04 16:50 ` Eli Zaretskii
  2017-02-05 15:35 ` bug#25618: AW: " michael.albinus
  2 siblings, 1 reply; 10+ messages in thread
From: Noam Postavsky @ 2017-02-04 14:01 UTC (permalink / raw)
  To: Tino Calancha; +Cc: 25618

On Sat, Feb 4, 2017 at 8:42 AM, Tino Calancha <tino.calancha@gmail.com> wrote:
>
> Prefer set a numeric limit of seconds than using the word 'few'.
>
> For instance, in my box the test `cl-seq-test-bug24264' in
> test/lisp/emacs-lisp/cl-seq-tests.el
> takes 7 s.
> With an explicit numeric threshold is easier to decide whether this test
> must be tagged as expensive or not.
>
> In following patch i define a few as 2-3 seconds.  Other people might
> prefer 4-5 or even higher.
> Please, suggest a proper value to consider a test as expensive.

Is it worth specifying this so precisely? The time elapsed depends on
the speed of the computer you run it on (also, whether Emacs is
compiled with optimization or not). A test that takes 2 seconds on
your computer might easily take 5 seconds on mine.





^ permalink raw reply	[flat|nested] 10+ messages in thread

* bug#25618: 25.1.91; More precise limit for tagging expensive tests
  2017-02-04 14:01 ` Noam Postavsky
@ 2017-02-04 14:13   ` Tino Calancha
  0 siblings, 0 replies; 10+ messages in thread
From: Tino Calancha @ 2017-02-04 14:13 UTC (permalink / raw)
  To: Noam Postavsky; +Cc: 25618, Tino Calancha



On Sat, 4 Feb 2017, Noam Postavsky wrote:

> On Sat, Feb 4, 2017 at 8:42 AM, Tino Calancha <tino.calancha@gmail.com> wrote:
>>
>> Prefer set a numeric limit of seconds than using the word 'few'.
>>
>> For instance, in my box the test `cl-seq-test-bug24264' in
>> test/lisp/emacs-lisp/cl-seq-tests.el
>> takes 7 s.
>> With an explicit numeric threshold is easier to decide whether this test
>> must be tagged as expensive or not.
>>
>> In following patch i define a few as 2-3 seconds.  Other people might
>> prefer 4-5 or even higher.
>> Please, suggest a proper value to consider a test as expensive.
>
> Is it worth specifying this so precisely? The time elapsed depends on
> the speed of the computer you run it on (also, whether Emacs is
> compiled with optimization or not). A test that takes 2 seconds on
> your computer might easily take 5 seconds on mine.
When i run the test suite, the test mentioned before takes significantly
longer than the others.  I feel like it could be considered as 
slightly expensive.
After reading 'some few seconds' in CONTRIBUTE i am still in doubt on 
that.  It could be more clear.





^ permalink raw reply	[flat|nested] 10+ messages in thread

* bug#25618: 25.1.91; More precise limit for tagging expensive tests
  2017-02-04 13:42 bug#25618: 25.1.91; More precise limit for tagging expensive tests Tino Calancha
  2017-02-04 14:01 ` Noam Postavsky
@ 2017-02-04 16:50 ` Eli Zaretskii
  2017-02-04 17:54   ` Tino Calancha
  2017-02-05  9:53   ` Michael Albinus
  2017-02-05 15:35 ` bug#25618: AW: " michael.albinus
  2 siblings, 2 replies; 10+ messages in thread
From: Eli Zaretskii @ 2017-02-04 16:50 UTC (permalink / raw)
  To: Tino Calancha; +Cc: 25618

> From: Tino Calancha <tino.calancha@gmail.com>
> Date: Sat, 04 Feb 2017 22:42:44 +0900
> 
> 
> Prefer set a numeric limit of seconds than using the word 'few'.
> 
> For instance, in my box the test `cl-seq-test-bug24264' in
> test/lisp/emacs-lisp/cl-seq-tests.el
> takes 7 s.
> With an explicit numeric threshold is easier to decide whether this test
> must be tagged as expensive or not.
> 
> In following patch i define a few as 2-3 seconds.  Other people might
> prefer 4-5 or even higher.
> Please, suggest a proper value to consider a test as expensive.

I think 2-3 sec is negligible, and 7 sec is not long enough to annoy.
10 sec or more is beginning to sound like it comes close to "too
long", and 20 sec is definitely too long.





^ permalink raw reply	[flat|nested] 10+ messages in thread

* bug#25618: 25.1.91; More precise limit for tagging expensive tests
  2017-02-04 16:50 ` Eli Zaretskii
@ 2017-02-04 17:54   ` Tino Calancha
  2017-02-10  8:49     ` Eli Zaretskii
  2017-02-05  9:53   ` Michael Albinus
  1 sibling, 1 reply; 10+ messages in thread
From: Tino Calancha @ 2017-02-04 17:54 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: 25618, Tino Calancha



On Sat, 4 Feb 2017, Eli Zaretskii wrote:

>> From: Tino Calancha <tino.calancha@gmail.com>
>> Date: Sat, 04 Feb 2017 22:42:44 +0900
>>
>>
>> Prefer set a numeric limit of seconds than using the word 'few'.
>>
>> For instance, in my box the test `cl-seq-test-bug24264' in
>> test/lisp/emacs-lisp/cl-seq-tests.el
>> takes 7 s.
>> With an explicit numeric threshold is easier to decide whether this test
>> must be tagged as expensive or not.
>>
>> In following patch i define a few as 2-3 seconds.  Other people might
>> prefer 4-5 or even higher.
>> Please, suggest a proper value to consider a test as expensive.
>
> I think 2-3 sec is negligible, and 7 sec is not long enough to annoy.
> 10 sec or more is beginning to sound like it comes close to "too
> long", and 20 sec is definitely too long.
Thank you.  Now is clear.
Are you OK with keep the sentence as it is now?
I mean, keep: 'some few seconds'.





^ permalink raw reply	[flat|nested] 10+ messages in thread

* bug#25618: 25.1.91; More precise limit for tagging expensive tests
  2017-02-04 16:50 ` Eli Zaretskii
  2017-02-04 17:54   ` Tino Calancha
@ 2017-02-05  9:53   ` Michael Albinus
  2017-02-05 13:53     ` Noam Postavsky
  1 sibling, 1 reply; 10+ messages in thread
From: Michael Albinus @ 2017-02-05  9:53 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: 25618, Tino Calancha

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Tino Calancha <tino.calancha@gmail.com>
>> Date: Sat, 04 Feb 2017 22:42:44 +0900
>> 
>> 
>> Prefer set a numeric limit of seconds than using the word 'few'.
>> 
>> For instance, in my box the test `cl-seq-test-bug24264' in
>> test/lisp/emacs-lisp/cl-seq-tests.el
>> takes 7 s.
>> With an explicit numeric threshold is easier to decide whether this test
>> must be tagged as expensive or not.
>> 
>> In following patch i define a few as 2-3 seconds.  Other people might
>> prefer 4-5 or even higher.
>> Please, suggest a proper value to consider a test as expensive.
>
> I think 2-3 sec is negligible, and 7 sec is not long enough to annoy.
> 10 sec or more is beginning to sound like it comes close to "too
> long", and 20 sec is definitely too long.

Unfortunately, there is no mean in ert to see how long a single test
runs. I often run expensive tests (well, I'm testing Tramp, filenotify,
autorevert :-) and it would help me to know the time for a single test
case run.

Maybe somebody could add it? I haven't the time just now; but if it
annoys me too much, I'll do it myself, later on, maybe.

This ticket might be a good place to remember this. Pls don't close it
after changing CONTRIBUTE.

Best regards, Michael.





^ permalink raw reply	[flat|nested] 10+ messages in thread

* bug#25618: 25.1.91; More precise limit for tagging expensive tests
  2017-02-05  9:53   ` Michael Albinus
@ 2017-02-05 13:53     ` Noam Postavsky
  0 siblings, 0 replies; 10+ messages in thread
From: Noam Postavsky @ 2017-02-05 13:53 UTC (permalink / raw)
  To: Michael Albinus; +Cc: 25618, Tino Calancha

On Sun, Feb 5, 2017 at 4:53 AM, Michael Albinus <michael.albinus@gmx.de> wrote:
> Maybe somebody could add it? I haven't the time just now; but if it
> annoys me too much, I'll do it myself, later on, maybe.
>
> This ticket might be a good place to remember this. Pls don't close it
> after changing CONTRIBUTE.

Could you instead open a separate bug? I've been through a lot of old
bugs threads that are difficult to understand because they branch into
multiple issues.





^ permalink raw reply	[flat|nested] 10+ messages in thread

* bug#25618: AW: Re: bug#25618: 25.1.91; More precise limit for tagging expensive tests
  2017-02-04 13:42 bug#25618: 25.1.91; More precise limit for tagging expensive tests Tino Calancha
  2017-02-04 14:01 ` Noam Postavsky
  2017-02-04 16:50 ` Eli Zaretskii
@ 2017-02-05 15:35 ` michael.albinus
  2 siblings, 0 replies; 10+ messages in thread
From: michael.albinus @ 2017-02-05 15:35 UTC (permalink / raw)
  To: Noam Postavsky; +Cc: 25618, Tino Calancha

[-- Attachment #1: Type: text/plain, Size: 979 bytes --]

Will do later this week. I'm almost offline next days, only email works for me.
Best regards, Michael. 


Von meinem Samsung Galaxy Smartphone gesendet.
-------- Ursprüngliche Nachricht --------Von: Noam Postavsky <npostavs@users.sourceforge.net> Datum: 05.02.17  14:53  (GMT+01:00) An: Michael Albinus <michael.albinus@gmx.de> Cc: Eli Zaretskii <eliz@gnu.org>, 25618@debbugs.gnu.org, Tino Calancha <tino.calancha@gmail.com> Betreff: Re: bug#25618: 25.1.91; More precise limit for tagging expensive tests 
On Sun, Feb 5, 2017 at 4:53 AM, Michael Albinus <michael.albinus@gmx.de> wrote:
> Maybe somebody could add it? I haven't the time just now; but if it
> annoys me too much, I'll do it myself, later on, maybe.
>
> This ticket might be a good place to remember this. Pls don't close it
> after changing CONTRIBUTE.

Could you instead open a separate bug? I've been through a lot of old
bugs threads that are difficult to understand because they branch into
multiple issues.

[-- Attachment #2: Type: text/html, Size: 1501 bytes --]

^ permalink raw reply	[flat|nested] 10+ messages in thread

* bug#25618: 25.1.91; More precise limit for tagging expensive tests
  2017-02-04 17:54   ` Tino Calancha
@ 2017-02-10  8:49     ` Eli Zaretskii
  2017-02-10  9:28       ` Tino Calancha
  0 siblings, 1 reply; 10+ messages in thread
From: Eli Zaretskii @ 2017-02-10  8:49 UTC (permalink / raw)
  To: Tino Calancha; +Cc: 25618

> From: Tino Calancha <tino.calancha@gmail.com>
> Date: Sun, 5 Feb 2017 02:54:03 +0900 (JST)
> cc: Tino Calancha <tino.calancha@gmail.com>, 25618@debbugs.gnu.org
> 
> On Sat, 4 Feb 2017, Eli Zaretskii wrote:
> 
> >> From: Tino Calancha <tino.calancha@gmail.com>
> >> Date: Sat, 04 Feb 2017 22:42:44 +0900
> >>
> >>
> >> Prefer set a numeric limit of seconds than using the word 'few'.
> >>
> >> For instance, in my box the test `cl-seq-test-bug24264' in
> >> test/lisp/emacs-lisp/cl-seq-tests.el
> >> takes 7 s.
> >> With an explicit numeric threshold is easier to decide whether this test
> >> must be tagged as expensive or not.
> >>
> >> In following patch i define a few as 2-3 seconds.  Other people might
> >> prefer 4-5 or even higher.
> >> Please, suggest a proper value to consider a test as expensive.
> >
> > I think 2-3 sec is negligible, and 7 sec is not long enough to annoy.
> > 10 sec or more is beginning to sound like it comes close to "too
> > long", and 20 sec is definitely too long.
> Thank you.  Now is clear.
> Are you OK with keep the sentence as it is now?
> I mean, keep: 'some few seconds'.

It's okay to leave it as is.  I also don't object to saying something
more specific, but 2-3 sec sound just too few to me.





^ permalink raw reply	[flat|nested] 10+ messages in thread

* bug#25618: 25.1.91; More precise limit for tagging expensive tests
  2017-02-10  8:49     ` Eli Zaretskii
@ 2017-02-10  9:28       ` Tino Calancha
  0 siblings, 0 replies; 10+ messages in thread
From: Tino Calancha @ 2017-02-10  9:28 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: 25618



On 02/10/2017 05:49 PM, Eli Zaretskii wrote:
>> From: Tino Calancha <tino.calancha@gmail.com>
>> Date: Sun, 5 Feb 2017 02:54:03 +0900 (JST)
>> cc: Tino Calancha <tino.calancha@gmail.com>, 25618@debbugs.gnu.org
>>
>> On Sat, 4 Feb 2017, Eli Zaretskii wrote:
>>
>>>> From: Tino Calancha <tino.calancha@gmail.com>
>>>> Date: Sat, 04 Feb 2017 22:42:44 +0900
>>>>
>>>>
>>>> Prefer set a numeric limit of seconds than using the word 'few'.
>>>>
>>>> For instance, in my box the test `cl-seq-test-bug24264' in
>>>> test/lisp/emacs-lisp/cl-seq-tests.el
>>>> takes 7 s.
>>>> With an explicit numeric threshold is easier to decide whether this test
>>>> must be tagged as expensive or not.
>>>>
>>>> In following patch i define a few as 2-3 seconds.  Other people might
>>>> prefer 4-5 or even higher.
>>>> Please, suggest a proper value to consider a test as expensive.
>>> I think 2-3 sec is negligible, and 7 sec is not long enough to annoy.
>>> 10 sec or more is beginning to sound like it comes close to "too
>>> long", and 20 sec is definitely too long.
>> Thank you.  Now is clear.
>> Are you OK with keep the sentence as it is now?
>> I mean, keep: 'some few seconds'.
> It's okay to leave it as is.  I also don't object to saying something
> more specific, but 2-3 sec sound just too few to me.
Thanks.  It's not a big thing.  I will leave it as it is.





^ permalink raw reply	[flat|nested] 10+ messages in thread

end of thread, other threads:[~2017-02-10  9:28 UTC | newest]

Thread overview: 10+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2017-02-04 13:42 bug#25618: 25.1.91; More precise limit for tagging expensive tests Tino Calancha
2017-02-04 14:01 ` Noam Postavsky
2017-02-04 14:13   ` Tino Calancha
2017-02-04 16:50 ` Eli Zaretskii
2017-02-04 17:54   ` Tino Calancha
2017-02-10  8:49     ` Eli Zaretskii
2017-02-10  9:28       ` Tino Calancha
2017-02-05  9:53   ` Michael Albinus
2017-02-05 13:53     ` Noam Postavsky
2017-02-05 15:35 ` bug#25618: AW: " michael.albinus

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/emacs.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).