From: "Ludovic Courtès" <ludo@gnu.org>
To: Maxim Cournoyer <maxim.cournoyer@gmail.com>
Cc: 74394@debbugs.gnu.org
Subject: [bug#74394] [PATCH 0/2] Skip slow tests by default and run 'check' in Git pre-push hook.
Date: Tue, 17 Dec 2024 15:51:33 +0100 [thread overview]
Message-ID: <87frmmnz0q.fsf@gnu.org> (raw)
In-Reply-To: <87zfkvrw38.fsf@gmail.com> (Maxim Cournoyer's message of "Tue, 17 Dec 2024 09:28:59 +0900")
Hi Maxim,
Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:
> Ludovic Courtès <ludo@gnu.org> writes:
[...]
>> I agree with the goal, of course, but not with the method: even without
>> expensive tests, “make check” is going to take maybe 5–10 minutes, and
>> having that happen when you run “git push” can be a terrible development
>> experience (especially since the developer most likely either already
>> ran the test suite or part of it right before, or pushes package changes
>> that have infinitely small probability of breaking “make check”).
>
> As I wrote, 'make check' with this change takes about 1 minute on my
> machine;
Right now, without your patch, we have:
--8<---------------cut here---------------start------------->8---
$ wget -qO- $(guix build --log-file guix --no-grafts)|gunzip |grep "\`check'"
starting phase `check'
phase `check' succeeded after 2049.2 seconds
--8<---------------cut here---------------end--------------->8---
More than 30mn on the fast machines of the build farm, and with some of
the expensive tests already skipped (those that require network access:
time-machine, pack -RR, etc.).
This patch is not dividing wall-clock time by 30, is it?
>> This variable itself may still be useful though (I’d call it
>> ‘RUN_EXPENSIVE_TESTS’ or something like that—that’s the name used in
>> Coreutils—, “expensive” being the key word). I would also keep it on by
>> default.
>
> One of the tests that was unbearably long when I measured was the
> time-machine test. It took about 20 minutes to fetch the git repository
> with guile-git and run the test (which does extra work compared to the
> CLI like validating each object). I don't think we want this kind of
> experience by default (because that'd probably convince people that
> running the test suite often is not a reasonable thing to do). The
> other tests were more reasonable, with the longer ones in the 2-3
> minutes range on my machine, IIRC. Perhaps we could have this 20 minute
> outlier skipped by default, maybe with a RUN_PROHIBITIVE_TESTS flag that
> would default to 0 (false).
Yeah okay, maybe we should skip them by default, and maybe we can find a
way to ensure developers do run them periodically.
> A long time ago I had read a blog post that argued that unit tests
> should be small and fast [0],
I actually agree. :-)
Thanks!
Ludo’.
prev parent reply other threads:[~2024-12-17 14:52 UTC|newest]
Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-11-17 12:03 [bug#74394] [PATCH 0/2] Skip slow tests by default and run 'check' in Git pre-push hook Maxim Cournoyer
2024-11-17 12:21 ` [bug#74394] [PATCH 1/2] build: Exclude expensive tests in check target by default Maxim Cournoyer
2024-11-17 12:21 ` [bug#74394] [PATCH 2/2] etc: Ensure test suite passes in pre-push git hook Maxim Cournoyer
2024-11-29 10:05 ` [bug#74394] [PATCH 0/2] Skip slow tests by default and run 'check' in Git pre-push hook Ludovic Courtès
2024-12-17 0:28 ` Maxim Cournoyer
2024-12-17 14:51 ` Ludovic Courtès [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87frmmnz0q.fsf@gnu.org \
--to=ludo@gnu.org \
--cc=74394@debbugs.gnu.org \
--cc=maxim.cournoyer@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this external index
https://git.savannah.gnu.org/cgit/guix.git
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.