all messages for Guix-related lists mirrored at yhetil.org
 help / color / mirror / code / Atom feed
From: "Ludovic Courtès" <ludo@gnu.org>
To: Maxim Cournoyer <maxim.cournoyer@gmail.com>
Cc: 74394@debbugs.gnu.org
Subject: [bug#74394] [PATCH 0/2] Skip slow tests by default and run 'check' in Git pre-push hook.
Date: Tue, 17 Dec 2024 15:51:33 +0100	[thread overview]
Message-ID: <87frmmnz0q.fsf@gnu.org> (raw)
In-Reply-To: <87zfkvrw38.fsf@gmail.com> (Maxim Cournoyer's message of "Tue, 17 Dec 2024 09:28:59 +0900")

Hi Maxim,

Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:

> Ludovic Courtès <ludo@gnu.org> writes:

[...]

>> I agree with the goal, of course, but not with the method: even without
>> expensive tests, “make check” is going to take maybe 5–10 minutes, and
>> having that happen when you run “git push” can be a terrible development
>> experience (especially since the developer most likely either already
>> ran the test suite or part of it right before, or pushes package changes
>> that have infinitely small probability of breaking “make check”).
>
> As I wrote, 'make check' with this change takes about 1 minute on my
> machine;

Right now, without your patch, we have:

--8<---------------cut here---------------start------------->8---
$ wget -qO- $(guix build --log-file guix --no-grafts)|gunzip |grep "\`check'"
starting phase `check'
phase `check' succeeded after 2049.2 seconds
--8<---------------cut here---------------end--------------->8---

More than 30mn on the fast machines of the build farm, and with some of
the expensive tests already skipped (those that require network access:
time-machine, pack -RR, etc.).

This patch is not dividing wall-clock time by 30, is it?

>> This variable itself may still be useful though (I’d call it
>> ‘RUN_EXPENSIVE_TESTS’ or something like that—that’s the name used in
>> Coreutils—, “expensive” being the key word).  I would also keep it on by
>> default.
>
> One of the tests that was unbearably long when I measured was the
> time-machine test.  It took about 20 minutes to fetch the git repository
> with guile-git and run the test (which does extra work compared to the
> CLI like validating each object).  I don't think we want this kind of
> experience by default (because that'd probably convince people that
> running the test suite often is not a reasonable thing to do).  The
> other tests were more reasonable, with the longer ones in the 2-3
> minutes range on my machine, IIRC.  Perhaps we could have this 20 minute
> outlier skipped by default, maybe with a RUN_PROHIBITIVE_TESTS flag that
> would default to 0 (false).

Yeah okay, maybe we should skip them by default, and maybe we can find a
way to ensure developers do run them periodically.

> A long time ago I had read a blog post that argued that unit tests
> should be small and fast [0],

I actually agree.  :-)

Thanks!

Ludo’.




      reply	other threads:[~2024-12-17 14:52 UTC|newest]

Thread overview: 6+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-11-17 12:03 [bug#74394] [PATCH 0/2] Skip slow tests by default and run 'check' in Git pre-push hook Maxim Cournoyer
2024-11-17 12:21 ` [bug#74394] [PATCH 1/2] build: Exclude expensive tests in check target by default Maxim Cournoyer
2024-11-17 12:21 ` [bug#74394] [PATCH 2/2] etc: Ensure test suite passes in pre-push git hook Maxim Cournoyer
2024-11-29 10:05 ` [bug#74394] [PATCH 0/2] Skip slow tests by default and run 'check' in Git pre-push hook Ludovic Courtès
2024-12-17  0:28   ` Maxim Cournoyer
2024-12-17 14:51     ` Ludovic Courtès [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=87frmmnz0q.fsf@gnu.org \
    --to=ludo@gnu.org \
    --cc=74394@debbugs.gnu.org \
    --cc=maxim.cournoyer@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this external index

	https://git.savannah.gnu.org/cgit/guix.git

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.