From: Dmitry Kurochkin <dmitry.kurochkin@gmail.com>
To: Tomi Ollila <tomi.ollila@iki.fi>, notmuch@notmuchmail.org
Subject: Re: [PATCH 4/9] test: add support for external executable dependencies
Date: Tue, 29 Nov 2011 02:13:53 +0400 [thread overview]
Message-ID: <87aa7fx5cu.fsf@gmail.com> (raw)
In-Reply-To: <87d3cbx6aa.fsf@gmail.com>
On Tue, 29 Nov 2011 01:53:49 +0400, Dmitry Kurochkin <dmitry.kurochkin@gmail.com> wrote:
> On Mon, 28 Nov 2011 23:16:27 +0200, Tomi Ollila <tomi.ollila@iki.fi> wrote:
> > On Thu, 17 Nov 2011 05:56:21 +0400, Dmitry Kurochkin <dmitry.kurochkin@gmail.com> wrote:
> > > There is existing support for general prerequisites in the test suite.
> > > But it is not very convenient to use: every test case has to keep
> > > track for it's dependencies and they have to be explicitly listed.
> > >
> > > The patch aims to add better support for a particular type of external
> > > dependencies: external executables. The main idea is to replace
> > > missing external binaries with shell functions that have the same
> > > name. These functions always fail and keep track of missing
> > > dependencies for a subtest. The result reporting functions later can
> > > check that an external binaries are missing and correctly report SKIP
> > > result instead of FAIL. The primary benefit is that the test cases do
> > > not need to declare their dependencies or be changed in any way.
> > > ---
> > > test/test-lib.sh | 49 +++++++++++++++++++++++++++++++++++++++++--------
> > > 1 files changed, 41 insertions(+), 8 deletions(-)
> > >
> > > diff --git a/test/test-lib.sh b/test/test-lib.sh
> > > index f21e45e..ab8c6fd 100755
> > > --- a/test/test-lib.sh
> > > +++ b/test/test-lib.sh
> > > @@ -526,40 +526,53 @@ notmuch_json_show_sanitize ()
> > > # - Implicitly by specifying the prerequisite tag in the calls to
> > > # test_expect_{success,failure,code}.
> > > #
> > > # The single parameter is the prerequisite tag (a simple word, in all
> > > # capital letters by convention).
> > >
> > > test_set_prereq () {
> > > satisfied="$satisfied$1 "
> > > }
> > > satisfied=" "
> > >
> > > test_have_prereq () {
> > > case $satisfied in
> > > *" $1 "*)
> > > : yes, have it ;;
> > > *)
> > > ! : nope ;;
> > > esac
> > > }
> > >
> > > +# declare prerequisite for the given external binary
> > > +test_declare_external_prereq () {
> > > + binary="$1"
> > > + test "$#" = 2 && name=$2 || name="$binary(1)"
> > > +
> > > + hash $binary 2>/dev/null || eval "
> > > +$1 () {
> > > + echo -n \"\$test_subtest_missing_external_prereqs_\" | grep -e \" $name \" ||
> > > + test_subtest_missing_external_prereqs_=\"$test_subtest_missing_external_prereqs_ $name\"
> > > + false
> > > +}"
> > > +}
> > > +
> >
> > Does this work right ?
>
> It does not.
>
> Moreover, there is a missing backslash before
> $test_subtest_missing_external_prereqs_ (and that is why I did not
> notice the bug during my poor testing).
>
> > ... the grep -e \" $name \" -- part requires
> > spaces on both side of, but next line does not write trailing space...
> > ... and is there leading space (and also the latest
> > $test_subtest_missing_external_prereqs_ (just before 'false') is evaluated) ?
> >
> > This could perhaps be written like the above test_set_prereq &
> > test_save_prereq:
> > ...
> > hash $binary 2>/dev/null || eval "
> > $binary () {
> > test_missing_external_prereq_${binary}_=t
> > case \$test_subtest_missing_external_prereqs_ in
> > *' $name '*) ;;
> > *) test_subtest_missing_external_prereqs_=\"\$test_subtest_missing_external_prereqs_$name \"
> > esac
> > false
> > }
> >
> > and test_subtest_missing_external_prereqs_=' ' done in test_reset_state_
> >
>
> Well, this approach would obviously be better, at least because it does
> work. But IMO it is too complex, and essentially has the same problem
> as the current code: it mixes the check with diagnostic message.
>
> We already have a proper way to check if dependency is missing already:
> test_missing_external_prereq_${binary}_. We should check it and add the
> binary to test_subtest_missing_external_prereqs_ if it is not set. And
> move setting of test_missing_external_prereq_${binary}_ below.
>
> > (I took some code from current git head.... also perhaps instead of
> > setting test_missing_external_prereq_${binary}_=t, (bash) associative
> > arrays could be taken into use -- the eval to read that information
> > is a bit hairy -- well, at least that part works for sure :D )
> >
>
> Yes! I like using hash here.
>
> > Hmm... how about this:
> >
> > hash $binary 2>/dev/null || eval "
> > $binary () {
> > if [ -z \"\${test_missing_external_prereq_[$binary]}\" ]
> > then
> > test_missing_external_prereq_[$binary]=t
> > test_subtest_missing_external_prereqs_=\"\$test_subtest_missing_external_prereqs_ $name\"
> > fi
> > false
> > }
> >
>
> There is some inconsistent indenting in the above code (mixed tabs and
> spaces). Also test_require_external_prereq() should be changed as well.
>
> Otherwise I like it. Would you please submit a patch?
>
Actually, test_missing_external_prereq_${binary}_ must be set outside of
the $binary() function. It indicates that the binary is missing, not
that it is required. I will send some patches to fix these bugs. Then
you can change it to use hashes.
Regards,
Dmitry
> Regards,
> Dmitry
>
> > and test_subtest_missing_external_prereqs_ cleared in test_reset_state_ like now.
> >
> > Tomi
next prev parent reply other threads:[~2011-11-28 22:14 UTC|newest]
Thread overview: 29+ messages / expand[flat|nested] mbox.gz Atom feed top
2011-11-17 1:56 [PATCH 0/9] test: (hopefully) better test prerequisites Dmitry Kurochkin
2011-11-17 1:56 ` [PATCH 1/9] test: move subtest variables reset into a dedicated function Dmitry Kurochkin
2011-11-17 1:56 ` [PATCH 2/9] test: set EMACS_SERVER variable only after dtach(1) was successfully started Dmitry Kurochkin
2011-11-17 9:14 ` Jameson Graef Rollins
2011-11-17 11:07 ` Dmitry Kurochkin
2011-11-17 1:56 ` [PATCH 3/9] test: add test state reset to test_expect_* functions that did not have it Dmitry Kurochkin
2011-11-17 1:56 ` [PATCH 4/9] test: add support for external executable dependencies Dmitry Kurochkin
2011-11-28 21:16 ` Tomi Ollila
2011-11-28 21:53 ` Dmitry Kurochkin
2011-11-28 22:13 ` Dmitry Kurochkin [this message]
2011-11-28 22:42 ` Dmitry Kurochkin
2011-11-17 1:56 ` [PATCH 5/9] test: fix "skipping test" verbose output Dmitry Kurochkin
2011-11-17 1:56 ` [PATCH 6/9] test: skip all subtests if external dependencies are missing during init Dmitry Kurochkin
2011-11-17 1:56 ` [PATCH 7/9] test: declare external dependencies for the tests Dmitry Kurochkin
2011-11-17 1:56 ` [PATCH 8/9] test: check if emacs is available in the beginning of test_emacs Dmitry Kurochkin
2011-11-17 9:43 ` Tomi Ollila
2011-11-17 11:13 ` Dmitry Kurochkin
2011-11-17 13:08 ` Dmitry Kurochkin
2011-11-17 1:56 ` [PATCH 9/9] test: fix "Stashing in notmuch-search" test when emacs is not available Dmitry Kurochkin
2011-11-17 9:14 ` [PATCH 0/9] test: (hopefully) better test prerequisites Jameson Graef Rollins
2011-11-17 11:20 ` Dmitry Kurochkin
2011-11-17 12:20 ` Tomi Ollila
2011-11-17 12:30 ` Tomi Ollila
2011-11-17 13:22 ` Dmitry Kurochkin
2011-11-17 14:02 ` Tomi Ollila
2011-11-17 15:17 ` Dmitry Kurochkin
2011-11-18 8:55 ` Tomi Ollila
2011-11-17 9:46 ` Thomas Jost
2011-11-17 11:45 ` Dmitry Kurochkin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://notmuchmail.org/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87aa7fx5cu.fsf@gmail.com \
--to=dmitry.kurochkin@gmail.com \
--cc=notmuch@notmuchmail.org \
--cc=tomi.ollila@iki.fi \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://yhetil.org/notmuch.git/
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).