From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from localhost (localhost [127.0.0.1]) by olra.theworths.org (Postfix) with ESMTP id B79EA429E25 for ; Thu, 12 Jan 2012 09:16:32 -0800 (PST) X-Virus-Scanned: Debian amavisd-new at olra.theworths.org X-Spam-Flag: NO X-Spam-Score: -0.7 X-Spam-Level: X-Spam-Status: No, score=-0.7 tagged_above=-999 required=5 tests=[RCVD_IN_DNSWL_LOW=-0.7] autolearn=disabled Received: from olra.theworths.org ([127.0.0.1]) by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id TaPfpgTqypxA for ; Thu, 12 Jan 2012 09:16:32 -0800 (PST) Received: from mail-ww0-f41.google.com (mail-ww0-f41.google.com [74.125.82.41]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) by olra.theworths.org (Postfix) with ESMTPS id 2C934431FB6 for ; Thu, 12 Jan 2012 09:16:32 -0800 (PST) Received: by wgbed3 with SMTP id ed3so429905wgb.2 for ; Thu, 12 Jan 2012 09:16:31 -0800 (PST) Received: by 10.180.19.130 with SMTP id f2mr1942278wie.12.1326388591030; Thu, 12 Jan 2012 09:16:31 -0800 (PST) Received: from localhost ([109.131.126.209]) by mx.google.com with ESMTPS id 28sm6497823wby.3.2012.01.12.09.16.29 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 12 Jan 2012 09:16:30 -0800 (PST) From: Pieter Praet To: Dmitry Kurochkin , notmuch@notmuchmail.org Subject: Re: [PATCH v3 00/7] test: (hopefully) better test prerequisites In-Reply-To: <1322418979-19180-1-git-send-email-dmitry.kurochkin@gmail.com> References: <87pqgd7d23.fsf@rocinante.cs.unb.ca> <1322418979-19180-1-git-send-email-dmitry.kurochkin@gmail.com> User-Agent: Notmuch/0.10.2+115~gadd29f6 (http://notmuchmail.org) Emacs/23.3.1 (x86_64-unknown-linux-gnu) Date: Thu, 12 Jan 2012 18:14:46 +0100 Message-ID: <87d3aolu61.fsf@praet.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-BeenThere: notmuch@notmuchmail.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Jan 2012 17:16:32 -0000 On Sun, 27 Nov 2011 22:36:12 +0400, Dmitry Kurochkin wrote: > Changes: > > v3 since v2: > > * rebased the remaining patches on current master > > v2 since v1: > > * Add test_require_external_prereq function to explicitly check for > external dependencies, use it in test_emacs. > > * Indenting fixes. > > * Use $binary instead of $1 in test_declare_external_prereq. > > Regards, > Dmitry > > _______________________________________________ > notmuch mailing list > notmuch@notmuchmail.org > http://notmuchmail.org/mailman/listinfo/notmuch Fantastic work Dmitry! My inept dabblings are thoroughly humbled by the elegance of this approach! All that's left to go from three to four nines, is ensuring correct $test_{count,success,fixed,broken,failure} reports (which can most definitively be categorized as "non-trivial")... Eg. with and without emacs(1), emacsclient(1), dtach(1), gpg(1) : #+begin_example Notmuch test suite complete. All 347 tests behaved as expected (2 expected failures). #+end_example #+begin_example Notmuch test suite complete. 266/312 tests passed. 1 broken test failed as expected. 5 tests failed. 40 tests skipped. #+end_example ... as well as a minor issue re test_emacs(), which will be addressed in a followup patch. Peace -- Pieter