On Wed, Aug 01 2012, David Bremner <david@tethera.net> wrote:
> As I mentioned on IRC, the test only fails on the Debian build machines
> (building in a clean chroot using sbuild is not enough) so it isn't
> really clear how to duplicate the it. Perhaps building in a clean
> virtual machine without networking would do it.  For which tests fail,
> see
>
> https://buildd.debian.org/status/fetch.php?pkg=notmuch&arch=i386&ver=0.13.2-1&stamp=1338740444
>
> I think the first things to fail are emacs tests. At a wild guess, it
> looks like all of the failing tests are related to emacs.

From a cursory look that does appear to be the case.  The non-emacs
tests that are also failing (json and crypto) are using
emacs_deliver_message.  Do we have any idea what's going on here?

> I don't think failing the build is the right thing to do, since there
> does not actually seem to be anything wrong with the resulting
> packages. So removing them from Debian because of some problems with
> running the test suite seems a bit extreme.  People not using
> notmuch-emacs would probably be especially annoyed ;).

But it also doesn't make much sense to me to run the tests as part of
the build only to ignore the result.  How do we know that the failing
tests aren't actually affecting the distributed packages?  The emacs
failures could be masking real failures of the emacs interface (which
users of notmuch-emacs would probably also find really annoying!).

jamie.