From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from localhost (localhost [127.0.0.1]) by arlo.cworth.org (Postfix) with ESMTP id 7A3106DE0209 for ; Mon, 30 Jul 2018 15:46:35 -0700 (PDT) X-Virus-Scanned: Debian amavisd-new at cworth.org X-Spam-Flag: NO X-Spam-Score: 0 X-Spam-Level: X-Spam-Status: No, score=0 tagged_above=-999 required=5 tests=[AWL=0.011, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=disabled Received: from arlo.cworth.org ([127.0.0.1]) by localhost (arlo.cworth.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id dHcqvsh2nd_D for ; Mon, 30 Jul 2018 15:46:34 -0700 (PDT) Received: from fethera.tethera.net (fethera.tethera.net [198.245.60.197]) by arlo.cworth.org (Postfix) with ESMTPS id 41E306DE092F for ; Mon, 30 Jul 2018 15:46:31 -0700 (PDT) Received: from remotemail by fethera.tethera.net with local (Exim 4.89) (envelope-from ) id 1fkGvu-0002Cc-AP; Mon, 30 Jul 2018 18:46:30 -0400 Received: (nullmailer pid 28751 invoked by uid 1000); Mon, 30 Jul 2018 22:45:56 -0000 From: David Bremner To: notmuch@notmuchmail.org Subject: [PATCH 15/15] test: add known broken test for multiple thread terms per message Date: Tue, 31 Jul 2018 06:45:55 +0800 Message-Id: <20180730224555.26047-16-david@tethera.net> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20180730224555.26047-1-david@tethera.net> References: <20180730224555.26047-1-david@tethera.net> X-BeenThere: notmuch@notmuchmail.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 22:46:35 -0000 Having multiple thread terms on a message document seems to be the underlying cause of some confusing results from notmuch search. The presence of these multiple thread terms is presumably an indexing bug, related to multiple files with the same message-id. The files here are synthesized from a reproducer for the problems in id:1523007700.l8xm6nm6af.naveen@linux.ibm.com. It isn't quite clear this is the same issue (the symptoms using notmuch-search are a bit different). --- test/.gitignore | 1 + test/Makefile.local | 7 ++++++- test/T720-database-schema.sh | 16 ++++++++++++++++ test/corpora/threading/mutant-ref/file1 | 9 +++++++++ test/corpora/threading/mutant-ref/file2 | 9 +++++++++ test/corpora/threading/mutant-ref/file3 | 9 +++++++++ test/corpora/threading/mutant-ref/file4 | 7 +++++++ test/corpora/threading/mutant-ref/file5 | 7 +++++++ test/term-report.cc | 22 ++++++++++++++++++++++ 9 files changed, 86 insertions(+), 1 deletion(-) create mode 100755 test/T720-database-schema.sh create mode 100644 test/corpora/threading/mutant-ref/file1 create mode 100644 test/corpora/threading/mutant-ref/file2 create mode 100644 test/corpora/threading/mutant-ref/file3 create mode 100644 test/corpora/threading/mutant-ref/file4 create mode 100644 test/corpora/threading/mutant-ref/file5 create mode 100644 test/term-report.cc diff --git a/test/.gitignore b/test/.gitignore index 73fe7e24..71bbd7ed 100644 --- a/test/.gitignore +++ b/test/.gitignore @@ -8,4 +8,5 @@ /make-db-version /test-results /ghost-report +/term-report /tmp.* diff --git a/test/Makefile.local b/test/Makefile.local index 1cf09778..c39feace 100644 --- a/test/Makefile.local +++ b/test/Makefile.local @@ -44,6 +44,9 @@ $(dir)/make-db-version: $(dir)/make-db-version.o $(dir)/ghost-report: $(dir)/ghost-report.o $(call quiet,CXX) $^ -o $@ $(LDFLAGS) $(XAPIAN_LDFLAGS) +$(dir)/term-report: $(dir)/term-report.o + $(call quiet,CXX) $^ -o $@ $(LDFLAGS) $(XAPIAN_LDFLAGS) + .PHONY: test check test_main_srcs=$(dir)/arg-test.c \ @@ -54,7 +57,9 @@ test_main_srcs=$(dir)/arg-test.c \ $(dir)/symbol-test.cc \ $(dir)/make-db-version.cc \ $(dir)/ghost-report.cc \ - $(dir)/message-id-parse.c + $(dir)/message-id-parse.c \ + $(dir)/term-report.cc + test_srcs=$(test_main_srcs) $(dir)/database-test.c diff --git a/test/T720-database-schema.sh b/test/T720-database-schema.sh new file mode 100755 index 00000000..a6a99239 --- /dev/null +++ b/test/T720-database-schema.sh @@ -0,0 +1,16 @@ +#!/usr/bin/env bash +test_description="database schema in lib/database.cc" + +. $(dirname "$0")/test-lib.sh || exit 1 + +add_email_corpus threading + +test_begin_subtest "every document has at most one thread term" +test_subtest_known_broken +${TEST_DIRECTORY}/term-report ${MAIL_DIR}/.notmuch/xapian | perl -ane 'pop(@F); printf "%d\n",scalar(grep { m/^G/ } @F);' | sort -u > OUTPUT +cat <> EXPECTED +0 +1 +EOF +test_expect_equal_file EXPECTED OUTPUT +test_done diff --git a/test/corpora/threading/mutant-ref/file1 b/test/corpora/threading/mutant-ref/file1 new file mode 100644 index 00000000..97f8db58 --- /dev/null +++ b/test/corpora/threading/mutant-ref/file1 @@ -0,0 +1,9 @@ +From: Alice +To: Daniel +Subject: leaf message +In-Reply-To: +References: +Message-ID: +Date: Thu, 16 Jun 2016 22:14:41 -0400 + +body diff --git a/test/corpora/threading/mutant-ref/file2 b/test/corpora/threading/mutant-ref/file2 new file mode 100644 index 00000000..2b2ccd1d --- /dev/null +++ b/test/corpora/threading/mutant-ref/file2 @@ -0,0 +1,9 @@ +From: Alice +To: Daniel +Subject: leaf message +In-Reply-To: +References: +Message-ID: +Date: Thu, 16 Jun 2016 22:14:41 -0400 + +body diff --git a/test/corpora/threading/mutant-ref/file3 b/test/corpora/threading/mutant-ref/file3 new file mode 100644 index 00000000..a8e705bc --- /dev/null +++ b/test/corpora/threading/mutant-ref/file3 @@ -0,0 +1,9 @@ +From: Alice +To: Daniel +Subject: leaf message +In-Reply-To: +References: +Message-ID: +Date: Thu, 16 Jun 2016 22:14:41 -0400 + +body diff --git a/test/corpora/threading/mutant-ref/file4 b/test/corpora/threading/mutant-ref/file4 new file mode 100644 index 00000000..3a0a5a13 --- /dev/null +++ b/test/corpora/threading/mutant-ref/file4 @@ -0,0 +1,7 @@ +From: Daniel +To: Alice +Subject: existing parent +Message-ID: +Date: Fri, 17 Jun 2016 22:14:41 -0400 + +body diff --git a/test/corpora/threading/mutant-ref/file5 b/test/corpora/threading/mutant-ref/file5 new file mode 100644 index 00000000..8f525d63 --- /dev/null +++ b/test/corpora/threading/mutant-ref/file5 @@ -0,0 +1,7 @@ +From: Daniel +To: Alice +Subject: existing parent +Message-ID: +Date: Fri, 17 Jun 2016 22:14:41 -0400 + +body diff --git a/test/term-report.cc b/test/term-report.cc new file mode 100644 index 00000000..88cd1bf5 --- /dev/null +++ b/test/term-report.cc @@ -0,0 +1,22 @@ +#include +#include +#include + +int main(int argc, char **argv) { + + if (argc < 2) { + std::cerr << "usage: term-report xapian-dir" << std::endl; + exit(1); + } + + Xapian::Database db(argv[1]); + for (Xapian::docid id(1); id < db.get_lastdocid(); id++) { + std::cout << id; + for (Xapian::TermIterator iter = db.termlist_begin(id); + iter != db.termlist_end(id); + iter++) { + std::cout << " " << *iter; + } + std::cout << std::endl; + } +} -- 2.18.0