Re: Database corruption after clean rebuild

[David Bremner <david@tethera.net>]

[-- Attachment #1: Type: text/plain, Size: 859 bytes --]

Javier Garcia <javiertury@gmail.com> writes:

> I can't build a healthy database for notmuch. My mail directory has
> quite a few mails, around 20,000.
>
> $ rm -rf ~/.mail/.notmuch
> $ notmuch new
> $ xapian-check ~/.mail/.notmuch/xapian/
>> docdata:
>> blocksize=8K items=63 firstunused=1 revision=2 levels=0 root=0
>> B-tree checked okay
>> docdata table structure checked OK
>>
>> termlist:
>> blocksize=8K items=43520 firstunused=8291 revision=2 levels=2 root=748
>> xapian-check: DatabaseError: 1 unused block(s) missing from the free
> list, first is 0
>

There was recently a similar report that turned out to be related to a
reference loop in the mail.  Do you actually have any symptoms of
database corruption other than the message about the free list? if not,
it might be worth trying the attached patch, which attempts to break
reference loops.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-WIP-test-patch-for-reference-loop-problem.patch --]
[-- Type: text/x-diff, Size: 1988 bytes --]

From 753c8d366f3ffde2a14de7157b55b27b555b39d8 Mon Sep 17 00:00:00 2001
From: David Bremner <david@tethera.net>
Date: Mon, 2 Apr 2018 08:02:05 -0300
Subject: [PATCH] WIP: test patch for reference loop problem

---
 lib/thread.cc | 22 ++++++++++++++++++++--
 1 file changed, 20 insertions(+), 2 deletions(-)

diff --git a/lib/thread.cc b/lib/thread.cc
index 3561b27f..356d63ce 100644
--- a/lib/thread.cc
+++ b/lib/thread.cc
@@ -391,10 +391,15 @@ static void
 _resolve_thread_relationships (notmuch_thread_t *thread)
 {
     notmuch_message_node_t *node;
-    notmuch_message_t *message, *parent;
+    notmuch_message_t *message, *first_message = NULL, *parent;
     const char *in_reply_to;
 
-    for (node = thread->message_list->head; node; node = node->next) {
+    node = thread->message_list->head;
+    if (node) {
+	first_message = node->message;
+	node = node->next;
+    }
+    for (; node; node = node->next) {
 	message = node->message;
 	in_reply_to = _notmuch_message_get_in_reply_to (message);
 	if (in_reply_to && strlen (in_reply_to) &&
@@ -406,6 +411,19 @@ _resolve_thread_relationships (notmuch_thread_t *thread)
 	    _notmuch_message_list_add_message (thread->toplevel_list, message);
     }
 
+    /* XXX: this is probably nonsense: if we didn't find any top level
+     * messages, choose one at random */
+    if (first_message) {
+	in_reply_to = _notmuch_message_get_in_reply_to (first_message);
+	if (thread->toplevel_list->head && in_reply_to && strlen (in_reply_to) &&
+	    g_hash_table_lookup_extended (thread->message_hash,
+					  in_reply_to, NULL,
+					  (void **) &parent))
+	    _notmuch_message_add_reply (parent, first_message);
+	else
+	    _notmuch_message_list_add_message (thread->toplevel_list, first_message);
+    }
+
     /* XXX: After scanning through the entire list looking for parents
      * via "In-Reply-To", we should do a second pass that looks at the
      * list of messages IDs in the "References" header instead. (And
-- 
2.16.3