From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from localhost (localhost [127.0.0.1]) by arlo.cworth.org (Postfix) with ESMTP id E79726DE025B for ; Fri, 6 Apr 2018 18:49:32 -0700 (PDT) X-Virus-Scanned: Debian amavisd-new at cworth.org X-Spam-Flag: NO X-Spam-Score: 0 X-Spam-Level: X-Spam-Status: No, score=0 tagged_above=-999 required=5 tests=[AWL=0.011, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=disabled Received: from arlo.cworth.org ([127.0.0.1]) by localhost (arlo.cworth.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 028o8S7he05M for ; Fri, 6 Apr 2018 18:49:32 -0700 (PDT) Received: from fethera.tethera.net (fethera.tethera.net [198.245.60.197]) by arlo.cworth.org (Postfix) with ESMTPS id D54CD6DE0259 for ; Fri, 6 Apr 2018 18:49:31 -0700 (PDT) Received: from remotemail by fethera.tethera.net with local (Exim 4.89) (envelope-from ) id 1f4cyu-0000yU-Vg; Fri, 06 Apr 2018 21:49:29 -0400 Received: (nullmailer pid 11976 invoked by uid 1000); Sat, 07 Apr 2018 01:49:27 -0000 From: David Bremner To: Javier Garcia , notmuch@notmuchmail.org Subject: Re: Database corruption after clean rebuild In-Reply-To: <8a311263-a46f-3327-4054-d8d863fa0169@gmail.com> References: <8a311263-a46f-3327-4054-d8d863fa0169@gmail.com> Date: Fri, 06 Apr 2018 22:49:27 -0300 Message-ID: <874lknwztk.fsf@tethera.net> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-BeenThere: notmuch@notmuchmail.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 07 Apr 2018 01:49:33 -0000 --=-=-= Content-Type: text/plain Javier Garcia writes: > I can't build a healthy database for notmuch. My mail directory has > quite a few mails, around 20,000. > > $ rm -rf ~/.mail/.notmuch > $ notmuch new > $ xapian-check ~/.mail/.notmuch/xapian/ >> docdata: >> blocksize=8K items=63 firstunused=1 revision=2 levels=0 root=0 >> B-tree checked okay >> docdata table structure checked OK >> >> termlist: >> blocksize=8K items=43520 firstunused=8291 revision=2 levels=2 root=748 >> xapian-check: DatabaseError: 1 unused block(s) missing from the free > list, first is 0 > There was recently a similar report that turned out to be related to a reference loop in the mail. Do you actually have any symptoms of database corruption other than the message about the free list? if not, it might be worth trying the attached patch, which attempts to break reference loops. --=-=-= Content-Type: text/x-diff Content-Disposition: inline; filename=0001-WIP-test-patch-for-reference-loop-problem.patch >From 753c8d366f3ffde2a14de7157b55b27b555b39d8 Mon Sep 17 00:00:00 2001 From: David Bremner Date: Mon, 2 Apr 2018 08:02:05 -0300 Subject: [PATCH] WIP: test patch for reference loop problem --- lib/thread.cc | 22 ++++++++++++++++++++-- 1 file changed, 20 insertions(+), 2 deletions(-) diff --git a/lib/thread.cc b/lib/thread.cc index 3561b27f..356d63ce 100644 --- a/lib/thread.cc +++ b/lib/thread.cc @@ -391,10 +391,15 @@ static void _resolve_thread_relationships (notmuch_thread_t *thread) { notmuch_message_node_t *node; - notmuch_message_t *message, *parent; + notmuch_message_t *message, *first_message = NULL, *parent; const char *in_reply_to; - for (node = thread->message_list->head; node; node = node->next) { + node = thread->message_list->head; + if (node) { + first_message = node->message; + node = node->next; + } + for (; node; node = node->next) { message = node->message; in_reply_to = _notmuch_message_get_in_reply_to (message); if (in_reply_to && strlen (in_reply_to) && @@ -406,6 +411,19 @@ _resolve_thread_relationships (notmuch_thread_t *thread) _notmuch_message_list_add_message (thread->toplevel_list, message); } + /* XXX: this is probably nonsense: if we didn't find any top level + * messages, choose one at random */ + if (first_message) { + in_reply_to = _notmuch_message_get_in_reply_to (first_message); + if (thread->toplevel_list->head && in_reply_to && strlen (in_reply_to) && + g_hash_table_lookup_extended (thread->message_hash, + in_reply_to, NULL, + (void **) &parent)) + _notmuch_message_add_reply (parent, first_message); + else + _notmuch_message_list_add_message (thread->toplevel_list, first_message); + } + /* XXX: After scanning through the entire list looking for parents * via "In-Reply-To", we should do a second pass that looks at the * list of messages IDs in the "References" header instead. (And -- 2.16.3 --=-=-=--