From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from localhost (localhost [127.0.0.1]) by arlo.cworth.org (Postfix) with ESMTP id 7A7F96DE0291 for ; Thu, 12 Jul 2018 04:22:51 -0700 (PDT) X-Virus-Scanned: Debian amavisd-new at cworth.org X-Spam-Flag: NO X-Spam-Score: -0.001 X-Spam-Level: X-Spam-Status: No, score=-0.001 tagged_above=-999 required=5 tests=[AWL=0.010, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=disabled Received: from arlo.cworth.org ([127.0.0.1]) by localhost (arlo.cworth.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1LAy2NpFyXiQ for ; Thu, 12 Jul 2018 04:22:50 -0700 (PDT) Received: from fethera.tethera.net (fethera.tethera.net [198.245.60.197]) by arlo.cworth.org (Postfix) with ESMTPS id 73F5A6DE0282 for ; Thu, 12 Jul 2018 04:22:50 -0700 (PDT) Received: from remotemail by fethera.tethera.net with local (Exim 4.89) (envelope-from ) id 1fdZgL-0005i2-Fr; Thu, 12 Jul 2018 07:22:45 -0400 Received: (nullmailer pid 32370 invoked by uid 1000); Thu, 12 Jul 2018 11:22:44 -0000 From: David Bremner To: Mike Hommey , notmuch@notmuchmail.org Cc: xapian-discuss@lists.xapian.org Subject: Re: Error while compacting: Bad position key In-Reply-To: <20180710031628.kowwcvfg6bmx4iyv@glandium.org> References: <20180710031628.kowwcvfg6bmx4iyv@glandium.org> X-List-To: notmuch Date: Thu, 12 Jul 2018 08:22:44 -0300 Message-ID: <87efg8r94r.fsf@tethera.net> MIME-Version: 1.0 Content-Type: text/plain X-BeenThere: notmuch@notmuchmail.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Jul 2018 11:22:51 -0000 Mike Hommey writes: > Hi, > > When running `notmuch compact` today, it stopped with the following > output: > > Compacting database... > compacting table postlist > Reduced by 25% 648656K (2498904K -> 1850248K) > compacting table docdata > Reduced by 15% 24K (152K -> 128K) > compacting table termlist > Reduced by 1% 27008K (2211800K -> 2184792K) > compacting table position > Error while compacting: Bad position key I had not seen anything like this before, but when I run xapian-check from xapian 1.4.6-2, I see termlist: B-tree checked okay doclen not within bounds doclen not within bounds doclen not within bounds doclen not within bounds doclen not within bounds doclen not within bounds doclen not within bounds doclen not within bounds [repeated 52498 times] OTOH, notmuch compact completes for me, so that might be unrelated. I'm copying the Xapian list for input. > Compaction failed: A Xapian exception occurred > > Running xapian-check says: > > docdata: > blocksize=8K items=2677 firstunused=19 revision=15425 levels=1 root=17 > B-tree checked okay > docdata table structure checked OK > > termlist: > blocksize=8K items=2986940 firstunused=276475 revision=15425 levels=2 root=271786 > B-tree checked okay > doclen not within bounds > (...) > doclen not within bounds > termlist table errors found: 107982 > > postlist: > blocksize=8K items=16090818 firstunused=312363 revision=15425 levels=3 root=249894 > B-tree checked okay > postlist table structure checked OK > > position: > blocksize=8K items=236476398 firstunused=653990 revision=15425 levels=3 root=598684 > xapian-check: DatabaseError: Block 459158 item 179: not in sorted order > What version of Xapian is this? I guess an obvious question is if there is some potential external cause of filesystem corruption (e.g. smart errrors, power loss at an inconvenient time). > Is there something I can do, or do I essentially need to completely > rebuild the database (and if so, what's the best way to do it?) > xapian-check has an option F to attempt to fix things. It might not help, but it shouldn't hurt. Otherwise something like % notmuch dump > backup.txt % mv Maildir/.notmuch $backup % notmuch new % notmuch restore < backup.txt will rebuild the database.