From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from localhost (localhost [127.0.0.1]) by arlo.cworth.org (Postfix) with ESMTP id 54FCA6DE0260 for ; Sat, 7 Apr 2018 01:39:38 -0700 (PDT) X-Virus-Scanned: Debian amavisd-new at cworth.org X-Spam-Flag: NO X-Spam-Score: 0.054 X-Spam-Level: X-Spam-Status: No, score=0.054 tagged_above=-999 required=5 tests=[AWL=0.174, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=disabled Received: from arlo.cworth.org ([127.0.0.1]) by localhost (arlo.cworth.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id BVqCTDIRjDvt for ; Sat, 7 Apr 2018 01:39:37 -0700 (PDT) Received: from mail-wm0-f49.google.com (mail-wm0-f49.google.com [74.125.82.49]) by arlo.cworth.org (Postfix) with ESMTPS id 767856DE025C for ; Sat, 7 Apr 2018 01:39:36 -0700 (PDT) Received: by mail-wm0-f49.google.com with SMTP id b127so7591375wmf.5 for ; Sat, 07 Apr 2018 01:39:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:to:references:from:message-id:date:user-agent:mime-version :in-reply-to:content-transfer-encoding:content-language; bh=teIOTNGFUlo2sgncHwB7TQoYMVqGW1wWsYPnSCykvw8=; b=fytDYt0TSBRzp76RAt0UpwzbeMr2KZKMeHoV/cMnRVdeNsbHpCocxZkchemJT6fjOK EnCQI38nSqo6x91aCfhz3wOWLpNt5mO8ni1nTkXq7I7aR8dTDvV1pmlSWjO5ANXB9dFb BI5/s+A6fTePQhQWwfuag6xN7ySiJIKQZKv67IPzvvg0HUv0kh7kd3rBQfwGLPReUDOP Gd/jlurDqUJUKYDBB0ECnFAGX1o8Pv2dF817TKRmoRX5ivP6ED7YSW2uE3b8DpZ8qZgu m5/w8mImO3mbAon6xaozXiEB4TbOUX4AXyS0FGW76R0+9sM5M2I8oQxLJ7Xq8bjdA+v7 Styg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding :content-language; bh=teIOTNGFUlo2sgncHwB7TQoYMVqGW1wWsYPnSCykvw8=; b=TB1sy0K/8pvvqv2b60z4xfMpf3Tr0Ls/lvJmii+Vr5a7r29ZJp4L7HXmqcM/QmOCOr //O9LrchXwVYfN08s2MQP+twpk5Elg2jAFtDaByoWKkjsF3+lT7lG775KPyk3YZO50HE jok1eBt6OeRZ2VreA1nYyVENPYn4OCY/SLx7iWQ3o7yW7fSEi6q1TT/YyocM7l6RfYbV lPvUZE02BVlfYxIC8Fku1dZ4f8J7WVJsC9oDf3wFjB5DEmffE0nWCfEuI62XuaRTEpxk THPviuTeWHJ8EHcK8IpUQj8jjMYdZmXgU95gDrJsizIn6WfKqQbDnhIQhp0csRj8N4tH zjJg== X-Gm-Message-State: ALQs6tA/edtGFtsY5vVSD1F2L4aNCRaMgyKnBmKvtEqMZAwyH3hPmEhG Vyo6zud0bEEsabSHP7dD3hC+a2ge X-Google-Smtp-Source: AIpwx4/WEYEON6gMdBnpcXp/BtJKbfflk6NzFthtylZZRoUJSshwEw8x9iV+6LEimp8f0v0/is2Axw== X-Received: by 10.80.232.72 with SMTP id k8mr11218452edn.38.1523090373378; Sat, 07 Apr 2018 01:39:33 -0700 (PDT) Received: from localhost.localdomain ([87.124.229.17]) by smtp.gmail.com with ESMTPSA id e24sm7684029edc.47.2018.04.07.01.39.31 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 07 Apr 2018 01:39:32 -0700 (PDT) Subject: Re: Database corruption after clean rebuild To: David Bremner , notmuch@notmuchmail.org References: <8a311263-a46f-3327-4054-d8d863fa0169@gmail.com> <874lknwztk.fsf@tethera.net> From: Javier Garcia Message-ID: Date: Sat, 7 Apr 2018 10:39:31 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.7.0 MIME-Version: 1.0 In-Reply-To: <874lknwztk.fsf@tethera.net> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Content-Language: en-US X-Mailman-Approved-At: Sat, 07 Apr 2018 02:53:38 -0700 X-BeenThere: notmuch@notmuchmail.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 07 Apr 2018 08:39:38 -0000 I've applied the path to notmuch 0.26.1 without success. $ rm -rf ~/.mail/.notmuch $ LD_LIBRARY_PATH=/hidden-path/notmuch-0.26.1/lib/:$LD_LIBRARY_PATH ./notmuch new    Found 20065 total files (that's not much mail).    Processed 20065 total files in 58s (341 files/sec.).    Added 19605 new messages to the database. $ xapian-check .mail/.notmuch/xapian/    docdata:    blocksize=8K items=63 firstunused=1 revision=2 levels=0 root=0    B-tree checked okay    docdata table structure checked OK    termlist:    blocksize=8K items=43520 firstunused=8293 revision=2 levels=2 root=748    xapian-check: DatabaseError: 1 unused block(s) missing from the free list, first is 0 With or without the patch, the "corrupted" database works fine most of the time. For instance this works: $ notmuch tag +new2 -- tag:new It's just that afew can't work with the db in this state, it complains saying the database is corrupted. $ rm -rf ~/.mail/.notmuch $ notmuch new $ afew -tn -vv       terminate called after throwing an instance of 'Xapian::DatabaseCorruptError'    Aborted (core dumped) Afew doesn't always crash, even though the database is always corrupted. Afew crashes when it's called just after a fresh notmuch database is built and randomly thereafter. The error is always the same. The following one-liner can solve most of afew crashes. It works well for the cases in which afew is called right after notmuch database creation. For random crashes is not as effective. $ notmuch tag +new2 -- tag:new On 07/04/18 03:49, David Bremner wrote: > Javier Garcia writes: > >> I can't build a healthy database for notmuch. My mail directory has >> quite a few mails, around 20,000. >> >> $ rm -rf ~/.mail/.notmuch >> $ notmuch new >> $ xapian-check ~/.mail/.notmuch/xapian/ >>> docdata: >>> blocksize=8K items=63 firstunused=1 revision=2 levels=0 root=0 >>> B-tree checked okay >>> docdata table structure checked OK >>> >>> termlist: >>> blocksize=8K items=43520 firstunused=8291 revision=2 levels=2 root=748 >>> xapian-check: DatabaseError: 1 unused block(s) missing from the free >> list, first is 0 >> > There was recently a similar report that turned out to be related to a > reference loop in the mail. Do you actually have any symptoms of > database corruption other than the message about the free list? if not, > it might be worth trying the attached patch, which attempts to break > reference loops. >