From: Gregor Zattler <telegraph@gmx.net>
To: notmuch@notmuchmail.org
Cc: xapian-discuss@lists.xapian.org
Subject: Re: Database corruption after clean rebuild
Date: Sun, 29 Apr 2018 12:17:12 +0200 [thread overview]
Message-ID: <87po2iqpuf.fsf@len.workgroup> (raw)
In-Reply-To: <5bde652e-8ffe-d43d-106d-a43c7a1b4126@gmail.com>
Hi notmuch developers,
I also had this database corruption, I waited for the fix to land
in notmuch 0.26.2, build it, moved the xapian directory away, did
a notmuch new and restored the tags from a dump. But the problem
remains:
~$ xapian-check ~/Mail/.notmuch/xapian
docdata:
blocksize=8K items=10841 firstunused=75 revision=82 levels=1 root=2
B-tree checked okay
docdata table structure checked OK
termlist:
blocksize=8K items=1893162 firstunused=368983 revision=82 levels=3 root=177608
xapian-check: DatabaseError: 1 unused block(s) missing from the free list, first is 0
this is very similar to the old database which I had moved away:
~$ xapian-check ~/Mail/.notmuch/xapian-2018-04-29-00-22/
docdata:
blocksize=8K items=10863 firstunused=78 revision=59623 levels=1 root=2
B-tree checked okay
docdata table structure checked OK
termlist:
blocksize=8K items=1894648 firstunused=360821 revision=59623 levels=3 root=360580
xapian-check: DatabaseError: 1 unused block(s) missing from the free list, first is 0
Now I did notmuch compact and the database check says there are
no errors.
This seems to me as if the fix had not helped or there is another problem.
$ notmuch --version
notmuch 0.26.2+26~g9e158fb
~$ xapian-compact --version
xapian-compact - xapian-core 1.4.3
Thanks for developing notmuch, Gregor
* Javier Garcia <javiertury@gmail.com> [2018-04-07; 17:09]:
> Unfortunately I can't share my emails without the approval of other
> parties. The minimum subsets that trigger the error are in the range of
> 1000-5000 mails, so asking each and everyone of them is out of my reach.
> I tried to replicate the problem using just spam folders without success.
>
> The following is a solid workaround I've stumbled upon. Afew no longer
> complains and database corruption is gone.
>
> $ notmuch compact
> $ xapian-check ~/.mail/.notmuch/xapian
> <check messages>
> No errors found
>
> I built xapian-core 1.50 but I can't compile notmuch 0.26.1 against it.
> I will wait and test again in a few weeks.
>
> If you are interested in my setup, the error happens with this minimal
> configuration.
>
> #~/.config/afew/config
> [Filter.1]
> query = 'folder:"//(INBOX|Inbox|inbox)$/" AND (NOT tag:inbox)'
> tags = +inbox;-new
> message = Messages in INBOX folder are tagged as inbox
>
> [Filter.2]
> query = '(NOT folder:"//(INBOX|Inbox|inbox)$/") AND (tag:inbox)'
> tags = -inbox
> message = Messages not in INBOX folder cannot be inbox
>
> #~/.notmuch-config
> [database]
> path=/home-path/.mail
> [new]
> tags=new
>
> On 07/04/18 12:51, David Bremner wrote:
>> Javier Garcia <javiertury@gmail.com> writes:
>>
>>> I've applied the path to notmuch 0.26.1 without success.
>>>
>>> $ rm -rf ~/.mail/.notmuch
>>> $ LD_LIBRARY_PATH=/hidden-path/notmuch-0.26.1/lib/:$LD_LIBRARY_PATH
>>> ./notmuch new
>>> Found 20065 total files (that's not much mail).
>>> Processed 20065 total files in 58s (341 files/sec.).
>>> Added 19605 new messages to the database.
>>>
>>> $ xapian-check .mail/.notmuch/xapian/
>>> docdata:
>>> blocksize=8K items=63 firstunused=1 revision=2 levels=0 root=0
>>> B-tree checked okay
>>> docdata table structure checked OK
>>> termlist:
>>> blocksize=8K items=43520 firstunused=8293 revision=2 levels=2 root=748
>>> xapian-check: DatabaseError: 1 unused block(s) missing from the free
>>> list, first is 0
>> OK, so probably not related to reference loops (although that patch is
>> not very well tested). It's not clear how notmuch is triggering it, but
>> this looks like the same bug in Xapian that olly fixed recently [1].
>>
>> A possible next step is to try building xapian master, and linking
>> notmuch against that.
>>
>> Maybe Patrick or Justus (in copy) has some idea why you're only seeing
>> problems in afew.
>>
>> Another debugging direction is to try to duplicate your problem with
>> some subset of mail that you're willing to share (bisection is the usual
>> strategy).
>>
>> [1] https://notmuchmail.org/pipermail/notmuch/2018/026369.html
>
> _______________________________________________
> notmuch mailing list
> notmuch@notmuchmail.org
> https://notmuchmail.org/mailman/listinfo/notmuch
Ciao; Gregor
--
-... --- .-. . -.. ..--.. ...-.-
next prev parent reply other threads:[~2018-04-29 10:22 UTC|newest]
Thread overview: 10+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-04-06 16:12 Database corruption after clean rebuild Javier Garcia
2018-04-07 1:49 ` David Bremner
2018-04-07 8:39 ` Javier Garcia
2018-04-07 10:51 ` David Bremner
2018-04-07 15:09 ` Javier Garcia
2018-04-07 15:17 ` David Bremner
2018-04-07 20:58 ` Olly Betts
2018-04-29 10:17 ` Gregor Zattler [this message]
2018-04-29 14:27 ` David Bremner
2018-04-29 18:54 ` Gregor Zattler
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://notmuchmail.org/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87po2iqpuf.fsf@len.workgroup \
--to=telegraph@gmx.net \
--cc=notmuch@notmuchmail.org \
--cc=xapian-discuss@lists.xapian.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://yhetil.org/notmuch.git/
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).