Xapian lockup when writing to the notmuch database

Xapian lockup when writing to the notmuch database

[Guyzmo]

Hello,

I have migrated my mail configuration from an old machine to a new one, which is a xen 
server with several VM instances. I have one VM dedicated to do mail stuff, and I want
to manage my Maildir using notmuch in another one. So I set up an NFSv4 share (with
sync option set) from the mail server, to have my Maildir accessible from the other VM. 
Of course, I made sure my user is able to have full privileges over the mounted share,
and there’s only one instance of notmuch running over that xapian db at all times.

All read only operations work just perfectly fine, I can even run notmuch-kz and have
it list my search mailboxes.

But when I do `notmuch new` it hangs. So I did ran it through gdb and here is the result:

% gdb notmuch
(gdb) run new
Starting program: /usr/local/bin/notmuch new
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/i386-linux-gnu/i686/cmov/libthread_db.so.1".
^C
Program received signal SIGINT, Interrupt.
0xb7fe1428 in __kernel_vsyscall ()
(gdb) bt
#0  0xb7fe1428 in __kernel_vsyscall ()
#1  0xb7d2a953 in read () from /lib/i386-linux-gnu/i686/cmov/libc.so.6
#2  0xb7adf538 in read (__nbytes=1, __buf=0xbfffedfc, __fd=<optimized out>) at /usr/include/i386-linux-gnu/bits/unistd.h:45
#3  FlintLock::lock (this=0x8079180, exclusive=true, explanation=...) at ../backends/flint_lock.cc:222
#4  0xb7b30a86 in ChertDatabase::get_database_write_lock (this=this@entry=0x80789a0, creating=creating@entry=false)
    at ../backends/chert/chert_database.cc:505
#5  0xb7b34fd2 in ChertDatabase::ChertDatabase (this=this@entry=0x80789a0, chert_dir=..., action=action@entry=1, block_size=block_size@entry=8192)
    at ../backends/chert/chert_database.cc:154
#6  0xb7b35554 in ChertWritableDatabase::ChertWritableDatabase (this=0x80789a0, dir=..., action=1, block_size=8192)
    at ../backends/chert/chert_database.cc:1036
#7  0xb7adcee4 in Xapian::WritableDatabase::WritableDatabase (this=0x80780c8, path=..., action=1) at ../backends/dbfactory.cc:490
#8  0xb7fb8c68 in notmuch_database_open_verbose (path=0x80780c8 "\340\327ŷ", mode=NOTMUCH_DATABASE_MODE_READ_WRITE, database=0xbffff230, 
    status_string=0xbffff248) at lib/database.cc:933
#9  0x08054e93 in notmuch_new_command (config=0x80745b8, argc=1, argv=0xbffff678) at notmuch-new.c:1008
#10 0x0804df87 in main (argc=2, argv=0xbffff674) at notmuch.c:421
(gdb) 

then I tried doing another write operation, doing a `notmuch tag` command

% gdb notmuch
(gdb) run tag +tag id:<a_unique_mail_id>
(gdb) bt
#0  0xb7fe1428 in __kernel_vsyscall ()
#1  0xb7d2a953 in read () from /lib/i386-linux-gnu/i686/cmov/libc.so.6
#2  0xb7adf538 in read (__nbytes=1, __buf=0xbfffefac, __fd=<optimized out>) at /usr/include/i386-linux-gnu/bits/unistd.h:45
#3  FlintLock::lock (this=0x8079360, exclusive=true, explanation=...) at ../backends/flint_lock.cc:222
#4  0xb7b30a86 in ChertDatabase::get_database_write_lock (this=this@entry=0x8078b80, creating=creating@entry=false)
    at ../backends/chert/chert_database.cc:505
#5  0xb7b34fd2 in ChertDatabase::ChertDatabase (this=this@entry=0x8078b80, chert_dir=..., action=action@entry=1, block_size=block_size@entry=8192)
    at ../backends/chert/chert_database.cc:154
#6  0xb7b35554 in ChertWritableDatabase::ChertWritableDatabase (this=0x8078b80, dir=..., action=1, block_size=8192)
    at ../backends/chert/chert_database.cc:1036
#7  0xb7adcee4 in Xapian::WritableDatabase::WritableDatabase (this=0x80780c8, path=..., action=1) at ../backends/dbfactory.cc:490
#8  0xb7fb8c68 in notmuch_database_open_verbose (path=0x80780c8 "\340\327ŷ", mode=NOTMUCH_DATABASE_MODE_READ_WRITE, database=0xbffff3e4, 
    status_string=0xbffff39c) at lib/database.cc:933
#9  0xb7fb975c in notmuch_database_open (path=0x8078068 "/home/guyzmo/Maildir", mode=mode@entry=NOTMUCH_DATABASE_MODE_READ_WRITE, 
    database=database@entry=0xbffff3e4) at lib/database.cc:848
#10 0x0805cbe4 in notmuch_tag_command (config=0x80745b8, argc=3, argv=0xbffff638) at notmuch-tag.c:262
#11 0x0804df87 in main (argc=4, argv=0xbffff634) at notmuch.c:421
(gdb) 

I’m running notmuch on a Debian Jessie (v7), on which I compiled latest notmuch
from git, using the repository’s libxapian which is v1.2.12-2.

Now, I could use a newer version of libxapian, either V1.2.19 from the debian 
backports (https://packages.debian.org/jessie/libxapian22), or get the latest
compiling it from sources, as it looks like the flintlock.cc compilation unit
has been rewritten since v1.2.12 — though it looks like the part where it hangs
is looking quite alike, but it’s hard to tell from the changes whether that 
would have an inpact.

So my question is whether this is a known issue, and is actually related to
the fact that I’m running it over NFS. I looked for NFS related comments on
xapian website and so far I found nothing looking alike my issue.

I’ll try testing with other versions of xapian and see it solves it. If the
issue is still on with HEAD of xapian’s sources, then I’ll do a bug report.
Until then, I’m just checking on this list for ideas and comments, in hope
for a working solution.

Cheers,

-- 
Guyzmo

Xapian lockup when writing to the notmuch database

[Guyzmo]

Hello,

I have migrated my mail configuration from an old machine to a new one, which is a xen 
server with several VM instances. I have one VM dedicated to do mail stuff, and I want
to manage my Maildir using notmuch in another one. So I set up an NFSv4 share (with
sync option set) from the mail server, to have my Maildir accessible from the other VM. 
Of course, I made sure my user is able to have full privileges over the mounted share,
and there’s only one instance of notmuch running over that xapian db at all times.

All read only operations work just perfectly fine, I can even run notmuch-kz and have
it list my search mailboxes.

But when I do `notmuch new` it hangs. So I did ran it through gdb and here is the result:

% gdb notmuch
(gdb) run new
Starting program: /usr/local/bin/notmuch new
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/i386-linux-gnu/i686/cmov/libthread_db.so.1".
^C
Program received signal SIGINT, Interrupt.
0xb7fe1428 in __kernel_vsyscall ()
(gdb) bt
#0  0xb7fe1428 in __kernel_vsyscall ()
#1  0xb7d2a953 in read () from /lib/i386-linux-gnu/i686/cmov/libc.so.6
#2  0xb7adf538 in read (__nbytes=1, __buf=0xbfffedfc, __fd=<optimized out>) at /usr/include/i386-linux-gnu/bits/unistd.h:45
#3  FlintLock::lock (this=0x8079180, exclusive=true, explanation=...) at ../backends/flint_lock.cc:222
#4  0xb7b30a86 in ChertDatabase::get_database_write_lock (this=this@entry=0x80789a0, creating=creating@entry=false)
  at ../backends/chert/chert_database.cc:505
#5  0xb7b34fd2 in ChertDatabase::ChertDatabase (this=this@entry=0x80789a0, chert_dir=..., action=action@entry=1, block_size=block_size@entry=8192)
  at ../backends/chert/chert_database.cc:154
#6  0xb7b35554 in ChertWritableDatabase::ChertWritableDatabase (this=0x80789a0, dir=..., action=1, block_size=8192)
  at ../backends/chert/chert_database.cc:1036
#7  0xb7adcee4 in Xapian::WritableDatabase::WritableDatabase (this=0x80780c8, path=..., action=1) at ../backends/dbfactory.cc:490
#8  0xb7fb8c68 in notmuch_database_open_verbose (path=0x80780c8 "\340\327ŷ", mode=NOTMUCH_DATABASE_MODE_READ_WRITE, database=0xbffff230, 
  status_string=0xbffff248) at lib/database.cc:933
#9  0x08054e93 in notmuch_new_command (config=0x80745b8, argc=1, argv=0xbffff678) at notmuch-new.c:1008
#10 0x0804df87 in main (argc=2, argv=0xbffff674) at notmuch.c:421
(gdb) 

then I tried doing another write operation, doing a `notmuch tag` command

% gdb notmuch
(gdb) run tag +tag id:<a_unique_mail_id>
(gdb) bt
#0  0xb7fe1428 in __kernel_vsyscall ()
#1  0xb7d2a953 in read () from /lib/i386-linux-gnu/i686/cmov/libc.so.6
#2  0xb7adf538 in read (__nbytes=1, __buf=0xbfffefac, __fd=<optimized out>) at /usr/include/i386-linux-gnu/bits/unistd.h:45
#3  FlintLock::lock (this=0x8079360, exclusive=true, explanation=...) at ../backends/flint_lock.cc:222
#4  0xb7b30a86 in ChertDatabase::get_database_write_lock (this=this@entry=0x8078b80, creating=creating@entry=false)
  at ../backends/chert/chert_database.cc:505
#5  0xb7b34fd2 in ChertDatabase::ChertDatabase (this=this@entry=0x8078b80, chert_dir=..., action=action@entry=1, block_size=block_size@entry=8192)
  at ../backends/chert/chert_database.cc:154
#6  0xb7b35554 in ChertWritableDatabase::ChertWritableDatabase (this=0x8078b80, dir=..., action=1, block_size=8192)
  at ../backends/chert/chert_database.cc:1036
#7  0xb7adcee4 in Xapian::WritableDatabase::WritableDatabase (this=0x80780c8, path=..., action=1) at ../backends/dbfactory.cc:490
#8  0xb7fb8c68 in notmuch_database_open_verbose (path=0x80780c8 "\340\327ŷ", mode=NOTMUCH_DATABASE_MODE_READ_WRITE, database=0xbffff3e4, 
  status_string=0xbffff39c) at lib/database.cc:933
#9  0xb7fb975c in notmuch_database_open (path=0x8078068 "/home/guyzmo/Maildir", mode=mode@entry=NOTMUCH_DATABASE_MODE_READ_WRITE, 
  database=database@entry=0xbffff3e4) at lib/database.cc:848
#10 0x0805cbe4 in notmuch_tag_command (config=0x80745b8, argc=3, argv=0xbffff638) at notmuch-tag.c:262
#11 0x0804df87 in main (argc=4, argv=0xbffff634) at notmuch.c:421
(gdb) 

I’m running notmuch on a Debian Wheezy, on which I compiled latest notmuch
from git, using the repository’s libxapian which is v1.2.12-2.

Now, I could use a newer version of libxapian, either V1.2.16 from the debian 
wheezy backports, or get the latest compiling it from sources, as it looks 
like the flintlock.cc compilation unit has been rewritten since v1.2.12
—  though it looks like the part where it hangs is looking quite alike, but 
it’s hard to tell from the changes whether that would have an inpact.

So my question is whether this is a known issue, and is actually related to
the fact that I’m running it over NFS. I looked for NFS related comments on
xapian website and so far I found nothing looking alike my issue.

I’ll try testing with other versions of xapian and see it solves it. If the
issue is still on with HEAD of xapian’s sources, then I’ll do a bug report.
Until then, I’m just checking on this list for ideas and comments, in hope
for a working solution.

Cheers,

-- 
Guyzmo

Re: Xapian lockup when writing to the notmuch database

[Guyzmo]

Hello list,

On Wed, Jan 06, 2016 at 03:49:28PM +0000, Guyzmo wrote:
[…]
> But when I do `notmuch new` it hangs. So I did ran it through gdb and here is the result:
[…]
> Until then, I’m just checking on this list for ideas and comments, in hope
> for a working solution.

Finally the working solution went from discussion with Bremner on IRC,
as he asked whether the rpc.lockd was indeed running and accessible, I
noticed that I meant the Maildir to be shared in NFSv4, and it ended
up in NFSv3, without lockd support.

After some configuration tweaking, and made sure it mounted in NFSv4,
it finally worked out!

Thank you Bremner, and thank you Olly for pinging me on IRC!

Cheers,

-- 
Guyzmo