A systematic way of handling Xapian lock errors?

A systematic way of handling Xapian lock errors?

[Matt Armstrong]

Simple notmuch commands like "notmuch tag" can fail to grab the Xapian
lock.  When this occurs they bail with:

A Xapian exception occurred opening database: Unable to get write lock on /example/.notmuch/xapian: already locked

I've noticed a few issues with this:

1) The notmuch command line doesn't define semantics for exit codes.  I
notice that notmuch exits with 1 for "xapian write lock error".  I
suspect this code is not reserved for write lock errors?  That would be
needed for any sensible downgrade of the write lock error into a soft
failure with a retry loop in things like shell scripts.

2) The notmuch Emacs client just bails whatever it was trying when this
occurs.  I run background mail fetching, which I expect is typical, and
this makes for an unpleasant experience as errors are thrown at me
randomly.  Since my post-injection filter commands run quickly, I'd
rather Emacs simply spin a short while against the lock trying to
perform the command.  Even a one second spin attempt is likely to
succeed most of the time.  But I would happily configure an infinite
retry loop -- Emacs can show me a spinning ball, and I'd rather not be
subject to odd effects due to failures.

To address both, has something like this ever been considered:

  notmuch --lock_timeout <seconds> COMMAND ARG...

Then frontends like Emacs could lean on retry logic written in C.  If
this seems workable, it is something I can try implementing.

Re: A systematic way of handling Xapian lock errors?

[David Bremner]

Matt Armstrong <marmstrong@google.com> writes:


> To address both, has something like this ever been considered:
>
>   notmuch --lock_timeout <seconds> COMMAND ARG...
>
> Then frontends like Emacs could lean on retry logic written in C.  If
> this seems workable, it is something I can try implementing.

notmuch in git master uses xapians lock retry if present (xapian
1.3+). Currently there is no timeout, but Olly seemed receptive to the
idea of adding one, so I'd suggest that's the right place to work on
improving this. You may find that just having unbounded lock retries
(i.e. blocking opens) is enough for your issues.

d

Re: A systematic way of handling Xapian lock errors?

[Matt Armstrong]

David Bremner <david@tethera.net> writes:

> Matt Armstrong <marmstrong@google.com> writes:
>
>
>> To address both, has something like this ever been considered:
>>
>>   notmuch --lock_timeout <seconds> COMMAND ARG...
>>
>> Then frontends like Emacs could lean on retry logic written in C.  If
>> this seems workable, it is something I can try implementing.
>
> notmuch in git master uses xapians lock retry if present (xapian
> 1.3+). Currently there is no timeout, but Olly seemed receptive to the
> idea of adding one, so I'd suggest that's the right place to work on
> improving this. You may find that just having unbounded lock retries
> (i.e. blocking opens) is enough for your issues.

Ah great.  I am still on Xapian 1.2 by way of running a relatively old
Linux distro.  Building a personal version of Xapian as we speak...