Notmuch suddenly stopped working

Notmuch suddenly stopped working

[Neeum Zawan]

Hi,

My setup:

Running version 0.24.2

I use maildir. Suddenly, a few days ago, notmuch stopped indexing. When
I run notmuch new, I get:

Processed 274 total files in almost no time.
Added 33 new messages to the database.

But it doesn't get added. No error message. A subsequent run gives me
the same result (or more "new messages" if new mail has arrived). I have
a post hook Python script, and it runs fine, but sees no new mails. I've
set up notmuch to tag all new mails with "new", but none of them are
tagged that way. I disabled the Python script - it made no difference.

I made no changes to my computer - did not reboot, did not install
anything, etc.

(BTW, usual search operations are working fine for items already in the
database). 

Help me debug!

Thanks,

-- 
85% of all statistics are wrong.


                    /\  /\               /\  /
                   /  \/  \ u e e n     /  \/  a w a z
                       >>>>>>mueen@nawaz.org<<<<<<
                                   anl

 

Re: Notmuch suddenly stopped working

[David Bremner]

Neeum Zawan <mailinglists@nawaz.org> writes:

> Hi,
>
> My setup:
>
> Running version 0.24.2
>
> I use maildir. Suddenly, a few days ago, notmuch stopped indexing. When
> I run notmuch new, I get:
>
> Processed 274 total files in almost no time.
> Added 33 new messages to the database.
>
> But it doesn't get added. No error message. A subsequent run gives me
> the same result (or more "new messages" if new mail has arrived). I have
> a post hook Python script, and it runs fine, but sees no new mails. I've
> set up notmuch to tag all new mails with "new", but none of them are
> tagged that way. I disabled the Python script - it made no difference.

The only things that happen after that message is closing the notmuch
database and running the post-new hook. IIUC, you said you disabled the
post-new hook so that leaves closing the database. Not sure what could
go wrong there, but some things to try

- check disk space
- run xapian-check on the xapian database
  Something like:
            % xapian-check ~/Maildir/.notmuch/xapian

- run in gdb and make sure it really runs the call to
  notmuch_database_destroy
  
- apply the following patch to notmuch-new.c and rebuild to get more
  information:

diff --git a/notmuch-new.c b/notmuch-new.c
index c4345705..447f9f7d 100644
--- a/notmuch-new.c
+++ b/notmuch-new.c
@@ -1280,7 +1280,8 @@ notmuch_new_command (notmuch_config_t *config, int argc, char *argv[])
        fprintf (stderr, "Note: A fatal error was encountered: %s\n",
                 notmuch_status_to_string (ret));
 
-    notmuch_database_destroy (notmuch);
+    print_status_database("notmuch new", notmuch,
+                         notmuch_database_destroy (notmuch));
 
     if (hooks && !ret && !interrupted)
        ret = notmuch_run_hook (db_path, "post-new");

- run notmuch new under strace

Re: Notmuch suddenly stopped working

[Neeum Zawaz]

David Bremner <david@tethera.net> writes:

> The only things that happen after that message is closing the notmuch
> database and running the post-new hook. IIUC, you said you disabled the
> post-new hook so that leaves closing the database. Not sure what could
> go wrong there, but some things to try

Would the post-new hook run if the database did not close? In my case
the post-new script was running. 

>
> - check disk space
> - run xapian-check on the xapian database

Disk space is fine.

xapian-check gives:

record:
baseA blocksize=8K items=85613 lastblock=285 revision=73183 levels=1 root=268
B-tree checked okay
record table structure checked OK

termlist:
baseA blocksize=8K items=171226 lastblock=68023 revision=73183 levels=2 root=27394
B-tree checked okay
termlist table structure checked OK

postlist:
baseA blocksize=8K items=2413415 lastblock=53350 revision=73183 levels=2 root=35
Failed to check B-tree: DatabaseError: Stored total free space value wrong
position:
baseB blocksize=8K items=22315184 lastblock=75964 revision=73183 levels=2 root=75964
B-tree checked okay
position table structure checked OK

spelling:
Lazily created, and not yet used.

synonym:
Lazily created, and not yet used.

Total errors found: 1

Not sure how serious that error is.

Now the weird thing: Overnight, it suddenly started working fine again!
I didn't do anything (not even the check command).

If it happens again, I'll try the patch you sent.

Thanks,


-- 
When I was young, they told me anyone could become president. Now I'm
beginning to believe it.


                    /\  /\               /\  /
                   /  \/  \ u e e n     /  \/  a w a z
                       >>>>>>mueen@nawaz.org<<<<<<
                                   anl

 

Re: Notmuch suddenly stopped working

[David Bremner]

Neeum Zawaz <mailinglists@nawaz.org> writes:

> David Bremner <david@tethera.net> writes:
>
>> The only things that happen after that message is closing the notmuch
>> database and running the post-new hook. IIUC, you said you disabled the
>> post-new hook so that leaves closing the database. Not sure what could
>> go wrong there, but some things to try
>
> Would the post-new hook run if the database did not close? In my case
> the post-new script was running. 
>

Currently that code does not abort if the database fails to close, so
yes, if could run the hook in either case.

>>
>> - check disk space
>> - run xapian-check on the xapian database
>
> Disk space is fine.
>
> xapian-check gives:
>
> postlist:
> baseA blocksize=8K items=2413415 lastblock=53350 revision=73183 levels=2 root=35
> Failed to check B-tree: DatabaseError: Stored total free space value wrong
> position:
> baseB blocksize=8K items=22315184 lastblock=75964 revision=73183 levels=2 root=75964
> B-tree checked okay
> position table structure checked OK

That looks a bit worrying. Maybe someone on the xapian-discuss [1] list
can advise you how serious the problem is. From a notmuch point of view
I would suggest backing up your database with 'notmuch dump',
reindexing, and restoring with 'notmuch restore'.


[1]: xapian-discuss@lists.xapian.org