bug report

bug report

[Andy Wills]

Hi,

I've been consistently getting an error of this form:

andy@andy-x200:~/work/talks/2017/icps$ notmuch new 
Processed 230 total files in 7s (29 files/sec.).
Added 212 new messages to the database. Removed 85 messages. Detected 18 file renames.
Error: A Xapian exception occurred flushing database: Value in posting list too large.

...and it seems that, since I've been getting this error, new emails
are not reliably being found by, for example M-x notmuch in emacs.

Can anyone help?

Best

Andy Wills
Psychology
Plymouth University, UK

-

Re: bug report

[David Bremner]

Andy Wills <home@willslab.co.uk> writes:

>
> andy@andy-x200:~/work/talks/2017/icps$ notmuch new 
> Processed 230 total files in 7s (29 files/sec.).
> Added 212 new messages to the database. Removed 85 messages. Detected 18 file renames.
> Error: A Xapian exception occurred flushing database: Value in posting list too large.
>
> ...and it seems that, since I've been getting this error, new emails
> are not reliably being found by, for example M-x notmuch in emacs.
>

Hi Andy;

What version of notmuch is that? From the error message it sounds
pre-2014. That particular code was rewritten in notmuch 0.19, so if my
guess is correct it might be fixed by newer notmuch.

d

Re: bug report

[David Bremner]

David Bremner <david@tethera.net> writes:

> Andy Wills <home@willslab.co.uk> writes:
>
>>
>> andy@andy-x200:~/work/talks/2017/icps$ notmuch new 
>> Processed 230 total files in 7s (29 files/sec.).
>> Added 212 new messages to the database. Removed 85 messages. Detected 18 file renames.
>> Error: A Xapian exception occurred flushing database: Value in posting list too large.
>>
>> ...and it seems that, since I've been getting this error, new emails
>> are not reliably being found by, for example M-x notmuch in emacs.
>>
>
> Hi Andy;
>
> What version of notmuch is that? From the error message it sounds
> pre-2014. That particular code was rewritten in notmuch 0.19, so if my
> guess is correct it might be fixed by newer notmuch.

The other generic advice is to run 'xapian-check' to check your database
for corruption. On Debian and derivatives (e.g. Ubuntu) this is in the
package 'xapian-tools'.

Re: bug report

[Rafael Avila de Espindola]

>>> andy@andy-x200:~/work/talks/2017/icps$ notmuch new 
>>> Processed 230 total files in 7s (29 files/sec.).
>>> Added 212 new messages to the database. Removed 85 messages. Detected 18 file renames.
>>> Error: A Xapian exception occurred flushing database: Value in posting list too large.

I have seen similar "Xapian exception". I don't remember the exact
message. It happens while running "notmuch new" while reading email on
emacs.

> The other generic advice is to run 'xapian-check' to check your database
> for corruption. On Debian and derivatives (e.g. Ubuntu) this is in the
> package 'xapian-tools'.

It found no errors for me.

Cheers,
Rafael

Re: bug report

[David Bremner]

Rafael Avila de Espindola <rafael.espindola@gmail.com> writes:

>>>> andy@andy-x200:~/work/talks/2017/icps$ notmuch new 
>>>> Processed 230 total files in 7s (29 files/sec.).
>>>> Added 212 new messages to the database. Removed 85 messages. Detected 18 file renames.
>>>> Error: A Xapian exception occurred flushing database: Value in posting list too large.
>
> I have seen similar "Xapian exception". I don't remember the exact
> message. It happens while running "notmuch new" while reading email on
> emacs.

It's hard to be sure without more details, but it sounds like a locking
problem, which could also be be fixed by upgrading to current xapian and
notmuch.

d

Re: bug report

[Rafael Avila de Espindola]

David Bremner <david@tethera.net> writes:

> It's hard to be sure without more details, but it sounds like a locking
> problem, which could also be be fixed by upgrading to current xapian and
> notmuch.

I currently have notmuch-0.23.5-1.fc25.x86_64 and
xapian-core-1.2.24-1.fc25.x86_64.

I haven't seen the error yesterday, so maybe it was solved already.

Thanks,
Rafael

Re: bug report

[David Bremner]

Rafael Avila de Espindola <rafael.espindola@gmail.com> writes:

> David Bremner <david@tethera.net> writes:
>
>> It's hard to be sure without more details, but it sounds like a locking
>> problem, which could also be be fixed by upgrading to current xapian and
>> notmuch.
>
> I currently have notmuch-0.23.5-1.fc25.x86_64 and
> xapian-core-1.2.24-1.fc25.x86_64.

You need notmuch built against xapian 1.4 to fix the locking error I was
thinking of.

d

Re: bug report

[David Bremner]

Andy Wills <home@willslab.co.uk> writes:

> Hi,
>
> I've been consistently getting an error of this form:
>
> andy@andy-x200:~/work/talks/2017/icps$ notmuch new 
> Processed 230 total files in 7s (29 files/sec.).
> Added 212 new messages to the database. Removed 85 messages. Detected 18 file renames.
> Error: A Xapian exception occurred flushing database: Value in posting list too large.
>
> ...and it seems that, since I've been getting this error, new emails
> are not reliably being found by, for example M-x notmuch in emacs.
>

Andy reported privately that an upgrade seems to have fixed the problem.

d

bug report

[Valentin Kuznetsov]

Hi,
I'm new to notmuch system, but I used the tool for quite some time which is
integrated with my mutt setup.

I found that the following query does not return any results
notmuch search CSBS-D-20-00016
while if I do this query
notmuch search CSBS-D-20-00015
I see my results. I provide debug output for both queries below. And in both cases I
have emails which contains these patterns in their subject fields. The only
difference I see is that the last digit is different. I can find those emails
easily with find/grep/etc. tools.

Could you please guide me how to resolve this?
Thanks
Valentin.

shell> notmuch search CSBS-D-20-00016
Query string is:
CSBS-D-20-00016
Exclude query is:
Query((Kdeleted OR Kspam))
Final query is:
Query(((Tmail AND ((csbs@1 PHRASE 4 d@2 PHRASE 4 20@3 PHRASE 4 00016@4) OR (Gcsbs@1 PHRASE 4 Gd@2 PHRASE 4 G20@3 PHRASE 4 G00016@4) OR (Kcsbs@1 PHRASE 4 Kd@2 PHRASE 4 K20@3 PHRASE 4 K00016@4) OR (Kcsbs@1 PHRASE 4 Kd@2 PHRASE 4 K20@3 PHRASE 4 K00016@4) OR (Qcsbs@1 PHRASE 4 Qd@2 PHRASE 4 Q20@3 PHRASE 4 Q00016@4) OR (Qcsbs@1 PHRASE 4 Qd@2 PHRASE 4 Q20@3 PHRASE 4 Q00016@4) OR (Pcsbs@1 PHRASE 4 Pd@2 PHRASE 4 P20@3 PHRASE 4 P00016@4) OR (XPROPERTYcsbs@1 PHRASE 4 XPROPERTYd@2 PHRASE 4 XPROPERTY20@3 PHRASE 4 XPROPERTY00016@4) OR (XFOLDER:csbs@1 PHRASE 4 XFOLDER:d@2 PHRASE 4 XFOLDER:20@3 PHRASE 4 XFOLDER:00016@4) OR (XFROMcsbs@1 PHRASE 4 XFROMd@2 PHRASE 4 XFROM20@3 PHRASE 4 XFROM00016@4) OR (XTOcsbs@1 PHRASE 4 XTOd@2 PHRASE 4 XTO20@3 PHRASE 4 XTO00016@4) OR (XATTACHMENTcsbs@1 PHRASE 4 XATTACHMENTd@2 PHRASE 4 XATTACHMENT20@3 PHRASE 4 XATTACHMENT00016@4) OR (XMIMETYPEcsbs@1 PHRASE 4 XMIMETYPEd@2 PHRASE 4 XMIMETYPE20@3 PHRASE 4 XMIMETYPE00016@4) OR (XSUBJECTcsbs@1 PHRASE 4 XSUBJECTd@2 PHRASE 4 X
 SUBJECT20@3 PHRASE 4 XSUBJECT00016@4))) AND_NOT (Kdeleted OR Kspam)))


shell> notmuch search CSBS-D-20-00015
Query string is:
CSBS-D-20-00015
Exclude query is:
Query((Kdeleted OR Kspam))
Final query is:
Query(((Tmail AND ((csbs@1 PHRASE 4 d@2 PHRASE 4 20@3 PHRASE 4 00015@4) OR (Gcsbs@1 PHRASE 4 Gd@2 PHRASE 4 G20@3 PHRASE 4 G00015@4) OR (Kcsbs@1 PHRASE 4 Kd@2 PHRASE 4 K20@3 PHRASE 4 K00015@4) OR (Kcsbs@1 PHRASE 4 Kd@2 PHRASE 4 K20@3 PHRASE 4 K00015@4) OR (Qcsbs@1 PHRASE 4 Qd@2 PHRASE 4 Q20@3 PHRASE 4 Q00015@4) OR (Qcsbs@1 PHRASE 4 Qd@2 PHRASE 4 Q20@3 PHRASE 4 Q00015@4) OR (Pcsbs@1 PHRASE 4 Pd@2 PHRASE 4 P20@3 PHRASE 4 P00015@4) OR (XPROPERTYcsbs@1 PHRASE 4 XPROPERTYd@2 PHRASE 4 XPROPERTY20@3 PHRASE 4 XPROPERTY00015@4) OR (XFOLDER:csbs@1 PHRASE 4 XFOLDER:d@2 PHRASE 4 XFOLDER:20@3 PHRASE 4 XFOLDER:00015@4) OR (XFROMcsbs@1 PHRASE 4 XFROMd@2 PHRASE 4 XFROM20@3 PHRASE 4 XFROM00015@4) OR (XTOcsbs@1 PHRASE 4 XTOd@2 PHRASE 4 XTO20@3 PHRASE 4 XTO00015@4) OR (XATTACHMENTcsbs@1 PHRASE 4 XATTACHMENTd@2 PHRASE 4 XATTACHMENT20@3 PHRASE 4 XATTACHMENT00015@4) OR (XMIMETYPEcsbs@1 PHRASE 4 XMIMETYPEd@2 PHRASE 4 XMIMETYPE20@3 PHRASE 4 XMIMETYPE00015@4) OR (XSUBJECTcsbs@1 PHRASE 4 XSUBJECTd@2 PHRASE 4 X
 SUBJECT20@3 PHRASE 4 XSUBJECT00015@4))) AND_NOT (Kdeleted OR Kspam)))
Query string is:
thread:000000000002e0e2
Exclude query is:
Query()
Final query is:
Query((Tmail AND 0 * G000000000002e0e2))
thread:000000000002e0e2      July 07 [2/2] Federica Legger, Valentin Kuznetsov; Fwd: CSBS-D-20-00015 - Ad Hoc from Author to Editorial Office (inbox replied)
Query string is:
thread:000000000002e0d7
Exclude query is:
Query()
Final query is:
Query((Tmail AND 0 * G000000000002e0d7))
thread:000000000002e0d7      July 07 [2/2] Federica Legger, Valentin Kuznetsov; Fwd: Submission Confirmation (inbox replied)
Query string is:
thread:000000000002e0cf
Exclude query is:
Query()
Final query is:
Query((Tmail AND 0 * G000000000002e0cf))
thread:000000000002e0cf      July 07 [1/1] Federica Legger; CSBS-D-20-00015 - Ad Hoc from Author to Editorial Office (inbox)

Re: bug report

[David Bremner]

Valentin Kuznetsov <vkuznet@protonmail.com> writes:

> Hi,
> I'm new to notmuch system, but I used the tool for quite some time which is
> integrated with my mutt setup.
>
> I found that the following query does not return any results
> notmuch search CSBS-D-20-00016
> while if I do this query
> notmuch search CSBS-D-20-00015
> I see my results. I provide debug output for both queries below. And in both cases I
> have emails which contains these patterns in their subject fields. The only
> difference I see is that the last digit is different. I can find those emails
> easily with find/grep/etc. tools.
>
> Could you please guide me how to resolve this?
> Thanks
> Valentin.

Hi Valentin;

Sorry your report got lost for so long. I can think a few things to try
if you are still interested in resolving it

0) Try your searches with the subject: prefix; if nothing else this
   should reduce the debugging output.
1) Try your searches with --exclude=false
2) Try a regular expression search subject:/CSBS-D-20-00016/

d