Bug report: crash of notmuch insert

Bug report: crash of notmuch insert

[Adam C. Emerson]

Hello.

For the last couple weeks I've been getting unpredictable crashes in
notmuch insert.

Once it starts happening, if I run notmuch compact it will stop
happening for a while, then start up again later.

I'm running it from procmail, in this case:

* .*
| reformail -f0 | notmuch insert +inbox

Though it happens for other sets of tags, too.

I am running notmuch 0.25.2 with Xapain version 1.4.4 on Arch
Linux. The problem started with the default build, and to debug it I
recompiled both packages with -O0 -g3.

The backtrace looks like:

Using host libthread_db library "/usr/lib/libthread_db.so.1".
Core was generated by `notmuch insert +inbox'.
Program terminated with signal SIGXFSZ, File size limit exceeded.
#0  0x00007fc116952143 in __pwrite64_nocancel () from /usr/lib/libc.so.6
(gdb) bt
#0  0x00007fc116952143 in __pwrite64_nocancel () at /usr/lib/libc.so.6
#1  0x00007fc1173fbe37 in io_write_block(int, char const*, unsigned long, long, long) (fd=6, p=0x56528e6723f8 "", n=8192, b=6250, o=51200000)
    at common/io_utils.cc:235
    #2  0x00007fc1173cb75a in GlassTable::write_block(unsigned int, unsigned char const*, bool) const (this=0x56528e657448, n=6250, p=0x56528e6723f8 "", appending=false) at backends/glass/glass_table.cc:222
    #3  0x00007fc1173cbc3a in GlassTable::block_to_cursor(Glass::Cursor*, int, unsigned int) const (this=0x56528e657448, C_=0x56528e657510, j=0, n=6186) at backends/glass/glass_table.cc:319
    #4  0x00007fc1173cc4e3 in GlassTable::find(Glass::Cursor*) const (this=0x56528e657448, C_=0x56528e657510) at backends/glass/glass_table.cc:546
    #5  0x00007fc1173ce33e in GlassTable::add(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool) (this=0x56528e657448, key="wrapctr\000%\257", tag="\263\a\241\003", already_compressed=false) at backends/glass/glass_table.cc:1313
    #6  0x00007fc1173bdca1 in GlassPositionListTable::set_positionlist(unsigned int, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) (this=0x56528e657448, did=9647, tname="wrapctr", s="\263\a\241\003") at backends/glass/glass_positionlist.h:70
    #7  0x00007fc1173bdb38 in Inverter::flush_pos_lists(GlassPositionListTable&) (this=0x56528e657e90, table=...)
        at backends/glass/glass_inverter.cc:232
	#8  0x00007fc1173add27 in GlassWritableDatabase::flush_postlist_changes() const (this=0x56528e656ec0) at backends/glass/glass_database.cc:1049
	#9  0x00007fc1173adbdc in GlassWritableDatabase::commit() (this=0x56528e656ec0) at backends/glass/glass_database.cc:1028
	#10 0x00007fc1173add76 in GlassWritableDatabase::close() (this=0x56528e656ec0) at backends/glass/glass_database.cc:1059
	#11 0x00007fc1172e9dd6 in Xapian::Database::close() (this=0x56528e656e10) at api/omdatabase.cc:138
	#12 0x000056528d4e8ed2 in notmuch_database_close(notmuch_database_t*) (notmuch=0x56528e65b970) at lib/database.cc:1147
	#13 0x000056528d4e98c0 in notmuch_database_destroy(notmuch_database_t*) (notmuch=0x56528e65b970) at lib/database.cc:1384
	#14 0x000056528d4d5f17 in notmuch_insert_command (config=0x56528e604a70, argc=2, argv=0x7ffd93381b30) at notmuch-insert.c:553
	#15 0x000056528d4d09d2 in main (argc=3, argv=0x7ffd93381b28) at notmuch.c:456

Thank you very much.

Re: Bug report: crash of notmuch insert

[David Bremner]

"Adam C. Emerson" <azure@fox.blue> writes:

> I am running notmuch 0.25.2 with Xapain version 1.4.4 on Arch
> Linux. The problem started with the default build, and to debug it I
> recompiled both packages with -O0 -g3.
>
> The backtrace looks like:
>
> Using host libthread_db library "/usr/lib/libthread_db.so.1".
> Core was generated by `notmuch insert +inbox'.
> Program terminated with signal SIGXFSZ, File size limit exceeded.

Have you ruled out an actual file size problem? This could be related to
the environment notmuch insert is running in; I have a vague memory of a
related issue encountered by a Postfix user.

d

Re: Bug report: crash of notmuch insert

[Tomi Ollila]

On Tue, Dec 12 2017, David Bremner wrote:

> "Adam C. Emerson" <azure@fox.blue> writes:
>
>> I am running notmuch 0.25.2 with Xapain version 1.4.4 on Arch
>> Linux. The problem started with the default build, and to debug it I
>> recompiled both packages with -O0 -g3.
>>
>> The backtrace looks like:
>>
>> Using host libthread_db library "/usr/lib/libthread_db.so.1".
>> Core was generated by `notmuch insert +inbox'.
>> Program terminated with signal SIGXFSZ, File size limit exceeded.
>
> Have you ruled out an actual file size problem? This could be related to
> the environment notmuch insert is running in; I have a vague memory of a
> related issue encountered by a Postfix user.

Adam,

Does `ulimit -f` help ? what does it print out ?

(I personally have never seen anything else than 'unlimited' there)

Tomi

>
> d

Re: Bug report: crash of notmuch insert

[Adam C. Emerson]

On 12/12/2017, Tomi Ollila wrote:
[snip]
> > Have you ruled out an actual file size problem? This could be related to
> > the environment notmuch insert is running in; I have a vague memory of a
> > related issue encountered by a Postfix user.
>
> Adam,
>
> Does `ulimit -f` help ? what does it print out ?
>
> (I personally have never seen anything else than 'unlimited' there)

Thank you! This fixed it. I wasn't setting a limit myself, but it
turns out that Postfix enforces its mailbox size by running deliver
with an rlimit on the maximum file size (not what I would have
expected). I just set mailbox_size_limit to 0 and I haven't had it
happen since.