[BUG] notmuch crashes on FreeBSD when synchronize_flags=true

[BUG] notmuch crashes on FreeBSD when synchronize_flags=true

[Simon Campese]

Hello,

I'm trying to setup notmuch on my FreeBSD machine (version 10.1). As
soon as I run 'notmuch new' (without an existing .notmuch directory),
notmuch crashes with the error message 'abort (core dumped)   notmuch
new'. An example output is: 

--- BEGIN shell-output ---

freebsd-desk% notmuch new
Found 50363 total files (that's not much mail).
zsh: abort (core dumped)  notmuch new
freebsd-desk%

--- END shell-output ---

If I put 'synchronize_flags=false' into my config, 'notmuch new' works fine and
indexes everything. Here's a backtrace from gdb:

--- BEGIN gdb-session ---

bsd-desk% gdb notmuch
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "amd64-marcel-freebsd"...(no debugging symbols found)...
(gdb) run new
Starting program: /usr/local/bin/notmuch new
(no debugging symbols found)...(no debugging symbols found)...(no debugging symbols found)...(no debugging symbols found)...(no debugging symbols found)...(no debugging symbols found)...(no debugging symbols found)...[New LWP 101945]
(no debugging symbols found)...(no debugging symbols found)...(no debugging symbols found)...(no debugging symbols found)...(no debugging symbols found)...(no debugging symbols found)...(no debugging symbols found)...(no debugging symbols found)...(no debugging symbols found)...(no debugging symbols found)...(no debugging symbols found)...(no debugging symbols found)...(no debugging symbols found)...(no debugging symbols found)...(no debugging symbols found)...(no debugging symbols found)...[New Thread 804406400 (LWP 101945/notmuch)]
(no debugging symbols found)...(no debugging symbols found)...(no debugging symbols found)...(no debuggingFound 50363 total files (that's not much mail).
Program received signal SIGABRT, Aborted.
[Switching to Thread 804406400 (LWP 101945/notmuch)]
0x00000008018d16ca in thr_kill () from /lib/libc.so.7
(gdb) bt
#0  0x00000008018d16ca in thr_kill () from /lib/libc.so.7
#1  0x00000008019a6149 in abort () from /lib/libc.so.7
#2  0x00000008022df42d in __cxa_rethrow () from /lib/libcxxrt.so.1
#3  0x0000000801c5a7e6 in Xapian::Document::Internal::remove_term () from /usr/local/lib/libxapian.so.22
#4  0x000000080084dd1c in notmuch_message_get_replies () from /usr/local/lib/libnotmuch.so.3
#5  0x000000080084f0cf in notmuch_message_remove_tag () from /usr/local/lib/libnotmuch.so.3
#6  0x000000080084f2d5 in notmuch_message_maildir_flags_to_tags () from /usr/local/lib/libnotmuch.so.3
#7  0x000000000040d3e5 in ?? ()
#8  0x000000000040ce48 in ?? ()
#9  0x000000000040ce48 in ?? ()
#10 0x000000000040ce48 in ?? ()
#11 0x000000000040c26b in ?? ()
#12 0x000000000040878b in ?? ()
#13 0x000000000040798f in ?? ()
#14 0x000000080063b000 in ?? ()
#15 0x0000000000000000 in ?? ()
(gdb)

--- END gdb-session ---

The error described above occurs, no matter if I install the notmuch
package (and its dependencies) via binary packages or build from
ports. I've tried version 0.18 and version 0.19. Also, it doesn't 
matter if I start with a maildir containing just a single mail, or my
real maildir with thousands of messages.

On my linux box (running the same verions of notmuch and its
dependencies), the error doesn't occur, so I guess it has something to
do with the different C++-Stacks (libcxxrt is FreeBSD's ABI library).
A quick glance at the notmuch code showed that
notmuch_message_get_replies() sometimes returns NULL. I'm not a C++
expert but could it be that this might be the problem here?

To give you some version numbers, this is what's currently installed:

notmuch: 0.19
xapian-core: 1.2.20
gmime: 2.6.20
talloc: 2.1.1

To reproduce the error, you can do the following:

- perform a fresh install of FreeBSD
- install the package or port, for example: 'pkg install notmuch'
- create a standard config file containing 'synchronize_flags=true' and
  pointing to some maildir
- run 'notmuch new' 

I would be very thankful for a fix (or hints what might be the problem).  


Thanks for your help,

Simon

Re: [BUG] notmuch crashes on FreeBSD when synchronize_flags=true

[David Bremner]

Simon Campese <notmuchmail_org@campese.de> writes:


> #4  0x000000080084dd1c in notmuch_message_get_replies () from /usr/local/lib/libnotmuch.so.3

Just confirm, this particular backtrace is from notmuch 0.18, right?
because 0.19 should build / be-linked-against libnotmuch.so.4.

> On my linux box (running the same verions of notmuch and its
> dependencies), the error doesn't occur, so I guess it has something to
> do with the different C++-Stacks (libcxxrt is FreeBSD's ABI library).
> A quick glance at the notmuch code showed that
> notmuch_message_get_replies() sometimes returns NULL. I'm not a C++
> expert but could it be that this might be the problem here?

No, I don't think so, that's a defined part of the API.

The usual problem on *BSD some incompatibility between glibc and BSD's
interpretation of POSIX. OTOH I couldn't identify something like that
from the backtrace you included.

I would be curious if the Xapian test suite runs in your environment.

> To give you some version numbers, this is what's currently installed:
>
> notmuch: 0.19
> xapian-core: 1.2.20

notmuch probably hasn't been very well tested against xapian 1.2.20,
since it was only release a few weeks ago, and isn't available widely
(or more precisely, in Debian) as binary packages.  Are you running that
on your linux box as well?

d

Re: [BUG] notmuch crashes on FreeBSD when synchronize_flags=true

[Simon Campese]

>> #4  0x000000080084dd1c in notmuch_message_get_replies () from
>> /usr/local/lib/libnotmuch.so.3

>Just confirm, this particular backtrace is from notmuch 0.18, right?
> because 0.19 should build / be-linked-against libnotmuch.so.4.

Yes, sorry, I mixed up my text snippets. The backtrace from notmuch 0.19
is analogous with the current libnotmuch referenced.

>I would be curious if the Xapian test suite runs in your environment.

I'm not very familiar with neither notmuch, nor xapian. Where can I find
this test suite? Are you referring to the 'test' directory in the
notmuch tree? If so, I have the impression that a lot has to be patched
to make it run under FreeBSD (I get a myriad of errors when trying to
compile). 

>> To give you some version numbers, this is what's currently installed:
>>
>> notmuch: 0.19
>> xapian-core: 1.2.20

>notmuch probably hasn't been very well tested against xapian 1.2.20,
>since it was only release a few weeks ago, and isn't available widely
>(or more precisely, in Debian) as binary packages.  Are you running that
>on your linux box as well?

Yes, on linux I have the exact same version numbers of notmuch and all
dependencies (including xapian) and everything works fine.

The bug has already been reported on the freebsd bug tracker:

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=193809

I've referenced this thread there, hopefully the problem will get sorted
out. Would be a shame if such a nice email program remains only
partially functional on BSD. 

Re: [BUG] notmuch crashes on FreeBSD when synchronize_flags=true

[Tomi Ollila]

On Sat, Mar 21 2015, Simon Campese <notmuchmail_org@campese.de> wrote:

> Hello,
>
> I'm trying to setup notmuch on my FreeBSD machine (version 10.1). As
> soon as I run 'notmuch new' (without an existing .notmuch directory),
> notmuch crashes with the error message 'abort (core dumped)   notmuch
> new'. An example output is: 
>
> --- BEGIN shell-output ---
>
> freebsd-desk% notmuch new
> Found 50363 total files (that's not much mail).
> zsh: abort (core dumped)  notmuch new
> freebsd-desk%
>
> --- END shell-output ---
>
> If I put 'synchronize_flags=false' into my config, 'notmuch new' works fine and
> indexes everything. Here's a backtrace from gdb:
>
> --- BEGIN gdb-session ---
>
> bsd-desk% gdb notmuch
> GNU gdb 6.1.1 [FreeBSD]
> Copyright 2004 Free Software Foundation, Inc.
> GDB is free software, covered by the GNU General Public License, and you are
> welcome to change it and/or distribute copies of it under certain conditions.
> Type "show copying" to see the conditions.
> There is absolutely no warranty for GDB.  Type "show warranty" for details.
> This GDB was configured as "amd64-marcel-freebsd"...(no debugging symbols found)...
> (gdb) run new
> Starting program: /usr/local/bin/notmuch new
> (no debugging symbols found)...(no debugging symbols found)...(no debugging symbols found)...(no debugging symbols found)...(no debugging symbols found)...(no debugging symbols found)...(no debugging symbols found)...[New LWP 101945]
> (no debugging symbols found)...(no debugging symbols found)...(no debugging symbols found)...(no debugging symbols found)...(no debugging symbols found)...(no debugging symbols found)...(no debugging symbols found)...(no debugging symbols found)...(no debugging symbols found)...(no debugging symbols found)...(no debugging symbols found)...(no debugging symbols found)...(no debugging symbols found)...(no debugging symbols found)...(no debugging symbols found)...(no debugging symbols found)...[New Thread 804406400 (LWP 101945/notmuch)]
> (no debugging symbols found)...(no debugging symbols found)...(no debugging symbols found)...(no debuggingFound 50363 total files (that's not much mail).
> Program received signal SIGABRT, Aborted.
> [Switching to Thread 804406400 (LWP 101945/notmuch)]
> 0x00000008018d16ca in thr_kill () from /lib/libc.so.7
> (gdb) bt
> #0  0x00000008018d16ca in thr_kill () from /lib/libc.so.7
> #1  0x00000008019a6149 in abort () from /lib/libc.so.7
> #2  0x00000008022df42d in __cxa_rethrow () from /lib/libcxxrt.so.1
> #3  0x0000000801c5a7e6 in Xapian::Document::Internal::remove_term () from /usr/local/lib/libxapian.so.22
> #4  0x000000080084dd1c in notmuch_message_get_replies () from /usr/local/lib/libnotmuch.so.3
> #5  0x000000080084f0cf in notmuch_message_remove_tag () from /usr/local/lib/libnotmuch.so.3
> #6  0x000000080084f2d5 in notmuch_message_maildir_flags_to_tags () from /usr/local/lib/libnotmuch.so.3
> #7  0x000000000040d3e5 in ?? ()
> #8  0x000000000040ce48 in ?? ()
> #9  0x000000000040ce48 in ?? ()
> #10 0x000000000040ce48 in ?? ()
> #11 0x000000000040c26b in ?? ()
> #12 0x000000000040878b in ?? ()
> #13 0x000000000040798f in ?? ()
> #14 0x000000080063b000 in ?? ()
> #15 0x0000000000000000 in ?? ()
> (gdb)
>
> --- END gdb-session ---

I downloaded (from somewhere) FreeBSD i386 LXDE VirtualBox (.ova) image
and after a while figuring out how to get stuff installed I got
notmuch 0.18.1 installed.

In case I create ~/mail/cur, ~/mail/new and ~/mail/tmp directories,
but mail(s) to ~/mail/new and execute notmuch new...

... notmuch coredumps and I get pretty much the same output as the


My guess is that there is a slight error how this part is displayed:

> #3  0x0000000801c5a7e6 in Xapian::Document::Internal::remove_term () from
> #4  0x000000080084dd1c in notmuch_message_get_replies () from
> #5  0x000000080084f0cf in notmuch_message_remove_tag () from
> #6  0x000000080084f2d5 in notmuch_message_maildir_flags_to_tags () from

-- I'd day #4 is actually _notmuch_message_remove_term () 

To resolve this further I suggest that SomeOne(tm) (that means you, Simon)
build notmuch like the following:

  make distclean
  CFLAGS='-ggdb -O0' ./configure
  make

(but no make install -- and next)

  rm -rf ~/mail/.notmuch
  gdb ./notmuch
  run new

and capture the gdb backtrace (let's hope Heisenberg does't intrude!)

(remember to have mails in ~/main/new and synchronize_flags in .notmuch-config.


Tomi

>
> I would be very thankful for a fix (or hints what might be the problem).  
>
>
> Thanks for your help,
>
> Simon

Re: [BUG] notmuch crashes on FreeBSD when synchronize_flags=true

[David Bremner]

Simon Campese <notmuchmail_org@campese.de> writes:

>>> #4  0x000000080084dd1c in notmuch_message_get_replies () from
>>> /usr/local/lib/libnotmuch.so.3
>
>>Just confirm, this particular backtrace is from notmuch 0.18, right?
>> because 0.19 should build / be-linked-against libnotmuch.so.4.
>
> Yes, sorry, I mixed up my text snippets. The backtrace from notmuch 0.19
> is analogous with the current libnotmuch referenced.
>
>>I would be curious if the Xapian test suite runs in your environment.
>
> I'm not very familiar with neither notmuch, nor xapian. Where can I find
> this test suite? Are you referring to the 'test' directory in the
> notmuch tree? If so, I have the impression that a lot has to be patched
> to make it run under FreeBSD (I get a myriad of errors when trying to
> compile). 

Well, it's a bit discouraging that the notmuch test suite doesn't
compile on FreeBSD, but no I refer the Xapian test suite, presumably
included with the xapian-core source.

I mentioned your troubles to Xapian upstream developer Olly Betts, and
he told me there were some known problems with the Xapian test suite
when compiled with clang. If you are using clang, it would be useful to
see if you can duplicate the problem with xapian and notmuch compiled
with gcc.

Re: [BUG] notmuch crashes on FreeBSD when synchronize_flags=true

[Tomi Ollila]

FYI (VirtualBox FreeBSD_10.1-64bit.7z from http://www.osboxes.org/freebsd/)
 
  root@osboxes:~/mail # rm -rf .notmuch/
  root@osboxes:~/mail # notmuch new
  Found 1 total files (that's not much mail).
  Warning: /root/mail/new/new.mail is an mbox containing a single message,
  likely caused by misconfigured mail delivery.  Support for single-message
  mboxes is deprecated and may be removed in the future.
  Abort (core dumped)

Built latest HEAD (bed8b6743) in $HOME/sw/notmuch

  root@osboxes:~/mail # rm -rf .notmuch/
  root@osboxes:~/mail # ~/sw/notmuch/notmuch new
  Found 2 total files (that's not much mail).
  Note: Ignoring non-mail file: /root/mail/notmuch.core
  Processed 2 total files in almost no time.
  Added 1 new message to the database.
  root@osboxes:~/mail #

OK, what worked :O (statically linked libnotmuch.4). Wat about shared...

  root@osboxes:~/mail # rm -rf .notmuch/ notmuch.core
  root@osboxes:~/mail # env LD_LIBRARY_PATH=$HOME/sw/notmuch/lib $HOME/sw/notmuch/notmuch-shared new
  Found 1 total files (that's not much mail).
  Abort (core dumped)

  root@osboxes:~/mail # gdb $HOME/sw/notmuch/notmuch-shared notmuch-shared.core
  GNU gdb 6.1.1 [FreeBSD]
  ...
  Core was generated by `notmuch-shared'.
  Program terminated with signal 6, Aborted.
  Reading symbols from /root/sw/notmuch/lib/libnotmuch.so.4...done.
  ...
  [New Thread 804406400 (LWP 100071/notmuch-shared)]
(gdb) bt
#0  0x00000008018d46ca in thr_kill () from /lib/libc.so.7
#1  0x00000008019a9149 in abort () from /lib/libc.so.7
#2  0x00000008022e242d in __cxa_rethrow () from /lib/libcxxrt.so.1
#3  0x0000000801c5d7d6 in Xapian::Document::Internal::remove_term ()
   from /usr/local/lib/libxapian.so.22
#4  0x00000008008500bc in _notmuch_message_remove_term
  (message=0x804468780, 
    prefix_name=0x800858414 "tag", value=<value optimized out>)
    at message.cc:1146
#5  0x000000080085158f in notmuch_message_remove_tag (message=0x804468780, 
    tag=0x800859167 "draft") at message.cc:1205
#6  0x0000000800851795 in notmuch_message_maildir_flags_to_tags (
    message=<value optimized out>) at message.cc:1312
#7  0x000000000040d9e5 in add_files (notmuch=0x8044542a0, 
    path=0x80445a3b0 "/root/mail/new", state=0x7fffffffe8b0)
    at notmuch-new.c:272
#8  0x000000000040d448 in add_files (notmuch=0x8044542a0, 
    path=0x804459fc0 "/root/mail", state=0x7fffffffe8b0) at
  notmuch-new.c:477
#9  0x000000000040c85f in notmuch_new_command (config=0x804468160, 
    argc=<value optimized out>, argv=<value optimized out>)
    at notmuch-new.c:1070
#10 0x0000000000408a23 in main (argc=<value optimized out>, 
    argv=0x7fffffffeb18) at notmuch.c:353

Due to just quick testing forgot my suggestion for CFLAGS... so...

   143 7:27 cd
   144 7:27 cd sw/notmuch
   145 7:27 make distclean
   146 7:28 gmake distclean
   147 7:28 env CFLAGS='-ggdb -O0' ./configure
   148 7:28 make
   149 7:28 gmake
   151 7:29 cd
   152 7:29 cd mail/
   153 7:29 ls
   154 7:29 rm notmuch-shared.core
   155 7:29 env LD_LIBRARY_PATH=$HOME/sw/notmuch/lib $HOME/sw/notmuch/notmuch-shared new
   156 7:29 gdb $HOME/sw/notmuch/notmuch-shared notmuch-shared.core

(gdb) bt
#0  0x00000008018f26ca in thr_kill () from /lib/libc.so.7
#1  0x00000008019c7149 in abort () from /lib/libc.so.7
#2  0x000000080230042d in __cxa_rethrow () from /lib/libcxxrt.so.1
#3  0x0000000801c7b7d6 in Xapian::Document::Internal::remove_term ()
   from /usr/local/lib/libxapian.so.22
#4  0x00000008008668b3 in _notmuch_message_remove_term
(message=0x804468780, 
    prefix_name=0x800874151 "tag", value=0x800874e97 "draft") at
    message.cc:1146
#5  0x0000000800868f76 in notmuch_message_remove_tag (message=0x804468780, 
    tag=0x800874e97 "draft") at message.cc:1205
#6  0x00000008008691d3 in notmuch_message_maildir_flags_to_tags (
    message=0x804468780) at message.cc:1312
#7  0x0000000000411295 in add_file (notmuch=0x8044542a0, 
    filename=0x80445c6e0 "/root/mail/new/new.mail", state=0x7fffffffe8b8)
    at notmuch-new.c:272
#8  0x00000000004102e2 in add_files (notmuch=0x8044542a0, 
    path=0x80445a3b0 "/root/mail/new", state=0x7fffffffe8b8)
    at notmuch-new.c:587
#9  0x000000000040fdad in add_files (notmuch=0x8044542a0, 
    path=0x804459fc0 "/root/mail", state=0x7fffffffe8b8) at
    notmuch-new.c:477
#10 0x000000000040ee81 in notmuch_new_command (config=0x804468160, argc=1, 
    argv=0x7fffffffeb20) at notmuch-new.c:1070
#11 0x0000000000409277 in main (argc=2, argv=0x7fffffffeb18) at
notmuch.c:353

Tomi

Re: [BUG] notmuch crashes on FreeBSD when synchronize_flags=true

[David Bremner]

Simon Campese <notmuchmail_org@campese.de> writes:

> Hello,
>
> I'm trying to setup notmuch on my FreeBSD machine (version 10.1). As
> soon as I run 'notmuch new' (without an existing .notmuch directory),
> notmuch crashes with the error message 'abort (core dumped)   notmuch
> new'. An example output is: 
>
> --- BEGIN shell-output ---
>
> freebsd-desk% notmuch new
> Found 50363 total files (that's not much mail).
> zsh: abort (core dumped)  notmuch new
> freebsd-desk%

According to

      https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=193809

this bug is only in the pre-compiled versions. I have just pushed a
patch to halt the build if c++filt is not detected, so at least those
wrong packages will not be built.  To actually build notmuch you'll need
to make sure c++filt is available.

d