notmuch seg. fault on _thread_add_message

notmuch seg. fault on _thread_add_message

[Kazuo Teramoto]

Hi!

Following an alot bug report [1] I created a small python script that
causes a segmentation fault in notmuch:
========================================================================
#!/usr/bin/env python2
import notmuch
db = notmuch.Database(mode=notmuch.Database.MODE.READ_WRITE)
q_new = notmuch.Query(db, 'tag:inbox')
for t in q_new.search_threads():
    pass
========================================================================

With gdb I get the following
========================================================================
GNU gdb (GDB) 7.3.1
Copyright (C) 2011 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-unknown-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /usr/bin/python2...(no debugging symbols found)...done.
(gdb) run /tmp/test_notmuch.py 
Starting program: /usr/bin/python2 /tmp/test_notmuch.py
[Thread debugging using libthread_db enabled]

(process:16579): GLib-GObject-CRITICAL **: gtype.c:2715: You forgot to call g_type_init()

(process:16579): GLib-GObject-CRITICAL **: g_object_newv: assertion `G_TYPE_IS_OBJECT (object_type)' failed

(process:16579): GLib-GObject-CRITICAL **: gtype.c:2715: You forgot to call g_type_init()

(process:16579): GLib-GObject-CRITICAL **: gtype.c:2715: You forgot to call g_type_init()

(process:16579): GLib-GObject-CRITICAL **: g_object_newv: assertion `G_TYPE_IS_OBJECT (object_type)' failed

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff4deb86e in decode_addrspec (in=0x7fffffffac80) at internet-address.c:1459
1459		((InternetAddressMailbox *) mailbox)->addr = addr->str;
(gdb) bt
#0  0x00007ffff4deb86e in decode_addrspec (in=0x7fffffffac80) at internet-address.c:1459
#1  0x00007ffff4debb1d in decode_address (in=0x7fffffffacf0) at internet-address.c:1558
#2  0x00007ffff4debe83 in internet_address_list_parse_string (str=0x7b1bc0 "Jani Nikula <jani@nikula.org>") at internet-address.c:1692
#3  0x00007ffff503257b in _thread_add_message (thread=0x7d87f0, message=0x6d9480) at lib/thread.cc:236
#4  0x00007ffff5032d7b in _notmuch_thread_create (ctx=0x7cadd0, notmuch=0x615590, seed_doc_id=6515, match_set=0xe65528, sort=NOTMUCH_SORT_NEWEST_FIRST) at lib/thread.cc:470
#5  0x00007ffff503165b in notmuch_threads_get (threads=0xe65510) at lib/query.cc:392
#6  0x00007ffff6636e34 in ffi_call_unix64 () from /usr/lib/libffi.so.5
#7  0x00007ffff6636855 in ffi_call () from /usr/lib/libffi.so.5
#8  0x00007ffff684a1f7 in _ctypes_callproc () from /usr/lib/python2.7/lib-dynload/_ctypes.so
#9  0x00007ffff6843a86 in ?? () from /usr/lib/python2.7/lib-dynload/_ctypes.so
#10 0x00007ffff7a67683 in PyObject_Call () from /usr/lib/libpython2.7.so.1.0
#11 0x00007ffff7afcbda in PyEval_EvalFrameEx () from /usr/lib/libpython2.7.so.1.0
#12 0x00007ffff7aff8ef in PyEval_EvalCodeEx () from /usr/lib/libpython2.7.so.1.0
#13 0x00007ffff7a8c15c in function_call () from /usr/lib/libpython2.7.so.1.0
#14 0x00007ffff7a67683 in PyObject_Call () from /usr/lib/libpython2.7.so.1.0
#15 0x00007ffff7a762bf in instancemethod_call () from /usr/lib/libpython2.7.so.1.0
#16 0x00007ffff7a67683 in PyObject_Call () from /usr/lib/libpython2.7.so.1.0
#17 0x00007ffff7abd002 in call_method () from /usr/lib/libpython2.7.so.1.0
#18 0x00007ffff7af9b1f in PyEval_EvalFrameEx () from /usr/lib/libpython2.7.so.1.0
#19 0x00007ffff7aff8ef in PyEval_EvalCodeEx () from /usr/lib/libpython2.7.so.1.0
#20 0x00007ffff7affa22 in PyEval_EvalCode () from /usr/lib/libpython2.7.so.1.0
#21 0x00007ffff7b19d8c in run_mod () from /usr/lib/libpython2.7.so.1.0
#22 0x00007ffff7b1ab90 in PyRun_FileExFlags () from /usr/lib/libpython2.7.so.1.0
#23 0x00007ffff7b1b60f in PyRun_SimpleFileExFlags () from /usr/lib/libpython2.7.so.1.0
#24 0x00007ffff7b2cd25 in Py_Main () from /usr/lib/libpython2.7.so.1.0
#25 0x00007ffff749b14d in __libc_start_main () from /lib/libc.so.6
#26 0x00000000004006a1 in _start ()
(gdb) list
1454			g_string_free (addr, TRUE);
1455			return NULL;
1456		}
1457		
1458		mailbox = g_object_newv (INTERNET_ADDRESS_TYPE_MAILBOX, 0, NULL);
1459		((InternetAddressMailbox *) mailbox)->addr = addr->str;
1460		g_string_free (addr, FALSE);
1461		
1462		return mailbox;
1463	}
(gdb) info locals
mailbox = 0x0
start = 0x7b1bcd "jani@nikula.org>"
inptr = 0x7b1bdc ">"
word = 0x7b1bcd "jani@nikula.org>"
got_local = 1
addr = 0x8f8d20
len = 4
(gdb) frame 3
#3  0x00007ffff503257b in _thread_add_message (thread=0x7d87f0, message=0x6d9480) at lib/thread.cc:236
236		list = internet_address_list_parse_string (from);
(gdb) l
231				 xstrdup (notmuch_message_get_message_id (message)),
232				 message);
233	
234	    from = notmuch_message_get_header (message, "from");
235	    if (from)
236		list = internet_address_list_parse_string (from);
237	
238	    if (list) {
239		address = internet_address_list_get_address (list, 0);
240		if (address) {
(gdb) info locals
list = 0x0
from = 0x7b1bc0 "Jani Nikula <jani@nikula.org>"
tags = 0x7b2850
tag = 0x7fffffffad60 "\300\033{"
address = 0x7b2850
author = 0x7fffffffafa8 ""
clean_author = 0x6d9480 "\220Ua"
(gdb) 
========================================================================

Is the NULL on mailbox ok? This code is from gmime. I'm using gmime
2.4.28 and notmuch from git (0.10+58~g1c81e8f).

Regards,
Kazuo.

[1]: https://github.com/pazz/alot/issues/142

-- 
“The journey is more important than the destination—that’s
part of life, if you only live for getting to the end, you’re
almost always disappointed.”

Donald E. Knuth

Re: notmuch seg. fault on _thread_add_message

[David Bremner]

On Sun, 4 Dec 2011 15:35:39 -0200, Kazuo Teramoto <kaz.rag@gmail.com> wrote:
> Hi!
> 
> Following an alot bug report [1] I created a small python script that
> causes a segmentation fault in notmuch:
> ========================================================================
> #!/usr/bin/env python2
> import notmuch
> db = notmuch.Database(mode=notmuch.Database.MODE.READ_WRITE)
> q_new = notmuch.Query(db, 'tag:inbox')
> for t in q_new.search_threads():
>     pass
> ========================================================================

As I mentioned previously, I suspect the message below should be fixed;
whether or not it fixes your bug I don't know. I'm not sure where
exactly g_type_init would/should be called in the library; there isn't
currently a global initialization call for libnotmuch. Perhaps there
should be?

> 
> (process:16579): GLib-GObject-CRITICAL **: gtype.c:2715: You forgot to call g_type_init()
> 

Re: notmuch seg. fault on _thread_add_message

[Austin Clements]

Quoth David Bremner on Dec 04 at  2:14 pm:
> On Sun, 4 Dec 2011 15:35:39 -0200, Kazuo Teramoto <kaz.rag@gmail.com> wrote:
> > Hi!
> > 
> > Following an alot bug report [1] I created a small python script that
> > causes a segmentation fault in notmuch:
> > ========================================================================
> > #!/usr/bin/env python2
> > import notmuch
> > db = notmuch.Database(mode=notmuch.Database.MODE.READ_WRITE)
> > q_new = notmuch.Query(db, 'tag:inbox')
> > for t in q_new.search_threads():
> >     pass
> > ========================================================================
> 
> As I mentioned previously, I suspect the message below should be fixed;
> whether or not it fixes your bug I don't know. I'm not sure where
> exactly g_type_init would/should be called in the library; there isn't
> currently a global initialization call for libnotmuch. Perhaps there
> should be?

I believe g_type_init can be safely called multiple times, which means
we could call it in notmuch_database_open.

[PATCH] lib: call g_type_init from notmuch_database_open

[David Bremner]

From: David Bremner <bremner@debian.org>

We want to make sure g_type_init is called before any GObject
functionality is used.
---
This seems to fix the segfault for me. Any other comments/experiences?

 lib/database.cc |    4 ++++
 1 files changed, 4 insertions(+), 0 deletions(-)

diff --git a/lib/database.cc b/lib/database.cc
index e4ef14e..98f101e 100644
--- a/lib/database.cc
+++ b/lib/database.cc
@@ -26,6 +26,7 @@
 #include <signal.h>
 
 #include <glib.h> /* g_free, GPtrArray, GHashTable */
+#include <glib-object.h> /* g_type_init */
 
 using namespace std;
 
@@ -600,6 +601,9 @@ notmuch_database_open (const char *path,
 	goto DONE;
     }
 
+    /* Initialize the GLib type system and threads */
+    g_type_init ();
+
     notmuch = talloc (NULL, notmuch_database_t);
     notmuch->exception_reported = FALSE;
     notmuch->path = talloc_strdup (notmuch, path);
-- 
1.7.7.3

[PATCH] test: add tests for python bindings

[David Bremner]

From: David Bremner <bremner@debian.org>

We start modestly, with a (slightly modified) test case from Kazuo
Teramoto. Originally it just made sure the bindings didn't crash; here
we check that by comparing the output with that of notmuch search.
---
 test/notmuch-test |    1 +
 test/python       |   20 ++++++++++++++++++++
 2 files changed, 21 insertions(+), 0 deletions(-)
 create mode 100755 test/python

diff --git a/test/notmuch-test b/test/notmuch-test
index adfd589..e77f3c5 100755
--- a/test/notmuch-test
+++ b/test/notmuch-test
@@ -46,6 +46,7 @@ TESTS="
   symbol-hiding
   search-folder-coherence
   atomicity
+  python
 "
 TESTS=${NOTMUCH_TESTS:=$TESTS}
 
diff --git a/test/python b/test/python
new file mode 100755
index 0000000..28a8aab
--- /dev/null
+++ b/test/python
@@ -0,0 +1,20 @@
+#!/usr/bin/env bash
+test_description="python bindings"
+. ./test-lib.sh
+
+add_email_corpus
+
+# This test relies on the output from the python Thread.__str__
+# method being byte compatible with that from notmuch search.
+
+test_begin_subtest "compare with notmuch search"
+python <<EOF > OUTPUT
+import notmuch
+db = notmuch.Database(mode=notmuch.Database.MODE.READ_WRITE)
+q_new = notmuch.Query(db, 'tag:inbox')
+for t in q_new.search_threads():
+    print unicode(t)
+EOF
+notmuch search tag:inbox > EXPECTED
+test_expect_equal_file OUTPUT EXPECTED
+test_done
-- 
1.7.7.3

[PATCH] lib: call g_type_init from notmuch_database_open

[bjoernb]

David Bremner david at tethera.net
Sun Dec 4 11:35:14 PST 2011
>This seems to fix the segfault for me. Any other comments/experiences?

this patch fixes the issue aswell for me. Thanks David.

Greetings,
bjoernb.

[PATCH] test: add tests for python bindings

[David Bremner]

From: David Bremner <bremner@debian.org>

We start modestly, with a (slightly modified) test case from Kazuo
Teramoto. Originally it just made sure the bindings didn't crash; here
we check that by comparing the output with that of notmuch search.
---
This version of the test is less ambitious, and just checks thread ids match.
 test/notmuch-test |    1 +
 test/python       |   17 +++++++++++++++++
 2 files changed, 18 insertions(+), 0 deletions(-)
 create mode 100755 test/python

diff --git a/test/notmuch-test b/test/notmuch-test
index 5aced5c..113ea7c 100755
--- a/test/notmuch-test
+++ b/test/notmuch-test
@@ -47,6 +47,7 @@ TESTS="
   symbol-hiding
   search-folder-coherence
   atomicity
+  python
 "
 TESTS=${NOTMUCH_TESTS:=$TESTS}
 
diff --git a/test/python b/test/python
new file mode 100755
index 0000000..179427b
--- /dev/null
+++ b/test/python
@@ -0,0 +1,17 @@
+#!/usr/bin/env bash
+test_description="python bindings"
+. ./test-lib.sh
+
+add_email_corpus
+
+test_begin_subtest "compare thread ids"
+python <<EOF | sort > OUTPUT
+import notmuch
+db = notmuch.Database(mode=notmuch.Database.MODE.READ_WRITE)
+q_new = notmuch.Query(db, 'tag:inbox')
+for t in q_new.search_threads():
+    print t.get_thread_id()
+EOF
+notmuch search --output=threads tag:inbox | sed s/^thread:// | sort > EXPECTED
+test_expect_equal_file OUTPUT EXPECTED
+test_done
-- 
1.7.7.3

Proposed bug fix release 0.10.2

[David Bremner]

On Sun,  4 Dec 2011 15:35:14 -0400, David Bremner <david@tethera.net> wrote:
> From: David Bremner <bremner@debian.org>
> 
> We want to make sure g_type_init is called before any GObject
> functionality is used.
> ---
> This seems to fix the segfault for me. Any other comments/experiences?
> 

I heard two positive reports, and no complaints, so I pushed a proposed
bug fix release to 

git://git.notmuchmail.org 

branch: release

I haven't tagged it yet, so let me know if there is something I missed.

This is just a bug fix release for this one segfault; I am tentatively
planning on freezing 0.11 around Christmas, with a release for New
Years.

d

Re: [PATCH] lib: call g_type_init from notmuch_database_open

[Sebastian Spaeth]

[-- Attachment #1: Type: text/plain, Size: 406 bytes --]

On Sun,  4 Dec 2011 15:35:14 -0400, David Bremner <david@tethera.net> wrote:
> From: David Bremner <bremner@debian.org>
> This seems to fix the segfault for me. Any other comments/experiences?

> +    /* Initialize the GLib type system and threads */
> +    g_type_init ();
> +


Thanks the patch looks sane, and relieves me from doing ugly things with
gmime from the python side of things.

+1

Sebastian

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]