v2 _notmuch_message_ensure_metadata exception handling

v2 _notmuch_message_ensure_metadata exception handling

[David Bremner]

I tried to address most of Jani's comments on v1 [1]. The bad news is
that the test is a bit delicate, it really does need the whole corpus.

I did decide to follow the suggestion to make the _reopen function
private for now. It would be nice if we could hide that complexity
from library users, I guess we'll have to see.


[1]: id:20170225034513.19427-1-david@tethera.net


Interdiff follows.

diff --git a/lib/database.cc b/lib/database.cc
index 1e958b65..ba440d4d 100644
--- a/lib/database.cc
+++ b/lib/database.cc
@@ -1135,7 +1135,7 @@ notmuch_database_close (notmuch_database_t *notmuch)
 }
 
 notmuch_status_t
-notmuch_database_reopen (notmuch_database_t *notmuch)
+_notmuch_database_reopen (notmuch_database_t *notmuch)
 {
     if (notmuch->mode != NOTMUCH_DATABASE_MODE_READ_ONLY)
 	return NOTMUCH_STATUS_UNSUPPORTED_OPERATION;
@@ -1147,8 +1147,8 @@ notmuch_database_reopen (notmuch_database_t *notmuch)
 	    _notmuch_database_log (notmuch, "Error: A Xapian exception reopening database: %s\n",
 				   error.get_msg ().c_str ());
 	    notmuch->exception_reported = TRUE;
-	    return NOTMUCH_STATUS_XAPIAN_EXCEPTION;
 	}
+	return NOTMUCH_STATUS_XAPIAN_EXCEPTION;
     }
 
     notmuch->view++;
diff --git a/lib/message.cc b/lib/message.cc
index bfb95917..007f1171 100644
--- a/lib/message.cc
+++ b/lib/message.cc
@@ -320,9 +320,8 @@ static void
 _notmuch_message_ensure_metadata (notmuch_message_t *message, void *field)
 {
     Xapian::TermIterator i, end;
-    notmuch_bool_t success = FALSE;
 
-    if ((field != NULL) && (message->last_view >= message->notmuch->view))
+    if (field && (message->last_view >= message->notmuch->view))
 	return;
 
     const char *thread_prefix = _find_prefix ("thread"),
@@ -338,7 +337,7 @@ _notmuch_message_ensure_metadata (notmuch_message_t *message, void *field)
      * slightly more costly than looking up individual fields if only
      * one field of the message object is actually used, it's a huge
      * win as more fields are used. */
-    for (int count=0; count < 3 && !success; count++) {
+    for (int count=0; count < 3; count++) {
 	try {
 	    i = message->doc.termlist_begin ();
 	    end = message->doc.termlist_end ();
@@ -408,13 +407,12 @@ _notmuch_message_ensure_metadata (notmuch_message_t *message, void *field)
 		message->in_reply_to = talloc_strdup (message, "");
 
 	    /* all the way without an exception */
-	    success = TRUE;
+	    break;
 	} catch (const Xapian::DatabaseModifiedError &error) {
-	    notmuch_status_t status = notmuch_database_reopen (message->notmuch);
+	    notmuch_status_t status = _notmuch_database_reopen (message->notmuch);
 	    if (status != NOTMUCH_STATUS_SUCCESS)
 		INTERNAL_ERROR ("unhandled error from notmuch_database_reopen: %s\n",
 				notmuch_status_to_string (status));
-	    success = FALSE;
 	} catch (const Xapian::Error &error) {
 	    INTERNAL_ERROR ("A Xapian exception occurred fetching message metadata: %s\n",
 			    error.get_msg().c_str());
diff --git a/lib/notmuch-private.h b/lib/notmuch-private.h
index 7b35fc5b..8587e86c 100644
--- a/lib/notmuch-private.h
+++ b/lib/notmuch-private.h
@@ -192,6 +192,9 @@ _notmuch_message_id_compressed (void *ctx, const char *message_id);
 notmuch_status_t
 _notmuch_database_ensure_writable (notmuch_database_t *notmuch);
 
+notmuch_status_t
+_notmuch_database_reopen (notmuch_database_t *notmuch);
+
 void
 _notmuch_database_log (notmuch_database_t *notmuch,
 		       const char *format, ...);
diff --git a/lib/notmuch.h b/lib/notmuch.h
index e9ed01dd..16da8be9 100644
--- a/lib/notmuch.h
+++ b/lib/notmuch.h
@@ -363,18 +363,6 @@ notmuch_status_t
 notmuch_database_close (notmuch_database_t *database);
 
 /**
- * Reopen a (read-only) database, synching the database view with that
- * on disk.
- *
- * @returns
- * - NOTMUCH_STATUS_UNSUPPORTED_OPERATION: database is not read only
- * - NOTMUCH_STATUS_XAPIAN_EXCEPTION: a Xapian exception occured trying to
- *   re-open the database.
- */
-notmuch_status_t
-notmuch_database_reopen (notmuch_database_t *database);
-
-/**
  * A callback invoked by notmuch_database_compact to notify the user
  * of the progress of the compaction process.
  */
diff --git a/test/T640-database-modified.sh b/test/T640-database-modified.sh
index 9599417c..516836b0 100755
--- a/test/T640-database-modified.sh
+++ b/test/T640-database-modified.sh
@@ -2,18 +2,22 @@
 test_description="DatabaseModifiedError handling"
 . ./test-lib.sh || exit 1
 
+# add enough messages to trigger the exception
 add_email_corpus
 
 test_begin_subtest "catching DatabaseModifiedError in _notmuch_message_ensure_metadata"
-test_C ${MAIL_DIR} <<'EOF'
+# it seems to need to be an early document to trigger the exception
+first_id=$(notmuch search --output=messages '*'| head -1 | sed s/^id://)
+
+test_C ${MAIL_DIR} <<EOF
 #include <unistd.h>
 #include <stdlib.h>
 #include <notmuch-test.h>
 #include <talloc.h>
+#include <assert.h>
 int
 main (int argc, char **argv)
 {
-    pid_t child;
     const char *path = argv[1];
 
     notmuch_database_t *rw_db, *ro_db;
@@ -23,7 +27,10 @@ main (int argc, char **argv)
     notmuch_tags_t *tags;
 
     EXPECT0 (notmuch_database_open (path, NOTMUCH_DATABASE_MODE_READ_ONLY, &ro_db));
-    EXPECT0 (notmuch_database_find_message (ro_db, "4EFC743A.3060609@april.org", &ro_message));
+    assert(ro_db);
+
+    EXPECT0 (notmuch_database_find_message (ro_db, "${first_id}", &ro_message));
+    assert(ro_message);
 
     EXPECT0 (notmuch_database_open (path, NOTMUCH_DATABASE_MODE_READ_WRITE, &rw_db));
     query = notmuch_query_create(rw_db, "");

[patch v2 1/4] test: add known broken test for uncaught DatabaseModifiedError

[David Bremner]

There are several of these to track down, but one that is in quite a
few code paths is _notmuch_message_ensure_metadata.
---
 test/T640-database-modified.sh | 67 ++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 67 insertions(+)
 create mode 100755 test/T640-database-modified.sh

diff --git a/test/T640-database-modified.sh b/test/T640-database-modified.sh
new file mode 100755
index 00000000..b5b3bc2b
--- /dev/null
+++ b/test/T640-database-modified.sh
@@ -0,0 +1,67 @@
+#!/usr/bin/env bash
+test_description="DatabaseModifiedError handling"
+. ./test-lib.sh || exit 1
+
+# add enough messages to trigger the exception
+add_email_corpus
+
+test_begin_subtest "catching DatabaseModifiedError in _notmuch_message_ensure_metadata"
+test_subtest_known_broken
+# it seems to need to be an early document to trigger the exception
+first_id=$(notmuch search --output=messages '*'| head -1 | sed s/^id://)
+
+test_C ${MAIL_DIR} <<EOF
+#include <unistd.h>
+#include <stdlib.h>
+#include <notmuch-test.h>
+#include <talloc.h>
+#include <assert.h>
+int
+main (int argc, char **argv)
+{
+    const char *path = argv[1];
+
+    notmuch_database_t *rw_db, *ro_db;
+    notmuch_messages_t *messages;
+    notmuch_message_t *message, *ro_message;
+    notmuch_query_t *query;
+    notmuch_tags_t *tags;
+
+    EXPECT0 (notmuch_database_open (path, NOTMUCH_DATABASE_MODE_READ_ONLY, &ro_db));
+    assert(ro_db);
+
+    EXPECT0 (notmuch_database_find_message (ro_db, "${first_id}", &ro_message));
+    assert(ro_message);
+
+    EXPECT0 (notmuch_database_open (path, NOTMUCH_DATABASE_MODE_READ_WRITE, &rw_db));
+    query = notmuch_query_create(rw_db, "");
+    EXPECT0 (notmuch_query_search_messages_st (query, &messages));
+
+    for (int count=0;
+	 notmuch_messages_valid (messages);
+	 notmuch_messages_move_to_next (messages), count++) {
+	message = notmuch_messages_get (messages);
+	for (int i=0; i<200; i++) {
+	    char *tag_str = talloc_asprintf(rw_db, "%d", i);
+	    EXPECT0 (notmuch_message_add_tag (message, tag_str));
+	    talloc_free (tag_str);
+	}
+    }
+
+    notmuch_database_close (rw_db);
+
+    tags = notmuch_message_get_tags (ro_message);
+    if (tags)
+	printf("SUCCESS\n");
+    return 0;
+}
+EOF
+
+cat <<'EOF' >EXPECTED
+== stdout ==
+SUCCESS
+== stderr ==
+EOF
+test_expect_equal_file EXPECTED OUTPUT
+
+test_done
-- 
2.11.0

[patch v2 2/4] lib: add _notmuch_database_reopen

[David Bremner]

The main expected use is to recover from a Xapian::DatabaseChanged
exception.
---
 lib/database-private.h |  4 ++++
 lib/database.cc        | 23 +++++++++++++++++++++++
 lib/notmuch-private.h  |  3 +++
 3 files changed, 30 insertions(+)

diff --git a/lib/database-private.h b/lib/database-private.h
index 2fb60f5e..06882439 100644
--- a/lib/database-private.h
+++ b/lib/database-private.h
@@ -207,6 +207,10 @@ struct _notmuch_database {
     unsigned long revision;
     const char *uuid;
 
+    /* Keep track of the number of times the database has been re-opened
+     * (or other global invalidations of notmuch's caching)
+     */
+    unsigned long view;
     Xapian::QueryParser *query_parser;
     Xapian::TermGenerator *term_gen;
     Xapian::ValueRangeProcessor *value_range_processor;
diff --git a/lib/database.cc b/lib/database.cc
index 386dcd17..ba440d4d 100644
--- a/lib/database.cc
+++ b/lib/database.cc
@@ -951,6 +951,7 @@ notmuch_database_open_verbose (const char *path,
 
     notmuch->mode = mode;
     notmuch->atomic_nesting = 0;
+    notmuch->view = 1;
     try {
 	string last_thread_id;
 	string last_mod;
@@ -1133,6 +1134,28 @@ notmuch_database_close (notmuch_database_t *notmuch)
     return status;
 }
 
+notmuch_status_t
+_notmuch_database_reopen (notmuch_database_t *notmuch)
+{
+    if (notmuch->mode != NOTMUCH_DATABASE_MODE_READ_ONLY)
+	return NOTMUCH_STATUS_UNSUPPORTED_OPERATION;
+
+    try {
+	notmuch->xapian_db->reopen ();
+    } catch (const Xapian::Error &error) {
+	if (! notmuch->exception_reported) {
+	    _notmuch_database_log (notmuch, "Error: A Xapian exception reopening database: %s\n",
+				   error.get_msg ().c_str ());
+	    notmuch->exception_reported = TRUE;
+	}
+	return NOTMUCH_STATUS_XAPIAN_EXCEPTION;
+    }
+
+    notmuch->view++;
+
+    return NOTMUCH_STATUS_SUCCESS;
+}
+
 static int
 unlink_cb (const char *path,
 	   unused (const struct stat *sb),
diff --git a/lib/notmuch-private.h b/lib/notmuch-private.h
index 7b35fc5b..8587e86c 100644
--- a/lib/notmuch-private.h
+++ b/lib/notmuch-private.h
@@ -192,6 +192,9 @@ _notmuch_message_id_compressed (void *ctx, const char *message_id);
 notmuch_status_t
 _notmuch_database_ensure_writable (notmuch_database_t *notmuch);
 
+notmuch_status_t
+_notmuch_database_reopen (notmuch_database_t *notmuch);
+
 void
 _notmuch_database_log (notmuch_database_t *notmuch,
 		       const char *format, ...);
-- 
2.11.0

[patch v2 3/4] lib: handle DatabaseModifiedError in _n_message_ensure_metadata

[David Bremner]

The retries are hardcoded to a small number, and error handling aborts
than propagating errors from notmuch_database_reopen. These are both
somewhat justified by the assumption that most things that can go
wrong in Xapian::Database::reopen are rare and fatal. Here's the brief
discussion with Xapian upstream:

   24-02-2017 08:12:57 < bremner> any intuition about how likely
      Xapian::Database::reopen is to fail? I'm catching a
      DatabaseModifiedError somewhere where handling any further errors is
      tricky, and wondering about treating a failed reopen as as "the
      impossible happened, stopping"

   24-02-2017 16:22:34 < olly> bremner: there should not be much scope for
    failure - stuff like out of memory or disk errors, which are probably a
    good enough excuse to stop
---
 lib/message.cc                 | 144 ++++++++++++++++++++++++-----------------
 test/T640-database-modified.sh |   1 -
 2 files changed, 83 insertions(+), 62 deletions(-)

diff --git a/lib/message.cc b/lib/message.cc
index 2fb67d85..9bafff0b 100644
--- a/lib/message.cc
+++ b/lib/message.cc
@@ -49,6 +49,9 @@ struct visible _notmuch_message {
     /* Message document modified since last sync */
     notmuch_bool_t modified;
 
+    /* last view of database the struct is synced with */
+    unsigned long last_view;
+
     Xapian::Document doc;
     Xapian::termcount termpos;
 };
@@ -110,6 +113,9 @@ _notmuch_message_create_for_document (const void *talloc_owner,
     message->flags = 0;
     message->lazy_flags = 0;
 
+    /* the message is initially not synchronized with Xapian */
+    message->last_view = 0;
+
     /* Each of these will be lazily created as needed. */
     message->message_id = NULL;
     message->thread_id = NULL;
@@ -314,6 +320,7 @@ static void
 _notmuch_message_ensure_metadata (notmuch_message_t *message)
 {
     Xapian::TermIterator i, end;
+
     const char *thread_prefix = _find_prefix ("thread"),
 	*tag_prefix = _find_prefix ("tag"),
 	*id_prefix = _find_prefix ("id"),
@@ -327,73 +334,88 @@ _notmuch_message_ensure_metadata (notmuch_message_t *message)
      * slightly more costly than looking up individual fields if only
      * one field of the message object is actually used, it's a huge
      * win as more fields are used. */
+    for (int count=0; count < 3; count++) {
+	try {
+	    i = message->doc.termlist_begin ();
+	    end = message->doc.termlist_end ();
+
+	    /* Get thread */
+	    if (!message->thread_id)
+		message->thread_id =
+		    _notmuch_message_get_term (message, i, end, thread_prefix);
+
+	    /* Get tags */
+	    assert (strcmp (thread_prefix, tag_prefix) < 0);
+	    if (!message->tag_list) {
+		message->tag_list =
+		    _notmuch_database_get_terms_with_prefix (message, i, end,
+							     tag_prefix);
+		_notmuch_string_list_sort (message->tag_list);
+	    }
 
-    i = message->doc.termlist_begin ();
-    end = message->doc.termlist_end ();
+	    /* Get id */
+	    assert (strcmp (tag_prefix, id_prefix) < 0);
+	    if (!message->message_id)
+		message->message_id =
+		    _notmuch_message_get_term (message, i, end, id_prefix);
+
+	    /* Get document type */
+	    assert (strcmp (id_prefix, type_prefix) < 0);
+	    if (! NOTMUCH_TEST_BIT (message->lazy_flags, NOTMUCH_MESSAGE_FLAG_GHOST)) {
+		i.skip_to (type_prefix);
+		/* "T" is the prefix "type" fields.  See
+		 * BOOLEAN_PREFIX_INTERNAL. */
+		if (*i == "Tmail")
+		    NOTMUCH_CLEAR_BIT (&message->flags, NOTMUCH_MESSAGE_FLAG_GHOST);
+		else if (*i == "Tghost")
+		    NOTMUCH_SET_BIT (&message->flags, NOTMUCH_MESSAGE_FLAG_GHOST);
+		else
+		    INTERNAL_ERROR ("Message without type term");
+		NOTMUCH_SET_BIT (&message->lazy_flags, NOTMUCH_MESSAGE_FLAG_GHOST);
+	    }
 
-    /* Get thread */
-    if (!message->thread_id)
-	message->thread_id =
-	    _notmuch_message_get_term (message, i, end, thread_prefix);
-
-    /* Get tags */
-    assert (strcmp (thread_prefix, tag_prefix) < 0);
-    if (!message->tag_list) {
-	message->tag_list =
-	    _notmuch_database_get_terms_with_prefix (message, i, end,
-						     tag_prefix);
-	_notmuch_string_list_sort (message->tag_list);
-    }
+	    /* Get filename list.  Here we get only the terms.  We lazily
+	     * expand them to full file names when needed in
+	     * _notmuch_message_ensure_filename_list. */
+	    assert (strcmp (type_prefix, filename_prefix) < 0);
+	    if (!message->filename_term_list && !message->filename_list)
+		message->filename_term_list =
+		    _notmuch_database_get_terms_with_prefix (message, i, end,
+							     filename_prefix);
 
-    /* Get id */
-    assert (strcmp (tag_prefix, id_prefix) < 0);
-    if (!message->message_id)
-	message->message_id =
-	    _notmuch_message_get_term (message, i, end, id_prefix);
-
-    /* Get document type */
-    assert (strcmp (id_prefix, type_prefix) < 0);
-    if (! NOTMUCH_TEST_BIT (message->lazy_flags, NOTMUCH_MESSAGE_FLAG_GHOST)) {
-	i.skip_to (type_prefix);
-	/* "T" is the prefix "type" fields.  See
-	 * BOOLEAN_PREFIX_INTERNAL. */
-	if (*i == "Tmail")
-	    NOTMUCH_CLEAR_BIT (&message->flags, NOTMUCH_MESSAGE_FLAG_GHOST);
-	else if (*i == "Tghost")
-	    NOTMUCH_SET_BIT (&message->flags, NOTMUCH_MESSAGE_FLAG_GHOST);
-	else
-	    INTERNAL_ERROR ("Message without type term");
-	NOTMUCH_SET_BIT (&message->lazy_flags, NOTMUCH_MESSAGE_FLAG_GHOST);
-    }
 
-    /* Get filename list.  Here we get only the terms.  We lazily
-     * expand them to full file names when needed in
-     * _notmuch_message_ensure_filename_list. */
-    assert (strcmp (type_prefix, filename_prefix) < 0);
-    if (!message->filename_term_list && !message->filename_list)
-	message->filename_term_list =
-	    _notmuch_database_get_terms_with_prefix (message, i, end,
-						     filename_prefix);
-
-
-    /* Get property terms. Mimic the setup with filenames above */
-    assert (strcmp (filename_prefix, property_prefix) < 0);
-    if (!message->property_map && !message->property_term_list)
-	message->property_term_list =
-	    _notmuch_database_get_terms_with_prefix (message, i, end,
-						     property_prefix);
-
-    /* Get reply to */
-    assert (strcmp (property_prefix, replyto_prefix) < 0);
-    if (!message->in_reply_to)
-	message->in_reply_to =
-	    _notmuch_message_get_term (message, i, end, replyto_prefix);
+	    /* Get property terms. Mimic the setup with filenames above */
+	    assert (strcmp (filename_prefix, property_prefix) < 0);
+	    if (!message->property_map && !message->property_term_list)
+		message->property_term_list =
+		    _notmuch_database_get_terms_with_prefix (message, i, end,
+							 property_prefix);
 
+	    /* Get reply to */
+	    assert (strcmp (property_prefix, replyto_prefix) < 0);
+	    if (!message->in_reply_to)
+		message->in_reply_to =
+		    _notmuch_message_get_term (message, i, end, replyto_prefix);
 
-    /* It's perfectly valid for a message to have no In-Reply-To
-     * header. For these cases, we return an empty string. */
-    if (!message->in_reply_to)
-	message->in_reply_to = talloc_strdup (message, "");
+
+	    /* It's perfectly valid for a message to have no In-Reply-To
+	     * header. For these cases, we return an empty string. */
+	    if (!message->in_reply_to)
+		message->in_reply_to = talloc_strdup (message, "");
+
+	    /* all the way without an exception */
+	    break;
+	} catch (const Xapian::DatabaseModifiedError &error) {
+	    notmuch_status_t status = _notmuch_database_reopen (message->notmuch);
+	    if (status != NOTMUCH_STATUS_SUCCESS)
+		INTERNAL_ERROR ("unhandled error from notmuch_database_reopen: %s\n",
+				notmuch_status_to_string (status));
+	} catch (const Xapian::Error &error) {
+	    INTERNAL_ERROR ("A Xapian exception occurred fetching message metadata: %s\n",
+			    error.get_msg().c_str());
+	}
+    }
+    message->last_view = message->notmuch->view;
 }
 
 void
diff --git a/test/T640-database-modified.sh b/test/T640-database-modified.sh
index b5b3bc2b..516836b0 100755
--- a/test/T640-database-modified.sh
+++ b/test/T640-database-modified.sh
@@ -6,7 +6,6 @@ test_description="DatabaseModifiedError handling"
 add_email_corpus
 
 test_begin_subtest "catching DatabaseModifiedError in _notmuch_message_ensure_metadata"
-test_subtest_known_broken
 # it seems to need to be an early document to trigger the exception
 first_id=$(notmuch search --output=messages '*'| head -1 | sed s/^id://)
 
-- 
2.11.0

[patch v2 4/4] lib/message.cc: use view number to invalidate cached metadata

[David Bremner]

Currently the view number is incremented by notmuch_database_reopen
---
 lib/message.cc | 25 +++++++++++--------------
 1 file changed, 11 insertions(+), 14 deletions(-)

diff --git a/lib/message.cc b/lib/message.cc
index 9bafff0b..007f1171 100644
--- a/lib/message.cc
+++ b/lib/message.cc
@@ -317,10 +317,13 @@ _notmuch_message_get_term (notmuch_message_t *message,
 }
 
 static void
-_notmuch_message_ensure_metadata (notmuch_message_t *message)
+_notmuch_message_ensure_metadata (notmuch_message_t *message, void *field)
 {
     Xapian::TermIterator i, end;
 
+    if (field && (message->last_view >= message->notmuch->view))
+	return;
+
     const char *thread_prefix = _find_prefix ("thread"),
 	*tag_prefix = _find_prefix ("tag"),
 	*id_prefix = _find_prefix ("id"),
@@ -470,8 +473,7 @@ _notmuch_message_get_doc_id (notmuch_message_t *message)
 const char *
 notmuch_message_get_message_id (notmuch_message_t *message)
 {
-    if (!message->message_id)
-	_notmuch_message_ensure_metadata (message);
+    _notmuch_message_ensure_metadata (message, message->message_id);
     if (!message->message_id)
 	INTERNAL_ERROR ("Message with document ID of %u has no message ID.\n",
 			message->doc_id);
@@ -546,16 +548,14 @@ notmuch_message_get_header (notmuch_message_t *message, const char *header)
 const char *
 _notmuch_message_get_in_reply_to (notmuch_message_t *message)
 {
-    if (!message->in_reply_to)
-	_notmuch_message_ensure_metadata (message);
+    _notmuch_message_ensure_metadata (message, message->in_reply_to);
     return message->in_reply_to;
 }
 
 const char *
 notmuch_message_get_thread_id (notmuch_message_t *message)
 {
-    if (!message->thread_id)
-	_notmuch_message_ensure_metadata (message);
+    _notmuch_message_ensure_metadata (message, message->thread_id);
     if (!message->thread_id)
 	INTERNAL_ERROR ("Message with document ID of %u has no thread ID.\n",
 			message->doc_id);
@@ -858,8 +858,7 @@ _notmuch_message_ensure_filename_list (notmuch_message_t *message)
     if (message->filename_list)
 	return;
 
-    if (!message->filename_term_list)
-	_notmuch_message_ensure_metadata (message);
+    _notmuch_message_ensure_metadata (message, message->filename_term_list);
 
     message->filename_list = _notmuch_string_list_create (message);
     node = message->filename_term_list->head;
@@ -953,7 +952,7 @@ notmuch_message_get_flag (notmuch_message_t *message,
 {
     if (flag == NOTMUCH_MESSAGE_FLAG_GHOST &&
 	! NOTMUCH_TEST_BIT (message->lazy_flags, flag))
-	_notmuch_message_ensure_metadata (message);
+	_notmuch_message_ensure_metadata (message, NULL);
 
     return NOTMUCH_TEST_BIT (message->flags, flag);
 }
@@ -994,8 +993,7 @@ notmuch_message_get_tags (notmuch_message_t *message)
 {
     notmuch_tags_t *tags;
 
-    if (!message->tag_list)
-	_notmuch_message_ensure_metadata (message);
+    _notmuch_message_ensure_metadata (message, message->tag_list);
 
     tags = _notmuch_tags_create (message, message->tag_list);
     /* _notmuch_tags_create steals the reference to the tag_list, but
@@ -1831,8 +1829,7 @@ _notmuch_message_ensure_property_map (notmuch_message_t *message)
     if (message->property_map)
 	return;
 
-    if (!message->property_term_list)
-	_notmuch_message_ensure_metadata (message);
+    _notmuch_message_ensure_metadata (message, message->property_term_list);
 
     message->property_map = _notmuch_string_map_create (message);
 
-- 
2.11.0

Re: v2 _notmuch_message_ensure_metadata exception handling

[Jani Nikula]

On Sat, 25 Feb 2017, David Bremner <david@tethera.net> wrote:
> I tried to address most of Jani's comments on v1 [1]. The bad news is
> that the test is a bit delicate, it really does need the whole corpus.

Too bad. I hope it's not too dependent on the environment and the phase
of the Moon, etc.

> I did decide to follow the suggestion to make the _reopen function
> private for now. It would be nice if we could hide that complexity
> from library users, I guess we'll have to see.

It's easy enough to expose that later if need be.

The series LGTM.

BR,
Jani.

Re: v2 _notmuch_message_ensure_metadata exception handling

[David Bremner]

Jani Nikula <jani@nikula.org> writes:

> On Sat, 25 Feb 2017, David Bremner <david@tethera.net> wrote:
>> I tried to address most of Jani's comments on v1 [1]. The bad news is
>> that the test is a bit delicate, it really does need the whole corpus.
>
> Too bad. I hope it's not too dependent on the environment and the phase
> of the Moon, etc.

We'll have to see. FWIW, the only problems I had were false positives,
i.e. passing when it should not.
>
> The series LGTM.

pushed to master