Second batch of API cleanup for exception safety.

Second batch of API cleanup for exception safety.

[David Bremner]

This is going to be a bit of a slog, so I'm going to send the patches
in batches, rather than waiting until I have one giant series.

I'm interested in feedback on the last two patches in particular
before I adjust notmuch to use the new API itself. An alternative
approach would be treat being a ghost_message as a seperate property
(and leave the old notmuch_bool_t API for the other flags). That seems
harder to migrate to cleanly, but I'm open to ideas.

[PATCH 01/11] test: remove unused backup_database calls

[David Bremner]

Since these backups are never restored, they should be safe to remove.
---
 test/T560-lib-error.sh | 2 --
 1 file changed, 2 deletions(-)

diff --git a/test/T560-lib-error.sh b/test/T560-lib-error.sh
index 81500536..5af3eab2 100755
--- a/test/T560-lib-error.sh
+++ b/test/T560-lib-error.sh
@@ -341,7 +341,6 @@ int main (int argc, char** argv)
    EXPECT0(notmuch_database_close (db));
 EOF
 
-backup_database
 test_begin_subtest "Handle getting message-id from closed database"
 cat c_head2 - c_tail <<'EOF' | test_C ${MAIL_DIR}
     {
@@ -358,7 +357,6 @@ cat <<EOF > EXPECTED
 EOF
 test_expect_equal_file EXPECTED OUTPUT
 
-backup_database
 test_begin_subtest "Handle getting thread-id from closed database"
 cat c_head2 - c_tail <<'EOF' | test_C ${MAIL_DIR}
     {
-- 
2.27.0

[PATCH 02/11] test: drop use of assert in closed db tests

[David Bremner]

Instead of printing the same static string for each test, can replace
the assert with something simpler (or at least easier to integrate
into the test suite).
---
 test/T560-lib-error.sh | 10 ++++------
 1 file changed, 4 insertions(+), 6 deletions(-)

diff --git a/test/T560-lib-error.sh b/test/T560-lib-error.sh
index 5af3eab2..6432faaa 100755
--- a/test/T560-lib-error.sh
+++ b/test/T560-lib-error.sh
@@ -322,7 +322,6 @@ cat <<EOF > c_head2
 #include <stdio.h>
 #include <notmuch.h>
 #include <notmuch-test.h>
-#include <assert.h>
 int main (int argc, char** argv)
 {
    notmuch_database_t *db;
@@ -337,7 +336,6 @@ int main (int argc, char** argv)
      exit (1);
    }
    EXPECT0(notmuch_database_find_message (db, id, &message));
-   assert(message != NULL);
    EXPECT0(notmuch_database_close (db));
 EOF
 
@@ -346,12 +344,12 @@ cat c_head2 - c_tail <<'EOF' | test_C ${MAIL_DIR}
     {
         const char *id2;
         id2=notmuch_message_get_message_id (message);
-        printf("%s\n%d\n", id, id2==NULL);
+        printf("%d\n%d\n", message != NULL, id2==NULL);
     }
 EOF
 cat <<EOF > EXPECTED
 == stdout ==
-1258471718-6781-1-git-send-email-dottedmag@dottedmag.net
+1
 1
 == stderr ==
 EOF
@@ -362,12 +360,12 @@ cat c_head2 - c_tail <<'EOF' | test_C ${MAIL_DIR}
     {
         const char *id2;
         id2=notmuch_message_get_thread_id (message);
-        printf("%s\n%d\n", id, id2==NULL);
+        printf("%d\n%d\n", message != NULL, id2==NULL);
     }
 EOF
 cat <<EOF > EXPECTED
 == stdout ==
-1258471718-6781-1-git-send-email-dottedmag@dottedmag.net
+1
 1
 == stderr ==
 EOF
-- 
2.27.0

[PATCH 03/11] lib/message: use LOG_XAPIAN_EXCEPTION in n_m_get_header

[David Bremner]

This is just for consistency, and a small reduction in the amount of
boilerplate.
---
 lib/message.cc | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

diff --git a/lib/message.cc b/lib/message.cc
index 3ca7b902..0551a427 100644
--- a/lib/message.cc
+++ b/lib/message.cc
@@ -572,9 +572,7 @@ notmuch_message_get_header (notmuch_message_t *message, const char *header)
 		return talloc_strdup (message, value.c_str ());
 
 	} catch (Xapian::Error &error) {
-	    _notmuch_database_log (notmuch_message_get_database (message), "A Xapian exception occurred when reading header: %s\n",
-				   error.get_msg ().c_str ());
-	    message->notmuch->exception_reported = true;
+	    LOG_XAPIAN_EXCEPTION (message, error);
 	    return NULL;
 	}
     }
-- 
2.27.0

[PATCH 04/11] test: add regression test for n_m_get_header

[David Bremner]

This function already catches Xapian exceptions, and we want to make
sure it stays that way.
---
 test/T560-lib-error.sh | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)

diff --git a/test/T560-lib-error.sh b/test/T560-lib-error.sh
index 6432faaa..777eb375 100755
--- a/test/T560-lib-error.sh
+++ b/test/T560-lib-error.sh
@@ -371,4 +371,20 @@ cat <<EOF > EXPECTED
 EOF
 test_expect_equal_file EXPECTED OUTPUT
 
+test_begin_subtest "Handle getting header from closed database"
+cat c_head2 - c_tail <<'EOF' | test_C ${MAIL_DIR}
+    {
+        const char *from;
+        from=notmuch_message_get_header (message, "from");
+        printf("%s\n%d\n", id, from == NULL);
+    }
+EOF
+cat <<EOF > EXPECTED
+== stdout ==
+1258471718-6781-1-git-send-email-dottedmag@dottedmag.net
+1
+== stderr ==
+EOF
+test_expect_equal_file EXPECTED OUTPUT
+
 test_done
-- 
2.27.0

[PATCH 05/11] lib/n_m_get_replies: doc return, initial regression test

[David Bremner]

We need to to set a query and retrieve the threads to meaningfully
test this function.
---
 lib/notmuch.h          |  5 ++++-
 test/T560-lib-error.sh | 18 ++++++++++++++++++
 2 files changed, 22 insertions(+), 1 deletion(-)

diff --git a/lib/notmuch.h b/lib/notmuch.h
index 0dc89547..037913f4 100644
--- a/lib/notmuch.h
+++ b/lib/notmuch.h
@@ -1402,7 +1402,10 @@ notmuch_message_get_thread_id (notmuch_message_t *message);
  * NULL. (Note that notmuch_messages_valid will accept that NULL
  * value as legitimate, and simply return FALSE for it.)
  *
- * The returned list will be destroyed when the thread is destroyed.
+ * This function also returns NULL if it triggers a Xapian exception.
+ *
+ * The returned list will be destroyed when the thread is
+ * destroyed.
  */
 notmuch_messages_t *
 notmuch_message_get_replies (notmuch_message_t *message);
diff --git a/test/T560-lib-error.sh b/test/T560-lib-error.sh
index 777eb375..d7caed5a 100755
--- a/test/T560-lib-error.sh
+++ b/test/T560-lib-error.sh
@@ -387,4 +387,22 @@ cat <<EOF > EXPECTED
 EOF
 test_expect_equal_file EXPECTED OUTPUT
 
+# XXX TODO: test on a message from notmuch_thread_get_toplevel_messages
+# XXX this test only tests the trivial code path
+test_begin_subtest "Handle getting replies from closed database"
+cat c_head2 - c_tail <<'EOF' | test_C ${MAIL_DIR}
+    {
+        notmuch_messages_t *replies;
+        replies = notmuch_message_get_replies (message);
+        printf("%d\n%d\n", message != NULL, replies==NULL);
+    }
+EOF
+cat <<EOF > EXPECTED
+== stdout ==
+1
+1
+== stderr ==
+EOF
+test_expect_equal_file EXPECTED OUTPUT
+
 test_done
-- 
2.27.0

[PATCH 06/11] lib: add known broken test for notmuch_message_get_filename

[David Bremner]

This will be fixed in the next commit
---
 test/T560-lib-error.sh | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

diff --git a/test/T560-lib-error.sh b/test/T560-lib-error.sh
index d7caed5a..7555197f 100755
--- a/test/T560-lib-error.sh
+++ b/test/T560-lib-error.sh
@@ -405,4 +405,21 @@ cat <<EOF > EXPECTED
 EOF
 test_expect_equal_file EXPECTED OUTPUT
 
+test_begin_subtest "Handle getting message filename from closed database"
+test_subtest_known_broken
+cat c_head2 - c_tail <<'EOF' | test_C ${MAIL_DIR}
+    {
+        const char *filename;
+        filename = notmuch_message_get_filename (message);
+        printf("%d\n%d\n", message != NULL, filename == NULL);
+    }
+EOF
+cat <<EOF > EXPECTED
+== stdout ==
+1
+1
+== stderr ==
+EOF
+test_expect_equal_file EXPECTED OUTPUT
+
 test_done
-- 
2.27.0

[PATCH 07/11] lib/n_m_g_filename: catch Xapian exceptions, document NULL return

[David Bremner]

This is the same machinery as applied for

     notmuch_message_get_{thread,message}_id
---
 lib/message.cc         | 7 ++++++-
 lib/notmuch.h          | 2 ++
 test/T560-lib-error.sh | 1 -
 3 files changed, 8 insertions(+), 2 deletions(-)

diff --git a/lib/message.cc b/lib/message.cc
index 0551a427..1a9eaffe 100644
--- a/lib/message.cc
+++ b/lib/message.cc
@@ -1122,7 +1122,12 @@ _notmuch_message_ensure_filename_list (notmuch_message_t *message)
 const char *
 notmuch_message_get_filename (notmuch_message_t *message)
 {
-    _notmuch_message_ensure_filename_list (message);
+    try {
+	_notmuch_message_ensure_filename_list (message);
+    } catch (Xapian::Error &error) {
+	LOG_XAPIAN_EXCEPTION (message, error);
+	return NULL;
+    }
 
     if (message->filename_list == NULL)
 	return NULL;
diff --git a/lib/notmuch.h b/lib/notmuch.h
index 037913f4..5c17ec7c 100644
--- a/lib/notmuch.h
+++ b/lib/notmuch.h
@@ -1433,6 +1433,8 @@ notmuch_message_count_files (notmuch_message_t *message);
  * this function will arbitrarily return a single one of those
  * filenames. See notmuch_message_get_filenames for returning the
  * complete list of filenames.
+ *
+ * This function returns NULL if it triggers a Xapian exception.
  */
 const char *
 notmuch_message_get_filename (notmuch_message_t *message);
diff --git a/test/T560-lib-error.sh b/test/T560-lib-error.sh
index 7555197f..57c74a2b 100755
--- a/test/T560-lib-error.sh
+++ b/test/T560-lib-error.sh
@@ -406,7 +406,6 @@ EOF
 test_expect_equal_file EXPECTED OUTPUT
 
 test_begin_subtest "Handle getting message filename from closed database"
-test_subtest_known_broken
 cat c_head2 - c_tail <<'EOF' | test_C ${MAIL_DIR}
     {
         const char *filename;
-- 
2.27.0

[PATCH 08/11] test: add known broken test for n_m_get_filenames

[David Bremner]

This will be fixed in the next commit
---
 test/T560-lib-error.sh | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

diff --git a/test/T560-lib-error.sh b/test/T560-lib-error.sh
index 57c74a2b..8d9b4cc0 100755
--- a/test/T560-lib-error.sh
+++ b/test/T560-lib-error.sh
@@ -421,4 +421,21 @@ cat <<EOF > EXPECTED
 EOF
 test_expect_equal_file EXPECTED OUTPUT
 
+test_begin_subtest "Handle getting all message filenames from closed database"
+test_subtest_known_broken
+cat c_head2 - c_tail <<'EOF' | test_C ${MAIL_DIR}
+    {
+        notmuch_filenames_t *filenames;
+        filenames = notmuch_message_get_filenames (message);
+        printf("%d\n%d\n", message != NULL, filenames == NULL);
+    }
+EOF
+cat <<EOF > EXPECTED
+== stdout ==
+1
+1
+== stderr ==
+EOF
+test_expect_equal_file EXPECTED OUTPUT
+
 test_done
-- 
2.27.0

[PATCH 09/11] lib: catch exceptions in n_m_get_filenames

[David Bremner]

This is essentially copied from the change to notmuch_message_get_filename
---
 lib/message.cc         | 7 ++++++-
 lib/notmuch.h          | 2 ++
 test/T560-lib-error.sh | 1 -
 3 files changed, 8 insertions(+), 2 deletions(-)

diff --git a/lib/message.cc b/lib/message.cc
index 1a9eaffe..8e43a393 100644
--- a/lib/message.cc
+++ b/lib/message.cc
@@ -1143,7 +1143,12 @@ notmuch_message_get_filename (notmuch_message_t *message)
 notmuch_filenames_t *
 notmuch_message_get_filenames (notmuch_message_t *message)
 {
-    _notmuch_message_ensure_filename_list (message);
+    try {
+	_notmuch_message_ensure_filename_list (message);
+    } catch (Xapian::Error &error) {
+	LOG_XAPIAN_EXCEPTION (message, error);
+	return NULL;
+    }
 
     return _notmuch_filenames_create (message, message->filename_list);
 }
diff --git a/lib/notmuch.h b/lib/notmuch.h
index 5c17ec7c..e544aafd 100644
--- a/lib/notmuch.h
+++ b/lib/notmuch.h
@@ -1448,6 +1448,8 @@ notmuch_message_get_filename (notmuch_message_t *message);
  *
  * Each filename in the iterator is an absolute filename, (the initial
  * component will match notmuch_database_get_path() ).
+ *
+ * This function returns NULL if it triggers a Xapian exception.
  */
 notmuch_filenames_t *
 notmuch_message_get_filenames (notmuch_message_t *message);
diff --git a/test/T560-lib-error.sh b/test/T560-lib-error.sh
index 8d9b4cc0..ce3526a4 100755
--- a/test/T560-lib-error.sh
+++ b/test/T560-lib-error.sh
@@ -422,7 +422,6 @@ EOF
 test_expect_equal_file EXPECTED OUTPUT
 
 test_begin_subtest "Handle getting all message filenames from closed database"
-test_subtest_known_broken
 cat c_head2 - c_tail <<'EOF' | test_C ${MAIL_DIR}
     {
         notmuch_filenames_t *filenames;
-- 
2.27.0

[PATCH 10/11] test: add known broken for n_m_get_flag on closed db

[David Bremner]

Exception caught in next commit. Note that FLAG_GHOST is the only one
that triggers the I/O code path.
---
 test/T560-lib-error.sh | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)

diff --git a/test/T560-lib-error.sh b/test/T560-lib-error.sh
index ce3526a4..68aadcdb 100755
--- a/test/T560-lib-error.sh
+++ b/test/T560-lib-error.sh
@@ -437,4 +437,22 @@ cat <<EOF > EXPECTED
 EOF
 test_expect_equal_file EXPECTED OUTPUT
 
+test_begin_subtest "Handle getting ghost flag from closed database"
+test_subtest_known_broken
+cat c_head2 - c_tail <<'EOF' | test_C ${MAIL_DIR}
+    {
+        notmuch_bool_t result;
+        result = notmuch_message_get_flag (message, NOTMUCH_MESSAGE_FLAG_GHOST);
+        printf("%d\n%d\n", message != NULL, result == FALSE);
+    }
+EOF
+cat <<EOF > EXPECTED
+== stdout ==
+1
+1
+== stderr ==
+EOF
+test_expect_equal_file EXPECTED OUTPUT
+
+
 test_done
-- 
2.27.0

Re: [PATCH 10/11] test: add known broken for n_m_get_flag on closed db

[David Bremner]

David Bremner <david@tethera.net> writes:

> Exception caught in next commit. Note that FLAG_GHOST is the only one
> that triggers the I/O code path.

Applied the first 10 patches of this series to master

d

[PATCH 11/11] lib: catch exceptions in n_m_get_flag, provide n_m_get_flag_st

[David Bremner]

It's not very nice to return FALSE for an error, so provide
notmuch_message_get_flag_st as a migration path.
---
 lib/message.cc         | 33 +++++++++++++++++++++++++++++----
 lib/notmuch.h          | 25 +++++++++++++++++++++++++
 test/T560-lib-error.sh | 18 +++++++++++++++++-
 3 files changed, 71 insertions(+), 5 deletions(-)

diff --git a/lib/message.cc b/lib/message.cc
index 8e43a393..81278d5e 100644
--- a/lib/message.cc
+++ b/lib/message.cc
@@ -1161,15 +1161,40 @@ notmuch_message_count_files (notmuch_message_t *message)
     return _notmuch_string_list_length (message->filename_list);
 }
 
+notmuch_status_t
+notmuch_message_get_flag_st (notmuch_message_t *message,
+			     notmuch_message_flag_t flag,
+			     notmuch_bool_t *is_set)
+{
+    if (! is_set)
+	return NOTMUCH_STATUS_NULL_POINTER;
+
+    try {
+	if (flag == NOTMUCH_MESSAGE_FLAG_GHOST &&
+	    ! NOTMUCH_TEST_BIT (message->lazy_flags, flag))
+	    _notmuch_message_ensure_metadata (message, NULL);
+    } catch (Xapian::Error &error) {
+	LOG_XAPIAN_EXCEPTION (message, error);
+	return NOTMUCH_STATUS_XAPIAN_EXCEPTION;
+    }
+
+    *is_set = NOTMUCH_TEST_BIT (message->flags, flag);
+    return NOTMUCH_STATUS_SUCCESS;
+}
+
 notmuch_bool_t
 notmuch_message_get_flag (notmuch_message_t *message,
 			  notmuch_message_flag_t flag)
 {
-    if (flag == NOTMUCH_MESSAGE_FLAG_GHOST &&
-	! NOTMUCH_TEST_BIT (message->lazy_flags, flag))
-	_notmuch_message_ensure_metadata (message, NULL);
+    notmuch_bool_t is_set;
+    notmuch_status_t status;
+
+    status = notmuch_message_get_flag_st (message, flag, &is_set);
 
-    return NOTMUCH_TEST_BIT (message->flags, flag);
+    if (status)
+	return FALSE;
+    else
+	return is_set;
 }
 
 void
diff --git a/lib/notmuch.h b/lib/notmuch.h
index e544aafd..c7b21060 100644
--- a/lib/notmuch.h
+++ b/lib/notmuch.h
@@ -1485,11 +1485,36 @@ typedef enum _notmuch_message_flag {
 
 /**
  * Get a value of a flag for the email corresponding to 'message'.
+ *
+ * returns FALSE in case of errors.
+ *
+ * @deprecated Deprecated as of libnotmuch 5.2 (notmuch 0.31). Please
+ * use notmuch_message_get_flag_st instead.
  */
+NOTMUCH_DEPRECATED(5,2)
 notmuch_bool_t
 notmuch_message_get_flag (notmuch_message_t *message,
 			  notmuch_message_flag_t flag);
 
+/**
+ * Get a value of a flag for the email corresponding to 'message'.
+ *
+ * @param message a message object
+ * @param flag flag to check
+ * @param is_set pointer to boolean to store flag value.
+ *
+ * @retval #NOTMUCH_STATUS_SUCCESS
+ * @retval #NOTMUCH_STATUS_NULL_POINTER is_set is NULL
+ * @retval #NOTMUCH_STATUS_XAPIAN_EXCEPTION Accessing the database
+ * triggered an exception.
+ *
+ * @since libnotmuch 5.2 (notmuch 0.31)
+ */
+notmuch_status_t
+notmuch_message_get_flag_st (notmuch_message_t *message,
+			     notmuch_message_flag_t flag,
+			     notmuch_bool_t *is_set);
+
 /**
  * Set a value of a flag for the email corresponding to 'message'.
  */
diff --git a/test/T560-lib-error.sh b/test/T560-lib-error.sh
index 68aadcdb..06b43ef2 100755
--- a/test/T560-lib-error.sh
+++ b/test/T560-lib-error.sh
@@ -438,7 +438,6 @@ EOF
 test_expect_equal_file EXPECTED OUTPUT
 
 test_begin_subtest "Handle getting ghost flag from closed database"
-test_subtest_known_broken
 cat c_head2 - c_tail <<'EOF' | test_C ${MAIL_DIR}
     {
         notmuch_bool_t result;
@@ -454,5 +453,22 @@ cat <<EOF > EXPECTED
 EOF
 test_expect_equal_file EXPECTED OUTPUT
 
+test_begin_subtest "Handle getting ghost flag from closed database (new API)"
+cat c_head2 - c_tail <<'EOF' | test_C ${MAIL_DIR}
+    {
+        notmuch_bool_t result;
+        notmuch_status_t status;
+        status = notmuch_message_get_flag_st (message, NOTMUCH_MESSAGE_FLAG_GHOST, &result);
+        printf("%d\n%d\n", message != NULL, status == NOTMUCH_STATUS_XAPIAN_EXCEPTION);
+    }
+EOF
+cat <<EOF > EXPECTED
+== stdout ==
+1
+1
+== stderr ==
+EOF
+test_expect_equal_file EXPECTED OUTPUT
+
 
 test_done
-- 
2.27.0

Re: [PATCH 11/11] lib: catch exceptions in n_m_get_flag, provide n_m_get_flag_st

[David Bremner]

David Bremner <david@tethera.net> writes:

> It's not very nice to return FALSE for an error, so provide
> notmuch_message_get_flag_st as a migration path.

applied to master, before the several patches needed to remove
deprecation warnings.