[PATCH] new: Improved workaround for mistaken new directories

[PATCH] new: Improved workaround for mistaken new directories

[Austin Clements]

Currently, notmuch new assumes any directory with a database mtime of
0 is new, but we don't set the mtime until after processing messages
and subdirectories in that directory.  Hence, anything that prevents
the mtime update (such as an interruption or the wall-clock logic
introduced in 8c39e8d6) will cause the next notmuch new to think the
directory is still new.

We work around this by setting the new directory's database mtime to
-1 before scanning anything in the new directory.  This also obviates
the need for the workaround used in 8c39e8d6.
---
 notmuch-new.c |   31 ++++++++++++++++++++-----------
 1 files changed, 20 insertions(+), 11 deletions(-)

diff --git a/notmuch-new.c b/notmuch-new.c
index c180591..7d17793 100644
--- a/notmuch-new.c
+++ b/notmuch-new.c
@@ -256,6 +256,25 @@ add_files_recursive (notmuch_database_t *notmuch,
 
     new_directory = db_mtime ? FALSE : TRUE;
 
+    /* XXX This is a temporary workaround.  If we don't update the
+     * database mtime until after processing messages in this
+     * directory, then a 0 mtime is *not* sufficient to indicate that
+     * this directory has no messages or subdirs in the database (for
+     * example, if an earlier run skipped the mtime update because
+     * fs_mtime == stat_time, or was interrupted before updating the
+     * mtime at the end).  To address this, we record a (bogus)
+     * non-zero value before processing any child messages so that a
+     * later run won't mistake this for a new directory (and, for
+     * example, fail to detect removed files and subdirs).
+     *
+     * A better solution would be for notmuch_database_get_directory
+     * to indicate if it really created a new directory or not, either
+     * by a new out-argument, or by recording this information and
+     * providing an accessor.
+     */
+    if (new_directory)
+	notmuch_directory_set_mtime (directory, -1);
+
     /* If the database knows about this directory, then we sort based
      * on strcmp to match the database sorting. Otherwise, we can do
      * inode-based sorting for faster filesystem operation. */
@@ -516,17 +535,7 @@ add_files_recursive (notmuch_database_t *notmuch,
      * when we stat'ed the directory, we skip updating the mtime in
      * the database because a message could be delivered later in this
      * same second.  This may lead to unnecessary re-scans, but it
-     * avoids overlooking messages.
-     *
-     * XXX Bug workaround: If this is a new directory, we *must*
-     * update the mtime; otherwise the next run will see the 0 mtime
-     * and think this is still a new directory containing no files or
-     * subdirs (which is unsound in general).  If fs_mtime ==
-     * stat_time, we set the database mtime to a bogus (but non-zero!)
-     * value to force a rescan.
-     */
-    if (new_directory && fs_mtime == stat_time)
-	fs_mtime = 1;
+     * avoids overlooking messages. */
     if (! interrupted && fs_mtime != stat_time) {
 	status = notmuch_directory_set_mtime (directory, fs_mtime);
 	if (status && ret == NOTMUCH_STATUS_SUCCESS)
-- 
1.7.5.1

Re: [PATCH] new: Improved workaround for mistaken new directories

[Carl Worth]

[-- Attachment #1: Type: text/plain, Size: 875 bytes --]

On Wed, 29 Jun 2011 19:00:01 -0400, Austin Clements <amdragon@MIT.EDU> wrote:
> We work around this by setting the new directory's database mtime to
> -1 before scanning anything in the new directory.  This also obviates
> the need for the workaround used in 8c39e8d6.

Thanks. This is pushed.

> +    /* XXX This is a temporary workaround.  If we don't update the
> +     * database mtime until after processing messages in this
...
> +     * A better solution would be for notmuch_database_get_directory
> +     * to indicate if it really created a new directory or not, either
> +     * by a new out-argument, or by recording this information and
> +     * providing an accessor.

A much better "XXX" comment! It tells the potential reader of the future
what to do to be able to remove the comment. Thanks so much.

-Carl

-- 
carl.d.worth@intel.com

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]