unofficial mirror of notmuch@notmuchmail.org
 help / color / mirror / code / Atom feed
From: Austin Clements <amdragon@MIT.EDU>
To: notmuch@notmuchmail.org
Subject: [PATCH v3 09/13] lib: Reorganize upgrade around document types
Date: Thu, 31 Jul 2014 22:09:59 -0400	[thread overview]
Message-ID: <1406859003-11561-10-git-send-email-amdragon@mit.edu> (raw)
In-Reply-To: <1406859003-11561-1-git-send-email-amdragon@mit.edu>

Rather than potentially making multiple passes over the same type of
data in the database, reorganize upgrade around each type of data that
may be upgraded.  This eliminates code duplication, will make
multi-version upgrades faster, and will let us improve progress
reporting.
---
 lib/database.cc | 72 +++++++++++++++++++++------------------------------------
 1 file changed, 26 insertions(+), 46 deletions(-)

diff --git a/lib/database.cc b/lib/database.cc
index 69d775f..31e6a93 100644
--- a/lib/database.cc
+++ b/lib/database.cc
@@ -1241,11 +1241,9 @@ notmuch_database_upgrade (notmuch_database_t *notmuch,
      * format. */
     notmuch->features = target_features;
 
-    /* Before version 1, each message document had its filename in the
-     * data field. Copy that into the new format by calling
-     * notmuch_message_add_filename.
-     */
-    if (new_features & NOTMUCH_FEATURE_FILE_TERMS) {
+    /* Perform per-message upgrades. */
+    if (new_features &
+	(NOTMUCH_FEATURE_FILE_TERMS | NOTMUCH_FEATURE_BOOL_FOLDER)) {
 	notmuch_query_t *query = notmuch_query_create (notmuch, "");
 	notmuch_messages_t *messages;
 	notmuch_message_t *message;
@@ -1264,13 +1262,27 @@ notmuch_database_upgrade (notmuch_database_t *notmuch,
 
 	    message = notmuch_messages_get (messages);
 
-	    filename = _notmuch_message_talloc_copy_data (message);
-	    if (filename && *filename != '\0') {
-		_notmuch_message_add_filename (message, filename);
-		_notmuch_message_clear_data (message);
-		_notmuch_message_sync (message);
+	    /* Before version 1, each message document had its
+	     * filename in the data field. Copy that into the new
+	     * format by calling notmuch_message_add_filename.
+	     */
+	    if (new_features & NOTMUCH_FEATURE_FILE_TERMS) {
+		filename = _notmuch_message_talloc_copy_data (message);
+		if (filename && *filename != '\0') {
+		    _notmuch_message_add_filename (message, filename);
+		    _notmuch_message_clear_data (message);
+		}
+		talloc_free (filename);
 	    }
-	    talloc_free (filename);
+
+	    /* Prior to version 2, the "folder:" prefix was
+	     * probabilistic and stemmed. Change it to the current
+	     * boolean prefix. Add "path:" prefixes while at it.
+	     */
+	    if (new_features & NOTMUCH_FEATURE_BOOL_FOLDER)
+		_notmuch_message_upgrade_folder (message);
+
+	    _notmuch_message_sync (message);
 
 	    notmuch_message_destroy (message);
 
@@ -1280,7 +1292,9 @@ notmuch_database_upgrade (notmuch_database_t *notmuch,
 	notmuch_query_destroy (query);
     }
 
-    /* Also, before version 1 we stored directory timestamps in
+    /* Perform per-directory upgrades. */
+
+    /* Before version 1 we stored directory timestamps in
      * XTIMESTAMP documents instead of the current XDIRECTORY
      * documents. So copy those as well. */
     if (new_features & NOTMUCH_FEATURE_DIRECTORY_DOCS) {
@@ -1322,40 +1336,6 @@ notmuch_database_upgrade (notmuch_database_t *notmuch,
 	}
     }
 
-    /*
-     * Prior to version 2, the "folder:" prefix was probabilistic and
-     * stemmed. Change it to the current boolean prefix. Add "path:"
-     * prefixes while at it.
-     */
-    if (new_features & NOTMUCH_FEATURE_BOOL_FOLDER) {
-	notmuch_query_t *query = notmuch_query_create (notmuch, "");
-	notmuch_messages_t *messages;
-	notmuch_message_t *message;
-
-	count = 0;
-	total = notmuch_query_count_messages (query);
-
-	for (messages = notmuch_query_search_messages (query);
-	     notmuch_messages_valid (messages);
-	     notmuch_messages_move_to_next (messages)) {
-	    if (do_progress_notify) {
-		progress_notify (closure, (double) count / total);
-		do_progress_notify = 0;
-	    }
-
-	    message = notmuch_messages_get (messages);
-
-	    _notmuch_message_upgrade_folder (message);
-	    _notmuch_message_sync (message);
-
-	    notmuch_message_destroy (message);
-
-	    count++;
-	}
-
-	notmuch_query_destroy (query);
-    }
-
     db->set_metadata ("features", _print_features (local, notmuch->features));
     db->set_metadata ("version", STRINGIFY (NOTMUCH_DATABASE_VERSION));
 
-- 
2.0.0

  parent reply	other threads:[~2014-08-01  2:10 UTC|newest]

Thread overview: 28+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2014-08-01  2:09 [PATCH v3 00/13] Implement and use database "features" Austin Clements
2014-08-01  2:09 ` [PATCH v3 01/13] test: Include generated dependencies for test sources Austin Clements
2014-08-01  2:09 ` [PATCH v3 02/13] util: Const version of strtok_len Austin Clements
2014-08-06 13:43   ` David Bremner
2014-08-01  2:09 ` [PATCH v3 03/13] new: Don't report version after upgrade Austin Clements
2014-08-23 15:39   ` Jani Nikula
2014-08-23 22:59     ` Austin Clements
2014-08-24 12:56       ` Jani Nikula
2014-08-01  2:09 ` [PATCH v3 04/13] lib: Database version 3: Introduce fine-grained "features" Austin Clements
2014-08-23 16:02   ` Jani Nikula
2014-08-24  0:57     ` Austin Clements
2014-08-23 22:21   ` David Bremner
2014-08-24  0:58     ` Austin Clements
2014-08-24  3:58       ` David Bremner
2014-08-24 22:16         ` Austin Clements
2014-08-24 23:03           ` David Bremner
2014-08-01  2:09 ` [PATCH v3 05/13] test: Tool to build DB with specific version and features Austin Clements
2014-08-23 16:03   ` Jani Nikula
2014-08-01  2:09 ` [PATCH v3 06/13] test: Tests for future version and unknown feature handling Austin Clements
2014-08-01  2:09 ` [PATCH v3 07/13] lib: Simplify upgrade code using a transaction Austin Clements
2014-08-01  2:09 ` [PATCH v3 08/13] lib: Use database features to drive upgrade Austin Clements
2014-08-01  2:09 ` Austin Clements [this message]
2014-08-01  2:10 ` [PATCH v3 10/13] lib: Report progress for combined upgrade operation Austin Clements
2014-08-01  2:10 ` [PATCH v3 11/13] lib: Support empty header values in database Austin Clements
2014-08-01  2:10 ` [PATCH v3 12/13] lib: Return an error from operations that require an upgrade Austin Clements
2014-08-01  2:10 ` [PATCH v3 13/13] lib: Update doc of notmuch_database_{needs_upgrade, upgrade} Austin Clements
2014-08-07 21:55 ` [PATCH v3 00/13] Implement and use database "features" Tomi Ollila
2014-08-08 18:18   ` Austin Clements

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://notmuchmail.org/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1406859003-11561-10-git-send-email-amdragon@mit.edu \
    --to=amdragon@mit.edu \
    --cc=notmuch@notmuchmail.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://yhetil.org/notmuch.git/

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).