From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from localhost (localhost [127.0.0.1]) by olra.theworths.org (Postfix) with ESMTP id D4B76431FC7 for ; Tue, 29 Jul 2014 09:49:18 -0700 (PDT) X-Virus-Scanned: Debian amavisd-new at olra.theworths.org X-Spam-Flag: NO X-Spam-Score: -2.3 X-Spam-Level: X-Spam-Status: No, score=-2.3 tagged_above=-999 required=5 tests=[RCVD_IN_DNSWL_MED=-2.3] autolearn=disabled Received: from olra.theworths.org ([127.0.0.1]) by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id dHmVjMPNZrTi for ; Tue, 29 Jul 2014 09:49:12 -0700 (PDT) Received: from dmz-mailsec-scanner-4.mit.edu (dmz-mailsec-scanner-4.mit.edu [18.9.25.15]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by olra.theworths.org (Postfix) with ESMTPS id 8A37A431FBC for ; Tue, 29 Jul 2014 09:48:31 -0700 (PDT) X-AuditID: 1209190f-f79f86d0000061c8-12-53d7d05d56c3 Received: from mailhub-auth-3.mit.edu ( [18.9.21.43]) (using TLS with cipher AES256-SHA (256/256 bits)) (Client did not present a certificate) by dmz-mailsec-scanner-4.mit.edu (Symantec Messaging Gateway) with SMTP id 83.B5.25032.D50D7D35; Tue, 29 Jul 2014 12:48:29 -0400 (EDT) Received: from outgoing.mit.edu (outgoing-auth-1.mit.edu [18.9.28.11]) by mailhub-auth-3.mit.edu (8.13.8/8.9.2) with ESMTP id s6TGmIu7030916; Tue, 29 Jul 2014 12:48:18 -0400 Received: from drake.dyndns.org (31-33-71.wireless.csail.mit.edu [128.31.33.71]) (authenticated bits=0) (User authenticated as amdragon@ATHENA.MIT.EDU) by outgoing.mit.edu (8.13.8/8.12.4) with ESMTP id s6TGmFXl030266 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=NOT); Tue, 29 Jul 2014 12:48:17 -0400 Received: from amthrax by drake.dyndns.org with local (Exim 4.77) (envelope-from ) id 1XCAZS-0007H2-UZ; Tue, 29 Jul 2014 12:48:14 -0400 From: Austin Clements To: notmuch@notmuchmail.org Subject: [PATCH v2 10/14] lib: Reorganize upgrade around document types Date: Tue, 29 Jul 2014 12:48:08 -0400 Message-Id: <1406652492-27803-11-git-send-email-amdragon@mit.edu> X-Mailer: git-send-email 2.0.0 In-Reply-To: <1406652492-27803-1-git-send-email-amdragon@mit.edu> References: <1406652492-27803-1-git-send-email-amdragon@mit.edu> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFtrFIsWRmVeSWpSXmKPExsUixCmqrRt74XqwwbdHyhar5/JYXL85k9mB yWPnrLvsHs9W3WIOYIrisklJzcksSy3St0vgynj+rI29YK1ixYUj5Q2Ma6S6GDk5JARMJI6s +ssEYYtJXLi3ng3EFhKYzSSxanZgFyMXkL2RUWLv3GeMEIljTBKPm0UhEnMZJT6vW8cCkmAT 0JDYtn85WJGIgLTEzruzWUFsZgFHic/7F4FNFRZwl3g59S3YNhYBVYl3U9vB4rxANcdbn7BB XCEn0XDjE5jNCRS/de8uE8RiB4lDszazTGDkX8DIsIpRNiW3Sjc3MTOnODVZtzg5MS8vtUjX RC83s0QvNaV0EyM4iCT5dzB+O6h0iFGAg1GJh5djwbVgIdbEsuLK3EOMkhxMSqK8evuuBwvx JeWnVGYkFmfEF5XmpBYfYpTgYFYS4eU7C5TjTUmsrEotyodJSXOwKInzvrW2ChYSSE8sSc1O TS1ILYLJynBwKEnwGp0HahQsSk1PrUjLzClBSDNxcIIM5wEa3gxSw1tckJhbnJkOkT/FqCgl zpt9DighAJLIKM2D64VF+StGcaBXhHndQdp5gAkCrvsV0GAmoMGsLmCDSxIRUlINjJIHFypP 9nkv837VN6vwE7NzfN3KP8xjjdi3+NiTbJPH6qpxs1JC++9MDUh4I2kj+qhQ3qIoms0zYm69 nGdRguyza4Jdjl/y43Z6/lJwsf+w+Z7VtCUz1pv/XON84cuF6c9tveZIynk3LeAN+XNz0TEd Lxnzr/7dFSsPPp0l2le5f3Jb0Qn1bCWW4oxEQy3mouJEAL46PhXNAgAA X-BeenThere: notmuch@notmuchmail.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Jul 2014 16:49:19 -0000 Rather than potentially making multiple passes over the same type of data in the database, reorganize upgrade around each type of data that may be upgraded. This eliminates code duplication, will make multi-version upgrades faster, and will let us improve progress reporting. --- lib/database.cc | 71 +++++++++++++++++++++------------------------------------ 1 file changed, 26 insertions(+), 45 deletions(-) diff --git a/lib/database.cc b/lib/database.cc index 5f68550..4940634 100644 --- a/lib/database.cc +++ b/lib/database.cc @@ -1237,11 +1237,9 @@ notmuch_database_upgrade (notmuch_database_t *notmuch, /* Perform the upgrade in a transaction. */ db->begin_transaction (true); - /* Before version 1, each message document had its filename in the - * data field. Copy that into the new format by calling - * notmuch_message_add_filename. - */ - if (new_features & NOTMUCH_FEATURE_FILE_TERMS) { + /* Perform per-message upgrades. */ + if (new_features & + (NOTMUCH_FEATURE_FILE_TERMS | NOTMUCH_FEATURE_BOOL_FOLDER)) { notmuch_query_t *query = notmuch_query_create (notmuch, ""); notmuch_messages_t *messages; notmuch_message_t *message; @@ -1260,13 +1258,28 @@ notmuch_database_upgrade (notmuch_database_t *notmuch, message = notmuch_messages_get (messages); - filename = _notmuch_message_talloc_copy_data (message); - if (filename && *filename != '\0') { - _notmuch_message_add_filename (message, filename); - _notmuch_message_clear_data (message); + /* Before version 1, each message document had its + * filename in the data field. Copy that into the new + * format by calling notmuch_message_add_filename. + */ + if (new_features & NOTMUCH_FEATURE_FILE_TERMS) { + filename = _notmuch_message_talloc_copy_data (message); + if (filename && *filename != '\0') { + _notmuch_message_add_filename (message, filename); + _notmuch_message_clear_data (message); + _notmuch_message_sync (message); + } + talloc_free (filename); + } + + /* Prior to version 2, the "folder:" prefix was + * probabilistic and stemmed. Change it to the current + * boolean prefix. Add "path:" prefixes while at it. + */ + if (new_features & NOTMUCH_FEATURE_BOOL_FOLDER) { + _notmuch_message_upgrade_folder (message); _notmuch_message_sync (message); } - talloc_free (filename); notmuch_message_destroy (message); @@ -1276,7 +1289,9 @@ notmuch_database_upgrade (notmuch_database_t *notmuch, notmuch_query_destroy (query); } - /* Also, before version 1 we stored directory timestamps in + /* Perform per-directory upgrades. */ + + /* Before version 1 we stored directory timestamps in * XTIMESTAMP documents instead of the current XDIRECTORY * documents. So copy those as well. */ if (new_features & NOTMUCH_FEATURE_DIRECTORY_DOCS) { @@ -1318,40 +1333,6 @@ notmuch_database_upgrade (notmuch_database_t *notmuch, } } - /* - * Prior to version 2, the "folder:" prefix was probabilistic and - * stemmed. Change it to the current boolean prefix. Add "path:" - * prefixes while at it. - */ - if (new_features & NOTMUCH_FEATURE_BOOL_FOLDER) { - notmuch_query_t *query = notmuch_query_create (notmuch, ""); - notmuch_messages_t *messages; - notmuch_message_t *message; - - count = 0; - total = notmuch_query_count_messages (query); - - for (messages = notmuch_query_search_messages (query); - notmuch_messages_valid (messages); - notmuch_messages_move_to_next (messages)) { - if (do_progress_notify) { - progress_notify (closure, (double) count / total); - do_progress_notify = 0; - } - - message = notmuch_messages_get (messages); - - _notmuch_message_upgrade_folder (message); - _notmuch_message_sync (message); - - notmuch_message_destroy (message); - - count++; - } - - notmuch_query_destroy (query); - } - notmuch->features = target_features; db->set_metadata ("features", _print_features (local, notmuch->features)); db->set_metadata ("version", STRINGIFY (NOTMUCH_DATABASE_VERSION)); -- 2.0.0