From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from localhost (localhost [127.0.0.1]) by arlo.cworth.org (Postfix) with ESMTP id B09976DE098B for ; Sun, 2 Apr 2017 06:17:03 -0700 (PDT) X-Virus-Scanned: Debian amavisd-new at cworth.org X-Spam-Flag: NO X-Spam-Score: -0.005 X-Spam-Level: X-Spam-Status: No, score=-0.005 tagged_above=-999 required=5 tests=[AWL=0.006, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=disabled Received: from arlo.cworth.org ([127.0.0.1]) by localhost (arlo.cworth.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Wz1IqqBjDsou for ; Sun, 2 Apr 2017 06:17:02 -0700 (PDT) Received: from fethera.tethera.net (fethera.tethera.net [198.245.60.197]) by arlo.cworth.org (Postfix) with ESMTPS id 7204F6DE0941 for ; Sun, 2 Apr 2017 06:17:01 -0700 (PDT) Received: from remotemail by fethera.tethera.net with local (Exim 4.84_2) (envelope-from ) id 1cufMd-0003Bh-Du; Sun, 02 Apr 2017 09:16:15 -0400 Received: (nullmailer pid 30011 invoked by uid 1000); Sun, 02 Apr 2017 13:16:56 -0000 From: David Bremner To: notmuch@notmuchmail.org Cc: Daniel Kahn Gillmor Subject: [rfc patch v2 3/5] add "notmuch reindex" subcommand Date: Sun, 2 Apr 2017 10:16:44 -0300 Message-Id: <20170402131646.29884-4-david@tethera.net> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20170402131646.29884-1-david@tethera.net> References: <20170402131646.29884-1-david@tethera.net> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: notmuch@notmuchmail.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Apr 2017 13:17:03 -0000 From: Daniel Kahn Gillmor This new subcommand takes a set of search terms, and re-indexes the list of matching messages. --- Makefile.local | 1 + doc/conf.py | 4 ++ doc/index.rst | 1 + doc/man1/notmuch-reindex.rst | 29 +++++++++ doc/man1/notmuch.rst | 4 +- doc/man7/notmuch-search-terms.rst | 7 +- notmuch-client.h | 3 + notmuch-reindex.c | 132 ++++++++++++++++++++++++++++++++++++++ notmuch.c | 2 + test/T700-reindex.sh | 21 ++++++ 10 files changed, 200 insertions(+), 4 deletions(-) create mode 100644 doc/man1/notmuch-reindex.rst create mode 100644 notmuch-reindex.c create mode 100755 test/T700-reindex.sh diff --git a/Makefile.local b/Makefile.local index 03eafaaa..c6e272bc 100644 --- a/Makefile.local +++ b/Makefile.local @@ -222,6 +222,7 @@ notmuch_client_srcs = \ notmuch-dump.c \ notmuch-insert.c \ notmuch-new.c \ + notmuch-reindex.c \ notmuch-reply.c \ notmuch-restore.c \ notmuch-search.c \ diff --git a/doc/conf.py b/doc/conf.py index a3d82696..aa864b3c 100644 --- a/doc/conf.py +++ b/doc/conf.py @@ -95,6 +95,10 @@ man_pages = [ u'incorporate new mail into the notmuch database', [notmuch_authors], 1), + ('man1/notmuch-reindex', 'notmuch-reindex', + u're-index matching messages', + [notmuch_authors], 1), + ('man1/notmuch-reply', 'notmuch-reply', u'constructs a reply template for a set of messages', [notmuch_authors], 1), diff --git a/doc/index.rst b/doc/index.rst index 344606d9..aa6c9f40 100644 --- a/doc/index.rst +++ b/doc/index.rst @@ -18,6 +18,7 @@ Contents: man5/notmuch-hooks man1/notmuch-insert man1/notmuch-new + man1/notmuch-reindex man1/notmuch-reply man1/notmuch-restore man1/notmuch-search diff --git a/doc/man1/notmuch-reindex.rst b/doc/man1/notmuch-reindex.rst new file mode 100644 index 00000000..6c786b85 --- /dev/null +++ b/doc/man1/notmuch-reindex.rst @@ -0,0 +1,29 @@ +=========== +notmuch-reindex +=========== + +SYNOPSIS +======== + +**notmuch** **reindex** [*option* ...] <*search-term*> ... + +DESCRIPTION +=========== + +Re-index all messages matching the search terms. + +See **notmuch-search-terms(7)** for details of the supported syntax for +<*search-term*\ >. + +The **reindex** command searches for all messages matching the +supplied search terms, and re-creates the full-text index on these +messages using the supplied options. + +SEE ALSO +======== + +**notmuch(1)**, **notmuch-config(1)**, **notmuch-count(1)**, +**notmuch-dump(1)**, **notmuch-hooks(5)**, **notmuch-insert(1)**, +**notmuch-new(1)**, +**notmuch-reply(1)**, **notmuch-restore(1)**, **notmuch-search(1)**, +**notmuch-search-terms(7)**, **notmuch-show(1)**, **notmuch-tag(1)** diff --git a/doc/man1/notmuch.rst b/doc/man1/notmuch.rst index fbd7f381..b2a8376e 100644 --- a/doc/man1/notmuch.rst +++ b/doc/man1/notmuch.rst @@ -149,8 +149,8 @@ SEE ALSO **notmuch-address(1)**, **notmuch-compact(1)**, **notmuch-config(1)**, **notmuch-count(1)**, **notmuch-dump(1)**, **notmuch-hooks(5)**, -**notmuch-insert(1)**, **notmuch-new(1)**, **notmuch-reply(1)**, -**notmuch-restore(1)**, **notmuch-search(1)**, +**notmuch-insert(1)**, **notmuch-new(1)**, **notmuch-reindex(1)**, +**notmuch-reply(1)**, **notmuch-restore(1)**, **notmuch-search(1)**, **notmuch-search-terms(7)**, **notmuch-show(1)**, **notmuch-tag(1)** The notmuch website: **https://notmuchmail.org** diff --git a/doc/man7/notmuch-search-terms.rst b/doc/man7/notmuch-search-terms.rst index 47cab48d..dd76972e 100644 --- a/doc/man7/notmuch-search-terms.rst +++ b/doc/man7/notmuch-search-terms.rst @@ -9,6 +9,8 @@ SYNOPSIS **notmuch** **dump** [--format=(batch-tag|sup)] [--] [--output=<*file*>] [--] [<*search-term*> ...] +**notmuch** **reindex** [option ...] <*search-term*> ... + **notmuch** **search** [option ...] <*search-term*> ... **notmuch** **show** [option ...] <*search-term*> ... @@ -421,5 +423,6 @@ SEE ALSO **notmuch(1)**, **notmuch-config(1)**, **notmuch-count(1)**, **notmuch-dump(1)**, **notmuch-hooks(5)**, **notmuch-insert(1)**, -**notmuch-new(1)**, **notmuch-reply(1)**, **notmuch-restore(1)**, -**notmuch-search(1)**, **notmuch-show(1)**, **notmuch-tag(1)** +**notmuch-new(1)**, **notmuch-reindex(1)**, **notmuch-reply(1)**, +**notmuch-restore(1)**, **notmuch-search(1)**, **notmuch-show(1)**, +**notmuch-tag(1)** diff --git a/notmuch-client.h b/notmuch-client.h index a6f70eae..ab7138c6 100644 --- a/notmuch-client.h +++ b/notmuch-client.h @@ -196,6 +196,9 @@ int notmuch_insert_command (notmuch_config_t *config, int argc, char *argv[]); int +notmuch_reindex_command (notmuch_config_t *config, int argc, char *argv[]); + +int notmuch_reply_command (notmuch_config_t *config, int argc, char *argv[]); int diff --git a/notmuch-reindex.c b/notmuch-reindex.c new file mode 100644 index 00000000..836a90a1 --- /dev/null +++ b/notmuch-reindex.c @@ -0,0 +1,132 @@ +/* notmuch - Not much of an email program, (just index and search) + * + * Copyright © 2016 Daniel Kahn Gillmor + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see http://www.gnu.org/licenses/ . + * + * Author: Daniel Kahn Gillmor + */ + +#include "notmuch-client.h" +#include "string-util.h" + +static volatile sig_atomic_t interrupted; + +static void +handle_sigint (unused (int sig)) +{ + static char msg[] = "Stopping... \n"; + + /* This write is "opportunistic", so it's okay to ignore the + * result. It is not required for correctness, and if it does + * fail or produce a short write, we want to get out of the signal + * handler as quickly as possible, not retry it. */ + IGNORE_RESULT (write (2, msg, sizeof (msg) - 1)); + interrupted = 1; +} + +/* reindex all messages matching 'query_string' using the passed-in indexopts + */ +static int +reindex_query (notmuch_database_t *notmuch, const char *query_string, + notmuch_param_t *indexopts) +{ + notmuch_query_t *query; + notmuch_messages_t *messages; + notmuch_message_t *message; + notmuch_status_t status; + + int ret = NOTMUCH_STATUS_SUCCESS; + + query = notmuch_query_create (notmuch, query_string); + if (query == NULL) { + fprintf (stderr, "Out of memory.\n"); + return 1; + } + + /* reindexing is not interested in any special sort order */ + notmuch_query_set_sort (query, NOTMUCH_SORT_UNSORTED); + + status = notmuch_query_search_messages (query, &messages); + if (print_status_query ("notmuch reindex", query, status)) + return status; + + for (; + notmuch_messages_valid (messages) && ! interrupted; + notmuch_messages_move_to_next (messages)) { + message = notmuch_messages_get (messages); + + notmuch_message_reindex(message, indexopts); + notmuch_message_destroy (message); + if (ret != NOTMUCH_STATUS_SUCCESS) + break; + } + + notmuch_query_destroy (query); + + return ret || interrupted; +} + +int +notmuch_reindex_command (notmuch_config_t *config, int argc, char *argv[]) +{ + char *query_string = NULL; + notmuch_database_t *notmuch; + struct sigaction action; + notmuch_bool_t try_decrypt = FALSE; + int opt_index; + int ret; + notmuch_param_t *indexopts = NULL; + + /* Set up our handler for SIGINT */ + memset (&action, 0, sizeof (struct sigaction)); + action.sa_handler = handle_sigint; + sigemptyset (&action.sa_mask); + action.sa_flags = SA_RESTART; + sigaction (SIGINT, &action, NULL); + + notmuch_opt_desc_t options[] = { + { NOTMUCH_OPT_INHERIT, (void *) ¬much_shared_options, NULL, 0, 0 }, + { 0, 0, 0, 0, 0 } + }; + + opt_index = parse_arguments (argc, argv, options, 1); + if (opt_index < 0) + return EXIT_FAILURE; + + notmuch_process_shared_options (argv[0]); + + if (notmuch_database_open (notmuch_config_get_database_path (config), + NOTMUCH_DATABASE_MODE_READ_WRITE, ¬much)) + return EXIT_FAILURE; + + notmuch_exit_if_unmatched_db_uuid (notmuch); + + query_string = query_string_from_args (config, argc-opt_index, argv+opt_index); + if (query_string == NULL) { + fprintf (stderr, "Out of memory\n"); + return EXIT_FAILURE; + } + + if (*query_string == '\0') { + fprintf (stderr, "Error: notmuch reindex requires at least one search term.\n"); + return EXIT_FAILURE; + } + + ret = reindex_query (notmuch, query_string, indexopts); + + notmuch_database_destroy (notmuch); + + return ret || interrupted ? EXIT_FAILURE : EXIT_SUCCESS; +} diff --git a/notmuch.c b/notmuch.c index 8e332ce6..201c7454 100644 --- a/notmuch.c +++ b/notmuch.c @@ -123,6 +123,8 @@ static command_t commands[] = { "Restore the tags from the given dump file (see 'dump')." }, { "compact", notmuch_compact_command, NOTMUCH_CONFIG_OPEN, "Compact the notmuch database." }, + { "reindex", notmuch_reindex_command, NOTMUCH_CONFIG_OPEN, + "Re-index all messages matching the search terms." }, { "config", notmuch_config_command, NOTMUCH_CONFIG_OPEN, "Get or set settings in the notmuch configuration file." }, { "help", notmuch_help_command, NOTMUCH_CONFIG_CREATE, /* create but don't save config */ diff --git a/test/T700-reindex.sh b/test/T700-reindex.sh new file mode 100755 index 00000000..32385a72 --- /dev/null +++ b/test/T700-reindex.sh @@ -0,0 +1,21 @@ +#!/usr/bin/env bash +test_description='reindexing messages' +. ./test-lib.sh || exit 1 + +add_email_corpus + +test_begin_subtest 'reindex preserves message-ids' +notmuch search --output=messages '*' > EXPECTED +# remove duplicate file +rm $MAIL_DIR/bar/18:2, +notmuch reindex '*' +notmuch search --output=messages '*' > OUTPUT +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest 'reindex preserves tags' +notmuch dump > EXPECTED +notmuch reindex '*' +notmuch dump > OUTPUT +test_expect_equal_file EXPECTED OUTPUT + +test_done -- 2.11.0