From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from localhost (localhost [127.0.0.1]) by arlo.cworth.org (Postfix) with ESMTP id 8208F6DE019C for ; Sat, 14 Oct 2017 23:48:18 -0700 (PDT) X-Virus-Scanned: Debian amavisd-new at cworth.org X-Spam-Flag: NO X-Spam-Score: -0.036 X-Spam-Level: X-Spam-Status: No, score=-0.036 tagged_above=-999 required=5 tests=[AWL=-0.036] autolearn=disabled Received: from arlo.cworth.org ([127.0.0.1]) by localhost (arlo.cworth.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1IZO7S_RvFE0 for ; Sat, 14 Oct 2017 23:48:17 -0700 (PDT) Received: from che.mayfirst.org (che.mayfirst.org [162.247.75.118]) by arlo.cworth.org (Postfix) with ESMTP id 870F16DE0243 for ; Sat, 14 Oct 2017 23:48:15 -0700 (PDT) Received: from fifthhorseman.net (ool-6c3a0662.static.optonline.net [108.58.6.98]) by che.mayfirst.org (Postfix) with ESMTPSA id 0AAA0F99D for ; Sun, 15 Oct 2017 02:48:14 -0400 (EDT) Received: by fifthhorseman.net (Postfix, from userid 1000) id A338021375; Sun, 15 Oct 2017 02:48:11 -0400 (EDT) From: Daniel Kahn Gillmor To: Notmuch Mail Subject: [PATCH v5 12/12] cli/reindex: add --try-decrypt=(true|false) Date: Sun, 15 Oct 2017 02:48:07 -0400 Message-Id: <20171015064807.14205-13-dkg@fifthhorseman.net> X-Mailer: git-send-email 2.14.2 In-Reply-To: <20171015064807.14205-1-dkg@fifthhorseman.net> References: <20171015064807.14205-1-dkg@fifthhorseman.net> X-BeenThere: notmuch@notmuchmail.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Oct 2017 06:48:18 -0000 Try to decrypt any encrypted parts of newly-discovered messages while re-indexing them. The cleartext of any successfully-decrypted messages will be indexed, with tags applied in the same form as from notmuch insert --try-decrypt=true. Note: if the deprecated crypto.gpg_path configuration option is set to anything other than "gpg", we ignore it (and print a warning on stderr, if built against gmime < 3.0). --- completion/notmuch-completion.bash | 10 +++++- doc/man1/notmuch-reindex.rst | 14 ++++++++ notmuch-reindex.c | 12 +++++-- test/T357-index-decryption.sh | 65 ++++++++++++++++++++++++++++++++++++++ 4 files changed, 98 insertions(+), 3 deletions(-) diff --git a/completion/notmuch-completion.bash b/completion/notmuch-completion.bash index 72a75a94..7aae4297 100644 --- a/completion/notmuch-completion.bash +++ b/completion/notmuch-completion.bash @@ -435,10 +435,18 @@ _notmuch_reindex() local cur prev words cword split _init_completion -s || return + $split && + case "${prev}" in + --try-decrypt) + COMPREPLY=( $( compgen -W "true false" -- "${cur}" ) ) + return + ;; + esac + ! $split && case "${cur}" in -*) - local options="${_notmuch_shared_options}" + local options="--try-decrypt= ${_notmuch_shared_options}" compopt -o nospace COMPREPLY=( $(compgen -W "$options" -- ${cur}) ) ;; diff --git a/doc/man1/notmuch-reindex.rst b/doc/man1/notmuch-reindex.rst index e39cc4ee..60a060a7 100644 --- a/doc/man1/notmuch-reindex.rst +++ b/doc/man1/notmuch-reindex.rst @@ -19,6 +19,20 @@ The **reindex** command searches for all messages matching the supplied search terms, and re-creates the full-text index on these messages using the supplied options. +Supported options for **reindex** include + + ``--try-decrypt=(true|false)`` + + If true, when encountering an encrypted message, try to + decrypt it while reindexing. If decryption is successful, + index the cleartext itself. Be aware that the index is likely + sufficient to reconstruct the cleartext of the message itself, + so please ensure that the notmuch message index is adequately + protected. DO NOT USE ``--try-decrypt=true`` without + considering the security of your index. + + See also ``index.try_decrypt`` in **notmuch-config(1)**. + SEE ALSO ======== diff --git a/notmuch-reindex.c b/notmuch-reindex.c index 57ff5904..2b06ec6b 100644 --- a/notmuch-reindex.c +++ b/notmuch-reindex.c @@ -89,7 +89,7 @@ notmuch_reindex_command (notmuch_config_t *config, int argc, char *argv[]) struct sigaction action; int opt_index; int ret; - notmuch_indexopts_t *indexopts = NULL; + notmuch_status_t status; /* Set up our handler for SIGINT */ memset (&action, 0, sizeof (struct sigaction)); @@ -99,6 +99,7 @@ notmuch_reindex_command (notmuch_config_t *config, int argc, char *argv[]) sigaction (SIGINT, &action, NULL); notmuch_opt_desc_t options[] = { + { .opt_inherit = notmuch_index_options }, { .opt_inherit = notmuch_shared_options }, { } }; @@ -115,6 +116,13 @@ notmuch_reindex_command (notmuch_config_t *config, int argc, char *argv[]) notmuch_exit_if_unmatched_db_uuid (notmuch); + status = notmuch_process_index_options (notmuch, config); + if (status != NOTMUCH_STATUS_SUCCESS) { + fprintf (stderr, "Error: Failed to process index options. (%s)\n", + notmuch_status_to_string (status)); + return EXIT_FAILURE; + } + query_string = query_string_from_args (config, argc-opt_index, argv+opt_index); if (query_string == NULL) { fprintf (stderr, "Out of memory\n"); @@ -126,7 +134,7 @@ notmuch_reindex_command (notmuch_config_t *config, int argc, char *argv[]) return EXIT_FAILURE; } - ret = reindex_query (notmuch, query_string, indexopts); + ret = reindex_query (notmuch, query_string, index_options.opts); notmuch_database_destroy (notmuch); diff --git a/test/T357-index-decryption.sh b/test/T357-index-decryption.sh index d7180797..cd97c48f 100755 --- a/test/T357-index-decryption.sh +++ b/test/T357-index-decryption.sh @@ -48,4 +48,69 @@ test_expect_equal \ "$output" \ "$expected" +# add a tag to all messages to ensure that it stays after reindexing +test_begin_subtest 'tagging all messages' +test_expect_success 'notmuch tag +blarney "encrypted message"' +test_begin_subtest "verify that tags have not changed" +output=$(notmuch search tag:blarney) +expected='thread:0000000000000001 2000-01-01 [1/1] Notmuch Test Suite; test encrypted message for cleartext index 001 (blarney encrypted inbox) +thread:0000000000000002 2000-01-01 [1/1] Notmuch Test Suite; test encrypted message for cleartext index 002 (blarney encrypted inbox)' +test_expect_equal \ + "$output" \ + "$expected" + +# see if first message shows up after reindexing with --try-decrypt=true (same $expected, untouched): +test_begin_subtest 'reindex old messages' +test_expect_success 'notmuch reindex --try-decrypt=true tag:encrypted and not property:index.decryption=success' +test_begin_subtest "reindexed encrypted message, including cleartext" +output=$(notmuch search wumpus) +test_expect_equal \ + "$output" \ + "$expected" + +# and the same search, but by property ($expected is untouched): +test_begin_subtest "emacs search by property for both messages" +output=$(notmuch search property:index.decryption=success) +test_expect_equal \ + "$output" \ + "$expected" + + +# try to remove cleartext indexing +test_begin_subtest 'reindex without cleartext' +test_expect_success 'notmuch reindex tag:encrypted and property:index.decryption=success' +test_begin_subtest "reindexed encrypted messages, without cleartext" +output=$(notmuch search wumpus) +expected='' +test_expect_equal \ + "$output" \ + "$expected" + +# and the same search, but by property ($expected is untouched): +test_begin_subtest "emacs search by property with both messages unindexed" +output=$(notmuch search property:index.decryption=success) +test_expect_equal \ + "$output" \ + "$expected" + +# ensure that the tags remain even when we are dropping the cleartext. +test_begin_subtest "verify that tags remain without cleartext" +output=$(notmuch search tag:blarney) +expected='thread:0000000000000001 2000-01-01 [1/1] Notmuch Test Suite; test encrypted message for cleartext index 001 (blarney encrypted inbox) +thread:0000000000000002 2000-01-01 [1/1] Notmuch Test Suite; test encrypted message for cleartext index 002 (blarney encrypted inbox)' +test_expect_equal \ + "$output" \ + "$expected" + + +# TODO: test removal of a message from the message store between +# indexing and reindexing. + +# TODO: insert the same message into the message store twice, index, +# remove one of them from the message store, and then reindex. +# reindexing should return a failure but the message should still be +# present? -- or what should the semantics be if you ask to reindex a +# message whose underlying files have been renamed or moved or +# removed? + test_done -- 2.14.2