From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from localhost (localhost [127.0.0.1]) by arlo.cworth.org (Postfix) with ESMTP id 7FADF6DE0C6D for ; Tue, 12 Sep 2017 16:02:02 -0700 (PDT) X-Virus-Scanned: Debian amavisd-new at cworth.org X-Spam-Flag: NO X-Spam-Score: -0.053 X-Spam-Level: X-Spam-Status: No, score=-0.053 tagged_above=-999 required=5 tests=[AWL=-0.053] autolearn=disabled Received: from arlo.cworth.org ([127.0.0.1]) by localhost (arlo.cworth.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id EK6_au7iqjx2 for ; Tue, 12 Sep 2017 16:02:01 -0700 (PDT) Received: from che.mayfirst.org (che.mayfirst.org [162.247.75.118]) by arlo.cworth.org (Postfix) with ESMTP id 7B3076DE096A for ; Tue, 12 Sep 2017 16:01:58 -0700 (PDT) Received: from fifthhorseman.net (unknown [38.109.115.130]) by che.mayfirst.org (Postfix) with ESMTPSA id 4EA52F99F for ; Tue, 12 Sep 2017 19:01:56 -0400 (EDT) Received: by fifthhorseman.net (Postfix, from userid 1000) id AECB120961; Tue, 12 Sep 2017 19:01:53 -0400 (EDT) From: Daniel Kahn Gillmor To: Notmuch Mail Subject: [PATCH 10/10] add --try-decrypt=(true|false) to notmuch reindex Date: Tue, 12 Sep 2017 19:01:53 -0400 Message-Id: <20170912230153.4175-10-dkg@fifthhorseman.net> X-Mailer: git-send-email 2.14.1 In-Reply-To: <20170912230153.4175-1-dkg@fifthhorseman.net> References: <20170912230153.4175-1-dkg@fifthhorseman.net> X-BeenThere: notmuch@notmuchmail.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Sep 2017 23:02:02 -0000 Try to decrypt any encrypted parts of newly-discovered messages while re-indexing them. The cleartext of any successfully-decrypted messages will be indexed, with tags applied in the same form as from notmuch insert --try-decrypt=true. Note: if the deprecated crypto.gpg_path configuration option is set to anything other than "gpg", we ignore it (and print a warning on stderr, if built against gmime < 3.0). --- completion/notmuch-completion.bash | 10 +++++- doc/man1/notmuch-reindex.rst | 14 ++++++++ notmuch-reindex.c | 23 ++++++++++++++ test/T357-index-decryption.sh | 65 ++++++++++++++++++++++++++++++++++++++ 4 files changed, 111 insertions(+), 1 deletion(-) diff --git a/completion/notmuch-completion.bash b/completion/notmuch-completion.bash index 72a75a94..ef79affe 100644 --- a/completion/notmuch-completion.bash +++ b/completion/notmuch-completion.bash @@ -435,10 +435,18 @@ _notmuch_reindex() local cur prev words cword split _init_completion -s || return + $split && + case "${prev}" in + --try-decrypt) + COMPREPLY=( $( compgen -W "true false" -- "${cur}" ) ) + return + ;; + esac + ! $split && case "${cur}" in -*) - local options="${_notmuch_shared_options}" + local options="--try-decrypt= ${_notmuch_shared_options}" compopt -o nospace COMPREPLY=( $(compgen -W "$options" -- ${cur}) ) ;; diff --git a/doc/man1/notmuch-reindex.rst b/doc/man1/notmuch-reindex.rst index e39cc4ee..60a060a7 100644 --- a/doc/man1/notmuch-reindex.rst +++ b/doc/man1/notmuch-reindex.rst @@ -19,6 +19,20 @@ The **reindex** command searches for all messages matching the supplied search terms, and re-creates the full-text index on these messages using the supplied options. +Supported options for **reindex** include + + ``--try-decrypt=(true|false)`` + + If true, when encountering an encrypted message, try to + decrypt it while reindexing. If decryption is successful, + index the cleartext itself. Be aware that the index is likely + sufficient to reconstruct the cleartext of the message itself, + so please ensure that the notmuch message index is adequately + protected. DO NOT USE ``--try-decrypt=true`` without + considering the security of your index. + + See also ``index.try_decrypt`` in **notmuch-config(1)**. + SEE ALSO ======== diff --git a/notmuch-reindex.c b/notmuch-reindex.c index bceac722..83cd0a57 100644 --- a/notmuch-reindex.c +++ b/notmuch-reindex.c @@ -90,6 +90,8 @@ notmuch_reindex_command (notmuch_config_t *config, int argc, char *argv[]) int opt_index; int ret; notmuch_indexopts_t *indexopts = NULL; + int try_decrypt = -1; + notmuch_status_t status; /* Set up our handler for SIGINT */ memset (&action, 0, sizeof (struct sigaction)); @@ -99,6 +101,7 @@ notmuch_reindex_command (notmuch_config_t *config, int argc, char *argv[]) sigaction (SIGINT, &action, NULL); notmuch_opt_desc_t options[] = { + { NOTMUCH_OPT_BOOLEAN, &try_decrypt, "try-decrypt", 0, 0 }, { NOTMUCH_OPT_INHERIT, (void *) ¬much_shared_options, NULL, 0, 0 }, { 0, 0, 0, 0, 0 } }; @@ -115,6 +118,26 @@ notmuch_reindex_command (notmuch_config_t *config, int argc, char *argv[]) notmuch_exit_if_unmatched_db_uuid (notmuch); + indexopts = notmuch_database_get_default_indexopts (notmuch); + if (!indexopts) + return EXIT_FAILURE; + + if (try_decrypt == TRUE || try_decrypt == FALSE) { + status = notmuch_indexopts_set_try_decrypt (indexopts, try_decrypt); + if (status) + fprintf (stderr, "Warning: failed to set --try-decrypt to %d (%s)\n", + try_decrypt, notmuch_status_to_string (status)); + } + +#if (GMIME_MAJOR_VERSION < 3) + if (notmuch_indexopts_get_try_decrypt (indexopts)) { + const char* gpg_path = notmuch_config_get_crypto_gpg_path (config); + if (gpg_path && strcmp(gpg_path, "gpg")) + fprintf (stderr, "Warning: deprecated crypto.gpg_path is set to '%s'\n" + "\tbut ignoring (use $PATH instead)\n", gpg_path); + } +#endif + query_string = query_string_from_args (config, argc-opt_index, argv+opt_index); if (query_string == NULL) { fprintf (stderr, "Out of memory\n"); diff --git a/test/T357-index-decryption.sh b/test/T357-index-decryption.sh index 7bbd81f6..3d18f5af 100755 --- a/test/T357-index-decryption.sh +++ b/test/T357-index-decryption.sh @@ -48,4 +48,69 @@ test_expect_equal \ "$output" \ "$expected" +# add a tag to all messages to ensure that it stays after reindexing +test_begin_subtest 'tagging all messages' +test_expect_success 'notmuch tag +blarney "encrypted message"' +test_begin_subtest "verify that tags have not changed" +output=$(notmuch search tag:blarney) +expected='thread:0000000000000001 2000-01-01 [1/1] Notmuch Test Suite; test encrypted message for cleartext index 001 (blarney encrypted inbox) +thread:0000000000000002 2000-01-01 [1/1] Notmuch Test Suite; test encrypted message for cleartext index 002 (blarney encrypted inbox)' +test_expect_equal \ + "$output" \ + "$expected" + +# see if first message shows up after reindexing with --try-decrypt=true (same $expected, untouched): +test_begin_subtest 'reindex old messages' +test_expect_success 'notmuch reindex --try-decrypt=true tag:encrypted and not property:index-decryption=success' +test_begin_subtest "reindexed encrypted message, including cleartext" +output=$(notmuch search wumpus) +test_expect_equal \ + "$output" \ + "$expected" + +# and the same search, but by property ($expected is untouched): +test_begin_subtest "emacs search by property for both messages" +output=$(notmuch search property:index-decryption=success) +test_expect_equal \ + "$output" \ + "$expected" + + +# try to remove cleartext indexing +test_begin_subtest 'reindex without cleartext' +test_expect_success 'notmuch reindex tag:encrypted and property:index-decryption=success' +test_begin_subtest "reindexed encrypted messages, without cleartext" +output=$(notmuch search wumpus) +expected='' +test_expect_equal \ + "$output" \ + "$expected" + +# and the same search, but by property ($expected is untouched): +test_begin_subtest "emacs search by property with both messages unindexed" +output=$(notmuch search property:index-decryption=success) +test_expect_equal \ + "$output" \ + "$expected" + +# ensure that the tags remain even when we are dropping the cleartext. +test_begin_subtest "verify that tags remain without cleartext" +output=$(notmuch search tag:blarney) +expected='thread:0000000000000001 2000-01-01 [1/1] Notmuch Test Suite; test encrypted message for cleartext index 001 (blarney encrypted inbox) +thread:0000000000000002 2000-01-01 [1/1] Notmuch Test Suite; test encrypted message for cleartext index 002 (blarney encrypted inbox)' +test_expect_equal \ + "$output" \ + "$expected" + + +# TODO: test removal of a message from the message store between +# indexing and reindexing. + +# TODO: insert the same message into the message store twice, index, +# remove one of them from the message store, and then reindex. +# reindexing should return a failure but the message should still be +# present? -- or what should the semantics be if you ask to reindex a +# message whose underlying files have been renamed or moved or +# removed? + test_done -- 2.14.1