From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from localhost (localhost [127.0.0.1]) by arlo.cworth.org (Postfix) with ESMTP id EE0B26DE145F for ; Sun, 26 May 2019 15:16:28 -0700 (PDT) X-Virus-Scanned: Debian amavisd-new at cworth.org X-Spam-Flag: NO X-Spam-Score: -0.177 X-Spam-Level: X-Spam-Status: No, score=-0.177 tagged_above=-999 required=5 tests=[AWL=0.024, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=disabled Received: from arlo.cworth.org ([127.0.0.1]) by localhost (arlo.cworth.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id GTb43-VYQIYQ for ; Sun, 26 May 2019 15:16:25 -0700 (PDT) Received: from che.mayfirst.org (che.mayfirst.org [162.247.75.118]) by arlo.cworth.org (Postfix) with ESMTPS id 510296DE13F0 for ; Sun, 26 May 2019 15:16:20 -0700 (PDT) DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/simple; d=fifthhorseman.net; i=@fifthhorseman.net; q=dns/txt; s=2019; t=1558908978; h=from : to : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : from; bh=IT6FT5DQMZGStYjgqGBti8m8q1fV9CXE9NGHO1j2QBE=; b=mnSupPypf6VDf49wXnNRi08KUSPO8PXf5Olcc1r7JlIDQosavxDsnTsy e0E4Ljyb3B3TqjTAt/O7OYTAAQwpAw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=fifthhorseman.net; i=@fifthhorseman.net; q=dns/txt; s=2019rsa; t=1558908978; h=from : to : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : from; bh=IT6FT5DQMZGStYjgqGBti8m8q1fV9CXE9NGHO1j2QBE=; b=m7nObd029kx9ZJ0AHcUCPE6cK0/dfd6HL9XCWWdkGvHdfGz6jXKIYpAz myHf0W+t5VyPPaTfU7/iAbjyljNfXbo1DqjsZmz0MKvWuOz8T+FCd5OSPS /UHO138DIAroEG+gXKsLRA7Oo3hAidedwe7zoBOYDLjinCXbHCmOTIvCAg QW/MxLSixk6e3k2cvE/n9zU7jGoO5FVMpKXJ68+jpFs13Gnh/w+cN3ziL2 MWO0LsPtg4L+iNmuJfAeTUjL15ehVl9bEUBUDQP09Bankjp/QrlXN5lN2M wY7gJfuCtBav8q9qekfkxFtrfW3JbBAQD6zyO/FqSu7MWXA9iMQpag== Received: from fifthhorseman.net (cpe-74-71-53-242.nyc.res.rr.com [74.71.53.242]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by che.mayfirst.org (Postfix) with ESMTPSA id 066B5F9A4 for ; Sun, 26 May 2019 18:16:17 -0400 (EDT) Received: by fifthhorseman.net (Postfix, from userid 1000) id 7281821101; Sun, 26 May 2019 18:16:14 -0400 (EDT) From: Daniel Kahn Gillmor To: Notmuch Mail Subject: [PATCH v2 10/17] indexing: record protected subject when indexing cleartext Date: Sun, 26 May 2019 18:16:03 -0400 Message-Id: <20190526221610.2833-11-dkg@fifthhorseman.net> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190526221610.2833-1-dkg@fifthhorseman.net> References: <20190526221610.2833-1-dkg@fifthhorseman.net> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: notmuch@notmuchmail.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 26 May 2019 22:16:29 -0000 When indexing the cleartext of an encrypted message, record any protected subject in the database, which should make it findable and visible in search. Signed-off-by: Daniel Kahn Gillmor --- lib/index.cc | 42 ++++++++++++++++++++++++++-------- lib/message.cc | 8 +++++++ lib/notmuch-private.h | 4 ++++ test/T356-protected-headers.sh | 16 +++++++++++++ 4 files changed, 61 insertions(+), 9 deletions(-) diff --git a/lib/index.cc b/lib/index.cc index f216ae5d..1fd9e67e 100644 --- a/lib/index.cc +++ b/lib/index.cc @@ -367,13 +367,15 @@ _index_content_type (notmuch_message_t *message, GMimeObject *part) static void _index_encrypted_mime_part (notmuch_message_t *message, notmuch_indexopts_t *indexopts, - GMimeMultipartEncrypted *part); + GMimeMultipartEncrypted *part, + _notmuch_message_crypto_t *msg_crypto); /* Callback to generate terms for each mime part of a message. */ static void _index_mime_part (notmuch_message_t *message, notmuch_indexopts_t *indexopts, - GMimeObject *part) + GMimeObject *part, + _notmuch_message_crypto_t *msg_crypto) { GMimeStream *stream, *filter; GMimeFilter *discard_non_term_filter; @@ -403,6 +405,8 @@ _index_mime_part (notmuch_message_t *message, _notmuch_message_add_term (message, "tag", "encrypted"); for (i = 0; i < g_mime_multipart_get_count (multipart); i++) { + notmuch_status_t status; + GMimeObject *child; if (GMIME_IS_MULTIPART_SIGNED (multipart)) { /* Don't index the signature, but index its content type. */ if (i == GMIME_MULTIPART_SIGNED_SIGNATURE) { @@ -419,7 +423,8 @@ _index_mime_part (notmuch_message_t *message, g_mime_multipart_get_part (multipart, i)); if (i == GMIME_MULTIPART_ENCRYPTED_CONTENT) { _index_encrypted_mime_part(message, indexopts, - GMIME_MULTIPART_ENCRYPTED (part)); + GMIME_MULTIPART_ENCRYPTED (part), + msg_crypto); } else { if (i != GMIME_MULTIPART_ENCRYPTED_VERSION) { _notmuch_database_log (notmuch_message_get_database (message), @@ -428,8 +433,13 @@ _index_mime_part (notmuch_message_t *message, } continue; } - _index_mime_part (message, indexopts, - g_mime_multipart_get_part (multipart, i)); + child = g_mime_multipart_get_part (multipart, i); + status = _notmuch_message_crypto_potential_payload (msg_crypto, child, part, i); + if (status) + _notmuch_database_log (notmuch_message_get_database (message), + "Warning: failed to mark the potential cryptographic payload (%s).\n", + notmuch_status_to_string (status)); + _index_mime_part (message, indexopts, child, msg_crypto); } return; } @@ -439,7 +449,7 @@ _index_mime_part (notmuch_message_t *message, mime_message = g_mime_message_part_get_message (GMIME_MESSAGE_PART (part)); - _index_mime_part (message, indexopts, g_mime_message_get_mime_part (mime_message)); + _index_mime_part (message, indexopts, g_mime_message_get_mime_part (mime_message), msg_crypto); return; } @@ -516,7 +526,8 @@ _index_mime_part (notmuch_message_t *message, static void _index_encrypted_mime_part (notmuch_message_t *message, notmuch_indexopts_t *indexopts, - GMimeMultipartEncrypted *encrypted_data) + GMimeMultipartEncrypted *encrypted_data, + _notmuch_message_crypto_t *msg_crypto) { notmuch_status_t status; GError *err = NULL; @@ -553,6 +564,10 @@ _index_encrypted_mime_part (notmuch_message_t *message, return; } if (decrypt_result) { + status = _notmuch_message_crypto_successful_decryption (msg_crypto); + if (status) + _notmuch_database_log_append (notmuch, "failed to mark the message as decrypted (%s)\n", + notmuch_status_to_string (status)); if (get_sk) { status = notmuch_message_add_property (message, "session-key", g_mime_decrypt_result_get_session_key (decrypt_result)); @@ -562,7 +577,8 @@ _index_encrypted_mime_part (notmuch_message_t *message, } g_object_unref (decrypt_result); } - _index_mime_part (message, indexopts, clear); + status = _notmuch_message_crypto_potential_payload (msg_crypto, clear, GMIME_OBJECT (encrypted_data), GMIME_MULTIPART_ENCRYPTED_CONTENT); + _index_mime_part (message, indexopts, clear, msg_crypto); g_object_unref (clear); status = notmuch_message_add_property (message, "index.decryption", "success"); @@ -606,6 +622,7 @@ _notmuch_message_index_file (notmuch_message_t *message, InternetAddressList *addresses; const char *subject; notmuch_status_t status; + _notmuch_message_crypto_t *msg_crypto; status = _notmuch_message_file_get_mime_message (message_file, &mime_message); @@ -628,7 +645,14 @@ _notmuch_message_index_file (notmuch_message_t *message, status = _notmuch_message_index_user_headers (message, mime_message); - _index_mime_part (message, indexopts, g_mime_message_get_mime_part (mime_message)); + msg_crypto = _notmuch_message_crypto_new (NULL); + _index_mime_part (message, indexopts, g_mime_message_get_mime_part (mime_message), msg_crypto); + if (msg_crypto && msg_crypto->payload_subject) { + _notmuch_message_gen_terms (message, "subject", msg_crypto->payload_subject); + _notmuch_message_update_subject (message, msg_crypto->payload_subject); + } + + talloc_free (msg_crypto); return NOTMUCH_STATUS_SUCCESS; } diff --git a/lib/message.cc b/lib/message.cc index dc4a96ad..9e1005a3 100644 --- a/lib/message.cc +++ b/lib/message.cc @@ -1238,6 +1238,14 @@ _notmuch_message_set_header_values (notmuch_message_t *message, message->modified = true; } +void +_notmuch_message_update_subject (notmuch_message_t *message, + const char *subject) +{ + message->doc.add_value (NOTMUCH_VALUE_SUBJECT, subject); + message->modified = true; +} + /* Upgrade a message to support NOTMUCH_FEATURE_LAST_MOD. The caller * must call _notmuch_message_sync. */ void diff --git a/lib/notmuch-private.h b/lib/notmuch-private.h index e46df9a8..6fc5b366 100644 --- a/lib/notmuch-private.h +++ b/lib/notmuch-private.h @@ -325,6 +325,10 @@ _notmuch_message_set_header_values (notmuch_message_t *message, const char *from, const char *subject); +void +_notmuch_message_update_subject (notmuch_message_t *message, + const char *subject); + void _notmuch_message_upgrade_last_mod (notmuch_message_t *message); diff --git a/test/T356-protected-headers.sh b/test/T356-protected-headers.sh index ff37f6bd..fee3b043 100755 --- a/test/T356-protected-headers.sh +++ b/test/T356-protected-headers.sh @@ -83,4 +83,20 @@ test_json_nodes <<<"$output" \ 'subject:["original"]["headers"]["Subject"]="This is a protected header"' \ 'reply-subject:["reply-headers"]["Subject"]="Re: Subject Unavailable"' +test_begin_subtest "protected subject is not indexed by default" +output=$(notmuch search --output=messages 'subject:"This is a protected header"') +test_expect_equal "$output" '' + +test_begin_subtest "reindex message with protected header" +test_expect_success 'notmuch reindex --decrypt=true id:protected-header@crypto.notmuchmail.org' + +test_begin_subtest "protected subject is indexed when cleartext is indexed" +output=$(notmuch search --output=messages 'subject:"This is a protected header"') +test_expect_equal "$output" 'id:protected-header@crypto.notmuchmail.org' + +test_begin_subtest "indexed protected subject is visible in search" +output=$(notmuch search --format=json 'id:protected-header@crypto.notmuchmail.org') +test_json_nodes <<<"$output" \ + 'subject:[0]["subject"]="This is a protected header"' + test_done -- 2.20.1