From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from localhost (localhost [127.0.0.1]) by arlo.cworth.org (Postfix) with ESMTP id 5F3076DE0EC5 for ; Mon, 27 May 2019 14:26:01 -0700 (PDT) X-Virus-Scanned: Debian amavisd-new at cworth.org X-Spam-Flag: NO X-Spam-Score: -0.249 X-Spam-Level: X-Spam-Status: No, score=-0.249 tagged_above=-999 required=5 tests=[AWL=-0.048, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=disabled Received: from arlo.cworth.org ([127.0.0.1]) by localhost (arlo.cworth.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gEVOZu1LSc7l for ; Mon, 27 May 2019 14:25:59 -0700 (PDT) Received: from che.mayfirst.org (che.mayfirst.org [162.247.75.118]) by arlo.cworth.org (Postfix) with ESMTPS id 092636DE0C7A for ; Mon, 27 May 2019 14:25:58 -0700 (PDT) DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/simple; d=fifthhorseman.net; i=@fifthhorseman.net; q=dns/txt; s=2019; t=1558992358; h=from : to : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : from; bh=SFGAUl6lQcU9cCkzxh3vfR/iabMQ3q0dcI4J88m+Pig=; b=In31S3GX8QvcG/JKwlgoQin9bLRseHOFy6IwbHoCzvdjCVsQ0nfg/Wwg GRN2yW30PztqBOWqPCf2F08dq6iGAw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=fifthhorseman.net; i=@fifthhorseman.net; q=dns/txt; s=2019rsa; t=1558992357; h=from : to : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : from; bh=SFGAUl6lQcU9cCkzxh3vfR/iabMQ3q0dcI4J88m+Pig=; b=u/sDun6QsLjY1suQzwtJ2p/jkC4mrlJpXscZvE3uR2VNkKBhwn20/Uk/ QE7aubyE/e+TM7s5rn7OwUdaquTl9QYxTlL9NnN775Y1AVIZaVxVdkbHdJ GW0xXuCgewmRtnt+1H6uoCDO8u/r3WqrEnn3/Uz8SP088ujCutXsJg8eKZ ffMXa038ofOlvGVEDHFdIqjEusoNYL9f4Udhkre/65dovxia2aOfhbEHbM zelvdyTrKCyNospLRfDLSULKNGlV1KJcZ0zNoHufmdp/WRvZXY/sPMvb8/ OjszLkgTdqwrlzLXWDOweb1CocrS6TtJ7U4yD8whkws450w7mQzqRA== Received: from fifthhorseman.net (cpe-74-71-53-242.nyc.res.rr.com [74.71.53.242]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by che.mayfirst.org (Postfix) with ESMTPSA id D8D70F99D for ; Mon, 27 May 2019 17:25:57 -0400 (EDT) Received: by fifthhorseman.net (Postfix, from userid 1000) id 4821C2024B; Mon, 27 May 2019 17:17:20 -0400 (EDT) From: Daniel Kahn Gillmor To: Notmuch Mail Subject: [PATCH v3 10/17] indexing: record protected subject when indexing cleartext Date: Mon, 27 May 2019 17:17:20 -0400 Message-Id: <20190527211720.6745-1-dkg@fifthhorseman.net> X-Mailer: git-send-email 2.20.1 In-Reply-To: <87y32sjity.fsf@tethera.net> References: <87y32sjity.fsf@tethera.net> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: notmuch@notmuchmail.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 May 2019 21:26:01 -0000 When indexing the cleartext of an encrypted message, record any protected subject in the database, which should make it findable and visible in search. Signed-off-by: Daniel Kahn Gillmor --- lib/index.cc | 46 +++++++++++++++++++++++++++------- lib/message.cc | 8 ++++++ lib/notmuch-private.h | 4 +++ test/T356-protected-headers.sh | 16 ++++++++++++ 4 files changed, 65 insertions(+), 9 deletions(-) diff --git a/lib/index.cc b/lib/index.cc index f216ae5d..2e0bacb1 100644 --- a/lib/index.cc +++ b/lib/index.cc @@ -367,13 +367,15 @@ _index_content_type (notmuch_message_t *message, GMimeObject *part) static void _index_encrypted_mime_part (notmuch_message_t *message, notmuch_indexopts_t *indexopts, - GMimeMultipartEncrypted *part); + GMimeMultipartEncrypted *part, + _notmuch_message_crypto_t *msg_crypto); /* Callback to generate terms for each mime part of a message. */ static void _index_mime_part (notmuch_message_t *message, notmuch_indexopts_t *indexopts, - GMimeObject *part) + GMimeObject *part, + _notmuch_message_crypto_t *msg_crypto) { GMimeStream *stream, *filter; GMimeFilter *discard_non_term_filter; @@ -403,6 +405,8 @@ _index_mime_part (notmuch_message_t *message, _notmuch_message_add_term (message, "tag", "encrypted"); for (i = 0; i < g_mime_multipart_get_count (multipart); i++) { + notmuch_status_t status; + GMimeObject *child; if (GMIME_IS_MULTIPART_SIGNED (multipart)) { /* Don't index the signature, but index its content type. */ if (i == GMIME_MULTIPART_SIGNED_SIGNATURE) { @@ -419,7 +423,8 @@ _index_mime_part (notmuch_message_t *message, g_mime_multipart_get_part (multipart, i)); if (i == GMIME_MULTIPART_ENCRYPTED_CONTENT) { _index_encrypted_mime_part(message, indexopts, - GMIME_MULTIPART_ENCRYPTED (part)); + GMIME_MULTIPART_ENCRYPTED (part), + msg_crypto); } else { if (i != GMIME_MULTIPART_ENCRYPTED_VERSION) { _notmuch_database_log (notmuch_message_get_database (message), @@ -428,8 +433,13 @@ _index_mime_part (notmuch_message_t *message, } continue; } - _index_mime_part (message, indexopts, - g_mime_multipart_get_part (multipart, i)); + child = g_mime_multipart_get_part (multipart, i); + status = _notmuch_message_crypto_potential_payload (msg_crypto, child, part, i); + if (status) + _notmuch_database_log (notmuch_message_get_database (message), + "Warning: failed to mark the potential cryptographic payload (%s).\n", + notmuch_status_to_string (status)); + _index_mime_part (message, indexopts, child, msg_crypto); } return; } @@ -439,7 +449,7 @@ _index_mime_part (notmuch_message_t *message, mime_message = g_mime_message_part_get_message (GMIME_MESSAGE_PART (part)); - _index_mime_part (message, indexopts, g_mime_message_get_mime_part (mime_message)); + _index_mime_part (message, indexopts, g_mime_message_get_mime_part (mime_message), msg_crypto); return; } @@ -516,7 +526,8 @@ _index_mime_part (notmuch_message_t *message, static void _index_encrypted_mime_part (notmuch_message_t *message, notmuch_indexopts_t *indexopts, - GMimeMultipartEncrypted *encrypted_data) + GMimeMultipartEncrypted *encrypted_data, + _notmuch_message_crypto_t *msg_crypto) { notmuch_status_t status; GError *err = NULL; @@ -553,6 +564,10 @@ _index_encrypted_mime_part (notmuch_message_t *message, return; } if (decrypt_result) { + status = _notmuch_message_crypto_successful_decryption (msg_crypto); + if (status) + _notmuch_database_log_append (notmuch, "failed to mark the message as decrypted (%s)\n", + notmuch_status_to_string (status)); if (get_sk) { status = notmuch_message_add_property (message, "session-key", g_mime_decrypt_result_get_session_key (decrypt_result)); @@ -562,7 +577,12 @@ _index_encrypted_mime_part (notmuch_message_t *message, } g_object_unref (decrypt_result); } - _index_mime_part (message, indexopts, clear); + status = _notmuch_message_crypto_potential_payload (msg_crypto, clear, GMIME_OBJECT (encrypted_data), GMIME_MULTIPART_ENCRYPTED_CONTENT); + if (status) + _notmuch_database_log_append (notmuch, + "failed to mark the potential cryptographic payload (%s).\n", + notmuch_status_to_string (status)); + _index_mime_part (message, indexopts, clear, msg_crypto); g_object_unref (clear); status = notmuch_message_add_property (message, "index.decryption", "success"); @@ -606,6 +626,7 @@ _notmuch_message_index_file (notmuch_message_t *message, InternetAddressList *addresses; const char *subject; notmuch_status_t status; + _notmuch_message_crypto_t *msg_crypto; status = _notmuch_message_file_get_mime_message (message_file, &mime_message); @@ -628,7 +649,14 @@ _notmuch_message_index_file (notmuch_message_t *message, status = _notmuch_message_index_user_headers (message, mime_message); - _index_mime_part (message, indexopts, g_mime_message_get_mime_part (mime_message)); + msg_crypto = _notmuch_message_crypto_new (NULL); + _index_mime_part (message, indexopts, g_mime_message_get_mime_part (mime_message), msg_crypto); + if (msg_crypto && msg_crypto->payload_subject) { + _notmuch_message_gen_terms (message, "subject", msg_crypto->payload_subject); + _notmuch_message_update_subject (message, msg_crypto->payload_subject); + } + + talloc_free (msg_crypto); return NOTMUCH_STATUS_SUCCESS; } diff --git a/lib/message.cc b/lib/message.cc index dc4a96ad..9e1005a3 100644 --- a/lib/message.cc +++ b/lib/message.cc @@ -1238,6 +1238,14 @@ _notmuch_message_set_header_values (notmuch_message_t *message, message->modified = true; } +void +_notmuch_message_update_subject (notmuch_message_t *message, + const char *subject) +{ + message->doc.add_value (NOTMUCH_VALUE_SUBJECT, subject); + message->modified = true; +} + /* Upgrade a message to support NOTMUCH_FEATURE_LAST_MOD. The caller * must call _notmuch_message_sync. */ void diff --git a/lib/notmuch-private.h b/lib/notmuch-private.h index e46df9a8..6fc5b366 100644 --- a/lib/notmuch-private.h +++ b/lib/notmuch-private.h @@ -325,6 +325,10 @@ _notmuch_message_set_header_values (notmuch_message_t *message, const char *from, const char *subject); +void +_notmuch_message_update_subject (notmuch_message_t *message, + const char *subject); + void _notmuch_message_upgrade_last_mod (notmuch_message_t *message); diff --git a/test/T356-protected-headers.sh b/test/T356-protected-headers.sh index 29b0c42a..714c847c 100755 --- a/test/T356-protected-headers.sh +++ b/test/T356-protected-headers.sh @@ -90,4 +90,20 @@ test_json_nodes <<<"$output" \ 'subject:["original"]["headers"]["Subject"]="This is a protected header"' \ 'reply-subject:["reply-headers"]["Subject"]="Re: This is a protected header"' +test_begin_subtest "protected subject is not indexed by default" +output=$(notmuch search --output=messages 'subject:"This is a protected header"') +test_expect_equal "$output" '' + +test_begin_subtest "reindex message with protected header" +test_expect_success 'notmuch reindex --decrypt=true id:protected-header@crypto.notmuchmail.org' + +test_begin_subtest "protected subject is indexed when cleartext is indexed" +output=$(notmuch search --output=messages 'subject:"This is a protected header"') +test_expect_equal "$output" 'id:protected-header@crypto.notmuchmail.org' + +test_begin_subtest "indexed protected subject is visible in search" +output=$(notmuch search --format=json 'id:protected-header@crypto.notmuchmail.org') +test_json_nodes <<<"$output" \ + 'subject:[0]["subject"]="This is a protected header"' + test_done -- 2.20.1