From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from localhost (localhost [127.0.0.1]) by arlo.cworth.org (Postfix) with ESMTP id 758846DE0EB0 for ; Mon, 27 May 2019 15:40:37 -0700 (PDT) X-Virus-Scanned: Debian amavisd-new at cworth.org X-Spam-Flag: NO X-Spam-Score: -0.265 X-Spam-Level: X-Spam-Status: No, score=-0.265 tagged_above=-999 required=5 tests=[AWL=-0.064, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=disabled Received: from arlo.cworth.org ([127.0.0.1]) by localhost (arlo.cworth.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id HrG7EDYNSUGN for ; Mon, 27 May 2019 15:40:36 -0700 (PDT) Received: from che.mayfirst.org (che.mayfirst.org [162.247.75.118]) by arlo.cworth.org (Postfix) with ESMTPS id C7E1D6DE0EC6 for ; Mon, 27 May 2019 15:40:35 -0700 (PDT) DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/simple; d=fifthhorseman.net; i=@fifthhorseman.net; q=dns/txt; s=2019; t=1558996833; h=from : to : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : from; bh=IT6FT5DQMZGStYjgqGBti8m8q1fV9CXE9NGHO1j2QBE=; b=vnjtJd2yq1wVO7dQKWME4Qpkirn+1w6yBbhXhHM2TvSrMyPc7jfxSmdn 7nc8Cv5KIjTjYGWoukv/ixyMWXjxDQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=fifthhorseman.net; i=@fifthhorseman.net; q=dns/txt; s=2019rsa; t=1558996833; h=from : to : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : from; bh=IT6FT5DQMZGStYjgqGBti8m8q1fV9CXE9NGHO1j2QBE=; b=QuGh+OT6IYVKHXWZ6DUjzjuLTY4re2LNUmXiyoE5paq9RDMa3joJPadA GLIywlh4SrnDBPKDuVQyHdG800eoqRwJ3mxvVinf1SWw0FQWax59S2V38M oP0hvKbHko609jdcFED9UIYj9k0BTgQyg8prK+bmdO8xEA6YNFuDOxbOan K9nVtupOceas1iUhnk/WixuMXiVW2ih0nUy4b93jwpSgec7WS0EuRE9Idx OgmP413NoV18lgFPKm/t9HswtcqS5YuzkvNa7OZZzOMbfENCNVT8wQvYjE 0XjVfIzsbnvR7e6sO/0rPIm0PViOVUBJZVb3SaPRIb0lRVCLSBaNwg== Received: from fifthhorseman.net (cpe-74-71-53-242.nyc.res.rr.com [74.71.53.242]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by che.mayfirst.org (Postfix) with ESMTPSA id 42612F99E for ; Mon, 27 May 2019 18:40:33 -0400 (EDT) Received: by fifthhorseman.net (Postfix, from userid 1000) id 5B2B820312; Mon, 27 May 2019 18:40:28 -0400 (EDT) From: Daniel Kahn Gillmor To: Notmuch Mail Subject: [PATCH v4 10/17] indexing: record protected subject when indexing cleartext Date: Mon, 27 May 2019 18:40:28 -0400 Message-Id: <20190527224028.24858-1-dkg@fifthhorseman.net> X-Mailer: git-send-email 2.20.1 In-Reply-To: <87y32sjity.fsf@tethera.net> References: <87y32sjity.fsf@tethera.net> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: notmuch@notmuchmail.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 May 2019 22:40:37 -0000 When indexing the cleartext of an encrypted message, record any protected subject in the database, which should make it findable and visible in search. Signed-off-by: Daniel Kahn Gillmor --- lib/index.cc | 42 ++++++++++++++++++++++++++-------- lib/message.cc | 8 +++++++ lib/notmuch-private.h | 4 ++++ test/T356-protected-headers.sh | 16 +++++++++++++ 4 files changed, 61 insertions(+), 9 deletions(-) diff --git a/lib/index.cc b/lib/index.cc index f216ae5d..1fd9e67e 100644 --- a/lib/index.cc +++ b/lib/index.cc @@ -367,13 +367,15 @@ _index_content_type (notmuch_message_t *message, GMimeObject *part) static void _index_encrypted_mime_part (notmuch_message_t *message, notmuch_indexopts_t *indexopts, - GMimeMultipartEncrypted *part); + GMimeMultipartEncrypted *part, + _notmuch_message_crypto_t *msg_crypto); /* Callback to generate terms for each mime part of a message. */ static void _index_mime_part (notmuch_message_t *message, notmuch_indexopts_t *indexopts, - GMimeObject *part) + GMimeObject *part, + _notmuch_message_crypto_t *msg_crypto) { GMimeStream *stream, *filter; GMimeFilter *discard_non_term_filter; @@ -403,6 +405,8 @@ _index_mime_part (notmuch_message_t *message, _notmuch_message_add_term (message, "tag", "encrypted"); for (i = 0; i < g_mime_multipart_get_count (multipart); i++) { + notmuch_status_t status; + GMimeObject *child; if (GMIME_IS_MULTIPART_SIGNED (multipart)) { /* Don't index the signature, but index its content type. */ if (i == GMIME_MULTIPART_SIGNED_SIGNATURE) { @@ -419,7 +423,8 @@ _index_mime_part (notmuch_message_t *message, g_mime_multipart_get_part (multipart, i)); if (i == GMIME_MULTIPART_ENCRYPTED_CONTENT) { _index_encrypted_mime_part(message, indexopts, - GMIME_MULTIPART_ENCRYPTED (part)); + GMIME_MULTIPART_ENCRYPTED (part), + msg_crypto); } else { if (i != GMIME_MULTIPART_ENCRYPTED_VERSION) { _notmuch_database_log (notmuch_message_get_database (message), @@ -428,8 +433,13 @@ _index_mime_part (notmuch_message_t *message, } continue; } - _index_mime_part (message, indexopts, - g_mime_multipart_get_part (multipart, i)); + child = g_mime_multipart_get_part (multipart, i); + status = _notmuch_message_crypto_potential_payload (msg_crypto, child, part, i); + if (status) + _notmuch_database_log (notmuch_message_get_database (message), + "Warning: failed to mark the potential cryptographic payload (%s).\n", + notmuch_status_to_string (status)); + _index_mime_part (message, indexopts, child, msg_crypto); } return; } @@ -439,7 +449,7 @@ _index_mime_part (notmuch_message_t *message, mime_message = g_mime_message_part_get_message (GMIME_MESSAGE_PART (part)); - _index_mime_part (message, indexopts, g_mime_message_get_mime_part (mime_message)); + _index_mime_part (message, indexopts, g_mime_message_get_mime_part (mime_message), msg_crypto); return; } @@ -516,7 +526,8 @@ _index_mime_part (notmuch_message_t *message, static void _index_encrypted_mime_part (notmuch_message_t *message, notmuch_indexopts_t *indexopts, - GMimeMultipartEncrypted *encrypted_data) + GMimeMultipartEncrypted *encrypted_data, + _notmuch_message_crypto_t *msg_crypto) { notmuch_status_t status; GError *err = NULL; @@ -553,6 +564,10 @@ _index_encrypted_mime_part (notmuch_message_t *message, return; } if (decrypt_result) { + status = _notmuch_message_crypto_successful_decryption (msg_crypto); + if (status) + _notmuch_database_log_append (notmuch, "failed to mark the message as decrypted (%s)\n", + notmuch_status_to_string (status)); if (get_sk) { status = notmuch_message_add_property (message, "session-key", g_mime_decrypt_result_get_session_key (decrypt_result)); @@ -562,7 +577,8 @@ _index_encrypted_mime_part (notmuch_message_t *message, } g_object_unref (decrypt_result); } - _index_mime_part (message, indexopts, clear); + status = _notmuch_message_crypto_potential_payload (msg_crypto, clear, GMIME_OBJECT (encrypted_data), GMIME_MULTIPART_ENCRYPTED_CONTENT); + _index_mime_part (message, indexopts, clear, msg_crypto); g_object_unref (clear); status = notmuch_message_add_property (message, "index.decryption", "success"); @@ -606,6 +622,7 @@ _notmuch_message_index_file (notmuch_message_t *message, InternetAddressList *addresses; const char *subject; notmuch_status_t status; + _notmuch_message_crypto_t *msg_crypto; status = _notmuch_message_file_get_mime_message (message_file, &mime_message); @@ -628,7 +645,14 @@ _notmuch_message_index_file (notmuch_message_t *message, status = _notmuch_message_index_user_headers (message, mime_message); - _index_mime_part (message, indexopts, g_mime_message_get_mime_part (mime_message)); + msg_crypto = _notmuch_message_crypto_new (NULL); + _index_mime_part (message, indexopts, g_mime_message_get_mime_part (mime_message), msg_crypto); + if (msg_crypto && msg_crypto->payload_subject) { + _notmuch_message_gen_terms (message, "subject", msg_crypto->payload_subject); + _notmuch_message_update_subject (message, msg_crypto->payload_subject); + } + + talloc_free (msg_crypto); return NOTMUCH_STATUS_SUCCESS; } diff --git a/lib/message.cc b/lib/message.cc index dc4a96ad..9e1005a3 100644 --- a/lib/message.cc +++ b/lib/message.cc @@ -1238,6 +1238,14 @@ _notmuch_message_set_header_values (notmuch_message_t *message, message->modified = true; } +void +_notmuch_message_update_subject (notmuch_message_t *message, + const char *subject) +{ + message->doc.add_value (NOTMUCH_VALUE_SUBJECT, subject); + message->modified = true; +} + /* Upgrade a message to support NOTMUCH_FEATURE_LAST_MOD. The caller * must call _notmuch_message_sync. */ void diff --git a/lib/notmuch-private.h b/lib/notmuch-private.h index e46df9a8..6fc5b366 100644 --- a/lib/notmuch-private.h +++ b/lib/notmuch-private.h @@ -325,6 +325,10 @@ _notmuch_message_set_header_values (notmuch_message_t *message, const char *from, const char *subject); +void +_notmuch_message_update_subject (notmuch_message_t *message, + const char *subject); + void _notmuch_message_upgrade_last_mod (notmuch_message_t *message); diff --git a/test/T356-protected-headers.sh b/test/T356-protected-headers.sh index ff37f6bd..fee3b043 100755 --- a/test/T356-protected-headers.sh +++ b/test/T356-protected-headers.sh @@ -83,4 +83,20 @@ test_json_nodes <<<"$output" \ 'subject:["original"]["headers"]["Subject"]="This is a protected header"' \ 'reply-subject:["reply-headers"]["Subject"]="Re: Subject Unavailable"' +test_begin_subtest "protected subject is not indexed by default" +output=$(notmuch search --output=messages 'subject:"This is a protected header"') +test_expect_equal "$output" '' + +test_begin_subtest "reindex message with protected header" +test_expect_success 'notmuch reindex --decrypt=true id:protected-header@crypto.notmuchmail.org' + +test_begin_subtest "protected subject is indexed when cleartext is indexed" +output=$(notmuch search --output=messages 'subject:"This is a protected header"') +test_expect_equal "$output" 'id:protected-header@crypto.notmuchmail.org' + +test_begin_subtest "indexed protected subject is visible in search" +output=$(notmuch search --format=json 'id:protected-header@crypto.notmuchmail.org') +test_json_nodes <<<"$output" \ + 'subject:[0]["subject"]="This is a protected header"' + test_done -- 2.20.1