From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from localhost (localhost [127.0.0.1]) by arlo.cworth.org (Postfix) with ESMTP id 574296DE0BB6 for ; Sun, 8 Sep 2019 20:27:38 -0700 (PDT) X-Virus-Scanned: Debian amavisd-new at cworth.org X-Spam-Flag: NO X-Spam-Score: -1.534 X-Spam-Level: X-Spam-Status: No, score=-1.534 tagged_above=-999 required=5 tests=[AWL=0.967, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001] autolearn=disabled Received: from arlo.cworth.org ([127.0.0.1]) by localhost (arlo.cworth.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Eac324Z3-tq7 for ; Sun, 8 Sep 2019 20:27:34 -0700 (PDT) Received: from che.mayfirst.org (che.mayfirst.org [162.247.75.118]) by arlo.cworth.org (Postfix) with ESMTPS id BD9556DE096A for ; Sun, 8 Sep 2019 20:27:34 -0700 (PDT) DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/simple; d=fifthhorseman.net; i=@fifthhorseman.net; q=dns/txt; s=2019; t=1567999653; h=from : to : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : from; bh=owg0lDZwCi5AB+hNz2U5G2nxwz1/wiy/VCk8LHKEhy8=; b=Ah7iBvlQdO3mK5E3BTRtwqRwJKRVyLkyVrqnTmluq3xsUFHONKg233uo lyY48h4PHdqRlGS17PZFaBWmLRnLDA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=fifthhorseman.net; i=@fifthhorseman.net; q=dns/txt; s=2019rsa; t=1567999653; h=from : to : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : from; bh=owg0lDZwCi5AB+hNz2U5G2nxwz1/wiy/VCk8LHKEhy8=; b=F54eRXfDI5QBUuNYe+1+tdqOgHA4cCqSuzSycFk0iMQngCS804pGD656 uIJnwX9DoIribs+LU6XHk+zaZy/sEiY6GYawjruOHe6Mdx+EO94XBwX5nz KbHVazmm3YQV01QIv5o16hWCSqbF2j3Ax9w00Kq4+WZ+vldJU6Km4yX1AK 3X5/pjojdTUneIUCeh3AfWs0OQRX/9cVBaPZJTu4Q5Fir1F4tTwesn6udF 09lOqT4CJjrSv8FDl7cyU+INZmp6viwqqZhvgtChdXzG0oL8jBDaZHPNuW vnTeBbdp+gbyKjVeFs2QVVuYwU66tmfS77K/30hOaT26x3xdwJUGag== Received: from fifthhorseman.net (unknown [IPv6:2001:470:1f07:60d:c41:39ff:fef3:974f]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by che.mayfirst.org (Postfix) with ESMTPSA id 91A08F9A7 for ; Sun, 8 Sep 2019 23:27:33 -0400 (EDT) Received: by fifthhorseman.net (Postfix, from userid 1000) id 10CB6204F4; Sun, 8 Sep 2019 23:27:29 -0400 (EDT) From: Daniel Kahn Gillmor To: Notmuch Mail Subject: [PATCH v4 3/4] index: repair "Mixed Up" messages before indexing. Date: Sun, 8 Sep 2019 23:27:25 -0400 Message-Id: <20190909032726.8931-4-dkg@fifthhorseman.net> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20190909032726.8931-1-dkg@fifthhorseman.net> References: <20190909032726.8931-1-dkg@fifthhorseman.net> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: notmuch@notmuchmail.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Sep 2019 03:27:38 -0000 When encountering a message that has been mangled in the "mixed up" way by an intermediate MTA, notmuch should instead repair it and index the repaired form. When it does this, it also associates the index.repaired=mixedup property with the message. If a problem is found with this repair process, or an improved repair process is proposed later, this should make it easy for people to reindex the relevant message. The property will also hopefully make it easier to diagnose this particular problem in the future. Signed-off-by: Daniel Kahn Gillmor --- doc/man7/notmuch-properties.rst | 6 ++++++ lib/index.cc | 22 +++++++++++++++++----- test/T351-pgpmime-mangling.sh | 2 -- 3 files changed, 23 insertions(+), 7 deletions(-) diff --git a/doc/man7/notmuch-properties.rst b/doc/man7/notmuch-properties.rst index e2db2ef5..a7d91d67 100644 --- a/doc/man7/notmuch-properties.rst +++ b/doc/man7/notmuch-properties.rst @@ -127,6 +127,12 @@ of its normal activity. found in that message, since it was able to index the built-in protected headers directly. + ``index.repaired=mixedup`` indicates the repair of a "Mixed Up" + encrypted PGP/MIME message, a mangling typically produced by + Microsoft's Exchange MTA. See + https://tools.ietf.org/html/draft-dkg-openpgp-pgpmime-message-mangling + for more information. + SEE ALSO ======== diff --git a/lib/index.cc b/lib/index.cc index 1301d78a..158ba5cf 100644 --- a/lib/index.cc +++ b/lib/index.cc @@ -387,11 +387,20 @@ _index_mime_part (notmuch_message_t *message, GMimeContentType *content_type; char *body; const char *charset; + GMimeObject *repaired_part = NULL; if (! part) { _notmuch_database_log (notmuch_message_get_database (message), "Warning: Not indexing empty mime part.\n"); - return; + goto DONE; + } + + repaired_part = _notmuch_repair_mixed_up_mangled (part); + if (repaired_part) { + /* This was likely "Mixed Up" in transit! We will instead use + * the more likely-to-be-correct variant. */ + notmuch_message_add_property (message, "index.repaired", "mixedup"); + part = repaired_part; } _index_content_type (message, part); @@ -444,7 +453,7 @@ _index_mime_part (notmuch_message_t *message, } _index_mime_part (message, indexopts, toindex, msg_crypto); } - return; + goto DONE; } if (GMIME_IS_MESSAGE_PART (part)) { @@ -454,14 +463,14 @@ _index_mime_part (notmuch_message_t *message, _index_mime_part (message, indexopts, g_mime_message_get_mime_part (mime_message), msg_crypto); - return; + goto DONE; } if (! (GMIME_IS_PART (part))) { _notmuch_database_log (notmuch_message_get_database (message), "Warning: Not indexing unknown mime part: %s.\n", g_type_name (G_OBJECT_TYPE (part))); - return; + goto DONE; } disposition = g_mime_object_get_content_disposition (part); @@ -475,7 +484,7 @@ _index_mime_part (notmuch_message_t *message, /* XXX: Would be nice to call out to something here to parse * the attachment into text and then index that. */ - return; + goto DONE; } byte_array = g_byte_array_new (); @@ -521,6 +530,9 @@ _index_mime_part (notmuch_message_t *message, free (body); } + DONE: + if (repaired_part) + g_object_unref (repaired_part); } /* descend (if desired) into the cleartext part of an encrypted MIME diff --git a/test/T351-pgpmime-mangling.sh b/test/T351-pgpmime-mangling.sh index f65b8a24..4555f937 100755 --- a/test/T351-pgpmime-mangling.sh +++ b/test/T351-pgpmime-mangling.sh @@ -21,7 +21,6 @@ test_json_nodes <<<"$output" \ 'body:["original"]'"$bodytext" test_begin_subtest "repaired 'Mixed-up' messages can be found with index.repaired=mixedup" -test_subtest_known_broken output=$(notmuch search --output=messages property:index.repaired=mixedup) test_expect_equal "$output" id:mixed-up@mangling.notmuchmail.org @@ -29,7 +28,6 @@ test_begin_subtest "index cleartext of 'Mixed-Up' mangled PGP/MIME message" test_expect_success 'notmuch reindex --decrypt=true id:mixed-up@mangling.notmuchmail.org' test_begin_subtest "search cleartext of 'Mixed-Up' mangled PGP/MIME message" -test_subtest_known_broken output=$(notmuch search --output=messages body:password) test_expect_equal "$output" id:mixed-up@mangling.notmuchmail.org -- 2.23.0