[PATCH 0/3] mda cleanups for idiotic mail policies

[PATCH 0/3] mda cleanups for idiotic mail policies

[Eric Wong]

Hopefully these will make things easier for folks on mail hosts
like Yahoo which implement horrible DMARC policies to prevent
mailing lists from working properly.

ref: http://mid.gmane.org/20150911225542.GA1225@survex.com

Eric Wong (3):
      mda: keep DKIM crap
      filter: do not strip Reply-To
      filter: more consistent labeling of rejections

[PATCH 1/3] mda: keep DKIM crap

[Eric Wong]

This is probably unsafe
---
 lib/PublicInbox/MDA.pm | 8 --------
 1 file changed, 8 deletions(-)

diff --git a/lib/PublicInbox/MDA.pm b/lib/PublicInbox/MDA.pm
index 8c2d6ed..25d6ae5 100644
--- a/lib/PublicInbox/MDA.pm
+++ b/lib/PublicInbox/MDA.pm
@@ -70,14 +70,6 @@ sub set_list_headers {
 			x-confirm-reading-to x-pmrqc)) {
 		$simple->header_set($h);
 	}
-
-	# Remove any "DomainKeys" (or similar) header lines.
-	# Any modifications (including List-Id) will cause a message
-	# to appear invalid
-	foreach my $h (qw(domainkey-signature dkim-signature
-			authentication-results)) {
-		$simple->header_set($h);
-	}
 }
 
 # returns a 3-element array: name, email, date
-- 
EW

[PATCH 2/3] filter: do not strip Reply-To

[Eric Wong]

It's often part of idiotic policies to prevent mailing lists
from working at all.
---
 lib/PublicInbox/Filter.pm | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/lib/PublicInbox/Filter.pm b/lib/PublicInbox/Filter.pm
index cab6aad..6f28e01 100644
--- a/lib/PublicInbox/Filter.pm
+++ b/lib/PublicInbox/Filter.pm
@@ -29,10 +29,10 @@ sub run {
 	# kill potentially bad/confusing headers
 	# Note: ssoma already does this, but since we mangle the message,
 	# we should do this before it gets to ssoma.
-	# We also kill Mail-{Followup,Reply}-To and Reply-To headers due to
+	# We also kill Mail-{Followup,Reply}-To headers due to
 	# the nature of public-inbox having no real subscribers.
 	foreach my $d (qw(status lines content-length
-			mail-followup-to mail-reply-to reply-to)) {
+			mail-followup-to mail-reply-to)) {
 		$mime->header_set($d);
 	}
 
-- 
EW

[PATCH 3/3] filter: more consistent labeling of rejections

[Eric Wong]

While we're at it, reject non-plain-text top-level messages,
too.  They probably do not exist in practice, but we cannot
afford to scrub given policies implemented by overzealous
mail providers.

While we're at it, update the comment for strip_multipart.
---
 lib/PublicInbox/Filter.pm | 11 +++++------
 1 file changed, 5 insertions(+), 6 deletions(-)

diff --git a/lib/PublicInbox/Filter.pm b/lib/PublicInbox/Filter.pm
index 6f28e01..01052d0 100644
--- a/lib/PublicInbox/Filter.pm
+++ b/lib/PublicInbox/Filter.pm
@@ -13,6 +13,7 @@ use Email::Filter;
 use IPC::Run;
 our $VERSION = '0.0.1';
 use constant NO_HTML => '*** We only accept plain-text email, no HTML ***';
+use constant TEXT_ONLY => '*** We only accept plain-text email ***';
 
 # start with the same defaults as mailman
 our $BAD_EXT = qr/\.(exe|bat|cmd|com|pif|scr|vbs|cpl|zip)\s*\z/i;
@@ -49,6 +50,7 @@ sub run {
 	} elsif ($content_type =~ m!\bmultipart/!i) {
 		return strip_multipart($mime, $content_type, $filter);
 	} else {
+		$filter->reject(TEXT_ONLY) if $filter;
 		replace_body($mime, "$content_type message scrubbed");
 		return 0;
 	}
@@ -108,10 +110,7 @@ sub dump_html {
 	}
 }
 
-# this is to correct user errors and not expected to cover all corner cases
-# if users don't want to hit this, they should be sending text/plain messages
-# unfortunately, too many people send HTML mail and we'll attempt to convert
-# it to something safer, smaller and harder-to-spy-on-users-with.
+# this is to correct old archives during import.
 sub strip_multipart {
 	my ($mime, $content_type, $filter) = @_;
 
@@ -152,7 +151,7 @@ sub strip_multipart {
 			if (recheck_type_ok($part)) {
 				push @keep, $part;
 			} elsif ($filter) {
-				$filter->reject('no attachments')
+				$filter->reject(TEXT_ONLY);
 			} else {
 				$rejected++;
 			}
@@ -164,7 +163,7 @@ sub strip_multipart {
 				push @keep, $part;
 			}
 		} else {
-			$filter->reject('no attachments') if $filter;
+			$filter->reject(TEXT_ONLY) if $filter;
 			# reject everything else, including non-PGP signatures
 			$rejected++;
 		}
-- 
EW