From: Eric Wong <e@80x24.org>
To: meta@public-inbox.org
Subject: [PATCH] nntp: filter out duplicate Message-IDs for leafnode
Date: Fri, 14 Jun 2019 00:53:28 +0000 [thread overview]
Message-ID: <20190614005328.2474-1-e@80x24.org> (raw)
It's the unfortunate reality that there are some clients which
reuse Message-IDs (in which we generate + use another) or set
multiple Message-IDs on their own. While the v2 format
addresses that, NNTP clients such as leafnode are not always
prepared to deal with that case.
So, ensure NNTP clients only see a single Message-ID, and
show the others as 'X-Alt-Message-ID'.
---
lib/PublicInbox/NNTP.pm | 16 ++++++++++++++++
t/nntpd.t | 17 +++++++++++++++++
2 files changed, 33 insertions(+)
diff --git a/lib/PublicInbox/NNTP.pm b/lib/PublicInbox/NNTP.pm
index d409e78..8a31b91 100644
--- a/lib/PublicInbox/NNTP.pm
+++ b/lib/PublicInbox/NNTP.pm
@@ -438,6 +438,22 @@ sub set_nntp_headers ($$$$$) {
# reason. We'll fake the shortest one possible.
$hdr->header_set('Path', 'y');
+ # leafnode (and maybe other NNTP clients) have trouble dealing
+ # with v2 messages which have multiple Message-IDs (either due
+ # to our own content-based dedupe or buggy git-send-email versions).
+ my @mids = $hdr->header('Message-ID');
+ if (scalar(@mids) > 1) {
+ my $mid0 = "<$mid>";
+ $hdr->header_set('Message-ID', $mid0);
+ my @alt = $hdr->header('X-Alt-Message-ID');
+ my %seen = map { $_ => 1 } (@alt, $mid0);
+ foreach my $m (@mids) {
+ next if $seen{$m}++;
+ push @alt, $m;
+ }
+ $hdr->header_set('X-Alt-Message-ID', @alt);
+ }
+
# clobber some
my $xref = xref($self, $ng, $n, $mid);
$hdr->header_set('Xref', $xref);
diff --git a/t/nntpd.t b/t/nntpd.t
index cce21ee..a95fb6f 100644
--- a/t/nntpd.t
+++ b/t/nntpd.t
@@ -250,6 +250,23 @@ Date: Fri, 02 Oct 1993 00:00:00 +0000
my $expect = qr/\AMessage-ID: /i . qr/\Q<$long_hdr>\E/;
ok(scalar(grep(/$expect/, @$hdr)), 'Message-ID not folded');
ok(scalar(grep(/^Path:/, @$hdr)), 'Path: header found');
+
+ # it's possible for v2 messages to have 2+ Message-IDs,
+ # but leafnode can't handle it
+ if ($version != 1) {
+ my @mids = ("<$long_hdr>", '<2mid@wtf>');
+ $for_leafnode->header_set('Message-ID', @mids);
+ $for_leafnode->body_set('not-a-dupe');
+ my $warn = '';
+ $SIG{__WARN__} = sub { $warn .= join('', @_) };
+ $im->add($for_leafnode);
+ $im->done;
+ like($warn, qr/reused/, 'warned for reused MID');
+ $hdr = $n->head('<2mid@wtf>');
+ my @hmids = grep(/\AMessage-ID: /i, @$hdr);
+ is(scalar(@hmids), 1, 'Single Message-ID in header');
+ like($hmids[0], qr/: <2mid\@wtf>/, 'got expected mid');
+ }
}
# pipelined requests:
--
EW
reply other threads:[~2019-06-14 0:53 UTC|newest]
Thread overview: [no followups] expand[flat|nested] mbox.gz Atom feed
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://public-inbox.org/README
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20190614005328.2474-1-e@80x24.org \
--to=e@80x24.org \
--cc=meta@public-inbox.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).