From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net X-Spam-Level: X-Spam-Status: No, score=-4.0 required=3.0 tests=ALL_TRUSTED,BAYES_00 shortcircuit=no autolearn=ham autolearn_force=no version=3.4.2 Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id 4BE8F1F4B4; Thu, 10 Dec 2020 20:55:40 +0000 (UTC) Date: Thu, 10 Dec 2020 20:55:40 +0000 From: Eric Wong To: Konstantin Ryabitsev Cc: meta@public-inbox.org Subject: Re: Extra newline when retrieving messages Message-ID: <20201210205540.GA10371@dcvr> References: <20201210202145.7agtcmrtl5jec42d@chatter.i7.local> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <20201210202145.7agtcmrtl5jec42d@chatter.i7.local> List-Id: Konstantin Ryabitsev wrote: > Hello: > > While investigating why some of the messages retrieved via > lore.kernel.org were failing DKIM checks, I realized that > public-inbox-httpd appends an extra newline to message bodies. This > newline isn't present in git backends, just in messages retrieved via > (at least) public-inbox-httpd. It looks like a regression from commit dbdc7a42dd885523 (2016-04-11); which now doesn't make sense, though very little is making sense to me nowadays :< The following should undo it, unless somebody DKIM signed a message which lacked any trailing "\n" at all. I'm not sure if it's possible to get a message lacking a trailing "\n" through any MTA, though. So I think the following should fix it, but I'm having even more trouble than usual trusting anything I do: ---------8<---------- Subject: [PATCH] mbox: only add a trailing newline if one does not exist In retrospect, commit dbdc7a42dd885523 from 2016-04-11 did not make sense since the CR would've been added before the "From " line anyways. Fixes: dbdc7a42dd885523 ("mbox: unconditionally add trailing newline") Reported-by: Konstantin Ryabitsev Link: https://public-inbox.org/meta/20201210202145.7agtcmrtl5jec42d@chatter.i7.local/ --- lib/PublicInbox/Mbox.pm | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/lib/PublicInbox/Mbox.pm b/lib/PublicInbox/Mbox.pm index c88350c9..1895d7e6 100644 --- a/lib/PublicInbox/Mbox.pm +++ b/lib/PublicInbox/Mbox.pm @@ -133,7 +133,11 @@ sub msg_body ($) { # https://www.loc.gov/preservation/digital/formats/fdd/fdd000385.shtml # https://web.archive.org/http://www.qmail.org/man/man5/mbox.html $$bdy =~ s/^(>*From )/>$1/gm; - $$bdy .= "\n"; + + # some messages lack a trailing newline, there needs to be one + # before the "From " line for the next message + $$bdy .= "\n" if substr($$bdy, -1, 1) ne "\n"; + $$bdy; } sub thread_cb {