Re: Bug related to (maybe?) / in Message-Id

[Eric Wong <e@80x24.org>]

Uwe Kleine-König <u.kleine-koenig@pengutronix.de> wrote:
> Hello Eric,
> 
> first of all: Thanks for your quick answer.

No problem.  I've been stuck on coderepo indexing stuff and needed the
distraction :<

> Playing around with slashes got my admin and me on the right trail:
> https://httpd.apache.org/docs/current/mod/core.html#allowencodedslashes
> 
> We set that to "On" and now it (mostly) works. Maybe it's worth adding
> this hint to the documentation even though apache isn't the most
> recommended setup? Maybe other servers have a similar security setting?

Thanks for figuring that out.  Perhaps a note in
Documentation/public-inbox.cgi.pod suffices (patch welcome :).

> I wrote "mostly" because
> 
> 	https://lore.ptxdist.org/ptxdist/Y+07h0l%2FzJJAgs9s@falbala.internal.home.lespocky.de/
> 	https://lore.ptxdist.org/ptxdist/Y+07h0l%2FzJJAgs9s@falbala.internal.home.lespocky.de
> 	https://lore.ptxdist.org/ptxdist/Y+07h0l/zJJAgs9s@falbala.internal.home.lespocky.de/
> 
> work as expected;
> 
> 	https://lore.ptxdist.org/ptxdist/Y+07h0l/zJJAgs9s@falbala.internal.home.lespocky.de
> 
> however does not, that yields a short "Not Found".

Most importantly, public-inbox itself only generates the first URL,
right? (`%2F' and trailing `/')

> With the patch applied the logged stuff for these URLs is mostly
> identical. REMOTE_PORT differs which is expected. Otherwise only
> PATH_INFO, PATH_TRANSLATED and REQUEST_URI differ. They are
> respectively:

Actually, only REQUEST_URI matters for mail retrieval URLs,
<snip irrelevant lines>

> 	"REQUEST_URI" => "/ptxdist/Y+07h0l%2FzJJAgs9s\@falbala.internal.home.lespocky.de/",
> 	"REQUEST_URI" => "/ptxdist/Y+07h0l%2FzJJAgs9s\@falbala.internal.home.lespocky.de",
> 	"REQUEST_URI" => "/ptxdist/Y+07h0l/zJJAgs9s\@falbala.internal.home.lespocky.de/",

I'm pleasantly surprised the 3rd case works, actually :>
I do hope we're not generating those URLs on our own.

> 	"REQUEST_URI" => "/ptxdist/Y+07h0l/zJJAgs9s\@falbala.internal.home.lespocky.de",

> . Not sure making the last URL work is easily possible (and worth the
> effort)? If a Message-Id ends in "/T" or similar the result will always
> be ambigous?

I think the last case is fixable and should be fixed.  It's
reasonable a user unfamiliar w/o URL encoding knowledge would
paste `/' w/o encoding it to `%2F', first

> One thing I just noticed is:
> 
> $ curl https://lore.ptxdist.org/ptxdist/Y+07h0l/zJJAgs9s@falbala.internal.home.lespocky.de/T
> Redirecting to https://lore.ptxdist.org/ptxdist/Y+07h0l/zJJAgs9s@falbala.internal.home.lespocky.de/T
> 
> which makes Firefox say: "The page isn’t redirecting properly". It works
> fine with the / replaced by %2F:
> 
> $ curl https://lore.ptxdist.org/ptxdist/Y+07h0l%2fzJJAgs9s@falbala.internal.home.lespocky.de/T
> Redirecting to https://lore.ptxdist.org/ptxdist/Y+07h0l%2fzJJAgs9s@falbala.internal.home.lespocky.de/T/#u

Yeah, that's another case that should work.

I don't really like the current giant if/elsif block
of route possibilities in lib/PublicInbox/WWW.pm but have
some ideas on how to improve it...  But a lot on my plate
already and way overdue for another release :<


It's a tragedy that mutt changed their default Message-ID format :<
Newer versions allow overriding $message_id_format in muttrc,
but mutt 2.0.5 in Debian 11 won't have it.

Fwiw, I run mutt on several machines/arches/OSes, getting them
all to the same new version of mutt will take many years and I
don't want compile + keep numerous binaries up-to-date in the
mean time.

I also use msmtp and distribute my muttrc + ~/bin to all of
them, so I wrote this msmtp wrapper which is mutt-version-agnostic:

$ cat $HOME/bin/msmtp-msgid
eval 'exec perl -w -S $0 ${1+"$@"}'
if 0; # running under some shell
# in muttrc:
# set sendmail = msmtp-msgid
use v5.12;
my ($hdr, $bdy) = split(/\n\n/, do { local $/; <STDIN> }, 2);
if ($hdr !~ /^Message-ID:\s*<\d{14}\b/aims) {
	my $h;
	if ($hdr =~ /^Message-ID:[^@]+\@([^>]+)>/ims) {
		$h = $1;
	} else {
		require Sys::Hostname;
		$h = Sys::Hostname::hostname();
	}
	require POSIX;
	require Time::HiRes;
	my ($time, $msec) = Time::HiRes::gettimeofday();
	my $msgid = POSIX::strftime("%Y%m%d%H%M%S.M$msec", gmtime($time));
	$hdr =~ s/^Message-ID:[^\n]+/Message-ID: <$msgid\@$h>/msi;
}
my $pid = open(my $fh, '|-', 'msmtp', @ARGV) // die "popen: $!";
print $fh $hdr, "\n\n", $bdy or die "print $!";
close $fh or die "msmtp \$?=$?";
__END__