"Deep-linking prevented" when clicking xz attachment at lore.kernel.org

"Deep-linking prevented" when clicking xz attachment at lore.kernel.org

[Vlastimil Babka]

Hi,

When opening:
https://lore.kernel.org/all/20220309021531.GA22223@xsang-OptiPlex-9020/

And clicking at [-- Attachment #4: dmesg.xz --]
Which links to
https://lore.kernel.org/all/20220309021531.GA22223@xsang-OptiPlex-9020/4-dmesg.xz

I get with a high probability a "Deep-linking prevented" response instead of
the attachment. In Firefox and Chrome. Sometimes it does succeed and
provides the attachment. When opening the attachment URL directly so that
there's no Referer, it succeeds reliably. Other people confirmed this too.
Reporting here per Konstantin's advise.

Thanks,
Vlastimil

[PATCH] www: loosen deep-linking prevention

[Eric Wong]

Vlastimil Babka <vbabka@suse.cz> wrote:
> Hi,
> 
> When opening:
> https://lore.kernel.org/all/20220309021531.GA22223@xsang-OptiPlex-9020/
> 
> And clicking at [-- Attachment #4: dmesg.xz --]
> Which links to
> https://lore.kernel.org/all/20220309021531.GA22223@xsang-OptiPlex-9020/4-dmesg.xz
> 
> I get with a high probability a "Deep-linking prevented" response instead of
> the attachment. In Firefox and Chrome. Sometimes it does succeed and
> provides the attachment. When opening the attachment URL directly so that
> there's no Referer, it succeeds reliably. Other people confirmed this too.
> Reporting here per Konstantin's advise.

Not sure exactly why this is on the browser side, but I think
the patch below fixes it.  I've deployed to
<https://yhbt.net/lore/>, and tested going through
<https://80x24.org/deep_link.html> via lynx and dillo (w3m
doesn't send Referer in this case)

------------8<---------
Subject: [PATCH] www: loosen deep-linking prevention

Apparently some browsers can set a Referer: header which fails
to match.  I'm not certain why, but making "$schema://$HOST_PORT"
matches case-insensitive seems more correct regardless.

In case that doesn't work, we'll also allow bypassing deep-link
prevention via a POST form button.

Reported-by: Vlastimil Babka <vbabka@suse.cz>
Link: https://public-inbox.org/meta/93ebfbd1-9924-481c-4edc-9b232d1e995c@suse.cz/
---
 lib/PublicInbox/WWW.pm       |  6 +++++-
 lib/PublicInbox/WwwAttach.pm | 18 ++++++++++++------
 2 files changed, 17 insertions(+), 7 deletions(-)

diff --git a/lib/PublicInbox/WWW.pm b/lib/PublicInbox/WWW.pm
index a282784a..755d7558 100644
--- a/lib/PublicInbox/WWW.pm
+++ b/lib/PublicInbox/WWW.pm
@@ -1,4 +1,4 @@
-# Copyright (C) 2014-2021 all contributors <meta@public-inbox.org>
+# Copyright (C) all contributors <meta@public-inbox.org>
 # License: AGPL-3.0+ <https://www.gnu.org/licenses/agpl-3.0.txt>
 #
 # Main web interface for mailing list archives
@@ -64,6 +64,10 @@ sub call {
 				serve_git($ctx, $epoch, $path);
 		} elsif ($path_info =~ m!$INBOX_RE/(\w+)\.sql\.gz\z!o) {
 			return get_altid_dump($ctx, $1, $2);
+		} elsif ($path_info =~ m!$INBOX_RE/$MID_RE/$ATTACH_RE\z!o) {
+			my ($idx, $fn) = ($3, $4);
+			return invalid_inbox_mid($ctx, $1, $2) ||
+				get_attach($ctx, $idx, $fn);
 		} elsif ($path_info =~ m!$INBOX_RE/!o) {
 			return invalid_inbox($ctx, $1) || mbox_results($ctx);
 		}
diff --git a/lib/PublicInbox/WwwAttach.pm b/lib/PublicInbox/WwwAttach.pm
index c17394af..87844bf3 100644
--- a/lib/PublicInbox/WwwAttach.pm
+++ b/lib/PublicInbox/WwwAttach.pm
@@ -1,4 +1,4 @@
-# Copyright (C) 2016-2021 all contributors <meta@public-inbox.org>
+# Copyright (C) all contributors <meta@public-inbox.org>
 # License: AGPL-3.0+ <https://www.gnu.org/licenses/agpl-3.0.txt>
 
 # For retrieving attachments from messages in the WWW interface
@@ -11,16 +11,17 @@ use PublicInbox::Eml;
 sub referer_match ($) {
 	my ($ctx) = @_;
 	my $env = $ctx->{env};
-	my $referer = $env->{HTTP_REFERER} // '';
+	return 1 if $env->{REQUEST_METHOD} eq 'POST';
+	my $referer = lc($env->{HTTP_REFERER} // '');
 	return 1 if $referer eq ''; # no referer is always OK for wget/curl
 
 	# prevent deep-linking from other domains on some browsers (Firefox)
 	# n.b.: $ctx->{ibx}->base_url($env) with INBOX_URL won't work
 	# with dillo, we can only match "$url_scheme://$HTTP_HOST/" without
 	# path components
-	my $base_url = $env->{'psgi.url_scheme'} . '://' .
+	my $base_url = lc($env->{'psgi.url_scheme'} . '://' .
 			($env->{HTTP_HOST} //
-			 "$env->{SERVER_NAME}:$env->{SERVER_PORT}") . '/';
+			 "$env->{SERVER_NAME}:$env->{SERVER_PORT}") . '/');
 	index($referer, $base_url) == 0;
 }
 
@@ -46,8 +47,13 @@ sub get_attach_i { # ->each_part callback
 			$part = $part->body;
 		} else {
 			$res->[0] = 403;
-			$res->[1]->[1] = 'text/plain';
-			$part = "Deep-linking prevented\n";
+			$res->[1]->[1] = 'text/html';
+			$part = <<"";
+<html><head><title>download
+attachment</title><body><pre>Deep-linking prevented</pre><form
+method=post\naction=""><input type=submit value="Download attachment"
+/></form></body></html>
+
 		}
 	}
 	push @{$res->[1]}, 'Content-Length', length($part);