[PATCH] www: sort all /$INBOX/ topics by Received: timestamp

[Eric Wong <e@80x24.org>]

Kyle Meyer <kyle@kyleam.com> wrote:
> As noted by Ihor on the Org list (<87pmarnhlr.fsf@localhost>), Org's
> public-inbox archive has a thread pinned to the top of the $inbox/
> overview:
> 
>  * https://yhetil.org/orgmode/
> 
> That thread has a message where a sender set his Date to a future date:
> 
>   $ curl -fSsL https://yhetil.org/orgmode/ZT2vNKsf3Lp5xit3@protected.localdomain/raw | \
>     grep Date:
>   Date: Sun, 29 Oct 2023 04:02:44 +0300
> 
> Based on grepping around the public-inbox tree and Git history, I know
> that there are spots in public-inbox that prefer a date from the
> Received headers over the one from the Date header.  Would that make
> sense to do here too to reduce the chances of a date from the future
> pinning a thread?  (Or perhaps that's already the intention and
> something's off here?)

Thanks for the report.  The previous pinning prevention only
prevented older messages from being pinned to the /$INBOX/ landing
page.  It didn't prevent recent, future-looking messages from
being pinned to the landing page, which seems to be what
happened in your case.

Does this untested patch fix it?

--------8<------
Subject: [PATCH] www: sort all /$INBOX/ topics by Received: timestamp

Our previous pinning prevention only worked to prevent older
(non-most-recent) topics from being pinned to the landing page,
but not the most recent window of messages.

We still sort messages within threads by Date: because that
makes git-send-email patchsets display more nicely, but we
don't want recent topics pinned due to future Date: headers.

I nearly switched sort_ds() back to sorting by Received: until
I looked back on commit 8e52e5fdea416d6fda0b8d301144af0c043a5a76
(use both Date: and Received: times, 2018-03-21) and was reminded
git-send-email relies on Date: for large series, so I added a
note about it for sort_ds().

Reported-by: Kyle Meyer <kyle@kyleam.com>
Link: https://public-inbox.org/meta/87edr5gx63.fsf@kyleam.com/
---
 lib/PublicInbox/View.pm | 18 +++++++++++-------
 1 file changed, 11 insertions(+), 7 deletions(-)

diff --git a/lib/PublicInbox/View.pm b/lib/PublicInbox/View.pm
index b8d6d85e..e5f748f7 100644
--- a/lib/PublicInbox/View.pm
+++ b/lib/PublicInbox/View.pm
@@ -1019,6 +1019,8 @@ sub _skel_ghost {
 	1;
 }
 
+# note: we favor Date: here because git-send-email increments it
+# to preserve [PATCH $N/$M] ordering in series (it can't control Received:)
 sub sort_ds {
 	@{$_[0]} = sort {
 		(eval { $a->topmost->{ds} } || 0) <=>
@@ -1040,9 +1042,10 @@ sub acc_topic { # walk_thread callback
 	if ($has_blob) {
 		my $subj = subject_normalized($smsg->{subject});
 		$subj = '(no subject)' if $subj eq '';
+		my $ts = $smsg->{ts};
 		my $ds = $smsg->{ds};
 		if ($level == 0) { # new, top-level topic
-			my $topic = [ $ds, 1, { $subj => $mid }, $subj ];
+			my $topic = [ $ts, $ds, 1, { $subj => $mid }, $subj ];
 			$ctx->{-cur_topic} = $topic;
 			push @{$ctx->{order}}, $topic;
 			return 1;
@@ -1050,10 +1053,11 @@ sub acc_topic { # walk_thread callback
 
 		# continue existing topic
 		my $topic = $ctx->{-cur_topic}; # should never be undef
-		$topic->[0] = $ds if $ds > $topic->[0];
-		$topic->[1]++; # bump N+ message counter
-		my $seen = $topic->[2];
-		if (scalar(@$topic) == 3) { # parent was a ghost
+		$topic->[0] = $ts if $ts > $topic->[0];
+		$topic->[1] = $ds if $ds > $topic->[1];
+		$topic->[2]++; # bump N+ message counter
+		my $seen = $topic->[3];
+		if (scalar(@$topic) == 4) { # parent was a ghost
 			push @$topic, $subj;
 		} elsif (!defined($seen->{$subj})) {
 			push @$topic, $level, $subj; # @extra messages
@@ -1061,7 +1065,7 @@ sub acc_topic { # walk_thread callback
 		$seen->{$subj} = $mid; # latest for subject
 	} else { # ghost message
 		return 1 if $level != 0; # ignore child ghosts
-		my $topic = $ctx->{-cur_topic} = [ -666, 0, {} ];
+		my $topic = $ctx->{-cur_topic} = [ -666, -666, 0, {} ];
 		push @{$ctx->{order}}, $topic;
 	}
 	1;
@@ -1082,7 +1086,7 @@ sub dump_topics {
 	}
 	# sort by recency, this allows new posts to "bump" old topics...
 	foreach my $topic (sort { $b->[0] <=> $a->[0] } @$order) {
-		my ($ds, $n, $seen, $top_subj, @extra) = @$topic;
+		my ($ts, $ds, $n, $seen, $top_subj, @extra) = @$topic;
 		@$topic = ();
 		next unless defined $top_subj;  # ghost topic
 		my $mid = delete $seen->{$top_subj};