future date in Date header pinning thread to top of $inbox/

future date in Date header pinning thread to top of $inbox/

[Kyle Meyer]

As noted by Ihor on the Org list (<87pmarnhlr.fsf@localhost>), Org's
public-inbox archive has a thread pinned to the top of the $inbox/
overview:

 * https://yhetil.org/orgmode/

That thread has a message where a sender set his Date to a future date:

  $ curl -fSsL https://yhetil.org/orgmode/ZT2vNKsf3Lp5xit3@protected.localdomain/raw | \
    grep Date:
  Date: Sun, 29 Oct 2023 04:02:44 +0300

Based on grepping around the public-inbox tree and Git history, I know
that there are spots in public-inbox that prefer a date from the
Received headers over the one from the Date header.  Would that make
sense to do here too to reduce the chances of a date from the future
pinning a thread?  (Or perhaps that's already the intention and
something's off here?)

Thanks.

[PATCH] www: sort all /$INBOX/ topics by Received: timestamp

[Eric Wong]

Kyle Meyer <kyle@kyleam.com> wrote:
> As noted by Ihor on the Org list (<87pmarnhlr.fsf@localhost>), Org's
> public-inbox archive has a thread pinned to the top of the $inbox/
> overview:
> 
>  * https://yhetil.org/orgmode/
> 
> That thread has a message where a sender set his Date to a future date:
> 
>   $ curl -fSsL https://yhetil.org/orgmode/ZT2vNKsf3Lp5xit3@protected.localdomain/raw | \
>     grep Date:
>   Date: Sun, 29 Oct 2023 04:02:44 +0300
> 
> Based on grepping around the public-inbox tree and Git history, I know
> that there are spots in public-inbox that prefer a date from the
> Received headers over the one from the Date header.  Would that make
> sense to do here too to reduce the chances of a date from the future
> pinning a thread?  (Or perhaps that's already the intention and
> something's off here?)

Thanks for the report.  The previous pinning prevention only
prevented older messages from being pinned to the /$INBOX/ landing
page.  It didn't prevent recent, future-looking messages from
being pinned to the landing page, which seems to be what
happened in your case.

Does this untested patch fix it?

--------8<------
Subject: [PATCH] www: sort all /$INBOX/ topics by Received: timestamp

Our previous pinning prevention only worked to prevent older
(non-most-recent) topics from being pinned to the landing page,
but not the most recent window of messages.

We still sort messages within threads by Date: because that
makes git-send-email patchsets display more nicely, but we
don't want recent topics pinned due to future Date: headers.

I nearly switched sort_ds() back to sorting by Received: until
I looked back on commit 8e52e5fdea416d6fda0b8d301144af0c043a5a76
(use both Date: and Received: times, 2018-03-21) and was reminded
git-send-email relies on Date: for large series, so I added a
note about it for sort_ds().

Reported-by: Kyle Meyer <kyle@kyleam.com>
Link: https://public-inbox.org/meta/87edr5gx63.fsf@kyleam.com/
---
 lib/PublicInbox/View.pm | 18 +++++++++++-------
 1 file changed, 11 insertions(+), 7 deletions(-)

diff --git a/lib/PublicInbox/View.pm b/lib/PublicInbox/View.pm
index b8d6d85e..e5f748f7 100644
--- a/lib/PublicInbox/View.pm
+++ b/lib/PublicInbox/View.pm
@@ -1019,6 +1019,8 @@ sub _skel_ghost {
 	1;
 }
 
+# note: we favor Date: here because git-send-email increments it
+# to preserve [PATCH $N/$M] ordering in series (it can't control Received:)
 sub sort_ds {
 	@{$_[0]} = sort {
 		(eval { $a->topmost->{ds} } || 0) <=>
@@ -1040,9 +1042,10 @@ sub acc_topic { # walk_thread callback
 	if ($has_blob) {
 		my $subj = subject_normalized($smsg->{subject});
 		$subj = '(no subject)' if $subj eq '';
+		my $ts = $smsg->{ts};
 		my $ds = $smsg->{ds};
 		if ($level == 0) { # new, top-level topic
-			my $topic = [ $ds, 1, { $subj => $mid }, $subj ];
+			my $topic = [ $ts, $ds, 1, { $subj => $mid }, $subj ];
 			$ctx->{-cur_topic} = $topic;
 			push @{$ctx->{order}}, $topic;
 			return 1;
@@ -1050,10 +1053,11 @@ sub acc_topic { # walk_thread callback
 
 		# continue existing topic
 		my $topic = $ctx->{-cur_topic}; # should never be undef
-		$topic->[0] = $ds if $ds > $topic->[0];
-		$topic->[1]++; # bump N+ message counter
-		my $seen = $topic->[2];
-		if (scalar(@$topic) == 3) { # parent was a ghost
+		$topic->[0] = $ts if $ts > $topic->[0];
+		$topic->[1] = $ds if $ds > $topic->[1];
+		$topic->[2]++; # bump N+ message counter
+		my $seen = $topic->[3];
+		if (scalar(@$topic) == 4) { # parent was a ghost
 			push @$topic, $subj;
 		} elsif (!defined($seen->{$subj})) {
 			push @$topic, $level, $subj; # @extra messages
@@ -1061,7 +1065,7 @@ sub acc_topic { # walk_thread callback
 		$seen->{$subj} = $mid; # latest for subject
 	} else { # ghost message
 		return 1 if $level != 0; # ignore child ghosts
-		my $topic = $ctx->{-cur_topic} = [ -666, 0, {} ];
+		my $topic = $ctx->{-cur_topic} = [ -666, -666, 0, {} ];
 		push @{$ctx->{order}}, $topic;
 	}
 	1;
@@ -1082,7 +1086,7 @@ sub dump_topics {
 	}
 	# sort by recency, this allows new posts to "bump" old topics...
 	foreach my $topic (sort { $b->[0] <=> $a->[0] } @$order) {
-		my ($ds, $n, $seen, $top_subj, @extra) = @$topic;
+		my ($ts, $ds, $n, $seen, $top_subj, @extra) = @$topic;
 		@$topic = ();
 		next unless defined $top_subj;  # ghost topic
 		my $mid = delete $seen->{$top_subj};

Re: [PATCH] www: sort all /$INBOX/ topics by Received: timestamp

[Kyle Meyer]

[ Ihor, I carried Bastien's email address over from your
  <87pmarnhlr.fsf@localhost>, but it's being reported as unknown user.
  Replacing with known good one. ]

Eric Wong writes:

> Does this untested patch fix it?

Yes, looks good in a quick local test.  Thank you for the speedy fix!

Re: [PATCH] www: sort all /$INBOX/ topics by Received: timestamp

[Ihor Radchenko]

Eric Wong <e@80x24.org> writes:

> Thanks for the report.  The previous pinning prevention only
> prevented older messages from being pinned to the /$INBOX/ landing
> page.  It didn't prevent recent, future-looking messages from
> being pinned to the landing page, which seems to be what
> happened in your case.
>
> Does this untested patch fix it?

I am looking at https://list.orgmode.org/ and the thread still displays
the future date "2023-10-29  1:04 UTC" as the last update time. Though
ordering suggests that the future time was ignored when building recent
thread list.


-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

[PATCH] www: display Received: timestamp for dumped topics

[Kyle Meyer]

Ihor Radchenko writes:

> Eric Wong <e@80x24.org> writes:
[...]
>> Does this untested patch fix it?
>
> I am looking at https://list.orgmode.org/ and the thread still displays
> the future date "2023-10-29  1:04 UTC" as the last update time. Though
> ordering suggests that the future time was ignored when building recent
> thread list.

The "it" that I reported is the sorting, so as far as I can see it's
fixed.  With the "last update time", I suppose you're talking about the
second line below:

  [FEATURE REQUEST] Timezone support in org-mode datestamps and org-agenda
   2023-10-29  1:04 UTC  (110+ messages)
                                        ` [POLL] Proposed syntax for [...]

I don't care too much one way or the other, but it makes sense to me for
that to match the value that's actually used to sort the messages.
Eric, what about something like this?

-- >8 --
Subject: [PATCH] www: display Received: timestamp for dumped topics

As a follow-up to 0f19ca7e3ca6fc56 (www: sort all /$INBOX/ topics by
Received: timestamp, 2023-02-04), display the Received: timestamp
rather than Date: timestamp when dumping the topic so that the
displayed date aligns with the sorting.

Reported-by: Ihor Radchenko <yantar92@posteo.net>
Link: https://public-inbox.org/meta/87o7q7di5i.fsf@localhost/
---
 lib/PublicInbox/View.pm | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/lib/PublicInbox/View.pm b/lib/PublicInbox/View.pm
index e5f748f7..384d7894 100644
--- a/lib/PublicInbox/View.pm
+++ b/lib/PublicInbox/View.pm
@@ -1093,7 +1093,7 @@ sub dump_topics {
 		my $href = mid_href($mid);
 		my $prev_subj = [ split(/ /, $top_subj) ];
 		$top_subj = ascii_html($top_subj);
-		$ds = fmt_ts($ds);
+		$ts = fmt_ts($ts);
 
 		# $n isn't the total number of posts on the topic,
 		# just the number of posts in the current results window
@@ -1107,7 +1107,7 @@ sub dump_topics {
 		}
 
 		my $s = "<a\nhref=\"$href/T/$anchor\">$top_subj</a>\n" .
-			" $ds UTC $n\n";
+			" $ts UTC $n\n";
 		while (@extra) {
 			my $level = shift @extra;
 			my $subj = shift @extra; # already normalized

base-commit: 0f19ca7e3ca6fc568a8713c5ececdb4d9fa577e6
-- 
2.39.1

Re: [PATCH] www: display Received: timestamp for dumped topics

[Eric Wong]

Kyle Meyer <kyle@kyleam.com> wrote:
> Ihor Radchenko writes:
> 
> > Eric Wong <e@80x24.org> writes:
> [...]
> >> Does this untested patch fix it?
> >
> > I am looking at https://list.orgmode.org/ and the thread still displays
> > the future date "2023-10-29  1:04 UTC" as the last update time. Though
> > ordering suggests that the future time was ignored when building recent
> > thread list.
> 
> The "it" that I reported is the sorting, so as far as I can see it's
> fixed.  With the "last update time", I suppose you're talking about the
> second line below:
> 
>   [FEATURE REQUEST] Timezone support in org-mode datestamps and org-agenda
>    2023-10-29  1:04 UTC  (110+ messages)
>                                         ` [POLL] Proposed syntax for [...]
> 
> I don't care too much one way or the other, but it makes sense to me for
> that to match the value that's actually used to sort the messages.
> Eric, what about something like this?

I'm slightly against it.  Date: is what's shown in every other
message view, and I think that's what most MUAs do, too.  I'm
fine with wrong dates being shown as long as it's not abusable
to pin a message to the top of /$INBOX/.

Aside from git-send-email behavior mentioned upthread, Date:
should also be favored in general because it's consistent across
recipients, and across independently-run archives w/o common
history.

I think the best we can do is point out to senders their clocks
are wrong, as this has been the case going back decades.

Re: [PATCH] www: display Received: timestamp for dumped topics

[Kyle Meyer]

Eric Wong writes:

> I'm slightly against it.  Date: is what's shown in every other
> message view, and I think that's what most MUAs do, too.  I'm
> fine with wrong dates being shown as long as it's not abusable
> to pin a message to the top of /$INBOX/.

Right, that seems like a good place to draw the line.

> Aside from git-send-email behavior mentioned upthread, Date:
> should also be favored in general because it's consistent across
> recipients, and across independently-run archives w/o common
> history.
>
> I think the best we can do is point out to senders their clocks
> are wrong, as this has been the case going back decades.

Fair enough.  Thanks.

Re: [PATCH] www: display Received: timestamp for dumped topics

[Ihor Radchenko]

Eric Wong <e@80x24.org> writes:

>>   [FEATURE REQUEST] Timezone support in org-mode datestamps and org-agenda
>>    2023-10-29  1:04 UTC  (110+ messages)
>>                                         ` [POLL] Proposed syntax for [...]
>> 
>> I don't care too much one way or the other, but it makes sense to me for
>> that to match the value that's actually used to sort the messages.
>> Eric, what about something like this?
>
> I'm slightly against it.  Date: is what's shown in every other
> message view, and I think that's what most MUAs do, too.  I'm
> fine with wrong dates being shown as long as it's not abusable
> to pin a message to the top of /$INBOX/.

Fair point.

The only inconvenience is that update time of the thread will remain in
future even when new messages arrive. But it is minor.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>