* Query to see all new "topics"
@ 2023-11-07 18:07 Konstantin Ryabitsev
2023-11-09 2:45 ` [RFC] www: add topics.html endpoint [was: Query to see all new "topics"] Eric Wong
0 siblings, 1 reply; 6+ messages in thread
From: Konstantin Ryabitsev @ 2023-11-07 18:07 UTC (permalink / raw)
To: meta
Hello:
Following the discussion on the ksummit list [1], I wanted to give someone a query
they could use to keep an eye on any new threads. Is there a xapian query that
can be used to effectively say "return just top-level messages and exclude any
follow-ups"? It's not quite as simple as "s:* AND NOT s:Re:" because we also
want to exclude threaded patches. Some kind of equivalent of "any messages
without an in-reply-to/references header"?
-K
[1] https://lore.kernel.org/ksummit/20231106-venomous-raccoon-of-wealth-acc57c@nitro/T/
^ permalink raw reply [flat|nested] 6+ messages in thread
* [RFC] www: add topics.html endpoint [was: Query to see all new "topics"]
2023-11-07 18:07 Query to see all new "topics" Konstantin Ryabitsev
@ 2023-11-09 2:45 ` Eric Wong
2023-11-09 18:10 ` Konstantin Ryabitsev
0 siblings, 1 reply; 6+ messages in thread
From: Eric Wong @ 2023-11-09 2:45 UTC (permalink / raw)
To: Konstantin Ryabitsev; +Cc: meta
Konstantin Ryabitsev <konstantin@linuxfoundation.org> wrote:
> Hello:
>
> Following the discussion on the ksummit list [1], I wanted to give someone a query
> they could use to keep an eye on any new threads. Is there a xapian query that
> can be used to effectively say "return just top-level messages and exclude any
> follow-ups"? It's not quite as simple as "s:* AND NOT s:Re:" because we also
> want to exclude threaded patches. Some kind of equivalent of "any messages
> without an in-reply-to/references header"?
Not easily with current Xapian schema..
It can get kinda close but you don't get the thread root with:
https://yhbt.net/lore/all/?q=rt:yesterday..&o=-1&t=1
The above isn't very useful IMHO, and also very expensive...
SQLite can actually do it pretty quickly, but it's WWW-only
(patch below): https://yhbt.net/lore/all/topics.html
I don't know if it can work with the way lei is supposed to dump
output for MUAs to consume... So maybe a custom TUI is the way
forward, but that comes with all the problems with
developing+maintaining a TUI I wrote about[1] previously...
-------8<------
Subject: [PATCH] www: add topics.html endpoint
This seems like a easy (but WWW-specific) way to get recent
topics as suggested by Konstantin. Perhaps an Atom endpoint
will also be useful.
To do this with Xapian would require a new columns and
reindexing; and I'm not sure if the current lei handling of
search results by dumping results to a format readable by common
MUAs would work well with this.
Suggested-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
Link: https://public-inbox.org/meta/20231107-skilled-cobra-of-swiftness-a6ff26@meerkat/
---
MANIFEST | 1 +
lib/PublicInbox/WWW.pm | 9 ++++++
lib/PublicInbox/WwwStream.pm | 1 +
lib/PublicInbox/WwwTopics.pm | 55 ++++++++++++++++++++++++++++++++++++
t/extindex-psgi.t | 6 ++++
t/plack.t | 9 ++++--
6 files changed, 78 insertions(+), 3 deletions(-)
create mode 100644 lib/PublicInbox/WwwTopics.pm
diff --git a/MANIFEST b/MANIFEST
index 51dcffaf..e1c3dc97 100644
--- a/MANIFEST
+++ b/MANIFEST
@@ -371,6 +371,7 @@ lib/PublicInbox/WwwListing.pm
lib/PublicInbox/WwwStatic.pm
lib/PublicInbox/WwwStream.pm
lib/PublicInbox/WwwText.pm
+lib/PublicInbox/WwwTopics.pm
lib/PublicInbox/XapClient.pm
lib/PublicInbox/XapHelper.pm
lib/PublicInbox/XapHelperCxx.pm
diff --git a/lib/PublicInbox/WWW.pm b/lib/PublicInbox/WWW.pm
index d2bd68ea..dcaf93cb 100644
--- a/lib/PublicInbox/WWW.pm
+++ b/lib/PublicInbox/WWW.pm
@@ -101,6 +101,8 @@ sub call {
invalid_inbox($ctx, $1) || get_atom($ctx);
} elsif ($path_info =~ m!$INBOX_RE/new\.html\z!o) {
invalid_inbox($ctx, $1) || get_new($ctx);
+ } elsif ($path_info =~ m!$INBOX_RE/topics\.html\z!o) {
+ invalid_inbox($ctx, $1) || get_topics($ctx);
} elsif ($path_info =~ m!$INBOX_RE/description\z!o) {
get_description($ctx, $1);
} elsif ($path_info =~ m!$INBOX_RE/(?:(?:git/)?([0-9]+)(?:\.git)?/)?
@@ -270,6 +272,13 @@ sub get_new {
PublicInbox::Feed::new_html($ctx);
}
+# /$INBOX/topics.html -> HTML only
+sub get_topics {
+ my ($ctx) = @_;
+ require PublicInbox::WwwTopics;
+ PublicInbox::WwwTopics::topics_html($ctx) || r404($ctx);
+}
+
# /$INBOX/?r=$GIT_COMMIT -> HTML only
sub get_index {
my ($ctx) = @_;
diff --git a/lib/PublicInbox/WwwStream.pm b/lib/PublicInbox/WwwStream.pm
index 4cbdda99..3a1d6edf 100644
--- a/lib/PublicInbox/WwwStream.pm
+++ b/lib/PublicInbox/WwwStream.pm
@@ -113,6 +113,7 @@ sub html_top ($) {
qq(<a\nid=mirror) .
qq(\nhref="${upfx}_/text/mirror/">mirror</a>$code / ).
qq(<a\nhref="$atom">Atom feed</a>);
+ $links .= delete($ctx->{-html_more_links}) if $ctx->{-html_more_links};
if ($ibx->isrch) {
my $q_val = delete($ctx->{-q_value_html}) // '';
$q_val = qq(\nvalue="$q_val") if $q_val ne '';
diff --git a/lib/PublicInbox/WwwTopics.pm b/lib/PublicInbox/WwwTopics.pm
new file mode 100644
index 00000000..5605cfbe
--- /dev/null
+++ b/lib/PublicInbox/WwwTopics.pm
@@ -0,0 +1,55 @@
+# Copyright (C) all contributors <meta@public-inbox.org>
+# License: AGPL-3.0+ <https://www.gnu.org/licenses/agpl-3.0.txt>
+
+package PublicInbox::WwwTopics;
+use v5.12;
+use autodie qw(open);
+use PublicInbox::Hval qw(ascii_html mid_href fmt_ts);
+use PublicInbox::WwwStream;
+
+sub add_topic_line ($$$) {
+ my (undef, $prev, $nr) = @_;
+ my $s = ascii_html($prev->{subject});
+ $s = '(no subject)' if $s eq '';
+ $_[0] .= "\n".fmt_ts($prev->{ds}).
+ qq{ <a\nhref="}.mid_href($prev->{mid}).qq{/#r">$s</a>};
+ $_[0] .= " $nr+ messages" if $nr > 1;
+}
+
+sub topics_html { # GET /$INBOX_NAME/topics.html
+ my ($ctx) = @_;
+ my $over = $ctx->{ibx}->over or
+ return $ctx->{www}->can('need')->($ctx,'Overview');
+
+ # XXX there is likely faster ways to do this.
+ # OTOH SQLite tends to be faster with multiple simple queries
+ # rather than more complex ones
+ my $msgs = $over->do_get(<<EOS, { limit => 10000 });
+SELECT num,ts,ds,tid,ddd FROM over WHERE tid IN
+(SELECT DISTINCT(tid) FROM over WHERE tid > 0 ORDER BY tid DESC LIMIT 200)
+AND +num > 0
+ORDER BY tid,ts ASC
+EOS
+ # can't use SQL to filter references since our schema wasn't designed
+ # for it, but our SQL sorts by ascending time to favor top-level
+ # messages while our final result (post-references filter) favors
+ # recent messages
+ chomp($ctx->{-html_more_links} = <<EOM);
+\n- recent:[<a href="./">subjects (threaded)</a>|topics] (all times UTC)
+EOM
+ my $buf = '<pre>';
+ my ($nr, $prev);
+ while (my $smsg = pop @$msgs) {
+ if ($prev && $smsg->{tid} != $prev->{tid}) {
+ add_topic_line($buf, $prev, $nr);
+ $nr = 0;
+ }
+ ++$nr;
+ $prev = $smsg;
+ }
+ add_topic_line($buf, $prev, $nr) if $prev;
+ $buf .= '</pre>';
+ PublicInbox::WwwStream::html_oneshot($ctx, 200, $buf);
+}
+
+1;
diff --git a/t/extindex-psgi.t b/t/extindex-psgi.t
index f71210a5..9e0c7dc3 100644
--- a/t/extindex-psgi.t
+++ b/t/extindex-psgi.t
@@ -118,6 +118,12 @@ my $client = sub {
is($res->code, 404, '404 on out-of-range mid2tid query');
$res = $cb->(POST("/m2t/t\@1/?q=s:unrelated&x=m"));
is($res->code, 404, '404 on cross-thread search');
+
+
+ $res = $cb->(GET('/m2t/topics.html'));
+ is($res->code, 200, 'topics.html on basic v2');
+ $res = $cb->(GET('/all/topics.html'));
+ is($res->code, 200, 'topics.html on extindex');
};
test_psgi(sub { $www->call(@_) }, $client);
%$env = (%$env, TMPDIR => $tmpdir, PI_CONFIG => $pi_config);
diff --git a/t/plack.t b/t/plack.t
index 7f80f488..7ec35e7a 100644
--- a/t/plack.t
+++ b/t/plack.t
@@ -204,9 +204,12 @@ my $c1 = sub {
my $raw = PublicInbox::Eml->new(\$body);
is($raw->body_raw, $eml->body_raw, 'ISO-2022-JP body unmodified');
- $res = $cb->(GET($pfx . '/blah@example.com/t.mbox.gz'));
- is(501, $res->code, '501 when overview missing');
- like($res->content, qr!\bOverview\b!, 'overview omission noted');
+ for my $u (qw(blah@example.com/t.mbox.gz topics.html)) {
+ $res = $cb->(GET("$pfx/$u"));
+ is(501, $res->code, "501 on /$u when overview missing");
+ like($res->content, qr!\bOverview\b!,
+ "overview omission noted for /$u");
+ }
# legacy redirects
for my $t (qw(m f)) {
[1] https://public-inbox.org/meta/20230922203353.M780211@dcvr/
^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [RFC] www: add topics.html endpoint [was: Query to see all new "topics"]
2023-11-09 2:45 ` [RFC] www: add topics.html endpoint [was: Query to see all new "topics"] Eric Wong
@ 2023-11-09 18:10 ` Konstantin Ryabitsev
2023-11-10 3:09 ` [RFC v2] www: add topics_(new|active).(html|atom) endpoints Eric Wong
0 siblings, 1 reply; 6+ messages in thread
From: Konstantin Ryabitsev @ 2023-11-09 18:10 UTC (permalink / raw)
To: Eric Wong; +Cc: meta
On Thu, Nov 09, 2023 at 02:45:08AM +0000, Eric Wong wrote:
> This seems like a easy (but WWW-specific) way to get recent
> topics as suggested by Konstantin. Perhaps an Atom endpoint
> will also be useful.
Yes, actually thinking about this some more, perhaps it makes sense to expose
this as an RSS feed feature (maybe even exclusively as an RSS feed feature?).
Have two different feeds:
- new topics: just all the new threads
- hot topics: NN most active threads (kinda lkml.org's "hottest messages")
Have this available per-list and for the extindex -- I think this would be
a great feature that we can point people at as a mechanism to keep an eye on
overall activity.
I haven't tried your patch yet -- doubt I will be able before coming back from
Plumbers next week.
-K
^ permalink raw reply [flat|nested] 6+ messages in thread
* [RFC v2] www: add topics_(new|active).(html|atom) endpoints
2023-11-09 18:10 ` Konstantin Ryabitsev
@ 2023-11-10 3:09 ` Eric Wong
2023-11-10 17:16 ` Konstantin Ryabitsev
0 siblings, 1 reply; 6+ messages in thread
From: Eric Wong @ 2023-11-10 3:09 UTC (permalink / raw)
To: Konstantin Ryabitsev; +Cc: meta
Konstantin Ryabitsev <konstantin@linuxfoundation.org> wrote:
> On Thu, Nov 09, 2023 at 02:45:08AM +0000, Eric Wong wrote:
> > This seems like a easy (but WWW-specific) way to get recent
> > topics as suggested by Konstantin. Perhaps an Atom endpoint
> > will also be useful.
>
> Yes, actually thinking about this some more, perhaps it makes sense to expose
> this as an RSS feed feature (maybe even exclusively as an RSS feed feature?).
I assume Atom is OK? I don't know of any widely-used feed readers
which only do RSS without Atom support. IIRC Atom is less ambiguous
and supports the in-reply-to extension.
That said, the Atom feeds generated by this RFC includes full
messages because that's the easiest way to tie into our existing
Atom generation code, so it's currently slower than the HTML
version which never retrieves git blobs.
> Have two different feeds:
>
> - new topics: just all the new threads
> - hot topics: NN most active threads (kinda lkml.org's "hottest messages")
I'm not sure if `hot' means it's the most read (not just replied-to);
but tracking read counts isn't something that scales on decentralized
systems. So I'm naming it "active" instead...
> Have this available per-list and for the extindex -- I think this would be
> a great feature that we can point people at as a mechanism to keep an eye on
> overall activity.
Yeah, lots of the WWW and lei code works transparently between extindex
and regular inboxes:
extindex:
https://yhbt.net/lore/all/topics_new.atom
https://yhbt.net/lore/all/topics_active.atom
https://yhbt.net/lore/all/topics_new.html
https://yhbt.net/lore/all/topics_active.html
v2:
https://yhbt.net/lore/lkml/topics_new.atom
https://yhbt.net/lore/lkml/topics_active.atom
https://yhbt.net/lore/lkml/topics_new.html
https://yhbt.net/lore/lkml/topics_active.html
v1:
https://public-inbox.org/git/topics_new.atom
https://public-inbox.org/git/topics_active.atom
https://public-inbox.org/git/topics_new.html
https://public-inbox.org/git/topics_active.html
> I haven't tried your patch yet -- doubt I will be able before coming back from
> Plumbers next week.
No worries. master seems pretty stable these days and I should
really get my brain around getting cindex wired up to WWW to cut
a release...
Anyways, this replaces my prior RFC. I've remembered to use
"GROUP BY" so the SQL is a bit faster than before.
-----------8<----------
Subject: [RFC v2] www: add topics_(new|active).(html|atom) endpoints
This seems like a easy (but WWW-specific) way to get recently
created and recently active topics as suggested by Konstantin.
To do this with Xapian will require a new columns and
reindexing; and I'm not sure if the current lei handling of
search results by dumping results to a format readable by common
MUAs would work well with this. A new TUI may be required...
Suggested-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
Link: https://public-inbox.org/meta/20231107-skilled-cobra-of-swiftness-a6ff26@meerkat/
---
MANIFEST | 1 +
lib/PublicInbox/WWW.pm | 15 +++++-
lib/PublicInbox/WwwAtomStream.pm | 11 ++--
lib/PublicInbox/WwwStream.pm | 1 +
lib/PublicInbox/WwwTopics.pm | 86 ++++++++++++++++++++++++++++++++
t/extindex-psgi.t | 8 +++
t/plack.t | 10 ++--
7 files changed, 122 insertions(+), 10 deletions(-)
create mode 100644 lib/PublicInbox/WwwTopics.pm
diff --git a/MANIFEST b/MANIFEST
index 51dcffaf..e1c3dc97 100644
--- a/MANIFEST
+++ b/MANIFEST
@@ -371,6 +371,7 @@ lib/PublicInbox/WwwListing.pm
lib/PublicInbox/WwwStatic.pm
lib/PublicInbox/WwwStream.pm
lib/PublicInbox/WwwText.pm
+lib/PublicInbox/WwwTopics.pm
lib/PublicInbox/XapClient.pm
lib/PublicInbox/XapHelper.pm
lib/PublicInbox/XapHelperCxx.pm
diff --git a/lib/PublicInbox/WWW.pm b/lib/PublicInbox/WWW.pm
index d2bd68ea..6b616bd4 100644
--- a/lib/PublicInbox/WWW.pm
+++ b/lib/PublicInbox/WWW.pm
@@ -101,6 +101,9 @@ sub call {
invalid_inbox($ctx, $1) || get_atom($ctx);
} elsif ($path_info =~ m!$INBOX_RE/new\.html\z!o) {
invalid_inbox($ctx, $1) || get_new($ctx);
+ } elsif ($path_info =~
+ m!$INBOX_RE/topics_(new|active)\.(atom|html)\z!o) {
+ get_topics($ctx, $1, $2, $3);
} elsif ($path_info =~ m!$INBOX_RE/description\z!o) {
get_description($ctx, $1);
} elsif ($path_info =~ m!$INBOX_RE/(?:(?:git/)?([0-9]+)(?:\.git)?/)?
@@ -270,6 +273,13 @@ sub get_new {
PublicInbox::Feed::new_html($ctx);
}
+# /$INBOX/topics_(new|active).(html|atom)
+sub get_topics {
+ my ($ctx, $ibx_name, $category, $type) = @_;
+ require PublicInbox::WwwTopics;
+ PublicInbox::WwwTopics::response($ctx, $ibx_name, $category, $type);
+}
+
# /$INBOX/?r=$GIT_COMMIT -> HTML only
sub get_index {
my ($ctx) = @_;
@@ -338,11 +348,12 @@ sub get_altid_dump {
}
sub need {
- my ($ctx, $extra) = @_;
+ my ($ctx, $extra, $upref) = @_;
require PublicInbox::WwwStream;
+ $upref //= '../';
PublicInbox::WwwStream::html_oneshot($ctx, 501, <<EOF);
<pre>$extra is not available for this public-inbox
-<a\nhref="../">Return to index</a></pre>
+<a\nhref="$upref">Return to index</a></pre>
EOF
}
diff --git a/lib/PublicInbox/WwwAtomStream.pm b/lib/PublicInbox/WwwAtomStream.pm
index 737cc6cb..26b366f5 100644
--- a/lib/PublicInbox/WwwAtomStream.pm
+++ b/lib/PublicInbox/WwwAtomStream.pm
@@ -99,15 +99,16 @@ sub atom_header {
$base_url .= '?' . $search_q->qs_html(x => undef);
$self_url .= '?' . $search_q->qs_html;
$page_id = to_uuid("q\n".$query);
+ } elsif (defined(my $cat = $ctx->{topic_category})) {
+ $title = title_tag("$cat topics - ".$ibx->description);
+ $self_url .= "topics_$cat.atom";
} else {
$title = title_tag($ibx->description);
$self_url .= 'new.atom';
- if (defined(my $addr = $ibx->{-primary_address})) {
- $page_id = "mailto:$addr";
- } else {
- $page_id = to_uuid($self_url);
- }
+ my $addr = $ibx->{-primary_address};
+ $page_id = "mailto:$addr" if defined $addr;
}
+ $page_id //= to_uuid($self_url);
qq(<?xml version="1.0" encoding="us-ascii"?>\n) .
qq(<feed\nxmlns="http://www.w3.org/2005/Atom"\n) .
qq(xmlns:thr="http://purl.org/syndication/thread/1.0">) .
diff --git a/lib/PublicInbox/WwwStream.pm b/lib/PublicInbox/WwwStream.pm
index 4cbdda99..3a1d6edf 100644
--- a/lib/PublicInbox/WwwStream.pm
+++ b/lib/PublicInbox/WwwStream.pm
@@ -113,6 +113,7 @@ sub html_top ($) {
qq(<a\nid=mirror) .
qq(\nhref="${upfx}_/text/mirror/">mirror</a>$code / ).
qq(<a\nhref="$atom">Atom feed</a>);
+ $links .= delete($ctx->{-html_more_links}) if $ctx->{-html_more_links};
if ($ibx->isrch) {
my $q_val = delete($ctx->{-q_value_html}) // '';
$q_val = qq(\nvalue="$q_val") if $q_val ne '';
diff --git a/lib/PublicInbox/WwwTopics.pm b/lib/PublicInbox/WwwTopics.pm
new file mode 100644
index 00000000..ad85a46d
--- /dev/null
+++ b/lib/PublicInbox/WwwTopics.pm
@@ -0,0 +1,86 @@
+# Copyright (C) all contributors <meta@public-inbox.org>
+# License: AGPL-3.0+ <https://www.gnu.org/licenses/agpl-3.0.txt>
+
+package PublicInbox::WwwTopics;
+use v5.12;
+use PublicInbox::Hval qw(ascii_html mid_href fmt_ts);
+
+sub add_topic_html ($$) {
+ my (undef, $smsg) = @_;
+ my $s = ascii_html($smsg->{subject});
+ $s = '(no subject)' if $s eq '';
+ $_[0] .= "\n".fmt_ts($smsg->{'MAX(ds)'} // $smsg->{ds}) .
+ qq{ <a\nhref="}.mid_href($smsg->{mid}).qq{/#r">$s</a>};
+ my $nr = $smsg->{'COUNT(num)'};
+ $_[0] .= " $nr+ messages" if $nr > 1;
+}
+
+# n.b. the `SELECT DISTINCT(tid)' subquery is critical for performance
+# with giant inboxes and extindices
+sub topics_new ($) {
+ $_[0]->do_get(<<EOS);
+SELECT ds,ddd,COUNT(num) FROM over WHERE tid IN
+(SELECT DISTINCT(tid) FROM over WHERE tid > 0 ORDER BY ts DESC LIMIT 200)
+AND +num > 0
+GROUP BY tid
+ORDER BY ds ASC
+EOS
+}
+
+sub topics_active ($) {
+ $_[0]->do_get(<<EOS);
+SELECT ddd,MAX(ds),COUNT(num) FROM over WHERE tid IN
+(SELECT DISTINCT(tid) FROM over WHERE tid > 0 ORDER BY ts DESC LIMIT 200)
+AND +num > 0
+GROUP BY tid
+ORDER BY ds ASC
+EOS
+}
+
+sub topics_i { pop @{$_[0]->{msgs}} }
+
+sub topics_atom { # GET /$INBOX_NAME/topics_(new|active).atom
+ my ($ctx) = @_;
+ require PublicInbox::WwwAtomStream;
+ my ($hdr, $smsg, $val);
+ $_->{ds} //= $_->{'MAX(ds)'} // 0 for @{$ctx->{msgs}};
+ PublicInbox::WwwAtomStream->response($ctx, \&topics_i);
+}
+
+sub topics_html { # GET /$INBOX_NAME/topics_(new|active).html
+ my ($ctx) = @_;
+ require PublicInbox::WwwStream;
+ my $buf = '<pre>';
+ $ctx->{-html_more_links} = qq{\n- recent:[<a
+href="./">subjects (threaded)</a>|};
+
+ if ($ctx->{topic_category} eq 'new') {
+ $ctx->{-html_more_links} .= qq{<b>topics (new)</b>|<a
+href="./topics_active.html">topics (active)</a>]};
+ } else { # topic_category eq "active" - topics with recent replies
+ $ctx->{-html_more_links} .= qq{<a
+href="./topics_new.html">topics (new)</a>|<b>topics (active)</b>]};
+ }
+ # can't use SQL to filter references since our schema wasn't designed
+ # for it, but our SQL sorts by ascending time to favor top-level
+ # messages while our final result (post-references filter) favors
+ # recent messages
+ my $msgs = delete $ctx->{msgs};
+ add_topic_html($buf, pop @$msgs) while scalar(@$msgs);
+ $buf .= '</pre>';
+ PublicInbox::WwwStream::html_oneshot($ctx, 200, $buf);
+}
+
+sub response {
+ my ($ctx, $ibx_name, $category, $type) = @_;
+ my ($ret, $over);
+ $ret = PublicInbox::WWW::invalid_inbox($ctx, $ibx_name) and return $ret;
+ $over = $ctx->{ibx}->over or
+ return PublicInbox::WWW::need($ctx, 'Overview', './');
+ $ctx->{msgs} = $category eq 'new' ? topics_new($over) :
+ topics_active($over);
+ $ctx->{topic_category} = $category;
+ $type eq 'atom' ? topics_atom($ctx) : topics_html($ctx);
+}
+
+1;
diff --git a/t/extindex-psgi.t b/t/extindex-psgi.t
index f71210a5..896c46ff 100644
--- a/t/extindex-psgi.t
+++ b/t/extindex-psgi.t
@@ -118,6 +118,14 @@ my $client = sub {
is($res->code, 404, '404 on out-of-range mid2tid query');
$res = $cb->(POST("/m2t/t\@1/?q=s:unrelated&x=m"));
is($res->code, 404, '404 on cross-thread search');
+
+
+ for my $c (qw(new active)) {
+ $res = $cb->(GET("/m2t/topics_$c.html"));
+ is($res->code, 200, "topics_$c.html on basic v2");
+ $res = $cb->(GET("/all/topics_$c.html"));
+ is($res->code, 200, "topics_$c.html on extindex");
+ }
};
test_psgi(sub { $www->call(@_) }, $client);
%$env = (%$env, TMPDIR => $tmpdir, PI_CONFIG => $pi_config);
diff --git a/t/plack.t b/t/plack.t
index 7f80f488..07cab12a 100644
--- a/t/plack.t
+++ b/t/plack.t
@@ -204,9 +204,13 @@ my $c1 = sub {
my $raw = PublicInbox::Eml->new(\$body);
is($raw->body_raw, $eml->body_raw, 'ISO-2022-JP body unmodified');
- $res = $cb->(GET($pfx . '/blah@example.com/t.mbox.gz'));
- is(501, $res->code, '501 when overview missing');
- like($res->content, qr!\bOverview\b!, 'overview omission noted');
+ for my $u (qw(blah@example.com/t.mbox.gz topics_new.html
+ topics_active.html)) {
+ $res = $cb->(GET("$pfx/$u"));
+ is(501, $res->code, "501 on /$u when overview missing");
+ like($res->content, qr!\bOverview\b!,
+ "overview omission noted for /$u");
+ }
# legacy redirects
for my $t (qw(m f)) {
^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [RFC v2] www: add topics_(new|active).(html|atom) endpoints
2023-11-10 3:09 ` [RFC v2] www: add topics_(new|active).(html|atom) endpoints Eric Wong
@ 2023-11-10 17:16 ` Konstantin Ryabitsev
2023-11-10 22:23 ` Eric Wong
0 siblings, 1 reply; 6+ messages in thread
From: Konstantin Ryabitsev @ 2023-11-10 17:16 UTC (permalink / raw)
To: Eric Wong; +Cc: meta
On Fri, Nov 10, 2023 at 03:09:59AM +0000, Eric Wong wrote:
> > Yes, actually thinking about this some more, perhaps it makes sense to expose
> > this as an RSS feed feature (maybe even exclusively as an RSS feed feature?).
>
> I assume Atom is OK? I don't know of any widely-used feed readers
> which only do RSS without Atom support. IIRC Atom is less ambiguous
> and supports the in-reply-to extension.
Yes, sorry, I know they aren't the same thing, but in my head Atom is just a
form of RSS (perhaps for the same reason why everyone says "rss reader" but
nobody says "atom reader").
> That said, the Atom feeds generated by this RFC includes full
> messages because that's the easiest way to tie into our existing
> Atom generation code, so it's currently slower than the HTML
> version which never retrieves git blobs.
That's fine, actually, because this lets people read the full message to
figure out if they are interested in the rest of the thread or not.
> > Have two different feeds:
> >
> > - new topics: just all the new threads
> > - hot topics: NN most active threads (kinda lkml.org's "hottest messages")
>
> I'm not sure if `hot' means it's the most read (not just replied-to);
> but tracking read counts isn't something that scales on decentralized
> systems. So I'm naming it "active" instead...
Sounds good to me.
> > Have this available per-list and for the extindex -- I think this would be
> > a great feature that we can point people at as a mechanism to keep an eye on
> > overall activity.
>
> Yeah, lots of the WWW and lei code works transparently between extindex
> and regular inboxes:
>
> extindex:
> https://yhbt.net/lore/all/topics_new.atom
> https://yhbt.net/lore/all/topics_active.atom
> https://yhbt.net/lore/all/topics_new.html
> https://yhbt.net/lore/all/topics_active.html
>
> v2:
> https://yhbt.net/lore/lkml/topics_new.atom
> https://yhbt.net/lore/lkml/topics_active.atom
> https://yhbt.net/lore/lkml/topics_new.html
> https://yhbt.net/lore/lkml/topics_active.html
>
> v1:
> https://public-inbox.org/git/topics_new.atom
> https://public-inbox.org/git/topics_active.atom
> https://public-inbox.org/git/topics_new.html
> https://public-inbox.org/git/topics_active.html
This is great, thank you!
-K
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [RFC v2] www: add topics_(new|active).(html|atom) endpoints
2023-11-10 17:16 ` Konstantin Ryabitsev
@ 2023-11-10 22:23 ` Eric Wong
0 siblings, 0 replies; 6+ messages in thread
From: Eric Wong @ 2023-11-10 22:23 UTC (permalink / raw)
To: Konstantin Ryabitsev; +Cc: meta
Konstantin Ryabitsev <konstantin@linuxfoundation.org> wrote:
> On Fri, Nov 10, 2023 at 03:09:59AM +0000, Eric Wong wrote:
> > That said, the Atom feeds generated by this RFC includes full
> > messages because that's the easiest way to tie into our existing
> > Atom generation code, so it's currently slower than the HTML
> > version which never retrieves git blobs.
>
> That's fine, actually, because this lets people read the full message to
> figure out if they are interested in the rest of the thread or not.
Alright, pushed out as commit 363c043a8a3f379a69802fc566112fcd8f1e750c
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2023-11-10 22:25 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2023-11-07 18:07 Query to see all new "topics" Konstantin Ryabitsev
2023-11-09 2:45 ` [RFC] www: add topics.html endpoint [was: Query to see all new "topics"] Eric Wong
2023-11-09 18:10 ` Konstantin Ryabitsev
2023-11-10 3:09 ` [RFC v2] www: add topics_(new|active).(html|atom) endpoints Eric Wong
2023-11-10 17:16 ` Konstantin Ryabitsev
2023-11-10 22:23 ` Eric Wong
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).