* Query to see all new "topics" @ 2023-11-07 18:07 Konstantin Ryabitsev 2023-11-09 2:45 ` [RFC] www: add topics.html endpoint [was: Query to see all new "topics"] Eric Wong 0 siblings, 1 reply; 6+ messages in thread From: Konstantin Ryabitsev @ 2023-11-07 18:07 UTC (permalink / raw) To: meta Hello: Following the discussion on the ksummit list [1], I wanted to give someone a query they could use to keep an eye on any new threads. Is there a xapian query that can be used to effectively say "return just top-level messages and exclude any follow-ups"? It's not quite as simple as "s:* AND NOT s:Re:" because we also want to exclude threaded patches. Some kind of equivalent of "any messages without an in-reply-to/references header"? -K [1] https://lore.kernel.org/ksummit/20231106-venomous-raccoon-of-wealth-acc57c@nitro/T/ ^ permalink raw reply [flat|nested] 6+ messages in thread
* [RFC] www: add topics.html endpoint [was: Query to see all new "topics"] 2023-11-07 18:07 Query to see all new "topics" Konstantin Ryabitsev @ 2023-11-09 2:45 ` Eric Wong 2023-11-09 18:10 ` Konstantin Ryabitsev 0 siblings, 1 reply; 6+ messages in thread From: Eric Wong @ 2023-11-09 2:45 UTC (permalink / raw) To: Konstantin Ryabitsev; +Cc: meta Konstantin Ryabitsev <konstantin@linuxfoundation.org> wrote: > Hello: > > Following the discussion on the ksummit list [1], I wanted to give someone a query > they could use to keep an eye on any new threads. Is there a xapian query that > can be used to effectively say "return just top-level messages and exclude any > follow-ups"? It's not quite as simple as "s:* AND NOT s:Re:" because we also > want to exclude threaded patches. Some kind of equivalent of "any messages > without an in-reply-to/references header"? Not easily with current Xapian schema.. It can get kinda close but you don't get the thread root with: https://yhbt.net/lore/all/?q=rt:yesterday..&o=-1&t=1 The above isn't very useful IMHO, and also very expensive... SQLite can actually do it pretty quickly, but it's WWW-only (patch below): https://yhbt.net/lore/all/topics.html I don't know if it can work with the way lei is supposed to dump output for MUAs to consume... So maybe a custom TUI is the way forward, but that comes with all the problems with developing+maintaining a TUI I wrote about[1] previously... -------8<------ Subject: [PATCH] www: add topics.html endpoint This seems like a easy (but WWW-specific) way to get recent topics as suggested by Konstantin. Perhaps an Atom endpoint will also be useful. To do this with Xapian would require a new columns and reindexing; and I'm not sure if the current lei handling of search results by dumping results to a format readable by common MUAs would work well with this. Suggested-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org> Link: https://public-inbox.org/meta/20231107-skilled-cobra-of-swiftness-a6ff26@meerkat/ --- MANIFEST | 1 + lib/PublicInbox/WWW.pm | 9 ++++++ lib/PublicInbox/WwwStream.pm | 1 + lib/PublicInbox/WwwTopics.pm | 55 ++++++++++++++++++++++++++++++++++++ t/extindex-psgi.t | 6 ++++ t/plack.t | 9 ++++-- 6 files changed, 78 insertions(+), 3 deletions(-) create mode 100644 lib/PublicInbox/WwwTopics.pm diff --git a/MANIFEST b/MANIFEST index 51dcffaf..e1c3dc97 100644 --- a/MANIFEST +++ b/MANIFEST @@ -371,6 +371,7 @@ lib/PublicInbox/WwwListing.pm lib/PublicInbox/WwwStatic.pm lib/PublicInbox/WwwStream.pm lib/PublicInbox/WwwText.pm +lib/PublicInbox/WwwTopics.pm lib/PublicInbox/XapClient.pm lib/PublicInbox/XapHelper.pm lib/PublicInbox/XapHelperCxx.pm diff --git a/lib/PublicInbox/WWW.pm b/lib/PublicInbox/WWW.pm index d2bd68ea..dcaf93cb 100644 --- a/lib/PublicInbox/WWW.pm +++ b/lib/PublicInbox/WWW.pm @@ -101,6 +101,8 @@ sub call { invalid_inbox($ctx, $1) || get_atom($ctx); } elsif ($path_info =~ m!$INBOX_RE/new\.html\z!o) { invalid_inbox($ctx, $1) || get_new($ctx); + } elsif ($path_info =~ m!$INBOX_RE/topics\.html\z!o) { + invalid_inbox($ctx, $1) || get_topics($ctx); } elsif ($path_info =~ m!$INBOX_RE/description\z!o) { get_description($ctx, $1); } elsif ($path_info =~ m!$INBOX_RE/(?:(?:git/)?([0-9]+)(?:\.git)?/)? @@ -270,6 +272,13 @@ sub get_new { PublicInbox::Feed::new_html($ctx); } +# /$INBOX/topics.html -> HTML only +sub get_topics { + my ($ctx) = @_; + require PublicInbox::WwwTopics; + PublicInbox::WwwTopics::topics_html($ctx) || r404($ctx); +} + # /$INBOX/?r=$GIT_COMMIT -> HTML only sub get_index { my ($ctx) = @_; diff --git a/lib/PublicInbox/WwwStream.pm b/lib/PublicInbox/WwwStream.pm index 4cbdda99..3a1d6edf 100644 --- a/lib/PublicInbox/WwwStream.pm +++ b/lib/PublicInbox/WwwStream.pm @@ -113,6 +113,7 @@ sub html_top ($) { qq(<a\nid=mirror) . qq(\nhref="${upfx}_/text/mirror/">mirror</a>$code / ). qq(<a\nhref="$atom">Atom feed</a>); + $links .= delete($ctx->{-html_more_links}) if $ctx->{-html_more_links}; if ($ibx->isrch) { my $q_val = delete($ctx->{-q_value_html}) // ''; $q_val = qq(\nvalue="$q_val") if $q_val ne ''; diff --git a/lib/PublicInbox/WwwTopics.pm b/lib/PublicInbox/WwwTopics.pm new file mode 100644 index 00000000..5605cfbe --- /dev/null +++ b/lib/PublicInbox/WwwTopics.pm @@ -0,0 +1,55 @@ +# Copyright (C) all contributors <meta@public-inbox.org> +# License: AGPL-3.0+ <https://www.gnu.org/licenses/agpl-3.0.txt> + +package PublicInbox::WwwTopics; +use v5.12; +use autodie qw(open); +use PublicInbox::Hval qw(ascii_html mid_href fmt_ts); +use PublicInbox::WwwStream; + +sub add_topic_line ($$$) { + my (undef, $prev, $nr) = @_; + my $s = ascii_html($prev->{subject}); + $s = '(no subject)' if $s eq ''; + $_[0] .= "\n".fmt_ts($prev->{ds}). + qq{ <a\nhref="}.mid_href($prev->{mid}).qq{/#r">$s</a>}; + $_[0] .= " $nr+ messages" if $nr > 1; +} + +sub topics_html { # GET /$INBOX_NAME/topics.html + my ($ctx) = @_; + my $over = $ctx->{ibx}->over or + return $ctx->{www}->can('need')->($ctx,'Overview'); + + # XXX there is likely faster ways to do this. + # OTOH SQLite tends to be faster with multiple simple queries + # rather than more complex ones + my $msgs = $over->do_get(<<EOS, { limit => 10000 }); +SELECT num,ts,ds,tid,ddd FROM over WHERE tid IN +(SELECT DISTINCT(tid) FROM over WHERE tid > 0 ORDER BY tid DESC LIMIT 200) +AND +num > 0 +ORDER BY tid,ts ASC +EOS + # can't use SQL to filter references since our schema wasn't designed + # for it, but our SQL sorts by ascending time to favor top-level + # messages while our final result (post-references filter) favors + # recent messages + chomp($ctx->{-html_more_links} = <<EOM); +\n- recent:[<a href="./">subjects (threaded)</a>|topics] (all times UTC) +EOM + my $buf = '<pre>'; + my ($nr, $prev); + while (my $smsg = pop @$msgs) { + if ($prev && $smsg->{tid} != $prev->{tid}) { + add_topic_line($buf, $prev, $nr); + $nr = 0; + } + ++$nr; + $prev = $smsg; + } + add_topic_line($buf, $prev, $nr) if $prev; + $buf .= '</pre>'; + PublicInbox::WwwStream::html_oneshot($ctx, 200, $buf); +} + +1; diff --git a/t/extindex-psgi.t b/t/extindex-psgi.t index f71210a5..9e0c7dc3 100644 --- a/t/extindex-psgi.t +++ b/t/extindex-psgi.t @@ -118,6 +118,12 @@ my $client = sub { is($res->code, 404, '404 on out-of-range mid2tid query'); $res = $cb->(POST("/m2t/t\@1/?q=s:unrelated&x=m")); is($res->code, 404, '404 on cross-thread search'); + + + $res = $cb->(GET('/m2t/topics.html')); + is($res->code, 200, 'topics.html on basic v2'); + $res = $cb->(GET('/all/topics.html')); + is($res->code, 200, 'topics.html on extindex'); }; test_psgi(sub { $www->call(@_) }, $client); %$env = (%$env, TMPDIR => $tmpdir, PI_CONFIG => $pi_config); diff --git a/t/plack.t b/t/plack.t index 7f80f488..7ec35e7a 100644 --- a/t/plack.t +++ b/t/plack.t @@ -204,9 +204,12 @@ my $c1 = sub { my $raw = PublicInbox::Eml->new(\$body); is($raw->body_raw, $eml->body_raw, 'ISO-2022-JP body unmodified'); - $res = $cb->(GET($pfx . '/blah@example.com/t.mbox.gz')); - is(501, $res->code, '501 when overview missing'); - like($res->content, qr!\bOverview\b!, 'overview omission noted'); + for my $u (qw(blah@example.com/t.mbox.gz topics.html)) { + $res = $cb->(GET("$pfx/$u")); + is(501, $res->code, "501 on /$u when overview missing"); + like($res->content, qr!\bOverview\b!, + "overview omission noted for /$u"); + } # legacy redirects for my $t (qw(m f)) { [1] https://public-inbox.org/meta/20230922203353.M780211@dcvr/ ^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [RFC] www: add topics.html endpoint [was: Query to see all new "topics"] 2023-11-09 2:45 ` [RFC] www: add topics.html endpoint [was: Query to see all new "topics"] Eric Wong @ 2023-11-09 18:10 ` Konstantin Ryabitsev 2023-11-10 3:09 ` [RFC v2] www: add topics_(new|active).(html|atom) endpoints Eric Wong 0 siblings, 1 reply; 6+ messages in thread From: Konstantin Ryabitsev @ 2023-11-09 18:10 UTC (permalink / raw) To: Eric Wong; +Cc: meta On Thu, Nov 09, 2023 at 02:45:08AM +0000, Eric Wong wrote: > This seems like a easy (but WWW-specific) way to get recent > topics as suggested by Konstantin. Perhaps an Atom endpoint > will also be useful. Yes, actually thinking about this some more, perhaps it makes sense to expose this as an RSS feed feature (maybe even exclusively as an RSS feed feature?). Have two different feeds: - new topics: just all the new threads - hot topics: NN most active threads (kinda lkml.org's "hottest messages") Have this available per-list and for the extindex -- I think this would be a great feature that we can point people at as a mechanism to keep an eye on overall activity. I haven't tried your patch yet -- doubt I will be able before coming back from Plumbers next week. -K ^ permalink raw reply [flat|nested] 6+ messages in thread
* [RFC v2] www: add topics_(new|active).(html|atom) endpoints 2023-11-09 18:10 ` Konstantin Ryabitsev @ 2023-11-10 3:09 ` Eric Wong 2023-11-10 17:16 ` Konstantin Ryabitsev 0 siblings, 1 reply; 6+ messages in thread From: Eric Wong @ 2023-11-10 3:09 UTC (permalink / raw) To: Konstantin Ryabitsev; +Cc: meta Konstantin Ryabitsev <konstantin@linuxfoundation.org> wrote: > On Thu, Nov 09, 2023 at 02:45:08AM +0000, Eric Wong wrote: > > This seems like a easy (but WWW-specific) way to get recent > > topics as suggested by Konstantin. Perhaps an Atom endpoint > > will also be useful. > > Yes, actually thinking about this some more, perhaps it makes sense to expose > this as an RSS feed feature (maybe even exclusively as an RSS feed feature?). I assume Atom is OK? I don't know of any widely-used feed readers which only do RSS without Atom support. IIRC Atom is less ambiguous and supports the in-reply-to extension. That said, the Atom feeds generated by this RFC includes full messages because that's the easiest way to tie into our existing Atom generation code, so it's currently slower than the HTML version which never retrieves git blobs. > Have two different feeds: > > - new topics: just all the new threads > - hot topics: NN most active threads (kinda lkml.org's "hottest messages") I'm not sure if `hot' means it's the most read (not just replied-to); but tracking read counts isn't something that scales on decentralized systems. So I'm naming it "active" instead... > Have this available per-list and for the extindex -- I think this would be > a great feature that we can point people at as a mechanism to keep an eye on > overall activity. Yeah, lots of the WWW and lei code works transparently between extindex and regular inboxes: extindex: https://yhbt.net/lore/all/topics_new.atom https://yhbt.net/lore/all/topics_active.atom https://yhbt.net/lore/all/topics_new.html https://yhbt.net/lore/all/topics_active.html v2: https://yhbt.net/lore/lkml/topics_new.atom https://yhbt.net/lore/lkml/topics_active.atom https://yhbt.net/lore/lkml/topics_new.html https://yhbt.net/lore/lkml/topics_active.html v1: https://public-inbox.org/git/topics_new.atom https://public-inbox.org/git/topics_active.atom https://public-inbox.org/git/topics_new.html https://public-inbox.org/git/topics_active.html > I haven't tried your patch yet -- doubt I will be able before coming back from > Plumbers next week. No worries. master seems pretty stable these days and I should really get my brain around getting cindex wired up to WWW to cut a release... Anyways, this replaces my prior RFC. I've remembered to use "GROUP BY" so the SQL is a bit faster than before. -----------8<---------- Subject: [RFC v2] www: add topics_(new|active).(html|atom) endpoints This seems like a easy (but WWW-specific) way to get recently created and recently active topics as suggested by Konstantin. To do this with Xapian will require a new columns and reindexing; and I'm not sure if the current lei handling of search results by dumping results to a format readable by common MUAs would work well with this. A new TUI may be required... Suggested-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org> Link: https://public-inbox.org/meta/20231107-skilled-cobra-of-swiftness-a6ff26@meerkat/ --- MANIFEST | 1 + lib/PublicInbox/WWW.pm | 15 +++++- lib/PublicInbox/WwwAtomStream.pm | 11 ++-- lib/PublicInbox/WwwStream.pm | 1 + lib/PublicInbox/WwwTopics.pm | 86 ++++++++++++++++++++++++++++++++ t/extindex-psgi.t | 8 +++ t/plack.t | 10 ++-- 7 files changed, 122 insertions(+), 10 deletions(-) create mode 100644 lib/PublicInbox/WwwTopics.pm diff --git a/MANIFEST b/MANIFEST index 51dcffaf..e1c3dc97 100644 --- a/MANIFEST +++ b/MANIFEST @@ -371,6 +371,7 @@ lib/PublicInbox/WwwListing.pm lib/PublicInbox/WwwStatic.pm lib/PublicInbox/WwwStream.pm lib/PublicInbox/WwwText.pm +lib/PublicInbox/WwwTopics.pm lib/PublicInbox/XapClient.pm lib/PublicInbox/XapHelper.pm lib/PublicInbox/XapHelperCxx.pm diff --git a/lib/PublicInbox/WWW.pm b/lib/PublicInbox/WWW.pm index d2bd68ea..6b616bd4 100644 --- a/lib/PublicInbox/WWW.pm +++ b/lib/PublicInbox/WWW.pm @@ -101,6 +101,9 @@ sub call { invalid_inbox($ctx, $1) || get_atom($ctx); } elsif ($path_info =~ m!$INBOX_RE/new\.html\z!o) { invalid_inbox($ctx, $1) || get_new($ctx); + } elsif ($path_info =~ + m!$INBOX_RE/topics_(new|active)\.(atom|html)\z!o) { + get_topics($ctx, $1, $2, $3); } elsif ($path_info =~ m!$INBOX_RE/description\z!o) { get_description($ctx, $1); } elsif ($path_info =~ m!$INBOX_RE/(?:(?:git/)?([0-9]+)(?:\.git)?/)? @@ -270,6 +273,13 @@ sub get_new { PublicInbox::Feed::new_html($ctx); } +# /$INBOX/topics_(new|active).(html|atom) +sub get_topics { + my ($ctx, $ibx_name, $category, $type) = @_; + require PublicInbox::WwwTopics; + PublicInbox::WwwTopics::response($ctx, $ibx_name, $category, $type); +} + # /$INBOX/?r=$GIT_COMMIT -> HTML only sub get_index { my ($ctx) = @_; @@ -338,11 +348,12 @@ sub get_altid_dump { } sub need { - my ($ctx, $extra) = @_; + my ($ctx, $extra, $upref) = @_; require PublicInbox::WwwStream; + $upref //= '../'; PublicInbox::WwwStream::html_oneshot($ctx, 501, <<EOF); <pre>$extra is not available for this public-inbox -<a\nhref="../">Return to index</a></pre> +<a\nhref="$upref">Return to index</a></pre> EOF } diff --git a/lib/PublicInbox/WwwAtomStream.pm b/lib/PublicInbox/WwwAtomStream.pm index 737cc6cb..26b366f5 100644 --- a/lib/PublicInbox/WwwAtomStream.pm +++ b/lib/PublicInbox/WwwAtomStream.pm @@ -99,15 +99,16 @@ sub atom_header { $base_url .= '?' . $search_q->qs_html(x => undef); $self_url .= '?' . $search_q->qs_html; $page_id = to_uuid("q\n".$query); + } elsif (defined(my $cat = $ctx->{topic_category})) { + $title = title_tag("$cat topics - ".$ibx->description); + $self_url .= "topics_$cat.atom"; } else { $title = title_tag($ibx->description); $self_url .= 'new.atom'; - if (defined(my $addr = $ibx->{-primary_address})) { - $page_id = "mailto:$addr"; - } else { - $page_id = to_uuid($self_url); - } + my $addr = $ibx->{-primary_address}; + $page_id = "mailto:$addr" if defined $addr; } + $page_id //= to_uuid($self_url); qq(<?xml version="1.0" encoding="us-ascii"?>\n) . qq(<feed\nxmlns="http://www.w3.org/2005/Atom"\n) . qq(xmlns:thr="http://purl.org/syndication/thread/1.0">) . diff --git a/lib/PublicInbox/WwwStream.pm b/lib/PublicInbox/WwwStream.pm index 4cbdda99..3a1d6edf 100644 --- a/lib/PublicInbox/WwwStream.pm +++ b/lib/PublicInbox/WwwStream.pm @@ -113,6 +113,7 @@ sub html_top ($) { qq(<a\nid=mirror) . qq(\nhref="${upfx}_/text/mirror/">mirror</a>$code / ). qq(<a\nhref="$atom">Atom feed</a>); + $links .= delete($ctx->{-html_more_links}) if $ctx->{-html_more_links}; if ($ibx->isrch) { my $q_val = delete($ctx->{-q_value_html}) // ''; $q_val = qq(\nvalue="$q_val") if $q_val ne ''; diff --git a/lib/PublicInbox/WwwTopics.pm b/lib/PublicInbox/WwwTopics.pm new file mode 100644 index 00000000..ad85a46d --- /dev/null +++ b/lib/PublicInbox/WwwTopics.pm @@ -0,0 +1,86 @@ +# Copyright (C) all contributors <meta@public-inbox.org> +# License: AGPL-3.0+ <https://www.gnu.org/licenses/agpl-3.0.txt> + +package PublicInbox::WwwTopics; +use v5.12; +use PublicInbox::Hval qw(ascii_html mid_href fmt_ts); + +sub add_topic_html ($$) { + my (undef, $smsg) = @_; + my $s = ascii_html($smsg->{subject}); + $s = '(no subject)' if $s eq ''; + $_[0] .= "\n".fmt_ts($smsg->{'MAX(ds)'} // $smsg->{ds}) . + qq{ <a\nhref="}.mid_href($smsg->{mid}).qq{/#r">$s</a>}; + my $nr = $smsg->{'COUNT(num)'}; + $_[0] .= " $nr+ messages" if $nr > 1; +} + +# n.b. the `SELECT DISTINCT(tid)' subquery is critical for performance +# with giant inboxes and extindices +sub topics_new ($) { + $_[0]->do_get(<<EOS); +SELECT ds,ddd,COUNT(num) FROM over WHERE tid IN +(SELECT DISTINCT(tid) FROM over WHERE tid > 0 ORDER BY ts DESC LIMIT 200) +AND +num > 0 +GROUP BY tid +ORDER BY ds ASC +EOS +} + +sub topics_active ($) { + $_[0]->do_get(<<EOS); +SELECT ddd,MAX(ds),COUNT(num) FROM over WHERE tid IN +(SELECT DISTINCT(tid) FROM over WHERE tid > 0 ORDER BY ts DESC LIMIT 200) +AND +num > 0 +GROUP BY tid +ORDER BY ds ASC +EOS +} + +sub topics_i { pop @{$_[0]->{msgs}} } + +sub topics_atom { # GET /$INBOX_NAME/topics_(new|active).atom + my ($ctx) = @_; + require PublicInbox::WwwAtomStream; + my ($hdr, $smsg, $val); + $_->{ds} //= $_->{'MAX(ds)'} // 0 for @{$ctx->{msgs}}; + PublicInbox::WwwAtomStream->response($ctx, \&topics_i); +} + +sub topics_html { # GET /$INBOX_NAME/topics_(new|active).html + my ($ctx) = @_; + require PublicInbox::WwwStream; + my $buf = '<pre>'; + $ctx->{-html_more_links} = qq{\n- recent:[<a +href="./">subjects (threaded)</a>|}; + + if ($ctx->{topic_category} eq 'new') { + $ctx->{-html_more_links} .= qq{<b>topics (new)</b>|<a +href="./topics_active.html">topics (active)</a>]}; + } else { # topic_category eq "active" - topics with recent replies + $ctx->{-html_more_links} .= qq{<a +href="./topics_new.html">topics (new)</a>|<b>topics (active)</b>]}; + } + # can't use SQL to filter references since our schema wasn't designed + # for it, but our SQL sorts by ascending time to favor top-level + # messages while our final result (post-references filter) favors + # recent messages + my $msgs = delete $ctx->{msgs}; + add_topic_html($buf, pop @$msgs) while scalar(@$msgs); + $buf .= '</pre>'; + PublicInbox::WwwStream::html_oneshot($ctx, 200, $buf); +} + +sub response { + my ($ctx, $ibx_name, $category, $type) = @_; + my ($ret, $over); + $ret = PublicInbox::WWW::invalid_inbox($ctx, $ibx_name) and return $ret; + $over = $ctx->{ibx}->over or + return PublicInbox::WWW::need($ctx, 'Overview', './'); + $ctx->{msgs} = $category eq 'new' ? topics_new($over) : + topics_active($over); + $ctx->{topic_category} = $category; + $type eq 'atom' ? topics_atom($ctx) : topics_html($ctx); +} + +1; diff --git a/t/extindex-psgi.t b/t/extindex-psgi.t index f71210a5..896c46ff 100644 --- a/t/extindex-psgi.t +++ b/t/extindex-psgi.t @@ -118,6 +118,14 @@ my $client = sub { is($res->code, 404, '404 on out-of-range mid2tid query'); $res = $cb->(POST("/m2t/t\@1/?q=s:unrelated&x=m")); is($res->code, 404, '404 on cross-thread search'); + + + for my $c (qw(new active)) { + $res = $cb->(GET("/m2t/topics_$c.html")); + is($res->code, 200, "topics_$c.html on basic v2"); + $res = $cb->(GET("/all/topics_$c.html")); + is($res->code, 200, "topics_$c.html on extindex"); + } }; test_psgi(sub { $www->call(@_) }, $client); %$env = (%$env, TMPDIR => $tmpdir, PI_CONFIG => $pi_config); diff --git a/t/plack.t b/t/plack.t index 7f80f488..07cab12a 100644 --- a/t/plack.t +++ b/t/plack.t @@ -204,9 +204,13 @@ my $c1 = sub { my $raw = PublicInbox::Eml->new(\$body); is($raw->body_raw, $eml->body_raw, 'ISO-2022-JP body unmodified'); - $res = $cb->(GET($pfx . '/blah@example.com/t.mbox.gz')); - is(501, $res->code, '501 when overview missing'); - like($res->content, qr!\bOverview\b!, 'overview omission noted'); + for my $u (qw(blah@example.com/t.mbox.gz topics_new.html + topics_active.html)) { + $res = $cb->(GET("$pfx/$u")); + is(501, $res->code, "501 on /$u when overview missing"); + like($res->content, qr!\bOverview\b!, + "overview omission noted for /$u"); + } # legacy redirects for my $t (qw(m f)) { ^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [RFC v2] www: add topics_(new|active).(html|atom) endpoints 2023-11-10 3:09 ` [RFC v2] www: add topics_(new|active).(html|atom) endpoints Eric Wong @ 2023-11-10 17:16 ` Konstantin Ryabitsev 2023-11-10 22:23 ` Eric Wong 0 siblings, 1 reply; 6+ messages in thread From: Konstantin Ryabitsev @ 2023-11-10 17:16 UTC (permalink / raw) To: Eric Wong; +Cc: meta On Fri, Nov 10, 2023 at 03:09:59AM +0000, Eric Wong wrote: > > Yes, actually thinking about this some more, perhaps it makes sense to expose > > this as an RSS feed feature (maybe even exclusively as an RSS feed feature?). > > I assume Atom is OK? I don't know of any widely-used feed readers > which only do RSS without Atom support. IIRC Atom is less ambiguous > and supports the in-reply-to extension. Yes, sorry, I know they aren't the same thing, but in my head Atom is just a form of RSS (perhaps for the same reason why everyone says "rss reader" but nobody says "atom reader"). > That said, the Atom feeds generated by this RFC includes full > messages because that's the easiest way to tie into our existing > Atom generation code, so it's currently slower than the HTML > version which never retrieves git blobs. That's fine, actually, because this lets people read the full message to figure out if they are interested in the rest of the thread or not. > > Have two different feeds: > > > > - new topics: just all the new threads > > - hot topics: NN most active threads (kinda lkml.org's "hottest messages") > > I'm not sure if `hot' means it's the most read (not just replied-to); > but tracking read counts isn't something that scales on decentralized > systems. So I'm naming it "active" instead... Sounds good to me. > > Have this available per-list and for the extindex -- I think this would be > > a great feature that we can point people at as a mechanism to keep an eye on > > overall activity. > > Yeah, lots of the WWW and lei code works transparently between extindex > and regular inboxes: > > extindex: > https://yhbt.net/lore/all/topics_new.atom > https://yhbt.net/lore/all/topics_active.atom > https://yhbt.net/lore/all/topics_new.html > https://yhbt.net/lore/all/topics_active.html > > v2: > https://yhbt.net/lore/lkml/topics_new.atom > https://yhbt.net/lore/lkml/topics_active.atom > https://yhbt.net/lore/lkml/topics_new.html > https://yhbt.net/lore/lkml/topics_active.html > > v1: > https://public-inbox.org/git/topics_new.atom > https://public-inbox.org/git/topics_active.atom > https://public-inbox.org/git/topics_new.html > https://public-inbox.org/git/topics_active.html This is great, thank you! -K ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [RFC v2] www: add topics_(new|active).(html|atom) endpoints 2023-11-10 17:16 ` Konstantin Ryabitsev @ 2023-11-10 22:23 ` Eric Wong 0 siblings, 0 replies; 6+ messages in thread From: Eric Wong @ 2023-11-10 22:23 UTC (permalink / raw) To: Konstantin Ryabitsev; +Cc: meta Konstantin Ryabitsev <konstantin@linuxfoundation.org> wrote: > On Fri, Nov 10, 2023 at 03:09:59AM +0000, Eric Wong wrote: > > That said, the Atom feeds generated by this RFC includes full > > messages because that's the easiest way to tie into our existing > > Atom generation code, so it's currently slower than the HTML > > version which never retrieves git blobs. > > That's fine, actually, because this lets people read the full message to > figure out if they are interested in the rest of the thread or not. Alright, pushed out as commit 363c043a8a3f379a69802fc566112fcd8f1e750c ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2023-11-10 22:25 UTC | newest] Thread overview: 6+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2023-11-07 18:07 Query to see all new "topics" Konstantin Ryabitsev 2023-11-09 2:45 ` [RFC] www: add topics.html endpoint [was: Query to see all new "topics"] Eric Wong 2023-11-09 18:10 ` Konstantin Ryabitsev 2023-11-10 3:09 ` [RFC v2] www: add topics_(new|active).(html|atom) endpoints Eric Wong 2023-11-10 17:16 ` Konstantin Ryabitsev 2023-11-10 22:23 ` Eric Wong
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).