From: Eric Wong <e@80x24.org>
To: meta@public-inbox.org
Subject: [PATCH 4/5] nntp: XPATH uses ->ALL extindex, too
Date: Sat, 28 Nov 2020 05:09:15 +0000 [thread overview]
Message-ID: <20201128050916.5586-5-e@80x24.org> (raw)
In-Reply-To: <20201128050916.5586-1-e@80x24.org>
Another 30-40% speedup when testing against a local
lore.kernel.org mirror. In either case, we'll consistently sort
the response for ease-of-testing and client-side
cache-friendliness.
---
lib/PublicInbox/NNTP.pm | 24 ++++++++++++++++++++----
t/extsearch.t | 4 +++-
2 files changed, 23 insertions(+), 5 deletions(-)
diff --git a/lib/PublicInbox/NNTP.pm b/lib/PublicInbox/NNTP.pm
index c014eff0..d314a3d1 100644
--- a/lib/PublicInbox/NNTP.pm
+++ b/lib/PublicInbox/NNTP.pm
@@ -994,12 +994,28 @@ sub cmd_xpath ($$) {
return r501 unless $mid =~ $ONE_MSGID;
$mid = $1;
my @paths;
- foreach my $ng (values %{$self->{nntpd}->{groups}}) {
- my $n = $ng->mm->num_for($mid);
- push @paths, "$ng->{newsgroup}/$n" if defined $n;
+ my $pi_cfg = $self->{nntpd}->{pi_config};
+ my $groups = $pi_cfg->{-by_newsgroup};
+ if (my $ALL = $pi_cfg->ALL) {
+ my ($id, $prev, %seen);
+ while (my $smsg = $ALL->over->next_by_mid($mid, \$id, \$prev)) {
+ my $xr3 = $ALL->over->get_xref3($smsg->{num});
+ for my $x (@$xr3) {
+ my ($ngname, $n) = split(/:/, $x);
+ $x = "$ngname/$n";
+ if ($groups->{$ngname} && !$seen{$x}++) {
+ push(@paths, $x);
+ }
+ }
+ }
+ } else { # slow path, no point in using long_response
+ for my $ibx (values %$groups) {
+ my $n = $ibx->mm->num_for($mid) // next;
+ push @paths, "$ibx->{newsgroup}/$n";
+ }
}
return '430 no such article on server' unless @paths;
- '223 '.join(' ', @paths);
+ '223 '.join(' ', sort(@paths));
}
sub res ($$) { do_write($_[0], $_[1] . "\r\n") }
diff --git a/t/extsearch.t b/t/extsearch.t
index f9f74e5c..b2439971 100644
--- a/t/extsearch.t
+++ b/t/extsearch.t
@@ -74,9 +74,11 @@ EOF
SKIP: {
require_mods(qw(Net::NNTP), 1);
my ($out, $err) = ("$home/nntpd.out.log", "$home/nntpd.err.log");
- my $cmd = [ '-nntpd', '-W0' ]; #, "--stdout=$out", "--stderr=$err" ];
+ my $cmd = [ '-nntpd', '-W0', "--stdout=$out", "--stderr=$err" ];
my $td = start_script($cmd, undef, { 3 => $sock });
my $n = Net::NNTP->new($host_port);
+ my @xp = $n->xpath('<testmessage@example.com>');
+ is_deeply(\@xp, [ qw(v1.example/1 v2.example/1) ]);
$n->group('v1.example');
my $res = $n->head(1);
@$res = grep(/^Xref: /, @$res);
next prev parent reply other threads:[~2020-11-28 5:09 UTC|newest]
Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-11-28 5:09 [PATCH 0/5] nntp: round 2 of ->ALL extindex speedups Eric Wong
2020-11-28 5:09 ` [PATCH 1/5] nntp: NEWGROUPS uses long_response Eric Wong
2020-11-28 5:09 ` [PATCH 2/5] nntp: speed up mid_lookup() using ->ALL extindex Eric Wong
2020-11-28 5:09 ` [PATCH 3/5] nntp: art_lookup: use mid_lookup and simplify Eric Wong
2020-11-28 5:09 ` Eric Wong [this message]
2020-11-28 5:09 ` [PATCH 5/5] nntpd: remove redundant {groups} shortcut Eric Wong
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://public-inbox.org/README
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20201128050916.5586-5-e@80x24.org \
--to=e@80x24.org \
--cc=meta@public-inbox.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).