unofficial mirror of meta@public-inbox.org
 help / color / mirror / Atom feed
From: Eric Wong <e@80x24.org>
To: meta@public-inbox.org
Subject: [PATCH 4/5] nntp: XPATH uses ->ALL extindex, too
Date: Sat, 28 Nov 2020 05:09:15 +0000	[thread overview]
Message-ID: <20201128050916.5586-5-e@80x24.org> (raw)
In-Reply-To: <20201128050916.5586-1-e@80x24.org>

Another 30-40% speedup when testing against a local
lore.kernel.org mirror.  In either case, we'll consistently sort
the response for ease-of-testing and client-side
cache-friendliness.
---
 lib/PublicInbox/NNTP.pm | 24 ++++++++++++++++++++----
 t/extsearch.t           |  4 +++-
 2 files changed, 23 insertions(+), 5 deletions(-)

diff --git a/lib/PublicInbox/NNTP.pm b/lib/PublicInbox/NNTP.pm
index c014eff0..d314a3d1 100644
--- a/lib/PublicInbox/NNTP.pm
+++ b/lib/PublicInbox/NNTP.pm
@@ -994,12 +994,28 @@ sub cmd_xpath ($$) {
 	return r501 unless $mid =~ $ONE_MSGID;
 	$mid = $1;
 	my @paths;
-	foreach my $ng (values %{$self->{nntpd}->{groups}}) {
-		my $n = $ng->mm->num_for($mid);
-		push @paths, "$ng->{newsgroup}/$n" if defined $n;
+	my $pi_cfg = $self->{nntpd}->{pi_config};
+	my $groups = $pi_cfg->{-by_newsgroup};
+	if (my $ALL = $pi_cfg->ALL) {
+		my ($id, $prev, %seen);
+		while (my $smsg = $ALL->over->next_by_mid($mid, \$id, \$prev)) {
+			my $xr3 = $ALL->over->get_xref3($smsg->{num});
+			for my $x (@$xr3) {
+				my ($ngname, $n) = split(/:/, $x);
+				$x = "$ngname/$n";
+				if ($groups->{$ngname} && !$seen{$x}++) {
+					push(@paths, $x);
+				}
+			}
+		}
+	} else { # slow path, no point in using long_response
+		for my $ibx (values %$groups) {
+			my $n = $ibx->mm->num_for($mid) // next;
+			push @paths, "$ibx->{newsgroup}/$n";
+		}
 	}
 	return '430 no such article on server' unless @paths;
-	'223 '.join(' ', @paths);
+	'223 '.join(' ', sort(@paths));
 }
 
 sub res ($$) { do_write($_[0], $_[1] . "\r\n") }
diff --git a/t/extsearch.t b/t/extsearch.t
index f9f74e5c..b2439971 100644
--- a/t/extsearch.t
+++ b/t/extsearch.t
@@ -74,9 +74,11 @@ EOF
 SKIP: {
 	require_mods(qw(Net::NNTP), 1);
 	my ($out, $err) = ("$home/nntpd.out.log", "$home/nntpd.err.log");
-	my $cmd = [ '-nntpd', '-W0' ]; #, "--stdout=$out", "--stderr=$err" ];
+	my $cmd = [ '-nntpd', '-W0', "--stdout=$out", "--stderr=$err" ];
 	my $td = start_script($cmd, undef, { 3 => $sock });
 	my $n = Net::NNTP->new($host_port);
+	my @xp = $n->xpath('<testmessage@example.com>');
+	is_deeply(\@xp, [ qw(v1.example/1 v2.example/1) ]);
 	$n->group('v1.example');
 	my $res = $n->head(1);
 	@$res = grep(/^Xref: /, @$res);

  parent reply	other threads:[~2020-11-28  5:09 UTC|newest]

Thread overview: 6+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-11-28  5:09 [PATCH 0/5] nntp: round 2 of ->ALL extindex speedups Eric Wong
2020-11-28  5:09 ` [PATCH 1/5] nntp: NEWGROUPS uses long_response Eric Wong
2020-11-28  5:09 ` [PATCH 2/5] nntp: speed up mid_lookup() using ->ALL extindex Eric Wong
2020-11-28  5:09 ` [PATCH 3/5] nntp: art_lookup: use mid_lookup and simplify Eric Wong
2020-11-28  5:09 ` Eric Wong [this message]
2020-11-28  5:09 ` [PATCH 5/5] nntpd: remove redundant {groups} shortcut Eric Wong

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://public-inbox.org/README

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20201128050916.5586-5-e@80x24.org \
    --to=e@80x24.org \
    --cc=meta@public-inbox.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).