From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net X-Spam-Level: X-Spam-Status: No, score=-4.0 required=3.0 tests=ALL_TRUSTED,BAYES_00 shortcircuit=no autolearn=ham autolearn_force=no version=3.4.2 Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id 889311F86C for ; Sat, 28 Nov 2020 05:09:16 +0000 (UTC) From: Eric Wong To: meta@public-inbox.org Subject: [PATCH 0/5] nntp: round 2 of ->ALL extindex speedups Date: Sat, 28 Nov 2020 05:09:11 +0000 Message-Id: <20201128050916.5586-1-e@80x24.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit List-Id: All of the O(n) iterations through newsgroups can either go through the ->ALL extindex or is broken out into long_response to not monopolize event loops. So 50-100K newsgroups ought to be usable... Further speeding up NEWGROUPS will require some MiscIdx indexing additions, but that no longer hogs up an event loop iteration. One remaining problems is startup time with 50-100K newsgroups; and that affects everything (especially -mda...) Eric Wong (5): nntp: NEWGROUPS uses long_response nntp: speed up mid_lookup() using ->ALL extindex nntp: art_lookup: use mid_lookup and simplify nntp: XPATH uses ->ALL extindex, too nntpd: remove redundant {groups} shortcut lib/PublicInbox/NNTP.pm | 125 +++++++++++++++++++++++++-------------- lib/PublicInbox/NNTPD.pm | 8 +-- t/extsearch.t | 4 +- t/nntp.t | 1 - 4 files changed, 87 insertions(+), 51 deletions(-)