From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: X-Spam-Status: No, score=-4.2 required=3.0 tests=ALL_TRUSTED,AWL,BAYES_00, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF shortcircuit=no autolearn=ham autolearn_force=no version=3.4.6 Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id 6C14F1F518 for ; Sun, 19 May 2024 21:55:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=80x24.org; s=selector1; t=1716155709; bh=w8Epl8R2e3Q912ibGaqW5SbSqfUXeXcg8hZfYtUzM5I=; h=From:To:Subject:Date:From; b=eCZIU6KjFkp39N9YI4dRpzv5RtqJ/tEKZxIvshP/tDcEfTSB4Y9REdRDswSwFhzcs nM4znatZEPZknoC3Uog7etUrO6Zpch5PtHy+MDrPT59uqqutByKK9o9CroWKT3uMGk U1KN+v3PisBG8Pq+/Zw23g7YoDLNvKE1G6E1oZDI= From: Eric Wong To: meta@public-inbox.org Subject: [PATCH 0/9] fixes noticed while working on indexheader Date: Sun, 19 May 2024 21:55:00 +0000 Message-Id: <20240519215509.2267117-1-e@80x24.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit List-Id: Still trying to figure out how to go about indexheader+altid across -extindex, multiple inboxes, and lei. But a bunch of improvements were found to existing behavior and we shouldn't have to worry about running out of FDs in xap_helper, anymore. Introducing khashl.h seems like a big code import atm; but hsearch(3) really sucks and std::map is too much alien-looking C++ (and chained hash tables have poor locality). The FUSE shim will certainly be C (and not C++) and perhaps URCU (for rculfhash) is too rare a dependency to count on, so having khashl available would help there. Eric Wong (9): config: dedupe ibx->{newsgroup} xap_helper: key search instances by -Q params, too xap_helper.h: use khashl.h instead of hsearch(3) xap_helper.h: use xcalloc to simplify error checking xap_helper.h: memoize Xapian handles with khashl xap_helper: expire DB handles when FD table is near full xap_helper: drop DB handles on EMFILE/ENFILE/etc... lei_saved_search: drop ->altid_map method www_text: fix /$INBOX/_/text/help/raw endpoint MANIFEST | 1 + lib/PublicInbox/Config.pm | 4 +- lib/PublicInbox/ExtSearchIdx.pm | 8 +- lib/PublicInbox/LeiSavedSearch.pm | 2 - lib/PublicInbox/Search.pm | 16 + lib/PublicInbox/WwwText.pm | 2 +- lib/PublicInbox/XapHelper.pm | 48 ++- lib/PublicInbox/XapHelperCxx.pm | 1 + lib/PublicInbox/khashl.h | 502 ++++++++++++++++++++++++++++++ lib/PublicInbox/xap_helper.h | 273 +++++++++------- lib/PublicInbox/xh_cidx.h | 79 +++-- t/psgi_text.t | 21 +- t/xap_helper.t | 23 ++ 13 files changed, 818 insertions(+), 162 deletions(-) create mode 100644 lib/PublicInbox/khashl.h