From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: X-Spam-Status: No, score=-4.2 required=3.0 tests=ALL_TRUSTED,AWL,BAYES_00, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, T_SCC_BODY_TEXT_LINE shortcircuit=no autolearn=ham autolearn_force=no version=3.4.6 Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id 257B11F406 for ; Thu, 30 Nov 2023 11:41:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=80x24.org; s=selector1; t=1701344469; bh=GKqPBzYG/yuxXXdnzJyDlRYVY0vwKDoPVGDuglK7B24=; h=From:To:Subject:Date:From; b=PCw1C+/BzqhRvNQc8c3wxbyf4sveGWw8oa0nRjkE3Cr7QEVo9frwJKC6u59/BD89e exVHplFDlXWviGrUVqx49oZg4ZBemPeUo0jzMrQTivwXLr+hvL7FgUEKXQR53CssHh w9wezQAzclaBgYGifEwXcnohmCpsErMPsY8kLxe0= From: Eric Wong To: meta@public-inbox.org Subject: [PATCH 00/15] various cindex fixes + speedups Date: Thu, 30 Nov 2023 11:40:53 +0000 Message-ID: <20231130114109.2577708-1-e@80x24.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit List-Id: Notable changes: 10/15 provides a huge speedup which will hopefully make future developments faster. 12/15 probably obsoletes libgit2 for extindex "all" users. 13/15 can save some memory with many inboxes while making configuration easier. Eric Wong (15): cindex: fix store_repo+repo_stored on no-op codesearch: allow inbox count to exceed matches config: reject newlines consistently in dir names cindex: only create {-cidx_err} field on failures cindex: keep batch pipe for pruning SHA-256 repos cindex: store extensions.objectFormat with repo data git: share unlinked pack checking code with gcf2 cindex: skip getpid guard for most OnDestroy use spawn: drop IO layer support from redirects cindex: speed up initial scan setup phase inbox: expire resources more aggressively git_async_cat: use git from "all" extindex if possible www_listing: support publicInbox.nameIsUrl inbox: shrink data structures for publicinbox.*.hide codesearch: use retry_reopen for WWW Documentation/public-inbox-config.pod | 19 +- lib/PublicInbox/CodeSearch.pm | 54 +++-- lib/PublicInbox/CodeSearchIdx.pm | 286 ++++++++++++++++---------- lib/PublicInbox/Config.pm | 32 ++- lib/PublicInbox/Gcf2.pm | 16 +- lib/PublicInbox/Git.pm | 27 +-- lib/PublicInbox/GitAsyncCat.pm | 8 +- lib/PublicInbox/Inbox.pm | 32 +-- lib/PublicInbox/MailDiff.pm | 3 +- lib/PublicInbox/SearchIdx.pm | 5 +- lib/PublicInbox/Spawn.pm | 32 +-- lib/PublicInbox/WwwListing.pm | 21 +- 12 files changed, 303 insertions(+), 232 deletions(-)