From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: X-Spam-Status: No, score=-4.0 required=3.0 tests=ALL_TRUSTED,BAYES_00 shortcircuit=no autolearn=ham autolearn_force=no version=3.4.2 Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id 0CA5D1F8C6 for ; Tue, 27 Jul 2021 10:44:31 +0000 (UTC) From: Eric Wong To: meta@public-inbox.org Subject: [PATCH/RFC 0/2] *index: sequential commits Date: Tue, 27 Jul 2021 10:44:28 +0000 Message-Id: <20210727104430.2160-1-e@80x24.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit List-Id: To avoid Xapian slowdowns with gigantic DBs, more shards seem necessary. But that can mean excessive parallelism, especially in writes. So introduce a middle road between fully parallel and fully serialized behavior by only serializing commits. 2/2 is lightly tested (and will probably take a bit...) Eric Wong (2): treewide: s/sequential_shard/sequential-shard/g index+extindex: support indexSequentialCommit Documentation/public-inbox-index.pod | 22 ++++++++++++++++++++++ lib/PublicInbox/Admin.pm | 4 ++-- lib/PublicInbox/Config.pm | 2 +- lib/PublicInbox/ExtSearchIdx.pm | 1 + lib/PublicInbox/LEI.pm | 4 ++-- lib/PublicInbox/V2Writable.pm | 23 ++++++++++++++++------- script/public-inbox-convert | 2 +- script/public-inbox-extindex | 1 + script/public-inbox-index | 13 ++++++++----- script/public-inbox-xcpdb | 4 ++-- t/v2mirror.t | 8 ++++---- 11 files changed, 60 insertions(+), 24 deletions(-)