From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: X-Spam-Status: No, score=-4.2 required=3.0 tests=ALL_TRUSTED,AWL,BAYES_00, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, T_SCC_BODY_TEXT_LINE shortcircuit=no autolearn=ham autolearn_force=no version=3.4.6 Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id 44A3A1F51A for ; Wed, 14 Aug 2024 00:16:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=80x24.org; s=selector1; t=1723594606; bh=CvB6OKMke+8dYbfsowusi4QnvXiD51FwUZTGIZ+AFuE=; h=From:To:Subject:Date:In-Reply-To:References:From; b=Bh5QPuhFHCbdIpmTiP5R2YpuDMl+gaf+MJxjOKroGPCfTeR7ZuYm3bh4v8La/Yn5L OMdHErWi4Y4IH2ZRW4IralV0l2cZtViKLzwh+7YE19dcRtBVikezRztugajrXWHlum 443BTMSu5wnmLqfSUS69OogYbzxfEzBwhLzXTHLs= From: Eric Wong To: meta@public-inbox.org Subject: [PATCH 4/5] lei_search: make missing Xapian docs for kw lookups Date: Wed, 14 Aug 2024 00:16:44 +0000 Message-Id: <20240814001645.395576-5-e@80x24.org> In-Reply-To: <20240814001645.395576-1-e@80x24.org> References: <20240814001645.395576-1-e@80x24.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit List-Id: Missing keyword entries should be non-fatal since Xapian data is always less important than what's in git and SQLite. As such, Xapian data has and remains written last, leaving the possibility of documents being missing from Xapian but present in SQLite and git. This improves recovery dealing with badly interrupted or failed imports due to bugs or hardware failures. --- lib/PublicInbox/LeiSearch.pm | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/lib/PublicInbox/LeiSearch.pm b/lib/PublicInbox/LeiSearch.pm index 684668c5..4519306d 100644 --- a/lib/PublicInbox/LeiSearch.pm +++ b/lib/PublicInbox/LeiSearch.pm @@ -145,7 +145,15 @@ sub kw_changed { } if (!defined($cur_kw) && $@) { $docids = join(', num:', @$docids); - croak "E: num:$docids keyword lookup failure: $@"; + # this may happen if a previous import was incomplete since + # we commit changes to Xapian last + if (ref($@) =~ /::DocNotFoundError\b/) { + warn <