From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <e@80x24.org>
X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net
X-Spam-Level: 
X-Spam-ASN:  
X-Spam-Status: No, score=-4.2 required=3.0 tests=ALL_TRUSTED,BAYES_00,
	DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,
	T_SCC_BODY_TEXT_LINE shortcircuit=no autolearn=ham autolearn_force=no
	version=3.4.2
Received: from localhost (dcvr.yhbt.net [127.0.0.1])
	by dcvr.yhbt.net (Postfix) with ESMTP id 04D7A1F54E
	for <meta@public-inbox.org>; Thu, 18 Aug 2022 07:22:47 +0000 (UTC)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=80x24.org;
	s=selector1; t=1660807367;
	bh=IgvD/xGxdLbQ/3TwVXPPMtY43oaejhkWJn4Cs9E//qY=;
	h=Date:From:To:Subject:References:In-Reply-To:From;
	b=IakObESSei2MlmgZpptqxutdhZzwwKVkr2cQmIq4C9gEgD2AzR3XdOxpyoHvxx8UR
	 siLA3W+/Y6mFR+TYgZT+h3/R8YnMsyBBbWGPuiG6emnFEKoE1K6CJx+blU149LqVhd
	 eJ3avlrLqFdxPZ06n+P+zOwIJSGTM92SaIgBLSGY=
Date: Thu, 18 Aug 2022 07:22:46 +0000
From: Eric Wong <e@80x24.org>
To: meta@public-inbox.org
Subject: Re: [PATCH 4/4] lei reindex: new command to reindex lei/store
Message-ID: <20220818072246.GA28873@dcvr>
References: <20220817093317.3820774-1-e@80x24.org>
 <20220817093317.3820774-5-e@80x24.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <20220817093317.3820774-5-e@80x24.org>
List-Id: <meta.public-inbox.org>

Eric Wong <e@80x24.org> wrote:
> index d49746cb..277ed6bd 100644
> --- a/lib/PublicInbox/LeiStore.pm
> +++ b/lib/PublicInbox/LeiStore.pm
> @@ -335,6 +335,36 @@ sub _docids_and_maybe_kw ($$) {
>  	($docids, [ sort keys %$kw ]);
>  }
>  
> +sub _reindex_1 { # git->cat_async callback
> +	my ($bref, $hex, $type, $size, $smsg) = @_;
> +	my ($self, $eidx, $tl) = delete @$smsg{qw(-self -eidx -tl)};
> +	$bref //= _lms_rw($self)->local_blob($hex, 1);
> +	if ($bref) {
> +		my $eml = PublicInbox::Eml->new($bref);
> +		$smsg->{-merge_vmd} = 1; # preserve existing keywords
> +		$eidx->idx_shard($smsg->{num})->index_eml($eml, $smsg);
> +	} else {
> +		warn("E: $type $hex\n");

This path has been worrying me a bit, I hit it quite a bit on
one of my systems since there was a time when external-only
messages were fully-indexed inside lei/store.  Nowadays,
duplicate indexing is avoided...