From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: X-Spam-Status: No, score=-4.2 required=3.0 tests=ALL_TRUSTED,AWL,BAYES_00, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF shortcircuit=no autolearn=ham autolearn_force=no version=3.4.6 Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id BB2EE1F4D0 for ; Thu, 12 Dec 2024 10:10:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=80x24.org; s=selector1; t=1733998240; bh=Ki3bSlHXROnzc1KEn/ZHJoPNJ2Br7y9KN+hsjGn97R4=; h=From:To:Subject:Date:In-Reply-To:References:From; b=wx+Xww1f1NC0dpnikU9Un2CLjq++gtw75ECfxmzEwpPXG5Getck4xsBKGSQjMtqqe OxEwnosYdVEfXzg3hHDlAlweO32h8HjhvJSlArLhl2lkoOTaq4fdXZEOpFU9a6d6Dy 0fMJI5gQ0JQdaT1DH4kT8nxAWnvWRd0bF6zrhhCY= From: Eric Wong To: meta@public-inbox.org Subject: [PATCH 1/3] searchidx: update_checkpoint: take bytes arg directly Date: Thu, 12 Dec 2024 10:10:38 +0000 Message-ID: <20241212101040.1109774-2-e@80x24.org> In-Reply-To: <20241212101040.1109774-1-e@80x24.org> References: <20241212101040.1109774-1-e@80x24.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit List-Id: Passing $smsg limits flexibility in case we reuse it for deletes and/or commit search. --- lib/PublicInbox/ExtSearchIdx.pm | 2 +- lib/PublicInbox/SearchIdx.pm | 6 +++--- lib/PublicInbox/V2Writable.pm | 2 +- 3 files changed, 5 insertions(+), 5 deletions(-) diff --git a/lib/PublicInbox/ExtSearchIdx.pm b/lib/PublicInbox/ExtSearchIdx.pm index fe2f5d2e..1c2a9a26 100644 --- a/lib/PublicInbox/ExtSearchIdx.pm +++ b/lib/PublicInbox/ExtSearchIdx.pm @@ -125,7 +125,7 @@ sub attach_config { sub check_batch_limit ($) { my ($req) = @_; # set flag for PublicInbox::V2Writable::index_todo: - update_checkpoint $req->{self}, $req->{new_smsg} and + update_checkpoint $req->{self}, $req->{new_smsg}->{bytes} and ${$req->{need_checkpoint}} = 1; } diff --git a/lib/PublicInbox/SearchIdx.pm b/lib/PublicInbox/SearchIdx.pm index 3a85f552..0d9acd20 100644 --- a/lib/PublicInbox/SearchIdx.pm +++ b/lib/PublicInbox/SearchIdx.pm @@ -788,8 +788,8 @@ sub is_bad_blob ($$$$) { } sub update_checkpoint ($$) { - my ($self, $smsg) = @_; - ($self->{transact_bytes} += $smsg->{bytes}) >= $self->{batch_bytes} and + my ($self, $bytes) = @_; + ($self->{transact_bytes} += $bytes) >= $self->{batch_bytes} and return 1; my $now = now; my $next = $self->{next_checkpoint} //= $now + $CHECKPOINT_INTVL; @@ -803,7 +803,7 @@ sub index_both { # git->cat_async callback my $smsg = bless { blob => $oid }, 'PublicInbox::Smsg'; $smsg->set_bytes($$bref, $size); my $self = $sync->{sidx}; - ${$sync->{need_checkpoint}} = 1 if update_checkpoint $self, $smsg; + ${$sync->{need_checkpoint}} = 1 if update_checkpoint $self, $smsg->{bytes}; local $self->{current_info} = "$self->{current_info}: $oid"; my $eml = PublicInbox::Eml->new($bref); $smsg->{num} = index_mm($self, $eml, $oid, $sync) or diff --git a/lib/PublicInbox/V2Writable.pm b/lib/PublicInbox/V2Writable.pm index 5f3bfde5..c894b648 100644 --- a/lib/PublicInbox/V2Writable.pm +++ b/lib/PublicInbox/V2Writable.pm @@ -107,7 +107,7 @@ sub do_idx ($$$) { my $idx = idx_shard($self, $smsg->{num}); $idx->index_eml($eml, $smsg); } - update_checkpoint $self, $smsg; + update_checkpoint $self, $smsg->{bytes}; } # returns undef on duplicate or spam