From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: X-Spam-Status: No, score=-4.0 required=3.0 tests=ALL_TRUSTED,BAYES_00 shortcircuit=no autolearn=ham autolearn_force=no version=3.4.0 Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id 127BB1FAF4 for ; Wed, 28 Feb 2018 23:42:09 +0000 (UTC) From: "Eric Wong (Contractor, The Linux Foundation)" To: meta@public-inbox.org Subject: [PATCH 17/21] v2writable: commit to skeleton via remote partitions Date: Wed, 28 Feb 2018 23:41:58 +0000 Message-Id: <20180228234202.8839-18-e@80x24.org> In-Reply-To: <20180228234202.8839-1-e@80x24.org> References: <20180228234202.8839-1-e@80x24.org> List-Id: We need to ensure Xapian transaction commits are made to remote partitions before associated commits hit the skeleton DB. This causes unnecessary commits to be made to the skeleton DB; but they're mostly harmless. Further work will be necessary to ensure proper ordering and avoidance of unnecessary commits. --- lib/PublicInbox/SearchIdxPart.pm | 1 + lib/PublicInbox/V2Writable.pm | 13 ++++++------- 2 files changed, 7 insertions(+), 7 deletions(-) diff --git a/lib/PublicInbox/SearchIdxPart.pm b/lib/PublicInbox/SearchIdxPart.pm index 6025fc4..2f577ec 100644 --- a/lib/PublicInbox/SearchIdxPart.pm +++ b/lib/PublicInbox/SearchIdxPart.pm @@ -45,6 +45,7 @@ sub partition_worker_loop ($$$) { if ($line eq "commit\n") { $xdb->commit_transaction if $txn; $txn = undef; + $self->{skeleton}->remote_commit; } elsif ($line eq "close\n") { $self->_xdb_release; $xdb = $txn = undef; diff --git a/lib/PublicInbox/V2Writable.pm b/lib/PublicInbox/V2Writable.pm index ff3b657..73110ff 100644 --- a/lib/PublicInbox/V2Writable.pm +++ b/lib/PublicInbox/V2Writable.pm @@ -131,22 +131,21 @@ sub searchidx_checkpoint { # order matters, we can only close {skel} after all partitions # are done because the partitions also write to {skel} - if (my $parts = $self->{idx_parts}) { foreach my $idx (@$parts) { - $idx->remote_commit; + $idx->remote_commit; # propagates commit to skel $idx->remote_close unless $more; } delete $self->{idx_parts} unless $more; } if (my $skel = $self->{skel}) { - $skel->{mm}->{dbh}->commit; + my $dbh = $skel->{mm}->{dbh}; + $dbh->commit; if ($more) { - $skel->{mm}->{dbh}->begin_work; - } - $skel->remote_commit; - unless ($more) { + $dbh->begin_work; + } else { + $skel->remote_commit; # XXX should be unnecessary... $skel->remote_close; delete $self->{skel}; } -- EW