From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: X-Spam-Status: No, score=-4.2 required=3.0 tests=ALL_TRUSTED,AWL,BAYES_00, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, T_SCC_BODY_TEXT_LINE shortcircuit=no autolearn=ham autolearn_force=no version=3.4.6 Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id DE6B91F723 for ; Thu, 30 Nov 2023 11:41:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=80x24.org; s=selector1; t=1701344470; bh=EmXOBkZLfuNnifbga1YGWE894on3AnnENe+958eXtlE=; h=From:To:Subject:Date:In-Reply-To:References:From; b=GXOzNP01cmtNoSz6BTPEhal/hVbSHsEGqDUSJenZgTOszyOou0Shu1G9FY98ux066 HVPpXnqG547jEE5JgwyuzBCYDCOay5EqblKtH4UV5UaGmKNLJxnbjFxQlIKN9V/dh6 dhFz/me5SeM+CQUIXZmUMdN5zuEkEh6GEphhBt3E= From: Eric Wong To: meta@public-inbox.org Subject: [PATCH 11/15] inbox: expire resources more aggressively Date: Thu, 30 Nov 2023 11:41:04 +0000 Message-ID: <20231130114109.2577708-12-e@80x24.org> In-Reply-To: <20231130114109.2577708-1-e@80x24.org> References: <20231130114109.2577708-1-e@80x24.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit List-Id: We no longer trigger git cleanups from the Inbox package since `git cat-file' users have their own cleanup to support git coderepos not associated with any inbox. This change means we unconditionally expire SQLite and Xapian FDs and some internal caches regardless of git activity. The old logic was irrelevant to Gcf2 (libgit2) users anyways since we couldn't determine whether or not an inbox was active based on {inflight} git requests, and upcoming changes will make it inaccurate for all extindex/cindex users as well. Opening SQLite and Xapian DBs is fairly cheap; so it's a small price to pay to reduce memory use and fragmentation. --- lib/PublicInbox/Inbox.pm | 24 ++++++++---------------- 1 file changed, 8 insertions(+), 16 deletions(-) diff --git a/lib/PublicInbox/Inbox.pm b/lib/PublicInbox/Inbox.pm index e71ef6d2..7af0ad90 100644 --- a/lib/PublicInbox/Inbox.pm +++ b/lib/PublicInbox/Inbox.pm @@ -12,28 +12,20 @@ use List::Util qw(max); use Carp qw(croak); use PublicInbox::Compat qw(uniqstr); -# returns true if further checking is required +# in case DBs get replaced (Xapcmd does it for v1) sub check_inodes ($) { for (qw(over mm)) { $_[0]->{$_}->check_inodes if $_[0]->{$_} } } +# search/over/mm hold onto FDs and description+cloneurl may get updated. +# creating long-lived allocations in the same phase as short-lived +# allocations also leads to fragmentation, so we don't want some stuff +# living too long. sub do_cleanup { my ($ibx) = @_; - my $live; - if (defined $ibx->{git}) { - $live = $ibx->isa(__PACKAGE__) ? $ibx->{git}->cleanup(1) - : $ibx->{git}->cleanup_if_unlinked; - delete($ibx->{git}) unless $live; - } - if ($live) { - check_inodes($ibx); - } else { - delete(@$ibx{qw(over mm description cloneurl - -imap_url -nntp_url -pop3_url)}); - } - my $srch = $ibx->{search} // $ibx; + my ($srch) = delete @$ibx{qw(search over mm description cloneurl)}; + $srch //= $ibx; # extsearch delete @$srch{qw(xdb qp)}; - PublicInbox::DS::add_uniq_timer($ibx+0, 5, \&do_cleanup, $ibx) if $live; } sub _cleanup_later ($) { @@ -370,7 +362,7 @@ sub unsubscribe_unlock { # called by inotify sub on_unlock { my ($self) = @_; - check_inodes($self); + check_inodes($self); # DB files may be replaced while holding lock my $subs = $self->{unlock_subs} or return; for my $obj (values %$subs) { eval { $obj->on_inbox_unlock($self) };