From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: X-Spam-Status: No, score=-4.0 required=3.0 tests=ALL_TRUSTED,BAYES_00 shortcircuit=no autolearn=ham autolearn_force=no version=3.4.2 Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id E03CE1F8C8 for ; Tue, 28 Sep 2021 23:11:06 +0000 (UTC) From: Eric Wong To: meta@public-inbox.org Subject: [PATCH 0/3] www: more aggressive cleanup of SQLite DBs Date: Tue, 28 Sep 2021 23:11:03 +0000 Message-Id: <20210928231106.5166-1-e@80x24.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit List-Id: While extindex allows the elimination of per-inbox Xapian DB shards, per-inbox over.sqlite3 and msgmap.sqlite3 are still required, and they can waste system resources when there are hundreds/thousands of inboxes and most of them sit idle.. Since reopening them isn't expensive, close them aggressively to reduce memory pressure and the likelyhood of swapping. We'll also take advantage of the new ->add_uniq_timer API to amortize the cost of cleanups and avoid globals in the inbox class. Eric Wong (3): www: do not bump {over} refcnt on long responses inbox: rewrite cleanup to be more aggressive inbox: drop memoization/preload, cleanup expires caches lib/PublicInbox/Git.pm | 12 ++++--- lib/PublicInbox/GzipFilter.pm | 6 ++++ lib/PublicInbox/Inbox.pm | 60 ++++++++++++++--------------------- lib/PublicInbox/Mbox.pm | 29 +++++++---------- lib/PublicInbox/View.pm | 6 ++-- lib/PublicInbox/WWW.pm | 9 ------ t/nntp.t | 1 - 7 files changed, 52 insertions(+), 71 deletions(-)