From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: X-Spam-Status: No, score=-4.0 required=3.0 tests=ALL_TRUSTED,BAYES_00 shortcircuit=no autolearn=ham autolearn_force=no version=3.4.2 Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id D75AB1FA0A for ; Sun, 10 May 2020 22:37:15 +0000 (UTC) From: Eric Wong To: meta@public-inbox.org Subject: [PATCH 3/5] overidx: document the SQLite PRAGMA we use Date: Sun, 10 May 2020 22:37:13 +0000 Message-Id: <20200510223715.19254-4-e@yhbt.net> In-Reply-To: <20200510223715.19254-1-e@yhbt.net> References: <20200510223715.19254-1-e@yhbt.net> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit List-Id: This ought to prevent cargo-culting the cache_size PRAGMA into smaller SQLite DBs we might use. --- lib/PublicInbox/OverIdx.pm | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/lib/PublicInbox/OverIdx.pm b/lib/PublicInbox/OverIdx.pm index acbf2c8de60..cb15baadf2b 100644 --- a/lib/PublicInbox/OverIdx.pm +++ b/lib/PublicInbox/OverIdx.pm @@ -21,8 +21,16 @@ use PublicInbox::Search; sub dbh_new { my ($self) = @_; my $dbh = $self->SUPER::dbh_new(1); + + # TRUNCATE reduces I/O compared to the default (DELETE) $dbh->do('PRAGMA journal_mode = TRUNCATE'); + + # 80000 pages (80MiB on SQLite <3.12.0, 320MiB on 3.12.0+) + # was found to be good in 2018 during the large LKML import + # at the time. This ought to be configurable based on HW + # and inbox size; I suspect it's overkill for many inboxes. $dbh->do('PRAGMA cache_size = 80000'); + create_tables($dbh); $dbh; }