From: Eric Wong <e@80x24.org>
To: meta@public-inbox.org
Subject: Re: httpd memory usage?
Date: Mon, 27 Sep 2021 07:10:10 +0000 [thread overview]
Message-ID: <20210927071010.GA27389@dcvr> (raw)
In-Reply-To: <20210904235305.GA22009@dcvr>
Eric Wong <e@80x24.org> wrote:
> Is it high for anybody else? I'm not seeing it for small
> instances (e.g. the instance that's powering
> public-inbox.org/{git,meta} is fine, but with lots of entries my
> lore mirror @ https://yhbt.net/lore/ the main sbrk heap seems to
> be growing slowly without bounds.
I suspect it's long-lived SQLite connections and associated
caches/etc (and having short-lived allocations mixed in with
long-term ones).
Anyways, I'm testing the below on yhbt.net/lore, but
there's other SQLite knobs worth investigating...
Preloading all DBs isn't realistic, either, due to the
potential number of inboxes.
diff --git a/lib/PublicInbox/Inbox.pm b/lib/PublicInbox/Inbox.pm
index 1d5fc708..8fcf6e81 100644
--- a/lib/PublicInbox/Inbox.pm
+++ b/lib/PublicInbox/Inbox.pm
@@ -43,7 +43,7 @@ sub cleanup_task () {
for my $git (@{$ibx->{-repo_objs}}) {
$again = 1 if $git->cleanup;
}
- check_inodes($ibx);
+ delete @$ibx{qw(over mm)};
$next->{"$ibx"} = $ibx if $again;
}
$CLEANUP = $next;
> I think I'll need to finish porting mwrap over to Perl/XS
> (from Ruby/C)...
Yeah, still on my TODO list. But right now lei IMAP(S) on
high-latency links is a terrible...
next prev parent reply other threads:[~2021-09-27 7:10 UTC|newest]
Thread overview: 17+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-09-04 23:53 httpd memory usage? Eric Wong
2021-09-27 7:10 ` Eric Wong [this message]
2021-10-04 0:07 ` [PATCH 0/2] www: fix ref cycles when threading extindex Eric Wong
2021-10-04 0:07 ` [PATCH 1/2] t/thread-cycle: make Email::Simple optional Eric Wong
2021-10-04 0:07 ` [PATCH 2/2] www: fix ref cycle from threading w/ extindex Eric Wong
2021-10-04 22:51 ` [PATCH 0/2] www: fix ref cycles when threading extindex Eric Wong
2021-10-05 11:33 ` Encode.pm leak Eric Wong
2021-10-12 10:59 ` Encode.pm leak in v2.87..v3.12 Eric Wong
2021-10-13 10:16 ` [PATCH 0/7] workaround Encode leak, several test fixes Eric Wong
2021-10-13 10:16 ` [PATCH 1/7] xt/perf-msgview: drop unnecessary use_ok Eric Wong
2021-10-13 10:16 ` [PATCH 2/7] test_common: hoist out tail_f sub Eric Wong
2021-10-13 10:16 ` [PATCH 3/7] t/www_listing: require opt-in for grokmirror tests Eric Wong
2021-10-13 10:16 ` [PATCH 4/7] eml: avoid Encode 2.87..3.12 leak Eric Wong
2021-10-13 10:16 ` [PATCH 5/7] t/lei-mirror: avoid reading ~/.public-inbox/config in test Eric Wong
2021-10-13 10:16 ` [PATCH 6/7] t/git: avoid "once" warning for async_warn Eric Wong
2021-10-13 10:16 ` [PATCH 7/7] t/nntpd-tls: change diag() to like() assertion Eric Wong
2021-11-04 0:17 ` httpd memory usage? Eric Wong
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://public-inbox.org/README
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210927071010.GA27389@dcvr \
--to=e@80x24.org \
--cc=meta@public-inbox.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).