From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net X-Spam-Level: X-Spam-Status: No, score=-4.0 required=3.0 tests=ALL_TRUSTED,BAYES_00 shortcircuit=no autolearn=ham autolearn_force=no version=3.4.2 Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id 591901F55B; Fri, 15 May 2020 05:23:14 +0000 (UTC) Date: Fri, 15 May 2020 05:23:14 +0000 From: Eric Wong To: Konstantin Ryabitsev Cc: meta@public-inbox.org Subject: Re: how's memory use? May 2020 edition Message-ID: <20200515052314.GA5471@dcvr> References: <20200512083734.GA13688@dcvr> <20200514205748.nsdv444ft4oqndqh@chatter.i7.local> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <20200514205748.nsdv444ft4oqndqh@chatter.i7.local> List-Id: Konstantin Ryabitsev wrote: > On Tue, May 12, 2020 at 08:37:34AM +0000, Eric Wong wrote: > > Hey all, if possible; I'd like to know the memory use of your > > daemons (particularly -httpd), relevant pmap(1) (or equivalent) > > output, and version of public-inbox in use. > > This is on lore.kernel.org. We upgraded to 1.5.0 yesterday, so this is > only after a day of running, but this usually covers a lot of traffic. > We run with -W4, hence 4 different outputs: Thanks for the info! I forgot to ask in the original post, but having an idea of active connections per worker might be useful too. I rarely see more than a few dozen, myself. > # pgrep -f public-inbox-httpd | xargs pmap | grep anon Would've been easier for humans to read the output if each process were individually broken out, but I can figure it out from addresses below :> > 0000000002093000 23568K rw--- [ anon ] OK, that looks like the heap of the master. > 00007f981c6dc000 84K rw--- [ anon ] > 00007f981d2c5000 4K rw--- [ anon ] > 00007f982802f000 20K rw--- [ anon ] > 00007f982824c000 16K rw--- [ anon ] > 00007f982865c000 184K rw--- [ anon ] > 00007f9828da8000 8K rw--- [ anon ] > 00007f9828fc2000 8K rw--- [ anon ] > 00007f9829351000 4K rw--- [ anon ] > 00007f982953f000 160K rw--- [ anon ] > 00007f9829572000 4K rw--- [ anon ] > 00007f9829575000 4K rw--- [ anon ] > 00007fffddbe2000 8K r-x-- [ anon ] > ffffffffff600000 4K r-x-- [ anon ] Probably somethings used by glibc internally, or maybe SQLite, Xapian. Good thing is the above mappings now get shared with children and are copy-on-write OK, onto another process: > 0000000002093000 23568K rw--- [ anon ] That looks inherited with the parent. > 0000000003797000 235060K rw--- [ anon ] Ah, so that's probably the main heap after forking (I forget my 64-bit process uses -W0, so no workers in that setup). Anyways, 250-300MB seems a lot better than things were for lore few months ago (closer to ~1G per worker, IIRC?). I've still got a some pure Perl ideas (and plenty with Inline::C), though I'll probably prioritize other things, first, such as IMAP. > 00007f981a8fd000 2736K rw--- [ anon ] Not sure where the above comes from, but it's an odd allocation that seems to get pulled in by most other workers, independently. > 00007f981c6dc000 84K rw--- [ anon ] > 00007f981d2c5000 4K rw--- [ anon ] > 00007f982802f000 20K rw--- [ anon ] > 00007f982824c000 16K rw--- [ anon ] > 00007f982865c000 184K rw--- [ anon ] > 00007f9828da8000 8K rw--- [ anon ] > 00007f9828fc2000 8K rw--- [ anon ] > 00007f9829351000 4K rw--- [ anon ] > 00007f982953f000 160K rw--- [ anon ] > 00007f9829572000 4K rw--- [ anon ] > 00007f9829575000 4K rw--- [ anon ] > 00007fffddbe2000 8K r-x-- [ anon ] > ffffffffff600000 4K r-x-- [ anon ] That all looks shared from the parent, good. > 0000000002093000 23568K rw--- [ anon ] > 0000000003797000 216568K rw--- [ anon ] OK, similar to the other worker. > 00007f98196cc000 4724K rw--- [ anon ] > 00007f9819b69000 4876K rw--- [ anon ] > 00007f981a02c000 2736K rw--- [ anon ] > 00007f981aeb5000 3496K rw--- [ anon ] > 00007f981b350000 1628K rw--- [ anon ] Weird, going to need to source dive into other dependencies to figure this out, but also not a lot compared to the main 200MB+ heap, either. I've got some odd ones like those, too, and they seem to persist... > 0000000002093000 23568K rw--- [ anon ] > 0000000003797000 241724K rw--- [ anon ] > 00007f981a8e2000 2736K rw--- [ anon ] > 00007f981b16f000 1964K rw--- [ anon ] > 00007f981b35a000 1300K rw--- [ anon ] Ditto for these mysterious allocations > 0000000002093000 23568K rw--- [ anon ] > 0000000003797000 202632K rw--- [ anon ] Probably the least busy process, and no odd >1MB mappings. Anyways, things seem looking much better than they were in the past. Regexp matching and split() for MIME is still a problem, and some lists like linux-mtd having some giant multi-MB spam that gets crawled... Thanks again for the info!