From: Eric Wong <e@80x24.org>
To: meta@public-inbox.org
Subject: fragmentation notes [was: [PATCH 5/5] www: reduce fragmentation ...]
Date: Wed, 12 Jun 2024 11:43:59 +0000 [thread overview]
Message-ID: <20240612114359.M679242@dcvr> (raw)
In-Reply-To: <20240606074416.3900983-6-e@80x24.org>
Eric Wong <e@80x24.org> wrote:
> Preliminary testing shows this appears to reduce RSS by roughly
> 20-40% under both glibc malloc (using a tiny
> MALLOC_MMAP_THRESHOLD_=67000) on 32-bit and jemalloc 5.2.1 on
> 64-bit with standard settings.
This patch and 4fedc5caff06 (gzip_filter: use zlib DEF_MEM_LEVEL for gzip)
seem to be fairly substantial improvement in reducing glibc fragmentation;
but more is being worked on...
MALLOC_MMAP_THRESHOLD_ should probably be 65553 for both 32 and 64-bit.
This comes from the default Linux pipe size (65536) we use for sysread,
the extra NUL byte Perl adds to all scalars, and 16 bytes of glibc
overhead:
65536 + 1 + 16 = 65553
glibc w/o the mmap threshold clamp remains awful out-of-the-box
compared to jemalloc; and git eats my memory in ways unrelated
to malloc :<
prev parent reply other threads:[~2024-06-12 11:43 UTC|newest]
Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-06-06 7:44 [PATCH 0/5] cleanups + posible major fragmentation reduction Eric Wong
2024-06-06 7:44 ` [PATCH 1/5] treewide: use \*STD(IN|OUT|ERR) consistently Eric Wong
2024-06-06 7:44 ` [PATCH 2/5] git: decouple git_version from git_exe Eric Wong
2024-06-06 7:44 ` [PATCH 3/5] test_common: use cached git lookup to reduce stat(2) Eric Wong
2024-06-06 7:44 ` [PATCH 4/5] treewide: use cached git executable lookup Eric Wong
2024-06-06 7:44 ` [PATCH 5/5] www: reduce fragmentation in /t/ and /T/ endpoints Eric Wong
2024-06-12 11:43 ` Eric Wong [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://public-inbox.org/README
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20240612114359.M679242@dcvr \
--to=e@80x24.org \
--cc=meta@public-inbox.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).