From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: X-Spam-Status: No, score=-4.2 required=3.0 tests=ALL_TRUSTED,AWL,BAYES_00, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, T_SCC_BODY_TEXT_LINE shortcircuit=no autolearn=ham autolearn_force=no version=3.4.6 Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id B04E31F7BE for ; Wed, 12 Jun 2024 11:43:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=80x24.org; s=selector1; t=1718192639; bh=d/pXyCsTBRro9Ll5hldj/+Wi+C9wsTrYKcVid8fJ5Dc=; h=Date:From:To:Subject:References:In-Reply-To:From; b=Iur9IZ7qq9d4eaz3e+BPJoYE52RWkpJ3/QjIo8Xk5H40JH22jLgRFCBc7k1L+a5Ta cP1+WGhWlxd5pOx1qlwfYZDxO82apgPxTr2d5j4WHYlcroK5EZhaFdHlzTgG7055F7 f+iGbK23udpyHPBvTTwOS+a4WeJ5Y2qSDR0TZRjQ= Date: Wed, 12 Jun 2024 11:43:59 +0000 From: Eric Wong To: meta@public-inbox.org Subject: fragmentation notes [was: [PATCH 5/5] www: reduce fragmentation ...] Message-ID: <20240612114359.M679242@dcvr> References: <20240606074416.3900983-1-e@80x24.org> <20240606074416.3900983-6-e@80x24.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20240606074416.3900983-6-e@80x24.org> List-Id: Eric Wong wrote: > Preliminary testing shows this appears to reduce RSS by roughly > 20-40% under both glibc malloc (using a tiny > MALLOC_MMAP_THRESHOLD_=67000) on 32-bit and jemalloc 5.2.1 on > 64-bit with standard settings. This patch and 4fedc5caff06 (gzip_filter: use zlib DEF_MEM_LEVEL for gzip) seem to be fairly substantial improvement in reducing glibc fragmentation; but more is being worked on... MALLOC_MMAP_THRESHOLD_ should probably be 65553 for both 32 and 64-bit. This comes from the default Linux pipe size (65536) we use for sysread, the extra NUL byte Perl adds to all scalars, and 16 bytes of glibc overhead: 65536 + 1 + 16 = 65553 glibc w/o the mmap threshold clamp remains awful out-of-the-box compared to jemalloc; and git eats my memory in ways unrelated to malloc :<