From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: X-Spam-Status: No, score=-4.2 required=3.0 tests=ALL_TRUSTED,AWL,BAYES_00, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, T_SCC_BODY_TEXT_LINE shortcircuit=no autolearn=ham autolearn_force=no version=3.4.6 Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id 9A1911F44D for ; Mon, 11 Mar 2024 19:40:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=80x24.org; s=selector1; t=1710186012; bh=VGAVaW5bd0gz9KH0V5ysDG2sYPZB61PAzCWwbBFRCA4=; h=From:To:Subject:Date:From; b=rkffisLxVmalWdJAYov/uiRarDf/+jZv9sxzo5Y+YoN/eFtZwXn+0RSogfYuisX1H dHqPIeKnPLfYAgu2LSgL7v3SMRyMAXfuASiRZMbzkInCwouC7sQsK/GN86Of5n7r18 JNqAylpjbL6YVFac4R0TEsuYMHuoRhIZPkKToxHA= From: Eric Wong To: meta@public-inbox.org Subject: [PATCH 0/4] memory reductions for WWW + solver Date: Mon, 11 Mar 2024 19:40:08 +0000 Message-ID: <20240311194012.1266143-1-e@80x24.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit List-Id: 1/4 gets rid of some overload caused by parallel solver invocations under heavy (likely bot) traffic crawling yhbt.net/lore with many coderepos enabled and joined to inboxes. 2/4 is a large reduction in allocations from loading coderepo <=> inbox associations, 4/4 is smaller. I found 2/4 with Devel::Mwrap and noticed 4/4 while working on 2/4. 3/4 is just a doc update but I've been successfully using jemalloc on my lore+gko mirror for a week or two, now (and I plan to experiment with making glibc||dlmalloc more resistant to fragmentation) Eric Wong (4): www: use a dedicated limiter for blob solver codesearch: deduplicate {ibx_score} name pairs doc: tuning: note reduced fragmentation w/ jemalloc codesearch: deduplicate $git->{nick} field Documentation/public-inbox-tuning.pod | 5 +++ examples/public-inbox-netd@.service | 2 ++ lib/PublicInbox/CodeSearch.pm | 14 ++++++-- lib/PublicInbox/SolverGit.pm | 15 +++++---- lib/PublicInbox/ViewVCS.pm | 48 ++++++++++++++++++++++----- 5 files changed, 66 insertions(+), 18 deletions(-)