unofficial mirror of meta@public-inbox.org
 help / color / mirror / Atom feed
* [PATCH] dedupe inbox names, coderepo nicks + git dirs
@ 2024-03-04 21:10 Eric Wong
  0 siblings, 0 replies; only message in thread
From: Eric Wong @ 2024-03-04 21:10 UTC (permalink / raw)
  To: meta

Inbox names, coderepo nicks, git_dir values are used heavily
as hash keys by the read-only coderepo WWW pieces.

Relying on CoW for mutable scalars on newer Perl doesn't work
well since CoW for those scalars are limited to 256 CoW references
and blow past that number when mapping thousands of coderepos
and inboxes to each other.  Instead, make the hash key up-front
and get the resulting string to point directly to the pointer
used by the hash key.
---
 This is only one teeny step in reducing memory usage.  It's
 really ridiculous with the coderepo stuff, heavy traffic and
 both jemalloc and glibc seem to struggle.  Oh, and the kernel
 is complaining about tcp_mem sysctl being too low on my 32-bit
 host with a whopping 1G of RAM...

 lib/PublicInbox/Config.pm | 6 ++++--
 lib/PublicInbox/Git.pm    | 3 ++-
 2 files changed, 6 insertions(+), 3 deletions(-)

diff --git a/lib/PublicInbox/Config.pm b/lib/PublicInbox/Config.pm
index 607197f6..d6300610 100644
--- a/lib/PublicInbox/Config.pm
+++ b/lib/PublicInbox/Config.pm
@@ -379,7 +379,8 @@ sub fill_coderepo {
 		$git->{cgit_url} = $cgits = _array($cgits);
 		$self->{"$pfx.cgiturl"} = $cgits;
 	}
-	$git->{nick} = $nick;
+	my %dedupe = ($nick => undef);
+	($git->{nick}) = keys %dedupe;
 	$git;
 }
 
@@ -486,7 +487,8 @@ sub _fill_ibx {
 	}
 
 	return unless valid_foo_name($name, 'publicinbox');
-	$ibx->{name} = $name;
+	my %dedupe = ($name => undef);
+	($ibx->{name}) = keys %dedupe; # used as a key everywhere
 	$ibx->{-pi_cfg} = $self;
 	$ibx = PublicInbox::Inbox->new($ibx);
 	foreach (@{$ibx->{address}}) {
diff --git a/lib/PublicInbox/Git.pm b/lib/PublicInbox/Git.pm
index f125b029..af12f141 100644
--- a/lib/PublicInbox/Git.pm
+++ b/lib/PublicInbox/Git.pm
@@ -96,7 +96,8 @@ sub new {
 	$git_dir =~ tr!/!/!s;
 	chop $git_dir;
 	# may contain {-tmp} field for File::Temp::Dir
-	bless { git_dir => $git_dir }, $class
+	my %dedupe = ($git_dir => undef);
+	bless { git_dir => (keys %dedupe)[0] }, $class
 }
 
 sub git_path ($$) {

^ permalink raw reply related	[flat|nested] only message in thread

only message in thread, other threads:[~2024-03-04 21:10 UTC | newest]

Thread overview: (only message) (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2024-03-04 21:10 [PATCH] dedupe inbox names, coderepo nicks + git dirs Eric Wong

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).