unofficial mirror of meta@public-inbox.org
 help / color / mirror / Atom feed
* [PATCH 0/3] limiter updates
@ 2025-01-28  8:31 Eric Wong
  2025-01-28  8:31 ` [PATCH 1/3] limiter: ignore unparseable rlimit Eric Wong
                   ` (2 more replies)
  0 siblings, 3 replies; 4+ messages in thread
From: Eric Wong @ 2025-01-28  8:31 UTC (permalink / raw)
  To: meta

2/3 should make managing a bunch of inboxes easier by easily
allowing them to share a common limiter (instead of having to
assign a named limiter to each inbox).

1/3 and 3/3 are just minor fixes and cleanups.

Eric Wong (3):
  limiter: ignore unparseable rlimit
  www: configurable "-httpbackend" named limiter
  git_http_backend: use default limiter from Qspawn

 Documentation/public-inbox-config.pod |  7 +++++--
 lib/PublicInbox/Cgit.pm               |  2 +-
 lib/PublicInbox/GitHTTPBackend.pm     | 14 ++++++--------
 lib/PublicInbox/Limiter.pm            |  3 ++-
 lib/PublicInbox/WWW.pm                |  4 ++--
 lib/PublicInbox/WwwCoderepo.pm        |  3 ++-
 6 files changed, 18 insertions(+), 15 deletions(-)

^ permalink raw reply	[flat|nested] 4+ messages in thread

* [PATCH 1/3] limiter: ignore unparseable rlimit
  2025-01-28  8:31 [PATCH 0/3] limiter updates Eric Wong
@ 2025-01-28  8:31 ` Eric Wong
  2025-01-28  8:31 ` [PATCH 2/3] www: configurable "-httpbackend" named limiter Eric Wong
  2025-01-28  8:31 ` [PATCH 3/3] git_http_backend: use default limiter from Qspawn Eric Wong
  2 siblings, 0 replies; 4+ messages in thread
From: Eric Wong @ 2025-01-28  8:31 UTC (permalink / raw)
  To: meta

Instead of blindly accepting whatever bogus values the config
gave us, ignore it and use the `W:' prefix to be consistent
with the rest of our codebase.
---
 lib/PublicInbox/Limiter.pm | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/lib/PublicInbox/Limiter.pm b/lib/PublicInbox/Limiter.pm
index a8d08fc3..b515de98 100644
--- a/lib/PublicInbox/Limiter.pm
+++ b/lib/PublicInbox/Limiter.pm
@@ -29,7 +29,8 @@ sub setup_rlimit {
 		if (scalar(@rlimit) == 1) {
 			push @rlimit, $rlimit[0];
 		} elsif (scalar(@rlimit) != 2) {
-			warn "could not parse $k: $v\n";
+			warn "W: could not parse $k: $v (ignored)\n";
+			next;
 		}
 		my $inf = $v =~ /\binfinity\b/i ?
 			$PublicInbox::Spawn::RLIMITS{RLIM_INFINITY} // eval {

^ permalink raw reply related	[flat|nested] 4+ messages in thread

* [PATCH 2/3] www: configurable "-httpbackend" named limiter
  2025-01-28  8:31 [PATCH 0/3] limiter updates Eric Wong
  2025-01-28  8:31 ` [PATCH 1/3] limiter: ignore unparseable rlimit Eric Wong
@ 2025-01-28  8:31 ` Eric Wong
  2025-01-28  8:31 ` [PATCH 3/3] git_http_backend: use default limiter from Qspawn Eric Wong
  2 siblings, 0 replies; 4+ messages in thread
From: Eric Wong @ 2025-01-28  8:31 UTC (permalink / raw)
  To: meta

The default limiter being hard-coded to 32 is excessive for
smaller systems.  With dozens or hundreds of inboxes,
configuring all inboxes to point to a user-defined limiter
is tedious, as well.  So give WWW admins the opportunity
to configure the limiter for all git-http-backend(1) processes
with one `publicinbox.-httpbackend.max' knob.
---
 Documentation/public-inbox-config.pod |  7 +++++--
 lib/PublicInbox/Cgit.pm               |  2 +-
 lib/PublicInbox/GitHTTPBackend.pm     | 12 +++++++-----
 lib/PublicInbox/WWW.pm                |  4 ++--
 lib/PublicInbox/WwwCoderepo.pm        |  3 ++-
 5 files changed, 17 insertions(+), 11 deletions(-)

diff --git a/Documentation/public-inbox-config.pod b/Documentation/public-inbox-config.pod
index 6c4d633a..6f8ec1f3 100644
--- a/Documentation/public-inbox-config.pod
+++ b/Documentation/public-inbox-config.pod
@@ -561,8 +561,11 @@ C<RLIMIT_*> keys may be set to enforce resource limits for
 a particular limiter (L<BSD::Resource(3pm)> is required).
 
 Default named-limiters are prefixed with "-".  Currently,
-the "-cgit" named limiter is reserved for instances spawning
-cgit via C<publicinbox.cgitrc>
+the C<-cgit> named limiter is reserved for instances spawning
+cgit via C<publicinbox.cgitrc>.  The C<-httpbackend> named
+limiter (in public-inbox 2.0+) governs all L<git-http-backend(1)>
+processes for inboxes and coderepos for a given public-inbox
+config file.
 
 =over 8
 
diff --git a/lib/PublicInbox/Cgit.pm b/lib/PublicInbox/Cgit.pm
index 78fc9ca0..480d1df6 100644
--- a/lib/PublicInbox/Cgit.pm
+++ b/lib/PublicInbox/Cgit.pm
@@ -93,7 +93,7 @@ sub call {
 	if ($path_info =~ m!\A/(.+?)/($PublicInbox::GitHTTPBackend::ANY)\z!ox) {
 		my ($nick, $path) = ($1, $2);
 		if (my $git = $self->{pi_cfg}->get_coderepo($nick)) {
-			return serve($env, $git, $path);
+			return serve($env, $git, $path, $self->{pi_cfg});
 		}
 	} elsif ($path_info =~ m!$self->{static}! &&
 		 defined($cgit_data = $self->{cgit_data})) {
diff --git a/lib/PublicInbox/GitHTTPBackend.pm b/lib/PublicInbox/GitHTTPBackend.pm
index ac610d4b..5dc07143 100644
--- a/lib/PublicInbox/GitHTTPBackend.pm
+++ b/lib/PublicInbox/GitHTTPBackend.pm
@@ -31,13 +31,13 @@ our $ANY = join('|', @binary, @text, 'git-upload-pack');
 my $TEXT = join('|', @text);
 
 sub serve {
-	my ($env, $git, $path) = @_;
+	my ($env, $git, $path, $pi_cfg) = @_;
 
 	# Documentation/technical/http-protocol.txt in git.git
 	# requires one and exactly one query parameter:
 	if ($env->{QUERY_STRING} =~ /\Aservice=git-[A-Za-z0-9_]+-pack\z/ ||
 				$path =~ /\Agit-[A-Za-z0-9_]+-pack\z/) {
-		my $ok = serve_smart($env, $git, $path);
+		my $ok = serve_smart($env, $git, $path, $pi_cfg);
 		return $ok if $ok;
 	}
 
@@ -86,8 +86,8 @@ sub ghb_parse_hdr { # header parser for Qspawn
 }
 
 # returns undef if 403 so it falls back to dumb HTTP
-sub serve_smart {
-	my ($env, $git, $path) = @_;
+sub serve_smart ($$$;$) {
+	my ($env, $git, $path, $pi_cfg) = @_;
 	my %env = %ENV;
 	# GIT_COMMITTER_NAME, GIT_COMMITTER_EMAIL
 	# may be set in the server-process and are passed as-is
@@ -101,7 +101,9 @@ sub serve_smart {
 		my $val = $env->{$name};
 		$env{$name} = $val if defined $val;
 	}
-	my $limiter = $git->{-httpbackend_limiter} || $default_limiter;
+	my $limiter = $git->{-httpbackend_limiter} //
+		($pi_cfg ? $pi_cfg->limiter('-httpbackend') : undef) //
+		$default_limiter;
 	$env{GIT_HTTP_EXPORT_ALL} = '1';
 	$env{PATH_TRANSLATED} = "$git->{git_dir}/$path";
 	my $rdr = input_prepare($env) or return r(500);
diff --git a/lib/PublicInbox/WWW.pm b/lib/PublicInbox/WWW.pm
index 420132d9..05ec2416 100644
--- a/lib/PublicInbox/WWW.pm
+++ b/lib/PublicInbox/WWW.pm
@@ -487,10 +487,10 @@ sub msg_page {
 
 sub serve_git {
 	my ($ctx, $epoch, $path) = @_;
-	my $env = $ctx->{env};
 	my $ibx = $ctx->{ibx};
 	my $git = defined $epoch ? $ibx->git_epoch($epoch) : $ibx->git;
-	$git ? PublicInbox::GitHTTPBackend::serve($env, $git, $path) : r404();
+	$git ? PublicInbox::GitHTTPBackend::serve($ctx->{env},
+		$git, $path, $ctx->{www}->{pi_cfg}) : r404();
 }
 
 sub mbox_results {
diff --git a/lib/PublicInbox/WwwCoderepo.pm b/lib/PublicInbox/WwwCoderepo.pm
index 8be8ada0..fa3c4acf 100644
--- a/lib/PublicInbox/WwwCoderepo.pm
+++ b/lib/PublicInbox/WwwCoderepo.pm
@@ -335,7 +335,8 @@ sub srv { # endpoint called by PublicInbox::WWW
 	my $pi_cfg = $self->{pi_cfg};
 	if ($path_info =~ m!\A/(.+?)/($PublicInbox::GitHTTPBackend::ANY)\z!x and
 		($git = $pi_cfg->get_coderepo($1))) {
-			PublicInbox::GitHTTPBackend::serve($ctx->{env},$git,$2);
+			PublicInbox::GitHTTPBackend::serve($ctx->{env},
+							$git, $2, $pi_cfg);
 	} elsif ($path_info =~ m!\A/(.+?)/\z! and
 			($ctx->{git} = $pi_cfg->get_coderepo($1))) {
 		$ctx->{wcr} = $self;

^ permalink raw reply related	[flat|nested] 4+ messages in thread

* [PATCH 3/3] git_http_backend: use default limiter from Qspawn
  2025-01-28  8:31 [PATCH 0/3] limiter updates Eric Wong
  2025-01-28  8:31 ` [PATCH 1/3] limiter: ignore unparseable rlimit Eric Wong
  2025-01-28  8:31 ` [PATCH 2/3] www: configurable "-httpbackend" named limiter Eric Wong
@ 2025-01-28  8:31 ` Eric Wong
  2 siblings, 0 replies; 4+ messages in thread
From: Eric Wong @ 2025-01-28  8:31 UTC (permalink / raw)
  To: meta

There's no reason for us to have another default limiter object
when the one in Qspawn already exists.
---
 lib/PublicInbox/GitHTTPBackend.pm | 6 +-----
 1 file changed, 1 insertion(+), 5 deletions(-)

diff --git a/lib/PublicInbox/GitHTTPBackend.pm b/lib/PublicInbox/GitHTTPBackend.pm
index 5dc07143..172de2bc 100644
--- a/lib/PublicInbox/GitHTTPBackend.pm
+++ b/lib/PublicInbox/GitHTTPBackend.pm
@@ -15,9 +15,6 @@ use PublicInbox::Tmpfile;
 use PublicInbox::WwwStatic qw(r @NO_CACHE);
 use Carp ();
 
-# 32 is same as the git-daemon connection limit
-my $default_limiter = PublicInbox::Limiter->new(32);
-
 # n.b. serving "description" and "cloneurl" should be innocuous enough to
 # not cause problems.  serving "config" might...
 my @text = qw[HEAD info/refs info/attributes
@@ -102,8 +99,7 @@ sub serve_smart ($$$;$) {
 		$env{$name} = $val if defined $val;
 	}
 	my $limiter = $git->{-httpbackend_limiter} //
-		($pi_cfg ? $pi_cfg->limiter('-httpbackend') : undef) //
-		$default_limiter;
+		($pi_cfg ? $pi_cfg->limiter('-httpbackend') : undef);
 	$env{GIT_HTTP_EXPORT_ALL} = '1';
 	$env{PATH_TRANSLATED} = "$git->{git_dir}/$path";
 	my $rdr = input_prepare($env) or return r(500);

^ permalink raw reply related	[flat|nested] 4+ messages in thread

end of thread, other threads:[~2025-01-28  8:31 UTC | newest]

Thread overview: 4+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-01-28  8:31 [PATCH 0/3] limiter updates Eric Wong
2025-01-28  8:31 ` [PATCH 1/3] limiter: ignore unparseable rlimit Eric Wong
2025-01-28  8:31 ` [PATCH 2/3] www: configurable "-httpbackend" named limiter Eric Wong
2025-01-28  8:31 ` [PATCH 3/3] git_http_backend: use default limiter from Qspawn Eric Wong

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).