* [PATCH 0/3] limiter updates
@ 2025-01-28 8:31 Eric Wong
2025-01-28 8:31 ` [PATCH 1/3] limiter: ignore unparseable rlimit Eric Wong
` (2 more replies)
0 siblings, 3 replies; 4+ messages in thread
From: Eric Wong @ 2025-01-28 8:31 UTC (permalink / raw)
To: meta
2/3 should make managing a bunch of inboxes easier by easily
allowing them to share a common limiter (instead of having to
assign a named limiter to each inbox).
1/3 and 3/3 are just minor fixes and cleanups.
Eric Wong (3):
limiter: ignore unparseable rlimit
www: configurable "-httpbackend" named limiter
git_http_backend: use default limiter from Qspawn
Documentation/public-inbox-config.pod | 7 +++++--
lib/PublicInbox/Cgit.pm | 2 +-
lib/PublicInbox/GitHTTPBackend.pm | 14 ++++++--------
lib/PublicInbox/Limiter.pm | 3 ++-
lib/PublicInbox/WWW.pm | 4 ++--
lib/PublicInbox/WwwCoderepo.pm | 3 ++-
6 files changed, 18 insertions(+), 15 deletions(-)
^ permalink raw reply [flat|nested] 4+ messages in thread
* [PATCH 1/3] limiter: ignore unparseable rlimit
2025-01-28 8:31 [PATCH 0/3] limiter updates Eric Wong
@ 2025-01-28 8:31 ` Eric Wong
2025-01-28 8:31 ` [PATCH 2/3] www: configurable "-httpbackend" named limiter Eric Wong
2025-01-28 8:31 ` [PATCH 3/3] git_http_backend: use default limiter from Qspawn Eric Wong
2 siblings, 0 replies; 4+ messages in thread
From: Eric Wong @ 2025-01-28 8:31 UTC (permalink / raw)
To: meta
Instead of blindly accepting whatever bogus values the config
gave us, ignore it and use the `W:' prefix to be consistent
with the rest of our codebase.
---
lib/PublicInbox/Limiter.pm | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/lib/PublicInbox/Limiter.pm b/lib/PublicInbox/Limiter.pm
index a8d08fc3..b515de98 100644
--- a/lib/PublicInbox/Limiter.pm
+++ b/lib/PublicInbox/Limiter.pm
@@ -29,7 +29,8 @@ sub setup_rlimit {
if (scalar(@rlimit) == 1) {
push @rlimit, $rlimit[0];
} elsif (scalar(@rlimit) != 2) {
- warn "could not parse $k: $v\n";
+ warn "W: could not parse $k: $v (ignored)\n";
+ next;
}
my $inf = $v =~ /\binfinity\b/i ?
$PublicInbox::Spawn::RLIMITS{RLIM_INFINITY} // eval {
^ permalink raw reply related [flat|nested] 4+ messages in thread
* [PATCH 2/3] www: configurable "-httpbackend" named limiter
2025-01-28 8:31 [PATCH 0/3] limiter updates Eric Wong
2025-01-28 8:31 ` [PATCH 1/3] limiter: ignore unparseable rlimit Eric Wong
@ 2025-01-28 8:31 ` Eric Wong
2025-01-28 8:31 ` [PATCH 3/3] git_http_backend: use default limiter from Qspawn Eric Wong
2 siblings, 0 replies; 4+ messages in thread
From: Eric Wong @ 2025-01-28 8:31 UTC (permalink / raw)
To: meta
The default limiter being hard-coded to 32 is excessive for
smaller systems. With dozens or hundreds of inboxes,
configuring all inboxes to point to a user-defined limiter
is tedious, as well. So give WWW admins the opportunity
to configure the limiter for all git-http-backend(1) processes
with one `publicinbox.-httpbackend.max' knob.
---
Documentation/public-inbox-config.pod | 7 +++++--
lib/PublicInbox/Cgit.pm | 2 +-
lib/PublicInbox/GitHTTPBackend.pm | 12 +++++++-----
lib/PublicInbox/WWW.pm | 4 ++--
lib/PublicInbox/WwwCoderepo.pm | 3 ++-
5 files changed, 17 insertions(+), 11 deletions(-)
diff --git a/Documentation/public-inbox-config.pod b/Documentation/public-inbox-config.pod
index 6c4d633a..6f8ec1f3 100644
--- a/Documentation/public-inbox-config.pod
+++ b/Documentation/public-inbox-config.pod
@@ -561,8 +561,11 @@ C<RLIMIT_*> keys may be set to enforce resource limits for
a particular limiter (L<BSD::Resource(3pm)> is required).
Default named-limiters are prefixed with "-". Currently,
-the "-cgit" named limiter is reserved for instances spawning
-cgit via C<publicinbox.cgitrc>
+the C<-cgit> named limiter is reserved for instances spawning
+cgit via C<publicinbox.cgitrc>. The C<-httpbackend> named
+limiter (in public-inbox 2.0+) governs all L<git-http-backend(1)>
+processes for inboxes and coderepos for a given public-inbox
+config file.
=over 8
diff --git a/lib/PublicInbox/Cgit.pm b/lib/PublicInbox/Cgit.pm
index 78fc9ca0..480d1df6 100644
--- a/lib/PublicInbox/Cgit.pm
+++ b/lib/PublicInbox/Cgit.pm
@@ -93,7 +93,7 @@ sub call {
if ($path_info =~ m!\A/(.+?)/($PublicInbox::GitHTTPBackend::ANY)\z!ox) {
my ($nick, $path) = ($1, $2);
if (my $git = $self->{pi_cfg}->get_coderepo($nick)) {
- return serve($env, $git, $path);
+ return serve($env, $git, $path, $self->{pi_cfg});
}
} elsif ($path_info =~ m!$self->{static}! &&
defined($cgit_data = $self->{cgit_data})) {
diff --git a/lib/PublicInbox/GitHTTPBackend.pm b/lib/PublicInbox/GitHTTPBackend.pm
index ac610d4b..5dc07143 100644
--- a/lib/PublicInbox/GitHTTPBackend.pm
+++ b/lib/PublicInbox/GitHTTPBackend.pm
@@ -31,13 +31,13 @@ our $ANY = join('|', @binary, @text, 'git-upload-pack');
my $TEXT = join('|', @text);
sub serve {
- my ($env, $git, $path) = @_;
+ my ($env, $git, $path, $pi_cfg) = @_;
# Documentation/technical/http-protocol.txt in git.git
# requires one and exactly one query parameter:
if ($env->{QUERY_STRING} =~ /\Aservice=git-[A-Za-z0-9_]+-pack\z/ ||
$path =~ /\Agit-[A-Za-z0-9_]+-pack\z/) {
- my $ok = serve_smart($env, $git, $path);
+ my $ok = serve_smart($env, $git, $path, $pi_cfg);
return $ok if $ok;
}
@@ -86,8 +86,8 @@ sub ghb_parse_hdr { # header parser for Qspawn
}
# returns undef if 403 so it falls back to dumb HTTP
-sub serve_smart {
- my ($env, $git, $path) = @_;
+sub serve_smart ($$$;$) {
+ my ($env, $git, $path, $pi_cfg) = @_;
my %env = %ENV;
# GIT_COMMITTER_NAME, GIT_COMMITTER_EMAIL
# may be set in the server-process and are passed as-is
@@ -101,7 +101,9 @@ sub serve_smart {
my $val = $env->{$name};
$env{$name} = $val if defined $val;
}
- my $limiter = $git->{-httpbackend_limiter} || $default_limiter;
+ my $limiter = $git->{-httpbackend_limiter} //
+ ($pi_cfg ? $pi_cfg->limiter('-httpbackend') : undef) //
+ $default_limiter;
$env{GIT_HTTP_EXPORT_ALL} = '1';
$env{PATH_TRANSLATED} = "$git->{git_dir}/$path";
my $rdr = input_prepare($env) or return r(500);
diff --git a/lib/PublicInbox/WWW.pm b/lib/PublicInbox/WWW.pm
index 420132d9..05ec2416 100644
--- a/lib/PublicInbox/WWW.pm
+++ b/lib/PublicInbox/WWW.pm
@@ -487,10 +487,10 @@ sub msg_page {
sub serve_git {
my ($ctx, $epoch, $path) = @_;
- my $env = $ctx->{env};
my $ibx = $ctx->{ibx};
my $git = defined $epoch ? $ibx->git_epoch($epoch) : $ibx->git;
- $git ? PublicInbox::GitHTTPBackend::serve($env, $git, $path) : r404();
+ $git ? PublicInbox::GitHTTPBackend::serve($ctx->{env},
+ $git, $path, $ctx->{www}->{pi_cfg}) : r404();
}
sub mbox_results {
diff --git a/lib/PublicInbox/WwwCoderepo.pm b/lib/PublicInbox/WwwCoderepo.pm
index 8be8ada0..fa3c4acf 100644
--- a/lib/PublicInbox/WwwCoderepo.pm
+++ b/lib/PublicInbox/WwwCoderepo.pm
@@ -335,7 +335,8 @@ sub srv { # endpoint called by PublicInbox::WWW
my $pi_cfg = $self->{pi_cfg};
if ($path_info =~ m!\A/(.+?)/($PublicInbox::GitHTTPBackend::ANY)\z!x and
($git = $pi_cfg->get_coderepo($1))) {
- PublicInbox::GitHTTPBackend::serve($ctx->{env},$git,$2);
+ PublicInbox::GitHTTPBackend::serve($ctx->{env},
+ $git, $2, $pi_cfg);
} elsif ($path_info =~ m!\A/(.+?)/\z! and
($ctx->{git} = $pi_cfg->get_coderepo($1))) {
$ctx->{wcr} = $self;
^ permalink raw reply related [flat|nested] 4+ messages in thread
* [PATCH 3/3] git_http_backend: use default limiter from Qspawn
2025-01-28 8:31 [PATCH 0/3] limiter updates Eric Wong
2025-01-28 8:31 ` [PATCH 1/3] limiter: ignore unparseable rlimit Eric Wong
2025-01-28 8:31 ` [PATCH 2/3] www: configurable "-httpbackend" named limiter Eric Wong
@ 2025-01-28 8:31 ` Eric Wong
2 siblings, 0 replies; 4+ messages in thread
From: Eric Wong @ 2025-01-28 8:31 UTC (permalink / raw)
To: meta
There's no reason for us to have another default limiter object
when the one in Qspawn already exists.
---
lib/PublicInbox/GitHTTPBackend.pm | 6 +-----
1 file changed, 1 insertion(+), 5 deletions(-)
diff --git a/lib/PublicInbox/GitHTTPBackend.pm b/lib/PublicInbox/GitHTTPBackend.pm
index 5dc07143..172de2bc 100644
--- a/lib/PublicInbox/GitHTTPBackend.pm
+++ b/lib/PublicInbox/GitHTTPBackend.pm
@@ -15,9 +15,6 @@ use PublicInbox::Tmpfile;
use PublicInbox::WwwStatic qw(r @NO_CACHE);
use Carp ();
-# 32 is same as the git-daemon connection limit
-my $default_limiter = PublicInbox::Limiter->new(32);
-
# n.b. serving "description" and "cloneurl" should be innocuous enough to
# not cause problems. serving "config" might...
my @text = qw[HEAD info/refs info/attributes
@@ -102,8 +99,7 @@ sub serve_smart ($$$;$) {
$env{$name} = $val if defined $val;
}
my $limiter = $git->{-httpbackend_limiter} //
- ($pi_cfg ? $pi_cfg->limiter('-httpbackend') : undef) //
- $default_limiter;
+ ($pi_cfg ? $pi_cfg->limiter('-httpbackend') : undef);
$env{GIT_HTTP_EXPORT_ALL} = '1';
$env{PATH_TRANSLATED} = "$git->{git_dir}/$path";
my $rdr = input_prepare($env) or return r(500);
^ permalink raw reply related [flat|nested] 4+ messages in thread
end of thread, other threads:[~2025-01-28 8:31 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-01-28 8:31 [PATCH 0/3] limiter updates Eric Wong
2025-01-28 8:31 ` [PATCH 1/3] limiter: ignore unparseable rlimit Eric Wong
2025-01-28 8:31 ` [PATCH 2/3] www: configurable "-httpbackend" named limiter Eric Wong
2025-01-28 8:31 ` [PATCH 3/3] git_http_backend: use default limiter from Qspawn Eric Wong
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).