From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: AS2116 193.90.0.0/16 X-Spam-Status: No, score=-2.9 required=3.0 tests=AWL,BAYES_00, RCVD_IN_MSPIKE_BL,RCVD_IN_MSPIKE_ZBI,RCVD_IN_XBL,RDNS_NONE,SPF_FAIL, SPF_HELO_FAIL,URIBL_BLOCKED shortcircuit=no autolearn=no autolearn_force=no version=3.4.0 Received: from 80x24.org (unknown [193.90.12.86]) by dcvr.yhbt.net (Postfix) with ESMTP id 6E2331F99C for ; Sat, 21 May 2016 03:03:28 +0000 (UTC) From: Eric Wong To: meta@public-inbox.org Subject: [PATCH 1/2] http: reduce over-buffering for getline responses Date: Sat, 21 May 2016 03:03:16 +0000 Message-Id: <20160521030317.24152-2-e@80x24.org> In-Reply-To: <20160521030317.24152-1-e@80x24.org> References: <20160521030317.24152-1-e@80x24.org> List-Id: By switching to a "pull"-based I/O model for reading application responses, we should be able to throttle buffering to slow clients more effectively and avoid wasting precious RAM. This will also allow us to more Danga::Socket-specific knowledge out of the PSGI application and keep it confined to PublicInbox::HTTP. --- lib/PublicInbox/HTTP.pm | 26 ++++++++++++++++++++++---- 1 file changed, 22 insertions(+), 4 deletions(-) diff --git a/lib/PublicInbox/HTTP.pm b/lib/PublicInbox/HTTP.pm index 1ef3fb3..f69056f 100644 --- a/lib/PublicInbox/HTTP.pm +++ b/lib/PublicInbox/HTTP.pm @@ -205,7 +205,7 @@ sub response_write { if ($alive) { $self->event_write; # watch for readability if done } else { - $self->write(sub { $self->close }); + Danga::Socket::write($self, sub { $self->close }); } if (my $obj = $env->{'pi-httpd.inbox'}) { # grace period for reaping resources @@ -215,9 +215,27 @@ sub response_write { $self->{env} = undef; }; - if (defined $res->[2]) { - Plack::Util::foreach($res->[2], $write); - $close->(); + if (defined(my $body = $res->[2])) { + if (ref $body eq 'ARRAY') { + $write->($_) foreach @$body; + $close->(); + } else { + my $pull; + $pull = sub { + local $/ = \8192; + while (defined(my $buf = $body->getline)) { + $write->($buf); + if ($self->{write_buf}) { + $self->write($pull); + return; + } + } + $pull = undef; + $body->close(); + $close->(); + }; + $pull->(); + } } else { # this is returned to the calling application: Plack::Util::inline_object(write => $write, close => $close); -- EW