From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: ludo@gnu.org (Ludovic =?UTF-8?Q?Court=C3=A8s?=) Newsgroups: gmane.lisp.guile.bugs Subject: bug#21093: Web server: response bodies systematically loaded in memory Date: Fri, 24 Jun 2016 10:58:21 +0200 Message-ID: <87r3bn7yky.fsf__1484.19924448735$1466758777$gmane$org@gnu.org> References: <87a8uriysr.fsf@gnu.org> <874m8jhuj7.fsf@pobox.com> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Trace: ger.gmane.org 1466758777 5960 80.91.229.3 (24 Jun 2016 08:59:37 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Fri, 24 Jun 2016 08:59:37 +0000 (UTC) Cc: 21093@debbugs.gnu.org, guile-devel@gnu.org To: Andy Wingo Original-X-From: bug-guile-bounces+guile-bugs=m.gmane.org@gnu.org Fri Jun 24 10:59:20 2016 Return-path: Envelope-to: guile-bugs@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1bGMxG-00082u-SJ for guile-bugs@m.gmane.org; Fri, 24 Jun 2016 10:59:15 +0200 Original-Received: from localhost ([::1]:42070 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bGMxG-00005Z-3O for guile-bugs@m.gmane.org; Fri, 24 Jun 2016 04:59:14 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:47613) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bGMx9-00005P-Kt for bug-guile@gnu.org; Fri, 24 Jun 2016 04:59:08 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bGMx4-0005aI-C9 for bug-guile@gnu.org; Fri, 24 Jun 2016 04:59:06 -0400 Original-Received: from debbugs.gnu.org ([208.118.235.43]:41067) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bGMx4-0005aD-8k for bug-guile@gnu.org; Fri, 24 Jun 2016 04:59:02 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1bGMx4-0002ir-3t for bug-guile@gnu.org; Fri, 24 Jun 2016 04:59:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: ludo@gnu.org (Ludovic =?UTF-8?Q?Court=C3=A8s?=) Original-Sender: "Debbugs-submit" Resent-CC: bug-guile@gnu.org Resent-Date: Fri, 24 Jun 2016 08:59:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 21093 X-GNU-PR-Package: guile X-GNU-PR-Keywords: Original-Received: via spool by 21093-submit@debbugs.gnu.org id=B21093.146675872110436 (code B ref 21093); Fri, 24 Jun 2016 08:59:02 +0000 Original-Received: (at 21093) by debbugs.gnu.org; 24 Jun 2016 08:58:41 +0000 Original-Received: from localhost ([127.0.0.1]:53404 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bGMwj-0002iG-2G for submit@debbugs.gnu.org; Fri, 24 Jun 2016 04:58:41 -0400 Original-Received: from eggs.gnu.org ([208.118.235.92]:56386) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bGMwh-0002i2-PO for 21093@debbugs.gnu.org; Fri, 24 Jun 2016 04:58:40 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bGMwb-0005M2-PL for 21093@debbugs.gnu.org; Fri, 24 Jun 2016 04:58:34 -0400 Original-Received: from fencepost.gnu.org ([2001:4830:134:3::e]:55297) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bGMwS-0005I5-Sa; Fri, 24 Jun 2016 04:58:24 -0400 Original-Received: from pluto.bordeaux.inria.fr ([193.50.110.57]:47830 helo=pluto) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.82) (envelope-from ) id 1bGMwS-0005Mq-30; Fri, 24 Jun 2016 04:58:24 -0400 X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 7 Messidor an 224 de la =?UTF-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-unknown-linux-gnu In-Reply-To: <874m8jhuj7.fsf@pobox.com> (Andy Wingo's message of "Fri, 24 Jun 2016 10:15:40 +0200") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.5 (gnu/linux) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 208.118.235.43 X-BeenThere: bug-guile@gnu.org List-Id: "Bug reports for GUILE, GNU's Ubiquitous Extension Language" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-guile-bounces+guile-bugs=m.gmane.org@gnu.org Original-Sender: "bug-guile" Xref: news.gmane.org gmane.lisp.guile.bugs:8215 Archived-At: Andy Wingo skribis: > On Mon 20 Jul 2015 00:10, ludo@gnu.org (Ludovic Court=C3=A8s) writes: > >> The =E2=80=98sanitize-response=E2=80=99 procedure systematically loads t= he whole >> response body in memory, which causes obvious scalability issues (this >> is in 2.0.11.) >> >> In particular, when a request handler returns a port-taking procedure as >> its second return value, =E2=80=98sanitize-request=E2=80=99 will just ca= ll that >> procedure passing it a string output port. >> >> This procedure should instead be called from the server implementation= =E2=80=99s >> =E2=80=98write=E2=80=99 hook, but that would necessitate an API change. >> >> Thoughts? > > How would you set the Content-Length? Just leave it off and set > Connection: close ? Set it in the headers perhaps? Then you have to > verify later, which I dunno how nice that is. Maybe it is OK. I think it could work this way: 1. By default, provide no =E2=80=98Content-Length=E2=80=99 and force chun= ked encoding (so that the recipient can make sure it received everything.) This is useful for data generated on the fly. 2. Provide an optional mechanism allowing the user to specify the content length upfront. Useful for statically-generated data that cannot fit in memory. I haven=E2=80=99t thought yet about the actual API (I=E2=80=99ll be happy t= o do so as time permits; to be clear, I don=E2=80=99t consider it a 2.0.12 blocker.) Ludo=E2=80=99.