From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Jason Earl Newsgroups: gmane.lisp.guile.bugs Subject: bug#13857: Unhandled case in module/web/response.scm Date: Sat, 02 Mar 2013 00:21:55 -0700 Message-ID: <878v669sh8.fsf@notengoamigos.org> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Trace: ger.gmane.org 1362240937 1793 80.91.229.3 (2 Mar 2013 16:15:37 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Sat, 2 Mar 2013 16:15:37 +0000 (UTC) To: 13857@debbugs.gnu.org Original-X-From: bug-guile-bounces+guile-bugs=m.gmane.org@gnu.org Sat Mar 02 17:16:01 2013 Return-path: Envelope-to: guile-bugs@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1UBp6O-0003re-Bf for guile-bugs@m.gmane.org; Sat, 02 Mar 2013 17:16:00 +0100 Original-Received: from localhost ([::1]:59002 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UBp62-0004Fr-Sp for guile-bugs@m.gmane.org; Sat, 02 Mar 2013 11:15:38 -0500 Original-Received: from eggs.gnu.org ([208.118.235.92]:45250) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UBhFh-0001x4-7M for bug-guile@gnu.org; Sat, 02 Mar 2013 02:53:07 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1UBhFd-0004M0-AF for bug-guile@gnu.org; Sat, 02 Mar 2013 02:53:05 -0500 Original-Received: from debbugs.gnu.org ([140.186.70.43]:50456) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UBhFd-0004Lv-6x for bug-guile@gnu.org; Sat, 02 Mar 2013 02:53:01 -0500 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.72) (envelope-from ) id 1UBhFe-0004Ly-Lc for bug-guile@gnu.org; Sat, 02 Mar 2013 02:53:02 -0500 X-Loop: help-debbugs@gnu.org Resent-From: Jason Earl Original-Sender: debbugs-submit-bounces@debbugs.gnu.org Resent-CC: bug-guile@gnu.org Resent-Date: Sat, 02 Mar 2013 07:53:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 13857 X-GNU-PR-Package: guile X-GNU-PR-Keywords: X-Debbugs-Original-To: bug-guile@gnu.org Original-Received: via spool by submit@debbugs.gnu.org id=B.136221074716693 (code B ref -1); Sat, 02 Mar 2013 07:53:02 +0000 Original-Received: (at submit) by debbugs.gnu.org; 2 Mar 2013 07:52:27 +0000 Original-Received: from localhost ([127.0.0.1]:54565 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1UBhF4-0004LA-U2 for submit@debbugs.gnu.org; Sat, 02 Mar 2013 02:52:27 -0500 Original-Received: from eggs.gnu.org ([208.118.235.92]:33791) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1UBhF2-0004Ky-Gs for submit@debbugs.gnu.org; Sat, 02 Mar 2013 02:52:25 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1UBhEs-0004GV-1D for submit@debbugs.gnu.org; Sat, 02 Mar 2013 02:52:17 -0500 Original-Received: from lists.gnu.org ([208.118.235.17]:55350) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UBhEr-0004GR-Tn for submit@debbugs.gnu.org; Sat, 02 Mar 2013 02:52:13 -0500 Original-Received: from eggs.gnu.org ([208.118.235.92]:45139) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UBhEo-0001YX-Ui for bug-guile@gnu.org; Sat, 02 Mar 2013 02:52:13 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1UBhEm-0004F7-1o for bug-guile@gnu.org; Sat, 02 Mar 2013 02:52:10 -0500 Original-Received: from mailout04.yourhostingaccount.com ([65.254.253.34]:55579) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UBhEl-0004F1-Tq for bug-guile@gnu.org; Sat, 02 Mar 2013 02:52:07 -0500 Original-Received: from mailscan01.yourhostingaccount.com ([10.1.15.1] helo=mailscan01.yourhostingaccount.com) by mailout04.yourhostingaccount.com with esmtp (Exim) id 1UBglZ-0003Yo-Fc for bug-guile@gnu.org; Sat, 02 Mar 2013 02:21:57 -0500 Original-Received: from impout01.yourhostingaccount.com ([10.1.55.1] helo=impout01.yourhostingaccount.com) by mailscan01.yourhostingaccount.com with esmtp (Exim) id 1UBglZ-0000cn-Gb for bug-guile@gnu.org; Sat, 02 Mar 2013 02:21:57 -0500 Original-Received: from authsmtp16.yourhostingaccount.com ([10.1.18.16]) by impout01.yourhostingaccount.com with NO UCE id 6XMx1l00B0LoD9W01XMxWp; Sat, 02 Mar 2013 02:21:57 -0500 X-Authority-Analysis: v=2.0 cv=EJGEIilC c=1 sm=1 a=iZ6JCboNNYXqehhoEOpvzg==:17 a=yy498_rWaDoA:10 a=DQlLCnKHR5QA:10 a=ftrz5CoAZfYA:10 a=-Fa43yUS0V0A:10 a=Nu5xS2u5AAAA:8 a=In_pQzuoZTgA:10 a=AaSqleDyAAAA:8 a=WMBVOS53j7HJJfUqS6IA:9 a=6z-DOhwwYRTa0kwD8qQA:9 a=QEXdDO2ut3YA:10 a=byfbOFXIYgAA:10 a=v3WRp2jgJj85H8Vg:21 a=NB-hjByzGQ0GwJlb:21 a=rr09IRaFdp_zbtag880A:9 a=ltXMujfcukYoTVgNOHxvog==:117 X-EN-OrigOutIP: 10.1.18.16 X-EN-IMPSID: 6XMx1l00B0LoD9W01XMxWp Original-Received: from [204.228.144.110] (helo=hack) by authsmtp16.yourhostingaccount.com with esmtpsa (TLSv1:AES128-SHA:128) (Exim) id 1UBglZ-00017q-5C for bug-guile@gnu.org; Sat, 02 Mar 2013 02:21:57 -0500 Face: iVBORw0KGgoAAAANSUhEUgAAADAAAAAwBAMAAAClLOS0AAAAAXNSR0IArs4c6QAAABhQTFRF AwMDLi4uU1NTc3NzkJCQurq60NDQ+/v7B9hRZgAAAAFiS0dEAIgFHUgAAAAJcEhZcwAAEpsAABKb AeCSWQwAAAAHdElNRQfZCBsWCSS75/xCAAABhklEQVQ4y42UTVPCMBCGt9UfAAzeYRTO1RHP6Nhy dugmdyCbu9r277vZpKGQMmMO3bbP5GPfdzegkBQRYYWGX1TtPkihAdvJaHXXGeq6Jnw3oCPYL2eb AYgzmglAtutBewZfwONhBCwcyE0CcOJAhgmoQUCZbO4BpCAslQLlgUnzKBy4G0nw4MB6BLScSD4m Sdd+PNkBEJmJTBVi/w0lyqivIg79kCUs51pe+hFA+zzJVhebB+COnekRUHhfEmDAK5CAk4CcEnDw vtwE5tZS5v+b28WN49pvkMJLAUsyXXH56FTEz4ofEFRGllsN5b826hSNcpY+6mCtdks1L69WSvSd 87FcDIUvBlcUcwd+5dzn8pF22PGp5CWPBUc+QZ4hP0D1JRolIfkBb3Xoj2OvrvFg3bdB9ENdgKyM M5RsBlXfatEPvfVTVWhOv+c9n0oWnZ/beevzASn/bBcuAN3YH465dpKclvlmeGXsp7PK++EuF+cH hUtGIiAhGlSGo0EkvpY4KGX+AA6qZu/FF1GEAAAAAElFTkSuQmCC User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.3.50 (gnu/linux) X-EN-UserInfo: f8a5a3c49e1c4664ba81facb1022c4a9:67ddfe7aeaee6d1ea5b788d961d42633 X-EN-AuthUser: jearl@notengoamigos.org X-EN-OrigIP: 204.228.144.110 X-EN-OrigHost: unknown X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.13 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 140.186.70.43 X-Mailman-Approved-At: Sat, 02 Mar 2013 11:15:36 -0500 X-BeenThere: bug-guile@gnu.org List-Id: "Bug reports for GUILE, GNU's Ubiquitous Extension Language" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-guile-bounces+guile-bugs=m.gmane.org@gnu.org Original-Sender: bug-guile-bounces+guile-bugs=m.gmane.org@gnu.org Xref: news.gmane.org gmane.lisp.guile.bugs:6816 Archived-At: --=-=-= Content-Type: text/plain response.scm does not seem to handle the case where the server does not specify a content length. Here's a minimal example that should work, but doesn't: --8<---------------cut here---------------start------------->8--- #!/usr/local/bin/guile -s !# (use-modules (srfi srfi-8) ((web uri) #:select (string->uri)) ((web client) #:select (http-get))) (receive (res-headers res-body) (http-get (string->uri "http://www.blogger.com/feeds/4777343509834060826/posts/default")) (display res-body) (newline)) --8<---------------cut here---------------end--------------->8--- Now the reason that I started experimenting with guile in the first place was that I wanted to learn more about scheme, and fixing this seemed like a good opportunity at a practical application of my basic scheme skills. So I did a little debugging and created this little patch that fixes this issue. --=-=-= Content-Type: text/x-diff; charset=utf-8 Content-Disposition: inline; filename=0001-Handle-case-where-server-does-not-specify-a-Content-.patch Content-Transfer-Encoding: quoted-printable >From 74af03051fec22e01376f3f4597eb8de2dab87b5 Mon Sep 17 00:00:00 2001 From: Jason Douglas Earl Date: Fri, 1 Mar 2013 23:58:41 -0700 Subject: [PATCH] Handle case where server does not specify a Content-Length. * module/web/response.scm (response-body-port, make-undelimited-input-port): The server does not have to return a content length. Handle that case. --- module/web/response.scm | 28 +++++++++++++++++++++++++--- 1 file changed, 25 insertions(+), 3 deletions(-) diff --git a/module/web/response.scm b/module/web/response.scm index 7e14f4d..33dcf1e 100644 --- a/module/web/response.scm +++ b/module/web/response.scm @@ -264,6 +264,26 @@ closes PORT, unless KEEP-ALIVE? is true." =20 (make-custom-binary-input-port "delimited input port" read! #f #f close)) =20 +(define (make-undelimited-input-port port keep-alive?) + "Return an input port that reads from PORT, until EOF. Closing the +returned port closes PORT, unless KEEP-ALIVE? is true." + (define bytes-read 0) + + (define (read! bv start count) + (let ((ret (get-bytevector-n! port bv start count))) + (if (eof-object? ret) + 0 + (begin + (set! bytes-read (+ bytes-read ret)) + ret)))) + + (define close + (and (not keep-alive?) + (lambda () + (close port)))) + + (make-custom-binary-input-port "undelimited input port" read! #f #f clos= e)) + (define* (response-body-port r #:key (decode? #t) (keep-alive? #t)) "Return an input port from which the body of R can be read. The encoding of the returned port is set according to R's =E2=80=98content-typ= e=E2=80=99 @@ -277,9 +297,11 @@ response port." (make-chunked-input-port (response-port r) #:keep-alive? keep-alive?) (let ((len (response-content-length r))) - (and len - (make-delimited-input-port (response-port r) - len keep-alive?))))) + (if len + (make-delimited-input-port (response-port r) + len keep-alive?) + (make-undelimited-input-port (response-port r) + keep-alive?))))) =20 (when (and decode? port) (match (response-content-type r) --=20 1.7.10.4 --=-=-= Content-Type: text/plain With that patch my little test program works. Now, please forgive my ignorance. I probably misunderstand what "delimited" means in this context, and I am probably using it incorrectly. I would be shocked if this even slightly resembles how this should be fixed. I have been lurking on the guile mailing lists for a few months and I don't understand half of what you guys are talking about (which is actually why I am so keen to play with guile). Sharing this patch seemed to be the easiest way to explain what is happening. I am not even going to pretend that I have spent a great deal of time reading the HTTP 1.1 protocol specs, but it does appear that the server does not have to return a Content-Length. I certainly have run across servers that don't. Poking at this issue has been quite a bit of fun for me. So, thanks for all of your hard work on the system. Now I must admit that I am interested in seeing how (and if) this gets fixed. Jason --=-=-=--