bug#13857: Unhandled case in module/web/response.scm

[Jason Earl <jearl@notengoamigos.org>]

On Sat, Mar 02 2013, Daniel Hartwig wrote:

> Hello
>
> Which version of guile are you using?  Is it from recent git?  There
> are perhaps related fixes to the web modules made since January.

Dang it.  I knew that there was stuff that I had forgotten to mention.
I am using the head of the stable-2.0 branch from git.  I would be happy
to use the master branch if it makes a difference, but I would have to
update libgc on the machines that I hack on.  However, the diff between
the web module in the stable-2.0 branch and master is pretty small.  I
am pretty confident that it would fail in a similar manner.

I actually have had this problem since before January, and I switched to
using guile straight out of git because I was hoping the fixes that Andy
made would solve my problem.  I considered filing a bug then, but I sort
of wanted to learn how to use the guile debugging tools.

> On 2 March 2013 15:21, Jason Earl <jearl@notengoamigos.org> wrote:
>>
>> response.scm does not seem to handle the case where the server does not
>> specify a content length.  Here's a minimal example that should work,
>> but doesn't:
>
> For non-chunked responses, Content-Length is _almost_ always present.
>
>>
>> --8<---------------cut here---------------start------------->8---
>> #!/usr/local/bin/guile -s
>> !#
>>
>> (use-modules (srfi srfi-8)
>>              ((web uri)    #:select (string->uri))
>>              ((web client) #:select (http-get)))
>>
>> (receive (res-headers res-body)
>>     (http-get (string->uri "http://www.blogger.com/feeds/4777343509834060826/posts/default"))
>>   (display res-body)
>>   (newline))
>> --8<---------------cut here---------------end--------------->8---
>
> On my testing, this server is using chunked transfer encoding in the
> response, and your patch should have no effect on that?

Did you actually try using guile as the web client?  I am just asking
because when I tested using Firefox with live http headers I would get
different headers than when I used guile.

Here's the relevant part of the trace I get when using the head of
stable-2.0:

--8<---------------cut here---------------start------------->8---
trace: |  |  (read-response-body #<<response> version: (1 . 1) code: 200 reason-phrase: …>)
trace: |  |  |  (response-body-port #<<response> version: (1 . 1) code: 200 reason-phra…> …)
trace: |  |  |  |  (response-transfer-encoding #<<response> version: (1 . 1) code: 200 r…>)
trace: |  |  |  |  |  (assq transfer-encoding ((p3p . "CP=\"This is not a P3P policy…") …))
trace: |  |  |  |  |  #f
trace: |  |  |  |  ()
trace: |  |  |  |  (member (chunked) ())
trace: |  |  |  |  #f
trace: |  |  |  |  (response-content-length #<<response> version: (1 . 1) code: 200 reas…>)
trace: |  |  |  |  |  (assq content-length ((p3p . "CP=\"This is not a P3P policy! S…") …))
trace: |  |  |  |  |  #f
trace: |  |  |  |  #f
trace: |  |  |  #f
trace: |  |  |  (and=> #f #<procedure get-bytevector-all (_)>)
trace: |  |  |  #f
trace: |  |  |  (eof-object? #f)
trace: |  |  |  #f
trace: |  |  #f
--8<---------------cut here---------------end--------------->8---

And here's the headers that I get when I connect with guile:

--8<---------------cut here---------------start------------->8---
#<<response> version: (1 . 1) code: 200 reason-phrase: "OK" headers: ((p3p . "CP=\"This is not a P3P policy! See http://www.google.com/support/accounts/bin/answer.py?hl=en&answer=151657 for more info.\"") (content-type application/atom+xml (charset . "UTF-8")) (expires . #<date nanosecond: 0 second: 40 minute: 50 hour: 4 day: 3 month: 3 year: 2013 zone-offset: 0>) (date . #<date nanosecond: 0 second: 40 minute: 50 hour: 4 day: 3 month: 3 year: 2013 zone-offset: 0>) (cache-control private (max-age . 0)) (last-modified . #<date nanosecond: 0 second: 43 minute: 58 hour: 23 day: 19 month: 2 year: 2013 zone-offset: 0>) (etag "AkQGQnoyfil7ImA9WhBSE0w." . #f) (x-content-type-options . "nosniff") (x-xss-protection . "1; mode=block") (server . "GSE") (connection close)) port: #<closed: file 0>>
--8<---------------cut here---------------end--------------->8---

>> Now the reason that I started experimenting with guile in the first
>> place was that I wanted to learn more about scheme, and fixing this
>> seemed like a good opportunity at a practical application of my basic
>> scheme skills.
>>
>> So I did a little debugging and created this little patch that fixes
>> this issue.

[...]

>> Poking at this issue has been quite a bit of fun for me.  So, thanks
>> for all of your hard work on the system.  Now I must admit that I am
>> interested in seeing how (and if) this gets fixed.
>>
>> Jason
>>
>
> Your undelimited port has only one feature on top of a regular port:
> handle #:keep-alive?.  Note that this HTTP option is not usable unless
> the response includes a Content-Length header :-).

Apparently I did not make it clear enough how green I was :).

Seriously though, I appreciate you taking the time to look into this for
me.  If my little test program works for you with *any* version of guile
that you happen to have lying around then there is probably something
wrong at my end.  I would like to think that I have uncovered an actual
bug, but I could easily be wrong.  I tested this on four different
machines, but they are all similar enough that it could still easily be
some configuration issue I have on my end.

If you have already tried the example program that I sent and you get
something besides #f then I am truly sorry to have wasted your precious
time.  If that is the case then I would appreciate hearing about it, of
course.

Jason