Using guile's web client

Using guile's web client

[Chris Vine]

Hi,

I have been playing around with guile-2.0.5's high-level web modules.
In order to see if I understood them, I tried about the simplest thing
I could think of: a GET request to checkip.dyndns.com to obtain my IP
address.

The following code works correctly provided I leave the commented out
code commented out:

(use-modules
 (web uri)
 (web response)
 (rnrs bytevectors)
 (web client))

((lambda()
   (let ((uri (build-uri 'http
                         #:host "checkip.dyndns.com"
                         #:port 80
                         #:path "/")))
     (display uri)
     (newline)
     (let ((r (http-get uri #:keep-alive? #t)))
       (if r
           (begin
             (display r)(newline)
;;           (display (read-response-body r))(newline)
             )
           (begin
             (display "Can't obtain response body")
             (newline)))))))

It outputs:

#<<uri> scheme: http userinfo: #f host: "checkip.dyndns.com" port: 80 path: "/" query: #f fragment: #f>
#<<response> version: (1 . 1) code: 200 reason-phrase: "OK" headers: ((content-type text/html) (server . "DynDNS-CheckIP/1.0") (connection close) (cache-control no-cache) (pragma no-cache) (content-length . 105)) port: #<input-output: socket 6>>

The contents of the response object mirrors the response obtained by
node.js with the equivalent code written in javascript, including the
content length, so the response object appears to be correct.  However,
if I try to obtain the response body by uncommenting the call to
read-response-body, I get an error:

Backtrace:
In ice-9/boot-9.scm:
 149: 10 [catch #t #<catch-closure 81e9280> ...]
 157: 9 [#<procedure 81b38c0 ()>]
In unknown file:
   ?: 8 [catch-closure]
In ice-9/boot-9.scm:
  63: 7 [call-with-prompt prompt0 ...]
In ice-9/eval.scm:
 407: 6 [eval # #]
In ice-9/boot-9.scm:
2111: 5 [save-module-excursion #<procedure 81a1060 at ice-9/boot-9.scm:3646:3 ()>]
3653: 4 [#<procedure 81a1060 at ice-9/boot-9.scm:3646:3 ()>]
In unknown file:
   ?: 3 [load-compiled/vm "/home/chris/.cache/guile/ccache/2.0-LE-4-2.0/home/chris/src/scheme/my-ip-test.scm.go"]
In /home/chris/src/scheme/my-ip-test.scm:
  18: 2 [#<procedure 81ffd90 ()>]
In web/response.scm:
 223: 1 [read-response-body #]
In unknown file:
   ?: 0 [bytevector-length #<eof>]

ERROR: In procedure bytevector-length:
ERROR: In procedure scm_c_bytevector_length: Wrong type argument in position 1 (expecting bytevector): #<eof>

The error is in the procedure call to read-response-body.  Is this a
bug in guile-2, or am I doing something wrong?

Chris

Re: Using guile's web client

[Ian Price]

Chris Vine <chris@cvine.freeserve.co.uk> writes:

> ((lambda()
>    (let ((uri (build-uri 'http
>                          #:host "checkip.dyndns.com"
>                          #:port 80
>                          #:path "/")))
>      (display uri)
>      (newline)
>      (let ((r (http-get uri #:keep-alive? #t)))
^^ there's your problem
>        (if r
>            (begin
>              (display r)(newline)
> ;;           (display (read-response-body r))(newline)
>              )
>            (begin
>              (display "Can't obtain response body")
>              (newline)))))))
snip.

>
> The error is in the procedure call to read-response-body.  Is this a
> bug in guile-2, or am I doing something wrong?
It's actually debatable whether or not this is your fault :). http-get
returns 2 values: the response and the body. When you call
read-response-body on the port, you are trying to read information that
has already been read, and so it is coming up with an (eof-object)
rather than the body. However, I notice that the manual doesn't actually
say that it returns two values, so that'll need to be fixed.

The correct way to write this code snippet is to use call-with-values
(or sugar such as receive or let-values), instead of let.

(let ((uri (build-uri 'http
                      #:host "checkip.dyndns.com"
                      #:port 80
                      #:path "/")))
  (call-with-values
      (lambda ()
        (http-get uri #:keep-alive? #t))
    (lambda (request body)
      ...)))

the request should (I think) always be a request object, so you don't
need to check it with if. Though, the body may be #f, so it's worth
checking that.


-- 
Ian Price

"Programming is like pinball. The reward for doing it well is
the opportunity to do it again" - from "The Wizardy Compiled"

Re: Using guile's web client

[Nala Ginrut]

[-- Attachment #1: Type: text/plain, Size: 3604 bytes --]

I'm afraid it's a real bug.
read-response-body didn't check <eof> before it call bytevector-length.
But actually, all the data has already been read. When it encounters <eof>,
it should return the body immediately. The bug is, it checks
bytevector-length first which cause (bytevector-length <eof>).
I didn't realize this bug before, because I read the body by block which
always constrains the bytevector length within the range of content-length.

But I'm not going to send a patch for this bug. Because I've sent another
patch to rewrite the whole read-response-body which get rid of this bug
already. And it will return the received body data when any error or break
happens. The old one didn't do this job.
Here is the thread:
http://lists.gnu.org/archive/html/guile-devel/2012-03/msg00116.html
What do you think?

Regard.

On Thu, Mar 15, 2012 at 8:05 AM, Chris Vine <chris@cvine.freeserve.co.uk>wrote:

> Hi,
>
> I have been playing around with guile-2.0.5's high-level web modules.
> In order to see if I understood them, I tried about the simplest thing
> I could think of: a GET request to checkip.dyndns.com to obtain my IP
> address.
>
> The following code works correctly provided I leave the commented out
> code commented out:
>
> (use-modules
>  (web uri)
>  (web response)
>  (rnrs bytevectors)
>  (web client))
>
> ((lambda()
>   (let ((uri (build-uri 'http
>                         #:host "checkip.dyndns.com"
>                         #:port 80
>                         #:path "/")))
>     (display uri)
>     (newline)
>     (let ((r (http-get uri #:keep-alive? #t)))
>       (if r
>           (begin
>             (display r)(newline)
> ;;           (display (read-response-body r))(newline)
>             )
>           (begin
>             (display "Can't obtain response body")
>             (newline)))))))
>
> It outputs:
>
> #<<uri> scheme: http userinfo: #f host: "checkip.dyndns.com" port: 80
> path: "/" query: #f fragment: #f>
> #<<response> version: (1 . 1) code: 200 reason-phrase: "OK" headers:
> ((content-type text/html) (server . "DynDNS-CheckIP/1.0") (connection
> close) (cache-control no-cache) (pragma no-cache) (content-length . 105))
> port: #<input-output: socket 6>>
>
> The contents of the response object mirrors the response obtained by
> node.js with the equivalent code written in javascript, including the
> content length, so the response object appears to be correct.  However,
> if I try to obtain the response body by uncommenting the call to
> read-response-body, I get an error:
>
> Backtrace:
> In ice-9/boot-9.scm:
>  149: 10 [catch #t #<catch-closure 81e9280> ...]
>  157: 9 [#<procedure 81b38c0 ()>]
> In unknown file:
>   ?: 8 [catch-closure]
> In ice-9/boot-9.scm:
>  63: 7 [call-with-prompt prompt0 ...]
> In ice-9/eval.scm:
>  407: 6 [eval # #]
> In ice-9/boot-9.scm:
> 2111: 5 [save-module-excursion #<procedure 81a1060 at
> ice-9/boot-9.scm:3646:3 ()>]
> 3653: 4 [#<procedure 81a1060 at ice-9/boot-9.scm:3646:3 ()>]
> In unknown file:
>   ?: 3 [load-compiled/vm
> "/home/chris/.cache/guile/ccache/2.0-LE-4-2.0/home/chris/src/scheme/my-ip-test.scm.go"]
> In /home/chris/src/scheme/my-ip-test.scm:
>  18: 2 [#<procedure 81ffd90 ()>]
> In web/response.scm:
>  223: 1 [read-response-body #]
> In unknown file:
>   ?: 0 [bytevector-length #<eof>]
>
> ERROR: In procedure bytevector-length:
> ERROR: In procedure scm_c_bytevector_length: Wrong type argument in
> position 1 (expecting bytevector): #<eof>
>
> The error is in the procedure call to read-response-body.  Is this a
> bug in guile-2, or am I doing something wrong?
>
> Chris
>
>

[-- Attachment #2: Type: text/html, Size: 4813 bytes --]

Re: Using guile's web client

[Ian Price]

Nala Ginrut <nalaginrut@gmail.com> writes:

> I'm afraid it's a real bug.
No, it's not. See my response to him. :)

-- 
Ian Price

"Programming is like pinball. The reward for doing it well is
the opportunity to do it again" - from "The Wizardy Compiled"

Re: Using guile's web client

[Nala Ginrut]

[-- Attachment #1: Type: text/plain, Size: 532 bytes --]

I must say NO to NO.
You may try http-get directly in this situation and see if it returns. No
matter how you handle the values, it never return because of the error I
mentioned.

On Thu, Mar 15, 2012 at 10:11 AM, Ian Price <ianprice90@googlemail.com>wrote:

> Nala Ginrut <nalaginrut@gmail.com> writes:
>
> > I'm afraid it's a real bug.
> No, it's not. See my response to him. :)
>
> --
> Ian Price
>
> "Programming is like pinball. The reward for doing it well is
> the opportunity to do it again" - from "The Wizardy Compiled"
>

[-- Attachment #2: Type: text/html, Size: 965 bytes --]

Re: Using guile's web client

[Nala Ginrut]

[-- Attachment #1: Type: text/plain, Size: 720 bytes --]

To clarify, he used wrong usage of http-get is another issue need to be
noticed.

On Thu, Mar 15, 2012 at 11:55 AM, Nala Ginrut <nalaginrut@gmail.com> wrote:

> I must say NO to NO.
> You may try http-get directly in this situation and see if it returns. No
> matter how you handle the values, it never return because of the error I
> mentioned.
>
>
> On Thu, Mar 15, 2012 at 10:11 AM, Ian Price <ianprice90@googlemail.com>wrote:
>
>> Nala Ginrut <nalaginrut@gmail.com> writes:
>>
>> > I'm afraid it's a real bug.
>> No, it's not. See my response to him. :)
>>
>> --
>> Ian Price
>>
>> "Programming is like pinball. The reward for doing it well is
>> the opportunity to do it again" - from "The Wizardy Compiled"
>>
>
>

[-- Attachment #2: Type: text/html, Size: 1390 bytes --]

Re: Using guile's web client

[Chris Vine]

On Thu, 15 Mar 2012 01:14:31 +0000
Ian Price <ianprice90@googlemail.com> wrote:
[snip]
> It's actually debatable whether or not this is your fault :). http-get
> returns 2 values: the response and the body. When you call
> read-response-body on the port, you are trying to read information
> that has already been read, and so it is coming up with an
> (eof-object) rather than the body. However, I notice that the manual
> doesn't actually say that it returns two values, so that'll need to
> be fixed.

Ah thanks, that's great.  My "fault" was in assuming that in the absence
of documentation, the http-get procedure does what is "obvious" (ie what
other similar high-level http libraries do) and leaves the reading of
the body to read-response-body.  I have now looked at the source for
the web module and that reveals all.

It also follows then that I don't need the keep-alive on the http-get
procedure call.

A version of read-response-body which reads what's available would be
handy, so that data could be read as it comes in if you are using guile
with an event loop, say via guile's select procedure (I couldn't see
one but I may well have missed it). That is obviously irrelevant to my
toy trial.

Chris

Re: Using guile's web client

[Ian Price]

Chris Vine <chris@cvine.freeserve.co.uk> writes:

> Ah thanks, that's great.  My "fault" was in assuming that in the absence
> of documentation, the http-get procedure does what is "obvious" (ie what
> other similar high-level http libraries do) and leaves the reading of
> the body to read-response-body.  I have now looked at the source for
> the web module and that reveals all.
Well, when in doubt, I test it out at the repl. This would have
clarified things immediately :)

-- 
Ian Price

"Programming is like pinball. The reward for doing it well is
the opportunity to do it again" - from "The Wizardy Compiled"