is (web client) ready for use even for the simplest task?

is (web client) ready for use even for the simplest task?

[Darren Hoo]

(use-modules (web client))

(http-post "http://www.google.com/") 

the POST request is sent without the Content-Length header

OK, let's add something to the body

(http-post "http://www.google.com/" #:body "")

Howcome the request now becomes an http GET request:

GET / HTTP/1.1
Content-Type: text/plain;charset=utf-8
Host: www.google.com
Connection: close

This is really ridiculous.

Re: is (web client) ready for use even for the simplest task?

[Aleix Conchillo Flaqué]

I have used (web client) in both guile-oauth (with a twitter client
example) and guile-xmlrpc.

So, yes, I think it is ready for the simplest task and much more. This
doesn't mean it might have bugs, as every software does.

Aleix

On Mon, Sep 9, 2013 at 9:13 PM, Darren Hoo <darren.hoo@gmail.com> wrote:
>
> (use-modules (web client))
>
> (http-post "http://www.google.com/")
>
> the POST request is sent without the Content-Length header
>
> OK, let's add something to the body
>
> (http-post "http://www.google.com/" #:body "")
>
> Howcome the request now becomes an http GET request:
>
> GET / HTTP/1.1
> Content-Type: text/plain;charset=utf-8
> Host: www.google.com
> Connection: close
>
> This is really ridiculous.
>
>

Re: is (web client) ready for use even for the simplest task?

[Nala Ginrut]

This bug had a fix but didn't push, I think it's in master now.

On Tue, 2013-09-10 at 12:13 +0800, Darren Hoo wrote:
> (use-modules (web client))
> 
> (http-post "http://www.google.com/") 
> 
> the POST request is sent without the Content-Length header
> 
> OK, let's add something to the body
> 
> (http-post "http://www.google.com/" #:body "")
> 
> Howcome the request now becomes an http GET request:
> 
> GET / HTTP/1.1
> Content-Type: text/plain;charset=utf-8
> Host: www.google.com
> Connection: close
> 
> This is really ridiculous.
> 
> 

Re: is (web client) ready for use even for the simplest task?

[Mark H Weaver]

Darren Hoo <darren.hoo@gmail.com> writes:

> (use-modules (web client))
>
> (http-post "http://www.google.com/") 
>
> the POST request is sent without the Content-Length header

RFC 2616 makes it clear that the Content-Length header is optional.

> OK, let's add something to the body
>
> (http-post "http://www.google.com/" #:body "")
>
> Howcome the request now becomes an http GET request:
>
> GET / HTTP/1.1
> Content-Type: text/plain;charset=utf-8
> Host: www.google.com
> Connection: close

I just applied a fix for this to the stable-2.0 branch in git.  In the
meantime, the workaround is to explicitly pass a content-type header
that specifies the charset, like this:

(http-post "http://www.google.com/"
           #:body ""
           #:headers '((content-type text/plain (charset . "utf-8"))))

> This is really ridiculous.

You found a bug.  It happens.  There's no need to be obnoxious about it.

      Mark

Re: is (web client) ready for use even for the simplest task?

[Nala Ginrut]

On Tue, 2013-09-10 at 05:19 -0400, Mark H Weaver wrote:
> Darren Hoo <darren.hoo@gmail.com> writes:
> 
> > (use-modules (web client))
> >
> > (http-post "http://www.google.com/") 
> >
> > the POST request is sent without the Content-Length header
> 
> RFC 2616 makes it clear that the Content-Length header is optional.
> 

I have to mention that Firefox seems not happy if the header not contain
Content-Length. When I tested Artanis redirect function under FF, it's
just halt anyway, but chrome is OK. And I fixed the issue by adding this
to response:
Content-Length: 0

That is so-called ridiculous.

But RFC 2616 wrote:
---------------------------cut----------------------------
body is not defined by this specification, but might be defined by
   future extensions to HTTP. Content negotiation MAY be used to select
   the appropriate response format. If no response body is included, the
   response MUST include a Content-Length field with a field-value of
   "0".
---------------------------end----------------------------



> > OK, let's add something to the body
> >
> > (http-post "http://www.google.com/" #:body "")
> >
> > Howcome the request now becomes an http GET request:
> >
> > GET / HTTP/1.1
> > Content-Type: text/plain;charset=utf-8
> > Host: www.google.com
> > Connection: close
> 
> I just applied a fix for this to the stable-2.0 branch in git.  In the
> meantime, the workaround is to explicitly pass a content-type header
> that specifies the charset, like this:
> 
> (http-post "http://www.google.com/"
>            #:body ""
>            #:headers '((content-type text/plain (charset . "utf-8"))))
> 
> > This is really ridiculous.
> 
> You found a bug.  It happens.  There's no need to be obnoxious about it.
> 
>       Mark
> 

Re: is (web client) ready for use even for the simplest task?

[Mark H Weaver]

Mark H Weaver <mhw@netris.org> writes:

>> GET / HTTP/1.1
>> Content-Type: text/plain;charset=utf-8
>> Host: www.google.com
>> Connection: close
>
> I just applied a fix for this to the stable-2.0 branch in git.  In the
> meantime, the workaround is to explicitly pass a content-type header
> that specifies the charset, like this:
>
> (http-post "http://www.google.com/"
>            #:body ""
>            #:headers '((content-type text/plain (charset . "utf-8"))))

Sorry, this wasn't quite enough if the body is non-empty.  In order to
work around the bug, you also need to explicitly pass the content-length
header, like this:

  (use-modules (web client) (rnrs bytevectors))

  (let ((bv (string->utf8 "test")))
    (http-post "http://www.google.com/"
               #:body bv
               #:headers `((content-type text/plain (charset . "utf-8"))
                           (content-length . ,(bytevector-length bv)))))

More generally, when using (web client) for anything other than GET, you
must explicitly provide a content-type header with charset, and also a
content-length (if appropriate).

The bug is fixed in stable-2.0 git.

      Mark

Re: is (web client) ready for use even for the simplest task?

[Ian Price]

Nala Ginrut <nalaginrut@gmail.com> writes:

> But RFC 2616 wrote:
> ---------------------------cut----------------------------
> body is not defined by this specification, but might be defined by
>    future extensions to HTTP. Content negotiation MAY be used to select
>    the appropriate response format. If no response body is included, the
>    response MUST include a Content-Length field with a field-value of
>    "0".
> ---------------------------end----------------------------

This section is a red herring, since it refers to the response to an
OPTIONS request, not to a POST request itself.

Section 4.3 Message Body says

   The presence of a message-body in a request is signaled by the
   inclusion of a Content-Length or Transfer-Encoding header field in
   the request's message-headers.

What this means is, if you don't send a content length or transfer
encoding header, then the server should not expect a body. I will note,
in particular, that not sending a message is a distinct operation from
sending a message of length 0.

-- 
Ian Price -- shift-reset.com

"Programming is like pinball. The reward for doing it well is
the opportunity to do it again" - from "The Wizardy Compiled"

Re: is (web client) ready for use even for the simplest task?

[Mark H Weaver]

Mark H Weaver <mhw@netris.org> writes:

>> (http-post "http://www.google.com/") 
>>
>> the POST request is sent without the Content-Length header
>
> RFC 2616 makes it clear that the Content-Length header is optional.

Sorry, I should have qualified this statement with "in this case",
i.e. when there's no body.  (web client) does add a Content-Length
header when there's a non-empty body.

It should also add a Content-Length header when an empty body is
provided, but it doesn't.  I'll fix this soon.  Anyway, the work-around
I provided adds a Content-Length header explicitly, so it works around
this problem too.

For the record, I didn't write this code; I'm just fixing it.

     Mark

Re: is (web client) ready for use even for the simplest task?

[Andy Wingo]

On Tue 10 Sep 2013 11:19, Mark H Weaver <mhw@netris.org> writes:

> Darren Hoo <darren.hoo@gmail.com> writes:
>
>> OK, let's add something to the body
>>
>> (http-post "http://www.google.com/" #:body "")
>>
>> Howcome the request now becomes an http GET request:
>>
>> GET / HTTP/1.1
>> Content-Type: text/plain;charset=utf-8
>> Host: www.google.com
>> Connection: close

That was totally my fault, FWIW.  Sorry about that!

>> This is really ridiculous.
>
> You found a bug.  It happens.  There's no need to be obnoxious about it.

Agreed.  But now that that's settled, have fun with Guile :)

Andy
-- 
http://wingolog.org/