From: "Jarosław Rzeszótko" <sztywny@gmail.com>
To: Stefan Monnier <monnier@iro.umontreal.ca>
Cc: 16220@debbugs.gnu.org
Subject: bug#16220: url-http.el: Not conforming to HTTP spec
Date: Tue, 24 Dec 2013 17:31:57 +0100 [thread overview]
Message-ID: <CAHnYAZbefHaSr1eTb6V-Lwin6+V5zNmoP6AQhdLYCGq-kmRJoQ@mail.gmail.com> (raw)
In-Reply-To: <jwvvbye2tue.fsf-monnier+emacsbugs@gnu.org>
Hi,
There are more issues besides just the extra "\r\n", the url-http.el
library is super confusing. I read the documentation for url-retrieve,
which makes doing a HTTP request look really straightforward, and then
I spent several hours finding out what the hell happens. For example,
the documentation for url-retrieve says:
"The variables `url-request-data', `url-request-method' and
`url-request-extra-headers' can be dynamically bound around the
request; dynamic binding of other variables doesn't necessarily take
effect."
The problem is that url-http.el sets a lot of headers by default that
can not be overwritten in any other way then dynamically overshadowing
some variables. For example, all connections are keep-alive by
default, which is confusing in itself already, futhermore you can't
just pass "Connection" . "close' in url-request-extra-headers, because
url-http.el joins the default and "extra" headers instead of merging
the "extra" ones and having them overwrite the default ones. So try do
something like this:
(let ((url-request-method "GET")
(url-request-extra-headers '(("Connection" . "close"))))
(url-retrieve-synchronously "http://www.google.com/"))
And what is sent is this:
GET / HTTP/1.1
Connection: keep-alive
...
Connection: close
Which again isn't valid HTTP and the behaviour of the HTTP server in
this case is undefined and implementation specific. The only way to
workaround this is doing this:
(let ((url-http-attempt-keepalives nil)
(url-request-method "GET")
(url-request-extra-headers '(("Connection" . "close"))))
(url-retrieve-synchronously "http://www.google.com/"))
Which is only clear by reading url-http.el where again it is described
in a very confusing way:
(defvar url-http-attempt-keepalives t
"Whether to use a single TCP connection multiple times in HTTP.
This is only useful when debugging the HTTP subsystem. Setting to
nil will explicitly close the connection to the server after every
request.")
This is all the more irritating so many of the headers are set by
default using the variables url-vars.el. Why those things are at all
variables is a mystery to me. Then there are messages that appear in
the minibuffer and there is no way to disable them. In the end it is
much easier to do HTTP requests manually using make-network-process
then it is with the url library, it really isn't a good general
purpose HTTP component, there are too many completely opaque things
happening behind the scenes, like the hashmap of persistent tcp
connections that is maintained behind the scenes. Didn't anyone else
run into problems with it?
Cheers,
Jarosław Rzeszótko
2013/12/24 Stefan Monnier <monnier@iro.umontreal.ca>:
>> At the end of every HTTP request to be made with url-http.el and containing
>> a body, an unnecessary "\r\n" is appended, and additionally those two
>> characters are not used in the calculation of the Content-Length header.
>
> Looks like an old workaround. Could someone get rid of this?
>
>
> Stefan
next prev parent reply other threads:[~2013-12-24 16:31 UTC|newest]
Thread overview: 17+ messages / expand[flat|nested] mbox.gz Atom feed top
2013-12-22 20:52 bug#16220: url-http.el: Not conforming to HTTP spec Jarosław Rzeszótko
2013-12-22 21:55 ` Jarosław Rzeszótko
2013-12-22 22:33 ` Ted Zlatanov
2013-12-23 6:51 ` Jarosław Rzeszótko
2013-12-23 13:08 ` Ted Zlatanov
2013-12-24 7:50 ` Lars Ingebrigtsen
2013-12-24 8:28 ` Jarosław Rzeszótko
2013-12-24 14:43 ` Stefan Monnier
2013-12-24 16:31 ` Jarosław Rzeszótko [this message]
2014-01-02 2:21 ` Stefan Monnier
2014-01-03 18:06 ` Jarosław Rzeszótko
2014-01-05 9:57 ` Lars Magne Ingebrigtsen
2014-01-05 13:25 ` Jarosław Rzeszótko
2014-01-06 15:06 ` Stefan Monnier
2014-01-07 19:30 ` Jarosław Rzeszótko
2014-01-08 18:29 ` Stefan Monnier
2014-01-18 22:35 ` Paul Eggert
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CAHnYAZbefHaSr1eTb6V-Lwin6+V5zNmoP6AQhdLYCGq-kmRJoQ@mail.gmail.com \
--to=sztywny@gmail.com \
--cc=16220@debbugs.gnu.org \
--cc=monnier@iro.umontreal.ca \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this external index
https://git.savannah.gnu.org/cgit/emacs.git
https://git.savannah.gnu.org/cgit/emacs/org-mode.git
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.