From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Stefan Monnier Newsgroups: gmane.emacs.bugs Subject: bug#16220: url-http.el: Not conforming to HTTP spec Date: Wed, 01 Jan 2014 21:21:04 -0500 Message-ID: References: NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain X-Trace: ger.gmane.org 1388629334 30916 80.91.229.3 (2 Jan 2014 02:22:14 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Thu, 2 Jan 2014 02:22:14 +0000 (UTC) Cc: 16220@debbugs.gnu.org To: =?UTF-8?Q?Jaros=C5=82aw_?= =?UTF-8?Q?Rzesz=C3=B3tko?= Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Thu Jan 02 03:22:20 2014 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1VyXvQ-0008IW-5X for geb-bug-gnu-emacs@m.gmane.org; Thu, 02 Jan 2014 03:22:20 +0100 Original-Received: from localhost ([::1]:43081 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1VyXvP-0006Wm-Os for geb-bug-gnu-emacs@m.gmane.org; Wed, 01 Jan 2014 21:22:19 -0500 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:55604) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1VyXvG-0006Wg-21 for bug-gnu-emacs@gnu.org; Wed, 01 Jan 2014 21:22:17 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1VyXv8-0002Ui-Os for bug-gnu-emacs@gnu.org; Wed, 01 Jan 2014 21:22:10 -0500 Original-Received: from debbugs.gnu.org ([140.186.70.43]:42887) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1VyXv8-0002Ud-Lj for bug-gnu-emacs@gnu.org; Wed, 01 Jan 2014 21:22:02 -0500 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.80) (envelope-from ) id 1VyXv8-0008Cd-4A for bug-gnu-emacs@gnu.org; Wed, 01 Jan 2014 21:22:02 -0500 X-Loop: help-debbugs@gnu.org Resent-From: Stefan Monnier Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Thu, 02 Jan 2014 02:22:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 16220 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: Original-Received: via spool by 16220-submit@debbugs.gnu.org id=B16220.138862926931458 (code B ref 16220); Thu, 02 Jan 2014 02:22:02 +0000 Original-Received: (at 16220) by debbugs.gnu.org; 2 Jan 2014 02:21:09 +0000 Original-Received: from localhost ([127.0.0.1]:56906 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VyXuG-0008BK-Pu for submit@debbugs.gnu.org; Wed, 01 Jan 2014 21:21:09 -0500 Original-Received: from ironport2-out.teksavvy.com ([206.248.154.181]:46769) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VyXuD-0008BB-SQ for 16220@debbugs.gnu.org; Wed, 01 Jan 2014 21:21:06 -0500 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Av4EABK/CFHO+J7K/2dsb2JhbABEvw4Xc4IeAQEEAVYjBQsLBy0SFBgNJIgeBsEtkQoDiGGcGYFegxU X-IPAS-Result: Av4EABK/CFHO+J7K/2dsb2JhbABEvw4Xc4IeAQEEAVYjBQsLBy0SFBgNJIgeBsEtkQoDiGGcGYFegxU X-IronPort-AV: E=Sophos;i="4.84,565,1355115600"; d="scan'208";a="43737613" Original-Received: from 206-248-158-202.dsl.teksavvy.com (HELO pastel.home) ([206.248.158.202]) by ironport2-out.teksavvy.com with ESMTP/TLS/ADH-AES256-SHA; 01 Jan 2014 21:21:05 -0500 Original-Received: by pastel.home (Postfix, from userid 20848) id D154A60051; Wed, 1 Jan 2014 21:21:04 -0500 (EST) In-Reply-To: ("=?UTF-8?Q?Jaros=C5=82aw_?= =?UTF-8?Q?Rzesz=C3=B3tko?="'s message of "Tue, 24 Dec 2013 17:31:57 +0100") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.3.50 (gnu/linux) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 140.186.70.43 X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.bugs:82811 Archived-At: > The problem is that url-http.el sets a lot of headers by default that > can not be overwritten in any other way then dynamically overshadowing > some variables. Indeed, this is ugly. Improvements welcome. > For example, all connections are keep-alive by > default, which is confusing in itself already, Not sure why it should be a problem. > (let ((url-request-method "GET") > (url-request-extra-headers '(("Connection" . "close")))) > (url-retrieve-synchronously "http://www.google.com/")) > And what is sent is this: > GET / HTTP/1.1 > Connection: keep-alive > ... > Connection: close > Which again isn't valid HTTP and the behaviour of the HTTP server in > this case is undefined and implementation specific. The only way to > workaround this is doing this: > (let ((url-http-attempt-keepalives nil) > (url-request-method "GET") > (url-request-extra-headers '(("Connection" . "close")))) > (url-retrieve-synchronously "http://www.google.com/")) Yuck! We can probably fix this fairly easily by letting url-request-extra-headers override (rather than just add to) other headers. > This is all the more irritating so many of the headers are set by > default using the variables url-vars.el. Why those things are at all > variables is a mystery to me. Probably partly historical evolution (there was no place to add new "parameters", so adding dynamic vars was an easy way to add more control without breaking existing code). > In the end it is much easier to do HTTP requests manually using > make-network-process then it is with the url library, I think that's misleading: the URL library is supposed to deal with things like proxies and redirections, which "manual requests via make-network-process" probably won't handle. > Didn't anyone else run into problems with it? Apparently not yet. But I agree that the API might deserve a redesign (IIRC another problem is in the way headers in the answer are returned to the caller, which does not work consistently across different kinds of URLs (ftp, http, file, imap, ...)). Stefan