bug#8931: 24.0.50; url-http.el terminating HTTP packets with extra carriage return and newline

bug#8931: 24.0.50; url-http.el terminating HTTP packets with extra carriage return and newline

[Nicolas Avrutin]

[-- Attachment #1.1: Type: text/plain, Size: 1081 bytes --]

`browse-url-emacs' and `url-retrieve-synchronously' (and probably other
functions) appear to be generating malformed HTTP packets by appending an
extra \r\n to the packet, resulting in them being terminated with "\r\n\r\n"
instead of "\r\n". Some pages (such as http://www.google.com) to still
return properly and just send back a packet saying that the sent packet was
malformed. Other pages, in my case last.fm's client API (access via
emms-lastfm-client) return 400 Bad Request. In addition, this causes
Wireshark to report the packet as "HEAD / HTTP/1.1 Continuation or non-HTTP
traffic".

Removing the "\r\n" on line 341 of url-http.el (see attached patch) removes
the duplicate "'\r\n" and appears to fix the issue. Calls to last.fm's
client API return 200 OK as expected and Wireshark properly reports the
packet as "HEAD / HTTP/1.1". Google, however, still reports a malformed
packet.

This bug is preset in Emacs 23.1, but is not in 23.2. My patch restores the
line to what it was in 23.2, but I do not know if it was changed for a
reason or in error.

--
Nicolas Avrutin

[-- Attachment #1.2: Type: text/html, Size: 1367 bytes --]

[-- Attachment #2: url-http.patch --]
[-- Type: text/x-patch, Size: 557 bytes --]

*** url-http.el	2011-06-24 15:11:53.186238893 -0400
--- url-http.el.patched	2011-06-24 15:11:00.232907619 -0400
***************
*** 338,344 ****
               ;; End request
               "\r\n"
               ;; Any data
!              url-http-data "\r\n"))
             ""))
      (url-http-debug "Request is: \n%s" request)
      request))
--- 338,344 ----
               ;; End request
               "\r\n"
               ;; Any data
!              url-http-data))
             ""))
      (url-http-debug "Request is: \n%s" request)
      request))

bug#8931: 24.0.50; url-http.el terminating HTTP packets with extra carriage return and newline

[Lars Magne Ingebrigtsen]

Nicolas Avrutin <nicolasavru@gmail.com> writes:

> This bug is preset in Emacs 23.1, but is not in 23.2. My patch
> restores the line to what it was in 23.2, but I do not know if it was
> changed for a reason or in error.

Thanks for the patch; I've now applied it to Emacs 24.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

bug#8931: url-http-create-request needs extra CRLF for https

[Christopher J. White]

There is a tug of war going on here.  

Prior to 100681, the code had no CRLF after url-http-data.  Mark
Hershberger submitted a patch to add the CRLF because without it, POSTs
with content to https urls just hang:

  http://bzr.savannah.gnu.org/lh/emacs/trunk/revision/100681

This change was undone by Nicola Avrutin in 104908:

  http://bzr.savannah.gnu.org/lh/emacs/trunk/revision/104908

The problem was that with no url-http-data, the request ends with a
double CRLF, which apparently confuses some servers.

I recently came across the exact same problem (with url-http.el
without the extra CRL), and Mark's patch above works just fine for me
(also POSTs to https).

I'm thinking the correct solution is to only add the CRLF if there is
http-data:

-    url-http-data))
+    url-http-data (if url-http-data "\r\n")))

...cj

 

bug#8931: May be related to 9592

[Christopher J. White]

Further testing with this indicates this may not be a complete fix.
The fix works reliably for a single connection, but when making
multiple POST requests, an older process is grabbed for reuse, but then
leads to an error as the process seems to terminate. 

See:  http://debbugs.gnu.org/cgi/bugreport.cgi?bug=9592

bug#8931: url-http-create-request needs extra CRLF for https

[Chong Yidong]

"Christopher J. White" <chris@grierwhite.com> writes:

> I'm thinking the correct solution is to only add the CRLF if there is
> http-data:
>
> -    url-http-data))
> +    url-http-data (if url-http-data "\r\n")))

Yes, this looks right.  Committed to trunk.