From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: "Christopher J. White" Newsgroups: gmane.emacs.bugs Subject: bug#10768: 23.3; url-http misses data when last few bytes are in 2nd packet and content-length is used Date: Thu, 9 Feb 2012 15:14:43 -0500 Message-ID: <20120209151443.000011a6@unknown> References: <20120208235321.00003609@unknown> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Trace: dough.gmane.org 1328818567 17860 80.91.229.3 (9 Feb 2012 20:16:07 GMT) X-Complaints-To: usenet@dough.gmane.org NNTP-Posting-Date: Thu, 9 Feb 2012 20:16:07 +0000 (UTC) Cc: 10768@debbugs.gnu.org To: Andreas Schwab Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Thu Feb 09 21:16:06 2012 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([140.186.70.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1RvaPV-0007DX-QU for geb-bug-gnu-emacs@m.gmane.org; Thu, 09 Feb 2012 21:16:06 +0100 Original-Received: from localhost ([::1]:45402 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1RvaPV-0005NG-1E for geb-bug-gnu-emacs@m.gmane.org; Thu, 09 Feb 2012 15:16:05 -0500 Original-Received: from eggs.gnu.org ([140.186.70.92]:54246) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1RvaPP-0005MD-Gz for bug-gnu-emacs@gnu.org; Thu, 09 Feb 2012 15:16:03 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1RvaPJ-0002Lw-Ny for bug-gnu-emacs@gnu.org; Thu, 09 Feb 2012 15:15:59 -0500 Original-Received: from debbugs.gnu.org ([140.186.70.43]:58687) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1RvaPJ-0002Lj-Bj for bug-gnu-emacs@gnu.org; Thu, 09 Feb 2012 15:15:53 -0500 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.72) (envelope-from ) id 1RvaQP-0008MA-P1 for bug-gnu-emacs@gnu.org; Thu, 09 Feb 2012 15:17:01 -0500 X-Loop: help-debbugs@gnu.org Resent-From: "Christopher J. White" Original-Sender: debbugs-submit-bounces@debbugs.gnu.org Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Thu, 09 Feb 2012 20:17:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 10768 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: Original-Received: via spool by 10768-submit@debbugs.gnu.org id=B10768.132881856332034 (code B ref 10768); Thu, 09 Feb 2012 20:17:01 +0000 Original-Received: (at 10768) by debbugs.gnu.org; 9 Feb 2012 20:16:03 +0000 Original-Received: from localhost ([127.0.0.1]:34077 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1RvaPR-0008KQ-9j for submit@debbugs.gnu.org; Thu, 09 Feb 2012 15:16:02 -0500 Original-Received: from mail237c25.carrierzone.com ([64.29.147.233]:51254) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1RvaPL-0008K9-RB for 10768@debbugs.gnu.org; Thu, 09 Feb 2012 15:15:58 -0500 X-POP-User: ajgrier.grierwhite.com Original-Received: from unknown (pool-108-49-4-207.bstnma.east.verizon.net [108.49.4.207]) by mail237c25.carrierzone.com (8.13.6/8.13.1) with ESMTP id q19KEhFD012084; Thu, 9 Feb 2012 20:14:45 GMT In-Reply-To: X-Mailer: Claws Mail 3.8.0cvs6 (GTK+ 2.16.6; i586-pc-mingw32msvc) X-CSC: 0 X-CHA: v=1.1 cv=eOIpK0TUyV1bo+1wo/1fQvFrj8DjMnYhoAngSdkCsuU= c=1 sm=1 a=hdTLrLAGt2gA:10 a=3pyGGIT-qAEA:10 a=kj9zAlcOel0A:10 a=iW5S9OHVd49uhICS+jddKg==:17 a=JJJ-DeXCAAAA:8 a=G0BRwk1oAAAA:8 a=tBb2bbeoAAAA:8 a=zF1JnFZfGnnx8OOGMiAA:9 a=yR2xMyxR96uYDJB_MEoA:7 a=CjuIK1q_8ugA:10 a=uyOpwfsfqbsA:10 a=YuKU6ANggZ8A:10 a=iW5S9OHVd49uhICS+jddKg==:117 X-CTCH-Spam: Unknown X-CTCH-RefID: str=0001.0A020203.4F342936.0003,ss=1,re=0.000,fgs=0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.13 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 2) X-Received-From: 140.186.70.43 X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.bugs:56737 Archived-At: Hi Andreas, I have created a small set of files with lengths varying from 1200 to 1225, and from 2658 to 2662 and put them up on my personal web server. (e.g. http://www.grierwhite.com/url-http-1220.txt) I chose lengths that just barely stretch from 1 to 2 payload packets, and from 2 to 3 packets. 1200-1207 -- 1 packets 1208-1225 -- 2 packets, 1-18 bytes in the 2nd 2658-2659 -- 2 packets 2660-2662 -- 3 packets, 1-3 bytes in the 3rd Prior to the change, the results are as follows: 1200-1207 -- pass 1208-1217 -- fail 1217-1225 -- pass 2658-2662 -- pass This indicates that the failure occurs when there are up to 10 bytes in the 2nd packet. This correlates with the fact that the header is 10 lines, thus will replace 10 \r characters and shorten the buffer by that amount. Consistent with the results I found with the original server toodledo.com that only has 8 header lines and only has the issue up to 8 bytes in the 2nd packet. It is interesting to note that when there are 3 packets and just a few bytes in the last packet, the failure does *not* occur. I think this may be explained by the fact that url-http-wait-for-headers-change-function may be called a second time, thus all position references are recomputed. Andreas, can you verify this logic? If this is correct, the problem only occurs for responses that just border on 2 packets. After the change, all lengths pass. Here is the lisp code that tests the lengths, feel free to run this as you should be able to reproduce this from any machine that can access the internet. I don't have a trunk version of emacs (24), but looking at the trunk copy of url-http.el, it appears to suffer from the same problem. (defun test-url-http-patch (minlen maxlen) (let ((passed 0) (failed 0)) (do ((len minlen (+ 1 len))) ((> len maxlen) nil) (set-buffer (url-retrieve-synchronously (format "http://www.grierwhite.com/url-http-%d.txt" len))) (goto-char (point-min)) (re-search-forward "^\n") (let ((recvlen (- (point-max) (point)))) (if (= len recvlen) (setq passed (1+ passed)) (setq failed (1+ failed)) (message "FAILED orig-len: %d, received len: %d" len recvlen)))) (message "Passed: %d/%d" passed (+ passed failed)))) ;; Run without the patch (test-url-http-patch 1200 1225) "Passed: 16/26" (test-url-http-patch 2658 2662) "Passed: 5/5" ;; Run with the patch (test-url-http-patch 1200 1225) "Passed: 26/26" (test-url-http-patch 2658 2662) "Passed: 5/5" ...cj On Thu, 09 Feb 2012 16:43:27 +0100 Andreas Schwab wrote: > Does this help? > > diff --git a/lisp/url/url-http.el b/lisp/url/url-http.el > index b43ed76..140824f 100644 > --- a/lisp/url/url-http.el > +++ b/lisp/url/url-http.el > @@ -352,11 +352,14 @@ request.") > ;; Parsing routines > (defun url-http-clean-headers () > "Remove trailing \r from header lines. > -This allows us to use `mail-fetch-field', etc." > +This allows us to use `mail-fetch-field', etc. > +Return the number of characters removed." > (declare (special url-http-end-of-headers)) > - (goto-char (point-min)) > - (while (re-search-forward "\r$" url-http-end-of-headers t) > - (replace-match ""))) > + (let ((end (marker-position url-http-end-of-headers))) > + (goto-char (point-min)) > + (while (re-search-forward "\r$" url-http-end-of-headers t) > + (replace-match "")) > + (- end url-http-end-of-headers))) > > (defun url-http-handle-authentication (proxy) > (declare (special status success url-http-method url-http-data > @@ -1051,7 +1054,7 @@ the end of the document." > (setq url-http-end-of-headers (set-marker (make-marker) > (point)) > end-of-headers t) > - (url-http-clean-headers))) > + (setq nd (- nd (url-http-clean-headers)))) > > (if (not end-of-headers) > ;; Haven't seen the end of the headers yet, need to wait > > Andreas. >