From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Chong Yidong Newsgroups: gmane.emacs.devel Subject: Re: URL not following some 302 redirects after recent changes Date: Wed, 31 Jan 2007 19:21:12 -0500 Message-ID: <87bqkek9s7.fsf@stupidchicken.com> References: <871wnw2opm.fsf@x3y2z1.net> <87tzy7wbsj.fsf@x3y2z1.net> NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: sea.gmane.org 1170289380 29021 80.91.229.12 (1 Feb 2007 00:23:00 GMT) X-Complaints-To: usenet@sea.gmane.org NNTP-Posting-Date: Thu, 1 Feb 2007 00:23:00 +0000 (UTC) Cc: emacs-devel@gnu.org To: Diane Murray , Magnus Henoch Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Thu Feb 01 01:22:44 2007 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([199.232.76.165]) by lo.gmane.org with esmtp (Exim 4.50) id 1HCPit-0002e5-Al for ged-emacs-devel@m.gmane.org; Thu, 01 Feb 2007 01:22:43 +0100 Original-Received: from localhost ([127.0.0.1] helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1HCPis-0000Z6-QV for ged-emacs-devel@m.gmane.org; Wed, 31 Jan 2007 19:22:42 -0500 Original-Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1HCPih-0000Yw-HQ for emacs-devel@gnu.org; Wed, 31 Jan 2007 19:22:31 -0500 Original-Received: from [199.232.76.173] (helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1HCPih-0000Yd-0l for emacs-devel@gnu.org; Wed, 31 Jan 2007 19:22:31 -0500 Original-Received: from south-station-annex.mit.edu ([18.72.1.2]) by monty-python.gnu.org with esmtps (TLS-1.0:DHE_RSA_AES_256_CBC_SHA:32) (Exim 4.52) id 1HCPig-0006mu-Nv for emacs-devel@gnu.org; Wed, 31 Jan 2007 19:22:30 -0500 Original-Received: from central-city-carrier-station.mit.edu (CENTRAL-CITY-CARRIER-STATION.MIT.EDU [18.7.7.72]) by south-station-annex.mit.edu (8.13.6/8.9.2) with ESMTP id l110LP9M003414; Wed, 31 Jan 2007 19:21:26 -0500 (EST) Original-Received: from outgoing-legacy.mit.edu (OUTGOING-LEGACY.MIT.EDU [18.7.22.104]) by central-city-carrier-station.mit.edu (8.13.6/8.9.2) with ESMTP id l110LFJ7002774; Wed, 31 Jan 2007 19:21:15 -0500 (EST) Original-Received: from localhost (MAIN-TWELVE-TWO-TWENTY-SEVEN.MIT.EDU [18.19.5.227]) ) by outgoing-legacy.mit.edu (8.13.6/8.12.4) with ESMTP id l110LCk7026958; Wed, 31 Jan 2007 19:21:12 -0500 (EST) Original-Received: from cyd by localhost with local (Exim 3.36 #1 (Debian)) id 1HCPhQ-0001pE-00; Wed, 31 Jan 2007 19:21:12 -0500 In-Reply-To: <87tzy7wbsj.fsf@x3y2z1.net> (Diane Murray's message of "Wed\, 31 Jan 2007 14\:44\:44 +0100") User-Agent: Gnus/5.11 (Gnus v5.11) Emacs/22.0.93 (gnu/linux) X-Scanned-By: MIMEDefang 2.42 X-Spam-Score: -2.599 X-detected-kernel: Solaris 9.1 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.devel:65726 Archived-At: Diane Murray writes: > On Wed, 22 Nov 2006 03:35:17 +0100 I wrote: >> Sometime after 2006-10-26 URL redirects stopped working correctly >> (Emacs CVS of 2006-09-19 and 2006-10-26 works, 2006-10-31 and >> 2006-11-19 don't work), perhaps due to changes made in revision 1.36 >> of url-http.el. I think I know the problem. In url-http-async-sentinel, before sending the http request, the code attempts to change the process sentinel to url-http-end-of-document-sentinel, which is supposed to parse the headers when the connection is closed: (cond ((string= (substring why 0 4) "open") (set-process-sentinel proc 'url-http-end-of-document-sentinel) (process-send-string proc (url-http-create-request))) However, attempting to change the process sentinel in the sentinel itself has no effect (see the elisp manual). That's why the headers don't get parsed when the connection is closed. The reason HTTP 1.1 does not have this problem is that it reports the total length of the document, so the headers get parsed as soon as we realise, in url-http-content-length-after-change-function, that we are at the end of the document. For HTTP 1.0, we have to wait till the connection is closed before realizing that we are at the end of the document. A quick fix is to introduce a variable url-http-catch-end-of-document, and have url-http-async-sentinel call url-http-end-of-document-sentinel instead of doing its usual job if that is non-nil. Then url-http-async-sentinel can set url-http-catch-end-of-document instead of trying to change the process sentinel. The patch below implements this. I will commit it if there are no objections in the next couple of days. *** emacs/lisp/url/url-http.el.~1.49.~ 2007-01-21 08:38:56.000000000 -0500 --- emacs/lisp/url/url-http.el 2007-01-31 19:12:53.000000000 -0500 *************** *** 30,35 **** --- 30,36 ---- (defvar url-http-extra-headers) (defvar url-http-target-url) (defvar url-http-proxy) + (defvar url-http-catch-end-of-document) (require 'url-gw) (require 'url-util) (require 'url-parse) *************** *** 1118,1123 **** --- 1119,1125 ---- url-http-extra-headers url-http-data url-http-target-url + url-http-catch-end-of-document url-http-proxy)) (set (make-local-variable var) nil)) *************** *** 1132,1137 **** --- 1134,1140 ---- url-callback-arguments cbargs url-http-after-change-function 'url-http-wait-for-headers-change-function url-http-target-url url-current-object + url-http-catch-end-of-document nil url-http-proxy url-using-proxy) (set-process-buffer connection buffer) *************** *** 1140,1145 **** --- 1143,1149 ---- (cond ((eq status 'connect) ;; Asynchronous connection + (setq url-http-catch-end-of-document nil) (set-process-sentinel connection 'url-http-async-sentinel)) ((eq status 'failed) ;; Asynchronous connection failed *************** *** 1153,1170 **** (declare (special url-callback-arguments)) ;; We are performing an asynchronous connection, and a status change ;; has occurred. ! (with-current-buffer (process-buffer proc) ! (cond ! ((string= (substring why 0 4) "open") ! (set-process-sentinel proc 'url-http-end-of-document-sentinel) ! (process-send-string proc (url-http-create-request))) ! (t ! (setf (car url-callback-arguments) ! (nconc (list :error (list 'error 'connection-failed why ! :host (url-host (or url-http-proxy url-current-object)) ! :service (url-port (or url-http-proxy url-current-object)))) ! (car url-callback-arguments))) ! (url-http-activate-callback))))) ;; Since Emacs 19/20 does not allow you to change the ;; `after-change-functions' hook in the midst of running them, we fake --- 1157,1176 ---- (declare (special url-callback-arguments)) ;; We are performing an asynchronous connection, and a status change ;; has occurred. ! (if url-http-catch-end-of-document ! (url-http-end-of-document-sentinel proc why) ! (with-current-buffer (process-buffer proc) ! (cond ! ((string= (substring why 0 4) "open") ! (setq url-http-catch-end-of-document t) ! (process-send-string proc (url-http-create-request))) ! (t ! (setf (car url-callback-arguments) ! (nconc (list :error (list 'error 'connection-failed why ! :host (url-host (or url-http-proxy url-current-object)) ! :service (url-port (or url-http-proxy url-current-object)))) ! (car url-callback-arguments))) ! (url-http-activate-callback)))))) ;; Since Emacs 19/20 does not allow you to change the ;; `after-change-functions' hook in the midst of running them, we fake