From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: lo2net Newsgroups: gmane.emacs.bugs Subject: bug#11788: [babc40c4] still fails to implement HTTPS over HTTP proxy properly Date: Sat, 25 Jul 2015 00:32:28 +0800 Message-ID: <87io998qjn.fsf@gmail.com> References: <87siua8hf1.fsf@violet.siamics.net> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Trace: ger.gmane.org 1437755622 1331 80.91.229.3 (24 Jul 2015 16:33:42 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Fri, 24 Jul 2015 16:33:42 +0000 (UTC) To: schwab@linux-m68k.org, ivan@siamics.net, 11788@debbugs.gnu.org Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Fri Jul 24 18:33:31 2015 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1ZIfua-0004J3-4X for geb-bug-gnu-emacs@m.gmane.org; Fri, 24 Jul 2015 18:33:28 +0200 Original-Received: from localhost ([::1]:46033 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZIfuZ-0006bG-97 for geb-bug-gnu-emacs@m.gmane.org; Fri, 24 Jul 2015 12:33:27 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:44618) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZIfuE-00064N-LZ for bug-gnu-emacs@gnu.org; Fri, 24 Jul 2015 12:33:08 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ZIfuA-0005Fk-JX for bug-gnu-emacs@gnu.org; Fri, 24 Jul 2015 12:33:06 -0400 Original-Received: from debbugs.gnu.org ([140.186.70.43]:56704) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZIfuA-0005Ff-HP for bug-gnu-emacs@gnu.org; Fri, 24 Jul 2015 12:33:02 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.80) (envelope-from ) id 1ZIfuA-00011O-CQ for bug-gnu-emacs@gnu.org; Fri, 24 Jul 2015 12:33:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: lo2net Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Fri, 24 Jul 2015 16:33:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 11788 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: X-Debbugs-Original-To: Andreas Schwab , Ivan Shmakov , 11788@debbugs.gnu.org, bug-gnu-emacs@gnu.org Original-Received: via spool by 11788-submit@debbugs.gnu.org id=B11788.14377555673871 (code B ref 11788); Fri, 24 Jul 2015 16:33:02 +0000 Original-Received: (at 11788) by debbugs.gnu.org; 24 Jul 2015 16:32:47 +0000 Original-Received: from localhost ([127.0.0.1]:58146 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1ZIftu-00010L-0H for submit@debbugs.gnu.org; Fri, 24 Jul 2015 12:32:46 -0400 Original-Received: from mail-qk0-f177.google.com ([209.85.220.177]:36744) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1ZIftr-00010D-Ll for 11788@debbugs.gnu.org; Fri, 24 Jul 2015 12:32:44 -0400 Original-Received: by qkdv3 with SMTP id v3so17437469qkd.3 for <11788@debbugs.gnu.org>; Fri, 24 Jul 2015 09:32:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:to:subject:references:date:in-reply-to:message-id:user-agent :mime-version:content-type; bh=cGdNuAorJFJiWGV7I5fs4uUkUrq5c9cDNvgzZ7EH6WY=; b=Y5BSTFFPrJYphAPbr7A0M75R4Uk2tBu/NSG8BgXTZNT8m5lBUhmwRpl9/75iuPJ6hY sGZQ++5yi0Vc2DLAaeIAuI2AOmCiXdnguXIfUH9yu++OYn1OM9HLAuC2C1xEgPvLSnb0 Fx6ynTnKYyGZfRTopLMk+nuOa5Z1U2FEdfwR2VgkYAKZ0EoobOl4dfSQsw09ZOt27vPN GRfLdrbhdKCn5AqwUw/ApHjhs/CfhKE/7j6StLdHjn6Bbqnc48qdTqWlY9Cu8FWMj67U 84eUsfDjLP87f/UXxY+J2mwiO7w+iA6GjAuuGFhT5PkR0Ff51C/12VQgYdOAXrqj89An wuZw== X-Received: by 10.140.31.194 with SMTP id f60mr21624314qgf.23.1437755563140; Fri, 24 Jul 2015 09:32:43 -0700 (PDT) Original-Received: from StormPC.yourcompany.com (ec2-54-159-240-135.compute-1.amazonaws.com. [54.159.240.135]) by smtp.gmail.com with ESMTPSA id 75sm4286581qkr.47.2015.07.24.09.32.34 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 24 Jul 2015 09:32:42 -0700 (PDT) In-Reply-To: <87siua8hf1.fsf@violet.siamics.net> (Ivan Shmakov's message of "Tue, 03 Dec 2013 08:31:14 +0000") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.0.50 (gnu/linux) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 140.186.70.43 X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.bugs:105111 Archived-At: --=-=-= Content-Type: text/plain Hi all, I've wrote a patch to fix this, it works fine for me with gnutls support enabled, so I thought it may be useful. PS: If without gnutls support, it needs to be modified to use external program with https via proxy support(e.g. openssl s_client post-May 2015 release version: http://rt.openssl.org/Ticket/Display.html?id=2651) other than just throw an error. But I think very few people will need this since this bug report stayed with outstanding status for such a long time. Here is the patch: --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=0001-fix-url-https-over-proxy-implement.patch >From 50ad7c62ac8e34a43b8201702153fd6548d09a10 Mon Sep 17 00:00:00 2001 From: lo2net Date: Fri, 24 Jul 2015 23:19:26 +0800 Subject: [PATCH] fix url https over proxy implement --- lisp/url/url-http.el | 106 ++++++++++++++++++++++++++++++++++++++++++++------- 1 file changed, 92 insertions(+), 14 deletions(-) diff --git a/lisp/url/url-http.el b/lisp/url/url-http.el index 6a7d8e2..2bcd0cf 100644 --- a/lisp/url/url-http.el +++ b/lisp/url/url-http.el @@ -207,7 +207,7 @@ request.") ;; `url-open-stream' needs a buffer in which to do things ;; like authentication. But we use another buffer afterwards. (unwind-protect - (let ((proc (url-open-stream host buf host port gateway-method))) + (let ((proc (url-open-stream host buf (if url-using-proxy (url-host url-using-proxy) host) (if url-using-proxy (url-port url-using-proxy) port) gateway-method))) ;; url-open-stream might return nil. (when (processp proc) ;; Drop the temp buffer link before killing the buffer. @@ -912,6 +912,7 @@ should be shown to the user." ;; These unfortunately cannot be macros... please ignore them! (defun url-http-idle-sentinel (proc why) "Remove (now defunct) process PROC from the list of open connections." + (url-http-debug "url-http-idle-sentinel(%s %s)" proc why) (maphash (lambda (key val) (if (memq proc val) (puthash key (delq proc val) url-http-open-connections))) @@ -922,7 +923,7 @@ should be shown to the user." ;; and (ii) closed connection due to reusing a HTTP connection which ;; we believed was still alive, but which the server closed on us. ;; We handle case (ii) by calling `url-http' again. - (url-http-debug "url-http-end-of-document-sentinel in buffer (%s)" + (url-http-debug "url-http-end-of-document-sentinel(%s %s) in buffer (%s)" proc why (process-buffer proc)) (url-http-idle-sentinel proc why) (when (buffer-name (process-buffer proc)) @@ -936,7 +937,11 @@ should be shown to the user." (erase-buffer) (let ((url-request-method url-http-method) (url-request-extra-headers url-http-extra-headers) - (url-request-data url-http-data)) + (url-request-data url-http-data) + (url-using-proxy (url-find-proxy-for-url url-current-object (url-host url-current-object))) + ) + (when url-using-proxy + (setq url-using-proxy (url-generic-parse-url url-using-proxy))) (url-http url-current-object url-callback-function url-callback-arguments (current-buffer))))) ((url-http-parse-headers) @@ -1217,16 +1222,16 @@ overriding the value of `url-gateway-method'." (nsm-noninteractive (or url-request-noninteractive (and (boundp 'url-http-noninteractive) url-http-noninteractive))) - (connection (url-http-find-free-connection host port gateway-method)) + (connection (url-http-find-free-connection (url-host url) (url-port url) gateway-method)) (buffer (or retry-buffer (generate-new-buffer - (format " *http %s:%d*" host port))))) + (format " *http %s:%d*" (url-host url) (url-port url)))))) (if (not connection) ;; Failed to open the connection for some reason (progn (kill-buffer buffer) (setq buffer nil) - (error "Could not create connection to %s:%d" host port)) + (error "Could not create connection to %s:%d" (url-host url) (url-port url))) (with-current-buffer buffer (mm-disable-multibyte) (setq url-current-object url @@ -1274,22 +1279,91 @@ overriding the value of `url-gateway-method'." (set-process-buffer connection buffer) (set-process-filter connection 'url-http-generic-filter) + + (url-http-debug "url-http() status: %s" (process-status connection)) + (pcase (process-status connection) (`connect ;; Asynchronous connection (set-process-sentinel connection 'url-http-async-sentinel)) (`failed ;; Asynchronous connection failed - (error "Could not create connection to %s:%d" host port)) + (error "Could not create connection to %s:%d" (url-host url) (url-port url))) (_ + (if (and url-http-proxy (string= "https" (url-type url-current-object))) + (url-https-proxy-connect connection) + (set-process-sentinel connection 'url-http-end-of-document-sentinel) - (process-send-string connection (url-http-create-request)))))) - buffer)) + (process-send-string connection (url-http-create-request)) + ) + ) + ) + )) + + buffer) + ) +(defun url-https-proxy-connect (connection) + ;; https proxy + (setq url-http-after-change-function 'url-https-proxy-after-change-function) + (process-send-string connection (format (concat "CONNECT %s:%d HTTP/1.1\r\n" + "Host: %s\r\n" + "\r\n") + (url-host url-current-object) (or (url-port url-current-object) 443) (url-host url-current-object))) + ) +(defun url-https-proxy-after-change-function (st nd length) + (let* ((process-buffer (current-buffer)) + (proc (get-buffer-process process-buffer)) + ) + (goto-char (point-min)) + (when (re-search-forward "^\r?\n" nil t) + (backward-char 1) + ;; Saw the end of the headers + (setq url-http-end-of-headers (set-marker (make-marker) (point))) + (url-http-parse-response) + (cond + ((null url-http-response-status) + ;; We got back a headerless malformed response from the + ;; server. + (url-http-activate-callback) + (error "Malformed response from proxy, fail!")) + ((= url-http-response-status 200) + (if (gnutls-available-p) + (condition-case e + (let ((tls-connection (gnutls-negotiate + :process proc + :hostname (url-host url-current-object) + :verify-error nil))) + (with-current-buffer process-buffer + (erase-buffer)) + (set-process-buffer tls-connection process-buffer) + (setq url-http-after-change-function 'url-http-wait-for-headers-change-function) + (set-process-filter tls-connection 'url-http-generic-filter) + (process-send-string tls-connection (url-http-create-request)) + ) + (gnutls-error + (url-http-activate-callback) + (error "gnutls-error: %s" e)) + (error + (url-http-activate-callback) + (error "error: %s" e)) + ) + (error "error: gnutls support needed!") + ) + ) + (t + ;; (setq url-http-connection-opened nil) + (url-http-activate-callback) + (error "error response: %d\n" url-http-response-status)) + ) + ) + ) + ) (defun url-http-async-sentinel (proc why) ;; We are performing an asynchronous connection, and a status change ;; has occurred. + (url-http-debug "url-http-async-sentinel(%s %s)" proc why) (when (buffer-name (process-buffer proc)) (with-current-buffer (process-buffer proc) (cond @@ -1298,11 +1372,15 @@ overriding the value of `url-gateway-method'." (url-http-end-of-document-sentinel proc why)) ((string= (substring why 0 4) "open") (setq url-http-connection-opened t) - (condition-case error - (process-send-string proc (url-http-create-request)) - (file-error - (setq url-http-connection-opened nil) - (message "HTTP error: %s" error)))) + (if (and url-http-proxy (string= "https" (url-type url-current-object))) + (url-https-proxy-connect proc) + (condition-case error + (process-send-string proc (url-http-create-request)) + (file-error + (setq url-http-connection-opened nil) + (message "HTTP error: %s" error))) + ) + ) (t (setf (car url-callback-arguments) (nconc (list :error (list 'error 'connection-failed why -- 2.4.6 --=-=-= Content-Type: text/plain If anything goes wrong, please let me know, thanks! --=-=-=--