From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Tao Fang Newsgroups: gmane.emacs.bugs Subject: bug#11788: [babc40c4] still fails to implement HTTPS over HTTP proxy properly Date: Tue, 22 Mar 2016 23:31:43 +0800 Message-ID: <8760wesf4w.fsf@gmail.com> References: <87siua8hf1.fsf@violet.siamics.net> <87io998qjn.fsf@gmail.com> <87lh8iz041.fsf@gnus.org> <83k2o1k6z0.fsf@gnu.org> <8737uj7vzw.fsf@gmail.com> <83bn97dgo6.fsf@gnu.org> <87a8m8dafw.fsf@engster.org> <831t7krazs.fsf@gnu.org> <87d1qvwxo0.fsf@gmail.com> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Trace: ger.gmane.org 1458660806 4212 80.91.229.3 (22 Mar 2016 15:33:26 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Tue, 22 Mar 2016 15:33:26 +0000 (UTC) Cc: 11788@debbugs.gnu.org, schwab@linux-m68k.org, ivan@siamics.net, David Engster To: Lars Magne Ingebrigtsen Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Tue Mar 22 16:33:15 2016 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1aiOJ0-00078G-Ie for geb-bug-gnu-emacs@m.gmane.org; Tue, 22 Mar 2016 16:33:14 +0100 Original-Received: from localhost ([::1]:37787 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aiOIz-0006L6-Rq for geb-bug-gnu-emacs@m.gmane.org; Tue, 22 Mar 2016 11:33:13 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:53671) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aiOIv-0006Kf-CU for bug-gnu-emacs@gnu.org; Tue, 22 Mar 2016 11:33:10 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1aiOIp-0003XR-BJ for bug-gnu-emacs@gnu.org; Tue, 22 Mar 2016 11:33:09 -0400 Original-Received: from debbugs.gnu.org ([208.118.235.43]:34795) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aiOIp-0003XN-7P for bug-gnu-emacs@gnu.org; Tue, 22 Mar 2016 11:33:03 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1aiOIo-0005tj-HF; Tue, 22 Mar 2016 11:33:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Tao Fang Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org, Magnus Henoch Resent-Date: Tue, 22 Mar 2016 15:33:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 11788 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: patch Original-Received: via spool by 11788-submit@debbugs.gnu.org id=B11788.145866073922612 (code B ref 11788); Tue, 22 Mar 2016 15:33:02 +0000 Original-Received: (at 11788) by debbugs.gnu.org; 22 Mar 2016 15:32:19 +0000 Original-Received: from localhost ([127.0.0.1]:60155 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1aiOI3-0005sb-UZ for submit@debbugs.gnu.org; Tue, 22 Mar 2016 11:32:19 -0400 Original-Received: from mail-qg0-f46.google.com ([209.85.192.46]:34244) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1aiOHy-0005sF-Rz for 11788@debbugs.gnu.org; Tue, 22 Mar 2016 11:32:14 -0400 Original-Received: by mail-qg0-f46.google.com with SMTP id w104so181192384qge.1 for <11788@debbugs.gnu.org>; Tue, 22 Mar 2016 08:32:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:to:cc:subject:references:user-agent:date:in-reply-to :message-id:mime-version; bh=EIF+Uo0UjPKwurcIhN3Uz1zRY0c8u5PGFoOq/ahNX0U=; b=Pp1iMvgUYK1YG62H1Rc7hBXlv7rW9IVFy7pwdlkXBk6ygcWWGx1RLOB0c2zQn5uVaQ A/LS17CCy7j0Ainlwy7oLJxoH1U2E5UhWp2lHnYprM5xwlyy3owMM4Dk9pdJpY1zyNPO TkK0YXT1GeOwX2vpWtJgsWC/oSqQbZ38WMmidzjUTKSdPzLAcmPZDnyNRfOVn8Nq0GHs YaW6mYb3yk4EpFLpoF8p1/dCiAVEbeL+mbGDXsLIlMibU5fnbSERWcXRg3ZteZH4ixwv uJYYlPCrhqIUs61/ZZ/+8xDNw6HxODjQHEWaZ0BQXd98O0zilTAt/Ex+wv3ivvY15U1x VD/g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:references:user-agent:date :in-reply-to:message-id:mime-version; bh=EIF+Uo0UjPKwurcIhN3Uz1zRY0c8u5PGFoOq/ahNX0U=; b=SISe4XsjS9Kt/Hmf59IRmDLKTVtxeCzw67HOk8cCynmlDEi+k4e1HHLF0GyI6OiEPY D+yKgt/vAg1+fYmzRBb3HsqU0R3+2VOI1KbqSk/o06ILEr1JX33Ny9khiBLCk1wSFOBE pLTb+X9pwHGERL3VEd5ZQFzZM62HuxVDzKw2XxOgyjTGdwVrFIn2lK/u25p1A44V+r69 JhYxvremglQdICgCfX7q/9+ESZSCm6ebCZ48w7vNPlpZj9gUKgjsK+S8a8/eP44NHT/l S+AafLLeB2DKe0le3rGqBEiyUhNoQt/BaIYZxNnScr3g09U3GVD0WLpHccvcQmi32nJE QhLQ== X-Gm-Message-State: AD7BkJKpv4KrRxAkZSyDRdWYvqD+LyxnBfXTnAKogPO5e7/fmndJfGgvQK14jYPKjdbcFQ== X-Received: by 10.140.227.81 with SMTP id x78mr50754204qhb.6.1458660725289; Tue, 22 Mar 2016 08:32:05 -0700 (PDT) Original-Received: from StormPC.yourcompany.com (ec2-52-3-137-119.compute-1.amazonaws.com. [52.3.137.119]) by smtp.gmail.com with ESMTPSA id g50sm14838876qgg.40.2016.03.22.08.31.48 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 22 Mar 2016 08:32:04 -0700 (PDT) User-Agent: Emacs/Gnus In-Reply-To: (Lars Magne Ingebrigtsen's message of "Sun, 20 Mar 2016 12:21:09 +0100") X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 208.118.235.43 X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.bugs:115326 Archived-At: --=-=-= Content-Type: text/plain Lars Magne Ingebrigtsen writes: > Throughout the code, the lines seem to be too long. They should > preferably not be more than 80 characters long (unless there's an > absolute need). > After negotiation, you should probably call `nsm-verify-connection'. > Uhm... and that's it. Oh, and a NEWS entry saying that url now > supports HTTPS proxies would be nice, and a ChangeLog style commit > message. Done with it. Here is the patch file: --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=0001-Fix-url-https-over-proxy-implement.-Bug-11788.patch >From 172363d31b3ad5f45da44aa09652d0e0779ef5f2 Mon Sep 17 00:00:00 2001 From: Tao Fang Date: Tue, 22 Mar 2016 22:39:51 +0800 Subject: [PATCH] Fix url https over proxy implement. (Bug#11788) * lisp/url/url-http.el: Fix url https over proxy implement. (Bug#11788) * etc/NEWS: Mention this. --- etc/NEWS | 3 ++ lisp/url/url-http.el | 105 ++++++++++++++++++++++++++++++++++++++++++++------- 2 files changed, 94 insertions(+), 14 deletions(-) diff --git a/etc/NEWS b/etc/NEWS index 4414625..7d2cc92 100644 --- a/etc/NEWS +++ b/etc/NEWS @@ -1193,6 +1193,9 @@ plist will contain a :peer element that has the output of programmatically delete all cookies, or cookies from a specific domain. ++++ +*** The URL package now support https over proxy. + ** Tramp +++ diff --git a/lisp/url/url-http.el b/lisp/url/url-http.el index 33f6d11..4f180ed 100644 --- a/lisp/url/url-http.el +++ b/lisp/url/url-http.el @@ -197,7 +197,14 @@ request.") ;; `url-open-stream' needs a buffer in which to do things ;; like authentication. But we use another buffer afterwards. (unwind-protect - (let ((proc (url-open-stream host buf host port gateway-method))) + (let ((proc (url-open-stream host buf + (if url-using-proxy + (url-host url-using-proxy) + host) + (if url-using-proxy + (url-port url-using-proxy) + port) + gateway-method))) ;; url-open-stream might return nil. (when (processp proc) ;; Drop the temp buffer link before killing the buffer. @@ -925,7 +932,13 @@ should be shown to the user." (erase-buffer) (let ((url-request-method url-http-method) (url-request-extra-headers url-http-extra-headers) - (url-request-data url-http-data)) + (url-request-data url-http-data) + (url-using-proxy (url-find-proxy-for-url + url-current-object + (url-host url-current-object)))) + (when url-using-proxy + (setq url-using-proxy + (url-generic-parse-url url-using-proxy))) (url-http url-current-object url-callback-function url-callback-arguments (current-buffer))))) ((url-http-parse-headers) @@ -1209,17 +1222,20 @@ The return value of this function is the retrieval buffer." (nsm-noninteractive (or url-request-noninteractive (and (boundp 'url-http-noninteractive) url-http-noninteractive))) - (connection (url-http-find-free-connection host port gateway-method)) + (connection (url-http-find-free-connection (url-host url) + (url-port url) + gateway-method)) (mime-accept-string url-mime-accept-string) (buffer (or retry-buffer (generate-new-buffer - (format " *http %s:%d*" host port))))) + (format " *http %s:%d*" (url-host url) (url-port url)))))) (if (not connection) ;; Failed to open the connection for some reason (progn (kill-buffer buffer) (setq buffer nil) - (error "Could not create connection to %s:%d" host port)) + (error "Could not create connection to %s:%d" (url-host url) + (url-port url))) (with-current-buffer buffer (mm-disable-multibyte) (setq url-current-object url @@ -1275,13 +1291,72 @@ The return value of this function is the retrieval buffer." (set-process-sentinel connection 'url-http-async-sentinel)) (`failed ;; Asynchronous connection failed - (error "Could not create connection to %s:%d" host port)) + (error "Could not create connection to %s:%d" (url-host url) + (url-port url))) (_ - (set-process-sentinel connection - 'url-http-end-of-document-sentinel) - (process-send-string connection (url-http-create-request)))))) + (if (and url-http-proxy (string= "https" + (url-type url-current-object))) + (url-https-proxy-connect connection) + (set-process-sentinel connection + 'url-http-end-of-document-sentinel) + (process-send-string connection (url-http-create-request))))))) buffer)) +(defun url-https-proxy-connect (connection) + (setq url-http-after-change-function 'url-https-proxy-after-change-function) + (process-send-string connection (format (concat "CONNECT %s:%d HTTP/1.1\r\n" + "Host: %s\r\n" + "\r\n") + (url-host url-current-object) + (or (url-port url-current-object) + url-https-default-port) + (url-host url-current-object)))) + +(defun url-https-proxy-after-change-function (st nd length) + (let* ((process-buffer (current-buffer)) + (proc (get-buffer-process process-buffer))) + (goto-char (point-min)) + (when (re-search-forward "^\r?\n" nil t) + (backward-char 1) + ;; Saw the end of the headers + (setq url-http-end-of-headers (set-marker (make-marker) (point))) + (url-http-parse-response) + (cond + ((null url-http-response-status) + ;; We got back a headerless malformed response from the + ;; server. + (url-http-activate-callback) + (error "Malformed response from proxy, fail!")) + ((= url-http-response-status 200) + (if (gnutls-available-p) + (condition-case e + (let ((tls-connection (gnutls-negotiate + :process proc + :hostname (url-host url-current-object) + :verify-error nil))) + ;; check certificate validity + (setq tls-connection + (nsm-verify-connection tls-connection + (url-host url-current-object) + (url-port url-current-object))) + (with-current-buffer process-buffer (erase-buffer)) + (set-process-buffer tls-connection process-buffer) + (setq url-http-after-change-function + 'url-http-wait-for-headers-change-function) + (set-process-filter tls-connection 'url-http-generic-filter) + (process-send-string tls-connection + (url-http-create-request))) + (gnutls-error + (url-http-activate-callback) + (error "gnutls-error: %s" e)) + (error + (url-http-activate-callback) + (error "error: %s" e))) + (error "error: gnutls support needed!"))) + (t + (url-http-activate-callback) + (message "error response: %d" url-http-response-status)))))) + (defun url-http-async-sentinel (proc why) ;; We are performing an asynchronous connection, and a status change ;; has occurred. @@ -1293,11 +1368,13 @@ The return value of this function is the retrieval buffer." (url-http-end-of-document-sentinel proc why)) ((string= (substring why 0 4) "open") (setq url-http-connection-opened t) - (condition-case error - (process-send-string proc (url-http-create-request)) - (file-error - (setq url-http-connection-opened nil) - (message "HTTP error: %s" error)))) + (if (and url-http-proxy (string= "https" (url-type url-current-object))) + (url-https-proxy-connect proc) + (condition-case error + (process-send-string proc (url-http-create-request)) + (file-error + (setq url-http-connection-opened nil) + (message "HTTP error: %s" error))))) (t (setf (car url-callback-arguments) (nconc (list :error (list 'error 'connection-failed why -- 2.7.4 --=-=-= Content-Type: text/plain Feel free to modify the patch, I'm not very familiar with that :) But I hope the bug can be fixed ASAP since I don't want to modify it every time when udpate emacs daily build. -- Emacs/Gnus --=-=-=--