From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED.blaine.gmane.org!not-for-mail From: Collin Day Newsgroups: gmane.emacs.bugs Subject: bug#35969: 26.2, Excorporate Date: Tue, 9 Jul 2019 16:53:39 -0600 Message-ID: References: Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Injection-Info: blaine.gmane.org; posting-host="blaine.gmane.org:195.159.176.226"; logging-data="8207"; mail-complaints-to="usenet@blaine.gmane.org" Cc: Andreas Schwab , 35969@debbugs.gnu.org To: Thomas Fitzsimmons Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Wed Jul 10 00:55:42 2019 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([209.51.188.17]) by blaine.gmane.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.89) (envelope-from ) id 1hkz1R-0001zx-Db for geb-bug-gnu-emacs@m.gmane.org; Wed, 10 Jul 2019 00:55:41 +0200 Original-Received: from localhost ([::1]:56672 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hkz1Q-0000nA-ET for geb-bug-gnu-emacs@m.gmane.org; Tue, 09 Jul 2019 18:55:40 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:51024) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hkz0z-0000n2-97 for bug-gnu-emacs@gnu.org; Tue, 09 Jul 2019 18:55:15 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hkz0v-0006uD-4r for bug-gnu-emacs@gnu.org; Tue, 09 Jul 2019 18:55:11 -0400 Original-Received: from debbugs.gnu.org ([209.51.188.43]:53903) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1hkz0o-0006pB-8A for bug-gnu-emacs@gnu.org; Tue, 09 Jul 2019 18:55:05 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1hkz0o-0001QK-2q for bug-gnu-emacs@gnu.org; Tue, 09 Jul 2019 18:55:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Collin Day Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Tue, 09 Jul 2019 22:55:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 35969 X-GNU-PR-Package: emacs Original-Received: via spool by 35969-submit@debbugs.gnu.org id=B35969.15627128455404 (code B ref 35969); Tue, 09 Jul 2019 22:55:02 +0000 Original-Received: (at 35969) by debbugs.gnu.org; 9 Jul 2019 22:54:05 +0000 Original-Received: from localhost ([127.0.0.1]:34491 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hkyzn-0001Oj-NJ for submit@debbugs.gnu.org; Tue, 09 Jul 2019 18:54:05 -0400 Original-Received: from mail-wr1-f48.google.com ([209.85.221.48]:45931) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hkyzk-0001OV-SL for 35969@debbugs.gnu.org; Tue, 09 Jul 2019 18:53:58 -0400 Original-Received: by mail-wr1-f48.google.com with SMTP id f9so436243wre.12 for <35969@debbugs.gnu.org>; Tue, 09 Jul 2019 15:53:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=E599mgmW7Soo3zoHF5cYc/R3YYjRMLPyGw1EFlSc3Tw=; b=OoL2ImJxf9LP3lPA656QGUmad1CfhSLj8gYE+jQIn5JX0vIHZbOm9cNR/ct+vkF/az E2TelBXgV5JzJp9wO56KtgG7qXS566xVzMpWndJ5lrxbPAq4ufX5Q+QUKWVkHXnqF0eJ kY+zSdGUOg7ODai9MuBTNO5x7v1N8Fc59440FMEvXGByfJSjL8r7NVrf/jHlKe2j2q6q vAHDA9FyNNGAcZpNGdYBThJ80nGdQxbl2DXE1+MboF5o9n5ZfL/myevivMHYCap1fkQn LGTWFZJ22sn+c+5sFVIyfY0/XsRZGybFRbpvBNu6yvhyXYeV/F+0AUSHgYb3uPvqIUQ0 gdXg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=E599mgmW7Soo3zoHF5cYc/R3YYjRMLPyGw1EFlSc3Tw=; b=jGNMAxDLbU7Jo8QV89Xqt3bE296y9IHtRhRvWohYGGX72XIQXY8HR1dKqbwzjA2rsd UJC0Mwoic7ebj9uyhD7pw7oT6V2fQseB3addBMrkYSRcmGKVcjAwrrj7nyhh++08yr+N zVagGQQquOftai0YeK69RkBadVPRgIYrgawBSklUN8mimAgfUsDX/4STYhdD6dqYdXCT qqfBHLsc9FHiCoZPeExqgs9gvQlxILuQDCsISh/jlrIUlJsET6ZnInded8k/+VDnJ8YK AkdTo3Y03+rh3WbcTNqJR/RgxWVcd4ta3Mo3IOB4UZf89ELXNkPvFmIRlFWYAL4BwRUi 0yjQ== X-Gm-Message-State: APjAAAWW68GppKeXHGjm69mXcYay/HEQkFB/sgwf7bIp7VXKOH/fxpe5 SHSSL1Dg0pHAZ0i+pQ6Z4ss2Oje9i8FjBGTj0Sg= X-Google-Smtp-Source: APXvYqzKd+KV9I2SSVzBpHIbjzsMautB4uRxWTq3vYpDZtoxihyfoyU19JO08IbbEd0JcwGF6Vp7/f7y+ehrdRnvkDA= X-Received: by 2002:adf:dfc4:: with SMTP id q4mr26479886wrn.54.1562712830764; Tue, 09 Jul 2019 15:53:50 -0700 (PDT) In-Reply-To: X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.51.188.43 X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.org gmane.emacs.bugs:162573 Archived-At: Did a git pull and followed instructions above. There are some issues. First, I need to run M-x package-install excorporate four times because I see (each line after each invocation) package--with-response-buffer-1: https://elpa.gnu.org/packages/url-http-ntlm-2.0.4.el: Method not allowed package--with-response-buffer-1: https://elpa.gnu.org/packages/fsm-0.2.1.el: Method not allowed package--with-response-buffer-1: https://elpa.gnu.org/packages/excorporate-0.8.3.tar: Method not allowed After the 4th call it compiles and becomes available. M-x customize-group excorporate, set up for no autoconfig right now, hit apply for current sessions, save not availiable.... M-x excorporate enter uname and password Contacting host: outlook.office365.com:443 error in process filter: exco--parse-xml-in-current-buffer: Server response is not an XML document error in process filter: Server response is not an XML documen *http outlook.office.365.com:443* HTTP/1.1 401 Unauthorized Server: Microsoft-IIS/10.0 request-id: f8e80f64-b51d-4d66-88ef-4ee7f09814f3 X-WSSecurity-Enabled: True X-WSSecurity-For: Logon X-FederationTrustTokenIssuerUri: urn:federation:MicrosoftOnline X-WSSecurity-SymmetricKey-Enabled: True X-WSSecurity-X509Cert-Enabled: True X-OAuth-Enabled: True X-Powered-By: ASP.NET X-FEServer: SN4PR0401CA0031 WWW-Authenticate: Basic Realm="" Date: Tue, 09 Jul 2019 22:48:22 GMT Content-Length: 0 URL-DEBUG http -> Saw end of headers... ( *http outlook.office365.com:443*) http -> url-http-parse-response called in ( *http outlook.office365.com:443*) http -> Got a content-length, being smart about document end. http -> Got 0-length content-length, activating callback immediately. http -> Marking connection as free: outlook.office365.com:443 # http -> url-http-parse-headers called in ( *http outlook.office365.com:443*) http -> url-http-parse-response called in ( *http outlook.office365.com:443*) http -> Parsed HTTP headers: class=4 status=401 http -> Handling normal authentication http -> Found existing connection: outlook.office365.com:443 # http -> Reusing existing connection: outlook.office365.com:443 http -> Marking connection as busy: outlook.office365.com:443 # http -> getting referer from buffer: buffer:# target-url:#s(url "https" nil nil "outlook.office365.com" nil "/EWS/Services.wsdl" nil nil t nil t t) lastloc:nil http -> Finished parsing HTTP headers: nil http -> Spinning waiting for headers... http -> Calling after change function `url-https-proxy-after-change-function' for `#' http -> url-http-parse-response called in ( *http outlook.office365.com:443*-535296) http -> error response: 400 http -> Marking connection as free: outlook.office365.com:443 # http -> Activating callback in buffer ( *http outlook.office365.com:443*-535296): #[257 "p\303\304\305\306\307 !\310\"\311$\216 @\312=\203$\0\313\301\314\315\316\302\"#\210\317\300\320\"\202/\0\313\301\321\322 #\210\317\300\323\")\207" [fsm-exco--fsm-0 (:identifier ("my email" . "https://outlook.office365.com/EWS/Excahnge.asmx") :mail-address "my email" :retrying nil :autodiscovery-urls nil :service-url "https://outlook.office365.com/EWS/Excahnge.asmx" :service-xml nil :service-wsdl nil :next-state-after-success nil :failure-message nil :server-version nil) "https://outlook.office365.com/EWS/Services.wsdl" make-byte-code 0 "\301\300!\205 \0\302\300!\207" vconcat vector [buffer-live-p kill-buffer] 2 :error plist-put :failure-message format "Failed to retrieve %s" fsm-send :unrecoverable-error :service-xml exco--parse-xml-in-current-buffer :success] 8 " (fn STATUS)"] ((:peer (:certificates ((:version 3 :serial-number "0a:48:28:08:de:bc:a4:10:92:0f:87:6a:63:05:a1:e9" :issuer "C=US,O=DigiCert Inc,CN=DigiCert Cloud Services CA-1" :valid-from "2018-11-18" :valid-to "2020-11-18" :subject "C=US,ST=Washington,L=Redmond,O=Microsoft Corporation,CN=outlook.com" :public-key-algorithm "RSA" :certificate-security-level "Medium" :signature-algorithm "RSA-SHA256" :public-key-id "sha1:ae:b5:9e:3f:4e:c3:72:a8:c0:fc:24:4e:24:b4:d6:8d:99:b0:b9:e1" :certificate-id "sha1:05:b8:36:4c:33:e5:63:7e:fd:88:e0:a3:b8:7e:7d:cf:6f:8d:d0:d4") (:version 3 :serial-number "01:9e:c1:c6:bd:3f:59:7b:b2:0c:33:38:e5:51:d8:77" :issuer "C=US,O=DigiCert Inc,OU=www.digicert.com,CN=DigiCert Global Root CA" :valid-from "2015-08-04" :valid-to "2030-08-04" :subject "C=US,O=DigiCert Inc,CN=DigiCert Cloud Services CA-1" :public-key-algorithm "RSA" :certificate-security-level "Medium" :signature-algorithm "RSA-SHA256" :public-key-id "sha1:ac:13:d5:79:03:66:f3:cc:ff:fe:7c:23:71:a7:61:5d:39:20:89:6e" :certificate-id "sha1:81:b6:8d:6c:d2:f2:21:f8:f5:34:e6:77:52:3b:b2:36:bb:a1:dc:56")) :certificate (:version 3 :serial-number "0a:48:28:08:de:bc:a4:10:92:0f:87:6a:63:05:a1:e9" :issuer "C=US,O=DigiCert Inc,CN=DigiCert Cloud Services CA-1" :valid-from "2018-11-18" :valid-to "2020-11-18" :subject "C=US,ST=Washington,L=Redmond,O=Microsoft Corporation,CN=outlook.com" :public-key-algorithm "RSA" :certificate-security-level "Medium" :signature-algorithm "RSA-SHA256" :public-key-id "sha1:ae:b5:9e:3f:4e:c3:72:a8:c0:fc:24:4e:24:b4:d6:8d:99:b0:b9:e1" :certificate-id "sha1:05:b8:36:4c:33:e5:63:7e:fd:88:e0:a3:b8:7e:7d:cf:6f:8d:d0:d4") :key-exchange "ECDHE-RSA" :protocol "TLS1.2" :cipher "AES-256-GCM" :mac "AEAD"))) So it appears not to work, at least on the current head as of today.... Thanks! -C On Tue, Jul 9, 2019 at 4:09 PM Collin Day wrote: > > Sorry, I have not had the chance to. A lot has been going on at my place of employment. It has crossed my mind, and as soon as I have a chance, I will try what you suggested above. Thanks, sorry for the inconvenience. > > On Tue, Jul 9, 2019, 3:53 PM Thomas Fitzsimmons wrote: >> >> Hi, >> >> Were you able to complete building Emacs? I'd like to know if my patch >> is needed on top of Andreas's to make Excorporate work through your >> proxy. >> >> I'm not sure why the built version would interfere with the current >> installation. If you were doing "make install" to a common prefix, that >> might explain it. Instead you can try something like this: >> >> cd emacs-master [your github.com checkout] >> make >> mkdir test-home >> HOME=`pwd`/test-home ./src/emacs -Q >> >> That will ensure you're only running the built Emacs, and completely >> ignoring the packages installed in your home directory. >> >> Thomas >> >> "tenspd137 ." writes: >> >> > I haven't tried the first item on the list yet - ie reloading Andreas' >> > function. i have tried to build emacs from >> > https://github.com/emacs-mirror/emacs.git because I can't reach the >> > official repo from my firewall for whatever reason. I am sure it is >> > something here. Anyway, I was having problems because my built >> > version (27.0.x) was getting seemingly mixed up with my current >> > installation. I can try again later, but right now things are pretty >> > busy at work. >> > >> > Thanks. >> > >> > On Tue, Jun 18, 2019 at 10:26 PM Thomas Fitzsimmons wrote: >> >> >> >> Hi, >> >> >> >> "tenspd137 ." writes: >> >> >> >> > So - I am not sure if I did it correctly, but I copied this function >> >> > with Andreas' changes into a file: >> >> > >> >> > (defun url-https-proxy-after-change-function (_st _nd _length) >> >> > (let* ((process-buffer (current-buffer)) >> >> > (proc (get-buffer-process process-buffer))) >> >> > (goto-char (point-min)) >> >> > (when (re-search-forward "^\r?\n" nil t) >> >> > (backward-char 1) >> >> > ;; Saw the end of the headers >> >> > (setq url-http-end-of-headers (set-marker (make-marker) (point))) >> >> > (url-http-parse-response) >> >> > (cond >> >> > ((null url-http-response-status) >> >> > ;; We got back a headerless malformed response from the >> >> > ;; server. >> >> > (url-http-activate-callback) >> >> > (error "Malformed response from proxy, fail!")) >> >> > ((= url-http-response-status 200) >> >> > (if (gnutls-available-p) >> >> > (condition-case e >> >> > (let ((tls-connection (gnutls-negotiate >> >> > :process proc >> >> > :hostname (url-host url-current-object) >> >> > :verify-error nil))) >> >> > ;; check certificate validity >> >> > (setq tls-connection >> >> > (nsm-verify-connection tls-connection >> >> > (url-host url-current-object) >> >> > (url-port url-current-object))) >> >> > (with-current-buffer process-buffer (erase-buffer)) >> >> > (set-process-buffer tls-connection process-buffer) >> >> > (setq url-http-after-change-function >> >> > 'url-http-wait-for-headers-change-function) >> >> > (set-process-filter tls-connection 'url-http-generic-filter) >> >> > (process-send-string tls-connection >> >> > ;; Use the non-proxy form of the request >> >> > (let (url-http-proxy) >> >> > (url-http-create-request)))) >> >> > (gnutls-error >> >> > (url-http-activate-callback) >> >> > (error "gnutls-error: %s" e)) >> >> > (error >> >> > (url-http-activate-callback) >> >> > (error "error: %s" e))) >> >> > (error "error: gnutls support needed!"))) >> >> > (t >> >> > (url-http-debug "error response: %d" url-http-response-status) >> >> > (url-http-activate-callback)))))) >> >> > >> >> > and then loaded it before running excorporate. After that, I did M-x >> >> > excorporate, and the minibuffer returns: error in process filter: >> >> > Server response is not an XML document >> >> >> >> In this scenario, if you immediately (without restarting Emacs/reloading >> >> anything) re-run M-x excorporate does it still fail? I just want to >> >> make sure that's not a transient failure. If it does fail the second >> >> time, can you post the HTTP response from the server? >> >> >> >> > When I do the similar test by loading the url-http-create-request with >> >> > Thomas's changes, I can get a connection and grab my schedule >> >> > through the proxy. >> >> >> >> OK. >> >> >> >> > Let me know if I need to try something different. >> >> >> >> Are you in a position to build Emacs master tip and retry the experiment >> >> without patching anything? >> >> >> >> Thanks, >> >> Thomas >> >> >> >> > On Mon, Jun 17, 2019 at 4:08 PM Thomas Fitzsimmons wrote: >> >> >> >> >> >> Hi, >> >> >> >> >> >> Good to hear that the patch I posted worked! >> >> >> >> >> >> Yes, that's the patch that Andreas's commit >> >> >> 84613dae5c34ea742dd9a3e56f5acb55f604b483 applied. From what I can tell, >> >> >> you will not have that in Emacs 26.2. >> >> >> >> >> >> Can you try reverting my patch and applying Andreas's patch, and see if >> >> >> Excorporate still works through the proxy? >> >> >> >> >> >> Thanks, >> >> >> Thomas >> >> >> >> >> >> "tenspd137 ." writes: >> >> >> >> >> >> > The patch Thomas seems to work from behind the proxy. My current >> >> >> > emacs version is 26.2, so I would think it would include the commit >> >> >> > Andreas is talking about.... I went and looked it up - is this the >> >> >> > correct commit? >> >> >> > >> >> >> > diff --git a/lisp/url/url-http.el b/lisp/url/url-http.el >> >> >> > index 53798f7..817c5ce 100644 >> >> >> > --- a/lisp/url/url-http.el >> >> >> > +++ b/lisp/url/url-http.el >> >> >> > @@ -1412,7 +1412,9 @@ The return value of this function is the >> >> >> > retrieval buffer." >> >> >> > 'url-http-wait-for-headers-change-function) >> >> >> > (set-process-filter tls-connection 'url-http-generic-filter) >> >> >> > (process-send-string tls-connection >> >> >> > - (url-http-create-request))) >> >> >> > + ;; Use the non-proxy form of the request >> >> >> > + (let (url-http-proxy) >> >> >> > + (url-http-create-request)))) >> >> >> > (gnutls-error >> >> >> > (url-http-activate-callback) >> >> >> > (error "gnutls-error: %s" e)) >> >> >> > >> >> >> > Thanks! >> >> >> > >> >> >> > -C >> >> >> > >> >> >> > >> >> >> > On Sat, Jun 15, 2019 at 1:41 AM Andreas Schwab wrote: >> >> >> >> >> >> >> >> On Jun 14 2019, Thomas Fitzsimmons wrote: >> >> >> >> >> >> >> >> > diff --git a/lisp/url/url-http.el b/lisp/url/url-http.el >> >> >> >> > index 00803a103a..723d111d58 100644 >> >> >> >> > --- a/lisp/url/url-http.el >> >> >> >> > +++ b/lisp/url/url-http.el >> >> >> >> > @@ -329,7 +329,10 @@ url-http-create-request >> >> >> >> > ;; The request >> >> >> >> > (or url-http-method "GET") " " >> >> >> >> > (url-http--encode-string >> >> >> >> > - (if using-proxy (url-recreate-url url-http-target-url) real-fname)) >> >> >> >> > + (if (and using-proxy >> >> >> >> > + (not (equal "https" (url-type url-http-target-url)))) >> >> >> >> > + (url-recreate-url url-http-target-url) >> >> >> >> > + real-fname)) >> >> >> >> >> >> >> >> That should already be handled by commit 84613dae5c. >> >> >> >> >> >> >> >> Andreas. >> >> >> >> >> >> >> >> -- >> >> >> >> Andreas Schwab, schwab@linux-m68k.org >> >> >> >> GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510 2552 DF73 E780 A9DA AEC1 >> >> >> >> "And now for something completely different."