* bug#35969: 26.2, Excorporate
@ 2019-05-28 21:25 tenspd137 .
2019-05-29 1:04 ` Thomas Fitzsimmons
0 siblings, 1 reply; 42+ messages in thread
From: tenspd137 . @ 2019-05-28 21:25 UTC (permalink / raw)
To: 35969
Hello,
I was trying to use Excorporate through a proxy, and it fails. I am
pretty sure that my configuration is correct because I am able to
retrieve *a* file when I use wget and https://server/EWS/Exchange.asmx
and I can get a Services.wsdl file with
https://server/EWS/Exchange.asmx.
If it helps, here is info from URL-DEBUG (with some identifying info removed):
http -> Contacting host: outlook.office365.com:443
http -> Marking connection as busy: outlook.office365.com:443
#<process outlook.office365.com<1>>
http -> Calling after change function
`url-https-proxy-after-change-function' for `#<process
outlook.office365.com<1>>'
http -> url-http-parse-response called in ( *http
outlook.office365.com:443*-608159)
http -> Request is:
GET https://outlook.office365.com/EWS/Services.wsdl HTTP/1.1
MIME-Version: 1.0
Connection: close
Extension: Security/Digest Security/SSL
Host: outlook.office365.com
Accept-encoding: gzip
Accept-charset: utf-8;q=1, gb2312;q=0.5, iso-8859-1;q=0.5, big5;q=0.5,
iso-2022-jp;q=0.5, shift_jis;q=0.5, euc-tw;q=0.5, euc-jp;q=0.5,
euc-jis-2004;q=0.5, euc-kr;q=0.5, us-ascii;q=0.5, utf-7;q=0.5,
hz-gb-2312;q=0.5, big5-hkscs;q=0.5, gbk;q=0.5, gb18030;q=0.5,
iso-8859-5;q=0.5, koi8-r;q=0.5, koi8-u;q=0.5, cp866;q=0.5,
koi8-t;q=0.5, windows-1251;q=0.5, cp855;q=0.5, iso-8859-2;q=0.5,
iso-8859-3;q=0.5, iso-8859-4;q=0.5, iso-8859-9;q=0.5,
iso-8859-10;q=0.5, iso-8859-13;q=0.5, iso-8859-14;q=0.5,
iso-8859-15;q=0.5, windows-1250;q=0.5, windows-1252;q=0.5,
windows-1254;q=0.5, windows-1257;q=0.5, cp775;q=0.5, cp850;q=0.5,
cp852;q=0.5, cp857;q=0.5, cp858;q=0.5, cp860;q=0.5, cp861;q=0.5,
cp863;q=0.5, cp865;q=0.5, cp437;q=0.5, macintosh;q=0.5, next;q=0.5,
hp-roman8;q=0.5, adobe-standard-encoding;q=0.5, iso-8859-16;q=0.5,
iso-8859-7;q=0.5, windows-1253;q=0.5, cp737;q=0.5, cp851;q=0.5,
cp869;q=0.5, iso-8859-8;q=0.5, windows-1255;q=0.5, cp862;q=0.5,
iso-2022-jp-2004;q=0.5, cp874;q=0.5, iso-8859-11;q=0.5, viscii;q=0.5,
windows-1258;q=0.5, iso-8859-6;q=0.5, windows-1256;q=0.5,
iso-2022-cn;q=0.5, iso-2022-cn-ext;q=0.5, iso-2022-jp-2;q=0.5,
iso-2022-kr;q=0.5, utf-16le;q=0.5, utf-16be;q=0.5, utf-16;q=0.5,
x-ctext;q=0.5
Accept: */*
User-Agent: URL/Emacs Emacs/26.2 (X11; x86_64-pc-linux-gnu)
Authorization: Basic ZGF2aWQuYy5kYXlAaHAuY29tOmwxaTNuN2VTa3kxMzcwJSU=
Cookie: ClientId=8998C5691CD143E784857A0D01537963; OIDC=1
http -> Calling after change function
`url-http-wait-for-headers-change-function' for `#<process
outlook.office365.com<1>>'
http -> url-http-wait-for-headers-change-function ( *http
outlook.office365.com:443*-608159)
http -> Saw end of headers... ( *http outlook.office365.com:443*-608159)
http -> url-http-parse-response called in ( *http
outlook.office365.com:443*-608159)
http -> Got a content-length, being smart about document end.
http -> Calling initial content-length for extra data at end of headers
http -> Marking connection as free: outlook.office365.com:443
#<process outlook.office365.com<1>>
http -> url-http-parse-headers called in ( *http
outlook.office365.com:443*-608159)
http -> url-http-parse-response called in ( *http
outlook.office365.com:443*-608159)
http -> Parsed HTTP headers: class=4 status=404
http -> Finished parsing HTTP headers: t
http -> Marking connection as free: outlook.office365.com:443
#<process outlook.office365.com<1>>
http -> Activating callback in buffer ( *http
outlook.office365.com:443*-608159): #[257
"p\303\304\305\306\307 !\310\"\311$\216
@\312=\203$\0\313\301\314\315\316\302\"#\210\317\300\320\"\202/\0\313\301\321\322
#\210\317\300\323\")\207" [fsm-exco--fsm-127 (:identifier
("my-work-email" . "https://outlook.office365.com/EWS/Exchange.asmx")
:mail-address "my-work-email" :retrying nil :autodiscovery-urls nil
:service-url "https://outlook.office365.com/EWS/Exchange.asmx"
:service-xml nil :service-wsdl nil :next-state-after-success nil
:failure-message nil :server-version nil)
"https://outlook.office365.com/EWS/Services.wsdl" make-byte-code 0
"\301\300!\205 \0\302\300!\207" vconcat vector [buffer-live-p
kill-buffer] 2 :error plist-put :failure-message format "Failed to
retrieve %s" fsm-send :unrecoverable-error :service-xml
exco--parse-xml-in-current-buffer :success] 8 "
(fn STATUS)"] ((:error (error http 404) :peer (:certificate (:version
3 :serial-number "03:fa:5c:d7:7f:8a:96:cf:9a:bf:bc:16:5d:a9:8a:0c"
:issuer "C=US,O=DigiCert Inc,CN=DigiCert Cloud Services CA-1"
:valid-from "2018-10-03" :valid-to "2020-10-03" :subject
"C=US,ST=Washington,L=Redmond,O=Microsoft Corporation,CN=outlook.com"
:public-key-algorithm "RSA" :certificate-security-level "Medium"
:signature-algorithm "RSA-SHA256" :public-key-id
"sha1:b5:44:50:ef:ea:49:4c:ea:39:ed:03:22:32:cd:82:54:72:b2:b2:7d"
:certificate-id
"sha1:fb:51:e0:1f:ed:b9:83:bf:0f:41:df:af:4e:4f:25:82:48:84:40:eb")
:key-exchange "ECDHE-RSA" :protocol "TLS1.2" :cipher "AES-256-GCM"
:mac "AEAD")))
http -> Spinning waiting for headers...
excorporate -> Failed: Failed to retrieve
https://outlook.office365.com/EWS/Services.wsdl
Thanks for any help / suggestions.
^ permalink raw reply [flat|nested] 42+ messages in thread* bug#35969: 26.2, Excorporate 2019-05-28 21:25 bug#35969: 26.2, Excorporate tenspd137 . @ 2019-05-29 1:04 ` Thomas Fitzsimmons 2019-05-29 16:17 ` tenspd137 . 0 siblings, 1 reply; 42+ messages in thread From: Thomas Fitzsimmons @ 2019-05-29 1:04 UTC (permalink / raw) To: tenspd137 .; +Cc: 35969 Hi, "tenspd137 ." <dcday137@gmail.com> writes: > I was trying to use Excorporate through a proxy, and it fails. I am > pretty sure that my configuration is correct because I am able to > retrieve *a* file when I use wget and https://server/EWS/Exchange.asmx > and I can get a Services.wsdl file with > https://server/EWS/Exchange.asmx. Thanks for reporting. Is just Emacs (not Excorporate) able to retrieve the file through the proxy? That is, starting from an "emacs -Q" session, can you try just: M-: (setq url-debug t) M-: (url-retrieve-synchronously "https://server/EWS/Exchange.asmx") and see what the *URL-DEBUG* and " *http server:443*" (note leading space character) buffers contain afterwards? Thanks, Thomas ^ permalink raw reply [flat|nested] 42+ messages in thread
* bug#35969: 26.2, Excorporate 2019-05-29 1:04 ` Thomas Fitzsimmons @ 2019-05-29 16:17 ` tenspd137 . 2019-05-29 17:02 ` Thomas Fitzsimmons 0 siblings, 1 reply; 42+ messages in thread From: tenspd137 . @ 2019-05-29 16:17 UTC (permalink / raw) To: Thomas Fitzsimmons; +Cc: 35969 Hi - thanks for the reply. Here are the steps I took (I had to configure the proxy as well....): M-: (setq url-proxy-services '(("http" . "web-proxy:8080") ("https" . "web-proxy:8080"))) M-: (setq url-debug t) M-: (url-retrieve-synchronously "https://server/EWS/Exchange.asmx") -Asks for name and password-, I fill them in..... The buffer contents are: " *http server:443*" HTTP/1.1 401 Unauthorized Server: Microsoft-IIS/10.0 request-id: 12d85c25-a5f7-49d5-a794-ef282ec5e6af X-WSSecurity-Enabled: True X-WSSecurity-For: Logon X-FederationTrustTokenIssuerUri: urn:federation:MicrosoftOnline X-WSSecurity-SymmetricKey-Enabled: True X-WSSecurity-X509Cert-Enabled: True X-OAuth-Enabled: True X-Powered-By: ASP.NET X-FEServer: SN6PR16CA0070 WWW-Authenticate: Basic Realm="" Date: Wed, 29 May 2019 16:00:44 GMT Connection: close Content-Length: 0 and "*URL-DEBUG*" http -> Contacting host: outlook.office365.com:443 http -> Marking connection as busy: outlook.office365.com:443 #<process outlook.office365.com> retrieval -> Spinning in url-retrieve-synchronously: nil (#<buffer *http outlook.office365.com:443*>) http -> Calling after change function `url-https-proxy-after-change-function' for `#<process outlook.office365.com>' http -> url-http-parse-response called in ( *http outlook.office365.com:443*) http -> Request is: GET https://outlook.office365.com/EWS/Exchange.asmx HTTP/1.1 MIME-Version: 1.0 Connection: close Extension: Security/Digest Security/SSL Host: outlook.office365.com Accept-encoding: gzip Accept: */* User-Agent: URL/Emacs Emacs/26.2 (X11; x86_64-pc-linux-gnu) Cookie: ClientId=8998C5691CD143E784857A0D01537963; OIDC=1 retrieval -> Spinning in url-retrieve-synchronously: nil (#<buffer *http outlook.office365.com:443*>) http -> Calling after change function `url-http-wait-for-headers-change-function' for `#<process outlook.office365.com>' http -> url-http-wait-for-headers-change-function ( *http outlook.office365.com:443*) http -> Saw end of headers... ( *http outlook.office365.com:443*) http -> url-http-parse-response called in ( *http outlook.office365.com:443*) http -> Got a content-length, being smart about document end. http -> Got 0-length content-length, activating callback immediately. http -> Marking connection as free: outlook.office365.com:443 #<process outlook.office365.com> http -> url-http-parse-headers called in ( *http outlook.office365.com:443*) http -> url-http-parse-response called in ( *http outlook.office365.com:443*) http -> Parsed HTTP headers: class=4 status=401 http -> Handling normal authentication http -> Contacting host: outlook.office365.com:443 http -> Marking connection as busy: outlook.office365.com:443 #<process outlook.office365.com> http -> Finished parsing HTTP headers: nil http -> Spinning waiting for headers... retrieval -> Spinning in url-retrieve-synchronously: nil (#<buffer *http outlook.office365.com:443*>) http -> Calling after change function `url-https-proxy-after-change-function' for `#<process outlook.office365.com>' http -> url-http-parse-response called in ( *http outlook.office365.com:443*-229815) http -> Request is: GET https://outlook.office365.com/EWS/Exchange.asmx HTTP/1.1 MIME-Version: 1.0 Connection: close Extension: Security/Digest Security/SSL Host: outlook.office365.com Accept-encoding: gzip Accept: */* User-Agent: URL/Emacs Emacs/26.2 (X11; x86_64-pc-linux-gnu) Cookie: ClientId=8998C5691CD143E784857A0D01537963; OIDC=1 Authorization: Basic ZGF2aWQuYy5kYXlAaHAuY29tOmwxaTNuN2VTa3kxMzcwJSU= http -> Calling after change function `url-http-wait-for-headers-change-function' for `#<process outlook.office365.com>' http -> url-http-wait-for-headers-change-function ( *http outlook.office365.com:443*-229815) http -> Saw end of headers... ( *http outlook.office365.com:443*-229815) http -> url-http-parse-response called in ( *http outlook.office365.com:443*-229815) http -> Got a content-length, being smart about document end. http -> Calling initial content-length for extra data at end of headers http -> Marking connection as free: outlook.office365.com:443 #<process outlook.office365.com> http -> url-http-parse-headers called in ( *http outlook.office365.com:443*-229815) http -> url-http-parse-response called in ( *http outlook.office365.com:443*-229815) http -> Parsed HTTP headers: class=4 status=400 http -> Finished parsing HTTP headers: t http -> Marking connection as free: outlook.office365.com:443 #<process outlook.office365.com> http -> Activating callback in buffer ( *http outlook.office365.com:443*-229815): #[128 "\302\303\304p#\210\300\305\240\210\301p\240\207" [(t) (#<buffer *http outlook.office365.com:443*>) url-debug retrieval "Synchronous fetching done (%S)" t] 5 " (fn &rest IGNORED)"] ((:error (error http 400) :peer (:certificate (:version 3 :serial-number "04:78:00:ec:6e:d6:46:74:40:83:3c:83:58:2c:0c:eb" :issuer "C=US,O=DigiCert Inc,CN=DigiCert Cloud Services CA-1" :valid-from "2018-11-19" :valid-to "2020-11-19" :subject "C=US,ST=Washington,L=Redmond,O=Microsoft Corporation,CN=outlook.com" :public-key-algorithm "RSA" :certificate-security-level "Medium" :signature-algorithm "RSA-SHA256" :public-key-id "sha1:18:91:f8:bd:ec:41:11:cd:a8:c8:d0:25:00:0d:a5:e4:10:b4:67:dd" :certificate-id "sha1:f2:1d:79:fd:e3:61:5c:02:2d:48:62:fd:cd:87:b8:6a:d6:60:b6:06") :key-exchange "ECDHE-RSA" :protocol "TLS1.2" :cipher "AES-256-GCM" :mac "AEAD"))) retrieval -> Synchronous fetching done (#<buffer *http outlook.office365.com:443*-229815>) http -> Spinning waiting for headers... But - just for reference - here is what is in the file I receive when I use wget: wget --user me@work --password xxxxxxxxxxx https://outlook.office365.com/EWS/Exchange.asmx <HTML><HEAD><link rel="alternate" type="text/xml" href="https://tu4pr8401mb1231.namprd84.prod.outlook.com:444/EWS/Exchange.asmx?disco"/><STYLE type="text/css">#content{ FONT-SIZE: 0.7em; PADDING-BOTTOM: 2em; MARGIN-LEFT: 30px}BODY{MARGIN-TOP: 0px; MARGIN-LEFT: 0px; COLOR: #000000; FONT-FAMILY: Verdana; BACKGROUND-COLOR: white}P{MARGIN-TOP: 0px; MARGIN-BOTTOM: 12px; COLOR: #000000; FONT-FAMILY: Verdana}PRE{BORDER-RIGHT: #f0f0e0 1px solid; PADDING-RIGHT: 5px; BORDER-TOP: #f0f0e0 1px solid; MARGIN-TOP: -5px; PADDING-LEFT: 5px; FONT-SIZE: 1.2em; PADDING-BOTTOM: 5px; BORDER-LEFT: #f0f0e0 1px solid; PADDING-TOP: 5px; BORDER-BOTTOM: #f0f0e0 1px solid; FONT-FAMILY: Courier New; BACKGROUND-COLOR: #e5e5cc}.heading1{MARGIN-TOP: 0px; PADDING-LEFT: 15px; FONT-WEIGHT: normal; FONT-SIZE: 26px; MARGIN-BOTTOM: 0px; PADDING-BOTTOM: 3px; MARGIN-LEFT: -30px; WIDTH: 100%; COLOR: #ffffff; PADDING-TOP: 10px; FONT-FAMILY: Tahoma; BACKGROUND-COLOR: #003366}.intro{MARGIN-LEFT: -15px}</STYLE><TITLE>Service</TITLE></HEAD><BODY><DIV id="content"><P class="heading1">Service</P><BR/><P class="intro">You have created a service.<P class='intro'>To test this service, you will need to create a client and use it to call the service. You can do this using the svcutil.exe tool from the command line with the following syntax:</P> <BR/><PRE>svcutil.exe <A HREF="https://tu4pr8401mb1231.namprd84.prod.outlook.com:444/EWS/Services.wsdl">https://tu4pr8401mb1231.namprd84.prod.outlook.com:444/EWS/Services.wsdl</A></PRE></P><P class="intro"/>This will generate a configuration file and a code file that contains the client class. Add the two files to your client application and use the generated client class to call the Service. For example:<BR/><P class='intro'><B>C#</B></P><PRE><font color="blue">class </font><font color="teal">Test </font>{ <font color="blue"> static void </font>Main() { <font color="teal">HelloClient</font> client = <font color="blue">new </font><font color="teal">HelloClient</font>(); <font color="green"> // Use the 'client' variable to call operations on the service. </font><font color="green"> // Always close the client. </font> client.Close(); } } </PRE><BR/><P class='intro'><B>Visual Basic</B></P><PRE><font color="blue">Class </font><font color="teal">Test </font><font color="blue"> Shared Sub </font>Main() <font color="blue"> Dim </font>client As <font color="teal">HelloClient</font> = <font color="blue">New </font><font color="teal">HelloClient</font>() <font color="green"> ' Use the 'client' variable to call operations on the service. </font><font color="green"> ' Always close the client. </font> client.Close() <font color="blue"> End Sub Thank you for taking a look and helping me learn some debug tricks. -C PS - Sorry for sending to you twice, I should have hit reply to all to put this in the bug tracker.... On Tue, May 28, 2019 at 7:04 PM Thomas Fitzsimmons <fitzsim@fitzsim.org> wrote: > > Hi, > > "tenspd137 ." <dcday137@gmail.com> writes: > > > I was trying to use Excorporate through a proxy, and it fails. I am > > pretty sure that my configuration is correct because I am able to > > retrieve *a* file when I use wget and https://server/EWS/Exchange.asmx > > and I can get a Services.wsdl file with > > https://server/EWS/Exchange.asmx. > > Thanks for reporting. Is just Emacs (not Excorporate) able to retrieve > the file through the proxy? That is, starting from an "emacs -Q" > session, can you try just: > > M-: (setq url-debug t) > M-: (url-retrieve-synchronously "https://server/EWS/Exchange.asmx") > > and see what the *URL-DEBUG* and " *http server:443*" (note leading > space character) buffers contain afterwards? > > Thanks, > Thomas ^ permalink raw reply [flat|nested] 42+ messages in thread
* bug#35969: 26.2, Excorporate 2019-05-29 16:17 ` tenspd137 . @ 2019-05-29 17:02 ` Thomas Fitzsimmons 2019-05-29 18:13 ` tenspd137 . 0 siblings, 1 reply; 42+ messages in thread From: Thomas Fitzsimmons @ 2019-05-29 17:02 UTC (permalink / raw) To: tenspd137 .; +Cc: 35969 Hi, "tenspd137 ." <dcday137@gmail.com> writes: > Hi - thanks for the reply. Here are the steps I took (I had to > configure the proxy as well....): > > M-: (setq url-proxy-services '(("http" . "web-proxy:8080") ("https" . "web-proxy:8080"))) > M-: (setq url-debug t) > M-: (url-retrieve-synchronously "https://server/EWS/Exchange.asmx") > > -Asks for name and password-, I fill them in..... OK, I think the buffer contents you pasted are fine/expected; it's the web site saying you're unauthorized, which then causes "url" to prompt for credentials. That part seems to have worked through the proxies. Then the url library created a second buffer: " *http outlook.office365.com:443*-229815" which I'm inferring from the logs contained "400 Bad Request" headers. I'm not sure why that might be; I haven't tried Emacs's url proxy support before. That said, I'd like to keep trying to fix this since I'd like Excorporate to work via proxies. You could try an unauthenticated site to see if that works through the proxies, e.g.: [same initial steps] M-: (url-retrieve-synchronously "https://www.gnu.org/") and then a different authenticated site, like this test page: M-: (url-retrieve-synchronously "https://httpbin.org/basic-auth/user/passwd") Username: user Password: passwd If that all works, then we'll have to figure out why url proxy support is not working with the Exchange server specifically. [...] > But - just for reference - here is what is in the file I receive when > I use wget: > > wget --user me@work --password xxxxxxxxxxx https://outlook.office365.com/EWS/Exchange.asmx > > [successful output] Are you configuring wget to use the same proxies as in the Emacs attempt, via the http_proxy and https_proxy environment variables? Thanks, Thomas ^ permalink raw reply [flat|nested] 42+ messages in thread
* bug#35969: 26.2, Excorporate 2019-05-29 17:02 ` Thomas Fitzsimmons @ 2019-05-29 18:13 ` tenspd137 . 2019-05-29 20:53 ` Thomas Fitzsimmons 0 siblings, 1 reply; 42+ messages in thread From: tenspd137 . @ 2019-05-29 18:13 UTC (permalink / raw) To: Thomas Fitzsimmons; +Cc: 35969 Thomas - For the unauthenticated site, the buffer " *http www.gnu.org:443*" returns HTTP/1.1 200 OK Date: Wed, 29 May 2019 17:31:43 GMT Server: Apache/2.4.7 Content-Location: home.html ..... [successfuol output] ..... and URL-DEBUG http -> Contacting host: www.gnu.org:443 http -> Marking connection as busy: www.gnu.org:443 #<process www.gnu.org> retrieval -> Spinning in url-retrieve-synchronously: nil (#<buffer *http www.gnu.org:443*>) retrieval -> Spinning in url-retrieve-synchronously: nil (#<buffer *http www.gnu.org:443*>) http -> Calling after change function `url-https-proxy-after-change-function' for `#<process www.gnu.org>' http -> url-http-parse-response called in ( *http www.gnu.org:443*) http -> Request is: GET https://www.gnu.org/ HTTP/1.1 MIME-Version: 1.0 Connection: close Extension: Security/Digest Security/SSL Host: www.gnu.org Accept-encoding: gzip Accept: */* User-Agent: URL/Emacs Emacs/26.2 (X11; x86_64-pc-linux-gnu) retrieval -> Spinning in url-retrieve-synchronously: nil (#<buffer *http www.gnu.org:443*>) http -> Calling after change function `url-http-wait-for-headers-change-function' for `#<process www.gnu.org>' http -> url-http-wait-for-headers-change-function ( *http www.gnu.org:443*) http -> Saw end of headers... ( *http www.gnu.org:443*) http -> url-http-parse-response called in ( *http www.gnu.org:443*) http -> Got a content-length, being smart about document end. http -> Calling initial content-length for extra data at end of headers http -> Spinning waiting for headers... http -> Calling after change function `url-http-content-length-after-change-function' for `#<process www.gnu.org>' http -> Calling after change function `url-http-content-length-after-change-function' for `#<process www.gnu.org>' http -> Calling after change function `url-http-content-length-after-change-function' for `#<process www.gnu.org>' http -> Calling after change function `url-http-content-length-after-change-function' for `#<process www.gnu.org>' http -> Calling after change function `url-http-content-length-after-change-function' for `#<process www.gnu.org>' http -> Marking connection as free: www.gnu.org:443 #<process www.gnu.org> http -> url-http-parse-headers called in ( *http www.gnu.org:443*) http -> url-http-parse-response called in ( *http www.gnu.org:443*) http -> Parsed HTTP headers: class=2 status=200 http -> Finished parsing HTTP headers: t http -> Marking connection as free: www.gnu.org:443 #<process www.gnu.org> http -> Activating callback in buffer ( *http www.gnu.org:443*): #[128 "\302\303\304p#\210\300\305\240\210\301p\240\207" [(nil) (#<buffer *http www.gnu.org:443*>) url-debug retrieval "Synchronous fetching done (%S)" t] 5 " (fn &rest IGNORED)"] ((:peer (:certificate (:version 3 :serial-number "03:da:73:db:ff:c6:23:a4:16:f3:45:6f:fe:05:8e:04:b8:c2" :issuer "C=US,O=Let's Encrypt,CN=Let's Encrypt Authority X3" :valid-from "2019-04-02" :valid-to "2019-07-01" :subject "CN=emacs.org" :public-key-algorithm "RSA" :certificate-security-level "Medium" :signature-algorithm "RSA-SHA256" :public-key-id "sha1:89:0c:f7:08:4e:65:16:0b:43:08:43:eb:e3:33:77:fa:fb:1c:a5:70" :certificate-id "sha1:0d:61:97:26:9a:58:d7:04:de:52:1f:29:ca:45:55:ec:67:a9:b1:60") :key-exchange "ECDHE-RSA" :protocol "TLS1.2" :cipher "AES-128-GCM" :mac "AEAD"))) retrieval -> Synchronous fetching done (#<buffer *http www.gnu.org:443*>) The authenticated site buffer " *http httpbin.org:443" returns HTTP/1.1 401 UNAUTHORIZED Access-Control-Allow-Credentials: true Access-Control-Allow-Origin: * Date: Wed, 29 May 2019 17:38:27 GMT Referrer-Policy: no-referrer-when-downgrade Server: nginx WWW-Authenticate: Basic realm="Fake Realm" X-Content-Type-Options: nosniff X-Frame-Options: DENY X-XSS-Protection: 1; mode=block Content-Length: 0 Connection: Close and the URL-DEBUG buffer gives http -> Contacting host: httpbin.org:443 http -> Marking connection as busy: httpbin.org:443 #<process httpbin.org> retrieval -> Spinning in url-retrieve-synchronously: nil (#<buffer *http httpbin.org:443*>) http -> Calling after change function `url-https-proxy-after-change-function' for `#<process httpbin.org>' http -> url-http-parse-response called in ( *http httpbin.org:443*) http -> Request is: GET https://httpbin.org/basic-auth/user/passwd HTTP/1.1 MIME-Version: 1.0 Connection: close Extension: Security/Digest Security/SSL Host: httpbin.org Accept-encoding: gzip Accept: */* User-Agent: URL/Emacs Emacs/26.2 (X11; x86_64-pc-linux-gnu) retrieval -> Spinning in url-retrieve-synchronously: nil (#<buffer *http httpbin.org:443*>) http -> Calling after change function `url-http-wait-for-headers-change-function' for `#<process httpbin.org>' http -> url-http-wait-for-headers-change-function ( *http httpbin.org:443*) http -> Saw end of headers... ( *http httpbin.org:443*) http -> url-http-parse-response called in ( *http httpbin.org:443*) http -> Got a content-length, being smart about document end. http -> Got 0-length content-length, activating callback immediately. http -> Marking connection as free: httpbin.org:443 #<process httpbin.org> http -> url-http-parse-headers called in ( *http httpbin.org:443*) http -> url-http-parse-response called in ( *http httpbin.org:443*) http -> Parsed HTTP headers: class=4 status=401 http -> Handling normal authentication http -> Contacting host: httpbin.org:443 http -> Marking connection as busy: httpbin.org:443 #<process httpbin.org> http -> Finished parsing HTTP headers: nil http -> Spinning waiting for headers... retrieval -> Spinning in url-retrieve-synchronously: nil (#<buffer *http httpbin.org:443*>) http -> Calling after change function `url-https-proxy-after-change-function' for `#<process httpbin.org>' http -> url-http-parse-response called in ( *http httpbin.org:443*-54346) http -> Request is: GET https://httpbin.org/basic-auth/user/passwd HTTP/1.1 MIME-Version: 1.0 Connection: close Extension: Security/Digest Security/SSL Host: httpbin.org Accept-encoding: gzip Accept: */* User-Agent: URL/Emacs Emacs/26.2 (X11; x86_64-pc-linux-gnu) Authorization: Basic dXNlcjpwYXNzd2Q= http -> Calling after change function `url-http-wait-for-headers-change-function' for `#<process httpbin.org>' http -> url-http-wait-for-headers-change-function ( *http httpbin.org:443*-54346) http -> Saw end of headers... ( *http httpbin.org:443*-54346) http -> url-http-parse-response called in ( *http httpbin.org:443*-54346) http -> Got a content-length, being smart about document end. http -> Calling initial content-length for extra data at end of headers http -> Marking connection as free: httpbin.org:443 #<process httpbin.org> http -> url-http-parse-headers called in ( *http httpbin.org:443*-54346) http -> url-http-parse-response called in ( *http httpbin.org:443*-54346) http -> Parsed HTTP headers: class=2 status=200 http -> Finished parsing HTTP headers: t http -> Marking connection as free: httpbin.org:443 #<process httpbin.org> http -> Activating callback in buffer ( *http httpbin.org:443*-54346): #[128 "\302\303\304p#\210\300\305\240\210\301p\240\207" [(t) (#<buffer *http httpbin.org:443*>) url-debug retrieval "Synchronous fetching done (%S)" t] 5 " (fn &rest IGNORED)"] ((:peer (:certificate (:version 3 :serial-number "06:0c:5d:3e:f4:d0:2e:f5:c1:ac:09:8e:b3:0e:39:ab" :issuer "C=US,O=Amazon,OU=Server CA 1B,CN=Amazon" :valid-from "2019-02-17" :valid-to "2020-03-17" :subject "CN=httpbin.org" :public-key-algorithm "RSA" :certificate-security-level "Medium" :signature-algorithm "RSA-SHA256" :public-key-id "sha1:1b:e3:b6:57:70:0a:15:df:dc:05:20:f2:66:da:8e:2f:74:4b:e8:ed" :certificate-id "sha1:cd:9a:4d:21:af:57:65:e6:d3:be:3a:d0:1c:dc:88:26:b8:96:bb:f7") :key-exchange "ECDHE-RSA" :protocol "TLS1.2" :cipher "AES-128-GCM" :mac "AEAD"))) retrieval -> Synchronous fetching done (#<buffer *http httpbin.org:443*-54346>) http -> Spinning waiting for headers... Just as a check: echo $http_proxy http://web-proxy:8080 echo $https_proxy http://web-proxy:8080 using wget, I get a file with the following contents: wget --user user --password passwd https://httpbin.org/basic-auth/user/passwd --2019-05-29 11:46:25-- https://httpbin.org/basic-auth/user/passwd Resolving web-proxy... xx.xx.xxx.xxx Connecting to web-proxy|xx.xx.xxx.xxx]:8080... connected. Proxy request sent, awaiting response... 401 UNAUTHORIZED Authentication selected: Basic realm="Fake Realm" Reusing existing connection to httpbin.org:443. Proxy request sent, awaiting response... 200 OK Length: 47 [application/json] Saving to: ‘passwd’ passwd 100%[======================================================================================================================>] 47 --.-KB/s in 0s 2019-05-29 11:46:26 (3.59 MB/s) - ‘passwd’ saved [47/47] cat passwd { "authenticated": true, "user": "user" } Thanks! -C } On Wed, May 29, 2019 at 11:02 AM Thomas Fitzsimmons <fitzsim@fitzsim.org> wrote: > > Hi, > > "tenspd137 ." <dcday137@gmail.com> writes: > > > Hi - thanks for the reply. Here are the steps I took (I had to > > configure the proxy as well....): > > > > M-: (setq url-proxy-services '(("http" . "web-proxy:8080") ("https" . "web-proxy:8080"))) > > M-: (setq url-debug t) > > M-: (url-retrieve-synchronously "https://server/EWS/Exchange.asmx") > > > > -Asks for name and password-, I fill them in..... > > OK, I think the buffer contents you pasted are fine/expected; it's the > web site saying you're unauthorized, which then causes "url" to prompt > for credentials. That part seems to have worked through the proxies. > > Then the url library created a second buffer: > > " *http outlook.office365.com:443*-229815" > > which I'm inferring from the logs contained "400 Bad Request" headers. > I'm not sure why that might be; I haven't tried Emacs's url proxy > support before. That said, I'd like to keep trying to fix this since > I'd like Excorporate to work via proxies. > > You could try an unauthenticated site to see if that works through the > proxies, e.g.: > > [same initial steps] > M-: (url-retrieve-synchronously "https://www.gnu.org/") > > and then a different authenticated site, like this test page: > > M-: (url-retrieve-synchronously "https://httpbin.org/basic-auth/user/passwd") > > Username: user > Password: passwd > > If that all works, then we'll have to figure out why url proxy support > is not working with the Exchange server specifically. > > [...] > > > But - just for reference - here is what is in the file I receive when > > I use wget: > > > > wget --user me@work --password xxxxxxxxxxx https://outlook.office365.com/EWS/Exchange.asmx > > > > [successful output] > > Are you configuring wget to use the same proxies as in the Emacs > attempt, via the http_proxy and https_proxy environment variables? > > Thanks, > Thomas ^ permalink raw reply [flat|nested] 42+ messages in thread
* bug#35969: 26.2, Excorporate 2019-05-29 18:13 ` tenspd137 . @ 2019-05-29 20:53 ` Thomas Fitzsimmons 2019-05-29 21:44 ` tenspd137 . 0 siblings, 1 reply; 42+ messages in thread From: Thomas Fitzsimmons @ 2019-05-29 20:53 UTC (permalink / raw) To: tenspd137 .; +Cc: 35969 Hi, "tenspd137 ." <dcday137@gmail.com> writes: > For the unauthenticated site, the buffer " *http www.gnu.org:443*" returns > > HTTP/1.1 200 OK > Date: Wed, 29 May 2019 17:31:43 GMT > Server: Apache/2.4.7 > Content-Location: home.html > ..... > [successful output] [...] > The authenticated site buffer " *http httpbin.org:443" returns [...] > http -> Parsed HTTP headers: class=2 status=200 OK, success through the proxy (both in Emacs and wget) for the simple no and basic authentication cases, interesting. Are you in a position to test authenticating against "https://server/EWS/Exchange.asmx" from within Emacs, without going through the proxy? (I realize this may not be possible in your setup.) I guess I'll have to set up a localhost proxy to debug this further. Thomas ^ permalink raw reply [flat|nested] 42+ messages in thread
* bug#35969: 26.2, Excorporate 2019-05-29 20:53 ` Thomas Fitzsimmons @ 2019-05-29 21:44 ` tenspd137 . 2019-05-29 22:01 ` Thomas Fitzsimmons 0 siblings, 1 reply; 42+ messages in thread From: tenspd137 . @ 2019-05-29 21:44 UTC (permalink / raw) To: Thomas Fitzsimmons; +Cc: 35969 Sorry for questioning - but I don't think it is successful in the basic authentication case - unless I am misreading some of the output. As for testing authentication against no proxy - yes, I can. In fact, I actually have. It works. At work we have a general office lan and a development lan. The bulk of my work is on the development lan, which is where I spend most my time. The office lan is less restricted proxy wise, but I don't use it for various reasons. I took a laptop (running M$ no less) and tried running emacs / excorporate with the same settings minus the proxy setup and had no problem, so it works, just wrong computer. I would be happy to repeat the same steps and grab you some log files if you like, but it might have to wait until tomorrow morning. Sorry I am no good at debugging this stuff myself, but I really don't know what is even supposed to happen under the hood. I probably should have thought of comparing log files myself earlier. Basically, I am trying to condense all of my stuff to the computer I use most so I don't have to constantly switch back and forth. Again, thanks for your time! -C On Wed, May 29, 2019 at 2:54 PM Thomas Fitzsimmons <fitzsim@fitzsim.org> wrote: > > Hi, > > "tenspd137 ." <dcday137@gmail.com> writes: > > > For the unauthenticated site, the buffer " *http www.gnu.org:443*" returns > > > > HTTP/1.1 200 OK > > Date: Wed, 29 May 2019 17:31:43 GMT > > Server: Apache/2.4.7 > > Content-Location: home.html > > ..... > > [successful output] > [...] > > > The authenticated site buffer " *http httpbin.org:443" returns > > [...] > > > http -> Parsed HTTP headers: class=2 status=200 > > OK, success through the proxy (both in Emacs and wget) for the simple no > and basic authentication cases, interesting. > > Are you in a position to test authenticating against > "https://server/EWS/Exchange.asmx" from within Emacs, without going > through the proxy? (I realize this may not be possible in your setup.) > > I guess I'll have to set up a localhost proxy to debug this further. > > Thomas ^ permalink raw reply [flat|nested] 42+ messages in thread
* bug#35969: 26.2, Excorporate 2019-05-29 21:44 ` tenspd137 . @ 2019-05-29 22:01 ` Thomas Fitzsimmons 2019-05-29 22:10 ` tenspd137 . 0 siblings, 1 reply; 42+ messages in thread From: Thomas Fitzsimmons @ 2019-05-29 22:01 UTC (permalink / raw) To: tenspd137 .; +Cc: 35969 Hi, "tenspd137 ." <dcday137@gmail.com> writes: > Sorry for questioning - but I don't think it is successful in the > basic authentication case - unless I am misreading some of the output. No problem; I quoted the "status=200" line which I think indicates that it was successful. But url creates another buffer and puts the authentication response in there; to be sure, you'd need to check that. I think for httpbin.org, it will contain the headers and then: { "authenticated": true, "user": "user" } just like your wget experiment. For your prior test run, the secondary buffer is mentioned in *URL-DEBUG* here: > retrieval -> Synchronous fetching done (#<buffer *http httpbin.org:443*-54346>) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Can you redo the httpbin.org authentication test within Emacs, through the proxies, and paste the contents of that secondary/response buffer? It would be helpful if you could paste the redacted secondary/response buffer contents for the failed Exchange authentication too. > As for testing authentication against no proxy - yes, I can. In fact, > I actually have. It works. [...] OK, good to know, thanks. The use case makes sense. Thanks, Thomas ^ permalink raw reply [flat|nested] 42+ messages in thread
* bug#35969: 26.2, Excorporate 2019-05-29 22:01 ` Thomas Fitzsimmons @ 2019-05-29 22:10 ` tenspd137 . 2019-05-30 17:06 ` tenspd137 . 0 siblings, 1 reply; 42+ messages in thread From: tenspd137 . @ 2019-05-29 22:10 UTC (permalink / raw) To: Thomas Fitzsimmons; +Cc: 35969 Yes - I will try to do it tomorrow morning. Thanks again! -C On Wed, May 29, 2019 at 4:01 PM Thomas Fitzsimmons <fitzsim@fitzsim.org> wrote: > > Hi, > > "tenspd137 ." <dcday137@gmail.com> writes: > > > Sorry for questioning - but I don't think it is successful in the > > basic authentication case - unless I am misreading some of the output. > > No problem; I quoted the "status=200" line which I think indicates that > it was successful. But url creates another buffer and puts the > authentication response in there; to be sure, you'd need to check that. > I think for httpbin.org, it will contain the headers and then: > > { > "authenticated": true, > "user": "user" > } > > just like your wget experiment. > > For your prior test run, the secondary buffer is mentioned in > *URL-DEBUG* here: > > > retrieval -> Synchronous fetching done (#<buffer *http httpbin.org:443*-54346>) > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > > Can you redo the httpbin.org authentication test within Emacs, through > the proxies, and paste the contents of that secondary/response buffer? > > It would be helpful if you could paste the redacted secondary/response > buffer contents for the failed Exchange authentication too. > > > As for testing authentication against no proxy - yes, I can. In fact, > > I actually have. It works. > > [...] > > OK, good to know, thanks. The use case makes sense. > > Thanks, > Thomas ^ permalink raw reply [flat|nested] 42+ messages in thread
* bug#35969: 26.2, Excorporate 2019-05-29 22:10 ` tenspd137 . @ 2019-05-30 17:06 ` tenspd137 . 2019-05-30 17:25 ` tenspd137 . 0 siblings, 1 reply; 42+ messages in thread From: tenspd137 . @ 2019-05-30 17:06 UTC (permalink / raw) To: Thomas Fitzsimmons; +Cc: 35969 Thomas- Here is the httpbin.org authentication test buffer (through emacs / proxy) HTTP/1.1 401 UNAUTHORIZED Access-Control-Allow-Credentials: true Access-Control-Allow-Origin: * Date: Thu, 30 May 2019 15:32:25 GMT Referrer-Policy: no-referrer-when-downgrade Server: nginx WWW-Authenticate: Basic realm="Fake Realm" X-Content-Type-Options: nosniff X-Frame-Options: DENY X-XSS-Protection: 1; mode=block Content-Length: 0 Connection: Close and here is the failed excahnge.asmx buffer HTTP/1.1 401 Unauthorized Server: Microsoft-IIS/10.0 request-id: fb0b957e-f55c-49b9-9fb5-fd9a58a5f31a X-WSSecurity-Enabled: True X-WSSecurity-For: Logon X-FederationTrustTokenIssuerUri: urn:federation:MicrosoftOnline X-WSSecurity-SymmetricKey-Enabled: True X-WSSecurity-X509Cert-Enabled: True X-OAuth-Enabled: True X-Powered-By: ASP.NET X-FEServer: SN6PR1501CA0010 WWW-Authenticate: Basic Realm="" Date: Thu, 30 May 2019 15:36:43 GMT Connection: close Content-Length: 0 and, as a bonus, the result of url-exchange.asmx from Emacs not using a proxy on the "other" lan using url-retrieve-synchronusly HTTP/1.1 401 Unauthorized Server: Microsoft-IIS/10.0 request-id: 369ea2d9-8dce-408c-a38e-fb8035f98065 X-WSSecurity-Enabled: True X-WSSecurity-For: Logon X-FederationTrustTokenIssuerUri: urn:federation:MicrosoftOnline X-WSSecurity-SymmetricKey-Enabled: True X-WSSecurity-X509Cert-Enabled: True X-OAuth-Enabled: True X-Powered-By: ASP.NET X-FEServer: SN4PR0201CA0037 WWW-Authenticate: Basic Realm="" Date: Thu, 30 May 2019 16:40:48 GMT Content-Length: 0 I tried to get logs from a successful exorporate useage, but the http result buffers appear to come and go, and if I try URL-DEBUG, a lot happens and eventually emacs on my laptop locks up, but if you want me to try anything specific or there is a way to look at things step by step, just let me know. Thanks! -C On Wed, May 29, 2019 at 4:10 PM tenspd137 . <dcday137@gmail.com> wrote: > > Yes - I will try to do it tomorrow morning. Thanks again! > > -C > > On Wed, May 29, 2019 at 4:01 PM Thomas Fitzsimmons <fitzsim@fitzsim.org> wrote: > > > > Hi, > > > > "tenspd137 ." <dcday137@gmail.com> writes: > > > > > Sorry for questioning - but I don't think it is successful in the > > > basic authentication case - unless I am misreading some of the output. > > > > No problem; I quoted the "status=200" line which I think indicates that > > it was successful. But url creates another buffer and puts the > > authentication response in there; to be sure, you'd need to check that. > > I think for httpbin.org, it will contain the headers and then: > > > > { > > "authenticated": true, > > "user": "user" > > } > > > > just like your wget experiment. > > > > For your prior test run, the secondary buffer is mentioned in > > *URL-DEBUG* here: > > > > > retrieval -> Synchronous fetching done (#<buffer *http httpbin.org:443*-54346>) > > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > > > > Can you redo the httpbin.org authentication test within Emacs, through > > the proxies, and paste the contents of that secondary/response buffer? > > > > It would be helpful if you could paste the redacted secondary/response > > buffer contents for the failed Exchange authentication too. > > > > > As for testing authentication against no proxy - yes, I can. In fact, > > > I actually have. It works. > > > > [...] > > > > OK, good to know, thanks. The use case makes sense. > > > > Thanks, > > Thomas ^ permalink raw reply [flat|nested] 42+ messages in thread
* bug#35969: 26.2, Excorporate 2019-05-30 17:06 ` tenspd137 . @ 2019-05-30 17:25 ` tenspd137 . [not found] ` <CAB=m8wL0JNK+dQv68nddHuJ8yvHdsV9ehTaGJ00NK4spe1sQAQ@mail.gmail.com> 0 siblings, 1 reply; 42+ messages in thread From: tenspd137 . @ 2019-05-30 17:25 UTC (permalink / raw) To: Thomas Fitzsimmons; +Cc: 35969 So, I don't know what this is indicative of, but if I try to use emacs through a proxy and use the httpbin.org example, the result buffer shows: HTTP/1.1 401 UNAUTHORIZED Access-Control-Allow-Credentials: true Access-Control-Allow-Origin: * Date: Thu, 30 May 2019 17:22:48 GMT Referrer-Policy: no-referrer-when-downgrade Server: nginx WWW-Authenticate: Basic realm="Fake Realm" X-Content-Type-Options: nosniff X-Frame-Options: DENY X-XSS-Protection: 1; mode=block Content-Length: 0 Connection: Close but then if I repeat the url-retrieve-synchrounously call HTTP/1.1 200 OK Access-Control-Allow-Credentials: true Access-Control-Allow-Origin: * Content-Encoding: gzip Content-Type: application/json Date: Thu, 30 May 2019 17:23:59 GMT Referrer-Policy: no-referrer-when-downgrade Server: nginx X-Content-Type-Options: nosniff X-Frame-Options: DENY X-XSS-Protection: 1; mode=block Content-Length: 59 Connection: Close { "authenticated": true, "user": "user" } So - it need two calls - one to authenticate and one to retrieve? Thanks! -C On Thu, May 30, 2019 at 11:06 AM tenspd137 . <dcday137@gmail.com> wrote: > > Thomas- > > Here is the httpbin.org authentication test buffer (through emacs / proxy) > > HTTP/1.1 401 UNAUTHORIZED > Access-Control-Allow-Credentials: true > Access-Control-Allow-Origin: * > Date: Thu, 30 May 2019 15:32:25 GMT > Referrer-Policy: no-referrer-when-downgrade > Server: nginx > WWW-Authenticate: Basic realm="Fake Realm" > X-Content-Type-Options: nosniff > X-Frame-Options: DENY > X-XSS-Protection: 1; mode=block > Content-Length: 0 > Connection: Close > > and here is the failed excahnge.asmx buffer > > HTTP/1.1 401 Unauthorized > Server: Microsoft-IIS/10.0 > request-id: fb0b957e-f55c-49b9-9fb5-fd9a58a5f31a > X-WSSecurity-Enabled: True > X-WSSecurity-For: Logon > X-FederationTrustTokenIssuerUri: urn:federation:MicrosoftOnline > X-WSSecurity-SymmetricKey-Enabled: True > X-WSSecurity-X509Cert-Enabled: True > X-OAuth-Enabled: True > X-Powered-By: ASP.NET > X-FEServer: SN6PR1501CA0010 > WWW-Authenticate: Basic Realm="" > Date: Thu, 30 May 2019 15:36:43 GMT > Connection: close > Content-Length: 0 > > and, as a bonus, the result of url-exchange.asmx from Emacs not using > a proxy on the "other" lan using url-retrieve-synchronusly > > HTTP/1.1 401 Unauthorized > Server: Microsoft-IIS/10.0 > request-id: 369ea2d9-8dce-408c-a38e-fb8035f98065 > X-WSSecurity-Enabled: True > X-WSSecurity-For: Logon > X-FederationTrustTokenIssuerUri: urn:federation:MicrosoftOnline > X-WSSecurity-SymmetricKey-Enabled: True > X-WSSecurity-X509Cert-Enabled: True > X-OAuth-Enabled: True > X-Powered-By: ASP.NET > X-FEServer: SN4PR0201CA0037 > WWW-Authenticate: Basic Realm="" > Date: Thu, 30 May 2019 16:40:48 GMT > Content-Length: 0 > > I tried to get logs from a successful exorporate useage, but the http > result buffers appear to come and go, and if I try URL-DEBUG, a lot > happens and eventually emacs on my laptop locks up, but if you want me > to try anything specific or there is a way to look at things step by > step, just let me know. > > Thanks! > > -C > > > > On Wed, May 29, 2019 at 4:10 PM tenspd137 . <dcday137@gmail.com> wrote: > > > > Yes - I will try to do it tomorrow morning. Thanks again! > > > > -C > > > > On Wed, May 29, 2019 at 4:01 PM Thomas Fitzsimmons <fitzsim@fitzsim.org> wrote: > > > > > > Hi, > > > > > > "tenspd137 ." <dcday137@gmail.com> writes: > > > > > > > Sorry for questioning - but I don't think it is successful in the > > > > basic authentication case - unless I am misreading some of the output. > > > > > > No problem; I quoted the "status=200" line which I think indicates that > > > it was successful. But url creates another buffer and puts the > > > authentication response in there; to be sure, you'd need to check that. > > > I think for httpbin.org, it will contain the headers and then: > > > > > > { > > > "authenticated": true, > > > "user": "user" > > > } > > > > > > just like your wget experiment. > > > > > > For your prior test run, the secondary buffer is mentioned in > > > *URL-DEBUG* here: > > > > > > > retrieval -> Synchronous fetching done (#<buffer *http httpbin.org:443*-54346>) > > > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > > > > > > Can you redo the httpbin.org authentication test within Emacs, through > > > the proxies, and paste the contents of that secondary/response buffer? > > > > > > It would be helpful if you could paste the redacted secondary/response > > > buffer contents for the failed Exchange authentication too. > > > > > > > As for testing authentication against no proxy - yes, I can. In fact, > > > > I actually have. It works. > > > > > > [...] > > > > > > OK, good to know, thanks. The use case makes sense. > > > > > > Thanks, > > > Thomas ^ permalink raw reply [flat|nested] 42+ messages in thread
[parent not found: <CAB=m8wL0JNK+dQv68nddHuJ8yvHdsV9ehTaGJ00NK4spe1sQAQ@mail.gmail.com>]
[parent not found: <m3blze4872.fsf@fitzsim.org>]
[parent not found: <CAB=m8wJfGveEzVm46j2MLGFupdv2Lwn3VwL02_RAdirzDJKcSg@mail.gmail.com>]
[parent not found: <CAB=m8wKv0dWgRctuc+bWyocNOPFertCt9K4SOB2Q_W5cPU_tTw@mail.gmail.com>]
[parent not found: <m3tvd5p8zt.fsf@fitzsim.org>]
[parent not found: <CAB=m8wLKtY_38m3mJ18_qhacRUNVCU-4GEFMAKXAdbc1Xamnxg@mail.gmail.com>]
[parent not found: <m3d0jtp0c7.fsf@fitzsim.org>]
[parent not found: <CAB=m8wL6KqATgAddiC3H7kzLSnbe5xTeAWrq5eqvr6n6aU-OKw@mail.gmail.com>]
[parent not found: <m34l55oywz.fsf@fitzsim.org>]
[parent not found: <CAB=m8wKHstZNzTZxj4m-UuOg9LNeu4LRrhysOaaQrSd6z1DE0A@mail.gmail.com>]
[parent not found: <m3woi1nfxw.fsf@fitzsim.org>]
* bug#35969: 26.2, Excorporate [not found] ` <m3woi1nfxw.fsf@fitzsim.org> @ 2019-06-14 20:13 ` tenspd137 . 2019-06-14 20:16 ` tenspd137 . 2019-06-14 21:02 ` Thomas Fitzsimmons 0 siblings, 2 replies; 42+ messages in thread From: tenspd137 . @ 2019-06-14 20:13 UTC (permalink / raw) To: Thomas Fitzsimmons, 35969 Thomas - I was able to try stepping through an Emacs/proxy/Exchange test in an emacs -Q session. After setting the proxy and configuring the debugger to step through url-http and url-http-async-sentilnel, the only thing I noticed is that it appears url-http-async-sentinel is not being called. I also put the url-https-proxy-connect override you gave me earlier into a file, loaded it and set the debugger to run through that as well as set up proxies. A broken down list of steps: 1. Load file containing proxy and altered url-https-proxyconnect, set debugger to run through it when hit. 2. set up url-http and url-http-async-sentinel to be picked up by debugger 3. eval (url-retrieve-synchronously "https://outlook.office365.com/EWS/Exchange.asmx"), step through url-http 4. Input username and password when asked 5. Continue stepping until end url-http-async-sentinel is never called. " *http* ... -####" has text indicating failure. url-https-proxy-connect is indeed called. I don't know how to look at the actual <process>, if I try to C-x C-e or M-: "connection", it goes into the debugger. The url's, etc look good as far as I can tell. Not sure what else I can do. If there are certain pieces of url-http you want me to look at, just let me know, but not really knowing what has to happen under the hood, I am not going to be able to do much else. Thanks! -C On Tue, Jun 4, 2019 at 4:28 PM Thomas Fitzsimmons <fitzsim@fitzsim.org> wrote: > > "tenspd137 ." <dcday137@gmail.com> writes: > > > Oh - I see. Because you want to look at the *HEADERS* flying around > > in that case. Understood. > > Right. > > > Quick question - using emacs -Q on my proxied machine > > > > (setq url-http-proxy "myproxy") doesn't seem to work, so I have been just doing: > > > > (setq url-proxy-services '(("http" . "myproxy") > > ("https" . "myproxy"))) > > > > then, I evaluate: > > > > (defun url-https-proxy-connect (connection) > > (setq url-http-after-change-function 'url-https-proxy-after-change-function) > > (message "THIS WAS CALLED: url-https-proxy-connect") > > (process-send-string connection (format (concat "CONNECT %s:%d HTTP/1.1\r\n" > > "Host: %s\r\n" > > "\r\n") > > (url-host url-current-object) > > (or (url-port url-current-object) > > url-https-default-port) > > (url-host url-current-object)))) > > > > which, if I understand correctly, adds the messaging to the orgiinal > > url-https-proxy-connect. > > That's correct. > > > Then I evaluate > > > > (url-retrieve-synchronously "https://outlook.office365.com/EWS/Exchange.asmx") > > > > after sending username and password, THIS WAS CALLED does not appear > > in the messages. Am I using it correctly, or is this not being > > called? I have to ask, because at this point in my emacs usage, I > > always bet first that I have done something wrong. > > I think you're doing everything correctly, so this suggests that Emacs > isn't doing any proxy handling, or at least that it is not initiating > the "CONNECT" protocol. > > From the wget output in your other email, it shows wget > connecting to the proxy with CONNECT. This protocol keeps the TLS > "tunnel" through the proxy open, and as far as I know, is required for > Exchange authentication to work through a proxy. > > Then the logs also show: > > Connection: Keep-Alive > Proxy-Connection: Keep-Alive > > which are important; even when you set Connection keep-alive, Emacs was > still sending both Connection close and Connection keep-alive which is > probably not valid. > > You may have reached the limit of what Emacs can currently do for > proxies, but it's up to someone (maybe me) to eventually implement > CONNECT properly if it isn't already. > > A next step would be to repeat your Emacs/proxy/Exchange experiments > until you get url-https-proxy-connect to be called or figure out why it > isn't being called. > > The only two callers are url-http and url-http-async-sentinel. To step > through them with edebug do: > > C-h f url-http RET C-x o TAB RET C-u C-M-x > > Likewise for url-http-async-sentinel, then redo your experiment and step > through the functions with SPC and eval stuff with C-x e to see where > they go off the rails before calling url-https-proxy-connect. > > > Still looking into wget verbosity.... > > I saw the other email, nice; is that output just with the -d option? > > BTW, it seems like Emacs is supposed to read the _proxy environment > variables the way wget does. Have you tried running emacs -Q in an > environment that has http_proxy and https_proxy set, and then not > setting any of the Emacs proxy variables and doing the Exchange > authentication experiment? > > Thanks, > Thomas ^ permalink raw reply [flat|nested] 42+ messages in thread
* bug#35969: 26.2, Excorporate 2019-06-14 20:13 ` tenspd137 . @ 2019-06-14 20:16 ` tenspd137 . 2019-06-14 20:22 ` Noam Postavsky 2019-06-14 21:02 ` Thomas Fitzsimmons 1 sibling, 1 reply; 42+ messages in thread From: tenspd137 . @ 2019-06-14 20:16 UTC (permalink / raw) To: Thomas Fitzsimmons, 35969 *if I try to C-x C-e or M-: "connection", it goes into the debugger.* I mean it gives me a backtrace telling me the variable is empty, even though later in lisp expressions, it is evaluated. I am assuming it is roughly equivalent to a C++ pointer, but I don't know how to query it, etc. Thanks! -C On Fri, Jun 14, 2019 at 2:13 PM tenspd137 . <dcday137@gmail.com> wrote: > > Thomas - > > I was able to try stepping through an Emacs/proxy/Exchange test in an > emacs -Q session. After setting the proxy and configuring the > debugger to step through url-http and url-http-async-sentilnel, the > only thing I noticed is that it appears url-http-async-sentinel is not > being called. I also put the url-https-proxy-connect override you > gave me earlier into a file, loaded it and set the debugger to run > through that as well as set up proxies. A broken down list of steps: > > 1. Load file containing proxy and altered url-https-proxyconnect, set > debugger to run through it when hit. > 2. set up url-http and url-http-async-sentinel to be picked up by debugger > 3. eval (url-retrieve-synchronously > "https://outlook.office365.com/EWS/Exchange.asmx"), step through > url-http > 4. Input username and password when asked > 5. Continue stepping until end > > url-http-async-sentinel is never called. " *http* ... -####" has text > indicating failure. url-https-proxy-connect is indeed called. I don't > know how to look at the actual <process>, if I try to C-x C-e or M-: > "connection", it goes into the debugger. The url's, etc look good as > far as I can tell. Not sure what else I can do. If there are certain > pieces of url-http you want me to look at, just let me know, but not > really knowing what has to happen under the hood, I am not going to be > able to do much else. > > Thanks! > > -C > > > > On Tue, Jun 4, 2019 at 4:28 PM Thomas Fitzsimmons <fitzsim@fitzsim.org> wrote: > > > > "tenspd137 ." <dcday137@gmail.com> writes: > > > > > Oh - I see. Because you want to look at the *HEADERS* flying around > > > in that case. Understood. > > > > Right. > > > > > Quick question - using emacs -Q on my proxied machine > > > > > > (setq url-http-proxy "myproxy") doesn't seem to work, so I have been just doing: > > > > > > (setq url-proxy-services '(("http" . "myproxy") > > > ("https" . "myproxy"))) > > > > > > then, I evaluate: > > > > > > (defun url-https-proxy-connect (connection) > > > (setq url-http-after-change-function 'url-https-proxy-after-change-function) > > > (message "THIS WAS CALLED: url-https-proxy-connect") > > > (process-send-string connection (format (concat "CONNECT %s:%d HTTP/1.1\r\n" > > > "Host: %s\r\n" > > > "\r\n") > > > (url-host url-current-object) > > > (or (url-port url-current-object) > > > url-https-default-port) > > > (url-host url-current-object)))) > > > > > > which, if I understand correctly, adds the messaging to the orgiinal > > > url-https-proxy-connect. > > > > That's correct. > > > > > Then I evaluate > > > > > > (url-retrieve-synchronously "https://outlook.office365.com/EWS/Exchange.asmx") > > > > > > after sending username and password, THIS WAS CALLED does not appear > > > in the messages. Am I using it correctly, or is this not being > > > called? I have to ask, because at this point in my emacs usage, I > > > always bet first that I have done something wrong. > > > > I think you're doing everything correctly, so this suggests that Emacs > > isn't doing any proxy handling, or at least that it is not initiating > > the "CONNECT" protocol. > > > > From the wget output in your other email, it shows wget > > connecting to the proxy with CONNECT. This protocol keeps the TLS > > "tunnel" through the proxy open, and as far as I know, is required for > > Exchange authentication to work through a proxy. > > > > Then the logs also show: > > > > Connection: Keep-Alive > > Proxy-Connection: Keep-Alive > > > > which are important; even when you set Connection keep-alive, Emacs was > > still sending both Connection close and Connection keep-alive which is > > probably not valid. > > > > You may have reached the limit of what Emacs can currently do for > > proxies, but it's up to someone (maybe me) to eventually implement > > CONNECT properly if it isn't already. > > > > A next step would be to repeat your Emacs/proxy/Exchange experiments > > until you get url-https-proxy-connect to be called or figure out why it > > isn't being called. > > > > The only two callers are url-http and url-http-async-sentinel. To step > > through them with edebug do: > > > > C-h f url-http RET C-x o TAB RET C-u C-M-x > > > > Likewise for url-http-async-sentinel, then redo your experiment and step > > through the functions with SPC and eval stuff with C-x e to see where > > they go off the rails before calling url-https-proxy-connect. > > > > > Still looking into wget verbosity.... > > > > I saw the other email, nice; is that output just with the -d option? > > > > BTW, it seems like Emacs is supposed to read the _proxy environment > > variables the way wget does. Have you tried running emacs -Q in an > > environment that has http_proxy and https_proxy set, and then not > > setting any of the Emacs proxy variables and doing the Exchange > > authentication experiment? > > > > Thanks, > > Thomas ^ permalink raw reply [flat|nested] 42+ messages in thread
* bug#35969: 26.2, Excorporate 2019-06-14 20:16 ` tenspd137 . @ 2019-06-14 20:22 ` Noam Postavsky 2019-06-14 20:32 ` tenspd137 . 0 siblings, 1 reply; 42+ messages in thread From: Noam Postavsky @ 2019-06-14 20:22 UTC (permalink / raw) To: tenspd137 .; +Cc: Thomas Fitzsimmons, 35969 "tenspd137 ." <dcday137@gmail.com> writes: > *if I try to C-x C-e or M-: "connection", it goes into the debugger.* > > I mean it gives me a backtrace telling me the variable is empty, even > though later in lisp expressions, it is evaluated. If you're stepping with debug or edebug, use `e' (bound to debugger-eval-expression or edebug-eval-expression, respectively) to evaluate expressions in the context of the code you are stepping through. ^ permalink raw reply [flat|nested] 42+ messages in thread
* bug#35969: 26.2, Excorporate 2019-06-14 20:22 ` Noam Postavsky @ 2019-06-14 20:32 ` tenspd137 . 2019-06-14 20:45 ` tenspd137 . 0 siblings, 1 reply; 42+ messages in thread From: tenspd137 . @ 2019-06-14 20:32 UTC (permalink / raw) To: Noam Postavsky; +Cc: Thomas Fitzsimmons, 35969 Ok - so yes, when I hit e and type "connection", it tells me #<process outlook.office365.com>, so I must have been doing it out of context. On Fri, Jun 14, 2019 at 2:22 PM Noam Postavsky <npostavs@gmail.com> wrote: > > "tenspd137 ." <dcday137@gmail.com> writes: > > > *if I try to C-x C-e or M-: "connection", it goes into the debugger.* > > > > I mean it gives me a backtrace telling me the variable is empty, even > > though later in lisp expressions, it is evaluated. > > If you're stepping with debug or edebug, use `e' (bound to > debugger-eval-expression or edebug-eval-expression, respectively) to > evaluate expressions in the context of the code you are stepping > through. ^ permalink raw reply [flat|nested] 42+ messages in thread
* bug#35969: 26.2, Excorporate 2019-06-14 20:32 ` tenspd137 . @ 2019-06-14 20:45 ` tenspd137 . 0 siblings, 0 replies; 42+ messages in thread From: tenspd137 . @ 2019-06-14 20:45 UTC (permalink / raw) To: Thomas Fitzsimmons, 35969 One thing I noticed is that after entering my username and password, the url-current-object did not contain a user or password. Of course, it could also be that authentication happens during the prompt and that the username ans password is not supposed to be saved there for security reasons. Either way, let me know if there are certain values you want me to look at. Thanks! -C On Fri, Jun 14, 2019 at 2:32 PM tenspd137 . <dcday137@gmail.com> wrote: > > Ok - so yes, when I hit e and type "connection", it tells me #<process > outlook.office365.com>, so I must have been doing it out of context. > > On Fri, Jun 14, 2019 at 2:22 PM Noam Postavsky <npostavs@gmail.com> wrote: > > > > "tenspd137 ." <dcday137@gmail.com> writes: > > > > > *if I try to C-x C-e or M-: "connection", it goes into the debugger.* > > > > > > I mean it gives me a backtrace telling me the variable is empty, even > > > though later in lisp expressions, it is evaluated. > > > > If you're stepping with debug or edebug, use `e' (bound to > > debugger-eval-expression or edebug-eval-expression, respectively) to > > evaluate expressions in the context of the code you are stepping > > through. ^ permalink raw reply [flat|nested] 42+ messages in thread
* bug#35969: 26.2, Excorporate 2019-06-14 20:13 ` tenspd137 . 2019-06-14 20:16 ` tenspd137 . @ 2019-06-14 21:02 ` Thomas Fitzsimmons 2019-06-14 21:48 ` tenspd137 . 1 sibling, 1 reply; 42+ messages in thread From: Thomas Fitzsimmons @ 2019-06-14 21:02 UTC (permalink / raw) To: tenspd137 .; +Cc: 35969 Hi, Thanks for following up with these further test results. "tenspd137 ." <dcday137@gmail.com> writes: > I was able to try stepping through an Emacs/proxy/Exchange test in an > emacs -Q session. After setting the proxy and configuring the > debugger to step through url-http and url-http-async-sentilnel, the > only thing I noticed is that it appears url-http-async-sentinel is not > being called. OK, that's probably expected. I listed it for completeness (all the call sites of url-https-proxy-connect), but I guess it would only be called under url-retrieve, not under url-retrieve-synchronously. > I also put the url-https-proxy-connect override you gave me earlier > into a file, loaded it and set the debugger to run through that as > well as set up proxies. A broken down list of steps: > > 1. Load file containing proxy and altered url-https-proxyconnect, set > debugger to run through it when hit. > 2. set up url-http and url-http-async-sentinel to be picked up by debugger > 3. eval (url-retrieve-synchronously > "https://outlook.office365.com/EWS/Exchange.asmx"), step through > url-http > 4. Input username and password when asked > 5. Continue stepping until end > > url-http-async-sentinel is never called. " *http* ... -####" has text > indicating failure. url-https-proxy-connect is indeed called. OK, it's good to know that url-http ultimately calls url-https-proxy-connect. Unfortunately, despite this, proxying does not work. > I don't know how to look at the actual <process>, if I try to C-x C-e > or M-: "connection", it goes into the debugger. The url's, etc look > good as far as I can tell. Not sure what else I can do. If there are > certain pieces of url-http you want me to look at, just let me know, > but not really knowing what has to happen under the hood, I am not > going to be able to do much else. I think I'll probably have to set up my own test environment. Are you in control of the proxy and its configuration? If so, can you provide rough configuration instructions (proxy software, version, relevant config settings)? If not, that's OK, I can try setting up generic proxy software. Thanks, Thomas ^ permalink raw reply [flat|nested] 42+ messages in thread
* bug#35969: 26.2, Excorporate 2019-06-14 21:02 ` Thomas Fitzsimmons @ 2019-06-14 21:48 ` tenspd137 . 2019-06-14 22:07 ` tenspd137 . 0 siblings, 1 reply; 42+ messages in thread From: tenspd137 . @ 2019-06-14 21:48 UTC (permalink / raw) To: Thomas Fitzsimmons; +Cc: 35969 Sorry - I am not in control of my proxy. So, before I give up on this for today, I was going over the wget logs from trying the same thing. One thing I noticed is: Connection: Keep-Alive Proxy-Connection: Keep-Alive I see stuff in the emacs buffers mentioning Connection, and it says closed (unless marking the connection as busy is the same as keeping it open), but I don't see anything mentioning the proxy connection....? Maybe another place to look? Just some ideas... Thanks! -C On Fri, Jun 14, 2019 at 3:02 PM Thomas Fitzsimmons <fitzsim@fitzsim.org> wrote: > > Hi, > > Thanks for following up with these further test results. > > "tenspd137 ." <dcday137@gmail.com> writes: > > > I was able to try stepping through an Emacs/proxy/Exchange test in an > > emacs -Q session. After setting the proxy and configuring the > > debugger to step through url-http and url-http-async-sentilnel, the > > only thing I noticed is that it appears url-http-async-sentinel is not > > being called. > > OK, that's probably expected. I listed it for completeness (all the > call sites of url-https-proxy-connect), but I guess it would only be > called under url-retrieve, not under url-retrieve-synchronously. > > > I also put the url-https-proxy-connect override you gave me earlier > > into a file, loaded it and set the debugger to run through that as > > well as set up proxies. A broken down list of steps: > > > > 1. Load file containing proxy and altered url-https-proxyconnect, set > > debugger to run through it when hit. > > 2. set up url-http and url-http-async-sentinel to be picked up by debugger > > 3. eval (url-retrieve-synchronously > > "https://outlook.office365.com/EWS/Exchange.asmx"), step through > > url-http > > 4. Input username and password when asked > > 5. Continue stepping until end > > > > url-http-async-sentinel is never called. " *http* ... -####" has text > > indicating failure. url-https-proxy-connect is indeed called. > > OK, it's good to know that url-http ultimately calls > url-https-proxy-connect. Unfortunately, despite this, proxying does not > work. > > > I don't know how to look at the actual <process>, if I try to C-x C-e > > or M-: "connection", it goes into the debugger. The url's, etc look > > good as far as I can tell. Not sure what else I can do. If there are > > certain pieces of url-http you want me to look at, just let me know, > > but not really knowing what has to happen under the hood, I am not > > going to be able to do much else. > > I think I'll probably have to set up my own test environment. Are you > in control of the proxy and its configuration? If so, can you provide > rough configuration instructions (proxy software, version, relevant > config settings)? If not, that's OK, I can try setting up generic proxy > software. > > Thanks, > Thomas ^ permalink raw reply [flat|nested] 42+ messages in thread
* bug#35969: 26.2, Excorporate 2019-06-14 21:48 ` tenspd137 . @ 2019-06-14 22:07 ` tenspd137 . 2019-06-14 23:07 ` tenspd137 . 0 siblings, 1 reply; 42+ messages in thread From: tenspd137 . @ 2019-06-14 22:07 UTC (permalink / raw) To: Thomas Fitzsimmons; +Cc: 35969 Just some more things I noticed from the wget log vs. the emacs buffers: emacs: Accept-encoding:gzip, wget:Accept-encoding: identity emacs: GET https://outlook.office365.com/EWS/Exchange.asmx HTTP/1.1, wget: GET /EWS/Exchange.asmx HTTP/1.1 Don't know if any of that is helpful, but there it is. Thanks! -C On Fri, Jun 14, 2019 at 3:48 PM tenspd137 . <dcday137@gmail.com> wrote: > > Sorry - I am not in control of my proxy. So, before I give up on this > for today, I was going over the wget logs from trying the same thing. > One thing I noticed is: > > Connection: Keep-Alive > Proxy-Connection: Keep-Alive > > I see stuff in the emacs buffers mentioning Connection, and it says > closed (unless marking the connection as busy is the same as keeping > it open), but I don't see anything mentioning the proxy > connection....? Maybe another place to look? > > Just some ideas... > > Thanks! > > -C > > On Fri, Jun 14, 2019 at 3:02 PM Thomas Fitzsimmons <fitzsim@fitzsim.org> wrote: > > > > Hi, > > > > Thanks for following up with these further test results. > > > > "tenspd137 ." <dcday137@gmail.com> writes: > > > > > I was able to try stepping through an Emacs/proxy/Exchange test in an > > > emacs -Q session. After setting the proxy and configuring the > > > debugger to step through url-http and url-http-async-sentilnel, the > > > only thing I noticed is that it appears url-http-async-sentinel is not > > > being called. > > > > OK, that's probably expected. I listed it for completeness (all the > > call sites of url-https-proxy-connect), but I guess it would only be > > called under url-retrieve, not under url-retrieve-synchronously. > > > > > I also put the url-https-proxy-connect override you gave me earlier > > > into a file, loaded it and set the debugger to run through that as > > > well as set up proxies. A broken down list of steps: > > > > > > 1. Load file containing proxy and altered url-https-proxyconnect, set > > > debugger to run through it when hit. > > > 2. set up url-http and url-http-async-sentinel to be picked up by debugger > > > 3. eval (url-retrieve-synchronously > > > "https://outlook.office365.com/EWS/Exchange.asmx"), step through > > > url-http > > > 4. Input username and password when asked > > > 5. Continue stepping until end > > > > > > url-http-async-sentinel is never called. " *http* ... -####" has text > > > indicating failure. url-https-proxy-connect is indeed called. > > > > OK, it's good to know that url-http ultimately calls > > url-https-proxy-connect. Unfortunately, despite this, proxying does not > > work. > > > > > I don't know how to look at the actual <process>, if I try to C-x C-e > > > or M-: "connection", it goes into the debugger. The url's, etc look > > > good as far as I can tell. Not sure what else I can do. If there are > > > certain pieces of url-http you want me to look at, just let me know, > > > but not really knowing what has to happen under the hood, I am not > > > going to be able to do much else. > > > > I think I'll probably have to set up my own test environment. Are you > > in control of the proxy and its configuration? If so, can you provide > > rough configuration instructions (proxy software, version, relevant > > config settings)? If not, that's OK, I can try setting up generic proxy > > software. > > > > Thanks, > > Thomas ^ permalink raw reply [flat|nested] 42+ messages in thread
* bug#35969: 26.2, Excorporate 2019-06-14 22:07 ` tenspd137 . @ 2019-06-14 23:07 ` tenspd137 . 2019-06-14 23:14 ` tenspd137 . 0 siblings, 1 reply; 42+ messages in thread From: tenspd137 . @ 2019-06-14 23:07 UTC (permalink / raw) To: Thomas Fitzsimmons; +Cc: 35969 I think I figured it out. I created a function that changed the line in url-http-create-request so that the request looks like: GET /EWS/Exchange.asmx HTTP/1.1 <----- *no server / host* MIME-Version: 1.0 Connection: close Extension: Security/Digest Security/SSL Host: outlook.office365.com Accept-encoding: gzip Accept: */* User-Agent: URL/Emacs Emacs/26.2 (X11; x86_64-pc-linux-gnu) Cookie: OIDC=1; ClientId=8998C5691CD143E784857A0D01537963 Authorization: Basic ZGF2aWQuYy5kYXlAaHAuY29tOlMxa3kzbGk3bmUwNzMxJSU= instead of GET https://outlook.office365.com/EWS/Exchange.asmx HTTP/1.1 MIME-Version: 1.0 Connection: close Extension: Security/Digest Security/SSL Host: outlook.office365.com Accept-encoding: gzip Accept: */* User-Agent: URL/Emacs Emacs/26.2 (X11; x86_64-pc-linux-gnu) Cookie: OIDC=1; ClientId=8998C5691CD143E784857A0D01537963 Authorization: Basic ZGF2aWQuYy5kYXlAaHAuY29tOlMxa3kzbGk3bmUwNzMxJSU= and the results buffer gives: *http....*-##### HTTP/1.1 200 OK Cache-Control: private Content-Length: 1213 Content-Type: text/html; charset=UTF-8 Content-Encoding: gzip Vary: Accept-Encoding Server: Microsoft-IIS/10.0 request-id: bcd31568-29fd-44e4-935f-35b54d697f33 X-CalculatedFETarget: CY4PR18CU003.internal.outlook.com X-BackEndHttpStatus: 200 Set-Cookie: exchangecookie=6c731ced364846f2a979bfaa84496f1e; expires=Sun, 14-Jun-2020 23:02:57 GMT; path=/; secure; HttpOnly X-FEProxyInfo: CY4PR18CA0059.NAMPRD18.PROD.OUTLOOK.COM X-CalculatedBETarget: CS1PR8401MB1223.NAMPRD84.PROD.OUTLOOK.COM X-BackEndHttpStatus: 200 X-RUM-Validated: 1 X-AspNet-Version: 4.0.30319 X-BeSku: Gen9 X-DiagInfo: CS1PR8401MB1223 X-BEServer: CS1PR8401MB1223 X-FEServer: CY4PR18CA0059 X-Powered-By: ASP.NET X-FEServer: SN4PR0501CA0003 Date: Fri, 14 Jun 2019 23:02:56 GMT Connection: close <HTML><HEAD><link rel="alternate" type="text/xml" href="https://cs1pr8401mb1223.namprd84.prod.outlook.com:444/EWS/Exchange.asmx?disco"/><STYLE type="text/css">#content{ FONT-SIZE: 0.7em; PADDING-BOTTOM: 2em; MARGIN-LEFT: 30px}BODY{MARGIN-TOP: 0px; MARGIN-LEFT: 0px; COLOR: #000000; FONT-FAMILY: Verdana; BACKGROUND-COLOR: white}P{MARGIN-TOP: 0px; MARGIN-BOTTOM: 12px; COLOR: #000000; FONT-FAMILY: Verdana}PRE{BORDER-RIGHT: #f0f0e0 1px solid; PADDING-RIGHT: 5px; BORDER-TOP: #f0f0e0 1px solid; MARGIN-TOP: -5px; PADDING-LEFT: 5px; FONT-SIZE: 1.2em; PADDING-BOTTOM: 5px; BORDER-LEFT: #f0f0e0 1px solid; PADDING-TOP: 5px; BORDER-BOTTOM: #f0f0e0 1px solid; FONT-FAMILY: Courier New; BACKGROUND-COLOR: #e5e5cc}.heading1{MARGIN-TOP: 0px; PADDING-LEFT: 15px; FONT-WEIGHT: normal; FONT-SIZE: 26px; MARGIN-BOTTOM: 0px; PADDING-BOTTOM: 3px; MARGIN-LEFT: -30px; WIDTH: 100%; COLOR: #ffffff; PADDING-TOP: 10px; FONT-FAMILY: Tahoma; BACKGROUND-COLOR: #003366}.intro{MARGIN-LEFT: -15px}</STYLE><TITLE>Service</TITLE></HEAD><BODY><DIV id="content"><P class="heading1">Service</P><BR/><P class="intro">You have created a service.<P class='intro'>To test this service, you will need to create a client and use it to call the service. You can do this using the svcutil.exe tool from the command line with the following syntax:</P> <BR/><PRE>svcutil.exe <A HREF="https://cs1pr8401mb1223.namprd84.prod.outlook.com:444/EWS/Services.wsdl">https://cs1pr8401mb1223.namprd84.prod.outlook.com:444/EWS/Services.wsdl</A></PRE></P><P class="intro"/>This will generate a configuration file and a code file that contains the client class. Add the two files to your client application and use the generated client class to call the Service. For example:<BR/><P class='intro'><B>C#</B></P><PRE><font color="blue">class </font><font color="teal">Test </font>{ <font color="blue"> static void </font>Main() { <font color="teal">HelloClient</font> client = <font color="blue">new </font><font color="teal">HelloClient</font>(); <font color="green"> // Use the 'client' variable to call operations on the service. </font><font color="green"> // Always close the client. </font> client.Close(); } } </PRE><BR/><P class='intro'><B>Visual Basic</B></P><PRE><font color="blue">Class </font><font color="teal">Test </font><font color="blue"> Shared Sub </font>Main() <font color="blue"> Dim </font>client As <font color="teal">HelloClient</font> = <font color="blue">New </font><font color="teal">HelloClient</font>() <font color="green"> ' Use the 'client' variable to call operations on the service. </font><font color="green"> ' Always close the client. </font> client.Close() <font color="blue"> End Sub </font><font color="blue">End Class</font></PRE></DIV></BODY></HTML> which matches the file saved with wget. That has to be worth something.... Thanks! -C On Fri, Jun 14, 2019 at 4:07 PM tenspd137 . <dcday137@gmail.com> wrote: > > Just some more things I noticed from the wget log vs. the emacs buffers: > > emacs: Accept-encoding:gzip, wget:Accept-encoding: identity > > emacs: GET https://outlook.office365.com/EWS/Exchange.asmx HTTP/1.1, > wget: GET /EWS/Exchange.asmx HTTP/1.1 > > Don't know if any of that is helpful, but there it is. > > Thanks! > > -C > > On Fri, Jun 14, 2019 at 3:48 PM tenspd137 . <dcday137@gmail.com> wrote: > > > > Sorry - I am not in control of my proxy. So, before I give up on this > > for today, I was going over the wget logs from trying the same thing. > > One thing I noticed is: > > > > Connection: Keep-Alive > > Proxy-Connection: Keep-Alive > > > > I see stuff in the emacs buffers mentioning Connection, and it says > > closed (unless marking the connection as busy is the same as keeping > > it open), but I don't see anything mentioning the proxy > > connection....? Maybe another place to look? > > > > Just some ideas... > > > > Thanks! > > > > -C > > > > On Fri, Jun 14, 2019 at 3:02 PM Thomas Fitzsimmons <fitzsim@fitzsim.org> wrote: > > > > > > Hi, > > > > > > Thanks for following up with these further test results. > > > > > > "tenspd137 ." <dcday137@gmail.com> writes: > > > > > > > I was able to try stepping through an Emacs/proxy/Exchange test in an > > > > emacs -Q session. After setting the proxy and configuring the > > > > debugger to step through url-http and url-http-async-sentilnel, the > > > > only thing I noticed is that it appears url-http-async-sentinel is not > > > > being called. > > > > > > OK, that's probably expected. I listed it for completeness (all the > > > call sites of url-https-proxy-connect), but I guess it would only be > > > called under url-retrieve, not under url-retrieve-synchronously. > > > > > > > I also put the url-https-proxy-connect override you gave me earlier > > > > into a file, loaded it and set the debugger to run through that as > > > > well as set up proxies. A broken down list of steps: > > > > > > > > 1. Load file containing proxy and altered url-https-proxyconnect, set > > > > debugger to run through it when hit. > > > > 2. set up url-http and url-http-async-sentinel to be picked up by debugger > > > > 3. eval (url-retrieve-synchronously > > > > "https://outlook.office365.com/EWS/Exchange.asmx"), step through > > > > url-http > > > > 4. Input username and password when asked > > > > 5. Continue stepping until end > > > > > > > > url-http-async-sentinel is never called. " *http* ... -####" has text > > > > indicating failure. url-https-proxy-connect is indeed called. > > > > > > OK, it's good to know that url-http ultimately calls > > > url-https-proxy-connect. Unfortunately, despite this, proxying does not > > > work. > > > > > > > I don't know how to look at the actual <process>, if I try to C-x C-e > > > > or M-: "connection", it goes into the debugger. The url's, etc look > > > > good as far as I can tell. Not sure what else I can do. If there are > > > > certain pieces of url-http you want me to look at, just let me know, > > > > but not really knowing what has to happen under the hood, I am not > > > > going to be able to do much else. > > > > > > I think I'll probably have to set up my own test environment. Are you > > > in control of the proxy and its configuration? If so, can you provide > > > rough configuration instructions (proxy software, version, relevant > > > config settings)? If not, that's OK, I can try setting up generic proxy > > > software. > > > > > > Thanks, > > > Thomas ^ permalink raw reply [flat|nested] 42+ messages in thread
* bug#35969: 26.2, Excorporate 2019-06-14 23:07 ` tenspd137 . @ 2019-06-14 23:14 ` tenspd137 . 2019-06-15 0:14 ` Thomas Fitzsimmons 2019-06-15 0:36 ` Thomas Fitzsimmons 0 siblings, 2 replies; 42+ messages in thread From: tenspd137 . @ 2019-06-14 23:14 UTC (permalink / raw) To: Thomas Fitzsimmons; +Cc: 35969 Sorry - I was working fast: *I created a function that changed the line in url-http-create-request so that the request looks like:* In a file I did: (defun thefile() "/EWS/Exchange.asmx") (copied from url-http library) (defun url-http-create-request (&optional ref-url) ..... ;; This was done with a call to `format'. Concatenating parts has ;; the advantage of keeping the parts of each header together and ;; allows us to elide null lines directly, at the cost of making ;; the layout less clear. (setq request (concat ;; The request (or url-http-method "GET") " " (url-http--encode-string (if using-proxy (thefile) real-fname)) <------------Changed this to "hardcode" the request as proof of concept " HTTP/" url-http-version "\r\n" ;; Version of MIME we speak "MIME-Version: 1.0\r\n" ;; (maybe) Try to keep the connection open "Connection: " (if (or using-proxy ..... ....... ....... Loaded the file and then ran (url-retriev-synchronously "https://......") Thanks! -C On Fri, Jun 14, 2019 at 5:07 PM tenspd137 . <dcday137@gmail.com> wrote: > > I think I figured it out. I created a function that changed the line > in url-http-create-request so that the request looks like: > GET /EWS/Exchange.asmx HTTP/1.1 <----- *no server / host* > MIME-Version: 1.0 > Connection: close > Extension: Security/Digest Security/SSL > Host: outlook.office365.com > Accept-encoding: gzip > Accept: */* > User-Agent: URL/Emacs Emacs/26.2 (X11; x86_64-pc-linux-gnu) > Cookie: OIDC=1; ClientId=8998C5691CD143E784857A0D01537963 > Authorization: Basic ZGF2aWQuYy5kYXlAaHAuY29tOlMxa3kzbGk3bmUwNzMxJSU= > > instead of > > GET https://outlook.office365.com/EWS/Exchange.asmx HTTP/1.1 > MIME-Version: 1.0 > Connection: close > Extension: Security/Digest Security/SSL > Host: outlook.office365.com > Accept-encoding: gzip > Accept: */* > User-Agent: URL/Emacs Emacs/26.2 (X11; x86_64-pc-linux-gnu) > Cookie: OIDC=1; ClientId=8998C5691CD143E784857A0D01537963 > Authorization: Basic ZGF2aWQuYy5kYXlAaHAuY29tOlMxa3kzbGk3bmUwNzMxJSU= > > and the results buffer gives: *http....*-##### > HTTP/1.1 200 OK > Cache-Control: private > Content-Length: 1213 > Content-Type: text/html; charset=UTF-8 > Content-Encoding: gzip > Vary: Accept-Encoding > Server: Microsoft-IIS/10.0 > request-id: bcd31568-29fd-44e4-935f-35b54d697f33 > X-CalculatedFETarget: CY4PR18CU003.internal.outlook.com > X-BackEndHttpStatus: 200 > Set-Cookie: exchangecookie=6c731ced364846f2a979bfaa84496f1e; > expires=Sun, 14-Jun-2020 23:02:57 GMT; path=/; secure; HttpOnly > X-FEProxyInfo: CY4PR18CA0059.NAMPRD18.PROD.OUTLOOK.COM > X-CalculatedBETarget: CS1PR8401MB1223.NAMPRD84.PROD.OUTLOOK.COM > X-BackEndHttpStatus: 200 > X-RUM-Validated: 1 > X-AspNet-Version: 4.0.30319 > X-BeSku: Gen9 > X-DiagInfo: CS1PR8401MB1223 > X-BEServer: CS1PR8401MB1223 > X-FEServer: CY4PR18CA0059 > X-Powered-By: ASP.NET > X-FEServer: SN4PR0501CA0003 > Date: Fri, 14 Jun 2019 23:02:56 GMT > Connection: close > > <HTML><HEAD><link rel="alternate" type="text/xml" > href="https://cs1pr8401mb1223.namprd84.prod.outlook.com:444/EWS/Exchange.asmx?disco"/><STYLE > type="text/css">#content{ FONT-SIZE: 0.7em; PADDING-BOTTOM: 2em; > MARGIN-LEFT: 30px}BODY{MARGIN-TOP: 0px; MARGIN-LEFT: 0px; COLOR: > #000000; FONT-FAMILY: Verdana; BACKGROUND-COLOR: white}P{MARGIN-TOP: > 0px; MARGIN-BOTTOM: 12px; COLOR: #000000; FONT-FAMILY: > Verdana}PRE{BORDER-RIGHT: #f0f0e0 1px solid; PADDING-RIGHT: 5px; > BORDER-TOP: #f0f0e0 1px solid; MARGIN-TOP: -5px; PADDING-LEFT: 5px; > FONT-SIZE: 1.2em; PADDING-BOTTOM: 5px; BORDER-LEFT: #f0f0e0 1px solid; > PADDING-TOP: 5px; BORDER-BOTTOM: #f0f0e0 1px solid; FONT-FAMILY: > Courier New; BACKGROUND-COLOR: #e5e5cc}.heading1{MARGIN-TOP: 0px; > PADDING-LEFT: 15px; FONT-WEIGHT: normal; FONT-SIZE: 26px; > MARGIN-BOTTOM: 0px; PADDING-BOTTOM: 3px; MARGIN-LEFT: -30px; WIDTH: > 100%; COLOR: #ffffff; PADDING-TOP: 10px; FONT-FAMILY: Tahoma; > BACKGROUND-COLOR: #003366}.intro{MARGIN-LEFT: > -15px}</STYLE><TITLE>Service</TITLE></HEAD><BODY><DIV id="content"><P > class="heading1">Service</P><BR/><P class="intro">You have created a > service.<P class='intro'>To test this service, you will need to create > a client and use it to call the service. You can do this using the > svcutil.exe tool from the command line with the following syntax:</P> > <BR/><PRE>svcutil.exe <A > HREF="https://cs1pr8401mb1223.namprd84.prod.outlook.com:444/EWS/Services.wsdl">https://cs1pr8401mb1223.namprd84.prod.outlook.com:444/EWS/Services.wsdl</A></PRE></P><P > class="intro"/>This will generate a configuration file and a code file > that contains the client class. Add the two files to your client > application and use the generated client class to call the Service. > For example:<BR/><P class='intro'><B>C#</B></P><PRE><font > color="blue">class </font><font color="teal">Test > </font>{ > <font color="blue"> static void </font>Main() > { > <font color="teal">HelloClient</font> client = <font > color="blue">new </font><font color="teal">HelloClient</font>(); > > <font color="green"> // Use the 'client' variable to call > operations on the service. > > </font><font color="green"> // Always close the client. > </font> client.Close(); > } > } > </PRE><BR/><P class='intro'><B>Visual Basic</B></P><PRE><font > color="blue">Class </font><font color="teal">Test > </font><font color="blue"> Shared Sub </font>Main() > <font color="blue"> Dim </font>client As <font > color="teal">HelloClient</font> = <font color="blue">New </font><font > color="teal">HelloClient</font>() > <font color="green"> ' Use the 'client' variable to call > operations on the service. > > </font><font color="green"> ' Always close the client. > </font> client.Close() > <font color="blue"> End Sub > </font><font color="blue">End Class</font></PRE></DIV></BODY></HTML> > > which matches the file saved with wget. > > That has to be worth something.... > > Thanks! > > -C > > On Fri, Jun 14, 2019 at 4:07 PM tenspd137 . <dcday137@gmail.com> wrote: > > > > Just some more things I noticed from the wget log vs. the emacs buffers: > > > > emacs: Accept-encoding:gzip, wget:Accept-encoding: identity > > > > emacs: GET https://outlook.office365.com/EWS/Exchange.asmx HTTP/1.1, > > wget: GET /EWS/Exchange.asmx HTTP/1.1 > > > > Don't know if any of that is helpful, but there it is. > > > > Thanks! > > > > -C > > > > On Fri, Jun 14, 2019 at 3:48 PM tenspd137 . <dcday137@gmail.com> wrote: > > > > > > Sorry - I am not in control of my proxy. So, before I give up on this > > > for today, I was going over the wget logs from trying the same thing. > > > One thing I noticed is: > > > > > > Connection: Keep-Alive > > > Proxy-Connection: Keep-Alive > > > > > > I see stuff in the emacs buffers mentioning Connection, and it says > > > closed (unless marking the connection as busy is the same as keeping > > > it open), but I don't see anything mentioning the proxy > > > connection....? Maybe another place to look? > > > > > > Just some ideas... > > > > > > Thanks! > > > > > > -C > > > > > > On Fri, Jun 14, 2019 at 3:02 PM Thomas Fitzsimmons <fitzsim@fitzsim.org> wrote: > > > > > > > > Hi, > > > > > > > > Thanks for following up with these further test results. > > > > > > > > "tenspd137 ." <dcday137@gmail.com> writes: > > > > > > > > > I was able to try stepping through an Emacs/proxy/Exchange test in an > > > > > emacs -Q session. After setting the proxy and configuring the > > > > > debugger to step through url-http and url-http-async-sentilnel, the > > > > > only thing I noticed is that it appears url-http-async-sentinel is not > > > > > being called. > > > > > > > > OK, that's probably expected. I listed it for completeness (all the > > > > call sites of url-https-proxy-connect), but I guess it would only be > > > > called under url-retrieve, not under url-retrieve-synchronously. > > > > > > > > > I also put the url-https-proxy-connect override you gave me earlier > > > > > into a file, loaded it and set the debugger to run through that as > > > > > well as set up proxies. A broken down list of steps: > > > > > > > > > > 1. Load file containing proxy and altered url-https-proxyconnect, set > > > > > debugger to run through it when hit. > > > > > 2. set up url-http and url-http-async-sentinel to be picked up by debugger > > > > > 3. eval (url-retrieve-synchronously > > > > > "https://outlook.office365.com/EWS/Exchange.asmx"), step through > > > > > url-http > > > > > 4. Input username and password when asked > > > > > 5. Continue stepping until end > > > > > > > > > > url-http-async-sentinel is never called. " *http* ... -####" has text > > > > > indicating failure. url-https-proxy-connect is indeed called. > > > > > > > > OK, it's good to know that url-http ultimately calls > > > > url-https-proxy-connect. Unfortunately, despite this, proxying does not > > > > work. > > > > > > > > > I don't know how to look at the actual <process>, if I try to C-x C-e > > > > > or M-: "connection", it goes into the debugger. The url's, etc look > > > > > good as far as I can tell. Not sure what else I can do. If there are > > > > > certain pieces of url-http you want me to look at, just let me know, > > > > > but not really knowing what has to happen under the hood, I am not > > > > > going to be able to do much else. > > > > > > > > I think I'll probably have to set up my own test environment. Are you > > > > in control of the proxy and its configuration? If so, can you provide > > > > rough configuration instructions (proxy software, version, relevant > > > > config settings)? If not, that's OK, I can try setting up generic proxy > > > > software. > > > > > > > > Thanks, > > > > Thomas ^ permalink raw reply [flat|nested] 42+ messages in thread
* bug#35969: 26.2, Excorporate 2019-06-14 23:14 ` tenspd137 . @ 2019-06-15 0:14 ` Thomas Fitzsimmons 2019-06-15 0:47 ` tenspd137 . 2019-06-15 0:36 ` Thomas Fitzsimmons 1 sibling, 1 reply; 42+ messages in thread From: Thomas Fitzsimmons @ 2019-06-15 0:14 UTC (permalink / raw) To: tenspd137 .; +Cc: 35969 Hi, Yes, this seems really promising. If you just replace (thefile) with real-fname (and re-eval your url-http-create-request), does Excorporate then work through the proxy? Thomas "tenspd137 ." <dcday137@gmail.com> writes: > Sorry - I was working fast: > > *I created a function that changed the line > in url-http-create-request so that the request looks like:* > > In a file I did: > > (defun thefile() "/EWS/Exchange.asmx") > > (copied from url-http library) > (defun url-http-create-request (&optional ref-url) > ..... > ;; This was done with a call to `format'. Concatenating parts has > ;; the advantage of keeping the parts of each header together and > ;; allows us to elide null lines directly, at the cost of making > ;; the layout less clear. > (setq request > (concat > ;; The request > (or url-http-method "GET") " " > (url-http--encode-string > (if using-proxy (thefile) real-fname)) > <------------Changed this to "hardcode" the request as proof of > concept > " HTTP/" url-http-version "\r\n" > ;; Version of MIME we speak > "MIME-Version: 1.0\r\n" > ;; (maybe) Try to keep the connection open > "Connection: " (if (or using-proxy ..... > ....... > ....... > > Loaded the file and then ran (url-retriev-synchronously "https://......") > > Thanks! > > -C > > On Fri, Jun 14, 2019 at 5:07 PM tenspd137 . <dcday137@gmail.com> wrote: >> >> I think I figured it out. I created a function that changed the line >> in url-http-create-request so that the request looks like: >> GET /EWS/Exchange.asmx HTTP/1.1 <----- *no server / host* >> MIME-Version: 1.0 >> Connection: close >> Extension: Security/Digest Security/SSL >> Host: outlook.office365.com >> Accept-encoding: gzip >> Accept: */* >> User-Agent: URL/Emacs Emacs/26.2 (X11; x86_64-pc-linux-gnu) >> Cookie: OIDC=1; ClientId=8998C5691CD143E784857A0D01537963 >> Authorization: Basic ZGF2aWQuYy5kYXlAaHAuY29tOlMxa3kzbGk3bmUwNzMxJSU= >> >> instead of >> >> GET https://outlook.office365.com/EWS/Exchange.asmx HTTP/1.1 >> MIME-Version: 1.0 >> Connection: close >> Extension: Security/Digest Security/SSL >> Host: outlook.office365.com >> Accept-encoding: gzip >> Accept: */* >> User-Agent: URL/Emacs Emacs/26.2 (X11; x86_64-pc-linux-gnu) >> Cookie: OIDC=1; ClientId=8998C5691CD143E784857A0D01537963 >> Authorization: Basic ZGF2aWQuYy5kYXlAaHAuY29tOlMxa3kzbGk3bmUwNzMxJSU= >> >> and the results buffer gives: *http....*-##### >> HTTP/1.1 200 OK >> Cache-Control: private >> Content-Length: 1213 >> Content-Type: text/html; charset=UTF-8 >> Content-Encoding: gzip >> Vary: Accept-Encoding >> Server: Microsoft-IIS/10.0 >> request-id: bcd31568-29fd-44e4-935f-35b54d697f33 >> X-CalculatedFETarget: CY4PR18CU003.internal.outlook.com >> X-BackEndHttpStatus: 200 >> Set-Cookie: exchangecookie=6c731ced364846f2a979bfaa84496f1e; >> expires=Sun, 14-Jun-2020 23:02:57 GMT; path=/; secure; HttpOnly >> X-FEProxyInfo: CY4PR18CA0059.NAMPRD18.PROD.OUTLOOK.COM >> X-CalculatedBETarget: CS1PR8401MB1223.NAMPRD84.PROD.OUTLOOK.COM >> X-BackEndHttpStatus: 200 >> X-RUM-Validated: 1 >> X-AspNet-Version: 4.0.30319 >> X-BeSku: Gen9 >> X-DiagInfo: CS1PR8401MB1223 >> X-BEServer: CS1PR8401MB1223 >> X-FEServer: CY4PR18CA0059 >> X-Powered-By: ASP.NET >> X-FEServer: SN4PR0501CA0003 >> Date: Fri, 14 Jun 2019 23:02:56 GMT >> Connection: close >> >> <HTML><HEAD><link rel="alternate" type="text/xml" >> href="https://cs1pr8401mb1223.namprd84.prod.outlook.com:444/EWS/Exchange.asmx?disco"/><STYLE >> type="text/css">#content{ FONT-SIZE: 0.7em; PADDING-BOTTOM: 2em; >> MARGIN-LEFT: 30px}BODY{MARGIN-TOP: 0px; MARGIN-LEFT: 0px; COLOR: >> #000000; FONT-FAMILY: Verdana; BACKGROUND-COLOR: white}P{MARGIN-TOP: >> 0px; MARGIN-BOTTOM: 12px; COLOR: #000000; FONT-FAMILY: >> Verdana}PRE{BORDER-RIGHT: #f0f0e0 1px solid; PADDING-RIGHT: 5px; >> BORDER-TOP: #f0f0e0 1px solid; MARGIN-TOP: -5px; PADDING-LEFT: 5px; >> FONT-SIZE: 1.2em; PADDING-BOTTOM: 5px; BORDER-LEFT: #f0f0e0 1px solid; >> PADDING-TOP: 5px; BORDER-BOTTOM: #f0f0e0 1px solid; FONT-FAMILY: >> Courier New; BACKGROUND-COLOR: #e5e5cc}.heading1{MARGIN-TOP: 0px; >> PADDING-LEFT: 15px; FONT-WEIGHT: normal; FONT-SIZE: 26px; >> MARGIN-BOTTOM: 0px; PADDING-BOTTOM: 3px; MARGIN-LEFT: -30px; WIDTH: >> 100%; COLOR: #ffffff; PADDING-TOP: 10px; FONT-FAMILY: Tahoma; >> BACKGROUND-COLOR: #003366}.intro{MARGIN-LEFT: >> -15px}</STYLE><TITLE>Service</TITLE></HEAD><BODY><DIV id="content"><P >> class="heading1">Service</P><BR/><P class="intro">You have created a >> service.<P class='intro'>To test this service, you will need to create >> a client and use it to call the service. You can do this using the >> svcutil.exe tool from the command line with the following syntax:</P> >> <BR/><PRE>svcutil.exe <A >> HREF="https://cs1pr8401mb1223.namprd84.prod.outlook.com:444/EWS/Services.wsdl">https://cs1pr8401mb1223.namprd84.prod.outlook.com:444/EWS/Services.wsdl</A></PRE></P><P >> class="intro"/>This will generate a configuration file and a code file >> that contains the client class. Add the two files to your client >> application and use the generated client class to call the Service. >> For example:<BR/><P class='intro'><B>C#</B></P><PRE><font >> color="blue">class </font><font color="teal">Test >> </font>{ >> <font color="blue"> static void </font>Main() >> { >> <font color="teal">HelloClient</font> client = <font >> color="blue">new </font><font color="teal">HelloClient</font>(); >> >> <font color="green"> // Use the 'client' variable to call >> operations on the service. >> >> </font><font color="green"> // Always close the client. >> </font> client.Close(); >> } >> } >> </PRE><BR/><P class='intro'><B>Visual Basic</B></P><PRE><font >> color="blue">Class </font><font color="teal">Test >> </font><font color="blue"> Shared Sub </font>Main() >> <font color="blue"> Dim </font>client As <font >> color="teal">HelloClient</font> = <font color="blue">New </font><font >> color="teal">HelloClient</font>() >> <font color="green"> ' Use the 'client' variable to call >> operations on the service. >> >> </font><font color="green"> ' Always close the client. >> </font> client.Close() >> <font color="blue"> End Sub >> </font><font color="blue">End Class</font></PRE></DIV></BODY></HTML> >> >> which matches the file saved with wget. >> >> That has to be worth something.... >> >> Thanks! >> >> -C >> >> On Fri, Jun 14, 2019 at 4:07 PM tenspd137 . <dcday137@gmail.com> wrote: >> > >> > Just some more things I noticed from the wget log vs. the emacs buffers: >> > >> > emacs: Accept-encoding:gzip, wget:Accept-encoding: identity >> > >> > emacs: GET https://outlook.office365.com/EWS/Exchange.asmx HTTP/1.1, >> > wget: GET /EWS/Exchange.asmx HTTP/1.1 >> > >> > Don't know if any of that is helpful, but there it is. >> > >> > Thanks! >> > >> > -C >> > >> > On Fri, Jun 14, 2019 at 3:48 PM tenspd137 . <dcday137@gmail.com> wrote: >> > > >> > > Sorry - I am not in control of my proxy. So, before I give up on this >> > > for today, I was going over the wget logs from trying the same thing. >> > > One thing I noticed is: >> > > >> > > Connection: Keep-Alive >> > > Proxy-Connection: Keep-Alive >> > > >> > > I see stuff in the emacs buffers mentioning Connection, and it says >> > > closed (unless marking the connection as busy is the same as keeping >> > > it open), but I don't see anything mentioning the proxy >> > > connection....? Maybe another place to look? >> > > >> > > Just some ideas... >> > > >> > > Thanks! >> > > >> > > -C >> > > >> > > On Fri, Jun 14, 2019 at 3:02 PM Thomas Fitzsimmons <fitzsim@fitzsim.org> wrote: >> > > > >> > > > Hi, >> > > > >> > > > Thanks for following up with these further test results. >> > > > >> > > > "tenspd137 ." <dcday137@gmail.com> writes: >> > > > >> > > > > I was able to try stepping through an Emacs/proxy/Exchange test in an >> > > > > emacs -Q session. After setting the proxy and configuring the >> > > > > debugger to step through url-http and url-http-async-sentilnel, the >> > > > > only thing I noticed is that it appears url-http-async-sentinel is not >> > > > > being called. >> > > > >> > > > OK, that's probably expected. I listed it for completeness (all the >> > > > call sites of url-https-proxy-connect), but I guess it would only be >> > > > called under url-retrieve, not under url-retrieve-synchronously. >> > > > >> > > > > I also put the url-https-proxy-connect override you gave me earlier >> > > > > into a file, loaded it and set the debugger to run through that as >> > > > > well as set up proxies. A broken down list of steps: >> > > > > >> > > > > 1. Load file containing proxy and altered url-https-proxyconnect, set >> > > > > debugger to run through it when hit. >> > > > > 2. set up url-http and url-http-async-sentinel to be picked up by debugger >> > > > > 3. eval (url-retrieve-synchronously >> > > > > "https://outlook.office365.com/EWS/Exchange.asmx"), step through >> > > > > url-http >> > > > > 4. Input username and password when asked >> > > > > 5. Continue stepping until end >> > > > > >> > > > > url-http-async-sentinel is never called. " *http* ... -####" has text >> > > > > indicating failure. url-https-proxy-connect is indeed called. >> > > > >> > > > OK, it's good to know that url-http ultimately calls >> > > > url-https-proxy-connect. Unfortunately, despite this, proxying does not >> > > > work. >> > > > >> > > > > I don't know how to look at the actual <process>, if I try to C-x C-e >> > > > > or M-: "connection", it goes into the debugger. The url's, etc look >> > > > > good as far as I can tell. Not sure what else I can do. If there are >> > > > > certain pieces of url-http you want me to look at, just let me know, >> > > > > but not really knowing what has to happen under the hood, I am not >> > > > > going to be able to do much else. >> > > > >> > > > I think I'll probably have to set up my own test environment. Are you >> > > > in control of the proxy and its configuration? If so, can you provide >> > > > rough configuration instructions (proxy software, version, relevant >> > > > config settings)? If not, that's OK, I can try setting up generic proxy >> > > > software. >> > > > >> > > > Thanks, >> > > > Thomas ^ permalink raw reply [flat|nested] 42+ messages in thread
* bug#35969: 26.2, Excorporate 2019-06-15 0:14 ` Thomas Fitzsimmons @ 2019-06-15 0:47 ` tenspd137 . 0 siblings, 0 replies; 42+ messages in thread From: tenspd137 . @ 2019-06-15 0:47 UTC (permalink / raw) To: Thomas Fitzsimmons; +Cc: 35969 [-- Attachment #1: Type: text/plain, Size: 11080 bytes --] I'll be able to try on Monday.... Thanks! C On Fri, Jun 14, 2019, 6:14 PM Thomas Fitzsimmons <fitzsim@fitzsim.org> wrote: > Hi, > > Yes, this seems really promising. If you just replace (thefile) with > real-fname (and re-eval your url-http-create-request), does Excorporate > then work through the proxy? > > Thomas > > "tenspd137 ." <dcday137@gmail.com> writes: > > > Sorry - I was working fast: > > > > *I created a function that changed the line > > in url-http-create-request so that the request looks like:* > > > > In a file I did: > > > > (defun thefile() "/EWS/Exchange.asmx") > > > > (copied from url-http library) > > (defun url-http-create-request (&optional ref-url) > > ..... > > ;; This was done with a call to `format'. Concatenating parts has > > ;; the advantage of keeping the parts of each header together and > > ;; allows us to elide null lines directly, at the cost of making > > ;; the layout less clear. > > (setq request > > (concat > > ;; The request > > (or url-http-method "GET") " " > > (url-http--encode-string > > (if using-proxy (thefile) real-fname)) > > <------------Changed this to "hardcode" the request as proof of > > concept > > " HTTP/" url-http-version "\r\n" > > ;; Version of MIME we speak > > "MIME-Version: 1.0\r\n" > > ;; (maybe) Try to keep the connection open > > "Connection: " (if (or using-proxy ..... > > ....... > > ....... > > > > Loaded the file and then ran (url-retriev-synchronously "https:// > ......") > > > > Thanks! > > > > -C > > > > On Fri, Jun 14, 2019 at 5:07 PM tenspd137 . <dcday137@gmail.com> wrote: > >> > >> I think I figured it out. I created a function that changed the line > >> in url-http-create-request so that the request looks like: > >> GET /EWS/Exchange.asmx HTTP/1.1 <----- *no server / host* > >> MIME-Version: 1.0 > >> Connection: close > >> Extension: Security/Digest Security/SSL > >> Host: outlook.office365.com > >> Accept-encoding: gzip > >> Accept: */* > >> User-Agent: URL/Emacs Emacs/26.2 (X11; x86_64-pc-linux-gnu) > >> Cookie: OIDC=1; ClientId=8998C5691CD143E784857A0D01537963 > >> Authorization: Basic ZGF2aWQuYy5kYXlAaHAuY29tOlMxa3kzbGk3bmUwNzMxJSU= > >> > >> instead of > >> > >> GET https://outlook.office365.com/EWS/Exchange.asmx HTTP/1.1 > >> MIME-Version: 1.0 > >> Connection: close > >> Extension: Security/Digest Security/SSL > >> Host: outlook.office365.com > >> Accept-encoding: gzip > >> Accept: */* > >> User-Agent: URL/Emacs Emacs/26.2 (X11; x86_64-pc-linux-gnu) > >> Cookie: OIDC=1; ClientId=8998C5691CD143E784857A0D01537963 > >> Authorization: Basic ZGF2aWQuYy5kYXlAaHAuY29tOlMxa3kzbGk3bmUwNzMxJSU= > >> > >> and the results buffer gives: *http....*-##### > >> HTTP/1.1 200 OK > >> Cache-Control: private > >> Content-Length: 1213 > >> Content-Type: text/html; charset=UTF-8 > >> Content-Encoding: gzip > >> Vary: Accept-Encoding > >> Server: Microsoft-IIS/10.0 > >> request-id: bcd31568-29fd-44e4-935f-35b54d697f33 > >> X-CalculatedFETarget: CY4PR18CU003.internal.outlook.com > >> X-BackEndHttpStatus: 200 > >> Set-Cookie: exchangecookie=6c731ced364846f2a979bfaa84496f1e; > >> expires=Sun, 14-Jun-2020 23:02:57 GMT; path=/; secure; HttpOnly > >> X-FEProxyInfo: CY4PR18CA0059.NAMPRD18.PROD.OUTLOOK.COM > >> X-CalculatedBETarget: CS1PR8401MB1223.NAMPRD84.PROD.OUTLOOK.COM > >> X-BackEndHttpStatus: 200 > >> X-RUM-Validated: 1 > >> X-AspNet-Version: 4.0.30319 > >> X-BeSku: Gen9 > >> X-DiagInfo: CS1PR8401MB1223 > >> X-BEServer: CS1PR8401MB1223 > >> X-FEServer: CY4PR18CA0059 > >> X-Powered-By: ASP.NET > >> X-FEServer: SN4PR0501CA0003 > >> Date: Fri, 14 Jun 2019 23:02:56 GMT > >> Connection: close > >> > >> <HTML><HEAD><link rel="alternate" type="text/xml" > >> href=" > https://cs1pr8401mb1223.namprd84.prod.outlook.com:444/EWS/Exchange.asmx?disco > "/><STYLE > >> type="text/css">#content{ FONT-SIZE: 0.7em; PADDING-BOTTOM: 2em; > >> MARGIN-LEFT: 30px}BODY{MARGIN-TOP: 0px; MARGIN-LEFT: 0px; COLOR: > >> #000000; FONT-FAMILY: Verdana; BACKGROUND-COLOR: white}P{MARGIN-TOP: > >> 0px; MARGIN-BOTTOM: 12px; COLOR: #000000; FONT-FAMILY: > >> Verdana}PRE{BORDER-RIGHT: #f0f0e0 1px solid; PADDING-RIGHT: 5px; > >> BORDER-TOP: #f0f0e0 1px solid; MARGIN-TOP: -5px; PADDING-LEFT: 5px; > >> FONT-SIZE: 1.2em; PADDING-BOTTOM: 5px; BORDER-LEFT: #f0f0e0 1px solid; > >> PADDING-TOP: 5px; BORDER-BOTTOM: #f0f0e0 1px solid; FONT-FAMILY: > >> Courier New; BACKGROUND-COLOR: #e5e5cc}.heading1{MARGIN-TOP: 0px; > >> PADDING-LEFT: 15px; FONT-WEIGHT: normal; FONT-SIZE: 26px; > >> MARGIN-BOTTOM: 0px; PADDING-BOTTOM: 3px; MARGIN-LEFT: -30px; WIDTH: > >> 100%; COLOR: #ffffff; PADDING-TOP: 10px; FONT-FAMILY: Tahoma; > >> BACKGROUND-COLOR: #003366}.intro{MARGIN-LEFT: > >> -15px}</STYLE><TITLE>Service</TITLE></HEAD><BODY><DIV id="content"><P > >> class="heading1">Service</P><BR/><P class="intro">You have created a > >> service.<P class='intro'>To test this service, you will need to create > >> a client and use it to call the service. You can do this using the > >> svcutil.exe tool from the command line with the following syntax:</P> > >> <BR/><PRE>svcutil.exe <A > >> HREF=" > https://cs1pr8401mb1223.namprd84.prod.outlook.com:444/EWS/Services.wsdl"> > https://cs1pr8401mb1223.namprd84.prod.outlook.com:444/EWS/Services.wsdl > </A></PRE></P><P > >> class="intro"/>This will generate a configuration file and a code file > >> that contains the client class. Add the two files to your client > >> application and use the generated client class to call the Service. > >> For example:<BR/><P class='intro'><B>C#</B></P><PRE><font > >> color="blue">class </font><font color="teal">Test > >> </font>{ > >> <font color="blue"> static void </font>Main() > >> { > >> <font color="teal">HelloClient</font> client = <font > >> color="blue">new </font><font color="teal">HelloClient</font>(); > >> > >> <font color="green"> // Use the 'client' variable to call > >> operations on the service. > >> > >> </font><font color="green"> // Always close the client. > >> </font> client.Close(); > >> } > >> } > >> </PRE><BR/><P class='intro'><B>Visual Basic</B></P><PRE><font > >> color="blue">Class </font><font color="teal">Test > >> </font><font color="blue"> Shared Sub </font>Main() > >> <font color="blue"> Dim </font>client As <font > >> color="teal">HelloClient</font> = <font color="blue">New </font><font > >> color="teal">HelloClient</font>() > >> <font color="green"> ' Use the 'client' variable to call > >> operations on the service. > >> > >> </font><font color="green"> ' Always close the client. > >> </font> client.Close() > >> <font color="blue"> End Sub > >> </font><font color="blue">End Class</font></PRE></DIV></BODY></HTML> > >> > >> which matches the file saved with wget. > >> > >> That has to be worth something.... > >> > >> Thanks! > >> > >> -C > >> > >> On Fri, Jun 14, 2019 at 4:07 PM tenspd137 . <dcday137@gmail.com> wrote: > >> > > >> > Just some more things I noticed from the wget log vs. the emacs > buffers: > >> > > >> > emacs: Accept-encoding:gzip, wget:Accept-encoding: identity > >> > > >> > emacs: GET https://outlook.office365.com/EWS/Exchange.asmx HTTP/1.1, > >> > wget: GET /EWS/Exchange.asmx HTTP/1.1 > >> > > >> > Don't know if any of that is helpful, but there it is. > >> > > >> > Thanks! > >> > > >> > -C > >> > > >> > On Fri, Jun 14, 2019 at 3:48 PM tenspd137 . <dcday137@gmail.com> > wrote: > >> > > > >> > > Sorry - I am not in control of my proxy. So, before I give up on > this > >> > > for today, I was going over the wget logs from trying the same > thing. > >> > > One thing I noticed is: > >> > > > >> > > Connection: Keep-Alive > >> > > Proxy-Connection: Keep-Alive > >> > > > >> > > I see stuff in the emacs buffers mentioning Connection, and it says > >> > > closed (unless marking the connection as busy is the same as keeping > >> > > it open), but I don't see anything mentioning the proxy > >> > > connection....? Maybe another place to look? > >> > > > >> > > Just some ideas... > >> > > > >> > > Thanks! > >> > > > >> > > -C > >> > > > >> > > On Fri, Jun 14, 2019 at 3:02 PM Thomas Fitzsimmons < > fitzsim@fitzsim.org> wrote: > >> > > > > >> > > > Hi, > >> > > > > >> > > > Thanks for following up with these further test results. > >> > > > > >> > > > "tenspd137 ." <dcday137@gmail.com> writes: > >> > > > > >> > > > > I was able to try stepping through an Emacs/proxy/Exchange test > in an > >> > > > > emacs -Q session. After setting the proxy and configuring the > >> > > > > debugger to step through url-http and url-http-async-sentilnel, > the > >> > > > > only thing I noticed is that it appears url-http-async-sentinel > is not > >> > > > > being called. > >> > > > > >> > > > OK, that's probably expected. I listed it for completeness (all > the > >> > > > call sites of url-https-proxy-connect), but I guess it would only > be > >> > > > called under url-retrieve, not under url-retrieve-synchronously. > >> > > > > >> > > > > I also put the url-https-proxy-connect override you gave me > earlier > >> > > > > into a file, loaded it and set the debugger to run through that > as > >> > > > > well as set up proxies. A broken down list of steps: > >> > > > > > >> > > > > 1. Load file containing proxy and altered > url-https-proxyconnect, set > >> > > > > debugger to run through it when hit. > >> > > > > 2. set up url-http and url-http-async-sentinel to be picked up > by debugger > >> > > > > 3. eval (url-retrieve-synchronously > >> > > > > "https://outlook.office365.com/EWS/Exchange.asmx"), step > through > >> > > > > url-http > >> > > > > 4. Input username and password when asked > >> > > > > 5. Continue stepping until end > >> > > > > > >> > > > > url-http-async-sentinel is never called. " *http* ... -####" > has text > >> > > > > indicating failure. url-https-proxy-connect is indeed called. > >> > > > > >> > > > OK, it's good to know that url-http ultimately calls > >> > > > url-https-proxy-connect. Unfortunately, despite this, proxying > does not > >> > > > work. > >> > > > > >> > > > > I don't know how to look at the actual <process>, if I try to > C-x C-e > >> > > > > or M-: "connection", it goes into the debugger. The url's, etc > look > >> > > > > good as far as I can tell. Not sure what else I can do. If > there are > >> > > > > certain pieces of url-http you want me to look at, just let me > know, > >> > > > > but not really knowing what has to happen under the hood, I am > not > >> > > > > going to be able to do much else. > >> > > > > >> > > > I think I'll probably have to set up my own test environment. > Are you > >> > > > in control of the proxy and its configuration? If so, can you > provide > >> > > > rough configuration instructions (proxy software, version, > relevant > >> > > > config settings)? If not, that's OK, I can try setting up > generic proxy > >> > > > software. > >> > > > > >> > > > Thanks, > >> > > > Thomas > [-- Attachment #2: Type: text/html, Size: 17068 bytes --] ^ permalink raw reply [flat|nested] 42+ messages in thread
* bug#35969: 26.2, Excorporate 2019-06-14 23:14 ` tenspd137 . 2019-06-15 0:14 ` Thomas Fitzsimmons @ 2019-06-15 0:36 ` Thomas Fitzsimmons 2019-06-15 7:41 ` Andreas Schwab 1 sibling, 1 reply; 42+ messages in thread From: Thomas Fitzsimmons @ 2019-06-15 0:36 UTC (permalink / raw) To: tenspd137 .; +Cc: 35969 Hi, "tenspd137 ." <dcday137@gmail.com> writes: > (defun thefile() "/EWS/Exchange.asmx") > > (copied from url-http library) > (defun url-http-create-request (&optional ref-url) > ..... > ;; This was done with a call to `format'. Concatenating parts has > ;; the advantage of keeping the parts of each header together and > ;; allows us to elide null lines directly, at the cost of making > ;; the layout less clear. > (setq request > (concat > ;; The request > (or url-http-method "GET") " " > (url-http--encode-string > (if using-proxy (thefile) real-fname)) <------------Changed this to "hardcode" the request as proof of concept Starting from your results I searched the web and turned up a Stack Overflow post [1] which pointed me to the wget source code: if (proxy #ifdef HAVE_SSL /* When using SSL over proxy, CONNECT establishes a direct connection to the HTTPS server. Therefore use the same argument as when talking to the server directly. */ && u->scheme != SCHEME_HTTPS #endif ) meth_arg = xstrdup (u->url); else meth_arg = url_full_path (u); I think that comment explains why the url connection to the HTTPS Exchange server is failing in your case. Applying the same logic in the location you found would give: diff --git a/lisp/url/url-http.el b/lisp/url/url-http.el index 00803a103a..723d111d58 100644 --- a/lisp/url/url-http.el +++ b/lisp/url/url-http.el @@ -329,7 +329,10 @@ url-http-create-request ;; The request (or url-http-method "GET") " " (url-http--encode-string - (if using-proxy (url-recreate-url url-http-target-url) real-fname)) + (if (and using-proxy + (not (equal "https" (url-type url-http-target-url)))) + (url-recreate-url url-http-target-url) + real-fname)) " HTTP/" url-http-version "\r\n" ;; Version of MIME we speak "MIME-Version: 1.0\r\n" Can you try out that patch and a) see if url-retrieve-synchronously still works, and if so, b) see if Excorporate works through the proxy with it? Thanks, Thomas 1. https://stackoverflow.com/questions/20050052/what-is-the-correct-url-format-to-send-to-an-http-proxy-server ^ permalink raw reply related [flat|nested] 42+ messages in thread
* bug#35969: 26.2, Excorporate 2019-06-15 0:36 ` Thomas Fitzsimmons @ 2019-06-15 7:41 ` Andreas Schwab 2019-06-17 16:31 ` tenspd137 . 2019-07-31 21:07 ` Thomas Fitzsimmons 0 siblings, 2 replies; 42+ messages in thread From: Andreas Schwab @ 2019-06-15 7:41 UTC (permalink / raw) To: Thomas Fitzsimmons; +Cc: tenspd137 ., 35969 On Jun 14 2019, Thomas Fitzsimmons <fitzsim@fitzsim.org> wrote: > diff --git a/lisp/url/url-http.el b/lisp/url/url-http.el > index 00803a103a..723d111d58 100644 > --- a/lisp/url/url-http.el > +++ b/lisp/url/url-http.el > @@ -329,7 +329,10 @@ url-http-create-request > ;; The request > (or url-http-method "GET") " " > (url-http--encode-string > - (if using-proxy (url-recreate-url url-http-target-url) real-fname)) > + (if (and using-proxy > + (not (equal "https" (url-type url-http-target-url)))) > + (url-recreate-url url-http-target-url) > + real-fname)) That should already be handled by commit 84613dae5c. Andreas. -- Andreas Schwab, schwab@linux-m68k.org GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510 2552 DF73 E780 A9DA AEC1 "And now for something completely different." ^ permalink raw reply [flat|nested] 42+ messages in thread
* bug#35969: 26.2, Excorporate 2019-06-15 7:41 ` Andreas Schwab @ 2019-06-17 16:31 ` tenspd137 . 2019-06-17 22:08 ` Thomas Fitzsimmons 2019-07-31 21:07 ` Thomas Fitzsimmons 1 sibling, 1 reply; 42+ messages in thread From: tenspd137 . @ 2019-06-17 16:31 UTC (permalink / raw) To: Andreas Schwab; +Cc: Thomas Fitzsimmons, 35969 The patch Thomas seems to work from behind the proxy. My current emacs version is 26.2, so I would think it would include the commit Andreas is talking about.... I went and looked it up - is this the correct commit? diff --git a/lisp/url/url-http.el b/lisp/url/url-http.el index 53798f7..817c5ce 100644 --- a/lisp/url/url-http.el +++ b/lisp/url/url-http.el @@ -1412,7 +1412,9 @@ The return value of this function is the retrieval buffer." 'url-http-wait-for-headers-change-function) (set-process-filter tls-connection 'url-http-generic-filter) (process-send-string tls-connection - (url-http-create-request))) + ;; Use the non-proxy form of the request + (let (url-http-proxy) + (url-http-create-request)))) (gnutls-error (url-http-activate-callback) (error "gnutls-error: %s" e)) Thanks! -C On Sat, Jun 15, 2019 at 1:41 AM Andreas Schwab <schwab@linux-m68k.org> wrote: > > On Jun 14 2019, Thomas Fitzsimmons <fitzsim@fitzsim.org> wrote: > > > diff --git a/lisp/url/url-http.el b/lisp/url/url-http.el > > index 00803a103a..723d111d58 100644 > > --- a/lisp/url/url-http.el > > +++ b/lisp/url/url-http.el > > @@ -329,7 +329,10 @@ url-http-create-request > > ;; The request > > (or url-http-method "GET") " " > > (url-http--encode-string > > - (if using-proxy (url-recreate-url url-http-target-url) real-fname)) > > + (if (and using-proxy > > + (not (equal "https" (url-type url-http-target-url)))) > > + (url-recreate-url url-http-target-url) > > + real-fname)) > > That should already be handled by commit 84613dae5c. > > Andreas. > > -- > Andreas Schwab, schwab@linux-m68k.org > GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510 2552 DF73 E780 A9DA AEC1 > "And now for something completely different." ^ permalink raw reply related [flat|nested] 42+ messages in thread
* bug#35969: 26.2, Excorporate 2019-06-17 16:31 ` tenspd137 . @ 2019-06-17 22:08 ` Thomas Fitzsimmons 2019-06-18 16:34 ` tenspd137 . 0 siblings, 1 reply; 42+ messages in thread From: Thomas Fitzsimmons @ 2019-06-17 22:08 UTC (permalink / raw) To: tenspd137 .; +Cc: Andreas Schwab, 35969 Hi, Good to hear that the patch I posted worked! Yes, that's the patch that Andreas's commit 84613dae5c34ea742dd9a3e56f5acb55f604b483 applied. From what I can tell, you will not have that in Emacs 26.2. Can you try reverting my patch and applying Andreas's patch, and see if Excorporate still works through the proxy? Thanks, Thomas "tenspd137 ." <dcday137@gmail.com> writes: > The patch Thomas seems to work from behind the proxy. My current > emacs version is 26.2, so I would think it would include the commit > Andreas is talking about.... I went and looked it up - is this the > correct commit? > > diff --git a/lisp/url/url-http.el b/lisp/url/url-http.el > index 53798f7..817c5ce 100644 > --- a/lisp/url/url-http.el > +++ b/lisp/url/url-http.el > @@ -1412,7 +1412,9 @@ The return value of this function is the > retrieval buffer." > 'url-http-wait-for-headers-change-function) > (set-process-filter tls-connection 'url-http-generic-filter) > (process-send-string tls-connection > - (url-http-create-request))) > + ;; Use the non-proxy form of the request > + (let (url-http-proxy) > + (url-http-create-request)))) > (gnutls-error > (url-http-activate-callback) > (error "gnutls-error: %s" e)) > > Thanks! > > -C > > > On Sat, Jun 15, 2019 at 1:41 AM Andreas Schwab <schwab@linux-m68k.org> wrote: >> >> On Jun 14 2019, Thomas Fitzsimmons <fitzsim@fitzsim.org> wrote: >> >> > diff --git a/lisp/url/url-http.el b/lisp/url/url-http.el >> > index 00803a103a..723d111d58 100644 >> > --- a/lisp/url/url-http.el >> > +++ b/lisp/url/url-http.el >> > @@ -329,7 +329,10 @@ url-http-create-request >> > ;; The request >> > (or url-http-method "GET") " " >> > (url-http--encode-string >> > - (if using-proxy (url-recreate-url url-http-target-url) real-fname)) >> > + (if (and using-proxy >> > + (not (equal "https" (url-type url-http-target-url)))) >> > + (url-recreate-url url-http-target-url) >> > + real-fname)) >> >> That should already be handled by commit 84613dae5c. >> >> Andreas. >> >> -- >> Andreas Schwab, schwab@linux-m68k.org >> GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510 2552 DF73 E780 A9DA AEC1 >> "And now for something completely different." ^ permalink raw reply [flat|nested] 42+ messages in thread
* bug#35969: 26.2, Excorporate 2019-06-17 22:08 ` Thomas Fitzsimmons @ 2019-06-18 16:34 ` tenspd137 . 2019-06-19 4:26 ` Thomas Fitzsimmons 0 siblings, 1 reply; 42+ messages in thread From: tenspd137 . @ 2019-06-18 16:34 UTC (permalink / raw) To: Thomas Fitzsimmons; +Cc: Andreas Schwab, 35969 So - I am not sure if I did it correctly, but I copied this function with Andreas' changes into a file: (defun url-https-proxy-after-change-function (_st _nd _length) (let* ((process-buffer (current-buffer)) (proc (get-buffer-process process-buffer))) (goto-char (point-min)) (when (re-search-forward "^\r?\n" nil t) (backward-char 1) ;; Saw the end of the headers (setq url-http-end-of-headers (set-marker (make-marker) (point))) (url-http-parse-response) (cond ((null url-http-response-status) ;; We got back a headerless malformed response from the ;; server. (url-http-activate-callback) (error "Malformed response from proxy, fail!")) ((= url-http-response-status 200) (if (gnutls-available-p) (condition-case e (let ((tls-connection (gnutls-negotiate :process proc :hostname (url-host url-current-object) :verify-error nil))) ;; check certificate validity (setq tls-connection (nsm-verify-connection tls-connection (url-host url-current-object) (url-port url-current-object))) (with-current-buffer process-buffer (erase-buffer)) (set-process-buffer tls-connection process-buffer) (setq url-http-after-change-function 'url-http-wait-for-headers-change-function) (set-process-filter tls-connection 'url-http-generic-filter) (process-send-string tls-connection ;; Use the non-proxy form of the request (let (url-http-proxy) (url-http-create-request)))) (gnutls-error (url-http-activate-callback) (error "gnutls-error: %s" e)) (error (url-http-activate-callback) (error "error: %s" e))) (error "error: gnutls support needed!"))) (t (url-http-debug "error response: %d" url-http-response-status) (url-http-activate-callback)))))) and then loaded it before running excorporate. After that, I did M-x excorporate, and the minibuffer returns: error in process filter: Server response is not an XML document When I do the similar test by loading the url-http-create-request with Thiomase's changes, I can get a connection and grab my schedule through the proxy. Let me know if I need to try something different. Thanks! -C On Mon, Jun 17, 2019 at 4:08 PM Thomas Fitzsimmons <fitzsim@fitzsim.org> wrote: > > Hi, > > Good to hear that the patch I posted worked! > > Yes, that's the patch that Andreas's commit > 84613dae5c34ea742dd9a3e56f5acb55f604b483 applied. From what I can tell, > you will not have that in Emacs 26.2. > > Can you try reverting my patch and applying Andreas's patch, and see if > Excorporate still works through the proxy? > > Thanks, > Thomas > > "tenspd137 ." <dcday137@gmail.com> writes: > > > The patch Thomas seems to work from behind the proxy. My current > > emacs version is 26.2, so I would think it would include the commit > > Andreas is talking about.... I went and looked it up - is this the > > correct commit? > > > > diff --git a/lisp/url/url-http.el b/lisp/url/url-http.el > > index 53798f7..817c5ce 100644 > > --- a/lisp/url/url-http.el > > +++ b/lisp/url/url-http.el > > @@ -1412,7 +1412,9 @@ The return value of this function is the > > retrieval buffer." > > 'url-http-wait-for-headers-change-function) > > (set-process-filter tls-connection 'url-http-generic-filter) > > (process-send-string tls-connection > > - (url-http-create-request))) > > + ;; Use the non-proxy form of the request > > + (let (url-http-proxy) > > + (url-http-create-request)))) > > (gnutls-error > > (url-http-activate-callback) > > (error "gnutls-error: %s" e)) > > > > Thanks! > > > > -C > > > > > > On Sat, Jun 15, 2019 at 1:41 AM Andreas Schwab <schwab@linux-m68k.org> wrote: > >> > >> On Jun 14 2019, Thomas Fitzsimmons <fitzsim@fitzsim.org> wrote: > >> > >> > diff --git a/lisp/url/url-http.el b/lisp/url/url-http.el > >> > index 00803a103a..723d111d58 100644 > >> > --- a/lisp/url/url-http.el > >> > +++ b/lisp/url/url-http.el > >> > @@ -329,7 +329,10 @@ url-http-create-request > >> > ;; The request > >> > (or url-http-method "GET") " " > >> > (url-http--encode-string > >> > - (if using-proxy (url-recreate-url url-http-target-url) real-fname)) > >> > + (if (and using-proxy > >> > + (not (equal "https" (url-type url-http-target-url)))) > >> > + (url-recreate-url url-http-target-url) > >> > + real-fname)) > >> > >> That should already be handled by commit 84613dae5c. > >> > >> Andreas. > >> > >> -- > >> Andreas Schwab, schwab@linux-m68k.org > >> GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510 2552 DF73 E780 A9DA AEC1 > >> "And now for something completely different." ^ permalink raw reply [flat|nested] 42+ messages in thread
* bug#35969: 26.2, Excorporate 2019-06-18 16:34 ` tenspd137 . @ 2019-06-19 4:26 ` Thomas Fitzsimmons 2019-06-20 20:34 ` tenspd137 . 0 siblings, 1 reply; 42+ messages in thread From: Thomas Fitzsimmons @ 2019-06-19 4:26 UTC (permalink / raw) To: tenspd137 .; +Cc: Andreas Schwab, 35969 Hi, "tenspd137 ." <dcday137@gmail.com> writes: > So - I am not sure if I did it correctly, but I copied this function > with Andreas' changes into a file: > > (defun url-https-proxy-after-change-function (_st _nd _length) > (let* ((process-buffer (current-buffer)) > (proc (get-buffer-process process-buffer))) > (goto-char (point-min)) > (when (re-search-forward "^\r?\n" nil t) > (backward-char 1) > ;; Saw the end of the headers > (setq url-http-end-of-headers (set-marker (make-marker) (point))) > (url-http-parse-response) > (cond > ((null url-http-response-status) > ;; We got back a headerless malformed response from the > ;; server. > (url-http-activate-callback) > (error "Malformed response from proxy, fail!")) > ((= url-http-response-status 200) > (if (gnutls-available-p) > (condition-case e > (let ((tls-connection (gnutls-negotiate > :process proc > :hostname (url-host url-current-object) > :verify-error nil))) > ;; check certificate validity > (setq tls-connection > (nsm-verify-connection tls-connection > (url-host url-current-object) > (url-port url-current-object))) > (with-current-buffer process-buffer (erase-buffer)) > (set-process-buffer tls-connection process-buffer) > (setq url-http-after-change-function > 'url-http-wait-for-headers-change-function) > (set-process-filter tls-connection 'url-http-generic-filter) > (process-send-string tls-connection > ;; Use the non-proxy form of the request > (let (url-http-proxy) > (url-http-create-request)))) > (gnutls-error > (url-http-activate-callback) > (error "gnutls-error: %s" e)) > (error > (url-http-activate-callback) > (error "error: %s" e))) > (error "error: gnutls support needed!"))) > (t > (url-http-debug "error response: %d" url-http-response-status) > (url-http-activate-callback)))))) > > and then loaded it before running excorporate. After that, I did M-x > excorporate, and the minibuffer returns: error in process filter: > Server response is not an XML document In this scenario, if you immediately (without restarting Emacs/reloading anything) re-run M-x excorporate does it still fail? I just want to make sure that's not a transient failure. If it does fail the second time, can you post the HTTP response from the server? > When I do the similar test by loading the url-http-create-request with > Thomas's changes, I can get a connection and grab my schedule > through the proxy. OK. > Let me know if I need to try something different. Are you in a position to build Emacs master tip and retry the experiment without patching anything? Thanks, Thomas > On Mon, Jun 17, 2019 at 4:08 PM Thomas Fitzsimmons <fitzsim@fitzsim.org> wrote: >> >> Hi, >> >> Good to hear that the patch I posted worked! >> >> Yes, that's the patch that Andreas's commit >> 84613dae5c34ea742dd9a3e56f5acb55f604b483 applied. From what I can tell, >> you will not have that in Emacs 26.2. >> >> Can you try reverting my patch and applying Andreas's patch, and see if >> Excorporate still works through the proxy? >> >> Thanks, >> Thomas >> >> "tenspd137 ." <dcday137@gmail.com> writes: >> >> > The patch Thomas seems to work from behind the proxy. My current >> > emacs version is 26.2, so I would think it would include the commit >> > Andreas is talking about.... I went and looked it up - is this the >> > correct commit? >> > >> > diff --git a/lisp/url/url-http.el b/lisp/url/url-http.el >> > index 53798f7..817c5ce 100644 >> > --- a/lisp/url/url-http.el >> > +++ b/lisp/url/url-http.el >> > @@ -1412,7 +1412,9 @@ The return value of this function is the >> > retrieval buffer." >> > 'url-http-wait-for-headers-change-function) >> > (set-process-filter tls-connection 'url-http-generic-filter) >> > (process-send-string tls-connection >> > - (url-http-create-request))) >> > + ;; Use the non-proxy form of the request >> > + (let (url-http-proxy) >> > + (url-http-create-request)))) >> > (gnutls-error >> > (url-http-activate-callback) >> > (error "gnutls-error: %s" e)) >> > >> > Thanks! >> > >> > -C >> > >> > >> > On Sat, Jun 15, 2019 at 1:41 AM Andreas Schwab <schwab@linux-m68k.org> wrote: >> >> >> >> On Jun 14 2019, Thomas Fitzsimmons <fitzsim@fitzsim.org> wrote: >> >> >> >> > diff --git a/lisp/url/url-http.el b/lisp/url/url-http.el >> >> > index 00803a103a..723d111d58 100644 >> >> > --- a/lisp/url/url-http.el >> >> > +++ b/lisp/url/url-http.el >> >> > @@ -329,7 +329,10 @@ url-http-create-request >> >> > ;; The request >> >> > (or url-http-method "GET") " " >> >> > (url-http--encode-string >> >> > - (if using-proxy (url-recreate-url url-http-target-url) real-fname)) >> >> > + (if (and using-proxy >> >> > + (not (equal "https" (url-type url-http-target-url)))) >> >> > + (url-recreate-url url-http-target-url) >> >> > + real-fname)) >> >> >> >> That should already be handled by commit 84613dae5c. >> >> >> >> Andreas. >> >> >> >> -- >> >> Andreas Schwab, schwab@linux-m68k.org >> >> GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510 2552 DF73 E780 A9DA AEC1 >> >> "And now for something completely different." ^ permalink raw reply [flat|nested] 42+ messages in thread
* bug#35969: 26.2, Excorporate 2019-06-19 4:26 ` Thomas Fitzsimmons @ 2019-06-20 20:34 ` tenspd137 . 2019-07-09 21:52 ` Thomas Fitzsimmons 0 siblings, 1 reply; 42+ messages in thread From: tenspd137 . @ 2019-06-20 20:34 UTC (permalink / raw) To: Thomas Fitzsimmons; +Cc: Andreas Schwab, 35969 I haven't tried the first item on the list yet - ie reloading Andreas' function. i have tried to build emacs from https://github.com/emacs-mirror/emacs.git because I can't reach the official repo from my firewall for whatever reason. I am sure it is something here. Anyway, I was having problems because my built version (27.0.x) was getting seemingly mixed up with my current installation. I can try again later, but right now things are pretty busy at work. Thanks. On Tue, Jun 18, 2019 at 10:26 PM Thomas Fitzsimmons <fitzsim@fitzsim.org> wrote: > > Hi, > > "tenspd137 ." <dcday137@gmail.com> writes: > > > So - I am not sure if I did it correctly, but I copied this function > > with Andreas' changes into a file: > > > > (defun url-https-proxy-after-change-function (_st _nd _length) > > (let* ((process-buffer (current-buffer)) > > (proc (get-buffer-process process-buffer))) > > (goto-char (point-min)) > > (when (re-search-forward "^\r?\n" nil t) > > (backward-char 1) > > ;; Saw the end of the headers > > (setq url-http-end-of-headers (set-marker (make-marker) (point))) > > (url-http-parse-response) > > (cond > > ((null url-http-response-status) > > ;; We got back a headerless malformed response from the > > ;; server. > > (url-http-activate-callback) > > (error "Malformed response from proxy, fail!")) > > ((= url-http-response-status 200) > > (if (gnutls-available-p) > > (condition-case e > > (let ((tls-connection (gnutls-negotiate > > :process proc > > :hostname (url-host url-current-object) > > :verify-error nil))) > > ;; check certificate validity > > (setq tls-connection > > (nsm-verify-connection tls-connection > > (url-host url-current-object) > > (url-port url-current-object))) > > (with-current-buffer process-buffer (erase-buffer)) > > (set-process-buffer tls-connection process-buffer) > > (setq url-http-after-change-function > > 'url-http-wait-for-headers-change-function) > > (set-process-filter tls-connection 'url-http-generic-filter) > > (process-send-string tls-connection > > ;; Use the non-proxy form of the request > > (let (url-http-proxy) > > (url-http-create-request)))) > > (gnutls-error > > (url-http-activate-callback) > > (error "gnutls-error: %s" e)) > > (error > > (url-http-activate-callback) > > (error "error: %s" e))) > > (error "error: gnutls support needed!"))) > > (t > > (url-http-debug "error response: %d" url-http-response-status) > > (url-http-activate-callback)))))) > > > > and then loaded it before running excorporate. After that, I did M-x > > excorporate, and the minibuffer returns: error in process filter: > > Server response is not an XML document > > In this scenario, if you immediately (without restarting Emacs/reloading > anything) re-run M-x excorporate does it still fail? I just want to > make sure that's not a transient failure. If it does fail the second > time, can you post the HTTP response from the server? > > > When I do the similar test by loading the url-http-create-request with > > Thomas's changes, I can get a connection and grab my schedule > > through the proxy. > > OK. > > > Let me know if I need to try something different. > > Are you in a position to build Emacs master tip and retry the experiment > without patching anything? > > Thanks, > Thomas > > > On Mon, Jun 17, 2019 at 4:08 PM Thomas Fitzsimmons <fitzsim@fitzsim.org> wrote: > >> > >> Hi, > >> > >> Good to hear that the patch I posted worked! > >> > >> Yes, that's the patch that Andreas's commit > >> 84613dae5c34ea742dd9a3e56f5acb55f604b483 applied. From what I can tell, > >> you will not have that in Emacs 26.2. > >> > >> Can you try reverting my patch and applying Andreas's patch, and see if > >> Excorporate still works through the proxy? > >> > >> Thanks, > >> Thomas > >> > >> "tenspd137 ." <dcday137@gmail.com> writes: > >> > >> > The patch Thomas seems to work from behind the proxy. My current > >> > emacs version is 26.2, so I would think it would include the commit > >> > Andreas is talking about.... I went and looked it up - is this the > >> > correct commit? > >> > > >> > diff --git a/lisp/url/url-http.el b/lisp/url/url-http.el > >> > index 53798f7..817c5ce 100644 > >> > --- a/lisp/url/url-http.el > >> > +++ b/lisp/url/url-http.el > >> > @@ -1412,7 +1412,9 @@ The return value of this function is the > >> > retrieval buffer." > >> > 'url-http-wait-for-headers-change-function) > >> > (set-process-filter tls-connection 'url-http-generic-filter) > >> > (process-send-string tls-connection > >> > - (url-http-create-request))) > >> > + ;; Use the non-proxy form of the request > >> > + (let (url-http-proxy) > >> > + (url-http-create-request)))) > >> > (gnutls-error > >> > (url-http-activate-callback) > >> > (error "gnutls-error: %s" e)) > >> > > >> > Thanks! > >> > > >> > -C > >> > > >> > > >> > On Sat, Jun 15, 2019 at 1:41 AM Andreas Schwab <schwab@linux-m68k.org> wrote: > >> >> > >> >> On Jun 14 2019, Thomas Fitzsimmons <fitzsim@fitzsim.org> wrote: > >> >> > >> >> > diff --git a/lisp/url/url-http.el b/lisp/url/url-http.el > >> >> > index 00803a103a..723d111d58 100644 > >> >> > --- a/lisp/url/url-http.el > >> >> > +++ b/lisp/url/url-http.el > >> >> > @@ -329,7 +329,10 @@ url-http-create-request > >> >> > ;; The request > >> >> > (or url-http-method "GET") " " > >> >> > (url-http--encode-string > >> >> > - (if using-proxy (url-recreate-url url-http-target-url) real-fname)) > >> >> > + (if (and using-proxy > >> >> > + (not (equal "https" (url-type url-http-target-url)))) > >> >> > + (url-recreate-url url-http-target-url) > >> >> > + real-fname)) > >> >> > >> >> That should already be handled by commit 84613dae5c. > >> >> > >> >> Andreas. > >> >> > >> >> -- > >> >> Andreas Schwab, schwab@linux-m68k.org > >> >> GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510 2552 DF73 E780 A9DA AEC1 > >> >> "And now for something completely different." ^ permalink raw reply [flat|nested] 42+ messages in thread
* bug#35969: 26.2, Excorporate 2019-06-20 20:34 ` tenspd137 . @ 2019-07-09 21:52 ` Thomas Fitzsimmons 2019-07-09 22:09 ` Collin Day 0 siblings, 1 reply; 42+ messages in thread From: Thomas Fitzsimmons @ 2019-07-09 21:52 UTC (permalink / raw) To: tenspd137 .; +Cc: Andreas Schwab, 35969 Hi, Were you able to complete building Emacs? I'd like to know if my patch is needed on top of Andreas's to make Excorporate work through your proxy. I'm not sure why the built version would interfere with the current installation. If you were doing "make install" to a common prefix, that might explain it. Instead you can try something like this: cd emacs-master [your github.com checkout] make mkdir test-home HOME=`pwd`/test-home ./src/emacs -Q That will ensure you're only running the built Emacs, and completely ignoring the packages installed in your home directory. Thomas "tenspd137 ." <dcday137@gmail.com> writes: > I haven't tried the first item on the list yet - ie reloading Andreas' > function. i have tried to build emacs from > https://github.com/emacs-mirror/emacs.git because I can't reach the > official repo from my firewall for whatever reason. I am sure it is > something here. Anyway, I was having problems because my built > version (27.0.x) was getting seemingly mixed up with my current > installation. I can try again later, but right now things are pretty > busy at work. > > Thanks. > > On Tue, Jun 18, 2019 at 10:26 PM Thomas Fitzsimmons <fitzsim@fitzsim.org> wrote: >> >> Hi, >> >> "tenspd137 ." <dcday137@gmail.com> writes: >> >> > So - I am not sure if I did it correctly, but I copied this function >> > with Andreas' changes into a file: >> > >> > (defun url-https-proxy-after-change-function (_st _nd _length) >> > (let* ((process-buffer (current-buffer)) >> > (proc (get-buffer-process process-buffer))) >> > (goto-char (point-min)) >> > (when (re-search-forward "^\r?\n" nil t) >> > (backward-char 1) >> > ;; Saw the end of the headers >> > (setq url-http-end-of-headers (set-marker (make-marker) (point))) >> > (url-http-parse-response) >> > (cond >> > ((null url-http-response-status) >> > ;; We got back a headerless malformed response from the >> > ;; server. >> > (url-http-activate-callback) >> > (error "Malformed response from proxy, fail!")) >> > ((= url-http-response-status 200) >> > (if (gnutls-available-p) >> > (condition-case e >> > (let ((tls-connection (gnutls-negotiate >> > :process proc >> > :hostname (url-host url-current-object) >> > :verify-error nil))) >> > ;; check certificate validity >> > (setq tls-connection >> > (nsm-verify-connection tls-connection >> > (url-host url-current-object) >> > (url-port url-current-object))) >> > (with-current-buffer process-buffer (erase-buffer)) >> > (set-process-buffer tls-connection process-buffer) >> > (setq url-http-after-change-function >> > 'url-http-wait-for-headers-change-function) >> > (set-process-filter tls-connection 'url-http-generic-filter) >> > (process-send-string tls-connection >> > ;; Use the non-proxy form of the request >> > (let (url-http-proxy) >> > (url-http-create-request)))) >> > (gnutls-error >> > (url-http-activate-callback) >> > (error "gnutls-error: %s" e)) >> > (error >> > (url-http-activate-callback) >> > (error "error: %s" e))) >> > (error "error: gnutls support needed!"))) >> > (t >> > (url-http-debug "error response: %d" url-http-response-status) >> > (url-http-activate-callback)))))) >> > >> > and then loaded it before running excorporate. After that, I did M-x >> > excorporate, and the minibuffer returns: error in process filter: >> > Server response is not an XML document >> >> In this scenario, if you immediately (without restarting Emacs/reloading >> anything) re-run M-x excorporate does it still fail? I just want to >> make sure that's not a transient failure. If it does fail the second >> time, can you post the HTTP response from the server? >> >> > When I do the similar test by loading the url-http-create-request with >> > Thomas's changes, I can get a connection and grab my schedule >> > through the proxy. >> >> OK. >> >> > Let me know if I need to try something different. >> >> Are you in a position to build Emacs master tip and retry the experiment >> without patching anything? >> >> Thanks, >> Thomas >> >> > On Mon, Jun 17, 2019 at 4:08 PM Thomas Fitzsimmons <fitzsim@fitzsim.org> wrote: >> >> >> >> Hi, >> >> >> >> Good to hear that the patch I posted worked! >> >> >> >> Yes, that's the patch that Andreas's commit >> >> 84613dae5c34ea742dd9a3e56f5acb55f604b483 applied. From what I can tell, >> >> you will not have that in Emacs 26.2. >> >> >> >> Can you try reverting my patch and applying Andreas's patch, and see if >> >> Excorporate still works through the proxy? >> >> >> >> Thanks, >> >> Thomas >> >> >> >> "tenspd137 ." <dcday137@gmail.com> writes: >> >> >> >> > The patch Thomas seems to work from behind the proxy. My current >> >> > emacs version is 26.2, so I would think it would include the commit >> >> > Andreas is talking about.... I went and looked it up - is this the >> >> > correct commit? >> >> > >> >> > diff --git a/lisp/url/url-http.el b/lisp/url/url-http.el >> >> > index 53798f7..817c5ce 100644 >> >> > --- a/lisp/url/url-http.el >> >> > +++ b/lisp/url/url-http.el >> >> > @@ -1412,7 +1412,9 @@ The return value of this function is the >> >> > retrieval buffer." >> >> > 'url-http-wait-for-headers-change-function) >> >> > (set-process-filter tls-connection 'url-http-generic-filter) >> >> > (process-send-string tls-connection >> >> > - (url-http-create-request))) >> >> > + ;; Use the non-proxy form of the request >> >> > + (let (url-http-proxy) >> >> > + (url-http-create-request)))) >> >> > (gnutls-error >> >> > (url-http-activate-callback) >> >> > (error "gnutls-error: %s" e)) >> >> > >> >> > Thanks! >> >> > >> >> > -C >> >> > >> >> > >> >> > On Sat, Jun 15, 2019 at 1:41 AM Andreas Schwab <schwab@linux-m68k.org> wrote: >> >> >> >> >> >> On Jun 14 2019, Thomas Fitzsimmons <fitzsim@fitzsim.org> wrote: >> >> >> >> >> >> > diff --git a/lisp/url/url-http.el b/lisp/url/url-http.el >> >> >> > index 00803a103a..723d111d58 100644 >> >> >> > --- a/lisp/url/url-http.el >> >> >> > +++ b/lisp/url/url-http.el >> >> >> > @@ -329,7 +329,10 @@ url-http-create-request >> >> >> > ;; The request >> >> >> > (or url-http-method "GET") " " >> >> >> > (url-http--encode-string >> >> >> > - (if using-proxy (url-recreate-url url-http-target-url) real-fname)) >> >> >> > + (if (and using-proxy >> >> >> > + (not (equal "https" (url-type url-http-target-url)))) >> >> >> > + (url-recreate-url url-http-target-url) >> >> >> > + real-fname)) >> >> >> >> >> >> That should already be handled by commit 84613dae5c. >> >> >> >> >> >> Andreas. >> >> >> >> >> >> -- >> >> >> Andreas Schwab, schwab@linux-m68k.org >> >> >> GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510 2552 DF73 E780 A9DA AEC1 >> >> >> "And now for something completely different." ^ permalink raw reply [flat|nested] 42+ messages in thread
* bug#35969: 26.2, Excorporate 2019-07-09 21:52 ` Thomas Fitzsimmons @ 2019-07-09 22:09 ` Collin Day 2019-07-09 22:53 ` Collin Day 0 siblings, 1 reply; 42+ messages in thread From: Collin Day @ 2019-07-09 22:09 UTC (permalink / raw) To: Thomas Fitzsimmons; +Cc: Andreas Schwab, 35969 [-- Attachment #1: Type: text/plain, Size: 8234 bytes --] Sorry, I have not had the chance to. A lot has been going on at my place of employment. It has crossed my mind, and as soon as I have a chance, I will try what you suggested above. Thanks, sorry for the inconvenience. On Tue, Jul 9, 2019, 3:53 PM Thomas Fitzsimmons <fitzsim@fitzsim.org> wrote: > Hi, > > Were you able to complete building Emacs? I'd like to know if my patch > is needed on top of Andreas's to make Excorporate work through your > proxy. > > I'm not sure why the built version would interfere with the current > installation. If you were doing "make install" to a common prefix, that > might explain it. Instead you can try something like this: > > cd emacs-master [your github.com checkout] > make > mkdir test-home > HOME=`pwd`/test-home ./src/emacs -Q > > That will ensure you're only running the built Emacs, and completely > ignoring the packages installed in your home directory. > > Thomas > > "tenspd137 ." <dcday137@gmail.com> writes: > > > I haven't tried the first item on the list yet - ie reloading Andreas' > > function. i have tried to build emacs from > > https://github.com/emacs-mirror/emacs.git because I can't reach the > > official repo from my firewall for whatever reason. I am sure it is > > something here. Anyway, I was having problems because my built > > version (27.0.x) was getting seemingly mixed up with my current > > installation. I can try again later, but right now things are pretty > > busy at work. > > > > Thanks. > > > > On Tue, Jun 18, 2019 at 10:26 PM Thomas Fitzsimmons <fitzsim@fitzsim.org> > wrote: > >> > >> Hi, > >> > >> "tenspd137 ." <dcday137@gmail.com> writes: > >> > >> > So - I am not sure if I did it correctly, but I copied this function > >> > with Andreas' changes into a file: > >> > > >> > (defun url-https-proxy-after-change-function (_st _nd _length) > >> > (let* ((process-buffer (current-buffer)) > >> > (proc (get-buffer-process process-buffer))) > >> > (goto-char (point-min)) > >> > (when (re-search-forward "^\r?\n" nil t) > >> > (backward-char 1) > >> > ;; Saw the end of the headers > >> > (setq url-http-end-of-headers (set-marker (make-marker) > (point))) > >> > (url-http-parse-response) > >> > (cond > >> > ((null url-http-response-status) > >> > ;; We got back a headerless malformed response from the > >> > ;; server. > >> > (url-http-activate-callback) > >> > (error "Malformed response from proxy, fail!")) > >> > ((= url-http-response-status 200) > >> > (if (gnutls-available-p) > >> > (condition-case e > >> > (let ((tls-connection (gnutls-negotiate > >> > :process proc > >> > :hostname (url-host > url-current-object) > >> > :verify-error nil))) > >> > ;; check certificate validity > >> > (setq tls-connection > >> > (nsm-verify-connection tls-connection > >> > (url-host > url-current-object) > >> > (url-port > url-current-object))) > >> > (with-current-buffer process-buffer (erase-buffer)) > >> > (set-process-buffer tls-connection process-buffer) > >> > (setq url-http-after-change-function > >> > 'url-http-wait-for-headers-change-function) > >> > (set-process-filter tls-connection > 'url-http-generic-filter) > >> > (process-send-string tls-connection > >> > ;; Use the non-proxy form of > the request > >> > (let (url-http-proxy) > >> > (url-http-create-request)))) > >> > (gnutls-error > >> > (url-http-activate-callback) > >> > (error "gnutls-error: %s" e)) > >> > (error > >> > (url-http-activate-callback) > >> > (error "error: %s" e))) > >> > (error "error: gnutls support needed!"))) > >> > (t > >> > (url-http-debug "error response: %d" url-http-response-status) > >> > (url-http-activate-callback)))))) > >> > > >> > and then loaded it before running excorporate. After that, I did M-x > >> > excorporate, and the minibuffer returns: error in process filter: > >> > Server response is not an XML document > >> > >> In this scenario, if you immediately (without restarting Emacs/reloading > >> anything) re-run M-x excorporate does it still fail? I just want to > >> make sure that's not a transient failure. If it does fail the second > >> time, can you post the HTTP response from the server? > >> > >> > When I do the similar test by loading the url-http-create-request with > >> > Thomas's changes, I can get a connection and grab my schedule > >> > through the proxy. > >> > >> OK. > >> > >> > Let me know if I need to try something different. > >> > >> Are you in a position to build Emacs master tip and retry the experiment > >> without patching anything? > >> > >> Thanks, > >> Thomas > >> > >> > On Mon, Jun 17, 2019 at 4:08 PM Thomas Fitzsimmons < > fitzsim@fitzsim.org> wrote: > >> >> > >> >> Hi, > >> >> > >> >> Good to hear that the patch I posted worked! > >> >> > >> >> Yes, that's the patch that Andreas's commit > >> >> 84613dae5c34ea742dd9a3e56f5acb55f604b483 applied. From what I can > tell, > >> >> you will not have that in Emacs 26.2. > >> >> > >> >> Can you try reverting my patch and applying Andreas's patch, and see > if > >> >> Excorporate still works through the proxy? > >> >> > >> >> Thanks, > >> >> Thomas > >> >> > >> >> "tenspd137 ." <dcday137@gmail.com> writes: > >> >> > >> >> > The patch Thomas seems to work from behind the proxy. My current > >> >> > emacs version is 26.2, so I would think it would include the commit > >> >> > Andreas is talking about.... I went and looked it up - is this the > >> >> > correct commit? > >> >> > > >> >> > diff --git a/lisp/url/url-http.el b/lisp/url/url-http.el > >> >> > index 53798f7..817c5ce 100644 > >> >> > --- a/lisp/url/url-http.el > >> >> > +++ b/lisp/url/url-http.el > >> >> > @@ -1412,7 +1412,9 @@ The return value of this function is the > >> >> > retrieval buffer." > >> >> > 'url-http-wait-for-headers-change-function) > >> >> > (set-process-filter tls-connection 'url-http-generic-filter) > >> >> > (process-send-string tls-connection > >> >> > - (url-http-create-request))) > >> >> > + ;; Use the non-proxy form of the request > >> >> > + (let (url-http-proxy) > >> >> > + (url-http-create-request)))) > >> >> > (gnutls-error > >> >> > (url-http-activate-callback) > >> >> > (error "gnutls-error: %s" e)) > >> >> > > >> >> > Thanks! > >> >> > > >> >> > -C > >> >> > > >> >> > > >> >> > On Sat, Jun 15, 2019 at 1:41 AM Andreas Schwab < > schwab@linux-m68k.org> wrote: > >> >> >> > >> >> >> On Jun 14 2019, Thomas Fitzsimmons <fitzsim@fitzsim.org> wrote: > >> >> >> > >> >> >> > diff --git a/lisp/url/url-http.el b/lisp/url/url-http.el > >> >> >> > index 00803a103a..723d111d58 100644 > >> >> >> > --- a/lisp/url/url-http.el > >> >> >> > +++ b/lisp/url/url-http.el > >> >> >> > @@ -329,7 +329,10 @@ url-http-create-request > >> >> >> > ;; The request > >> >> >> > (or url-http-method "GET") " " > >> >> >> > (url-http--encode-string > >> >> >> > - (if using-proxy (url-recreate-url > url-http-target-url) real-fname)) > >> >> >> > + (if (and using-proxy > >> >> >> > + (not (equal "https" (url-type > url-http-target-url)))) > >> >> >> > + (url-recreate-url url-http-target-url) > >> >> >> > + real-fname)) > >> >> >> > >> >> >> That should already be handled by commit 84613dae5c. > >> >> >> > >> >> >> Andreas. > >> >> >> > >> >> >> -- > >> >> >> Andreas Schwab, schwab@linux-m68k.org > >> >> >> GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510 2552 DF73 E780 > A9DA AEC1 > >> >> >> "And now for something completely different." > [-- Attachment #2: Type: text/html, Size: 12352 bytes --] ^ permalink raw reply [flat|nested] 42+ messages in thread
* bug#35969: 26.2, Excorporate 2019-07-09 22:09 ` Collin Day @ 2019-07-09 22:53 ` Collin Day 2019-07-10 0:08 ` Thomas Fitzsimmons 2019-07-13 14:37 ` Thomas Fitzsimmons 0 siblings, 2 replies; 42+ messages in thread From: Collin Day @ 2019-07-09 22:53 UTC (permalink / raw) To: Thomas Fitzsimmons; +Cc: Andreas Schwab, 35969 Did a git pull and followed instructions above. There are some issues. First, I need to run M-x package-install <RET> excorporate <RET> four times because I see (each line after each invocation) package--with-response-buffer-1: https://elpa.gnu.org/packages/url-http-ntlm-2.0.4.el: Method not allowed package--with-response-buffer-1: https://elpa.gnu.org/packages/fsm-0.2.1.el: Method not allowed package--with-response-buffer-1: https://elpa.gnu.org/packages/excorporate-0.8.3.tar: Method not allowed After the 4th call it compiles and becomes available. M-x customize-group <RET> excorporate, set up for no autoconfig right now, hit apply for current sessions, save not availiable.... M-x excorporate <RET> enter uname and password Contacting host: outlook.office365.com:443 error in process filter: exco--parse-xml-in-current-buffer: Server response is not an XML document error in process filter: Server response is not an XML documen *http outlook.office.365.com:443* HTTP/1.1 401 Unauthorized Server: Microsoft-IIS/10.0 request-id: f8e80f64-b51d-4d66-88ef-4ee7f09814f3 X-WSSecurity-Enabled: True X-WSSecurity-For: Logon X-FederationTrustTokenIssuerUri: urn:federation:MicrosoftOnline X-WSSecurity-SymmetricKey-Enabled: True X-WSSecurity-X509Cert-Enabled: True X-OAuth-Enabled: True X-Powered-By: ASP.NET X-FEServer: SN4PR0401CA0031 WWW-Authenticate: Basic Realm="" Date: Tue, 09 Jul 2019 22:48:22 GMT Content-Length: 0 URL-DEBUG http -> Saw end of headers... ( *http outlook.office365.com:443*) http -> url-http-parse-response called in ( *http outlook.office365.com:443*) http -> Got a content-length, being smart about document end. http -> Got 0-length content-length, activating callback immediately. http -> Marking connection as free: outlook.office365.com:443 #<process outlook.office365.com> http -> url-http-parse-headers called in ( *http outlook.office365.com:443*) http -> url-http-parse-response called in ( *http outlook.office365.com:443*) http -> Parsed HTTP headers: class=4 status=401 http -> Handling normal authentication http -> Found existing connection: outlook.office365.com:443 #<process outlook.office365.com> http -> Reusing existing connection: outlook.office365.com:443 http -> Marking connection as busy: outlook.office365.com:443 #<process outlook.office365.com> http -> getting referer from buffer: buffer:#<buffer *http outlook.office365.com:443*> target-url:#s(url "https" nil nil "outlook.office365.com" nil "/EWS/Services.wsdl" nil nil t nil t t) lastloc:nil http -> Finished parsing HTTP headers: nil http -> Spinning waiting for headers... http -> Calling after change function `url-https-proxy-after-change-function' for `#<process outlook.office365.com>' http -> url-http-parse-response called in ( *http outlook.office365.com:443*-535296) http -> error response: 400 http -> Marking connection as free: outlook.office365.com:443 #<process outlook.office365.com> http -> Activating callback in buffer ( *http outlook.office365.com:443*-535296): #[257 "p\303\304\305\306\307 !\310\"\311$\216 @\312=\203$\0\313\301\314\315\316\302\"#\210\317\300\320\"\202/\0\313\301\321\322 #\210\317\300\323\")\207" [fsm-exco--fsm-0 (:identifier ("my email" . "https://outlook.office365.com/EWS/Excahnge.asmx") :mail-address "my email" :retrying nil :autodiscovery-urls nil :service-url "https://outlook.office365.com/EWS/Excahnge.asmx" :service-xml nil :service-wsdl nil :next-state-after-success nil :failure-message nil :server-version nil) "https://outlook.office365.com/EWS/Services.wsdl" make-byte-code 0 "\301\300!\205 \0\302\300!\207" vconcat vector [buffer-live-p kill-buffer] 2 :error plist-put :failure-message format "Failed to retrieve %s" fsm-send :unrecoverable-error :service-xml exco--parse-xml-in-current-buffer :success] 8 " (fn STATUS)"] ((:peer (:certificates ((:version 3 :serial-number "0a:48:28:08:de:bc:a4:10:92:0f:87:6a:63:05:a1:e9" :issuer "C=US,O=DigiCert Inc,CN=DigiCert Cloud Services CA-1" :valid-from "2018-11-18" :valid-to "2020-11-18" :subject "C=US,ST=Washington,L=Redmond,O=Microsoft Corporation,CN=outlook.com" :public-key-algorithm "RSA" :certificate-security-level "Medium" :signature-algorithm "RSA-SHA256" :public-key-id "sha1:ae:b5:9e:3f:4e:c3:72:a8:c0:fc:24:4e:24:b4:d6:8d:99:b0:b9:e1" :certificate-id "sha1:05:b8:36:4c:33:e5:63:7e:fd:88:e0:a3:b8:7e:7d:cf:6f:8d:d0:d4") (:version 3 :serial-number "01:9e:c1:c6:bd:3f:59:7b:b2:0c:33:38:e5:51:d8:77" :issuer "C=US,O=DigiCert Inc,OU=www.digicert.com,CN=DigiCert Global Root CA" :valid-from "2015-08-04" :valid-to "2030-08-04" :subject "C=US,O=DigiCert Inc,CN=DigiCert Cloud Services CA-1" :public-key-algorithm "RSA" :certificate-security-level "Medium" :signature-algorithm "RSA-SHA256" :public-key-id "sha1:ac:13:d5:79:03:66:f3:cc:ff:fe:7c:23:71:a7:61:5d:39:20:89:6e" :certificate-id "sha1:81:b6:8d:6c:d2:f2:21:f8:f5:34:e6:77:52:3b:b2:36:bb:a1:dc:56")) :certificate (:version 3 :serial-number "0a:48:28:08:de:bc:a4:10:92:0f:87:6a:63:05:a1:e9" :issuer "C=US,O=DigiCert Inc,CN=DigiCert Cloud Services CA-1" :valid-from "2018-11-18" :valid-to "2020-11-18" :subject "C=US,ST=Washington,L=Redmond,O=Microsoft Corporation,CN=outlook.com" :public-key-algorithm "RSA" :certificate-security-level "Medium" :signature-algorithm "RSA-SHA256" :public-key-id "sha1:ae:b5:9e:3f:4e:c3:72:a8:c0:fc:24:4e:24:b4:d6:8d:99:b0:b9:e1" :certificate-id "sha1:05:b8:36:4c:33:e5:63:7e:fd:88:e0:a3:b8:7e:7d:cf:6f:8d:d0:d4") :key-exchange "ECDHE-RSA" :protocol "TLS1.2" :cipher "AES-256-GCM" :mac "AEAD"))) So it appears not to work, at least on the current head as of today.... Thanks! -C On Tue, Jul 9, 2019 at 4:09 PM Collin Day <dcday137@gmail.com> wrote: > > Sorry, I have not had the chance to. A lot has been going on at my place of employment. It has crossed my mind, and as soon as I have a chance, I will try what you suggested above. Thanks, sorry for the inconvenience. > > On Tue, Jul 9, 2019, 3:53 PM Thomas Fitzsimmons <fitzsim@fitzsim.org> wrote: >> >> Hi, >> >> Were you able to complete building Emacs? I'd like to know if my patch >> is needed on top of Andreas's to make Excorporate work through your >> proxy. >> >> I'm not sure why the built version would interfere with the current >> installation. If you were doing "make install" to a common prefix, that >> might explain it. Instead you can try something like this: >> >> cd emacs-master [your github.com checkout] >> make >> mkdir test-home >> HOME=`pwd`/test-home ./src/emacs -Q >> >> That will ensure you're only running the built Emacs, and completely >> ignoring the packages installed in your home directory. >> >> Thomas >> >> "tenspd137 ." <dcday137@gmail.com> writes: >> >> > I haven't tried the first item on the list yet - ie reloading Andreas' >> > function. i have tried to build emacs from >> > https://github.com/emacs-mirror/emacs.git because I can't reach the >> > official repo from my firewall for whatever reason. I am sure it is >> > something here. Anyway, I was having problems because my built >> > version (27.0.x) was getting seemingly mixed up with my current >> > installation. I can try again later, but right now things are pretty >> > busy at work. >> > >> > Thanks. >> > >> > On Tue, Jun 18, 2019 at 10:26 PM Thomas Fitzsimmons <fitzsim@fitzsim.org> wrote: >> >> >> >> Hi, >> >> >> >> "tenspd137 ." <dcday137@gmail.com> writes: >> >> >> >> > So - I am not sure if I did it correctly, but I copied this function >> >> > with Andreas' changes into a file: >> >> > >> >> > (defun url-https-proxy-after-change-function (_st _nd _length) >> >> > (let* ((process-buffer (current-buffer)) >> >> > (proc (get-buffer-process process-buffer))) >> >> > (goto-char (point-min)) >> >> > (when (re-search-forward "^\r?\n" nil t) >> >> > (backward-char 1) >> >> > ;; Saw the end of the headers >> >> > (setq url-http-end-of-headers (set-marker (make-marker) (point))) >> >> > (url-http-parse-response) >> >> > (cond >> >> > ((null url-http-response-status) >> >> > ;; We got back a headerless malformed response from the >> >> > ;; server. >> >> > (url-http-activate-callback) >> >> > (error "Malformed response from proxy, fail!")) >> >> > ((= url-http-response-status 200) >> >> > (if (gnutls-available-p) >> >> > (condition-case e >> >> > (let ((tls-connection (gnutls-negotiate >> >> > :process proc >> >> > :hostname (url-host url-current-object) >> >> > :verify-error nil))) >> >> > ;; check certificate validity >> >> > (setq tls-connection >> >> > (nsm-verify-connection tls-connection >> >> > (url-host url-current-object) >> >> > (url-port url-current-object))) >> >> > (with-current-buffer process-buffer (erase-buffer)) >> >> > (set-process-buffer tls-connection process-buffer) >> >> > (setq url-http-after-change-function >> >> > 'url-http-wait-for-headers-change-function) >> >> > (set-process-filter tls-connection 'url-http-generic-filter) >> >> > (process-send-string tls-connection >> >> > ;; Use the non-proxy form of the request >> >> > (let (url-http-proxy) >> >> > (url-http-create-request)))) >> >> > (gnutls-error >> >> > (url-http-activate-callback) >> >> > (error "gnutls-error: %s" e)) >> >> > (error >> >> > (url-http-activate-callback) >> >> > (error "error: %s" e))) >> >> > (error "error: gnutls support needed!"))) >> >> > (t >> >> > (url-http-debug "error response: %d" url-http-response-status) >> >> > (url-http-activate-callback)))))) >> >> > >> >> > and then loaded it before running excorporate. After that, I did M-x >> >> > excorporate, and the minibuffer returns: error in process filter: >> >> > Server response is not an XML document >> >> >> >> In this scenario, if you immediately (without restarting Emacs/reloading >> >> anything) re-run M-x excorporate does it still fail? I just want to >> >> make sure that's not a transient failure. If it does fail the second >> >> time, can you post the HTTP response from the server? >> >> >> >> > When I do the similar test by loading the url-http-create-request with >> >> > Thomas's changes, I can get a connection and grab my schedule >> >> > through the proxy. >> >> >> >> OK. >> >> >> >> > Let me know if I need to try something different. >> >> >> >> Are you in a position to build Emacs master tip and retry the experiment >> >> without patching anything? >> >> >> >> Thanks, >> >> Thomas >> >> >> >> > On Mon, Jun 17, 2019 at 4:08 PM Thomas Fitzsimmons <fitzsim@fitzsim.org> wrote: >> >> >> >> >> >> Hi, >> >> >> >> >> >> Good to hear that the patch I posted worked! >> >> >> >> >> >> Yes, that's the patch that Andreas's commit >> >> >> 84613dae5c34ea742dd9a3e56f5acb55f604b483 applied. From what I can tell, >> >> >> you will not have that in Emacs 26.2. >> >> >> >> >> >> Can you try reverting my patch and applying Andreas's patch, and see if >> >> >> Excorporate still works through the proxy? >> >> >> >> >> >> Thanks, >> >> >> Thomas >> >> >> >> >> >> "tenspd137 ." <dcday137@gmail.com> writes: >> >> >> >> >> >> > The patch Thomas seems to work from behind the proxy. My current >> >> >> > emacs version is 26.2, so I would think it would include the commit >> >> >> > Andreas is talking about.... I went and looked it up - is this the >> >> >> > correct commit? >> >> >> > >> >> >> > diff --git a/lisp/url/url-http.el b/lisp/url/url-http.el >> >> >> > index 53798f7..817c5ce 100644 >> >> >> > --- a/lisp/url/url-http.el >> >> >> > +++ b/lisp/url/url-http.el >> >> >> > @@ -1412,7 +1412,9 @@ The return value of this function is the >> >> >> > retrieval buffer." >> >> >> > 'url-http-wait-for-headers-change-function) >> >> >> > (set-process-filter tls-connection 'url-http-generic-filter) >> >> >> > (process-send-string tls-connection >> >> >> > - (url-http-create-request))) >> >> >> > + ;; Use the non-proxy form of the request >> >> >> > + (let (url-http-proxy) >> >> >> > + (url-http-create-request)))) >> >> >> > (gnutls-error >> >> >> > (url-http-activate-callback) >> >> >> > (error "gnutls-error: %s" e)) >> >> >> > >> >> >> > Thanks! >> >> >> > >> >> >> > -C >> >> >> > >> >> >> > >> >> >> > On Sat, Jun 15, 2019 at 1:41 AM Andreas Schwab <schwab@linux-m68k.org> wrote: >> >> >> >> >> >> >> >> On Jun 14 2019, Thomas Fitzsimmons <fitzsim@fitzsim.org> wrote: >> >> >> >> >> >> >> >> > diff --git a/lisp/url/url-http.el b/lisp/url/url-http.el >> >> >> >> > index 00803a103a..723d111d58 100644 >> >> >> >> > --- a/lisp/url/url-http.el >> >> >> >> > +++ b/lisp/url/url-http.el >> >> >> >> > @@ -329,7 +329,10 @@ url-http-create-request >> >> >> >> > ;; The request >> >> >> >> > (or url-http-method "GET") " " >> >> >> >> > (url-http--encode-string >> >> >> >> > - (if using-proxy (url-recreate-url url-http-target-url) real-fname)) >> >> >> >> > + (if (and using-proxy >> >> >> >> > + (not (equal "https" (url-type url-http-target-url)))) >> >> >> >> > + (url-recreate-url url-http-target-url) >> >> >> >> > + real-fname)) >> >> >> >> >> >> >> >> That should already be handled by commit 84613dae5c. >> >> >> >> >> >> >> >> Andreas. >> >> >> >> >> >> >> >> -- >> >> >> >> Andreas Schwab, schwab@linux-m68k.org >> >> >> >> GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510 2552 DF73 E780 A9DA AEC1 >> >> >> >> "And now for something completely different." ^ permalink raw reply [flat|nested] 42+ messages in thread
* bug#35969: 26.2, Excorporate 2019-07-09 22:53 ` Collin Day @ 2019-07-10 0:08 ` Thomas Fitzsimmons 2019-07-10 0:11 ` Thomas Fitzsimmons 2019-07-13 14:37 ` Thomas Fitzsimmons 1 sibling, 1 reply; 42+ messages in thread From: Thomas Fitzsimmons @ 2019-07-10 0:08 UTC (permalink / raw) To: Collin Day; +Cc: Andreas Schwab, 35969 OK, can you apply my patch, then run "make" again and confirm it works? Thanks, Thomas Collin Day <dcday137@gmail.com> writes: > Did a git pull and followed instructions above. > > There are some issues. First, I need to run M-x package-install <RET> > excorporate <RET> four times because I see (each line after each > invocation) > > package--with-response-buffer-1: > https://elpa.gnu.org/packages/url-http-ntlm-2.0.4.el: Method not > allowed > package--with-response-buffer-1: > https://elpa.gnu.org/packages/fsm-0.2.1.el: Method not allowed > package--with-response-buffer-1: > https://elpa.gnu.org/packages/excorporate-0.8.3.tar: Method not > allowed > > After the 4th call it compiles and becomes available. > > M-x customize-group <RET> excorporate, set up for no autoconfig right > now, hit apply for current sessions, save not availiable.... > > M-x excorporate <RET> > > enter uname and password > > Contacting host: outlook.office365.com:443 > error in process filter: exco--parse-xml-in-current-buffer: Server > response is not an XML document > error in process filter: Server response is not an XML documen > > *http outlook.office.365.com:443* [...] > So it appears not to work, at least on the current head as of today.... > > Thanks! > > -C > > On Tue, Jul 9, 2019 at 4:09 PM Collin Day <dcday137@gmail.com> wrote: >> >> Sorry, I have not had the chance to. A lot has been going on at my >> place of employment. It has crossed my mind, and as soon as I have >> a chance, I will try what you suggested above. Thanks, sorry for >> the inconvenience. >> >> On Tue, Jul 9, 2019, 3:53 PM Thomas Fitzsimmons <fitzsim@fitzsim.org> wrote: >>> >>> Hi, >>> >>> Were you able to complete building Emacs? I'd like to know if my patch >>> is needed on top of Andreas's to make Excorporate work through your >>> proxy. >>> >>> I'm not sure why the built version would interfere with the current >>> installation. If you were doing "make install" to a common prefix, that >>> might explain it. Instead you can try something like this: >>> >>> cd emacs-master [your github.com checkout] >>> make >>> mkdir test-home >>> HOME=`pwd`/test-home ./src/emacs -Q >>> >>> That will ensure you're only running the built Emacs, and completely >>> ignoring the packages installed in your home directory. >>> >>> Thomas >>> >>> "tenspd137 ." <dcday137@gmail.com> writes: >>> >>> > I haven't tried the first item on the list yet - ie reloading Andreas' >>> > function. i have tried to build emacs from >>> > https://github.com/emacs-mirror/emacs.git because I can't reach the >>> > official repo from my firewall for whatever reason. I am sure it is >>> > something here. Anyway, I was having problems because my built >>> > version (27.0.x) was getting seemingly mixed up with my current >>> > installation. I can try again later, but right now things are pretty >>> > busy at work. >>> > >>> > Thanks. >>> > >>> > On Tue, Jun 18, 2019 at 10:26 PM Thomas Fitzsimmons <fitzsim@fitzsim.org> wrote: >>> >> >>> >> Hi, >>> >> >>> >> "tenspd137 ." <dcday137@gmail.com> writes: >>> >> >>> >> > So - I am not sure if I did it correctly, but I copied this function >>> >> > with Andreas' changes into a file: >>> >> > >>> >> > (defun url-https-proxy-after-change-function (_st _nd _length) >>> >> > (let* ((process-buffer (current-buffer)) >>> >> > (proc (get-buffer-process process-buffer))) >>> >> > (goto-char (point-min)) >>> >> > (when (re-search-forward "^\r?\n" nil t) >>> >> > (backward-char 1) >>> >> > ;; Saw the end of the headers >>> >> > (setq url-http-end-of-headers (set-marker (make-marker) (point))) >>> >> > (url-http-parse-response) >>> >> > (cond >>> >> > ((null url-http-response-status) >>> >> > ;; We got back a headerless malformed response from the >>> >> > ;; server. >>> >> > (url-http-activate-callback) >>> >> > (error "Malformed response from proxy, fail!")) >>> >> > ((= url-http-response-status 200) >>> >> > (if (gnutls-available-p) >>> >> > (condition-case e >>> >> > (let ((tls-connection (gnutls-negotiate >>> >> > :process proc >>> >> > :hostname (url-host url-current-object) >>> >> > :verify-error nil))) >>> >> > ;; check certificate validity >>> >> > (setq tls-connection >>> >> > (nsm-verify-connection tls-connection >>> >> > (url-host url-current-object) >>> >> > (url-port url-current-object))) >>> >> > (with-current-buffer process-buffer (erase-buffer)) >>> >> > (set-process-buffer tls-connection process-buffer) >>> >> > (setq url-http-after-change-function >>> >> > 'url-http-wait-for-headers-change-function) >>> >> > (set-process-filter tls-connection 'url-http-generic-filter) >>> >> > (process-send-string tls-connection >>> >> > ;; Use the non-proxy form of the request >>> >> > (let (url-http-proxy) >>> >> > (url-http-create-request)))) >>> >> > (gnutls-error >>> >> > (url-http-activate-callback) >>> >> > (error "gnutls-error: %s" e)) >>> >> > (error >>> >> > (url-http-activate-callback) >>> >> > (error "error: %s" e))) >>> >> > (error "error: gnutls support needed!"))) >>> >> > (t >>> >> > (url-http-debug "error response: %d" url-http-response-status) >>> >> > (url-http-activate-callback)))))) >>> >> > >>> >> > and then loaded it before running excorporate. After that, I did M-x >>> >> > excorporate, and the minibuffer returns: error in process filter: >>> >> > Server response is not an XML document >>> >> >>> >> In this scenario, if you immediately (without restarting Emacs/reloading >>> >> anything) re-run M-x excorporate does it still fail? I just want to >>> >> make sure that's not a transient failure. If it does fail the second >>> >> time, can you post the HTTP response from the server? >>> >> >>> >> > When I do the similar test by loading the url-http-create-request with >>> >> > Thomas's changes, I can get a connection and grab my schedule >>> >> > through the proxy. >>> >> >>> >> OK. >>> >> >>> >> > Let me know if I need to try something different. >>> >> >>> >> Are you in a position to build Emacs master tip and retry the experiment >>> >> without patching anything? >>> >> >>> >> Thanks, >>> >> Thomas >>> >> >>> >> > On Mon, Jun 17, 2019 at 4:08 PM Thomas Fitzsimmons <fitzsim@fitzsim.org> wrote: >>> >> >> >>> >> >> Hi, >>> >> >> >>> >> >> Good to hear that the patch I posted worked! >>> >> >> >>> >> >> Yes, that's the patch that Andreas's commit >>> >> >> 84613dae5c34ea742dd9a3e56f5acb55f604b483 applied. From what I can tell, >>> >> >> you will not have that in Emacs 26.2. >>> >> >> >>> >> >> Can you try reverting my patch and applying Andreas's patch, and see if >>> >> >> Excorporate still works through the proxy? >>> >> >> >>> >> >> Thanks, >>> >> >> Thomas >>> >> >> >>> >> >> "tenspd137 ." <dcday137@gmail.com> writes: >>> >> >> >>> >> >> > The patch Thomas seems to work from behind the proxy. My current >>> >> >> > emacs version is 26.2, so I would think it would include the commit >>> >> >> > Andreas is talking about.... I went and looked it up - is this the >>> >> >> > correct commit? >>> >> >> > >>> >> >> > diff --git a/lisp/url/url-http.el b/lisp/url/url-http.el >>> >> >> > index 53798f7..817c5ce 100644 >>> >> >> > --- a/lisp/url/url-http.el >>> >> >> > +++ b/lisp/url/url-http.el >>> >> >> > @@ -1412,7 +1412,9 @@ The return value of this function is the >>> >> >> > retrieval buffer." >>> >> >> > 'url-http-wait-for-headers-change-function) >>> >> >> > (set-process-filter tls-connection 'url-http-generic-filter) >>> >> >> > (process-send-string tls-connection >>> >> >> > - (url-http-create-request))) >>> >> >> > + ;; Use the non-proxy form of the request >>> >> >> > + (let (url-http-proxy) >>> >> >> > + (url-http-create-request)))) >>> >> >> > (gnutls-error >>> >> >> > (url-http-activate-callback) >>> >> >> > (error "gnutls-error: %s" e)) >>> >> >> > >>> >> >> > Thanks! >>> >> >> > >>> >> >> > -C >>> >> >> > >>> >> >> > >>> >> >> > On Sat, Jun 15, 2019 at 1:41 AM Andreas Schwab <schwab@linux-m68k.org> wrote: >>> >> >> >> >>> >> >> >> On Jun 14 2019, Thomas Fitzsimmons <fitzsim@fitzsim.org> wrote: >>> >> >> >> >>> >> >> >> > diff --git a/lisp/url/url-http.el b/lisp/url/url-http.el >>> >> >> >> > index 00803a103a..723d111d58 100644 >>> >> >> >> > --- a/lisp/url/url-http.el >>> >> >> >> > +++ b/lisp/url/url-http.el >>> >> >> >> > @@ -329,7 +329,10 @@ url-http-create-request >>> >> >> >> > ;; The request >>> >> >> >> > (or url-http-method "GET") " " >>> >> >> >> > (url-http--encode-string >>> >> >> >> > - (if using-proxy (url-recreate-url url-http-target-url) real-fname)) >>> >> >> >> > + (if (and using-proxy >>> >> >> >> > + (not (equal "https" (url-type url-http-target-url)))) >>> >> >> >> > + (url-recreate-url url-http-target-url) >>> >> >> >> > + real-fname)) >>> >> >> >> >>> >> >> >> That should already be handled by commit 84613dae5c. >>> >> >> >> >>> >> >> >> Andreas. >>> >> >> >> >>> >> >> >> -- >>> >> >> >> Andreas Schwab, schwab@linux-m68k.org >>> >> >> >> GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510 2552 DF73 E780 A9DA AEC1 >>> >> >> >> "And now for something completely different." ^ permalink raw reply [flat|nested] 42+ messages in thread
* bug#35969: 26.2, Excorporate 2019-07-10 0:08 ` Thomas Fitzsimmons @ 2019-07-10 0:11 ` Thomas Fitzsimmons 2019-07-10 15:35 ` Collin Day 0 siblings, 1 reply; 42+ messages in thread From: Thomas Fitzsimmons @ 2019-07-10 0:11 UTC (permalink / raw) To: Collin Day; +Cc: Andreas Schwab, 35969 I think there should be no need to remove test-home, so you won't have to re-run the M-x package-install steps. Thomas Thomas Fitzsimmons <fitzsim@fitzsim.org> writes: > OK, can you apply my patch, then run "make" again and confirm it works? > > Thanks, > Thomas > > Collin Day <dcday137@gmail.com> writes: > >> Did a git pull and followed instructions above. >> >> There are some issues. First, I need to run M-x package-install <RET> >> excorporate <RET> four times because I see (each line after each >> invocation) >> >> package--with-response-buffer-1: >> https://elpa.gnu.org/packages/url-http-ntlm-2.0.4.el: Method not >> allowed >> package--with-response-buffer-1: >> https://elpa.gnu.org/packages/fsm-0.2.1.el: Method not allowed >> package--with-response-buffer-1: >> https://elpa.gnu.org/packages/excorporate-0.8.3.tar: Method not >> allowed >> >> After the 4th call it compiles and becomes available. >> >> M-x customize-group <RET> excorporate, set up for no autoconfig right >> now, hit apply for current sessions, save not availiable.... >> >> M-x excorporate <RET> >> >> enter uname and password >> >> Contacting host: outlook.office365.com:443 >> error in process filter: exco--parse-xml-in-current-buffer: Server >> response is not an XML document >> error in process filter: Server response is not an XML documen >> >> *http outlook.office.365.com:443* > > [...] > >> So it appears not to work, at least on the current head as of today.... >> >> Thanks! >> >> -C >> >> On Tue, Jul 9, 2019 at 4:09 PM Collin Day <dcday137@gmail.com> wrote: >>> >>> Sorry, I have not had the chance to. A lot has been going on at my >>> place of employment. It has crossed my mind, and as soon as I have >>> a chance, I will try what you suggested above. Thanks, sorry for >>> the inconvenience. >>> >>> On Tue, Jul 9, 2019, 3:53 PM Thomas Fitzsimmons <fitzsim@fitzsim.org> wrote: >>>> >>>> Hi, >>>> >>>> Were you able to complete building Emacs? I'd like to know if my patch >>>> is needed on top of Andreas's to make Excorporate work through your >>>> proxy. >>>> >>>> I'm not sure why the built version would interfere with the current >>>> installation. If you were doing "make install" to a common prefix, that >>>> might explain it. Instead you can try something like this: >>>> >>>> cd emacs-master [your github.com checkout] >>>> make >>>> mkdir test-home >>>> HOME=`pwd`/test-home ./src/emacs -Q >>>> >>>> That will ensure you're only running the built Emacs, and completely >>>> ignoring the packages installed in your home directory. >>>> >>>> Thomas >>>> >>>> "tenspd137 ." <dcday137@gmail.com> writes: >>>> >>>> > I haven't tried the first item on the list yet - ie reloading Andreas' >>>> > function. i have tried to build emacs from >>>> > https://github.com/emacs-mirror/emacs.git because I can't reach the >>>> > official repo from my firewall for whatever reason. I am sure it is >>>> > something here. Anyway, I was having problems because my built >>>> > version (27.0.x) was getting seemingly mixed up with my current >>>> > installation. I can try again later, but right now things are pretty >>>> > busy at work. >>>> > >>>> > Thanks. >>>> > >>>> > On Tue, Jun 18, 2019 at 10:26 PM Thomas Fitzsimmons <fitzsim@fitzsim.org> wrote: >>>> >> >>>> >> Hi, >>>> >> >>>> >> "tenspd137 ." <dcday137@gmail.com> writes: >>>> >> >>>> >> > So - I am not sure if I did it correctly, but I copied this function >>>> >> > with Andreas' changes into a file: >>>> >> > >>>> >> > (defun url-https-proxy-after-change-function (_st _nd _length) >>>> >> > (let* ((process-buffer (current-buffer)) >>>> >> > (proc (get-buffer-process process-buffer))) >>>> >> > (goto-char (point-min)) >>>> >> > (when (re-search-forward "^\r?\n" nil t) >>>> >> > (backward-char 1) >>>> >> > ;; Saw the end of the headers >>>> >> > (setq url-http-end-of-headers (set-marker (make-marker) (point))) >>>> >> > (url-http-parse-response) >>>> >> > (cond >>>> >> > ((null url-http-response-status) >>>> >> > ;; We got back a headerless malformed response from the >>>> >> > ;; server. >>>> >> > (url-http-activate-callback) >>>> >> > (error "Malformed response from proxy, fail!")) >>>> >> > ((= url-http-response-status 200) >>>> >> > (if (gnutls-available-p) >>>> >> > (condition-case e >>>> >> > (let ((tls-connection (gnutls-negotiate >>>> >> > :process proc >>>> >> > :hostname (url-host url-current-object) >>>> >> > :verify-error nil))) >>>> >> > ;; check certificate validity >>>> >> > (setq tls-connection >>>> >> > (nsm-verify-connection tls-connection >>>> >> > (url-host url-current-object) >>>> >> > (url-port url-current-object))) >>>> >> > (with-current-buffer process-buffer (erase-buffer)) >>>> >> > (set-process-buffer tls-connection process-buffer) >>>> >> > (setq url-http-after-change-function >>>> >> > 'url-http-wait-for-headers-change-function) >>>> >> > (set-process-filter tls-connection 'url-http-generic-filter) >>>> >> > (process-send-string tls-connection >>>> >> > ;; Use the non-proxy form of the request >>>> >> > (let (url-http-proxy) >>>> >> > (url-http-create-request)))) >>>> >> > (gnutls-error >>>> >> > (url-http-activate-callback) >>>> >> > (error "gnutls-error: %s" e)) >>>> >> > (error >>>> >> > (url-http-activate-callback) >>>> >> > (error "error: %s" e))) >>>> >> > (error "error: gnutls support needed!"))) >>>> >> > (t >>>> >> > (url-http-debug "error response: %d" url-http-response-status) >>>> >> > (url-http-activate-callback)))))) >>>> >> > >>>> >> > and then loaded it before running excorporate. After that, I did M-x >>>> >> > excorporate, and the minibuffer returns: error in process filter: >>>> >> > Server response is not an XML document >>>> >> >>>> >> In this scenario, if you immediately (without restarting Emacs/reloading >>>> >> anything) re-run M-x excorporate does it still fail? I just want to >>>> >> make sure that's not a transient failure. If it does fail the second >>>> >> time, can you post the HTTP response from the server? >>>> >> >>>> >> > When I do the similar test by loading the url-http-create-request with >>>> >> > Thomas's changes, I can get a connection and grab my schedule >>>> >> > through the proxy. >>>> >> >>>> >> OK. >>>> >> >>>> >> > Let me know if I need to try something different. >>>> >> >>>> >> Are you in a position to build Emacs master tip and retry the experiment >>>> >> without patching anything? >>>> >> >>>> >> Thanks, >>>> >> Thomas >>>> >> >>>> >> > On Mon, Jun 17, 2019 at 4:08 PM Thomas Fitzsimmons <fitzsim@fitzsim.org> wrote: >>>> >> >> >>>> >> >> Hi, >>>> >> >> >>>> >> >> Good to hear that the patch I posted worked! >>>> >> >> >>>> >> >> Yes, that's the patch that Andreas's commit >>>> >> >> 84613dae5c34ea742dd9a3e56f5acb55f604b483 applied. From what I can tell, >>>> >> >> you will not have that in Emacs 26.2. >>>> >> >> >>>> >> >> Can you try reverting my patch and applying Andreas's patch, and see if >>>> >> >> Excorporate still works through the proxy? >>>> >> >> >>>> >> >> Thanks, >>>> >> >> Thomas >>>> >> >> >>>> >> >> "tenspd137 ." <dcday137@gmail.com> writes: >>>> >> >> >>>> >> >> > The patch Thomas seems to work from behind the proxy. My current >>>> >> >> > emacs version is 26.2, so I would think it would include the commit >>>> >> >> > Andreas is talking about.... I went and looked it up - is this the >>>> >> >> > correct commit? >>>> >> >> > >>>> >> >> > diff --git a/lisp/url/url-http.el b/lisp/url/url-http.el >>>> >> >> > index 53798f7..817c5ce 100644 >>>> >> >> > --- a/lisp/url/url-http.el >>>> >> >> > +++ b/lisp/url/url-http.el >>>> >> >> > @@ -1412,7 +1412,9 @@ The return value of this function is the >>>> >> >> > retrieval buffer." >>>> >> >> > 'url-http-wait-for-headers-change-function) >>>> >> >> > (set-process-filter tls-connection 'url-http-generic-filter) >>>> >> >> > (process-send-string tls-connection >>>> >> >> > - (url-http-create-request))) >>>> >> >> > + ;; Use the non-proxy form of the request >>>> >> >> > + (let (url-http-proxy) >>>> >> >> > + (url-http-create-request)))) >>>> >> >> > (gnutls-error >>>> >> >> > (url-http-activate-callback) >>>> >> >> > (error "gnutls-error: %s" e)) >>>> >> >> > >>>> >> >> > Thanks! >>>> >> >> > >>>> >> >> > -C >>>> >> >> > >>>> >> >> > >>>> >> >> > On Sat, Jun 15, 2019 at 1:41 AM Andreas Schwab <schwab@linux-m68k.org> wrote: >>>> >> >> >> >>>> >> >> >> On Jun 14 2019, Thomas Fitzsimmons <fitzsim@fitzsim.org> wrote: >>>> >> >> >> >>>> >> >> >> > diff --git a/lisp/url/url-http.el b/lisp/url/url-http.el >>>> >> >> >> > index 00803a103a..723d111d58 100644 >>>> >> >> >> > --- a/lisp/url/url-http.el >>>> >> >> >> > +++ b/lisp/url/url-http.el >>>> >> >> >> > @@ -329,7 +329,10 @@ url-http-create-request >>>> >> >> >> > ;; The request >>>> >> >> >> > (or url-http-method "GET") " " >>>> >> >> >> > (url-http--encode-string >>>> >> >> >> > - (if using-proxy (url-recreate-url url-http-target-url) real-fname)) >>>> >> >> >> > + (if (and using-proxy >>>> >> >> >> > + (not (equal "https" (url-type url-http-target-url)))) >>>> >> >> >> > + (url-recreate-url url-http-target-url) >>>> >> >> >> > + real-fname)) >>>> >> >> >> >>>> >> >> >> That should already be handled by commit 84613dae5c. >>>> >> >> >> >>>> >> >> >> Andreas. >>>> >> >> >> >>>> >> >> >> -- >>>> >> >> >> Andreas Schwab, schwab@linux-m68k.org >>>> >> >> >> GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510 2552 DF73 E780 A9DA AEC1 >>>> >> >> >> "And now for something completely different." ^ permalink raw reply [flat|nested] 42+ messages in thread
* bug#35969: 26.2, Excorporate 2019-07-10 0:11 ` Thomas Fitzsimmons @ 2019-07-10 15:35 ` Collin Day 0 siblings, 0 replies; 42+ messages in thread From: Collin Day @ 2019-07-10 15:35 UTC (permalink / raw) To: Thomas Fitzsimmons; +Cc: Andreas Schwab, 35969 When I add your patch, there is still an error: Contacting host: outlook.office365.com:443 error in process filter: exco--parse-xml-in-current-buffer: Server response is not an XML document error in process filter: Server response is not an XML document * http outlook,office365.com:443* HTTP/1.1 401 Unauthorized Server: Microsoft-IIS/10.0 request-id: 33a1707c-bed3-4edd-98c6-bd1443827acf X-WSSecurity-Enabled: True X-WSSecurity-For: Logon X-FederationTrustTokenIssuerUri: urn:federation:MicrosoftOnline X-WSSecurity-SymmetricKey-Enabled: True X-WSSecurity-X509Cert-Enabled: True X-OAuth-Enabled: True X-Powered-By: ASP.NET X-FEServer: SN4PR0601CA0013 WWW-Authenticate: Basic Realm="" Date: Wed, 10 Jul 2019 15:27:49 GMT Content-Length: 0 URL-DEBUG http -> Contacting host: outlook.office365.com:443 http -> Marking connection as busy: outlook.office365.com:443 #<process outlook.office365.com> http -> getting referer from buffer: buffer:#<buffer *scratch*> target-url:#s(url "https" nil nil "outlook.office365.com" nil "/EWS/Services.wsdl" nil nil t nil t t) lastloc:nil http -> Calling after change function `url-https-proxy-after-change-function' for `#<process outlook.office365.com>' http -> url-http-parse-response called in ( *http outlook.office365.com:443*) http -> Request is: GET /EWS/Services.wsdl HTTP/1.1 MIME-Version: 1.0 Connection: keep-alive Extension: Security/Digest Security/SSL Host: outlook.office365.com Accept-encoding: gzip Accept: */* User-Agent: URL/Emacs Emacs/27.0.50 (X11; x86_64-pc-linux-gnu) http -> Calling after change function `url-http-wait-for-headers-change-function' for `#<process outlook.office365.com>' http -> url-http-wait-for-headers-change-function ( *http outlook.office365.com:443*) http -> Saw end of headers... ( *http outlook.office365.com:443*) http -> url-http-parse-response called in ( *http outlook.office365.com:443*) http -> Got a content-length, being smart about document end. http -> Got 0-length content-length, activating callback immediately. http -> Marking connection as free: outlook.office365.com:443 #<process outlook.office365.com> http -> url-http-parse-headers called in ( *http outlook.office365.com:443*) http -> url-http-parse-response called in ( *http outlook.office365.com:443*) http -> Parsed HTTP headers: class=4 status=401 http -> Handling normal authentication http -> Found existing connection: outlook.office365.com:443 #<process outlook.office365.com> http -> Reusing existing connection: outlook.office365.com:443 http -> Marking connection as busy: outlook.office365.com:443 #<process outlook.office365.com> http -> getting referer from buffer: buffer:#<buffer *http outlook.office365.com:443*> target-url:#s(url "https" nil nil "outlook.office365.com" nil "/EWS/Services.wsdl" nil nil t nil t t) lastloc:nil http -> Finished parsing HTTP headers: nil http -> Spinning waiting for headers... http -> Calling after change function `url-https-proxy-after-change-function' for `#<process outlook.office365.com>' http -> url-http-parse-response called in ( *http outlook.office365.com:443*-293116) http -> error response: 400 http -> Marking connection as free: outlook.office365.com:443 #<process outlook.office365.com> http -> Activating callback in buffer ( *http outlook.office365.com:443*-293116): #[257 "p\303\304\305\306\307 !\310\"\311$\216 @\312=\203$\0\313\301\314\315\316\302\"#\210\317\300\320\"\202/\0\313\301\321\322 #\210\317\300\323\")\207" [fsm-exco--fsm-0 (:identifier ("my email" . "https://outlook.office365.com/EWS/Exchange.asmx") :mail-address "my email" :retrying nil :autodiscovery-urls nil :service-url "https://outlook.office365.com/EWS/Exchange.asmx" :service-xml nil :service-wsdl nil :next-state-after-success nil :failure-message nil :server-version nil) "https://outlook.office365.com/EWS/Services.wsdl" make-byte-code 0 "\301\300!\205 \0\302\300!\207" vconcat vector [buffer-live-p kill-buffer] 2 :error plist-put :failure-message format "Failed to retrieve %s" fsm-send :unrecoverable-error :service-xml exco--parse-xml-in-current-buffer :success] 8 " (fn STATUS)"] ((:peer (:certificates ((:version 3 :serial-number "0b:be:0b:85:09:74:2b:7b:f8:7f:80:dc:14:67:6d:54" :issuer "C=US,O=DigiCert Inc,CN=DigiCert Cloud Services CA-1" :valid-from "2018-11-19" :valid-to "2020-11-19" :subject "C=US,ST=Washington,L=Redmond,O=Microsoft Corporation,CN=outlook.com" :public-key-algorithm "RSA" :certificate-security-level "Medium" :signature-algorithm "RSA-SHA256" :public-key-id "sha1:35:1a:fe:e2:6c:d9:35:08:3c:95:a2:c1:12:a1:2a:8e:7f:87:fa:07" :certificate-id "sha1:9a:44:16:e2:84:8e:3d:26:86:f0:23:a2:ff:f2:d0:24:28:2f:18:8f") (:version 3 :serial-number "01:9e:c1:c6:bd:3f:59:7b:b2:0c:33:38:e5:51:d8:77" :issuer "C=US,O=DigiCert Inc,OU=www.digicert.com,CN=DigiCert Global Root CA" :valid-from "2015-08-04" :valid-to "2030-08-04" :subject "C=US,O=DigiCert Inc,CN=DigiCert Cloud Services CA-1" :public-key-algorithm "RSA" :certificate-security-level "Medium" :signature-algorithm "RSA-SHA256" :public-key-id "sha1:ac:13:d5:79:03:66:f3:cc:ff:fe:7c:23:71:a7:61:5d:39:20:89:6e" :certificate-id "sha1:81:b6:8d:6c:d2:f2:21:f8:f5:34:e6:77:52:3b:b2:36:bb:a1:dc:56")) :certificate (:version 3 :serial-number "0b:be:0b:85:09:74:2b:7b:f8:7f:80:dc:14:67:6d:54" :issuer "C=US,O=DigiCert Inc,CN=DigiCert Cloud Services CA-1" :valid-from "2018-11-19" :valid-to "2020-11-19" :subject "C=US,ST=Washington,L=Redmond,O=Microsoft Corporation,CN=outlook.com" :public-key-algorithm "RSA" :certificate-security-level "Medium" :signature-algorithm "RSA-SHA256" :public-key-id "sha1:35:1a:fe:e2:6c:d9:35:08:3c:95:a2:c1:12:a1:2a:8e:7f:87:fa:07" :certificate-id "sha1:9a:44:16:e2:84:8e:3d:26:86:f0:23:a2:ff:f2:d0:24:28:2f:18:8f") :key-exchange "ECDHE-RSA" :protocol "TLS1.2" :cipher "AES-256-GCM" :mac "AEAD"))) url-http.el with patch from describe function, just to make sure patch is applied: ... (setq request (concat ;; The request (or url-http-method "GET") " " (url-http--encode-string ;; (if using-proxy (url-recreate-url url-http-target-url) real-fname)) (if (and using-proxy (not (equal "https" (url-type url-http-target-url)))) (url-recreate-url url-http-target-url) real-fname)) " HTTP/" url-http-version "\r\n" ... output from make: make[2]: Entering directory '/home/dayd/projects/emacs/lisp' ELC url/url-http.elc Sorry if the extra info is extraneous, just trying to grab as much as I can for you. I think there is something else wrong at this point though - like I mentioned it seems strange I had to run package-install 4 times to get excorporate to install in the first place. Thanks! -C On Tue, Jul 9, 2019 at 6:11 PM Thomas Fitzsimmons <fitzsim@fitzsim.org> wrote: > > I think there should be no need to remove test-home, so you won't have > to re-run the M-x package-install steps. > > Thomas > > Thomas Fitzsimmons <fitzsim@fitzsim.org> writes: > > > OK, can you apply my patch, then run "make" again and confirm it works? > > > > Thanks, > > Thomas > > > > Collin Day <dcday137@gmail.com> writes: > > > >> Did a git pull and followed instructions above. > >> > >> There are some issues. First, I need to run M-x package-install <RET> > >> excorporate <RET> four times because I see (each line after each > >> invocation) > >> > >> package--with-response-buffer-1: > >> https://elpa.gnu.org/packages/url-http-ntlm-2.0.4.el: Method not > >> allowed > >> package--with-response-buffer-1: > >> https://elpa.gnu.org/packages/fsm-0.2.1.el: Method not allowed > >> package--with-response-buffer-1: > >> https://elpa.gnu.org/packages/excorporate-0.8.3.tar: Method not > >> allowed > >> > >> After the 4th call it compiles and becomes available. > >> > >> M-x customize-group <RET> excorporate, set up for no autoconfig right > >> now, hit apply for current sessions, save not availiable.... > >> > >> M-x excorporate <RET> > >> > >> enter uname and password > >> > >> Contacting host: outlook.office365.com:443 > >> error in process filter: exco--parse-xml-in-current-buffer: Server > >> response is not an XML document > >> error in process filter: Server response is not an XML documen > >> > >> *http outlook.office.365.com:443* > > > > [...] > > > >> So it appears not to work, at least on the current head as of today.... > >> > >> Thanks! > >> > >> -C > >> > >> On Tue, Jul 9, 2019 at 4:09 PM Collin Day <dcday137@gmail.com> wrote: > >>> > >>> Sorry, I have not had the chance to. A lot has been going on at my > >>> place of employment. It has crossed my mind, and as soon as I have > >>> a chance, I will try what you suggested above. Thanks, sorry for > >>> the inconvenience. > >>> > >>> On Tue, Jul 9, 2019, 3:53 PM Thomas Fitzsimmons <fitzsim@fitzsim.org> wrote: > >>>> > >>>> Hi, > >>>> > >>>> Were you able to complete building Emacs? I'd like to know if my patch > >>>> is needed on top of Andreas's to make Excorporate work through your > >>>> proxy. > >>>> > >>>> I'm not sure why the built version would interfere with the current > >>>> installation. If you were doing "make install" to a common prefix, that > >>>> might explain it. Instead you can try something like this: > >>>> > >>>> cd emacs-master [your github.com checkout] > >>>> make > >>>> mkdir test-home > >>>> HOME=`pwd`/test-home ./src/emacs -Q > >>>> > >>>> That will ensure you're only running the built Emacs, and completely > >>>> ignoring the packages installed in your home directory. > >>>> > >>>> Thomas > >>>> > >>>> "tenspd137 ." <dcday137@gmail.com> writes: > >>>> > >>>> > I haven't tried the first item on the list yet - ie reloading Andreas' > >>>> > function. i have tried to build emacs from > >>>> > https://github.com/emacs-mirror/emacs.git because I can't reach the > >>>> > official repo from my firewall for whatever reason. I am sure it is > >>>> > something here. Anyway, I was having problems because my built > >>>> > version (27.0.x) was getting seemingly mixed up with my current > >>>> > installation. I can try again later, but right now things are pretty > >>>> > busy at work. > >>>> > > >>>> > Thanks. > >>>> > > >>>> > On Tue, Jun 18, 2019 at 10:26 PM Thomas Fitzsimmons <fitzsim@fitzsim.org> wrote: > >>>> >> > >>>> >> Hi, > >>>> >> > >>>> >> "tenspd137 ." <dcday137@gmail.com> writes: > >>>> >> > >>>> >> > So - I am not sure if I did it correctly, but I copied this function > >>>> >> > with Andreas' changes into a file: > >>>> >> > > >>>> >> > (defun url-https-proxy-after-change-function (_st _nd _length) > >>>> >> > (let* ((process-buffer (current-buffer)) > >>>> >> > (proc (get-buffer-process process-buffer))) > >>>> >> > (goto-char (point-min)) > >>>> >> > (when (re-search-forward "^\r?\n" nil t) > >>>> >> > (backward-char 1) > >>>> >> > ;; Saw the end of the headers > >>>> >> > (setq url-http-end-of-headers (set-marker (make-marker) (point))) > >>>> >> > (url-http-parse-response) > >>>> >> > (cond > >>>> >> > ((null url-http-response-status) > >>>> >> > ;; We got back a headerless malformed response from the > >>>> >> > ;; server. > >>>> >> > (url-http-activate-callback) > >>>> >> > (error "Malformed response from proxy, fail!")) > >>>> >> > ((= url-http-response-status 200) > >>>> >> > (if (gnutls-available-p) > >>>> >> > (condition-case e > >>>> >> > (let ((tls-connection (gnutls-negotiate > >>>> >> > :process proc > >>>> >> > :hostname (url-host url-current-object) > >>>> >> > :verify-error nil))) > >>>> >> > ;; check certificate validity > >>>> >> > (setq tls-connection > >>>> >> > (nsm-verify-connection tls-connection > >>>> >> > (url-host url-current-object) > >>>> >> > (url-port url-current-object))) > >>>> >> > (with-current-buffer process-buffer (erase-buffer)) > >>>> >> > (set-process-buffer tls-connection process-buffer) > >>>> >> > (setq url-http-after-change-function > >>>> >> > 'url-http-wait-for-headers-change-function) > >>>> >> > (set-process-filter tls-connection 'url-http-generic-filter) > >>>> >> > (process-send-string tls-connection > >>>> >> > ;; Use the non-proxy form of the request > >>>> >> > (let (url-http-proxy) > >>>> >> > (url-http-create-request)))) > >>>> >> > (gnutls-error > >>>> >> > (url-http-activate-callback) > >>>> >> > (error "gnutls-error: %s" e)) > >>>> >> > (error > >>>> >> > (url-http-activate-callback) > >>>> >> > (error "error: %s" e))) > >>>> >> > (error "error: gnutls support needed!"))) > >>>> >> > (t > >>>> >> > (url-http-debug "error response: %d" url-http-response-status) > >>>> >> > (url-http-activate-callback)))))) > >>>> >> > > >>>> >> > and then loaded it before running excorporate. After that, I did M-x > >>>> >> > excorporate, and the minibuffer returns: error in process filter: > >>>> >> > Server response is not an XML document > >>>> >> > >>>> >> In this scenario, if you immediately (without restarting Emacs/reloading > >>>> >> anything) re-run M-x excorporate does it still fail? I just want to > >>>> >> make sure that's not a transient failure. If it does fail the second > >>>> >> time, can you post the HTTP response from the server? > >>>> >> > >>>> >> > When I do the similar test by loading the url-http-create-request with > >>>> >> > Thomas's changes, I can get a connection and grab my schedule > >>>> >> > through the proxy. > >>>> >> > >>>> >> OK. > >>>> >> > >>>> >> > Let me know if I need to try something different. > >>>> >> > >>>> >> Are you in a position to build Emacs master tip and retry the experiment > >>>> >> without patching anything? > >>>> >> > >>>> >> Thanks, > >>>> >> Thomas > >>>> >> > >>>> >> > On Mon, Jun 17, 2019 at 4:08 PM Thomas Fitzsimmons <fitzsim@fitzsim.org> wrote: > >>>> >> >> > >>>> >> >> Hi, > >>>> >> >> > >>>> >> >> Good to hear that the patch I posted worked! > >>>> >> >> > >>>> >> >> Yes, that's the patch that Andreas's commit > >>>> >> >> 84613dae5c34ea742dd9a3e56f5acb55f604b483 applied. From what I can tell, > >>>> >> >> you will not have that in Emacs 26.2. > >>>> >> >> > >>>> >> >> Can you try reverting my patch and applying Andreas's patch, and see if > >>>> >> >> Excorporate still works through the proxy? > >>>> >> >> > >>>> >> >> Thanks, > >>>> >> >> Thomas > >>>> >> >> > >>>> >> >> "tenspd137 ." <dcday137@gmail.com> writes: > >>>> >> >> > >>>> >> >> > The patch Thomas seems to work from behind the proxy. My current > >>>> >> >> > emacs version is 26.2, so I would think it would include the commit > >>>> >> >> > Andreas is talking about.... I went and looked it up - is this the > >>>> >> >> > correct commit? > >>>> >> >> > > >>>> >> >> > diff --git a/lisp/url/url-http.el b/lisp/url/url-http.el > >>>> >> >> > index 53798f7..817c5ce 100644 > >>>> >> >> > --- a/lisp/url/url-http.el > >>>> >> >> > +++ b/lisp/url/url-http.el > >>>> >> >> > @@ -1412,7 +1412,9 @@ The return value of this function is the > >>>> >> >> > retrieval buffer." > >>>> >> >> > 'url-http-wait-for-headers-change-function) > >>>> >> >> > (set-process-filter tls-connection 'url-http-generic-filter) > >>>> >> >> > (process-send-string tls-connection > >>>> >> >> > - (url-http-create-request))) > >>>> >> >> > + ;; Use the non-proxy form of the request > >>>> >> >> > + (let (url-http-proxy) > >>>> >> >> > + (url-http-create-request)))) > >>>> >> >> > (gnutls-error > >>>> >> >> > (url-http-activate-callback) > >>>> >> >> > (error "gnutls-error: %s" e)) > >>>> >> >> > > >>>> >> >> > Thanks! > >>>> >> >> > > >>>> >> >> > -C > >>>> >> >> > > >>>> >> >> > > >>>> >> >> > On Sat, Jun 15, 2019 at 1:41 AM Andreas Schwab <schwab@linux-m68k.org> wrote: > >>>> >> >> >> > >>>> >> >> >> On Jun 14 2019, Thomas Fitzsimmons <fitzsim@fitzsim.org> wrote: > >>>> >> >> >> > >>>> >> >> >> > diff --git a/lisp/url/url-http.el b/lisp/url/url-http.el > >>>> >> >> >> > index 00803a103a..723d111d58 100644 > >>>> >> >> >> > --- a/lisp/url/url-http.el > >>>> >> >> >> > +++ b/lisp/url/url-http.el > >>>> >> >> >> > @@ -329,7 +329,10 @@ url-http-create-request > >>>> >> >> >> > ;; The request > >>>> >> >> >> > (or url-http-method "GET") " " > >>>> >> >> >> > (url-http--encode-string > >>>> >> >> >> > - (if using-proxy (url-recreate-url url-http-target-url) real-fname)) > >>>> >> >> >> > + (if (and using-proxy > >>>> >> >> >> > + (not (equal "https" (url-type url-http-target-url)))) > >>>> >> >> >> > + (url-recreate-url url-http-target-url) > >>>> >> >> >> > + real-fname)) > >>>> >> >> >> > >>>> >> >> >> That should already be handled by commit 84613dae5c. > >>>> >> >> >> > >>>> >> >> >> Andreas. > >>>> >> >> >> > >>>> >> >> >> -- > >>>> >> >> >> Andreas Schwab, schwab@linux-m68k.org > >>>> >> >> >> GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510 2552 DF73 E780 A9DA AEC1 > >>>> >> >> >> "And now for something completely different." ^ permalink raw reply [flat|nested] 42+ messages in thread
* bug#35969: 26.2, Excorporate 2019-07-09 22:53 ` Collin Day 2019-07-10 0:08 ` Thomas Fitzsimmons @ 2019-07-13 14:37 ` Thomas Fitzsimmons 1 sibling, 0 replies; 42+ messages in thread From: Thomas Fitzsimmons @ 2019-07-13 14:37 UTC (permalink / raw) To: Collin Day; +Cc: Andreas Schwab, 35969 Hi Collin, Collin Day <dcday137@gmail.com> writes: > Did a git pull and followed instructions above. > > There are some issues. First, I need to run M-x package-install <RET> > excorporate <RET> four times because I see (each line after each > invocation) > > package--with-response-buffer-1: > https://elpa.gnu.org/packages/url-http-ntlm-2.0.4.el: Method not > allowed > package--with-response-buffer-1: > https://elpa.gnu.org/packages/fsm-0.2.1.el: Method not allowed > package--with-response-buffer-1: > https://elpa.gnu.org/packages/excorporate-0.8.3.tar: Method not > allowed > > After the 4th call it compiles and becomes available. Thanks for persisting with this. Did you configure the proxy settings in Emacs before attempting package-install? If so, try this in your newly-built Emacs, emacs -Q, with HOME=`pwd`/test-home: (progn (setq url-debug t) (url-retrieve-synchronously "https://httpbin.org/basic-auth/user/passwd") (dolist (p (seq-filter (lambda (b) (string-match " *http*" (buffer-name b))) (buffer-list))) (message "HTTP result buffer: \"%s\"\n%s" (buffer-name p) (with-current-buffer p (buffer-string)))) "check *Messages*") Username is user, password is passwd. That will establish whether even basic authentication over HTTPS is working through the proxy with your new Emacs build. Thomas ^ permalink raw reply [flat|nested] 42+ messages in thread
* bug#35969: 26.2, Excorporate 2019-06-15 7:41 ` Andreas Schwab 2019-06-17 16:31 ` tenspd137 . @ 2019-07-31 21:07 ` Thomas Fitzsimmons 2019-07-31 21:20 ` Andreas Schwab 1 sibling, 1 reply; 42+ messages in thread From: Thomas Fitzsimmons @ 2019-07-31 21:07 UTC (permalink / raw) To: Andreas Schwab; +Cc: Collin Day, 35969 Hi, I found a proxy server to test against. I've now replicated Collin's findings. Andreas Schwab <schwab@linux-m68k.org> writes: > On Jun 14 2019, Thomas Fitzsimmons <fitzsim@fitzsim.org> wrote: > >> diff --git a/lisp/url/url-http.el b/lisp/url/url-http.el >> index 00803a103a..723d111d58 100644 >> --- a/lisp/url/url-http.el >> +++ b/lisp/url/url-http.el >> @@ -329,7 +329,10 @@ url-http-create-request >> ;; The request >> (or url-http-method "GET") " " >> (url-http--encode-string >> - (if using-proxy (url-recreate-url url-http-target-url) real-fname)) >> + (if (and using-proxy >> + (not (equal "https" (url-type url-http-target-url)))) >> + (url-recreate-url url-http-target-url) >> + real-fname)) For discussion purposes, let's call the above "patch T"... > That should already be handled by commit 84613dae5c. ... and this commit "patch A", which is: diff --git a/lisp/url/url-http.el b/lisp/url/url-http.el index 53798f77c3..817c5ce3b3 100644 --- a/lisp/url/url-http.el +++ b/lisp/url/url-http.el @@ -1412,7 +1412,9 @@ url-https-proxy-after-change-function 'url-http-wait-for-headers-change-function) (set-process-filter tls-connection 'url-http-generic-filter) (process-send-string tls-connection - (url-http-create-request))) + ;; Use the non-proxy form of the request + (let (url-http-proxy) + (url-http-create-request)))) (gnutls-error (url-http-activate-callback) (error "gnutls-error: %s" e)) I tried on Emacs 26.2 and master tip, and in both cases, Excorporate worked with patch T applied and patch A not present (26.2) or reverted (master); it failed with any other combination of the patches (A and T, A only, neither A nor T). One difference I noticed is that with A applied, the Connection header is set to keep-alive and the connection is reused, whereas with just T, Connection is set to close and the connection is re-established. The attached patch fixes it. Andreas, do you have a test case that patch A fixed, and if so, can you retest with the proposed fix? Thanks, Thomas diff --git a/lisp/url/url-http.el b/lisp/url/url-http.el index 838f0a30c1..eb054cd65a 100644 --- a/lisp/url/url-http.el +++ b/lisp/url/url-http.el @@ -1436,7 +1436,8 @@ url-https-proxy-after-change-function (set-process-filter tls-connection 'url-http-generic-filter) (process-send-string tls-connection ;; Use the non-proxy form of the request - (let (url-http-proxy) + (let (url-http-proxy + url-http-attempt-keepalives) (url-http-create-request)))) (gnutls-error (url-http-activate-callback) ^ permalink raw reply related [flat|nested] 42+ messages in thread
* bug#35969: 26.2, Excorporate 2019-07-31 21:07 ` Thomas Fitzsimmons @ 2019-07-31 21:20 ` Andreas Schwab 2019-07-31 22:28 ` Collin Day 2019-08-01 1:58 ` Thomas Fitzsimmons 0 siblings, 2 replies; 42+ messages in thread From: Andreas Schwab @ 2019-07-31 21:20 UTC (permalink / raw) To: Thomas Fitzsimmons; +Cc: Collin Day, 35969 On Jul 31 2019, Thomas Fitzsimmons <fitzsim@fitzsim.org> wrote: > Andreas, do you have a test case that patch A fixed, and if so, can > you retest with the proposed fix? Any https request over proxy will do. Andreas. -- Andreas Schwab, schwab@linux-m68k.org GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510 2552 DF73 E780 A9DA AEC1 "And now for something completely different." ^ permalink raw reply [flat|nested] 42+ messages in thread
* bug#35969: 26.2, Excorporate 2019-07-31 21:20 ` Andreas Schwab @ 2019-07-31 22:28 ` Collin Day 2019-08-01 1:58 ` Thomas Fitzsimmons 1 sibling, 0 replies; 42+ messages in thread From: Collin Day @ 2019-07-31 22:28 UTC (permalink / raw) To: Andreas Schwab; +Cc: Thomas Fitzsimmons, 35969 [-- Attachment #1: Type: text/plain, Size: 595 bytes --] Sorry I have been unresponsive. I am in the middle of changing jobs.... good news that you can recreate it. C On Wed, Jul 31, 2019, 3:20 PM Andreas Schwab <schwab@linux-m68k.org> wrote: > On Jul 31 2019, Thomas Fitzsimmons <fitzsim@fitzsim.org> wrote: > > > Andreas, do you have a test case that patch A fixed, and if so, can > > you retest with the proposed fix? > > Any https request over proxy will do. > > Andreas. > > -- > Andreas Schwab, schwab@linux-m68k.org > GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510 2552 DF73 E780 A9DA AEC1 > "And now for something completely different." > [-- Attachment #2: Type: text/html, Size: 1122 bytes --] ^ permalink raw reply [flat|nested] 42+ messages in thread
* bug#35969: 26.2, Excorporate 2019-07-31 21:20 ` Andreas Schwab 2019-07-31 22:28 ` Collin Day @ 2019-08-01 1:58 ` Thomas Fitzsimmons 2019-08-16 3:40 ` Thomas Fitzsimmons 1 sibling, 1 reply; 42+ messages in thread From: Thomas Fitzsimmons @ 2019-08-01 1:58 UTC (permalink / raw) To: Andreas Schwab; +Cc: Collin Day, 35969 Andreas Schwab <schwab@linux-m68k.org> writes: > On Jul 31 2019, Thomas Fitzsimmons <fitzsim@fitzsim.org> wrote: > >> Andreas, do you have a test case that patch A fixed, and if so, can >> you retest with the proposed fix? > > Any https request over proxy will do. OK, I tested the attached patch; it works for a request to an uncached HTTPS website, and it works for Excorporate accessing an HTTPS server. I moved the URL form logic to url-http-create-request because that function refers to using-proxy (set to url-http-proxy on entry) in multiple places, not just during URL recreation. When url-https-proxy-after-change-function was setting using-proxy to nil for the entire duration of the url-http-create-request, it was interfering with "Connection" handling later in the function: "Connection: " (if (or using-proxy (not url-http-attempt-keepalives)) "close" "keep-alive") Does this look OK to push to master? Thanks, Thomas diff --git a/lisp/url/url-http.el b/lisp/url/url-http.el index 838f0a30c1..354cc56de4 100644 --- a/lisp/url/url-http.el +++ b/lisp/url/url-http.el @@ -329,7 +329,9 @@ url-http-create-request ;; The request (or url-http-method "GET") " " (url-http--encode-string - (if using-proxy (url-recreate-url url-http-target-url) real-fname)) + (if (and using-proxy + (not (equal "https" (url-type url-http-target-url)))) + (url-recreate-url url-http-target-url) real-fname)) " HTTP/" url-http-version "\r\n" ;; Version of MIME we speak "MIME-Version: 1.0\r\n" @@ -1435,9 +1437,7 @@ url-https-proxy-after-change-function 'url-http-wait-for-headers-change-function) (set-process-filter tls-connection 'url-http-generic-filter) (process-send-string tls-connection - ;; Use the non-proxy form of the request - (let (url-http-proxy) - (url-http-create-request)))) + (url-http-create-request))) (gnutls-error (url-http-activate-callback) (error "gnutls-error: %s" e)) ^ permalink raw reply related [flat|nested] 42+ messages in thread
* bug#35969: 26.2, Excorporate 2019-08-01 1:58 ` Thomas Fitzsimmons @ 2019-08-16 3:40 ` Thomas Fitzsimmons 0 siblings, 0 replies; 42+ messages in thread From: Thomas Fitzsimmons @ 2019-08-16 3:40 UTC (permalink / raw) To: Andreas Schwab; +Cc: Collin Day, 35969-done Thomas Fitzsimmons <fitzsim@fitzsim.org> writes: > Andreas Schwab <schwab@linux-m68k.org> writes: > >> On Jul 31 2019, Thomas Fitzsimmons <fitzsim@fitzsim.org> wrote: >> >>> Andreas, do you have a test case that patch A fixed, and if so, can >>> you retest with the proposed fix? >> >> Any https request over proxy will do. > > OK, I tested the attached patch; it works for a request to an uncached > HTTPS website, and it works for Excorporate accessing an HTTPS server. > > I moved the URL form logic to url-http-create-request because that > function refers to using-proxy (set to url-http-proxy on entry) in > multiple places, not just during URL recreation. When > url-https-proxy-after-change-function was setting using-proxy to nil for > the entire duration of the url-http-create-request, it was interfering > with "Connection" handling later in the function: > > "Connection: " (if (or using-proxy > (not url-http-attempt-keepalives)) > "close" "keep-alive") > > Does this look OK to push to master? I didn't hear any objections, so I pushed the change. Thanks, Thomas ^ permalink raw reply [flat|nested] 42+ messages in thread
end of thread, other threads:[~2019-08-16 3:40 UTC | newest]
Thread overview: 42+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-05-28 21:25 bug#35969: 26.2, Excorporate tenspd137 .
2019-05-29 1:04 ` Thomas Fitzsimmons
2019-05-29 16:17 ` tenspd137 .
2019-05-29 17:02 ` Thomas Fitzsimmons
2019-05-29 18:13 ` tenspd137 .
2019-05-29 20:53 ` Thomas Fitzsimmons
2019-05-29 21:44 ` tenspd137 .
2019-05-29 22:01 ` Thomas Fitzsimmons
2019-05-29 22:10 ` tenspd137 .
2019-05-30 17:06 ` tenspd137 .
2019-05-30 17:25 ` tenspd137 .
[not found] ` <CAB=m8wL0JNK+dQv68nddHuJ8yvHdsV9ehTaGJ00NK4spe1sQAQ@mail.gmail.com>
[not found] ` <m3blze4872.fsf@fitzsim.org>
[not found] ` <CAB=m8wJfGveEzVm46j2MLGFupdv2Lwn3VwL02_RAdirzDJKcSg@mail.gmail.com>
[not found] ` <CAB=m8wKv0dWgRctuc+bWyocNOPFertCt9K4SOB2Q_W5cPU_tTw@mail.gmail.com>
[not found] ` <m3tvd5p8zt.fsf@fitzsim.org>
[not found] ` <CAB=m8wLKtY_38m3mJ18_qhacRUNVCU-4GEFMAKXAdbc1Xamnxg@mail.gmail.com>
[not found] ` <m3d0jtp0c7.fsf@fitzsim.org>
[not found] ` <CAB=m8wL6KqATgAddiC3H7kzLSnbe5xTeAWrq5eqvr6n6aU-OKw@mail.gmail.com>
[not found] ` <m34l55oywz.fsf@fitzsim.org>
[not found] ` <CAB=m8wKHstZNzTZxj4m-UuOg9LNeu4LRrhysOaaQrSd6z1DE0A@mail.gmail.com>
[not found] ` <m3woi1nfxw.fsf@fitzsim.org>
2019-06-14 20:13 ` tenspd137 .
2019-06-14 20:16 ` tenspd137 .
2019-06-14 20:22 ` Noam Postavsky
2019-06-14 20:32 ` tenspd137 .
2019-06-14 20:45 ` tenspd137 .
2019-06-14 21:02 ` Thomas Fitzsimmons
2019-06-14 21:48 ` tenspd137 .
2019-06-14 22:07 ` tenspd137 .
2019-06-14 23:07 ` tenspd137 .
2019-06-14 23:14 ` tenspd137 .
2019-06-15 0:14 ` Thomas Fitzsimmons
2019-06-15 0:47 ` tenspd137 .
2019-06-15 0:36 ` Thomas Fitzsimmons
2019-06-15 7:41 ` Andreas Schwab
2019-06-17 16:31 ` tenspd137 .
2019-06-17 22:08 ` Thomas Fitzsimmons
2019-06-18 16:34 ` tenspd137 .
2019-06-19 4:26 ` Thomas Fitzsimmons
2019-06-20 20:34 ` tenspd137 .
2019-07-09 21:52 ` Thomas Fitzsimmons
2019-07-09 22:09 ` Collin Day
2019-07-09 22:53 ` Collin Day
2019-07-10 0:08 ` Thomas Fitzsimmons
2019-07-10 0:11 ` Thomas Fitzsimmons
2019-07-10 15:35 ` Collin Day
2019-07-13 14:37 ` Thomas Fitzsimmons
2019-07-31 21:07 ` Thomas Fitzsimmons
2019-07-31 21:20 ` Andreas Schwab
2019-07-31 22:28 ` Collin Day
2019-08-01 1:58 ` Thomas Fitzsimmons
2019-08-16 3:40 ` Thomas Fitzsimmons
Code repositories for project(s) associated with this public inbox https://git.savannah.gnu.org/cgit/emacs.git This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).