bug#50113: Excorporate: Communicating with domain that requires SSO?

bug#50113: Excorporate: Communicating with domain that requires SSO?

[Justin Abrahms]

[-- Attachment #1: Type: text/plain, Size: 674 bytes --]

Hi. 

I'm trying to setup excorporate for my new job. In looking at the outlook client that came pre-installed on my PC, I see that our EWS url uses outlook365. This is the error I see in my *Messages* buffer:

Contacting host: outlook.office365.com:443
error in process filter: exco--parse-xml-in-current-buffer: Server response is not an XML document
error in process filter: Server response is not an XML document

When looking at *URL-DEBUG*, I see that it's being redirected to my company's SSO endpoint. In looking around the docs & internet, I'm not seeing how others deal with needing to do SSO, so I'm reaching out here in hopes someone has ideas.

Thanks,
 -justin

[-- Attachment #2: Type: text/html, Size: 968 bytes --]

bug#50113: Excorporate: Communicating with domain that requires SSO?

[Thomas Fitzsimmons]

Hi Justin,

"Justin Abrahms" <justin@abrah.ms> writes:

> I'm trying to setup excorporate for my new job. In looking at the
> outlook client that came pre-installed on my PC, I see that our EWS
> url uses outlook365. This is the error I see in my *Messages* buffer:
>
> Contacting host: outlook.office365.com:443
> error in process filter: exco--parse-xml-in-current-buffer: Server response is not an XML document
> error in process filter: Server response is not an XML document
>
> When looking at *URL-DEBUG*, I see that it's being redirected to my
> company's SSO endpoint. In looking around the docs & internet, I'm not
> seeing how others deal with needing to do SSO, so I'm reaching out
> here in hopes someone has ideas.

Thanks for filing this issue.

I haven't had to figure out single-sign-on authentication for
Excorporate yet, but it's good to have a bug report about it so that we
can work on solutions, assuming they exist.

The first thing you can try is to access the EWS URL in Firefox.  Does
browsing to it result in you being transferred to the single-sign-on
page?  And if you authenticate, do you then see a page that says
something like:

"Service

You have created a service.

[...]"?

Thomas

bug#50113: Excorporate: Communicating with domain that requires SSO?

[Justin Abrahms]

[-- Attachment #1: Type: text/plain, Size: 1857 bytes --]

I'm not able to get to the 'you created a service' page.

If I go to:

https://outlook.office365.com/EWS/Exchange.asmx

I'm prompted for a username + password. These are not my actual username + password, as those don't work. I believe if I was able to set up an application-specific password.. I'd be able to use that as the password. Unfortunately, those are disallowed per my administrator. When I abandon the attempted sign-in, I get the butter-y yellow page that I'd expect for an EWS site.

 -justin

On Mon, Aug 23, 2021, at 2:11 PM, Thomas Fitzsimmons wrote:
> Hi Justin,
> 
> "Justin Abrahms" <justin@abrah.ms> writes:
> 
> > I'm trying to setup excorporate for my new job. In looking at the
> > outlook client that came pre-installed on my PC, I see that our EWS
> > url uses outlook365. This is the error I see in my *Messages* buffer:
> >
> > Contacting host: outlook.office365.com:443
> > error in process filter: exco--parse-xml-in-current-buffer: Server response is not an XML document
> > error in process filter: Server response is not an XML document
> >
> > When looking at *URL-DEBUG*, I see that it's being redirected to my
> > company's SSO endpoint. In looking around the docs & internet, I'm not
> > seeing how others deal with needing to do SSO, so I'm reaching out
> > here in hopes someone has ideas.
> 
> Thanks for filing this issue.
> 
> I haven't had to figure out single-sign-on authentication for
> Excorporate yet, but it's good to have a bug report about it so that we
> can work on solutions, assuming they exist.
> 
> The first thing you can try is to access the EWS URL in Firefox.  Does
> browsing to it result in you being transferred to the single-sign-on
> page?  And if you authenticate, do you then see a page that says
> something like:
> 
> "Service
> 
> You have created a service.
> 
> [...]"?
> 
> Thomas
> 

[-- Attachment #2: Type: text/html, Size: 2746 bytes --]

bug#50113: Excorporate: Communicating with domain that requires SSO?

[Thomas Fitzsimmons]

OK, thanks for trying.

Maybe Excorporate could use oauth2.el to authenticate via OAuth2, but
I've never tried that.

As a starting point, are you able to make DavMail retrieve your calendar
items?  If yes, check if it uses, or can be configured to use, EWS
rather than WebDAV.  If you can make DavMail + EWS work somehow, then
there's hope for Excorporate in your environment, with some oauth2.el
integration effort.

Thomas

"Justin Abrahms" <justin@abrah.ms> writes:

> I'm not able to get to the 'you created a service' page.
>
> If I go to:
>
> https://outlook.office365.com/EWS/Exchange.asmx
>
> I'm prompted for a username + password. These are not my actual username + password, as those don't work. I believe if I was able to set up
> an application-specific password.. I'd be able to use that as the password. Unfortunately, those are disallowed per my administrator. When I
> abandon the attempted sign-in, I get the butter-y yellow page that I'd expect for an EWS site.
>
>  -justin
>
> On Mon, Aug 23, 2021, at 2:11 PM, Thomas Fitzsimmons wrote:
>
>  Hi Justin,
>
>  "Justin Abrahms" <justin@abrah.ms> writes:
>
>  > I'm trying to setup excorporate for my new job. In looking at the
>  > outlook client that came pre-installed on my PC, I see that our EWS
>  > url uses outlook365. This is the error I see in my *Messages* buffer:
>  >
>  > Contacting host: outlook.office365.com:443
>  > error in process filter: exco--parse-xml-in-current-buffer: Server response is not an XML document
>  > error in process filter: Server response is not an XML document
>  >
>  > When looking at *URL-DEBUG*, I see that it's being redirected to my
>  > company's SSO endpoint. In looking around the docs & internet, I'm not
>  > seeing how others deal with needing to do SSO, so I'm reaching out
>  > here in hopes someone has ideas.
>
>  Thanks for filing this issue.
>
>  I haven't had to figure out single-sign-on authentication for
>  Excorporate yet, but it's good to have a bug report about it so that we
>  can work on solutions, assuming they exist.
>
>  The first thing you can try is to access the EWS URL in Firefox.  Does
>  browsing to it result in you being transferred to the single-sign-on
>  page?  And if you authenticate, do you then see a page that says
>  something like:
>
>  "Service
>
>  You have created a service.
>
>  [...]"?
>
>  Thomas

bug#50113: Excorporate: Communicating with domain that requires SSO?

[Lars Ingebrigtsen]

Thomas Fitzsimmons <fitzsim@fitzsim.org> writes:

> As a starting point, are you able to make DavMail retrieve your calendar
> items?  If yes, check if it uses, or can be configured to use, EWS
> rather than WebDAV.  If you can make DavMail + EWS work somehow, then
> there's hope for Excorporate in your environment, with some oauth2.el
> integration effort.

(I'm going through old bug reports that unfortunately weren't resolved
at the time.)

This was a year ago.  Skimming this bug report, it's not clear to me
whether there's anything to be done on the Emacs side here?

bug#50113: Excorporate: Communicating with domain that requires SSO?

[Thomas Fitzsimmons]

Hi Lars,

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Thomas Fitzsimmons <fitzsim@fitzsim.org> writes:
>
>> As a starting point, are you able to make DavMail retrieve your calendar
>> items?  If yes, check if it uses, or can be configured to use, EWS
>> rather than WebDAV.  If you can make DavMail + EWS work somehow, then
>> there's hope for Excorporate in your environment, with some oauth2.el
>> integration effort.
>
> (I'm going through old bug reports that unfortunately weren't resolved
> at the time.)
>
> This was a year ago.  Skimming this bug report, it's not clear to me
> whether there's anything to be done on the Emacs side here?

Thank you for following up on this report (amazing progress on nx10%!).
I do see a way forward for implementing this, at least the parts that
Emacs can control, so let's leave this open.

Thanks,
Thomas

bug#50113: Excorporate: Communicating with domain that requires SSO?

[Lars Ingebrigtsen]

Thomas Fitzsimmons <fitzsim@fitzsim.org> writes:

> I do see a way forward for implementing this, at least the parts that
> Emacs can control, so let's leave this open.

Great!

bug#50113: Excorporate: Communicating with domain that requires SSO?

[Thomas Fitzsimmons]

Hi,

ParetoOptimal <pareto.optimal@mailfence.com> writes:

>> As a starting point, are you able to make DavMail retrieve your calendar
>> items? 
>
> Hello! I'm able to use davmail successfully, but I'm not sure how to
> verify how my calendar is working.

OK, thanks for following up.

> I guess I'll try following
> https://davmail.sourceforge.net/thunderbirdcalendarsetup.html and verify
> it works with thunderbird?

Sure, that'd be proof that something is working.

> I did try going to the caldav url fom the instructions above:
>
> http://localhost:1080/users/mail@company.com/calendar
>
> But i received an error, so it's very possible that may not work or may
> need a new url.
>
>> If yes, check if it uses, or can be configured to use, EWS
>> rather than WebDAV.
>
> You mean if davmail can be configured to expose the calendar as EWS
> rather than WebDAV or caldav format right? Because excorporate is only
> capable of consuming EWS?

No; my eventual goal is to help you get Excorporate communicating
directly with the Exchange server, without needing DavMail.  Today,
Excorporate can only do that using EWS, not WebDAV.

I was asking whether you can configure DavMail to use EWS, instead of
WebDAV, to communicate with the Exchange server.  If you can get DavMail
communicating with the Exchange server using EWS (independent of
Excorporate), then that proves a) your administrators allow access to
the EWS APIs, and b) DavMail can successfully authenticate to your
Exchange server.

If (a) works, and (b) works using OAuth 2.0, then that's what this bug
report intends to eventually provide support for in Emacs.

If (a) does not work, then either Excorporate will not work, or someone
could add WebDAV support to Excorporate.

>> If you can make DavMail + EWS work somehow, then
>> there's hope for Excorporate in your environment, with some oauth2.el
>> integration effort.
>
> If DavMail is working, what is the need for oauth2.el?

Interesting timing; I just pushed the start of url-http-oauth
yesterday, but it's only the skeleton so far:

   https://git.sr.ht/~fitzsim/url-http-oauth

url-retrieve operations are buried deep within Excorporate and
soap-client.  So oauth2.el's crucial TODO item about integrating with
the URL library is relevant.  It was never clear to me how to use
oauth2.el to good effect, assuming a dependence on `url-retrieve'.  So
I'm writing another mode, similar to url-http-ntlm.  This new mode will
use the encrypted auth-source database to store Bearer tokens instead of
plstore.

Initially I will validate the mode with a demo package that
authenticates to https://meta.sr.ht/query.  Sourcehut has implemented
OAuth 2.0 in a perfectly Free Software-friendly way.  So I will validate
that it works first.

This should result in a URL add-on library that is well-integrated with
Emacs, and maintainable, at least for one hospitable OAuth 2.0 provider.

> It seems like a sensible lowest-effort way to get excorporate working
> with SSO would be to totally depend on DavMail.

One of Excorporate's main mandates is to have no middle-process.
Similar to how Gnus doesn't need an external process to communicate with
IMAP and SMTP servers.  At this point I consider an
Excorporate-to-DavMail integration a non-goal; it's not something I'd be
interested in maintaining, anyway.

Definitely try to get Emacs working with your calendar, and depend on
DavMail exclusively.  But there are probably better ways to do that,
which don't involve Excorporate at all.

> I tried to research whether this is possible, but unsurprisingly
> searching "davmail calendar EWS" gave poor results.

It seems to be covered here (via Searx):

   https://davmail.sourceforge.net/faq.html

A search for "davmail calendar Excorporate" would probably eventually
lead you back to this bug report.

Thomas

bug#50113: Excorporate: Communicating with domain that requires SSO?

[Thomas Fitzsimmons]

Hi,

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Thomas Fitzsimmons <fitzsim@fitzsim.org> writes:
>
>> I do see a way forward for implementing this, at least the parts that
>> Emacs can control, so let's leave this open.
>
> Great!

Excorporate 1.1.0, with OAuth 2.0 support, is now available on GNU ELPA.
I'm closing this bug report.  Please test the new release and create new
bug reports if necessary.

Thomas