mbsync with XOAUTH2 SASL mechanism

mbsync with XOAUTH2 SASL mechanism

[Peter Polidoro]

I am trying to setup an oauth2 email account to work with Emacs 
using mbsync (from the isync guix package) and mu4e.

I setup oauth2ms to fetch the token and setup mbsync to use 
oauth2ms for the PassCmd and XOAUTH2 for the AuthMechs.

Now when I run mbsync, I get the error:

IMAP error: selected SASL mechanism(s) not available;
   selected: XOAUTH2
   available: SCRAM-SHA-1 SCRAM-SHA-256 GS2-IAKERB GS2-KRB5 GSSAPI 
   GSS-SPNEGO DIGEST-MD5 EXTERNAL OTP CRAM-MD5 PLAIN ANONYMOUS

I found instructions online saying I need to install the xoauth2 
sasl plugin from https://github.com/moriyoshi/cyrus-sasl-xoauth2

What is the proper Guix way of getting mbsync to work with 
XOAUTH2? Should I try to package cyrus-sasl-xoauth2 or modify the 
isync package or something else? Thanks!

Re: mbsync with XOAUTH2 SASL mechanism

[Joshua Branson]

Peter Polidoro <peter@polidoro.io> writes:

> I am trying to setup an oauth2 email account to work with Emacs using mbsync
> (from the isync guix package) and mu4e.
>
> I setup oauth2ms to fetch the token and setup mbsync to use oauth2ms for the
> PassCmd and XOAUTH2 for the AuthMechs.
>
> Now when I run mbsync, I get the error:
>
> IMAP error: selected SASL mechanism(s) not available;
>   selected: XOAUTH2
>   available: SCRAM-SHA-1 SCRAM-SHA-256 GS2-IAKERB GS2-KRB5 GSSAPI    GSS-SPNEGO
>  DIGEST-MD5 EXTERNAL OTP CRAM-MD5 PLAIN ANONYMOUS
>
> I found instructions online saying I need to install the xoauth2 sasl plugin
> from https://github.com/moriyoshi/cyrus-sasl-xoauth2
>
> What is the proper Guix way of getting mbsync to work with XOAUTH2? Should I try
> to package cyrus-sasl-xoauth2 or modify the isync package or something else?
> Thanks!
>

Man this sounds complicated!  haha.  I use isync too...but I do the
really really lazy (insecure) way via ~/.authinfo.

Joshua

Re: mbsync with XOAUTH2 SASL mechanism

[Peter Polidoro]

> Man this sounds complicated!  haha.  I use isync too...but I do the
> really really lazy (insecure) way via ~/.authinfo.
> 
> Joshua

I wish I did not have to have such a complicated setup. My work email account has just stopped allowing basic password authentication, however, so I can no longer use Emacs for my work email until I figure this out.

I do not know if OAuth2 refers to something proprietary, if so I apologize for bringing it up here. My only goal is to be able to use Emacs rather than proprietary software for my work email.

I submitted a patch for a “cyrus-sasl-xoauth2” package that may allow this to work, but I do not yet know enough about Guix packaging to complete the package. If anyone has a similar problem and has advice or can help I would really appreciate it. Thanks!

Re: mbsync with XOAUTH2 SASL mechanism

[Felix Lechner via]

Hi Peter,

On Sun, Nov 13, 2022 at 11:05 AM Peter Polidoro <peter@polidoro.io> wrote:
>
> I do not know if OAuth2 refers to something proprietary

While I cannot help much with your issue, the "Open Authentication"
standard is open--although too complex even for some insiders:

Eran Hammer resigned from his role of lead author for the OAuth 2.0
project, withdrew from the IETF working group, and removed his name
from the specification in July 2012. Hammer cited a conflict between
web and enterprise cultures as his reason for leaving, noting that
IETF is a community that is "all about enterprise use cases" and "not
capable of simple". "What is now offered is a blueprint for an
authorization protocol", he noted, "that is the enterprise way",
providing a "whole new frontier to sell consulting services and
integration solutions". In comparing OAuth 2.0 with OAuth 1.0,
Hammer points out that it has become "more complex, less
interoperable, less useful, more incomplete, and most importantly,
less secure". He explains how architectural changes for 2.0 unbound
tokens from clients, removed all signatures and cryptography at a
protocol level and added expiring tokens (because tokens could not be
revoked) while complicating the processing of authorization. Numerous
items were left unspecified or unlimited in the specification because
"as has been the nature of this working group, no issue is too small
to get stuck on or leave open for each implementation to decide."
(internal quotes removed) [1]

Kind regards
Felix Lechner

[1] https://en.wikipedia.org/wiki/OAuth

Re: mbsync with XOAUTH2 SASL mechanism

[Tobias Geerinckx-Rice]

[-- Attachment #1: Type: text/plain, Size: 217 bytes --]

Joshua Branson 写道：
> really really lazy (insecure) way via ~/.authinfo.

I'll keep this tangent short:

  ~ λ file .authinfo.gpg 
  .authinfo.gpg: data

(There is no step 2.)

Kind regards,

T G-R

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 247 bytes --]

Re: mbsync with XOAUTH2 SASL mechanism

[jbranso]

November 14, 2022 6:09 PM, "Tobias Geerinckx-Rice" <me@tobias.gr> wrote:

> Joshua Branson 写道：
> 
>> really really lazy (insecure) way via ~/.authinfo.
> 
> I'll keep this tangent short:
> 
> ~ λ file .authinfo.gpg
> .authinfo.gpg: data
> 
> (There is no step 2.)

hahaha!  what up friend?  My problem with that method 
(and yes I was once stupid enough to pull this off), was that I had 
created such a file with my gpg key, and then lost that key.  All my
passwords gone.  :(  Sad day.  

I'll tell you what, I'll go ahead and try to use a .authinfo.gpg again
and try password based encryption.  Can't hurt as long as I remember the password
somewhere.

Joshua

> 
> Kind regards,
> 
> T G-R

Re: mbsync with XOAUTH2 SASL mechanism

[Giovanni Biscuolo]

[-- Attachment #1: Type: text/plain, Size: 1260 bytes --]

Hello Peter,

have you solved your problem?

I never tested this, but I'll have to do...

Peter Polidoro <peter@polidoro.io> writes:

> I am trying to setup an oauth2 email account to work with Emacs 
> using mbsync (from the isync guix package) and mu4e.

[...]

> What is the proper Guix way of getting mbsync to work with 
> XOAUTH2? Should I try to package cyrus-sasl-xoauth2 or modify the 
> isync package or something else? Thanks!

I can't help with packaging and integrating cyrus-sasl-xoauth2 with
isync but maybe getmail6 (packaged in Guix) is able to get your emails
from your enterprise IMAP server

This howto is for getmail 5.6 but AFAIU should also work for getmail6
(it's mentioned in the official getmail6 documentation [1]) 

Last but not least, please consider that if you can (and if your company
server/postmaster allows it) it's much better to use an "app password"
method instead of Oauth2
https://pypi.org/project/getmail/#oauth2-privacy-policy

HTH! Gio'

P.S.: please give us feedback if you can, I think your is a common
problem among Guix users



[1] https://getmail6.org/configuration.html#retriever-parameters (search
for "use_xoauth2"

-- 
Giovanni Biscuolo

Xelera IT Infrastructures

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 849 bytes --]

Re: mbsync with XOAUTH2 SASL mechanism

[Peter Polidoro]

Giovanni Biscuolo <g@xelera.eu> writes:

> have you solved your problem?

No, I hate to admit that I have given up in frustration.

My work email unfortunately uses office365. I work for a large 
nonprofit science foundation. I wish they only used free software, 
but some of the enterprise software is proprietary. I used to be 
able to read and write my work email with Emacs, but after 
Microsoft changed their policies, that no longer works.

I found several sets of instructions online for getting outlook365 
OAuth2 working with Emacs, such as this one:

https://sites.uw.edu/bxf4/2022/09/01/getting-uw-outlook-365-oauth2-to-work-with-emacs-mu4e-mbsync-and-msmtp/

I submitted a cyrus-sasl-xoauth2 guix package, but the guix side 
is not the frustrating part.

The frustrating part is that all of the instructions online say 
you need to create an "Azure Active Directory App". I created one 
and it seemed to work fine, but after a couple of weeks it expired 
and then I kept getting emails from Microsoft saying I needed to 
pay them money to keep the Azure app running. I really do not want 
to subscribe to anything Microsoft related, even if my work pays 
for it. That link references another authentication app from 
Thunderbird, perhaps there is a way to get something like that 
working with Emacs, but I could not find any detailed instructions 
to do so.

Right now I am able to read and write personal emails in Emacs, 
but for all of my work emails I am forced to use Outlook in a web 
browser.

> Last but not least, please consider that if you can (and if your 
> company
> server/postmaster allows it) it's much better to use an "app 
> password"
> method instead of Oauth2
> https://pypi.org/project/getmail/#oauth2-privacy-policy

I wish. That is the problem. App passwords used to be allowed by 
office365, but they changed that policy.

Re: mbsync with XOAUTH2 SASL mechanism

[Timo Wilken]

Hi Peter, hi Giovanni,

I had the same problem with having to register an "app" to access my
emails (but with neomutt, not Emacs).

Instead, I ended up "borrowing" Thunderbird's client key and secret,
which has worked fine so far.

Maybe I'm being a bit paranoid, but I don't want to post the literal
key here. You can copy it from mailnews/base/src/OAuth2Providers.jsm
in Thunderbird's source tree (look for "login.microsoftonline.com" in
the kIssuers variable near line 140). Send me an email privately if
you can't find it.

I hope that helps,
Timo

On Tue, Jan 24, 2023 at 07:41:51AM -0500, Peter Polidoro wrote:
> 
> Giovanni Biscuolo <g@xelera.eu> writes:
> 
> > have you solved your problem?
> 
> No, I hate to admit that I have given up in frustration.
> 
> My work email unfortunately uses office365. I work for a large nonprofit
> science foundation. I wish they only used free software, but some of the
> enterprise software is proprietary. I used to be able to read and write my
> work email with Emacs, but after Microsoft changed their policies, that no
> longer works.
> 
> I found several sets of instructions online for getting outlook365 OAuth2
> working with Emacs, such as this one:
> 
> https://sites.uw.edu/bxf4/2022/09/01/getting-uw-outlook-365-oauth2-to-work-with-emacs-mu4e-mbsync-and-msmtp/
> 
> I submitted a cyrus-sasl-xoauth2 guix package, but the guix side is not the
> frustrating part.
> 
> The frustrating part is that all of the instructions online say you need to
> create an "Azure Active Directory App". I created one and it seemed to work
> fine, but after a couple of weeks it expired and then I kept getting emails
> from Microsoft saying I needed to pay them money to keep the Azure app
> running. I really do not want to subscribe to anything Microsoft related,
> even if my work pays for it. That link references another authentication app
> from Thunderbird, perhaps there is a way to get something like that working
> with Emacs, but I could not find any detailed instructions to do so.
> 
> Right now I am able to read and write personal emails in Emacs, but for all
> of my work emails I am forced to use Outlook in a web browser.
> 
> > Last but not least, please consider that if you can (and if your company
> > server/postmaster allows it) it's much better to use an "app password"
> > method instead of Oauth2
> > https://pypi.org/project/getmail/#oauth2-privacy-policy
> 
> I wish. That is the problem. App passwords used to be allowed by office365,
> but they changed that policy.
> 

Re: mbsync with XOAUTH2 SASL mechanism

[Giovanni Biscuolo]

[-- Attachment #1: Type: text/plain, Size: 635 bytes --]

Hi,

Giovanni Biscuolo <g@xelera.eu> writes:

[...]

> This howto is for getmail 5.6 but AFAIU should also work for getmail6
> (it's mentioned in the official getmail6 documentation [1])

I forgot to mention the howto!

https://www.bytereef.org/howto/oauth2/getmail.html

it contains detailed instructions on how to configure getmail to get the
initial access and refresh tokens (they must be periodically "manually"
refreshed, unfortunately)

HTH! Gio'

[...]

> [1] https://getmail6.org/configuration.html#retriever-parameters (search
> for "use_xoauth2"

-- 
Giovanni Biscuolo

Xelera IT Infrastructures

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 849 bytes --]

Re: mbsync with XOAUTH2 SASL mechanism

[Giovanni Biscuolo]

[-- Attachment #1: Type: text/plain, Size: 777 bytes --]

Hi Timo,

Timo Wilken <guix@twilken.net> writes:

[...]

> Instead, I ended up "borrowing" Thunderbird's client key and secret,
> which has worked fine so far.
>
> Maybe I'm being a bit paranoid, but I don't want to post the literal
> key here. You can copy it from mailnews/base/src/OAuth2Providers.jsm
> in Thunderbird's source tree (look for "login.microsoftonline.com" in
> the kIssuers variable near line 140).

thanks for sharing this trick!

this adds a big dependency in our software stack, but it works, so why
not! :-)

a couple questions:

- have you tried getmail6 with the mentioned howto?

- to refresh the tokens do you have to periodically run Thunderbird?

Thanks! Gio'

[...]

-- 
Giovanni Biscuolo

Xelera IT Infrastructures

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 849 bytes --]

Re: mbsync with XOAUTH2 SASL mechanism

[Timo Wilken]

Hi Giovanni!



On 26 January 2023 11:19:22 CET, Giovanni Biscuolo <g@xelera.eu> wrote:
>Timo Wilken <guix@twilken.net> writes:
>> Instead, I ended up "borrowing" Thunderbird's client key and secret,
>> which has worked fine so far.
>
>thanks for sharing this trick!
>
>this adds a big dependency in our software stack, but it works, so why
>not! :-)

Just to be clear: I did not install Thunderbird. I do not run Thunderbird. I only copy-pasted the client key from its source code into my own scripts. See below for details.

>a couple questions:
>
>- have you tried getmail6 with the mentioned howto?

No, I have not, sorry.

>- to refresh the tokens do you have to periodically run Thunderbird?

No, I do not run Thunderbird. I use neomutt to read my mail, and specifically for XOAUTH2 I use the "mutt_oauth2.py" script that comes with neomutt: https://github.com/neomutt/neomutt/blob/main/contrib/oauth2/mutt_oauth2.py

This script handles everything XOAUTH2-related, including refreshing tokens when needed. I just call it with its --client-id and --client-secret parameters (among other params; I am on my phone at the moment and cannot look up the full invocation I use), using the values I got from Thunderbirds source code.

(See also the neomutt documentation: https://neomutt.org/guide/optionalfeatures#6-%C2%A0oauthbearer-and-xoauth2-support)

Perhaps you can adapt this script to your usecase.

Cheers,
Timo