bug#72992: 29.4; towards xoauth2 support in Emacs

[Xiyue Deng <manphiz@gmail.com>]

Andrew Cohen <acohen@ust.hk> writes:

>>>>>> "XD" == Xiyue Deng <manphiz@gmail.com> writes:
>
>     XD> Hi Andrew, Andrew Cohen <acohen@ust.hk> writes:
>
>     >>>>>>> "XD" == Xiyue Deng <dengxiyue@gmail.com> writes:
>     >> 
>
> [...]
>
>     XD> The basic support is actually in the Emacs core already,
>     XD> e.g. for Gnus nnimap[2] and smtpmail[3].  However, this assumes
>     XD> one to put the access_token in place of `:secret' in the
>     XD> auth-source file as Emacs uses password as the access_token in
>     XD> both places.  However, access_token expires quite frequently
>     XD> (e.g. about 1 hour for Gmail) and without refreshing it
>     XD> automatically it is practically impossible to use conveniently.
>     XD> Hence the propose hack and the following suggestion.
>     >> 
>     >> 
>     >> This isn't actually true. When I added the support many years
>     >> ago, I updated auth-source so that the :secret field can be a
>     >> function, and this is how you should be using the current xoauth
>     >> support.
>
>     XD> Thanks for pointing this out!  I found the place where `:secret'
>     XD> is handled as a function[1].  However, this requires a user to
>     XD> implement the oauth2 logic oneself, which I'm afraid is a bit
>     XD> too low-level and error-prone.  (Actually, can I actually put a
>     XD> lisp function in auth-source.gpg?)  
>
> I don't think you have to do anything low level, and I don't think there
> is anything error prone here; you can use the functions from oauth
> themselves (oauth2.el can create its own plstores, but I prefer to use
> auth-source.el to manage the stores).  The only things needed are a call
> to oauth2-refresh-access to get a new token, and then
> oauth2-token-access-token to return the new access token.
>

Yes, I'm not worried about power users.  I just think that the average
Emacs user would be hesitant on writing ELisp themselves to enable
xoauth2 login (hence low-level), especially when they don't have
anything to copy from (yet).  Many Gnus users are not programmers and
would prefer writing "(nnimap-authenticator 'xoauth2)" and expect it to
work.  But I believe you don't object providing that convenience OOTB
either.

> The function I wrote computes the refresh time to decide when to create
> a new token. This logic could easily be put into oauth2 instead.
>

I am planning on adding this to oauth2 as well.  Will ask for your
review when that happens.

> And yes, you can put the lisp function in auth-source.gpg (this is what
> I do).
>

TIL!  (I used to have a handwritten script to get the values for
offlineimap.  Guess we should all be using `auth-info-password')

> By the way there are some significant bugs in auth-source.el which I
> have fixed in my personal tree but haven't yet pushed. I have so little
> time for emacs at the moment, but I'll try to get around to it. And
> there is one major deficiency in auth-source.el that I want to deal
> with: obfuscation of the :secret. When Ted originally wrote
> auth-source.el he wrapped the :secret in a closure so that the secret
> itself wasn't visible in memory. At the time he did this, closures
> weren't fully part of emacs, and their implementation at the time didn't
> expose the contents of the closure in bytecode. But the current official
> implementation does, so this obfuscation trick no longer works. I want
> to remove it since it no longer works and might lead to confusion. 
>

Looking forward to it!

>     XD> Maybe auth-source source can host a helper function that checks
>     XD> if `:secret' is not set and xaouth2 is preferred (e.g. `:auth'
>     XD> is `xoauth2') and all required credentials are available it will
>     XD> get the access_token and put it `:secret' (or basically my hacky
>     XD> advice :)
>
> I think this isn't the right way to go. Currently xoauth2 is one of
> several supported SASL methods.  The logic is supposed to be to try them
> in a certain order, but this hasn't worked properly for some
> time. Nobody has noticed since almost everyone uses only the basic
> method. In gnus there has always been a server variable,
> nnimap-authenticator, that chooses the preferred sasl method, which is
> how the current support for xaouth2 is designed to work.  I think this
> is the right way to handle this (rather than relying on some specific
> form of the auth-source entry) but it would be good to fix the logic in
> nnimap.el to allow multiple methods to be tried.
>

Right.  The `:auth' trick I did is just to workaround the restriction
that `nnimap-login' chooses basic method over other methods, and I'd
prefer a better built-in support in auth-source myself.  As you
mentioned, maybe it can be remodeled after `smtpmail-try-auth-method' to
so that the login method is chosen on demand instead of trial-and-error.

> [...]
>
>     XD> P.S. Is your set up mentioned in Bug#72358 still working for
>     XD> outlook.com emails?  After reaching out to an MS representative
>     XD> they mentioned that token refresh was disabled[3] for
>     XD> outlook.com so I just gave up.  Maybe it still works for Outlook
>     XD> Org emails?
>
> Yes, it still works perfectly. I suspect that the information they gave
> you isn't fully accurate :)

Thanks for confirming!  I'll follow-up in private to try to figure this
out if you don't mind.

> -- 
> Andrew Cohen

-- 
Xiyue Deng