gmail+imap+smtp (oauth2)

gmail+imap+smtp (oauth2)

[Uwe Brauer]

Hi

I am forced to use gmail at my university (the features google offer are
a bit different to the personal accounts)
and also for some private stuff.

There was a discussion in 2022 about the following issue, but then
google dropped this subject because of Covid, however now it seems to
back:

Now google keeps sending me message that from 30th of May 3rd party
packages cannot connect anymore to imap+smtp via the traditional way.

It is my understanding that it might be possible make emacs (gnus) work
with gmail under those circumstances, that is using oauth2.

Has anybody got that to work?

If so, can he/she share its setting?

Thanks

Uwe Brauer 


-- 
I strongly condemn Putin's war of aggression against the Ukraine.
I support to deliver weapons to Ukraine's military. 
I support the ban of Russia from SWIFT.
I support the EU membership of the Ukraine. 

Re: gmail+imap+smtp (oauth2)

[Jostein Kjønigsen]

[-- Attachment #1: Type: text/plain, Size: 1234 bytes --]

On 03.05.2022 07:59, Uwe Brauer wrote:
>
> Hi
>
> I am forced to use gmail at my university (the features google offer are
> a bit different to the personal accounts)
> and also for some private stuff.
>
> There was a discussion in 2022 about the following issue, but then
> google dropped this subject because of Covid, however now it seems to
> back:
>
> Now google keeps sending me message that from 30th of May 3rd party
> packages cannot connect anymore to imap+smtp via the traditional way.
>
> It is my understanding that it might be possible make emacs (gnus) work
> with gmail under those circumstances, that is using oauth2.
>
> Has anybody got that to work?
>
> If so, can he/she share its setting?
>
> Thanks
>
> Uwe Brauer
>
>
My understanding may not be perfect, but having considered a similar 
conundrum for a work-integration, I landed on the conclusion that SMTP 
and IMAP should keep working as long as you use app-passwords for 
logging in to your account. If so, that should probably be adequate for 
Emacs too.

Or is my understanding of this situation incorrect?

-- 
Vennlig hilsen
*Jostein Kjønigsen*

jostein@kjonigsen.net 🍵 jostein@gmail.com
https://jostein.kjønigsen.no <https://jostein.kjønigsen.no>

[-- Attachment #2: Type: text/html, Size: 1863 bytes --]

Re: gmail+imap+smtp (oauth2)

[Uwe Brauer]

[-- Attachment #1: Type: text/plain, Size: 846 bytes --]


> On 03.05.2022 07:59, Uwe Brauer wrote:
> My understanding may not be perfect, but having considered a similar
> conundrum for a work-integration, I landed on the conclusion that SMTP
> and IMAP should keep working as long as you use app-passwords for
> logging in to your account. If so, that should probably be adequate
> for Emacs too.


I am not to understand the meaning of  app-passwords.

My email account has a password, that is used when connecting via ssl
or tls. But gmail will need a different protocol in the future, this is
why I am worried.


> Or is my understanding of this situation incorrect?

-- 
I strongly condemn Putin's war of aggression against the Ukraine.
I support to deliver weapons to Ukraine's military. 
I support the ban of Russia from SWIFT.
I support the EU membership of the Ukraine. 

[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 5673 bytes --]

Re: gmail+imap+smtp (oauth2)

[Robert Pluim]

>>>>> On Tue, 03 May 2022 22:44:46 +0200, Uwe Brauer <oub@mat.ucm.es> said:

    >> On 03.05.2022 07:59, Uwe Brauer wrote:
    >> My understanding may not be perfect, but having considered a similar
    >> conundrum for a work-integration, I landed on the conclusion that SMTP
    >> and IMAP should keep working as long as you use app-passwords for
    >> logging in to your account. If so, that should probably be adequate
    >> for Emacs too.

    Uwe> I am not to understand the meaning of  app-passwords.

    Uwe> My email account has a password, that is used when connecting via ssl
    Uwe> or tls. But gmail will need a different protocol in the future, this is
    Uwe> why I am worried.

Your *Google* account has a password, which is what you use to connect
to eg gmail.com using a browser. You can also create one or more
separate 'app password' on Google, which you then use for a specific
connection to Google, eg SMTP. See
<https://support.google.com/accounts/answer/185833?hl=en>

Robert
-- 

Re: gmail+imap+smtp (oauth2)

[Tim Cross]

Uwe Brauer <oub@mat.ucm.es> writes:

> [[S/MIME Signed Part:Undecided]]
>
>> On 03.05.2022 07:59, Uwe Brauer wrote:
>> My understanding may not be perfect, but having considered a similar
>> conundrum for a work-integration, I landed on the conclusion that SMTP
>> and IMAP should keep working as long as you use app-passwords for
>> logging in to your account. If so, that should probably be adequate
>> for Emacs too.
>
>
> I am not to understand the meaning of  app-passwords.
>

There is nothing too concerning here. An application password is just
another password you can use to authenticate for a specific purpose. You
could in fact have 2 application passwords - 1 for IMAP and 1 for SMTP.
These application passwords are typically much longer than most user
passwords and you don't get to select it - the password is generated for
you. 

Your 'normal' Google password will only work as part of a 2FA
authentication process. You enter your account name (google email
address usually), your password and then a 2nd factor (fingerprint, code
from an authentication app, hardware key etc).  If your IMAP or SMTP/MUA
clients support oauth2, you don't need application passwords, you can
just use your google username/password and 2nd factor. Problem is, many
clients, particularly older ones, don't support oauth2. The situation is
further compounded by the fact that oauth2 implementations vary enough
that it is very difficult to just implement a 'generic' oauth2 solution
in a high level, easy to use manner. For example, it is possible to
configure Emacs' SMTP to use oauth2, but it is clunky, requires a
significant level of udnerstanding of oauth2 and manual steps
(such as getting a token from a web page and putting it into your configuration).

> My email account has a password, that is used when connecting via ssl
> or tls. But gmail will need a different protocol in the future, this is
> why I am worried.
>

A 'different protocol' is perhaps over stating it. The SMTP protocol
itself isn't changing, but the authentication mechanism is. At present,
I think we are in a transition stage and like many transitions, things
are a bit rough around the edges. At some point, I suspect Google will
remove application passwords. However, I expect this will only happen
once the alternative authentication mecahnisms have stabilised and
standardised more than they are at present. By that time, I would also
expect many of the clients will support the new authentication
protocols. 

Re: gmail+imap+smtp (oauth2)

[Uwe Brauer]

[-- Attachment #1: Type: text/plain, Size: 2663 bytes --]

>>> "TC" == Tim Cross <theophilusx@gmail.com> writes:

> Uwe Brauer <oub@mat.ucm.es> writes:

>> [[S/MIME Signed Part:Undecided]]
>> 
>>> On 03.05.2022 07:59, Uwe Brauer wrote:
>>> My understanding may not be perfect, but having considered a similar
>>> conundrum for a work-integration, I landed on the conclusion that SMTP
>>> and IMAP should keep working as long as you use app-passwords for
>>> logging in to your account. If so, that should probably be adequate
>>> for Emacs too.
>> 
>> 
>> I am not to understand the meaning of  app-passwords.
>> 

> There is nothing too concerning here. An application password is just
> another password you can use to authenticate for a specific purpose. You
> could in fact have 2 application passwords - 1 for IMAP and 1 for SMTP.
> These application passwords are typically much longer than most user
> passwords and you don't get to select it - the password is generated for
> you. 

> Your 'normal' Google password will only work as part of a 2FA
> authentication process. You enter your account name (google email
> address usually), your password and then a 2nd factor (fingerprint, code
> from an authentication app, hardware key etc).  If your IMAP or SMTP/MUA
> clients support oauth2, you don't need application passwords, you can
> just use your google username/password and 2nd factor. Problem is, many
> clients, particularly older ones, don't support oauth2. The situation is
> further compounded by the fact that oauth2 implementations vary enough
> that it is very difficult to just implement a 'generic' oauth2 solution
> in a high level, easy to use manner. For example, it is possible to
> configure Emacs' SMTP to use oauth2, but it is clunky, requires a
> significant level of udnerstanding of oauth2 and manual steps
> (such as getting a token from a web page and putting it into your configuration).

Hm, but did you successfully configure gnus with your  app password in order to
receive (imap) and to send (smtp).

If so, could you please share your configuration.

Or are you saying it is simply to use

(setq gnus-select-method (list 'nnnil "")) 
(setq gnus-secondary-select-methods nil)
(setq gnus-secondary-select-methods
      '(
	(nnimap "UCMgmail"
		(nnimap-address "imap.gmail.com")
		(nnimap-server-port 993)
                ;; (nnimap-authinfo-file "~/.authinfo.gpg")
		(nnimap-authinfo-file "~/.authinfo")
		(nnimap-stream ssl)
                ;;(nnimap-stream starttls)
                (nnimap-fetch-partial-articles "text/")
		(nnir-search-engine imap))))

And put in authinfo my app password instead my «normal» one?

[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 5673 bytes --]

Re: gmail+imap+smtp (oauth2)

[Robert Pluim]

>>>>> On Thu, 05 May 2022 14:57:52 +0200, Uwe Brauer <oub@mat.ucm.es> said:
    Uwe> Hm, but did you successfully configure gnus with your  app password in order to
    Uwe> receive (imap) and to send (smtp).

Yes

    Uwe> If so, could you please share your configuration.

    Uwe> Or are you saying it is simply to use

    Uwe> (setq gnus-select-method (list 'nnnil "")) 
    Uwe> (setq gnus-secondary-select-methods nil)
    Uwe> (setq gnus-secondary-select-methods
    Uwe>       '(
    Uwe> 	(nnimap "UCMgmail"
    Uwe> 		(nnimap-address "imap.gmail.com")
    Uwe> 		(nnimap-server-port 993)
    Uwe>                 ;; (nnimap-authinfo-file "~/.authinfo.gpg")
    Uwe> 		(nnimap-authinfo-file "~/.authinfo")
    Uwe> 		(nnimap-stream ssl)
    Uwe>                 ;;(nnimap-stream starttls)
    Uwe>                 (nnimap-fetch-partial-articles "text/")
    Uwe> 		(nnir-search-engine imap))))

    Uwe> And put in authinfo my app password instead my «normal» one?

Yes

Robert
-- 

Re: gmail+imap+smtp (oauth2)

[Uwe Brauer]

[-- Attachment #1: Type: text/plain, Size: 1512 bytes --]

>>> "RP" == Robert Pluim <rpluim@gmail.com> writes:

>>>>>> On Thu, 05 May 2022 14:57:52 +0200, Uwe Brauer <oub@mat.ucm.es> said:
Uwe> Hm, but did you successfully configure gnus with your  app password in order to
Uwe> receive (imap) and to send (smtp).

> Yes

Uwe> If so, could you please share your configuration.

Uwe> Or are you saying it is simply to use

Uwe> (setq gnus-select-method (list 'nnnil "")) 
Uwe> (setq gnus-secondary-select-methods nil)
Uwe> (setq gnus-secondary-select-methods
Uwe> '(
Uwe> (nnimap "UCMgmail"
Uwe> (nnimap-address "imap.gmail.com")
Uwe> (nnimap-server-port 993)
Uwe> ;; (nnimap-authinfo-file "~/.authinfo.gpg")
Uwe> (nnimap-authinfo-file "~/.authinfo")
Uwe> (nnimap-stream ssl)
Uwe> ;;(nnimap-stream starttls)
Uwe> (nnimap-fetch-partial-articles "text/")
Uwe> (nnir-search-engine imap))))

Uwe> And put in authinfo my app password instead my «normal» one?

> Yes

I might give it a try but what is the syntax in teh authinfo file

My original setting is 
machine smtp.gmail.com login oub@ucm.es port 587 password passwd
machine UCMgmail login oub@ucm.es password passwd port  imaps


Tassilo suggested

machine imap.gmail.com login USER@gmail.com password APP_PWD force yes


So I am a bit confused.


> Robert

-- 
I strongly condemn Putin's war of aggression against the Ukraine.
I support to deliver weapons to Ukraine's military. 
I support the ban of Russia from SWIFT.
I support the EU membership of the Ukraine. 

[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 5673 bytes --]

Re: gmail+imap+smtp (oauth2)

[Robert Pluim]

>>>>> On Sun, 08 May 2022 16:36:31 +0200, Uwe Brauer <oub@mat.ucm.es> said:

    Uwe> I might give it a try but what is the syntax in teh authinfo file

The auth-source info documentation has plenty of gory details

    Uwe> My original setting is 
    Uwe> machine smtp.gmail.com login oub@ucm.es port 587 password passwd
    Uwe> machine UCMgmail login oub@ucm.es password passwd port  imaps


    Uwe> Tassilo suggested

    Uwe> machine imap.gmail.com login USER@gmail.com password APP_PWD force yes

Most fields in .authinfo are optional, so

machine foo login bar
machine foo login bar port baz

are both legal. For gmail I have

machine imap.gmail.com login USER password APP_PWD port imaps

(the "force yes" stuff is only needed for servers that donʼt advertise
that you need to authenticate)

Robert
-- 

Re: gmail+imap+smtp (oauth2)

[Uwe Brauer]

[-- Attachment #1: Type: text/plain, Size: 884 bytes --]


Uwe> I might give it a try but what is the syntax in teh authinfo file

> The auth-source info documentation has plenty of gory details

I am just digging in, not that clearly written or at least my use case
is not easily found.

Uwe> My original setting is 
Uwe> machine smtp.gmail.com login oub@ucm.es port 587 password passwd
Uwe> machine UCMgmail login oub@ucm.es password passwd port  imaps


Uwe> Tassilo suggested

Uwe> machine imap.gmail.com login USER@gmail.com password APP_PWD force 
yes

> Most fields in .authinfo are optional, so

> machine foo login bar
> machine foo login bar port baz

> are both legal. For gmail I have

> machine imap.gmail.com login USER password APP_PWD port imaps

Ok what confuses me if it is better to use smtps or specifying the port
directly as in 587, or 465.

But I might give it a try either way

thanks

[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 5673 bytes --]

Re: gmail+imap+smtp (oauth2)

[Robert Pluim]

>>>>> On Sun, 08 May 2022 18:40:51 +0200, Uwe Brauer <oub@mat.ucm.es> said:

    Uwe> I might give it a try but what is the syntax in teh authinfo file

    >> The auth-source info documentation has plenty of gory details

    Uwe> I am just digging in, not that clearly written or at least my use case
    Uwe> is not easily found.

Now Iʼm hurt, I tried to make those docs clear :-)

If you tell me which bits are unclear, we can rework them.

    Uwe> My original setting is 
    Uwe> machine smtp.gmail.com login oub@ucm.es port 587 password passwd
    Uwe> machine UCMgmail login oub@ucm.es password passwd port  imaps


    Uwe> Tassilo suggested

    Uwe> machine imap.gmail.com login USER@gmail.com password APP_PWD force 
    Uwe> yes

    >> Most fields in .authinfo are optional, so

    >> machine foo login bar
    >> machine foo login bar port baz

    >> are both legal. For gmail I have

    >> machine imap.gmail.com login USER password APP_PWD port imaps

    Uwe> Ok what confuses me if it is better to use smtps or specifying the port
    Uwe> directly as in 587, or 465.

Either will work, Emacs will convert from the string representation to
the integer one. These days I think 'submission' (587) is preferred to
'smtps' (465), except that smtp.gmail.com doesnʼt do TLS on 587, only
STARTTLS, so I stubbornly stick to 465.

Robert
-- 

Re: gmail+imap+smtp (oauth2)

[Uwe Brauer]

Uwe> I might give it a try but what is the syntax in teh authinfo file


Uwe> I am just digging in, not that clearly written or at least my use case
Uwe> is not easily found.

> Now Iʼm hurt, I tried to make those docs clear :-)

Oops, I am deeply sorry 😱

> If you tell me which bits are unclear, we can rework them.

I will do in the next days, ok?

Thanks for your information, that has been helpful

Re: gmail+imap+smtp (oauth2)

[Robert Pluim]

>>>>> On Tue, 10 May 2022 08:29:11 +0200, Uwe Brauer <oub@mat.ucm.es> said:

    Uwe> I might give it a try but what is the syntax in teh authinfo file


    Uwe> I am just digging in, not that clearly written or at least my use case
    Uwe> is not easily found.

    >> Now Iʼm hurt, I tried to make those docs clear :-)

    Uwe> Oops, I am deeply sorry 😱

Humour over email never works very well. Iʼm not really hurt.

    >> If you tell me which bits are unclear, we can rework them.

    Uwe> I will do in the next days, ok?

ok

    Uwe> Thanks for your information, that has been helpful

Youʼre welcome (or is it 'no problem' these days?)

Robert
-- 

[app password does not work (at the moment)] (was: gmail+imap+smtp (oauth2))

[Uwe Brauer]

[-- Attachment #1: Type: text/plain, Size: 3235 bytes --]

>>> "RP" == Robert Pluim <rpluim@gmail.com> writes:

>>>>>> On Tue, 10 May 2022 08:29:11 +0200, Uwe Brauer <oub@mat.ucm.es> said:
Uwe> I might give it a try but what is the syntax in teh authinfo file


Uwe> I am just digging in, not that clearly written or at least my use case
Uwe> is not easily found.

>>> Now Iʼm hurt, I tried to make those docs clear :-)

Uwe> Oops, I am deeply sorry 😱

> Humour over email never works very well. Iʼm not really hurt.

Very true, and I was not «deeply sorry» 😉

>>> If you tell me which bits are unclear, we can rework them.

Uwe> I will do in the next days, ok?

> ok

Uwe> Thanks for your information, that has been helpful

Ok, here we go 2nd of June, my private gmail account does not allow
access via imap using 995 or similar. Since this might be a problem
faced my other Emacs users, I think it is appropriate to discuss this
issue here.


So I tried to follow the instructions found in 
https://support.google.com/mail/answer/185833?hl=en#

I activate 2-step verification and I generate an app password. Now one
remark and one question

    1. The generated 16 char password is actually much *shorter* than my
       imap password. This is supposed to enhance security 🙃

    2. I am not sure I understand: do I have to generate such password
       each time I want to access, via gnus my gmail account? I hope
       not.

Now I changed my authinfo entries to this 

machine smtp.gmail.com login oub.oub.oub@gmail.com port 587 password  XXXX
machine gmail login oub.oub.oub password XXXXX port imaps


Where XXXX is the new app password freshly generated.


I followed what you told me 

,----
| 
|     Uwe> If so, could you please share your configuration.
| 
|     Uwe> Or are you saying it is simply to use
| 
|     Uwe> (setq gnus-select-method (list 'nnnil "")) 
|     Uwe> (setq gnus-secondary-select-methods nil)
|     Uwe> (setq gnus-secondary-select-methods
|     Uwe>       '(
|     Uwe> 	(nnimap "UCMgmail"
|     Uwe> 		(nnimap-address "imap.gmail.com")
|     Uwe> 		(nnimap-server-port 993)
|     Uwe>                 ;; (nnimap-authinfo-file "~/.authinfo.gpg")
|     Uwe> 		(nnimap-authinfo-file "~/.authinfo")
|     Uwe> 		(nnimap-stream ssl)
|     Uwe>                 ;;(nnimap-stream starttls)
|     Uwe>                 (nnimap-fetch-partial-articles "text/")
|     Uwe> 		(nnir-search-engine imap))))
| 
|     Uwe> And put in authinfo my app password instead my «normal» one?
| 
| Yes
| 
| Robert
`----

Be it as it may, it *does not work* 

Here is my gnus group buffer



    [ Imap Gmail 2408064 ] 
           *: nnimap+gmail:INBOX

And I cannot open the group INBOX, the message is 

Opening nnimap server on gmail...failed: NO (AUTHENTICATIONFAILED) Invalid credentials (Failure)


But the password is the app password.


So any help would be strongly appreciated. The other alternative is
using oath2. Gosh.


Uwe Brauer 



-- 
I strongly condemn Putin's war of aggression against the Ukraine.
I support to deliver weapons to Ukraine's military. 
I support the ban of Russia from SWIFT.
I support the EU membership of the Ukraine. 

[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 5673 bytes --]

[SOLVED (magic?)] (was: [app password does not work (at the moment)])

[Uwe Brauer]

[-- Attachment #1: Type: text/plain, Size: 2617 bytes --]

>>> "UB" == Uwe Brauer <oub@mat.ucm.es> writes:

>>> "RP" == Robert Pluim <rpluim@gmail.com> writes:
>>>>>>> On Tue, 10 May 2022 08:29:11 +0200, Uwe Brauer <oub@mat.ucm.es> 

> So I tried to follow the instructions found in 
> https://support.google.com/mail/answer/185833?hl=en#

> I activate 2-step verification and I generate an app password. Now one
> remark and one question

>     1. The generated 16 char password is actually much *shorter* than my
>        imap password. This is supposed to enhance security 🙃

>     2. I am not sure I understand: do I have to generate such password
>        each time I want to access, via gnus my gmail account? I hope
>        not.

> Now I changed my authinfo entries to this 

> machine smtp.gmail.com login oub.oub.oub@gmail.com port 587 password  XXXX
> machine gmail login oub.oub.oub password XXXXX port imaps


> Where XXXX is the new app password freshly generated.


> I followed what you told me 

> ,----
> | 
> |     Uwe> If so, could you please share your configuration.
> | 
> |     Uwe> Or are you saying it is simply to use
> | 
> |     Uwe> (setq gnus-select-method (list 'nnnil "")) 
> |     Uwe> (setq gnus-secondary-select-methods nil)
> |     Uwe> (setq gnus-secondary-select-methods
> |     Uwe>       '(
> |     Uwe> 	(nnimap "UCMgmail"
> |     Uwe> 		(nnimap-address "imap.gmail.com")
> |     Uwe> 		(nnimap-server-port 993)
> |     Uwe>                 ;; (nnimap-authinfo-file "~/.authinfo.gpg")
> |     Uwe> 		(nnimap-authinfo-file "~/.authinfo")
> |     Uwe> 		(nnimap-stream ssl)
> |     Uwe>                 ;;(nnimap-stream starttls)
> |     Uwe>                 (nnimap-fetch-partial-articles "text/")
> |     Uwe> 		(nnir-search-engine imap))))
> | 
> |     Uwe> And put in authinfo my app password instead my «normal» one?
> | 
> | Yes
> | 
> | Robert
> `----

> Be it as it may, it *does not work* 

Well, you are not going to belive, it but after makeing the following minor change:

machine smtp.gmail.com login oub.oub.oub@gmail.com port 587 password  XXXXX
machine imap.gmail.com login oub.oub.oub password XXXX port imaps

After  *restarting emacs*, 

I can now again access my private gmail account, 

Sorry for the noise.

May I suggest to add a section to the authinfo info file, describing the
use of gmail app-passwords.

If you like I can give it a try (I would send you my proposal off list)
and you can decide for yourself.

Sometimes a novice (like me concerning gmail app passwords) might mention
things that seem trivial to the expert.


[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 5673 bytes --]

Re: [SOLVED (magic?)]

[Robert Pluim]

>>>>> On Thu, 02 Jun 2022 17:37:13 +0200, Uwe Brauer <oub@mat.ucm.es> said:
    >> |     Uwe> (setq gnus-select-method (list 'nnnil "")) 
    >> |     Uwe> (setq gnus-secondary-select-methods nil)
    >> |     Uwe> (setq gnus-secondary-select-methods
    >> |     Uwe>       '(
    >> |     Uwe> 	(nnimap "UCMgmail"
    >> |     Uwe> 		(nnimap-address "imap.gmail.com")
    >> |     Uwe> 		(nnimap-server-port 993)
    >> |     Uwe>                 ;; (nnimap-authinfo-file "~/.authinfo.gpg")
    >> |     Uwe> 		(nnimap-authinfo-file "~/.authinfo")
    >> |     Uwe> 		(nnimap-stream ssl)
    >> |     Uwe>                 ;;(nnimap-stream starttls)
    >> |     Uwe>                 (nnimap-fetch-partial-articles "text/")
    >> |     Uwe> 		(nnir-search-engine imap))))
    >> | 
    >> |     Uwe> And put in authinfo my app password instead my «normal» one?
    >> | 
    >> | Yes
    >> | 
    >> | Robert
    >> `----

    >> Be it as it may, it *does not work* 

    Uwe> Well, you are not going to belive, it but after makeing the following minor change:

    Uwe> machine smtp.gmail.com login oub.oub.oub@gmail.com port 587 password  XXXXX
    Uwe> machine imap.gmail.com login oub.oub.oub password XXXX port imaps

You made the 'machine' value match your nnimap-address? That seems
sensible.

    Uwe> After  *restarting emacs*, 

    Uwe> I can now again access my private gmail account, 

    Uwe> Sorry for the noise.

    Uwe> May I suggest to add a section to the authinfo info file, describing the
    Uwe> use of gmail app-passwords.

It sounds more like we need to document better that .authinfo entries
can sometimes be cached, so `auth-source-forget-all-cached' might need
to be called when messing with it (itʼs currently described in the
'help for developers' section).

    Uwe> If you like I can give it a try (I would send you my proposal off list)
    Uwe> and you can decide for yourself.

    Uwe> Sometimes a novice (like me concerning gmail app passwords) might mention
    Uwe> things that seem trivial to the expert.

Itʼs not that theyʼre trivial or non-trivial: changing .authinfo
happens rarely enough that any such caching issues are not obvious.

Robert
-- 

Re: [SOLVED (magic?)]

[Uwe Brauer]

[-- Attachment #1: Type: text/plain, Size: 1263 bytes --]




Uwe> Well, you are not going to belive, it but after makeing the following minor change:

Uwe> machine smtp.gmail.com login oub.oub.oub@gmail.com port 587 password  XXXXX
Uwe> machine imap.gmail.com login oub.oub.oub password XXXX port imaps

> You made the 'machine' value match your nnimap-address? That seems
> sensible.

Well this is not the end of the story, although it worked it screwed up
other gmail accounts I access. So the final solution was this 


machine gmail login oub.oub.oub@gmail.com password YYYY port imaps
machine smtp.gmail.com login oub.oub.oub@gmail.com port 587 password  YYYY
machine User2 login user2@gmail.com password XXXX port imaps
machine User2 login user2@gmail.com port 587 password  XXXX

And I am still not sure whether this is the best solution, since it
looks a bit incoherent.



> It sounds more like we need to document better that .authinfo entries
> can sometimes be cached, so `auth-source-forget-all-cached' might need
> to be called when messing with it (itʼs currently described in the
> 'help for developers' section).

Oh, I did not know even about the `auth-source-forget-all-cached'
function. Since I restarted emacs, may the cache was the source of my problems!?


Uwe 

[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 5673 bytes --]

Re: [SOLVED (magic?)]

[Robert Pluim]

>>>>> On Mon, 06 Jun 2022 08:49:26 +0200, Uwe Brauer <oub@mat.ucm.es> said:

    Uwe> Well, you are not going to belive, it but after makeing the following minor change:

    Uwe> machine smtp.gmail.com login oub.oub.oub@gmail.com port 587 password  XXXXX
    Uwe> machine imap.gmail.com login oub.oub.oub password XXXX port imaps

    >> You made the 'machine' value match your nnimap-address? That seems
    >> sensible.

    Uwe> Well this is not the end of the story, although it worked it screwed up
    Uwe> other gmail accounts I access. So the final solution was this 


    Uwe> machine gmail login oub.oub.oub@gmail.com password YYYY port imaps
    Uwe> machine smtp.gmail.com login oub.oub.oub@gmail.com port 587 password  YYYY
    Uwe> machine User2 login user2@gmail.com password XXXX port imaps
    Uwe> machine User2 login user2@gmail.com port 587 password  XXXX

    Uwe> And I am still not sure whether this is the best solution, since it
    Uwe> looks a bit incoherent.

(info "(auth) Multiple GMail accounts with Gnus") recommends keying
off the name of the secondary server only, not the nnimap-address.

    >> It sounds more like we need to document better that .authinfo entries
    >> can sometimes be cached, so `auth-source-forget-all-cached' might need
    >> to be called when messing with it (itʼs currently described in the
    >> 'help for developers' section).

    Uwe> Oh, I did not know even about the `auth-source-forget-all-cached'
    Uwe> function. Since I restarted emacs, may the cache was the source of my problems!?

Itʼs possible. Would it have helped you if there had been a reference
to that part of the documentation somewhere else in the auth-source manual?

Robert
-- 

Re: [SOLVED (magic?)] (was: [app password does not work (at the moment)])

[Tomas Hlavaty]

On Thu 02 Jun 2022 at 17:37, Uwe Brauer <oub@mat.ucm.es> wrote:
> I can now again access my private gmail account,

I thought you were forced to use google by your university but I see
that that is not the case.  You have a choice.
https://www.stallman.org/google.html

Re: [SOLVED (magic?)] (was: [app password does not work (at the moment)])

[tomas]

[-- Attachment #1: Type: text/plain, Size: 793 bytes --]

On Mon, Jun 06, 2022 at 08:55:42PM +0200, Tomas Hlavaty wrote:
> On Thu 02 Jun 2022 at 17:37, Uwe Brauer <oub@mat.ucm.es> wrote:
> > I can now again access my private gmail account,
> 
> I thought you were forced to use google by your university but I see
> that that is not the case.  You have a choice.
> https://www.stallman.org/google.html

This choice would entail losing his job?

Look. I really hate Google. I can afford to avoid them fairly thoroughly [1].
But perhaps the choice for other people isn't that easy. Expecting others
to play heroes might not be... friendly. Instead, you could try to help
them to mitigate their situation as far as possible.

Cheers

[1] I still exchange mails with people on gmail, so they are collecting my
   data this way.
-- 
t

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: [SOLVED (magic?)] (was: [app password does not work (at the moment)])

[Tomas Hlavaty]

On Mon 06 Jun 2022 at 21:07, <tomas@tuxteam.de> wrote:
> On Mon, Jun 06, 2022 at 08:55:42PM +0200, Tomas Hlavaty wrote:
>> On Thu 02 Jun 2022 at 17:37, Uwe Brauer <oub@mat.ucm.es> wrote:
>> > I can now again access my private gmail account,
>> 
>> I thought you were forced to use google by your university but I see
>> that that is not the case.  You have a choice.
>> https://www.stallman.org/google.html
>
> This choice would entail losing his job?

Why would he lose his job?

Re: [SOLVED (magic?)] (was: [app password does not work (at the moment)])

[tomas]

[-- Attachment #1: Type: text/plain, Size: 1732 bytes --]

On Mon, Jun 06, 2022 at 09:37:03PM +0200, Tomas Hlavaty wrote:
> On Mon 06 Jun 2022 at 21:07, <tomas@tuxteam.de> wrote:
> > On Mon, Jun 06, 2022 at 08:55:42PM +0200, Tomas Hlavaty wrote:
> >> On Thu 02 Jun 2022 at 17:37, Uwe Brauer <oub@mat.ucm.es> wrote:
> >> > I can now again access my private gmail account,
> >> 
> >> I thought you were forced to use google by your university but I see
> >> that that is not the case.  You have a choice.
> >> https://www.stallman.org/google.html
> >
> > This choice would entail losing his job?
> 
> Why would he lose his job?

Because his employer insists on handling job-related mails via gmail?

See, i've seen several employers: most of them want to keep control
of their work related mails in some way.

The "tradidional" way of handling this is by setting up their own
mail infrastructore (whatever that means, the spectrum covers some
space, from "just" setting up an SMTP server to offering (voluntary
or enforced) some horrible web mail service; some allow forwarding
to user chosen infrastructure, some don't).

The gist is that they want some accountability of important mails.

Anyway, the next step is for those employers to fall for the cloud
fallacy: outsource above mentioned infrastructure. Still, the condition
stands. Either you take part of that infrastructure or you go look
for another job [1].

Showing up at the office from time to time isn't optional either,
for many jobs after all.

Cheers

[1] No, this doesn't justify forcing people to use gmail, and I
   know that I'd raise a stink if my employer wanted me to do
   that. But I'd *never* try to tell other people how much of
   a hero they are expected to be.

-- 
t

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: [SOLVED (magic?)] (was: [app password does not work (at the moment)])

[Tomas Hlavaty]

On Tue 07 Jun 2022 at 06:35, tomas@tuxteam.de wrote:
> On Mon, Jun 06, 2022 at 09:37:03PM +0200, Tomas Hlavaty wrote:
>> On Mon 06 Jun 2022 at 21:07, <tomas@tuxteam.de> wrote:
>> > On Mon, Jun 06, 2022 at 08:55:42PM +0200, Tomas Hlavaty wrote:
>> >> On Thu 02 Jun 2022 at 17:37, Uwe Brauer <oub@mat.ucm.es> wrote:
>> >> > I can now again access my private gmail account,
>> >> 
>> >> I thought you were forced to use google by your university but I see
>> >> that that is not the case.  You have a choice.
>> >> https://www.stallman.org/google.html
>> >
>> > This choice would entail losing his job?
>> 
>> Why would he lose his job?
>
> Because his employer insists on handling job-related mails via gmail?
>
> See, i've seen several employers: most of them want to keep control
> of their work related mails in some way.

He said "private":

    I can now again access my private gmail account,

In civilised world, employers do not dictate brands for private stuff.
One has a choice.

[Clarification] (was: [SOLVED (magic?)])

[Uwe Brauer]

[-- Attachment #1: Type: text/plain, Size: 2838 bytes --]


> On Tue 07 Jun 2022 at 06:35, tomas@tuxteam.de wrote:

> He said "private":

>     I can now again access my private gmail account,

> In civilised world, employers do not dictate brands for private stuff.
> One has a choice.

Ok before this discussion is going to escalate, some clarifications

    1. Since some 10 years or so, my university uses gmail. That is a
       fact. Now I did not really care, because we had plenty of disk
       space out of a sudden and I used free software to access my
       account (either gnus or thunderbird/seamonkey). However, Tim
       Cross pointed out, this is not "really" a gmail account, more a
       service google provides for academic institutions. Now the
       university requires me to send official email, using the official
       "from" as provided my the university. There might be tricks to
       use other SMTP servers but as other, I think Tim Cross being one
       of them pointed out, users pointed out, this approach run into
       problems concerning SPAM filters.

       The information policy of my university concerning technical
       details about the email accounts is murky at best. However they
       told us, that we need to switch to 2 Factor identification and
       *maybe* cannot access anymore imap and smtp the «usual» way. Till
       now this did not happen, but it might in any moment. Fortunately
       as Tim and other pointed out, I can use, what google calls a app
       password that I have to generate. I find this a bizarre
       design/security decision since this password is considerably
       shorter than my original imaps/smtps password. Be it as it may
       that seems to be a working  alternative and allows me to continue
       to use gnus.

    2. I do indeed also posses a private gmail account (actually 2, one
       of them is very old, the other I used just for the notify
       extension of mercurial to send notifications to my collaborates
       when pushing to public repository). Again I did not really care
       about the things RMS mentions correctly on this web page, because
       they don't apply to me, I still access my email using free
       software and I encrypt my private mails so that google cannot
       scan them. Now, since 1 of June indeed one cannot access anymore
       the private accounts (at least I cannot) via imaps/smtps and this
       is why I was worried and wrote my email. The app password
       approach however works. Now, if google decides to discontinue its
       app password approach and oauth2 is not going to work, in that
       case I have to switch to any other provider, because I need to
       access my email with free software.

I hope to have clarified any misunderstandings

Regards

Uwe Brauer

[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 5673 bytes --]

Re: [Clarification] (was: [SOLVED (magic?)])

[Yuri Khan]

On Tue, 7 Jun 2022 at 14:16, Uwe Brauer <oub@mat.ucm.es> wrote:

>        as Tim and other pointed out, I can use, what google calls a app
>        password that I have to generate. I find this a bizarre
>        design/security decision since this password is considerably
>        shorter than my original imaps/smtps password.

I can try to explain the idea of app passwords, and then maybe they
will not seem as bizarre to you.

What we start with is a single Google account, with a single password,
and all client applications using this password. Easy to configure,
bad for security: most users will choose a weak password and store it
in many configuration points, and if it leaks or is stolen from any of
those, the whole account is compromised. The attacker can use your
master password to log in and change your password, and then you are
locked out.

On the other hand, with app passwords, each password is constrained to
a single client application. You generate a password for your email
client and it can connect with that password. If that password gets
stolen, the attacker has temporary access to your data. They cannot
change your password and lock you out. When you find out, you revoke
the leaked password and generate a new one, and then the attacker is
locked out and your account is no longer compromised.

Re: [Clarification]

[Uwe Brauer]

[-- Attachment #1: Type: text/plain, Size: 1522 bytes --]

>>> "YK" == Yuri Khan <yuri.v.khan@gmail.com> writes:

> On Tue, 7 Jun 2022 at 14:16, Uwe Brauer <oub@mat.ucm.es> wrote:
>> as Tim and other pointed out, I can use, what google calls a app
>> password that I have to generate. I find this a bizarre
>> design/security decision since this password is considerably
>> shorter than my original imaps/smtps password.

> I can try to explain the idea of app passwords, and then maybe they
> will not seem as bizarre to you.

> What we start with is a single Google account, with a single password,
> and all client applications using this password. Easy to configure,
> bad for security: most users will choose a weak password and store it
> in many configuration points, and if it leaks or is stolen from any of
> those, the whole account is compromised. The attacker can use your
> master password to log in and change your password, and then you are
> locked out.

> On the other hand, with app passwords, each password is constrained to
> a single client application. You generate a password for your email
> client and it can connect with that password. If that password gets
> stolen, the attacker has temporary access to your data. They cannot
> change your password and lock you out. When you find out, you revoke
> the leaked password and generate a new one, and then the attacker is
> locked out and your account is no longer compromised.


I see point taken, thanks.

However for my taste they are too short, I'd prefer them at least to
have 32 chars or better more.

[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 5673 bytes --]

Re: [SOLVED (magic?)] (was: [app password does not work (at the moment)])

[tomas]

[-- Attachment #1: Type: text/plain, Size: 450 bytes --]

On Tue, Jun 07, 2022 at 07:52:40AM +0200, Tomas Hlavaty wrote:

[...]

> > See, i've seen several employers: most of them want to keep control
> > of their work related mails in some way.
> 
> He said "private":
> 
>     I can now again access my private gmail account,

Ah, I must have overlooked that.

> In civilised world, employers do not dictate brands for private stuff.
> One has a choice.

Yes, of course.

Cheers
-- 
t

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: [SOLVED (magic?)] (was: [app password does not work (at the moment)])

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > But I'd *never* try to tell other people how much of
  >    a hero they are expected to be.

I agree that we shoudn't _demand_ that others be heroic, but we should
continue to encourage resistance.  Let's watch out for the danger of
agreeing with those who disparage the idea of resistance.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: [SOLVED (magic?)]

[Byung-Hee HWANG]

Dear tomas,

<tomas@tuxteam.de> writes:

> (... thanks ...)
> [1] I still exchange mails with people on gmail, so they are collecting my
>    data this way.

In real life, stealing personal information is common in South Korea
[*]. No one will claim it. So Google's data collection is not bad thing
in here South Korea, i think.

(this is only personal opnion) 

[*] 2022-06-07: <<toss>> and customer's data
https://www.hankyung.com/economy/article/2022060762846

Sincerely, Gnus fan Byung-Hee

-- 
^고맙습니다 _地平天成_ 감사합니다_^))//

Re: [SOLVED (magic?)]

[Tomas Hlavaty]

On Tue 07 Jun 2022 at 14:44, Byung-Hee HWANG <soyeomul@doraji.xyz> wrote:
> In real life, stealing personal information is common in South Korea
> [*]. No one will claim it. So Google's data collection is not bad thing
> in here South Korea, i think.
>
> (this is only personal opnion) 
>
> [*] 2022-06-07: <<toss>> and customer's data
> https://www.hankyung.com/economy/article/2022060762846

In Berlin, bicycle theft is common.
By your logic, it is not a bad thing?
On the other hand, from your logic, I can see why
"stealing personal information is common in South Korea".

Re: [SOLVED (magic?)]

[tomas]

[-- Attachment #1: Type: text/plain, Size: 1213 bytes --]

On Tue, Jun 07, 2022 at 08:04:45AM +0200, Tomas Hlavaty wrote:
> On Tue 07 Jun 2022 at 14:44, Byung-Hee HWANG <soyeomul@doraji.xyz> wrote:
> > In real life, stealing personal information is common in South Korea
> > [*]. No one will claim it. So Google's data collection is not bad thing
> > in here South Korea, i think.
> >
> > (this is only personal opnion) 
> >
> > [*] 2022-06-07: <<toss>> and customer's data
> > https://www.hankyung.com/economy/article/2022060762846
> 
> In Berlin, bicycle theft is common.
> By your logic, it is not a bad thing?
> On the other hand, from your logic, I can see why
> "stealing personal information is common in South Korea".

This is disingenuous. "Stealing" information is very different from
stealing a physical object. In the latter case the owner loses access
to it.

You are reproducing the content industry's worst lies without knowing
it. Intellectual property is intellectual poverty :)

That doesn't mean I'm fine with google sticking its dirty nose into
my private affairs, of course.

That said, having lived in Berlin for ten years, my bicycle wasn't
stolen once. In my time before, I lost like five of them!

Cheers
-- 
t
> 

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: [SOLVED (magic?)]

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > You are reproducing the content [sic] industry's worst lies without knowing
  > it. Intellectual property [sic] is intellectual poverty :)

It's much worse than that!  "Intellectual property" is a confused
overgeneralization -- it misrepresents reality and spreads confusion,
by lumping together laws that we should rigorously treat as separate
and unrelated.

Please see https://gnu.org/philosophy/not-ipr.html and join me in
rejecting that term.

I suggest rejecting "content," too.
See https://gnu.org/philosophy/words-to-avoid.html.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: [SOLVED (magic?)]

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> Cc: emacs-devel@gnu.org
> Date: Thu, 09 Jun 2022 18:29:39 -0400
> 
>   > You are reproducing the content [sic] industry's worst lies without knowing
>   > it. Intellectual property [sic] is intellectual poverty :)
> 
> It's much worse than that!  "Intellectual property" is a confused
> overgeneralization -- it misrepresents reality and spreads confusion,
> by lumping together laws that we should rigorously treat as separate
> and unrelated.
> 
> Please see https://gnu.org/philosophy/not-ipr.html and join me in
> rejecting that term.

That article talks about "intellectual property" in relation to laws
that (allegedly) govern it.  While "intellectual property" is many
times abused by establishments with some vested interests, the term
also has a meaning which has nothing to do with laws, copyright,
trademarks, patents and the like: when it is used to refer to ideas
that constitute trade secrets of some organization, which uses these
ideas and secrets to get a competitive edge in its products and
business transactions.  There's (AFAIK; IANAL) no laws covering this,
but companies will generally guard those secrets very closely and
refrain from disclosing them, definitely not to their competitors.

P.S. This discussion doesn't really belong to emacs-devel.

Re: [SOLVED (magic?)]

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

Trade secrecy is one of the several forms of power that
the overgeneralization "intellectual property" is said to include.
As you said, it is very different from all the others.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: [SOLVED (magic?)]

[tomas]

[-- Attachment #1: Type: text/plain, Size: 1610 bytes --]

On Sat, Jun 11, 2022 at 08:44:28PM -0400, Richard Stallman wrote:
> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
> 
> Trade secrecy is one of the several forms of power that
> the overgeneralization "intellectual property" is said to include.
> As you said, it is very different from all the others.

As I see it, "intellectual property" is an umbrella concept clumping
together unrelated things (patents, copyrights, trade secrets, brand
rights) which may or may not make sense in the context.

In my eyes, it's desperate capitalism: available land has stopped
"growing" (it did between the 13th and the 18th century, in some
pretty perverse way [1]). But capitalism wants growth, so it needs
something else to grow into.

Now you can produce virtulally (really!) unlimited amounts of videos
on Youtube [2], so there's your growth.

Until people realise that those videos aren't any worth unless there
are other people willing to watch them.

It's going to be a difficult adaptation phase for us. I don't even
know whether we'll survive that. But knowing a lot of young and
optimistic people I think I haven't the moral right to not try.

Now let me get down from my soapbox :)

Cheers

[1] it involved exterminating people and other living creatures,
   but somehow we managed to "forget" about that

[2] this is, of course, just a placeholder for intellectual
   property. Biased? Yes :)

-- 
t

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: [SOLVED (magic?)]

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > As I see it, "intellectual property" is an umbrella concept clumping
  > together unrelated things (patents, copyrights, trade secrets, brand
  > rights) which may or may not make sense in the context.

That is true.  However, that practic of lumping them together
causes fundamental confusion, and it is very harmful.  We need to
oppose lumping them together.

https://gnu.org/philosophy/not-ipr.html explains this.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: [SOLVED (magic?)]

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > In real life, stealing personal information is common in South Korea
  > [*]. No one will claim it. So Google's data collection is not bad thing
  > in here South Korea, i think.

I don't follow the reasoning here.  If harmful use of other people's
personal information is rife, that doesn't make it ok.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: [SOLVED (magic?)] (was: [app password does not work (at the moment)])

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Look. I really hate Google. I can afford to avoid them fairly thoroughly [1].
  > But perhaps the choice for other people isn't that easy. Expecting others
  > to play heroes might not be... friendly. Instead, you could try to help
  > them to mitigate their situation as far as possible.

I agree with that.  I encourage people to be small-h heroic
by setting an example -- not by rebuking them if the don't.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: gmail+imap+smtp (oauth2)

[Tim Cross]

Uwe Brauer <oub@mat.ucm.es> writes:

> And put in authinfo my app password instead my «normal» one?

Yep, that is all you need to do - replace your 'normal' google password
with an app password and your set to go. All you have to do is generate
the app password, which is done in the settings section of your google
account. 

I don't use gnus, butr rather mu4e and mbsync. However, the principals
are the same - I have my SMTP entry in authinfo.gpg, which uses my
google username and app password. For mbsync, I have a gpg file with my
app password in it and use that as part of my .mbsyncrc. 

Re: gmail+imap+smtp (oauth2)

[Filipp Gunbin]

On 05/05/2022 14:57 +0200, Uwe Brauer wrote:

> Hm, but did you successfully configure gnus with your  app password in order to
> receive (imap) and to send (smtp).
>
> If so, could you please share your configuration.
>
> Or are you saying it is simply to use
>
> (setq gnus-select-method (list 'nnnil "")) 
> (setq gnus-secondary-select-methods nil)
> (setq gnus-secondary-select-methods
>       '(
> 	(nnimap "UCMgmail"
> 		(nnimap-address "imap.gmail.com")
> 		(nnimap-server-port 993)
>                 ;; (nnimap-authinfo-file "~/.authinfo.gpg")
> 		(nnimap-authinfo-file "~/.authinfo")
> 		(nnimap-stream ssl)
>                 ;;(nnimap-stream starttls)
>                 (nnimap-fetch-partial-articles "text/")
> 		(nnir-search-engine imap))))
>
> And put in authinfo my app password instead my «normal» one?

Just FTR, with Outlook from Office365 (we had that at work), it is just
that - you create an app password in web interface (yes, non-free JS),
and then just put it in authinfo instead of your "account password".

Filipp

Re: gmail+imap+smtp (oauth2)

[Jorge A. Alfaro-Murillo]

On Thu, May 05 2022, Filipp Gunbin wrote:

>
> Just FTR, with Outlook from Office365 (we had that at work), it 
> is just that - you create an app password in web interface (yes, 
> non-free JS), and then just put it in authinfo instead of your 
> "account password".

Just to let you know that (sadly) we have Office365 at my 
institution but that they do not allow app passwords. It is 
something that IT has to allow from the Office 365 Admin 
Center. When I called them about it, they told me that they were 
only supporting email clients that had 2-factor authentication.

I haven't been able to use gnus with my work email (@yale.edu) 
since then. I wonder if the same is true for other institutions 
that use Google Workspace.

FYI, two free open-source email projects thunderbird (MPL-2.0) and 
fairmail (GPL3) work with 2-factor authentication. Is there anyway 
to use their method of authentication in gnus?
-- 
Jorge.

Re: gmail+imap+smtp (oauth2)

[Thomas Fitzsimmons]

Hi Jorge,

"Jorge A. Alfaro-Murillo" <jorge@democraciareal.org> writes:

> On Thu, May 05 2022, Filipp Gunbin wrote:
>
>> Just FTR, with Outlook from Office365 (we had that at work), it is
>> just that - you create an app password in web interface (yes,
>> non-free JS), and then just put it in authinfo instead of your
>> "account password".
>
> Just to let you know that (sadly) we have Office365 at my institution
> but that they do not allow app passwords. It is something that IT has
> to allow from the Office 365 Admin Center. When I called them about
> it, they told me that they were only supporting email clients that had
> 2-factor authentication.
>
> I haven't been able to use gnus with my work email (@yale.edu) since
> then. I wonder if the same is true for other institutions that use
> Google Workspace.
>
> FYI, two free open-source email projects thunderbird (MPL-2.0) and
> fairmail (GPL3) work with 2-factor authentication. Is there anyway to
> use their method of authentication in gnus?

In my case I had to request that my Microsoft Office 365 administrators
leave IMAP and SMTP enabled.  Apparently each of these protocols can be
enabled or disabled for individual users, groups, etc., or
organization-wide.  (This is another variable that frustrates the
writing of a generic set of instructions for "Using Emacs with MSO365".)
This setting is orthogonal to whether Microsoft-Office-365-OAuth-2.0 is
required for each of the protocols, as far as I can tell.

Assuming you can have your administrators enable IMAP and SMTP for your
accounts, and assuming that application passwords are not enabled, using
Gnus for email is still achievable in my experience.

Ideally I would like to use the Emacs IMAP and SMTP implementations,
with the oauth2 package from GNU ELPA.  However, I haven't got there
yet.  I haven't yet tackled whether oauth2.el supports
Microsoft-Office-365's-take-on-OAuth-2.0-as-configured-by-my-organization
(there should probably be a standard for naming OAuth 2.0 implementation
and configuration variants; you know, to limit confusion).

However, an interim solution I've landed on is to use Gnus's nnmaildir
backend for reading a local archive of my IMAP emails, smtpmail for
SMTP, and mbsync for synchronizing between the IMAP server and the local
archive.

Microsoft-Office-365-OAuth-2.0 is handled by a script, oauth2ms, which
smtpmail and mbsync call out to:

    https://github.com/harishkrupo/oauth2ms

Besides providing the script, the oauth2ms repository has thorough
documentation that explains all the Emacs-vs-MSO365 considerations
better than any other resource I've seen.

Thomas

Re: gmail+imap+smtp (oauth2)

[Tim Cross]

"Jorge A. Alfaro-Murillo" <jorge@democraciareal.org> writes:

> On Thu, May 05 2022, Filipp Gunbin wrote:
>
>>
>> Just FTR, with Outlook from Office365 (we had that at work), it is just that -
>> you create an app password in web interface (yes, non-free JS), and then just
>> put it in authinfo instead of your "account password".
>
> Just to let you know that (sadly) we have Office365 at my institution but that
> they do not allow app passwords. It is something that IT has to allow from the
> Office 365 Admin Center. When I called them about it, they told me that they
> were only supporting email clients that had 2-factor authentication.
>
> I haven't been able to use gnus with my work email (@yale.edu) since then. I
> wonder if the same is true for other institutions that use Google Workspace.
>

Yes, this is an issue for institutions that use Google or Office365. The
ability to use app passwords is a configuration option for the
institution. Both Google and MS will recommend against enabling that
option. Far too many IT departmenbts in large institutions will just
follow Google/MS advice because they don't understand the issues and
because they are not prepared to stick the neck out and go against
Google/MS recommendations. 

> FYI, two free open-source email projects thunderbird (MPL-2.0) and fairmail
> (GPL3) work with 2-factor authentication. Is there anyway to use their method of
> authentication in gnus?

From what I've read, it is suggested both these projects are not fully
compliant with the T&C of Google/MS. This is something some people have
attempted to get clarified, but fail to get any response. I think this
is precisely the area where the FSF could assist as they might be able
to at least get the issue looked at by senior enough Google/MS
executives to get a definitive answer. 

As I understand it, the key issue regards the application ID. Google's
T&C imply this must be kept secret (it is an ID assigned by Google once
your application has been approved and is used in the code). Problem
being, how can you have that ID be in the code and be secret. 

Re: gmail+imap+smtp (oauth2)

[Tomas Hlavaty]

On Fri 06 May 2022 at 10:43, Tim Cross <theophilusx@gmail.com> wrote:
> As I understand it, the key issue regards the application ID. Google's
> T&C imply this must be kept secret (it is an ID assigned by Google once
> your application has been approved and is used in the code). Problem
> being, how can you have that ID be in the code and be secret. 

Is the application id created by google when the oauth2 is configured by
a university?

Re: gmail+imap+smtp (oauth2)

[Tim Cross]

Tomas Hlavaty <tom@logand.com> writes:

> On Fri 06 May 2022 at 10:43, Tim Cross <theophilusx@gmail.com> wrote:
>> As I understand it, the key issue regards the application ID. Google's
>> T&C imply this must be kept secret (it is an ID assigned by Google once
>> your application has been approved and is used in the code). Problem
>> being, how can you have that ID be in the code and be secret. 
>
> Is the application id created by google when the oauth2 is configured by
> a university?

No. The application ID is provided by Google once the application has
been approved by them. The flow sort of goes 

1. Register wiht google as a developer. This gives you a developer ID
which yu can use as an application ID. 
2. Develop your application which uses oath2 to connect to google. 
3. Submit your application for approval by google.
4. Once approved, Google gives you an application ID which is used by
your application.
5. Release your application

Problem is, Google T&C require that the application ID is kept secret.
For open source, this is a problem because we cannot add the applicaiton
ID and keep it secret while making the code open source. 

Re: gmail+imap+smtp (oauth2)

[Stefan Monnier]

> Problem is, Google T&C require that the application ID is kept secret.
> For open source, this is a problem because we cannot add the applicaiton
> ID and keep it secret while making the code open source.

FWIW, it's also a problem for proprietary applications since the secret
will necessarily be somewhere inside the executable as well.  It's a bit
harder to find, and can be obfuscated to some extent, but as long as you
can run the code inside a debugger and you have enough time on your
hands to reverse engineer the workings of that part of the code you can
also extract the application ID.


        Stefan

Re: gmail+imap+smtp (oauth2)

[tomas]

[-- Attachment #1: Type: text/plain, Size: 1101 bytes --]

On Fri, May 06, 2022 at 07:38:12AM -0400, Stefan Monnier wrote:
> > Problem is, Google T&C require that the application ID is kept secret.
> > For open source, this is a problem because we cannot add the applicaiton
> > ID and keep it secret while making the code open source.
> 
> FWIW, it's also a problem for proprietary applications since the secret
> will necessarily be somewhere inside the executable as well.  It's a bit
> harder to find, and can be obfuscated to some extent, but as long as you
> can run the code inside a debugger and you have enough time on your
> hands to reverse engineer the workings of that part of the code you can
> also extract the application ID.

We know. They know. We know they know. They know we know they know.
We've been down this drm-keys-embedded-in-application tango so many
times already-

Now someone explain to me: why do they nevertheless do it?

Is it for having some T&C where they basically say "we'll
kick you out whenever we think we want to"?

I'm hard-pressed to come up with a friendlier interpretation.

Cheers
-- 
t

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: gmail+imap+smtp (oauth2)

[Lars Ingebrigtsen]

<tomas@tuxteam.de> writes:

> Now someone explain to me: why do they nevertheless do it?
>
> Is it for having some T&C where they basically say "we'll
> kick you out whenever we think we want to"?

Yes.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: gmail+imap+smtp (oauth2)

[Stefan Monnier]

> We know.  They know.  We know they know.  They know we know they know.
> We've been down this drm-keys-embedded-in-application tango so many
> times already-
>
> Now someone explain to me: why do they nevertheless do it?

Regardless of this "why", I think what it means is that it's perfectly
OK to use it in Free Software, with the "secret" key in plain sight.

I think from Google's point of view all it means is that we're more
prone to DoS attacks where someone abuses the "secret" key causing
Google to revoke the key.  Google won't suffer from it, we will, so they
don't care.


        Stefan

Re: gmail+imap+smtp (oauth2)

[Tim Cross]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> We know.  They know.  We know they know.  They know we know they know.
>> We've been down this drm-keys-embedded-in-application tango so many
>> times already-
>>
>> Now someone explain to me: why do they nevertheless do it?
>
> Regardless of this "why", I think what it means is that it's perfectly
> OK to use it in Free Software, with the "secret" key in plain sight.
>
> I think from Google's point of view all it means is that we're more
> prone to DoS attacks where someone abuses the "secret" key causing
> Google to revoke the key.  Google won't suffer from it, we will, so they
> don't care.
>

Agree. I also suspect this is the same rationale being used by
Thunderbird and others. 

Re: gmail+imap+smtp (oauth2)

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Regardless of this "why", I think what it means is that it's perfectly
  > OK to use it in Free Software, with the "secret" key in plain sight.

  > I think from Google's point of view all it means is that we're more
  > prone to DoS attacks where someone abuses the "secret" key causing
  > Google to revoke the key.  Google won't suffer from it, we will, so they
  > don't care.

I think that is plausible.  However, the idea that these secret
application keys were designed for web dis-services, rather than for
programs you can install, seems plausible to me too.

Because a program and a service are fundamentally different, using a
single term (such as "app") for both of them creates devastating confusion.
When davmail was mentioned, I needed to know which one it was, and you
saw it was not easy to explain the question.

You can help educate people about ths difference.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

gmail+imap+smtp (davmail)

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

Can someone please establish practically what good davmail can do for
the task of using Gmail via Rmail or Gnus, without running any nonfree
software?

This is a purely practical question.  My hope is to find an answer
that is factual and certain, replacing all speculation with facts.

For this purpose, let's distinguish between (a) configuring the Gmail
account and (b) using that account to transfer mail.  We know that (a)
requires running nonfree JS code; the question is what we can do about
(b).

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: gmail+imap+smtp (davmail)

[Eric S Fraga]

On Wednesday, 11 May 2022 at 05:01, Richard Stallman wrote:
> Can someone please establish practically what good davmail can do for
> the task of using Gmail via Rmail or Gnus, without running any nonfree
> software?

davmail is only for Exchange (or other webdav services), as was pointed
out to me elsewhere in this thread.  It is not suitable for gmail, is my
understanding.

-- 
Eric S Fraga via gnus (Emacs 29.0.50 2022-05-10) on Debian 11.3

Re: gmail+imap+smtp (davmail)

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > davmail is only for Exchange (or other webdav services), as was pointed
  > out to me elsewhere in this thread.  It is not suitable for gmail, is my
  > understanding.

Oh well.

Thanks for explaining.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: gmail+imap+smtp (oauth2)

[Tim Cross]

<tomas@tuxteam.de> writes:

> [[PGP Signed Part:Undecided]]
> On Fri, May 06, 2022 at 07:38:12AM -0400, Stefan Monnier wrote:
>> > Problem is, Google T&C require that the application ID is kept secret.
>> > For open source, this is a problem because we cannot add the applicaiton
>> > ID and keep it secret while making the code open source.
>> 
>> FWIW, it's also a problem for proprietary applications since the secret
>> will necessarily be somewhere inside the executable as well.  It's a bit
>> harder to find, and can be obfuscated to some extent, but as long as you
>> can run the code inside a debugger and you have enough time on your
>> hands to reverse engineer the workings of that part of the code you can
>> also extract the application ID.
>
> We know. They know. We know they know. They know we know they know.
> We've been down this drm-keys-embedded-in-application tango so many
> times already-
>
> Now someone explain to me: why do they nevertheless do it?
>
> Is it for having some T&C where they basically say "we'll
> kick you out whenever we think we want to"?
>
> I'm hard-pressed to come up with a friendlier interpretation.
>

Problem is, your looking for a rational reason. The reality is T&C are
typically written by non-technical people (often lawyers) who don't
really understand (Or care to understand) the finer points and
implications. They work in broad brushstrokes and plan to adjust/change
if there is enough stink. HOwever, in this case, the percentage of
voerall users impacted is unlikely to even register a a blip o thier
radar. 

Bootm line, we are being done in by bureaucracy and bad communication. 

Re: gmail+imap+smtp (oauth2)

[Eric S Fraga]

On Friday,  6 May 2022 at 22:49, Tim Cross wrote:
> Bootm line, we are being done in by bureaucracy and bad communication. 

Story of my life (currently). ;-)

-- 
Eric S Fraga via gnus (Emacs 29.0.50 2022-05-03) on Debian 11.3

Re: gmail+imap+smtp (oauth2)

[tomas]

[-- Attachment #1: Type: text/plain, Size: 1275 bytes --]

On Fri, May 06, 2022 at 10:49:15PM +1000, Tim Cross wrote:
> 
> <tomas@tuxteam.de> writes:

[...]

> > Is it for having some T&C where they basically say "we'll
> > kick you out whenever we think we want to"?
> >
> > I'm hard-pressed to come up with a friendlier interpretation.
> >
> 
> Problem is, your looking for a rational reason. The reality is T&C are
> typically written by non-technical people (often lawyers) who don't
> really understand (Or care to understand) the finer points and
> implications.

...but who, working for Google, should have ready access to some who
do.

>                They work in broad brushstrokes and plan to adjust/change
> if there is enough stink. HOwever, in this case, the percentage of
> voerall users impacted is unlikely to even register a a blip o thier
> radar. 
> 
> Bootm line, we are being done in by bureaucracy and bad communication. 

OK, you came up with a friendlier explanation. You seem to be
a friendlier person than me :-)

I still tend to not quite believe that. To me, this is an instance
of "any sufficiently advanced form of malice is indistinguishable
from stupidity" [1]. IOW, plausible deniability.

Cheers

[1] A CRISPR monster between Clarke's Third and Hanlon.
-- 
t

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: gmail+imap+smtp (oauth2)

[Tim Cross]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> Problem is, Google T&C require that the application ID is kept secret.
>> For open source, this is a problem because we cannot add the applicaiton
>> ID and keep it secret while making the code open source.
>
> FWIW, it's also a problem for proprietary applications since the secret
> will necessarily be somewhere inside the executable as well.  It's a bit
> harder to find, and can be obfuscated to some extent, but as long as you
> can run the code inside a debugger and you have enough time on your
> hands to reverse engineer the workings of that part of the code you can
> also extract the application ID.
>

Yes, that is a flaw. However, requiring the application ID to be kept
secret is really the error - it isn't necessary and doesn't improve the
security. From what I've read, it was never the intention of the
designers of oauth that this value be kept secret. It really exists
mainly as an auditing/debugging/troublshooting aid, not part of the
authn/authz process. 

I think this is why some people are trying to get clarification from
Google as it is likely their reference to what must be kept secret only
includes the applicaiton ID by error/oversight. (I was told this
confusion originally occured because of ambiguity in the original oauth
documentation, which has subsequently been fixed/clarified). Problem is,
most users cannot get past the lower level helpdesk staff or get their
issue in front of someone who can actually look at it and do something
and even if you could, getting them to care enough to do something is
unlikely - the percentage of users impacted is likley just too small
compared to other issues they are also dealing with. 

Re: gmail+imap+smtp (oauth2)

[Tim Cross]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> Problem is, Google T&C require that the application ID is kept secret.
>> For open source, this is a problem because we cannot add the applicaiton
>> ID and keep it secret while making the code open source.
>
> FWIW, it's also a problem for proprietary applications since the secret
> will necessarily be somewhere inside the executable as well.  It's a bit
> harder to find, and can be obfuscated to some extent, but as long as you
> can run the code inside a debugger and you have enough time on your
> hands to reverse engineer the workings of that part of the code you can
> also extract the application ID.
>

Yes, that is a flaw. However, requiring the application ID to be kept
secret is really the error - it isn't necessary and doesn't improve the
security. From what I've read, it was never the intention of the
designers of oauth that this value be kept secret. It really exists
mainly as an auditing/debugging/troublshooting aid, not part of the
authn/authz process. 

I think this is why some people are trying to get clarification from
Google as it is likely their reference to what must be kept secret only
includes the applicaiton ID by error/oversight. (I was told this
confusion originally occured because of ambiguity in the original oauth
documentation, which has subsequently been fixed/clarified). Problem is,
most users cannot get past the lower level helpdesk staff or get their
issue in front of someone who can actually look at it and do something
and even if you could, getting them to care enough to do something is
unlikely - the percentage of users impacted is likley just too small
compared to other issues they are also dealing with. 

Re: gmail+imap+smtp (oauth2)

[Tomas Hlavaty]

On Fri 06 May 2022 at 22:34, Tim Cross <theophilusx@gmail.com> wrote:
> Yes, that is a flaw. However, requiring the application ID to be kept
> secret is really the error - it isn't necessary and doesn't improve the
> security. From what I've read, it was never the intention of the
> designers of oauth that this value be kept secret.

the intention is mentioned on their website:

https://www.oauth.com/oauth2-servers/client-registration/client-id-secret/

   The client_id is a public identifier for apps. Even though it’s
   public, it’s best that it isn’t guessable by third parties, so many
   implementations use something like a 32-character hex string. If the
   client ID is guessable, it makes it slightly easier to craft phishing
   attacks against arbitrary applications.

people here think about it in terms of programs
but if you think about it in terms of services, this issue disappears
it looks like the authors of oauth2 had services in mind

Re: gmail+imap+smtp (oauth2)

[Tomas Hlavaty]

On Fri 06 May 2022 at 07:38, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
>> Problem is, Google T&C require that the application ID is kept secret.
>> For open source, this is a problem because we cannot add the applicaiton
>> ID and keep it secret while making the code open source.
> FWIW, it's also a problem for proprietary applications since the secret
> will necessarily be somewhere inside the executable as well.

it is not a problem for proprietary services

Re: gmail+imap+smtp (oauth2)

[Tomas Hlavaty]

On Fri 06 May 2022 at 19:04, Tim Cross <theophilusx@gmail.com> wrote:
>> Is the application id created by google when the oauth2 is configured by
>> a university?
>
> No. The application ID is provided by Google once the application has
> been approved by them.

does application id mean client_id?

> The flow sort of goes 
>
> 1. Register wiht google as a developer. This gives you a developer ID
> which yu can use as an application ID. 
> 2. Develop your application which uses oath2 to connect to google. 
> 3. Submit your application for approval by google.
> 4. Once approved, Google gives you an application ID which is used by
> your application.
> 5. Release your application

understand

and where does the university, which uses goggle mail to which a student
or teacher connects, fit in this?
how does the university configure their mail?
or has the university no say in this at all?

> Problem is, Google T&C require that the application ID is kept secret.

it seems to be oauth2 thing, not google:

https://www.oauth.com/oauth2-servers/client-registration/client-id-secret/

   The client_id is a public identifier for apps. Even though it’s
   public, it’s best that it isn’t guessable by third parties, so many
   implementations use something like a 32-character hex string. If the
   client ID is guessable, it makes it slightly easier to craft phishing
   attacks against arbitrary applications.  It must also be unique
   across all clients that the authorization server handles.

   For each registered application, you’ll need to store the public
   client_id and the private client_secret. Because these are
   essentially equivalent to a username and password

oauth2 recommends keeping client_id (equivalent to username) secret
even though they call it a public identifier

> For open source, this is a problem because we cannot add the applicaiton
> ID and keep it secret while making the code open source. 

it seems to me that oauth2 protocol is not open at all
it might be open in a sense that anybody can read the spec and implement it
but not in a sense that anybody can read the spec, implement it and use it
(unlike other protocols like smtp or imap)
one of the features seems to be that there is a (usually extra) party with special role
having absolute authority about who to let through the gate

Re: gmail+imap+smtp (oauth2)

[Tim Cross]

Tomas Hlavaty <tom@logand.com> writes:

> On Fri 06 May 2022 at 19:04, Tim Cross <theophilusx@gmail.com> wrote:
>>> Is the application id created by google when the oauth2 is configured by
>>> a university?
>>
>> No. The application ID is provided by Google once the application has
>> been approved by them.
>
> does application id mean client_id?

No, At least, not necessarily and not on its own. It might form part of
the client ID. Note that the oath2 spec uses the term 'client' in at
least 3 different contexts. For example, the client is also the web
application running on the server, not simply the end-user's client
program. 

An implementaiton can define any set of 'attributes' it wants to use as
part of the authorization process. This can include an applicaiton id, a
user id (for end user clients) or any other data you might want in order
to assess an authorisation request (IP address, time of day, GEOIP data,
whatever). 

The client id being referred to on that site is mainly talking about a
client id which can be used to idnetify specific clients. What Google
wants with the application id is a way to do both analytics and a way to
have a big kill switch should they find a particular class of client has
soemthing suspect about it (e.g. source of DDoS attack, critical
security flaw, badly behaved/written app which is adversely consuming
resources etc). 
>
>> The flow sort of goes 
>>
>> 1. Register wiht google as a developer. This gives you a developer ID
>> which yu can use as an application ID. 
>> 2. Develop your application which uses oath2 to connect to google. 
>> 3. Submit your application for approval by google.
>> 4. Once approved, Google gives you an application ID which is used by
>> your application.
>> 5. Release your application
>
> understand
>
> and where does the university, which uses goggle mail to which a student
> or teacher connects, fit in this?
> how does the university configure their mail?
> or has the university no say in this at all?
>

No, the University has considerable say. Typically, it has control over
things like availability of POP3/IMAP, ability to forward mail to other
servers, whether 2FA is required or not, level/type of spam filtering
(warning, quarantine etc). Google (and same for MS and office365) will
provide defaults and recommendations - typically very conservative (no
POP/IMAP, 2FA, no app passwrods, no mail forwarding etc). Most IT
departments will just following these recommendations (mainly because
they want to be able to say, when things go wrong, they were following
Google/MS recommendations). 

>> Problem is, Google T&C require that the application ID is kept secret.
>
> it seems to be oauth2 thing, not google:
>
> https://www.oauth.com/oauth2-servers/client-registration/client-id-secret/
>
>    The client_id is a public identifier for apps. Even though it’s
>    public, it’s best that it isn’t guessable by third parties, so many
>    implementations use something like a 32-character hex string. If the
>    client ID is guessable, it makes it slightly easier to craft phishing
>    attacks against arbitrary applications.  It must also be unique
>    across all clients that the authorization server handles.
>
>    For each registered application, you’ll need to store the public
>    client_id and the private client_secret. Because these are
>    essentially equivalent to a username and password
>
> oauth2 recommends keeping client_id (equivalent to username) secret
> even though they call it a public identifier
>

Putting aside that applicaiton ID and client ID are not the same thing,
not the bit 

> It must also be unique across all clients that the authorization server handles.

which is confusing because the term client is used to mean different
things. This was exactly the sort of thing which led to the confusion
regarding client id. 

>> For open source, this is a problem because we cannot add the applicaiton
>> ID and keep it secret while making the code open source. 
>
> it seems to me that oauth2 protocol is not open at all
> it might be open in a sense that anybody can read the spec and implement it
> but not in a sense that anybody can read the spec, implement it and use it
> (unlike other protocols like smtp or imap)

I disagree. The spec is open and anyone can implmeent it in any language
they want and can license it according to any license they want.
Google's implementation may not be open, but someone else could just as
easily write an implementation which is. 

Note also that the actual specification is based on standard web
technologies and has no requirement for Javascript - all
communicaiton/negotiation can be done just using standard web form
encoded data. 

> one of the features seems to be that there is a (usually extra) party with special role
> having absolute authority about who to let through the gate

How is that any different to any other service (including SMTP). I
cannot just use any SMTP server (these days). I have to be authorised to
do so and the service provider I use has full authority to reject my
messages or to refuse acceptring any messages sent to my on that server. 

Even if you had a fully open source oauth2 implementaiton, there will
still be a party (resource owner) who controls access and
approves/rejects requests for access. 

The oauth2 standard is deliberately light on low level details. This
does mean that different implementations may not be interoperable with
each other (this is even outlined in the spec and is deliberate). It
isn't trying to define a networking protocol which is interoperable with
other sites like SMTP or SSH. However, that doesn't mean it isn't open.
It is a spec and it is an open spec. How individual sites decide to
implement that spec is up to the individual site.

Where the real power of oauth2 comes into play is with the separaton of
identity provider (IdP) and service provider (SP). It is this power
which has enabled "login wiht XXX" functionality to develop. What would
really make all of this work well is for some group (like the FSF or
EFF) to setup an ethical IdP which Google (and MS, Facebook, Github,
Gitlab etc) would trust to perform the client authorisation and provide
the client authorization token which you then use to get the access
token from Google. This woulud allow all of your sensitive identity
information to be managed by someone you have trust in. Google can already
do this sort of things for their enterprise customers (i.e. your home
institution manages your credentials, not google). However, IAM is hard
and most institutions are more than happy to allow Google (or MS Azure
or ...) to do this for them. 

Anyway, this has gotten way off topic, so I'll leave this thread alone
from here. I don't think I have anything else to add. 

Re: gmail+imap+smtp (oauth2)

[Stefan Monnier]

> Most IT departments will just following these recommendations (mainly
> because they want to be able to say, when things go wrong, they were
> following Google/MS recommendations). 

Indeed, most IT departments willing to stick their neck out will likely
opt for other providers anyway.


        Stefan

Re: gmail+imap+smtp (oauth2)

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > I disagree. The spec is open and anyone can implmeent it in any language
  > they want and can license it according to any license they want.
  > Google's implementation may not be open, but someone else could just as
  > easily write an implementation which is. 

I have to point out that whether a protocol is "open" is not what we
are concerned about.  We do not advocate "open".

Our concern is whether the Oauth2 protocol can be used to communicate
with Gmail without using any non-libre software.

Free (libre) and open are different ideas and not equivalent.  See
https://gnu.org/philosophy/open-source-misses-the-point.html.



-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: gmail+imap+smtp (oauth2)

[Tim Cross]

Richard Stallman <rms@gnu.org> writes:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>   > I disagree. The spec is open and anyone can implmeent it in any language
>   > they want and can license it according to any license they want.
>   > Google's implementation may not be open, but someone else could just as
>   > easily write an implementation which is. 
>
> I have to point out that whether a protocol is "open" is not what we
> are concerned about.  We do not advocate "open".
>
> Our concern is whether the Oauth2 protocol can be used to communicate
> with Gmail without using any non-libre software.
>
> Free (libre) and open are different ideas and not equivalent.  See
> https://gnu.org/philosophy/open-source-misses-the-point.html.

The answer is yes. You could implement an oauth2 client which would not
require running any non-libre software. However, this does not solve the
issue of the application ID being secret. This is essentially what
thunderbird have done (and it looks like, ignored the application ID
secrecy issue). 

Re: gmail+imap+smtp (oauth2)

[Tomas Hlavaty]

On Sun 08 May 2022 at 19:36, Richard Stallman <rms@gnu.org> wrote:
> Our concern is whether the Oauth2 protocol can be used to communicate
> with Gmail without using any non-libre software.

oauth2 mandates whitelist of allowed clients
the issue is getting on and staying on that whitelist controlled by google

Re: gmail+imap+smtp (oauth2)

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > Our concern is whether the Oauth2 protocol can be used to communicate
  > > with Gmail without using any non-libre software.

  > oauth2 mandates whitelist of allowed clients
  > the issue is getting on and staying on that whitelist controlled by google

There is a leap between the concern I mentined and your reply.
I don't know how to relate the two.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: gmail+imap+smtp (oauth2)

[Tomas Hlavaty]

On Wed 11 May 2022 at 05:04, Richard Stallman <rms@gnu.org> wrote:
>   > > Our concern is whether the Oauth2 protocol can be used to communicate
>   > > with Gmail without using any non-libre software.
>
>   > oauth2 mandates whitelist of allowed clients
>   > the issue is getting on and staying on that whitelist controlled by google
>
> There is a leap between the concern I mentined and your reply.
> I don't know how to relate the two.

I think Oauth2 protocol could theoretically be used to communicate with
Gmail without using any non-libre software.

Writing an oauth2 client software is not that difficult.

But having an oauth2 client software is useless without having the
software on a whitelist.  In the case of gmail, on two whitelists.

1) client_id whitelist

   oauth2 requires the client_id parameter.  It means getting the
   software whitelisted by X.  Who X is depends on the use-case.  It is
   not clear to me yet, who X is in this particular use-case, Google or
   the school/university?

   oauth2 says client_id should be secret.
   Having the client_id public would obviously break the whitelist.

2) application id whitelist

   In case of gmail, google apparently requires the application id parameter.

   Google T&C apparently says application id should be secret.
   Having the application id public would obviously break the whitelist.

This is not a problem for services where there usually is a business
incentive to put those parameters on the whitelist(s) and keep those
values secret, per service.

In case of programs, this gets problematic.

For example, what is an application for the purpose of application id?
If oauth2 client software was a standalone program, would it be a value
specific to that program?  Or would it be a value specific to the actual
mail client, e.g. gnus?  Would each fork (what would that mean exactly?)
need to get own application id?  Etc

Some people suggested ignoring Googles T&C, citing Thunderbird as
precedent.  That seems wrong.  The way to comply for a program seems
that each user should get his own application id.  Likely inconvenient,
that is why they decided to ignore (or dispute?) Google T&C and
apparently publish the application id.

What incentive would an organisation have to deal with whitelisting programs?

Similar for client_id.

Re: gmail+imap+smtp (oauth2)

[Tomas Hlavaty]

On Thu 12 May 2022 at 01:38, Tomas Hlavaty <tom@logand.com> wrote:
> In the case of gmail, on two whitelists.

i think that application id is indeed the same thing as client_id
so there is one whitelist
(google uses client_id and client_secret)

> Some people suggested ignoring Googles T&C, citing Thunderbird as
> precedent.

i had a look and both thunderbird and davmail publish client_id
thunderbird also publishes client_secret

cd /tmp
wget
http://releases.mozilla.org/pub/thunderbird/releases/101.0b2/source/thunderbird-101.0b2.source.tar.xz
tar xvf thunderbird-101.0b2.source.tar.xz
cd /tmp/thunderbird-101.0
grep -r -C 4 accounts.google.com/o/oauth2/auth

cd /tmp
wget --content-disposition https://sourceforge.net/projects/davmail/files/davmail/6.0.1/davmail-src-6.0.1-3390.tgz/download
tar zxvf davmail-src-6.0.1-3390.tgz
cd /tmp/davmail-src-6.0.1-3390
grep -r -i clientid

Re: gmail+imap+smtp (oauth2)

[Thomas Fitzsimmons]

Hi Tomas,

Tomas Hlavaty <tom@logand.com> writes:

> On Wed 11 May 2022 at 05:04, Richard Stallman <rms@gnu.org> wrote:
>>   > > Our concern is whether the Oauth2 protocol can be used to communicate
>>   > > with Gmail without using any non-libre software.
>>
>>   > oauth2 mandates whitelist of allowed clients
>>   > the issue is getting on and staying on that whitelist controlled by google
>>
>> There is a leap between the concern I mentined and your reply.
>> I don't know how to relate the two.
>
> I think Oauth2 protocol could theoretically be used to communicate with
> Gmail without using any non-libre software.
>
> Writing an oauth2 client software is not that difficult.

How would you go about doing that, such that the gmail.com login process
could be done natively within Emacs, without calling out to a web
browser?

For comparison, in my experience, Thunderbird embeds a Firefox browser
window to do the login step for MSO365 (required on the first connection
to retrieve the OAuth 2.0 refresh token).  I assume that's how
Thunderbird handles initial Gmail authentication too.

It would be helpful if there were an Emacs-native way of performing this
step, i.e., a method that doesn't require JavaScript or non-free
software.

To replicate a basic IMAP authentication workflow, the minibuffer would
prompt:

Username for gmail.com:
Password for <username>@gmail.com:
and optionally:
OTP for <username>@gmail.com:

Alternatively, the username and password would be retrieved from
authinfo.gpg, and Emacs would prompt for the OTP in the minibuffer
dynamically.

Thomas

Re: gmail+imap+smtp (oauth2)

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > It would be helpful if there were an Emacs-native way of performing this
  > step, i.e., a method that doesn't require JavaScript or non-free
  > software.

Running the nonfree JavaScript is not only an inconvenience.  It is an
injustice.  The first aim here is to be able to fetch and send email
without running any nonfree software on your machine(s).

That includes the nonfree JavaScript code that Gmail sends.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: gmail+imap+smtp (oauth2)

[Tomas Hlavaty]

On Sat 07 May 2022 at 04:55, Tim Cross <theophilusx@gmail.com> wrote:
> No, the University has considerable say. Typically, it has control over

how does one obtain the client_id?
from the university?

> Putting aside that applicaiton ID and client ID are not the same thing,
> not the bit

they seem to be instances of the same concept, whitelist items
one whitelist item for the university controlled whitelist
one whitelist item for google controlled whitelist

> I disagree. The spec is open and anyone can implmeent it in any
> language

that is true but not very useful due to the whitelists

>> one of the features seems to be that there is a (usually extra) party with special role
>> having absolute authority about who to let through the gate
>
> How is that any different to any other service (including SMTP).

traditionally, there used to be two actors involved in client-server
protocols like smtp and there were no whitelists

oauth2 involves more actors and whitelist(s)

> Even if you had a fully open source oauth2 implementaiton, there will
> still be a party (resource owner) who controls access and
> approves/rejects requests for access.

so far people could use whatever smtp client they chose
without having to whitelist it
(it was even possible to manually enter commands)

with oauth2, people cannot use whatever smtp client they choose;
that's why this thread exists

Re: gmail+imap+smtp (oauth2)

[Tomas Hlavaty]

On Thu 12 May 2022 at 09:10, Tomas Hlavaty <tom@logand.com> wrote:
>> Putting aside that applicaiton ID and client ID are not the same thing,

that seems wrong, it looks like those are the same thing, the client_id
parameter, see:

cd /tmp
wget http://releases.mozilla.org/pub/thunderbird/releases/101.0b2/source/thunderbird-101.0b2.source.tar.xz
tar xvf thunderbird-101.0b2.source.tar.xz
cd /tmp/thunderbird-101.0
grep -r -C 4 accounts.google.com/o/oauth2/auth

gmail requires client_id and client_secret
both likely assigned by google

Re: gmail+imap+smtp (oauth2)

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > I think this
  > is precisely the area where the FSF could assist as they might be able
  > to at least get the issue looked at by senior enough Google/MS
  > executives to get a definitive answer. 

We could try to do that, but we would need a precise and clear description
of the scenario and the question.  We're not experts on that.

I don't think Google was willing to confront this question when it
arose in 2019.  It would be necessary to raise a loud campaign.

If it turns out that other acceptable solutions do work in the free world.
I think it would be better to recommend them.



-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: gmail+imap+smtp (oauth2)

[Eric S Fraga]

On Thursday,  5 May 2022 at 16:13, Jorge A. Alfaro-Murillo wrote:
> I haven't been able to use gnus with my work email (@yale.edu) since
> then. I wonder if the same is true for other institutions that use
> Google Workspace.

My institution is the same: Exchange without the possibility of app
passwords.

My solution, which has been working just fine for a year now, is to use
davmail [1] to be in the middle between gnus and the Exchange server.
It was easy to install and doesn't get in the way.  The instructions are
fairly straightforward, as much as anything that deals with MS can be.

HTH,
eric

Footnotes:
[1]  http://davmail.sourceforge.net/

-- 
Eric S Fraga with org 9.5.3 in Emacs 29.0.50 on Debian 11.3

Re: gmail+imap+smtp (oauth2)

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > My solution, which has been working just fine for a year now, is to use
  > davmail [1] to be in the middle between gnus and the Exchange server.
  > It was easy to install and doesn't get in the way.

This raises a few questions:

1. What kind of thing is devmail?  Is it a program?  Is it a service?

2. If it is a program, is it entirely free/libre code?
What operating systems can it run on?

3. Does using it require running nonfree software?

4. How much of a wizard do you need to be, to use it successfully?

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: gmail+imap+smtp (oauth2)

[tomas]

[-- Attachment #1: Type: text/plain, Size: 1147 bytes --]

On Sun, May 08, 2022 at 07:37:52PM -0400, Richard Stallman wrote:
> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
> 
>   > My solution, which has been working just fine for a year now, is to use
>   > davmail [1] to be in the middle between gnus and the Exchange server.
>   > It was easy to install and doesn't get in the way.
> 
> This raises a few questions:
> 
> 1. What kind of thing is devmail?  Is it a program?  Is it a service?

It's called "davmail". Microsoft offers access to its mailboxes as
a WEBDAV interface. davmail seems to be an interface program to that.

It also can do, as far as I can see,  the other mailbox protocols
(POP, IMAP...) and perhaps more.

> 2. If it is a program, is it entirely free/libre code?
> What operating systems can it run on?

According to [1], it is GPL (probably v2).

No idea about the other two.

Cheers

[1] https://en.wikipedia.org/wiki/Comparison_of_CalDAV_and_CardDAV_implementations
-- 
t

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: gmail+imap+smtp (oauth2)

[Eric S Fraga]

On Sunday,  8 May 2022 at 19:37, Richard Stallman wrote:
> This raises a few questions:
>
> 1. What kind of thing is devmail?  Is it a program?  Is it a service?

Yes, both: a program that provides a service.  I'm not sure how to
differentiate between these two classifications...

> 2. If it is a program, is it entirely free/libre code?

Their web page, http://davmail.sourceforge.net/licenses.html, indicates
GPLv2.

> What operating systems can it run on?

Linux, Windows, Mac OS.

> 3. Does using it require running nonfree software?

I do not believe so; at least, I did not have to install anything that I
would consider non-free.

> 4. How much of a wizard do you need to be, to use it successfully?

I followed the instructions on the web page and everything worked as
expected.  However, should anything go wrong, the system does look
somewhat forbidding.  But, for me, nothing went wrong so it was
straightforward.

-- 
Eric S Fraga via gnus (Emacs 29.0.50 2022-05-03) on Debian 11.3

Re: gmail+imap+smtp (oauth2)

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > 1. What kind of thing is devmail?  Is it a program?  Is it a service?

  > Yes, both: a program that provides a service.

I'll try to clear up the distinction I'm making.

                                                   I'm not sure how to
  > differentiate between these two classifications...

A program and a service are totally different kinds of things.

A program is a list of instructions.  It can be stored in a file.
Copies of that file can be made and distributed.  They can be run.
You could get a copy of a program and run it yourself.

It is impossible to do ANY of those things to a service.  Consider
Twitter.  You can't put Twitter in a file.  It is impossible to copy
Twitter (what would that mean?), or "run Twitter" (what would that
mean?).

A service is something which does things, and which you can
communicate with.  You can't "run" it or "copy" it, but you can talk
with it.  It may do some things at your request; it may answer some
kinds of questions.  Or not.

A service can be operated by a computer.  Or by a human being.  In
principle, you as a user communicating with the service can't
necessarily tell how it is operated, and it may not matter anyway.

If a service is operated by running a program on a computer, for
clarity of our thinking we need to keep the distinction between the
service and the program.  What's true of one is probably not true of
the other.

When you talked about "using devmail", which of these did you do>
Did you talk with a service set up by someone else?
Did you run the devmail program on a computer of yours?
Both?


-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: gmail+imap+smtp (oauth2)

[Eric S Fraga]

On Monday,  9 May 2022 at 19:20, Richard Stallman wrote:
>   > Yes, both: a program that provides a service.
>
> I'll try to clear up the distinction I'm making.

Thank you although my answer is still valid, in my opinion.

davmail is a program.  It sits between you and the web dav service you
wish to access but for various reasons cannot do so directly, hence
allowing you (e.g. via gnus in Emacs) to access web dav services
transparently.

-- 
Eric S Fraga via gnus (Emacs 29.0.50 2022-05-10) on Debian 11.3

Re: gmail+imap+smtp (oauth2)

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > davmail is a program.  It sits between you and the web dav service you
  > wish to access but for various reasons cannot do so directly, hence
  > allowing you (e.g. via gnus in Emacs) to access web dav services
  > transparently.

Thanks for clearing it up.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: gmail+imap+smtp (oauth2)

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

I see that davmail is a program; it is not a service.  It is something
you can install and run, not something outside your machine that your
machine can talk with.

Every program does some sort of job.  Can you summarize in 10 or 20
lines the job that davmail does?  No need for little details -- we
don't need those to evaluate what davmail can do for us.  But we need
to know at a broad level what job it does, or we can't think about it.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: gmail+imap+smtp (oauth2)

[Eric S Fraga]

On Thursday, 12 May 2022 at 06:36, Richard Stallman wrote:
> I see that davmail is a program; it is not a service.  It is something
> you can install and run, not something outside your machine that your
> machine can talk with.

Sure but it acts as an intermediary to talk to something outside your
machine which is why I consider this to be both program and service in
your parlance.

> Can you summarize in 10 or 20 lines the job that davmail does?  No
> need for little details -- we don't need those to evaluate what
> davmail can do for us.  But we need to know at a broad level what job
> it does, or we can't think about it.

I quote from the website for davmail [1]:

   "The main goal of DavMail is to provide standard compliant protocols
    in front of proprietary Exchange. This means LDAP for global address
    book, SMTP to send messages, IMAP to browse messages on the server
    in any folder, POP to retrieve inbox messages only, Caldav for
    calendar support and Carddav for personal contacts sync. Thus any
    standard compliant client can be used with Microsoft Exchange.

    DavMail gateway is implemented in java and should run on any
    platform. Releases are tested on Windows, Linux (Ubuntu) and Mac
    OSX. Tested successfully with the Iphone (gateway running on a
    server)."

HTH,
eric

Footnotes:
[1]  http://davmail.sourceforge.net/

-- 
Eric S Fraga via gnus (Emacs 29.0.50 2022-05-10) on Debian 11.3

Re: gmail+imap+smtp (oauth2)

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

    > Can you summarize in 10 or 20 lines the job that davmail does?  No
    > need for little details -- we don't need those to evaluate what
    > davmail can do for us.  But we need to know at a broad level what job
    > it does, or we can't think about it.

    I quote from the website for davmail [1]:

       "The main goal of DavMail is to provide standard compliant protocols
        in front of proprietary Exchange.

Thanks.  That explains clearly what job davmail does.

It seems that davmail is a program that you run onyour machine, and it
communicates over the internet with an internet service.

Stretching the word "service" to include davmail makes the crucial
distinction less clear.  For clarity's sake, let's no call it that.

ISTR that davmail is free software.  If so, this might be a good
solution for accessing Microsoft Exchange.  If only there were
something similar for Gmail.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: gmail+imap+smtp (oauth2)

[Jorge A. Alfaro-Murillo]

On Fri, May 06 2022, Eric S Fraga wrote:

> On Thursday,  5 May 2022 at 16:13, Jorge A. Alfaro-Murillo 
> wrote:
>> I haven't been able to use gnus with my work email (@yale.edu) 
>> since then. I wonder if the same is true for other institutions 
>> that use Google Workspace.
>
> My institution is the same: Exchange without the possibility of 
> app passwords.
>
> My solution, which has been working just fine for a year now, is 
> to use davmail [1] to be in the middle between gnus and the 
> Exchange server.  It was easy to install and doesn't get in the 
> way.  The instructions are fairly straightforward, as much as 
> anything that deals with MS can be.

Thank you! I'll take a look. Does it allow you to send emails with 
smtpmail too?

Would you mind sharing your gnus/smtpmail setup after davmail is 
running?
-- 
Jorge.

Re: gmail+imap+smtp (oauth2)

[Eric S Fraga]

Apologies: I replied off-list by mistake.  Here is my reply:


On Thursday, 12 May 2022 at 10:12, Jorge A. Alfaro-Murillo wrote:
> Thank you! I'll take a look. Does it allow you to send emails with
> smtpmail too?

Supposedly but I have not tried this yet.  I will have to soon, mind
you, as our Exchange server will be requiring multi-factor
authentication for SMTP as well in a little while.

> Would you mind sharing your gnus/smtpmail setup after davmail is
> running?

I have this as one of my mail-sources:

(list 'pop
      :server "localhost"
      :user "my exchange user id"
      :port 1110)

I use pop to retrieve the email from the Exchange server.  The port is
for davmail on the localhost, as noted in the instructions.

I then have

(setq gnus-secondary-select-methods
      '((nnml "outlook"
              (gnus-search-engine gnus-search-notmuch
                                  (remove-prefix "/home/ucecesf/Mail")
                                  (config-file "/home/ucecesf/.notmuch-config")
                                  )
              (get-new-mail t))
     ...)) 

This is all I had to do.

HTH,
eric

-- 
Eric S Fraga via gnus (Emacs 29.0.50 2022-05-10) on Debian 11.3

Re: gmail+imap+smtp (oauth2)

[Roland Winkler]

On Fri, May 13 2022, Eric S Fraga wrote:
> On Thursday, 12 May 2022 at 10:12, Jorge A. Alfaro-Murillo wrote:
>> Thank you! I'll take a look. Does it allow you to send emails with
>> smtpmail too?
>
> Supposedly but I have not tried this yet.  I will have to soon, mind
> you, as our Exchange server will be requiring multi-factor
> authentication for SMTP as well in a little while.

Davmail lets me talk to the office365 SMTP server which for me requires
MFA.  With Gnus I use

         (setq smtpmail-smtp-user "foo@bar.com"
               smtpmail-smtp-server "localhost"
               smtpmail-smtp-service 1025))

in addition to the configuration of davmail.

Re: gmail+imap+smtp (oauth2)

[Eric S Fraga]

On Friday, 13 May 2022 at 13:49, Roland Winkler wrote:
> Davmail lets me talk to the office365 SMTP server which for me requires
> MFA.  With Gnus I use
>
>          (setq smtpmail-smtp-user "foo@bar.com"
>                smtpmail-smtp-server "localhost"
>                smtpmail-smtp-service 1025))

Thank you.  This is very helpful.
-- 
Eric S Fraga via gnus (Emacs 29.0.50 2022-05-10) on Debian 11.3

Re: gmail+imap+smtp (oauth2)

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > your password and then a 2nd factor (fingerprint, code
  > from an authentication app, hardware key etc).

I have to ask whether there is an option for the second factor that
isn't unjust.

Google should not know your fingerprint, and anyway, can you give it
to Google without nonfree software?  I doubt it.

Is there any accepted authentication app which is free software and
runs on a free GNU/Linux system?  The word "app" suggests that they
run on nonfree snoopphone operating systems.

As for hardware keys, in principle there could be one that is
acceptable.  I investigated the Yubikey some years ago and ISTR I
found it requires nonfree software.  Which option can people
recommend?

It sounds like there may be other options not listed.  Is any of them
acceptable?

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: gmail+imap+smtp (oauth2)

[Stefan Monnier]

> Is there any accepted authentication app which is free software and
> runs on a free GNU/Linux system?  The word "app" suggests that they
> run on nonfree snoopphone operating systems.

Yes, what people often call "authentication app" is a misnomer which
I resent (it's often accompanied with an application name or a link to
it, suggesting that the software (typically proprietary, of course) is
the one and only you can use).  It really (usually) means the
authentication uses the TOTP protocol, for which there are several Free
Software implementations readily available.

> As for hardware keys, in principle there could be one that is
> acceptable.  I investigated the Yubikey some years ago and ISTR I
> found it requires nonfree software.  Which option can people
> recommend?

AFAIK the Somu (and presumably other Solokeys) does not require any
proprietary software.  I'd be happy to hear confirmation from someone
who actually dug into the matter, tho.


        Stefan

Re: gmail+imap+smtp (oauth2)

[Stefan Monnier]

> authentication uses the TOTP protocol, for which there are several Free
> Software implementations readily available.

Including one in ELisp

    https://github.com/juergenhoetzel/emacs-totp

which we could add to GNU ELPA, I believe.


        Stefan

Re: gmail+imap+smtp (oauth2)

[Uwe Brauer]

[-- Attachment #1: Type: text/plain, Size: 1309 bytes --]

>>> "SM" == Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> authentication uses the TOTP protocol, for which there are several Free
>> Software implementations readily available.

> Including one in ELisp

>     https://github.com/juergenhoetzel/emacs-totp

> which we could add to GNU ELPA, I believe.

Now I am bit confused, as Robert confirmed it would be enough to have

,----
| 
| 
|  (setq gnus-select-method (list 'nnnil "")) 
|  (setq gnus-secondary-select-methods nil)
|  (setq gnus-secondary-select-methods
|        '(
|  	(nnimap "UCMgmail"
|  		(nnimap-address "imap.gmail.com")
|  		(nnimap-server-port 993)
|                  ;; (nnimap-authinfo-file "~/.authinfo.gpg")
|  		(nnimap-authinfo-file "~/.authinfo")
|  		(nnimap-stream ssl)
|                  ;;(nnimap-stream starttls)
|                  (nnimap-fetch-partial-articles "text/")
|  		(nnir-search-engine imap))))
| 
|  And put in authinfo my app password instead my «normal» one?
|
| Robert> yes
`----

So where is emacs-totp needed in this workflow?

>         Stefan


-- 
I strongly condemn Putin's war of aggression against the Ukraine.
I support to deliver weapons to Ukraine's military. 
I support the ban of Russia from SWIFT.
I support the EU membership of the Ukraine. 

[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 5673 bytes --]

Re: gmail+imap+smtp (oauth2)

[Tim Cross]

Uwe Brauer <oub@mat.ucm.es> writes:

> [[S/MIME Signed Part:Undecided]]
>>>> "SM" == Stefan Monnier <monnier@iro.umontreal.ca> writes:
>
>>> authentication uses the TOTP protocol, for which there are several Free
>>> Software implementations readily available.
>
>> Including one in ELisp
>
>>     https://github.com/juergenhoetzel/emacs-totp
>
>> which we could add to GNU ELPA, I believe.
>
> Now I am bit confused, as Robert confirmed it would be enough to have
>
> ,----
> | 
> | 
> |  (setq gnus-select-method (list 'nnnil "")) 
> |  (setq gnus-secondary-select-methods nil)
> |  (setq gnus-secondary-select-methods
> |        '(
> |  	(nnimap "UCMgmail"
> |  		(nnimap-address "imap.gmail.com")
> |  		(nnimap-server-port 993)
> |                  ;; (nnimap-authinfo-file "~/.authinfo.gpg")
> |  		(nnimap-authinfo-file "~/.authinfo")
> |  		(nnimap-stream ssl)
> |                  ;;(nnimap-stream starttls)
> |                  (nnimap-fetch-partial-articles "text/")
> |  		(nnir-search-engine imap))))
> | 
> |  And put in authinfo my app password instead my «normal» one?
> |
> | Robert> yes
> `----
>
> So where is emacs-totp needed in this workflow?
>

It isn't - at least not directly. 

The TOTP protocol is what authentication apps use to generate unique one
time passwords. You can use such a technique with some 2FA setups for
the 2nd factor. You only use the 2FA authentication to login to your
google account to generate the application passwords. The suggestion is
that you could use the emacs implementation instead of Google
Authenticator or Facebook authenticator etc. 

Re: gmail+imap+smtp (oauth2)

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  >   It really (usually) means the
  > authentication uses the TOTP protocol, for which there are several Free
  > Software implementations readily available.

If someone can confirm that this method works, using no nonfree software,
that would be very helpful.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: gmail+imap+smtp (oauth2)

[Brian Cully via Emacs development discussions.]

Richard Stallman <rms@gnu.org> writes:

>   >   It really (usually) means the
>   > authentication uses the TOTP protocol, for which there are 
>   > several Free
>   > Software implementations readily available.
>
> If someone can confirm that this method works, using no nonfree 
> software,
> that would be very helpful.

I use KeePassXC to generate TOTP tokens, and it’s licensed under 
the GPLv2 and v3.

-bjc

Re: gmail+imap+smtp (oauth2)

[Ihor Radchenko]

Richard Stallman <rms@gnu.org> writes:

> As for hardware keys, in principle there could be one that is
> acceptable.  I investigated the Yubikey some years ago and ISTR I
> found it requires nonfree software.  Which option can people
> recommend?

I am using Yubikey and I did not have to install any kind of nonfree
software, except libyubikey licenced under GPL-2
(https://github.com/Yubico/yubico-c)

Best,
Ihor

Re: gmail+imap+smtp (oauth2)

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > I am using Yubikey and I did not have to install any kind of nonfree
  > software, except libyubikey licenced under GPL-2
  > (https://github.com/Yubico/yubico-c)

I'm glad to find that memory was mistaken.

Can anyone set up a 100%-libre system to use a Yubikey to access Gmail
with GNOME, and verify that this is a solution we can recommend?

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: gmail+imap+smtp (oauth2)

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > I landed on the conclusion that SMTP 
  > and IMAP should keep working as long as you use app-passwords for 
  > logging in to your account.

Can you explain what "app-passwords" are?  I have never used Gmail,
and I don't need to know technical details, but I have to think
about the ethical implications of this.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: gmail+imap+smtp (oauth2)

[Tim Cross]

Richard Stallman <rms@gnu.org> writes:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>   > I landed on the conclusion that SMTP 
>   > and IMAP should keep working as long as you use app-passwords for 
>   > logging in to your account.
>
> Can you explain what "app-passwords" are?  I have never used Gmail,
> and I don't need to know technical details, but I have to think
> about the ethical implications of this.

Google introduced the concept of app passwords back when they first
implemented 2FA. Basically, they are just a password based
authentication workflow which can be used with applications that do not
support 2FA or oauth2 based authentication and authorisation. Google
generates a long complex password which you can then use to authenticate
instead of your 'normal' password (and 2nd factor for 2FA). The app
passwords cannot be used to login via 'standard' web based mechanisms
(2FA/oauth2). 

You have to log into your google account and enable app passwords and
then generate 1 (or more) app passwords which you then use for imap/smtp
authentication instead of your 'normal' google password. 

Each app password can be given a name - for example, I have one called
'emacs' which is the password I use to connect to imap/smtp from Emacs.
You can view what app passwords you have defined and when they were last
used by logging into your google account and checking your settings
page. You cannot see the actual password though - that is only available
when you first create the password. If you forget it or lose it, you
have to create a new one (and delete the old one of course). 

Google is removing access to imap/smtp using your main google
login/password and will require 2FA and oauth2 for all web based
authn/authz. However, their documentation implies that app passwords
will remain as the standad solution for applications which cannot do 2FA
or oauth2. 

I don't know if app passwords are available for institution/enterprise
google users. It is possible that may be a configuration option and up
to the individual organisations to enable/disable. Google's advice would
be not to enable them unless there is a demonstrated need. Many larger
organisations will just follow Google's advice as they don't want users
using applications they haven't 'approved'.

I don't think there are any significant ethical considerations
associated with app passwords (in addition to those associated with
using Google/Gmail that is). It is likely that setting the app password
via the Google account settings page involves non-free Javascript, but I
think that boat sailed when you initially sign up for a gmail account
anyway. Some will probably have issue with the fact you cannot set the
specific app password and have no insight into the algorithm google uses
to generate the password, which are reasonable criticisms (though
experience has shown many people do better with even flawed password
generators than self selected passwords). At the end of the day, if you
trust Google with your email data, it probably isn't a long stretch to
trust they will generate a reasonably good password 

Re: gmail+imap+smtp (oauth2)

[tomas]

[-- Attachment #1: Type: text/plain, Size: 1478 bytes --]

On Wed, May 04, 2022 at 12:05:37PM +1000, Tim Cross wrote:

[...]

> I don't think there are any significant ethical considerations
> associated with app passwords (in addition to those associated with
> using Google/Gmail that is) [...]

First, thanks for your clear explanation. It took me a while to
wrap my head around that concept the first time I stumbled upon
it (it was, BTW, a free application).

Then, I have been thinking hard about the question I quoted above,
as every app and her sister (even free ones!) is now copying this
pattern.

What this is based on is mistrust of the user: she ain't going to
manage her passwords properly anyway, is she?

This makes a lot of sense for big wigs like Google, Facebook et al,
which thrive on having reams of users, because their marginal gains
per user are extremely thin. Having a password recovery service
incurs costs, so the more control is taken from those pesky unreliable
users the better.

What this leads to is, in my eyes, fatal: first, this narrative of
the dumb user is strenghtened (I'm on the brink of thinking that
this is /intentional/), second, there's no motivation to make users
smarter.

In one short phrase: take the user out of the equation.

(That's BTW why I'm wary of all those 2FA schemes).

Whether this has anything to do with free software ideals or not
is stuff for another discussion. But I don't want to derail this
thread even more :-)

Cheers
-- 
t

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: gmail+imap+smtp (oauth2)

[Thomas Fitzsimmons]

Hi Tim,

Tim Cross <theophilusx@gmail.com> writes:

> Richard Stallman <rms@gnu.org> writes:
>
>> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
>> [[[ whether defending the US Constitution against all enemies,     ]]]
>> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>>
>>   > I landed on the conclusion that SMTP 
>>   > and IMAP should keep working as long as you use app-passwords for 
>>   > logging in to your account.
>>
>> Can you explain what "app-passwords" are?  I have never used Gmail,
>> and I don't need to know technical details, but I have to think
>> about the ethical implications of this.

[...]

> I don't think there are any significant ethical considerations
> associated with app passwords (in addition to those associated with
> using Google/Gmail that is). It is likely that setting the app password
> via the Google account settings page involves non-free Javascript, but I
> think that boat sailed when you initially sign up for a gmail account
> anyway.

One issue with OAuth2 schemes is that they periodically force the user
through a web-browser-only authentication process that requires non-free
JavaScript, in order to get a refresh token.

(I'm hoping someone can prove me wrong, and point me to a command-line
procedure using only free software that allows me to get a refresh token
when required.  We're told OAuth2 is a modern standard, right?  So there
should be a modern, standard way of doing the same things as the
JavaScript authentication blobs... right?)

There are two issues, which I think should be considered separately:

One-time registration requiring non-free JavaScript (1).

Subsequently requiring non-free JavaScript for authentication to use
IMAP or SMTP protocols (2).

See the discussion in this bug report, closed wontfix:

   https://debbugs.gnu.org/cgi/bugreport.cgi?bug=41386

I'm hoping the FSF will study and comment on the issue in general, given
that gmail.com, such a large email provider, is making this OAuth2
change.  To me, issue (2) seems like a high priority one for Free
Software.  Keep in mind that avoiding issue (1) isn't always optional,
from an employee/student perspective.

Thomas

Re: gmail+imap+smtp (oauth2)

[Stefan Monnier]

> (I'm hoping someone can prove me wrong, and point me to a command-line
> procedure using only free software that allows me to get a refresh token
> when required.  We're told OAuth2 is a modern standard, right?  So there
> should be a modern, standard way of doing the same things as the
> JavaScript authentication blobs... right?)

My understanding is that current definitions of "modern" sound very much
like "JavaScript authentication blobs".


        Stefan

Re: gmail+imap+smtp (oauth2)

[Robert Pluim]

>>>>> On Wed, 04 May 2022 10:38:41 -0400, Stefan Monnier <monnier@iro.umontreal.ca> said:

    >> (I'm hoping someone can prove me wrong, and point me to a command-line
    >> procedure using only free software that allows me to get a refresh token
    >> when required.  We're told OAuth2 is a modern standard, right?  So there
    >> should be a modern, standard way of doing the same things as the
    >> JavaScript authentication blobs... right?)

    Stefan> My understanding is that current definitions of "modern" sound very much
    Stefan> like "JavaScript authentication blobs".

...with some random semi-redundant impenetrable XML crap wrapped
around it (maybe not for OAuth2, but certainly for other
authentication protocols).

Robert
-- 

Re: gmail+imap+smtp (oauth2)

[Tim Cross]

Thomas Fitzsimmons <fitzsim@fitzsim.org> writes:

> Hi Tim,
>
> Tim Cross <theophilusx@gmail.com> writes:
>
>> Richard Stallman <rms@gnu.org> writes:
>>
>>> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
>>> [[[ whether defending the US Constitution against all enemies,     ]]]
>>> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>>>
>>>   > I landed on the conclusion that SMTP 
>>>   > and IMAP should keep working as long as you use app-passwords for 
>>>   > logging in to your account.
>>>
>>> Can you explain what "app-passwords" are?  I have never used Gmail,
>>> and I don't need to know technical details, but I have to think
>>> about the ethical implications of this.
>
> [...]
>
>> I don't think there are any significant ethical considerations
>> associated with app passwords (in addition to those associated with
>> using Google/Gmail that is). It is likely that setting the app password
>> via the Google account settings page involves non-free Javascript, but I
>> think that boat sailed when you initially sign up for a gmail account
>> anyway.
>
> One issue with OAuth2 schemes is that they periodically force the user
> through a web-browser-only authentication process that requires non-free
> JavaScript, in order to get a refresh token.
>
> (I'm hoping someone can prove me wrong, and point me to a command-line
> procedure using only free software that allows me to get a refresh token
> when required.  We're told OAuth2 is a modern standard, right?  So there
> should be a modern, standard way of doing the same things as the
> JavaScript authentication blobs... right?)
>
> There are two issues, which I think should be considered separately:
>
> One-time registration requiring non-free JavaScript (1).
>
> Subsequently requiring non-free JavaScript for authentication to use
> IMAP or SMTP protocols (2).
>
> See the discussion in this bug report, closed wontfix:
>
>    https://debbugs.gnu.org/cgi/bugreport.cgi?bug=41386
>
> I'm hoping the FSF will study and comment on the issue in general, given
> that gmail.com, such a large email provider, is making this OAuth2
> change.  To me, issue (2) seems like a high priority one for Free
> Software.  Keep in mind that avoiding issue (1) isn't always optional,
> from an employee/student perspective.
>

I think your confusing oauth2 as an authn/authz framework/standard and
its implementation as done by Google. There is nothing which requires
oauth2 be implemented in Javascript or that requies it to use non-free
software. 

Re: gmail+imap+smtp (oauth2)

[Thomas Fitzsimmons]

Hi Tim,

Tim Cross <theophilusx@gmail.com> writes:

> Thomas Fitzsimmons <fitzsim@fitzsim.org> writes:

[...]

>> Tim Cross <theophilusx@gmail.com> writes:
>>
>>> Richard Stallman <rms@gnu.org> writes:
>>>
>>>> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
>>>> [[[ whether defending the US Constitution against all enemies,     ]]]
>>>> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>>>>
>>>>   > I landed on the conclusion that SMTP 
>>>>   > and IMAP should keep working as long as you use app-passwords for 
>>>>   > logging in to your account.
>>>>
>>>> Can you explain what "app-passwords" are?  I have never used Gmail,
>>>> and I don't need to know technical details, but I have to think
>>>> about the ethical implications of this.
>>
>> [...]
>>
>>> I don't think there are any significant ethical considerations
>>> associated with app passwords (in addition to those associated with
>>> using Google/Gmail that is). It is likely that setting the app password
>>> via the Google account settings page involves non-free Javascript, but I
>>> think that boat sailed when you initially sign up for a gmail account
>>> anyway.
>>
>> One issue with OAuth2 schemes is that they periodically force the user
>> through a web-browser-only authentication process that requires non-free
>> JavaScript, in order to get a refresh token.
>>
>> (I'm hoping someone can prove me wrong, and point me to a command-line
>> procedure using only free software that allows me to get a refresh token
>> when required.  We're told OAuth2 is a modern standard, right?  So there
>> should be a modern, standard way of doing the same things as the
>> JavaScript authentication blobs... right?)
>>
>> There are two issues, which I think should be considered separately:
>>
>> One-time registration requiring non-free JavaScript (1).
>>
>> Subsequently requiring non-free JavaScript for authentication to use
>> IMAP or SMTP protocols (2).
>>
>> See the discussion in this bug report, closed wontfix:
>>
>>    https://debbugs.gnu.org/cgi/bugreport.cgi?bug=41386
>>
>> I'm hoping the FSF will study and comment on the issue in general, given
>> that gmail.com, such a large email provider, is making this OAuth2
>> change.  To me, issue (2) seems like a high priority one for Free
>> Software.  Keep in mind that avoiding issue (1) isn't always optional,
>> from an employee/student perspective.
>>
>
> I think your confusing oauth2 as an authn/authz framework/standard and
> its implementation as done by Google.

Are you aware of any email provider that advertises "OAuth 2.0" but that
does not require non-free JavaScript blobs for authentication?

> There is nothing which requires oauth2 be implemented in Javascript or
> that requies it to use non-free software.

So can you describe how to get an OAuth 2.0 gmail.com refresh token
without the use of non-free JavaScript?

Thanks,
Thomas

Re: gmail+imap+smtp (oauth2)

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Are you aware of any email provider that advertises "OAuth 2.0" but that
  > does not require non-free JavaScript blobs for authentication?

That is a very important question, because if there is one,
we can advise to solve the problem by switching to that service.
If there isn't one, that means the problem is MORE GRAVE, not less grave.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: gmail+imap+smtp (oauth2)

[Tomas Hlavaty]

On Wed 04 May 2022 at 11:41, Thomas Fitzsimmons <fitzsim@fitzsim.org> wrote:
>>> One issue with OAuth2 schemes is that they periodically force the user
>>> through a web-browser-only authentication process that requires non-free
>>> JavaScript, in order to get a refresh token.
[...]
>> There is nothing which requires oauth2 be implemented in Javascript or
>> that requies it to use non-free software.

exactly

getting the bearer token is just a few https requests

> So can you describe how to get an OAuth 2.0 gmail.com refresh token
> without the use of non-free JavaScript?

i don't use gmail so i cannot answer this specific question

but as pointed out, the real issue is not so much on the client side
but on the server side

Re: gmail+imap+smtp (oauth2)

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > but as pointed out, the real issue is not so much on the client side
  > but on the server side

Since the issue is whether we can use free software to make GNUS talk
with Gmail, naturally the issue has a "server side".  
Would you please identify more precisely what "real issue" you mean?
For instance, quote the text that described this issue?

There have been so many messages about this that I can't identify one
of them based on a generally description of what it said.


-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: gmail+imap+smtp (oauth2)

[Tim Cross]

Richard Stallman <rms@gnu.org> writes:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>   > but as pointed out, the real issue is not so much on the client side
>   > but on the server side
>
> Since the issue is whether we can use free software to make GNUS talk
> with Gmail, naturally the issue has a "server side".  
> Would you please identify more precisely what "real issue" you mean?
> For instance, quote the text that described this issue?
>
> There have been so many messages about this that I can't identify one
> of them based on a generally description of what it said.

Your looking for a clear concise short explaination for a larger complex
set of problems. There are multiple issues. 

Big Question: Can you use gmail only using libre software. Anser NO. 

Subsequent questions: Can you minimise the use of non-libre software
Yes. 

At this stage, I do not know of any way to create/register a google
account which does not require Javascript and the status of that
javascript is unknown, but can be expected to be non-free. Once you have
created an account, the only way to access your account 'settings' page
is to login to the Google site, again requiring use of non-free
javascript. 

Google login authentication supports a number of different 2FA schemes.
Some are non-free (SMS, Google Authenticator). Some are free
(keyPassXC). It is up to the user to select which scheme is used. 

Until recently, once your account was registered, you could use libre
tools to access your messages and send new ones via IMAP and SMTP.
However, you do have to use the non-free account settings page to turn
these services on and you must not enable 2FA. Once they are enabled,
you don't need to use the non-free login/settings pages again (unless
you want to change your password or other settings).

Google has started enforcing 2FA (now mandatory on all new accounts). If
you have 2FA, you cannot use your 'normal' Google username/password with
IMAP and SMTP. At this point you have 2 choices. This is where the main
issue for this thread started. The choices are -

1. Use application passwords. These are a 'special' password you create
using your google settings page (running non-free software). Once you
have the applicaiton password, you can use IMAP/SMTP with libre clients,
using the application password in place of your 'normal' password. At
this point, your retrieval and sending of messages can be done using
only libre software. 

2. Use a Google oauth2 compliant client to obtain an oath2 access token
which you then use as your password in your libre IMAP/SMTP client.
However, at this time, there doesn't seem to be any libre Google oauth2
client we can use. If there was, it would be theoretically possible to
access your emails and send new ones using only libre software and avoid
needing to login to the non-free settings page to setup application
passwords. 

The issue with having a libre oauth2 client is that the client needs to
be approved by Google and issued with an application ID which is
supplied as part of the client authorisation request. The Google T&C
state that this value must be kept secret. If we put this ID into the
source code, it won't be secret and therefore not compliant with
Google's T&C. 

It has been argued that the interpretation of the T&C is misleading or
ambiguous and the applicaiton ID does not need to be kept secret (or
does not need to be 'as secret' as something like a password). Other
projects, like thunderbird, appear to be adopting this position and have
incorporated oauth2 authentication, eliminating the need for applicaiton
passwords or the need for users to use the non-free Google login page in
order to access IMAP/SMTP. The risk for them is that if Google decides
their application has not complied with the T&c, they will cancel the
application ID and thunderbird will stop working with Google. 

Personally, I think the thunderbird position is the right one. It
minimises the need to use non-free software and I think it is unlikely
Google will cancel their application ID. Even if they do, all the user
then needs to do is setup application passwords and use them. 

What might be good is if the FSF could get clarification from Google
regarding the T&C requirements for application ID. I suspect Google's
intention with the T&C is that developers should not publicise the
application ID i.e. having it embedded in source code is OK, having it
referenced on the web site homepage is not. As Stefan pointed out, even
with closed source software, an embedded ID of this type can still be
extracted by anyone with sufficient patience and knowledge on how to use
a debugger. 

There are some risks associated with requesting clarification. If google
comes back and categorically states the application ID cannot be
embedded in an open source program and we then go ahead and do it, I
guess Google could use that as a pretext for more serious legal action.
It would not be possible to argue it was an error and would likely be
seen as a deliberate breach of their T&C. A situation FSF lawyers would
probably find unacceptable. 

Re: gmail+imap+smtp (oauth2)

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

Thanks very much for spelling out the whole situation clearly.

(Where does TOTP fit into this picture?)

  > At this stage, I do not know of any way to create/register a google
  > account which does not require Javascript and the status of that
  > javascript is unknown, but can be expected to be non-free. Once you have
  > created an account, the only way to access your account 'settings' page
  > is to login to the Google site, again requiring use of non-free
  > javascript. 

This is an injustice, of course.  It is one reason to refuse to use
Gmail.  It may be possible to write free replacement Javascript code
and use that instead.  But it doesn't pertain to Emacs in particular,
so we don't need to go into it here.

In case a school demands you have a Gmail account, it would be useful
if we had instructions to send to the staff, saying, "You may create
the account, choose a password, and tell it to me.  (Since it will
only be for email to and from the school, it makes no difference to me
that school staff will know the password.)  Please choose an account
name with no resemblance to my name.  Please set the account settings
as follows so that my software can access the account."

  > Google has started enforcing 2FA (now mandatory on all new accounts).

If 2FA is enabled, in which situations does the user have to do the
2FA procedure?  And how many times?  Only once, for setup -- or
repeatedly?

This, I think, is where the possibility of using hardware keys such as
the Yubikey, is pertinent,

> Personally, I think the thunderbird position is the right one. It
  > minimises the need to use non-free software and I think it is unlikely
  > Google will cancel their application ID. Even if they do, all the user
  > then needs to do is setup application passwords and use them. 

I tend to agree, except that the FSF can't do it by making a contract
with Google that we intend not to keep.

  > What might be good is if the FSF could get clarification from Google
  > regarding the T&C requirements for application ID.

Actually I doubt that Google would respond.  Also,

  > There are some risks associated with requesting clarification. If google
  > comes back and categorically states the application ID cannot be
  > embedded in *a free* program

that is a cogent reason not to ask,

(We shun the term "open source" because it implies rejection of our moral
stance.  Likewise the term "closed source", which also rejects it.
See https://gnu.org/philosophy/open-source-misses-the-point.html.)


-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: gmail+imap+smtp (oauth2)

[Tim Cross]

Richard Stallman <rms@gnu.org> writes:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
> Thanks very much for spelling out the whole situation clearly.
>
> (Where does TOTP fit into this picture?)

For the 2FA process. Instead of using something like Google
Authenticator, you could use an open source TOTP client. Your still
going to have to use the non-free Javascript based UI, your just using
less non-free software. However, as they say, you cannot be a little bit
pregnant! 

>
>   > At this stage, I do not know of any way to create/register a google
>   > account which does not require Javascript and the status of that
>   > javascript is unknown, but can be expected to be non-free. Once you have
>   > created an account, the only way to access your account 'settings' page
>   > is to login to the Google site, again requiring use of non-free
>   > javascript. 
>
> This is an injustice, of course.  It is one reason to refuse to use
> Gmail.  It may be possible to write free replacement Javascript code
> and use that instead.  But it doesn't pertain to Emacs in particular,
> so we don't need to go into it here.
>

True. It is also possible Google does provide an API which a school or
institution could use to create a custom account registration and
settings update site. Note also that many larger institutions may
actually integrate Google into their in-house identity and access
management (IAM) system (it is one of the strengths of oauth2, being
able to integrate with 3rd party identity providers). However, few (if
any) of the commercial IAM solutions are based on libre software (there are
some Universities who have been working on an IAM solution which is
based on libre packages, but sadly, too many Universities have ignorant
administrators who still see proprietary = quality, libre = amateur. 

> In case a school demands you have a Gmail account, it would be useful
> if we had instructions to send to the staff, saying, "You may create
> the account, choose a password, and tell it to me.  (Since it will
> only be for email to and from the school, it makes no difference to me
> that school staff will know the password.)  Please choose an account
> name with no resemblance to my name.  Please set the account settings
> as follows so that my software can access the account."
>

There is no way any institution would support such a workflow. Apart
from the additional resource demands, it would raise lots of questions
regarding staff knowing student's email passwords. In many
schools/Universities, email is considered an official record and many
critical workflows are based around it (enrolling, unenrolling,
assignment submission, various approval processes etc). 

>   > Google has started enforcing 2FA (now mandatory on all new accounts).
>
> If 2FA is enabled, in which situations does the user have to do the
> 2FA procedure?  And how many times?  Only once, for setup -- or
> repeatedly?
>

Basically, every time you connect from a new device or new
browser and any time you want to modify your settings. Of course, you
can change that and enforce 2FA every time you try to access a service. 

> This, I think, is where the possibility of using hardware keys such as
> the Yubikey, is pertinent,
>

Re: gmail+imap+smtp (oauth2)

[Tomas Hlavaty]

On Mon 09 May 2022 at 10:01, Tim Cross <theophilusx@gmail.com> wrote:
>> In case a school demands you have a Gmail account, it would be useful
> [...]
> There is no way any institution would support such a workflow. Apart
> from the additional resource demands, it would raise lots of questions
> regarding staff knowing student's email passwords. In many
> schools/Universities, email is considered an official record and many
> critical workflows are based around it (enrolling, unenrolling,
> assignment submission, various approval processes etc). 

When a school/university demands gmail account
and google locks me out of my gmail account,
what happens?

Re: gmail+imap+smtp (oauth2)

[Tim Cross]

Tomas Hlavaty <tom@logand.com> writes:

> On Mon 09 May 2022 at 10:01, Tim Cross <theophilusx@gmail.com> wrote:
>>> In case a school demands you have a Gmail account, it would be useful
>> [...]
>> There is no way any institution would support such a workflow. Apart
>> from the additional resource demands, it would raise lots of questions
>> regarding staff knowing student's email passwords. In many
>> schools/Universities, email is considered an official record and many
>> critical workflows are based around it (enrolling, unenrolling,
>> assignment submission, various approval processes etc). 
>
> When a school/university demands gmail account
> and google locks me out of my gmail account,
> what happens?

When a school/university makes a decision to use Google as their email
provider, it isn't 'normal' google - it is your school/university's
email, essentially hosted by google. As such, your institution controls
access, not google. Google just provides the service to your
school/university. Often, the setup involves integration with your
school/university IAM system i.e. your 'identity' (your username) is
managed by the school/university. This integration makes it easier for
existing school/university workflows to continue i.e. onboarding of new
students/staff, removal of accounts when students/staff leave etc. It
also makes integration with other services, such as on-line LMS (Moodle,
Blackboard etc) easier as there is just one 'meta directory' of all
accounts. This is where oauth2 shows its strengths. Your institituion
essentially becomes a identity provider which Google trusts. When you
request authorisation credentials, they are provided by your
institutions IAM system. Your client then submits those authorisation
credentials to get an access token from Google which you then submit to
the Google service you want to access (i.e. email). 

So, if your locked out, it is because your institution has decided to
lock you out, not google. 

Of course, the real solution here is that schools/universitites should
just get out of the business of providing email services to students.
Instead, they should update their workflows to allow a student to
specify what their email is and just leave it at that. Nearly every
student who goes to a University or school already has an email address.
Being forced to have a new one is not necessarily a 'benefit' (things
have progressed since the 80's where most people didn't have an email
account). 

For staff, slightly different situation. Email is considered official
records of the institution, so it is fair enough that the email account
you get as a staff member is actually part of your work relationship
with the institution and not your private email. Unfortunately, it isn't
that clean cut, especially for researchers etc who often associate their
email address with published articles/research. Things get messy when
the individual changes institutions. That is why there are a number of
initiatives for having institutional independent identifiers that can be
used. Unfortunately, there are more than one such scheme, so it will
likely take a few more years before everything becomes standardised and
consistent. (things get even more complex with respect to research data,
especially when that data is collected through complex funding from
private and public bodies - identifying who actually owns the data and
who is responsible for on-going costs associated with storing/hosting
valuable research data can be complex and difficult to manage).

Re: gmail+imap+smtp (oauth2)

[Tomas Hlavaty]

On Tue 10 May 2022 at 17:51, Tim Cross <theophilusx@gmail.com> wrote:
> Tomas Hlavaty <tom@logand.com> writes:
>> When a school/university demands gmail account
>> and google locks me out of my gmail account,
>> what happens?
>
> When a school/university makes a decision to use Google as their email
> provider, it isn't 'normal' google - it is your school/university's
> email, essentially hosted by google. As such, your institution controls
> access, not google. Google just provides the service to your
> school/university. Often, the setup involves integration with your
> school/university IAM system i.e. your 'identity' (your username) is
> managed by the school/university. This integration makes it easier for
> existing school/university workflows to continue i.e. onboarding of new
> students/staff, removal of accounts when students/staff leave etc. It
> also makes integration with other services, such as on-line LMS (Moodle,
> Blackboard etc) easier as there is just one 'meta directory' of all
> accounts. This is where oauth2 shows its strengths. Your institituion
> essentially becomes a identity provider which Google trusts. When you
> request authorisation credentials, they are provided by your
> institutions IAM system. Your client then submits those authorisation
> credentials to get an access token from Google which you then submit to
> the Google service you want to access (i.e. email).

thank you for the explanation

who is in charge of giving out oauth2 client_id?
does it mean that the university gives out client_id?

is it still google who gives out application id
(the google specific and required additional parameter for the oauth2 client?)

> So, if your locked out, it is because your institution has decided to
> lock you out, not google.

that's good to know

Re: gmail+imap+smtp (oauth2)

[Tim Cross]

Tomas Hlavaty <tom@logand.com> writes:

> On Tue 10 May 2022 at 17:51, Tim Cross <theophilusx@gmail.com> wrote:
>> Tomas Hlavaty <tom@logand.com> writes:
>>> When a school/university demands gmail account
>>> and google locks me out of my gmail account,
>>> what happens?
>>
>> When a school/university makes a decision to use Google as their email
>> provider, it isn't 'normal' google - it is your school/university's
>> email, essentially hosted by google. As such, your institution controls
>> access, not google. Google just provides the service to your
>> school/university. Often, the setup involves integration with your
>> school/university IAM system i.e. your 'identity' (your username) is
>> managed by the school/university. This integration makes it easier for
>> existing school/university workflows to continue i.e. onboarding of new
>> students/staff, removal of accounts when students/staff leave etc. It
>> also makes integration with other services, such as on-line LMS (Moodle,
>> Blackboard etc) easier as there is just one 'meta directory' of all
>> accounts. This is where oauth2 shows its strengths. Your institituion
>> essentially becomes a identity provider which Google trusts. When you
>> request authorisation credentials, they are provided by your
>> institutions IAM system. Your client then submits those authorisation
>> credentials to get an access token from Google which you then submit to
>> the Google service you want to access (i.e. email).
>
> thank you for the explanation
>
> who is in charge of giving out oauth2 client_id?
> does it mean that the university gives out client_id?
>
> is it still google who gives out application id
> (the google specific and required additional parameter for the oauth2 client?)
>

It really depends on exactly how the integration has been configured.
However, the typical model is that your institution would provide the
client authorisation credentials i.e. your institution will say if you
are or are not a valid user and what resources your permitted to access.
This will typically happen with Google acting as a sort of proxy. The
process will go something like 

1. You connect to a login service. This could be a service provided by
Google or one provided by your institution. If it is one provided by
Google, you will login with something like
username@your.institution.edu. Google will then pass the information you
provide in the login (minimum of username and password, but possibly
(likely) a 2FA token as well) to your institutions IAM system. Your
institution will verify the user, password and any additional
information and if all passes, returns an authorisation token to google,
who in turn return it to you. Your client then presents the autorisation
token to Google who then gives you an access token (possibly including a
refresh token). Your client then presents the access token to get access
to whatever service you wanted (i.e. email) and your client does what it
needs to do. In a sense, this is very similar to those web sites which
say "Login wiht Facebook. Github, Google, whatever. The site your trying
to access really just proxies the initial verification process, passing
the details you enter on to the respective ID provider who does the
actual validation. 

So, your institutions IAM system is responsible for the initial
assessment to determine if the client request is permitted. Your
institution could require any number of IDs, really up to them. In this
scenario, it is unlikely they will do application vetting and provide
application IDs, so they would not be required. They could however,
require a different ID - for example, student number. This is the key
point about oauth2 - it isn't prescriptive about exactly what is
involved in the client approval cycle. The oauth2 specification just
mapps out the authorisation workflow, not the speicifics of what
information is passed in the requests/responses - for oaht2, these are
just payloads. 

In the case of Google, they want to be able to control 'classes' of
applications. For them, they want protection against a vulnerable
application being used to compromise large numbers of thier users. For
example, a new email client in the play store which has been approved
and given an application ID. Later, a major security hole is found in
that applicaiton. If it is severe enough, google might suspend the
application ID associated with that application to prevent further
exploits until the application is patched. 

Note that what I've outlined is at a very high level. It is based on a
number of projects I have worked on integrating systems with various
services and identity management systems. As usual, the devil is in the
details. Where oauth2 helps is that it standardises the workflow and
communication processes. It avoids going into details about the lower
level specifics (crypto algorithms, credential attributes, etc). It
focuses on defining the parties involved and the way data flows between
those parties and to some extent, the http headers and/or web form
encoding. All the non-free javascript involved with Google has nothing
to do with oauth2 - that is all just UI stuff google uses on it web
login page i.e. is just the Google specific web page interface. I
suspect (but have not verified) that Google probably provides an API
interface where the data could be sent directly, bypassing the Google UI
page, simply by generating the correctly formatted form data and posting
it to the correct URL (which would require an applicaiton ID). My guess
is this is how other applications (like thunderbird) work. I have heard
of a number of people who have registered themselves as developers with
Google so that they can get a developer ID, which you can use as an
application ID, in order to craft their own process. Only problem with
this is that it requires registering with Google and requires greater
technical understanding than most average users have/want. 

Re: gmail+imap+smtp (oauth2)

[Eric S Fraga]

On Tuesday, 10 May 2022 at 17:51, Tim Cross wrote:
> Of course, the real solution here is that schools/universitites should
> just get out of the business of providing email services to students.

Although I agree with this sentiment, we are told (in my institution) to
always communicate with students using their "official" email, for the
same reason you give for staff having an institutional email:

> Email is considered official records of the institution, so it is fair
> enough that the email account you get as a staff member is actually
> part of your work relationship with the institution and not your
> private email. 

We do need to evolve to universal email addresses, not just for staff.
But there are many reasons why this will not happen including anonymity
when desired.

My main problem with all of this is that most institutions are choosing
one of either MS or Google to provide the services, and neither has a
particularly good track record in following any standards.  Hence making
it difficult for those of us not using their preferred access
methods/tools.

-- 
Eric S Fraga via gnus (Emacs 29.0.50 2022-05-10) on Debian 11.3

Re: gmail+imap+smtp (oauth2)

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > For the 2FA process. Instead of using something like Google
  > Authenticator, you could use an open source TOTP client. Your still
  > going to have to use the non-free Javascript based UI,

From what you've said, it seems that maybe this need only be done once:

  > > If 2FA is enabled, in which situations does the user have to do the
  > > 2FA procedure?  And how many times?  Only once, for setup -- or
  > > repeatedly?
  > >

  > Basically, every time you connect from a new device or new
  > browser and any time you want to modify your settings. 

If all you are doing is transfering mail to/from Gmail using free
software, I think you won't have to change this once it is set up.

However, I am concerned about "connect from a new device".  What does
"device" mean, if you are running a free program on GNU/Linux on a PC?


-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: gmail+imap+smtp (oauth2)

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > (there are
  > some Universities who have been working on an IAM solution which is
  > based on libre packages,

Can anyone give me a pointer to that?

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: gmail+imap+smtp (oauth2)

[Tim Cross]

Richard Stallman <rms@gnu.org> writes:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>   > (there are
>   > some Universities who have been working on an IAM solution which is
>   > based on libre packages,
>
> Can anyone give me a pointer to that?

It has been 5 years since I left that sector, so I don't have any
current links. There is an Educause IAM group which would likely be able
to provide current details. The projects I was referring to were within
Educause and JASIG (which I believe is now Apereo). The openIAM and WS02
sites might also be worth checking out. There was also a project made up
of a few mid/large US Unis who were developing an IAM system, but I
cannot for the life of me recall the name of the project and my search
fu failed me tonight! For some reason, Portland, Harvard and Soutwestern
all come to mind. However, lots can change in 5 years!

Educause IAM group https://www.educause.edu/community/identity-and-access-management-community-group
Apereo http://apereo.org
openIAM https://www.openiam.com

Re: gmail+imap+smtp (oauth2)

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

Thanks for trying to inform me.  Alas, the broad search necessary to
take it up from there is more than I could possibly do.

Does anyone kmow someone in his field that we could plausibly ask for
more mdern information?

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: gmail+imap+smtp (oauth2)

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > In case a school demands you have a Gmail account, it would be useful
  > > if we had instructions to send to the staff, saying, "You may create
  > > the account, choose a password, and tell it to me.  (Since it will
  > > only be for email to and from the school, it makes no difference to me
  > > that school staff will know the password.)  Please choose an account
  > > name with no resemblance to my name.  Please set the account settings
  > > as follows so that my software can access the account."
  > >

  > There is no way any institution would support such a workflow. Apart
  > from the additional resource demands,

It looks like we are miscommunicating.  I think you're thinking about
a change in the university's general practices.  I'm talking about
demanding that some staff person do this once -- for you -- as a
special thing.

Perse will be shocked and say, "But then I would know your password!"
That gives you a chance to shock per again by saying, "That's ok -- I
trust a university employee more than I trust Google."

Then you can say that you refuse to run Google's proprietary software
on your computer, and likewise Microsoft and Apple, because trusting them
that far is against your principles.


-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: gmail+imap+smtp (oauth2)

[Tim Cross]

Richard Stallman <rms@gnu.org> writes:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>   > > In case a school demands you have a Gmail account, it would be useful
>   > > if we had instructions to send to the staff, saying, "You may create
>   > > the account, choose a password, and tell it to me.  (Since it will
>   > > only be for email to and from the school, it makes no difference to me
>   > > that school staff will know the password.)  Please choose an account
>   > > name with no resemblance to my name.  Please set the account settings
>   > > as follows so that my software can access the account."
>   > >
>
>   > There is no way any institution would support such a workflow. Apart
>   > from the additional resource demands,
>
> It looks like we are miscommunicating.  I think you're thinking about
> a change in the university's general practices.  I'm talking about
> demanding that some staff person do this once -- for you -- as a
> special thing.
>
> Perse will be shocked and say, "But then I would know your password!"
> That gives you a chance to shock per again by saying, "That's ok -- I
> trust a university employee more than I trust Google."
>
> Then you can say that you refuse to run Google's proprietary software
> on your computer, and likewise Microsoft and Apple, because trusting them
> that far is against your principles.

and the staff member would say "Sorry, I'm not permitted to do that" or
"Sorry, I cannot do that". 

Things have changed a lot since the 70s. Staff in Universities are far
more restricted on what they are allowed to do and what they can do these
days. There have been numerous cases of staff 'selling' degrees,
entering fake transcriptions, altering results, etc. Due to the
reputational damage such things can cause, lots has been done to lock
down what they are able to do, increased monitoring of what they do and
detailed policy on what they are allowed to do. As a student (or more
accurately, a prospective student), your unlikely to even get close to
someone who could setup/create the account for you. 

This is also ignoring the huge workload imposed on staff these days. It
isn't like the old days where you would talk to the assistant in the
school/department who could do anything. These days, your probably
connected to some unfortunate call centre worker with a set script of
canned responses who will just refer you to some web page. 

Re: gmail+imap+smtp (oauth2)

[Tim Cross]

Tim Cross <theophilusx@gmail.com> writes:

> Richard Stallman <rms@gnu.org> writes:
>
>> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
>> [[[ whether defending the US Constitution against all enemies,     ]]]
>> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>>
>>   > > In case a school demands you have a Gmail account, it would be useful
>>   > > if we had instructions to send to the staff, saying, "You may create
>>   > > the account, choose a password, and tell it to me.  (Since it will
>>   > > only be for email to and from the school, it makes no difference to me
>>   > > that school staff will know the password.)  Please choose an account
>>   > > name with no resemblance to my name.  Please set the account settings
>>   > > as follows so that my software can access the account."
>>   > >
>>
>>   > There is no way any institution would support such a workflow. Apart
>>   > from the additional resource demands,
>>
>> It looks like we are miscommunicating.  I think you're thinking about
>> a change in the university's general practices.  I'm talking about
>> demanding that some staff person do this once -- for you -- as a
>> special thing.
>>
>> Perse will be shocked and say, "But then I would know your password!"
>> That gives you a chance to shock per again by saying, "That's ok -- I
>> trust a university employee more than I trust Google."
>>
>> Then you can say that you refuse to run Google's proprietary software
>> on your computer, and likewise Microsoft and Apple, because trusting them
>> that far is against your principles.
>

I also forgot the one other extremely important point. Your assumption
is that when your asking the staff person to create your account that
the password is just for email. That is extremely unlikely to be the
case. These days, the student will have one login and one password
(likely with 2FA) for everything. They don't have separate accounts for
accessing learning materials, admin/billing/enrolment/transcripts,
email, library, network access etc. All these services are integrated
under a 'same login' (slightly different to single sign on, though many
of the systems would also be SSO).

As for telling them about your principals... Well quite honestly, I
doubt any admin or service desk person or any staff generally will care
about your principals. The likely response will be "Oh well, then you
had better find a different school". Remember, ultimately, it is no skin
off their nose if you enrol or not - their pay packet will be the same. 

Re: gmail+imap+smtp (oauth2)

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > As for telling them about your principals... Well quite honestly, I
  > doubt any admin or service desk person or any staff generally will care
  > about your principals. The likely response will be "Oh well, then you
  > had better find a different school".

I think you're assuming that everyone is Shaw's "reasonable man" who
adapts himself to circumstances.  I'm urging people to be the
"unreasonable man" on whom all progress depends -- and all resistance
to oppression, too.  In the long term, failing to resist ensures total
loss for nearly everyone.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: gmail+imap+smtp (oauth2)

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > Then you can say that you refuse to run Google's proprietary software
  > > on your computer, and likewise Microsoft and Apple, because trusting them
  > > that far is against your principles.

  > and the staff member would say "Sorry, I'm not permitted to do that" or
  > "Sorry, I cannot do that". 

If so, then you ask to talk with the manager.  After that, you ask to
talk with whoever is in charge of admissions.

Just because winning freedom is not easy, that doesn't mean we should
give up at the first hurdle.

By the way, in the 1970s university admissions didn't give students
(let alone prospective students) email accounts.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: gmail+imap+smtp (oauth2)

[Eric S Fraga]

On Friday, 13 May 2022 at 11:10, Richard Stallman wrote:
> By the way, in the 1970s university admissions didn't give students
> (let alone prospective students) email accounts.

Sure but in the 1970s all communication between student (prospective or
current) and the institution was by post.  By the time I was a postgrad,
I got given a pigeon hole by the institution.

I don't see why this is relevant to anything

-- 
Eric S Fraga via gnus (Emacs 29.0.50 2022-05-10) on Debian 11.3

Re: gmail+imap+smtp (oauth2)

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > By the way, in the 1970s university admissions didn't give students
  > > (let alone prospective students) email accounts.

  > Sure but in the 1970s all communication between student (prospective or
  > current) and the institution was by post.  By the time I was a postgrad,
  > I got given a pigeon hole by the institution.

  > I don't see why this is relevant to anything

It corrects an inaccurate presupposition in the message I replied to.
It may not be crucially relevant to this issue, but I did not want to
accept it as true.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: gmail+imap+smtp (oauth2)

[chad]

[-- Attachment #1: Type: text/plain, Size: 2047 bytes --]

There is a small additional wrinkle to Tim Cross' excellent summary:

On Sat, May 7, 2022 at 12:17 AM Tim Cross <theophilusx@gmail.com> wrote:

>
> 2. Use a Google oauth2 compliant client to obtain an oath2 access token
> which you then use as your password in your libre IMAP/SMTP client.
> However, at this time, there doesn't seem to be any libre Google oauth2
> client we can use. If there was, it would be theoretically possible to
> access your emails and send new ones using only libre software and avoid
> needing to login to the non-free settings page to setup application
> passwords.


There exist libre mail client software packages that use oauth2 to talk to
Google/Gmail. For example, nmh, which can be used inside emacs with mh-e.
That said, these packages have practical downsides to pure libre usage,
including both Thunderbird-style "we think we know what you mean" reading
of the terms and conditions as well as user-setup instructions that include
running probably non-libre Google javascript.

In the end, it would be great if the FSF (or someone else) could get Google
to go "on the record" about the details of the Terms & Conditions, but
that's been the state for several years now, and it doesn't seem likely.
(In fact, it's a common refrain among frequent gmail users running Chrome,
Android, and iOS apps that gmail is basically unresponsive except to
periodically push out undesired changes.) Still, it would be great, hope
springs eternal, etc.

In terms of practical advice for people who want to use emacs and are stuck
at least part-time on gmail (or "Google Workspace", or whatever it's called
this month), the various middleman approaches ala mbsync or davmail are
functional. The main complication there is in trying to automate things
that Google (MS Office 365, etc) design as (frequently changing)
interactive web pages, and the results are often fragile or confusing. I'm
a firm proponent of Hanlon's Razor, but it's hard to believe that this
outcome isn't at least tacitly accepted.

Hope that helps,
~Chad

[-- Attachment #2: Type: text/html, Size: 2539 bytes --]

Re: gmail+imap+smtp (oauth2)

[tomas]

[-- Attachment #1: Type: text/plain, Size: 2354 bytes --]

On Sat, May 14, 2022 at 05:43:33PM -0400, chad wrote:
> There is a small additional wrinkle to Tim Cross' excellent summary:
> [about uncertainty] I'm
> a firm proponent of Hanlon's Razor, but it's hard to believe that this
> outcome isn't at least tacitly accepted.

I've vented my opinion several times here, so I fear I'm repeating
myself.

I think bigcorps have, to some extent, given up on controlling [0]
users via proprietary software. Free software has won, in some
strange way (a bit like the Monkey's Paw [1], returning as Open
Source).

To protect their business model, they have to devise other ways.
One of them is to interpose themselves as monopolistic "services"
between people and their everyday tasks: information retrieval
(Google), communications (Facebook, Twitter et al), business
contact (Google, Microsoft, etc.), mobile connectivity (Google,
Apple), marketplace (Amazon)... I could go on.

What we are seeing here is the "service" of identity management,
something up to now reserved to nation states (they used to
issue passports, remember?)

Now I'm coming back to you, Chad: as long as they are trying
to conquer market share, this kind of uncertainty is useful:
they get the "cheaters", i.e. those who are disgusted by the
very concept. Currently those Firefox users. They formally
break the T&C, but we don't go after them... yet.

To offer a historical parallel: I remember a time (must have
been around Windows 3.x) where everyone knew that using a
number divisible by seven [2] gave you a valid Windows license
key, so you could install an illegal copy. I'm convinced that
back then, Microsoft /wanted/ to see its user base extended
by this huge population. They weren't going to pay anyway,
they were those trained in Windows advocating for it at
work, they could be cracked on at will, and oh, the complaining
about those "criminals" "stealing" billions and billions
"worth" of software. What's not to like?

I don't believe Hanlon applied then. I don't believe Hanlon
applies here (that's why I came up with this mix of Hanlon's
razor and Clarke's Third).

Cheers

[0] Control sounds sooo... evil. It's just about business,
   crisp and cold.
[1] https://en.wikipedia.org/wiki/The_Monkey%27s_Paw
[2] the concrete details may be wrong: it's a while
   ago

-- 
t

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: gmail+imap+smtp (oauth2)

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > One issue with OAuth2 schemes is that they periodically force the user
  > through a web-browser-only authentication process that requires non-free
  > JavaScript, in order to get a refresh token.

Is there any disagreement about whether Gmail requires this?

If it does, is it a requirement of Oauth2, or only specifically of Gmail?


-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: gmail+imap+smtp (oauth2)

[Tim Cross]

Richard Stallman <rms@gnu.org> writes:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>   > One issue with OAuth2 schemes is that they periodically force the user
>   > through a web-browser-only authentication process that requires non-free
>   > JavaScript, in order to get a refresh token.
>
> Is there any disagreement about whether Gmail requires this?
>
> If it does, is it a requirement of Oauth2, or only specifically of Gmail?

It is not a requirement of oauth2. It is part of Google's
implementation. 

As to whether it is the only mechanism available to get the necessary
token, I don't know. Someone would need to dig into Google's API. I
suspect there are other API based methods as there are 3rd party apps
which use google oauth2 and don't require the user to use Google's web
interface. I don't know if google has any restrictions on accessing the
API (such as being registered or approved). 

Re: gmail+imap+smtp (oauth2)

[Óscar Fuentes]

Tim Cross <theophilusx@gmail.com> writes:

> Google introduced the concept of app passwords back when they first
> implemented 2FA. Basically, they are just a password based
> authentication workflow which can be used with applications that do not
> support 2FA or oauth2 based authentication and authorisation.

[snip]

> Google is removing access to imap/smtp using your main google
> login/password and will require 2FA and oauth2 for all web based
> authn/authz. However, their documentation implies that app passwords
> will remain as the standad solution for applications which cannot do 2FA
> or oauth2. 

If I read this right:

https://support.google.com/accounts/answer/185833?hl=en

app passwords require to have 2FA turned on. So you have to go through
the 2FA process anyway with app passwords.

Re: gmail+imap+smtp (oauth2)

[Robert Pluim]

>>>>> On Wed, 04 May 2022 17:35:45 +0200, Óscar Fuentes <ofv@wanadoo.es> said:

    Óscar> Tim Cross <theophilusx@gmail.com> writes:
    >> Google introduced the concept of app passwords back when they first
    >> implemented 2FA. Basically, they are just a password based
    >> authentication workflow which can be used with applications that do not
    >> support 2FA or oauth2 based authentication and authorisation.

    Óscar> [snip]

    >> Google is removing access to imap/smtp using your main google
    >> login/password and will require 2FA and oauth2 for all web based
    >> authn/authz. However, their documentation implies that app passwords
    >> will remain as the standad solution for applications which cannot do 2FA
    >> or oauth2. 

    Óscar> If I read this right:

    Óscar> https://support.google.com/accounts/answer/185833?hl=en

    Óscar> app passwords require to have 2FA turned on. So you have to go through
    Óscar> the 2FA process anyway with app passwords.

When creating the app password, maybe (itʼs been so long since I
created my app password for Gnus that I donʼt remember). But *using*
the app password just requires presenting it using the normal
IMAP/SMTP/etc authentication protocols.

Robert
-- 

Re: gmail+imap+smtp (oauth2)

[Óscar Fuentes]

Robert Pluim <rpluim@gmail.com> writes:

>     Óscar> If I read this right:
>
>     Óscar> https://support.google.com/accounts/answer/185833?hl=en
>
>     Óscar> app passwords require to have 2FA turned on. So you have to go through
>     Óscar> the 2FA process anyway with app passwords.
>
> When creating the app password, maybe (itʼs been so long since I
> created my app password for Gnus that I donʼt remember). But *using*
> the app password just requires presenting it using the normal
> IMAP/SMTP/etc authentication protocols.

Good to know, thanks. So you don't have to bear with 2FA each time you
access your e-mail, but the privacy concern doesn't change: you must
give your phone number to Google.

Re: gmail+imap+smtp (oauth2)

[Tim Cross]

Óscar Fuentes <ofv@wanadoo.es> writes:

> Robert Pluim <rpluim@gmail.com> writes:
>
>>     Óscar> If I read this right:
>>
>>     Óscar> https://support.google.com/accounts/answer/185833?hl=en
>>
>>     Óscar> app passwords require to have 2FA turned on. So you have to go through
>>     Óscar> the 2FA process anyway with app passwords.
>>
>> When creating the app password, maybe (itʼs been so long since I
>> created my app password for Gnus that I donʼt remember). But *using*
>> the app password just requires presenting it using the normal
>> IMAP/SMTP/etc authentication protocols.
>
> Good to know, thanks. So you don't have to bear with 2FA each time you
> access your e-mail, but the privacy concern doesn't change: you must
> give your phone number to Google.

Not sure if that is true. Google originally used SMS based 2FA (as did
many service providers originally). However, that approach is no longer
preferred (I think they still support it, but don't recommend it).
However, I think they now support a couple of different 2FA schemes,
such as using an authenticator app you register by scanning a QR code
and I think they now also support hardware keys like yubikey. They don't
need yuour phone number for these methods (but I'm not sure if they
still require you to provide it).

Still, if privacy is your concern, I'm not sure you should be using
google anyway! Worrying about them having your phone number is minor
compared to all the data they have about you in your emails (which
probably also have your phone number in some of them anyway). 

Re: gmail+imap+smtp (oauth2)

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Good to know, thanks. So you don't have to bear with 2FA each time you
  > access your e-mail, but the privacy concern doesn't change: you must
  > give your phone number to Google.

Does it have to be _your_ phone number, or can it be any phone
that you can answer at the time of creating the app password?
Will Google ever phone you again on the same number?

Another question is, does setting up the app password require
a computer running nonfree software?  (For instance, a mobile phone.)
Can you do this with a landline?


-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: gmail+imap+smtp (oauth2)

[Brian Cully via Emacs development discussions.]

Richard Stallman <rms@gnu.org> writes:

> Does it have to be _your_ phone number, or can it be any phone
> that you can answer at the time of creating the app password?
> Will Google ever phone you again on the same number?
>
> Another question is, does setting up the app password require
> a computer running nonfree software?  (For instance, a mobile 
> phone.)
> Can you do this with a landline?

It does not have to be your phone number, no. They will send an 
SMS to whatever number you choose (so long as its not on a 
blacklist somewhere). You do not need to run non-free software, 
presuming you can receive SMS with non-free software (there are 
services for such things).

However, you have the choice of using TOTP for 2FA[1], in which 
case you can use any number of free applications to generate codes 
for you. If you use SMS as your 2FA, Google will send messages to 
you periodically as you attempt to log in to services (though only 
with your main password - not app passwords). Once converted to 
TOTP, though, I do not believe Google will ever try to contact you 
again. I have set up Google accounts using burner numbers and 
converted them to TOTP without any issue over the years. However, 
there’s obviously no guarantee that Google will continue to allow 
this in the future.

-bjc

[1] There may be restrictions I’m not aware of when using TOTP, as 
I’d set mine up a long time ago. You may, for instance, need to be 
able to receive SMS in order to do the initial TOTP setup. You 
definitely need to use SMS to do initial account setup.

Re: gmail+imap+smtp (oauth2)

[Stefan Monnier]

> SMS in order to do the initial TOTP setup. You definitely need to use SMS to
> do initial account setup.

FWIW< I seem to remember that the initial setup could be done with
a landline: you'd receive a phone call with an automatic voice spelling
out a number you'd then have to enter into a web form.


        Stefan

Re: gmail+imap+smtp (oauth2)

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > FWIW< I seem to remember that the initial setup could be done with
  > a landline: you'd receive a phone call with an automatic voice spelling
  > out a number you'd then have to enter into a web form.

While that avoids the SMS, I suspect it requires running nonfree
software (JS code).

It would be useful for someone to check that.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: gmail+imap+smtp (oauth2)

[Tim Cross]

Brian Cully via "Emacs development discussions." <emacs-devel@gnu.org> writes:

> Richard Stallman <rms@gnu.org> writes:
>
>> Does it have to be _your_ phone number, or can it be any phone
>> that you can answer at the time of creating the app password?
>> Will Google ever phone you again on the same number?
>>
>> Another question is, does setting up the app password require
>> a computer running nonfree software?  (For instance, a mobile phone.)
>> Can you do this with a landline?
>
> It does not have to be your phone number, no. They will send an SMS to whatever
> number you choose (so long as its not on a blacklist somewhere). You do not need
> to run non-free software, presuming you can receive SMS with non-free software
> (there are services for such things).
>
> However, you have the choice of using TOTP for 2FA[1], in which case you can use
> any number of free applications to generate codes for you. If you use SMS as
> your 2FA, Google will send messages to you periodically as you attempt to log in
> to services (though only with your main password - not app passwords). Once
> converted to TOTP, though, I do not believe Google will ever try to contact you
> again. I have set up Google accounts using burner numbers and converted them to
> TOTP without any issue over the years. However, there’s obviously no guarantee
> that Google will continue to allow this in the future.
>
> -bjc
>
> [1] There may be restrictions I’m not aware of when using TOTP, as I’d set mine
> up a long time ago. You may, for instance, need to be able to receive SMS in
> order to do the initial TOTP setup. You definitely need to use SMS to do initial
> account setup.

The SMS workflow is not Google's preferred 2FA. When Google first rolled
out 2FA, SMS based codes were widely used and it was one of the
techniques recommended by NIST. However, due to issues with number
spoofing and social engineering of Telco service desks to redirect
numbers etc, NIST now recommends against SMS based 2FA. 

I'm not sure about whehter you still require SMS in initial account
setup for Google. It has been too long since I moved my 2FA to use
non-SMS based techniques. I do notice that when I do login to google
from a new device/browser, the SMS option is still shown as one option,
but it is not the default/preferred option, only a fallback one. 

I do still wonder though - if your so concerned about privacy and google
having your phone number, how you can be comfortable with them having
your email data? 

Re: gmail+imap+smtp (oauth2)

[Brian Cully via Emacs development discussions.]

Tim Cross <theophilusx@gmail.com> writes:
> I do still wonder though - if your so concerned about privacy 
> and google
> having your phone number, how you can be comfortable with them 
> having
> your email data?

I don’t use the accounts for email, I use them as alternate 
identities for other services. I’m sure Google can piece things 
together to figure out I’m behind everything, should they care to, 
but in these instances it’s not Google I’m worried about.

-bjc

Re: gmail+imap+smtp (oauth2)

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > I do still wonder though - if your so concerned about privacy and google
  > having your phone number, how you can be comfortable with them having
  > your email data? 

Maybe they WERE NOT comfortable with that.
Someone told us that his school required students to use Gmail,
and did not allow them to forward the mail elsewhere.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: gmail+imap+smtp (oauth2)

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > I have set up Google accounts using burner numbers and 
  > converted them to TOTP without any issue over the years. However, 
  > there’s obviously no guarantee that Google will continue to allow 
  > this in the future.

Would you like to set up TOTP using libre software, on a 100%-libre
system, with Javascript disabled, and verify that it works -- to
verify for sure that it can be done and really works?

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: gmail+imap+smtp (oauth2)

[Brian Cully via Emacs development discussions.]

Richard Stallman <rms@gnu.org> writes:

> Would you like to set up TOTP using libre software, on a 
> 100%-libre
> system, with Javascript disabled, and verify that it works -- to
> verify for sure that it can be done and really works?

See my previous response about TOTP software.

Regarding the other matter: you cannot log in to Google (via 
https://accounts.google.com/) with Javascript disabled. It renders 
the following message:

--8<---------------cut here---------------start------------->8---
Couldn’t sign you in

The browser you’re using doesn’t support JavaScript, or has 
JavaScript turned off.

To keep your Google Account secure, try signing in on a browser 
that has JavaScript turned on.
--8<---------------cut here---------------end--------------->8---

-bjc

Re: gmail+imap+smtp (oauth2)

[Tim Cross]

Robert Pluim <rpluim@gmail.com> writes:

>>>>>> On Wed, 04 May 2022 17:35:45 +0200, Óscar Fuentes <ofv@wanadoo.es> said:
>
>     Óscar> Tim Cross <theophilusx@gmail.com> writes:
>     >> Google introduced the concept of app passwords back when they first
>     >> implemented 2FA. Basically, they are just a password based
>     >> authentication workflow which can be used with applications that do not
>     >> support 2FA or oauth2 based authentication and authorisation.
>
>     Óscar> [snip]
>
>     >> Google is removing access to imap/smtp using your main google
>     >> login/password and will require 2FA and oauth2 for all web based
>     >> authn/authz. However, their documentation implies that app passwords
>     >> will remain as the standad solution for applications which cannot do 2FA
>     >> or oauth2. 
>
>     Óscar> If I read this right:
>
>     Óscar> https://support.google.com/accounts/answer/185833?hl=en
>
>     Óscar> app passwords require to have 2FA turned on. So you have to go through
>     Óscar> the 2FA process anyway with app passwords.
>
> When creating the app password, maybe (itʼs been so long since I
> created my app password for Gnus that I donʼt remember). But *using*
> the app password just requires presenting it using the normal
> IMAP/SMTP/etc authentication protocols.
>

2FA has nothing to do with the application passwords (at least not
directly). You just need to have enabled it in order to enable
application passwords for your account. 

Re: gmail+imap+smtp (oauth2)

[Tim Cross]

Óscar Fuentes <ofv@wanadoo.es> writes:

> Tim Cross <theophilusx@gmail.com> writes:
>
>> Google introduced the concept of app passwords back when they first
>> implemented 2FA. Basically, they are just a password based
>> authentication workflow which can be used with applications that do not
>> support 2FA or oauth2 based authentication and authorisation.
>
> [snip]
>
>> Google is removing access to imap/smtp using your main google
>> login/password and will require 2FA and oauth2 for all web based
>> authn/authz. However, their documentation implies that app passwords
>> will remain as the standad solution for applications which cannot do 2FA
>> or oauth2. 
>
> If I read this right:
>
> https://support.google.com/accounts/answer/185833?hl=en
>
> app passwords require to have 2FA turned on. So you have to go through
> the 2FA process anyway with app passwords.

All new google accounts have to do 2FA already. I suspect they will soon
require existing accounts to enable it as well. Enabling 2FA is very
simple and it works well. Basically, you only need to do the 2FA login
the first time you connect from a new device or browser. 2FA based
authentication is becoming the norm and I appeciate the additional
protection.

Re: gmail+imap+smtp (oauth2)

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > All new google accounts have to do 2FA already. I suspect they will soon
  > require existing accounts to enable it as well. Enabling 2FA is very
  > simple and it works well. Basically, you only need to do the 2FA login
  > the first time you connect from a new device or browser. 2FA based
  > authentication is becoming the norm

I'll take your word for all those things, and they are all
significant.  However, in the free software movement, we have an
additional overriding condition: it must not require people to run
nonfree software.


-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: gmail+imap+smtp (oauth2)

[Cesar Crusius]

[-- Attachment #1: Type: text/plain, Size: 1224 bytes --]

Uwe Brauer <oub@mat.ucm.es> writes:

> Hi
>
> I am forced to use gmail at my university (the features google offer are
> a bit different to the personal accounts)
> and also for some private stuff.
>
> There was a discussion in 2022 about the following issue, but then
> google dropped this subject because of Covid, however now it seems to
> back:
>
> Now google keeps sending me message that from 30th of May 3rd party
> packages cannot connect anymore to imap+smtp via the traditional way.
>
> It is my understanding that it might be possible make emacs (gnus) work
> with gmail under those circumstances, that is using oauth2.
>
> Has anybody got that to work?
>
> If so, can he/she share its setting?

Google seems to be clamping down on OAuth access methods and those of us using it to access GMail have been getting a message that our OAuth clients will be blocked starting October 3rd. This is because we're using "out of bound OAuth2". From what I can tell, the packages need to be rewritten to be "compliant," and from what I remember from previous discussions, making them compliant may be a non-trivial task involving registering an "official" application and so on.

-- 
Cesar Crusius

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 658 bytes --]

Re: gmail+imap+smtp (oauth2)

[Tim Cross]

Cesar Crusius <cesar.crusius@gmail.com> writes:

> [[PGP Signed Part:Undecided]]
> Uwe Brauer <oub@mat.ucm.es> writes:
>
>> Hi
>>
>> I am forced to use gmail at my university (the features google offer are
>> a bit different to the personal accounts)
>> and also for some private stuff.
>>
>> There was a discussion in 2022 about the following issue, but then
>> google dropped this subject because of Covid, however now it seems to
>> back:
>>
>> Now google keeps sending me message that from 30th of May 3rd party
>> packages cannot connect anymore to imap+smtp via the traditional way.
>>
>> It is my understanding that it might be possible make emacs (gnus) work
>> with gmail under those circumstances, that is using oauth2.
>>
>> Has anybody got that to work?
>>
>> If so, can he/she share its setting?
>
> Google seems to be clamping down on OAuth access methods and those of us using
> it to access GMail have been getting a message that our OAuth clients will be
> blocked starting October 3rd. This is because we're using "out of bound OAuth2".
> From what I can tell, the packages need to be rewritten to be "compliant," and
> from what I remember from previous discussions, making them compliant may be a
> non-trivial task involving registering an "official" application and so on.

From what I recall from previous discussions, the issue centres around
Google's T&C and interpretation of those terms & conditions. 

Google requires that apps using oauth2 to access their services be
approved by them and assigned an application ID. The problem is that the
T&C require that the oauth2 tokens must be kept secret. However, this is an
issue because you cannot have both open source code and a secret
applicaiton ID token embedded in the source code.

It has been suggested by some that this is a misintgerpretation of the
T&C and that the application ID is possibly not one of the toekns which
must remain secret (in which case, it could be embedded in the code).
Attempts to get clarification on this point from Google have failed to
get a response. This is possibly something the FSF could assist with. In
particular, they could get the necessary clarification and if there is a
problem, clearly articulate the issues to Google and ask them what their
plans/solution is for open source applications. (at this point, I
suspect their solution is application passwords).