bug#9113: 24.0.50; auth-sources: .authinfo versus .authinfo.gpg

bug#9113: 24.0.50; auth-sources: .authinfo versus .authinfo.gpg

[Roland Winkler]

If an authinfo file does not exists and the user has not customized
anything, something like smtpmail will create a new file .authinfo
with the appropriate entry.

I suggest that instead the code should try first to generate a file
.authinfo.gpg and if this fails it should warn the user that Emacs
is going to create a file .authinfo, which can be very unsafe.

In this context, the doc string of auth-sources is, unfortunately,
not too helpful:

  See the auth.info manual for details.
  [snip]
  It's best to customize this with `M-x customize-variable' because
  the choices can get pretty complex."

The default value of auth-sources should be such that the user is,
at least, on the safe side.


In GNU Emacs 24.0.50.1 (x86_64-unknown-linux-gnu, GTK+ Version 2.20.1)
 of 2011-07-16 on regnitz
Windowing system distributor `The X.Org Foundation', version 11.0.10706000
Important settings:
  value of $LC_ALL: nil
  value of $LC_COLLATE: C
  value of $LC_CTYPE: nil
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: en_GB.utf8
  value of $LANG: en_US.ISO-8859-15
  value of $XMODIFIERS: nil
  locale-coding-system: iso-latin-9-unix
  default enable-multibyte-characters: t

Major mode: Mail

bug#9113: 24.0.50; auth-sources: .authinfo versus .authinfo.gpg

[Ted Zlatanov]

On Sun, 17 Jul 2011 22:08:22 -0500 "Roland Winkler" <winkler@gnu.org> wrote: 

RW> If an authinfo file does not exists and the user has not customized
RW> anything, something like smtpmail will create a new file .authinfo
RW> with the appropriate entry.

RW> I suggest that instead the code should try first to generate a file
RW> .authinfo.gpg and if this fails it should warn the user that Emacs
RW> is going to create a file .authinfo, which can be very unsafe.

RW> In this context, the doc string of auth-sources is, unfortunately,
RW> not too helpful:

RW>   See the auth.info manual for details.
RW>   [snip]
RW>   It's best to customize this with `M-x customize-variable' because
RW>   the choices can get pretty complex."

RW> The default value of auth-sources should be such that the user is,
RW> at least, on the safe side.

The Emacs maintainers asked me to make the default unencrypted.  I don't
think they will change their position.

Ted

bug#9113: 24.0.50; auth-sources: .authinfo versus .authinfo.gpg

[Stefan Monnier]

> The Emacs maintainers asked me to make the default unencrypted.  I don't
> think they will change their position.

I can't remember exactly how we got there.  But I do agree that saving
a password unencrypted by default is not a good idea.


        Stefan

bug#9113: 24.0.50; auth-sources: .authinfo versus .authinfo.gpg

[Ted Zlatanov]

On Wed, 25 Jan 2012 21:02:12 -0500 Stefan Monnier <monnier@iro.umontreal.ca> wrote: 

>> The Emacs maintainers asked me to make the default unencrypted.  I don't
>> think they will change their position.

SM> I can't remember exactly how we got there.  But I do agree that saving
SM> a password unencrypted by default is not a good idea.

I don't recall exactly either.  But here's how we can proceed.  We have
several options:

1) go back to authinfo.gpg as the first choice

2) use unencrypted authinfo with encrypted password tokens, which looks like this:

machine supertest password gpg:jA0EAwMC2tUEaZgM7A5gyWM/owySdCOS/cjoFCuf8LI1d1kYX7z6cjsNkakM04u1geh/iesqyH3XQFI+SEVLb/oEC/EoQ0LIgRRoBiLyu9XZWN1ytY7MQxpPZniFz13oGV4/Dwl8yrP3Hba5LfQpHy2FZRM=

3) work on the libnettle support (automatic if we use GnuTLS) so the
external GPG executable is not needed to generate encrypted password
tokens or encrypted authinfo files

4) use Daiki Ueno's plist storage format (already in auth-source but not
well tested AFAIK)

5) ask the user if he has no authinfo file what he wants to do, and
choose sensible defaults from the above depending on whether EPA/EPG and
GPG; or libnettle are available.  If we do that, `auth-sources' will be
set to 'ask by default.

Additionally, we should decide if any of this is happening for 24.1.  I
would really prefer to make the default more secure for 24.1.

Ted

bug#9113: 24.0.50; auth-sources: .authinfo versus .authinfo.gpg

[Stefan Monnier]

>>> The Emacs maintainers asked me to make the default unencrypted.  I don't
>>> think they will change their position.
SM> I can't remember exactly how we got there.  But I do agree that saving
SM> a password unencrypted by default is not a good idea.
> I don't recall exactly either.  But here's how we can proceed.  We have
> several options:
> 1) go back to authinfo.gpg as the first choice

I'm not sure what this means: how does it fix the problem, what other
consequences does it have?  E.g. will Emacs end up asking for my
password to read autoinfo.gpg even though the thing it's looking for is
not there?

> 2) use unencrypted authinfo with encrypted password tokens, which
>    looks like this:

> machine supertest password
> gpg:jA0EAwMC2tUEaZgM7A5gyWM/owySdCOS/cjoFCuf8LI1d1kYX7z6cjsNkakM04u1geh/iesqyH3XQFI+SEVLb/oEC/EoQ0LIgRRoBiLyu9XZWN1ytY7MQxpPZniFz13oGV4/Dwl8yrP3Hba5LfQpHy2FZRM=

That might be a good option.

> Additionally, we should decide if any of this is happening for 24.1.  I
> would really prefer to make the default more secure for 24.1.

IIRC for 23 the default was to keep the password for the current session
and not to store it in any file at all.  I think it's a better default
than writing it in clear in some file, so at least for 24.1 reverting to
the Emacs-23 default is very attractive.

Another option (the better long-term option) is to use an external
keychain service to handle these issues.  That's what we should focus on
for the "next time".


        Stefan

bug#9113: 24.0.50; auth-sources: .authinfo versus .authinfo.gpg

[Lars Ingebrigtsen]

Stefan Monnier <monnier@IRO.UMontreal.CA> writes:

> I'm not sure what this means: how does it fix the problem, what other
> consequences does it have?  E.g. will Emacs end up asking for my
> password to read autoinfo.gpg even though the thing it's looking for is
> not there?

Yes.  That was the major reason for not using .authinfo.gpg.

>> 2) use unencrypted authinfo with encrypted password tokens, which
>>    looks like this:
>
>> machine supertest password
>> gpg:jA0EAwMC2tUEaZgM7A5gyWM/owySdCOS/cjoFCuf8LI1d1kYX7z6cjsNkakM04u1geh/iesqyH3XQFI+SEVLb/oEC/EoQ0LIgRRoBiLyu9XZWN1ytY7MQxpPZniFz13oGV4/Dwl8yrP3Hba5LfQpHy2FZRM=
>
> That might be a good option.

Yes.  But it will require the user to type in a password to get to the
password.  :-)  And again, programs like Firefox defaults to storing the
passwords in non-encrypted files, so I don't really see why Emacs should
be more difficult to use than Firefox.

> IIRC for 23 the default was to keep the password for the current session
> and not to store it in any file at all.  I think it's a better default
> than writing it in clear in some file, so at least for 24.1 reverting to
> the Emacs-23 default is very attractive.

Well, Emacs 23 just made you write the .authinfo file by hand.  Emacs 24
prompts you for whether you want to store the password or not.  If you
don't want to, say "n".

-- 
(domestic pets only, the antidote for overdose, milk.)
  http://lars.ingebrigtsen.no  *  Sent from my Rome

bug#9113: 24.0.50; auth-sources: .authinfo versus .authinfo.gpg

[Achim Gratz]

Ted Zlatanov <tzz@lifelogs.com> writes:
> 2) use unencrypted authinfo with encrypted password tokens, which looks like this:
>
> machine supertest password gpg:jA0EAwMC2tUEaZgM7A5gyWM/owySdCOS/cjoFCuf8LI1d1kYX7z6cjsNkakM04u1geh/iesqyH3XQFI+SEVLb/oEC/EoQ0LIgRRoBiLyu9XZWN1ytY7MQxpPZniFz13oGV4/Dwl8yrP3Hba5LfQpHy2FZRM=

That looks appealing.  Can it work with ssh-agent also?


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Waldorf MIDI Implementation & additional documentation:
http://Synth.Stromeko.net/Downloads.html#WaldorfDocs

bug#9113: 24.0.50; auth-sources: .authinfo versus .authinfo.gpg, bug#9113: 24.0.50; auth-sources: .authinfo versus .authinfo.gpg, bug#9113: 24.0.50; auth-sources: .authinfo versus .authinfo.gpg

[Ted Zlatanov]

>>> 2) use unencrypted authinfo with encrypted password tokens, which
>>> looks like this:
>> 
>>> machine supertest password
>>> gpg:jA0EAwMC2tUEaZgM7A5gyWM/owySdCOS/cjoFCuf8LI1d1kYX7z6cjsNkakM04u1geh/iesqyH3XQFI+SEVLb/oEC/EoQ0LIgRRoBiLyu9XZWN1ytY7MQxpPZniFz13oGV4/Dwl8yrP3Hba5LfQpHy2FZRM=

On Thu, 26 Jan 2012 18:53:46 +0100 Achim Gratz <Stromeko@nexgo.de> wrote: 

AG> That looks appealing.  Can it work with ssh-agent also?

No, unfortunately.

On Thu, 26 Jan 2012 12:28:47 -0500 Stefan Monnier <monnier@IRO.UMontreal.CA> wrote: 

SM> That might be a good option.

It works fairly well but it's hacky, and can't be shared with other
programs.  I'd like to implement it with libnettle at least, so it
doesn't depend on the external gpg utility.  But yes, we could do this
one and it would work on all platforms with libnettle.

On Thu, 26 Jan 2012 18:52:25 +0100 Lars Ingebrigtsen <larsi@gnus.org> wrote: 

LI> Yes.  But it will require the user to type in a password to get to the
LI> password.  :-)  And again, programs like Firefox defaults to storing the
LI> passwords in non-encrypted files, so I don't really see why Emacs should
LI> be more difficult to use than Firefox.

The encryption doesn't have to be strong.  It could use a well-known
secret that the user can override, rather than an actual passphrase, and
then no questions will be asked.

SM> Another option (the better long-term option) is to use an external
SM> keychain service to handle these issues.  That's what we should focus on
SM> for the "next time".

Do you mean gpg-agent or the OS keychain?  Neither is available on all
platforms consistently.

>> IIRC for 23 the default was to keep the password for the current session
>> and not to store it in any file at all.  I think it's a better default
>> than writing it in clear in some file, so at least for 24.1 reverting to
>> the Emacs-23 default is very attractive.

LI> Well, Emacs 23 just made you write the .authinfo file by hand.  Emacs 24
LI> prompts you for whether you want to store the password or not.  If you
LI> don't want to, say "n".

One possible flow:

If the user says `y' then we can ask (if `auth-sources' is 'ask) 
"Do you want to keep your passwords in a GPG-encrypted file?"

If they say `y' then set `auth-sources' to "~/.authinfo.gpg" and check
that EPA/EPG are enabled. If GPG is not available, what do we do? Use
libnettle? Or explain and pretend they said `n'?

If they say `n' then set `auth-sources' to "~/.authinfo".

So it's one extra step.  But it is getting unwieldy.

Ted

bug#9113: 24.0.50; auth-sources: .authinfo versus .authinfo.gpg, bug#9113: 24.0.50; auth-sources: .authinfo versus .authinfo.gpg, bug#9113: 24.0.50; auth-sources: .authinfo versus .authinfo.gpg

[Stefan Monnier]

SM> That might be a good option.
> It works fairly well but it's hacky, and can't be shared with other
> programs.

Indeed, it's a major downside.

> I'd like to implement it with libnettle at least, so it doesn't depend
> on the external gpg utility.

But that would make it work even less with other programs.

LI> Yes.  But it will require the user to type in a password to get to the
LI> password.  :-)  And again, programs like Firefox defaults to storing the
LI> passwords in non-encrypted files, so I don't really see why Emacs should
LI> be more difficult to use than Firefox.

I don't know about you, but I don't let Firefox store my mailbox's
password.  I have a lot of passwords stored in Firefox's database, but
they're all things I don't really care about (e.g. passwords to log into
some stupid web-forums).

SM> Another option (the better long-term option) is to use an external
SM> keychain service to handle these issues.  That's what we should focus on
SM> for the "next time".
> Do you mean gpg-agent or the OS keychain?

I mean the keychain.

> Neither is available on all platforms consistently.

AFAIK all platforms have a keychain nowadays and it's the best place to
put sensitive passwords such as the ones used to access your IMAP server.

>>> IIRC for 23 the default was to keep the password for the current session
>>> and not to store it in any file at all.  I think it's a better default
>>> than writing it in clear in some file, so at least for 24.1 reverting to
>>> the Emacs-23 default is very attractive.
LI> Well, Emacs 23 just made you write the .authinfo file by hand.  Emacs 24
LI> prompts you for whether you want to store the password or not.  If you
LI> don't want to, say "n".

Yes, I guess it's good enough.

> One possible flow:
> If the user says `y' then we can ask (if `auth-sources' is 'ask) 
> "Do you want to keep your passwords in a GPG-encrypted file?"

> If they say `y' then set `auth-sources' to "~/.authinfo.gpg" and check
> that EPA/EPG are enabled. If GPG is not available, what do we do? Use
> libnettle? Or explain and pretend they said `n'?

If GPG is not available, ask a different question, as in "It will be
saved in cleartext, is that OK?"


        Stefan

bug#9113: 24.0.50; auth-sources: .authinfo versus .authinfo.gpg, bug#9113: 24.0.50; auth-sources: .authinfo versus .authinfo.gpg, bug#9113: 24.0.50; auth-sources: .authinfo versus .authinfo.gpg

[Lars Ingebrigtsen]

Stefan Monnier <monnier@IRO.UMontreal.CA> writes:

> I don't know about you, but I don't let Firefox store my mailbox's
> password.  I have a lot of passwords stored in Firefox's database, but
> they're all things I don't really care about (e.g. passwords to log into
> some stupid web-forums).

I think it's fairly normal to let your mail reader store your email
password.  So replace Firefox with Thunderbird or Mail.app, and the
passwords will (again) be unencrypted, I think?  Or does (say) OS X (or
Ubuntu) start a key chain when you log in, and then Thunderbird consults
that when it connects to the IMAP server?

-- 
(domestic pets only, the antidote for overdose, milk.)
  http://lars.ingebrigtsen.no  *  Sent from my Rome

bug#9113: 24.0.50; auth-sources: .authinfo versus .authinfo.gpg, bug#9113: 24.0.50; auth-sources: .authinfo versus .authinfo.gpg, bug#9113: 24.0.50; auth-sources: .authinfo versus .authinfo.gpg

[Stefan Monnier]

> Or does (say) OS X (or Ubuntu) start a key chain when you log in, and
> then Thunderbird consults that when it connects to the IMAP server?

Exactly.  So, yes, I want Emacs to support the system's keychain tool,
since it's the right solution for the job.


        Stefan

bug#9113: 24.0.50; auth-sources: .authinfo versus .authinfo.gpg, bug#9113: 24.0.50; auth-sources: .authinfo versus .authinfo.gpg, bug#9113: 24.0.50; auth-sources: .authinfo versus .authinfo.gpg

[Lars Ingebrigtsen]

Stefan Monnier <monnier@IRO.UMontreal.CA> writes:

> Exactly.  So, yes, I want Emacs to support the system's keychain tool,
> since it's the right solution for the job.

If that's possible, then it would indeed be a lot better than stashing
the credentials in a file.

-- 
(domestic pets only, the antidote for overdose, milk.)
  http://lars.ingebrigtsen.no  *  Sent from my Rome

bug#9113: 24.0.50; auth-sources: .authinfo versus .authinfo.gpg, bug#9113: 24.0.50; auth-sources: .authinfo versus .authinfo.gpg, bug#9113: 24.0.50; auth-sources: .authinfo versus .authinfo.gpg

[Michael Albinus]

Stefan Monnier <monnier@IRO.UMontreal.CA> writes:

>> Or does (say) OS X (or Ubuntu) start a key chain when you log in, and
>> then Thunderbird consults that when it connects to the IMAP server?
>
> Exactly.  So, yes, I want Emacs to support the system's keychain tool,
> since it's the right solution for the job.

auth-sources.el supports already secrets.el, which is an interface to
Gnome keyring and KWallet, respectively.

The problem is, that there is no default under which name a password is
stored there. Evrery application seems to use its own naming scheme.

>         Stefan

Best regards, Michael.

bug#9113: 24.0.50; auth-sources: .authinfo versus .authinfo.gpg, bug#9113: 24.0.50; auth-sources: .authinfo versus .authinfo.gpg, bug#9113: 24.0.50; auth-sources: .authinfo versus .authinfo.gpg

[Stefan Monnier]

>>> Or does (say) OS X (or Ubuntu) start a key chain when you log in, and
>>> then Thunderbird consults that when it connects to the IMAP server?
>> Exactly.  So, yes, I want Emacs to support the system's keychain tool,
>> since it's the right solution for the job.
> auth-sources.el supports already secrets.el, which is an interface to
> Gnome keyring and KWallet, respectively.

So that's what we should use by default when available.

> The problem is, that there is no default under which name a password is
> stored there.  Every application seems to use its own naming scheme.

While it is probably a problem for users, I don't think it's a problem
for Emacs: it just means that the password you store with one
application won't automatically work in some other application when
accessing the same service on the same host.


        Stefan

bug#9113: 24.0.50; auth-sources: .authinfo versus .authinfo.gpg, bug#9113: 24.0.50; auth-sources: .authinfo versus .authinfo.gpg, bug#9113: 24.0.50; auth-sources: .authinfo versus .authinfo.gpg

[Ted Zlatanov]

On Tue, 31 Jan 2012 12:51:48 -0500 Stefan Monnier <monnier@IRO.UMontreal.CA> wrote: 

>>>> Or does (say) OS X (or Ubuntu) start a key chain when you log in, and
>>>> then Thunderbird consults that when it connects to the IMAP server?
>>> Exactly.  So, yes, I want Emacs to support the system's keychain tool,
>>> since it's the right solution for the job.
>> auth-sources.el supports already secrets.el, which is an interface to
>> Gnome keyring and KWallet, respectively.

SM> So that's what we should use by default when available.

I don't think secrets.el has a probing function to decide if there's
something that can talk to us via the Secrets API.  If that was
possible, we could make it the first choice.  But I'm concerned that
then we *automatically* pick one solution for some users and another for
others, and I'm going to have to support that.

On Mac OS X, I would really like to use the system keychain.  But the
bindings were never finished and I don't know enough to do it myself.

>> The problem is, that there is no default under which name a password is
>> stored there.  Every application seems to use its own naming scheme.

SM> While it is probably a problem for users, I don't think it's a problem
SM> for Emacs: it just means that the password you store with one
SM> application won't automatically work in some other application when
SM> accessing the same service on the same host.

I chose to use the netrc/authinfo format to be compatible with other
applications; I could have used something much more capable otherwise.
Similarly for keychains I think we should try to be consistent with
Firefox and Chrome, at least for HTTP/HTTPS and probably in general.
Compatibility with those applications is a big benefit to our users.

Ted

bug#9113: 24.0.50; auth-sources: .authinfo versus .authinfo.gpg

[Michael Albinus]

Ted Zlatanov <tzz@lifelogs.com> writes:

> I don't think secrets.el has a probing function to decide if there's
> something that can talk to us via the Secrets API.

(ignore-errors
  (require 'secrets)
  secrets-enabled)

> Ted

Best regards, Michael.

bug#9113: 24.0.50; auth-sources: .authinfo versus .authinfo.gpg, bug#9113: 24.0.50; auth-sources: .authinfo versus .authinfo.gpg, bug#9113: 24.0.50; auth-sources: .authinfo versus .authinfo.gpg

[Daiki Ueno]

Ted Zlatanov <tzz@lifelogs.com> writes:

>>>> 2) use unencrypted authinfo with encrypted password tokens, which
>>>> looks like this:
>>> 
>>>> machine supertest password
>>>> gpg:jA0EAwMC2tUEaZgM7A5gyWM/owySdCOS/cjoFCuf8LI1d1kYX7z6cjsNkakM04u1geh/iesqyH3XQFI+SEVLb/oEC/EoQ0LIgRRoBiLyu9XZWN1ytY7MQxpPZniFz13oGV4/Dwl8yrP3Hba5LfQpHy2FZRM=
>
> It works fairly well but it's hacky, and can't be shared with other
> programs.  I'd like to implement it with libnettle at least, so it
> doesn't depend on the external gpg utility.  But yes, we could do this
> one and it would work on all platforms with libnettle.

I remember there were a couple of concerns:

(1) it also doesn't work with GnuPG2 at all (have you tested it?)
(2) even with libnettle, you need to implement OpenPGP packet handling
    if you want to keep the data compatibility with GPG (I don't think
    it is a good idea to reinvent another encrypted data format with
    plist as you proposed)

BTW,

>>> IIRC for 23 the default was to keep the password for the current session
>>> and not to store it in any file at all.  I think it's a better default
>>> than writing it in clear in some file, so at least for 24.1 reverting to
>>> the Emacs-23 default is very attractive.
>
> LI> Well, Emacs 23 just made you write the .authinfo file by hand.  Emacs 24
> LI> prompts you for whether you want to store the password or not.  If you
> LI> don't want to, say "n".

Even then, it is combersome for me to type "n" to proceed to the next
step (i.e. accessing smtp, etc).  Firefox allows user to keep browsing
password protected Web pages without answering the question immediately.

How about:

(1) add M-x auth-source-save command to save passwords manually
(2) (message "Type \\[auth-source-save] to save your passwords to file")
    instead of the question

Regards,
-- 
Daiki Ueno

bug#9113: 24.0.50; auth-sources: .authinfo versus .authinfo.gpg

[Ted Zlatanov]

On Fri, 27 Jan 2012 10:47:32 +0900 Daiki Ueno <ueno@unixuser.org> wrote: 

DU> Ted Zlatanov <tzz@lifelogs.com> writes:
>>>>> 2) use unencrypted authinfo with encrypted password tokens, which
>>>>> looks like this:
>>>> 
>>>>> machine supertest password
>>>>> gpg:jA0EAwMC2tUEaZgM7A5gyWM/owySdCOS/cjoFCuf8LI1d1kYX7z6cjsNkakM04u1geh/iesqyH3XQFI+SEVLb/oEC/EoQ0LIgRRoBiLyu9XZWN1ytY7MQxpPZniFz13oGV4/Dwl8yrP3Hba5LfQpHy2FZRM=
>> 
>> It works fairly well but it's hacky, and can't be shared with other
>> programs.  I'd like to implement it with libnettle at least, so it
>> doesn't depend on the external gpg utility.  But yes, we could do this
>> one and it would work on all platforms with libnettle.

DU> I remember there were a couple of concerns:

DU> (1) it also doesn't work with GnuPG2 at all (have you tested it?)

No, I haven't tested it.

DU> (2) even with libnettle, you need to implement OpenPGP packet handling
DU>     if you want to keep the data compatibility with GPG (I don't think
DU>     it is a good idea to reinvent another encrypted data format with
DU>     plist as you proposed)

Perhaps it would be OK to generate OpenPGP packets using libnettle, so
we are compatible with GPG.  That would be a decent amount of work but
it would suddenly remove Emacs's dependency on an external utility and
make it work on all platforms with GnuTLS support.  I think that's a
really good direction now that we have libnettle!  Are you interested in
working on it with me, and do you see any potential problems with this
approach?

DU> How about:

DU> (1) add M-x auth-source-save command to save passwords manually
DU> (2) (message "Type \\[auth-source-save] to save your passwords to file")
DU>     instead of the question

That's a very good suggestion, since currently the saving functionality
is done as a closure we pass back (internally this closure opens the
file, adds the line, then closes it, so it doesn't care about the
contents and thus is safe to call in any order).  So we could simply
queue those closures and then call something to save them.  But all the
prompting and UI has to be redesigned so it would be a lot of work for
me.  I'd like some more opinions on this.

On Thu, 26 Jan 2012 16:41:19 -0500 Stefan Monnier <monnier@IRO.UMontreal.CA> wrote: 

>> I'd like to implement it with libnettle at least, so it doesn't depend
>> on the external gpg utility.

SM> But that would make it work even less with other programs.

Yes.  I like Ueno-san's suggestion of generating OpenPGP packets
ourselves.  We can let the user decide whether he prefers encrypted
password tokens, encrypting the whole file, or leaving it in the clear.
Maybe we could even talk to the GPG agent for credentials.

SM> Another option (the better long-term option) is to use an external
SM> keychain service to handle these issues.  That's what we should focus on
SM> for the "next time".
>> Do you mean gpg-agent or the OS keychain?

SM> I mean the keychain.

>> Neither is available on all platforms consistently.

SM> AFAIK all platforms have a keychain nowadays and it's the best place to
SM> put sensitive passwords such as the ones used to access your IMAP server.

I don't think GNU/Linux has anything beyond the Secrets API, and that
depends on many optional components.

Mac OS X has a standard keychain, which someone attempted to support in
Emacs but didn't get it finished.  It's not too complicated.

W32 has some functionality (see
http://msdn.microsoft.com/en-us/library/aa380261(v=VS.85).aspx and
http://stackoverflow.com/questions/442923/windows-equivalent-of-os-x-keychain_
for some discussion) but not a fully capable keychain.

I don't know about the other platforms we support, but I hope this shows
that we should support but not rely on OS keychains.  `auth-sources'
reflects that by making them optional choices but not the defaults.

>>>> IIRC for 23 the default was to keep the password for the current session
>>>> and not to store it in any file at all.  I think it's a better default
>>>> than writing it in clear in some file, so at least for 24.1 reverting to
>>>> the Emacs-23 default is very attractive.
LI> Well, Emacs 23 just made you write the .authinfo file by hand.  Emacs 24
LI> prompts you for whether you want to store the password or not.  If you
LI> don't want to, say "n".

SM> Yes, I guess it's good enough.

>> One possible flow:
>> If the user says `y' then we can ask (if `auth-sources' is 'ask) 
>> "Do you want to keep your passwords in a GPG-encrypted file?"

>> If they say `y' then set `auth-sources' to "~/.authinfo.gpg" and check
>> that EPA/EPG are enabled. If GPG is not available, what do we do? Use
>> libnettle? Or explain and pretend they said `n'?

SM> If GPG is not available, ask a different question, as in "It will be
SM> saved in cleartext, is that OK?"

I think we'll need something on top of EPA/EPG if we support OpenPGP
packets with libnettle, an encryption services wrapper, which we can ask
"can we encrypt?" "can we encrypt a file with external GPG?" "can we
encrypt a file with internal OpenPGP and libnettle?" and so on.  Once we
have that wrapper API we can build the user interaction easily, without
ad-hoc checks.

This is getting a little long for the bug report, do you want to move it
to emacs-devel?

Ted

bug#9113: 24.0.50; auth-sources: .authinfo versus .authinfo.gpg

[Daiki Ueno]

Ted Zlatanov <tzz@lifelogs.com> writes:

> I think we'll need something on top of EPA/EPG if we support OpenPGP
> packets with libnettle,

I don't think it is a good idea to expose full cryptographic functions
in libnettle into Elisp, simply because there is no real use-case for
them except auth-source.

If you really want them and you think your problem can only be solved
with that approach, I would rather suggest to add gpg-encrypt-simple and
gpg-decrypt-simple in C level, which generates OpenPGP packets but only
supports single fixed algorithm and parameters.

So, anyway, this topic is not quite relevant to EPA/EPG from my
standpoint.

Regards,
-- 
Daiki Ueno

bug#9113: 24.0.50; auth-sources: .authinfo versus .authinfo.gpg, bug#9113: 24.0.50; auth-sources: .authinfo versus .authinfo.gpg, bug#9113: 24.0.50; auth-sources: .authinfo versus .authinfo.gpg

[Lars Ingebrigtsen]

Ted Zlatanov <tzz@lifelogs.com> writes:

> The encryption doesn't have to be strong.  It could use a well-known
> secret that the user can override, rather than an actual passphrase, and
> then no questions will be asked.

Sure.  This is what Firefox (etc.) does, and (most) people seem to be
satisfied with that.  On the other hand, this is just obscuring the
passwords, so the difference between this and, say,

machine smtp.gmail.com user foo password base64:c2VjcmV0

isn't huge.  (I mean, it is a real difference, but I'm not quite sure
whether it's a difference with a distinction.  :-)

So perhaps auth-source should just base64-encode password tokens by
default for Emacs 24.1?  That would give the users less of an "EEK"
feeling if they're looking at this file, and somebody is looking over
their shoulders...

-- 
(domestic pets only, the antidote for overdose, milk.)
  http://lars.ingebrigtsen.no  *  Sent from my Rome

bug#9113: 24.0.50; auth-sources: .authinfo versus .authinfo.gpg, bug#9113: 24.0.50; auth-sources: .authinfo versus .authinfo.gpg, bug#9113: 24.0.50; auth-sources: .authinfo versus .authinfo.gpg

[Chong Yidong]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> So perhaps auth-source should just base64-encode password tokens by
> default for Emacs 24.1?  That would give the users less of an "EEK"
> feeling if they're looking at this file, and somebody is looking over
> their shoulders...

Or we could rot13 it ;-)

bug#9113: 24.0.50; auth-sources: .authinfo versus .authinfo.gpg, bug#9113: 24.0.50; auth-sources: .authinfo versus .authinfo.gpg, bug#9113: 24.0.50; auth-sources: .authinfo versus .authinfo.gpg

[Lars Ingebrigtsen]

Chong Yidong <cyd@gnu.org> writes:

> Or we could rot13 it ;-)

For extra security: Double rot13.

-- 
(domestic pets only, the antidote for overdose, milk.)
  http://lars.ingebrigtsen.no  *  Sent from my Rome

bug#9113: 24.0.50; auth-sources: .authinfo versus .authinfo.gpg, bug#9113: 24.0.50; auth-sources: .authinfo versus .authinfo.gpg, bug#9113: 24.0.50; auth-sources: .authinfo versus .authinfo.gpg

[Kevin Rodgers]

On 1/31/12 4:57 AM, Lars Ingebrigtsen wrote:
> Chong Yidong<cyd@gnu.org>  writes:
>
>> Or we could rot13 it ;-)
>
> For extra security: Double rot13.

To fully support the Unicode BMP: rot32768

-- 
Kevin Rodgers
Denver, Colorado, USA

bug#9113: 24.0.50; auth-sources: .authinfo versus .authinfo.gpg

[Ted Zlatanov]

On Mon, 30 Jan 2012 17:33:47 +0100 Lars Ingebrigtsen <larsi@gnus.org> wrote: 

LI> Ted Zlatanov <tzz@lifelogs.com> writes:
>> The encryption doesn't have to be strong.  It could use a well-known
>> secret that the user can override, rather than an actual passphrase, and
>> then no questions will be asked.

LI> Sure.  This is what Firefox (etc.) does, and (most) people seem to be
LI> satisfied with that.  On the other hand, this is just obscuring the
LI> passwords, so the difference between this and, say,

LI> machine smtp.gmail.com user foo password base64:c2VjcmV0

LI> isn't huge.  (I mean, it is a real difference, but I'm not quite sure
LI> whether it's a difference with a distinction.  :-)

LI> So perhaps auth-source should just base64-encode password tokens by
LI> default for Emacs 24.1?  That would give the users less of an "EEK"
LI> feeling if they're looking at this file, and somebody is looking over
LI> their shoulders...

On Tue, 31 Jan 2012 14:55:57 +0800 Chong Yidong <cyd@gnu.org> wrote: 

CY> Or we could rot13 it ;-)

Base64 or ROT-13 would make the encryption trivial to crack *and* would
make the tokens unusable by other programs.  I don't think it's a good
compromise.

On Tue, 31 Jan 2012 10:00:32 +0100 Michael Albinus <michael.albinus@gmx.de> wrote: 

MA> The problem is, that there is no default under which name a password
MA> is stored [in the Secrets API]. Evrery application seems to use its
MA> own naming scheme.

We can probably work around that.  I'm more concerned that there is no
standard keychain for GNU/Linux or W32.  These are completely optional
services, up to the administrator and the user to install and activate.
On most server machines, for instance, you won't find a desktop
environment with a keychain or a GPG agent, although you may find a SSH
agent.  This solution is guaranteed to work only for Mac OS X.

On Mon, 30 Jan 2012 23:21:19 +0100 Lars Ingebrigtsen <larsi@gnus.org> wrote: 

LI> Stefan Monnier <monnier@IRO.UMontreal.CA> writes:

>> Exactly.  So, yes, I want Emacs to support the system's keychain tool,
>> since it's the right solution for the job.

LI> If that's possible, then it would indeed be a lot better than stashing
LI> the credentials in a file.

I'm not convinced it's better, see above.  In addition, it's hardly
portable: how would the user take his credentials to another machine?
Another platform?  It seems like a lock-in situation which I am not keen
to impose on our users.

As a default, it seems that storing the credential data in a temporary
in-memory auth-source backend *by default* is the best solution.  

Then on exit or on `auth-source-save', if there is something in the
in-memory backend, we can ask the user if he wants to save the passwords
and where, with all the consequent UI choices.  The user can pick a
plain file, or a plain file with password tokens, or a GPG-encrypted
file (with or without external support), or the platform's keychain
service, if available.  At that time the UI can modify `auth-sources'
for the user.

Ted

bug#9113: 24.0.50; auth-sources: .authinfo versus .authinfo.gpg

[Michael Albinus]

Ted Zlatanov <tzz@lifelogs.com> writes:

> As a default, it seems that storing the credential data in a temporary
> in-memory auth-source backend *by default* is the best solution.  

You use already password-cache.el in auth-source.el. It could be made
public by allowing a :cache entry in `auth-sources'.

> Then on exit or on `auth-source-save', if there is something in the
> in-memory backend, we can ask the user if he wants to save the passwords
> and where, with all the consequent UI choices.  The user can pick a
> plain file, or a plain file with password tokens, or a GPG-encrypted
> file (with or without external support), or the platform's keychain
> service, if available.  At that time the UI can modify `auth-sources'
> for the user.

Too complicate. If a user decides for cached passwords, she shouldn't be
asked for saving. It is convenient enough to enter a password only once
during a session.

> Ted

Best regards, Michael.

bug#9113: 24.0.50; auth-sources: .authinfo versus .authinfo.gpg

[Ted Zlatanov]

On Tue, 31 Jan 2012 12:37:17 +0100 Michael Albinus <michael.albinus@gmx.de> wrote: 

MA> Ted Zlatanov <tzz@lifelogs.com> writes:
>> As a default, it seems that storing the credential data in a temporary
>> in-memory auth-source backend *by default* is the best solution.  

MA> You use already password-cache.el in auth-source.el. It could be made
MA> public by allowing a :cache entry in `auth-sources'.

OK.

>> Then on exit or on `auth-source-save', if there is something in the
>> in-memory backend, we can ask the user if he wants to save the passwords
>> and where, with all the consequent UI choices.  The user can pick a
>> plain file, or a plain file with password tokens, or a GPG-encrypted
>> file (with or without external support), or the platform's keychain
>> service, if available.  At that time the UI can modify `auth-sources'
>> for the user.

MA> Too complicate. If a user decides for cached passwords, she shouldn't be
MA> asked for saving. It is convenient enough to enter a password only once
MA> during a session.

I'm not convinced but it does make sense, and would make the experience
simpler for the user.  I've asked on the Gnus mailing list for opinions
and anyone interested can post them here too.

Ted

bug#9113: 24.0.50; auth-sources: .authinfo versus .authinfo.gpg

[Roland Winkler]

On Thu Jan 26 2012 Ted Zlatanov wrote:
> I don't recall exactly either.  But here's how we can proceed.  We have
> several options:
> 
> 1) go back to authinfo.gpg as the first choice
> 
> 2) use unencrypted authinfo with encrypted password tokens, which looks like
> this:
> 
> machine supertest password
> gpg:jA0EAwMC2tUEaZgM7A5gyWM/owySdCOS/cjoFCuf8LI1d1kYX7z6cjsNkakM04u1geh/iesqyH3XQFI+SEVLb/oEC/EoQ0LIgRRoBiLyu9XZWN1ytY7MQxpPZniFz13oGV4/Dwl8yrP3Hba5LfQpHy2FZRM=
> 
> 3) work on the libnettle support (automatic if we use GnuTLS) so the
> external GPG executable is not needed to generate encrypted password
> tokens or encrypted authinfo files
> 
> 4) use Daiki Ueno's plist storage format (already in auth-source but not
> well tested AFAIK)
> 
> 5) ask the user if he has no authinfo file what he wants to do, and
> choose sensible defaults from the above depending on whether EPA/EPG and
> GPG; or libnettle are available.  If we do that, `auth-sources' will be
> set to 'ask by default.

For me, being a user who does not know too much about the subtleties
of "smart solutions" for this problem, it would already be helpful
if the relevant docstrings / info pages / a *Warnings* buffer
contained a warning like

  It is highly recommended to store the file .authinfo as an
  encrypted file as .authinfo.gpg, though in some cases such a
  solution can be inconvenient or otherwise problematic.

On the other hand, describe-variable currently gives for
auth-sources

  auth-sources is a variable defined in `auth-source.el'.
  Its value is ("~/.authinfo" "~/.authinfo.gpg" "~/.netrc")
  
  Documentation:
  List of authentication sources.
  
  The default will get login and password information from
  "~/.authinfo.gpg", which you should set up with the EPA/EPG
  packages to be encrypted.  If that file doesn't exist, it will
  try the unencrypted version "~/.authinfo" and the famous
  "~/.netrc" file.
  
  See the auth.info manual for details.

What general scheme of precedence is implemented here if
auth-sources is a list and the "default value" in this list is not
the first or last one, but the second? Or is this just a bug in the
docstring? 

For this problem, I cannot find helpful comments in the auth.info
manual either. I suggest that the docstring of auth-sources should
provide a hyperlink to the relevant section of the auth.info manual.

Roland

bug#9113: 24.0.50; auth-sources: .authinfo versus .authinfo.gpg

[Lars Ingebrigtsen]

"Roland Winkler" <winkler@gnu.org> writes:

>   It is highly recommended to store the file .authinfo as an
>   encrypted file as .authinfo.gpg, though in some cases such a
>   solution can be inconvenient or otherwise problematic.

I would say "it's highly discouraged", because putting your passwords
into the .authinfo.gpg file will render your Emacs virtually unusable
for reading mail/news/etc.  (By default.)

I mean, unless you think typing in a password three gazillion times is
OK.

-- 
(domestic pets only, the antidote for overdose, milk.)
  http://lars.ingebrigtsen.no  *  Sent from my Rome

bug#9113: 24.0.50; auth-sources: .authinfo versus .authinfo.gpg

[Roland Winkler]

On Sat Jan 28 2012 Lars Ingebrigtsen wrote:
> "Roland Winkler" <winkler@gnu.org> writes:
> 
> >   It is highly recommended to store the file .authinfo as an
> >   encrypted file as .authinfo.gpg, though in some cases such a
> >   solution can be inconvenient or otherwise problematic.
> 
> I would say "it's highly discouraged", because putting your
> passwords into the .authinfo.gpg file will render your Emacs
> virtually unusable for reading mail/news/etc. (By default.)
> 
> I mean, unless you think typing in a password three gazillion
> times is OK.

But then it appears to me that elsewhere there is a problem:

Why is it necessary that Emacs reads this file three gazillion
times? I would assume: reading the encrypted file once and holding
the content in memory cannot be more unsecure than storing the
sensitive information in an unencrypted file.

With an unencrypted file, the passwords are definitely lost /
exposed if my laptop is lost or stolen. With an encrypted file, a
thief needs to access the memory of a running (or dumped) emacs
process, which appears less likely to me.

In any case, how are ssh-agent and gpg-agent handling passphrases
that are given to them?

What am I missing here?

Roland

bug#9113: 24.0.50; auth-sources: .authinfo versus .authinfo.gpg

[Lars Ingebrigtsen]

"Roland Winkler" <winkler@gnu.org> writes:

> But then it appears to me that elsewhere there is a problem:
>
> Why is it necessary that Emacs reads this file three gazillion
> times? I would assume: reading the encrypted file once and holding
> the content in memory cannot be more unsecure than storing the
> sensitive information in an unencrypted file.

Yes, that's more secure.  Now that you mention it, perhaps we did fix
the aggressive password prompting?  I seem to remember adding a cache at
some point...

Anyway, having to enter a password for (say) sending email, even if your
SMTP server isn't password-protected (as you have to do with
.authinfo.gpg) isn't particularly ideal.

So I think the .authinfo.gpg concept isn't a good thing.  (But
encrypting tokens in the .authinfo file might be.)

And perhaps the password token in .authinfo should always be obscured,
at least, to avoid accidentally spilling the passwords (visually) if you
do a grep .* or something.  (This is what all the other
password-hoarding applications like Firefox, Chrome, etc do by default.)

-- 
(domestic pets only, the antidote for overdose, milk.)
  http://lars.ingebrigtsen.no  *  Sent from my Rome

bug#9113: 24.0.50; auth-sources: .authinfo versus .authinfo.gpg

[Roland Winkler]

On Mon Jan 30 2012 Lars Ingebrigtsen wrote:
> Anyway, having to enter a password for (say) sending email, even if your
> SMTP server isn't password-protected (as you have to do with
> .authinfo.gpg) isn't particularly ideal.

Again, it appears to me that such a problem could be solved
completely differently. Why couldn't one tell auth-source (say, via
a user variable) for which cases it can find a password in
.authinfo(.gpg)? Or the other way round: a user variable telling
authinfo for which cases it should not seek a password in
.authinfo(.gpg)? Or various variations of such a solution...

I'd guess that any solution dealing with .authinfo(.gpg) even when
this file is not required is asking for trouble in one or the other
way.

Roland