auth-source change default spec

auth-source change default spec

[Tim Cross]

I've recently run into a minor problem with the auth-source library
which I think is due to the default SPEC for auth-sources. I wanted
some feedbak before logging a bug request and also wanted to make this
possible issue visible asap given the need to get defaults sorted for
the next release.

The current default sorces spec (taken from recent emacs bzr sources) is

("~/.authinfo" "~/.authinfo.gpg" "~/.netrc")

I think it should be changed to have .authinfo.gpg first in the list.
The reason is that if you already have a .authinfo.gpg file and then
attempt to access a resource for which you don't yet have credentials
and the search criteria specifies the :create option, because
.authinfo is first, it will attempt to save the credentials in the
.authinfo file and not .authinfo.gpg. If you have things configured to
ask if you want to save (the default) it will ask if you want to save
to .authinfo even when it is aware you have a .authinfo.gpg file. It
does not appear to give you an option to change this.  If you just
accept the defaults and you do use .authinfo.gpg, things will break
when you add new credentials because it will create a .authinfo file.
Subsequent searches will never see the credentials you already have in
your .authinfo.gpg file as the search stops it has found the .authinfo
file.

I also think that putting the GPG version first would encourage better
practices. On many systems, especially GNU Linux, gpg will already be
installed. I guess it may be an issue on other platforms, but still
think it is better to go for the more secure solution as the default,
even if that does create some additional work for those who don't want
to bothwer with encryptiong and are happy with a less secure approach.

If this is not acceptable, I think the auth-source library may need to
be enhanced so that it defaults to the gpg version of the file for
saving when it knows one already exists.

Tim


-- 
Tim Cross

Re: auth-source change default spec

[Tim Cross]

P.S. I also forgot to mention. It looks like auth-source.el has

(require 'assoc)

which emacs tells me is an obsolete package. Should this be addressed
prior to emacs 24?

Tim


On 28 April 2012 10:45, Tim Cross <theophilusx@gmail.com> wrote:
> I've recently run into a minor problem with the auth-source library
> which I think is due to the default SPEC for auth-sources. I wanted
> some feedbak before logging a bug request and also wanted to make this
> possible issue visible asap given the need to get defaults sorted for
> the next release.
>
> The current default sorces spec (taken from recent emacs bzr sources) is
>
> ("~/.authinfo" "~/.authinfo.gpg" "~/.netrc")
>
> I think it should be changed to have .authinfo.gpg first in the list.
> The reason is that if you already have a .authinfo.gpg file and then
> attempt to access a resource for which you don't yet have credentials
> and the search criteria specifies the :create option, because
> .authinfo is first, it will attempt to save the credentials in the
> .authinfo file and not .authinfo.gpg. If you have things configured to
> ask if you want to save (the default) it will ask if you want to save
> to .authinfo even when it is aware you have a .authinfo.gpg file. It
> does not appear to give you an option to change this.  If you just
> accept the defaults and you do use .authinfo.gpg, things will break
> when you add new credentials because it will create a .authinfo file.
> Subsequent searches will never see the credentials you already have in
> your .authinfo.gpg file as the search stops it has found the .authinfo
> file.
>
> I also think that putting the GPG version first would encourage better
> practices. On many systems, especially GNU Linux, gpg will already be
> installed. I guess it may be an issue on other platforms, but still
> think it is better to go for the more secure solution as the default,
> even if that does create some additional work for those who don't want
> to bothwer with encryptiong and are happy with a less secure approach.
>
> If this is not acceptable, I think the auth-source library may need to
> be enhanced so that it defaults to the gpg version of the file for
> saving when it knows one already exists.
>
> Tim
>
>
> --
> Tim Cross



-- 
Tim Cross

Re: auth-source change default spec

[Stefan Monnier]

> P.S. I also forgot to mention. It looks like auth-source.el has
> (require 'assoc)

> which emacs tells me is an obsolete package. Should this be addressed
> prior to emacs 24?

It's only obsolete in the trunk (i.e. Emacs-24.2), so it doesn't need to
be addressed for Emacs-24.1 (I have a patch ready for 24.2, BTW).


        Stefan

Re: auth-source change default spec

[Tim Cross]

OK, only mentioned the obsolete package issue in case it needed to be
noted, but sounds like you have it in hand.

What about the .authinfo v .authinfo.gpg situation. I think this is a
bit more of a problem as it can esily cause confusion for the end user
when adding new credentials and breaks already working setups?

Tim


On 29 April 2012 01:08, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
>> P.S. I also forgot to mention. It looks like auth-source.el has
>> (require 'assoc)
>
>> which emacs tells me is an obsolete package. Should this be addressed
>> prior to emacs 24?
>
> It's only obsolete in the trunk (i.e. Emacs-24.2), so it doesn't need to
> be addressed for Emacs-24.1 (I have a patch ready for 24.2, BTW).
>
>
>        Stefan



-- 
Tim Cross

Re: auth-source change default spec

[Ted Zlatanov]

On Sat, 28 Apr 2012 10:45:37 +1000 Tim Cross <theophilusx@gmail.com> wrote: 

TC> I've recently run into a minor problem with the auth-source library
TC> which I think is due to the default SPEC for auth-sources. I wanted
TC> some feedbak before logging a bug request and also wanted to make this
TC> possible issue visible asap given the need to get defaults sorted for
TC> the next release.

TC> The current default sorces spec (taken from recent emacs bzr sources) is

TC> ("~/.authinfo" "~/.authinfo.gpg" "~/.netrc")

TC> I think it should be changed to have .authinfo.gpg first in the
TC> list.

Could you please read through Emacs bug #9113?  It deals with this issue
at length.

http://comments.gmane.org/gmane.emacs.bugs/49377

I had the .gpg file first originally and would still like it to be
first, but the objections are quite reasonable.

TC> The reason is that if you already have a .authinfo.gpg file and then
TC> attempt to access a resource for which you don't yet have credentials
TC> and the search criteria specifies the :create option, because
TC> .authinfo is first, it will attempt to save the credentials in the
TC> .authinfo file and not .authinfo.gpg. If you have things configured to
TC> ask if you want to save (the default) it will ask if you want to save
TC> to .authinfo even when it is aware you have a .authinfo.gpg file. It
TC> does not appear to give you an option to change this.  If you just
TC> accept the defaults and you do use .authinfo.gpg, things will break
TC> when you add new credentials because it will create a .authinfo
TC> file.

I don't think anything is broken.  auth-source is simply respecting
`auth-sources' as it's supposed to.  Preferring the second source
because of some attribute (e.g. "it has the .gpg extension") is much
worse in terms of usability.

TC> Subsequent searches will never see the credentials you already have in
TC> your .authinfo.gpg file as the search stops it has found the .authinfo
TC> file.

I think that's correct behavior (most `auth-source-search' calls pass
:max 1 to get this effect).

TC> If this is not acceptable, I think the auth-source library may need to
TC> be enhanced so that it defaults to the gpg version of the file for
TC> saving when it knows one already exists.

I think it's much, much simpler to just ask the user to put the .gpg
file first in `auth-sources'.

Ted

Re: auth-source change default spec

[Richard Riley]

Ted Zlatanov <tzz@lifelogs.com> writes:

> On Sat, 28 Apr 2012 10:45:37 +1000 Tim Cross <theophilusx@gmail.com> wrote: 
>
> TC> I've recently run into a minor problem with the auth-source library
> TC> which I think is due to the default SPEC for auth-sources. I wanted
> TC> some feedbak before logging a bug request and also wanted to make this
> TC> possible issue visible asap given the need to get defaults sorted for
> TC> the next release.
>
> TC> The current default sorces spec (taken from recent emacs bzr sources) is
>
> TC> ("~/.authinfo" "~/.authinfo.gpg" "~/.netrc")
>
> TC> I think it should be changed to have .authinfo.gpg first in the
> TC> list.
>
> Could you please read through Emacs bug #9113?  It deals with this issue
> at length.
>
> http://comments.gmane.org/gmane.emacs.bugs/49377
>
> I had the .gpg file first originally and would still like it to be
> first, but the objections are quite reasonable.
>
> TC> The reason is that if you already have a .authinfo.gpg file and then
> TC> attempt to access a resource for which you don't yet have credentials
> TC> and the search criteria specifies the :create option, because
> TC> .authinfo is first, it will attempt to save the credentials in the
> TC> .authinfo file and not .authinfo.gpg. If you have things configured to
> TC> ask if you want to save (the default) it will ask if you want to save
> TC> to .authinfo even when it is aware you have a .authinfo.gpg file. It
> TC> does not appear to give you an option to change this.  If you just
> TC> accept the defaults and you do use .authinfo.gpg, things will break
> TC> when you add new credentials because it will create a .authinfo
> TC> file.
>
> I don't think anything is broken.  auth-source is simply respecting
> `auth-sources' as it's supposed to.  Preferring the second source
> because of some attribute (e.g. "it has the .gpg extension") is much
> worse in terms of usability.

I would strongly disagree. I would expect it should default to the most
secure. And allow fall through on the search. Should you really want,
for some really obscure reason, to prefer a plain text file for secure
passwords over the .gpg then some sort of override could be
implemented. I know I'd be pretty miffed if I saved passwords thinking
they were going into .gpg only to have them read out to me at a later
date by someone who got hold of the plaintext file.

Re: auth-source change default spec

[Tim Cross]

Hi Ted,

I looked at that bug report and I think the options you offered were
good. I disagree with the useability argument i.e. that asking the
user to enter their gpg passphrase in order to search their
.authinfo.gpg file is too much to ask and that if firefox doesnt use
encrypted files to store passwords why should we. Security always has
a small element of inconvenience and asking to enter a passphrase is
not too much. Just because firefox uses poor practice in storing
sensitive data doesn't justify emacs doing the same.

For me, the main question relating to this and usability is to what
extent other platforms, like windows, will have the necessary
encryption facilities available such that having the encrypted version
as default will not result in really broken or inconvenient behaviour
for them. Not being a windows user, I cannot assess this issue.

However, this is perhaps getting off point for my main issue.

Regardless of the style of authinfo file being used, my issue is that
the library appears to only use the first choice in the auth-sources
list even when it knows (at least should) there is a gpg file. In this
situation, it should default to the gpg version, not to the first item
in the auth-sources spec.

Make not mistake, the current way things work does cause problems for
users. A couple of us recently spent some hours trying to work out why
things were breaking after changing some code to use auth-sources.

Unfortunately,, I don't think asking the user to edit the auth-sources
list is the right answer. Some packages will automatically create
entries for authinfo. It is likely they are unaware of auth-sources or
the configuration variables. Asking them to modify the default is
possibly expecting too much.

I think this can be resolved fairly easily. If auth-sources has
already found a .authinfo.gpg file in it's initial search, then that
should become the default file to sotre new credentials, regardless of
what is first in auth-sources. In addition, it would be good to allow
the user to hange the destination filename at the prompt when asked if
they want to save the current credentials.

Not also, the auth-sources manual is a bit misleading. It states that
the gpg version will be searched first. If I understand correctly,
this is not the case - it depends on auth-sources.

I will also need to check the meaning of :max 1 - I thought that meant
the search should return a maximum of one result, not, as seems to be
implied by the text in that bug report, that the library would only
search a max of 1 file. Another (less desirable) solution would be for
the library to continue to search all files until either it found a
match or ran out of files. This would at least stop the bug we ran
into beause auth-source created a .authinfo file when we already had
an .authinfo.gpg file.

Having said all that, the library is a good addittion and I appreciate
the work which has gone into it.

Tim

On 30 April 2012 22:51, Richard Riley <rileyrg@gmail.com> wrote:
> Ted Zlatanov <tzz@lifelogs.com> writes:
>
>> On Sat, 28 Apr 2012 10:45:37 +1000 Tim Cross <theophilusx@gmail.com> wrote:
>>
>> TC> I've recently run into a minor problem with the auth-source library
>> TC> which I think is due to the default SPEC for auth-sources. I wanted
>> TC> some feedbak before logging a bug request and also wanted to make this
>> TC> possible issue visible asap given the need to get defaults sorted for
>> TC> the next release.
>>
>> TC> The current default sorces spec (taken from recent emacs bzr sources) is
>>
>> TC> ("~/.authinfo" "~/.authinfo.gpg" "~/.netrc")
>>
>> TC> I think it should be changed to have .authinfo.gpg first in the
>> TC> list.
>>
>> Could you please read through Emacs bug #9113?  It deals with this issue
>> at length.
>>
>> http://comments.gmane.org/gmane.emacs.bugs/49377
>>
>> I had the .gpg file first originally and would still like it to be
>> first, but the objections are quite reasonable.
>>
>> TC> The reason is that if you already have a .authinfo.gpg file and then
>> TC> attempt to access a resource for which you don't yet have credentials
>> TC> and the search criteria specifies the :create option, because
>> TC> .authinfo is first, it will attempt to save the credentials in the
>> TC> .authinfo file and not .authinfo.gpg. If you have things configured to
>> TC> ask if you want to save (the default) it will ask if you want to save
>> TC> to .authinfo even when it is aware you have a .authinfo.gpg file. It
>> TC> does not appear to give you an option to change this.  If you just
>> TC> accept the defaults and you do use .authinfo.gpg, things will break
>> TC> when you add new credentials because it will create a .authinfo
>> TC> file.
>>
>> I don't think anything is broken.  auth-source is simply respecting
>> `auth-sources' as it's supposed to.  Preferring the second source
>> because of some attribute (e.g. "it has the .gpg extension") is much
>> worse in terms of usability.
>
> I would strongly disagree. I would expect it should default to the most
> secure. And allow fall through on the search. Should you really want,
> for some really obscure reason, to prefer a plain text file for secure
> passwords over the .gpg then some sort of override could be
> implemented. I know I'd be pretty miffed if I saved passwords thinking
> they were going into .gpg only to have them read out to me at a later
> date by someone who got hold of the plaintext file.
>
>
>
>
>
>



-- 
Tim Cross

Re: auth-source change default spec

[Ted Zlatanov]

On Mon, 30 Apr 2012 14:51:24 +0200 Richard Riley <rileyrg@gmail.com> wrote: 

RR> I would expect [`auth-sources'] should default to the most
RR> secure. And allow fall through on the search. Should you really
RR> want, for some really obscure reason, to prefer a plain text file
RR> for secure passwords over the .gpg then some sort of override could
RR> be implemented. I know I'd be pretty miffed if I saved passwords
RR> thinking they were going into .gpg only to have them read out to me
RR> at a later date by someone who got hold of the plaintext file.

The prompt clearly says "add entry to authinfo/authinfo.gpg?" when
entries are added automatically, and if you edit manually you should
know what you're editing.

The fall-through works just fine.  But if an entry is in the first file
in `auth-sources', and :max 1 is specified, there's no need to open a
bunch of files when we have an answer already.

On Tue, 1 May 2012 08:41:49 +1000 Tim Cross <theophilusx@gmail.com> wrote: 

TC> For me, the main question relating to this and usability is to what
TC> extent other platforms, like windows, will have the necessary
TC> encryption facilities available such that having the encrypted version
TC> as default will not result in really broken or inconvenient behaviour
TC> for them. Not being a windows user, I cannot assess this issue.

I can tell you it doesn't work on W32 and many other platforms.  We'd
need a native implementation of the OpenPGP packet format (probably
through libnettle).  If we had that, we could definitely use .gpg as the
default everywhere.  libcurl (thus Git and curl, among others) would
still be unable to use it, but at least you'd have Emacs-side consistency.

TC> Regardless of the style of authinfo file being used, my issue is that
TC> the library appears to only use the first choice in the auth-sources
TC> list even when it knows (at least should) there is a gpg file. In this
TC> situation, it should default to the gpg version, not to the first item
TC> in the auth-sources spec.

I disagree.  `auth-sources' specifies a search order and should not be
rearranged by the auth-source.el library.

TC> Unfortunately,, I don't think asking the user to edit the auth-sources
TC> list is the right answer.

I think it's the only answer.  Why is it a problem to ask the user to do
`M-x customize-variable auth-sources'?  It's completely analogous with
the rest of Emacs' customization UI.

TC> I think this can be resolved fairly easily. If auth-sources has
TC> already found a .authinfo.gpg file in it's initial search, then that
TC> should become the default file to sotre new credentials, regardless
TC> of what is first in auth-sources. 

I don't want to change the simple, clear logic of "save to the first
place you can."  Otherwise you'd have to pick between saving to
plist-store, Secrets API, or .gpg file.  Which one is right?

TC> In addition, it would be good to allow the user to [change] the
TC> destination filename at the prompt when asked if they want to save
TC> the current credentials.

That's a good idea.  Maybe offer a choice between the various entries in
`auth-sources' that can save.

TC> Not also, the auth-sources manual is a bit misleading. It states that
TC> the gpg version will be searched first. If I understand correctly,
TC> this is not the case - it depends on auth-sources.

Thanks, I've updated the manual in the Gnus repo and it should make its
way to Emacs.

TC> I will also need to check the meaning of :max 1 - I thought that meant
TC> the search should return a maximum of one result, not, as seems to be
TC> implied by the text in that bug report, that the library would only
TC> search a max of 1 file. Another (less desirable) solution would be for
TC> the library to continue to search all files until either it found a
TC> match or ran out of files. This would at least stop the bug we ran
TC> into beause auth-source created a .authinfo file when we already had
TC> an .authinfo.gpg file.

`auth-source-search' will pass :max to each parser, which will stop when
it finds enough entries.  So if there are 3 files and the second one has
the entry, `auth-source-search' will open 2 files for parsing.  :max
definitely does not refer to the number of files to check.

Ted

Re: auth-source change default spec

[Tim Cross]

Hi  Ted,

I'm a bit concerned I've not been clear enough on the precise issue we
encountered as some of what you wrote appears to contradict the
experience we had. I will try to re-state the issue to ensure we are
all talking about the same thing.

Recently, I added support for auth-sources to a package which provides
integration with various google services. Previously, the password was
only requested interactively when you used one of the services.
However, the addition by google of two step authentication and special
application passwords has made this approach less appealing.

Adding the support for auth-sources was trivial and essentially, we
used a version of the example given in the auth-sources manual.
Initial testing worked fine. However, we then found a user who ran
into problems. Once they had run the code and added their google
credentials, other systems, such as smtpmail, ceased to work.

After some debugging, we found the problem. The new credentials had
been added to ~/.authinfo, but the user already had a .authinfo.gpg
file.

It seems that if the user has both an .authinfo and an .authinfo.gpg
file, auth-sources is not searching both files. Note that the
.authinfo file only contains the entries that were just added and that
the previous entries are still only in the .authinfo.gpg file. Note
also that the entries are all for different resources. The evidence is
that auth-source-search is NOT searching all identified files, only
the first one i.e. authinfo not authinfo.gpg even if it does not find
a match for the requested resource in .authinfo. From what you write,
I get the impression that this is not the expected behaviour - the
search is supposed to search all available files until (in the case of
:max 1) at least one match if found. This does not seem to be the
case.

If the search functionality had searched all avaialable files, the
issue probably would not have come up. However, this does highlight my
other concern. If I have a .authinfo.gpg file and auth-sources knows I
have the file (it has already prompted for the passphrase in the
initial search) and has failed to find the resource and has prompted
for the values and then prompts to save those values, I think it
should save them (or at least offer to save them) to the most secure
version it knows about i.e. .authinfo.gpg. With the existing setup, it
is very easy for the user to be under a false sense of security - they
have setup an .authinfo.gpg file, obviously have the necessary
supporting programs etc and I think they should expect that a program
which offers to save new credentials will use the more secure method
when it already knows the gpg file exists. My suggestion to make
.authinfo.gpg the first default was another way to ensure this, though
given the lack of support for such encryption on some platforms, the
better solution would be to use the most secure method auth-sources
knows is available by default. or at the very least use whatever the
user is using.

The problem with expecting users being required to edit the
auth-sources file is that they may encounter the auth-sources library
as a side effect of running some other program. The user is not
required to configure anything in order to enable auth-sources. I
think this creates a slight inconsistency. To obtain secure behaviour,
the user must edit a value they may not even know about for a library
they have made no concious decision to use. However, if the above two
points were addressed i.e. searching and using the most secure file
known about by default, this is much less of an issue IMO.

Tim

2012/5/2 Ted Zlatanov <tzz@lifelogs.com>:
> On Mon, 30 Apr 2012 14:51:24 +0200 Richard Riley <rileyrg@gmail.com> wrote:
>
> RR> I would expect [`auth-sources'] should default to the most
> RR> secure. And allow fall through on the search. Should you really
> RR> want, for some really obscure reason, to prefer a plain text file
> RR> for secure passwords over the .gpg then some sort of override could
> RR> be implemented. I know I'd be pretty miffed if I saved passwords
> RR> thinking they were going into .gpg only to have them read out to me
> RR> at a later date by someone who got hold of the plaintext file.
>
> The prompt clearly says "add entry to authinfo/authinfo.gpg?" when
> entries are added automatically, and if you edit manually you should
> know what you're editing.
>
> The fall-through works just fine.  But if an entry is in the first file
> in `auth-sources', and :max 1 is specified, there's no need to open a
> bunch of files when we have an answer already.
>
> On Tue, 1 May 2012 08:41:49 +1000 Tim Cross <theophilusx@gmail.com> wrote:
>
> TC> For me, the main question relating to this and usability is to what
> TC> extent other platforms, like windows, will have the necessary
> TC> encryption facilities available such that having the encrypted version
> TC> as default will not result in really broken or inconvenient behaviour
> TC> for them. Not being a windows user, I cannot assess this issue.
>
> I can tell you it doesn't work on W32 and many other platforms.  We'd
> need a native implementation of the OpenPGP packet format (probably
> through libnettle).  If we had that, we could definitely use .gpg as the
> default everywhere.  libcurl (thus Git and curl, among others) would
> still be unable to use it, but at least you'd have Emacs-side consistency.
>
> TC> Regardless of the style of authinfo file being used, my issue is that
> TC> the library appears to only use the first choice in the auth-sources
> TC> list even when it knows (at least should) there is a gpg file. In this
> TC> situation, it should default to the gpg version, not to the first item
> TC> in the auth-sources spec.
>
> I disagree.  `auth-sources' specifies a search order and should not be
> rearranged by the auth-source.el library.
>
> TC> Unfortunately,, I don't think asking the user to edit the auth-sources
> TC> list is the right answer.
>
> I think it's the only answer.  Why is it a problem to ask the user to do
> `M-x customize-variable auth-sources'?  It's completely analogous with
> the rest of Emacs' customization UI.
>
> TC> I think this can be resolved fairly easily. If auth-sources has
> TC> already found a .authinfo.gpg file in it's initial search, then that
> TC> should become the default file to sotre new credentials, regardless
> TC> of what is first in auth-sources.
>
> I don't want to change the simple, clear logic of "save to the first
> place you can."  Otherwise you'd have to pick between saving to
> plist-store, Secrets API, or .gpg file.  Which one is right?
>
> TC> In addition, it would be good to allow the user to [change] the
> TC> destination filename at the prompt when asked if they want to save
> TC> the current credentials.
>
> That's a good idea.  Maybe offer a choice between the various entries in
> `auth-sources' that can save.
>
> TC> Not also, the auth-sources manual is a bit misleading. It states that
> TC> the gpg version will be searched first. If I understand correctly,
> TC> this is not the case - it depends on auth-sources.
>
> Thanks, I've updated the manual in the Gnus repo and it should make its
> way to Emacs.
>
> TC> I will also need to check the meaning of :max 1 - I thought that meant
> TC> the search should return a maximum of one result, not, as seems to be
> TC> implied by the text in that bug report, that the library would only
> TC> search a max of 1 file. Another (less desirable) solution would be for
> TC> the library to continue to search all files until either it found a
> TC> match or ran out of files. This would at least stop the bug we ran
> TC> into beause auth-source created a .authinfo file when we already had
> TC> an .authinfo.gpg file.
>
> `auth-source-search' will pass :max to each parser, which will stop when
> it finds enough entries.  So if there are 3 files and the second one has
> the entry, `auth-source-search' will open 2 files for parsing.  :max
> definitely does not refer to the number of files to check.
>
> Ted
>
>



-- 
Tim Cross

Re: auth-source change default spec

[Ted Zlatanov]

On Wed, 2 May 2012 07:41:19 +1000 Tim Cross <theophilusx@gmail.com> wrote: 

TC> It seems that if the user has both an .authinfo and an .authinfo.gpg
TC> file, auth-sources is not searching both files. Note that the
TC> .authinfo file only contains the entries that were just added and that
TC> the previous entries are still only in the .authinfo.gpg file. Note
TC> also that the entries are all for different resources. The evidence is
TC> that auth-source-search is NOT searching all identified files, only
TC> the first one i.e. authinfo not authinfo.gpg even if it does not find
TC> a match for the requested resource in .authinfo. From what you write,
TC> I get the impression that this is not the expected behaviour - the
TC> search is supposed to search all available files until (in the case of
TC> :max 1) at least one match if found. This does not seem to be the
TC> case.

This is either a bug or a misconfiguration; I just tested it and the
search went to the second file for me.  Please open a bug, set
`auth-source-debug' to 'trivia, and attach the value of `auth-sources'
and the content of *Messages* to show the log output when the bug
occurs.  If you can attach the edited .authinfo* files, even better.

TC> If I have a .authinfo.gpg file and auth-sources knows I have the
TC> file (it has already prompted for the passphrase in the initial
TC> search) and has failed to find the resource and has prompted for the
TC> values and then prompts to save those values, I think it should save
TC> them (or at least offer to save them) to the most secure version it
TC> knows about i.e. .authinfo.gpg. With the existing setup, it is very
TC> easy for the user to be under a false sense of security - they have
TC> setup an .authinfo.gpg file, obviously have the necessary supporting
TC> programs etc and I think they should expect that a program which
TC> offers to save new credentials will use the more secure method when
TC> it already knows the gpg file exists

There are 4 typical backends (plist-store, Secrets API, and netrc files
which can be .gpg or not), and the place where the file is stored may be
a factor too (e.g. a .gpg file on a NFS server may be considered worse
than unencrypted locally).  So we don't know what the user considers secure.

I'd rather change the "add entry" prompt to offer a choice of places to
save the new entry, so the user can choose.  This is a rare event so it
makes sense to bring up a menu.  I really don't think the auth-source
library should decide what's most secure!

What's a good multiple-choice menu library in Emacs?  Or do I roll my
own?  I can't solve it with a one-line prompt so it has to be fairly
intelligent yet work in a TTY.  I need it to be built-in, not external.

The choices would be:

Save to:

(show contents of `auth-sources' here with an explanation)

(1) [first `auth-sources' entry]
(2) [second `auth-sources' entry]
...

(e) Customize `auth-sources'

(c) Cancel

TC> The problem with expecting users being required to edit the
TC> auth-sources file is that they may encounter the auth-sources
TC> library as a side effect of running some other program. 

...

TC> The user is not required to configure anything in order to enable
TC> auth-sources. I think this creates a slight inconsistency. To obtain
TC> secure behaviour, the user must edit a value they may not even know
TC> about for a library they have made no concious decision to
TC> use.

You are right, but making the user configure `auth-sources' the first
time it's used would really annoy many Emacs users.  I'd rather make it
work OK by default, prompt for the save place as you suggested above,
and rely on the user to be curious and customize their environment.

So if I get the menu question addressed, I'll add that menu.

Ted

menu system in recent Emacs? (was: auth-source change default spec)

[Ted Zlatanov]

I think the menu question may be lost in the auth-source discussion, so
can anyone help me with the following?

Thanks
Ted

On Wed, 02 May 2012 08:25:44 -0400 Ted Zlatanov <tzz@lifelogs.com> wrote: 

TZ> What's a good multiple-choice menu library in Emacs?  Or do I roll my
TZ> own?  I can't solve it with a one-line prompt so it has to be fairly
TZ> intelligent yet work in a TTY.  I need it to be built-in, not external.

TZ> The choices would be:

TZ> Save to:

TZ> (show contents of `auth-sources' here with an explanation)

TZ> (1) [first `auth-sources' entry]
TZ> (2) [second `auth-sources' entry]
TZ> ...

TZ> (e) Customize `auth-sources'

TZ> (c) Cancel

Re: menu system in recent Emacs?

[Davis Herring]

> TZ> What's a good multiple-choice menu library in Emacs?  Or do I roll my
> TZ> own?  I can't solve it with a one-line prompt so it has to be fairly
> TZ> intelligent yet work in a TTY.  I need it to be built-in, not external.

How about `tmm-prompt'?  It's what M-` uses.

Davis

-- 
This product is sold by volume, not by mass.  If it appears too dense or
too sparse, it is because mass-energy conversion has occurred during
shipping.

Re: menu system in recent Emacs?

[Ted Zlatanov]

On Wed, 02 May 2012 08:37:17 -0600 Davis Herring <herring@lanl.gov> wrote: 

TZ> What's a good multiple-choice menu library in Emacs?  Or do I roll my
TZ> own?  I can't solve it with a one-line prompt so it has to be fairly
TZ> intelligent yet work in a TTY.  I need it to be built-in, not external.

DH> How about `tmm-prompt'?  It's what M-` uses.

Looks usable, e.g.

(tmm-prompt '("top" ("menu" ("choice 3" . 3) ("choice 4" . 4) ("choice 5" . 5))))

The only weird things are how it picks the shortcuts and the choices are
not visually highlighted when you use the arrow keys.

For the keyboard shortcuts, it would be nice if I could just number them
sequentially.  Do I need to use the keymap version of the MENU parameter
for that?

Ted

Re: menu system in recent Emacs?

[Eric Schulte]

Ted Zlatanov <tzz@lifelogs.com> writes:

> On Wed, 02 May 2012 20:52:35 +0200 Tassilo Horn <tassilo@member.fsf.org> wrote: 
>
> TH> Ted Zlatanov <tzz@lifelogs.com> writes:
> TZ> What's a good multiple-choice menu library in Emacs?
>
> TH> What about the `widget' library which is used by customize?
>
> That's nice for UI work, but really complicated for just choosing one of
> 3-10 options, and it doesn't pop up a temporary buffer like `tmm-prompt'.
>

You may not want to (require 'org), but the following is my favorite
multiple-choice prompt.

  (org-icompleting-read
   "Pick a color: "
   '("red" "white" "blue" "black" "brown"))

-- 
Eric Schulte
http://cs.unm.edu/~eschulte/

Re: menu system in recent Emacs?

[Tassilo Horn]

Ted Zlatanov <tzz@lifelogs.com> writes:

> TZ> What's a good multiple-choice menu library in Emacs?

What about the `widget' library which is used by customize?

,----[ (info "(widget)radio-button-choice") ]
| Syntax:
| 
|      TYPE ::= (radio-button-choice [KEYWORD ARGUMENT]...  TYPE ... )
| 
|    The component types specify the choices, with one radio button for
| each.  The widget's value will be that of the chosen TYPE argument.
| This widget matches any value that matches at least one of the
| specified TYPE arguments.
`----

Bye,
Tassilo

Re: menu system in recent Emacs?

[Ted Zlatanov]

On Wed, 02 May 2012 20:52:35 +0200 Tassilo Horn <tassilo@member.fsf.org> wrote: 

TH> Ted Zlatanov <tzz@lifelogs.com> writes:
TZ> What's a good multiple-choice menu library in Emacs?

TH> What about the `widget' library which is used by customize?

That's nice for UI work, but really complicated for just choosing one of
3-10 options, and it doesn't pop up a temporary buffer like `tmm-prompt'.

Ted

Re: menu system in recent Emacs?

[Ted Zlatanov]

On Wed, 02 May 2012 14:00:58 -0400 Eric Schulte <eric.schulte@gmx.com> wrote: 

ES> Ted Zlatanov <tzz@lifelogs.com> writes:
>> On Wed, 02 May 2012 20:52:35 +0200 Tassilo Horn <tassilo@member.fsf.org> wrote: 
>> 
TH> Ted Zlatanov <tzz@lifelogs.com> writes:
TZ> What's a good multiple-choice menu library in Emacs?
>> 
TH> What about the `widget' library which is used by customize?
>> 
>> That's nice for UI work, but really complicated for just choosing one of
>> 3-10 options, and it doesn't pop up a temporary buffer like `tmm-prompt'.
>> 

ES> You may not want to (require 'org), but the following is my favorite
ES> multiple-choice prompt.

ES>   (org-icompleting-read
ES>    "Pick a color: "
ES>    '("red" "white" "blue" "black" "brown"))

That uses `completing-read' if ido-mode and iswitchb are not enabled.

I like `completing-read'.  But it doesn't show the list of choices by
default, which will be confusing to users.  Also it's hard to insert
explanatory text, in this case "these are the entries from
`auth-sources' where you can save your new auth entry".  Can that be
done somehow?  I don't see a way.

Ted

Re: menu system in recent Emacs?

[Thien-Thi Nguyen]

() Ted Zlatanov <tzz@lifelogs.com>
() Wed, 02 May 2012 17:34:38 -0400

   Also it's hard to insert explanatory text [...].
   Can that be done somehow?  I don't see a way.

You can include that text in the prompt, with newlines for readability.

(completing-read "Here are some choices:\n- A\t- B\n- C\t- D\nChoose one: "
                 '("A" "B" "C" "D"))

Re: menu system in recent Emacs?

[Ted Zlatanov]

On Thu, 03 May 2012 01:52:50 +0200 Thien-Thi Nguyen <ttn@gnuvola.org> wrote: 

TN> () Ted Zlatanov <tzz@lifelogs.com>
TN> () Wed, 02 May 2012 17:34:38 -0400

TN>    Also it's hard to insert explanatory text [...].
TN>    Can that be done somehow?  I don't see a way.

TN> You can include that text in the prompt, with newlines for readability.

TN> (completing-read "Here are some choices:\n- A\t- B\n- C\t- D\nChoose one: "
TN>                  '("A" "B" "C" "D"))

I see, it's obvious now that you point it out :)  I can use that.

I would really like it to show the list of choices immediately, as if
the user pressed `TAB'.  Can that be fed into the function somehow as a
synthetic keypress?  I'd prefer to avoid modifying `completing-read' in
the source, so my code will work with older Emacsen as well.

Ted

Re: menu system in recent Emacs?

[Thien-Thi Nguyen]

() Ted Zlatanov <tzz@lifelogs.com>
() Wed, 02 May 2012 20:53:31 -0400

   I would really like it to show the list of choices immediately,
   as if the user pressed `TAB'.  Can that be fed into the
   function somehow as a synthetic keypress?

That would be:

(progn
  (push ?\t unread-command-events)
  (completing-read 
   "Here are some choices:\n- A\t- B\n- C\t- D\nChoose one: "
   '("A" "B" "C" "D")))

which can now be simplified to:

(progn
  (push ?\t unread-command-events)
  (completing-read 
   "CHOICE EXPLANATION\nChoose one: "
   '("A" "B" "C" "D")))

Re: menu system in recent Emacs?

[Ted Zlatanov]

On Thu, 03 May 2012 03:12:27 +0200 Thien-Thi Nguyen <ttn@gnuvola.org> wrote: 

TN> () Ted Zlatanov <tzz@lifelogs.com>
TN> () Wed, 02 May 2012 20:53:31 -0400

TZ>    I would really like it to show the list of choices immediately,
TZ>    as if the user pressed `TAB'.  Can that be fed into the
TZ>    function somehow as a synthetic keypress?

TN> (progn
TN>   (push ?\t unread-command-events)
TN>   (completing-read 
TN>    "CHOICE EXPLANATION\nChoose one: "
TN>    '("A" "B" "C" "D")))

Thank you, I didn't know about `unread-command-events'.  I'll use that code.

Ted

Re: menu system in recent Emacs?

[Stefan Monnier]

>    I would really like it to show the list of choices immediately,
>    as if the user pressed `TAB'.  Can that be fed into the
>    function somehow as a synthetic keypress?

> That would be:

> (progn
>   (push ?\t unread-command-events)
>   (completing-read 
>    "Here are some choices:\n- A\t- B\n- C\t- D\nChoose one: "

Note: this will only work when the minibuffer can grow to have
enough lines.  E.g. my minibuffer-only frame would display this prompt
in a way that's rather incomprehensible.

> (progn
>   (push ?\t unread-command-events)
>   (completing-read 
>    "CHOICE EXPLANATION\nChoose one: "
>    '("A" "B" "C" "D")))

That's pretty ugly.
I recommend something more like

   (minibuffer-with-setup-hook #'minibuffer-completion-help
     (completing-read ...))

As for the explanation, you could add an `annotation-function' metadata
to the completion-table.  Something like

   (let ((table (lambda (string pred action)
                  (if (eq action 'metadata)
                      `(metadata (annotation-function
                                  . ,(lambda (s)
                                       (cond
                                        ((equal s "A") " explanation a")
                                        ((equal s "B") " explanation b")
                                        ((equal s "C") " explanation c")
                                        ((equal s "D") " explanation d")))))
                    (complete-with-action action '("A" "B" "C" "D")
                                          string pred)))))
     (minibuffer-with-setup-hook #'minibuffer-completion-help
       (completing-read "prompt:" table)))
     

-- Stefan

Re: auth-source change default spec

[Tim Cross]

On 2 May 2012 22:25, Ted Zlatanov <tzz@lifelogs.com> wrote:
> On Wed, 2 May 2012 07:41:19 +1000 Tim Cross <theophilusx@gmail.com> wrote:
>
> TC> It seems that if the user has both an .authinfo and an .authinfo.gpg
> TC> file, auth-sources is not searching both files. Note that the
> TC> .authinfo file only contains the entries that were just added and that
> TC> the previous entries are still only in the .authinfo.gpg file. Note
> TC> also that the entries are all for different resources. The evidence is
> TC> that auth-source-search is NOT searching all identified files, only
> TC> the first one i.e. authinfo not authinfo.gpg even if it does not find
> TC> a match for the requested resource in .authinfo. From what you write,
> TC> I get the impression that this is not the expected behaviour - the
> TC> search is supposed to search all available files until (in the case of
> TC> :max 1) at least one match if found. This does not seem to be the
> TC> case.
>
> This is either a bug or a misconfiguration; I just tested it and the
> search went to the second file for me.  Please open a bug, set
> `auth-source-debug' to 'trivia, and attach the value of `auth-sources'
> and the content of *Messages* to show the log output when the bug
> occurs.  If you can attach the edited .authinfo* files, even better.
>

OK, will update emacs and log a bug report with recipe if I am still
able to reproduce the issue.

> TC> If I have a .authinfo.gpg file and auth-sources knows I have the
> TC> file (it has already prompted for the passphrase in the initial
> TC> search) and has failed to find the resource and has prompted for the
> TC> values and then prompts to save those values, I think it should save
> TC> them (or at least offer to save them) to the most secure version it
> TC> knows about i.e. .authinfo.gpg. With the existing setup, it is very
> TC> easy for the user to be under a false sense of security - they have
> TC> setup an .authinfo.gpg file, obviously have the necessary supporting
> TC> programs etc and I think they should expect that a program which
> TC> offers to save new credentials will use the more secure method when
> TC> it already knows the gpg file exists
>
> There are 4 typical backends (plist-store, Secrets API, and netrc files
> which can be .gpg or not), and the place where the file is stored may be
> a factor too (e.g. a .gpg file on a NFS server may be considered worse
> than unencrypted locally).  So we don't know what the user considers secure.
>
> I'd rather change the "add entry" prompt to offer a choice of places to
> save the new entry, so the user can choose.  This is a rare event so it
> makes sense to bring up a menu.  I really don't think the auth-source
> library should decide what's most secure!
>
> What's a good multiple-choice menu library in Emacs?  Or do I roll my
> own?  I can't solve it with a one-line prompt so it has to be fairly
> intelligent yet work in a TTY.  I need it to be built-in, not external.
>
> The choices would be:
>
> Save to:
>
> (show contents of `auth-sources' here with an explanation)
>
> (1) [first `auth-sources' entry]
> (2) [second `auth-sources' entry]
> ...
>
> (e) Customize `auth-sources'
>
> (c) Cancel
>
> TC> The problem with expecting users being required to edit the
> TC> auth-sources file is that they may encounter the auth-sources
> TC> library as a side effect of running some other program.
>
> ...
>
> TC> The user is not required to configure anything in order to enable
> TC> auth-sources. I think this creates a slight inconsistency. To obtain
> TC> secure behaviour, the user must edit a value they may not even know
> TC> about for a library they have made no concious decision to
> TC> use.
>
> You are right, but making the user configure `auth-sources' the first
> time it's used would really annoy many Emacs users.  I'd rather make it
> work OK by default, prompt for the save place as you suggested above,
> and rely on the user to be curious and customize their environment.
>
> So if I get the menu question addressed, I'll add that menu.
>
> Ted
>
>
I agree. The ability for the user to specify/choose where the entry is
saved would be an adequate solution and provides a reasonable outcome
for the majority of users regardless of their level of concern or
configuration sophistication re: security and auth-sources.
Unfortunately, I can't recommend the best approach for menu
generation/handling.

thanks for being receptive to my comments/suggestions and the effort put in.

Tim



-- 
Tim Cross

Re: menu system in recent Emacs?

[Thierry Volpiatto]

Thien-Thi Nguyen <ttn@gnuvola.org> writes:

> () Ted Zlatanov <tzz@lifelogs.com>
> () Wed, 02 May 2012 20:53:31 -0400
>
>    I would really like it to show the list of choices immediately,
>    as if the user pressed `TAB'.  Can that be fed into the
>    function somehow as a synthetic keypress?
>
> That would be:
>
> (progn
>   (push ?\t unread-command-events)
>   (completing-read 
>    "Here are some choices:\n- A\t- B\n- C\t- D\nChoose one: "
>    '("A" "B" "C" "D")))
>
> which can now be simplified to:
>
> (progn
>   (push ?\t unread-command-events)
>   (completing-read 
>    "CHOICE EXPLANATION\nChoose one: "
>    '("A" "B" "C" "D")))

I think you want to use cons instead of push here, what you want is
(t . event) like described in doc, isn't it?


-- 
  Thierry
Get my Gnupg key:
gpg --keyserver pgp.mit.edu --recv-keys 59F29997 

Re: menu system in recent Emacs?

[Stefan Reichör]

Eric Schulte <eric.schulte@gmx.com> writes:

> Ted Zlatanov <tzz@lifelogs.com> writes:
>
>> On Wed, 02 May 2012 20:52:35 +0200 Tassilo Horn <tassilo@member.fsf.org> wrote: 
>>
>> TH> Ted Zlatanov <tzz@lifelogs.com> writes:
>> TZ> What's a good multiple-choice menu library in Emacs?
>>
>> TH> What about the `widget' library which is used by customize?
>>
>> That's nice for UI work, but really complicated for just choosing one of
>> 3-10 options, and it doesn't pop up a temporary buffer like `tmm-prompt'.
>>
>
> You may not want to (require 'org), but the following is my favorite
> multiple-choice prompt.
>
>   (org-icompleting-read
>    "Pick a color: "
>    '("red" "white" "blue" "black" "brown"))

There is also ido - it shows all candidates:

  (ido-completing-read
   "Pick a color: "
   '("red" "white" "blue" "black" "brown"))


Stefan.

Re: menu system in recent Emacs?

[Thierry Volpiatto]

Thierry Volpiatto <thierry.volpiatto@gmail.com> writes:

> Thien-Thi Nguyen <ttn@gnuvola.org> writes:
>
>> () Ted Zlatanov <tzz@lifelogs.com>
>> () Wed, 02 May 2012 20:53:31 -0400
>>
>>    I would really like it to show the list of choices immediately,
>>    as if the user pressed `TAB'.  Can that be fed into the
>>    function somehow as a synthetic keypress?
>>
>> That would be:
>>
>> (progn
>>   (push ?\t unread-command-events)
>>   (completing-read 
>>    "Here are some choices:\n- A\t- B\n- C\t- D\nChoose one: "
>>    '("A" "B" "C" "D")))
>>
>> which can now be simplified to:
>>
>> (progn
>>   (push ?\t unread-command-events)
>>   (completing-read 
>>    "CHOICE EXPLANATION\nChoose one: "
>>    '("A" "B" "C" "D")))
>
> I think you want to use cons instead of push here, what you want is
> (t . event) like described in doc, isn't it?
Ok, forget that, I understand what you want to do here.
This was not working in helm-mode, it is now fixed.

-- 
  Thierry
Get my Gnupg key:
gpg --keyserver pgp.mit.edu --recv-keys 59F29997 

Re: menu system in recent Emacs?

[Thien-Thi Nguyen]

() Thierry Volpiatto <thierry.volpiatto@gmail.com>
() Thu, 03 May 2012 07:21:43 +0200

   I think you want to use cons instead of push here,
   what you want is (t . event) like described in doc, isn't it?

I've just now read (info "(elisp) Event Input Misc")
which says, in part:

     Normally you add events to the front of this list, so that the
     events most recently unread will be reread first.

     Events read from this list are not normally added to the current
     command's key sequence (as returned by e.g. `this-command-keys'),
     as the events will already have been added once as they were read
     for the first time.  An element of the form `(`t' . EVENT)' forces
     EVENT to be added to the current command's key sequence.

So, no, i do not want to use ‘cons’ because:

- ‘push’ is side-effecting, the equivalent of ‘cons’ then ‘setq’,
  which is what we want (here, the value of the sub-expression is
  unimportant);

- (t . EVENT) frobs ‘this-command-keys’, which doesn't concern me.

Perhaps you mistook ‘?\t’ (CHARACTER TABULATION, U+0009) for ‘t’?

Re: menu system in recent Emacs?

[Thierry Volpiatto]

Thien-Thi Nguyen <ttn@gnuvola.org> writes:

> () Thierry Volpiatto <thierry.volpiatto@gmail.com>
> () Thu, 03 May 2012 07:21:43 +0200
>
>    I think you want to use cons instead of push here,
>    what you want is (t . event) like described in doc, isn't it?
>
> I've just now read (info "(elisp) Event Input Misc")
> which says, in part:
>
>      Normally you add events to the front of this list, so that the
>      events most recently unread will be reread first.
>
>      Events read from this list are not normally added to the current
>      command's key sequence (as returned by e.g. `this-command-keys'),
>      as the events will already have been added once as they were read
>      for the first time.  An element of the form `(`t' . EVENT)' forces
>      EVENT to be added to the current command's key sequence.
>
> So, no, i do not want to use ‘cons’ because:
>
> - ‘push’ is side-effecting, the equivalent of ‘cons’ then ‘setq’,
>   which is what we want (here, the value of the sub-expression is
>   unimportant);
Yes I understood this afterward, thanks.

> - (t . EVENT) frobs ‘this-command-keys’, which doesn't concern me.
>
> Perhaps you mistook ‘?\t’ (CHARACTER TABULATION, U+0009) for ‘t’?
>
>

-- 
  Thierry
Get my Gnupg key:
gpg --keyserver pgp.mit.edu --recv-keys 59F29997 

Re: menu system in recent Emacs?

[Ted Zlatanov]

On Thu, 03 May 2012 08:18:21 +0200 Stefan Reichör <stefan@xsteve.at> wrote: 

SR> There is also ido - it shows all candidates:

   (ido-completing-read
    "Pick a color: "
    '("red" "white" "blue" "black" "brown"))

That's nice, shows the list immediately, uses colors, and doesn't
require `ido-mode' to be on.  It has one problem, though: it's not
widely used in Emacs, unlike `completing-read', so I'm concerned that it
wouldn't be familiar to new users.  The ido UI is too different from
standard completion in Emacs and elsewhere, e.g. shell completion.  For
instance, TAB will complete AND select the choice if it's unique.
`completing-read' and most others will complete on TAB but require RET
to select.  Another difference is that the up and down arrows select
previous input, but left and right arrows select choices.

So I will stick with `completing-read' to make it easier for the users
and they can simply use ido-mode (which provides `ido-completing-read'
to replace `completing-read' IIUC) to get the colors and ido UI.

Ted