bug#29575: 25.3; Secret Service API treats labels as unique

bug#29575: 25.3; Secret Service API treats labels as unique

[Allen Li]

The Secret Service API [1] treats labels as unique keys for each
secret item in a collection.  However, labels are not required to be
unique in a collection [2], the attribute key/value pairs are.

It is perfectly valid to have multiple secrets with the same label, in
which case Emacs's Secret Service API is not able to retrieve all but
the most recently created (?) secret.

This can be demonstrated by creating two such secrets using the
secret-tool utility:

secret-tool store --label=Test1 id foo
secret-tool store --label=Test1 id bar

You can see how the attributes uniquely identify secrets:

secret-tool store --label=Test2 id foo  # This overwrites the first secret.

Implementation idea: Use attribute plists instead of label strings to
uniquely identify secret items.

This would require creating a new copy of the API to preserve backward
compatibility.

[1]: https://www.gnu.org/software/emacs/manual/html_node/auth/Secret-Service-API.html

[2]: https://specifications.freedesktop.org/secret-service/re02.html

In GNU Emacs 25.3.1 (x86_64-pc-linux-gnu, GTK+ Version 3.22.19)
 of 2017-09-16 built on juergen
Windowing system distributor 'The X.Org Foundation', version 11.0.11905000
Configured using:
 'configure --prefix=/usr --sysconfdir=/etc --libexecdir=/usr/lib
 --localstatedir=/var --with-x-toolkit=gtk3 --with-xft --with-modules
 'CFLAGS=-march=x86-64 -mtune=generic -O2 -pipe -fstack-protector-strong
 -fno-plt' CPPFLAGS=-D_FORTIFY_SOURCE=2
 LDFLAGS=-Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now'

Configured features:
XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND GPM DBUS GCONF GSETTINGS
NOTIFY ACL GNUTLS LIBXML2 FREETYPE M17N_FLT LIBOTF XFT ZLIB
TOOLKIT_SCROLL_BARS GTK3 X11 MODULES

bug#29575: 25.3; Secret Service API treats labels as unique

[Michael Albinus]

Allen Li <vianchielfaura@gmail.com> writes:

Hi Allen,

> The Secret Service API [1] treats labels as unique keys for each
> secret item in a collection.  However, labels are not required to be
> unique in a collection [2], the attribute key/value pairs are.
>
> It is perfectly valid to have multiple secrets with the same label, in
> which case Emacs's Secret Service API is not able to retrieve all but
> the most recently created (?) secret.
>
> This can be demonstrated by creating two such secrets using the
> secret-tool utility:
>
> secret-tool store --label=Test1 id foo
> secret-tool store --label=Test1 id bar
>
> You can see how the attributes uniquely identify secrets:
>
> secret-tool store --label=Test2 id foo  # This overwrites the first secret.

First of all: do you have a use case in mind for this? Whether we'll
extend the Secret Service API depends on the real need.

> Implementation idea: Use attribute plists instead of label strings to
> uniquely identify secret items.

Well, inside the org.freedesktop.Secret.{Service,Collection,Item}
interfaces, an item is identified by an object path. We could extend our
interface to allow both label and object path as item, and to throw away
the "unique label rule" inside collections.

> This would require creating a new copy of the API to preserve backward
> compatibility.

The change proposed above would be backward compatible.

Best regards, Michael.

bug#29575: 25.3; Secret Service API treats labels as unique

[Allen Li]

On Mon, Dec 11, 2017 at 5:02 AM, Michael Albinus <michael.albinus@gmx.de> wrote:
> Allen Li <vianchielfaura@gmail.com> writes:
>
> Hi Allen,
>
>> The Secret Service API [1] treats labels as unique keys for each
>> secret item in a collection.  However, labels are not required to be
>> unique in a collection [2], the attribute key/value pairs are.
>>
>> It is perfectly valid to have multiple secrets with the same label, in
>> which case Emacs's Secret Service API is not able to retrieve all but
>> the most recently created (?) secret.
>>
>> This can be demonstrated by creating two such secrets using the
>> secret-tool utility:
>>
>> secret-tool store --label=Test1 id foo
>> secret-tool store --label=Test1 id bar
>>
>> You can see how the attributes uniquely identify secrets:
>>
>> secret-tool store --label=Test2 id foo  # This overwrites the first secret.
>
> First of all: do you have a use case in mind for this? Whether we'll
> extend the Secret Service API depends on the real need.

Yes, I plan on implementing a personal password manager using the API.
I am not planning on a design that requires duplicate labels, but I
hesitated on starting my project because I predict implementing other
tools that interact with the same underlying keyring service that use
duplicate labels for implementation simplicity or other reasons.

>> Implementation idea: Use attribute plists instead of label strings to
>> uniquely identify secret items.
>
> Well, inside the org.freedesktop.Secret.{Service,Collection,Item}
> interfaces, an item is identified by an object path. We could extend our
> interface to allow both label and object path as item, and to throw away
> the "unique label rule" inside collections.

That sounds like a better starting idea.  One problem that comes to
mind is that the object path could be a valid label value, I think.  I
don’t think the specification places any guarantees on the object path
either, e.g. if another program modifies an Item, does that change the
object path from under us?  That would cause race bugs.

>
>> This would require creating a new copy of the API to preserve backward
>> compatibility.
>
> The change proposed above would be backward compatible.
>
> Best regards, Michael.

bug#29575: 25.3; Secret Service API treats labels as unique

[Michael Albinus]

Allen Li <vianchielfaura@gmail.com> writes:

Hi Allen,

> Yes, I plan on implementing a personal password manager using the API.

Is it a standalone program, or an Emacs package? In the latter case I
recommend to try auth-sources.el. It is the default "password manager"
in Emacs. The Secret Service API is integrated as one backend, although
there's room for improvement.

>> Well, inside the org.freedesktop.Secret.{Service,Collection,Item}
>> interfaces, an item is identified by an object path. We could extend our
>> interface to allow both label and object path as item, and to throw away
>> the "unique label rule" inside collections.
>
> That sounds like a better starting idea.  One problem that comes to
> mind is that the object path could be a valid label value, I think.

That's possible, yes. But I doubt it will happen in reality. At least in
Emacs we could check, that a label doesn't look like an object path, and
suppress such items for being considered.

> I don’t think the specification places any guarantees on the object
> path either, e.g. if another program modifies an Item, does that
> change the object path from under us?  That would cause race bugs.

secrets.el registers for several signals already, like
"org.freedesktop.DBus.NameOwnerChanged",
"org.freedesktop.Secret.Service.CollectionCreated" and
"org.freedesktop.Secret.Service.CollectionDeleted".

We could also register for the signals
"org.freedesktop.Secret.Collection.ItemCreated",
"org.freedesktop.Secret.Collection.ItemDeleted" and
"org.freedesktop.Secret.Collection.ItemChanged" in order to be notified
about such changes.

I don't know whether I have sufficient time to work on secrets.el next
weeks. Would you be interested in providing respective patches for secrets.el?

Best regards, Michael.

bug#29575: 25.3; Secret Service API treats labels as unique

[Allen Li]

On Tue, Dec 12, 2017 at 12:35 AM, Michael Albinus
<michael.albinus@gmx.de> wrote:
> Allen Li <vianchielfaura@gmail.com> writes:
>
> Hi Allen,
>
>> Yes, I plan on implementing a personal password manager using the API.
>
> Is it a standalone program, or an Emacs package? In the latter case I
> recommend to try auth-sources.el. It is the default "password manager"
> in Emacs. The Secret Service API is integrated as one backend, although
> there's room for improvement.

I envision a frontend to a password store, so not a password manager
in the sense that Emacs Lisp code calls out to it to retrieve
passwords.  My understanding is that auth-source.el fulfills the
latter role.  I want more something to store secrets that I can recall
interactively through various frontends, one of which would be through
Emacs.

>>> Well, inside the org.freedesktop.Secret.{Service,Collection,Item}
>>> interfaces, an item is identified by an object path. We could extend our
>>> interface to allow both label and object path as item, and to throw away
>>> the "unique label rule" inside collections.
>>
>> That sounds like a better starting idea.  One problem that comes to
>> mind is that the object path could be a valid label value, I think.
>
> That's possible, yes. But I doubt it will happen in reality. At least in
> Emacs we could check, that a label doesn't look like an object path, and
> suppress such items for being considered.
>
>> I don’t think the specification places any guarantees on the object
>> path either, e.g. if another program modifies an Item, does that
>> change the object path from under us?  That would cause race bugs.
>
> secrets.el registers for several signals already, like
> "org.freedesktop.DBus.NameOwnerChanged",
> "org.freedesktop.Secret.Service.CollectionCreated" and
> "org.freedesktop.Secret.Service.CollectionDeleted".
>
> We could also register for the signals
> "org.freedesktop.Secret.Collection.ItemCreated",
> "org.freedesktop.Secret.Collection.ItemDeleted" and
> "org.freedesktop.Secret.Collection.ItemChanged" in order to be notified
> about such changes.
>
> I don't know whether I have sufficient time to work on secrets.el next
> weeks. Would you be interested in providing respective patches for secrets.el?

Sorry, I can’t commit to that.

>
> Best regards, Michael.

bug#29575: 25.3; Secret Service API treats labels as unique

[Ted Zlatanov]

On Tue, 12 Dec 2017 19:41:50 -0800 Allen Li <vianchielfaura@gmail.com> wrote: 

AL> I envision a frontend to a password store, so not a password manager
AL> in the sense that Emacs Lisp code calls out to it to retrieve
AL> passwords.  My understanding is that auth-source.el fulfills the
AL> latter role.  I want more something to store secrets that I can recall
AL> interactively through various frontends, one of which would be through
AL> Emacs.

Consider the password-store (http://passwordstore.org/) utility, for
which there's an auth-sources backend in lisp/auth-sources-pass.el

Thanks
Ted

bug#29575: 25.3; Secret Service API treats labels as unique

[Michael Albinus]

Michael Albinus <michael.albinus@gmx.de> writes:

Hi Allen,

>>> Well, inside the org.freedesktop.Secret.{Service,Collection,Item}
>>> interfaces, an item is identified by an object path. We could extend our
>>> interface to allow both label and object path as item, and to throw away
>>> the "unique label rule" inside collections.
>>
>> That sounds like a better starting idea.  One problem that comes to
>> mind is that the object path could be a valid label value, I think.
>
> That's possible, yes. But I doubt it will happen in reality. At least in
> Emacs we could check, that a label doesn't look like an object path, and
> suppress such items for being considered.

Finally, I found the time to implement this. `secrets-create-item' does
allow labels now which exist already in a collection. `secrets-item-path',
`secrets-get-secret', `secrets-get-attributes', 'secrets-get-attribute'
and `secrets-delete-item' allow also object paths as argument.

Pushed to the master branch. Could you pls check whether this fits your needs?

Best regards, Michael.

bug#29575: 25.3; Secret Service API treats labels as unique

[Allen Li]

Thanks.  It doesn't look like there is a way to get the object path by
searching attributes?  That means there's no way to get the object path for
a secret added from a cooperating application (that may not have a unique
label).  secrets-search-items returns labels.  Ideally there would be a
variant of secrets-search-items that returned object paths.

Also, the docstring for secrets-create-items now has this line which I
think is misleading:

The label ITEM must not be unique in COLLECTION.

Should it be:

The label ITEM does not have to be unique in COLLECTION.
On Tue, May 15, 2018 at 5:56 AM Michael Albinus <michael.albinus@gmx.de>
wrote:

> Michael Albinus <michael.albinus@gmx.de> writes:

> Hi Allen,

> >>> Well, inside the org.freedesktop.Secret.{Service,Collection,Item}
> >>> interfaces, an item is identified by an object path. We could extend
our
> >>> interface to allow both label and object path as item, and to throw
away
> >>> the "unique label rule" inside collections.
> >>
> >> That sounds like a better starting idea.  One problem that comes to
> >> mind is that the object path could be a valid label value, I think.
> >
> > That's possible, yes. But I doubt it will happen in reality. At least in
> > Emacs we could check, that a label doesn't look like an object path, and
> > suppress such items for being considered.

> Finally, I found the time to implement this. `secrets-create-item' does
> allow labels now which exist already in a collection. `secrets-item-path',
> `secrets-get-secret', `secrets-get-attributes', 'secrets-get-attribute'
> and `secrets-delete-item' allow also object paths as argument.

> Pushed to the master branch. Could you pls check whether this fits your
needs?

> Best regards, Michael.

bug#29575: 25.3; Secret Service API treats labels as unique

[Michael Albinus]

Allen Li <darkfeline@felesatra.moe> writes:

Hi Allen,

> Thanks.  It doesn't look like there is a way to get the object path by
> searching attributes?  That means there's no way to get the object path for
> a secret added from a cooperating application (that may not have a unique
> label).  secrets-search-items returns labels.  Ideally there would be a
> variant of secrets-search-items that returned object paths.

Indeed. I've added the function `secrets-search-item-paths'.

> Also, the docstring for secrets-create-items now has this line which I
> think is misleading:
>
> The label ITEM must not be unique in COLLECTION.
>
> Should it be:
>
> The label ITEM does not have to be unique in COLLECTION.

Yes, sounds better. I've adapted the docstring.

All changes are pushed to master. Thanks for your feedback!

Best regards, Michael.

bug#29575: 25.3; Secret Service API treats labels as unique

[Michael Albinus]

Michael Albinus <michael.albinus@gmx.de> writes:

Hi Allen,

>> Thanks.  It doesn't look like there is a way to get the object path by
>> searching attributes?  That means there's no way to get the object path for
>> a secret added from a cooperating application (that may not have a unique
>> label).  secrets-search-items returns labels.  Ideally there would be a
>> variant of secrets-search-items that returned object paths.
>
> Indeed. I've added the function `secrets-search-item-paths'.
>
>> Also, the docstring for secrets-create-items now has this line which I
>> think is misleading:
>>
>> The label ITEM must not be unique in COLLECTION.
>>
>> Should it be:
>>
>> The label ITEM does not have to be unique in COLLECTION.
>
> Yes, sounds better. I've adapted the docstring.
>
> All changes are pushed to master. Thanks for your feedback!

Does it work for you? I'd like to close bugs which are fixed for a long
time; this is one of them.

Best regards, Michael.

bug#29575: 25.3; Secret Service API treats labels as unique

[Allen Li]

Yes, it works. I think this needs to be documented in the auth-source
manual, but otherwise all the functionality seems to be there and
working, thanks.
On Wed, Sep 5, 2018 at 2:00 AM Michael Albinus <michael.albinus@gmx.de> wrote:
>
> Michael Albinus <michael.albinus@gmx.de> writes:
>
> Hi Allen,
>
> >> Thanks.  It doesn't look like there is a way to get the object path by
> >> searching attributes?  That means there's no way to get the object path for
> >> a secret added from a cooperating application (that may not have a unique
> >> label).  secrets-search-items returns labels.  Ideally there would be a
> >> variant of secrets-search-items that returned object paths.
> >
> > Indeed. I've added the function `secrets-search-item-paths'.
> >
> >> Also, the docstring for secrets-create-items now has this line which I
> >> think is misleading:
> >>
> >> The label ITEM must not be unique in COLLECTION.
> >>
> >> Should it be:
> >>
> >> The label ITEM does not have to be unique in COLLECTION.
> >
> > Yes, sounds better. I've adapted the docstring.
> >
> > All changes are pushed to master. Thanks for your feedback!
>
> Does it work for you? I'd like to close bugs which are fixed for a long
> time; this is one of them.
>
> Best regards, Michael.

bug#29575: 25.3; Secret Service API treats labels as unique

[Michael Albinus]

Version: 27.1

Allen Li <darkfeline@felesatra.moe> writes:

Hi Allen,

> Yes, it works. I think this needs to be documented in the auth-source
> manual, but otherwise all the functionality seems to be there and
> working, thanks.

Thanks for checking. I've updated the auth.texi manual, closing the bug.

Best regards, Michael.