unofficial mirror of bug-gnu-emacs@gnu.org 
 help / color / mirror / code / Atom feed
* bug#59538: 29.0.50; Gnus setup broken by commit 2cf9e699ef0fc43a4eadaf00a1ed2f876765c64d
@ 2022-11-24 10:31 João Távora
  2022-11-24 15:01 ` bug#58985: " J.P.
  0 siblings, 1 reply; 4+ messages in thread
From: João Távora @ 2022-11-24 10:31 UTC (permalink / raw)
  To: 59538; +Cc: F. Jason Park

Hi Maintainers,

Commit 2cf9e699ef0fc43a4eadaf00a1ed2f876765c64d breaks my Gnus setup.

    Author: F. Jason Park <jp@neverwas.me>
    Date:   Tue Nov 1 22:46:24 2022 -0700

    Make auth-source-pass behave more like other backends

I've reached this conclusion through 'git bisect'.  I.e. the commit which
immediately precedes it is not broken.

I haven't investigated why, but I do use 'pass' (www.passwordstore.org) to
  (require 'auth-source)
  (auth-source-pass-enable)
  (setq auth-sources '(password-store)) ;; don't use anything else

store my passwords securely.

This is my pass-related setup, which is pretty simple:

After the commit, M-x gnus is unable to connect to my local imap server.  There
is very little debug information.

João
Date: Thu, 24 Nov 2022 10:30:38 +0000
Message-ID: <87wn7kve41.fsf@gmail.com>





^ permalink raw reply	[flat|nested] 4+ messages in thread

* bug#58985: 29.0.50; Gnus setup broken by commit 2cf9e699ef0fc43a4eadaf00a1ed2f876765c64d
  2022-11-24 10:31 bug#59538: 29.0.50; Gnus setup broken by commit 2cf9e699ef0fc43a4eadaf00a1ed2f876765c64d João Távora
@ 2022-11-24 15:01 ` J.P.
  2022-11-24 15:38   ` João Távora
  0 siblings, 1 reply; 4+ messages in thread
From: J.P. @ 2022-11-24 15:01 UTC (permalink / raw)
  To: João Távora; +Cc: Kai Tetzlaff, 58985

Hi João,

João Távora <joaotavora@gmail.com> writes:

> Hi Maintainers,
>
> Commit 2cf9e699ef0fc43a4eadaf00a1ed2f876765c64d breaks my Gnus setup.
>
>     Author: F. Jason Park <jp@neverwas.me>
>     Date:   Tue Nov 1 22:46:24 2022 -0700
>
>     Make auth-source-pass behave more like other backends
>
> I've reached this conclusion through 'git bisect'.  I.e. the commit which
> immediately precedes it is not broken.

Sorry about that. I feel not great that you spent precious man hours
bisecting on my account.

The new option `auth-source-pass-extra-query-params' is behind the
breakage you're witnessing. It tries to make auth-source-pass adhere as
closely as possible ("bug for bug") to the auth-source reference
backend, netrc (but only to the extent that the other backends already
do). The idea was to make searches closer to being backend agnostic and
thus more predictable. And auth-source-pass was the lone holdout in
terms of conforming behavior. But, alas, it's looking like the quest for
uniformity has come at the cost of usability for everyday
auth-source-pass users, which is regrettable and surely a deal breaker
for keeping it enabled by default.

> I haven't investigated why, but I do use 'pass' (www.passwordstore.org) to
>   (require 'auth-source)
>   (auth-source-pass-enable)
>   (setq auth-sources '(password-store)) ;; don't use anything else
>
> store my passwords securely.
>
> This is my pass-related setup, which is pretty simple:
>
> After the commit, M-x gnus is unable to connect to my local imap server.  There
> is very little debug information.

If we were actually gonna try and debug this, I'd probably ask you for
the names of the affected items in your ~/.password-store and the query
params passed to `auth-source-search' and maybe also whatever's printed
to *Messages* when a query is performed with `auth-source-debug' turned
on.

However, I think it's probably best to forgo all that and do what I was
leaning toward from the outset, and that's keeping the new behavior off
by default in Emacs 29. It's looking liable to cause too much churn for
too many folks [1]. Thus, unless anyone objects or has anything else to
add, I will do this in the next 24 hours or so. Apologies again for the
disruption and the time spent bisecting.

J.P.


[1] https://debbugs.gnu.org/cgi/bugreport.cgi?bug=58985#114





^ permalink raw reply	[flat|nested] 4+ messages in thread

* bug#58985: 29.0.50; Gnus setup broken by commit 2cf9e699ef0fc43a4eadaf00a1ed2f876765c64d
  2022-11-24 15:01 ` bug#58985: " J.P.
@ 2022-11-24 15:38   ` João Távora
  2022-11-25 14:23     ` J.P.
  0 siblings, 1 reply; 4+ messages in thread
From: João Távora @ 2022-11-24 15:38 UTC (permalink / raw)
  To: J.P.; +Cc: Kai Tetzlaff, 58985

"J.P." <jp@neverwas.me> writes:

> If we were actually gonna try and debug this, I'd probably ask you for
> the names of the affected items in your ~/.password-store and the query
> params passed to `auth-source-search' and maybe also whatever's printed
> to *Messages* when a query is performed with `auth-source-debug' turned
> on.

The affected item is, I believe, ~/.password-store/local-gmail:imap.gpg
and likely also ~/.password-store/smtp.gmail.com:465.gpg.  When I set
auth-source-debug to t, these lines appeared in *Messages*

auth-source-pass: final result: nil
auth-source-search: found 0 results (max 1) matching (:max 1 :host ("local-gmail" "localhost") :port ("imap" "imap" "143") :user "joaotavora@gmail.com" :require (:user :secret) :create t)
auth-source-pass: final result: nil
auth-source-search: CREATED 0 results (max 1) matching (:max 1 :host ("local-gmail" "localhost") :port ("imap" "imap" "143") :user "joaotavora@gmail.com" :require (:user :secret) :create t)
Opening nnimap server on local-gmail...failed: 

> However, I think it's probably best to forgo all that and do what I was
> leaning toward from the outset, and that's keeping the new behavior off
> by default in Emacs 29. It's looking liable to cause too much churn for
> too many folks [1]. Thus, unless anyone objects or has anything else to
> add, I will do this in the next 24 hours or so. Apologies again for the
> disruption and the time spent bisecting.

No problem, and thanks for understanding.  I think it is indeed better
if you make this opt-in. I can then opt into it and help you debug the
root cause.  But in the meantime, my email won't be broken :-)

João





^ permalink raw reply	[flat|nested] 4+ messages in thread

* bug#58985: 29.0.50; Gnus setup broken by commit 2cf9e699ef0fc43a4eadaf00a1ed2f876765c64d
  2022-11-24 15:38   ` João Távora
@ 2022-11-25 14:23     ` J.P.
  0 siblings, 0 replies; 4+ messages in thread
From: J.P. @ 2022-11-25 14:23 UTC (permalink / raw)
  To: João Távora; +Cc: Kai Tetzlaff, 58985

[-- Attachment #1: Type: text/plain, Size: 2926 bytes --]

João Távora <joaotavora@gmail.com> writes:

> The affected item is, I believe, ~/.password-store/local-gmail:imap.gpg
> and likely also ~/.password-store/smtp.gmail.com:465.gpg.  When I set
> auth-source-debug to t, these lines appeared in *Messages*
>
> auth-source-pass: final result: nil
> auth-source-search: found 0 results (max 1) matching (:max 1 :host
> ("local-gmail" "localhost") :port ("imap" "imap" "143") :user
> "joaotavora@gmail.com" :require (:user :secret) :create t)
> auth-source-pass: final result: nil
> auth-source-search: CREATED 0 results (max 1) matching (:max 1 :host
> ("local-gmail" "localhost") :port ("imap" "imap" "143") :user
> "joaotavora@gmail.com" :require (:user :secret) :create t)
> Opening nnimap server on local-gmail...failed: 

This was helpful, thanks. It seems

  :require (:user ...)

is clashing with the absence of a "user" component in the affected file
names.

Among other things, the commit in question tries to provide a way of
honoring the `:require' keyword in a manner befitting the doc string of
`auth-source-search':

  :require (A B C) means that only results that contain those
  tokens will be returned.  Thus for instance requiring :secret
  will ensure that any results will actually have a :secret
  property.

The other back ends more or less do the same. (Take a peek at the
attached examples if you're bored.) So, I guess the takeaway here, at
least as things stand, is basically this: if for some reason you really
wanted to enable the option, you'd need to rename the affected files.
Either

  ~/.password-store/joaotavora@gmail.com@local-gmail:imap.gpg

or

  ~/.password-store/local-gmail:imap/joaotavora@gmail.com.gpg

should do it. Alternatively, if the gnus function that calls
`auth-source-search' were somehow configurable (guessing no), you could
omit the `:require's altogether, increase the `:max' value, and
prioritize the results, which is what ERC does (or tries to do).

>> However, I think it's probably best to forgo all that and do what I was
>> leaning toward from the outset, and that's keeping the new behavior off
>> by default in Emacs 29. It's looking liable to cause too much churn for
>> too many folks [1]. Thus, unless anyone objects or has anything else to
>> add, I will do this in the next 24 hours or so. Apologies again for the
>> disruption and the time spent bisecting.
>
> No problem, and thanks for understanding.

Thank YOU for understanding. (All I did was break your email.)

> I think it is indeed better if you make this opt-in. I can then opt
> into it and help you debug the root cause. But in the meantime, my
> email won't be broken :-)

I've pushed the change, but you may need to clear your auth-source cache
or restart your session to see any effect. Please let me know if that
doesn't do it. And thanks for all your work on Emacs!

J.P.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-POC-Compare-require-among-auth-source-backends.patch --]
[-- Type: text/x-patch, Size: 14765 bytes --]

From 5b0f0c108578cbae5f0804fe1daa60599a71d4bf Mon Sep 17 00:00:00 2001
From: "F. Jason Park" <jp@neverwas.me>
Date: Thu, 24 Nov 2022 21:03:03 -0800
Subject: [PATCH] [POC] Compare :require among auth-source backends

---
 test/lisp/auth-source-pass-tests.el | 305 ++++++++++++++++++++++++++++
 1 file changed, 305 insertions(+)

diff --git a/test/lisp/auth-source-pass-tests.el b/test/lisp/auth-source-pass-tests.el
index 1107e09b51..ed88cf5476 100644
--- a/test/lisp/auth-source-pass-tests.el
+++ b/test/lisp/auth-source-pass-tests.el
@@ -751,6 +751,311 @@ auth-source-pass-extra-query-keywords--user-priorities
                          (:host "g" :user "u" :port 2 :secret "@") ; **
                          (:host "g" :user "u" :port 2 :secret "/"))))))))
 
+;;;; :require demo
+
+;; Swapping out smtp.gmail.com and 465 with local-gmail and imap
+;; doesn't change the outcome of any cases below.
+
+;; netrc
+
+(ert-deftest auth-source-pass-extra-query-keywords--netrc-joao ()
+  (ert-with-temp-file netrc-file
+    :text "\
+machine local-gmail port imap password a
+machine smtp.gmail.com port 465 password b
+"
+    (let* ((auth-sources (list netrc-file))
+           (auth-source-do-cache nil)
+           (results (auth-source-search :max 1
+                                        :host '("local-gmail" "localhost")
+                                        :port '("imap" "imap" "143")
+                                        :user "joaotavora@gmail.com"
+                                        :require '(:user :secret))))
+      (should-not results))))
+
+(ert-deftest auth-source-pass-extra-query-keywords--netrc-joao-user ()
+  (ert-with-temp-file netrc-file
+    :text "\
+machine local-gmail login joaotavora@gmail.com port imap password a
+machine smtp.gmail.com login joaotavora@gmail.com port 465 password b
+"
+    (let* ((auth-sources (list netrc-file))
+           (auth-source-do-cache nil)
+           (results (auth-source-search :max 1
+                                        :host '("local-gmail" "localhost")
+                                        :port '("imap" "imap" "143")
+                                        :user "joaotavora@gmail.com"
+                                        :require '(:user :secret))))
+      (dolist (result results)
+        (setf (plist-get result :secret) (auth-info-password result)))
+      (should (equal results '(( :host "local-gmail"
+                                 :user "joaotavora@gmail.com"
+                                 :port "imap"
+                                 :secret "a")))))))
+
+;; plstore
+
+(require 'epg)
+
+(ert-deftest auth-source-pass-extra-query-keywords--plstore-joao ()
+  (ert-with-temp-file plstore-file
+    :suffix ".plist"
+    :text "\
+;;; public entries -*- mode: plstore -*-
+((\"7c0d0c60eba2b3da2feebff6a007934b73d6ba8c\"
+  :secret-secret t
+  :host \"local-gmail\"
+  :port \"imap\")
+ (\"e579bb71ac879a2fbe90462be686ec090bcb995f\"
+  :secret-secret t
+  :host \"smtp.gmail.com\"
+  :port \"465\"))
+;;; secret entries
+((\"7c0d0c60eba2b3da2feebff6a007934b73d6ba8c\" :secret \"a\")
+ (\"e579bb71ac879a2fbe90462be686ec090bcb995f\" :secret \"b\"))
+"
+    (cl-letf (((symbol-function 'epg-decrypt-string)
+               (lambda (&rest r) (prin1-to-string (cadr r))))
+              ((symbol-function 'epg-find-configuration)
+               (lambda (&rest _) '((program . "/bin/true")))))
+      (let* ((auth-sources (list plstore-file))
+             (auth-source-do-cache nil)
+             (results (auth-source-search :max 1
+                                          :host '("local-gmail" "localhost")
+                                          :port '("imap" "imap" "143")
+                                          :user "joaotavora@gmail.com"
+                                          :require '(:user :secret))))
+        (should-not results)))))
+
+(ert-deftest auth-source-pass-extra-query-keywords--plstore-joao-user ()
+  (ert-with-temp-file plstore-file
+    :suffix ".plist"
+    :text "\
+;;; public entries -*- mode: plstore -*-
+((\"b0d8e1b370cff2d4c71cd503905d1bfa80247a82\"
+  :secret-secret t
+  :host \"local-gmail\"
+  :user \"joaotavora@gmail.com\"
+  :port \"imap\")
+ (\"5d05df976779ae4690254c6572c1652748ac4b58\"
+  :secret-secret t
+  :host \"smtp.gmail.com\"
+  :user \"joaotavora@gmail.com\"
+  :port \"465\"))
+;;; secret entries
+((\"b0d8e1b370cff2d4c71cd503905d1bfa80247a82\" :secret \"a\")
+ (\"5d05df976779ae4690254c6572c1652748ac4b58\" :secret \"b\"))
+"
+    (cl-letf (((symbol-function 'epg-decrypt-string)
+               (lambda (&rest r) (prin1-to-string (cadr r))))
+              ((symbol-function 'epg-find-configuration)
+               (lambda (&rest _) '((program . "/bin/true")))))
+      (let* ((auth-sources (list plstore-file))
+             (auth-source-do-cache nil)
+             (results (auth-source-search :max 1
+                                          :host '("local-gmail" "localhost")
+                                          :port '("imap" "imap" "143")
+                                          :user "joaotavora@gmail.com"
+                                          :require '(:user :secret))))
+        (dolist (result results)
+          (setf (plist-get result :secret) (auth-info-password result)))
+        (should (equal results '(( :login nil
+                                   :secret "a"
+                                   :host "local-gmail"
+                                   :user "joaotavora@gmail.com"
+                                   :port "imap"))))))))
+
+;; json
+
+(ert-deftest auth-source-pass-extra-query-keywords--json-joao ()
+  (ert-with-temp-file json-store
+    :suffix ".json"
+    :text "\
+[{\"host\":\"local-gmail\",
+  \"port\":\"imap\",
+  \"secret\":\"a\"},
+ {\"host\":\"smtp.gmail.com\",
+  \"port\":\"465\",
+  \"secret\":\"b\"}]
+"
+    (let ((auth-sources (list json-store))
+          (auth-source-do-cache nil))
+      (should-not (auth-source-search :max 1
+                                      :host '("local-gmail" "localhost")
+                                      :port '("imap" "imap" "143")
+                                      :user "joaotavora@gmail.com"
+                                      :require '(:user :secret))))))
+
+(ert-deftest auth-source-pass-extra-query-keywords--json-joao-user ()
+  (ert-with-temp-file json-store
+    :suffix ".json"
+    :text "\
+[{\"host\":\"local-gmail\",
+  \"port\":\"imap\",
+  \"user\":\"joaotavora@gmail.com\",
+  \"secret\":\"a\"},
+ {\"host\":\"smtp.gmail.com\",
+  \"port\":\"465\",
+  \"user\":\"joaotavora@gmail.com\",
+  \"secret\":\"b\"}]
+"
+    (let* ((auth-sources (list json-store))
+           (auth-source-do-cache nil)
+           (results (auth-source-search :max 1
+                                        :host '("local-gmail" "localhost")
+                                        :port '("imap" "imap" "143")
+                                        :user "joaotavora@gmail.com"
+                                        :require '(:user :secret))))
+      (dolist (result results)
+        (setf (plist-get result :secret) (auth-info-password result)))
+      (should (equal results
+                     '(( :host "local-gmail"
+                         :port "imap"
+                         :user "joaotavora@gmail.com"
+                         :secret "a")))))))
+
+;; secrets
+
+;; With the secrets backend, any "attribute" (keyword) specified in
+;; the query must exist in the entry, so `:require' is implied.  The
+;; mocked search below is so contrived as to be meaningless, but it
+;; nevertheless emphasizes the "must match" nature of the upstream
+;; SearchItems DBus method (of the Secret Service API's collections
+;; interface: org.freedesktop.Secret.Collection.SearchItems).
+;;
+;; https://specifications.freedesktop.org/secret-service/latest/re02.html
+;; #org.freedesktop.Secret.Collection.SearchItems
+
+(require 'secrets)
+
+(ert-deftest auth-source-pass-extra-query-keywords--secrets-joao ()
+  (let ((auth-sources '("secrets:Test"))
+        (auth-source-do-cache nil)
+        (entries '(("nil@local-gmail:imap"
+                    (:host . "local-gmail")
+                    (:port . "imap")
+                    (:xdg:schema . "org.freedesktop.Secret.Generic"))
+                   ("nil@smtp.gmail.com:465"
+                    (:host . "smtp.gmail.com")
+                    (:port . "465")
+                    (:xdg:schema . "org.freedesktop.Secret.Generic"))))
+        (secrets '(("nil@local-gmail:imap" . "a")
+                   ("nil@smtp.gmail.com:465" . "b"))))
+
+    (cl-letf (((symbol-function 'secrets-search-items)
+               (lambda (_ &rest r)
+                 (mapcan (lambda (s)
+                           (and (seq-every-p (pcase-lambda (`(,k . ,v))
+                                               (equal v (alist-get k (cdr s))))
+                                             (map-pairs r))
+                                (list (car s))))
+                         entries)))
+              ((symbol-function 'secrets-get-secret)
+               (lambda (_ label) (assoc-default label secrets)))
+              ((symbol-function 'secrets-get-attributes)
+               (lambda (_ label) (assoc-default label entries))))
+
+      (should-not (auth-source-search :max 1
+                                      :host '("local-gmail" "localhost")
+                                      :port '("imap" "imap" "143")
+                                      :user "joaotavora@gmail.com"
+                                      :require '(:user :secret))))))
+
+(ert-deftest auth-source-pass-extra-query-keywords--secrets-joao-user ()
+  (let ((auth-sources '("secrets:Test"))
+        (auth-source-do-cache nil)
+        (entries '(("joaotavora@gmail.com@local-gmail:imap"
+                    (:host . "local-gmail")
+                    (:user . "joaotavora@gmail.com")
+                    (:port . "imap")
+                    (:xdg:schema . "org.freedesktop.Secret.Generic"))
+                   ("joaotavora@gmail.com@smtp.gmail.com:465"
+                    (:host . "smtp.gmail.com")
+                    (:user . "joaotavora@gmail.com")
+                    (:port . "465")
+                    (:xdg:schema . "org.freedesktop.Secret.Generic"))))
+        (secrets '(("joaotavora@gmail.com@local-gmail:imap" . "a")
+                   ("joaotavora@gmail.com@smtp.gmail.com:465" . "b"))))
+
+    (cl-letf (((symbol-function 'secrets-search-items)
+               (lambda (_ &rest r)
+                 (mapcan (lambda (s)
+                           (and (seq-every-p (pcase-lambda (`(,k . ,v))
+                                               (equal v (alist-get k (cdr s))))
+                                             (map-pairs r))
+                                (list (car s))))
+                         entries)))
+              ((symbol-function 'secrets-get-secret)
+               (lambda (_ label) (assoc-default label secrets)))
+              ((symbol-function 'secrets-get-attributes)
+               (lambda (_ label) (assoc-default label entries))))
+
+      (let ((results (auth-source-search :max 1
+                                         :host '("local-gmail" "localhost")
+                                         :port '("imap" "imap" "143")
+                                         :user "joaotavora@gmail.com"
+                                         :require '(:user :secret))))
+        (dolist (result results)
+          (setf (plist-get result :secret) (auth-info-password result)))
+        (should (equal results
+                       '(( :login nil
+                           :secret "a"
+                           :host "local-gmail"
+                           :user "joaotavora@gmail.com"
+                           :port "imap"
+                           :xdg:schema "org.freedesktop.Secret.Generic"))))))))
+
+;; Pass
+
+(ert-deftest auth-source-pass-extra-query-keywords--pass--joao ()
+  (auth-source-pass--with-store '(("smtp.gmail.com:465" (secret . "a"))
+                                  ("local-gmail:imap" (secret . "b")))
+    (auth-source-pass-enable)
+    (let ((auth-source-pass-extra-query-keywords t))
+      (should-not (auth-source-search :max 1
+                                      :host '("local-gmail" "localhost")
+                                      :port '("imap" "imap" "143")
+                                      :user "joaotavora@gmail.com"
+                                      :require '(:user :secret))))))
+
+(ert-deftest auth-source-pass-extra-query-keywords--pass--joao-user ()
+  ;; "suffix" syntax
+  (auth-source-pass--with-store '(("smtp.gmail.com:465/joaotavora@gmail.com"
+                                   (secret . "a"))
+                                  ("local-gmail:imap/joaotavora@gmail.com"
+                                   (secret . "b")))
+    (auth-source-pass-enable)
+    (let* ((auth-source-pass-extra-query-keywords t)
+           (results (auth-source-search :max 1
+                                        :host '("local-gmail" "localhost")
+                                        :port '("imap" "imap" "143")
+                                        :user "joaotavora@gmail.com"
+                                        :require '(:user :secret))))
+      (dolist (result results)
+        (setf (plist-get result :secret) (auth-info-password result)))
+      (should (equal results '(( :host "local-gmail"
+                                 :user "joaotavora@gmail.com"
+                                 :port "imap"
+                                 :secret "b"))))))
+  ;; "prefix" syntax
+  (auth-source-pass--with-store '(("joaotavora@gmail.com@smtp.gmail.com:465"
+                                   (secret . "a"))
+                                  ("joaotavora@gmail.com@local-gmail:imap"
+                                   (secret . "b")))
+    (auth-source-pass-enable)
+    (let* ((auth-source-pass-extra-query-keywords t)
+           (results (auth-source-search :max 1
+                                        :host '("local-gmail" "localhost")
+                                        :port '("imap" "imap" "143")
+                                        :user "joaotavora@gmail.com"
+                                        :require '(:user :secret))))
+      (dolist (result results)
+        (setf (plist-get result :secret) (auth-info-password result)))
+      (should (equal results '(( :host "local-gmail"
+                                 :user "joaotavora@gmail.com"
+                                 :port "imap"
+                                 :secret "b")))))))
+
 (provide 'auth-source-pass-tests)
 
 ;;; auth-source-pass-tests.el ends here
-- 
2.38.1


^ permalink raw reply related	[flat|nested] 4+ messages in thread

end of thread, other threads:[~2022-11-25 14:23 UTC | newest]

Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-11-24 10:31 bug#59538: 29.0.50; Gnus setup broken by commit 2cf9e699ef0fc43a4eadaf00a1ed2f876765c64d João Távora
2022-11-24 15:01 ` bug#58985: " J.P.
2022-11-24 15:38   ` João Távora
2022-11-25 14:23     ` J.P.

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/emacs.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).