url-auth-tests hang if authinfo is encrypted

url-auth-tests hang if authinfo is encrypted

[Colin Baxter]

Hello,

My ~/.authinfo is encrypted to ~/.authinfo.gpg; consequently,
url-auth-tests hangs, waiting for a password that never comes. If I
decrypt the file to ~/.authinfo then the test succeeds.

Is there anyway the test could be made to prompt the user to enter the
password? Failing that then perhaps the test ought to fail cleanly when
~/.authinfo.gpg is found.

Encrypting the ~/.authinfo is quite common, I believe.

Thanks,

Colin.

Re: url-auth-tests hang if authinfo is encrypted

[Eli Zaretskii]

> From: Colin Baxter <m43cap@yandex.com>
> Cc: 
> Date: Mon, 03 Apr 2017 20:20:44 +0100
> 
> My ~/.authinfo is encrypted to ~/.authinfo.gpg; consequently,
> url-auth-tests hangs, waiting for a password that never comes. If I
> decrypt the file to ~/.authinfo then the test succeeds.

What branch are you running? master or emacs-25?

Re: url-auth-tests hang if authinfo is encrypted

[Colin Baxter]

On Mon, Apr 03 2017, Eli Zaretskii wrote:

>> From: Colin Baxter <m43cap@yandex.com>
>> Cc: 
>> Date: Mon, 03 Apr 2017 20:20:44 +0100
>> 
>> My ~/.authinfo is encrypted to ~/.authinfo.gpg; consequently,
>> url-auth-tests hangs, waiting for a password that never comes. If I
>> decrypt the file to ~/.authinfo then the test succeeds.
>
> What branch are you running? master or emacs-25?

Master, on 3.2.0-4-686-pae #1 SMP Debian 3.2.86-1 i686 GNU/Linux. Sorry,
I should have sent this information first. For me, the file ~/.authinfo
just has to exist - it can be empty - for url-auth-tests to succeed.

Best wishes, Colin.

Re: url-auth-tests hang if authinfo is encrypted

[Eli Zaretskii]

> From: Colin Baxter <m43cap@yandex.com>
> Cc: emacs-devel@gnu.org
> Date: Tue, 04 Apr 2017 07:15:30 +0100
> 
> On Mon, Apr 03 2017, Eli Zaretskii wrote:
> 
> >> From: Colin Baxter <m43cap@yandex.com>
> >> Cc: 
> >> Date: Mon, 03 Apr 2017 20:20:44 +0100
> >> 
> >> My ~/.authinfo is encrypted to ~/.authinfo.gpg; consequently,
> >> url-auth-tests hangs, waiting for a password that never comes. If I
> >> decrypt the file to ~/.authinfo then the test succeeds.
> >
> > What branch are you running? master or emacs-25?
> 
> Master, on 3.2.0-4-686-pae #1 SMP Debian 3.2.86-1 i686 GNU/Linux. Sorry,
> I should have sent this information first. For me, the file ~/.authinfo
> just has to exist - it can be empty - for url-auth-tests to succeed.

Jarno, any chance you could look into this issue?  TIA.

[PATCH] Fix potentially hanging url-auth tests

[Jarno Malmari]

* test/lisp/url/url-auth-tests.el (url-auth-test-digest-auth-retrieve-cache):
(url-auth-test-digest-auth, url-auth-test-digest-auth-opaque):
Set auth-sources variable to nil.
---
 test/lisp/url/url-auth-tests.el | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/test/lisp/url/url-auth-tests.el b/test/lisp/url/url-auth-tests.el
index 30636db..47aafab 100644
--- a/test/lisp/url/url-auth-tests.el
+++ b/test/lisp/url/url-auth-tests.el
@@ -137,7 +137,9 @@ url-auth-test-challenges
   "Check how the entry point retrieves cached authentication.
 Essential is how realms and paths are matched."
 
-  (let* ((url-digest-auth-storage
+  (let* ((auth-sources nil) ; Don't use any user authinfo files.  They
+                            ; may even hang the test run if encrypted.
+         (url-digest-auth-storage
           '(("example.org:80"
              ("/path/auth1" "auth1user" "key")
              ("/path" "pathuser" "key")
@@ -231,7 +233,8 @@ url-auth-test-challenges
 cnonce is used for generating them which is not mocked by the
 test and cannot be passed by arguments to `url-digest-auth'."
   (dolist (challenge url-auth-test-challenges)
-    (let* ((attrs (append
+    (let* ((auth-sources nil) ; Don't use any user authinfo files.
+           (attrs (append
                    (list (cons "nonce" (plist-get challenge :nonce)))
                    (if (plist-get challenge :qop)
                        (list (cons "qop" (plist-get challenge :qop))))))
@@ -276,7 +279,8 @@ url-auth-test-challenges
 (ert-deftest url-auth-test-digest-auth-opaque ()
   "Check that `opaque' value is added to result when presented by
 the server."
-  (let* ((url-digest-auth-storage
+  (let* ((auth-sources nil) ; Don't use any user authinfo files.
+         (url-digest-auth-storage
           '(("example.org:80" ("/" "user" "key"))))
          (attrs (list (cons "nonce" "anynonce")))
          auth)
-- 
2.7.0.25.gfc10eb5

Re: [PATCH] Fix potentially hanging url-auth tests

[Colin Baxter]

Dear Jarno,

On Sat, Apr 15 2017, Jarno Malmari wrote:

> * test/lisp/url/url-auth-tests.el (url-auth-test-digest-auth-retrieve-cache):
> (url-auth-test-digest-auth, url-auth-test-digest-auth-opaque):
> Set auth-sources variable to nil.
> ---
>  test/lisp/url/url-auth-tests.el | 10 +++++++---
>  1 file changed, 7 insertions(+), 3 deletions(-)
>
> diff --git a/test/lisp/url/url-auth-tests.el b/test/lisp/url/url-auth-tests.el
> index 30636db..47aafab 100644
> --- a/test/lisp/url/url-auth-tests.el
> +++ b/test/lisp/url/url-auth-tests.el
> @@ -137,7 +137,9 @@ url-auth-test-challenges
>    "Check how the entry point retrieves cached authentication.
>  Essential is how realms and paths are matched."
>  
> -  (let* ((url-digest-auth-storage
> +  (let* ((auth-sources nil) ; Don't use any user authinfo files.  They
> +                            ; may even hang the test run if encrypted.
> +         (url-digest-auth-storage
>            '(("example.org:80"
>               ("/path/auth1" "auth1user" "key")
>               ("/path" "pathuser" "key")
> @@ -231,7 +233,8 @@ url-auth-test-challenges
>  cnonce is used for generating them which is not mocked by the
>  test and cannot be passed by arguments to `url-digest-auth'."
>    (dolist (challenge url-auth-test-challenges)
> -    (let* ((attrs (append
> +    (let* ((auth-sources nil) ; Don't use any user authinfo files.
> +           (attrs (append
>                     (list (cons "nonce" (plist-get challenge :nonce)))
>                     (if (plist-get challenge :qop)
>                         (list (cons "qop" (plist-get challenge :qop))))))
> @@ -276,7 +279,8 @@ url-auth-test-challenges
>  (ert-deftest url-auth-test-digest-auth-opaque ()
>    "Check that `opaque' value is added to result when presented by
>  the server."
> -  (let* ((url-digest-auth-storage
> +  (let* ((auth-sources nil) ; Don't use any user authinfo files.
> +         (url-digest-auth-storage
>            '(("example.org:80" ("/" "user" "key"))))
>           (attrs (list (cons "nonce" "anynonce")))
>           auth)


This patch works for me. Great. Thank you.

Best wishes,

Colin.

--
Colin Baxter
m43cap@yandex.com
Key fingerprint = 68A8 799C 0230 16E7 BF68  2A27 BBFA 2492 91F5 41C8

Re: [PATCH] Fix potentially hanging url-auth tests

[Jarno Malmari]

On Sat, 15 Apr 2017, at 18:36, Colin Baxter wrote:
> This patch works for me. Great. Thank you.

Good to hear. Thanks for testing.

As you can see, the fix is local to the test itself. I think this is the
right approach since it is the responsibility of the test to know what
side effects the function calls have (reading the file .authinfo in this
case). The thing I'm not happy about is the possibility of reintroducing
the issue in the future. There's no check for that, but hopefully the
code comments give a clue what to avoid.

Sure, other testing facilities discussed earlier (customized HOME, or a
way to detect if run inside a test) might be useful but probably not
warranted solely by the issue here.

Re: url-auth-tests hang if authinfo is encrypted

[Ted Zlatanov]

On Mon, 03 Apr 2017 20:20:44 +0100 Colin Baxter <m43cap@yandex.com> wrote: 

CB> My ~/.authinfo is encrypted to ~/.authinfo.gpg; consequently,
CB> url-auth-tests hangs, waiting for a password that never comes. If I
CB> decrypt the file to ~/.authinfo then the test succeeds.

CB> Is there anyway the test could be made to prompt the user to enter the
CB> password? Failing that then perhaps the test ought to fail cleanly when
CB> ~/.authinfo.gpg is found.

CB> Encrypting the ~/.authinfo is quite common, I believe.

I think the user's personal setup shouldn't affect the tests.

The solution can be either automatic (auth-source knows it's inside a
test, or EPG doesn't try to decrypt in a test environment) or explicit
(tests override `auth-sources' and/or EPG). For the automatic
option, I can add the code to auth-source if anyone can suggest how to
tell it's inside a test environment.

Maybe others can think of other options?

Ted

Re: url-auth-tests hang if authinfo is encrypted

[Andreas Schwab]

On Apr 04 2017, Ted Zlatanov <tzz@lifelogs.com> wrote:

> The solution can be either automatic (auth-source knows it's inside a
> test, or EPG doesn't try to decrypt in a test environment) or explicit
> (tests override `auth-sources' and/or EPG). For the automatic
> option, I can add the code to auth-source if anyone can suggest how to
> tell it's inside a test environment.
>
> Maybe others can think of other options?

I think the test environment should set up a separate HOME for emacs to
read all its startup files.

Andreas.

-- 
Andreas Schwab, SUSE Labs, schwab@suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

Re: url-auth-tests hang if authinfo is encrypted

[Colin Baxter]

Dear Ted,

On Tue, Apr 04 2017, Ted Zlatanov wrote:

> On Mon, 03 Apr 2017 20:20:44 +0100 Colin Baxter <m43cap@yandex.com> wrote: 
>
> CB> My ~/.authinfo is encrypted to ~/.authinfo.gpg; consequently,
> CB> url-auth-tests hangs, waiting for a password that never comes. If I
> CB> decrypt the file to ~/.authinfo then the test succeeds.
>
> CB> Is there anyway the test could be made to prompt the user to enter the
> CB> password? Failing that then perhaps the test ought to fail cleanly when
> CB> ~/.authinfo.gpg is found.
>
> CB> Encrypting the ~/.authinfo is quite common, I believe.
>
> I think the user's personal setup shouldn't affect the tests.
>
> The solution can be either automatic (auth-source knows it's inside a
> test, or EPG doesn't try to decrypt in a test environment) or explicit
> (tests override `auth-sources' and/or EPG). For the automatic
> option, I can add the code to auth-source if anyone can suggest how to
> tell it's inside a test environment.
>
> Maybe others can think of other options?
>

Would it help if users like me were to move their encrypted authinfo
away from $HOME by using something like

(setq auth-sources
    '((:source "~/some/other/path/to/.authinfo.gpg")))

in their ~/.emacs?

Perhaps I am being naive.

Best wishes, Colin.

Re: url-auth-tests hang if authinfo is encrypted

[Colin Baxter]

Dear Ted,
------------ snip -----------
>>
>> I think the user's personal setup shouldn't affect the tests.
>>
>> The solution can be either automatic (auth-source knows it's inside a
>> test, or EPG doesn't try to decrypt in a test environment) or explicit
>> (tests override `auth-sources' and/or EPG). For the automatic
>> option, I can add the code to auth-source if anyone can suggest how to
>> tell it's inside a test environment.
>>
>> Maybe others can think of other options?
>>
>
> Would it help if users like me were to move their encrypted authinfo
> away from $HOME by using something like
>
> (setq auth-sources
>     '((:source "~/some/other/path/to/.authinfo.gpg")))
>
> in their ~/.emacs?
>

This certainly works  - for me at least. The url-auth-tests pass
successfully. I suppose though the change might be a too major to be
accepted as standard.

Best wishes, Colin.

Re: url-auth-tests hang if authinfo is encrypted

[Ted Zlatanov]

On Wed, 05 Apr 2017 07:43:09 +0100 Colin Baxter <m43cap@yandex.com> wrote: 

>> Ted Zlatanov wrote:
>>> I think the user's personal setup shouldn't affect the tests.
>>> 
>>> The solution can be either automatic (auth-source knows it's inside a
>>> test, or EPG doesn't try to decrypt in a test environment) or explicit
>>> (tests override `auth-sources' and/or EPG). For the automatic
>>> option, I can add the code to auth-source if anyone can suggest how to
>>> tell it's inside a test environment.
>>> 
>>> Maybe others can think of other options?
>>> 
>> 
>> Would it help if users like me were to move their encrypted authinfo
>> away from $HOME by using something like
>> 
>> (setq auth-sources
>> '((:source "~/some/other/path/to/.authinfo.gpg")))
>> 
>> in their ~/.emacs?

CB> This certainly works  - for me at least. The url-auth-tests pass
CB> successfully. I suppose though the change might be a too major to be
CB> accepted as standard.

Your suggestion is the explicit option, which has the downside that
every test has to do it, but yeah that would work.

On Tue, 04 Apr 2017 18:04:39 +0200 Andreas Schwab <schwab@suse.de> wrote: 

AS> I think the test environment should set up a separate HOME for emacs to
AS> read all its startup files.

I think that's better than the explicit option. At least for this case,
and probably in general. Can anyone think of reasons not to do it?

Ted

Re: url-auth-tests hang if authinfo is encrypted

[Jarno Malmari]

>>> Ted Zlatanov wrote:
>>>> I think the user's personal setup shouldn't affect the tests.

Yes, definitely should not.

>>> Would it help if users like me were to move their encrypted authinfo
>>> away from $HOME by using something like
>>> 
>>> (setq auth-sources
>>> '((:source "~/some/other/path/to/.authinfo.gpg")))
>>> 
>>> in their ~/.emacs?
>
> CB> This certainly works  - for me at least. The url-auth-tests pass
> CB> successfully. I suppose though the change might be a too major to be
> CB> accepted as standard.

How are people running the tests?

The tests are currently not testing the contents of any authinfo files
so setting auth-sources to nil would work. For example:
emacs -Q -batch --eval='(setq auth-sources nil)' -l ert -l
test/lisp/url/url-auth-tests.el -f ert-run-tests-batch-and-exit

> AS> I think the test environment should set up a separate HOME for emacs to
> AS> read all its startup files.
>
> I think that's better than the explicit option. At least for this case,
> and probably in general. Can anyone think of reasons not to do it?

Separate HOME should then always point to empty or test specific
directory. Shared directory would sound like trouble. Test specific one
would allow adding url-auth-tests.el specific authinfo content, if such
would be added in the future.

Re: url-auth-tests hang if authinfo is encrypted

[Colin Baxter]

Dear Jarno,

On Thu, Apr 06 2017, Jarno Malmari wrote:
>>>> Ted Zlatanov wrote:
>>>>> I think the user's personal setup shouldn't affect the tests.
>
> Yes, definitely should not.
>
>>>> Would it help if users like me were to move their encrypted authinfo
>>>> away from $HOME by using something like
>>>> 
>>>> (setq auth-sources
>>>> '((:source "~/some/other/path/to/.authinfo.gpg")))
>>>> 
>>>> in their ~/.emacs?
>>
>> CB> This certainly works  - for me at least. The url-auth-tests pass
>> CB> successfully. I suppose though the change might be a too major to be
>> CB> accepted as standard.
>
> How are people running the tests?
>
> The tests are currently not testing the contents of any authinfo files
> so setting auth-sources to nil would work. For example:
> emacs -Q -batch --eval='(setq auth-sources nil)' -l ert -l
> test/lisp/url/url-auth-tests.el -f ert-run-tests-batch-and-exit

This is wonderfully elegant, but a simple-minded user - like me - could
perhaps blindly use 'make check' and be puzzled by the result.

Best wishes, Colin.

Re: url-auth-tests hang if authinfo is encrypted

[Eli Zaretskii]

> From: Colin Baxter <m43cap@yandex.com>
> Date: Fri, 07 Apr 2017 08:22:31 +0100
> Cc: , emacs-devel@gnu.org
> 
> > How are people running the tests?
> >
> > The tests are currently not testing the contents of any authinfo files
> > so setting auth-sources to nil would work. For example:
> > emacs -Q -batch --eval='(setq auth-sources nil)' -l ert -l
> > test/lisp/url/url-auth-tests.el -f ert-run-tests-batch-and-exit
> 
> This is wonderfully elegant, but a simple-minded user - like me - could
> perhaps blindly use 'make check' and be puzzled by the result.

I frequently run individual tests like this:

  cd test && make SOMETEST.log

For example, "make lisp/files-tests.log" will run just the tests in
files-tests.el.  (Sometimes, I need to remove an existing .log file,
if Make says "SOMETEST.log is up-to-date".)

Re: url-auth-tests hang if authinfo is encrypted

[Colin Baxter]

On Fri, Apr 07 2017, Eli Zaretskii wrote:

>> From: Colin Baxter <m43cap@yandex.com>
>> Date: Fri, 07 Apr 2017 08:22:31 +0100
>> Cc: , emacs-devel@gnu.org
>> 
>> > How are people running the tests?
>> >
>> > The tests are currently not testing the contents of any authinfo files
>> > so setting auth-sources to nil would work. For example:
>> > emacs -Q -batch --eval='(setq auth-sources nil)' -l ert -l
>> > test/lisp/url/url-auth-tests.el -f ert-run-tests-batch-and-exit
>> 
>> This is wonderfully elegant, but a simple-minded user - like me - could
>> perhaps blindly use 'make check' and be puzzled by the result.
>
> I frequently run individual tests like this:
>
>   cd test && make SOMETEST.log
>
> For example, "make lisp/files-tests.log" will run just the tests in
> files-tests.el.  (Sometimes, I need to remove an existing .log file,
> if Make says "SOMETEST.log is up-to-date".)

I'll try this in future, thank you.

Colin.