tramp-auto-auth.el --- TRAMP automatic authentication library

tramp-auto-auth.el --- TRAMP automatic authentication library

[Bruno Félix Rezende Ribeiro]

[-- Attachment #1: Type: text/plain, Size: 1755 bytes --]


Hello Emacs developers,

Please, find attached ‘tramp-auto-auth.el’.  Quoting from its commentary
header section:

  This library provides ‘tramp-auto-auth-mode’: a global minor mode
  whose purpose is to automatically feed TRAMP sub-processes with
  passwords for paths matching regexps.  This is useful in situations
  where interactive user input is not desirable or feasible.  For
  instance, in sub-nets with large number of hosts or whose hosts have
  dynamic IPs assigned to them.  In those cases it’s not practical to
  query passwords using the ‘auth-source’ library, since this would
  require each host to be listed explicitly and immutably in a Netrc
  file.  Another scenario where this mode is useful are non-interactive
  Emacs sessions (like those used for batch processing or by evaluating
  ‘:async’ Org Babel source blocks) in which it’s impossible for the
  user to answer a password-asking prompt.

This library has proved extremely useful for my work using Emacs to
access remote machines in the aforementioned conditions and whose
authentication policies I couldn’t change.  For quite some time, I’ve
searched extensively for ways of accomplishing the same task with a
practical setup, to no avail.  Perhaps you could comment on alternative
approaches you had experience with but I failed to contemplate.

I’d like to discuss this implementation’s deficiencies and get your
insights and perspectives onto the problem this code is supposed to
solve.  I also would like to discuss how equivalent functionality could
make its way to the standard distribution, in case other people find it
useful enough to be worth supporting out-of-the-box.


Thanks in advance,
Bruno


[-- Attachment #2: tramp-auto-auth.el --]
[-- Type: application/emacs-lisp, Size: 4524 bytes --]

[-- Attachment #3: Type: text/plain, Size: 255 bytes --]


-- 
 88888  FFFFF Bruno Félix Rezende Ribeiro (oitofelix) [0x28D618AF]
 8   8  F     http://oitofelix.freeshell.org/
 88888  FFFF  mailto:oitofelix@gnu.org
 8   8  F     irc://chat.freenode.org/oitofelix
 88888  F     xmpp://oitofelix@riseup.net

Re: tramp-auto-auth.el --- TRAMP automatic authentication library

[Michael Albinus]

Bruno Félix Rezende Ribeiro <oitofelix@gnu.org> writes:

> Hello Emacs developers,

Hi Bruno,

> Please, find attached ‘tramp-auto-auth.el’.  Quoting from its commentary
> header section:
>
>   This library provides ‘tramp-auto-auth-mode’: a global minor mode
>   whose purpose is to automatically feed TRAMP sub-processes with
>   passwords for paths matching regexps.  This is useful in situations
>   where interactive user input is not desirable or feasible.  For
>   instance, in sub-nets with large number of hosts or whose hosts have
>   dynamic IPs assigned to them.  In those cases it’s not practical to
>   query passwords using the ‘auth-source’ library, since this would
>   require each host to be listed explicitly and immutably in a Netrc
>   file.  Another scenario where this mode is useful are non-interactive
>   Emacs sessions (like those used for batch processing or by evaluating
>   ‘:async’ Org Babel source blocks) in which it’s impossible for the
>   user to answer a password-asking prompt.

Thanks for this.

Frankly, I'm not enthusiastic adding cleartext passwords into
Tramp. This has all the security flaws you know, and is good for
problems. At least in core Tramp it shouldn't be propagated.

> This library has proved extremely useful for my work using Emacs to
> access remote machines in the aforementioned conditions and whose
> authentication policies I couldn’t change.  For quite some time, I’ve
> searched extensively for ways of accomplishing the same task with a
> practical setup, to no avail.  Perhaps you could comment on alternative
> approaches you had experience with but I failed to contemplate.

Emacs has the password infrastructure auth-source.el and
password-cache.el, which do their best to avoid password related
problems. If they do not fit your needs, they shall be extended.

The best approach would be if auth-source would support regular
expressions for the declarations of items. Instead of applying declaring

     machine MYMACHINE login MYLOGINNAME password MYPASSWORD port MYPORT

one would declare something like

     machine-regexp REGEXP login MYLOGINNAME password MYPASSWORD port MYPORT

I have used the netrc syntax, and I have add a new keyword; but any
other backend shall work also. I've shortly scanned debbugs; there
doesn't exist (yet) a request to support regular expressions. Maybe you
write a bug report about?

Another approach could be to use different Tramp methods. You could
declare

     password MYPASSWORD port method1
     password OTHERPASSWORD port method2

In Tramp, you would declare new methods method1 and method2, derived
from (for example) ssh. Then you can open /method1:host:/path/to/file
for a host which uses MYPASSWORD, and you can open
/method2:otherhost:/path/to/file for a host which uses OTHERPASSWORD.

> Thanks in advance,
> Bruno

Best regards, Michael.

Re: tramp-auto-auth.el --- TRAMP automatic authentication library

[Bruno Félix Rezende Ribeiro]

[-- Attachment #1: Type: text/plain, Size: 2384 bytes --]

Hello Michael and other GNU Emacs developers,

Thanks for your reply.

Michael Albinus <michael.albinus@gmx.de> writes:

> Frankly, I'm not enthusiastic adding cleartext passwords into
> Tramp. This has all the security flaws you know, and is good for
> problems. At least in core Tramp it shouldn't be propagated.

Please, find attached the implementation of tramp-auto-auth.el using
exclusively the auth-source library.

I did as you suggested except that I didn’t add a new keyword nor made
any change to auth-source.el.

Quoting from the commentary section:

   When a TRAMP prompt is encountered, ‘tramp-auto-auth-mode’ queries
   the alist ‘tramp-auto-auth-alist’ for the auth-source spec value
   whose regexp key matches the correspondent TRAMP path.  This spec
   is then used to query the auth-source library for a presumably
   phony entry exclusively dedicated to the whole class of TRAMP
   paths matching that regexp.

   To make use of the automatic authentication feature, on the Lisp
   side the variable ‘tramp-auto-auth-alist’ must be customized to
   hold the path regexps and their respective auth-source specs, and
   then ‘tramp-auto-auth-mode’ must be enabled.  For example:

   ---- ~/.emacs.el -------------------------------------------------
   (require 'tramp-auto-auth)

   (add-to-list
    'tramp-auto-auth-alist
    '("root@10\\.0\\." .
      (:host "Funny-Machines" :user "root" :port "ssh")))

   (tramp-auto-auth-mode)
   ------------------------------------------------------------------

   After this, just put the respective sacred secret in an
   authentication source supported by auth-source library.  For
   instance:

   ---- ~/.authinfo.gpg ---------------------------------------------
   machine Funny-Machines login root password "$r00tP#sWD!" port ssh
   ------------------------------------------------------------------

   In case you are feeling lazy or the secret is not so secret (nor so
   sacred) -- or for any reason you need to do it all from Lisp --
   it’s enough to:

   (auth-source-remember '(:host "Funny-Machines" :user "root" :port "ssh")
   		         '((:secret "$r00tP#sWD!")))

   And happy TRAMPing!


Is this feature in this form suitable for inclusion in the TRAMP
standard distribution?

Please, let me know what you think.


[-- Attachment #2: tramp-auto-auth.el --]
[-- Type: application/emacs-lisp, Size: 6584 bytes --]

[-- Attachment #3: Type: text/plain, Size: 257 bytes --]



-- 
 88888  FFFFF Bruno Félix Rezende Ribeiro (oitofelix) [0x28D618AF]
 8   8  F     http://oitofelix.freeshell.org/
 88888  FFFF  mailto:oitofelix@gnu.org
 8   8  F     irc://chat.freenode.org/oitofelix
 88888  F     xmpp://oitofelix@riseup.net

Re: tramp-auto-auth.el --- TRAMP automatic authentication library

[Michael Albinus]

Bruno Félix Rezende Ribeiro <oitofelix@gnu.org> writes:

> Hello Michael and other GNU Emacs developers,

Hi Bruno,

>> Frankly, I'm not enthusiastic adding cleartext passwords into
>> Tramp. This has all the security flaws you know, and is good for
>> problems. At least in core Tramp it shouldn't be propagated.
>
> Please, find attached the implementation of tramp-auto-auth.el using
> exclusively the auth-source library.

Thanks for this! It looks better now to my eyes.

> I did as you suggested except that I didn’t add a new keyword nor made
> any change to auth-source.el.
>
> Quoting from the commentary section:
>
>    When a TRAMP prompt is encountered, ‘tramp-auto-auth-mode’ queries
>    the alist ‘tramp-auto-auth-alist’ for the auth-source spec value
>    whose regexp key matches the correspondent TRAMP path.  This spec
>    is then used to query the auth-source library for a presumably
>    phony entry exclusively dedicated to the whole class of TRAMP
>    paths matching that regexp.

Thinking about this, I believe we could use such a mechanism at broader
level. You manage just one Tramp resource (passwords). WIBNI you could
cluster remote hosts also for other resources? For example, in order to
say "users for a given host share the same password if they access via
'ssh' or 'sftp' or 'scp'". Or if you say "the connection property [1]
\"remote-shell\" of a given list of hosts shall be \"/bin/bash\"". Or if
you say "the connection-local variable [2] `tramp-remote-path' for a
given list of hosts shall contain \"/appli/pub/bin\"".

[1] (info "(tramp) Predefined connection information")
[2] (info "(tramp) Remote programs")

Then you could declare just clusters. I would start with a cluster name
(a string), and a list of regular expressions which identify the remote
hosts. Using your example, one would declare

(add-to-list 'tramp-clusters '("Funny-Machines" "root@10\\.0\\." "..."))

For every resource, be it a password, a connection property, or a
connection-local variable, Tramp would always check whether there is a
setting of that resource for the host in question, and if not, whether
there is a setting in a cluster the host belongs to.

This broader approach wouldn't be implemented by an own package via
advising Tramp functions, but in Tramp itself. For the beginning, one
could start with managing passwords this way.

> Is this feature in this form suitable for inclusion in the TRAMP
> standard distribution?

Does this proposal makes sense to you? Would you like to work on this?

Just some comments on your code

> ;; Copyright (C) 2019 Bruno Félix Rezende Ribeiro <oitofelix@gnu.org>

This would be FSF copyrighted, if included in Emacs/Tramp.

> ;; Author: Bruno Félix Rezende Ribeiro <oitofelix@gnu.org>
> ;; Maintainer: Bruno Félix Rezende Ribeiro <oitofelix@gnu.org>

If there is an author, you don't need a maintainer.

> ;; Package-Version: 20190827.1316
> ;; Package-Requires: (tramp)

These entries are needed only in case it would be an ELPA package.

> ;; After this, just put the respective sacred secret in an
> ;; authentication source supported by auth-source library.  For
> ;; instance:
> ;;
> ;; ---- ~/.authinfo.gpg ---------------------------------------------
> ;; machine Funny-Machines login root password "$r00tP#sWD!" port ssh
> ;; ------------------------------------------------------------------

IIRC, neither "login" nor "port" keys are mandatory in auth-source. So
you could live just with "machine" and "password".

> ;; In case you are feeling lazy or the secret is not so secret (nor so
> ;; sacred) -- or for any reason you need to do it all from Lisp --
> ;; it’s enough to:
> ;;
> ;; (auth-source-remember '(:host "Funny-Machines" :user "root" :port "ssh")
> ;; 		         '((:secret "$r00tP#sWD!")))

I wouldn't write this into a Tramp doc. Refer to the "auth" Info pages.

> (defcustom tramp-auto-auth-alist

A defcustom should have a :version key. In case it will be added to
Tramp, :version "27.1" (the first Emacs version this user option has
appeared) would be OK.

>   :require 'tramp-auto-auth)

Why is this needed?

> ;;;###autoload

Please use ";;;###tramp-autoload". The user option makes only sense
after Tramp has been loaded.

> 	(advice-add #'tramp-action-password :around

Code, which is part of core Emacs, shall not advice other
functions. Advicing is intended for user-written Lisp.

Please ensure also, that you do not exceed the 80 chars/line limit, for
better readability.

Best regards, Michael.

Re: tramp-auto-auth.el --- TRAMP automatic authentication library

[Bruno Félix Rezende Ribeiro]

Hello Michael,

Michael Albinus <michael.albinus@gmx.de> writes:

> Thinking about this, I believe we could use such a mechanism at broader
> level. You manage just one Tramp resource (passwords). WIBNI you could
> cluster remote hosts also for other resources? For example, in order to
> say "users for a given host share the same password if they access via
> 'ssh' or 'sftp' or 'scp'". Or if you say "the connection property [1]
> \"remote-shell\" of a given list of hosts shall be \"/bin/bash\"". Or if
> you say "the connection-local variable [2] `tramp-remote-path' for a
> given list of hosts shall contain \"/appli/pub/bin\"".
>
> [1] (info "(tramp) Predefined connection information")
> [2] (info "(tramp) Remote programs")
>
> Then you could declare just clusters. I would start with a cluster name
> (a string), and a list of regular expressions which identify the remote
> hosts. Using your example, one would declare
>
> (add-to-list 'tramp-clusters '("Funny-Machines" "root@10\\.0\\." "..."))
>
> For every resource, be it a password, a connection property, or a
> connection-local variable, Tramp would always check whether there is a
> setting of that resource for the host in question, and if not, whether
> there is a setting in a cluster the host belongs to.
>
> This broader approach wouldn't be implemented by an own package via
> advising Tramp functions, but in Tramp itself. For the beginning, one
> could start with managing passwords this way.
>
>> Is this feature in this form suitable for inclusion in the TRAMP
>> standard distribution?
>
> Does this proposal makes sense to you? Would you like to work on this?

Yes, it does, and I’d like to work on it.  However, it’s not clear to me
if the code I already wrote would be used as an interim solution.  Most
of the remarks you made below seems to me to imply that’s the case,
however this conflicts with the fact that code in core Emacs should not
advice other functions.  I think it’s most likely you are just giving me
general advice as if it were to be included, but I’m not entirely sure;
because (maybe) this code could be included in GNU ELPA.  Could you
please clarify this matter?


> Just some comments on your code
>
>> ;; Copyright (C) 2019 Bruno Félix Rezende Ribeiro <oitofelix@gnu.org>
>
> This would be FSF copyrighted, if included in Emacs/Tramp.

All right.  I’ve already assigned my copyright to the FSF for past and
future contributions to GNU Emacs.


>> ;; Author: Bruno Félix Rezende Ribeiro <oitofelix@gnu.org>
>> ;; Maintainer: Bruno Félix Rezende Ribeiro <oitofelix@gnu.org>
>
> If there is an author, you don't need a maintainer.

Indeed.


>> ;; Package-Version: 20190827.1316
>> ;; Package-Requires: (tramp)
>
> These entries are needed only in case it would be an ELPA package.

I see.  These are there because I’ve uploaded this source file to a
local ELPA.


>> ;; After this, just put the respective sacred secret in an
>> ;; authentication source supported by auth-source library.  For
>> ;; instance:
>> ;;
>> ;; ---- ~/.authinfo.gpg ---------------------------------------------
>> ;; machine Funny-Machines login root password "$r00tP#sWD!" port ssh
>> ;; ------------------------------------------------------------------
>
> IIRC, neither "login" nor "port" keys are mandatory in auth-source. So
> you could live just with "machine" and "password".

OK.


>> ;; In case you are feeling lazy or the secret is not so secret (nor so
>> ;; sacred) -- or for any reason you need to do it all from Lisp --
>> ;; it’s enough to:
>> ;;
>> ;; (auth-source-remember '(:host "Funny-Machines" :user "root" :port "ssh")
>> ;; 		         '((:secret "$r00tP#sWD!")))
>
> I wouldn't write this into a Tramp doc. Refer to the "auth" Info pages.

My intention with this was to document the way I found for another
private library to setup an auth-source entry automatically without
fiddling with user’s Netrc files.

I couldn’t find any reference to this function in auth-source’s manual.
Perhaps a section about this technique is needed there before we can
refer to it?


>> (defcustom tramp-auto-auth-alist
>
> A defcustom should have a :version key. In case it will be added to
> Tramp, :version "27.1" (the first Emacs version this user option has
> appeared) would be OK.

Noted.


>>   :require 'tramp-auto-auth)
>
> Why is this needed?

Quoting from the documentation on ‘:global’ in ‘(elisp) Defining Minor
Modes’:

  One of the effects of making a minor mode global is that the MODE
  variable becomes a customization variable.  Toggling it through the
  Customize interface turns the mode on and off, and its value can be
  saved for future Emacs sessions.  For the saved variable to work, you
  should ensure that the ‘define-minor-mode’ form is evaluated each time
  Emacs starts; for packages that are not part of Emacs, the easiest way
  to do this is to specify a ‘:require’ keyword.

Since ‘tramp-auto-auth’ is not part of Emacs, I thought that was needed.


>> ;;;###autoload
>
> Please use ";;;###tramp-autoload". The user option makes only sense
> after Tramp has been loaded.

Where is this ‘;;;###<package>-autoload’ cookie feature documented?  I
couldn’t find it.


>> 	(advice-add #'tramp-action-password :around
>
> Code, which is part of core Emacs, shall not advice other
> functions. Advicing is intended for user-written Lisp.

I couldn’t find any other way to obtain the same effect.  Do you have
any suggestion?


> Please ensure also, that you do not exceed the 80 chars/line limit, for
> better readability.

Sure.


Best regards,

-- 
 88888  FFFFF Bruno Félix Rezende Ribeiro (oitofelix) [0x28D618AF]
 8   8  F     http://oitofelix.freeshell.org/
 88888  FFFF  mailto:oitofelix@gnu.org
 8   8  F     irc://chat.freenode.org/oitofelix
 88888  F     xmpp://oitofelix@riseup.net

Re: tramp-auto-auth.el --- TRAMP automatic authentication library

[Michael Albinus]

Bruno Félix Rezende Ribeiro <oitofelix@gnu.org> writes:

> Hello Michael,

Hi Bruno,

> However, it’s not clear to me
> if the code I already wrote would be used as an interim solution.

Is this needed? Tramp is available as ELPA package, and as soon as you
provide patches, it will be there (after next sync).

> I think it’s most likely you are just giving me general advice as if
> it were to be included, but I’m not entirely sure; because (maybe)
> this code could be included in GNU ELPA.  Could you please clarify
> this matter?

My proposal is to improve vanilla Tramp.

>>> ;; In case you are feeling lazy or the secret is not so secret (nor so
>>> ;; sacred) -- or for any reason you need to do it all from Lisp --
>>> ;; it’s enough to:
>>> ;;
>>> ;; (auth-source-remember '(:host "Funny-Machines" :user "root" :port "ssh")
>>> ;; 		         '((:secret "$r00tP#sWD!")))
>>
>> I wouldn't write this into a Tramp doc. Refer to the "auth" Info pages.
>
> My intention with this was to document the way I found for another
> private library to setup an auth-source entry automatically without
> fiddling with user’s Netrc files.
>
> I couldn’t find any reference to this function in auth-source’s manual.
> Perhaps a section about this technique is needed there before we can
> refer to it?

Yes, auth.texi is the place to update. Patches welcome.

>>>   :require 'tramp-auto-auth)
>>
>> Why is this needed?
>
> Quoting from the documentation on ‘:global’ in ‘(elisp) Defining Minor
> Modes’:
>
>   One of the effects of making a minor mode global is that the MODE
>   variable becomes a customization variable.  Toggling it through the
>   Customize interface turns the mode on and off, and its value can be
>   saved for future Emacs sessions.  For the saved variable to work, you
>   should ensure that the ‘define-minor-mode’ form is evaluated each time
>   Emacs starts; for packages that are not part of Emacs, the easiest way
>   to do this is to specify a ‘:require’ keyword.
>
> Since ‘tramp-auto-auth’ is not part of Emacs, I thought that was needed.

When we have added your code to Tramp, it will also be part of
Emacs. Well, using it via ELPA for older Emacsen might still require
this technique. So maybe you keep it (changing to :require 'tramp).

>>> ;;;###autoload
>>
>> Please use ";;;###tramp-autoload". The user option makes only sense
>> after Tramp has been loaded.
>
> Where is this ‘;;;###<package>-autoload’ cookie feature documented?  I
> couldn’t find it.

Read the documentation of `generate-autoload-cookie' and
`generated-autoload-file' in (info "(elisp) Autoload")

>>> 	(advice-add #'tramp-action-password :around
>>
>> Code, which is part of core Emacs, shall not advice other
>> functions. Advicing is intended for user-written Lisp.
>
> I couldn’t find any other way to obtain the same effect.  Do you have
> any suggestion?

Change `tramp-read-passwd' accordingly.

> Best regards,

Best regards, Michael.

Re: tramp-auto-auth.el --- TRAMP automatic authentication library

[Stefan Monnier]

> cluster remote hosts also for other resources? For example, in order to
> say "users for a given host share the same password if they access via
> 'ssh' or 'sftp' or 'scp'". Or if you say "the connection property [1]

FWIW, additionally to ssh, sftp, and scp, one might also want to include
protocols like IMAP or SMTP.

Which then suggests that maybe doing this outside/above Tramp (e.g. in
auth-source?), so it can be used by both Gnus and Tramp, would be
even better.


        Stefan

Re: tramp-auto-auth.el --- TRAMP automatic authentication library

[Michael Albinus]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

Hi Stefan,

>> cluster remote hosts also for other resources? For example, in order to
>> say "users for a given host share the same password if they access via
>> 'ssh' or 'sftp' or 'scp'". Or if you say "the connection property [1]
>
> FWIW, additionally to ssh, sftp, and scp, one might also want to include
> protocols like IMAP or SMTP.
>
> Which then suggests that maybe doing this outside/above Tramp (e.g. in
> auth-source?), so it can be used by both Gnus and Tramp, would be
> even better.

Yes. The downside for Tramp would be, that it isn't backward compatible
anymore.

>         Stefan

Best regards, Michael.