bug#67937: 30.0.50; auth-source-pass relies on epa-file being enabled

["Arsen Arsenović via Bug reports for GNU Emacs, the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>]

[-- Attachment #1: Type: text/plain, Size: 9531 bytes --]

Hi Michael,

Please keep the bug tracker in CC.

Michael Albinus <michael.albinus@gmx.de> writes:

>>> A .gpg file could be taken from a remote location. In that case, you
>>> have two file name handlers, which must cooperate: epa-file-handler, and
>>> tramp-file-name-handler.
>>
>> No, just one: tramp-file-name-handler.  epa-file-handler has nothing to
>> do with remote file access.
>
> So you don't knmow the principle of file name handlers.

I do not appreciate this dismissal.  I am very aware that
epa-file-handler is called in that scenario, that it processes the file,
and that it temporarily inhibits itself in order to cause TRAMP to
access a remote file.  I was trying to imply that epa-file-handler has
nothing to do with remote access there, it simply recurses as normal.

> The idea is to provide an alternative implementation for several basic
> primitive functions in Emacs. You can still use those functions, like
> insert-file-contents, but the file name handler uses its own
> implementation if it detects, that the file is special (on a remote
> host, compressed, whatever). And the file name handlers are combined
> if the file is special in different ways.
>
> Make a test: There is a file, let's say file.gpg. You open it via 'M-x
> find-file RET /path/to/file.gpg'. In the corresponding buffer you'll see
> the contents of decrypted file.gpg, thanks to the file name handler
> epa-file-handler.
>
>>> Furthermore, a .gpg file could be compressed, like file.gpg.gz.
>>
>> No, it cannot, not in a pass store.  Here's an example:
>>
>>   ~/.password-store$ touch thing.gpg.gz
>>   ~/.password-store$ pass show thing
>>   Error: thing is not in the password store.
>
> Now compress the file on the shell to, let's say, foo.gpg.gz. Open this
> file like 'M-x find-file /path/to/file.gpg.gz'. You should see now the
> contents of foo.gpg, again, due to file name handlers jka-compr-handler
> and epa-file-handler. Actually, this doesn't work ATM, due to an error
> in jka-compr-handler (the temporary file should keep the suffix of the
> original file), I'll check next days.

I know, this is how I keep journals (well, not compressed, but I have
encrypted files in my Org setup).  This isn't relevant here because...

> Of course, it doesn't work in your example on *shell* level. However, on
> Emacs level it should work, because jka-compr-handler should return an
> uncompressed temp file /tmp/xyz.gpg, and epa-file-handler should know
> how to handle this gpg file.

... the reference implementation does not do this, and it has no reason
to do this, because the 'storage format' in question does not support
compression, and so, auth-source-pass has no reason to implement this.

> And when calling 'pass show', that temp file /tmp/xyz.gpg should be used.

Nothing here calls 'pass show', mind you.  'pass show' was called in my
example not to demonstrate something Emacs does, but what the reference
implementation for the job auth-source-pass implements does.

Please read through auth-source-pass.el: it is a reimplementation.  This
is fine, of course.  It is short, so it should be pretty easy to read
through.

>> In general, a pass file is _specifically_ a gpg-encrypted file (and it
>> says so in the manual), and, indeed, pass assumes so, a lot:
>
> Of course. jka-compr-handler is responsible to convert the compressed
> file.gpg.gz to the uncompressed temp file xyz.gpg.

Indeed.  However, this does not matter, as a pass file is specifically
only a gpg-encrypted file, without any compression.

>> ... as does auth-source-pass:
>>
>> (defun auth-source-pass--read-entry (entry)
>>   "Return a string with the file content of ENTRY."
>>   (with-temp-buffer
>>     (insert-file-contents (expand-file-name
>>                            (format "%s.gpg" entry)
>>                            auth-source-pass-filename))
>>     (buffer-substring-no-properties (point-min) (point-max))))
>>
>> ;; TODO: add tests for that when `assess-with-filesystem' is included
>> ;; in Emacs
>> (defun auth-source-pass-entries ()
>>   "Return a list of all password store entries."
>>   (let ((store-dir (expand-file-name auth-source-pass-filename)))
>>     (mapcar
>>      (lambda (file) (file-name-sans-extension (file-relative-name file store-dir)))
>>      (directory-files-recursively store-dir "\\.gpg\\'"))))
>>
>> This is fine, of course, not making this assumption would be
>> unreasonable because of what the format of pass stores is.
>
> This must be enhanced then. A compressed (or remote) gpg file shall be
> acceptable as well.

No, there's no reason to "enhance" this - in fact, doing so could only
complicate this logic, for no reason.  Pass stores do not contain
compressed files (besides, I am not sure there is even a general
solution for this "enhancement").

The only possible result of this "enhancement" is a difference of
behavior between auth-source-pass and the reference implementation of
pass.  I don't know about you, but I don't feel comfortable with a
noncompatible implementation handling my passwords.

>> I do understand that pass also does not cover TRAMP the same way it does
>> not cover compressed files, but I don't believe this is relevant here:
>> when we discuss a filesystem hierarchy, the TRAMP handler serves to
>> remap it to a remote location, while the EPA file handler serves to
>> _alter contents_.  This is quite different.
>
> The same scenario: Tramp shall provide a local temp file xyz.gpg of the
> remote file /ssh:remotehost:/path/to/file.gpg.gz.

I'm not sure I follow.  How is this related to the paragraph above it?

I was trying to show a distinction between the jobs of epa-file-handler
and tramp-file-name-handler: the former "remaps" the contents of a file,
while the latter "remaps" the filesystem layout, which are very distinct
jobs, and to justify why the reference implementation not supporting
either is irrelevant.

We don't benefit from customizing of the former in auth-source-pass (it
would be akin to being able to change what cons does, IMO).

>> Emacs recognizes this: '-literally' file operations support TRAMP, but
>> not the content-altering handlers.  This is neat, I think.
>
> I don't know whether the other handlers need to support
> insert-file-contents-literally. To be investigated.

All the ones relevant to retrieving file contents unchanged ought to, or
ought to be altered to.

In the paragraph above, I said "'-literally' file operations support
TRAMP".  I meant this vice-versa, of course: TRAMP implements the
'-literally' file operations.  epa-file intentionally omits that
implementation, as far as I can see.

> But likely, it is sufficient, that they support file-local-copy. See:
>
> --8<---------------cut here---------------start------------->8---
> (file-local-copy "~/foo.gpg") = nil
> (file-local-copy "~/foo.gpg.gz") => "/tmp/jka-comeyhZmp"
> (file-local-copy "/ssh::~/foo.gpg") => "/home/albinus/.cache/emacs/tramp.fxF6zy.gpg"
> --8<---------------cut here---------------end--------------->8---
>
> Yes, there is the error, that jka-compr-handler does not keep the
> suffix, but this will be fixed. And with this, you will always ensure,
> that you have a local gpg file, a copy of the compressed or remote file.
>
>>> No, it doesn't make sense to bypass the file name handler machinery.
>>
>> Indeed - I have not implied otherwise.  There are useful handlers.
>> epa-file is not one of them for this use-case.
>
> My experience over some decades is, that whenever we don't use the
> file name handler mechanism where we should, we will run into trouble
> midterm.

And I argue we should not _specifically_ for the file contents.  For
reaching pass entries, of course, we should.

To reiterate my point, pass files are not just some files on the disk,
they are specifically .gpg encrypted files, and as such, have only one
reasonable thing to do with their contents: decrypt them.  There is
nothing else reasonable to do, so getting the file contents literally
and decrypting them seems like the most robust and correct approach.

The current approach does precisely this (it is not possible for
anything else to happen, because all files in a pass store are .gpg
files), but it relies on file-name-handler-alist being set correctly,
which is demonstrably unreliable.

It is only the contents of a pass file that are special, not how the
file is found, 'insert-file-contents-literally' provides the right
abstraction to get the file contents as-is and allow the user to treat
it specially.

Alternatively, if you insist, we could have auth-source-pass always add
the epa-file-handler in a let, because the only reasonable thing to do
with pass entries is decrypt them.  I recall that this option was not
favored.  I think it's still preferable to relying on epa-file being
enabled without ensuring so, though I'd still prefer if that dependency
wasn't present.

I do not think that checking for the presence of a file handler is
effective (because, firstly, it won't fix this issue in the general
case, and secondly, it is not useful), nor that checking for a specific
file handler being present is effective (because that'd be a roundabout
way to simply force the epa-file handler or not rely on it).

I'd also love to hear from the authors of auth-source-pass.

Have a lovely day.
-- 
Arsen Arsenović

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 381 bytes --]