Fix needed for communication with gpg-agent

Fix needed for communication with gpg-agent

[Richard Stallman]

Would someone please implement this code, needed for correct
redisplay after running pinentry?  Then please ack.

------- Start of forwarded message -------
From: Werner Koch <wk@gnupg.org>
To: rms@gnu.org
Subject: Redraw after calling gpg (was: [bug] PGG shows ?? when prompt for passphrase)
CC: ueno@unixuser.org
Organisation: g10 Code GmbH
Date: Wed, 14 Feb 2007 19:02:56 +0100
In-Reply-To: <E1HFfUN-0001ps-TY@fencepost.gnu.org> (Richard Stallman's message
	of "Fri\, 09 Feb 2007 18\:49\:11 -0500")
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=failed 
	version=3.0.4

On Sat, 10 Feb 2007 00:49, rms@gnu.org said:

> Can you write a self-contained proposal?  I will forward it to the
> other Emacs developers to ask someone to implement it or suggest
> changes.

gpg-agent is a tool to take care of passphrase caching, private key
operations and smart card access.  It works as a dameon started for
eahc user and calls a tool called Pinentry if it requires input from
the user (Passphrase or PIN).  This works very well on X11 as Pinentry
can create its own windows soley under its own control.

Without X, Pinentry falls back to a curses implementation (there is
also a curses only version of Pinentry).  This works pretty reliable
but poses one problem: The screen needs a redraw as there is no
portable way to save and restore the old screen content.  What can be
done is to redraw the screen every time after calling a crypto
operation which uses gpg or gpgsm.  This is of course not optimal as
in most cases the passphrase has been cached and an expensive redraw
is not required.

To solve this problem the latest SVN Pinentry (-r 155) and GnuPG (-r
4434) have a new feature: Pinentry will touch a file if the curses
version has been used right before returning control to gpg-agent and
eventually to the calling appliaction.  The file we touch is by
default the socket gpg-agent is listening fro requests by gpg or
gpgsm.

An application calling gpg and expecting thyat a pinentry might pop up
(i.e. signing or decrypting) should record the mtime of the socket
file and compare that after the call to gpg.  If it changed a redraw
should be done.  What a applaiction needs to do:

  1. Get the value of the envvar GPG_AGENT_INFO.  This envvar is set by
     gpg-agent and the central information on how to access gpg-agent.
     It works similar to the way ssh-agent works.  Example:
  
       $ echo $GPG_AGENT_INFO
       /tmp/gpg-0Hg6cf/S.gpg-agent:22391:1
  
  2. Parse that value and pass the first part to file-attributes.
     Record the mtime.
  
  3. Call gpg or gpgsm
  
  4. Do a file-attributes again on that file and compare the mtime
     against the recorded one.
  
  5. It there is a mismatch a redraw should be done.
  

Cronjobs invoking gpg (or ssh when --enable-ssh-support is used with
gpg-agent) might also lead to a pinentry pop up.  We can't do much
about this but I don't think that it is a real problem.  Such a
Pinentry will only pop up rarely and hitting C-l is much easier than
any other method.

if you want to test this system and are using X, you can add a line
like

  pinentry-program /usr/local/bin/pinentry-curses

into ~/.gnupg/gpg-agent.conf and give gpg-agent a HUP.


Salam-Shalom,

   Werner
------- End of forwarded message -------

Re: Fix needed for communication with gpg-agent

[Chong Yidong]

Richard Stallman <rms@gnu.org> writes:

> Would someone please implement this code, needed for correct
> redisplay after running pinentry?  Then please ack.
>
> ...
>
> Without X, Pinentry falls back to a curses implementation (there is
> also a curses only version of Pinentry).  This works pretty reliable
> but poses one problem: The screen needs a redraw as there is no
> portable way to save and restore the old screen content.  What can be
> done is to redraw the screen every time after calling a crypto
> operation which uses gpg or gpgsm.  This is of course not optimal as
> in most cases the passphrase has been cached and an expensive redraw
> is not required.

I don't see any reasonable way to do this.

Roughly speaking, when Emacs is run in a console (i.e. as a curses
program), it monopolizes terminal input and output.  I don't know of
any way to call the curses version of pinentry, have it temporarily
"take over" the terminal, and return control to Emacs once it's done.

I suggest filing this in TODO or PROBLEMS.  There is a kind of
workaround: if you first use gpg-agent outside of Emacs to cache your
passphrase, subsequent use of gpg-agent from in Emacs will work since
gpg-agent no longer needs to prompt for your passphrase.

Re: Fix needed for communication with gpg-agent

[Miles Bader]

Chong Yidong <cyd@stupidchicken.com> writes:
> I don't see any reasonable way to do this.
>
> Roughly speaking, when Emacs is run in a console (i.e. as a curses
> program), it monopolizes terminal input and output.  I don't know of
> any way to call the curses version of pinentry, have it temporarily
> "take over" the terminal, and return control to Emacs once it's done.

Yeah.

One possible way might be to have emacs start its own gpg-agent daemon,
specifying a custom pinentry program that uses emacs itself for
interaction (Emacs would set the GPG_AGENT_INFO environment variable to
communicate that to invocations of gpg).  That seems a wee-bit
heavyweight, since Emacs generally doesn't know whether you're going to
run gpg, but I suppose it might be a reasonable option for people that
use a long-running Emacs session.

[This is all post-release stuff though, obviously...]

-Miles

-- 
Fast, small, soon; pick any 2.

Re: Fix needed for communication with gpg-agent

[Chong Yidong]

Miles Bader <miles@gnu.org> writes:

> One possible way might be to have emacs start its own gpg-agent daemon,
> specifying a custom pinentry program that uses emacs itself for
> interaction (Emacs would set the GPG_AGENT_INFO environment variable to
> communicate that to invocations of gpg).  That seems a wee-bit
> heavyweight, since Emacs generally doesn't know whether you're going to
> run gpg, but I suppose it might be a reasonable option for people that
> use a long-running Emacs session.
>
> [This is all post-release stuff though, obviously...]

Another solution might be to cannibalize the ansi-term code, and have
the pinentry program run in an elisp terminal.  I looked into this,
and concluded that it is a big job.  It should be left till after the
release.

Re: Fix needed for communication with gpg-agent

[Chong Yidong]

> Another solution might be to cannibalize the ansi-term code, and have
> the pinentry program run in an elisp terminal.  I looked into this,
> and concluded that it is a big job.  It should be left till after the
> release.

FWIW, here is a patch that implements the idea I outlined above.  I
have tested it, and it works with the curses version of pinentry.

Admittedly, it is a rather naughty solution: GPG is called through an
elisp terminal (as implemented by `make-term' in term.el), with a
modified process filter whose purpose is to send GPG the string to be
encrypted when required.  The terminal is run in a recursive editing
level, so that the user can input to pinentry if necessary; the
process sentinel is modified to call `exit-recursive-edit' when GPG
exits, to return control to pgg-gpg-process-region.

*** emacs/lisp/pgg-gpg.el.~1.22.~	2007-02-18 11:07:52.000000000 -0500
--- emacs/lisp/pgg-gpg.el	2007-02-22 12:23:11.000000000 -0500
***************
*** 60,65 ****
--- 60,67 ----
  (defvar pgg-gpg-user-id nil
    "GnuPG ID of your default identity.")
  
+ (defvar pgg-gpg-term-input nil)
+ 
  (defun pgg-gpg-process-region (start end passphrase program args)
    (let* ((use-agent (and (null passphrase) (pgg-gpg-use-agent-p)))
  	 (output-file-name (pgg-make-temp-file "pgg-output"))
***************
*** 83,114 ****
        (erase-buffer))
      (unwind-protect
  	(progn
! 	  (set-default-file-modes 448)
! 	  (let ((coding-system-for-write 'binary))
! 	    (setq process
! 		  (apply #'start-process "*GnuPG*" errors-buffer
! 			 program args)))
! 	  (set-process-sentinel process #'ignore)
! 	  (when passphrase
! 	    (setq passphrase-with-newline (concat passphrase "\n"))
! 	    (if pgg-passphrase-coding-system
! 		(progn
! 		  (setq encoded-passphrase-with-new-line
! 			(encode-coding-string
! 			 passphrase-with-newline
! 			 (coding-system-change-eol-conversion
! 			  pgg-passphrase-coding-system 'unix)))
! 		  (pgg-clear-string passphrase-with-newline))
! 	      (setq encoded-passphrase-with-new-line passphrase-with-newline
! 		    passphrase-with-newline nil))
! 	    (process-send-string process encoded-passphrase-with-new-line))
! 	  (process-send-region process start end)
! 	  (process-send-eof process)
! 	  (while (eq 'run (process-status process))
! 	    (accept-process-output process 5))
! 	  (setq status (process-status process)
! 		exit-status (process-exit-status process))
! 	  (delete-process process)
  	  (with-current-buffer (get-buffer-create output-buffer)
  	    (buffer-disable-undo)
  	    (erase-buffer)
--- 85,130 ----
        (erase-buffer))
      (unwind-protect
  	(progn
! 	  (if (and (not window-system) use-agent)
! 	      (let ((term-term-name "vt100")
! 		    (buf (current-buffer)))
! 		(apply #'make-term "GPGTERM" "gpg" nil args)
! 		(switch-to-buffer "*GPGTERM*")
! 		(set (make-local-variable 'pgg-gpg-term-input)
! 		     (list buf start end))
! 		(setq process (get-buffer-process "*GPGTERM*"))
! 		(term-char-mode)
! 		(set-process-filter process 'pgg-gpg-filter)
! 		(set-process-sentinel process 'pgg-gpg-sentinel)
! 		(recursive-edit)
! 		(setq exit-status (process-exit-status process)))
! 	    (set-default-file-modes 448)
! 	    (let ((coding-system-for-write 'binary))
! 	      (setq process
! 		    (apply #'start-process "*GnuPG*" errors-buffer
! 			   program args)))
! 	    (set-process-sentinel process #'ignore)
! 	    (accept-process-output process nil nil 1)
! 	    (when passphrase
! 	      (setq passphrase-with-newline (concat passphrase "\n"))
! 	      (if pgg-passphrase-coding-system
! 		  (progn
! 		    (setq encoded-passphrase-with-new-line
! 			  (encode-coding-string
! 			   passphrase-with-newline
! 			   (coding-system-change-eol-conversion
! 			    pgg-passphrase-coding-system 'unix)))
! 		    (pgg-clear-string passphrase-with-newline))
! 		(setq encoded-passphrase-with-new-line passphrase-with-newline
! 		      passphrase-with-newline nil))
! 	      (process-send-string process encoded-passphrase-with-new-line))
! 	    (process-send-region process start end)
! 	    (process-send-eof process)
! 	    (while (eq 'run (process-status process))
! 	      (accept-process-output process 5))
! 	    (setq status (process-status process)
! 		  exit-status (process-exit-status process))
! 	    (delete-process process))
  	  (with-current-buffer (get-buffer-create output-buffer)
  	    (buffer-disable-undo)
  	    (erase-buffer)
***************
*** 132,137 ****
--- 148,174 ----
  	  (delete-file output-file-name))
        (set-default-file-modes orig-mode))))
  
+ (defun pgg-gpg-filter (proc str)
+   (if (and (string-match "writing to" str) pgg-gpg-term-input)
+       (let ((buf   (nth 0 pgg-gpg-term-input))
+ 	    (start (nth 1 pgg-gpg-term-input))
+ 	    (end   (nth 2 pgg-gpg-term-input)))
+ 	(erase-buffer)
+ 	(with-current-buffer buf
+ 	  (process-send-region proc start end)
+ 	  (process-send-eof proc)
+ 	  (process-send-eof proc)
+ 	  (setq pgg-gpg-filter-args nil)))
+     (term-emulate-terminal proc str)))
+ 
+ (defun pgg-gpg-sentinel (proc msg)
+   (let ((buffer (process-buffer proc)))
+     (when (memq (process-status proc) '(signal exit))
+       (delete-process proc)
+       (unless (null (buffer-name buffer))
+ 	(kill-buffer buffer))
+       (exit-recursive-edit))))
+ 
  (defun pgg-gpg-possibly-cache-passphrase (passphrase &optional key notruncate)
    (if (and passphrase
  	   pgg-cache-passphrase

Re: Fix needed for communication with gpg-agent

[Chong Yidong]

   FWIW, here is a patch that implements the idea I outlined above.  I
   have tested it, and it works with the curses version of pinentry.

   Admittedly, it is a rather naughty solution: GPG is called through an
   elisp terminal (as implemented by `make-term' in term.el), with a
   modified process filter whose purpose is to send GPG the string to be
   encrypted when required.  The terminal is run in a recursive editing
   level, so that the user can input to pinentry if necessary; the
   process sentinel is modified to call `exit-recursive-edit' when GPG
   exits, to return control to pgg-gpg-process-region.

Whoops, minor typo in that patch.  Please consider this one, which
works on all configurations I have tested.

*** emacs/lisp/pgg-gpg.el.~1.22.~	2007-02-18 11:07:52.000000000 -0500
--- emacs/lisp/pgg-gpg.el	2007-02-22 12:41:44.000000000 -0500
***************
*** 60,65 ****
--- 60,67 ----
  (defvar pgg-gpg-user-id nil
    "GnuPG ID of your default identity.")
  
+ (defvar pgg-gpg-term-input nil)
+ 
  (defun pgg-gpg-process-region (start end passphrase program args)
    (let* ((use-agent (and (null passphrase) (pgg-gpg-use-agent-p)))
  	 (output-file-name (pgg-make-temp-file "pgg-output"))
***************
*** 84,114 ****
      (unwind-protect
  	(progn
  	  (set-default-file-modes 448)
! 	  (let ((coding-system-for-write 'binary))
! 	    (setq process
! 		  (apply #'start-process "*GnuPG*" errors-buffer
! 			 program args)))
! 	  (set-process-sentinel process #'ignore)
! 	  (when passphrase
! 	    (setq passphrase-with-newline (concat passphrase "\n"))
! 	    (if pgg-passphrase-coding-system
! 		(progn
! 		  (setq encoded-passphrase-with-new-line
! 			(encode-coding-string
! 			 passphrase-with-newline
! 			 (coding-system-change-eol-conversion
! 			  pgg-passphrase-coding-system 'unix)))
! 		  (pgg-clear-string passphrase-with-newline))
! 	      (setq encoded-passphrase-with-new-line passphrase-with-newline
! 		    passphrase-with-newline nil))
! 	    (process-send-string process encoded-passphrase-with-new-line))
! 	  (process-send-region process start end)
! 	  (process-send-eof process)
! 	  (while (eq 'run (process-status process))
! 	    (accept-process-output process 5))
! 	  (setq status (process-status process)
! 		exit-status (process-exit-status process))
! 	  (delete-process process)
  	  (with-current-buffer (get-buffer-create output-buffer)
  	    (buffer-disable-undo)
  	    (erase-buffer)
--- 86,129 ----
      (unwind-protect
  	(progn
  	  (set-default-file-modes 448)
! 	  (if (and (not window-system) use-agent)
! 	      (let ((term-term-name "vt100")
! 		    (buf (current-buffer)))
! 		(apply #'make-term "GPGTERM" "gpg" nil args)
! 		(switch-to-buffer "*GPGTERM*")
! 		(set (make-local-variable 'pgg-gpg-term-input)
! 		     (list buf start end))
! 		(setq process (get-buffer-process "*GPGTERM*"))
! 		(term-char-mode)
! 		(set-process-filter process 'pgg-gpg-filter)
! 		(set-process-sentinel process 'pgg-gpg-sentinel)
! 		(recursive-edit)
! 		(setq exit-status (process-exit-status process)))
! 	    (let ((coding-system-for-write 'binary))
! 	      (setq process
! 		    (apply #'start-process "*GnuPG*" errors-buffer
! 			   program args)))
! 	    (set-process-sentinel process #'ignore)
! 	    (when passphrase
! 	      (setq passphrase-with-newline (concat passphrase "\n"))
! 	      (if pgg-passphrase-coding-system
! 		  (progn
! 		    (setq encoded-passphrase-with-new-line
! 			  (encode-coding-string
! 			   passphrase-with-newline
! 			   (coding-system-change-eol-conversion
! 			    pgg-passphrase-coding-system 'unix)))
! 		    (pgg-clear-string passphrase-with-newline))
! 		(setq encoded-passphrase-with-new-line passphrase-with-newline
! 		      passphrase-with-newline nil))
! 	      (process-send-string process encoded-passphrase-with-new-line))
! 	    (process-send-region process start end)
! 	    (process-send-eof process)
! 	    (while (eq 'run (process-status process))
! 	      (accept-process-output process 5))
! 	    (setq status (process-status process)
! 		  exit-status (process-exit-status process))
! 	    (delete-process process))
  	  (with-current-buffer (get-buffer-create output-buffer)
  	    (buffer-disable-undo)
  	    (erase-buffer)
***************
*** 132,137 ****
--- 147,173 ----
  	  (delete-file output-file-name))
        (set-default-file-modes orig-mode))))
  
+ (defun pgg-gpg-filter (proc str)
+   (if (and (string-match "writing to" str) pgg-gpg-term-input)
+       (let ((buf   (nth 0 pgg-gpg-term-input))
+ 	    (start (nth 1 pgg-gpg-term-input))
+ 	    (end   (nth 2 pgg-gpg-term-input)))
+ 	(erase-buffer)
+ 	(with-current-buffer buf
+ 	  (process-send-region proc start end)
+ 	  (process-send-eof proc)
+ 	  (process-send-eof proc)
+ 	  (setq pgg-gpg-filter-args nil)))
+     (term-emulate-terminal proc str)))
+ 
+ (defun pgg-gpg-sentinel (proc msg)
+   (let ((buffer (process-buffer proc)))
+     (when (memq (process-status proc) '(signal exit))
+       (delete-process proc)
+       (unless (null (buffer-name buffer))
+ 	(kill-buffer buffer))
+       (exit-recursive-edit))))
+ 
  (defun pgg-gpg-possibly-cache-passphrase (passphrase &optional key notruncate)
    (if (and passphrase
  	   pgg-cache-passphrase

Re: Fix needed for communication with gpg-agent

[Werner Koch]

On Thu, 22 Feb 2007 18:31, cyd@stupidchicken.com said:

> Admittedly, it is a rather naughty solution: GPG is called through an
> elisp terminal (as implemented by `make-term' in term.el), with a
> modified process filter whose purpose is to send GPG the string to be

The design goal of pinnetry is to keep it small and thus more secure
than a huge application.  The GUI versions take additional care to
assemble the passphrase only in locked-from-swapping memory.  With all
that code running in emacs, I doubt that it makes sense at all to use
the curses versions of Pinentry.  Here is another idea: 

Is it possible to enhance server-start/emacsclient so that it does not
edit a file but asks for string and returns that one?  Pinentry could
then use this feature for user interaction.


Salam-Shalom,

   Werner

Re: Fix needed for communication with gpg-agent

[Chong Yidong]

Werner Koch <wk@gnupg.org> writes:

>> Admittedly, it is a rather naughty solution: GPG is called through an
>> elisp terminal (as implemented by `make-term' in term.el), with a
>> modified process filter whose purpose is to send GPG the string to be
>
> With all that code running in emacs, I doubt that it makes sense at
> all to use the curses versions of Pinentry.

In that case, I think Emacs should disable use of gpg-agent when
running in a console, except when gpg-agent already has a passphrase
cached.  Then all we need is some method for Emacs to determine if the
necessary passphrase is cached.  If gpg-agent does not have the
passphrase, Emacs will then prompting for the passphrase and send it
to GPG, without caching it in Elisp (i.e. subsequent calls to GPG will
require entering the passphrase again).

> Is it possible to enhance server-start/emacsclient so that it does not
> edit a file but asks for string and returns that one?  Pinentry could
> then use this feature for user interaction.

I'm not sure how this suggestion could work.

Re: Fix needed for communication with gpg-agent

[Sascha Wilde]

Chong Yidong <cyd@stupidchicken.com> wrote:

> Werner Koch <wk@gnupg.org> writes:
>
>>> Admittedly, it is a rather naughty solution: GPG is called through an
>>> elisp terminal (as implemented by `make-term' in term.el), with a
>>> modified process filter whose purpose is to send GPG the string to be
>>
>> With all that code running in emacs, I doubt that it makes sense at
>> all to use the curses versions of Pinentry.
>
> In that case, I think Emacs should disable use of gpg-agent when
> running in a console, except when gpg-agent already has a passphrase
> cached.  Then all we need is some method for Emacs to determine if the
> necessary passphrase is cached.

Even if it is possible to determine this (I don't know right now),
there is an additional problem:  the key caching of the gpg-agent
times out after an configurable interval, so after that time emacs
would suddenly stop using the agent -- this sounds like undesirable to
me... 

> If gpg-agent does not have the
> passphrase, Emacs will then prompting for the passphrase and send it
> to GPG, without caching it in Elisp (i.e. subsequent calls to GPG will
> require entering the passphrase again).

I think this suggestion is based on a misunderstanding -- the security
problems in the current implementation (when not using gpg-agent) has
nothing to do with caching, it comes form the fact, that emacs writes
the the passphrase to an temporary file (which is then feed to gpg).

>> Is it possible to enhance server-start/emacsclient so that it does not
>> edit a file but asks for string and returns that one?  Pinentry could
>> then use this feature for user interaction.
>
> I'm not sure how this suggestion could work.

I haven't fully understood this idea, too.  In general I doubt, that
it is an good idea to make gpg-agent depend on an running emacs for
passphrase input -- even if many emacs users are using emacs as there
primary working environment and therefor have it running all the
time -- not everyone does...

cheers
sascha
-- 
Sascha Wilde
Life's too short to read boring signatures

Re: Fix needed for communication with gpg-agent

[Chong Yidong]

Sascha Wilde <wilde@sha-bang.de> writes:

> I think this suggestion is based on a misunderstanding -- the security
> problems in the current implementation (when not using gpg-agent) has
> nothing to do with caching, it comes form the fact, that emacs writes
> the the passphrase to an temporary file (which is then feed to gpg).

Maybe I'm confused, but I think this is not the problem.

>From what I recall, the issue was that an older version of pgg used
the function call-process-region.  This was a genuine security hole,
since call-process-region uses a tempfile to communicate with the
process.

The current version of pgg in Emacs CVS uses start-process to create
an asynchronous gpg process, and communicates with it using
process-send-string.  On systems that support ptys, Emacs communicates
with asynchronous processes through ptys (see create_process in
process.c:1815), not tempfiles.

Unless there is some security risk in the way we use ptys that I'm not
aware of, I don't think there is a problem in the way we communicate
with gpg.

Re: Fix needed for communication with gpg-agent

[Sascha Wilde]

Chong Yidong <cyd@stupidchicken.com> wrote:
> Sascha Wilde <wilde@sha-bang.de> writes:
>
>> I think this suggestion is based on a misunderstanding -- the security
>> problems in the current implementation (when not using gpg-agent) has
>> nothing to do with caching, it comes form the fact, that emacs writes
>> the the passphrase to an temporary file (which is then feed to gpg).
>
> Maybe I'm confused, but I think this is not the problem.

I just had a look at the source, you are right, this problem was
fixed -- it seems it is me who was confused...

sascha
-- 
Sascha Wilde
Well, *my* brain likes to think it's vastly more powerful than any
finite Turing machine but it hasn't proven that to me...
  -- Christopher Koppler in comp.lang.lisp

Re: Fix needed for communication with gpg-agent

[Richard Stallman]

    I just had a look at the source, you are right, this problem was
    fixed -- it seems it is me who was confused...

That is good news.

Does this mean we have no particular need to recommend gpg-agent?  We
could recommend gpg-agent, just for its convenience, for those who use
X or use screen, but not say much about it (just an xref).
For tty users, they can go along doing what they used to do.

Do you agree with that solution?  Yidong, could you adapt your manual
patch for that?

Re: Fix needed for communication with gpg-agent

[Richard Stallman]

    I think this suggestion is based on a misunderstanding -- the security
    problems in the current implementation (when not using gpg-agent) has
    nothing to do with caching, it comes form the fact, that emacs writes
    the the passphrase to an temporary file (which is then feed to gpg).

Thanks for filling in that important detail.
(This is why I said we needed to involve you in the discussion
before deciding what to do.)

If all we have to do is avoid passing it thru a temporary file,
there must be lots of other ways to avoid that.  Could it be passed
through a pipe or a socket?

Someone says it already is:

    The current version of pgg in Emacs CVS uses start-process to create
    an asynchronous gpg process, and communicates with it using
    process-send-string.  On systems that support ptys, Emacs communicates
    with asynchronous processes through ptys (see create_process in
    process.c:1815), not tempfiles.

    Unless there is some security risk in the way we use ptys that I'm not
    aware of, I don't think there is a problem in the way we communicate
    with gpg.

Does this mean the problem is already fixed?

Re: Fix needed for communication with gpg-agent

[Sascha Wilde]

Richard Stallman <rms@gnu.org> wrote:

>     I think this suggestion is based on a misunderstanding -- the security
>     problems in the current implementation (when not using gpg-agent) has
>     nothing to do with caching, it comes form the fact, that emacs writes
>     the the passphrase to an temporary file (which is then feed to gpg).
[...]
> If all we have to do is avoid passing it thru a temporary file,
> there must be lots of other ways to avoid that.  Could it be passed
> through a pipe or a socket?
>
> Someone says it already is:
[...]
> Does this mean the problem is already fixed?

Yes.  This problem was already solved (as said in another mail, I
forgot it was).

But there are still some more subtle security problems left, which
IIRC were discussed in the original thread, too:  If emacs caches the
passphrase there is no way to protect the passphrase from being
written to swap, when the system decides to swap out parts of emacs.

pgp-agent and pinentry on the other hand are trying hard to prevent
the passphrase from getting swaped out or written to hd by any other
means.

So in conclusion: the main security problem was solved, but it is
still preferable to use gpg-agent.

sascha
-- 
Sascha Wilde : VI is to EMACS as masturbation is to making love:
             : effective and always available but probably not your
             : first choice...

Re: Fix needed for communication with gpg-agent

[Richard Stallman]

    But there are still some more subtle security problems left, which
    IIRC were discussed in the original thread, too:  If emacs caches the
    passphrase there is no way to protect the passphrase from being
    written to swap, when the system decides to swap out parts of emacs.

If we turn off caching of the passphrase in Emacs, does this problem
go away?

Re: Fix needed for communication with gpg-agent

[Chong Yidong]

Richard Stallman <rms@gnu.org> writes:

>     But there are still some more subtle security problems left, which
>     IIRC were discussed in the original thread, too:  If emacs caches the
>     passphrase there is no way to protect the passphrase from being
>     written to swap, when the system decides to swap out parts of emacs.
>
> If we turn off caching of the passphrase in Emacs, does this problem
> go away?

Not really.

The risk here occurs when you have a password stored in cleartext in
memory (for example, it is stored in the Lisp string just before we
are about to send it to gpg).  If memory get written to the swap file,
it can be read by root.

This is arguably a security hole because it makes it too easy for root
to find people's passwords (granted, root can easily steal passwords
anyway, but it arguably shouldn't be *this* easy.)

But if you are concerned enough about root stealing your passwords in
this way, you wouldn't mind running in X or performing the other
workarounds that exist for that problem.

Re: Fix needed for communication with gpg-agent

[Andreas Schwab]

Chong Yidong <cyd@stupidchicken.com> writes:

> This is arguably a security hole because it makes it too easy for root
> to find people's passwords (granted, root can easily steal passwords
> anyway, but it arguably shouldn't be *this* easy.)

If you are root you can read every process's memory, no matter where it is
stored.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Fix needed for communication with gpg-agent

[David Kastrup]

Andreas Schwab <schwab@suse.de> writes:

> Chong Yidong <cyd@stupidchicken.com> writes:
>
>> This is arguably a security hole because it makes it too easy for
>> root to find people's passwords (granted, root can easily steal
>> passwords anyway, but it arguably shouldn't be *this* easy.)
>
> If you are root you can read every process's memory, no matter where
> it is stored.

The danger is not as much root on an active system.  The danger is
that the passwords will be stored on disk where they will stay
semi-permanently, even if the disk gets sold, imaged, or stolen.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: Fix needed for communication with gpg-agent

[Richard Stallman]

    > If we turn off caching of the passphrase in Emacs, does this problem
    > go away?

    Not really.

    The risk here occurs when you have a password stored in cleartext in
    memory (for example, it is stored in the Lisp string just before we
    are about to send it to gpg).  If memory get written to the swap file,
    it can be read by root.

If the passphrase is cached in Emacs, I presume someone walking up to
your terminal could type commands at Emacs and find it.  Is that
right?

If so, does turning off caching prevent THAT problem?

If it does, should we document this?  Or turn off caching by default?
Or what?

Re: Fix needed for communication with gpg-agent

[Werner Koch]

On Mon, 26 Feb 2007 04:27, rms@gnu.org said:

> If so, does turning off caching prevent THAT problem?

Not really.  If someone is able to enter commands in Emacs, he will
also be able to read the memory of all user processes and thus find
cached passphrases.  Or he might install a key logger or a trojan or
read confidential documents directly.

The passphrase is in fact a last line of defense.  If someone can copy
the secret key he will often be able to figure out the passphrase
anyway (social engineering or dictionary attacks).

> If it does, should we document this?  Or turn off caching by default?
> Or what?

Explain that one should not leave a terminal alone and that zeroing
out swap partitions before selling a box is suggested.  Without
caching of passphrases, using crypto is to troublesome.



Shalom-Salam,

   Werner

Re: Fix needed for communication with gpg-agent

[Richard Stallman]

    > If so, does turning off caching prevent THAT problem?

    Not really.  If someone is able to enter commands in Emacs, he will
    also be able to read the memory of all user processes and thus find
    cached passphrases.

I do not understand this response.  How can anyone find cached
passphrases if caching is turned off?

Re: Fix needed for communication with gpg-agent

[Werner Koch]

On Tue, 27 Feb 2007 08:38, rms@gnu.org said:

> I do not understand this response.  How can anyone find cached
> passphrases if caching is turned off?

It seems this thread is about two different topics:

 - Using Pinentry
 - Using Emacs to send (and cache) a passphrase to gpg

I have not seen the beginning of this thread, so I don't know much
about the latter case.

I am always thinking of using the gpg-agent - eventually gpg-agent
will not only cache the passphrase but handle all secret keys.  Thus
this latter case will sooner or later be irrelevant as tehre won't be
any way for Emacs to tell gpg (actually gpg2 or gpgsm) a passphrase.


Shalom-Salam,

   Werner

Re: Fix needed for communication with gpg-agent

[Richard Stallman]

    It seems this thread is about two different topics:

     - Using Pinentry
     - Using Emacs to send (and cache) a passphrase to gpg

I see those as two parts of a slightly larger topic:

  What should we do _now_ in Emacs as regards use of gpg keys.

We want something that is secure enough and not too inconvenient.

    I am always thinking of using the gpg-agent - eventually gpg-agent
    will not only cache the passphrase but handle all secret keys.  Thus
    this latter case will sooner or later be irrelevant as tehre won't be
    any way for Emacs to tell gpg (actually gpg2 or gpgsm) a passphrase.

Please do NOT do this until we have implemented in Emacs a way to do
this, that works conveniently in all modes of using Emacs.

Re: Fix needed for communication with gpg-agent

[Werner Koch]

On Thu, 22 Feb 2007 19:40, cyd@stupidchicken.com said:

> In that case, I think Emacs should disable use of gpg-agent when
> running in a console, except when gpg-agent already has a passphrase
> cached.  Then all we need is some method for Emacs to determine if the

That is only a short term solution.  The goal of gpg-agent is to take
care of _all_ secret key[1] management.  gpg and gpgsm will delegate
all secret key operations to gpg-agent and nevermwork with a secret
key directly.  This is already the case for gpgsm, the S/MIME variant
of gpg.  As time permits this will be implemented in gpg too.  Using
gpg-agent as a passphrase cache for gpg is only a temporrary solution.

> > Is it possible to enhance server-start/emacsclient so that it does not
> > edit a file but asks for string and returns that one?  Pinentry could
> > then use this feature for user interaction.
>
> I'm not sure how this suggestion could work.

Recall that pinentry is called on the sole discretion of gpg-agent.
Only gpg-agent knows whether Pinentry needs to be called.

My suggestion is this:

 +-------+    +-----------+    +-----------+       +----------+
 | emacs | -> | gpg/gpgsm | -> | gpg-agent | ----> | pinentry |
 +-------+    +-----------+    +-----------+ (may) +----------+
     ^                                                  |
     |                                                  |
     +--------------------------------------------------+
           (some mechanism to loop back to emacs)

Pinentry uses this:

  if DISPLAY set
     Use GUI mode; no problem
  else if SOME_EMACS_ENVVAR set
     Loop back to emacs
  else 
     Use Curses

to decide wether that emacs loop back method is to be used.  It will
then ask emacs: Please create a form or a minibuffer with this or that
description and return the user input to me.

I don't know the emacsclient protocol and whether it can easiliy be
enhanced for that case.  To me this sounds like a viable solution
which does not require emacs to stop other processing if no pinentry
is required.


Shalom-Salam,

   Werner

Re: Fix needed for communication with gpg-agent

[Sascha Wilde]

Werner Koch <wk@gnupg.org> wrote:
> On Thu, 22 Feb 2007 19:40, cyd@stupidchicken.com said:
>> > Is it possible to enhance server-start/emacsclient so that it does not
>> > edit a file but asks for string and returns that one?  Pinentry could
>> > then use this feature for user interaction.
>>
>> I'm not sure how this suggestion could work.
>
> Recall that pinentry is called on the sole discretion of gpg-agent.
> Only gpg-agent knows whether Pinentry needs to be called.
>
> My suggestion is this:
>
>  +-------+    +-----------+    +-----------+       +----------+
>  | emacs | -> | gpg/gpgsm | -> | gpg-agent | ----> | pinentry |
>  +-------+    +-----------+    +-----------+ (may) +----------+
>      ^                                                  |
>      |                                                  |
>      +--------------------------------------------------+
>            (some mechanism to loop back to emacs)
>
> Pinentry uses this:
>
>   if DISPLAY set
>      Use GUI mode; no problem
>   else if SOME_EMACS_ENVVAR set
>      Loop back to emacs
>   else 
>      Use Curses

This looks very good to me, BUT: this would need a new (extended)
version of pinentry, which will probably not be available in most
major distributions for the near future, so I don't think this would
be a feasible solution for the 22 release.

> to decide wether that emacs loop back method is to be used.  It will
> then ask emacs: Please create a form or a minibuffer with this or that
> description and return the user input to me.
>
> I don't know the emacsclient protocol and whether it can easiliy be
> enhanced for that case.

I don't know that either, but we could always create a dedicated
socket for such an service, so yes, I think this would be the right
thing to do -- after the release.

cheers
sascha
-- 
Sascha Wilde
Nota bene: wenn Word für Längeres geeignet wäre, würde es schließlich
nicht Word, sondern Sentence, Page oder Article heißen
 	-- Matthias Mühlich in dctt

Re: Fix needed for communication with gpg-agent

[Chong Yidong]

Werner Koch <wk@gnupg.org> writes:

> [Pinentry] will then ask emacs: Please create a form or a minibuffer
> with this or that description and return the user input to me.

I agree with Sascha that this is not feasible for the Emacs 22
release, which is supposed to be soon.  In your opinion, of the
remaining choices, which is preferable?

1. Running GPG in an Elisp terminal, as in the patch I sent.
2. Documenting the problem in the PGG manual, and recommending using
   PGG and gpg-agent in X (for Emacs 22 anyway).

Unless you have some other solution...?

Re: Fix needed for communication with gpg-agent

[Werner Koch]

On Fri, 23 Feb 2007 17:23, cyd@stupidchicken.com said:

> 2. Documenting the problem in the PGG manual, and recommending using
>    PGG and gpg-agent in X (for Emacs 22 anyway).

This and for non-X one could suggest to use screen and start gpg-agent
using the option

 --keep-tty

  Ignore requests to change change the current @code{tty} respective the X
  window system's @code{DISPLAY} variable.  This is useful to lock the
  pinentry to pop up at the @code{tty} or display you started the agent.

on a different tty than Emacs.  I know that a least one hacker is
using it this way.


Salam-Shalom,

   Werner

Re: Fix needed for communication with gpg-agent

[Chong Yidong]

Werner Koch <wk@gnupg.org> writes:

> On Fri, 23 Feb 2007 17:23, cyd@stupidchicken.com said:
>
>> 2. Documenting the problem in the PGG manual, and recommending using
>>    PGG and gpg-agent in X (for Emacs 22 anyway).
>
> This and for non-X one could suggest to use screen and start gpg-agent
> using the option
>
>  --keep-tty

In that case, I suggest the following patch to the PGG documentation.
What do people think?

*** emacs/man/pgg.texi.~1.16.~	2007-01-17 22:03:18.000000000 -0500
--- emacs/man/pgg.texi	2007-02-23 14:35:46.000000000 -0500
***************
*** 229,236 ****
  @node Caching passphrase
  @section Caching passphrase
  
! PGG provides a simple passphrase caching mechanism.  If you want to
! arrange the interaction, set the variable @code{pgg-read-passphrase}.
  
  @defvar pgg-cache-passphrase
  If non-@code{nil}, store passphrases.  The default value of this
--- 229,279 ----
  @node Caching passphrase
  @section Caching passphrase
  
! When using GnuPG (gpg) as the PGP scheme, we recommend using a program
! called @code{gpg-agent} for caching passphrases@footnote{Actually
! @code{gpg-agent} does not cache passphrases but private keys.  On the
! other hand, from a user's point of view, this technical difference
! isn't visible.}.
! 
! @defvar pgg-gpg-use-agent
! If non-@code{nil}, attempt to use @code{gpg-agent} whenever possible.
! The default is @code{t}.  If @code{gpg-agent} is not running, or GnuPG
! is not the current PGP scheme, PGG's own passphrase-caching mechanism
! is used (see below).
! @end defvar
! 
! To use @code{gpg-agent} with PGG, you must first ensure that
! @code{gpg-agent} is running.  For example, if you are running in the X
! Window System, you can do this by putting the following line in your
! @file{.xsession} file:
! 
! @smallexample
! eval "$(gpg-agent --daemon)"
! @end smallexample
! 
! For more details on invoking @code{gpg-agent}, @xref{Invoking
! GPG-AGENT,,,gnupg,Using the GNU Privacy Guard}.
! 
! Whenever you perform a PGG operation that requires a GnuPG passphrase,
! GnuPG will contact @code{gpg-agent}, which prompts you for the
! passphrase.  Furthermore, @code{gpg-agent} ``caches'' the result, so
! that subsequent uses will not require you to enter the passphrase
! again.  (This cache usually expires after a certain time has passed;
! you can change this using the @code{--default-cache-ttl} option when
! invoking @code{gpg-agent}.)
! 
! If you are running in a X Window System environment, @code{gpg-agent}
! prompts you for a passphrase by opening a graphical window.  However,
! if you are running Emacs on a text terminal, @code{gpg-agent} has
! trouble receiving input from the terminal, since it is being sent to
! Emacs.  One workaround for this problem is to run @code{gpg-agent} on
! a different terminal from Emacs, with the @code{--keep-tty} option;
! this tells @code{gpg-agent} use its own terminal to prompt for
! passphrases.
! 
! When @code{gpg-agent} is not being used, PGG provides its own
! passphrase caching mechanism, which is controlled by the variable
! @code{pgg-read-passphrase}:
  
  @defvar pgg-cache-passphrase
  If non-@code{nil}, store passphrases.  The default value of this
***************
*** 243,256 ****
  Elapsed time for expiration in seconds.
  @end defvar
  
- @defvar pgg-gpg-use-agent
- When using GnuPG (gpg) as PGP scheme you can use @code{gpg-agent} for
- caching@footnote{Actually @code{gpg-agent} does not cache passphrases
- but private keys.  On the other hand, from a users point of view this
- technical difference isn't visible.}.  It defaults to @code{t}.
- Setting this to @code{nil} is not recommended.
- @end defvar
- 
  @node Default user identity
  @section Default user identity
  
--- 286,291 ----

Re: Fix needed for communication with gpg-agent

[Sascha Wilde]

Chong Yidong <cyd@stupidchicken.com> wrote:

> Werner Koch <wk@gnupg.org> writes:
>
>> On Fri, 23 Feb 2007 17:23, cyd@stupidchicken.com said:
>>
>>> 2. Documenting the problem in the PGG manual, and recommending using
>>>    PGG and gpg-agent in X (for Emacs 22 anyway).

Given the current situation (including the fact, that all of us want
to get Emacs 22 finally released), I agree.

>> This and for non-X one could suggest to use screen and start gpg-agent
>> using the option
>>
>>  --keep-tty
>
> In that case, I suggest the following patch to the PGG documentation.
> What do people think?

I like it.  IMO this is the way to do, Richard, what do you think?

cheers
sascha
-- 
Sascha Wilde  :  "I heard that if you play the Windows CD backward, you
              :  get a satanic message. But that's nothing compared to
              :  when you play it forward: It installs Windows...." 
              :  -- G. R. Gaudreau

Re: Fix needed for communication with gpg-agent

[Richard Stallman]

    > This and for non-X one could suggest to use screen and start gpg-agent
    > using the option
    >
    >  --keep-tty

    In that case, I suggest the following patch to the PGG documentation.
    What do people think?

That isn't an adequate solution, so please do not install this change.

Re: Fix needed for communication with gpg-agent

[Richard Stallman]

    This and for non-X one could suggest to use screen and start gpg-agent
    using the option

     --keep-tty

      Ignore requests to change change the current @code{tty} respective the X
      window system's @code{DISPLAY} variable.  This is useful to lock the
      pinentry to pop up at the @code{tty} or display you started the agent.

    on a different tty than Emacs.  I know that a least one hacker is
    using it this way.

That is not an acceptable solution
because it calls on the user to do something very unusual
merely in order to be able to use gpg.

Re: Fix needed for communication with gpg-agent

[Chong Yidong]

Richard Stallman <rms@gnu.org> writes:

>     This and for non-X one could suggest to use screen and start gpg-agent
>     using the option
>
>      --keep-tty
>
>       Ignore requests to change change the current @code{tty} respective the X
>       window system's @code{DISPLAY} variable.  This is useful to lock the
>       pinentry to pop up at the @code{tty} or display you started the agent.
>
>     on a different tty than Emacs.  I know that a least one hacker is
>     using it this way.
>
> That is not an acceptable solution because it calls on the user to
> do something very unusual merely in order to be able to use gpg.

This unusual approach is only if the user wants to use gpg-agent on
the console.

As I've pointed out, the way PGG communicates with gpg is quite
secure, since it is done using a pty instead of a tempfile.  The two
drawbacks to entering the passphrase through PGG is that (a) Emacs is
more complicated that pinentry, so it may be more probable that Emacs
contains an exploitable bug than pinentry, and (b) your cached
passphrase might be recoverable by someone else who uses your console
when you step away.

Note that in case (b), you're screwed anyway: once a malicious
attacker has access to your console, he can install such nastier
attacks (keyboard sniffers, etc) that Emacs/PGG may be the least of
your worries.

So we're left with reason (a).  I think it is sufficient to recommend
using gpg-agent under X; if the user does not want to use X, we can
say to either (i) use PGG's Elisp passphrase caching, and explain how
it's not necessarily the best thing to do because Emacs code may
harbor bugs (though we're not currently aware of any), or (ii) use the
above unusual setup if he wants to use gpg-agent.

Re: Fix needed for communication with gpg-agent

[Andreas Schwab]

Werner Koch <wk@gnupg.org> writes:

> Is it possible to enhance server-start/emacsclient so that it does not
> edit a file but asks for string and returns that one?  Pinentry could
> then use this feature for user interaction.

emacsclient --eval can execute arbitrary lisp code.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Fix needed for communication with gpg-agent

[Kim F. Storm]

Andreas Schwab <schwab@suse.de> writes:

> Werner Koch <wk@gnupg.org> writes:
>
>> Is it possible to enhance server-start/emacsclient so that it does not
>> edit a file but asks for string and returns that one?  Pinentry could
>> then use this feature for user interaction.
>
> emacsclient --eval can execute arbitrary lisp code.

But we must then assure that Emacs can execute that lisp code.
It may need special treatment since Emacs is currently waiting for
gpg to finish (so it's "sleeping" inside accept-process-output).


-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: Fix needed for communication with gpg-agent

[Stefan Monnier]

> Is it possible to enhance server-start/emacsclient so that it does not
> edit a file but asks for string and returns that one?  Pinentry could
> then use this feature for user interaction.

  emacsclient --eval '(read-passwd "gpg passwd: ")'

This looks like a very clean solution.  Except that you have to make sure
that Emacs is running a server, and if the user is running several Emacsen at
the same time you may not be able to get to that server.


        Stefan


PS: Also in case gpg is run on a different machine than Emacs, you may bump
into difficulties, although maybe this is sufficiently unlikely/impossible.

Re: Fix needed for communication with gpg-agent

[Miles Bader]

Stefan Monnier <monnier@iro.umontreal.ca> writes:
>   emacsclient --eval '(read-passwd "gpg passwd: ")'
>
> This looks like a very clean solution.  Except that you have to make sure
> that Emacs is running a server, and if the user is running several Emacsen at
> the same time you may not be able to get to that server.

Given Werner's proposed mechanism, pinentry is contacting emacs because
emacs asked it to (by setting the special environment variable) -- so
Emacs should be prepared to be contacted, and can communicate any needed
info in the environment variable.

-Miles

-- 
In New York, most people don't have cars, so if you want to kill a person, you
have to take the subway to their house.  And sometimes on the way, the train
is delayed and you get impatient, so you have to kill someone on the subway.
  [George Carlin]

Re: Fix needed for communication with gpg-agent

[Richard Stallman]

The motive for using gpg-agent and pinentry is to avoid ever having
the passphrase inside Emacs.  Wouldn't communicating with pinentry as
an Emacs subprocess defeat the purpose?

Re: Fix needed for communication with gpg-agent

[Richard Stallman]

    One possible way might be to have emacs start its own gpg-agent daemon,
    specifying a custom pinentry program that uses emacs itself for
    interaction (Emacs would set the GPG_AGENT_INFO environment variable to
    communicate that to invocations of gpg).  That seems a wee-bit
    heavyweight, since Emacs generally doesn't know whether you're going to
    run gpg, but I suppose it might be a reasonable option for people that
    use a long-running Emacs session.

Emacs could start this demon only the first time you try to use gpg.
Then it would not be wasteful.

We need to solve this problem one way or another now, because we
decided to fix a certain security hole by telling users to use
gpg-agent.  We don't need the most elegant possible fix, but we
need something reasonable to use.

Re: Fix needed for communication with gpg-agent

[Chong Yidong]

Richard Stallman <rms@gnu.org> writes:

> We need to solve this problem one way or another now, because we
> decided to fix a certain security hole by telling users to use
> gpg-agent.  We don't need the most elegant possible fix, but we
> need something reasonable to use.

Has anyone ever said that not using gpg-agent causes a security hole
(except for you)?

The strongest statement I've ever seen is that gpg-agent is highly
recommended, since it provides the most secure way of inputting
passphrases.  Basically, the worry is that someone could somehow
change the Elisp code in your Emacs session so that it records your
passphrase as you are entering it.  This is a non-zero but minuscule
risk.

In other words, if you want to be as secure as possible, use X.

Note that if someone is in a position to corrupt your Emacs session,
it is only a little more trouble to create and redirect you to a fake
version of gpg-agent that will intercept your passphrase anyway.  So
you are screwed even if you use gpg-agent in X.

As Ken Thompson once noted, all security risks are relative.

Re: Fix needed for communication with gpg-agent

[Richard Stallman]

    > We need to solve this problem one way or another now, because we
    > decided to fix a certain security hole by telling users to use
    > gpg-agent.  We don't need the most elegant possible fix, but we
    > need something reasonable to use.

    Has anyone ever said that not using gpg-agent causes a security hole
    (except for you)?

What a silly question!  I am not an expert on security, so such a
concern idea would NEVER originate from me.  Thus problem was
described in this list by others, a few months ago.

      Basically, the worry is that someone could somehow
    change the Elisp code in your Emacs session so that it records your
    passphrase as you are entering it.  This is a non-zero but minuscule
    risk.

I think he could also walk up to your terminal after you have entered
the passphrase, and get it out of data remaining in Emacs.

In the discussion when this was raised, people seemed to agree
it was a problem we should fix.  And the only fix was to avoid
storing passphrases in Emacs.

Re: Fix needed for communication with gpg-agent

[Chong Yidong]

Richard Stallman <rms@gnu.org> writes:

> I think he could also walk up to your terminal after you have entered
> the passphrase, and get it out of data remaining in Emacs.
>
> In the discussion when this was raised, people seemed to agree
> it was a problem we should fix.  And the only fix was to avoid
> storing passphrases in Emacs.

There is more than one way to fix that.

Since no one seems to have an idea of how to handle the
console/pinentry case, let's just disable password caching on text
terminals, and disabling use of gpg-agent on text terminals, by
default.  We can suggest a workaround in the PGG manual, and tell
people to enter a passphrase into pinentry before starting Emacs, if
they want to use PGG with gpg-agent on a text terminal.

Re: Fix needed for communication with gpg-agent

[Richard Stallman]

    Since no one seems to have an idea of how to handle the
    console/pinentry case, let's just disable password caching on text
    terminals, and disabling use of gpg-agent on text terminals, by
    default.  We can suggest a workaround in the PGG manual, and tell
    people to enter a passphrase into pinentry before starting Emacs, if
    they want to use PGG with gpg-agent on a text terminal.

Before I could accept this solution, I need the people who objected to
other solutions a few months ago to join the discussion now.

Re: Fix needed for communication with gpg-agent

[Kim F. Storm]

Chong Yidong <cyd@stupidchicken.com> writes:

> Richard Stallman <rms@gnu.org> writes:
>
>> Would someone please implement this code, needed for correct
>> redisplay after running pinentry?  Then please ack.
>>
>> ...
>>
>> Without X, Pinentry falls back to a curses implementation (there is
>> also a curses only version of Pinentry).  This works pretty reliable
>> but poses one problem: The screen needs a redraw as there is no
>> portable way to save and restore the old screen content.  What can be
>> done is to redraw the screen every time after calling a crypto
>> operation which uses gpg or gpgsm.  This is of course not optimal as
>> in most cases the passphrase has been cached and an expensive redraw
>> is not required.
>
> I don't see any reasonable way to do this.

So why does Werner say that "this works pretty reliable" ...?

>
> Roughly speaking, when Emacs is run in a console (i.e. as a curses
> program), it monopolizes terminal input and output.  I don't know of
> any way to call the curses version of pinentry, have it temporarily
> "take over" the terminal, and return control to Emacs once it's done.

Well, if Emacs just "sleeps" while it waits for gpg to finish, then we
can assume that pinentry has finished using the screen as well.

accept-process-output has a just-this-one argument which causes emacs
to wait without reading the keyboard (it will still run timers though).

Maybe we use that to wait for gpg + pinentry already?  I guess Emacs
cannot really continue without getting the result from gpg.

.. and if we waited for more than a few seconds for gpg to complete,
we can (safely) assume that the user was involved, and redraw the
screen.  (so no need to implement the "protocol".)

Of course, we only need to do any of this if "no X".

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: Fix needed for communication with gpg-agent

[Chong Yidong]

storm@cua.dk (Kim F. Storm) writes:

>> Roughly speaking, when Emacs is run in a console (i.e. as a curses
>> program), it monopolizes terminal input and output.  I don't know of
>> any way to call the curses version of pinentry, have it temporarily
>> "take over" the terminal, and return control to Emacs once it's done.
>
> Well, if Emacs just "sleeps" while it waits for gpg to finish, then we
> can assume that pinentry has finished using the screen as well.
>
> accept-process-output has a just-this-one argument which causes emacs
> to wait without reading the keyboard (it will still run timers though).

I don't think it works like that under curses (maybe you know
something I don't, tho).

Re: Fix needed for communication with gpg-agent

[Richard Stallman]

    > Roughly speaking, when Emacs is run in a console (i.e. as a curses
    > program), it monopolizes terminal input and output.  I don't know of
    > any way to call the curses version of pinentry, have it temporarily
    > "take over" the terminal, and return control to Emacs once it's done.

    Well, if Emacs just "sleeps" while it waits for gpg to finish, then we

I think the issue is about process groups and job control.
When pinentry runs, will it be ABLE to write to the tty?

Re: Fix needed for communication with gpg-agent

[Chong Yidong]

Richard Stallman <rms@gnu.org> writes:

>     > Roughly speaking, when Emacs is run in a console (i.e. as a curses
>     > program), it monopolizes terminal input and output.  I don't know of
>     > any way to call the curses version of pinentry, have it temporarily
>     > "take over" the terminal, and return control to Emacs once it's done.
>
>     Well, if Emacs just "sleeps" while it waits for gpg to finish, then we
>
> I think the issue is about process groups and job control.
> When pinentry runs, will it be ABLE to write to the tty?

Whatever pinentry writes is garbled when Emacs refreshes the tty
screen.  The more serious problem, however, is that pinentry cannot
READ from the tty since Emacs gobbles all tty input.

The only sane way to handle this is to run gpg in a terminal emulator,
but that is a rather big job.  Since it's unknown whether such an
approach is secure anyway, I suggest disabling passphrase caching on
the console and leaving the pinentry-on-the-console issue till after
the release.

Re: Fix needed for communication with gpg-agent

[Richard Stallman]

    Whatever pinentry writes is garbled when Emacs refreshes the tty
    screen.  The more serious problem, however, is that pinentry cannot
    READ from the tty since Emacs gobbles all tty input.

Can we modify call-process to optionally pass control of the tty
to the subprogram?

Re: Fix needed for communication with gpg-agent

[Sascha Wilde]

Richard Stallman <rms@gnu.org> wrote:

>     Roughly speaking, when Emacs is run in a console (i.e. as a curses
>     program), it monopolizes terminal input and output.  I don't know of
>     any way to call the curses version of pinentry, have it temporarily
>     "take over" the terminal, and return control to Emacs once it's done.
>
> I am not sure whether that is a problem or not.  Daiki-san and Sascha,
> can you tell us whether this is a problem?

Unfortunately, yes.

A few more in depth tests[0] tests show, that emacs some how catches
some of the input, which is meant to go to pinentry:  The pinentry
curses interface is already shown, when emacs still gets some of the
key events...

> Is this part implemented already?  What will happen when gpg-agent
> tries to run pinentry if you are using a tty?

See above, the curses version of pinentry is started and shows up in
the terminal, but under certain circumstances it is hard to impossible
to input the passphrase.

I'm not sure if this is an emacs problem or an pinentry issue, but I
would tend to guess, that it would be pinentries job to make sure that
it gets full control other the terminal -- I'm not sure if this is
actually possible in a portable way, though.

Werner, what do you think?

On the other hand: the redraw problem, that some people mentioned,
doesn't seem to be a serious issue to me, emacs would only need to do
a full screen updated when all of the following conditions are true:
- emacs runs on an non graphical console
- gpg-agent is used
- an cryptographic action, which might need passphrase input, was run
I don't think that it would a big performance problem if we would do a
redraw in this case.  (And it shouldn't be hard to implement.)

cheers
sascha

[0] I tested this when I wrote the first implementation of gpg-agent
    support for pgg, but obviously not thoroughly enough.
--
Sascha Wilde
God put me on earth to accomplish a certain number of things.
Right now I am so far behind I will never die.
                       -- Bill Waterson, Calvin and Hobbes

Re: Fix needed for communication with gpg-agent

[Kim F. Storm]

Sascha Wilde <wilde@sha-bang.de> writes:

> A few more in depth tests[0] tests show, that emacs some how catches
> some of the input, which is meant to go to pinentry:  The pinentry
> curses interface is already shown, when emacs still gets some of the
> key events...

So pinentry CAN read some input ... 

Since this was the main problem which we were concerned with, the
problem is reduced to find a way to block Emacs from reading from the
tty while pinentry needs it.

Here's one way that might work (untested):

Currently, pgg.el uses code like this:

       (while (eq 'run (process-status process))
	 (accept-process-output process 5))

Replacing this with the following would (in principle) prevent Emacs
from reading any input while waiting for the process to terminate.

          (if window-system
             (while (eq 'run (process-status process))
	       (accept-process-output process 5))
             ;; Don't read from tty in case pinentry needs it.
             (accept-process-output process nil nil 1))


But we may also need to define SYNC_INPUT to make this work reliable ??

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: Fix needed for communication with gpg-agent

[Chong Yidong]

storm@cua.dk (Kim F. Storm) writes:

> Currently, pgg.el uses code like this:
>
>        (while (eq 'run (process-status process))
> 	 (accept-process-output process 5))
>
> Replacing this with the following would (in principle) prevent Emacs
> from reading any input while waiting for the process to terminate.
>              (accept-process-output process nil nil 1))
>
>
> But we may also need to define SYNC_INPUT to make this work reliable ??

That accept-process-output is done after sending gpg the text that we
want to encrypy/decrypt.  Gpg calls pinentry *before* accepting the
text.

BTW, for anyone who would like to test out your ideas on this problem,
here is a recipe:

1. $ killall gpg-agent; eval `gpg-agent --daemon`;
2. $ emacs
3. [Put some text in the region]
4. M-x pgg-sign-region

The two behaviors of pinentry can be observed to depend on whether you
are on the console or in X.

Re: Fix needed for communication with gpg-agent

[Werner Koch]

On Thu, 22 Feb 2007 01:14, cyd@stupidchicken.com said:

> 1. $ killall gpg-agent; eval `gpg-agent --daemon`;
> 2. $ emacs

FWIW, it is usually easier to run it this way:

  $ gpg-agent --daemon sh
  $ emacs

or just

  $ gpg-agent --dameon emacs



Shalom-Salam,

   Werner

Re: Fix needed for communication with gpg-agent

[Kim F. Storm]

Chong Yidong <cyd@stupidchicken.com> writes:

> storm@cua.dk (Kim F. Storm) writes:
>
>> Currently, pgg.el uses code like this:
>>
>>        (while (eq 'run (process-status process))
>> 	 (accept-process-output process 5))
>>
>> Replacing this with the following would (in principle) prevent Emacs
>> from reading any input while waiting for the process to terminate.
>> [...]
>>              (accept-process-output process nil nil 1))
>>
>>
>> But we may also need to define SYNC_INPUT to make this work reliable ??
>
> That accept-process-output is done after sending gpg the text that we
> want to encrypy/decrypt.  Gpg calls pinentry *before* accepting the
> text.

The code looks like this:

(setq process (start-process ...))
(process-send process ...)
(process-send-region process ...)
(process-send-eof process)
(accept-process-output process ...)

If we use SYNC_INPUT, I don't see where Emacs would read any keyboard input
between start-process and accept-process-output.  I suppose that pinentry
uses the tty in that time-frame...

Would someone pls. test if it works with the change I suggested --
possibly after recompiling Emacs with CFLAGS=-DSYNC_INPUT ...


-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: Fix needed for communication with gpg-agent

[Werner Koch]

On Thu, 22 Feb 2007 00:15, storm@cua.dk said:

> Since this was the main problem which we were concerned with, the
> problem is reduced to find a way to block Emacs from reading from the
> tty while pinentry needs it.

So the problem is that pinentry does not have exclusive access to the
tty.   What we do is:

  /* Open the desired terminal if necessary.  */
  if (tty_name)
    {
      ttyfi = fopen (tty_name, "r");
      if (!ttyfi)
	return -1;
      ttyfo = fopen (tty_name, "w");
      if (!ttyfo)
	{
	  int err = errno;
	  fclose (ttyfi);
	  errno = err;
	  return -1;
	}
      screen = newterm (tty_type, ttyfo, ttyfi);
      set_term (screen);
    }
  else
    {
      if (!init_screen)
	{
	  init_screen = 1;
	  initscr ();
	}
      else
	clear ();
    }
  
  keypad (stdscr, TRUE); /* Enable keyboard mapping.  */
  nonl ();		/* Tell curses not to do NL->CR/NL on output.  */
  cbreak ();		/* Take input chars one at a time, no wait for \n.  */
  noecho ();		/* Don't echo input - in color.  */

TTY_NAME is taken from the envvar GPG_TTY.  Just an idea:  What about
having emacs allocating a new pty pass that to gpg/gpg-agent.  Emacs
could then monitor the pty and thus notice if pinentry is going to use
it.




Salam-Shalom,

   Werner

Re: Fix needed for communication with gpg-agent

[Richard Stallman]

    > Since this was the main problem which we were concerned with, the
    > problem is reduced to find a way to block Emacs from reading from the
    > tty while pinentry needs it.

It is not intrinsically necessary to block Emacs from reading.
It would be enough if the code in Emacs does not read from the tty
during that time.

However, enabling pinentry to read from the tty probably requires
setting the tty's process group to the one that pinentry is in.  That
will probably have the effect that Emacs can't read from the tty
during that time.  If so, that is ok.

Re: Fix needed for communication with gpg-agent

[Werner Koch]

On Fri, 23 Feb 2007 11:22, rms@gnu.org said:

> However, enabling pinentry to read from the tty probably requires
> setting the tty's process group to the one that pinentry is in.  That
> will probably have the effect that Emacs can't read from the tty
> during that time.  If so, that is ok.

The problem is that I have found no way to set pinentry as the
foreground process.  tcsetpgrp() won't work because pinentry runs in a
different session (is is being called by gpg-agent which has used
setsid).



Shalom-Salam,

   Werner

Re: Fix needed for communication with gpg-agent

[Chong Yidong]

Werner Koch <wk@gnupg.org> writes:

>> However, enabling pinentry to read from the tty probably requires
>> setting the tty's process group to the one that pinentry is in.  That
>> will probably have the effect that Emacs can't read from the tty
>> during that time.  If so, that is ok.
>
> The problem is that I have found no way to set pinentry as the
> foreground process.  tcsetpgrp() won't work because pinentry runs in a
> different session (is is being called by gpg-agent which has used
> setsid).

...and since processes are not allowed to migrate between sessions,
this approach is pretty much sunk.

Re: Fix needed for communication with gpg-agent

[Richard Stallman]

    The problem is that I have found no way to set pinentry as the
    foreground process.  tcsetpgrp() won't work because pinentry runs in a
    different session (is is being called by gpg-agent which has used
    setsid).

Alas, I don't remember what setsid does, so I can't think about this
question.  However, someone reported that pinentry was able to read
from the tty, and that the only problem was to stop Emacs from also
trying to read.  If that is true, setsid apparently is not a real
obstacle -- at least on one kind of system.  Maybe it is a real
obstacle on some other kinds of system.

Re: Fix needed for communication with gpg-agent

[Richard Stallman]

    So the problem is that pinentry does not have exclusive access to the
    tty.

It should not be hard to program Emacs to turn off the signal handler
for tty input while it calls gpg.  That way, Emacs won't notice there
is any input, and won't read any.

This could be a new function made as a variant of call-process.

Would this solve the problem?

Re: Fix needed for communication with gpg-agent

[Chong Yidong]

Richard Stallman <rms@gnu.org> writes:

> It should not be hard to program Emacs to turn off the signal handler
> for tty input while it calls gpg.  That way, Emacs won't notice there
> is any input, and won't read any.
>
> This could be a new function made as a variant of call-process.
>
> Would this solve the problem?

I think you're assuming that Emacs calls gpg as a synchronous process:
i.e. call gpg, turn off tty input, wait for gpg to finish, restore tty
input.  The way the PGG code is currently set up, Emacs calls gpg as
an asynchronous process, and uses process-send-string to send gpg the
input text that we want to encrypt.

One reason to avoid call-process is that it communicates with
processes using a tempfile, which is a genuine security hole.

So for your suggestion to work, we would have to (i) create a new
built-in function similar to call-process, (ii) change it to use a pty
rather than a tempfile, (iii) make it turn off the Emacs tty signal
handler input while the process is running, (iv) find some way to
prevent Emacs from re-drawing the terminal screen while the process is
running, and (v) re-write pgg-gpg.el to use this built-in function,
which will require a revamp of its existing logic.

Or, we could recommend using X.

Re: Fix needed for communication with gpg-agent

[Kim F. Storm]

Chong Yidong <cyd@stupidchicken.com> writes:

> Richard Stallman <rms@gnu.org> writes:
>
>> It should not be hard to program Emacs to turn off the signal handler
>> for tty input while it calls gpg.  That way, Emacs won't notice there
>> is any input, and won't read any.
>>
>> This could be a new function made as a variant of call-process.
>>
>> Would this solve the problem?
>
> I think you're assuming that Emacs calls gpg as a synchronous process:
> i.e. call gpg, turn off tty input, wait for gpg to finish, restore tty
> input.  The way the PGG code is currently set up, Emacs calls gpg as
> an asynchronous process, and uses process-send-string to send gpg the
> input text that we want to encrypt.

It is still "synchronous" in the sense that we wait for the process
to finish before continuing...

Noone has yet explained why using accept-process-output to wait just
for the pgp process and not process keyboard input while waiting is
not sufficient.

If we need to enable SYNC_INPUT for that to work, and we don't want to
do that in general, we could make a negative "just-this-one" arg mean
"wait for single process to terminate with input blocked."

I.e. a call like this:

   (accept-process-output process nil nil -1)


We would only have to do that on a terminal, as pinentry works fine on
X already.

Any readon that wouldn't work??

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: Fix needed for communication with gpg-agent

[Chong Yidong]

storm@cua.dk (Kim F. Storm) writes:

> Noone has yet explained why using accept-process-output to wait just
> for the pgp process and not process keyboard input while waiting is
> not sufficient.
>
> If we need to enable SYNC_INPUT for that to work, and we don't want to
> do that in general, we could make a negative "just-this-one" arg mean
> "wait for single process to terminate with input blocked."
>
> I.e. a call like this:
>
>    (accept-process-output process nil nil -1)
>
> We would only have to do that on a terminal, as pinentry works fine on
> X already.
>
> Any readon that wouldn't work??

Have you tried it?  When I tried, gpg quits immediately, with or
without SYNC_INPUT; I can't debug it because I don't understand how
this idea is supposed to work in the first place.  IIUC, incoming tty
input in stored by Emacs, never forwarded to the gpg process.

Re: Fix needed for communication with gpg-agent

[Kim F. Storm]

Chong Yidong <cyd@stupidchicken.com> writes:

> storm@cua.dk (Kim F. Storm) writes:
>
>> Noone has yet explained why using accept-process-output to wait just
>> for the pgp process and not process keyboard input while waiting is
>> not sufficient.
>>
>> If we need to enable SYNC_INPUT for that to work, and we don't want to
>> do that in general, we could make a negative "just-this-one" arg mean
>> "wait for single process to terminate with input blocked."
>>
>> I.e. a call like this:
>>
>>    (accept-process-output process nil nil -1)
>>
>> We would only have to do that on a terminal, as pinentry works fine on
>> X already.
>>
>> Any readon that wouldn't work??
>
> Have you tried it?  

No -- the version of gpg I have doesn't have the gpg-agent.
I haven't had time to upgrade.

>                     When I tried, gpg quits immediately, with or
> without SYNC_INPUT; I can't debug it because I don't understand how
> this idea is supposed to work in the first place.  IIUC, incoming tty
> input in stored by Emacs, never forwarded to the gpg process.

Ok, so you say that Emacs does monopolise the tty, reading all input (or
at least blocking anyone else from reading from the tty).

IIRC, someone else said that pinentry was able to read _something_ but
that it was very difficult to actually read a working password (because
Emacs would read some of the input).

So what's the truth here?


In any case, I support your proposal to document the current restrictions
for Emacs on terminals -- anything else seems too complex at this time.

After the release, I think we should work closely with the gpg team on
a long term solution for making Emacs 23 and pinentry co-operate also
on a terminal (maybe the multi-tty branch has some additional things
to consider too).

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: Fix needed for communication with gpg-agent

[Sascha Wilde]

storm@cua.dk (Kim F. Storm) wrote:
> Ok, so you say that Emacs does monopolise the tty, reading all input (or
> at least blocking anyone else from reading from the tty).
>
> IIRC, someone else said that pinentry was able to read _something_ but
> that it was very difficult to actually read a working password (because
> Emacs would read some of the input).
>
> So what's the truth here?

It was me who wrote the later -- and it describes exactly what I
experienced on my tests.

sascha
-- 
Sascha Wilde   -.-. ..- .-. .. --- ... .. - -.-- 
               -.- .. .-.. .-.. . -.. 
               - .... .
               -.-. .- -