pgg-gpg.el - pgg-gpg-process-region timing problem

pgg-gpg.el - pgg-gpg-process-region timing problem

[ken manheimer]

[-- Attachment #1: Type: text/plain, Size: 3349 bytes --]

allout encryption, which depends on pgg.el &c, has been failing due to
a weird timing problem i've narrowed down to the
pgg-gpg-process-region function.  this is in emacs 23.0.60 built from
a CVS HEAD checkout of an hour or so ago, and some earlier 23.0.60
versions.  i've reproduced the problem using emacs -q and regularly
launched emacs, etc.

the problem is that pgg-gpg-process-region often returns a nil status
despite successful decryption, unless a delay (eg, (sleep-for .3) is
inserted between process start and later interaction with the process.
 with the delay, the status is something like 207 or 208, while
without it the status is nil - yet, on examination the content is
sucessfully decrypted.

the size of the delay needs to be increased when the machine is
heavily loaded, and can be reduced (on my thinkpad t43 2 GHz laptop,
3Gb ram, running kubuntu 8.0.4 +) to no lower than (sleep-for .2).
rarely, the proper status is returned without the delay, depending on
the machine load and other things i can't identify.  considering the
real-time sensitivity of the problem, your mileage will almost
certainly vary!

i originally thought the problem might be in gpg, but now figure it
resides either in the emacs process mechanism, pgg's use of that, or
somewhere in the combination of emacs and pgg.

i've created a simple .el script, pggprob.el, with minimal code and
data to produce the problem, and a patch, pggprob-patch.txt, with a
single-line modification to pgg-gpg-process-region to work around the
problem.  both are attached.  since the workaround is merely a timing
delay, it's clearly not a fix!  i'm hoping this is enough to enable
others to reproduce the problem, and investigate...
--
ken
http://myriadicity.net

If Emacs crashed, and you have the Emacs process in the gdb debugger,
please include the output from the following gdb commands:
    `bt full' and `xbacktrace'.
If you would like to further debug the crash, please read the file
/home/klm/src/emacs-HEAD/etc/DEBUG for instructions.

In GNU Emacs 23.0.60.1 (i686-pc-linux-gnu, GTK+ Version 2.12.9)
 of 2008-06-22 on twist
Windowing system distributor `The X.Org Foundation', version 11.0.10400090
Important settings:
  value of $LC_ALL: nil
  value of $LC_COLLATE: nil
  value of $LC_CTYPE: nil
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: nil
  value of $LANG: en_US.UTF-8
  value of $XMODIFIERS: nil
  locale-coding-system: utf-8-unix
  default-enable-multibyte-characters: t

Major mode: Emacs-Lisp

Minor modes in effect:
  tooltip-mode: t
  tool-bar-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  global-auto-composition-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent input:
C-x C-f <M-backspace> <M-backspace> <M-backspace> l
i b / e m a <tab> a l l <tab> s c r <tab> p g g p <tab>
<return> M-x e v a l - c u r <tab> <return> M-. C-g
M-> C-x C-e M-x e m a c s <M-backspace> r e p o r t
- e m <tab> <return>

Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.
Source file `/home/klm/src/emacs-HEAD/lisp/pgg-gpg.el' newer than
byte-compiled file
Quit
Mark set
nil

[-- Attachment #2: pggprob.el --]
[-- Type: application/octet-stream, Size: 671 bytes --]

;; demonstrate timing problem in pgg-gpg-process-region

(setq passphrase "a"
      verifier "-----BEGIN PGP MESSAGE-----
Version: GnuPG v1.4.6 (GNU/Linux)

jA0EAwMCo72fNGB1rNxgySv+rxdkPv46nVxQ5fg4yT78ldgVcDseD7Mre4Fhjifn
VTA1/PdmetVfFb9+
=MXF6
-----END PGP MESSAGE-----"
      count 0)

(defun decrypt (passphrase text)
  (with-current-buffer (get-buffer-create "*pgg-test*")
    (erase-buffer)
    (insert text)
    (pgg-decrypt nil nil passphrase)))

;; the following yields nil w/out the patched delay, 208 with it.
;; examination of the actual results buffer shows that the verifier is
;; successfully decrypted in both cases, however.
(decrypt passphrase verifier)

[-- Attachment #3: pgg-patch.txt --]
[-- Type: text/plain, Size: 409 bytes --]

--- pgg-gpg.el.~1.27.~	2008-05-06 03:57:46.000000000 -0400
+++ pgg-gpg.el	2008-06-22 13:24:14.000000000 -0400
@@ -86,6 +86,7 @@
 	    (setq process
 		  (apply #'start-process "*GnuPG*" errors-buffer
 			 program args)))
+          (sleep-for .3)                ; klm: timing delay
 	  (set-process-sentinel process #'ignore)
 	  (when passphrase
 	    (setq passphrase-with-newline (concat passphrase "\n"))

Re: pgg-gpg.el - pgg-gpg-process-region timing problem

[Thien-Thi Nguyen]

() "ken manheimer" <ken.manheimer@gmail.com>
() Sun, 22 Jun 2008 13:37:11 -0400

   pggprob-patch.txt, with a single-line modification to
   pgg-gpg-process-region to work around the problem.  both
   are attached.  since the workaround is merely a timing
   delay, it's clearly not a fix!

Please find below an UNTESTED patch that addresses the
problem in two ways:
 - use a downward funarg sentinel for complete monitoring
 - reduce unnecessary data motion

Does it give good results?

(Authors added to cc.)

thi

Re: pgg-gpg.el - pgg-gpg-process-region timing problem

[Thien-Thi Nguyen]

ChangeLog entry:

	Use process sentinel to monitor gpg subprocess status.

	* pgg-gpg.el (pgg-gpg-process-region): Fix omission bug:
	Use process sentinel to update process status and exit-status.
	Reported by ken manheimer <ken.manheimer@gmail.com>.

patch:
*** pgg-gpg.el	6 May 2008 07:57:46 -0000	1.27
--- pgg-gpg.el	23 Jun 2008 08:40:50 -0000
***************
*** 70,111 ****
  	    ,@pgg-gpg-extra-args ,@args))
  	 (output-buffer pgg-output-buffer)
  	 (errors-buffer pgg-errors-buffer)
! 	 (orig-mode (default-file-modes))
! 	 (process-connection-type nil)
! 	 (inhibit-redisplay t)
! 	 process status exit-status
! 	 passphrase-with-newline
! 	 encoded-passphrase-with-new-line)
      (with-current-buffer (get-buffer-create errors-buffer)
        (buffer-disable-undo)
        (erase-buffer))
      (unwind-protect
! 	(progn
  	  (set-default-file-modes 448)
! 	  (let ((coding-system-for-write 'binary))
! 	    (setq process
! 		  (apply #'start-process "*GnuPG*" errors-buffer
! 			 program args)))
! 	  (set-process-sentinel process #'ignore)
  	  (when passphrase
! 	    (setq passphrase-with-newline (concat passphrase "\n"))
! 	    (if pgg-passphrase-coding-system
! 		(progn
! 		  (setq encoded-passphrase-with-new-line
  			(encode-coding-string
! 			 passphrase-with-newline
! 			 (coding-system-change-eol-conversion
! 			  pgg-passphrase-coding-system 'unix)))
! 		  (pgg-clear-string passphrase-with-newline))
! 	      (setq encoded-passphrase-with-new-line passphrase-with-newline
! 		    passphrase-with-newline nil))
! 	    (process-send-string process encoded-passphrase-with-new-line))
  	  (process-send-region process start end)
  	  (process-send-eof process)
! 	  (while (eq 'run (process-status process))
  	    (accept-process-output process 5))
- 	  (setq status (process-status process)
- 		exit-status (process-exit-status process))
  	  (delete-process process)
  	  (with-current-buffer (get-buffer-create output-buffer)
  	    (buffer-disable-undo)
--- 70,104 ----
  	    ,@pgg-gpg-extra-args ,@args))
  	 (output-buffer pgg-output-buffer)
  	 (errors-buffer pgg-errors-buffer)
! 	 (inhibit-redisplay t))
      (with-current-buffer (get-buffer-create errors-buffer)
        (buffer-disable-undo)
        (erase-buffer))
      (unwind-protect
! 	(let* ((coding-system-for-write 'binary)
! 	       (process (let ((orig (default-file-modes)))
  			  (set-default-file-modes 448)
! 			  (prog1 (apply #'start-process "*GnuPG*"
! 					errors-buffer program args)
! 			    (set-default-file-modes orig))))
! 	       (status (process-status process))
! 	       exit-status)
! 	  (set-process-sentinel
! 	   process (lambda (process reason)
! 		     (setq status (process-status process)
! 			   exit-status (process-exit-status process))))
  	  (when passphrase
! 	    (process-send-string
! 	     process (if pgg-passphrase-coding-system
  			 (encode-coding-string
! 			  passphrase
! 			  pgg-passphrase-coding-system)
! 		       passphrase))
! 	    (process-send-string process "\n"))
  	  (process-send-region process start end)
  	  (process-send-eof process)
! 	  (while (eq 'run status)
  	    (accept-process-output process 5))
  	  (delete-process process)
  	  (with-current-buffer (get-buffer-create output-buffer)
  	    (buffer-disable-undo)
***************
*** 120,134 ****
  		(error "%s exited abnormally: '%s'" program exit-status))
  	    (if (= 127 exit-status)
  		(error "%s could not be found" program))))
-       (if passphrase-with-newline
- 	  (pgg-clear-string passphrase-with-newline))
-       (if encoded-passphrase-with-new-line
- 	  (pgg-clear-string encoded-passphrase-with-new-line))
-       (if (and process (eq 'run (process-status process)))
- 	  (interrupt-process process))
        (if (file-exists-p output-file-name)
! 	  (delete-file output-file-name))
!       (set-default-file-modes orig-mode))))
  
  (defun pgg-gpg-possibly-cache-passphrase (passphrase &optional key notruncate)
    (if (and passphrase
--- 113,120 ----
  		(error "%s exited abnormally: '%s'" program exit-status))
  	    (if (= 127 exit-status)
  		(error "%s could not be found" program))))
        (if (file-exists-p output-file-name)
! 	  (delete-file output-file-name)))))
  
  (defun pgg-gpg-possibly-cache-passphrase (passphrase &optional key notruncate)
    (if (and passphrase

Re: pgg-gpg.el - pgg-gpg-process-region timing problem

[Thien-Thi Nguyen]

Delving into this, i got a little carried away... full func below.
This adds GOOD_PASSPHRASE checking (necessary for GNUPG 1.4.6, at
least) and some other coding-system related finessing, but removes
the sentinel proposed in the first attempt.  In its place, we zonk
the baleful `process-adaptive-read-buffering' and remove timeout
parameters from `accept-process-output' calls.

With this func things seem to work fine for `pgg-sign' on a small
buffer, and mostly for large buffers (270KB), but sometimes (~10%)
with large buffers, i see a "file error: bad address *GnuPG*".  Hmmm...

thi

______________________________________________________________________
(defun pgg-gpg-process-region (start end passphrase program args)
  (let* ((use-agent (and (null passphrase) (pgg-gpg-use-agent-p)))
	 (output-file-name (pgg-make-temp-file "pgg-output"))
	 (args
	  `("--status-fd" "2"
	    ,@(if use-agent '("--use-agent")
		(if passphrase '("--passphrase-fd" "0")))
	    "--yes" ; overwrite
	    "--output" ,output-file-name
	    ,@pgg-gpg-extra-args ,@args))
	 (output-buffer pgg-output-buffer)
	 (errors-buffer pgg-errors-buffer)
	 (orig-mode (default-file-modes))
	 (inhibit-redisplay t))
    (with-current-buffer (get-buffer-create errors-buffer)
      (buffer-disable-undo)
      (erase-buffer))
    (unwind-protect
	(let* ((coding-system-for-write 'binary)
	       ;; GNUPG 1.4.6 does not terminate on bad passphrase, eg:
	       ;;   [GNUPG:] BAD_PASSPHRASE (long hex # here)
	       ;;   gpg: skipped "ttn": bad passphrase
	       ;;   gpg: [stdin]: clearsign failed: bad passphrase
	       ;; so we need to check that condition ourselves and bail out.
	       ;;
	       ;; To check if the passphrase is accepted, we need to parse the
	       ;; errors-buffer, but `process-adaptive-read-buffering' non-nil
	       ;; sometimes prevents it from filling.  So turn it off.
	       (process-adaptive-read-buffering (not passphrase))
	       (process (progn
			  (set-default-file-modes 448)
			  (apply #'start-process "*GnuPG*"
				 errors-buffer program args)))
	       (status (process-status process))

	       exit-status)
	  (set-process-sentinel process nil)
	  (when passphrase
	    (let ((coding-system-for-write (or pgg-passphrase-coding-system
					       'binary)))
	      (process-send-string process passphrase))
	    (process-send-string process "\n")
	    ;; Bail out if passphrase is not accepted.
	    ;; MAINTAIN ME: Tested against GNUPG 1.4.6.
	    (let (result)
	      (while (not result)
		(accept-process-output process)
		(with-current-buffer errors-buffer
		  (save-excursion
		    (goto-char (point-min))
		    (when (re-search-forward
			   ;; BGM: BAD, GOOD, MISSING.
			   "^.GNUPG:. \\([BGM][A-Z]+\\)_PASSPHRASE"
			   nil t)
		      (setq result (match-string 1))))))
	      (unless (string= "GOOD" result)
		(error "Passphrase no good"))))
	  (process-send-region process start end)
	  (process-send-eof process)
	  ;; TODO: Re-enable `process-adaptive-read-buffering' here.
	  (while (eq 'run (setq status (process-status process)))
	    (accept-process-output process))
	  (delete-process process)
	  (setq exit-status (process-exit-status process))
	  (with-current-buffer (get-buffer-create output-buffer)
	    (buffer-disable-undo)
	    (erase-buffer)
	    (if (file-exists-p output-file-name)
		(let ((coding-system-for-read (if pgg-text-mode
						  'raw-text
						'binary)))
		  (insert-file-contents output-file-name)))
	    (set-buffer errors-buffer)
	    (if (memq status '(stop signal))
		(error "%s exited abnormally: '%s'" program exit-status))
	    (if (= 127 exit-status)
		(error "%s could not be found" program))))
      (if (file-exists-p output-file-name)
	  (delete-file output-file-name))
      (when (get-process "*GnuPG*")
	(kill-process "*GnuPG*"))
      (set-default-file-modes orig-mode))))

Re: pgg-gpg.el - pgg-gpg-process-region timing problem

[ken manheimer]

hi, thi.  thanks for the responses - i'm glad to see investigation into this.

i've tried both the patches, and both hang for me on the pggprob.el
test.  the hang is interruptable with Ctl-G keyboard quit.  i tried
stepping through the second patched version with edebug (including
disabling the '(inhibit-redisplay t)' let-binding, which thwarts
edebug stepping), and found the hang happens at the first
'(accept-process-output process)'.  (it hangs at the second one if i
don't provide a passphrase.)

i'm not sure what's happening.  in case it helps, gpg 1.4.6 is what's
running on my system.
-- 
ken
http://myriadicity.net



On Mon, Jun 23, 2008 at 9:17 AM, Thien-Thi Nguyen <ttn@gnuvola.org> wrote:
> Delving into this, i got a little carried away... full func below.
> This adds GOOD_PASSPHRASE checking (necessary for GNUPG 1.4.6, at
> least) and some other coding-system related finessing, but removes
> the sentinel proposed in the first attempt.  In its place, we zonk
> the baleful `process-adaptive-read-buffering' and remove timeout
> parameters from `accept-process-output' calls.
>
> With this func things seem to work fine for `pgg-sign' on a small
> buffer, and mostly for large buffers (270KB), but sometimes (~10%)
> with large buffers, i see a "file error: bad address *GnuPG*".  Hmmm...
>
> thi
>
> ______________________________________________________________________
> (defun pgg-gpg-process-region (start end passphrase program args)
>  (let* ((use-agent (and (null passphrase) (pgg-gpg-use-agent-p)))
>         (output-file-name (pgg-make-temp-file "pgg-output"))
>         (args
>          `("--status-fd" "2"
>            ,@(if use-agent '("--use-agent")
>                (if passphrase '("--passphrase-fd" "0")))
>            "--yes" ; overwrite
>            "--output" ,output-file-name
>            ,@pgg-gpg-extra-args ,@args))
>         (output-buffer pgg-output-buffer)
>         (errors-buffer pgg-errors-buffer)
>         (orig-mode (default-file-modes))
>         (inhibit-redisplay t))
>    (with-current-buffer (get-buffer-create errors-buffer)
>      (buffer-disable-undo)
>      (erase-buffer))
>    (unwind-protect
>        (let* ((coding-system-for-write 'binary)
>               ;; GNUPG 1.4.6 does not terminate on bad passphrase, eg:
>               ;;   [GNUPG:] BAD_PASSPHRASE (long hex # here)
>               ;;   gpg: skipped "ttn": bad passphrase
>               ;;   gpg: [stdin]: clearsign failed: bad passphrase
>               ;; so we need to check that condition ourselves and bail out.
>               ;;
>               ;; To check if the passphrase is accepted, we need to parse the
>               ;; errors-buffer, but `process-adaptive-read-buffering' non-nil
>               ;; sometimes prevents it from filling.  So turn it off.
>               (process-adaptive-read-buffering (not passphrase))
>               (process (progn
>                          (set-default-file-modes 448)
>                          (apply #'start-process "*GnuPG*"
>                                 errors-buffer program args)))
>               (status (process-status process))
>
>               exit-status)
>          (set-process-sentinel process nil)
>          (when passphrase
>            (let ((coding-system-for-write (or pgg-passphrase-coding-system
>                                               'binary)))
>              (process-send-string process passphrase))
>            (process-send-string process "\n")
>            ;; Bail out if passphrase is not accepted.
>            ;; MAINTAIN ME: Tested against GNUPG 1.4.6.
>            (let (result)
>              (while (not result)
>                (accept-process-output process)
>                (with-current-buffer errors-buffer
>                  (save-excursion
>                    (goto-char (point-min))
>                    (when (re-search-forward
>                           ;; BGM: BAD, GOOD, MISSING.
>                           "^.GNUPG:. \\([BGM][A-Z]+\\)_PASSPHRASE"
>                           nil t)
>                      (setq result (match-string 1))))))
>              (unless (string= "GOOD" result)
>                (error "Passphrase no good"))))
>          (process-send-region process start end)
>          (process-send-eof process)
>          ;; TODO: Re-enable `process-adaptive-read-buffering' here.
>          (while (eq 'run (setq status (process-status process)))
>            (accept-process-output process))
>          (delete-process process)
>          (setq exit-status (process-exit-status process))
>          (with-current-buffer (get-buffer-create output-buffer)
>            (buffer-disable-undo)
>            (erase-buffer)
>            (if (file-exists-p output-file-name)
>                (let ((coding-system-for-read (if pgg-text-mode
>                                                  'raw-text
>                                                'binary)))
>                  (insert-file-contents output-file-name)))
>            (set-buffer errors-buffer)
>            (if (memq status '(stop signal))
>                (error "%s exited abnormally: '%s'" program exit-status))
>            (if (= 127 exit-status)
>                (error "%s could not be found" program))))
>      (if (file-exists-p output-file-name)
>          (delete-file output-file-name))
>      (when (get-process "*GnuPG*")
>        (kill-process "*GnuPG*"))
>      (set-default-file-modes orig-mode))))

Re: pgg-gpg.el - pgg-gpg-process-region timing problem

[Daiki Ueno]

Please do NOT fix problems like this by guesswork.

>>>>> In <873an4b898.fsf@ambire.localdomain> 
>>>>>	Thien-Thi Nguyen <ttn@gnuvola.org> wrote:
> Delving into this, i got a little carried away... full func below.
> This adds GOOD_PASSPHRASE checking (necessary for GNUPG 1.4.6, at
> least)

Really?  I'm using 1.4.9 and it just exits immediately with
BAD_PASSPHRASE.  And besides, in the case --symmetric is given,
(GOOD|BAD)_PASSPHRASE should not be issued IIUC.

Perhaps you should first try to reproduce the exactly same problem as
ken manheimer reported, though I can't reproduce it on my system.

Regards,
-- 
Daiki Ueno

Re: pgg-gpg.el - pgg-gpg-process-region timing problem

[Thien-Thi Nguyen]

() Daiki Ueno <ueno@unixuser.org>
() Tue, 24 Jun 2008 10:59:52 +0900

   Please do NOT fix problems like this by guesswork.

OK.

   Really?

Yes, i see that for 1.4.6.

   Perhaps you should first try to reproduce the exactly same
   problem as ken manheimer reported, though I can't reproduce it
   on my system.

I will try to do so (but my machine is not that fast -- do you
have any tips for simulating speed?).

thi

Re: pgg-gpg.el - pgg-gpg-process-region timing problem

[Thien-Thi Nguyen]

() "ken manheimer" <ken.manheimer@gmail.com>
() Mon, 23 Jun 2008 13:44:52 -0400

   the hang happens at the first
   '(accept-process-output process)'.

What happens if you convert
  (accept-process-output process)
to
  (accept-process-output)
?

thi

Re: pgg-gpg.el - pgg-gpg-process-region timing problem

[Stefan Monnier]

>    the hang happens at the first
>    '(accept-process-output process)'.

> What happens if you convert
>   (accept-process-output process)
> to
>   (accept-process-output)
> ?

If that fixes it, it means either that `process' is not the right
process, or that we may have a bug in the C code.  So if that fixes the
bug, please don't just install such a workaround without trying to find
the root cause first.


        Stefan

Re: pgg-gpg.el - pgg-gpg-process-region timing problem

[Daiki Ueno]

>>>>> In <87y74v2o94.fsf@ambire.localdomain> 
>>>>>	Thien-Thi Nguyen <ttn@gnuvola.org> wrote:
>    Really?

> Yes, i see that for 1.4.6.

I couldn't confirm the phenomenon even with 1.4.6.  Now I suspect that
you are using different command line from the one pgg-gpg uses.

This may ask you passphrase several times:

gpg --status-fd 2 --decrypt test.gpg

But pgg-gpg uses something like:

gpg --passphrase-fd 0 --status-fd 2 --decrypt test.gpg

By the way, I think you would be happy if you were using M-x
epa-decrypt-file instead.  It can handle this case smarter -- i.e. it
will ask you the correct passphrase again if you mistyped it ;-)

Regards,
-- 
Daiki Ueno

Re: pgg-gpg.el - pgg-gpg-process-region timing problem

[ken manheimer]

[-- Attachment #1: Type: text/plain, Size: 5144 bytes --]

i am not sure how to proceed with this problem.  as demonstrated by
the pggprob.el script, i consistently get encryption and decryption
problems without the contrived delay in my pgg-gpg-process-region
patch.  (the problem does not happen if i just do a regular
pgg-decrypt - i think the crucial thing is that a symmetric passphrase
is being passed into pgg-gpg-process-region.)

i have tried everything i could think of to exclude local quirks from
the problem, including starting emacs with '--no-site-file -q' and
eliminating everything from my ~/.gnupg/gpg.conf file.  i have
reproduced the problem (and workaround) consistently in kubuntu 8.04
emacs-snapshot package and a recent CVS HEAD checkout.  in fact, i
have incidentally reinstalled my kubuntu 8.04 OS recently, on both my
main 2.0 Ghz T43 thinkpad and on a 750 Mhz a20p thinkpad.  the problem
continues consistently on the t43 but does *not* occur on the a20p,
which is not contrary to the sensitivities of a timing problem.

i have also tried thien-thi nguyen's alternative approaches, but
neither worked for me, hanging in different places.

i am pretty sure this is a real problem, and something that warrants
attention.  i realize that the difficulty in repeating the problem
elsewhere increases the challenge in addressing it, and am open to
cooperating with people to try things, if that would help.  the
consistent effectiveness of my workaround should help provide clues
about the nature of the problem, at least.

any suggestions?
--
ken
http://myriadicity.net

On Sun, Jun 22, 2008 at 1:37 PM, ken manheimer <ken.manheimer@gmail.com> wrote:
> allout encryption, which depends on pgg.el &c, has been failing due to
> a weird timing problem i've narrowed down to the
> pgg-gpg-process-region function.  this is in emacs 23.0.60 built from
> a CVS HEAD checkout of an hour or so ago, and some earlier 23.0.60
> versions.  i've reproduced the problem using emacs -q and regularly
> launched emacs, etc.
>
> the problem is that pgg-gpg-process-region often returns a nil status
> despite successful decryption, unless a delay (eg, (sleep-for .3) is
> inserted between process start and later interaction with the process.
>  with the delay, the status is something like 207 or 208, while
> without it the status is nil - yet, on examination the content is
> sucessfully decrypted.
>
> the size of the delay needs to be increased when the machine is
> heavily loaded, and can be reduced (on my thinkpad t43 2 GHz laptop,
> 3Gb ram, running kubuntu 8.0.4 +) to no lower than (sleep-for .2).
> rarely, the proper status is returned without the delay, depending on
> the machine load and other things i can't identify.  considering the
> real-time sensitivity of the problem, your mileage will almost
> certainly vary!
>
> i originally thought the problem might be in gpg, but now figure it
> resides either in the emacs process mechanism, pgg's use of that, or
> somewhere in the combination of emacs and pgg.
>
> i've created a simple .el script, pggprob.el, with minimal code and
> data to produce the problem, and a patch, pggprob-patch.txt, with a
> single-line modification to pgg-gpg-process-region to work around the
> problem.  both are attached.  since the workaround is merely a timing
> delay, it's clearly not a fix!  i'm hoping this is enough to enable
> others to reproduce the problem, and investigate...
> --
> ken
> http://myriadicity.net
>
> If Emacs crashed, and you have the Emacs process in the gdb debugger,
> please include the output from the following gdb commands:
>    `bt full' and `xbacktrace'.
> If you would like to further debug the crash, please read the file
> /home/klm/src/emacs-HEAD/etc/DEBUG for instructions.
>
> In GNU Emacs 23.0.60.1 (i686-pc-linux-gnu, GTK+ Version 2.12.9)
>  of 2008-06-22 on twist
> Windowing system distributor `The X.Org Foundation', version 11.0.10400090
> Important settings:
>  value of $LC_ALL: nil
>  value of $LC_COLLATE: nil
>  value of $LC_CTYPE: nil
>  value of $LC_MESSAGES: nil
>  value of $LC_MONETARY: nil
>  value of $LC_NUMERIC: nil
>  value of $LC_TIME: nil
>  value of $LANG: en_US.UTF-8
>  value of $XMODIFIERS: nil
>  locale-coding-system: utf-8-unix
>  default-enable-multibyte-characters: t
>
> Major mode: Emacs-Lisp
>
> Minor modes in effect:
>  tooltip-mode: t
>  tool-bar-mode: t
>  mouse-wheel-mode: t
>  menu-bar-mode: t
>  file-name-shadow-mode: t
>  global-font-lock-mode: t
>  font-lock-mode: t
>  blink-cursor-mode: t
>  global-auto-composition-mode: t
>  auto-composition-mode: t
>  auto-encryption-mode: t
>  auto-compression-mode: t
>  line-number-mode: t
>  transient-mark-mode: t
>
> Recent input:
> C-x C-f <M-backspace> <M-backspace> <M-backspace> l
> i b / e m a <tab> a l l <tab> s c r <tab> p g g p <tab>
> <return> M-x e v a l - c u r <tab> <return> M-. C-g
> M-> C-x C-e M-x e m a c s <M-backspace> r e p o r t
> - e m <tab> <return>
>
> Recent messages:
> For information about GNU Emacs and the GNU system, type C-h C-a.
> Source file `/home/klm/src/emacs-HEAD/lisp/pgg-gpg.el' newer than
> byte-compiled file
> Quit
> Mark set
> nil
>



-- 
ken
http://myriadicity.net

[-- Attachment #2: pggprob.el --]
[-- Type: application/octet-stream, Size: 671 bytes --]

;; demonstrate timing problem in pgg-gpg-process-region

(setq passphrase "a"
      verifier "-----BEGIN PGP MESSAGE-----
Version: GnuPG v1.4.6 (GNU/Linux)

jA0EAwMCo72fNGB1rNxgySv+rxdkPv46nVxQ5fg4yT78ldgVcDseD7Mre4Fhjifn
VTA1/PdmetVfFb9+
=MXF6
-----END PGP MESSAGE-----"
      count 0)

(defun decrypt (passphrase text)
  (with-current-buffer (get-buffer-create "*pgg-test*")
    (erase-buffer)
    (insert text)
    (pgg-decrypt nil nil passphrase)))

;; the following yields nil w/out the patched delay, 208 with it.
;; examination of the actual results buffer shows that the verifier is
;; successfully decrypted in both cases, however.
(decrypt passphrase verifier)

[-- Attachment #3: pgg-patch.txt --]
[-- Type: text/plain, Size: 409 bytes --]

--- pgg-gpg.el.~1.27.~	2008-05-06 03:57:46.000000000 -0400
+++ pgg-gpg.el	2008-06-22 13:24:14.000000000 -0400
@@ -86,6 +86,7 @@
 	    (setq process
 		  (apply #'start-process "*GnuPG*" errors-buffer
 			 program args)))
+          (sleep-for .3)                ; klm: timing delay
 	  (set-process-sentinel process #'ignore)
 	  (when passphrase
 	    (setq passphrase-with-newline (concat passphrase "\n"))

Re: pgg-gpg.el - pgg-gpg-process-region timing problem

[Chong Yidong]

> i am not sure how to proceed with this problem.  as demonstrated by
> the pggprob.el script, i consistently get encryption and decryption
> problems without the contrived delay in my pgg-gpg-process-region
> patch.  (the problem does not happen if i just do a regular
> pgg-decrypt - i think the crucial thing is that a symmetric passphrase
> is being passed into pgg-gpg-process-region.)

Here's one suggestion, though I don't know how useful it is: if it's
possible to run the same or similar version of pgg in Emacs 22, you
could and see if it appears there.  If not, we could narrow the problem
down to a change in Emacs' process code during Emacs 23 (though OTOH I
don't know of any that could lead to this).

Re: pgg-gpg.el - pgg-gpg-process-region timing problem

[ken manheimer]

[i've removed emacs-pretest-bug@ from the recipients - it was my
mistake to continue to cc that address in my correspondence about this
problem...]

On Thu, Jun 26, 2008 at 12:32 PM, Chong Yidong <cyd@stupidchicken.com> wrote:

>> i am not sure how to proceed with this problem.  as demonstrated by
>> the pggprob.el script, i consistently get encryption and decryption
>> problems without the contrived delay in my pgg-gpg-process-region
>> patch.  (the problem does not happen if i just do a regular
>> pgg-decrypt - i think the crucial thing is that a symmetric passphrase
>> is being passed into pgg-gpg-process-region.)
>
> Here's one suggestion, though I don't know how useful it is: if it's
> possible to run the same or similar version of pgg in Emacs 22, you
> could and see if it appears there.  If not, we could narrow the problem
> down to a change in Emacs' process code during Emacs 23 (though OTOH I
> don't know of any that could lead to this).

this is certainly worth trying.

it brings me to a feeble question - is there a stable CVS tag for
checkout of the currently released emacs version?  i see MAIN and
EMACS_22_BASE, neither of which is very descriptive.  (MAIN is too
generic to draw any conclusions, and EMACS_22_BASE sounds like
something that is built upon, not a destination in itself.)  have i
missed one?

(i know this is a digression, but i feel lost in a twisty maze of
passages, trying to find an authoritative listing of the most
significant emacs CVS repository tags.  i couldn't find anything in
the emacs project site on savannah or in the emacs wiki.  seems like
it would be useful to have such a listing associated with the
project!)
-- 
ken
http://myriadicity.net

Re: pgg-gpg.el - pgg-gpg-process-region timing problem

[Miles Bader]

"ken manheimer" <ken.manheimer@gmail.com> writes:
> is there a stable CVS tag for checkout of the currently released emacs
> version?  i see MAIN and EMACS_22_BASE, neither of which is very
> descriptive.  (MAIN is too generic to draw any conclusions, and
> EMACS_22_BASE sounds like something that is built upon, not a
> destination in itself.)

Despite the name, EMACS_22_BASE is actually the _tip_ of the release
branch.

-Miles

-- 
A zen-buddhist walked into a pizza shop and
said, "Make me one with everything."

Re: pgg-gpg.el - pgg-gpg-process-region timing problem

[ken manheimer]

On Fri, Jun 27, 2008 at 10:24 AM, Miles Bader <miles@gnu.org> wrote:
> "ken manheimer" <ken.manheimer@gmail.com> writes:
>> is there a stable CVS tag for checkout of the currently released emacs
>> version?  i see MAIN and EMACS_22_BASE, neither of which is very
>> descriptive.  (MAIN is too generic to draw any conclusions, and
>> EMACS_22_BASE sounds like something that is built upon, not a
>> destination in itself.)
>
> Despite the name, EMACS_22_BASE is actually the _tip_ of the release
> branch.

so, do i want to use EMACS_22_2, instead?  i'm trying that...

(i guess there's not single non-branch tag that tracks the current
release, like EMACS_RELEASE?)

> --
> A zen-buddhist walked into a pizza shop and
> said, "Make me one with everything."

ooh, ooh - that's not all!

the buddhist monk (as i heard it) gets his pizza and pays with a €10
(or $20 or whatever) and waits around for bit.  the pizza guy asks
"what's the matter?", and the monk says, "i was expecting some
change", to which the pizza guys replies:

  "you of all people should know, change comes from within."

:-)
-- 
ken
http://myriadicity.net

  it's frog soup night, and two frogs are sitting in a pot of water under
  which the fire was just lit.  one frog screams, "help, help, get me
  out of here!".  the other frog points at the first and shouts, "oh my
  god, a talking frog!"

Re: pgg-gpg.el - pgg-gpg-process-region timing problem

[Stefan Monnier]

>>> is there a stable CVS tag for checkout of the currently released emacs
>>> version?  i see MAIN and EMACS_22_BASE, neither of which is very
>>> descriptive.  (MAIN is too generic to draw any conclusions, and
>>> EMACS_22_BASE sounds like something that is built upon, not a
>>> destination in itself.)
>> 
>> Despite the name, EMACS_22_BASE is actually the _tip_ of the release
>> branch.

> so, do i want to use EMACS_22_2, instead?  i'm trying that...

Yes: the Emacs-22.2 tarball was cut from the content of the
EMACS_22_2 tag (at least, that's the intention).


        Stefan


PS: You can similarly check out EMACS_19_34, EMACS_20_4, ... tho those older
tags were applied retroactively and I don't think they give you a clean
tree from which you can built things right.  They may also include extra
files or omit necessary ones.

Re: pgg-gpg.el - pgg-gpg-process-region timing problem

[ken manheimer]

i've got some more datapoints, and they suggest that i'm experiencing
a local OS installation/incidentals problem.

first, as suggested by yidong, i've tested and reproduced the problemi
identically in an emacs 22 build from a CVS EMACS_22_2 tagged
checkout.  i need not refer to anything external but my pggprob.el
tester.  (i also noticed that i the first attempt in each session to
evoke the error does not demonstrate the problem.  it only shows on
the second and subsequent tries.)

conversely, i am unable to reproduce the problem on any other hosts
with much of my operating environment, but various instances of emacs
22 and 23 and different OS and gpg installations (including gentoo
under vmware, rhel 4, fedora 1).  in general there are too many
variables to draw any conclusions, unfortunately, but it's important
info.

all this kinda explains why i haven't heard any complaints from any
other allout encrypted-topics users, and why i may be unlikely to hear
such.  i still don't know what it is about my system that's causing
the problem.
-- 
ken
http://myriadicity.net

On Fri, Jun 27, 2008 at 10:01 AM, ken manheimer <ken.manheimer@gmail.com> wrote:
> [i've removed emacs-pretest-bug@ from the recipients - it was my
> mistake to continue to cc that address in my correspondence about this
> problem...]
>
> On Thu, Jun 26, 2008 at 12:32 PM, Chong Yidong <cyd@stupidchicken.com> wrote:
>
>>> i am not sure how to proceed with this problem.  as demonstrated by
>>> the pggprob.el script, i consistently get encryption and decryption
>>> problems without the contrived delay in my pgg-gpg-process-region
>>> patch.  (the problem does not happen if i just do a regular
>>> pgg-decrypt - i think the crucial thing is that a symmetric passphrase
>>> is being passed into pgg-gpg-process-region.)
>>
>> Here's one suggestion, though I don't know how useful it is: if it's
>> possible to run the same or similar version of pgg in Emacs 22, you
>> could and see if it appears there.  If not, we could narrow the problem
>> down to a change in Emacs' process code during Emacs 23 (though OTOH I
>> don't know of any that could lead to this).
>
> this is certainly worth trying.
>
> it brings me to a feeble question - is there a stable CVS tag for
> checkout of the currently released emacs version?  i see MAIN and
> EMACS_22_BASE, neither of which is very descriptive.  (MAIN is too
> generic to draw any conclusions, and EMACS_22_BASE sounds like
> something that is built upon, not a destination in itself.)  have i
> missed one?
>
> (i know this is a digression, but i feel lost in a twisty maze of
> passages, trying to find an authoritative listing of the most
> significant emacs CVS repository tags.  i couldn't find anything in
> the emacs project site on savannah or in the emacs wiki.  seems like
> it would be useful to have such a listing associated with the
> project!)
> --
> ken
> http://myriadicity.net