bug#6641: 23.2; signing email with smime using mml ignores extra certs in smime-keys

bug#6641: 23.2; signing email with smime using mml ignores extra certs in smime-keys

[Jay Berkenbilt]

This message includes a full analysis of the problem as well as a
workaround, but I don't have a complete fix.

BACKGROUND:

This background will be known to anyone who is familiar with how S/MIME
works, but since a lot of people aren't or have misconceptions, I'm
including it for completeness.

When sending a signed message with S/MIME, all certificates from the
signer's public key up the issuer chain all the way to (but not
including) a trusted CA key that ultimately signed the key must be
included in the message.  For example, suppose user "user@example.com"
has a key ("user-key") signed by "signer-key" which is in turn signed by
"trusted-ca", where "trusted-ca" is a CA key that is trusted by the
recipient.  In this case, the smime.p7s file must contain the user's
public key and also signer-key's public key.

The smime-keys variable is an association list that maps email addresses
to a list whose first element is a key file and whose second element, if
any, is a list of additional certificates to attach to the email.  For
example, in the above case, the user could set her smime-keys variable
to

'(("user@example.com" "~/keys/user-key.pem" ("~/keys/signer-key.pem")))

When customizing the smime-keys variable, this structure is correctly
created, and the function smime-sign-buffer uses it appropriately via a
call to smime-get-key-with-certs-by-email.

END OF BACKGROUND.

DETAILS:

If you compose mail with M-x message-mail and attempt to sign a message
with S/MIME, you will be required to have the smime-keys variable
appropriately configured.  If you don't, you will be prompted to
customize it.  That all works, so we'll just do this manually for
expediency.

 * Run emacs -Q

 * In the scratch buffer, type

(setq smime-keys `((,user-mail-address "/tmp/user.pem" ("/tmp/chain.pem"))))

   and hit C-j to evaluate.  Don't worry that the files don't exist.

 * Run M-x message-mail RET

 * Type C-c C-m s s

 * Observe that the following line has been inserted into the buffer
   (except replace ! with # below -- I don't want emacs to actually
   interpret this as an MML tag in this bug report!)

<!secure method=smime mode=sign keyfile="/tmp/user.pem">

 * Run M-x debug-on-entry RET smime-sign-buffer RET
       M-x debug-on-entry RET smime-sign-region RET

 * Enter something in To and Subject, and type a body.  Then send with
   C-c C-s.  (We won't actually send the message.)

 * Observe the stack trace:

     smime-sign-buffer("/tmp/user.pem")
     mml-smime-openssl-sign((part (sign . "smime") (keyfile . "/tmp/user.pem") (tag-location . 241) (contents . "test\n")))
     mml-smime-sign((part (sign . "smime") (keyfile . "/tmp/user.pem") (tag-location . 241) (contents . "test\n")))
     mml-smime-sign-buffer((part (sign . "smime") (keyfile . "/tmp/user.pem") (tag-location . 241) (contents . "test\n")))
     mml-generate-mime-1((part (sign . "smime") (keyfile . "/tmp/user.pem") (tag-location . 241) (contents . "test\n")))
     mml-generate-mime()
     message-encode-message-body()
     message-send-mail(nil)
     message-send-via-mail(nil)
     message-send(nil)
     call-interactively(message-send nil nil)

   See how smime-sign-buffer is called with the argument of
   "/tmp/user.pem", which is assigned to the parameter "keyfile".

   Observe this code in smime.el in the smime-sign-buffer function:

                (if keyfile
                    keyfile
                  (smime-get-key-with-certs-by-email
                   (completing-read
                    (concat "Sign using key"
                            (if smime-keys
                                (concat " (default " (caar smime-keys) "): ")
                              ": "))
                    smime-keys nil nil (car-safe (car-safe smime-keys)))))

   If keyfile is set, we use its value.  Otherwise, we call
   smime-get-key-with-certs-by-email.  Here we won't do that since keyfile
   is set.

 * Hit 'c' in the *Backtrace* window and observe the smime-sign-region
   is called with the third argument set to the key file.  From smime.el
   in smime-sign-region, we have:

  (let* ((certfiles (and (cdr-safe keyfile) (cadr keyfile)))
	 (keyfile (or (car-safe keyfile) keyfile))

   so certfiles will be nil, and the extra certificates will be
   ignored.

 * Hit 'q' in the *Backtrace* window to quit without sending the
   message.

 * In the *mail* buffer, remove ' keyfile="/tmp/user.pem"' from the
   buffer so you are left with

<!secure method=smime mode=sign>

   (again with #, not !)

 * Hit C-c C-s to send.

 * This time observe that smime-sign-buffer is nil.

 * Hit 'c' to continue.  This time, you are prompted for a key.  Accept
   the default.

 * Now observe that the third argument to smime-sign-region is
   ("/tmp/user.pem" ("/tmp/chain.pem")), which is what we want.

 * Hit 'q' to exit the debugger.

I'm not sure what the best fix is.  It looks like smime-sign-buffer
would actually do the right thing if called with keyfile as ("addr"
("cert" ...)) instead of just "addr" since it doesn't actually look at
keyfile and since smime-sign-region already accepts keyfile with either
structure.  The main problem would be encoding the additional cert files
as an attribute to the "secure" mml tag so that mml-smime-openssl-sign
could construct the appropriate keyfile argument.  The insertion of the
keyfile attribute is handled by mml-smime-openssl-sign-query.  It
wouldn't be a hard fix to code, but I'm going to resist the temptation
because someone more familiar with the code may have a better fix.  (If
you like this idea and want me to code it and send a patch, I would be
willing to do that.)  Another idea that occurred to me would be to have
an attribute like keyname, rather than keyfile, that would contain the
email address to use as a lookup key in smime-keys.  Since all functions
involved already require smime-keys to be set, this seems like it would
be safe, but I think it would overly complicate the code.  Anyway, I'll
leave the fix to the experts.

Finally, I will point out that the metadata below is specific to my
highly customized environment, but you've seen the recipe above to
reproduce this in emacs -Q.  Also, I will confirm that, by removing the
keyfile attribute to the secure mml tag and just entering the key when
prompted, I have successfully created S/MIME signed messages that can be
read and verified with mainstream mail clients like thunderbird and that
contain the require chain of certificates.

Automatically inserted data appears below.


In GNU Emacs 23.2.1 (x86_64-unknown-linux-gnu, GTK+ Version 2.10.4)
 of 2010-05-25 on motoko.argon.local
Windowing system distributor `The X.Org Foundation', version 11.0.70101000
configured using `configure  '--prefix=/opt/tps/packages/linux.x86_64.rhel5/emacs-23.2-1''

Important settings:
  value of $LC_ALL: nil
  value of $LC_COLLATE: nil
  value of $LC_CTYPE: nil
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: nil
  value of $LANG: en_US.UTF-8
  value of $XMODIFIERS: @im=none
  locale-coding-system: utf-8-unix
  default enable-multibyte-characters: t

Major mode: Message

Minor modes in effect:
  gnus-message-citation-mode: t
  flyspell-mode: t
  mml-mode: t
  which-function-mode: t
  tooltip-mode: t
  mouse-wheel-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  column-number-mode: t
  line-number-mode: t
  abbrev-mode: t

Recent input:
SPC C-g M-x i n f o SPC c o m <tab> m <tab> <backspace> 
SPC SPC C-g l l l l C-x b s m <tab> <return> C-x b 
m <tab> <tab> e <tab> <return> C-s < # s e c u r e 
C-s C-s C-s C-s C-s C-s C-s C-s C-s C-a C-r t a g s 
C-s C-s C-a C-x b * m <tab> <return> C-x b <return> 
C-x b s m <tab> <return> C-s k e y w a C-a M-< s m 
i m C-s s m i m e - s i g n - b u f f e r C-s C-s C-s 
C-s C-a M-x d e b u g SPC o n SPC e SPC <return> s 
m i m e SPC s i g n <tab> b <tab> <return> C-x b * 
m <tab> <return> C-c C-c s SPC <return> d C-n M-f M-f 
M-f M-f M-f M-f M-f M-f M-f M-f M-f M-f M-b M-b M-b 
M-b M-b M-b <switch-frame> C-x b <return> C-x b * m 
<tab> <return> C-n C-n C-n C-n C-n C-n C-n C-n C-n 
C-e C-b SPC k e y f i l e = " a " C-c C-g q <switch-frame> 
q <C-backspace> SPC k e y f i l e - <backspace> = " 
a " C-c C-c s <switch-frame> C-h f m m l SPC s m <tab> 
o p <tab> s <tab> <return> C-x o C-e M-b M-b <return> 
<switch-frame> q C-x b <return> C-x 2 C-x b <return> 
M-x s e n d SPC b <tab> <backspace> e m <tab> C-g C-x 
b s m <tab> C-g M-x b u g SPC SPC SPC C-g M-x r e p 
o r t <tab> b <backspace> <tab> b <tab> <return>

Recent messages:
Sending...
Entering debugger...
Type C-x 1 to delete the help window.
Back to top level.
Auto-saving...done
Quit [2 times]
Making completion list...
Quit
Making completion list...
C-c RET s s runs the command mml-secure-message-sign-smime

Load-path shadows:
/home/jberkenb/elisp/startup hides /opt/tps/packages/linux.x86_64.rhel5/emacs-23.2-1/share/emacs/23.2/lisp/startup

Features:
(shadow warnings emacsbug info debug ansi-color w3m-form w3m-symbol
gnus-async paranoid-backup newcomment fg-lisp-indent find-func apropos
q-valid-addr mail-extr sort multi-isearch mml-smime vc-accurev
gnus-bcklg gnus-cite help-mode view flyspell ispell gnus-draft
parse-time gnus-ml disp-table byte-opt bytecomp byte-compile nndraft
nnmh nnfolder gnus-agent gnus-srvr gnus-score score-mode nnvirtual
gnus-msg gnus-art mm-uu mml2015 epg-config mm-view smime dig nntp
gnus-cache gnus-sum gnus-group gnus-undo nndoc nnoo nnmail mail-source
format-spec gnus-start gnus-spec gnus-int gnus-range gnus-win gnus
gnus-ems vc-svn vc vc-dispatcher w3m browse-url doc-view image-mode
dired timezone w3m-hist w3m-fb w3m-ems w3m-ccl ccl w3m-favicon w3m-image
w3m-proc w3m-util qmime qmime-compose qmime-view which-func imenu
filecache server uniquify compile ange-ftp comint ring message ecomplete
rfc822 mml mml-sec password-cache mm-decode mm-bodies mm-encode mailcap
mail-parse rfc2231 rfc2047 rfc2045 qp ietf-drums mailabbrev nnheader
gnus-util netrc time-date mm-util mail-prsvr gmm-utils mailheader
canlock sha1 hex-util hashcash mail-utils cc-styles cc-align cc-engine
cc-vars cc-defs smtpmail sendmail regexp-opt project advice help-fns
advice-preload jka-compr cus-edit easymenu wid-edit cus-start cus-load
edmacro kmacro cl cl-19 tooltip ediff-hook vc-hooks lisp-float-type
mwheel x-win x-dnd font-setting tool-bar dnd fontset image fringe
lisp-mode register page menu-bar rfn-eshadow timer select scroll-bar
mldrag mouse jit-lock font-lock syntax facemenu font-core frame cham
georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese hebrew greek romanian slovak czech european ethiopic
indian cyrillic chinese case-table epa-hook jka-cmpr-hook help simple
abbrev loaddefs button minibuffer faces cus-face files text-properties
overlay md5 base64 format env code-pages mule custom widget
hashtable-print-readable backquote make-network-process dbusbind
system-font-setting font-render-setting gtk x-toolkit x multi-tty emacs)

bug#6641: 23.2; signing email with smime using mml ignores extra certs in smime-keys

[Lars Magne Ingebrigtsen]

Jay Berkenbilt <ejb@ql.org> writes:

> I'm not sure what the best fix is.  It looks like smime-sign-buffer
> would actually do the right thing if called with keyfile as ("addr"
> ("cert" ...)) instead of just "addr" since it doesn't actually look at
> keyfile and since smime-sign-region already accepts keyfile with either
> structure.

I'm not very familiar with this area.  Could you send a patch that would
fix this?

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

bug#6641: 23.2; signing email with smime using mml ignores extra certs in smime-keys

[Jay Berkenbilt]

Lars Magne Ingebrigtsen <larsi@gnus.org> wrote:

> Jay Berkenbilt <ejb@ql.org> writes:
>
>> I'm not sure what the best fix is.  It looks like smime-sign-buffer
>> would actually do the right thing if called with keyfile as ("addr"
>> ("cert" ...)) instead of just "addr" since it doesn't actually look at
>> keyfile and since smime-sign-region already accepts keyfile with either
>> structure.
>
> I'm not very familiar with this area.  Could you send a patch that would
> fix this?

I'll see what I can do.  I don't really use this functionality much...I
thought I was going to use it and didn't end up using it.  But I think I
understand it and might be able to come up with a patch.  Don't hold
your breath waiting for it though. :-)

-- 
Jay Berkenbilt <ejb@ql.org>

bug#6641: 23.2; signing email with smime using mml ignores extra certs in smime-keys

[Lars Ingebrigtsen]

Jay Berkenbilt <ejb@ql.org> writes:

> I'll see what I can do.  I don't really use this functionality much...I
> thought I was going to use it and didn't end up using it.  But I think I
> understand it and might be able to come up with a patch.  Don't hold
> your breath waiting for it though. :-)

Ok; and meanwhile I'm closing this bug (for now).

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen