SMIME signature creation/verification v2

SMIME signature creation/verification v2

[David Bremner]

This obsoletes 

     id:1439746876-23654-1-git-send-email-david@tethera.net

The main change is to split the creation of the protocol array into
two steps, the first in patch 2/8, so that (hopefully) it builds after
every patch in the series.  I also fixed a few small whitespace issue,
and added the first optional dependency in the debian packaging.

[Patch v2 1/8] crypto: refactor context creation to facilitate further work

[David Bremner]

From: Jani Nikula <jani@nikula.org>

Let the context creation functions decide how to handle multiple calls
and cache the crypto context. No functional changes.
---
 crypto.c | 19 ++++++++++---------
 1 file changed, 10 insertions(+), 9 deletions(-)

diff --git a/crypto.c b/crypto.c
index a6eb27d..1187ad7 100644
--- a/crypto.c
+++ b/crypto.c
@@ -22,14 +22,20 @@
 
 /* Create a GPG context (GMime 2.6) */
 static notmuch_crypto_context_t *
-create_gpg_context (const char *gpgpath)
+create_gpg_context (notmuch_crypto_t *crypto)
 {
     notmuch_crypto_context_t *gpgctx;
 
+    if (crypto->gpgctx)
+	return crypto->gpgctx;
+
     /* TODO: GMimePasswordRequestFunc */
-    gpgctx = g_mime_gpg_context_new (NULL, gpgpath ? gpgpath : "gpg");
-    if (! gpgctx)
+    gpgctx = g_mime_gpg_context_new (NULL, crypto->gpgpath ? crypto->gpgpath : "gpg");
+    if (! gpgctx) {
+	fprintf (stderr, "Failed to construct gpg context.\n");
 	return NULL;
+    }
+    crypto->gpgctx = gpgctx;
 
     g_mime_gpg_context_set_use_agent ((GMimeGpgContext *) gpgctx, TRUE);
     g_mime_gpg_context_set_always_trust ((GMimeGpgContext *) gpgctx, FALSE);
@@ -57,12 +63,7 @@ notmuch_crypto_get_context (notmuch_crypto_t *crypto, const char *protocol)
      */
     if (strcasecmp (protocol, "application/pgp-signature") == 0 ||
 	strcasecmp (protocol, "application/pgp-encrypted") == 0) {
-	if (! crypto->gpgctx) {
-	    crypto->gpgctx = create_gpg_context (crypto->gpgpath);
-	    if (! crypto->gpgctx)
-		fprintf (stderr, "Failed to construct gpg context.\n");
-	}
-	cryptoctx = crypto->gpgctx;
+	cryptoctx = create_gpg_context (crypto);
     } else {
 	fprintf (stderr, "Unknown or unsupported cryptographic protocol.\n");
     }
-- 
2.6.2

[Patch v2 2/8] crypto: make crypto ctx initialization an array

[David Bremner]

From: Jani Nikula <jani@nikula.org>

Make it trivial to add handlers for new protocols without duplicating
code. No functional changes.
---
 crypto.c | 27 +++++++++++++++++++++------
 1 file changed, 21 insertions(+), 6 deletions(-)

diff --git a/crypto.c b/crypto.c
index 1187ad7..da0289d 100644
--- a/crypto.c
+++ b/crypto.c
@@ -43,12 +43,27 @@ create_gpg_context (notmuch_crypto_t *crypto)
     return gpgctx;
 }
 
+static const struct {
+    const char *protocol;
+    notmuch_crypto_context_t *(*get_context) (notmuch_crypto_t *crypto);
+} protocols[] = {
+    {
+	.protocol = "application/pgp-signature",
+	.get_context = create_gpg_context,
+    },
+    {
+	.protocol = "application/pgp-encrypted",
+	.get_context = create_gpg_context,
+    },
+};
+
 /* for the specified protocol return the context pointer (initializing
  * if needed) */
 notmuch_crypto_context_t *
 notmuch_crypto_get_context (notmuch_crypto_t *crypto, const char *protocol)
 {
     notmuch_crypto_context_t *cryptoctx = NULL;
+    size_t i;
 
     if (! protocol) {
 	fprintf (stderr, "Cryptographic protocol is empty.\n");
@@ -61,14 +76,14 @@ notmuch_crypto_get_context (notmuch_crypto_t *crypto, const char *protocol)
      * parameter names as defined in this document are
      * case-insensitive."  Thus, we use strcasecmp for the protocol.
      */
-    if (strcasecmp (protocol, "application/pgp-signature") == 0 ||
-	strcasecmp (protocol, "application/pgp-encrypted") == 0) {
-	cryptoctx = create_gpg_context (crypto);
-    } else {
-	fprintf (stderr, "Unknown or unsupported cryptographic protocol.\n");
+    for (i = 0; i < ARRAY_SIZE (protocols); i++) {
+	if (strcasecmp (protocol, protocols[i].protocol) == 0)
+	    return protocols[i].get_context (crypto);
     }
 
-    return cryptoctx;
+    fprintf (stderr, "Unknown or unsupported cryptographic protocol.\n");
+
+    return NULL;
 }
 
 int
-- 
2.6.2

[Patch v2 3/8] cli: let the user know which protocol is unknown or unsupported

[David Bremner]

From: Jani Nikula <jani@nikula.org>

The current error message is not helpful.
---
 crypto.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/crypto.c b/crypto.c
index da0289d..feae949 100644
--- a/crypto.c
+++ b/crypto.c
@@ -81,7 +81,8 @@ notmuch_crypto_get_context (notmuch_crypto_t *crypto, const char *protocol)
 	    return protocols[i].get_context (crypto);
     }
 
-    fprintf (stderr, "Unknown or unsupported cryptographic protocol.\n");
+    fprintf (stderr, "Unknown or unsupported cryptographic protocol %s.\n",
+	     protocol);
 
     return NULL;
 }
-- 
2.6.2

[Patch v2 4/8] test: initial tests for S/MIME and notmuch-emacs

[David Bremner]

Test the ability of notmuch-mua-mail to send S/MIME signed (and
encrypted) messages; this really relies on existing functionality in
message-mode.

The dependency on openssl to generate keys seems acceptable since
that's the method I got to work for smime signing in emacs.

The generated keys and messages will later be useful for testing the
notmuch CLI.
---
 test/T355-smime.sh             | 53 ++++++++++++++++++++++++++++++++++++++++++
 test/smime/openssl-ca-req.conf | 13 +++++++++++
 test/smime/openssl-req.conf    | 13 +++++++++++
 test/test-lib.el               | 10 ++++++++
 test/test-lib.sh               |  1 +
 5 files changed, 90 insertions(+)
 create mode 100755 test/T355-smime.sh
 create mode 100644 test/smime/openssl-ca-req.conf
 create mode 100644 test/smime/openssl-req.conf

diff --git a/test/T355-smime.sh b/test/T355-smime.sh
new file mode 100755
index 0000000..5f3ff12
--- /dev/null
+++ b/test/T355-smime.sh
@@ -0,0 +1,53 @@
+#!/usr/bin/env bash
+
+test_description='S/MIME signature verification and decryption'
+. ./test-lib.sh
+
+test_require_external_prereq openssl
+
+test_begin_subtest "Generate CA Cert"
+openssl genpkey -algorithm RSA -out ca.key -pass pass:test -des3 1024
+openssl req -new -x509 -key ca.key -passin pass:test \
+	-config $TEST_DIRECTORY/smime/openssl-ca-req.conf -out ca.crt
+test_expect_equal "$(openssl verify ca.crt | tail -1)" "OK"
+
+test_begin_subtest "Generate User Cert"
+openssl genpkey -algorithm RSA  -out smime.key 1024
+openssl req -config $TEST_DIRECTORY/smime/openssl-req.conf \
+	-new -key smime.key -passin pass:test -nodes \
+	-out smime.csr
+openssl x509 -req -in smime.csr -passin pass:test -CA ca.crt -CAkey ca.key -set_serial 1 -out test_suite.crt -setalias "Self Signed SMIME" -addtrust emailProtection -addreject clientAuth -addreject serverAuth -trustout
+# we need one file with the cert and private key
+cat test_suite.crt smime.key > test_suite.pem
+test_expect_equal "$(openssl verify -purpose smimesign -CAfile ca.crt test_suite.pem)" "test_suite.pem: OK"
+
+test_expect_success 'emacs delivery of S/MIME signed message' \
+     'emacs_fcc_message \
+     "test signed message 001" \
+     "This is a test signed message." \
+     "(mml-secure-message-sign \"smime\")"'
+
+# Hard code the MML to avoid several interactive questions
+test_expect_success 'emacs delivery of S/MIME encrypted + signed message' \
+'emacs_fcc_message \
+    "test encrypted message 001" \
+    "<#secure method=smime mode=signencrypt keyfile=\\\"test_suite.pem\\\" certfile=\\\"test_suite.pem\\\">\nThis is a test encrypted message.\n"'
+
+test_begin_subtest "Signature verification (openssl)"
+notmuch show --format=raw subject:"test signed message 001" |\
+    openssl smime -verify -CAfile ca.crt 2>OUTPUT
+cat <<EOF > EXPECTED
+Verification successful
+EOF
+test_expect_equal_file OUTPUT EXPECTED
+
+test_begin_subtest "Decryption and signature verification (openssl)"
+notmuch show --format=raw subject:"test encrypted message 001" |\
+    openssl smime -decrypt -recip test_suite.pem |\
+    openssl smime -verify -CAfile ca.crt 2>OUTPUT
+cat <<EOF > EXPECTED
+Verification successful
+EOF
+test_expect_equal_file OUTPUT EXPECTED
+
+test_done
diff --git a/test/smime/openssl-ca-req.conf b/test/smime/openssl-ca-req.conf
new file mode 100644
index 0000000..49572ee
--- /dev/null
+++ b/test/smime/openssl-ca-req.conf
@@ -0,0 +1,13 @@
+ [ req ]
+ distinguished_name     = req_distinguished_name
+ prompt                 = no
+
+
+ [ req_distinguished_name ]
+ C                      = OZ
+ ST                     = Munchkinlandia
+ L                      = Emerald City
+ O                      = Organization Name
+ OU                     = Dept. of Fake Certs
+ CN                     = Fast Eddies Certs and Chips
+ emailAddress           = fake-ca@example.com
diff --git a/test/smime/openssl-req.conf b/test/smime/openssl-req.conf
new file mode 100644
index 0000000..c6b9de7
--- /dev/null
+++ b/test/smime/openssl-req.conf
@@ -0,0 +1,13 @@
+ [ req ]
+ distinguished_name     = req_distinguished_name
+ prompt                 = no
+
+
+ [ req_distinguished_name ]
+ C                      = OZ
+ ST                     = Munchkinlandia
+ L                      = Emerald City
+ O                      = Not much organization
+ OU                     = Dept. of Testing
+ CN                     = Notmuch Test Suite
+ emailAddress           = test_suite@notmuchmail.org
diff --git a/test/test-lib.el b/test/test-lib.el
index 04c8d63..596a705 100644
--- a/test/test-lib.el
+++ b/test/test-lib.el
@@ -188,3 +188,13 @@ nothing."
 ;; environments
 
 (setq mm-text-html-renderer 'html2text)
+
+;; Set some variables for S/MIME tests.
+
+(setq smime-keys '(("" "test_suite.pem" nil)))
+
+(setq mml-smime-use 'openssl)
+
+;; all test keys are without passphrase
+(eval-after-load 'smime
+  '(defun smime-ask-passphrase (cache)  nil))
diff --git a/test/test-lib.sh b/test/test-lib.sh
index 126911f..2e9a499 100644
--- a/test/test-lib.sh
+++ b/test/test-lib.sh
@@ -1325,4 +1325,5 @@ test_declare_external_prereq emacs
 test_declare_external_prereq ${TEST_EMACSCLIENT}
 test_declare_external_prereq gdb
 test_declare_external_prereq gpg
+test_declare_external_prereq openssl
 test_declare_external_prereq ${NOTMUCH_PYTHON}
-- 
2.6.2

[Patch v2 5/8] test: add broken S/MIME signature verification test for notmuch CLI

[David Bremner]

The test is pretty much cut and paste from the PGP/MIME version, with
obvious updates taken from notmuch output.  This also requires setting
up gpgsm infrastucture.
---
 test/T355-smime.sh | 50 ++++++++++++++++++++++++++++++++++++++++++++++++++
 test/test-lib.sh   |  1 +
 2 files changed, 51 insertions(+)

diff --git a/test/T355-smime.sh b/test/T355-smime.sh
index 5f3ff12..b3cc76e 100755
--- a/test/T355-smime.sh
+++ b/test/T355-smime.sh
@@ -3,7 +3,17 @@
 test_description='S/MIME signature verification and decryption'
 . ./test-lib.sh
 
+add_gpgsm_home ()
+{
+    local output
+    [ -d ${GNUPGHOME} ] && return
+    mkdir -m 0700 "$GNUPGHOME"
+    gpgsm --no-tty --import < test_suite.pem >"$GNUPGHOME"/import.log 2>&1
+    test_debug "cat $GNUPGHOME/import.log"
+}
+
 test_require_external_prereq openssl
+test_require_external_prereq gpgsm
 
 test_begin_subtest "Generate CA Cert"
 openssl genpkey -algorithm RSA -out ca.key -pass pass:test -des3 1024
@@ -21,6 +31,10 @@ openssl x509 -req -in smime.csr -passin pass:test -CA ca.crt -CAkey ca.key -set_
 cat test_suite.crt smime.key > test_suite.pem
 test_expect_equal "$(openssl verify -purpose smimesign -CAfile ca.crt test_suite.pem)" "test_suite.pem: OK"
 
+add_gpgsm_home
+
+FINGERPRINT=$(openssl x509 -fingerprint -in test_suite.crt -noout | sed -e 's/^.*=//' -e s/://g)
+
 test_expect_success 'emacs delivery of S/MIME signed message' \
      'emacs_fcc_message \
      "test signed message 001" \
@@ -41,6 +55,42 @@ Verification successful
 EOF
 test_expect_equal_file OUTPUT EXPECTED
 
+test_begin_subtest "signature verification (notmuch CLI)"
+test_subtest_known_broken
+output=$(notmuch show --format=json --verify subject:"test signed message 001" \
+    | notmuch_json_show_sanitize \
+    | sed -e 's|"created": [1234567890]*|"created": 946728000|' \
+	  -e 's|"expires": [1234567890]*|"expires": 424242424|' )
+expected='[[[{"id": "XXXXX",
+ "match": true,
+ "excluded": false,
+ "filename": "YYYYY",
+ "timestamp": 946728000,
+ "date_relative": "2000-01-01",
+ "tags": ["inbox","signed"],
+ "headers": {"Subject": "test signed message 001",
+ "From": "Notmuch Test Suite <test_suite@notmuchmail.org>",
+ "To": "test_suite@notmuchmail.org",
+ "Date": "Sat, 01 Jan 2000 12:00:00 +0000"},
+ "body": [{"id": 1,
+ "sigstatus": [{"status": "good",
+ "fingerprint": "'$FINGERPRINT'",
+ "expires": 424242424,
+ "created": 946728000}],
+ "content-type": "multipart/signed",
+ "content": [{"id": 2,
+ "content-type": "text/plain",
+ "content": "This is a test signed message.\n"},
+ {"id": 3,
+  "content-length": 1930,
+  "content-transfer-encoding": "base64",
+  "content-type": "application/x-pkcs7-signature",
+  "filename": "smime.p7s"}]}]},
+ []]]]'
+test_expect_equal_json \
+    "$output" \
+    "$expected"
+
 test_begin_subtest "Decryption and signature verification (openssl)"
 notmuch show --format=raw subject:"test encrypted message 001" |\
     openssl smime -decrypt -recip test_suite.pem |\
diff --git a/test/test-lib.sh b/test/test-lib.sh
index 2e9a499..0790698 100644
--- a/test/test-lib.sh
+++ b/test/test-lib.sh
@@ -1326,4 +1326,5 @@ test_declare_external_prereq ${TEST_EMACSCLIENT}
 test_declare_external_prereq gdb
 test_declare_external_prereq gpg
 test_declare_external_prereq openssl
+test_declare_external_prereq gpgsm
 test_declare_external_prereq ${NOTMUCH_PYTHON}
-- 
2.6.2

[Patch v2 6/8] cli: crypto: S/MIME verification support

[David Bremner]

From: Jani Nikula <jani@nikula.org>

notmuch-show --verify will now also process S/MIME multiparts if
encountered. Requires gmime-2.6 and gpgsm.

Based on work by Jameson Graef Rollins <jrollins@finestructure.net>.
---
 crypto.c           | 35 +++++++++++++++++++++++++++++++++++
 notmuch-client.h   |  7 +++++--
 test/T355-smime.sh |  1 -
 3 files changed, 40 insertions(+), 3 deletions(-)

diff --git a/crypto.c b/crypto.c
index feae949..3dabc97 100644
--- a/crypto.c
+++ b/crypto.c
@@ -43,6 +43,28 @@ create_gpg_context (notmuch_crypto_t *crypto)
     return gpgctx;
 }
 
+/* Create a PKCS7 context (GMime 2.6) */
+static notmuch_crypto_context_t *
+create_pkcs7_context (notmuch_crypto_t *crypto)
+{
+    notmuch_crypto_context_t *pkcs7ctx;
+
+    if (crypto->pkcs7ctx)
+	return crypto->pkcs7ctx;
+
+    /* TODO: GMimePasswordRequestFunc */
+    pkcs7ctx = g_mime_pkcs7_context_new (NULL);
+    if (! pkcs7ctx) {
+	fprintf (stderr, "Failed to construct pkcs7 context.\n");
+	return NULL;
+    }
+    crypto->pkcs7ctx = pkcs7ctx;
+
+    g_mime_pkcs7_context_set_always_trust ((GMimePkcs7Context *) pkcs7ctx,
+					   FALSE);
+
+    return pkcs7ctx;
+}
 static const struct {
     const char *protocol;
     notmuch_crypto_context_t *(*get_context) (notmuch_crypto_t *crypto);
@@ -55,6 +77,14 @@ static const struct {
 	.protocol = "application/pgp-encrypted",
 	.get_context = create_gpg_context,
     },
+    {
+	.protocol = "application/pkcs7-signature",
+	.get_context = create_pkcs7_context,
+    },
+    {
+	.protocol = "application/x-pkcs7-signature",
+	.get_context = create_pkcs7_context,
+    },
 };
 
 /* for the specified protocol return the context pointer (initializing
@@ -95,5 +125,10 @@ notmuch_crypto_cleanup (notmuch_crypto_t *crypto)
 	crypto->gpgctx = NULL;
     }
 
+    if (crypto->pkcs7ctx) {
+	g_object_unref (crypto->pkcs7ctx);
+	crypto->pkcs7ctx = NULL;
+    }
+
     return 0;
 }
diff --git a/notmuch-client.h b/notmuch-client.h
index 3bd2903..18e6c60 100644
--- a/notmuch-client.h
+++ b/notmuch-client.h
@@ -31,6 +31,8 @@
 #include <gmime/gmime.h>
 
 typedef GMimeCryptoContext notmuch_crypto_context_t;
+/* This is automatically included only since gmime 2.6.10 */
+#include <gmime/gmime-pkcs7-context.h>
 
 #include "notmuch.h"
 
@@ -70,6 +72,7 @@ typedef struct notmuch_show_format {
 
 typedef struct notmuch_crypto {
     notmuch_crypto_context_t* gpgctx;
+    notmuch_crypto_context_t* pkcs7ctx;
     notmuch_bool_t verify;
     notmuch_bool_t decrypt;
     const char *gpgpath;
@@ -407,8 +410,8 @@ struct mime_node {
 /* Construct a new MIME node pointing to the root message part of
  * message. If crypto->verify is true, signed child parts will be
  * verified. If crypto->decrypt is true, encrypted child parts will be
- * decrypted.  If crypto->gpgctx is NULL, it will be lazily
- * initialized.
+ * decrypted.  If the crypto contexts (crypto->gpgctx or
+ * crypto->pkcs7) are NULL, they will be lazily initialized.
  *
  * Return value:
  *
diff --git a/test/T355-smime.sh b/test/T355-smime.sh
index b3cc76e..caedf5e 100755
--- a/test/T355-smime.sh
+++ b/test/T355-smime.sh
@@ -56,7 +56,6 @@ EOF
 test_expect_equal_file OUTPUT EXPECTED
 
 test_begin_subtest "signature verification (notmuch CLI)"
-test_subtest_known_broken
 output=$(notmuch show --format=json --verify subject:"test signed message 001" \
     | notmuch_json_show_sanitize \
     | sed -e 's|"created": [1234567890]*|"created": 946728000|' \
-- 
2.6.2

[Patch v2 7/8] debian: Recommend gpgsm for S/MIME support

[David Bremner]

From: Jameson Graef Rollins <jrollins@finestructure.net>

---
 debian/control | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/debian/control b/debian/control
index 7e6a548..3e71ee4 100644
--- a/debian/control
+++ b/debian/control
@@ -31,7 +31,7 @@ Vcs-Browser: http://git.notmuchmail.org/git/notmuch
 Package: notmuch
 Architecture: any
 Depends: libnotmuch4 (= ${binary:Version}), ${shlibs:Depends}, ${misc:Depends}
-Recommends: notmuch-emacs | notmuch-vim | notmuch-mutt | alot,  gnupg-agent
+Recommends: notmuch-emacs | notmuch-vim | notmuch-mutt | alot,  gnupg-agent, gpgsm
 Description: thread-based email index, search and tagging
  Notmuch is a system for indexing, searching, reading, and tagging
  large collections of email messages in maildir or mh format. It uses
-- 
2.6.2

[Patch v2 8/8] debian: add gpgsm as build dependency

[David Bremner]

It's not needed for the actual build, but it is needed to run the
SMIME tests; <!nocheck> means it can be omitted if the tests are not
going to be run.
---
 debian/control | 1 +
 1 file changed, 1 insertion(+)

diff --git a/debian/control b/debian/control
index 3e71ee4..d08951c 100644
--- a/debian/control
+++ b/debian/control
@@ -22,6 +22,7 @@ Build-Depends:
  emacs23-nox | emacs23 (>=23~) | emacs23-lucid (>=23~),
  gdb [!s390x !ia64 !armel !ppc64el !mips !mipsel !mips64el],
  dtach (>= 0.8),
+ gpgsm <!nocheck>,
  bash-completion (>=1.9.0~)
 Standards-Version: 3.9.6
 Homepage: http://notmuchmail.org/
-- 
2.6.2