Re: Failing notmuch/SMIME test

[Daniel Kahn Gillmor <dkg@debian.org>]

[-- Attachment #1.1.1: Type: text/plain, Size: 2915 bytes --]

Hi Bremner, thanks for flagging this.

On Fri 2022-03-18 07:48:01 -0300, David Bremner wrote:
> One of the SMIME tests is failing for me.
>
> T355-smime: Testing S/MIME signature verification and decryption
>  FAIL   signature verification (notmuch CLI)
> 	--- T355-smime.4.expected	2022-03-18 10:31:31.877258855 +0000
> 	+++ T355-smime.4.output	2022-03-18 10:31:31.877258855 +0000
> 	@@ -24,7 +24,7 @@
> 	                         "sigstatus": [
> 	                             {
> 	                                 "created": 946728000,
> 	-                                "email": "<test_suite@notmuchmail.org>",
> 	+                                "email": "test_suite@notmuchmail.org",
> 	                                 "expires": 424242424,
> 	                                 "fingerprint": "616F46CD73834C63847756AF0DFB64A6E0972A47",
> 	                                 "status": "good",
> 	@@ -38,7 +38,7 @@
> 	                         "status": [
> 	                             {
> 	                                 "created": 946728000,
> 	-                                "email": "<test_suite@notmuchmail.org>",
> 	+                                "email": "test_suite@notmuchmail.org",
> 	                                 "expires": 424242424,
> 	                                 "fingerprint": "616F46CD73834C63847756AF0DFB64A6E0972A47",
> 	                                 "status": "good",
>
>
> I'm running gpgsm 2.2.27, gpgme 1.16.0, and gmime 3.2.9. At guess the
> change is due to the recent gmime upgrade, but that is pure speculation,
> I could not find anything in the gmime git log to back it up.

I think it is probably due to this squashed changeset in gmime:
https://github.com/jstedfast/gmime/commit/0ab298a0086c09c403b5d35effa73b59f271693d

(yes, this is my own proposed change, weirdly re-structured by github)

The root motivation for this change is some lack of clarity in the
underlying gpgme toolkit, which i've never been able to get resolved:
https://dev.gnupg.org/T5450

> The change looks innocuous enough, but of course it's enough to break
> the test, and I'm not sure how to make this consistent between
> versions.

I suppose the right way to fix this generically is a test which
abstracts out whether gmime reports an angle-addr or a addr-spec for
x.509 certs, and then adjust the tests to match.

I can try to send a patch for this, but it'll take me a while to swap
it all back in.

If anyone wants to propose a patch in the meantime, i'd also be happy to
review.

The simplest thing in the short term is probably to switch the test to
matching based on the bare e-mail address and assert a build-dep on
gmime 3.2.8 (see attached), but that seems a little bit extreme, since
gmime only released 3.2.9 recently (and 3.2.8 never made it out the door
via any formal channels, if i understand the history correctly).

I'll see whether i can make a better fix.

   --dkg


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.1.2: 0001-test-avoid-breakage-with-the-latest-gmime.patch --]
[-- Type: text/x-diff, Size: 2352 bytes --]

From df2b487c5db2af183a75bc32e3a3adf9b90c6316 Mon Sep 17 00:00:00 2001
From: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
Date: Sun, 20 Mar 2022 17:07:13 -0400
Subject: [PATCH] test: avoid breakage with the latest gmime

since gmime 3.2.7, it is now reporting the more stable "email" field from gpgme as the cert info:

https://github.com/jstedfast/gmime/commit/0ab298a0086c09c403b5d35effa73b59f271693d

However, this changes the minimum version of gmime quite dramatically.

A better fix would add a gmime test that normalizes the test results
appropriately, contingent on whether this change is made in gmime, but
i haven't written that test yet.
---
 configure          | 2 +-
 test/T355-smime.sh | 4 ++--
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/configure b/configure
index 36f3f606..d529c358 100755
--- a/configure
+++ b/configure
@@ -481,7 +481,7 @@ if [ ${have_xapian} = "0" ]; then
     errors=$((errors + 1))
 fi
 
-GMIME_MINVER=3.0.3
+GMIME_MINVER=3.2.8
 
 printf "Checking for GMime development files (>= $GMIME_MINVER)... "
 if pkg-config --exists "gmime-3.0 >= $GMIME_MINVER"; then
diff --git a/test/T355-smime.sh b/test/T355-smime.sh
index 31fa4b4e..c25f94f9 100755
--- a/test/T355-smime.sh
+++ b/test/T355-smime.sh
@@ -46,7 +46,7 @@ expected='[[[{"id": "XXXXX",
  "timestamp": 946728000,
  "date_relative": "2000-01-01",
  "tags": ["inbox","signed"],
- "crypto": {"signed": {"status": [{"fingerprint": "'$FINGERPRINT'", "status": "good","userid": "CN=Notmuch Test Suite", "email": "<test_suite@notmuchmail.org>", "expires": 424242424, "created": 946728000}]}},
+ "crypto": {"signed": {"status": [{"fingerprint": "'$FINGERPRINT'", "status": "good","userid": "CN=Notmuch Test Suite", "email": "test_suite@notmuchmail.org", "expires": 424242424, "created": 946728000}]}},
  "headers": {"Subject": "test signed message 001",
  "From": "Notmuch Test Suite <test_suite@notmuchmail.org>",
  "To": "test_suite@notmuchmail.org",
@@ -55,7 +55,7 @@ expected='[[[{"id": "XXXXX",
  "sigstatus": [{"fingerprint": "'$FINGERPRINT'",
  "status": "good",
  "userid": "CN=Notmuch Test Suite",
- "email": "<test_suite@notmuchmail.org>",
+ "email": "test_suite@notmuchmail.org",
  "expires": 424242424,
  "created": 946728000}],
  "content-type": "multipart/signed",
-- 
2.35.1


[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]