T350 test failures with gnupg-2.1.16

T350 test failures with gnupg-2.1.16

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 3597 bytes --]

Hello!

After updating to gnupg 2.1.16, T350-crypto.sh fails in some of the
signature verification steps with wrong content-length:

T350-crypto: Testing PGP/MIME signature verification and decryption
 PASS   emacs delivery of signed message
 FAIL   signature verification
        --- T350-crypto.2.expected      2016-11-22 18:59:48.341851653 +0000
        +++ T350-crypto.2.output        2016-11-22 18:59:48.341851653 +0000
        @@ -11,7 +11,7 @@
                                         "id": 2
                                     },
                                     {
        -                                "content-length": 280,
        +                                "content-length": 312,
                                         "content-type": "application/pgp-signature",
                                         "id": 3
                                     }
 FAIL   signature verification with full owner trust
        --- T350-crypto.3.expected      2016-11-22 18:59:48.393853469 +0000
        +++ T350-crypto.3.output        2016-11-22 18:59:48.393853469 +0000
        @@ -11,7 +11,7 @@
                                         "id": 2
                                     },
                                     {
        -                                "content-length": 280,
        +                                "content-length": 312,
                                         "content-type": "application/pgp-signature",
                                         "id": 3
                                     }
 FAIL   signature verification with signer key unavailable
        --- T350-crypto.4.expected      2016-11-22 18:59:48.445855285 +0000
        +++ T350-crypto.4.output        2016-11-22 18:59:48.445855285 +0000
        @@ -11,7 +11,7 @@
                                         "id": 2
                                     },
                                     {
        -                                "content-length": 280,
        +                                "content-length": 312,
                                         "content-type": "application/pgp-signature",
                                         "id": 3
                                     }
 PASS   emacs delivery of encrypted message with attachment
 PASS   decryption, --format=text
 PASS   decryption, --format=json
 PASS   decryption, --format=json, --part=4
 PASS   decrypt attachment (--part=5 --format=raw)
 PASS   decryption failure with missing key
 PASS   emacs delivery of encrypted + signed message
 PASS   decryption + signature verification
 PASS   reply to encrypted message
 PASS   Reply within emacs to an encrypted message
 FAIL   signature verification with revoked key
        --- T350-crypto.15.expected     2016-11-22 18:59:49.505892318 +0000
        +++ T350-crypto.15.output       2016-11-22 18:59:49.505892318 +0000
        @@ -11,7 +11,7 @@
                                         "id": 2
                                     },
                                     {
        -                                "content-length": 280,
        +                                "content-length": 312,
                                         "content-type": "application/pgp-signature",
                                         "id": 3
                                     }


Downgrading gpg to 2.1.15 makes the tests pass as expected.

Here is the NEWS for 2.1.16:

https://lists.gnu.org/archive/html/info-gnu/2016-11/msg00006.html

Let me know if I can provide any further information. Please CC me in
replies as I'm not subscribed to this list. Thanks!

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 454 bytes --]

Re: T350 test failures with gnupg-2.1.16

[David Bremner]

Marius Bakke <mbakke@fastmail.com> writes:

> Hello!
>
> After updating to gnupg 2.1.16, T350-crypto.sh fails in some of the
> signature verification steps with wrong content-length:
>
> T350-crypto: Testing PGP/MIME signature verification and decryption
>  PASS   emacs delivery of signed message
>  FAIL   signature verification
>         --- T350-crypto.2.expected      2016-11-22 18:59:48.341851653 +0000
>         +++ T350-crypto.2.output        2016-11-22 18:59:48.341851653 +0000
>         @@ -11,7 +11,7 @@
>                                          "id": 2
>                                      },
>                                      {
>         -                                "content-length": 280,
>         +                                "content-length": 312,
>                                          "content-type": "application/pgp-signature",

These failures are not duplicated for me in debian sid, also with gpg
2.1.16. From IRC I believe Marius is running GuixSD.

Re: T350 test failures with gnupg-2.1.16

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 1269 bytes --]

David Bremner <david@tethera.net> writes:

> Marius Bakke <mbakke@fastmail.com> writes:
>
>> Hello!
>>
>> After updating to gnupg 2.1.16, T350-crypto.sh fails in some of the
>> signature verification steps with wrong content-length:
>>
>> T350-crypto: Testing PGP/MIME signature verification and decryption
>>  PASS   emacs delivery of signed message
>>  FAIL   signature verification
>>         --- T350-crypto.2.expected      2016-11-22 18:59:48.341851653 +0000
>>         +++ T350-crypto.2.output        2016-11-22 18:59:48.341851653 +0000
>>         @@ -11,7 +11,7 @@
>>                                          "id": 2
>>                                      },
>>                                      {
>>         -                                "content-length": 280,
>>         +                                "content-length": 312,
>>                                          "content-type": "application/pgp-signature",
>
> These failures are not duplicated for me in debian sid, also with gpg
> 2.1.16. From IRC I believe Marius is running GuixSD.

This is correct. Strange that it's not reproducible on Debian. Any tips
for how to troubleshoot this further? Is the content-length based on
signature only? I'll see if I can extract the raw output somehow.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: T350 test failures with gnupg-2.1.16

[David Bremner]

Marius Bakke <mbakke@fastmail.com> writes:

> David Bremner <david@tethera.net> writes:
>
>> Marius Bakke <mbakke@fastmail.com> writes:
>>
>>> Hello!
>>>
>>> After updating to gnupg 2.1.16, T350-crypto.sh fails in some of the
>>> signature verification steps with wrong content-length:
>>>
>>> T350-crypto: Testing PGP/MIME signature verification and decryption
>>>  PASS   emacs delivery of signed message
>>>  FAIL   signature verification
>>>         --- T350-crypto.2.expected      2016-11-22 18:59:48.341851653 +0000
>>>         +++ T350-crypto.2.output        2016-11-22 18:59:48.341851653 +0000
>>>         @@ -11,7 +11,7 @@
>>>                                          "id": 2
>>>                                      },
>>>                                      {
>>>         -                                "content-length": 280,
>>>         +                                "content-length": 312,
>>>                                          "content-type": "application/pgp-signature",
>>
>> These failures are not duplicated for me in debian sid, also with gpg
>> 2.1.16. From IRC I believe Marius is running GuixSD.
>
> This is correct. Strange that it's not reproducible on Debian. Any tips
> for how to troubleshoot this further? Is the content-length based on
> signature only? I'll see if I can extract the raw output somehow.

You could start with the following, from inside tmp.T350-crypto

% grep -R "Subject: test signed message 001" mail 
% ../../devel/printmimestructure < mail/sent/cur/$the_file_matched_by_grep

That will tell us if the mismatch is in the created file or in the later
parsing.

Re: T350 test failures with gnupg-2.1.16

[Daniel Kahn Gillmor]

[-- Attachment #1: Type: text/plain, Size: 1066 bytes --]

On Tue 2016-11-22 14:22:09 -0500, Marius Bakke wrote:
> T350-crypto: Testing PGP/MIME signature verification and decryption
>  PASS   emacs delivery of signed message
>  FAIL   signature verification
>         --- T350-crypto.2.expected      2016-11-22 18:59:48.341851653 +0000
>         +++ T350-crypto.2.output        2016-11-22 18:59:48.341851653 +0000
>         @@ -11,7 +11,7 @@
>                                          "id": 2
>                                      },
>                                      {
>         -                                "content-length": 280,
>         +                                "content-length": 312,
>                                          "content-type": "application/pgp-signature",
>                                          "id": 3
>                                      }

If you could get me a copy of the actual application/pgp-signature part,
i'd be interested in looking at it.  Unlike bremner, i'm actually able
to duplicate this problem on debian sid, so i'll see what i can figure
out.

        --dkg

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 962 bytes --]

Re: T350 test failures with gnupg-2.1.16

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 3303 bytes --]

David Bremner <david@tethera.net> writes:

> Marius Bakke <mbakke@fastmail.com> writes:
>
>> David Bremner <david@tethera.net> writes:
>>
>>> Marius Bakke <mbakke@fastmail.com> writes:
>>>
>>>> Hello!
>>>>
>>>> After updating to gnupg 2.1.16, T350-crypto.sh fails in some of the
>>>> signature verification steps with wrong content-length:
>>>>
>>>> T350-crypto: Testing PGP/MIME signature verification and decryption
>>>>  PASS   emacs delivery of signed message
>>>>  FAIL   signature verification
>>>>         --- T350-crypto.2.expected      2016-11-22 18:59:48.341851653 +0000
>>>>         +++ T350-crypto.2.output        2016-11-22 18:59:48.341851653 +0000
>>>>         @@ -11,7 +11,7 @@
>>>>                                          "id": 2
>>>>                                      },
>>>>                                      {
>>>>         -                                "content-length": 280,
>>>>         +                                "content-length": 312,
>>>>                                          "content-type": "application/pgp-signature",
>>>
>>> These failures are not duplicated for me in debian sid, also with gpg
>>> 2.1.16. From IRC I believe Marius is running GuixSD.
>>
>> This is correct. Strange that it's not reproducible on Debian. Any tips
>> for how to troubleshoot this further? Is the content-length based on
>> signature only? I'll see if I can extract the raw output somehow.
>
> You could start with the following, from inside tmp.T350-crypto
>
> % grep -R "Subject: test signed message 001" mail 
> % ../../devel/printmimestructure < mail/sent/cur/$the_file_matched_by_grep
>
> That will tell us if the mismatch is in the created file or in the later
> parsing.

Thanks for this! It seems the signature is 32 bytes longer in 2.1.16.

../../devel/printmimestructure < mail/sent/cur/1479841188.2873_194073_1.localhost\:2\,S
└┬╴multipart/signed 778 bytes
 ├─╴text/plain 31 bytes
 └─╴application/pgp-signature [signature.asc] 312 bytes

vs...

../../devel/printmimestructure < mail/sent/cur/1479848474.6836_793177_1.localhost\:2\,S
└┬╴multipart/signed 747 bytes
 ├─╴text/plain 31 bytes
 └─╴application/pgp-signature [signature.asc] 280 bytes

The signatures of each email:

2.1.16:
--=-=-=
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iLMEAQEIAB0WIQRa6rEfXjPc6HXdt1ttkmEtlORjgQUCWDSVpAAKCRBtkmEtlORj
gf90A/4twA6txofm53BhqVAOUwdQNmA2H/yDhP29k6ctZ+XeTw77VZgrFMERoll7
lG6MEsH4JiMasJoevOohRsNmA9F3cEy5b38+c5KuaUlz5jVAKLZ4e8jkZmw2t8L+
hDbtLt7vzd72as8i9yNfKhf1DqAU9ayCJgXOMN4ql/uZqbWIqQ==
=o1L0
-----END PGP SIGNATURE-----
--=-=-=--

2.1.15:
--=-=-=
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iJwEAQEIAAYFAlg0shoACgkQbZJhLZTkY4H4ewQAoWTNwOtQAH/fwlgkqBuWLoWB
7CrrW3Lj1vEVaTRkaBIFP7NiYTDGZtWP6KCZ7G9HXsyprsg5HtVIp3wl4DHKmK/u
XipG0l3PNkSv9+SuUVxI4E9dj0kTJzNLqZaRYf3kmQJTs/jTyxQCuqPd1JF5kD9e
Nkd1585nFCNQAdNJgIE=
=8npB
-----END PGP SIGNATURE-----
--=-=-=--

Reading through the ChangeLog, I can't see anything obviously related.

One workaround could be setting an explicit key algorithm instead of
relying on the default. I'll have a go at this.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: T350 test failures with gnupg-2.1.16

[Daniel Kahn Gillmor]

[-- Attachment #1: Type: text/plain, Size: 1961 bytes --]

On Tue 2016-11-22 15:49:37 -0500, Daniel Kahn Gillmor wrote:
> On Tue 2016-11-22 14:22:09 -0500, Marius Bakke wrote:
>> T350-crypto: Testing PGP/MIME signature verification and decryption
>>  PASS   emacs delivery of signed message
>>  FAIL   signature verification
>>         --- T350-crypto.2.expected      2016-11-22 18:59:48.341851653 +0000
>>         +++ T350-crypto.2.output        2016-11-22 18:59:48.341851653 +0000
>>         @@ -11,7 +11,7 @@
>>                                          "id": 2
>>                                      },
>>                                      {
>>         -                                "content-length": 280,
>>         +                                "content-length": 312,
>>                                          "content-type": "application/pgp-signature",
>>                                          "id": 3
>>                                      }
>
> If you could get me a copy of the actual application/pgp-signature part,
> i'd be interested in looking at it.  Unlike bremner, i'm actually able
> to duplicate this problem on debian sid, so i'll see what i can figure
> out.

OK, the difference here is that 2.1.16 is automatically including the
full OpenPGP v4 fingerprint in the message signature.  This is part of
the ongoing discussion around revisions to the OpenPGP standard, and it
makes it easier for a mail user agent to tell whether it's missing the
key for verification or whether the signature is just bad.

so the length of the signature is extended by about 23 octets (1 octet
of subpacket length; 1 octet of subpacket type, 1 octet of fpr versio
number, and 20-octets of fingerprint), which becomes about 32 octets
after base64 encoding, hence the increase in content-length from 280 to
312 octets.

As for how to fix it -- i guess the right thing would be to make that
number variable -- as long as the signature is non-zero and it
validates, i think it'd be fine.

           --dkg

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 962 bytes --]

Re: T350 test failures with gnupg-2.1.16

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 2288 bytes --]

Daniel Kahn Gillmor <dkg@fifthhorseman.net> writes:

> On Tue 2016-11-22 15:49:37 -0500, Daniel Kahn Gillmor wrote:
>> On Tue 2016-11-22 14:22:09 -0500, Marius Bakke wrote:
>>> T350-crypto: Testing PGP/MIME signature verification and decryption
>>>  PASS   emacs delivery of signed message
>>>  FAIL   signature verification
>>>         --- T350-crypto.2.expected      2016-11-22 18:59:48.341851653 +0000
>>>         +++ T350-crypto.2.output        2016-11-22 18:59:48.341851653 +0000
>>>         @@ -11,7 +11,7 @@
>>>                                          "id": 2
>>>                                      },
>>>                                      {
>>>         -                                "content-length": 280,
>>>         +                                "content-length": 312,
>>>                                          "content-type": "application/pgp-signature",
>>>                                          "id": 3
>>>                                      }
>>
>> If you could get me a copy of the actual application/pgp-signature part,
>> i'd be interested in looking at it.  Unlike bremner, i'm actually able
>> to duplicate this problem on debian sid, so i'll see what i can figure
>> out.
>
> OK, the difference here is that 2.1.16 is automatically including the
> full OpenPGP v4 fingerprint in the message signature.  This is part of
> the ongoing discussion around revisions to the OpenPGP standard, and it
> makes it easier for a mail user agent to tell whether it's missing the
> key for verification or whether the signature is just bad.
>
> so the length of the signature is extended by about 23 octets (1 octet
> of subpacket length; 1 octet of subpacket type, 1 octet of fpr versio
> number, and 20-octets of fingerprint), which becomes about 32 octets
> after base64 encoding, hence the increase in content-length from 280 to
> 312 octets.
>
> As for how to fix it -- i guess the right thing would be to make that
> number variable -- as long as the signature is non-zero and it
> validates, i think it'd be fine.

Wow, good catch. I was about to bisect gnupg to figure out what changed
this behaviour..

I'm not familiar enough with the notmuch test framework to suggest a
fix, but glad to know this is not specific to Guix.

Thanks a lot for your help!

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

[PATCH] tests: account for varying-size OpenPGP signatures

[Daniel Kahn Gillmor]

GnuPG 2.1.16 is now injecting the full issuer fingerprint in its
signatures, which makes them about 32 octets larger when
ascii-armored.

This change in size means that the size of the MIME parts will vary
depending on the version of gpg that the user has installed.  at any
rate, the signature part should be non-zero, so we just test for that
instead of an exact size.
---
 test/T350-crypto.sh | 20 ++++++++++++--------
 1 file changed, 12 insertions(+), 8 deletions(-)

diff --git a/test/T350-crypto.sh b/test/T350-crypto.sh
index df2dc74..38f984f 100755
--- a/test/T350-crypto.sh
+++ b/test/T350-crypto.sh
@@ -37,7 +37,8 @@ test_expect_success 'emacs delivery of signed message' \
 test_begin_subtest "signature verification"
 output=$(notmuch show --format=json --verify subject:"test signed message 001" \
     | notmuch_json_show_sanitize \
-    | sed -e 's|"created": [1234567890]*|"created": 946728000|')
+    | sed -e 's|"created": [1234567890]*|"created": 946728000|' \
+          -e 's/"content-length": [1-9][0-9]*/"content-length": "NONZERO"/')
 expected='[[[{"id": "XXXXX",
  "match": true,
  "excluded": false,
@@ -59,7 +60,7 @@ expected='[[[{"id": "XXXXX",
  "content": "This is a test signed message.\n"},
  {"id": 3,
  "content-type": "application/pgp-signature",
- "content-length": 280}]}]},
+ "content-length": "NONZERO"}]}]},
  []]]]'
 test_expect_equal_json \
     "$output" \
@@ -71,7 +72,8 @@ echo "${FINGERPRINT}:6:" | gpg --no-tty --import-ownertrust >>"$GNUPGHOME"/trust
 gpg --no-tty --check-trustdb >>"$GNUPGHOME"/trust.log 2>&1
 output=$(notmuch show --format=json --verify subject:"test signed message 001" \
     | notmuch_json_show_sanitize \
-    | sed -e 's|"created": [1234567890]*|"created": 946728000|')
+    | sed -e 's|"created": [1234567890]*|"created": 946728000|'\
+          -e 's/"content-length": [1-9][0-9]*/"content-length": "NONZERO"/')
 expected='[[[{"id": "XXXXX",
  "match": true,
  "excluded": false,
@@ -94,7 +96,7 @@ expected='[[[{"id": "XXXXX",
  "content": "This is a test signed message.\n"},
  {"id": 3,
  "content-type": "application/pgp-signature",
- "content-length": 280}]}]},
+ "content-length": "NONZERO"}]}]},
  []]]]'
 test_expect_equal_json \
     "$output" \
@@ -105,7 +107,8 @@ test_begin_subtest "signature verification with signer key unavailable"
 mv "${GNUPGHOME}"{,.bak}
 output=$(notmuch show --format=json --verify subject:"test signed message 001" \
     | notmuch_json_show_sanitize \
-    | sed -e 's|"created": [1234567890]*|"created": 946728000|')
+    | sed -e 's|"created": [1234567890]*|"created": 946728000|' \
+          -e 's/"content-length": [1-9][0-9]*/"content-length": "NONZERO"/')
 expected='[[[{"id": "XXXXX",
  "match": true,
  "excluded": false,
@@ -127,7 +130,7 @@ expected='[[[{"id": "XXXXX",
  "content": "This is a test signed message.\n"},
  {"id": 3,
  "content-type": "application/pgp-signature",
- "content-length": 280}]}]},
+ "content-length": "NONZERO"}]}]},
  []]]]'
 test_expect_equal_json \
     "$output" \
@@ -348,7 +351,8 @@ y
     | gpg --no-tty --quiet --import
 output=$(notmuch show --format=json --verify subject:"test signed message 001" \
     | notmuch_json_show_sanitize \
-    | sed -e 's|"created": [1234567890]*|"created": 946728000|')
+    | sed -e 's|"created": [1234567890]*|"created": 946728000|' \
+          -e 's/"content-length": [1-9][0-9]*/"content-length": "NONZERO"/')
 expected='[[[{"id": "XXXXX",
  "match": true,
  "excluded": false,
@@ -370,7 +374,7 @@ expected='[[[{"id": "XXXXX",
  "content": "This is a test signed message.\n"},
  {"id": 3,
  "content-type": "application/pgp-signature",
- "content-length": 280}]}]},
+ "content-length": "NONZERO"}]}]},
  []]]]'
 test_expect_equal_json \
     "$output" \
-- 
2.10.2

Re: [PATCH] tests: account for varying-size OpenPGP signatures

[David Bremner]

Daniel Kahn Gillmor <dkg@fifthhorseman.net> writes:

> @@ -37,7 +37,8 @@ test_expect_success 'emacs delivery of signed message' \
>  test_begin_subtest "signature verification"
>  output=$(notmuch show --format=json --verify subject:"test signed message 001" \
>      | notmuch_json_show_sanitize \
> -    | sed -e 's|"created": [1234567890]*|"created": 946728000|')
> +    | sed -e 's|"created": [1234567890]*|"created": 946728000|' \
> +          -e 's/"content-length": [1-9][0-9]*/"content-length": "NONZERO"/')

there are a bunch of sed oneliners (or perl equivalent) collected in
test-lib.sh as functions test_*_sanitize. I wonder if that would be
worthwhile here, to have one place to update regexps etc...  Something
in the style of notmuch_show_sanitize wrapping a call to
notmuch_json_show_sanitize

[PATCH v2] tests: account for varying-size cryptographic signatures

[Daniel Kahn Gillmor]

GnuPG 2.1.16 is now injecting the full issuer fingerprint in its
signatures, which makes them about 32 octets larger when
ascii-armored.

This change in size means that the size of the MIME parts will vary
depending on the version of gpg that the user has installed.  at any
rate, the signature part should be non-zero (this is true for
basically any MIME part), so we just test for that instead of an exact
size.
---
 test/T350-crypto.sh | 22 ++++++++++------------
 test/T355-smime.sh  |  2 +-
 test/test-lib.sh    |  3 ++-
 3 files changed, 13 insertions(+), 14 deletions(-)

diff --git a/test/T350-crypto.sh b/test/T350-crypto.sh
index df2dc74..a1e5e20 100755
--- a/test/T350-crypto.sh
+++ b/test/T350-crypto.sh
@@ -59,7 +59,7 @@ expected='[[[{"id": "XXXXX",
  "content": "This is a test signed message.\n"},
  {"id": 3,
  "content-type": "application/pgp-signature",
- "content-length": 280}]}]},
+ "content-length": "NONZERO"}]}]},
  []]]]'
 test_expect_equal_json \
     "$output" \
@@ -94,7 +94,7 @@ expected='[[[{"id": "XXXXX",
  "content": "This is a test signed message.\n"},
  {"id": 3,
  "content-type": "application/pgp-signature",
- "content-length": 280}]}]},
+ "content-length": "NONZERO"}]}]},
  []]]]'
 test_expect_equal_json \
     "$output" \
@@ -127,7 +127,7 @@ expected='[[[{"id": "XXXXX",
  "content": "This is a test signed message.\n"},
  {"id": 3,
  "content-type": "application/pgp-signature",
- "content-length": 280}]}]},
+ "content-length": "NONZERO"}]}]},
  []]]]'
 test_expect_equal_json \
     "$output" \
@@ -197,7 +197,7 @@ expected='[[[{"id": "XXXXX",
  "content-type": "multipart/encrypted",
  "content": [{"id": 2,
  "content-type": "application/pgp-encrypted",
- "content-length": 11},
+ "content-length": "NONZERO"},
  {"id": 3,
  "content-type": "multipart/mixed",
  "content": [{"id": 4,
@@ -205,7 +205,7 @@ expected='[[[{"id": "XXXXX",
  "content": "This is a test encrypted message.\n"},
  {"id": 5,
  "content-type": "application/octet-stream",
- "content-length": 28,
+ "content-length": "NONZERO",
  "content-transfer-encoding": "base64",
  "filename": "TESTATTACHMENT"}]}]}]},
  []]]]'
@@ -234,11 +234,9 @@ test_expect_equal_file OUTPUT TESTATTACHMENT
 
 test_begin_subtest "decryption failure with missing key"
 mv "${GNUPGHOME}"{,.bak}
-# The length of the encrypted attachment varies so must be normalized.
 output=$(notmuch show --format=json --decrypt subject:"test encrypted message 001" \
     | notmuch_json_show_sanitize \
-    | sed -e 's|"created": [1234567890]*|"created": 946728000|' \
-    | sed -e 's|"content-length": 6[1234567890]*|"content-length": 652|')
+    | sed -e 's|"created": [1234567890]*|"created": 946728000|')
 expected='[[[{"id": "XXXXX",
  "match": true,
  "excluded": false,
@@ -255,10 +253,10 @@ expected='[[[{"id": "XXXXX",
  "content-type": "multipart/encrypted",
  "content": [{"id": 2,
  "content-type": "application/pgp-encrypted",
- "content-length": 11},
+ "content-length": "NONZERO"},
  {"id": 3,
  "content-type": "application/octet-stream",
- "content-length": 652}]}]},
+ "content-length": "NONZERO"}]}]},
  []]]]'
 test_expect_equal_json \
     "$output" \
@@ -295,7 +293,7 @@ expected='[[[{"id": "XXXXX",
  "content-type": "multipart/encrypted",
  "content": [{"id": 2,
  "content-type": "application/pgp-encrypted",
- "content-length": 11},
+ "content-length": "NONZERO"},
  {"id": 3,
  "content-type": "text/plain",
  "content": "This is another test encrypted message.\n"}]}]},
@@ -370,7 +368,7 @@ expected='[[[{"id": "XXXXX",
  "content": "This is a test signed message.\n"},
  {"id": 3,
  "content-type": "application/pgp-signature",
- "content-length": 280}]}]},
+ "content-length": "NONZERO"}]}]},
  []]]]'
 test_expect_equal_json \
     "$output" \
diff --git a/test/T355-smime.sh b/test/T355-smime.sh
index d942412..a8be45e 100755
--- a/test/T355-smime.sh
+++ b/test/T355-smime.sh
@@ -69,7 +69,7 @@ expected='[[[{"id": "XXXXX",
  "content-type": "text/plain",
  "content": "This is a test signed message.\n"},
  {"id": 3,
-  "content-length": 1922,
+  "content-length": "NONZERO",
   "content-transfer-encoding": "base64",
   "content-type": "application/x-pkcs7-signature",
   "filename": "smime.p7s"}]}]},
diff --git a/test/test-lib.sh b/test/test-lib.sh
index a12c6d0..f55d2c6 100644
--- a/test/test-lib.sh
+++ b/test/test-lib.sh
@@ -736,7 +736,8 @@ notmuch_json_show_sanitize ()
 	-e 's|"Date": "Fri, 05 Jan 2001 [^"]*0000"|"Date": "GENERATED_DATE"|g' \
 	-e 's|"filename": "signature.asc",||g' \
 	-e 's|"filename": "/[^"]*",|"filename": "YYYYY",|g' \
-	-e 's|"timestamp": 97.......|"timestamp": 42|g'
+	-e 's|"timestamp": 97.......|"timestamp": 42|g' \
+        -e 's|"content-length": [1-9][0-9]*|"content-length": "NONZERO"|g'
 }
 
 notmuch_emacs_error_sanitize ()
-- 
2.10.2

Re: [PATCH] tests: account for varying-size OpenPGP signatures

[Daniel Kahn Gillmor]

On Wed 2016-11-23 11:59:38 -0500, David Bremner wrote:
> there are a bunch of sed oneliners (or perl equivalent) collected in
> test-lib.sh as functions test_*_sanitize. I wonder if that would be
> worthwhile here, to have one place to update regexps etc...  Something
> in the style of notmuch_show_sanitize wrapping a call to
> notmuch_json_show_sanitize

Thanks for the suggestion, please see v2 of this patch.

     --dkg

Re: [PATCH v2] tests: account for varying-size cryptographic signatures

[David Bremner]

Daniel Kahn Gillmor <dkg@fifthhorseman.net> writes:

> GnuPG 2.1.16 is now injecting the full issuer fingerprint in its
> signatures, which makes them about 32 octets larger when
> ascii-armored.
>
> This change in size means that the size of the MIME parts will vary
> depending on the version of gpg that the user has installed.  at any
> rate, the signature part should be non-zero (this is true for
> basically any MIME part), so we just test for that instead of an exact
> size.

I've pushed Daniel's patch to release and master.

d