unofficial mirror of notmuch@notmuchmail.org
 help / color / mirror / code / Atom feed
* [PATCH] test: Add test for messages with missing headers
@ 2012-08-07 21:21 Austin Clements
  2012-08-07 22:43 ` Mark Walters
  2012-08-08  1:28 ` [PATCH v2] " Austin Clements
  0 siblings, 2 replies; 12+ messages in thread
From: Austin Clements @ 2012-08-07 21:21 UTC (permalink / raw)
  To: notmuch

Currently the JSON tests for search and show are broken because
notmuch attempts to dereference a NULL pointer.
---
Things to bikeshed:

* Should we include From and Subject in the headers object when there
  are no from or subject headers?  Currently the schema says that
  everything but those two and "Date" is optional (indeed, "To" is
  missing from the second message) but that was just post facto
  standardization.

* How should we format expected JSON in the test suite, now that we
  can format it however we want?  Here I've just pasted in the result
  of python -mjson.tool.  While that was very easy and the result is
  quite readable, it's the antithesis of compact and the keys have
  been alphabetized.

 test/missing-headers |  162 ++++++++++++++++++++++++++++++++++++++++++++++++++
 test/notmuch-test    |    1 +
 2 files changed, 163 insertions(+)
 create mode 100755 test/missing-headers

diff --git a/test/missing-headers b/test/missing-headers
new file mode 100755
index 0000000..744c04e
--- /dev/null
+++ b/test/missing-headers
@@ -0,0 +1,162 @@
+#!/usr/bin/env bash
+test_description='messages with missing headers'
+. ./test-lib.sh
+
+# Notmuch requires at least one of from, subject, or to or it will
+# ignore the file.  Generate two messages so that together they cover
+# all possible missing headers.  We also give one of the messages a
+# date to ensure stable result ordering.
+
+cat <<EOF > "${MAIL_DIR}/msg-2"
+To: Notmuch Test Suite <test_suite@notmuchmail.org>
+Date: Fri, 05 Jan 2001 15:43:57 +0000
+
+Body
+EOF
+
+cat <<EOF > "${MAIL_DIR}/msg-1"
+From: Notmuch Test Suite <test_suite@notmuchmail.org>
+
+Body
+EOF
+
+NOTMUCH_NEW
+
+test_begin_subtest "Search: text"
+output=$(notmuch search '*' | notmuch_search_sanitize)
+test_expect_equal "$output" "\
+thread:XXX   2001-01-05 [1/1] (null);  (inbox unread)
+thread:XXX   1970-01-01 [1/1] Notmuch Test Suite;  (inbox unread)"
+
+test_begin_subtest "Search: json"
+test_subtest_known_broken
+output=$(notmuch search --format=json '*' | notmuch_search_sanitize)
+test_expect_equal_json "$output" '
+[
+    {
+        "authors": "",
+        "date_relative": "2001-01-05",
+        "matched": 1,
+        "subject": "",
+        "tags": [
+            "inbox",
+            "unread"
+        ],
+        "thread": "XXX",
+        "timestamp": 978709437,
+        "total": 1
+    },
+    {
+        "authors": "Notmuch Test Suite",
+        "date_relative": "1970-01-01",
+        "matched": 1,
+        "subject": "",
+        "tags": [
+            "inbox",
+            "unread"
+        ],
+        "thread": "XXX",
+        "timestamp": 0,
+        "total": 1
+    }
+]'
+
+test_begin_subtest "Show: text"
+output=$(notmuch show '*')
+test_expect_equal "$output" "\
+\fmessage{ id:notmuch-sha1-7a6e4eac383ef958fcd3ebf2143db71b8ff01161 depth:0 match:1 excluded:0 filename:/tmp/nmtest/tmp.missing-headers/mail/msg-2
+\fheader{
+ (2001-01-05) (inbox unread)
+Subject: (null)
+From: (null)
+To: Notmuch Test Suite <test_suite@notmuchmail.org>
+Date: Fri, 05 Jan 2001 15:43:57 +0000
+\fheader}
+\fbody{
+\fpart{ ID: 1, Content-type: text/plain
+Body
+\fpart}
+\fbody}
+\fmessage}
+\fmessage{ id:notmuch-sha1-ca55943aff7a72baf2ab21fa74fab3d632401334 depth:0 match:1 excluded:0 filename:/tmp/nmtest/tmp.missing-headers/mail/msg-1
+\fheader{
+Notmuch Test Suite <test_suite@notmuchmail.org> (1970-01-01) (inbox unread)
+Subject: (null)
+From: Notmuch Test Suite <test_suite@notmuchmail.org>
+Date: Thu, 01 Jan 1970 00:00:00 +0000
+\fheader}
+\fbody{
+\fpart{ ID: 1, Content-type: text/plain
+Body
+\fpart}
+\fbody}
+\fmessage}"
+
+test_begin_subtest "Show: json"
+test_subtest_known_broken
+output=$(notmuch show --format=json '*' | notmuch_json_show_sanitize)
+test_expect_equal_json "$output" '
+[
+    [
+        [
+            {
+                "body": [
+                    {
+                        "content": "Body\n",
+                        "content-type": "text/plain",
+                        "id": 1
+                    }
+                ],
+                "date_relative": "2001-01-05",
+                "excluded": false,
+                "filename": "YYYYY",
+                "headers": {
+                    "Date": "Fri, 05 Jan 2001 15:43:57 +0000",
+                    "From": "",
+                    "Subject": "",
+                    "To": "Notmuch Test Suite <test_suite@notmuchmail.org>"
+                },
+                "id": "XXXXX",
+                "match": true,
+                "tags": [
+                    "inbox",
+                    "unread"
+                ],
+                "timestamp": 978709437
+            },
+            []
+        ]
+    ],
+    [
+        [
+            {
+                "body": [
+                    {
+                        "content": "Body\n",
+                        "content-type": "text/plain",
+                        "id": 1
+                    }
+                ],
+                "date_relative": "1970-01-01",
+                "excluded": false,
+                "filename": "YYYYY",
+                "headers": {
+                    "Date": "Thu, 01 Jan 1970 00:00:00 +0000",
+                    "From": "Notmuch Test Suite <test_suite@notmuchmail.org>",
+                    "Subject": ""
+                },
+                "id": "XXXXX",
+                "match": true,
+                "tags": [
+                    "inbox",
+                    "unread"
+                ],
+                "timestamp": 0
+            },
+            []
+        ]
+    ]
+]'
+
+
+test_done
diff --git a/test/notmuch-test b/test/notmuch-test
index ea39dfc..cc732c3 100755
--- a/test/notmuch-test
+++ b/test/notmuch-test
@@ -59,6 +59,7 @@ TESTS="
   emacs-address-cleaning
   emacs-hello
   emacs-show
+  missing-headers
 "
 TESTS=${NOTMUCH_TESTS:=$TESTS}
 
-- 
1.7.10

^ permalink raw reply related	[flat|nested] 12+ messages in thread

* Re: [PATCH] test: Add test for messages with missing headers
  2012-08-07 21:21 [PATCH] test: Add test for messages with missing headers Austin Clements
@ 2012-08-07 22:43 ` Mark Walters
  2012-08-08  1:28 ` [PATCH v2] " Austin Clements
  1 sibling, 0 replies; 12+ messages in thread
From: Mark Walters @ 2012-08-07 22:43 UTC (permalink / raw)
  To: Austin Clements, notmuch


Hi 

This generally looks good but there is a bug in the test from a
hardcoded /tmp/ path (see below). And as confirmation with the patch in
id:"874noe1o0r.fsf@qmul.ac.uk" the tests (modulo the test bug) pass.

On Tue, 07 Aug 2012, Austin Clements <amdragon@MIT.EDU> wrote:
> Currently the JSON tests for search and show are broken because
> notmuch attempts to dereference a NULL pointer.
> ---
> Things to bikeshed:
>
> * Should we include From and Subject in the headers object when there
>   are no from or subject headers?  Currently the schema says that
>   everything but those two and "Date" is optional (indeed, "To" is
>   missing from the second message) but that was just post facto
>   standardization.

I think Date and From are compulsory in an email but the others are not
(but I am unsure which Date and which From so that may be unhelpful).
If I am correct it might be sensible to always include those two.

> * How should we format expected JSON in the test suite, now that we
>   can format it however we want?  Here I've just pasted in the result
>   of python -mjson.tool.  While that was very easy and the result is
>   quite readable, it's the antithesis of compact and the keys have
>   been alphabetized.

I like this: making the tests readable is a big plus.

>  test/missing-headers |  162 ++++++++++++++++++++++++++++++++++++++++++++++++++
>  test/notmuch-test    |    1 +
>  2 files changed, 163 insertions(+)
>  create mode 100755 test/missing-headers
>
> diff --git a/test/missing-headers b/test/missing-headers
> new file mode 100755
> index 0000000..744c04e
> --- /dev/null
> +++ b/test/missing-headers
> @@ -0,0 +1,162 @@
> +#!/usr/bin/env bash
> +test_description='messages with missing headers'
> +. ./test-lib.sh
> +
> +# Notmuch requires at least one of from, subject, or to or it will
> +# ignore the file.  Generate two messages so that together they cover
> +# all possible missing headers.  We also give one of the messages a
> +# date to ensure stable result ordering.
> +
> +cat <<EOF > "${MAIL_DIR}/msg-2"
> +To: Notmuch Test Suite <test_suite@notmuchmail.org>
> +Date: Fri, 05 Jan 2001 15:43:57 +0000
> +
> +Body
> +EOF
> +
> +cat <<EOF > "${MAIL_DIR}/msg-1"
> +From: Notmuch Test Suite <test_suite@notmuchmail.org>
> +
> +Body
> +EOF
> +
> +NOTMUCH_NEW
> +
> +test_begin_subtest "Search: text"
> +output=$(notmuch search '*' | notmuch_search_sanitize)
> +test_expect_equal "$output" "\
> +thread:XXX   2001-01-05 [1/1] (null);  (inbox unread)
> +thread:XXX   1970-01-01 [1/1] Notmuch Test Suite;  (inbox unread)"
> +
> +test_begin_subtest "Search: json"
> +test_subtest_known_broken
> +output=$(notmuch search --format=json '*' | notmuch_search_sanitize)
> +test_expect_equal_json "$output" '
> +[
> +    {
> +        "authors": "",
> +        "date_relative": "2001-01-05",
> +        "matched": 1,
> +        "subject": "",
> +        "tags": [
> +            "inbox",
> +            "unread"
> +        ],
> +        "thread": "XXX",
> +        "timestamp": 978709437,
> +        "total": 1
> +    },
> +    {
> +        "authors": "Notmuch Test Suite",
> +        "date_relative": "1970-01-01",
> +        "matched": 1,
> +        "subject": "",
> +        "tags": [
> +            "inbox",
> +            "unread"
> +        ],
> +        "thread": "XXX",
> +        "timestamp": 0,
> +        "total": 1
> +    }
> +]'
> +
> +test_begin_subtest "Show: text"
> +output=$(notmuch show '*')
> +test_expect_equal "$output" "\
> +\fmessage{ id:notmuch-sha1-7a6e4eac383ef958fcd3ebf2143db71b8ff01161 depth:0 match:1 excluded:0 filename:/tmp/nmtest/tmp.missing-headers/mail/msg-2

The filename above has not been sanitised so contains your tmp path.

Best wishes

Mark

> +\fheader{
> + (2001-01-05) (inbox unread)
> +Subject: (null)
> +From: (null)
> +To: Notmuch Test Suite <test_suite@notmuchmail.org>
> +Date: Fri, 05 Jan 2001 15:43:57 +0000
> +\fheader}
> +\fbody{
> +\fpart{ ID: 1, Content-type: text/plain
> +Body
> +\fpart}
> +\fbody}
> +\fmessage}
> +\fmessage{ id:notmuch-sha1-ca55943aff7a72baf2ab21fa74fab3d632401334 depth:0 match:1 excluded:0 filename:/tmp/nmtest/tmp.missing-headers/mail/msg-1
> +\fheader{
> +Notmuch Test Suite <test_suite@notmuchmail.org> (1970-01-01) (inbox unread)
> +Subject: (null)
> +From: Notmuch Test Suite <test_suite@notmuchmail.org>
> +Date: Thu, 01 Jan 1970 00:00:00 +0000
> +\fheader}
> +\fbody{
> +\fpart{ ID: 1, Content-type: text/plain
> +Body
> +\fpart}
> +\fbody}
> +\fmessage}"
> +
> +test_begin_subtest "Show: json"
> +test_subtest_known_broken
> +output=$(notmuch show --format=json '*' | notmuch_json_show_sanitize)
> +test_expect_equal_json "$output" '
> +[
> +    [
> +        [
> +            {
> +                "body": [
> +                    {
> +                        "content": "Body\n",
> +                        "content-type": "text/plain",
> +                        "id": 1
> +                    }
> +                ],
> +                "date_relative": "2001-01-05",
> +                "excluded": false,
> +                "filename": "YYYYY",
> +                "headers": {
> +                    "Date": "Fri, 05 Jan 2001 15:43:57 +0000",
> +                    "From": "",
> +                    "Subject": "",
> +                    "To": "Notmuch Test Suite <test_suite@notmuchmail.org>"
> +                },
> +                "id": "XXXXX",
> +                "match": true,
> +                "tags": [
> +                    "inbox",
> +                    "unread"
> +                ],
> +                "timestamp": 978709437
> +            },
> +            []
> +        ]
> +    ],
> +    [
> +        [
> +            {
> +                "body": [
> +                    {
> +                        "content": "Body\n",
> +                        "content-type": "text/plain",
> +                        "id": 1
> +                    }
> +                ],
> +                "date_relative": "1970-01-01",
> +                "excluded": false,
> +                "filename": "YYYYY",
> +                "headers": {
> +                    "Date": "Thu, 01 Jan 1970 00:00:00 +0000",
> +                    "From": "Notmuch Test Suite <test_suite@notmuchmail.org>",
> +                    "Subject": ""
> +                },
> +                "id": "XXXXX",
> +                "match": true,
> +                "tags": [
> +                    "inbox",
> +                    "unread"
> +                ],
> +                "timestamp": 0
> +            },
> +            []
> +        ]
> +    ]
> +]'
> +
> +
> +test_done
> diff --git a/test/notmuch-test b/test/notmuch-test
> index ea39dfc..cc732c3 100755
> --- a/test/notmuch-test
> +++ b/test/notmuch-test
> @@ -59,6 +59,7 @@ TESTS="
>    emacs-address-cleaning
>    emacs-hello
>    emacs-show
> +  missing-headers
>  "
>  TESTS=${NOTMUCH_TESTS:=$TESTS}
>  
> -- 
> 1.7.10
>
> _______________________________________________
> notmuch mailing list
> notmuch@notmuchmail.org
> http://notmuchmail.org/mailman/listinfo/notmuch

^ permalink raw reply	[flat|nested] 12+ messages in thread

* [PATCH v2] test: Add test for messages with missing headers
  2012-08-07 21:21 [PATCH] test: Add test for messages with missing headers Austin Clements
  2012-08-07 22:43 ` Mark Walters
@ 2012-08-08  1:28 ` Austin Clements
  2012-08-08  7:36   ` Mark Walters
                     ` (3 more replies)
  1 sibling, 4 replies; 12+ messages in thread
From: Austin Clements @ 2012-08-08  1:28 UTC (permalink / raw)
  To: notmuch

Currently the JSON tests for search and show are broken because
notmuch attempts to dereference a NULL pointer.
---
This version fixes the "Show: text" test so that it sanitize its
output and doesn't hard-code my test paths.

 test/missing-headers |  162 ++++++++++++++++++++++++++++++++++++++++++++++++++
 test/notmuch-test    |    1 +
 2 files changed, 163 insertions(+)
 create mode 100755 test/missing-headers

diff --git a/test/missing-headers b/test/missing-headers
new file mode 100755
index 0000000..e79f922
--- /dev/null
+++ b/test/missing-headers
@@ -0,0 +1,162 @@
+#!/usr/bin/env bash
+test_description='messages with missing headers'
+. ./test-lib.sh
+
+# Notmuch requires at least one of from, subject, or to or it will
+# ignore the file.  Generate two messages so that together they cover
+# all possible missing headers.  We also give one of the messages a
+# date to ensure stable result ordering.
+
+cat <<EOF > "${MAIL_DIR}/msg-2"
+To: Notmuch Test Suite <test_suite@notmuchmail.org>
+Date: Fri, 05 Jan 2001 15:43:57 +0000
+
+Body
+EOF
+
+cat <<EOF > "${MAIL_DIR}/msg-1"
+From: Notmuch Test Suite <test_suite@notmuchmail.org>
+
+Body
+EOF
+
+NOTMUCH_NEW
+
+test_begin_subtest "Search: text"
+output=$(notmuch search '*' | notmuch_search_sanitize)
+test_expect_equal "$output" "\
+thread:XXX   2001-01-05 [1/1] (null);  (inbox unread)
+thread:XXX   1970-01-01 [1/1] Notmuch Test Suite;  (inbox unread)"
+
+test_begin_subtest "Search: json"
+test_subtest_known_broken
+output=$(notmuch search --format=json '*' | notmuch_search_sanitize)
+test_expect_equal_json "$output" '
+[
+    {
+        "authors": "",
+        "date_relative": "2001-01-05",
+        "matched": 1,
+        "subject": "",
+        "tags": [
+            "inbox",
+            "unread"
+        ],
+        "thread": "XXX",
+        "timestamp": 978709437,
+        "total": 1
+    },
+    {
+        "authors": "Notmuch Test Suite",
+        "date_relative": "1970-01-01",
+        "matched": 1,
+        "subject": "",
+        "tags": [
+            "inbox",
+            "unread"
+        ],
+        "thread": "XXX",
+        "timestamp": 0,
+        "total": 1
+    }
+]'
+
+test_begin_subtest "Show: text"
+output=$(notmuch show '*' | notmuch_show_sanitize)
+test_expect_equal "$output" "\
+\fmessage{ id:notmuch-sha1-7a6e4eac383ef958fcd3ebf2143db71b8ff01161 depth:0 match:1 excluded:0 filename:/XXX/mail/msg-2
+\fheader{
+ (2001-01-05) (inbox unread)
+Subject: (null)
+From: (null)
+To: Notmuch Test Suite <test_suite@notmuchmail.org>
+Date: Fri, 05 Jan 2001 15:43:57 +0000
+\fheader}
+\fbody{
+\fpart{ ID: 1, Content-type: text/plain
+Body
+\fpart}
+\fbody}
+\fmessage}
+\fmessage{ id:notmuch-sha1-ca55943aff7a72baf2ab21fa74fab3d632401334 depth:0 match:1 excluded:0 filename:/XXX/mail/msg-1
+\fheader{
+Notmuch Test Suite <test_suite@notmuchmail.org> (1970-01-01) (inbox unread)
+Subject: (null)
+From: Notmuch Test Suite <test_suite@notmuchmail.org>
+Date: Thu, 01 Jan 1970 00:00:00 +0000
+\fheader}
+\fbody{
+\fpart{ ID: 1, Content-type: text/plain
+Body
+\fpart}
+\fbody}
+\fmessage}"
+
+test_begin_subtest "Show: json"
+test_subtest_known_broken
+output=$(notmuch show --format=json '*' | notmuch_json_show_sanitize)
+test_expect_equal_json "$output" '
+[
+    [
+        [
+            {
+                "body": [
+                    {
+                        "content": "Body\n",
+                        "content-type": "text/plain",
+                        "id": 1
+                    }
+                ],
+                "date_relative": "2001-01-05",
+                "excluded": false,
+                "filename": "YYYYY",
+                "headers": {
+                    "Date": "Fri, 05 Jan 2001 15:43:57 +0000",
+                    "From": "",
+                    "Subject": "",
+                    "To": "Notmuch Test Suite <test_suite@notmuchmail.org>"
+                },
+                "id": "XXXXX",
+                "match": true,
+                "tags": [
+                    "inbox",
+                    "unread"
+                ],
+                "timestamp": 978709437
+            },
+            []
+        ]
+    ],
+    [
+        [
+            {
+                "body": [
+                    {
+                        "content": "Body\n",
+                        "content-type": "text/plain",
+                        "id": 1
+                    }
+                ],
+                "date_relative": "1970-01-01",
+                "excluded": false,
+                "filename": "YYYYY",
+                "headers": {
+                    "Date": "Thu, 01 Jan 1970 00:00:00 +0000",
+                    "From": "Notmuch Test Suite <test_suite@notmuchmail.org>",
+                    "Subject": ""
+                },
+                "id": "XXXXX",
+                "match": true,
+                "tags": [
+                    "inbox",
+                    "unread"
+                ],
+                "timestamp": 0
+            },
+            []
+        ]
+    ]
+]'
+
+
+test_done
diff --git a/test/notmuch-test b/test/notmuch-test
index ea39dfc..cc732c3 100755
--- a/test/notmuch-test
+++ b/test/notmuch-test
@@ -59,6 +59,7 @@ TESTS="
   emacs-address-cleaning
   emacs-hello
   emacs-show
+  missing-headers
 "
 TESTS=${NOTMUCH_TESTS:=$TESTS}
 
-- 
1.7.10

^ permalink raw reply related	[flat|nested] 12+ messages in thread

* Re: [PATCH v2] test: Add test for messages with missing headers
  2012-08-08  1:28 ` [PATCH v2] " Austin Clements
@ 2012-08-08  7:36   ` Mark Walters
  2012-08-08  8:08   ` Tomi Ollila
                     ` (2 subsequent siblings)
  3 siblings, 0 replies; 12+ messages in thread
From: Mark Walters @ 2012-08-08  7:36 UTC (permalink / raw)
  To: Austin Clements, notmuch


On Wed, 08 Aug 2012, Austin Clements <amdragon@MIT.EDU> wrote:
> Currently the JSON tests for search and show are broken because
> notmuch attempts to dereference a NULL pointer.
> ---
> This version fixes the "Show: text" test so that it sanitize its
> output and doesn't hard-code my test paths.

LGTM +1. Tests work as expected and all tests pass with
id:"874noe1o0r.fsf@qmul.ac.uk" applied.

Best wishes

Mark

>  test/missing-headers |  162 ++++++++++++++++++++++++++++++++++++++++++++++++++
>  test/notmuch-test    |    1 +
>  2 files changed, 163 insertions(+)
>  create mode 100755 test/missing-headers
>
> diff --git a/test/missing-headers b/test/missing-headers
> new file mode 100755
> index 0000000..e79f922
> --- /dev/null
> +++ b/test/missing-headers
> @@ -0,0 +1,162 @@
> +#!/usr/bin/env bash
> +test_description='messages with missing headers'
> +. ./test-lib.sh
> +
> +# Notmuch requires at least one of from, subject, or to or it will
> +# ignore the file.  Generate two messages so that together they cover
> +# all possible missing headers.  We also give one of the messages a
> +# date to ensure stable result ordering.
> +
> +cat <<EOF > "${MAIL_DIR}/msg-2"
> +To: Notmuch Test Suite <test_suite@notmuchmail.org>
> +Date: Fri, 05 Jan 2001 15:43:57 +0000
> +
> +Body
> +EOF
> +
> +cat <<EOF > "${MAIL_DIR}/msg-1"
> +From: Notmuch Test Suite <test_suite@notmuchmail.org>
> +
> +Body
> +EOF
> +
> +NOTMUCH_NEW
> +
> +test_begin_subtest "Search: text"
> +output=$(notmuch search '*' | notmuch_search_sanitize)
> +test_expect_equal "$output" "\
> +thread:XXX   2001-01-05 [1/1] (null);  (inbox unread)
> +thread:XXX   1970-01-01 [1/1] Notmuch Test Suite;  (inbox unread)"
> +
> +test_begin_subtest "Search: json"
> +test_subtest_known_broken
> +output=$(notmuch search --format=json '*' | notmuch_search_sanitize)
> +test_expect_equal_json "$output" '
> +[
> +    {
> +        "authors": "",
> +        "date_relative": "2001-01-05",
> +        "matched": 1,
> +        "subject": "",
> +        "tags": [
> +            "inbox",
> +            "unread"
> +        ],
> +        "thread": "XXX",
> +        "timestamp": 978709437,
> +        "total": 1
> +    },
> +    {
> +        "authors": "Notmuch Test Suite",
> +        "date_relative": "1970-01-01",
> +        "matched": 1,
> +        "subject": "",
> +        "tags": [
> +            "inbox",
> +            "unread"
> +        ],
> +        "thread": "XXX",
> +        "timestamp": 0,
> +        "total": 1
> +    }
> +]'
> +
> +test_begin_subtest "Show: text"
> +output=$(notmuch show '*' | notmuch_show_sanitize)
> +test_expect_equal "$output" "\
> +\fmessage{ id:notmuch-sha1-7a6e4eac383ef958fcd3ebf2143db71b8ff01161 depth:0 match:1 excluded:0 filename:/XXX/mail/msg-2
> +\fheader{
> + (2001-01-05) (inbox unread)
> +Subject: (null)
> +From: (null)
> +To: Notmuch Test Suite <test_suite@notmuchmail.org>
> +Date: Fri, 05 Jan 2001 15:43:57 +0000
> +\fheader}
> +\fbody{
> +\fpart{ ID: 1, Content-type: text/plain
> +Body
> +\fpart}
> +\fbody}
> +\fmessage}
> +\fmessage{ id:notmuch-sha1-ca55943aff7a72baf2ab21fa74fab3d632401334 depth:0 match:1 excluded:0 filename:/XXX/mail/msg-1
> +\fheader{
> +Notmuch Test Suite <test_suite@notmuchmail.org> (1970-01-01) (inbox unread)
> +Subject: (null)
> +From: Notmuch Test Suite <test_suite@notmuchmail.org>
> +Date: Thu, 01 Jan 1970 00:00:00 +0000
> +\fheader}
> +\fbody{
> +\fpart{ ID: 1, Content-type: text/plain
> +Body
> +\fpart}
> +\fbody}
> +\fmessage}"
> +
> +test_begin_subtest "Show: json"
> +test_subtest_known_broken
> +output=$(notmuch show --format=json '*' | notmuch_json_show_sanitize)
> +test_expect_equal_json "$output" '
> +[
> +    [
> +        [
> +            {
> +                "body": [
> +                    {
> +                        "content": "Body\n",
> +                        "content-type": "text/plain",
> +                        "id": 1
> +                    }
> +                ],
> +                "date_relative": "2001-01-05",
> +                "excluded": false,
> +                "filename": "YYYYY",
> +                "headers": {
> +                    "Date": "Fri, 05 Jan 2001 15:43:57 +0000",
> +                    "From": "",
> +                    "Subject": "",
> +                    "To": "Notmuch Test Suite <test_suite@notmuchmail.org>"
> +                },
> +                "id": "XXXXX",
> +                "match": true,
> +                "tags": [
> +                    "inbox",
> +                    "unread"
> +                ],
> +                "timestamp": 978709437
> +            },
> +            []
> +        ]
> +    ],
> +    [
> +        [
> +            {
> +                "body": [
> +                    {
> +                        "content": "Body\n",
> +                        "content-type": "text/plain",
> +                        "id": 1
> +                    }
> +                ],
> +                "date_relative": "1970-01-01",
> +                "excluded": false,
> +                "filename": "YYYYY",
> +                "headers": {
> +                    "Date": "Thu, 01 Jan 1970 00:00:00 +0000",
> +                    "From": "Notmuch Test Suite <test_suite@notmuchmail.org>",
> +                    "Subject": ""
> +                },
> +                "id": "XXXXX",
> +                "match": true,
> +                "tags": [
> +                    "inbox",
> +                    "unread"
> +                ],
> +                "timestamp": 0
> +            },
> +            []
> +        ]
> +    ]
> +]'
> +
> +
> +test_done
> diff --git a/test/notmuch-test b/test/notmuch-test
> index ea39dfc..cc732c3 100755
> --- a/test/notmuch-test
> +++ b/test/notmuch-test
> @@ -59,6 +59,7 @@ TESTS="
>    emacs-address-cleaning
>    emacs-hello
>    emacs-show
> +  missing-headers
>  "
>  TESTS=${NOTMUCH_TESTS:=$TESTS}
>  
> -- 
> 1.7.10
>
> _______________________________________________
> notmuch mailing list
> notmuch@notmuchmail.org
> http://notmuchmail.org/mailman/listinfo/notmuch

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [PATCH v2] test: Add test for messages with missing headers
  2012-08-08  1:28 ` [PATCH v2] " Austin Clements
  2012-08-08  7:36   ` Mark Walters
@ 2012-08-08  8:08   ` Tomi Ollila
  2012-08-08 15:46     ` Austin Clements
  2012-08-08 21:23   ` [PATCH] sprinters: bugfix when NULL passed for a string Mark Walters
  2012-08-12 19:50   ` [PATCH v2] test: Add test for messages with missing headers David Bremner
  3 siblings, 1 reply; 12+ messages in thread
From: Tomi Ollila @ 2012-08-08  8:08 UTC (permalink / raw)
  To: Austin Clements, notmuch

On Wed, Aug 08 2012, Austin Clements <amdragon@MIT.EDU> wrote:

> Currently the JSON tests for search and show are broken because
> notmuch attempts to dereference a NULL pointer.
> ---
> This version fixes the "Show: text" test so that it sanitize its
> output and doesn't hard-code my test paths.

+1

Tomi

w/ python json(.tool) the original order cannot be preserved as the parser
stores content into dictionary -- without sorting those came in some
internal python order... the following code could be used to use less
indentation, though:

python -c 'import sys,json; j = json.load(sys.stdin);
json.dump(j, sys.stdout, sort_keys=True, indent=2)' < input_file

The other "problem" with json.tool is that it converts non-ascii chars
to \uNNNN values :/.

What we could do is to dig a simple c json formatter -- someday in the
future, maybe -- but for *now* this is the best we can have :D

>
>  test/missing-headers |  162 ++++++++++++++++++++++++++++++++++++++++++++++++++
>  test/notmuch-test    |    1 +
>  2 files changed, 163 insertions(+)
>  create mode 100755 test/missing-headers
>
> diff --git a/test/missing-headers b/test/missing-headers
> new file mode 100755
> index 0000000..e79f922
> --- /dev/null
> +++ b/test/missing-headers
> @@ -0,0 +1,162 @@
> +#!/usr/bin/env bash
> +test_description='messages with missing headers'
> +. ./test-lib.sh
> +
> +# Notmuch requires at least one of from, subject, or to or it will
> +# ignore the file.  Generate two messages so that together they cover
> +# all possible missing headers.  We also give one of the messages a
> +# date to ensure stable result ordering.
> +
> +cat <<EOF > "${MAIL_DIR}/msg-2"
> +To: Notmuch Test Suite <test_suite@notmuchmail.org>
> +Date: Fri, 05 Jan 2001 15:43:57 +0000
> +
> +Body
> +EOF
> +
> +cat <<EOF > "${MAIL_DIR}/msg-1"
> +From: Notmuch Test Suite <test_suite@notmuchmail.org>
> +
> +Body
> +EOF
> +
> +NOTMUCH_NEW
> +
> +test_begin_subtest "Search: text"
> +output=$(notmuch search '*' | notmuch_search_sanitize)
> +test_expect_equal "$output" "\
> +thread:XXX   2001-01-05 [1/1] (null);  (inbox unread)
> +thread:XXX   1970-01-01 [1/1] Notmuch Test Suite;  (inbox unread)"
> +
> +test_begin_subtest "Search: json"
> +test_subtest_known_broken
> +output=$(notmuch search --format=json '*' | notmuch_search_sanitize)
> +test_expect_equal_json "$output" '
> +[
> +    {
> +        "authors": "",
> +        "date_relative": "2001-01-05",
> +        "matched": 1,
> +        "subject": "",
> +        "tags": [
> +            "inbox",
> +            "unread"
> +        ],
> +        "thread": "XXX",
> +        "timestamp": 978709437,
> +        "total": 1
> +    },
> +    {
> +        "authors": "Notmuch Test Suite",
> +        "date_relative": "1970-01-01",
> +        "matched": 1,
> +        "subject": "",
> +        "tags": [
> +            "inbox",
> +            "unread"
> +        ],
> +        "thread": "XXX",
> +        "timestamp": 0,
> +        "total": 1
> +    }
> +]'
> +
> +test_begin_subtest "Show: text"
> +output=$(notmuch show '*' | notmuch_show_sanitize)
> +test_expect_equal "$output" "\
> +\fmessage{ id:notmuch-sha1-7a6e4eac383ef958fcd3ebf2143db71b8ff01161 depth:0 match:1 excluded:0 filename:/XXX/mail/msg-2
> +\fheader{
> + (2001-01-05) (inbox unread)
> +Subject: (null)
> +From: (null)
> +To: Notmuch Test Suite <test_suite@notmuchmail.org>
> +Date: Fri, 05 Jan 2001 15:43:57 +0000
> +\fheader}
> +\fbody{
> +\fpart{ ID: 1, Content-type: text/plain
> +Body
> +\fpart}
> +\fbody}
> +\fmessage}
> +\fmessage{ id:notmuch-sha1-ca55943aff7a72baf2ab21fa74fab3d632401334 depth:0 match:1 excluded:0 filename:/XXX/mail/msg-1
> +\fheader{
> +Notmuch Test Suite <test_suite@notmuchmail.org> (1970-01-01) (inbox unread)
> +Subject: (null)
> +From: Notmuch Test Suite <test_suite@notmuchmail.org>
> +Date: Thu, 01 Jan 1970 00:00:00 +0000
> +\fheader}
> +\fbody{
> +\fpart{ ID: 1, Content-type: text/plain
> +Body
> +\fpart}
> +\fbody}
> +\fmessage}"
> +
> +test_begin_subtest "Show: json"
> +test_subtest_known_broken
> +output=$(notmuch show --format=json '*' | notmuch_json_show_sanitize)
> +test_expect_equal_json "$output" '
> +[
> +    [
> +        [
> +            {
> +                "body": [
> +                    {
> +                        "content": "Body\n",
> +                        "content-type": "text/plain",
> +                        "id": 1
> +                    }
> +                ],
> +                "date_relative": "2001-01-05",
> +                "excluded": false,
> +                "filename": "YYYYY",
> +                "headers": {
> +                    "Date": "Fri, 05 Jan 2001 15:43:57 +0000",
> +                    "From": "",
> +                    "Subject": "",
> +                    "To": "Notmuch Test Suite <test_suite@notmuchmail.org>"
> +                },
> +                "id": "XXXXX",
> +                "match": true,
> +                "tags": [
> +                    "inbox",
> +                    "unread"
> +                ],
> +                "timestamp": 978709437
> +            },
> +            []
> +        ]
> +    ],
> +    [
> +        [
> +            {
> +                "body": [
> +                    {
> +                        "content": "Body\n",
> +                        "content-type": "text/plain",
> +                        "id": 1
> +                    }
> +                ],
> +                "date_relative": "1970-01-01",
> +                "excluded": false,
> +                "filename": "YYYYY",
> +                "headers": {
> +                    "Date": "Thu, 01 Jan 1970 00:00:00 +0000",
> +                    "From": "Notmuch Test Suite <test_suite@notmuchmail.org>",
> +                    "Subject": ""
> +                },
> +                "id": "XXXXX",
> +                "match": true,
> +                "tags": [
> +                    "inbox",
> +                    "unread"
> +                ],
> +                "timestamp": 0
> +            },
> +            []
> +        ]
> +    ]
> +]'
> +
> +
> +test_done
> diff --git a/test/notmuch-test b/test/notmuch-test
> index ea39dfc..cc732c3 100755
> --- a/test/notmuch-test
> +++ b/test/notmuch-test
> @@ -59,6 +59,7 @@ TESTS="
>    emacs-address-cleaning
>    emacs-hello
>    emacs-show
> +  missing-headers
>  "
>  TESTS=${NOTMUCH_TESTS:=$TESTS}
>  
> -- 
> 1.7.10
>
> _______________________________________________
> notmuch mailing list
> notmuch@notmuchmail.org
> http://notmuchmail.org/mailman/listinfo/notmuch

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [PATCH v2] test: Add test for messages with missing headers
  2012-08-08  8:08   ` Tomi Ollila
@ 2012-08-08 15:46     ` Austin Clements
  2012-08-08 18:47       ` Tomi Ollila
  0 siblings, 1 reply; 12+ messages in thread
From: Austin Clements @ 2012-08-08 15:46 UTC (permalink / raw)
  To: Tomi Ollila; +Cc: notmuch

Quoth Tomi Ollila on Aug 08 at 11:08 am:
> On Wed, Aug 08 2012, Austin Clements <amdragon@MIT.EDU> wrote:
> 
> > Currently the JSON tests for search and show are broken because
> > notmuch attempts to dereference a NULL pointer.
> > ---
> > This version fixes the "Show: text" test so that it sanitize its
> > output and doesn't hard-code my test paths.
> 
> +1
> 
> Tomi
> 
> w/ python json(.tool) the original order cannot be preserved as the parser
> stores content into dictionary -- without sorting those came in some
> internal python order... the following code could be used to use less

To be fair, it can output them in original order, but doing so
requires Python 2.7:

python -c 'import sys,json,collections; json.dump(json.load(sys.stdin, object_pairs_hook=collections.OrderedDict), sys.stdout, indent=4)'

> indentation, though:
> 
> python -c 'import sys,json; j = json.load(sys.stdin);
> json.dump(j, sys.stdout, sort_keys=True, indent=2)' < input_file
> 
> The other "problem" with json.tool is that it converts non-ascii chars
> to \uNNNN values :/.

This one doesn't require 2.7.

python -c 'import sys,json,codecs; json.dump(json.load(sys.stdin), codecs.getwriter("utf8")(sys.stdout), indent=4, ensure_ascii=False)'

Though I think that, for test canonicalization, \uNNNN is probably
less error/locale prone.

> What we could do is to dig a simple c json formatter -- someday in the
> future, maybe -- but for *now* this is the best we can have :D

Not another JSON parser/printer!

> >
> >  test/missing-headers |  162 ++++++++++++++++++++++++++++++++++++++++++++++++++
> >  test/notmuch-test    |    1 +
> >  2 files changed, 163 insertions(+)
> >  create mode 100755 test/missing-headers
> >
> > diff --git a/test/missing-headers b/test/missing-headers
> > new file mode 100755
> > index 0000000..e79f922
> > --- /dev/null
> > +++ b/test/missing-headers
> > @@ -0,0 +1,162 @@
> > +#!/usr/bin/env bash
> > +test_description='messages with missing headers'
> > +. ./test-lib.sh
> > +
> > +# Notmuch requires at least one of from, subject, or to or it will
> > +# ignore the file.  Generate two messages so that together they cover
> > +# all possible missing headers.  We also give one of the messages a
> > +# date to ensure stable result ordering.
> > +
> > +cat <<EOF > "${MAIL_DIR}/msg-2"
> > +To: Notmuch Test Suite <test_suite@notmuchmail.org>
> > +Date: Fri, 05 Jan 2001 15:43:57 +0000
> > +
> > +Body
> > +EOF
> > +
> > +cat <<EOF > "${MAIL_DIR}/msg-1"
> > +From: Notmuch Test Suite <test_suite@notmuchmail.org>
> > +
> > +Body
> > +EOF
> > +
> > +NOTMUCH_NEW
> > +
> > +test_begin_subtest "Search: text"
> > +output=$(notmuch search '*' | notmuch_search_sanitize)
> > +test_expect_equal "$output" "\
> > +thread:XXX   2001-01-05 [1/1] (null);  (inbox unread)
> > +thread:XXX   1970-01-01 [1/1] Notmuch Test Suite;  (inbox unread)"
> > +
> > +test_begin_subtest "Search: json"
> > +test_subtest_known_broken
> > +output=$(notmuch search --format=json '*' | notmuch_search_sanitize)
> > +test_expect_equal_json "$output" '
> > +[
> > +    {
> > +        "authors": "",
> > +        "date_relative": "2001-01-05",
> > +        "matched": 1,
> > +        "subject": "",
> > +        "tags": [
> > +            "inbox",
> > +            "unread"
> > +        ],
> > +        "thread": "XXX",
> > +        "timestamp": 978709437,
> > +        "total": 1
> > +    },
> > +    {
> > +        "authors": "Notmuch Test Suite",
> > +        "date_relative": "1970-01-01",
> > +        "matched": 1,
> > +        "subject": "",
> > +        "tags": [
> > +            "inbox",
> > +            "unread"
> > +        ],
> > +        "thread": "XXX",
> > +        "timestamp": 0,
> > +        "total": 1
> > +    }
> > +]'
> > +
> > +test_begin_subtest "Show: text"
> > +output=$(notmuch show '*' | notmuch_show_sanitize)
> > +test_expect_equal "$output" "\
> > +\fmessage{ id:notmuch-sha1-7a6e4eac383ef958fcd3ebf2143db71b8ff01161 depth:0 match:1 excluded:0 filename:/XXX/mail/msg-2
> > +\fheader{
> > + (2001-01-05) (inbox unread)
> > +Subject: (null)
> > +From: (null)
> > +To: Notmuch Test Suite <test_suite@notmuchmail.org>
> > +Date: Fri, 05 Jan 2001 15:43:57 +0000
> > +\fheader}
> > +\fbody{
> > +\fpart{ ID: 1, Content-type: text/plain
> > +Body
> > +\fpart}
> > +\fbody}
> > +\fmessage}
> > +\fmessage{ id:notmuch-sha1-ca55943aff7a72baf2ab21fa74fab3d632401334 depth:0 match:1 excluded:0 filename:/XXX/mail/msg-1
> > +\fheader{
> > +Notmuch Test Suite <test_suite@notmuchmail.org> (1970-01-01) (inbox unread)
> > +Subject: (null)
> > +From: Notmuch Test Suite <test_suite@notmuchmail.org>
> > +Date: Thu, 01 Jan 1970 00:00:00 +0000
> > +\fheader}
> > +\fbody{
> > +\fpart{ ID: 1, Content-type: text/plain
> > +Body
> > +\fpart}
> > +\fbody}
> > +\fmessage}"
> > +
> > +test_begin_subtest "Show: json"
> > +test_subtest_known_broken
> > +output=$(notmuch show --format=json '*' | notmuch_json_show_sanitize)
> > +test_expect_equal_json "$output" '
> > +[
> > +    [
> > +        [
> > +            {
> > +                "body": [
> > +                    {
> > +                        "content": "Body\n",
> > +                        "content-type": "text/plain",
> > +                        "id": 1
> > +                    }
> > +                ],
> > +                "date_relative": "2001-01-05",
> > +                "excluded": false,
> > +                "filename": "YYYYY",
> > +                "headers": {
> > +                    "Date": "Fri, 05 Jan 2001 15:43:57 +0000",
> > +                    "From": "",
> > +                    "Subject": "",
> > +                    "To": "Notmuch Test Suite <test_suite@notmuchmail.org>"
> > +                },
> > +                "id": "XXXXX",
> > +                "match": true,
> > +                "tags": [
> > +                    "inbox",
> > +                    "unread"
> > +                ],
> > +                "timestamp": 978709437
> > +            },
> > +            []
> > +        ]
> > +    ],
> > +    [
> > +        [
> > +            {
> > +                "body": [
> > +                    {
> > +                        "content": "Body\n",
> > +                        "content-type": "text/plain",
> > +                        "id": 1
> > +                    }
> > +                ],
> > +                "date_relative": "1970-01-01",
> > +                "excluded": false,
> > +                "filename": "YYYYY",
> > +                "headers": {
> > +                    "Date": "Thu, 01 Jan 1970 00:00:00 +0000",
> > +                    "From": "Notmuch Test Suite <test_suite@notmuchmail.org>",
> > +                    "Subject": ""
> > +                },
> > +                "id": "XXXXX",
> > +                "match": true,
> > +                "tags": [
> > +                    "inbox",
> > +                    "unread"
> > +                ],
> > +                "timestamp": 0
> > +            },
> > +            []
> > +        ]
> > +    ]
> > +]'
> > +
> > +
> > +test_done
> > diff --git a/test/notmuch-test b/test/notmuch-test
> > index ea39dfc..cc732c3 100755
> > --- a/test/notmuch-test
> > +++ b/test/notmuch-test
> > @@ -59,6 +59,7 @@ TESTS="
> >    emacs-address-cleaning
> >    emacs-hello
> >    emacs-show
> > +  missing-headers
> >  "
> >  TESTS=${NOTMUCH_TESTS:=$TESTS}
> >  
> >
> > _______________________________________________
> > notmuch mailing list
> > notmuch@notmuchmail.org
> > http://notmuchmail.org/mailman/listinfo/notmuch

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [PATCH v2] test: Add test for messages with missing headers
  2012-08-08 15:46     ` Austin Clements
@ 2012-08-08 18:47       ` Tomi Ollila
  2012-08-08 19:10         ` Austin Clements
  0 siblings, 1 reply; 12+ messages in thread
From: Tomi Ollila @ 2012-08-08 18:47 UTC (permalink / raw)
  To: Austin Clements; +Cc: notmuch

On Wed, Aug 08 2012, Austin Clements <amdragon@MIT.EDU> wrote:

> Quoth Tomi Ollila on Aug 08 at 11:08 am:
>> On Wed, Aug 08 2012, Austin Clements <amdragon@MIT.EDU> wrote:
>> 
>> > Currently the JSON tests for search and show are broken because
>> > notmuch attempts to dereference a NULL pointer.
>> > ---
>> > This version fixes the "Show: text" test so that it sanitize its
>> > output and doesn't hard-code my test paths.
>> 
>> +1
>> 
>> Tomi
>> 
>> w/ python json(.tool) the original order cannot be preserved as the parser
>> stores content into dictionary -- without sorting those came in some
>> internal python order... the following code could be used to use less
>
> To be fair, it can output them in original order, but doing so
> requires Python 2.7:
>
> python -c 'import sys,json,collections; json.dump(json.load(sys.stdin,
> object_pairs_hook=collections.OrderedDict), sys.stdout, indent=4)'

ack.

>
>> indentation, though:
>> 
>> python -c 'import sys,json; j = json.load(sys.stdin);
>> json.dump(j, sys.stdout, sort_keys=True, indent=2)' < input_file
>> 
>> The other "problem" with json.tool is that it converts non-ascii chars
>> to \uNNNN values :/.
>
> This one doesn't require 2.7.
>
> python -c 'import sys,json,codecs; json.dump(json.load(sys.stdin),
> codecs.getwriter("utf8")(sys.stdout), indent=4, ensure_ascii=False)'

nice.

>
> Though I think that, for test canonicalization, \uNNNN is probably
> less error/locale prone.

true.

>
>> What we could do is to dig a simple c json formatter -- someday in the
>> future, maybe -- but for *now* this is the best we can have :D
>
> Not another JSON parser/printer!

Why not, one from scratch, without looking any source there is already >;)

Well, you're right; the json.tool version python 2.6 provides is good --
-- ordering and those \uNNNN:s doesn't really matter.
Probably thinking (or planning!) anything else at this time is waste of time.

At least these 2 questions remain:

Is the test requirement of python 2.6 suitable/acceptable ?

How do we pretty-print S-expression syntax ;) ?


Tomi


>
>> >
>> >  test/missing-headers |  162 ++++++++++++++++++++++++++++++++++++++++++++++++++
>> >  test/notmuch-test    |    1 +
>> >  2 files changed, 163 insertions(+)
>> >  create mode 100755 test/missing-headers
>> >
>> > diff --git a/test/missing-headers b/test/missing-headers
>> > new file mode 100755
>> > index 0000000..e79f922
>> > --- /dev/null
>> > +++ b/test/missing-headers
>> > @@ -0,0 +1,162 @@
>> > +#!/usr/bin/env bash
>> > +test_description='messages with missing headers'
>> > +. ./test-lib.sh
>> > +
>> > +# Notmuch requires at least one of from, subject, or to or it will
>> > +# ignore the file.  Generate two messages so that together they cover
>> > +# all possible missing headers.  We also give one of the messages a
>> > +# date to ensure stable result ordering.
>> > +
>> > +cat <<EOF > "${MAIL_DIR}/msg-2"
>> > +To: Notmuch Test Suite <test_suite@notmuchmail.org>
>> > +Date: Fri, 05 Jan 2001 15:43:57 +0000
>> > +
>> > +Body
>> > +EOF
>> > +
>> > +cat <<EOF > "${MAIL_DIR}/msg-1"
>> > +From: Notmuch Test Suite <test_suite@notmuchmail.org>
>> > +
>> > +Body
>> > +EOF
>> > +
>> > +NOTMUCH_NEW
>> > +
>> > +test_begin_subtest "Search: text"
>> > +output=$(notmuch search '*' | notmuch_search_sanitize)
>> > +test_expect_equal "$output" "\
>> > +thread:XXX   2001-01-05 [1/1] (null);  (inbox unread)
>> > +thread:XXX   1970-01-01 [1/1] Notmuch Test Suite;  (inbox unread)"
>> > +
>> > +test_begin_subtest "Search: json"
>> > +test_subtest_known_broken
>> > +output=$(notmuch search --format=json '*' | notmuch_search_sanitize)
>> > +test_expect_equal_json "$output" '
>> > +[
>> > +    {
>> > +        "authors": "",
>> > +        "date_relative": "2001-01-05",
>> > +        "matched": 1,
>> > +        "subject": "",
>> > +        "tags": [
>> > +            "inbox",
>> > +            "unread"
>> > +        ],
>> > +        "thread": "XXX",
>> > +        "timestamp": 978709437,
>> > +        "total": 1
>> > +    },
>> > +    {
>> > +        "authors": "Notmuch Test Suite",
>> > +        "date_relative": "1970-01-01",
>> > +        "matched": 1,
>> > +        "subject": "",
>> > +        "tags": [
>> > +            "inbox",
>> > +            "unread"
>> > +        ],
>> > +        "thread": "XXX",
>> > +        "timestamp": 0,
>> > +        "total": 1
>> > +    }
>> > +]'
>> > +
>> > +test_begin_subtest "Show: text"
>> > +output=$(notmuch show '*' | notmuch_show_sanitize)
>> > +test_expect_equal "$output" "\
>> > +\fmessage{ id:notmuch-sha1-7a6e4eac383ef958fcd3ebf2143db71b8ff01161 depth:0 match:1 excluded:0 filename:/XXX/mail/msg-2
>> > +\fheader{
>> > + (2001-01-05) (inbox unread)
>> > +Subject: (null)
>> > +From: (null)
>> > +To: Notmuch Test Suite <test_suite@notmuchmail.org>
>> > +Date: Fri, 05 Jan 2001 15:43:57 +0000
>> > +\fheader}
>> > +\fbody{
>> > +\fpart{ ID: 1, Content-type: text/plain
>> > +Body
>> > +\fpart}
>> > +\fbody}
>> > +\fmessage}
>> > +\fmessage{ id:notmuch-sha1-ca55943aff7a72baf2ab21fa74fab3d632401334 depth:0 match:1 excluded:0 filename:/XXX/mail/msg-1
>> > +\fheader{
>> > +Notmuch Test Suite <test_suite@notmuchmail.org> (1970-01-01) (inbox unread)
>> > +Subject: (null)
>> > +From: Notmuch Test Suite <test_suite@notmuchmail.org>
>> > +Date: Thu, 01 Jan 1970 00:00:00 +0000
>> > +\fheader}
>> > +\fbody{
>> > +\fpart{ ID: 1, Content-type: text/plain
>> > +Body
>> > +\fpart}
>> > +\fbody}
>> > +\fmessage}"
>> > +
>> > +test_begin_subtest "Show: json"
>> > +test_subtest_known_broken
>> > +output=$(notmuch show --format=json '*' | notmuch_json_show_sanitize)
>> > +test_expect_equal_json "$output" '
>> > +[
>> > +    [
>> > +        [
>> > +            {
>> > +                "body": [
>> > +                    {
>> > +                        "content": "Body\n",
>> > +                        "content-type": "text/plain",
>> > +                        "id": 1
>> > +                    }
>> > +                ],
>> > +                "date_relative": "2001-01-05",
>> > +                "excluded": false,
>> > +                "filename": "YYYYY",
>> > +                "headers": {
>> > +                    "Date": "Fri, 05 Jan 2001 15:43:57 +0000",
>> > +                    "From": "",
>> > +                    "Subject": "",
>> > +                    "To": "Notmuch Test Suite <test_suite@notmuchmail.org>"
>> > +                },
>> > +                "id": "XXXXX",
>> > +                "match": true,
>> > +                "tags": [
>> > +                    "inbox",
>> > +                    "unread"
>> > +                ],
>> > +                "timestamp": 978709437
>> > +            },
>> > +            []
>> > +        ]
>> > +    ],
>> > +    [
>> > +        [
>> > +            {
>> > +                "body": [
>> > +                    {
>> > +                        "content": "Body\n",
>> > +                        "content-type": "text/plain",
>> > +                        "id": 1
>> > +                    }
>> > +                ],
>> > +                "date_relative": "1970-01-01",
>> > +                "excluded": false,
>> > +                "filename": "YYYYY",
>> > +                "headers": {
>> > +                    "Date": "Thu, 01 Jan 1970 00:00:00 +0000",
>> > +                    "From": "Notmuch Test Suite <test_suite@notmuchmail.org>",
>> > +                    "Subject": ""
>> > +                },
>> > +                "id": "XXXXX",
>> > +                "match": true,
>> > +                "tags": [
>> > +                    "inbox",
>> > +                    "unread"
>> > +                ],
>> > +                "timestamp": 0
>> > +            },
>> > +            []
>> > +        ]
>> > +    ]
>> > +]'
>> > +
>> > +
>> > +test_done
>> > diff --git a/test/notmuch-test b/test/notmuch-test
>> > index ea39dfc..cc732c3 100755
>> > --- a/test/notmuch-test
>> > +++ b/test/notmuch-test
>> > @@ -59,6 +59,7 @@ TESTS="
>> >    emacs-address-cleaning
>> >    emacs-hello
>> >    emacs-show
>> > +  missing-headers
>> >  "
>> >  TESTS=${NOTMUCH_TESTS:=$TESTS}
>> >  
>> >
>> > _______________________________________________
>> > notmuch mailing list
>> > notmuch@notmuchmail.org
>> > http://notmuchmail.org/mailman/listinfo/notmuch
> _______________________________________________
> notmuch mailing list
> notmuch@notmuchmail.org
> http://notmuchmail.org/mailman/listinfo/notmuch

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [PATCH v2] test: Add test for messages with missing headers
  2012-08-08 18:47       ` Tomi Ollila
@ 2012-08-08 19:10         ` Austin Clements
  0 siblings, 0 replies; 12+ messages in thread
From: Austin Clements @ 2012-08-08 19:10 UTC (permalink / raw)
  To: Tomi Ollila; +Cc: notmuch

Quoth Tomi Ollila on Aug 08 at  9:47 pm:
> On Wed, Aug 08 2012, Austin Clements <amdragon@MIT.EDU> wrote:
> > Not another JSON parser/printer!
> 
> Why not, one from scratch, without looking any source there is already >;)
> 
> Well, you're right; the json.tool version python 2.6 provides is good --
> -- ordering and those \uNNNN:s doesn't really matter.
> Probably thinking (or planning!) anything else at this time is waste of time.
> 
> At least these 2 questions remain:
> 
> Is the test requirement of python 2.6 suitable/acceptable ?

Too late; it's already in master.  Besides, even *Debian stable* has
Python 2.6, which qualifies it as ubiquitous in my book.

> How do we pretty-print S-expression syntax ;) ?

Emacs of course!

echo '(1 2 (3 4) (5 6))' | \
  emacs -Q --batch --eval '(pp (read-from-minibuffer "" nil nil t))'

This may or may not be a good idea.

> Tomi

^ permalink raw reply	[flat|nested] 12+ messages in thread

* [PATCH] sprinters: bugfix when NULL passed for a string.
  2012-08-08  1:28 ` [PATCH v2] " Austin Clements
  2012-08-08  7:36   ` Mark Walters
  2012-08-08  8:08   ` Tomi Ollila
@ 2012-08-08 21:23   ` Mark Walters
  2012-08-08 21:28     ` Austin Clements
  2012-08-12 19:51     ` David Bremner
  2012-08-12 19:50   ` [PATCH v2] test: Add test for messages with missing headers David Bremner
  3 siblings, 2 replies; 12+ messages in thread
From: Mark Walters @ 2012-08-08 21:23 UTC (permalink / raw)
  To: Austin Clements, notmuch


The string function in a sprinter may be called with a NULL string
pointer (eg if a header is absent). This causes a segfault. We fix
this by checking for a null pointer in the string functions and update
the sprinter documentation.

At the moment some output when format=text is done directly rather than
via an sprinter: in that case a null pointer is passed to printf or
similar and a "(null)" appears in the output. That behaviour is not
changed in this patch.
---

This is the same as id:"874noe1o0r.fsf@qmul.ac.uk" except for being
based on top of the test in the parent message, and marking the broken
test fixed.

Best wishes

Mark

 sprinter-json.c      |    2 ++
 sprinter-text.c      |    2 ++
 sprinter.h           |    4 +++-
 test/missing-headers |    2 --
 4 files changed, 7 insertions(+), 3 deletions(-)

diff --git a/sprinter-json.c b/sprinter-json.c
index c9b6835..0a07790 100644
--- a/sprinter-json.c
+++ b/sprinter-json.c
@@ -118,6 +118,8 @@ json_string_len (struct sprinter *sp, const char *val, size_t len)
 static void
 json_string (struct sprinter *sp, const char *val)
 {
+    if (val == NULL)
+	val = "";
     json_string_len (sp, val, strlen (val));
 }
 
diff --git a/sprinter-text.c b/sprinter-text.c
index dfa54b5..10343be 100644
--- a/sprinter-text.c
+++ b/sprinter-text.c
@@ -38,6 +38,8 @@ text_string_len (struct sprinter *sp, const char *val, size_t len)
 static void
 text_string (struct sprinter *sp, const char *val)
 {
+    if (val == NULL)
+	val = "";
     text_string_len (sp, val, strlen (val));
 }
 
diff --git a/sprinter.h b/sprinter.h
index 5f43175..912a526 100644
--- a/sprinter.h
+++ b/sprinter.h
@@ -27,7 +27,9 @@ typedef struct sprinter {
      * a list or map, followed or preceded by separators).  For string
      * and string_len, the char * must be UTF-8 encoded.  string_len
      * allows non-terminated strings and strings with embedded NULs
-     * (though the handling of the latter is format-dependent).
+     * (though the handling of the latter is format-dependent). For
+     * string (but not string_len) the string pointer passed may be
+     * NULL.
      */
     void (*string) (struct sprinter *, const char *);
     void (*string_len) (struct sprinter *, const char *, size_t);
diff --git a/test/missing-headers b/test/missing-headers
index e79f922..f14b878 100755
--- a/test/missing-headers
+++ b/test/missing-headers
@@ -29,7 +29,6 @@ thread:XXX   2001-01-05 [1/1] (null);  (inbox unread)
 thread:XXX   1970-01-01 [1/1] Notmuch Test Suite;  (inbox unread)"
 
 test_begin_subtest "Search: json"
-test_subtest_known_broken
 output=$(notmuch search --format=json '*' | notmuch_search_sanitize)
 test_expect_equal_json "$output" '
 [
@@ -93,7 +92,6 @@ Body
 \fmessage}"
 
 test_begin_subtest "Show: json"
-test_subtest_known_broken
 output=$(notmuch show --format=json '*' | notmuch_json_show_sanitize)
 test_expect_equal_json "$output" '
 [
-- 
1.7.9.1

^ permalink raw reply related	[flat|nested] 12+ messages in thread

* Re: [PATCH] sprinters: bugfix when NULL passed for a string.
  2012-08-08 21:23   ` [PATCH] sprinters: bugfix when NULL passed for a string Mark Walters
@ 2012-08-08 21:28     ` Austin Clements
  2012-08-12 19:51     ` David Bremner
  1 sibling, 0 replies; 12+ messages in thread
From: Austin Clements @ 2012-08-08 21:28 UTC (permalink / raw)
  To: Mark Walters; +Cc: notmuch

LGTM!

Quoth Mark Walters on Aug 08 at 10:23 pm:
> 
> The string function in a sprinter may be called with a NULL string
> pointer (eg if a header is absent). This causes a segfault. We fix
> this by checking for a null pointer in the string functions and update
> the sprinter documentation.
> 
> At the moment some output when format=text is done directly rather than
> via an sprinter: in that case a null pointer is passed to printf or
> similar and a "(null)" appears in the output. That behaviour is not
> changed in this patch.
> ---
> 
> This is the same as id:"874noe1o0r.fsf@qmul.ac.uk" except for being
> based on top of the test in the parent message, and marking the broken
> test fixed.
> 
> Best wishes
> 
> Mark
> 
>  sprinter-json.c      |    2 ++
>  sprinter-text.c      |    2 ++
>  sprinter.h           |    4 +++-
>  test/missing-headers |    2 --
>  4 files changed, 7 insertions(+), 3 deletions(-)
> 
> diff --git a/sprinter-json.c b/sprinter-json.c
> index c9b6835..0a07790 100644
> --- a/sprinter-json.c
> +++ b/sprinter-json.c
> @@ -118,6 +118,8 @@ json_string_len (struct sprinter *sp, const char *val, size_t len)
>  static void
>  json_string (struct sprinter *sp, const char *val)
>  {
> +    if (val == NULL)
> +	val = "";
>      json_string_len (sp, val, strlen (val));
>  }
>  
> diff --git a/sprinter-text.c b/sprinter-text.c
> index dfa54b5..10343be 100644
> --- a/sprinter-text.c
> +++ b/sprinter-text.c
> @@ -38,6 +38,8 @@ text_string_len (struct sprinter *sp, const char *val, size_t len)
>  static void
>  text_string (struct sprinter *sp, const char *val)
>  {
> +    if (val == NULL)
> +	val = "";
>      text_string_len (sp, val, strlen (val));
>  }
>  
> diff --git a/sprinter.h b/sprinter.h
> index 5f43175..912a526 100644
> --- a/sprinter.h
> +++ b/sprinter.h
> @@ -27,7 +27,9 @@ typedef struct sprinter {
>       * a list or map, followed or preceded by separators).  For string
>       * and string_len, the char * must be UTF-8 encoded.  string_len
>       * allows non-terminated strings and strings with embedded NULs
> -     * (though the handling of the latter is format-dependent).
> +     * (though the handling of the latter is format-dependent). For
> +     * string (but not string_len) the string pointer passed may be
> +     * NULL.
>       */
>      void (*string) (struct sprinter *, const char *);
>      void (*string_len) (struct sprinter *, const char *, size_t);
> diff --git a/test/missing-headers b/test/missing-headers
> index e79f922..f14b878 100755
> --- a/test/missing-headers
> +++ b/test/missing-headers
> @@ -29,7 +29,6 @@ thread:XXX   2001-01-05 [1/1] (null);  (inbox unread)
>  thread:XXX   1970-01-01 [1/1] Notmuch Test Suite;  (inbox unread)"
>  
>  test_begin_subtest "Search: json"
> -test_subtest_known_broken
>  output=$(notmuch search --format=json '*' | notmuch_search_sanitize)
>  test_expect_equal_json "$output" '
>  [
> @@ -93,7 +92,6 @@ Body
>  \fmessage}"
>  
>  test_begin_subtest "Show: json"
> -test_subtest_known_broken
>  output=$(notmuch show --format=json '*' | notmuch_json_show_sanitize)
>  test_expect_equal_json "$output" '
>  [

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [PATCH v2] test: Add test for messages with missing headers
  2012-08-08  1:28 ` [PATCH v2] " Austin Clements
                     ` (2 preceding siblings ...)
  2012-08-08 21:23   ` [PATCH] sprinters: bugfix when NULL passed for a string Mark Walters
@ 2012-08-12 19:50   ` David Bremner
  3 siblings, 0 replies; 12+ messages in thread
From: David Bremner @ 2012-08-12 19:50 UTC (permalink / raw)
  To: Austin Clements, notmuch

Austin Clements <amdragon@MIT.EDU> writes:

> Currently the JSON tests for search and show are broken because
> notmuch attempts to dereference a NULL pointer.

pushed, 

d

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [PATCH] sprinters: bugfix when NULL passed for a string.
  2012-08-08 21:23   ` [PATCH] sprinters: bugfix when NULL passed for a string Mark Walters
  2012-08-08 21:28     ` Austin Clements
@ 2012-08-12 19:51     ` David Bremner
  1 sibling, 0 replies; 12+ messages in thread
From: David Bremner @ 2012-08-12 19:51 UTC (permalink / raw)
  To: Mark Walters, Austin Clements, notmuch

Mark Walters <markwalters1009@gmail.com> writes:

> The string function in a sprinter may be called with a NULL string
> pointer (eg if a header is absent). This causes a segfault. We fix
> this by checking for a null pointer in the string functions 

pushed, 

d

^ permalink raw reply	[flat|nested] 12+ messages in thread

end of thread, other threads:[~2012-08-12 19:51 UTC | newest]

Thread overview: 12+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2012-08-07 21:21 [PATCH] test: Add test for messages with missing headers Austin Clements
2012-08-07 22:43 ` Mark Walters
2012-08-08  1:28 ` [PATCH v2] " Austin Clements
2012-08-08  7:36   ` Mark Walters
2012-08-08  8:08   ` Tomi Ollila
2012-08-08 15:46     ` Austin Clements
2012-08-08 18:47       ` Tomi Ollila
2012-08-08 19:10         ` Austin Clements
2012-08-08 21:23   ` [PATCH] sprinters: bugfix when NULL passed for a string Mark Walters
2012-08-08 21:28     ` Austin Clements
2012-08-12 19:51     ` David Bremner
2012-08-12 19:50   ` [PATCH v2] test: Add test for messages with missing headers David Bremner

Code repositories for project(s) associated with this public inbox

	https://yhetil.org/notmuch.git/

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).