From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from localhost (localhost [127.0.0.1]) by olra.theworths.org (Postfix) with ESMTP id 2238F431FBD for ; Wed, 8 Aug 2012 08:46:13 -0700 (PDT) X-Virus-Scanned: Debian amavisd-new at olra.theworths.org X-Spam-Flag: NO X-Spam-Score: -0.7 X-Spam-Level: X-Spam-Status: No, score=-0.7 tagged_above=-999 required=5 tests=[RCVD_IN_DNSWL_LOW=-0.7] autolearn=disabled Received: from olra.theworths.org ([127.0.0.1]) by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id EPk1K47ghroK for ; Wed, 8 Aug 2012 08:46:12 -0700 (PDT) Received: from dmz-mailsec-scanner-4.mit.edu (DMZ-MAILSEC-SCANNER-4.MIT.EDU [18.9.25.15]) by olra.theworths.org (Postfix) with ESMTP id 19199431FAE for ; Wed, 8 Aug 2012 08:46:12 -0700 (PDT) X-AuditID: 1209190f-b7f306d0000008b4-51-502289c3e2eb Received: from mailhub-auth-1.mit.edu ( [18.9.21.35]) by dmz-mailsec-scanner-4.mit.edu (Symantec Messaging Gateway) with SMTP id 06.15.02228.3C982205; Wed, 8 Aug 2012 11:46:11 -0400 (EDT) Received: from outgoing.mit.edu (OUTGOING-AUTH.MIT.EDU [18.7.22.103]) by mailhub-auth-1.mit.edu (8.13.8/8.9.2) with ESMTP id q78FkAjX011907; Wed, 8 Aug 2012 11:46:11 -0400 Received: from awakening.csail.mit.edu (awakening.csail.mit.edu [18.26.4.91]) (authenticated bits=0) (User authenticated as amdragon@ATHENA.MIT.EDU) by outgoing.mit.edu (8.13.6/8.12.4) with ESMTP id q78Fk9l8018579 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=NOT); Wed, 8 Aug 2012 11:46:10 -0400 (EDT) Received: from amthrax by awakening.csail.mit.edu with local (Exim 4.77) (envelope-from ) id 1Sz8SW-0006ZA-Se; Wed, 08 Aug 2012 11:46:08 -0400 Date: Wed, 8 Aug 2012 11:46:08 -0400 From: Austin Clements To: Tomi Ollila Subject: Re: [PATCH v2] test: Add test for messages with missing headers Message-ID: <20120808154608.GD11179@mit.edu> References: <1344374501-28185-1-git-send-email-amdragon@mit.edu> <1344389313-7886-1-git-send-email-amdragon@mit.edu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFmphleLIzCtJLcpLzFFi42IR4hRV1j3cqRRg8HuVhsX1mzOZLd6snMfq wORx+OtCFo9nq24xBzBFcdmkpOZklqUW6dslcGV8mTuHsWCuTcWPSZuYGxgPaHQxcnJICJhI PJ74iRHCFpO4cG89WxcjF4eQwD5Gia5PGxkhnPWMEq82f4RyTjBJTJq+mAXCWcIoseJzN3MX IwcHi4CKxNf/7CCj2AQ0JLbtXw42VgQo/KBtPSuIzSwgLfHtdzMTiC0s4CHx4O18sDivgI7E vCtrWCFmzmGU+L+jnR0iIShxcuYTFohmLYkb/14ygewCGbT8HwdImFPAQKLl4UewmaJAu6ac 3MY2gVFoFpLuWUi6ZyF0L2BkXsUom5JbpZubmJlTnJqsW5ycmJeXWqRropebWaKXmlK6iREc 2JL8Oxi/HVQ6xCjAwajEwxudrBggxJpYVlyZe4hRkoNJSZTXsk0pQIgvKT+lMiOxOCO+qDQn tfgQowQHs5II79tgoBxvSmJlVWpRPkxKmoNFSZz3aspNfyGB9MSS1OzU1ILUIpisDAeHkgSv NzCChQSLUtNTK9Iyc0oQ0kwcnCDDeYCGb+0AGV5ckJhbnJkOkT/FqCglzssB0iwAksgozYPr hSWeV4ziQK8I8zKDVPEAkxZc9yugwUxAgw+Lgg0uSURISTUwqu5OsubWEL/J/+FsjcYig7xZ /2RZ/ZpEPY5cls403LX4tNai4zPKF/3lzn8k+dRGIuMTc1GZ/4JjSiU+P99XCEY/tMn/z/rc T6z38NuHi188y7j9p6BOikGEV7ckWFj7MIejY/SbZXwNaUnB92vb9KZHzBXavMfzteBSnh2P jfc13poYVC6rxFKckWioxVxUnAgAg5y81hcDAAA= Cc: notmuch@notmuchmail.org X-BeenThere: notmuch@notmuchmail.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Aug 2012 15:46:13 -0000 Quoth Tomi Ollila on Aug 08 at 11:08 am: > On Wed, Aug 08 2012, Austin Clements wrote: > > > Currently the JSON tests for search and show are broken because > > notmuch attempts to dereference a NULL pointer. > > --- > > This version fixes the "Show: text" test so that it sanitize its > > output and doesn't hard-code my test paths. > > +1 > > Tomi > > w/ python json(.tool) the original order cannot be preserved as the parser > stores content into dictionary -- without sorting those came in some > internal python order... the following code could be used to use less To be fair, it can output them in original order, but doing so requires Python 2.7: python -c 'import sys,json,collections; json.dump(json.load(sys.stdin, object_pairs_hook=collections.OrderedDict), sys.stdout, indent=4)' > indentation, though: > > python -c 'import sys,json; j = json.load(sys.stdin); > json.dump(j, sys.stdout, sort_keys=True, indent=2)' < input_file > > The other "problem" with json.tool is that it converts non-ascii chars > to \uNNNN values :/. This one doesn't require 2.7. python -c 'import sys,json,codecs; json.dump(json.load(sys.stdin), codecs.getwriter("utf8")(sys.stdout), indent=4, ensure_ascii=False)' Though I think that, for test canonicalization, \uNNNN is probably less error/locale prone. > What we could do is to dig a simple c json formatter -- someday in the > future, maybe -- but for *now* this is the best we can have :D Not another JSON parser/printer! > > > > test/missing-headers | 162 ++++++++++++++++++++++++++++++++++++++++++++++++++ > > test/notmuch-test | 1 + > > 2 files changed, 163 insertions(+) > > create mode 100755 test/missing-headers > > > > diff --git a/test/missing-headers b/test/missing-headers > > new file mode 100755 > > index 0000000..e79f922 > > --- /dev/null > > +++ b/test/missing-headers > > @@ -0,0 +1,162 @@ > > +#!/usr/bin/env bash > > +test_description='messages with missing headers' > > +. ./test-lib.sh > > + > > +# Notmuch requires at least one of from, subject, or to or it will > > +# ignore the file. Generate two messages so that together they cover > > +# all possible missing headers. We also give one of the messages a > > +# date to ensure stable result ordering. > > + > > +cat < "${MAIL_DIR}/msg-2" > > +To: Notmuch Test Suite > > +Date: Fri, 05 Jan 2001 15:43:57 +0000 > > + > > +Body > > +EOF > > + > > +cat < "${MAIL_DIR}/msg-1" > > +From: Notmuch Test Suite > > + > > +Body > > +EOF > > + > > +NOTMUCH_NEW > > + > > +test_begin_subtest "Search: text" > > +output=$(notmuch search '*' | notmuch_search_sanitize) > > +test_expect_equal "$output" "\ > > +thread:XXX 2001-01-05 [1/1] (null); (inbox unread) > > +thread:XXX 1970-01-01 [1/1] Notmuch Test Suite; (inbox unread)" > > + > > +test_begin_subtest "Search: json" > > +test_subtest_known_broken > > +output=$(notmuch search --format=json '*' | notmuch_search_sanitize) > > +test_expect_equal_json "$output" ' > > +[ > > + { > > + "authors": "", > > + "date_relative": "2001-01-05", > > + "matched": 1, > > + "subject": "", > > + "tags": [ > > + "inbox", > > + "unread" > > + ], > > + "thread": "XXX", > > + "timestamp": 978709437, > > + "total": 1 > > + }, > > + { > > + "authors": "Notmuch Test Suite", > > + "date_relative": "1970-01-01", > > + "matched": 1, > > + "subject": "", > > + "tags": [ > > + "inbox", > > + "unread" > > + ], > > + "thread": "XXX", > > + "timestamp": 0, > > + "total": 1 > > + } > > +]' > > + > > +test_begin_subtest "Show: text" > > +output=$(notmuch show '*' | notmuch_show_sanitize) > > +test_expect_equal "$output" "\ > > + message{ id:notmuch-sha1-7a6e4eac383ef958fcd3ebf2143db71b8ff01161 depth:0 match:1 excluded:0 filename:/XXX/mail/msg-2 > > + header{ > > + (2001-01-05) (inbox unread) > > +Subject: (null) > > +From: (null) > > +To: Notmuch Test Suite > > +Date: Fri, 05 Jan 2001 15:43:57 +0000 > > + header} > > + body{ > > + part{ ID: 1, Content-type: text/plain > > +Body > > + part} > > + body} > > + message} > > + message{ id:notmuch-sha1-ca55943aff7a72baf2ab21fa74fab3d632401334 depth:0 match:1 excluded:0 filename:/XXX/mail/msg-1 > > + header{ > > +Notmuch Test Suite (1970-01-01) (inbox unread) > > +Subject: (null) > > +From: Notmuch Test Suite > > +Date: Thu, 01 Jan 1970 00:00:00 +0000 > > + header} > > + body{ > > + part{ ID: 1, Content-type: text/plain > > +Body > > + part} > > + body} > > + message}" > > + > > +test_begin_subtest "Show: json" > > +test_subtest_known_broken > > +output=$(notmuch show --format=json '*' | notmuch_json_show_sanitize) > > +test_expect_equal_json "$output" ' > > +[ > > + [ > > + [ > > + { > > + "body": [ > > + { > > + "content": "Body\n", > > + "content-type": "text/plain", > > + "id": 1 > > + } > > + ], > > + "date_relative": "2001-01-05", > > + "excluded": false, > > + "filename": "YYYYY", > > + "headers": { > > + "Date": "Fri, 05 Jan 2001 15:43:57 +0000", > > + "From": "", > > + "Subject": "", > > + "To": "Notmuch Test Suite " > > + }, > > + "id": "XXXXX", > > + "match": true, > > + "tags": [ > > + "inbox", > > + "unread" > > + ], > > + "timestamp": 978709437 > > + }, > > + [] > > + ] > > + ], > > + [ > > + [ > > + { > > + "body": [ > > + { > > + "content": "Body\n", > > + "content-type": "text/plain", > > + "id": 1 > > + } > > + ], > > + "date_relative": "1970-01-01", > > + "excluded": false, > > + "filename": "YYYYY", > > + "headers": { > > + "Date": "Thu, 01 Jan 1970 00:00:00 +0000", > > + "From": "Notmuch Test Suite ", > > + "Subject": "" > > + }, > > + "id": "XXXXX", > > + "match": true, > > + "tags": [ > > + "inbox", > > + "unread" > > + ], > > + "timestamp": 0 > > + }, > > + [] > > + ] > > + ] > > +]' > > + > > + > > +test_done > > diff --git a/test/notmuch-test b/test/notmuch-test > > index ea39dfc..cc732c3 100755 > > --- a/test/notmuch-test > > +++ b/test/notmuch-test > > @@ -59,6 +59,7 @@ TESTS=" > > emacs-address-cleaning > > emacs-hello > > emacs-show > > + missing-headers > > " > > TESTS=${NOTMUCH_TESTS:=$TESTS} > > > > > > _______________________________________________ > > notmuch mailing list > > notmuch@notmuchmail.org > > http://notmuchmail.org/mailman/listinfo/notmuch