From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <dkg@fifthhorseman.net>
Received: from localhost (localhost [127.0.0.1])
 by arlo.cworth.org (Postfix) with ESMTP id 2C6436DE1416
 for <notmuch@notmuchmail.org>; Sun, 26 May 2019 15:16:23 -0700 (PDT)
X-Virus-Scanned: Debian amavisd-new at cworth.org
X-Spam-Flag: NO
X-Spam-Score: -0.176
X-Spam-Level: 
X-Spam-Status: No, score=-0.176 tagged_above=-999 required=5 tests=[AWL=0.025, 
 DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1,
 DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001]
 autolearn=disabled
Received: from arlo.cworth.org ([127.0.0.1])
 by localhost (arlo.cworth.org [127.0.0.1]) (amavisd-new, port 10024)
 with ESMTP id T8-aYoVEZd5M for <notmuch@notmuchmail.org>;
 Sun, 26 May 2019 15:16:20 -0700 (PDT)
Received: from che.mayfirst.org (che.mayfirst.org [162.247.75.118])
 by arlo.cworth.org (Postfix) with ESMTPS id EBAEA6DE1300
 for <notmuch@notmuchmail.org>; Sun, 26 May 2019 15:16:19 -0700 (PDT)
DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/simple; 
 d=fifthhorseman.net; i=@fifthhorseman.net; q=dns/txt; 
 s=2019; t=1558908977; h=from : to : cc : subject : date : 
 message-id : in-reply-to : references : mime-version : 
 content-transfer-encoding : from; 
 bh=OWO8GKDfmXUf4nJfYor75+OJbxK3x85Nxl8YYWXwnds=; 
 b=FaVwiRrx5ff/60GJ9GyaLD8XG4vpvM8+x3DI6vjlMq4pPKS4cgCjVNyB
 yBWM1cTxaLzmhUNvc9tnk6/T/rEuBQ==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=fifthhorseman.net; 
 i=@fifthhorseman.net; q=dns/txt; s=2019rsa; t=1558908977; 
 h=from : to : cc : subject : date : message-id : 
 in-reply-to : references : mime-version : 
 content-transfer-encoding : from; 
 bh=OWO8GKDfmXUf4nJfYor75+OJbxK3x85Nxl8YYWXwnds=; 
 b=V+z2I9Q99q+ibtzlxc3Y2cxgTrapOkGwyj8354RFnAC9J/4XaQVp4WcF
 tSBa+cyz1i7h/N4XnG8BctV70FQ9CY8tPpdnxPz4kHHtIoAY1jUAVFQkkE
 yAsySS4yLvsjFdWto0vJD4M9l1rRJpSl/91zQMxricsd9Q7O3+NoTgNn3r
 szqUECnp9o1vhAOChm8jdQV4n94QRKSX8mIWNaoSBIwC+LgF9IHmuZu/Y/
 G/aI3jeNprv5pi6bGhkMg+JmolU6AZdn6r5x6sOUzwzEisDe9jvQBJwdmY
 Mb/TrgAgDaJu2Q7U6MChRcvmGFlZv3DO37B5a5xz7hd29+VfdTD+cg==
Received: from fifthhorseman.net (cpe-74-71-53-242.nyc.res.rr.com
 [74.71.53.242])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
 (No client certificate requested)
 by che.mayfirst.org (Postfix) with ESMTPSA id 41A29F99D;
 Sun, 26 May 2019 18:16:17 -0400 (EDT)
Received: by fifthhorseman.net (Postfix, from userid 1000)
 id 4B9892076F; Sun, 26 May 2019 18:16:14 -0400 (EDT)
From: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
To: Notmuch Mail <notmuch@notmuchmail.org>
Subject: [PATCH v2 03/17] test: new test framework to compare json parts
Date: Sun, 26 May 2019 18:15:56 -0400
Message-Id: <20190526221610.2833-4-dkg@fifthhorseman.net>
X-Mailer: git-send-email 2.20.1
In-Reply-To: <20190526221610.2833-1-dkg@fifthhorseman.net>
References: <20190526221610.2833-1-dkg@fifthhorseman.net>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-BeenThere: notmuch@notmuchmail.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Use and development of the notmuch mail system."
 <notmuch.notmuchmail.org>
List-Unsubscribe: <https://notmuchmail.org/mailman/options/notmuch>,
 <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>
List-Archive: <http://notmuchmail.org/pipermail/notmuch/>
List-Post: <mailto:notmuch@notmuchmail.org>
List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>
List-Subscribe: <https://notmuchmail.org/mailman/listinfo/notmuch>,
 <mailto:notmuch-request@notmuchmail.org?subject=subscribe>
X-List-Received-Date: Sun, 26 May 2019 22:16:23 -0000

From: Jameson Graef Rollins <jrollins@finestructure.net>

This makes it easier to write fairly compact, readable tests of json
output, without needing to sanitize away parts that we don't care
about.

Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
---
 test/json_check_nodes.py | 113 +++++++++++++++++++++++++++++++++++++++
 test/test-lib.sh         |  24 +++++++++
 2 files changed, 137 insertions(+)
 create mode 100755 test/json_check_nodes.py

diff --git a/test/json_check_nodes.py b/test/json_check_nodes.py
new file mode 100755
index 00000000..afc19f4b
--- /dev/null
+++ b/test/json_check_nodes.py
@@ -0,0 +1,113 @@
+#!/usr/bin/env python
+import re
+import sys
+import json
+
+
+EXPR_RE = re.compile('(?P<label>[a-zA-Z0-9_-]+):(?P<address>[^=!]+)(?:(?P<type>[=!])(?P<val>.*))?', re.DOTALL|re.MULTILINE)
+
+
+if len(sys.argv) < 2:
+    sys.exit('usage: '+ sys.argv[0] + """ EXPR [EXPR]
+
+Takes json data on stdin and evaluates test expressions specified in
+arguments.  Each test is evaluated, and output is printed only if the
+test fails.  If any test fails the return value of execution will be
+non-zero.
+
+EXPR can be one of following types:
+
+Value test: test that object in json data found at address is equal to specified value:
+
+  label:address|value
+
+Existence test: test that dict or list in json data found at address
+does *not* contain the specified key:
+
+  label:address!key
+
+Extract: extract object from json data found at address and print
+
+  label:address
+
+Results are printed to stdout prefixed by expression label.  In all
+cases the test will fail if object does not exist in data.
+
+Example:
+
+0 $ echo '["a", "b", {"c": 1}]' | python3 json_check_nodes.py 'second_d:[1]="d"' 'no_c:[2]!"c"'
+second_d: value not equal: data[1] = 'b' != 'd'
+no_c: dict contains key: data[2]["c"] = "c"
+1 $
+
+""")
+
+
+# parse expressions from arguments
+exprs = []
+for expr in sys.argv[1:]:
+    m = re.match(EXPR_RE, expr)
+    if not m:
+        sys.exit("Invalid expression: {}".format(expr))
+    exprs.append(m)
+
+data = json.load(sys.stdin)
+
+fail = False
+
+for expr in exprs:
+    # print(expr.groups(),fail)
+
+    e = 'data{}'.format(expr.group('address'))
+    try:
+        val = eval(e)
+    except SyntaxError:
+        fail = True
+        print("{}: syntax error on evaluation of object: {}".format(
+            expr.group('label'), e))
+        continue
+    except:
+        fail = True
+        print("{}: object not found: data{}".format(
+            expr.group('label'), expr.group('address')))
+        continue
+
+    if expr.group('type') == '=':
+        try:
+            obj_val = json.loads(expr.group('val'))
+        except:
+            fail = True
+            print("{}: error evaluating value: {}".format(
+                expr.group('label'), expr.group('address')))
+            continue
+        if val != obj_val:
+            fail = True
+            print("{}: value not equal: data{} = {} != {}".format(
+                expr.group('label'), expr.group('address'), repr(val), repr(obj_val)))
+
+    elif expr.group('type') == '!':
+        if not isinstance(val, (dict, list)):
+            fail = True
+            print("{}: not a dict or a list: data{}".format(
+                expr.group('label'), expr.group('address')))
+            continue
+        try:
+            idx = json.loads(expr.group('val'))
+            if idx in val:
+                fail = True
+                print("{}: {} contains key: {}[{}] = {}".format(
+                    expr.group('label'), type(val), e, expr.group('val'), val[idx]))
+        except SyntaxError:
+            fail = True
+            print("{}: syntax error on evaluation of value: {}".format(
+                expr.group('label'), expr.group('val')))
+            continue
+
+
+    elif expr.group('type') is None:
+        print("{}: {}".format(expr.group('label'), val))
+
+
+if fail:
+    sys.exit(1)
+sys.exit(0)
diff --git a/test/test-lib.sh b/test/test-lib.sh
index ff18fae6..616cb674 100644
--- a/test/test-lib.sh
+++ b/test/test-lib.sh
@@ -507,6 +507,30 @@ test_sort_json () {
         "import sys, json; json.dump(sorted(json.load(sys.stdin)),sys.stdout)"
 }
 
+# test for json objects:
+# read the source of test/json_check_nodes.py (or the output when
+# invoking it without arguments) for an explanation of the syntax.
+test_json_nodes () {
+        exec 1>&6 2>&7		# Restore stdout and stderr
+	if [ -z "$inside_subtest" ]; then
+		error "bug in the test script: test_json_eval without test_begin_subtest"
+	fi
+	inside_subtest=
+	test "$#" > 0 ||
+	    error "bug in the test script: test_json_nodes needs at least 1 parameter"
+
+	if ! test_skip "$test_subtest_name"
+	then
+	    output=$(PYTHONIOENCODING=utf-8 $NOTMUCH_PYTHON "$TEST_DIRECTORY"/json_check_nodes.py "$@")
+		if [ "$?" = 0 ]
+		then
+			test_ok_
+		else
+			test_failure_ "$output"
+		fi
+	fi
+}
+
 test_emacs_expect_t () {
 	test "$#" = 1 ||
 	error "bug in the test script: not 1 parameter to test_emacs_expect_t"
-- 
2.20.1