unofficial mirror of notmuch@notmuchmail.org
 help / color / mirror / code / Atom feed
From: Austin Clements <amdragon@MIT.EDU>
To: notmuch@notmuchmail.org
Subject: [PATCH] test: Fix UTF-8 JSON tests in Python 3
Date: Tue,  4 Dec 2012 10:24:14 -0500	[thread overview]
Message-ID: <1354634654-9564-1-git-send-email-amdragon@mit.edu> (raw)
In-Reply-To: <87d2yp27zu.fsf@nexoid.at>

test_expect_equal_json uses json.tool from the system Python.  While
Python 2 wasn't picky about the encoding of stdin, Python 3 decodes
stdin strictly according to the environment.  Since we set LC_ALL=C
for the tests, Python 3's json.tool was assuming stdin would be in
ASCII and aborting when it couldn't decode the UTF-8 characters from
some of the JSON tests.  This patch sets the PYTHONIOENCODING
environment variable to utf-8 when invoking json.tool to override
Python's default encoding choice.
---
 test/test-lib.sh |    9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/test/test-lib.sh b/test/test-lib.sh
index f169785..9487526 100644
--- a/test/test-lib.sh
+++ b/test/test-lib.sh
@@ -534,8 +534,13 @@ test_expect_equal_file ()
 # canonicalized before diff'ing.  If an argument cannot be parsed, it
 # is used unchanged so that there's something to diff against.
 test_expect_equal_json () {
-    output=$(echo "$1" | python -mjson.tool || echo "$1")
-    expected=$(echo "$2" | python -mjson.tool || echo "$2")
+    # The test suite forces LC_ALL=C, but this causes Python 3 to
+    # decode stdin as ASCII.  We need to read JSON in UTF-8, so
+    # override Python's stdio encoding defaults.
+    output=$(echo "$1" | PYTHONIOENCODING=utf-8 python -mjson.tool \
+        || echo "$1")
+    expected=$(echo "$2" | PYTHONIOENCODING=utf-8 python -mjson.tool \
+        || echo "$2")
     shift 2
     test_expect_equal "$output" "$expected" "$@"
 }
-- 
1.7.10.4

  reply	other threads:[~2012-12-04 15:24 UTC|newest]

Thread overview: 7+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2012-12-04 13:46 [PATCH 1/1] Fixing failing JSON tests Peter Feigl
2012-12-04 14:35 ` Austin Clements
2012-12-04 14:43   ` Peter Feigl
2012-12-04 15:24     ` Austin Clements [this message]
2012-12-06 21:15       ` [PATCH] test: Fix UTF-8 JSON tests in Python 3 Tomi Ollila
2012-12-07 13:14       ` David Bremner
2012-12-08 13:37       ` David Bremner

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://notmuchmail.org/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1354634654-9564-1-git-send-email-amdragon@mit.edu \
    --to=amdragon@mit.edu \
    --cc=notmuch@notmuchmail.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://yhetil.org/notmuch.git/

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).