Support running test suite using python3

Support running test suite using python3

[David Bremner]

On systems where the default python is python2 (i.e. most of them),
there's not convenient way to try out the test suite with python3.
This series aims to make that easier by making

% PYTHON=python3 ./configure

propagate something useful to the test suite.

The last patch is not strictly needed, but it helped me figure out
what was wrong when the python3 version of sphinx was not installed.

[PATCH 1/4] test: make python tests compatible with python3

[David Bremner]

Making the test suite actually run them with python3 is left for
future work.
---
 test/T390-python.sh | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/test/T390-python.sh b/test/T390-python.sh
index 3f03a2e..26d0b97 100755
--- a/test/T390-python.sh
+++ b/test/T390-python.sh
@@ -11,7 +11,7 @@ db = notmuch.Database(mode=notmuch.Database.MODE.READ_ONLY)
 q_new = notmuch.Query(db, 'tag:inbox')
 q_new.set_sort(notmuch.Query.SORT.OLDEST_FIRST)
 for t in q_new.search_threads():
-    print t.get_thread_id()
+    print (t.get_thread_id())
 EOF
 notmuch search --sort=oldest-first --output=threads tag:inbox | sed s/^thread:// > EXPECTED
 test_expect_equal_file OUTPUT EXPECTED
@@ -23,7 +23,7 @@ db = notmuch.Database(mode=notmuch.Database.MODE.READ_ONLY)
 q_new = notmuch.Query(db, 'tag:inbox')
 q_new.set_sort(notmuch.Query.SORT.OLDEST_FIRST)
 for m in q_new.search_messages():
-    print m.get_message_id()
+    print (m.get_message_id())
 EOF
 notmuch search --sort=oldest-first --output=messages tag:inbox | sed s/^id:// > EXPECTED
 test_expect_equal_file OUTPUT EXPECTED
@@ -32,7 +32,7 @@ test_begin_subtest "get non-existent file"
 test_python <<EOF
 import notmuch
 db = notmuch.Database(mode=notmuch.Database.MODE.READ_ONLY)
-print db.find_message_by_filename("i-dont-exist")
+print (db.find_message_by_filename("i-dont-exist"))
 EOF
 test_expect_equal "$(cat OUTPUT)" "None"
 
-- 
2.1.4

[PATCH 2/4] configure: add ability to force python version via environment

[David Bremner]

This is makes it a bit easier to run our test suite under alternative
python versions.
---
 configure | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/configure b/configure
index 2065fcd..71eef6c 100755
--- a/configure
+++ b/configure
@@ -51,6 +51,7 @@ CXXFLAGS_for_sh=${CXXFLAGS:-${CFLAGS}}
 CXXFLAGS=${CXXFLAGS:-\$(CFLAGS)}
 LDFLAGS=${LDFLAGS:-}
 XAPIAN_CONFIG=${XAPIAN_CONFIG:-xapian-config}
+PYTHON=${PYTHON:-}
 
 # We don't allow the EMACS or GZIP Makefile variables inherit values
 # from the environment as we do with CC and CXX above. The reason is
@@ -407,7 +408,7 @@ fi
 printf "Checking for python... "
 have_python=0
 
-for name in python python2 python3; do
+for name in ${PYTHON} python python2 python3; do
     if command -v $name > /dev/null; then
 	have_python=1
 	python=$name
-- 
2.1.4

[PATCH 3/4] test: use the python interpreter in sh.configure

[David Bremner]

The configure script chooses "python" if it exists, so this could
change the version of python used to run the test suite.
---
 configure        |  2 ++
 test/test-lib.sh | 14 +++-----------
 2 files changed, 5 insertions(+), 11 deletions(-)

diff --git a/configure b/configure
index 71eef6c..1081061 100755
--- a/configure
+++ b/configure
@@ -111,6 +111,8 @@ Other environment variables can be used to control configure itself,
 	XAPIAN_CONFIG	The program to use to determine flags for
 			compiling and linking against the Xapian
 			library. [$XAPIAN_CONFIG]
+	PYTHON		Name of python command to use in
+			configure and the test suite.
 
 Additionally, various options can be specified on the configure
 command line.
diff --git a/test/test-lib.sh b/test/test-lib.sh
index 486d1c4..9b63a7a 100644
--- a/test/test-lib.sh
+++ b/test/test-lib.sh
@@ -621,9 +621,9 @@ test_expect_equal_json () {
     # The test suite forces LC_ALL=C, but this causes Python 3 to
     # decode stdin as ASCII.  We need to read JSON in UTF-8, so
     # override Python's stdio encoding defaults.
-    output=$(echo "$1" | PYTHONIOENCODING=utf-8 python -mjson.tool \
+    output=$(echo "$1" | PYTHONIOENCODING=utf-8 $NOTMUCH_PYTHON -mjson.tool \
         || echo "$1")
-    expected=$(echo "$2" | PYTHONIOENCODING=utf-8 python -mjson.tool \
+    expected=$(echo "$2" | PYTHONIOENCODING=utf-8 $NOTMUCH_PYTHON -mjson.tool \
         || echo "$2")
     shift 2
     test_expect_equal "$output" "$expected" "$@"
@@ -1153,14 +1153,8 @@ test_python() {
 	export LD_LIBRARY_PATH=$TEST_DIRECTORY/../lib
 	export PYTHONPATH=$TEST_DIRECTORY/../bindings/python
 
-	# Some distros (e.g. Arch Linux) ship Python 2.* as /usr/bin/python2,
-	# most others as /usr/bin/python. So first try python2, and fallback to
-	# python if python2 doesn't exist.
-	cmd=python2
-	[[ ${test_missing_external_prereq_[python2]} == t ]] && cmd=python
-
 	(echo "import sys; _orig_stdout=sys.stdout; sys.stdout=open('OUTPUT', 'w')"; cat) \
-		| $cmd -
+		| $NOTMUCH_PYTHON -
 }
 
 test_C () {
@@ -1320,5 +1314,3 @@ test_declare_external_prereq emacs
 test_declare_external_prereq ${TEST_EMACSCLIENT}
 test_declare_external_prereq gdb
 test_declare_external_prereq gpg
-test_declare_external_prereq python
-test_declare_external_prereq python2
-- 
2.1.4

[PATCH 4/4] test: redirect man output to /dev/null

[David Bremner]

In the case the these tests fail, they generate a bunch of output;
this output is not very interesting because it is just the successful
output of a man page. It does however make it hard to see what tests are actually failing, even with NOTMUCH_TEST_QUIET
---
 test/T010-help-test.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/test/T010-help-test.sh b/test/T010-help-test.sh
index caf8bdb..d7266ff 100755
--- a/test/T010-help-test.sh
+++ b/test/T010-help-test.sh
@@ -12,9 +12,9 @@ if [ $NOTMUCH_HAVE_MAN -eq 1 ]; then
     test_expect_success 'notmuch help tag' 'notmuch help tag'
 else
     test_expect_success 'notmuch --help tag (man pages not available)' \
-	'test_must_fail notmuch --help tag'
+	'test_must_fail notmuch --help tag >/dev/null'
     test_expect_success 'notmuch help tag (man pages not available)' \
-	'test_must_fail notmuch help tag'
+	'test_must_fail notmuch help tag >/dev/null'
 fi
 
 test_done
-- 
2.1.4

Re: [PATCH 3/4] test: use the python interpreter in sh.configure

[David Bremner]

David Bremner <david@tethera.net> writes:

> The configure script chooses "python" if it exists, so this could
> change the version of python used to run the test suite.

I pushed the other 3 patches in this series. This one at least needs a
correction for the commit subject.

d

[PATCH] test: use the python interpreter in sh.config

[David Bremner]

The configure script chooses "python" if both python and python{2,3}
exist exists, so this could change the version of python used to run
the test suite.

The checking for ${NOTMUCH_PYTHON} in the test suite is arguably
over-engineering, since the configure step will fail if it can't find
it.
---

Here is an updated version which fixes the subject typo, and actually
tests for the python binary read from sh.config. I'll probably merge this version unless there are objections

 configure           |  2 ++
 test/T390-python.sh |  2 ++
 test/test-lib.sh    | 15 ++++-----------
 3 files changed, 8 insertions(+), 11 deletions(-)

diff --git a/configure b/configure
index 56f550b..20fbed6 100755
--- a/configure
+++ b/configure
@@ -114,6 +114,8 @@ Other environment variables can be used to control configure itself,
 	XAPIAN_CONFIG	The program to use to determine flags for
 			compiling and linking against the Xapian
 			library. [$XAPIAN_CONFIG]
+	PYTHON		Name of python command to use in
+			configure and the test suite.
 
 Additionally, various options can be specified on the configure
 command line.
diff --git a/test/T390-python.sh b/test/T390-python.sh
index 26d0b97..c3f24f7 100755
--- a/test/T390-python.sh
+++ b/test/T390-python.sh
@@ -2,6 +2,8 @@
 test_description="python bindings"
 . ./test-lib.sh
 
+test_require_external_prereq ${NOTMUCH_PYTHON}
+
 add_email_corpus
 
 test_begin_subtest "compare thread ids"
diff --git a/test/test-lib.sh b/test/test-lib.sh
index 3466e9c..db3b6aa 100644
--- a/test/test-lib.sh
+++ b/test/test-lib.sh
@@ -621,9 +621,9 @@ test_expect_equal_json () {
     # The test suite forces LC_ALL=C, but this causes Python 3 to
     # decode stdin as ASCII.  We need to read JSON in UTF-8, so
     # override Python's stdio encoding defaults.
-    output=$(echo "$1" | PYTHONIOENCODING=utf-8 python -mjson.tool \
+    output=$(echo "$1" | PYTHONIOENCODING=utf-8 $NOTMUCH_PYTHON -mjson.tool \
         || echo "$1")
-    expected=$(echo "$2" | PYTHONIOENCODING=utf-8 python -mjson.tool \
+    expected=$(echo "$2" | PYTHONIOENCODING=utf-8 $NOTMUCH_PYTHON -mjson.tool \
         || echo "$2")
     shift 2
     test_expect_equal "$output" "$expected" "$@"
@@ -1153,14 +1153,8 @@ test_python() {
 	export LD_LIBRARY_PATH=$TEST_DIRECTORY/../lib
 	export PYTHONPATH=$TEST_DIRECTORY/../bindings/python
 
-	# Some distros (e.g. Arch Linux) ship Python 2.* as /usr/bin/python2,
-	# most others as /usr/bin/python. So first try python2, and fallback to
-	# python if python2 doesn't exist.
-	cmd=python2
-	[[ ${test_missing_external_prereq_[python2]} == t ]] && cmd=python
-
 	(echo "import sys; _orig_stdout=sys.stdout; sys.stdout=open('OUTPUT', 'w')"; cat) \
-		| $cmd -
+		| $NOTMUCH_PYTHON -
 }
 
 test_ruby() {
@@ -1325,5 +1319,4 @@ test_declare_external_prereq emacs
 test_declare_external_prereq ${TEST_EMACSCLIENT}
 test_declare_external_prereq gdb
 test_declare_external_prereq gpg
-test_declare_external_prereq python
-test_declare_external_prereq python2
+test_declare_external_prereq ${NOTMUCH_PYTHON}
-- 
2.1.4

Re: [PATCH] test: use the python interpreter in sh.config

[Tomi Ollila]

On Sun, Aug 02 2015, David Bremner <david@tethera.net> wrote:

> The configure script chooses "python" if both python and python{2,3}
> exist exists, so this could change the version of python used to run
> the test suite.

LGTM. tests pass. 

>
> The checking for ${NOTMUCH_PYTHON} in the test suite is arguably
> over-engineering, since the configure step will fail if it can't find
> it.

Unless we accidentally edit NOTMUCH_PYTHON (away) in configure...
(i.e. I parsonally think this comment part is unnecessary, but i'm not
against having it)

Tomi

> ---
>
> Here is an updated version which fixes the subject typo, and actually
> tests for the python binary read from sh.config. I'll probably merge this version unless there are objections
>
>  configure           |  2 ++
>  test/T390-python.sh |  2 ++
>  test/test-lib.sh    | 15 ++++-----------
>  3 files changed, 8 insertions(+), 11 deletions(-)
>
> diff --git a/configure b/configure
> index 56f550b..20fbed6 100755
> --- a/configure
> +++ b/configure
> @@ -114,6 +114,8 @@ Other environment variables can be used to control configure itself,
>  	XAPIAN_CONFIG	The program to use to determine flags for
>  			compiling and linking against the Xapian
>  			library. [$XAPIAN_CONFIG]
> +	PYTHON		Name of python command to use in
> +			configure and the test suite.
>  
>  Additionally, various options can be specified on the configure
>  command line.
> diff --git a/test/T390-python.sh b/test/T390-python.sh
> index 26d0b97..c3f24f7 100755
> --- a/test/T390-python.sh
> +++ b/test/T390-python.sh
> @@ -2,6 +2,8 @@
>  test_description="python bindings"
>  . ./test-lib.sh
>  
> +test_require_external_prereq ${NOTMUCH_PYTHON}
> +
>  add_email_corpus
>  
>  test_begin_subtest "compare thread ids"
> diff --git a/test/test-lib.sh b/test/test-lib.sh
> index 3466e9c..db3b6aa 100644
> --- a/test/test-lib.sh
> +++ b/test/test-lib.sh
> @@ -621,9 +621,9 @@ test_expect_equal_json () {
>      # The test suite forces LC_ALL=C, but this causes Python 3 to
>      # decode stdin as ASCII.  We need to read JSON in UTF-8, so
>      # override Python's stdio encoding defaults.
> -    output=$(echo "$1" | PYTHONIOENCODING=utf-8 python -mjson.tool \
> +    output=$(echo "$1" | PYTHONIOENCODING=utf-8 $NOTMUCH_PYTHON -mjson.tool \
>          || echo "$1")
> -    expected=$(echo "$2" | PYTHONIOENCODING=utf-8 python -mjson.tool \
> +    expected=$(echo "$2" | PYTHONIOENCODING=utf-8 $NOTMUCH_PYTHON -mjson.tool \
>          || echo "$2")
>      shift 2
>      test_expect_equal "$output" "$expected" "$@"
> @@ -1153,14 +1153,8 @@ test_python() {
>  	export LD_LIBRARY_PATH=$TEST_DIRECTORY/../lib
>  	export PYTHONPATH=$TEST_DIRECTORY/../bindings/python
>  
> -	# Some distros (e.g. Arch Linux) ship Python 2.* as /usr/bin/python2,
> -	# most others as /usr/bin/python. So first try python2, and fallback to
> -	# python if python2 doesn't exist.
> -	cmd=python2
> -	[[ ${test_missing_external_prereq_[python2]} == t ]] && cmd=python
> -
>  	(echo "import sys; _orig_stdout=sys.stdout; sys.stdout=open('OUTPUT', 'w')"; cat) \
> -		| $cmd -
> +		| $NOTMUCH_PYTHON -
>  }
>  
>  test_ruby() {
> @@ -1325,5 +1319,4 @@ test_declare_external_prereq emacs
>  test_declare_external_prereq ${TEST_EMACSCLIENT}
>  test_declare_external_prereq gdb
>  test_declare_external_prereq gpg
> -test_declare_external_prereq python
> -test_declare_external_prereq python2
> +test_declare_external_prereq ${NOTMUCH_PYTHON}
> -- 
> 2.1.4
>
> _______________________________________________
> notmuch mailing list
> notmuch@notmuchmail.org
> http://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] test: use the python interpreter in sh.config

[David Bremner]

David Bremner <david@tethera.net> writes:

> The configure script chooses "python" if both python and python{2,3}
> exist exists, so this could change the version of python used to run
> the test suite.
>
> The checking for ${NOTMUCH_PYTHON} in the test suite is arguably
> over-engineering, since the configure step will fail if it can't find
> it.

pushed.

d