[PATCH 00/24] test: separate source and build directories

[PATCH 00/24] test: separate source and build directories

[Jani Nikula]

This lengthy but mostly mechanical series separates all source and build
directory references in the test framework to be able to properly build
and run out-of-tree tests. At the end, the 'cp -a' of the test directory
from source to build can be dropped.

The general idea is that you have current working directory anywhere
within the build directory hierarchy when you run tests, and you
reference the notmuch-test or individual test scripts within the source
directory hierarchy using relative or absolute paths. Or just run 'make
test' in the build directory as usual.

Patch 6 adds a script that figures out the directories, exports them as
environment variables NOTMUCH_SRCDIR and NOTMUCH_BUILDDIR, and those are
used througout the series. The existing TEST_DIRECTORY variable now
specifically means $NOTMUCH_BUILDDIR/test.

Some of the patches toward the end could have been squashed together,
but I thought it's faster and easier to review this way.

BR,
Jani.


Jani Nikula (24):
  test: use corpora/default not corpus in T480-hex-escaping.sh
  build: use NOTMUCH_SRCDIR for referencing notmuch-test
  test: use $(dirname "$0") for sourcing test-lib.sh
  perf-test: use $(dirname "$0") for sourcing perf-test-lib.sh
  test: check for notmuch binary in test-lib.sh
  test: add a script to figure out source and build directories
  test: use source and build paths in test-lib.sh
  perf-test: use source and build paths in perf-test-lib.sh
  test: use source and build paths in test-lib-common.sh
  test: mkdir and cd to $TMP_DIRECTORY instead of $test for testing
  test: use source and build paths in notmuch-test
  test: use source and build paths in T000-basic.sh
  test: use source and build paths in T160-json.sh and T170-sexp.sh
  test: use source and build paths in T260-thread-order.sh
  test: use source and build paths in T310-emacs.sh
  test: use source and build paths in T350-crypto.sh
  test: use source and build paths in T355-smime.sh
  test: use source and build paths in T360-symbol-hiding.sh
  test: use source and build paths in T380-atomicity.sh
  test: use source and build paths in T4?0-emacs-*.sh
  test: use source and build paths in T480-hex-escaping.sh
  test: use source and build paths in T590-thread-breakage.sh
  build: don't copy the test directory for out-of-tree builds
  devel: add script to test out-of-tree builds

 configure                                     |  4 ---
 devel/out-of-tree-build-check.sh              | 16 ++++++++++
 performance-test/M00-new.sh                   |  2 +-
 performance-test/M01-dump-restore.sh          |  2 +-
 performance-test/M02-show.sh                  |  2 +-
 performance-test/M03-search.sh                |  2 +-
 performance-test/M04-reply.sh                 |  2 +-
 performance-test/M05-reindex.sh               |  2 +-
 performance-test/M06-insert.sh                |  2 +-
 performance-test/T00-new.sh                   |  2 +-
 performance-test/T01-dump-restore.sh          |  2 +-
 performance-test/T02-tag.sh                   |  2 +-
 performance-test/T03-reindex.sh               |  2 +-
 performance-test/perf-test-lib.sh             | 12 +++++---
 test/Makefile.local                           |  6 ++--
 test/T000-basic.sh                            | 18 +++--------
 test/T010-help-test.sh                        |  2 +-
 test/T020-compact.sh                          |  2 +-
 test/T030-config.sh                           |  2 +-
 test/T040-setup.sh                            |  2 +-
 test/T050-new.sh                              |  2 +-
 test/T060-count.sh                            |  2 +-
 test/T070-insert.sh                           |  2 +-
 test/T080-search.sh                           |  2 +-
 test/T090-search-output.sh                    |  2 +-
 test/T095-address.sh                          |  2 +-
 test/T100-search-by-folder.sh                 |  2 +-
 test/T110-search-position-overlap-bug.sh      |  2 +-
 test/T120-search-insufficient-from-quoting.sh |  2 +-
 test/T130-search-limiting.sh                  |  2 +-
 test/T140-excludes.sh                         |  2 +-
 test/T150-tagging.sh                          |  2 +-
 test/T160-json.sh                             |  6 ++--
 test/T170-sexp.sh                             |  6 ++--
 test/T180-text.sh                             |  2 +-
 test/T190-multipart.sh                        |  2 +-
 test/T200-thread-naming.sh                    |  2 +-
 test/T205-author-naming.sh                    |  2 +-
 test/T210-raw.sh                              |  2 +-
 test/T220-reply.sh                            |  2 +-
 test/T230-reply-to-sender.sh                  |  2 +-
 test/T240-dump-restore.sh                     |  2 +-
 test/T250-uuencode.sh                         |  2 +-
 test/T260-thread-order.sh                     |  4 +--
 test/T270-author-order.sh                     |  2 +-
 test/T280-from-guessing.sh                    |  2 +-
 test/T290-long-id.sh                          |  2 +-
 test/T300-encoding.sh                         |  2 +-
 test/T310-emacs.sh                            |  6 ++--
 test/T320-emacs-large-search-buffer.sh        |  2 +-
 test/T330-emacs-subject-to-filename.sh        |  2 +-
 test/T340-maildir-sync.sh                     |  2 +-
 test/T350-crypto.sh                           |  4 +--
 test/T355-smime.sh                            | 10 +++---
 test/T360-symbol-hiding.sh                    |  6 ++--
 test/T370-search-folder-coherence.sh          |  2 +-
 test/T380-atomicity.sh                        |  4 +--
 test/T390-python.sh                           |  2 +-
 test/T395-ruby.sh                             |  2 +-
 test/T400-hooks.sh                            |  2 +-
 test/T410-argument-parsing.sh                 |  2 +-
 test/T420-emacs-test-functions.sh             |  2 +-
 test/T430-emacs-address-cleaning.sh           |  2 +-
 test/T440-emacs-hello.sh                      |  4 +--
 test/T450-emacs-show.sh                       |  4 +--
 test/T455-emacs-charsets.sh                   |  2 +-
 test/T460-emacs-tree.sh                       |  4 +--
 test/T470-missing-headers.sh                  |  2 +-
 test/T480-hex-escaping.sh                     |  6 ++--
 test/T490-parse-time-string.sh                |  2 +-
 test/T500-search-date.sh                      |  2 +-
 test/T510-thread-replies.sh                   |  2 +-
 test/T520-show.sh                             |  2 +-
 test/T530-upgrade.sh                          |  2 +-
 test/T550-db-features.sh                      |  2 +-
 test/T560-lib-error.sh                        |  2 +-
 test/T570-revision-tracking.sh                |  2 +-
 test/T580-thread-search.sh                    |  2 +-
 test/T590-libconfig.sh                        |  2 +-
 test/T590-thread-breakage.sh                  |  6 ++--
 test/T600-named-queries.sh                    |  2 +-
 test/T610-message-property.sh                 |  2 +-
 test/T620-lock.sh                             |  2 +-
 test/T630-emacs-draft.sh                      |  2 +-
 test/T640-database-modified.sh                |  2 +-
 test/T650-regexp-query.sh                     |  2 +-
 test/T660-bad-date.sh                         |  2 +-
 test/T670-duplicate-mid.sh                    |  2 +-
 test/T680-html-indexing.sh                    |  2 +-
 test/T690-command-line-args.sh                |  2 +-
 test/T700-reindex.sh                          |  2 +-
 test/export-dirs.sh                           | 32 +++++++++++++++++++
 test/notmuch-test                             | 23 +++++++++-----
 test/test-lib-common.sh                       | 44 +++++++++------------------
 test/test-lib.sh                              | 27 ++++++++++------
 test/test-verbose                             |  2 +-
 96 files changed, 221 insertions(+), 179 deletions(-)
 create mode 100755 devel/out-of-tree-build-check.sh
 create mode 100644 test/export-dirs.sh

-- 
2.11.0

[PATCH 01/24] test: use corpora/default not corpus in T480-hex-escaping.sh

[Jani Nikula]

Turns out round trip tests didn't really round trip anything. Broken
by yours truly in 971cdc72cdb8 ("test: make it possible to have
multiple corpora"). Ooops.
---
 test/T480-hex-escaping.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/test/T480-hex-escaping.sh b/test/T480-hex-escaping.sh
index 86cf37292d85..18b56600480c 100755
--- a/test/T480-hex-escaping.sh
+++ b/test/T480-hex-escaping.sh
@@ -3,7 +3,7 @@ test_description="hex encoding and decoding"
 . ./test-lib.sh || exit 1
 
 test_begin_subtest "round trip"
-find $TEST_DIRECTORY/corpus -type f -print | sort | xargs cat > EXPECTED
+find $TEST_DIRECTORY/corpora/default -type f -print | sort | xargs cat > EXPECTED
 $TEST_DIRECTORY/hex-xcode --direction=encode < EXPECTED | $TEST_DIRECTORY/hex-xcode --direction=decode > OUTPUT
 test_expect_equal_file EXPECTED OUTPUT
 
@@ -25,7 +25,7 @@ $TEST_DIRECTORY/hex-xcode --direction=decode  < EXPECTED.$test_count |\
 test_expect_equal_file EXPECTED.$test_count OUTPUT.$test_count
 
 test_begin_subtest "round trip (in-place)"
-find $TEST_DIRECTORY/corpus -type f -print | sort | xargs cat > EXPECTED
+find $TEST_DIRECTORY/corpora/default -type f -print | sort | xargs cat > EXPECTED
 $TEST_DIRECTORY/hex-xcode --in-place --direction=encode < EXPECTED |\
      $TEST_DIRECTORY/hex-xcode --in-place --direction=decode > OUTPUT
 test_expect_equal_file EXPECTED OUTPUT
-- 
2.11.0

[PATCH 02/24] build: use NOTMUCH_SRCDIR for referencing notmuch-test

[Jani Nikula]

Don't assume the tests are always run from within the source tree.
---
 test/Makefile.local | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/test/Makefile.local b/test/Makefile.local
index 0df72c921654..1a0ab813f996 100644
--- a/test/Makefile.local
+++ b/test/Makefile.local
@@ -62,13 +62,13 @@ test-binaries: $(TEST_BINARIES)
 test:	all test-binaries
 ifeq ($V,)
 	@echo 'Use "$(MAKE) V=1" to see the details for passing and known broken tests.'
-	@env NOTMUCH_TEST_QUIET=1 ${test_src_dir}/notmuch-test $(OPTIONS)
+	@env NOTMUCH_TEST_QUIET=1 $(NOTMUCH_SRCDIR)/$(test_src_dir)/notmuch-test $(OPTIONS)
 else
 # The user has explicitly enabled quiet execution.
 ifeq ($V,0)
-	@env NOTMUCH_TEST_QUIET=1 ${test_src_dir}/notmuch-test $(OPTIONS)
+	@env NOTMUCH_TEST_QUIET=1 $(NOTMUCH_SRCDIR)/$(test_src_dir)/notmuch-test $(OPTIONS)
 else
-	@${test_src_dir}/notmuch-test $(OPTIONS)
+	@$(NOTMUCH_SRCDIR)/$(test_src_dir)/notmuch-test $(OPTIONS)
 endif
 endif
 
-- 
2.11.0

[PATCH 03/24] test: use $(dirname "$0") for sourcing test-lib.sh

[Jani Nikula]

Don't assume the tests are always run from within the source tree.
---
 test/T000-basic.sh                            | 2 +-
 test/T010-help-test.sh                        | 2 +-
 test/T020-compact.sh                          | 2 +-
 test/T030-config.sh                           | 2 +-
 test/T040-setup.sh                            | 2 +-
 test/T050-new.sh                              | 2 +-
 test/T060-count.sh                            | 2 +-
 test/T070-insert.sh                           | 2 +-
 test/T080-search.sh                           | 2 +-
 test/T090-search-output.sh                    | 2 +-
 test/T095-address.sh                          | 2 +-
 test/T100-search-by-folder.sh                 | 2 +-
 test/T110-search-position-overlap-bug.sh      | 2 +-
 test/T120-search-insufficient-from-quoting.sh | 2 +-
 test/T130-search-limiting.sh                  | 2 +-
 test/T140-excludes.sh                         | 2 +-
 test/T150-tagging.sh                          | 2 +-
 test/T160-json.sh                             | 2 +-
 test/T170-sexp.sh                             | 2 +-
 test/T180-text.sh                             | 2 +-
 test/T190-multipart.sh                        | 2 +-
 test/T200-thread-naming.sh                    | 2 +-
 test/T205-author-naming.sh                    | 2 +-
 test/T210-raw.sh                              | 2 +-
 test/T220-reply.sh                            | 2 +-
 test/T230-reply-to-sender.sh                  | 2 +-
 test/T240-dump-restore.sh                     | 2 +-
 test/T250-uuencode.sh                         | 2 +-
 test/T260-thread-order.sh                     | 2 +-
 test/T270-author-order.sh                     | 2 +-
 test/T280-from-guessing.sh                    | 2 +-
 test/T290-long-id.sh                          | 2 +-
 test/T300-encoding.sh                         | 2 +-
 test/T310-emacs.sh                            | 2 +-
 test/T320-emacs-large-search-buffer.sh        | 2 +-
 test/T330-emacs-subject-to-filename.sh        | 2 +-
 test/T340-maildir-sync.sh                     | 2 +-
 test/T350-crypto.sh                           | 2 +-
 test/T355-smime.sh                            | 2 +-
 test/T360-symbol-hiding.sh                    | 2 +-
 test/T370-search-folder-coherence.sh          | 2 +-
 test/T380-atomicity.sh                        | 2 +-
 test/T390-python.sh                           | 2 +-
 test/T395-ruby.sh                             | 2 +-
 test/T400-hooks.sh                            | 2 +-
 test/T410-argument-parsing.sh                 | 2 +-
 test/T420-emacs-test-functions.sh             | 2 +-
 test/T430-emacs-address-cleaning.sh           | 2 +-
 test/T440-emacs-hello.sh                      | 2 +-
 test/T450-emacs-show.sh                       | 2 +-
 test/T455-emacs-charsets.sh                   | 2 +-
 test/T460-emacs-tree.sh                       | 2 +-
 test/T470-missing-headers.sh                  | 2 +-
 test/T480-hex-escaping.sh                     | 2 +-
 test/T490-parse-time-string.sh                | 2 +-
 test/T500-search-date.sh                      | 2 +-
 test/T510-thread-replies.sh                   | 2 +-
 test/T520-show.sh                             | 2 +-
 test/T530-upgrade.sh                          | 2 +-
 test/T550-db-features.sh                      | 2 +-
 test/T560-lib-error.sh                        | 2 +-
 test/T570-revision-tracking.sh                | 2 +-
 test/T580-thread-search.sh                    | 2 +-
 test/T590-libconfig.sh                        | 2 +-
 test/T590-thread-breakage.sh                  | 2 +-
 test/T600-named-queries.sh                    | 2 +-
 test/T610-message-property.sh                 | 2 +-
 test/T620-lock.sh                             | 2 +-
 test/T630-emacs-draft.sh                      | 2 +-
 test/T640-database-modified.sh                | 2 +-
 test/T650-regexp-query.sh                     | 2 +-
 test/T660-bad-date.sh                         | 2 +-
 test/T670-duplicate-mid.sh                    | 2 +-
 test/T680-html-indexing.sh                    | 2 +-
 test/T690-command-line-args.sh                | 2 +-
 test/T700-reindex.sh                          | 2 +-
 test/test-verbose                             | 2 +-
 77 files changed, 77 insertions(+), 77 deletions(-)

diff --git a/test/T000-basic.sh b/test/T000-basic.sh
index a4efa3cab8b5..d514082d48e3 100755
--- a/test/T000-basic.sh
+++ b/test/T000-basic.sh
@@ -14,7 +14,7 @@ then
 	exit 1
 fi
 
-. ./test-lib.sh || exit 1
+. $(dirname "$0")/test-lib.sh || exit 1
 
 ################################################################
 # Test harness
diff --git a/test/T010-help-test.sh b/test/T010-help-test.sh
index 0c833de26217..da45d3aecf7a 100755
--- a/test/T010-help-test.sh
+++ b/test/T010-help-test.sh
@@ -1,7 +1,7 @@
 #!/usr/bin/env bash
 
 test_description="online help"
-. ./test-lib.sh || exit 1
+. $(dirname "$0")/test-lib.sh || exit 1
 
 test_begin_subtest 'notmuch --help'
 test_expect_success 'notmuch --help'
diff --git a/test/T020-compact.sh b/test/T020-compact.sh
index a3d7380e81c8..58cd2ba74cef 100755
--- a/test/T020-compact.sh
+++ b/test/T020-compact.sh
@@ -1,6 +1,6 @@
 #!/usr/bin/env bash
 test_description='"notmuch compact"'
-. ./test-lib.sh || exit 1
+. $(dirname "$0")/test-lib.sh || exit 1
 
 add_message '[subject]=One'
 add_message '[subject]=Two'
diff --git a/test/T030-config.sh b/test/T030-config.sh
index 35d757f6b227..e91c36597e3c 100755
--- a/test/T030-config.sh
+++ b/test/T030-config.sh
@@ -1,7 +1,7 @@
 #!/usr/bin/env bash
 
 test_description='"notmuch config"'
-. ./test-lib.sh || exit 1
+. $(dirname "$0")/test-lib.sh || exit 1
 
 test_begin_subtest "Get string value"
 test_expect_equal "$(notmuch config get user.name)" "Notmuch Test Suite"
diff --git a/test/T040-setup.sh b/test/T040-setup.sh
index 998bd6e0798a..56efe1d57b49 100755
--- a/test/T040-setup.sh
+++ b/test/T040-setup.sh
@@ -1,7 +1,7 @@
 #!/usr/bin/env bash
 
 test_description='"notmuch setup"'
-. ./test-lib.sh || exit 1
+. $(dirname "$0")/test-lib.sh || exit 1
 
 test_begin_subtest "Notmuch new without a config suggests notmuch setup"
 output=$(notmuch --config=new-notmuch-config new 2>&1)
diff --git a/test/T050-new.sh b/test/T050-new.sh
index 272ed417aa2e..2035d29f529a 100755
--- a/test/T050-new.sh
+++ b/test/T050-new.sh
@@ -1,6 +1,6 @@
 #!/usr/bin/env bash
 test_description='"notmuch new" in several variations'
-. ./test-lib.sh || exit 1
+. $(dirname "$0")/test-lib.sh || exit 1
 
 test_begin_subtest "No new messages"
 output=$(NOTMUCH_NEW --debug)
diff --git a/test/T060-count.sh b/test/T060-count.sh
index 4751440e94a1..0c0bf47309e9 100755
--- a/test/T060-count.sh
+++ b/test/T060-count.sh
@@ -1,6 +1,6 @@
 #!/usr/bin/env bash
 test_description='"notmuch count" for messages and threads'
-. ./test-lib.sh || exit 1
+. $(dirname "$0")/test-lib.sh || exit 1
 
 add_email_corpus
 
diff --git a/test/T070-insert.sh b/test/T070-insert.sh
index e56a9d21880e..f1650e623e35 100755
--- a/test/T070-insert.sh
+++ b/test/T070-insert.sh
@@ -1,6 +1,6 @@
 #!/usr/bin/env bash
 test_description='"notmuch insert"'
-. ./test-lib.sh || exit 1
+. $(dirname "$0")/test-lib.sh || exit 1
 
 test_require_external_prereq gdb
 
diff --git a/test/T080-search.sh b/test/T080-search.sh
index 3bb3dced8915..70f2854973a9 100755
--- a/test/T080-search.sh
+++ b/test/T080-search.sh
@@ -1,6 +1,6 @@
 #!/usr/bin/env bash
 test_description='"notmuch search" in several variations'
-. ./test-lib.sh || exit 1
+. $(dirname "$0")/test-lib.sh || exit 1
 
 add_email_corpus
 
diff --git a/test/T090-search-output.sh b/test/T090-search-output.sh
index dccefcb7f012..bf28d220a760 100755
--- a/test/T090-search-output.sh
+++ b/test/T090-search-output.sh
@@ -1,6 +1,6 @@
 #!/usr/bin/env bash
 test_description='various settings for "notmuch search --output="'
-. ./test-lib.sh || exit 1
+. $(dirname "$0")/test-lib.sh || exit 1
 
 add_email_corpus
 
diff --git a/test/T095-address.sh b/test/T095-address.sh
index 5931b1475e0b..f0291d29ec43 100755
--- a/test/T095-address.sh
+++ b/test/T095-address.sh
@@ -1,6 +1,6 @@
 #!/usr/bin/env bash
 test_description='"notmuch address" in several variants'
-. ./test-lib.sh || exit 1
+. $(dirname "$0")/test-lib.sh || exit 1
 
 add_email_corpus
 
diff --git a/test/T100-search-by-folder.sh b/test/T100-search-by-folder.sh
index 79c266e445f5..a090f3d247c4 100755
--- a/test/T100-search-by-folder.sh
+++ b/test/T100-search-by-folder.sh
@@ -1,6 +1,6 @@
 #!/usr/bin/env bash
 test_description='"notmuch search" by folder: and path: (with variations)'
-. ./test-lib.sh || exit 1
+. $(dirname "$0")/test-lib.sh || exit 1
 
 add_message '[dir]=bad' '[subject]="To the bone"'
 add_message '[dir]=.' '[subject]="Top level"'
diff --git a/test/T110-search-position-overlap-bug.sh b/test/T110-search-position-overlap-bug.sh
index 2a4238f95fc9..f4d5ee1479ba 100755
--- a/test/T110-search-position-overlap-bug.sh
+++ b/test/T110-search-position-overlap-bug.sh
@@ -18,7 +18,7 @@
 # id:3wd4o8wa7fx.fsf@testarossa.amd.com
 
 test_description='that notmuch does not overlap term positions'
-. ./test-lib.sh || exit 1
+. $(dirname "$0")/test-lib.sh || exit 1
 
 add_message '[to]="a@b.c, x@y.z"'
 
diff --git a/test/T120-search-insufficient-from-quoting.sh b/test/T120-search-insufficient-from-quoting.sh
index 4862d82644df..509fec8b0f2a 100755
--- a/test/T120-search-insufficient-from-quoting.sh
+++ b/test/T120-search-insufficient-from-quoting.sh
@@ -1,6 +1,6 @@
 #!/usr/bin/env bash
 test_description='messages with unquoted . in name'
-. ./test-lib.sh || exit 1
+. $(dirname "$0")/test-lib.sh || exit 1
 
 add_message \
   '[from]="Some.Name for Someone <bugs@quoting.com>"' \
diff --git a/test/T130-search-limiting.sh b/test/T130-search-limiting.sh
index c8986f4e4057..8a30e7abf87f 100755
--- a/test/T130-search-limiting.sh
+++ b/test/T130-search-limiting.sh
@@ -1,6 +1,6 @@
 #!/usr/bin/env bash
 test_description='"notmuch search" --offset and --limit parameters'
-. ./test-lib.sh || exit 1
+. $(dirname "$0")/test-lib.sh || exit 1
 
 add_email_corpus
 
diff --git a/test/T140-excludes.sh b/test/T140-excludes.sh
index f91d4d7f0cac..0cf69975f442 100755
--- a/test/T140-excludes.sh
+++ b/test/T140-excludes.sh
@@ -1,6 +1,6 @@
 #!/usr/bin/env bash
 test_description='"notmuch search, count and show" with excludes in several variations'
-. ./test-lib.sh || exit 1
+. $(dirname "$0")/test-lib.sh || exit 1
 
 # Generates a thread consisting of a top level message and 'length'
 # replies. The subject of the top message 'subject: top message"
diff --git a/test/T150-tagging.sh b/test/T150-tagging.sh
index 0d0a3b874526..6140c6768630 100755
--- a/test/T150-tagging.sh
+++ b/test/T150-tagging.sh
@@ -1,6 +1,6 @@
 #!/usr/bin/env bash
 test_description='"notmuch tag"'
-. ./test-lib.sh || exit 1
+. $(dirname "$0")/test-lib.sh || exit 1
 
 add_message '[subject]=One'
 add_message '[subject]=Two'
diff --git a/test/T160-json.sh b/test/T160-json.sh
index 07955a2bb90a..765b78a6746d 100755
--- a/test/T160-json.sh
+++ b/test/T160-json.sh
@@ -1,6 +1,6 @@
 #!/usr/bin/env bash
 test_description="--format=json output"
-. ./test-lib.sh || exit 1
+. $(dirname "$0")/test-lib.sh || exit 1
 
 test_begin_subtest "Show message: json"
 add_message "[subject]=\"json-show-subject\"" "[date]=\"Sat, 01 Jan 2000 12:00:00 -0000\"" "[bcc]=\"test_suite+bcc@notmuchmail.org\"" "[reply-to]=\"test_suite+replyto@notmuchmail.org\"" "[body]=\"json-show-message\""
diff --git a/test/T170-sexp.sh b/test/T170-sexp.sh
index db142e49683d..1125fdc91ec0 100755
--- a/test/T170-sexp.sh
+++ b/test/T170-sexp.sh
@@ -1,6 +1,6 @@
 #!/usr/bin/env bash
 test_description="--format=sexp output"
-. ./test-lib.sh || exit 1
+. $(dirname "$0")/test-lib.sh || exit 1
 
 test_begin_subtest "Show message: sexp"
 add_message "[subject]=\"sexp-show-subject\"" "[date]=\"Sat, 01 Jan 2000 12:00:00 -0000\"" "[bcc]=\"test_suite+bcc@notmuchmail.org\"" "[reply-to]=\"test_suite+replyto@notmuchmail.org\"" "[body]=\"sexp-show-message\""
diff --git a/test/T180-text.sh b/test/T180-text.sh
index 3a265dbd5a73..ad2cb1f37c9b 100755
--- a/test/T180-text.sh
+++ b/test/T180-text.sh
@@ -1,6 +1,6 @@
 #!/usr/bin/env bash
 test_description="--format=text output"
-. ./test-lib.sh || exit 1
+. $(dirname "$0")/test-lib.sh || exit 1
 
 test_begin_subtest "Show message: text"
 add_message "[subject]=\"text-show-subject\"" "[date]=\"Sat, 01 Jan 2000 12:00:00 -0000\"" "[body]=\"text-show-message\""
diff --git a/test/T190-multipart.sh b/test/T190-multipart.sh
index 94bb0570a3b2..f73535b7bf7e 100755
--- a/test/T190-multipart.sh
+++ b/test/T190-multipart.sh
@@ -1,6 +1,6 @@
 #!/usr/bin/env bash
 test_description="output of multipart message"
-. ./test-lib.sh || exit 1
+. $(dirname "$0")/test-lib.sh || exit 1
 
 cat <<EOF > embedded_message_body
 Content-Type: multipart/alternative; boundary="==-=-=="
diff --git a/test/T200-thread-naming.sh b/test/T200-thread-naming.sh
index 2167ba8ef973..594d301f7c3a 100755
--- a/test/T200-thread-naming.sh
+++ b/test/T200-thread-naming.sh
@@ -1,6 +1,6 @@
 #!/usr/bin/env bash
 test_description="naming of threads with changing subject"
-. ./test-lib.sh || exit 1
+. $(dirname "$0")/test-lib.sh || exit 1
 
 test_begin_subtest "Initial thread name (oldest-first search)"
 add_message '[subject]="thread-naming: Initial thread subject"' \
diff --git a/test/T205-author-naming.sh b/test/T205-author-naming.sh
index 69d8dc50ff86..68b85ced724e 100755
--- a/test/T205-author-naming.sh
+++ b/test/T205-author-naming.sh
@@ -1,6 +1,6 @@
 #!/usr/bin/env bash
 test_description="naming of authors with unusual addresses"
-. ./test-lib.sh || exit 1
+. $(dirname "$0")/test-lib.sh || exit 1
 
 test_begin_subtest "Add author with empty quoted real name"
 add_message '[subject]="author-naming: Initial thread subject"' \
diff --git a/test/T210-raw.sh b/test/T210-raw.sh
index 832a4ad311b7..99fdef72e634 100755
--- a/test/T210-raw.sh
+++ b/test/T210-raw.sh
@@ -1,7 +1,7 @@
 #!/usr/bin/env bash
 
 test_description='notmuch show --format=raw'
-. ./test-lib.sh || exit 1
+. $(dirname "$0")/test-lib.sh || exit 1
 
 add_message
 add_message
diff --git a/test/T220-reply.sh b/test/T220-reply.sh
index 4fb67ffbfc0d..ebe710f98c55 100755
--- a/test/T220-reply.sh
+++ b/test/T220-reply.sh
@@ -1,6 +1,6 @@
 #!/usr/bin/env bash
 test_description="\"notmuch reply\" in several variations"
-. ./test-lib.sh || exit 1
+. $(dirname "$0")/test-lib.sh || exit 1
 
 test_begin_subtest "Basic reply"
 add_message '[from]="Sender <sender@example.com>"' \
diff --git a/test/T230-reply-to-sender.sh b/test/T230-reply-to-sender.sh
index 608334dc023e..134a106365c4 100755
--- a/test/T230-reply-to-sender.sh
+++ b/test/T230-reply-to-sender.sh
@@ -1,6 +1,6 @@
 #!/usr/bin/env bash
 test_description="\"notmuch reply --reply-to=sender\" in several variations"
-. ./test-lib.sh || exit 1
+. $(dirname "$0")/test-lib.sh || exit 1
 
 test_begin_subtest "Basic reply-to-sender"
 add_message '[from]="Sender <sender@example.com>"' \
diff --git a/test/T240-dump-restore.sh b/test/T240-dump-restore.sh
index 75fb0b404bbe..0870ff921f1e 100755
--- a/test/T240-dump-restore.sh
+++ b/test/T240-dump-restore.sh
@@ -1,6 +1,6 @@
 #!/usr/bin/env bash
 test_description="\"notmuch dump\" and \"notmuch restore\""
-. ./test-lib.sh || exit 1
+. $(dirname "$0")/test-lib.sh || exit 1
 
 NOTMUCH_NEW > /dev/null
 test_begin_subtest "dump header"
diff --git a/test/T250-uuencode.sh b/test/T250-uuencode.sh
index 6f45d3959bf2..251c0b40f186 100755
--- a/test/T250-uuencode.sh
+++ b/test/T250-uuencode.sh
@@ -1,6 +1,6 @@
 #!/usr/bin/env bash
 test_description="handling of uuencoded data"
-. ./test-lib.sh || exit 1
+. $(dirname "$0")/test-lib.sh || exit 1
 
 add_message [subject]=uuencodetest '[date]="Sat, 01 Jan 2000 12:00:00 -0000"' \
 '[body]="This message is used to ensure that notmuch correctly handles a
diff --git a/test/T260-thread-order.sh b/test/T260-thread-order.sh
index 89f4d1be4816..7f71ce097eb5 100755
--- a/test/T260-thread-order.sh
+++ b/test/T260-thread-order.sh
@@ -1,6 +1,6 @@
 #!/usr/bin/env bash
 test_description="threading when messages received out of order"
-. ./test-lib.sh || exit 1
+. $(dirname "$0")/test-lib.sh || exit 1
 
 # Generate all single-root four message thread structures.  We'll use
 # this for multiple tests below.
diff --git a/test/T270-author-order.sh b/test/T270-author-order.sh
index 9124ece6f911..c28ecb028515 100755
--- a/test/T270-author-order.sh
+++ b/test/T270-author-order.sh
@@ -1,6 +1,6 @@
 #!/usr/bin/env bash
 test_description="author reordering;"
-. ./test-lib.sh || exit 1
+. $(dirname "$0")/test-lib.sh || exit 1
 
 test_begin_subtest "Adding parent message"
 generate_message [body]=findme [id]=new-parent-id [subject]=author-reorder-threadtest '[from]="User <user@example.com>"' '[date]="Sat, 01 Jan 2000 12:00:00 -0000"'
diff --git a/test/T280-from-guessing.sh b/test/T280-from-guessing.sh
index 7c562fb9e353..b87182323b72 100755
--- a/test/T280-from-guessing.sh
+++ b/test/T280-from-guessing.sh
@@ -1,6 +1,6 @@
 #!/usr/bin/env bash
 test_description="From line heuristics (with multiple configured addresses)"
-. ./test-lib.sh || exit 1
+. $(dirname "$0")/test-lib.sh || exit 1
 
 test_begin_subtest "Magic from guessing (nothing to go on)"
 add_message '[from]="Sender <sender@example.com>"' \
diff --git a/test/T290-long-id.sh b/test/T290-long-id.sh
index 1fb7c037ee52..5e3879f5dabf 100755
--- a/test/T290-long-id.sh
+++ b/test/T290-long-id.sh
@@ -1,6 +1,6 @@
 #!/usr/bin/env bash
 test_description="messages with ridiculously-long message IDs"
-. ./test-lib.sh || exit 1
+. $(dirname "$0")/test-lib.sh || exit 1
 
 test_begin_subtest "Referencing long ID before adding"
 generate_message '[subject]="Reference of ridiculously-long message ID"' \
diff --git a/test/T300-encoding.sh b/test/T300-encoding.sh
index 8d201c7e8aa4..2c656a1e0950 100755
--- a/test/T300-encoding.sh
+++ b/test/T300-encoding.sh
@@ -1,6 +1,6 @@
 #!/usr/bin/env bash
 test_description="encoding issues"
-. ./test-lib.sh || exit 1
+. $(dirname "$0")/test-lib.sh || exit 1
 
 test_begin_subtest "Message with text of unknown charset"
 add_message '[content-type]="text/plain; charset=unknown-8bit"' \
diff --git a/test/T310-emacs.sh b/test/T310-emacs.sh
index fde11790a600..b8a89a4ac4c9 100755
--- a/test/T310-emacs.sh
+++ b/test/T310-emacs.sh
@@ -1,7 +1,7 @@
 #!/usr/bin/env bash
 
 test_description="emacs interface"
-. ./test-lib.sh || exit 1
+. $(dirname "$0")/test-lib.sh || exit 1
 
 EXPECTED=$TEST_DIRECTORY/emacs.expected-output
 
diff --git a/test/T320-emacs-large-search-buffer.sh b/test/T320-emacs-large-search-buffer.sh
index e9d5e3584f87..f61e8a973fc5 100755
--- a/test/T320-emacs-large-search-buffer.sh
+++ b/test/T320-emacs-large-search-buffer.sh
@@ -1,6 +1,6 @@
 #!/usr/bin/env bash
 test_description="Emacs with large search results buffer"
-. ./test-lib.sh || exit 1
+. $(dirname "$0")/test-lib.sh || exit 1
 
 x=xxxxxxxxxx # 10
 x=$x$x$x$x$x$x$x$x$x$x # 100
diff --git a/test/T330-emacs-subject-to-filename.sh b/test/T330-emacs-subject-to-filename.sh
index 517fa8398261..eaf7c980903b 100755
--- a/test/T330-emacs-subject-to-filename.sh
+++ b/test/T330-emacs-subject-to-filename.sh
@@ -1,7 +1,7 @@
 #!/usr/bin/env bash
 
 test_description="emacs: mail subject to filename"
-. ./test-lib.sh || exit 1
+. $(dirname "$0")/test-lib.sh || exit 1
 
 # emacs server can't be started in a child process with $(test_emacs ...)
 test_emacs '(ignore)' > /dev/null
diff --git a/test/T340-maildir-sync.sh b/test/T340-maildir-sync.sh
index b473ae4e1c0f..7fece5f2302c 100755
--- a/test/T340-maildir-sync.sh
+++ b/test/T340-maildir-sync.sh
@@ -2,7 +2,7 @@
 
 test_description="maildir synchronization"
 
-. ./test-lib.sh || exit 1
+. $(dirname "$0")/test-lib.sh || exit 1
 
 # Create the expected maildir structure
 mkdir $MAIL_DIR/cur
diff --git a/test/T350-crypto.sh b/test/T350-crypto.sh
index 53bf4113d622..0e19bd08f7f3 100755
--- a/test/T350-crypto.sh
+++ b/test/T350-crypto.sh
@@ -5,7 +5,7 @@
 # - verification of signatures from expired/revoked keys
 
 test_description='PGP/MIME signature verification and decryption'
-. ./test-lib.sh || exit 1
+. $(dirname "$0")/test-lib.sh || exit 1
 
 add_gnupg_home ()
 {
diff --git a/test/T355-smime.sh b/test/T355-smime.sh
index 03d24581de2b..b813b6cad2e5 100755
--- a/test/T355-smime.sh
+++ b/test/T355-smime.sh
@@ -1,7 +1,7 @@
 #!/usr/bin/env bash
 
 test_description='S/MIME signature verification and decryption'
-. ./test-lib.sh || exit 1
+. $(dirname "$0")/test-lib.sh || exit 1
 
 add_gpgsm_home ()
 {
diff --git a/test/T360-symbol-hiding.sh b/test/T360-symbol-hiding.sh
index 9c6d4e647db5..68edc2d3a7ab 100755
--- a/test/T360-symbol-hiding.sh
+++ b/test/T360-symbol-hiding.sh
@@ -9,7 +9,7 @@
 
 test_description='exception symbol hiding'
 
-. ./test-lib.sh || exit 1
+. $(dirname "$0")/test-lib.sh || exit 1
 
 test_begin_subtest 'running test' run_test
 mkdir -p ${PWD}/fakedb/.notmuch
diff --git a/test/T370-search-folder-coherence.sh b/test/T370-search-folder-coherence.sh
index 8748b3d00eeb..0a2727e74fb4 100755
--- a/test/T370-search-folder-coherence.sh
+++ b/test/T370-search-folder-coherence.sh
@@ -1,6 +1,6 @@
 #!/usr/bin/env bash
 test_description='folder tags removed and added through file renames remain consistent'
-. ./test-lib.sh || exit 1
+. $(dirname "$0")/test-lib.sh || exit 1
 
 test_begin_subtest "No new messages"
 output=$(NOTMUCH_NEW)
diff --git a/test/T380-atomicity.sh b/test/T380-atomicity.sh
index a46a2df2a1fe..17a3e478aeca 100755
--- a/test/T380-atomicity.sh
+++ b/test/T380-atomicity.sh
@@ -1,6 +1,6 @@
 #!/usr/bin/env bash
 test_description='atomicity'
-. ./test-lib.sh || exit 1
+. $(dirname "$0")/test-lib.sh || exit 1
 
 # This script tests the effects of killing and restarting "notmuch
 # new" at arbitrary points.  If notmuch new is properly atomic, the
diff --git a/test/T390-python.sh b/test/T390-python.sh
index a9a61145699b..fd338743f44d 100755
--- a/test/T390-python.sh
+++ b/test/T390-python.sh
@@ -1,6 +1,6 @@
 #!/usr/bin/env bash
 test_description="python bindings"
-. ./test-lib.sh || exit 1
+. $(dirname "$0")/test-lib.sh || exit 1
 
 test_require_external_prereq ${NOTMUCH_PYTHON}
 
diff --git a/test/T395-ruby.sh b/test/T395-ruby.sh
index 52f9fe0f8bfd..a0b76eb89ca1 100755
--- a/test/T395-ruby.sh
+++ b/test/T395-ruby.sh
@@ -1,6 +1,6 @@
 #!/usr/bin/env bash
 test_description="ruby bindings"
-. ./test-lib.sh || exit 1
+. $(dirname "$0")/test-lib.sh || exit 1
 
 if [ "${NOTMUCH_HAVE_RUBY_DEV}" = "0" ]; then
     test_subtest_missing_external_prereq_["ruby development files"]=t
diff --git a/test/T400-hooks.sh b/test/T400-hooks.sh
index 7917a82f0154..49c690eb0360 100755
--- a/test/T400-hooks.sh
+++ b/test/T400-hooks.sh
@@ -1,6 +1,6 @@
 #!/usr/bin/env bash
 test_description='hooks'
-. ./test-lib.sh || exit 1
+. $(dirname "$0")/test-lib.sh || exit 1
 
 HOOK_DIR=${MAIL_DIR}/.notmuch/hooks
 
diff --git a/test/T410-argument-parsing.sh b/test/T410-argument-parsing.sh
index fad134e305c5..f22434814a75 100755
--- a/test/T410-argument-parsing.sh
+++ b/test/T410-argument-parsing.sh
@@ -1,6 +1,6 @@
 #!/usr/bin/env bash
 test_description="argument parsing"
-. ./test-lib.sh || exit 1
+. $(dirname "$0")/test-lib.sh || exit 1
 
 test_begin_subtest "sanity check"
 $TEST_DIRECTORY/arg-test  pos1  --keyword=one --string=foo pos2 --int=7 --flag=one --flag=three > OUTPUT
diff --git a/test/T420-emacs-test-functions.sh b/test/T420-emacs-test-functions.sh
index 955c5f7ff295..bfc10be3b7b5 100755
--- a/test/T420-emacs-test-functions.sh
+++ b/test/T420-emacs-test-functions.sh
@@ -1,7 +1,7 @@
 #!/usr/bin/env bash
 
 test_description="emacs test function sanity"
-. ./test-lib.sh || exit 1
+. $(dirname "$0")/test-lib.sh || exit 1
 
 test_begin_subtest "emacs test function sanity"
 test_emacs_expect_t 't'
diff --git a/test/T430-emacs-address-cleaning.sh b/test/T430-emacs-address-cleaning.sh
index 664b79d293fb..02d3b4117b94 100755
--- a/test/T430-emacs-address-cleaning.sh
+++ b/test/T430-emacs-address-cleaning.sh
@@ -1,7 +1,7 @@
 #!/usr/bin/env bash
 
 test_description="emacs address cleaning"
-. ./test-lib.sh || exit 1
+. $(dirname "$0")/test-lib.sh || exit 1
 
 test_begin_subtest "notmuch-test-address-clean part 1"
 test_emacs_expect_t '(notmuch-test-address-cleaning-1)'
diff --git a/test/T440-emacs-hello.sh b/test/T440-emacs-hello.sh
index ac214a5b2b45..76e795937c6c 100755
--- a/test/T440-emacs-hello.sh
+++ b/test/T440-emacs-hello.sh
@@ -1,7 +1,7 @@
 #!/usr/bin/env bash
 
 test_description="emacs notmuch-hello view"
-. ./test-lib.sh || exit 1
+. $(dirname "$0")/test-lib.sh || exit 1
 
 EXPECTED=$TEST_DIRECTORY/emacs.expected-output
 
diff --git a/test/T450-emacs-show.sh b/test/T450-emacs-show.sh
index db48c7d5b4c0..95babb1f0812 100755
--- a/test/T450-emacs-show.sh
+++ b/test/T450-emacs-show.sh
@@ -1,7 +1,7 @@
 #!/usr/bin/env bash
 
 test_description="emacs notmuch-show view"
-. ./test-lib.sh || exit 1
+. $(dirname "$0")/test-lib.sh || exit 1
 
 EXPECTED=$TEST_DIRECTORY/emacs-show.expected-output
 
diff --git a/test/T455-emacs-charsets.sh b/test/T455-emacs-charsets.sh
index 5d6d53a82326..cb1297caa5dc 100755
--- a/test/T455-emacs-charsets.sh
+++ b/test/T455-emacs-charsets.sh
@@ -1,7 +1,7 @@
 #!/usr/bin/env bash
 
 test_description="emacs notmuch-show charset handling"
-. ./test-lib.sh || exit 1
+. $(dirname "$0")/test-lib.sh || exit 1
 
 
 UTF8_YEN=$'\xef\xbf\xa5'
diff --git a/test/T460-emacs-tree.sh b/test/T460-emacs-tree.sh
index 958ff888a865..dc5978419be7 100755
--- a/test/T460-emacs-tree.sh
+++ b/test/T460-emacs-tree.sh
@@ -1,7 +1,7 @@
 #!/usr/bin/env bash
 
 test_description="emacs tree view interface"
-. ./test-lib.sh || exit 1
+. $(dirname "$0")/test-lib.sh || exit 1
 
 EXPECTED=$TEST_DIRECTORY/tree.expected-output
 
diff --git a/test/T470-missing-headers.sh b/test/T470-missing-headers.sh
index 32031e3174d3..4bf5d285c49b 100755
--- a/test/T470-missing-headers.sh
+++ b/test/T470-missing-headers.sh
@@ -1,6 +1,6 @@
 #!/usr/bin/env bash
 test_description='messages with missing headers'
-. ./test-lib.sh || exit 1
+. $(dirname "$0")/test-lib.sh || exit 1
 
 # Notmuch requires at least one of from, subject, or to or it will
 # ignore the file.  Generate two messages so that together they cover
diff --git a/test/T480-hex-escaping.sh b/test/T480-hex-escaping.sh
index 18b56600480c..28564c3c9da2 100755
--- a/test/T480-hex-escaping.sh
+++ b/test/T480-hex-escaping.sh
@@ -1,6 +1,6 @@
 #!/usr/bin/env bash
 test_description="hex encoding and decoding"
-. ./test-lib.sh || exit 1
+. $(dirname "$0")/test-lib.sh || exit 1
 
 test_begin_subtest "round trip"
 find $TEST_DIRECTORY/corpora/default -type f -print | sort | xargs cat > EXPECTED
diff --git a/test/T490-parse-time-string.sh b/test/T490-parse-time-string.sh
index ab90fcc5a76d..d1c70cfaf5d5 100755
--- a/test/T490-parse-time-string.sh
+++ b/test/T490-parse-time-string.sh
@@ -1,6 +1,6 @@
 #!/usr/bin/env bash
 test_description="date/time parser module"
-. ./test-lib.sh || exit 1
+. $(dirname "$0")/test-lib.sh || exit 1
 
 # Sanity/smoke tests for the date/time parser independent of notmuch
 
diff --git a/test/T500-search-date.sh b/test/T500-search-date.sh
index fc4ecdc30b16..5c5b99a0e2b0 100755
--- a/test/T500-search-date.sh
+++ b/test/T500-search-date.sh
@@ -1,6 +1,6 @@
 #!/usr/bin/env bash
 test_description="date:since..until queries"
-. ./test-lib.sh || exit 1
+. $(dirname "$0")/test-lib.sh || exit 1
 
 add_email_corpus
 
diff --git a/test/T510-thread-replies.sh b/test/T510-thread-replies.sh
index fa288bb19903..6837ff176630 100755
--- a/test/T510-thread-replies.sh
+++ b/test/T510-thread-replies.sh
@@ -9,7 +9,7 @@ test_description='test of proper handling of in-reply-to and references headers'
 # database is constructed properly, even in the presence of
 # non-RFC-compliant headers'
 
-. ./test-lib.sh || exit 1
+. $(dirname "$0")/test-lib.sh || exit 1
 
 test_begin_subtest "Use References when In-Reply-To is broken"
 add_message '[id]="foo@one.com"' \
diff --git a/test/T520-show.sh b/test/T520-show.sh
index fb232a32510f..16222650eafc 100755
--- a/test/T520-show.sh
+++ b/test/T520-show.sh
@@ -1,7 +1,7 @@
 #!/usr/bin/env bash
 test_description='"notmuch show"'
 
-. ./test-lib.sh || exit 1
+. $(dirname "$0")/test-lib.sh || exit 1
 
 add_email_corpus
 
diff --git a/test/T530-upgrade.sh b/test/T530-upgrade.sh
index f0fd1511d056..69ebec688464 100755
--- a/test/T530-upgrade.sh
+++ b/test/T530-upgrade.sh
@@ -1,7 +1,7 @@
 #!/usr/bin/env bash
 test_description="database upgrade"
 
-. ./test-lib.sh || exit 1
+. $(dirname "$0")/test-lib.sh || exit 1
 
 dbtarball=database-v1.tar.xz
 
diff --git a/test/T550-db-features.sh b/test/T550-db-features.sh
index f94a660d7508..9d5a9e70a598 100755
--- a/test/T550-db-features.sh
+++ b/test/T550-db-features.sh
@@ -1,7 +1,7 @@
 #!/usr/bin/env bash
 test_description="database version and feature compatibility"
 
-. ./test-lib.sh || exit 1
+. $(dirname "$0")/test-lib.sh || exit 1
 
 test_begin_subtest "future database versions abort open"
 ${TEST_DIRECTORY}/make-db-version ${MAIL_DIR} 9999 ""
diff --git a/test/T560-lib-error.sh b/test/T560-lib-error.sh
index a50eca807700..06a6b860ae8a 100755
--- a/test/T560-lib-error.sh
+++ b/test/T560-lib-error.sh
@@ -1,7 +1,7 @@
 #!/usr/bin/env bash
 test_description="error reporting for library"
 
-. ./test-lib.sh || exit 1
+. $(dirname "$0")/test-lib.sh || exit 1
 
 add_email_corpus
 
diff --git a/test/T570-revision-tracking.sh b/test/T570-revision-tracking.sh
index 76ad227944a0..a59e7c980fc9 100755
--- a/test/T570-revision-tracking.sh
+++ b/test/T570-revision-tracking.sh
@@ -1,7 +1,7 @@
 #!/usr/bin/env bash
 test_description="database revision tracking"
 
-. ./test-lib.sh || exit 1
+. $(dirname "$0")/test-lib.sh || exit 1
 
 add_email_corpus
 
diff --git a/test/T580-thread-search.sh b/test/T580-thread-search.sh
index 512559a3aff9..01aa3efd2e79 100755
--- a/test/T580-thread-search.sh
+++ b/test/T580-thread-search.sh
@@ -5,7 +5,7 @@
 
 test_description='test of searching by thread-id'
 
-. ./test-lib.sh || exit 1
+. $(dirname "$0")/test-lib.sh || exit 1
 
 add_email_corpus
 
diff --git a/test/T590-libconfig.sh b/test/T590-libconfig.sh
index 1b308693c527..46f3a76d574f 100755
--- a/test/T590-libconfig.sh
+++ b/test/T590-libconfig.sh
@@ -1,7 +1,7 @@
 #!/usr/bin/env bash
 test_description="library config API"
 
-. ./test-lib.sh || exit 1
+. $(dirname "$0")/test-lib.sh || exit 1
 
 add_email_corpus
 
diff --git a/test/T590-thread-breakage.sh b/test/T590-thread-breakage.sh
index 38abc2113c26..53932d63effe 100755
--- a/test/T590-thread-breakage.sh
+++ b/test/T590-thread-breakage.sh
@@ -19,7 +19,7 @@ test_description='thread breakage during reindexing'
 # works properly and attempted fixes to threading issues do not break
 # the expected contents of the index.
 
-. ./test-lib.sh || exit 1
+. $(dirname "$0")/test-lib.sh || exit 1
 
 message_a() {
     mkdir -p ${MAIL_DIR}/cur
diff --git a/test/T600-named-queries.sh b/test/T600-named-queries.sh
index 495b7699a0a7..59496c3e94a1 100755
--- a/test/T600-named-queries.sh
+++ b/test/T600-named-queries.sh
@@ -1,6 +1,6 @@
 #!/usr/bin/env bash
 test_description='named queries'
-. ./test-lib.sh || exit 1
+. $(dirname "$0")/test-lib.sh || exit 1
 
 QUERYSTR="date:2009-11-18..2009-11-18 and tag:unread"
 
diff --git a/test/T610-message-property.sh b/test/T610-message-property.sh
index ba5f55daf9b1..74b3f5a1811f 100755
--- a/test/T610-message-property.sh
+++ b/test/T610-message-property.sh
@@ -1,7 +1,7 @@
 #!/usr/bin/env bash
 test_description="message property API"
 
-. ./test-lib.sh || exit 1
+. $(dirname "$0")/test-lib.sh || exit 1
 
 add_email_corpus
 
diff --git a/test/T620-lock.sh b/test/T620-lock.sh
index f46475e8ddcc..085ffe4377c2 100755
--- a/test/T620-lock.sh
+++ b/test/T620-lock.sh
@@ -1,6 +1,6 @@
 #!/usr/bin/env bash
 test_description="locking"
-. ./test-lib.sh || exit 1
+. $(dirname "$0")/test-lib.sh || exit 1
 
 if [ "${NOTMUCH_HAVE_XAPIAN_DB_RETRY_LOCK}" = "0" ]; then
     test_subtest_missing_external_prereq_["lock retry support"]=t
diff --git a/test/T630-emacs-draft.sh b/test/T630-emacs-draft.sh
index cd9e33a7c405..d7903ce799da 100755
--- a/test/T630-emacs-draft.sh
+++ b/test/T630-emacs-draft.sh
@@ -1,6 +1,6 @@
 #!/usr/bin/env bash
 test_description="Emacs Draft Handling"
-. ./test-lib.sh || exit 1
+. $(dirname "$0")/test-lib.sh || exit 1
 
 add_email_corpus
 
diff --git a/test/T640-database-modified.sh b/test/T640-database-modified.sh
index e35e35f64289..274105c708fa 100755
--- a/test/T640-database-modified.sh
+++ b/test/T640-database-modified.sh
@@ -1,6 +1,6 @@
 #!/usr/bin/env bash
 test_description="DatabaseModifiedError handling"
-. ./test-lib.sh || exit 1
+. $(dirname "$0")/test-lib.sh || exit 1
 
 # add enough messages to trigger the exception
 add_email_corpus
diff --git a/test/T650-regexp-query.sh b/test/T650-regexp-query.sh
index d5def7649d0f..4085340f66d4 100755
--- a/test/T650-regexp-query.sh
+++ b/test/T650-regexp-query.sh
@@ -1,6 +1,6 @@
 #!/usr/bin/env bash
 test_description='regular expression searches'
-. ./test-lib.sh || exit 1
+. $(dirname "$0")/test-lib.sh || exit 1
 
 if [ $NOTMUCH_HAVE_XAPIAN_FIELD_PROCESSOR -eq 0 ]; then
     test_done
diff --git a/test/T660-bad-date.sh b/test/T660-bad-date.sh
index a98e11c88714..f65544b9ffdf 100755
--- a/test/T660-bad-date.sh
+++ b/test/T660-bad-date.sh
@@ -1,6 +1,6 @@
 #!/usr/bin/env bash
 test_description="parsing of bad dates"
-. ./test-lib.sh || exit 1
+. $(dirname "$0")/test-lib.sh || exit 1
 
 add_message [date]='"()"'
 
diff --git a/test/T670-duplicate-mid.sh b/test/T670-duplicate-mid.sh
index 21a9689ab870..c198c506378c 100755
--- a/test/T670-duplicate-mid.sh
+++ b/test/T670-duplicate-mid.sh
@@ -1,6 +1,6 @@
 #!/usr/bin/env bash
 test_description="duplicate message ids"
-. ./test-lib.sh || exit 1
+. $(dirname "$0")/test-lib.sh || exit 1
 
 add_message '[id]="duplicate"' '[subject]="message 1" [filename]=copy1'
 add_message '[id]="duplicate"' '[subject]="message 2" [filename]=copy2'
diff --git a/test/T680-html-indexing.sh b/test/T680-html-indexing.sh
index 74f33708be93..62ba84985231 100755
--- a/test/T680-html-indexing.sh
+++ b/test/T680-html-indexing.sh
@@ -1,6 +1,6 @@
 #!/usr/bin/env bash
 test_description="indexing of html parts"
-. ./test-lib.sh || exit 1
+. $(dirname "$0")/test-lib.sh || exit 1
 
 add_email_corpus html
 
diff --git a/test/T690-command-line-args.sh b/test/T690-command-line-args.sh
index a4f4b5f50f8d..9aa47611a935 100755
--- a/test/T690-command-line-args.sh
+++ b/test/T690-command-line-args.sh
@@ -1,7 +1,7 @@
 #!/usr/bin/env bash
 
 test_description="command line arguments"
-. ./test-lib.sh || exit 1
+. $(dirname "$0")/test-lib.sh || exit 1
 
 add_message
 
diff --git a/test/T700-reindex.sh b/test/T700-reindex.sh
index 051fbb3cef48..2b7bc6588b8d 100755
--- a/test/T700-reindex.sh
+++ b/test/T700-reindex.sh
@@ -1,6 +1,6 @@
 #!/usr/bin/env bash
 test_description='reindexing messages'
-. ./test-lib.sh || exit 1
+. $(dirname "$0")/test-lib.sh || exit 1
 
 add_email_corpus
 
diff --git a/test/test-verbose b/test/test-verbose
index 158be28a365e..8af6d9a97600 100755
--- a/test/test-verbose
+++ b/test/test-verbose
@@ -2,7 +2,7 @@
 
 test_description='the verbosity options of the test framework itself.'
 
-. ./test-lib.sh || exit 1
+. $(dirname "$0")/test-lib.sh || exit 1
 
 test_begin_subtest 'print something in test_expect_success and pass'
 test_expect_success '
-- 
2.11.0

[PATCH 04/24] perf-test: use $(dirname "$0") for sourcing perf-test-lib.sh

[Jani Nikula]

Don't assume the tests are always run from within the source tree.
---
 performance-test/M00-new.sh          | 2 +-
 performance-test/M01-dump-restore.sh | 2 +-
 performance-test/M02-show.sh         | 2 +-
 performance-test/M03-search.sh       | 2 +-
 performance-test/M04-reply.sh        | 2 +-
 performance-test/M05-reindex.sh      | 2 +-
 performance-test/M06-insert.sh       | 2 +-
 performance-test/T00-new.sh          | 2 +-
 performance-test/T01-dump-restore.sh | 2 +-
 performance-test/T02-tag.sh          | 2 +-
 performance-test/T03-reindex.sh      | 2 +-
 11 files changed, 11 insertions(+), 11 deletions(-)

diff --git a/performance-test/M00-new.sh b/performance-test/M00-new.sh
index a040a97e043a..aab36e6953e2 100755
--- a/performance-test/M00-new.sh
+++ b/performance-test/M00-new.sh
@@ -2,7 +2,7 @@
 
 test_description='notmuch new'
 
-. ./perf-test-lib.sh || exit 1
+. $(dirname "$0")/perf-test-lib.sh || exit 1
 
 # ensure initial 'notmuch new' is run by memory_start
 uncache_database
diff --git a/performance-test/M01-dump-restore.sh b/performance-test/M01-dump-restore.sh
index 8fea98242d2c..32ab8dc9c0b9 100755
--- a/performance-test/M01-dump-restore.sh
+++ b/performance-test/M01-dump-restore.sh
@@ -2,7 +2,7 @@
 
 test_description='dump and restore'
 
-. ./perf-test-lib.sh || exit 1
+. $(dirname "$0")/perf-test-lib.sh || exit 1
 
 memory_start
 
diff --git a/performance-test/M02-show.sh b/performance-test/M02-show.sh
index d73035ea1197..2e218fd3c4b6 100755
--- a/performance-test/M02-show.sh
+++ b/performance-test/M02-show.sh
@@ -2,7 +2,7 @@
 
 test_description='show'
 
-. ./perf-test-lib.sh || exit 1
+. $(dirname "$0")/perf-test-lib.sh || exit 1
 
 memory_start
 
diff --git a/performance-test/M03-search.sh b/performance-test/M03-search.sh
index 8d026eee3662..343f5c7cfc24 100755
--- a/performance-test/M03-search.sh
+++ b/performance-test/M03-search.sh
@@ -2,7 +2,7 @@
 
 test_description='search'
 
-. ./perf-test-lib.sh || exit 1
+. $(dirname "$0")/perf-test-lib.sh || exit 1
 
 memory_start
 
diff --git a/performance-test/M04-reply.sh b/performance-test/M04-reply.sh
index 0e1ce0876925..3c1205dbc412 100755
--- a/performance-test/M04-reply.sh
+++ b/performance-test/M04-reply.sh
@@ -2,7 +2,7 @@
 
 test_description='search'
 
-. ./perf-test-lib.sh || exit 1
+. $(dirname "$0")/perf-test-lib.sh || exit 1
 
 memory_start
 
diff --git a/performance-test/M05-reindex.sh b/performance-test/M05-reindex.sh
index d36e061b7549..17e2c824698a 100755
--- a/performance-test/M05-reindex.sh
+++ b/performance-test/M05-reindex.sh
@@ -2,7 +2,7 @@
 
 test_description='reindex'
 
-. ./perf-test-lib.sh || exit 1
+. $(dirname "$0")/perf-test-lib.sh || exit 1
 
 memory_start
 
diff --git a/performance-test/M06-insert.sh b/performance-test/M06-insert.sh
index 9fcc2a0c9ff1..5ae0656a505b 100755
--- a/performance-test/M06-insert.sh
+++ b/performance-test/M06-insert.sh
@@ -2,7 +2,7 @@
 
 test_description='search'
 
-. ./perf-test-lib.sh || exit 1
+. $(dirname "$0")/perf-test-lib.sh || exit 1
 
 memory_start
 
diff --git a/performance-test/T00-new.sh b/performance-test/T00-new.sh
index b9f211581f5a..687501294e2c 100755
--- a/performance-test/T00-new.sh
+++ b/performance-test/T00-new.sh
@@ -2,7 +2,7 @@
 
 test_description='notmuch new'
 
-. ./perf-test-lib.sh || exit 1
+. $(dirname "$0")/perf-test-lib.sh || exit 1
 
 uncache_database
 
diff --git a/performance-test/T01-dump-restore.sh b/performance-test/T01-dump-restore.sh
index 9cfd5cd624ac..12f12e660533 100755
--- a/performance-test/T01-dump-restore.sh
+++ b/performance-test/T01-dump-restore.sh
@@ -2,7 +2,7 @@
 
 test_description='dump and restore'
 
-. ./perf-test-lib.sh || exit 1
+. $(dirname "$0")/perf-test-lib.sh || exit 1
 
 time_start
 
diff --git a/performance-test/T02-tag.sh b/performance-test/T02-tag.sh
index dacb50b8bbf8..8c5dfd680316 100755
--- a/performance-test/T02-tag.sh
+++ b/performance-test/T02-tag.sh
@@ -2,7 +2,7 @@
 
 test_description='tagging'
 
-. ./perf-test-lib.sh || exit 1
+. $(dirname "$0")/perf-test-lib.sh || exit 1
 
 time_start
 
diff --git a/performance-test/T03-reindex.sh b/performance-test/T03-reindex.sh
index 7af2d22d3df8..d6d5c3c3ae83 100755
--- a/performance-test/T03-reindex.sh
+++ b/performance-test/T03-reindex.sh
@@ -2,7 +2,7 @@
 
 test_description='tagging'
 
-. ./perf-test-lib.sh || exit 1
+. $(dirname "$0")/perf-test-lib.sh || exit 1
 
 time_start
 
-- 
2.11.0

[PATCH 05/24] test: check for notmuch binary in test-lib.sh

[Jani Nikula]

Move notmuch executable check into common code. Redundant for
notmuch-test runs, but works also for individual tests.
---
 test/T000-basic.sh | 10 ----------
 test/test-lib.sh   |  7 +++++++
 2 files changed, 7 insertions(+), 10 deletions(-)

diff --git a/test/T000-basic.sh b/test/T000-basic.sh
index d514082d48e3..f3808c0753a2 100755
--- a/test/T000-basic.sh
+++ b/test/T000-basic.sh
@@ -4,16 +4,6 @@
 #
 
 test_description='the test framework itself.'
-
-################################################################
-# It appears that people try to run tests without building...
-
-if ! test -x ../notmuch
-then
-	echo >&2 'You do not seem to have built notmuch yet.'
-	exit 1
-fi
-
 . $(dirname "$0")/test-lib.sh || exit 1
 
 ################################################################
diff --git a/test/test-lib.sh b/test/test-lib.sh
index 350246492d8c..e3b40f21743d 100644
--- a/test/test-lib.sh
+++ b/test/test-lib.sh
@@ -26,6 +26,13 @@ fi
 # Make sure echo builtin does not expand backslash-escape sequences by default.
 shopt -u xpg_echo
 
+# It appears that people try to run tests without building...
+if ! test -x ../notmuch
+then
+	echo >&2 'You do not seem to have built notmuch yet.'
+	exit 1
+fi
+
 this_test=${0##*/}
 this_test=${this_test%.sh}
 this_test_bare=${this_test#T[0-9][0-9][0-9]-}
-- 
2.11.0

[PATCH 06/24] test: add a script to figure out source and build directories

[Jani Nikula]

Expect the tests to be run somewhere in the build directory
hierarchy. Allow for source and build directories to be passed through
the environment.
---
 test/export-dirs.sh | 32 ++++++++++++++++++++++++++++++++
 1 file changed, 32 insertions(+)
 create mode 100644 test/export-dirs.sh

diff --git a/test/export-dirs.sh b/test/export-dirs.sh
new file mode 100644
index 000000000000..0578b1e50194
--- /dev/null
+++ b/test/export-dirs.sh
@@ -0,0 +1,32 @@
+# Source this script to set and export NOTMUCH_SRCDIR and
+# NOTMUCH_BUILDDIR.
+#
+# For this to work, always have current directory somewhere within the
+# build directory hierarchy, and run the script sourcing this script
+# using a path (relative or absolute) to the source directory.
+
+if [[ -z "${NOTMUCH_SRCDIR}" ]]; then
+	export NOTMUCH_SRCDIR="$(cd "$(dirname "$0")"/.. && pwd)"
+fi
+
+find_builddir()
+{
+	local dir="$1"
+
+	while [[ -n "$dir" ]] && [[ "$dir" != "/" ]]; do
+		if [[ -x "$dir/notmuch" ]] && [[ ! -d "$dir/notmuch" ]]; then
+			echo "$dir"
+			break
+		fi
+		dir="$(dirname "$dir")"
+	done
+}
+
+if [[ -z "${NOTMUCH_BUILDDIR}" ]]; then
+	export NOTMUCH_BUILDDIR="$(find_builddir "$(pwd)")"
+
+	if [[ -z "${NOTMUCH_BUILDDIR}" ]]; then
+		echo "Run tests in a subdir of built notmuch tree." >&2
+		exit 1
+	fi
+fi
-- 
2.11.0

[PATCH 07/24] test: use source and build paths in test-lib.sh

[Jani Nikula]

Make a distinction between source and build directories.
---
 test/test-lib.sh | 22 ++++++++++++----------
 1 file changed, 12 insertions(+), 10 deletions(-)

diff --git a/test/test-lib.sh b/test/test-lib.sh
index e3b40f21743d..5ccae7aea28b 100644
--- a/test/test-lib.sh
+++ b/test/test-lib.sh
@@ -26,9 +26,11 @@ fi
 # Make sure echo builtin does not expand backslash-escape sequences by default.
 shopt -u xpg_echo
 
+# Ensure NOTMUCH_SRCDIR and NOTMUCH_BUILDDIR are set.
+. $(dirname "$0")/export-dirs.sh || exit 1
+
 # It appears that people try to run tests without building...
-if ! test -x ../notmuch
-then
+if [[ ! -x "$NOTMUCH_BUILDDIR/notmuch" ]]; then
 	echo >&2 'You do not seem to have built notmuch yet.'
 	exit 1
 fi
@@ -369,7 +371,7 @@ add_email_corpus ()
     if [ -d $TEST_DIRECTORY/corpora.mail/$corpus ]; then
 	cp -a $TEST_DIRECTORY/corpora.mail/$corpus ${MAIL_DIR}
     else
-	cp -a $TEST_DIRECTORY/corpora/$corpus ${MAIL_DIR}
+	cp -a $NOTMUCH_SRCDIR/test/corpora/$corpus ${MAIL_DIR}
 	notmuch new >/dev/null || die "'notmuch new' failed while adding email corpus"
 	mkdir -p $TEST_DIRECTORY/corpora.mail
 	cp -a ${MAIL_DIR} $TEST_DIRECTORY/corpora.mail/$corpus
@@ -918,8 +920,8 @@ export NOTMUCH_CONFIG=$NOTMUCH_CONFIG
 # --load		Force loading of notmuch.el and test-lib.el
 
 exec ${TEST_EMACS} --quick \
-	--directory "$TEST_DIRECTORY/../emacs" --load notmuch.el \
-	--directory "$TEST_DIRECTORY" --load test-lib.el \
+	--directory "$NOTMUCH_SRCDIR/emacs" --load notmuch.el \
+	--directory "$NOTMUCH_SRCDIR/test" --load test-lib.el \
 	"\$@"
 EOF
 	chmod a+x "$TMP_DIRECTORY/run_emacs"
@@ -934,8 +936,8 @@ test_emacs () {
 	test -z "$missing_dependencies" || return
 
 	if [ -z "$EMACS_SERVER" ]; then
-		emacs_tests="${this_test_bare}.el"
-		if [ -f "$TEST_DIRECTORY/$emacs_tests" ]; then
+		emacs_tests="$NOTMUCH_SRCDIR/test/${this_test_bare}.el"
+		if [ -f "$emacs_tests" ]; then
 			load_emacs_tests="--eval '(load \"$emacs_tests\")'"
 		else
 			load_emacs_tests=
@@ -978,14 +980,14 @@ test_python() {
 }
 
 test_ruby() {
-    MAIL_DIR=$MAIL_DIR ruby -I $TEST_DIRECTORY/../bindings/ruby> OUTPUT
+    MAIL_DIR=$MAIL_DIR ruby -I $NOTMUCH_SRCDIR/bindings/ruby> OUTPUT
 }
 
 test_C () {
     exec_file="test${test_count}"
     test_file="${exec_file}.c"
     cat > ${test_file}
-    ${TEST_CC} ${TEST_CFLAGS} -I${TEST_DIRECTORY} -I${NOTMUCH_SRCDIR}/lib -o ${exec_file} ${test_file} -L${TEST_DIRECTORY}/../lib/ -lnotmuch -ltalloc
+    ${TEST_CC} ${TEST_CFLAGS} -I${NOTMUCH_SRCDIR}/test -I${NOTMUCH_SRCDIR}/lib -o ${exec_file} ${test_file} -L${NOTMUCH_BUILDDIR}/lib/ -lnotmuch -ltalloc
     echo "== stdout ==" > OUTPUT.stdout
     echo "== stderr ==" > OUTPUT.stderr
     ./${exec_file} "$@" 1>>OUTPUT.stdout 2>>OUTPUT.stderr
@@ -1041,7 +1043,7 @@ test_init_ () {
 }
 
 
-. ./test-lib-common.sh || exit 1
+. "$NOTMUCH_SRCDIR/test/test-lib-common.sh" || exit 1
 
 if [ "${NOTMUCH_GMIME_MAJOR}" = 3 ]; then
     test_subtest_broken_gmime_3 () {
-- 
2.11.0

[PATCH 08/24] perf-test: use source and build paths in perf-test-lib.sh

[Jani Nikula]

Make a distinction between source and build directories.
---
 performance-test/perf-test-lib.sh | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)

diff --git a/performance-test/perf-test-lib.sh b/performance-test/perf-test-lib.sh
index c89d5aab39a0..56538abddcdb 100644
--- a/performance-test/perf-test-lib.sh
+++ b/performance-test/perf-test-lib.sh
@@ -1,4 +1,4 @@
-. ./version.sh || exit 1
+. $(dirname "$0")/version.sh || exit 1
 
 corpus_size=large
 
@@ -25,12 +25,16 @@ do
 		echo "error: unknown performance test option '$1'" >&2; exit 1 ;;
 	esac
 done
-. ../test/test-lib-common.sh || exit 1
+
+# Ensure NOTMUCH_SRCDIR and NOTMUCH_BUILDDIR are set.
+. $(dirname "$0")/../test/export-dirs.sh || exit 1
+
+. "$NOTMUCH_SRCDIR/test/test-lib-common.sh" || exit 1
 
 set -e
 
-if ! test -x ../notmuch
-then
+# It appears that people try to run tests without building...
+if [[ ! -x "$NOTMUCH_BUILDDIR/notmuch" ]]; then
 	echo >&2 'You do not seem to have built notmuch yet.'
 	exit 1
 fi
-- 
2.11.0

[PATCH 09/24] test: use source and build paths in test-lib-common.sh

[Jani Nikula]

Make a distinction between source and build directories.

The expectation is that anyone sourcing test-lib-common.sh has sourced
export-dirs.sh.
---
 test/test-lib-common.sh | 43 +++++++++++++++----------------------------
 1 file changed, 15 insertions(+), 28 deletions(-)

diff --git a/test/test-lib-common.sh b/test/test-lib-common.sh
index 984a7c1a5798..5133917dae9a 100644
--- a/test/test-lib-common.sh
+++ b/test/test-lib-common.sh
@@ -24,39 +24,26 @@
 #
 type die >/dev/null 2>&1 || die () { echo "$@" >&2; exit 1; }
 
-find_notmuch_path ()
-{
-    dir="$1"
-
-    while [ -n "$dir" ]; do
-	bin="$dir/notmuch"
-	if [ -x "$bin" ]; then
-	    echo "$dir"
-	    return
-	fi
-	dir="$(dirname "$dir")"
-	if [ "$dir" = "/" ]; then
-	    break
-	fi
-    done
-}
+if [[ -z "$NOTMUCH_SRCDIR" ]] || [[ -z "$NOTMUCH_BUILDDIR" ]]; then
+	echo "internal: srcdir or builddir not set" >&2
+	exit 1
+fi
 
 backup_database () {
     test_name=$(basename $0 .sh)
-    rm -rf notmuch-dir-backup."$test_name"
-    cp -pR ${MAIL_DIR}/.notmuch notmuch-dir-backup."${test_name}"
+    rm -rf $NOTMUCH_BUILDDIR/test/notmuch-dir-backup."$test_name"
+    cp -pR ${MAIL_DIR}/.notmuch $NOTMUCH_BUILDDIR/test/notmuch-dir-backup."${test_name}"
 }
 
 restore_database () {
     test_name=$(basename $0 .sh)
     rm -rf ${MAIL_DIR}/.notmuch
-    cp -pR notmuch-dir-backup."${test_name}" ${MAIL_DIR}/.notmuch
+    cp -pR $NOTMUCH_BUILDDIR/test/notmuch-dir-backup."${test_name}" ${MAIL_DIR}/.notmuch
 }
 
 # Test the binaries we have just built.  The tests are kept in
 # test/ subdirectory and are run in 'trash directory' subdirectory.
-TEST_DIRECTORY=$(pwd -P)
-notmuch_path=`find_notmuch_path "$TEST_DIRECTORY"`
+TEST_DIRECTORY=$NOTMUCH_BUILDDIR/test
 
 # Prepend $TEST_DIRECTORY/../lib to LD_LIBRARY_PATH, to make tests work
 # on systems where ../notmuch depends on LD_LIBRARY_PATH.
@@ -64,11 +51,11 @@ LD_LIBRARY_PATH=${TEST_DIRECTORY%/*}/lib${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}
 export LD_LIBRARY_PATH
 
 # configure output
-. $notmuch_path/sh.config || exit 1
+. "$NOTMUCH_BUILDDIR/sh.config" || exit 1
 
 # load OS specifics
-if [ -e ./test-lib-$PLATFORM.sh ]; then
-	. ./test-lib-$PLATFORM.sh || exit 1
+if [[ -e "$NOTMUCH_SRCDIR/test/test-lib-$PLATFORM.sh" ]]; then
+    . "$NOTMUCH_SRCDIR/test/test-lib-$PLATFORM.sh" || exit 1
 fi
 
 # Generate a new message in the mail directory, with a unique message
@@ -308,12 +295,12 @@ then
 	PATH=$GIT_VALGRIND/bin:$PATH
 	GIT_EXEC_PATH=$GIT_VALGRIND/bin
 	export GIT_VALGRIND
-	test -n "$notmuch_path" && MANPATH="$notmuch_path/doc/_build/man"
+	test -n "$NOTMUCH_BUILDDIR" && MANPATH="$NOTMUCH_BUILDDIR/doc/_build/man"
 else # normal case
-	if test -n "$notmuch_path"
+	if test -n "$NOTMUCH_BUILDDIR"
 		then
-			PATH="$notmuch_path:$PATH"
-			MANPATH="$notmuch_path/doc/_build/man"
+			PATH="$NOTMUCH_BUILDDIR:$PATH"
+			MANPATH="$NOTMUCH_BUILDDIR/doc/_build/man"
 		fi
 fi
 export PATH MANPATH
-- 
2.11.0

[PATCH 10/24] test: mkdir and cd to $TMP_DIRECTORY instead of $test for testing

[Jani Nikula]

$TMP_DIRECTORY is a full path, while $test is not.
---
 test/test-lib-common.sh | 1 -
 test/test-lib.sh        | 2 +-
 2 files changed, 1 insertion(+), 2 deletions(-)

diff --git a/test/test-lib-common.sh b/test/test-lib-common.sh
index 5133917dae9a..5e53348a9438 100644
--- a/test/test-lib-common.sh
+++ b/test/test-lib-common.sh
@@ -329,7 +329,6 @@ mkdir -p "${HOME}"
 MAIL_DIR="${TMP_DIRECTORY}/mail"
 export NOTMUCH_CONFIG="${TMP_DIRECTORY}/notmuch-config"
 
-mkdir -p "${test}"
 mkdir -p "${MAIL_DIR}"
 
 cat <<EOF >"${NOTMUCH_CONFIG}"
diff --git a/test/test-lib.sh b/test/test-lib.sh
index 5ccae7aea28b..0b4303f1d485 100644
--- a/test/test-lib.sh
+++ b/test/test-lib.sh
@@ -1066,7 +1066,7 @@ emacs_generate_script
 
 # Use -P to resolve symlinks in our working directory so that the cwd
 # in subprocesses like git equals our $PWD (for pathname comparisons).
-cd -P "$test" || error "Cannot set up test environment"
+cd -P "$TMP_DIRECTORY" || error "Cannot set up test environment"
 
 if test "$verbose" = "t"
 then
-- 
2.11.0

[PATCH 11/24] test: use source and build paths in notmuch-test

[Jani Nikula]

Make a distinction between source and build directories.
---
 test/notmuch-test | 23 ++++++++++++++++-------
 1 file changed, 16 insertions(+), 7 deletions(-)

diff --git a/test/notmuch-test b/test/notmuch-test
index 9d9df7dfedbe..ca68dd416cfc 100755
--- a/test/notmuch-test
+++ b/test/notmuch-test
@@ -15,12 +15,20 @@ if [ ${BASH_VERSINFO[0]} -lt 4 ]; then
     exit 1
 fi
 
-cd "$(dirname "$0")"
+# Ensure NOTMUCH_SRCDIR and NOTMUCH_BUILDDIR are set.
+. $(dirname "$0")/export-dirs.sh || exit 1
 
-TESTS=${NOTMUCH_TESTS:-T[0-9][0-9][0-9]-*.sh}
+TESTS=
+for test in $NOTMUCH_TESTS; do
+    TESTS="$TESTS $NOTMUCH_SRCDIR/test/$test"
+done
+
+if [[ -z "$TESTS" ]]; then
+    TESTS="$NOTMUCH_SRCDIR/test/T[0-9][0-9][0-9]-*.sh"
+fi
 
 # Clean up any results from a previous run
-rm -rf test-results
+rm -rf $NOTMUCH_BUILDDIR/test/test-results
 
 # Test for timeout utility
 if command -v timeout >/dev/null; then
@@ -33,12 +41,13 @@ fi
 trap 'e=$?; kill $!; exit $e' HUP INT TERM
 # Run the tests
 for test in $TESTS; do
-    $TEST_TIMEOUT_CMD ./$test "$@" &
+    $TEST_TIMEOUT_CMD $test "$@" &
     wait $!
     # If the test failed without producing results, then it aborted,
     # so we should abort, too.
     RES=$?
-    if [[ $RES != 0 && ! -e "test-results/${test%.sh}" ]]; then
+    testname=$(basename $test .sh)
+    if [[ $RES != 0 && ! -e "$NOTMUCH_BUILDDIR/test/test-results/$testname" ]]; then
         exit $RES
     fi
 done
@@ -46,10 +55,10 @@ trap - HUP INT TERM
 
 # Report results
 echo
-./aggregate-results.sh test-results/*
+$NOTMUCH_SRCDIR/test/aggregate-results.sh $NOTMUCH_BUILDDIR/test/test-results/*
 ev=$?
 
 # Clean up
-rm -rf test-results corpora.mail
+rm -rf $NOTMUCH_BUILDDIR/test/test-results $NOTMUCH_BUILDDIR/test/corpora.mail
 
 exit $ev
-- 
2.11.0

[PATCH 12/24] test: use source and build paths in T000-basic.sh

[Jani Nikula]

Make a distinction between source and build directories.
---
 test/T000-basic.sh | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/test/T000-basic.sh b/test/T000-basic.sh
index f3808c0753a2..7fbdcfa3036c 100755
--- a/test/T000-basic.sh
+++ b/test/T000-basic.sh
@@ -32,19 +32,19 @@ fi
 test_begin_subtest 'failure to clean up causes the test to fail'
 test_expect_code 2 'test_when_finished "(exit 2)"'
 
-EXPECTED=$TEST_DIRECTORY/test.expected-output
+EXPECTED=$NOTMUCH_SRCDIR/test/test.expected-output
 suppress_diff_date() {
     sed -e 's/\(.*\-\-\- test-verbose\.4\.\expected\).*/\1/' \
 	-e 's/\(.*\+\+\+ test-verbose\.4\.\output\).*/\1/'
 }
 
 test_begin_subtest "Ensure that test output is suppressed unless the test fails"
-output=$(cd $TEST_DIRECTORY; NOTMUCH_TEST_QUIET= ./test-verbose 2>&1 | suppress_diff_date)
+output=$(cd $TEST_DIRECTORY; NOTMUCH_TEST_QUIET= $NOTMUCH_SRCDIR/test/test-verbose 2>&1 | suppress_diff_date)
 expected=$(cat $EXPECTED/test-verbose-no | suppress_diff_date)
 test_expect_equal "$output" "$expected"
 
 test_begin_subtest "Ensure that -v does not suppress test output"
-output=$(cd $TEST_DIRECTORY; NOTMUCH_TEST_QUIET= ./test-verbose -v 2>&1 | suppress_diff_date)
+output=$(cd $TEST_DIRECTORY; NOTMUCH_TEST_QUIET= $NOTMUCH_SRCDIR/test/test-verbose -v 2>&1 | suppress_diff_date)
 expected=$(cat $EXPECTED/test-verbose-yes | suppress_diff_date)
 # Do not include the results of test-verbose in totals
 rm $TEST_DIRECTORY/test-results/test-verbose
-- 
2.11.0

[PATCH 13/24] test: use source and build paths in T160-json.sh and T170-sexp.sh

[Jani Nikula]

Make a distinction between source and build directories.
---
 test/T160-json.sh | 4 ++--
 test/T170-sexp.sh | 4 ++--
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/test/T160-json.sh b/test/T160-json.sh
index 765b78a6746d..91b98e5d6bdc 100755
--- a/test/T160-json.sh
+++ b/test/T160-json.sh
@@ -41,13 +41,13 @@ id="json-show-inline-attachment-filename@notmuchmail.org"
 emacs_fcc_message \
     "$subject" \
     'This is a test message with inline attachment with a filename' \
-    "(mml-attach-file \"$TEST_DIRECTORY/README\" nil nil \"inline\")
+    "(mml-attach-file \"$NOTMUCH_SRCDIR/test/README\" nil nil \"inline\")
      (message-goto-eoh)
      (insert \"Message-ID: <$id>\n\")"
 output=$(notmuch show --format=json "id:$id")
 filename=$(notmuch search --output=files "id:$id")
 # Get length of README after base64-encoding, minus additional newline.
-attachment_length=$(( $(base64 $TEST_DIRECTORY/README | wc -c) - 1 ))
+attachment_length=$(( $(base64 $NOTMUCH_SRCDIR/test/README | wc -c) - 1 ))
 test_expect_equal_json "$output" "[[[{\"id\": \"$id\", \"match\": true, \"excluded\": false, \"filename\": [\"$filename\"], \"timestamp\": 946728000, \"date_relative\": \"2000-01-01\", \"tags\": [\"inbox\"], \"headers\": {\"Subject\": \"$subject\", \"From\": \"Notmuch Test Suite <test_suite@notmuchmail.org>\", \"To\": \"test_suite@notmuchmail.org\", \"Date\": \"Sat, 01 Jan 2000 12:00:00 +0000\"}, \"body\": [{\"id\": 1, \"content-type\": \"multipart/mixed\", \"content\": [{\"id\": 2, \"content-type\": \"text/plain\", \"content\": \"This is a test message with inline attachment with a filename\"}, {\"id\": 3, \"content-type\": \"application/octet-stream\", \"content-length\": $attachment_length, \"content-transfer-encoding\": \"base64\", \"content-disposition\": \"inline\", \"filename\": \"README\"}]}]}, []]]]"
 
 test_begin_subtest "Search message: json, utf-8"
diff --git a/test/T170-sexp.sh b/test/T170-sexp.sh
index 1125fdc91ec0..c3dcf52a8294 100755
--- a/test/T170-sexp.sh
+++ b/test/T170-sexp.sh
@@ -37,13 +37,13 @@ id="sexp-show-inline-attachment-filename@notmuchmail.org"
 emacs_fcc_message \
     "$subject" \
     'This is a test message with inline attachment with a filename' \
-    "(mml-attach-file \"$TEST_DIRECTORY/README\" nil nil \"inline\")
+    "(mml-attach-file \"$NOTMUCH_SRCDIR/test/README\" nil nil \"inline\")
      (message-goto-eoh)
      (insert \"Message-ID: <$id>\n\")"
 output=$(notmuch show --format=sexp "id:$id")
 filename=$(notmuch search --output=files "id:$id")
 # Get length of README after base64-encoding, minus additional newline.
-attachment_length=$(( $(base64 $TEST_DIRECTORY/README | wc -c) - 1 ))
+attachment_length=$(( $(base64 $NOTMUCH_SRCDIR/test/README | wc -c) - 1 ))
 test_expect_equal "$output" "((((:id \"$id\" :match t :excluded nil :filename (\"$filename\") :timestamp 946728000 :date_relative \"2000-01-01\" :tags (\"inbox\") :headers (:Subject \"sexp-show-inline-attachment-filename\" :From \"Notmuch Test Suite <test_suite@notmuchmail.org>\" :To \"test_suite@notmuchmail.org\" :Date \"Sat, 01 Jan 2000 12:00:00 +0000\") :body ((:id 1 :content-type \"multipart/mixed\" :content ((:id 2 :content-type \"text/plain\" :content \"This is a test message with inline attachment with a filename\") (:id 3 :content-type \"application/octet-stream\" :content-disposition \"inline\" :filename \"README\" :content-transfer-encoding \"base64\" :content-length $attachment_length))))) ())))"
 
 test_done
-- 
2.11.0

[PATCH 14/24] test: use source and build paths in T260-thread-order.sh

[Jani Nikula]

Make a distinction between source and build directories.
---
 test/T260-thread-order.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/test/T260-thread-order.sh b/test/T260-thread-order.sh
index 7f71ce097eb5..fea612757c41 100755
--- a/test/T260-thread-order.sh
+++ b/test/T260-thread-order.sh
@@ -4,7 +4,7 @@ test_description="threading when messages received out of order"
 
 # Generate all single-root four message thread structures.  We'll use
 # this for multiple tests below.
-THREADS=$($NOTMUCH_PYTHON ${TEST_DIRECTORY}/gen-threads.py 4)
+THREADS=$($NOTMUCH_PYTHON ${NOTMUCH_SRCDIR}/test/gen-threads.py 4)
 nthreads=$(wc -l <<< "$THREADS")
 
 test_begin_subtest "Messages with one parent get linked in all delivery orders"
-- 
2.11.0

[PATCH 15/24] test: use source and build paths in T310-emacs.sh

[Jani Nikula]

Make a distinction between source and build directories.
---
 test/T310-emacs.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/test/T310-emacs.sh b/test/T310-emacs.sh
index b8a89a4ac4c9..7f1c75630ca9 100755
--- a/test/T310-emacs.sh
+++ b/test/T310-emacs.sh
@@ -3,13 +3,13 @@
 test_description="emacs interface"
 . $(dirname "$0")/test-lib.sh || exit 1
 
-EXPECTED=$TEST_DIRECTORY/emacs.expected-output
+EXPECTED=$NOTMUCH_SRCDIR/test/emacs.expected-output
 
 add_email_corpus
 
 # syntax errors in test-lib.el cause mysterious failures
 test_begin_subtest "Syntax of emacs test library"
-test_expect_success "${TEST_EMACS} -Q --batch --load $TEST_DIRECTORY/test-lib.el"
+test_expect_success "${TEST_EMACS} -Q --batch --load $NOTMUCH_SRCDIR/test/test-lib.el"
 
 test_begin_subtest "Basic notmuch-hello view in emacs"
 test_emacs '(notmuch-hello)
-- 
2.11.0

[PATCH 16/24] test: use source and build paths in T350-crypto.sh

[Jani Nikula]

Make a distinction between source and build directories.
---
 test/T350-crypto.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/test/T350-crypto.sh b/test/T350-crypto.sh
index 0e19bd08f7f3..ef3b5ec6ff18 100755
--- a/test/T350-crypto.sh
+++ b/test/T350-crypto.sh
@@ -14,7 +14,7 @@ add_gnupg_home ()
     _gnupg_exit () { gpgconf --kill all 2>/dev/null || true; }
     at_exit_function _gnupg_exit
     mkdir -m 0700 "$GNUPGHOME"
-    gpg --no-tty --import <$TEST_DIRECTORY/gnupg-secret-key.asc >"$GNUPGHOME"/import.log 2>&1
+    gpg --no-tty --import <$NOTMUCH_SRCDIR/test/gnupg-secret-key.asc >"$GNUPGHOME"/import.log 2>&1
     test_debug "cat $GNUPGHOME/import.log"
     if (gpg --quick-random --version >/dev/null 2>&1) ; then
 	echo quick-random >> "$GNUPGHOME"/gpg.conf
-- 
2.11.0

[PATCH 17/24] test: use source and build paths in T355-smime.sh

[Jani Nikula]

Make a distinction between source and build directories.
---
 test/T355-smime.sh | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/test/T355-smime.sh b/test/T355-smime.sh
index b813b6cad2e5..1523f17b9f6f 100755
--- a/test/T355-smime.sh
+++ b/test/T355-smime.sh
@@ -10,7 +10,7 @@ add_gpgsm_home ()
     _gnupg_exit () { gpgconf --kill all 2>/dev/null || true; }
     at_exit_function _gnupg_exit
     mkdir -m 0700 "$GNUPGHOME"
-    gpgsm --no-tty --no-common-certs-import --disable-dirmngr --import < $TEST_DIRECTORY/smime/test.crt >"$GNUPGHOME"/import.log 2>&1
+    gpgsm --no-tty --no-common-certs-import --disable-dirmngr --import < $NOTMUCH_SRCDIR/test/smime/test.crt >"$GNUPGHOME"/import.log 2>&1
     fpr=$(gpgsm  --list-key test_suite@notmuchmail.org | sed -n 's/.*fingerprint: //p')
     echo "$fpr S relax" >> $GNUPGHOME/trustlist.txt
     test_debug "cat $GNUPGHOME/import.log"
@@ -19,7 +19,7 @@ add_gpgsm_home ()
 test_require_external_prereq openssl
 test_require_external_prereq gpgsm
 
-cp $TEST_DIRECTORY/smime/key+cert.pem test_suite.pem
+cp $NOTMUCH_SRCDIR/test/smime/key+cert.pem test_suite.pem
 
 FINGERPRINT=$(openssl x509 -fingerprint -in test_suite.pem -noout | sed -e 's/^.*=//' -e s/://g)
 
@@ -41,7 +41,7 @@ test_expect_success \
 
 test_begin_subtest "Signature verification (openssl)"
 notmuch show --format=raw subject:"test signed message 001" |\
-    openssl smime -verify -CAfile $TEST_DIRECTORY/smime/test.crt 2>OUTPUT
+    openssl smime -verify -CAfile $NOTMUCH_SRCDIR/test/smime/test.crt 2>OUTPUT
 cat <<EOF > EXPECTED
 Verification successful
 EOF
@@ -86,7 +86,7 @@ test_expect_equal_json \
 test_begin_subtest "Decryption and signature verification (openssl)"
 notmuch show --format=raw subject:"test encrypted message 001" |\
     openssl smime -decrypt -recip test_suite.pem |\
-    openssl smime -verify -CAfile $TEST_DIRECTORY/smime/test.crt 2>OUTPUT
+    openssl smime -verify -CAfile $NOTMUCH_SRCDIR/test/smime/test.crt 2>OUTPUT
 cat <<EOF > EXPECTED
 Verification successful
 EOF
-- 
2.11.0

[PATCH 18/24] test: use source and build paths in T360-symbol-hiding.sh

[Jani Nikula]

Make a distinction between source and build directories.
---
 test/T360-symbol-hiding.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/test/T360-symbol-hiding.sh b/test/T360-symbol-hiding.sh
index 68edc2d3a7ab..43921cb4a275 100755
--- a/test/T360-symbol-hiding.sh
+++ b/test/T360-symbol-hiding.sh
@@ -26,8 +26,8 @@ test_begin_subtest 'checking output'
 test_expect_equal "$result" "$output"
 
 test_begin_subtest 'comparing existing to exported symbols'
-nm -P $TEST_DIRECTORY/../lib/libnotmuch.so | awk '$2 == "T" && $1 ~ "^notmuch" {print $1}' | sort | uniq > ACTUAL
-sed -n 's/^\(notmuch_[a-zA-Z0-9_]*\)[[:blank:]]*(.*/\1/p' $TEST_DIRECTORY/../lib/notmuch.h | sort | uniq > EXPORTED
+nm -P $NOTMUCH_BUILDDIR/lib/libnotmuch.so | awk '$2 == "T" && $1 ~ "^notmuch" {print $1}' | sort | uniq > ACTUAL
+sed -n 's/^\(notmuch_[a-zA-Z0-9_]*\)[[:blank:]]*(.*/\1/p' $NOTMUCH_SRCDIR/lib/notmuch.h | sort | uniq > EXPORTED
 test_expect_equal_file EXPORTED ACTUAL
 
 test_done
-- 
2.11.0

[PATCH 19/24] test: use source and build paths in T380-atomicity.sh

[Jani Nikula]

Make a distinction between source and build directories.
---
 test/T380-atomicity.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/test/T380-atomicity.sh b/test/T380-atomicity.sh
index 17a3e478aeca..45de22284cb8 100755
--- a/test/T380-atomicity.sh
+++ b/test/T380-atomicity.sh
@@ -64,7 +64,7 @@ if test_require_external_prereq gdb; then
     # -tty /dev/null works around a conflict between the 'timeout' wrapper
     # and gdb's attempt to control the TTY.
     export MAIL_DIR
-    ${TEST_GDB} -tty /dev/null -batch -x $TEST_DIRECTORY/atomicity.py notmuch 1>gdb.out 2>&1
+    ${TEST_GDB} -tty /dev/null -batch -x $NOTMUCH_SRCDIR/test/atomicity.py notmuch 1>gdb.out 2>&1
 
     # Get the final, golden output
     notmuch search '*' > expected
-- 
2.11.0

[PATCH 20/24] test: use source and build paths in T4?0-emacs-*.sh

[Jani Nikula]

Make a distinction between source and build directories.
---
 test/T440-emacs-hello.sh | 2 +-
 test/T450-emacs-show.sh  | 2 +-
 test/T460-emacs-tree.sh  | 2 +-
 3 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/test/T440-emacs-hello.sh b/test/T440-emacs-hello.sh
index 76e795937c6c..d23c1fca9876 100755
--- a/test/T440-emacs-hello.sh
+++ b/test/T440-emacs-hello.sh
@@ -3,7 +3,7 @@
 test_description="emacs notmuch-hello view"
 . $(dirname "$0")/test-lib.sh || exit 1
 
-EXPECTED=$TEST_DIRECTORY/emacs.expected-output
+EXPECTED=$NOTMUCH_SRCDIR/test/emacs.expected-output
 
 add_email_corpus
 
diff --git a/test/T450-emacs-show.sh b/test/T450-emacs-show.sh
index 95babb1f0812..d6aa5b4158b8 100755
--- a/test/T450-emacs-show.sh
+++ b/test/T450-emacs-show.sh
@@ -3,7 +3,7 @@
 test_description="emacs notmuch-show view"
 . $(dirname "$0")/test-lib.sh || exit 1
 
-EXPECTED=$TEST_DIRECTORY/emacs-show.expected-output
+EXPECTED=$NOTMUCH_SRCDIR/test/emacs-show.expected-output
 
 add_email_corpus
 
diff --git a/test/T460-emacs-tree.sh b/test/T460-emacs-tree.sh
index dc5978419be7..0f4e4503fd6e 100755
--- a/test/T460-emacs-tree.sh
+++ b/test/T460-emacs-tree.sh
@@ -3,7 +3,7 @@
 test_description="emacs tree view interface"
 . $(dirname "$0")/test-lib.sh || exit 1
 
-EXPECTED=$TEST_DIRECTORY/tree.expected-output
+EXPECTED=$NOTMUCH_SRCDIR/test/tree.expected-output
 
 add_email_corpus
 
-- 
2.11.0

[PATCH 21/24] test: use source and build paths in T480-hex-escaping.sh

[Jani Nikula]

Make a distinction between source and build directories.
---
 test/T480-hex-escaping.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/test/T480-hex-escaping.sh b/test/T480-hex-escaping.sh
index 28564c3c9da2..2c5bbb631305 100755
--- a/test/T480-hex-escaping.sh
+++ b/test/T480-hex-escaping.sh
@@ -3,7 +3,7 @@ test_description="hex encoding and decoding"
 . $(dirname "$0")/test-lib.sh || exit 1
 
 test_begin_subtest "round trip"
-find $TEST_DIRECTORY/corpora/default -type f -print | sort | xargs cat > EXPECTED
+find $NOTMUCH_SRCDIR/test/corpora/default -type f -print | sort | xargs cat > EXPECTED
 $TEST_DIRECTORY/hex-xcode --direction=encode < EXPECTED | $TEST_DIRECTORY/hex-xcode --direction=decode > OUTPUT
 test_expect_equal_file EXPECTED OUTPUT
 
@@ -25,7 +25,7 @@ $TEST_DIRECTORY/hex-xcode --direction=decode  < EXPECTED.$test_count |\
 test_expect_equal_file EXPECTED.$test_count OUTPUT.$test_count
 
 test_begin_subtest "round trip (in-place)"
-find $TEST_DIRECTORY/corpora/default -type f -print | sort | xargs cat > EXPECTED
+find $NOTMUCH_SRCDIR/test/corpora/default -type f -print | sort | xargs cat > EXPECTED
 $TEST_DIRECTORY/hex-xcode --in-place --direction=encode < EXPECTED |\
      $TEST_DIRECTORY/hex-xcode --in-place --direction=decode > OUTPUT
 test_expect_equal_file EXPECTED OUTPUT
-- 
2.11.0

[PATCH 22/24] test: use source and build paths in T590-thread-breakage.sh

[Jani Nikula]

Make a distinction between source and build directories.
---
 test/T590-thread-breakage.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/test/T590-thread-breakage.sh b/test/T590-thread-breakage.sh
index 53932d63effe..aeb82cf4e3f3 100755
--- a/test/T590-thread-breakage.sh
+++ b/test/T590-thread-breakage.sh
@@ -66,7 +66,7 @@ test_thread_count() {
 
 test_ghost_count() {
     test_begin_subtest "${2:-Expecting $1 ghosts(s)}"
-    ghosts=$(../ghost-report ${MAIL_DIR}/.notmuch/xapian)
+    ghosts=$($NOTMUCH_BUILDDIR/test/ghost-report ${MAIL_DIR}/.notmuch/xapian)
     test_expect_equal "$ghosts" "$1"
 }
 
@@ -111,7 +111,7 @@ test_content_count banana 0
 test_begin_subtest 'No ghosts should remain after deletion of second message'
 # this is known to fail; we are leaking ghost messages deliberately
 test_subtest_known_broken
-ghosts=$(../ghost-report ${MAIL_DIR}/.notmuch/xapian)
+ghosts=$($NOTMUCH_BUILDDIR/test/ghost-report ${MAIL_DIR}/.notmuch/xapian)
 test_expect_equal "$ghosts" "0"
 
 rm -f ${MAIL_DIR}/cur/a
-- 
2.11.0

[PATCH 23/24] build: don't copy the test directory for out-of-tree builds

[Jani Nikula]

It should now work without.
---
 configure | 4 ----
 1 file changed, 4 deletions(-)

diff --git a/configure b/configure
index c0a550c7f9fd..d3e30b53ea51 100755
--- a/configure
+++ b/configure
@@ -40,10 +40,6 @@ if [ "$srcdir" != "." ]; then
 	cp "$srcdir"/"$dir"/Makefile "$dir"
     done
 
-    # Easiest way to get the test suite to work is to just copy the
-    # whole thing into the build directory.
-    cp -a "$srcdir"/test/* test
-
     # Emacs only likes to generate compiled files next to the .el files
     # by default so copy these as well (which is not ideal).
     cp -a "$srcdir"/emacs/*.el emacs
-- 
2.11.0

[PATCH 24/24] devel: add script to test out-of-tree builds

[Jani Nikula]

Something I used for 'git bisect run', but we should really add this
as part of our process.

Can also be used for running out-of-tree tests with e.g.:

$ devel/out-of-tree-build-check.sh V=1 test
---
 devel/out-of-tree-build-check.sh | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)
 create mode 100755 devel/out-of-tree-build-check.sh

diff --git a/devel/out-of-tree-build-check.sh b/devel/out-of-tree-build-check.sh
new file mode 100755
index 000000000000..917752ff72cb
--- /dev/null
+++ b/devel/out-of-tree-build-check.sh
@@ -0,0 +1,16 @@
+#!/bin/sh
+# test out-of-tree builds in a temp directory
+# passes all args to make
+
+set -eu
+
+srcdir="$(cd "$(dirname "$0")"/.. && pwd)"
+builddir=$(mktemp -d)
+
+cd $builddir
+
+$srcdir/configure
+make "$@"
+
+cd $srcdir
+rm -rf $builddir
-- 
2.11.0

Re: [PATCH 24/24] devel: add script to test out-of-tree builds

[Tomi Ollila]

On Mon, Sep 25 2017, Jani Nikula wrote:

> Something I used for 'git bisect run', but we should really add this
> as part of our process.
>
> Can also be used for running out-of-tree tests with e.g.:
>
> $ devel/out-of-tree-build-check.sh V=1 test

This series looks pretty good (there are some ""-quote inconsistensies in
some of the smaller-numbered patches, but probably not adding to the
current state of the art -- just note that if these were my patches
that would be intolerable >;D)

So, IMO, the above can be pushed as is (On Fedora 26 the test results
are identical with and without these patches applied (855 succeeded,
7 expected failures, 2 tests skipped on one of my systems)

This last one (in addition to quote inconsistencies) has also slight
naming inconsistency compared w/ other files in that directory; To match
IMO this should be named as check-out-of-tree-build.sh... And since
I had to comment on this I'll express my nitpicks of the content, too :D:

Tomi

> ---
>  devel/out-of-tree-build-check.sh | 16 ++++++++++++++++
>  1 file changed, 16 insertions(+)
>  create mode 100755 devel/out-of-tree-build-check.sh
>
> diff --git a/devel/out-of-tree-build-check.sh b/devel/out-of-tree-build-check.sh
> new file mode 100755
> index 000000000000..917752ff72cb
> --- /dev/null
> +++ b/devel/out-of-tree-build-check.sh
> @@ -0,0 +1,16 @@
> +#!/bin/sh
> +# test out-of-tree builds in a temp directory
> +# passes all args to make
> +
> +set -eu
> +
> +srcdir="$(cd "$(dirname "$0")"/.. && pwd)"

quotes unnecessary above, but may clarify things...

> +builddir=$(mktemp -d)

... but the above is then inconsistent as it does not have ""-quotes

> +
> +cd $builddir

due to $(mktemp -d) not outputting any $IFS characters the above always
works without "$builddir" but imo not having quotes there is a bad example
(and inconsistent to what I'm going to write below)

> +$srcdir/configure

$srcdir, OTOH could have $IFS characters (usually spaces) there, so above
line would break in such a case

> +make "$@"
> +
> +cd $srcdir

ditto

> +rm -rf $builddir

consistency

> -- 
> 2.11.0

Re: [PATCH 24/24] devel: add script to test out-of-tree builds

[Tomi Ollila]

On Tue, Sep 26 2017, Tomi Ollila wrote:

>> +
>> +set -eu
>> +
>> +srcdir="$(cd "$(dirname "$0")"/.. && pwd)"
>
> quotes unnecessary above, but may clarify things...

outmost quotes, that is -- e.g. srcdir=$(cd "$(dirname "$0")"/.. && pwd)
-- other quotes are necessary

>
>> +builddir=$(mktemp -d)

Re: [PATCH 00/24] test: separate source and build directories

[David Bremner]

Jani Nikula <jani@nikula.org> writes:

> This lengthy but mostly mechanical series separates all source and build
> directory references in the test framework to be able to properly build
> and run out-of-tree tests. At the end, the 'cp -a' of the test directory
> from source to build can be dropped.

I pushed most of these, except 16, which needs to be manually moved to
test-lib.sh and 24 which is now obsolete.

Unfortunately this has introduced the following bug: running make test
twice without running make clean between causes T310-emacs.sh to be
terminated with signal 15 (maybe from a timeout?). So I guess something
is not being removed / shut dow by the test suite.

reverting a8cd1ec26b7ed1144  "fixes" the problem so I guess if you don't
have any better ideas I can do that for now.

d

[PATCH] test: remove --root option and fix TMP_DIRECTORY cleanup

[Jani Nikula]

The primary motivation here is to fix TMP_DIRECTORY cleanup prior to
running each test when the current working directory is not the test
subdirectory. Tests with failures would leave their TMP_DIRECTORY
directory behind for debugging, and repeated out-of-tree test runs
would have old temp directories. (This lead to e.g. T310-emacs.sh
hanging because emacs would prompt for overwriting files.)

We remove the likely anyway defunct --root test option while at it,
just to be on the safe side when doing 'rm -rf' on the TMP_DIRECTORY.
---
 test/README             | 9 ---------
 test/test-lib-common.sh | 8 ++------
 test/test-lib.sh        | 3 ---
 3 files changed, 2 insertions(+), 18 deletions(-)

diff --git a/test/README b/test/README
index 8e06f44241a6..b378c3ff3c5f 100644
--- a/test/README
+++ b/test/README
@@ -80,15 +80,6 @@ The following command-line options are available when running tests:
 	As the names depend on the tests' file names, it is safe to
 	run the tests with this option in parallel.
 
---root=<dir>::
-	This runs the testsuites specified under a separate directory.
-	However, caution is advised, as not all tests are maintained
-	with this relocation in mind, so some tests may behave
-	differently.
-
-	Pointing this argument at a tmpfs filesystem can improve the
-	speed of the test suite for some users.
-
 Certain tests require precomputed databases to complete. You can fetch these
 databases with
 
diff --git a/test/test-lib-common.sh b/test/test-lib-common.sh
index 5e53348a9438..4300eb65418f 100644
--- a/test/test-lib-common.sh
+++ b/test/test-lib-common.sh
@@ -307,13 +307,9 @@ export PATH MANPATH
 
 # Test repository
 test="tmp.$(basename "$0" .sh)"
-test -n "$root" && test="$root/$test"
-case "$test" in
-/*) TMP_DIRECTORY="$test" ;;
- *) TMP_DIRECTORY="$TEST_DIRECTORY/$test" ;;
-esac
+TMP_DIRECTORY="$TEST_DIRECTORY/$test"
 test ! -z "$debug" || remove_tmp=$TMP_DIRECTORY
-rm -fr "$test" || {
+rm -rf "$TMP_DIRECTORY" || {
 	GIT_EXIT_OK=t
 	echo >&6 "FATAL: Cannot prepare test area"
 	exit 1
diff --git a/test/test-lib.sh b/test/test-lib.sh
index 4619c327dd02..7926d2787710 100644
--- a/test/test-lib.sh
+++ b/test/test-lib.sh
@@ -160,9 +160,6 @@ do
 		valgrind=t; verbose=t; shift ;;
 	--tee)
 		shift ;; # was handled already
-	--root=*)
-		root=$(expr "z$1" : 'z[^=]*=\(.*\)')
-		shift ;;
 	*)
 		echo "error: unknown test option '$1'" >&2; exit 1 ;;
 	esac
-- 
2.11.0

Re: [PATCH] test: remove --root option and fix TMP_DIRECTORY cleanup

[Jani Nikula]

On Sat, 21 Oct 2017, Jani Nikula <jani@nikula.org> wrote:
> The primary motivation here is to fix TMP_DIRECTORY cleanup prior to
> running each test when the current working directory is not the test
> subdirectory. Tests with failures would leave their TMP_DIRECTORY
> directory behind for debugging, and repeated out-of-tree test runs
> would have old temp directories. (This lead to e.g. T310-emacs.sh
> hanging because emacs would prompt for overwriting files.)

To clarify, plain 'make test' no longer has its CWD in the test
subdirectory either, leading to the same problem as out-of-tree runs
here.

The problem noticed by David could also be reproduced by running
'test/T310-emacs.sh; test/T310-emacs.sh' but not 'cd test;
./T310-emacs.sh; ./T310-emacs.sh'.

BR,
Jani.

>
> We remove the likely anyway defunct --root test option while at it,
> just to be on the safe side when doing 'rm -rf' on the TMP_DIRECTORY.
> ---
>  test/README             | 9 ---------
>  test/test-lib-common.sh | 8 ++------
>  test/test-lib.sh        | 3 ---
>  3 files changed, 2 insertions(+), 18 deletions(-)
>
> diff --git a/test/README b/test/README
> index 8e06f44241a6..b378c3ff3c5f 100644
> --- a/test/README
> +++ b/test/README
> @@ -80,15 +80,6 @@ The following command-line options are available when running tests:
>  	As the names depend on the tests' file names, it is safe to
>  	run the tests with this option in parallel.
>  
> ---root=<dir>::
> -	This runs the testsuites specified under a separate directory.
> -	However, caution is advised, as not all tests are maintained
> -	with this relocation in mind, so some tests may behave
> -	differently.
> -
> -	Pointing this argument at a tmpfs filesystem can improve the
> -	speed of the test suite for some users.
> -
>  Certain tests require precomputed databases to complete. You can fetch these
>  databases with
>  
> diff --git a/test/test-lib-common.sh b/test/test-lib-common.sh
> index 5e53348a9438..4300eb65418f 100644
> --- a/test/test-lib-common.sh
> +++ b/test/test-lib-common.sh
> @@ -307,13 +307,9 @@ export PATH MANPATH
>  
>  # Test repository
>  test="tmp.$(basename "$0" .sh)"
> -test -n "$root" && test="$root/$test"
> -case "$test" in
> -/*) TMP_DIRECTORY="$test" ;;
> - *) TMP_DIRECTORY="$TEST_DIRECTORY/$test" ;;
> -esac
> +TMP_DIRECTORY="$TEST_DIRECTORY/$test"
>  test ! -z "$debug" || remove_tmp=$TMP_DIRECTORY
> -rm -fr "$test" || {
> +rm -rf "$TMP_DIRECTORY" || {
>  	GIT_EXIT_OK=t
>  	echo >&6 "FATAL: Cannot prepare test area"
>  	exit 1
> diff --git a/test/test-lib.sh b/test/test-lib.sh
> index 4619c327dd02..7926d2787710 100644
> --- a/test/test-lib.sh
> +++ b/test/test-lib.sh
> @@ -160,9 +160,6 @@ do
>  		valgrind=t; verbose=t; shift ;;
>  	--tee)
>  		shift ;; # was handled already
> -	--root=*)
> -		root=$(expr "z$1" : 'z[^=]*=\(.*\)')
> -		shift ;;
>  	*)
>  		echo "error: unknown test option '$1'" >&2; exit 1 ;;
>  	esac
> -- 
> 2.11.0

Re: [PATCH] test: remove --root option and fix TMP_DIRECTORY cleanup

[David Bremner]

Jani Nikula <jani@nikula.org> writes:

> The primary motivation here is to fix TMP_DIRECTORY cleanup prior to
> running each test when the current working directory is not the test
> subdirectory. Tests with failures would leave their TMP_DIRECTORY
> directory behind for debugging, and repeated out-of-tree test runs
> would have old temp directories. (This lead to e.g. T310-emacs.sh
> hanging because emacs would prompt for overwriting files.)
>
> We remove the likely anyway defunct --root test option while at it,
> just to be on the safe side when doing 'rm -rf' on the TMP_DIRECTORY.

This seems to fix the problem I reported, however applying

     id:cd03efa9c93ee54f7e7f7e166079062984ddd658.1506370901.git.jani@nikula.org

on top breaks

      ./devel/check-out-of-tree-build.sh test

I get lots of errors along the lines of

      Error opening database at
      /tmp/tmp.GD3HPpejbL/test/tmp.T350-crypto/mail/.notmuch: No such
      file or directory

So this patch seems to unbreak in-tree builds (and out of tree builds
where we copy the whole test hierarchy), and is probably worth applying
as is, but I wanted to give you a chance to respond before I proceed.

d

[PATCH v2] test: use source path in add_gnupg_home

[Jani Nikula]

Make a distinction between source and build directories.

---

v2 rebase, add_gnupg_home is now in test-lib.sh
---
 test/test-lib.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/test/test-lib.sh b/test/test-lib.sh
index 7926d2787710..5274e6e1cf2c 100644
--- a/test/test-lib.sh
+++ b/test/test-lib.sh
@@ -109,7 +109,7 @@ add_gnupg_home ()
     _gnupg_exit () { gpgconf --kill all 2>/dev/null || true; }
     at_exit_function _gnupg_exit
     mkdir -m 0700 "$GNUPGHOME"
-    gpg --no-tty --import <$TEST_DIRECTORY/gnupg-secret-key.asc >"$GNUPGHOME"/import.log 2>&1
+    gpg --no-tty --import <$NOTMUCH_SRCDIR/test/gnupg-secret-key.asc >"$GNUPGHOME"/import.log 2>&1
     test_debug "cat $GNUPGHOME/import.log"
     if (gpg --quick-random --version >/dev/null 2>&1) ; then
 	echo quick-random >> "$GNUPGHOME"/gpg.conf
-- 
2.11.0

Re: [PATCH] test: remove --root option and fix TMP_DIRECTORY cleanup

[Jani Nikula]

On Sat, 21 Oct 2017, David Bremner <david@tethera.net> wrote:
> Jani Nikula <jani@nikula.org> writes:
>
>> The primary motivation here is to fix TMP_DIRECTORY cleanup prior to
>> running each test when the current working directory is not the test
>> subdirectory. Tests with failures would leave their TMP_DIRECTORY
>> directory behind for debugging, and repeated out-of-tree test runs
>> would have old temp directories. (This lead to e.g. T310-emacs.sh
>> hanging because emacs would prompt for overwriting files.)
>>
>> We remove the likely anyway defunct --root test option while at it,
>> just to be on the safe side when doing 'rm -rf' on the TMP_DIRECTORY.
>
> This seems to fix the problem I reported, however applying
>
>      id:cd03efa9c93ee54f7e7f7e166079062984ddd658.1506370901.git.jani@nikula.org
>
> on top breaks
>
>       ./devel/check-out-of-tree-build.sh test
>
> I get lots of errors along the lines of
>
>       Error opening database at
>       /tmp/tmp.GD3HPpejbL/test/tmp.T350-crypto/mail/.notmuch: No such
>       file or directory
>
> So this patch seems to unbreak in-tree builds (and out of tree builds
> where we copy the whole test hierarchy), and is probably worth applying
> as is, but I wanted to give you a chance to respond before I proceed.

So these two should be applied first, in either order:

id:20171021192141.6666-1-jani@nikula.org
id:20171021115802.31197-1-jani@nikula.org

And then this should work on top:

id:cd03efa9c93ee54f7e7f7e166079062984ddd658.1506370901.git.jani@nikula.org

BR,
Jani.

[PATCH] NEWS: test suite out-of-tree builds

[Jani Nikula]

Out-of-tree builds now work and supersede --root option.
---
 NEWS | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/NEWS b/NEWS
index 3c8198d3ef12..c2ebb6447bd0 100644
--- a/NEWS
+++ b/NEWS
@@ -1,3 +1,16 @@
+Notmuch 0.26 (UNRELEASED)
+=========================
+
+Test Suite
+----------
+
+Out-of-tree builds
+
+  The test suite now works properly with out-of-tree builds, i.e. with
+  separate source and build directories. The --root option to tests
+  has been dropped. The same can now be achieved more reliably using
+  out-of-tree builds.
+
 Notmuch 0.25.1 (2017-09-11)
 ===========================
 
-- 
2.11.0

Re: [PATCH] test: remove --root option and fix TMP_DIRECTORY cleanup

[David Bremner]

Jani Nikula <jani@nikula.org> writes:


> So these two should be applied first, in either order:
>
> id:20171021192141.6666-1-jani@nikula.org
> id:20171021115802.31197-1-jani@nikula.org
>
> And then this should work on top:
>
> id:cd03efa9c93ee54f7e7f7e166079062984ddd658.1506370901.git.jani@nikula.org

Applied, and pushed.

d

Re: [PATCH] NEWS: test suite out-of-tree builds

[David Bremner]

Jani Nikula <jani@nikula.org> writes:

> Out-of-tree builds now work and supersede --root option.
> ---

pushed,

d