set test prereqs (Emacs, GDB, GPG) v4

set test prereqs (Emacs, GDB, GPG) v4

[Pieter Praet]

Rebased to current master.

Previous version:
  id:"1307016220-17509-1-git-send-email-pieter@praet.org"

Discussion:
  id:"1317660447-27520-1-git-send-email-schnouki@schnouki.net"

[PATCH 1/4] test: add 'GnuPG' prereq to dependent 'crypto' tests

[Pieter Praet]

Adds a new test that checks for the presence of 'gpg',
and adds that test as a prereq to all subsequent tests
that rely on GnuPG.

This causes tests with unmet dependencies to be skipped.

---
 test/crypto |   33 +++++++++++++++++++--------------
 1 files changed, 19 insertions(+), 14 deletions(-)

diff --git a/test/crypto b/test/crypto
index 0af4aa8..3795926 100755
--- a/test/crypto
+++ b/test/crypto
@@ -7,6 +7,11 @@
 test_description='PGP/MIME signature verification and decryption'
 . ./test-lib.sh
 
+# GnuPG is a prereq.
+test_expect_success "prereq: GnuPG is present" "which gpg" \
+    && test_set_prereq GPG
+
+
 add_gnupg_home ()
 {
     local output
@@ -31,7 +36,7 @@ FINGERPRINT=$(gpg --no-tty --list-secret-keys --with-colons --fingerprint | grep
 # although I can't figure out why
 add_email_corpus
 
-test_expect_success 'emacs delivery of signed message' \
+test_expect_success GPG 'emacs delivery of signed message' \
 'emacs_deliver_message \
     "test signed message 001" \
     "This is a test signed message." \
@@ -64,7 +69,7 @@ expected='[[[{"id": "XXXXX",
  {"id": 3,
  "content-type": "application/pgp-signature"}]}]},
  []]]]'
-test_expect_equal \
+test_expect_equal GPG \
     "$output" \
     "$expected"
 
@@ -99,7 +104,7 @@ expected='[[[{"id": "XXXXX",
  {"id": 3,
  "content-type": "application/pgp-signature"}]}]},
  []]]]'
-test_expect_equal \
+test_expect_equal GPG \
     "$output" \
     "$expected"
 
@@ -132,7 +137,7 @@ expected='[[[{"id": "XXXXX",
  {"id": 3,
  "content-type": "application/pgp-signature"}]}]},
  []]]]'
-test_expect_equal \
+test_expect_equal GPG \
     "$output" \
     "$expected"
 mv "${GNUPGHOME}"{.bak,}
@@ -141,7 +146,7 @@ mv "${GNUPGHOME}"{.bak,}
 cat <<EOF >TESTATTACHMENT
 This is a test file.
 EOF
-test_expect_success 'emacs delivery of encrypted message with attachment' \
+test_expect_success GPG 'emacs delivery of encrypted message with attachment' \
 'emacs_deliver_message \
     "test encrypted message 001" \
     "This is a test encrypted message.\n" \
@@ -175,7 +180,7 @@ Non-text part: application/octet-stream
 \fpart}
 \fbody}
 \fmessage}'
-test_expect_equal \
+test_expect_equal GPG \
     "$output" \
     "$expected"
 
@@ -210,7 +215,7 @@ expected='[[[{"id": "XXXXX",
  "content-type": "application/octet-stream",
  "filename": "TESTATTACHMENT"}]}]}]},
  []]]]'
-test_expect_equal \
+test_expect_equal GPG \
     "$output" \
     "$expected"
 
@@ -221,7 +226,7 @@ output=$(notmuch show --format=json --part=4 --decrypt subject:"test encrypted m
 expected='{"id": 4,
  "content-type": "text/plain",
  "content": "This is a test encrypted message.\n"}'
-test_expect_equal \
+test_expect_equal GPG \
     "$output" \
     "$expected"
 
@@ -231,7 +236,7 @@ notmuch show \
     --part=5 \
     --decrypt \
     subject:"test encrypted message 001" >OUTPUT
-test_expect_equal_file OUTPUT TESTATTACHMENT
+test_expect_equal_file GPG OUTPUT TESTATTACHMENT
 
 test_begin_subtest "decryption failure with missing key"
 mv "${GNUPGHOME}"{,.bak}
@@ -258,12 +263,12 @@ expected='[[[{"id": "XXXXX",
  {"id": 3,
  "content-type": "application/octet-stream"}]}]},
  []]]]'
-test_expect_equal \
+test_expect_equal GPG \
     "$output" \
     "$expected"
 mv "${GNUPGHOME}"{.bak,}
 
-test_expect_success 'emacs delivery of encrypted + signed message' \
+test_expect_success GPG 'emacs delivery of encrypted + signed message' \
 'emacs_deliver_message \
     "test encrypted message 002" \
     "This is another test encrypted message.\n" \
@@ -298,7 +303,7 @@ expected='[[[{"id": "XXXXX",
  "content-type": "text/plain",
  "content": "This is another test encrypted message.\n"}]}]},
  []]]]'
-test_expect_equal \
+test_expect_equal GPG \
     "$output" \
     "$expected"
 
@@ -310,7 +315,7 @@ Subject: Re: test encrypted message 002
 
 On 01 Jan 2000 12:00:00 -0000, Notmuch Test Suite <test_suite@notmuchmail.org> wrote:
 > This is another test encrypted message.'
-test_expect_equal \
+test_expect_equal GPG \
     "$output" \
     "$expected"
 
@@ -351,7 +356,7 @@ expected='[[[{"id": "XXXXX",
  {"id": 3,
  "content-type": "application/pgp-signature"}]}]},
  []]]]'
-test_expect_equal \
+test_expect_equal GPG \
     "$output" \
     "$expected"
 
-- 
1.7.7.1

[PATCH 2/4] test: add 'Emacs' prereq to dependent 'crypto' tests

[Pieter Praet]

Adds a new test that checks for the presence of 'emacs',
and adds that test as a prereq to all subsequent tests
that rely on Emacs.

This causes tests with unmet dependencies to be skipped.

---
 test/crypto |   17 ++++++++++++++---
 1 files changed, 14 insertions(+), 3 deletions(-)

diff --git a/test/crypto b/test/crypto
index 3795926..4a00c00 100755
--- a/test/crypto
+++ b/test/crypto
@@ -7,10 +7,21 @@
 test_description='PGP/MIME signature verification and decryption'
 . ./test-lib.sh
 
+# Emacs is a prereq.
+test_expect_success "prereq: Emacs is present" "which emacs" \
+    && test_set_prereq EMACS
+
 # GnuPG is a prereq.
 test_expect_success "prereq: GnuPG is present" "which gpg" \
     && test_set_prereq GPG
 
+# Some tests have multiple prereqs, but the test_expect_* functions
+# accept only a single argument as prereq tag, and using test_have_prereq
+# in and around tests causes various errors for me, so a dirty workaround
+# will have to do for the time being.
+test_have_prereq EMACS && test_have_prereq GPG \
+    && test_set_prereq EMACS+GPG
+
 
 add_gnupg_home ()
 {
@@ -36,7 +47,7 @@ FINGERPRINT=$(gpg --no-tty --list-secret-keys --with-colons --fingerprint | grep
 # although I can't figure out why
 add_email_corpus
 
-test_expect_success GPG 'emacs delivery of signed message' \
+test_expect_success EMACS+GPG 'emacs delivery of signed message' \
 'emacs_deliver_message \
     "test signed message 001" \
     "This is a test signed message." \
@@ -146,7 +157,7 @@ mv "${GNUPGHOME}"{.bak,}
 cat <<EOF >TESTATTACHMENT
 This is a test file.
 EOF
-test_expect_success GPG 'emacs delivery of encrypted message with attachment' \
+test_expect_success EMACS+GPG 'emacs delivery of encrypted message with attachment' \
 'emacs_deliver_message \
     "test encrypted message 001" \
     "This is a test encrypted message.\n" \
@@ -268,7 +279,7 @@ test_expect_equal GPG \
     "$expected"
 mv "${GNUPGHOME}"{.bak,}
 
-test_expect_success GPG 'emacs delivery of encrypted + signed message' \
+test_expect_success EMACS+GPG 'emacs delivery of encrypted + signed message' \
 'emacs_deliver_message \
     "test encrypted message 002" \
     "This is another test encrypted message.\n" \
-- 
1.7.7.1

[PATCH 3/4] test: add 'Emacs' prereq to dependent 'emacs' tests

[Pieter Praet]

Adds a new test that checks for the presence of 'emacs',
and adds that test as a prereq to all subsequent tests
that rely on Emacs.

This causes tests with unmet dependencies to be skipped.

---
 test/emacs |   65 ++++++++++++++++++++++++++++++++---------------------------
 1 files changed, 35 insertions(+), 30 deletions(-)

diff --git a/test/emacs b/test/emacs
index 0303d7d..8b1e5e4 100755
--- a/test/emacs
+++ b/test/emacs
@@ -3,6 +3,11 @@
 test_description="emacs interface"
 . test-lib.sh
 
+# Emacs is a prereq.
+test_expect_success "prereq: Emacs is present" "which emacs" \
+    && test_set_prereq EMACS
+
+
 EXPECTED=$TEST_DIRECTORY/emacs.expected-output
 
 add_email_corpus
@@ -10,7 +15,7 @@ add_email_corpus
 test_begin_subtest "Basic notmuch-hello view in emacs"
 test_emacs '(notmuch-hello)
 	    (test-output)'
-test_expect_equal_file OUTPUT $EXPECTED/notmuch-hello
+test_expect_equal_file EMACS OUTPUT $EXPECTED/notmuch-hello
 
 test_begin_subtest "Saved search with 0 results"
 test_emacs '(let ((notmuch-show-empty-saved-searches t)
@@ -20,20 +25,20 @@ test_emacs '(let ((notmuch-show-empty-saved-searches t)
 			("empty" . "tag:doesnotexist"))))
 	      (notmuch-hello)
 	      (test-output))'
-test_expect_equal_file OUTPUT $EXPECTED/notmuch-hello-with-empty
+test_expect_equal_file EMACS OUTPUT $EXPECTED/notmuch-hello-with-empty
 
 test_begin_subtest "No saved searches displayed (all with 0 results)"
 test_emacs '(let ((notmuch-saved-searches
 		   '\''(("empty" . "tag:doesnotexist"))))
 	      (notmuch-hello)
 	      (test-output))'
-test_expect_equal_file OUTPUT $EXPECTED/notmuch-hello-no-saved-searches
+test_expect_equal_file EMACS OUTPUT $EXPECTED/notmuch-hello-no-saved-searches
 
 test_begin_subtest "Basic notmuch-search view in emacs"
 test_emacs '(notmuch-search "tag:inbox")
 	    (notmuch-test-wait)
 	    (test-output)'
-test_expect_equal_file OUTPUT $EXPECTED/notmuch-search-tag-inbox
+test_expect_equal_file EMACS OUTPUT $EXPECTED/notmuch-search-tag-inbox
 
 test_begin_subtest "Navigation of notmuch-hello to search results"
 test_emacs '(notmuch-hello)
@@ -42,13 +47,13 @@ test_emacs '(notmuch-hello)
 	    (widget-button-press (point))
 	    (notmuch-test-wait)
 	    (test-output)'
-test_expect_equal_file OUTPUT $EXPECTED/notmuch-hello-view-inbox
+test_expect_equal_file EMACS OUTPUT $EXPECTED/notmuch-hello-view-inbox
 
 test_begin_subtest "Basic notmuch-show view in emacs"
 maildir_storage_thread=$(notmuch search --output=threads id:20091117190054.GU3165@dottiness.seas.harvard.edu)
 test_emacs "(notmuch-show \"$maildir_storage_thread\")
 	    (test-output)"
-test_expect_equal_file OUTPUT $EXPECTED/notmuch-show-thread-maildir-storage
+test_expect_equal_file EMACS OUTPUT $EXPECTED/notmuch-show-thread-maildir-storage
 
 test_begin_subtest "notmuch-show for message with invalid From"
 add_message "[subject]=\"message-with-invalid-from\"" \
@@ -64,7 +69,7 @@ Date: Tue, 05 Jan 2001 15:43:57 -0000
 
 This is just a test message (#1)
 EOF
-test_expect_equal_file OUTPUT EXPECTED
+test_expect_equal_file EMACS OUTPUT EXPECTED
 
 test_begin_subtest "Navigation of notmuch-search to thread view"
 test_emacs '(notmuch-search "tag:inbox")
@@ -74,7 +79,7 @@ test_emacs '(notmuch-search "tag:inbox")
 	    (notmuch-search-show-thread)
 	    (notmuch-test-wait)
 	    (test-output)'
-test_expect_equal_file OUTPUT $EXPECTED/notmuch-show-thread-maildir-storage
+test_expect_equal_file EMACS OUTPUT $EXPECTED/notmuch-show-thread-maildir-storage
 
 test_begin_subtest "Add tag from search view"
 os_x_darwin_thread=$(notmuch search --output=threads id:ddd65cda0911171950o4eea4389v86de9525e46052d3@mail.gmail.com)
@@ -82,26 +87,26 @@ test_emacs "(notmuch-search \"$os_x_darwin_thread\")
 	    (notmuch-test-wait)
 	    (notmuch-search-add-tag \"tag-from-search-view\")"
 output=$(notmuch search $os_x_darwin_thread | notmuch_search_sanitize)
-test_expect_equal "$output" "thread:XXX   2009-11-18 [4/4] Jjgod Jiang, Alexander Botero-Lowry; [notmuch] Mac OS X/Darwin compatibility issues (inbox tag-from-search-view unread)"
+test_expect_equal EMACS "$output" "thread:XXX   2009-11-18 [4/4] Jjgod Jiang, Alexander Botero-Lowry; [notmuch] Mac OS X/Darwin compatibility issues (inbox tag-from-search-view unread)"
 
 test_begin_subtest "Remove tag from search view"
 test_emacs "(notmuch-search \"$os_x_darwin_thread\")
 	    (notmuch-test-wait)
 	    (notmuch-search-remove-tag \"tag-from-search-view\")"
 output=$(notmuch search $os_x_darwin_thread | notmuch_search_sanitize)
-test_expect_equal "$output" "thread:XXX   2009-11-18 [4/4] Jjgod Jiang, Alexander Botero-Lowry; [notmuch] Mac OS X/Darwin compatibility issues (inbox unread)"
+test_expect_equal EMACS "$output" "thread:XXX   2009-11-18 [4/4] Jjgod Jiang, Alexander Botero-Lowry; [notmuch] Mac OS X/Darwin compatibility issues (inbox unread)"
 
 test_begin_subtest "Add tag from notmuch-show view"
 test_emacs "(notmuch-show \"$os_x_darwin_thread\")
 	    (notmuch-show-add-tag \"tag-from-show-view\")"
 output=$(notmuch search $os_x_darwin_thread | notmuch_search_sanitize)
-test_expect_equal "$output" "thread:XXX   2009-11-18 [4/4] Jjgod Jiang, Alexander Botero-Lowry; [notmuch] Mac OS X/Darwin compatibility issues (inbox tag-from-show-view unread)"
+test_expect_equal EMACS "$output" "thread:XXX   2009-11-18 [4/4] Jjgod Jiang, Alexander Botero-Lowry; [notmuch] Mac OS X/Darwin compatibility issues (inbox tag-from-show-view unread)"
 
 test_begin_subtest "Remove tag from notmuch-show view"
 test_emacs "(notmuch-show \"$os_x_darwin_thread\")
 	    (notmuch-show-remove-tag \"tag-from-show-view\")"
 output=$(notmuch search $os_x_darwin_thread | notmuch_search_sanitize)
-test_expect_equal "$output" "thread:XXX   2009-11-18 [4/4] Jjgod Jiang, Alexander Botero-Lowry; [notmuch] Mac OS X/Darwin compatibility issues (inbox unread)"
+test_expect_equal EMACS "$output" "thread:XXX   2009-11-18 [4/4] Jjgod Jiang, Alexander Botero-Lowry; [notmuch] Mac OS X/Darwin compatibility issues (inbox unread)"
 
 test_begin_subtest "Message with .. in Message-Id:"
 add_message [id]=123..456@example '[subject]="Message with .. in Message-Id"'
@@ -116,7 +121,7 @@ test_emacs '(notmuch-search "id:\"123..456@example\"")
 	    (notmuch-show-add-tag "show-remove")
 	    (notmuch-show-remove-tag "show-remove")'
 output=$(notmuch search 'id:"123..456@example"' | notmuch_search_sanitize)
-test_expect_equal "$output" "thread:XXX   2001-01-05 [1/1] Notmuch Test Suite; Message with .. in Message-Id (inbox search-add show-add)"
+test_expect_equal EMACS "$output" "thread:XXX   2001-01-05 [1/1] Notmuch Test Suite; Message with .. in Message-Id (inbox search-add show-add)"
 
 test_begin_subtest "Sending a message via (fake) SMTP"
 emacs_deliver_message \
@@ -140,7 +145,7 @@ Content-Type: text/plain; charset=us-ascii
 
 This is a test that messages are sent via SMTP
 EOF
-test_expect_equal_file OUTPUT EXPECTED
+test_expect_equal_file EMACS OUTPUT EXPECTED
 
 test_begin_subtest "Verify that sent messages are saved/searchable (via FCC)"
 notmuch new > /dev/null
@@ -157,7 +162,7 @@ To:
 Subject: 
 --text follows this line--
 EOF
-test_expect_equal_file OUTPUT EXPECTED
+test_expect_equal_file EMACS OUTPUT EXPECTED
 
 # Make another FCC maildir specific for the next test
 mkdir -p mail/sent-string/cur
@@ -175,7 +180,7 @@ Subject:
 Fcc: ${MAIL_DIR}/sent-string
 --text follows this line--
 EOF
-test_expect_equal_file OUTPUT EXPECTED
+test_expect_equal_file EMACS OUTPUT EXPECTED
 
 # Make more FCC maildirs specific for the next test
 mkdir -p mail/sent-list-match/cur
@@ -198,7 +203,7 @@ Subject:
 Fcc: ${MAIL_DIR}/sent-list-match
 --text follows this line--
 EOF
-test_expect_equal_file OUTPUT EXPECTED
+test_expect_equal_file EMACS OUTPUT EXPECTED
 
 # Make another FCC maildir specific for the next test
 mkdir -p mail/sent-list-catch-all/cur
@@ -218,7 +223,7 @@ Subject:
 Fcc: ${MAIL_DIR}/sent-list-catch-all
 --text follows this line--
 EOF
-test_expect_equal_file OUTPUT EXPECTED
+test_expect_equal_file EMACS OUTPUT EXPECTED
 
 test_begin_subtest "notmuch-fcc-dirs set to a list (no match)"
 test_emacs "(let ((notmuch-fcc-dirs
@@ -232,7 +237,7 @@ To:
 Subject: 
 --text follows this line--
 EOF
-test_expect_equal_file OUTPUT EXPECTED
+test_expect_equal_file EMACS OUTPUT EXPECTED
 
 test_begin_subtest "Reply within emacs"
 test_emacs '(notmuch-search "subject:\"testing message sent via SMTP\"")
@@ -250,26 +255,26 @@ Fcc: $(pwd)/mail/sent
 On 01 Jan 2000 12:00:00 -0000, Notmuch Test Suite <test_suite@notmuchmail.org> wrote:
 > This is a test that messages are sent via SMTP
 EOF
-test_expect_equal_file OUTPUT EXPECTED
+test_expect_equal_file EMACS OUTPUT EXPECTED
 
 test_begin_subtest "Save attachment from within emacs using notmuch-show-save-attachments"
 # save as archive to test that Emacs does not re-compress .gz
 test_emacs '(let ((standard-input "\"attachment1.gz\""))
 	      (notmuch-show "id:cf0c4d610911171136h1713aa59w9cf9aa31f052ad0a@mail.gmail.com")
 	      (notmuch-show-save-attachments))' > /dev/null 2>&1
-test_expect_equal_file attachment1.gz "$EXPECTED/attachment"
+test_expect_equal_file EMACS attachment1.gz "$EXPECTED/attachment"
 
 test_begin_subtest "Save attachment from within emacs using notmuch-show-save-part"
 # save as archive to test that Emacs does not re-compress .gz
 test_emacs '(let ((standard-input "\"attachment2.gz\""))
 	      (notmuch-show-save-part "id:cf0c4d610911171136h1713aa59w9cf9aa31f052ad0a@mail.gmail.com" 5))' > /dev/null 2>&1
-test_expect_equal_file attachment2.gz "$EXPECTED/attachment"
+test_expect_equal_file EMACS attachment2.gz "$EXPECTED/attachment"
 
 test_begin_subtest "View raw message within emacs"
 test_emacs '(notmuch-show "id:cf0c4d610911171136h1713aa59w9cf9aa31f052ad0a@mail.gmail.com")
 	    (notmuch-show-view-raw-message)
 	    (test-output)'
-test_expect_equal_file OUTPUT $EXPECTED/raw-message-cf0c4d-52ad0a
+test_expect_equal_file EMACS OUTPUT $EXPECTED/raw-message-cf0c4d-52ad0a
 
 test_begin_subtest "Hiding/showing signature in notmuch-show view"
 maildir_storage_thread=$(notmuch search --output=threads id:20091117190054.GU3165@dottiness.seas.harvard.edu)
@@ -279,7 +284,7 @@ test_emacs "(notmuch-show \"$maildir_storage_thread\")
 	    (search-backward \"Click/Enter to hide.\")
 	    (button-activate (button-at (point)))
 	    (test-output)"
-test_expect_equal_file OUTPUT $EXPECTED/notmuch-show-thread-maildir-storage
+test_expect_equal_file EMACS OUTPUT $EXPECTED/notmuch-show-thread-maildir-storage
 
 test_begin_subtest "Detection and hiding of top-post quoting of message"
 add_message '[subject]="The problem with top-posting"' \
@@ -326,13 +331,13 @@ Thanks for the advice! I will be sure to put it to good use.
 -Top Poster
 
 [ 9-line hidden original message. Click/Enter to show. ]" > EXPECTED
-test_expect_equal_file OUTPUT EXPECTED
+test_expect_equal_file EMACS OUTPUT EXPECTED
 
 test_begin_subtest "Hiding message in notmuch-show view"
 test_emacs '(notmuch-show "id:f35dbb950911171438k5df6eb56k77b6c0944e2e79ae@mail.gmail.com")
 	    (notmuch-show-toggle-message)
 	    (test-visible-output)'
-test_expect_equal_file OUTPUT $EXPECTED/notmuch-show-thread-with-hidden-messages
+test_expect_equal_file EMACS OUTPUT $EXPECTED/notmuch-show-thread-with-hidden-messages
 
 test_begin_subtest "Hiding message with visible citation in notmuch-show view"
 test_emacs '(notmuch-show "id:f35dbb950911171438k5df6eb56k77b6c0944e2e79ae@mail.gmail.com")
@@ -340,7 +345,7 @@ test_emacs '(notmuch-show "id:f35dbb950911171438k5df6eb56k77b6c0944e2e79ae@mail.
 	    (button-activate (button-at (point)))
 	    (notmuch-show-toggle-message)
 	    (test-visible-output)'
-test_expect_equal_file OUTPUT $EXPECTED/notmuch-show-thread-with-hidden-messages
+test_expect_equal_file EMACS OUTPUT $EXPECTED/notmuch-show-thread-with-hidden-messages
 
 test_begin_subtest 'Hiding message following HTML part'
 test_subtest_known_broken
@@ -366,14 +371,14 @@ test_emacs "(notmuch-show \"id:$id\") \
             (notmuch-show-next-message) \
             (notmuch-show-toggle-message) \
             (test-visible-output \"EXPECTED\")"
-test_expect_equal_file OUTPUT EXPECTED
+test_expect_equal_file EMACS OUTPUT EXPECTED
 
 test_begin_subtest "Refresh show buffer"
 test_emacs '(notmuch-show "id:f35dbb950911171438k5df6eb56k77b6c0944e2e79ae@mail.gmail.com")
 	    (test-visible-output "EXPECTED")
 	    (notmuch-show-refresh-view)
 	    (test-visible-output)'
-test_expect_equal_file OUTPUT EXPECTED
+test_expect_equal_file EMACS OUTPUT EXPECTED
 
 test_begin_subtest "Refresh modified show buffer"
 test_subtest_known_broken
@@ -384,6 +389,6 @@ test_emacs '(notmuch-show "id:f35dbb950911171438k5df6eb56k77b6c0944e2e79ae@mail.
 	    (test-visible-output "EXPECTED")
 	    (notmuch-show-refresh-view)
 	    (test-visible-output)'
-test_expect_equal_file OUTPUT EXPECTED
+test_expect_equal_file EMACS OUTPUT EXPECTED
 
 test_done
-- 
1.7.7.1

[PATCH 4/4] test: add 'Emacs' prereq to dependent 'emacs-large-search-buffer' tests

[Pieter Praet]

Adds a new test that checks for the presence of 'emacs',
and adds that test as a prereq to all subsequent tests
that rely on Emacs.

This causes tests with unmet dependencies to be skipped.

---
 test/emacs-large-search-buffer |    9 +++++++--
 1 files changed, 7 insertions(+), 2 deletions(-)

diff --git a/test/emacs-large-search-buffer b/test/emacs-large-search-buffer
index 6095e9d..5757380 100755
--- a/test/emacs-large-search-buffer
+++ b/test/emacs-large-search-buffer
@@ -2,6 +2,11 @@
 test_description="Emacs with large search results buffer"
 . test-lib.sh
 
+# Emacs is a prereq.
+test_expect_success "prereq: Emacs is present" "which emacs" \
+    && test_set_prereq EMACS
+
+
 x=xxxxxxxxxx # 10
 x=$x$x$x$x$x$x$x$x$x$x # 100
 x=$x$x$x$x$x$x$x$x$x # 900
@@ -27,7 +32,7 @@ test_emacs '(notmuch-search "*")
 	    (notmuch-test-wait)
 	    (test-output)'
 sed -i -e s',  *, ,g' -e 's/xxx*/[BLOB]/g' OUTPUT
-test_expect_equal_file OUTPUT EXPEXTED
+test_expect_equal_file EMACS OUTPUT EXPEXTED
 
 test_begin_subtest "Ensure that emacs doesn't drop error messages"
 test_emacs '(notmuch-search "--this-option-does-not-exist")
@@ -38,6 +43,6 @@ Error: Unexpected output from notmuch search:
 Unrecognized option: --this-option-does-not-exist
 End of search results. (process returned 1)
 EOF
-test_expect_equal_file OUTPUT EXPEXTED
+test_expect_equal_file EMACS OUTPUT EXPEXTED
 
 test_done
-- 
1.7.7.1

Re: set test prereqs (Emacs, GDB, GPG) v4

[Ali Polatel]

[-- Attachment #1: Type: text/plain, Size: 525 bytes --]

On Tue, Nov 01, 2011 at 08:49:10PM +0100, Pieter Praet wrote:
>Rebased to current master.
>
>Previous version:
>  id:"1307016220-17509-1-git-send-email-pieter@praet.org"
>
>Discussion:
>  id:"1317660447-27520-1-git-send-email-schnouki@schnouki.net"
>

Thanks for the nice work!
I want to share one thing which came up to my mind during reading
through them. Since we require bash for tests, we might go for using
bash's builtin 'type -P foo' instead of 'which foo'. Not that it's
important, just wondered "why not?"

		-alip

[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]

Re: [PATCH 1/4] test: add 'GnuPG' prereq to dependent 'crypto' tests

[Jameson Graef Rollins]

[-- Attachment #1: Type: text/plain, Size: 625 bytes --]

On Tue,  1 Nov 2011 20:49:11 +0100, Pieter Praet <pieter@praet.org> wrote:
> -test_expect_success 'emacs delivery of signed message' \
> +test_expect_success GPG 'emacs delivery of signed message' \

Hi, Pieter and Thomas.  Thanks for all the work on this, but I have one
issue.  Is there a way we can do this without adding a new argument to
every test function?  For some reason I really don't like that solution.
It seems too invasive.  Can't we have something that works more like
test_subtest_known_broken, that modifies the test environment, rather
than add an argument to every call of every testing function?

jamie.

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

Re: [PATCH 1/4] test: add 'GnuPG' prereq to dependent 'crypto' tests

[Pieter Praet]

On Tue, 01 Nov 2011 14:17:51 -0700, Jameson Graef Rollins <jrollins@finestructure.net> wrote:
> On Tue,  1 Nov 2011 20:49:11 +0100, Pieter Praet <pieter@praet.org> wrote:
> > -test_expect_success 'emacs delivery of signed message' \
> > +test_expect_success GPG 'emacs delivery of signed message' \
> 
> Hi, Pieter and Thomas.  Thanks for all the work on this, but I have one
> issue.  Is there a way we can do this without adding a new argument to
> every test function?  For some reason I really don't like that solution.
> It seems too invasive.  Can't we have something that works more like
> test_subtest_known_broken, that modifies the test environment, rather
> than add an argument to every call of every testing function?
> 
> jamie.

I've been thinking the very same thing.


We could use `test_have_prereq' to get rid of the argument, e.g.:

  #+begin_src sh
    test_have_prereq EMACS && \
    test_begin_subtest "blah"
    echo "doing stuff..."
    test_expect_equal_file OUTPUT EXPECTED
  #+end_src

...but still, everything between `test_begin_subtest' and `test_expect_*'
would be executed, so to err on the safe side, we could wrap the full body
of every test, e.g.:

  #+begin_src sh
    test_begin_subtest "blah"
    test_have_prereq EMACS && {
    ...
    echo "doing stuff..."
    ...
    }
    test_expect_equal_file OUTPUT EXPECTED
  #+end_src


Or... (I've given this zero thought, so please take it with a bag of
salt) we could run all tests from inside a "controller" which `eval's
their contents line by line, and skips a test entirely whenever $? > 0.


All of this is (still) excessively invasive in some way or another though...

Suggestions?


Peace


-- 
Pieter

Re: set test prereqs (Emacs, GDB, GPG) v4

[Pieter Praet]

On Tue, 1 Nov 2011 22:20:25 +0200, Ali Polatel <polatel@gmail.com> wrote:
> On Tue, Nov 01, 2011 at 08:49:10PM +0100, Pieter Praet wrote:
> >Rebased to current master.
> >
> >Previous version:
> >  id:"1307016220-17509-1-git-send-email-pieter@praet.org"
> >
> >Discussion:
> >  id:"1317660447-27520-1-git-send-email-schnouki@schnouki.net"
> >
> 
> Thanks for the nice work!
> I want to share one thing which came up to my mind during reading
> through them. Since we require bash for tests, we might go for using
> bash's builtin 'type -P foo' instead of 'which foo'. Not that it's
> important, just wondered "why not?"
> 
> 		-alip

Good suggestion!

In fact, we might as well immediately go for full POSIX compliance.

How do you feel about using `hash' ?  Besides being available in POSIX
sh, it has the added benefit of making subsequent lookups faster [*].


Peace

-- 
Pieter

[*] This statement only applies to the shell built-in.  Other types of
    hash may distort your perception of time, which is likely to result
    in significantly slower lookups.

[PATCH 0/6] Rebase of Pieter's "set test prereqs"

[Thomas Jost]

Hello list,

This is another rebased version of Pieter's series to add GPG and Emacs as test
prereqs, plus some additions on my own. (Rebased and posted as requested by
Pieter [1].)

Changes as compared to Pieter's patches (including parts from [2]):
- prereqs are not tested using test_expect_success as they were in Pieter's
  original patches, but using a new function called test_set_bin_prereq. I wrote
  this before the gdb prereq was added, hence the different way to set it.

- some fixes in Pieter's patches so that it actually works when gpg is not
  installed. Can't exactly remember what (...but you can just check his original
  patches), but in the end it was working fine in a chroot without gpg.

- since Emacs is now run using dtach, the emacs prereq also depends on dtach.
  The presence of emacs and dtach is also checked in the test_emacs() function
  of the test suite.

- testing for prereqs is now done using the "hash" built-in instead of "which",
  as suggested in [3].

Tested with and without dtach. A previous version of this series was also
without emacs/gpg in a chroot, but not this one :)

[1] id:"874ny4fcdz.fsf@praet.org"
[2] id:"1317660447-27520-1-git-send-email-schnouki@schnouki.net"
[3] id:"87zkgemodd.fsf@praet.org"

Pieter Praet (4):
  test: add 'GnuPG' prereq to dependent 'crypto' tests
  test: add 'Emacs' prereq to dependent 'crypto' tests
  test: add 'Emacs' prereq to dependent 'emacs' tests
  test: add 'Emacs' prereq to dependent 'emacs-large-search-buffer'
    tests

Thomas Jost (2):
  test: define a helper function for defining prereqs on executables
  test: check if emacs and dtach are available in test_emacs()

 test/crypto                    |   46 +++++++++++++++-------
 test/emacs                     |   82 +++++++++++++++++++++------------------
 test/emacs-large-search-buffer |    9 +++-
 test/test-lib.sh               |   17 ++++++++
 4 files changed, 99 insertions(+), 55 deletions(-)

-- 
1.7.7.3

[PATCH 1/6] test: define a helper function for defining prereqs on executables

[Thomas Jost]

While test_expect_success could be used to define these prereqs, this is
probably not a good idea: if a prereq is not available, using
test_expect_success would result in a test being reported as FAILED at the end
of the test suite (and its dependencies as skipped).

(Thanks to Pieter Praet for suggesting the use of "hash" instead of "which".)
---
 test/test-lib.sh |   13 +++++++++++++
 1 files changed, 13 insertions(+), 0 deletions(-)

diff --git a/test/test-lib.sh b/test/test-lib.sh
index 1ea7fa9..382934f 100755
--- a/test/test-lib.sh
+++ b/test/test-lib.sh
@@ -543,6 +543,19 @@ test_have_prereq () {
 	esac
 }
 
+test_set_bin_prereq () {
+	bin=$1
+	name=$2
+	prereq=$3
+	if hash $bin &>/dev/null
+	then
+		test_set_prereq $prereq
+	else
+		say_color info "%-6s" "INFO"
+		echo " Missing test prerequisite: $name"
+	fi
+}
+
 # You are not expected to call test_ok_ and test_failure_ directly, use
 # the text_expect_* functions instead.
 
-- 
1.7.7.3

[PATCH 2/6] test: check if emacs and dtach are available in test_emacs()

[Thomas Jost]

The test_emacs() function is run by the emacs tests even if the EMACS prereq is
not available. This causes an issue when running the test suite on a system
where dtach is not installed: the emacs server will not start, but since the
$EMACS_SERVER variable is set on the first call to test_emacs, subsequent calls
may result in attempts to run emacsclient, which will then report "can't find
socket".

This patch fixes this issue by making sure that dtach and emacs are available
before doing anything else in test_emacs().
---
 test/test-lib.sh |    4 ++++
 1 files changed, 4 insertions(+), 0 deletions(-)

diff --git a/test/test-lib.sh b/test/test-lib.sh
index 382934f..ca1f412 100755
--- a/test/test-lib.sh
+++ b/test/test-lib.sh
@@ -852,6 +852,10 @@ EOF
 }
 
 test_emacs () {
+	# First check that emacs and dtach are available
+	hash emacs &>/dev/null || return
+	hash dtach &>/dev/null || return
+
 	if [ -z "$EMACS_SERVER" ]; then
 		EMACS_SERVER="notmuch-test-suite-$$"
 		# start a detached session with an emacs server
-- 
1.7.7.3

[PATCH 3/6] test: add 'GnuPG' prereq to dependent 'crypto' tests

[Thomas Jost]

From: Pieter Praet <pieter@praet.org>

Adds a new test that checks for the presence of 'gpg',
and adds that test as a prereq to all subsequent tests
that rely on GnuPG.

This causes tests with unmet dependencies to be skipped.

Signed-off-by: Pieter Praet <pieter@praet.org>
---
 test/crypto |   35 ++++++++++++++++++++---------------
 1 files changed, 20 insertions(+), 15 deletions(-)

diff --git a/test/crypto b/test/crypto
index 0af4aa8..eca59b6 100755
--- a/test/crypto
+++ b/test/crypto
@@ -7,11 +7,16 @@
 test_description='PGP/MIME signature verification and decryption'
 . ./test-lib.sh
 
+# GnuPG is a prereq.
+test_set_bin_prereq gpg "GnuPG" GPG
+
+
 add_gnupg_home ()
 {
     local output
     [ -d ${GNUPGHOME} ] && return
     mkdir -m 0700 "$GNUPGHOME"
+    test_have_prereq GPG || return
     gpg --no-tty --import <$TEST_DIRECTORY/gnupg-secret-key.asc >"$GNUPGHOME"/import.log 2>&1
     test_debug "cat $GNUPGHOME/import.log"
     if (gpg --quick-random --version >/dev/null 2>&1) ; then
@@ -25,13 +30,13 @@ add_gnupg_home ()
 
 add_gnupg_home
 # get key fingerprint
-FINGERPRINT=$(gpg --no-tty --list-secret-keys --with-colons --fingerprint | grep '^fpr:' | cut -d: -f10)
+test_have_prereq GPG && FINGERPRINT=$(gpg --no-tty --list-secret-keys --with-colons --fingerprint | grep '^fpr:' | cut -d: -f10)
 
 # for some reason this is needed for emacs_deliver_message to work,
 # although I can't figure out why
 add_email_corpus
 
-test_expect_success 'emacs delivery of signed message' \
+test_expect_success GPG 'emacs delivery of signed message' \
 'emacs_deliver_message \
     "test signed message 001" \
     "This is a test signed message." \
@@ -64,7 +69,7 @@ expected='[[[{"id": "XXXXX",
  {"id": 3,
  "content-type": "application/pgp-signature"}]}]},
  []]]]'
-test_expect_equal \
+test_expect_equal GPG \
     "$output" \
     "$expected"
 
@@ -99,7 +104,7 @@ expected='[[[{"id": "XXXXX",
  {"id": 3,
  "content-type": "application/pgp-signature"}]}]},
  []]]]'
-test_expect_equal \
+test_expect_equal GPG \
     "$output" \
     "$expected"
 
@@ -132,7 +137,7 @@ expected='[[[{"id": "XXXXX",
  {"id": 3,
  "content-type": "application/pgp-signature"}]}]},
  []]]]'
-test_expect_equal \
+test_expect_equal GPG \
     "$output" \
     "$expected"
 mv "${GNUPGHOME}"{.bak,}
@@ -141,7 +146,7 @@ mv "${GNUPGHOME}"{.bak,}
 cat <<EOF >TESTATTACHMENT
 This is a test file.
 EOF
-test_expect_success 'emacs delivery of encrypted message with attachment' \
+test_expect_success GPG 'emacs delivery of encrypted message with attachment' \
 'emacs_deliver_message \
     "test encrypted message 001" \
     "This is a test encrypted message.\n" \
@@ -175,7 +180,7 @@ Non-text part: application/octet-stream
 \fpart}
 \fbody}
 \fmessage}'
-test_expect_equal \
+test_expect_equal GPG \
     "$output" \
     "$expected"
 
@@ -210,7 +215,7 @@ expected='[[[{"id": "XXXXX",
  "content-type": "application/octet-stream",
  "filename": "TESTATTACHMENT"}]}]}]},
  []]]]'
-test_expect_equal \
+test_expect_equal GPG \
     "$output" \
     "$expected"
 
@@ -221,7 +226,7 @@ output=$(notmuch show --format=json --part=4 --decrypt subject:"test encrypted m
 expected='{"id": 4,
  "content-type": "text/plain",
  "content": "This is a test encrypted message.\n"}'
-test_expect_equal \
+test_expect_equal GPG \
     "$output" \
     "$expected"
 
@@ -231,7 +236,7 @@ notmuch show \
     --part=5 \
     --decrypt \
     subject:"test encrypted message 001" >OUTPUT
-test_expect_equal_file OUTPUT TESTATTACHMENT
+test_expect_equal_file GPG OUTPUT TESTATTACHMENT
 
 test_begin_subtest "decryption failure with missing key"
 mv "${GNUPGHOME}"{,.bak}
@@ -258,12 +263,12 @@ expected='[[[{"id": "XXXXX",
  {"id": 3,
  "content-type": "application/octet-stream"}]}]},
  []]]]'
-test_expect_equal \
+test_expect_equal GPG \
     "$output" \
     "$expected"
 mv "${GNUPGHOME}"{.bak,}
 
-test_expect_success 'emacs delivery of encrypted + signed message' \
+test_expect_success GPG 'emacs delivery of encrypted + signed message' \
 'emacs_deliver_message \
     "test encrypted message 002" \
     "This is another test encrypted message.\n" \
@@ -298,7 +303,7 @@ expected='[[[{"id": "XXXXX",
  "content-type": "text/plain",
  "content": "This is another test encrypted message.\n"}]}]},
  []]]]'
-test_expect_equal \
+test_expect_equal GPG \
     "$output" \
     "$expected"
 
@@ -310,7 +315,7 @@ Subject: Re: test encrypted message 002
 
 On 01 Jan 2000 12:00:00 -0000, Notmuch Test Suite <test_suite@notmuchmail.org> wrote:
 > This is another test encrypted message.'
-test_expect_equal \
+test_expect_equal GPG \
     "$output" \
     "$expected"
 
@@ -351,7 +356,7 @@ expected='[[[{"id": "XXXXX",
  {"id": 3,
  "content-type": "application/pgp-signature"}]}]},
  []]]]'
-test_expect_equal \
+test_expect_equal GPG \
     "$output" \
     "$expected"
 
-- 
1.7.7.3

[PATCH 4/6] test: add 'Emacs' prereq to dependent 'crypto' tests

[Thomas Jost]

From: Pieter Praet <pieter@praet.org>

Adds a new test that checks for the presence of 'emacs',
and adds that test as a prereq to all subsequent tests
that rely on Emacs.

This causes tests with unmet dependencies to be skipped.

Right now, all crypto tests do depend on Emacs, because it
is used to generate the signed/encrypted messages that are
needed by the tests.

Signed-off-by: Pieter Praet <pieter@praet.org>
---
 test/crypto |   39 +++++++++++++++++++++++++--------------
 1 files changed, 25 insertions(+), 14 deletions(-)

diff --git a/test/crypto b/test/crypto
index eca59b6..085070b 100755
--- a/test/crypto
+++ b/test/crypto
@@ -7,9 +7,20 @@
 test_description='PGP/MIME signature verification and decryption'
 . ./test-lib.sh
 
+# Emacs is a prereq.
+test_set_bin_prereq dtach "dtach" DTACH
+test_have_prereq DTACH && test_set_bin_prereq emacs "Emacs" EMACS
+
 # GnuPG is a prereq.
 test_set_bin_prereq gpg "GnuPG" GPG
 
+# Some tests have multiple prereqs, but the test_expect_* functions
+# accept only a single argument as prereq tag, and using test_have_prereq
+# in and around tests causes various errors for me, so a dirty workaround
+# will have to do for the time being.
+test_have_prereq EMACS && test_have_prereq GPG \
+    && test_set_prereq EMACS+GPG
+
 
 add_gnupg_home ()
 {
@@ -36,7 +47,7 @@ test_have_prereq GPG && FINGERPRINT=$(gpg --no-tty --list-secret-keys --with-col
 # although I can't figure out why
 add_email_corpus
 
-test_expect_success GPG 'emacs delivery of signed message' \
+test_expect_success EMACS+GPG 'emacs delivery of signed message' \
 'emacs_deliver_message \
     "test signed message 001" \
     "This is a test signed message." \
@@ -69,7 +80,7 @@ expected='[[[{"id": "XXXXX",
  {"id": 3,
  "content-type": "application/pgp-signature"}]}]},
  []]]]'
-test_expect_equal GPG \
+test_expect_equal EMACS+GPG \
     "$output" \
     "$expected"
 
@@ -104,7 +115,7 @@ expected='[[[{"id": "XXXXX",
  {"id": 3,
  "content-type": "application/pgp-signature"}]}]},
  []]]]'
-test_expect_equal GPG \
+test_expect_equal EMACS+GPG \
     "$output" \
     "$expected"
 
@@ -137,7 +148,7 @@ expected='[[[{"id": "XXXXX",
  {"id": 3,
  "content-type": "application/pgp-signature"}]}]},
  []]]]'
-test_expect_equal GPG \
+test_expect_equal EMACS+GPG \
     "$output" \
     "$expected"
 mv "${GNUPGHOME}"{.bak,}
@@ -146,7 +157,7 @@ mv "${GNUPGHOME}"{.bak,}
 cat <<EOF >TESTATTACHMENT
 This is a test file.
 EOF
-test_expect_success GPG 'emacs delivery of encrypted message with attachment' \
+test_expect_success EMACS+GPG 'emacs delivery of encrypted message with attachment' \
 'emacs_deliver_message \
     "test encrypted message 001" \
     "This is a test encrypted message.\n" \
@@ -180,7 +191,7 @@ Non-text part: application/octet-stream
 \fpart}
 \fbody}
 \fmessage}'
-test_expect_equal GPG \
+test_expect_equal EMACS+GPG \
     "$output" \
     "$expected"
 
@@ -215,7 +226,7 @@ expected='[[[{"id": "XXXXX",
  "content-type": "application/octet-stream",
  "filename": "TESTATTACHMENT"}]}]}]},
  []]]]'
-test_expect_equal GPG \
+test_expect_equal EMACS+GPG \
     "$output" \
     "$expected"
 
@@ -226,7 +237,7 @@ output=$(notmuch show --format=json --part=4 --decrypt subject:"test encrypted m
 expected='{"id": 4,
  "content-type": "text/plain",
  "content": "This is a test encrypted message.\n"}'
-test_expect_equal GPG \
+test_expect_equal EMACS+GPG \
     "$output" \
     "$expected"
 
@@ -236,7 +247,7 @@ notmuch show \
     --part=5 \
     --decrypt \
     subject:"test encrypted message 001" >OUTPUT
-test_expect_equal_file GPG OUTPUT TESTATTACHMENT
+test_expect_equal_file EMACS+GPG OUTPUT TESTATTACHMENT
 
 test_begin_subtest "decryption failure with missing key"
 mv "${GNUPGHOME}"{,.bak}
@@ -263,12 +274,12 @@ expected='[[[{"id": "XXXXX",
  {"id": 3,
  "content-type": "application/octet-stream"}]}]},
  []]]]'
-test_expect_equal GPG \
+test_expect_equal EMACS+GPG \
     "$output" \
     "$expected"
 mv "${GNUPGHOME}"{.bak,}
 
-test_expect_success GPG 'emacs delivery of encrypted + signed message' \
+test_expect_success EMACS+GPG 'emacs delivery of encrypted + signed message' \
 'emacs_deliver_message \
     "test encrypted message 002" \
     "This is another test encrypted message.\n" \
@@ -303,7 +314,7 @@ expected='[[[{"id": "XXXXX",
  "content-type": "text/plain",
  "content": "This is another test encrypted message.\n"}]}]},
  []]]]'
-test_expect_equal GPG \
+test_expect_equal EMACS+GPG \
     "$output" \
     "$expected"
 
@@ -315,7 +326,7 @@ Subject: Re: test encrypted message 002
 
 On 01 Jan 2000 12:00:00 -0000, Notmuch Test Suite <test_suite@notmuchmail.org> wrote:
 > This is another test encrypted message.'
-test_expect_equal GPG \
+test_expect_equal EMACS+GPG \
     "$output" \
     "$expected"
 
@@ -356,7 +367,7 @@ expected='[[[{"id": "XXXXX",
  {"id": 3,
  "content-type": "application/pgp-signature"}]}]},
  []]]]'
-test_expect_equal GPG \
+test_expect_equal EMACS+GPG \
     "$output" \
     "$expected"
 
-- 
1.7.7.3

[PATCH 5/6] test: add 'Emacs' prereq to dependent 'emacs' tests

[Thomas Jost]

From: Pieter Praet <pieter@praet.org>

Adds a new test that checks for the presence of 'emacs',
and adds that test as a prereq to all subsequent tests
that rely on Emacs.

This causes tests with unmet dependencies to be skipped.

Signed-off-by: Pieter Praet <pieter@praet.org>

Conflicts:

	test/emacs
---
 test/emacs |   82 ++++++++++++++++++++++++++++++++---------------------------
 1 files changed, 44 insertions(+), 38 deletions(-)

diff --git a/test/emacs b/test/emacs
index 75a0a74..ea641af 100755
--- a/test/emacs
+++ b/test/emacs
@@ -3,6 +3,11 @@
 test_description="emacs interface"
 . test-lib.sh
 
+# Emacs is a prereq.
+test_set_bin_prereq dtach "dtach" DTACH
+test_have_prereq DTACH && test_set_bin_prereq emacs "Emacs" EMACS
+
+
 EXPECTED=$TEST_DIRECTORY/emacs.expected-output
 
 add_email_corpus
@@ -10,7 +15,7 @@ add_email_corpus
 test_begin_subtest "Basic notmuch-hello view in emacs"
 test_emacs '(notmuch-hello)
 	    (test-output)'
-test_expect_equal_file OUTPUT $EXPECTED/notmuch-hello
+test_expect_equal_file EMACS OUTPUT $EXPECTED/notmuch-hello
 
 test_begin_subtest "Saved search with 0 results"
 test_emacs '(let ((notmuch-show-empty-saved-searches t)
@@ -20,20 +25,20 @@ test_emacs '(let ((notmuch-show-empty-saved-searches t)
 			("empty" . "tag:doesnotexist"))))
 	      (notmuch-hello)
 	      (test-output))'
-test_expect_equal_file OUTPUT $EXPECTED/notmuch-hello-with-empty
+test_expect_equal_file EMACS OUTPUT $EXPECTED/notmuch-hello-with-empty
 
 test_begin_subtest "No saved searches displayed (all with 0 results)"
 test_emacs '(let ((notmuch-saved-searches
 		   '\''(("empty" . "tag:doesnotexist"))))
 	      (notmuch-hello)
 	      (test-output))'
-test_expect_equal_file OUTPUT $EXPECTED/notmuch-hello-no-saved-searches
+test_expect_equal_file EMACS OUTPUT $EXPECTED/notmuch-hello-no-saved-searches
 
 test_begin_subtest "Basic notmuch-search view in emacs"
 test_emacs '(notmuch-search "tag:inbox")
 	    (notmuch-test-wait)
 	    (test-output)'
-test_expect_equal_file OUTPUT $EXPECTED/notmuch-search-tag-inbox
+test_expect_equal_file EMACS OUTPUT $EXPECTED/notmuch-search-tag-inbox
 
 test_begin_subtest "Navigation of notmuch-hello to search results"
 test_emacs '(notmuch-hello)
@@ -42,13 +47,13 @@ test_emacs '(notmuch-hello)
 	    (widget-button-press (point))
 	    (notmuch-test-wait)
 	    (test-output)'
-test_expect_equal_file OUTPUT $EXPECTED/notmuch-hello-view-inbox
+test_expect_equal_file EMACS OUTPUT $EXPECTED/notmuch-hello-view-inbox
 
 test_begin_subtest "Basic notmuch-show view in emacs"
 maildir_storage_thread=$(notmuch search --output=threads id:20091117190054.GU3165@dottiness.seas.harvard.edu)
 test_emacs "(notmuch-show \"$maildir_storage_thread\")
 	    (test-output)"
-test_expect_equal_file OUTPUT $EXPECTED/notmuch-show-thread-maildir-storage
+test_expect_equal_file EMACS OUTPUT $EXPECTED/notmuch-show-thread-maildir-storage
 
 test_begin_subtest "notmuch-show for message with invalid From"
 add_message "[subject]=\"message-with-invalid-from\"" \
@@ -64,7 +69,7 @@ Date: Tue, 05 Jan 2001 15:43:57 -0000
 
 This is just a test message (#1)
 EOF
-test_expect_equal_file OUTPUT EXPECTED
+test_expect_equal_file EMACS OUTPUT EXPECTED
 
 test_begin_subtest "Navigation of notmuch-search to thread view"
 test_emacs '(notmuch-search "tag:inbox")
@@ -74,7 +79,7 @@ test_emacs '(notmuch-search "tag:inbox")
 	    (notmuch-search-show-thread)
 	    (notmuch-test-wait)
 	    (test-output)'
-test_expect_equal_file OUTPUT $EXPECTED/notmuch-show-thread-maildir-storage
+test_expect_equal_file EMACS OUTPUT $EXPECTED/notmuch-show-thread-maildir-storage
 
 test_begin_subtest "Add tag from search view"
 os_x_darwin_thread=$(notmuch search --output=threads id:ddd65cda0911171950o4eea4389v86de9525e46052d3@mail.gmail.com)
@@ -82,26 +87,26 @@ test_emacs "(notmuch-search \"$os_x_darwin_thread\")
 	    (notmuch-test-wait)
 	    (notmuch-search-add-tag \"tag-from-search-view\")"
 output=$(notmuch search $os_x_darwin_thread | notmuch_search_sanitize)
-test_expect_equal "$output" "thread:XXX   2009-11-18 [4/4] Jjgod Jiang, Alexander Botero-Lowry; [notmuch] Mac OS X/Darwin compatibility issues (inbox tag-from-search-view unread)"
+test_expect_equal EMACS "$output" "thread:XXX   2009-11-18 [4/4] Jjgod Jiang, Alexander Botero-Lowry; [notmuch] Mac OS X/Darwin compatibility issues (inbox tag-from-search-view unread)"
 
 test_begin_subtest "Remove tag from search view"
 test_emacs "(notmuch-search \"$os_x_darwin_thread\")
 	    (notmuch-test-wait)
 	    (notmuch-search-remove-tag \"tag-from-search-view\")"
 output=$(notmuch search $os_x_darwin_thread | notmuch_search_sanitize)
-test_expect_equal "$output" "thread:XXX   2009-11-18 [4/4] Jjgod Jiang, Alexander Botero-Lowry; [notmuch] Mac OS X/Darwin compatibility issues (inbox unread)"
+test_expect_equal EMACS "$output" "thread:XXX   2009-11-18 [4/4] Jjgod Jiang, Alexander Botero-Lowry; [notmuch] Mac OS X/Darwin compatibility issues (inbox unread)"
 
 test_begin_subtest "Add tag from notmuch-show view"
 test_emacs "(notmuch-show \"$os_x_darwin_thread\")
 	    (notmuch-show-add-tag \"tag-from-show-view\")"
 output=$(notmuch search $os_x_darwin_thread | notmuch_search_sanitize)
-test_expect_equal "$output" "thread:XXX   2009-11-18 [4/4] Jjgod Jiang, Alexander Botero-Lowry; [notmuch] Mac OS X/Darwin compatibility issues (inbox tag-from-show-view unread)"
+test_expect_equal EMACS "$output" "thread:XXX   2009-11-18 [4/4] Jjgod Jiang, Alexander Botero-Lowry; [notmuch] Mac OS X/Darwin compatibility issues (inbox tag-from-show-view unread)"
 
 test_begin_subtest "Remove tag from notmuch-show view"
 test_emacs "(notmuch-show \"$os_x_darwin_thread\")
 	    (notmuch-show-remove-tag \"tag-from-show-view\")"
 output=$(notmuch search $os_x_darwin_thread | notmuch_search_sanitize)
-test_expect_equal "$output" "thread:XXX   2009-11-18 [4/4] Jjgod Jiang, Alexander Botero-Lowry; [notmuch] Mac OS X/Darwin compatibility issues (inbox unread)"
+test_expect_equal EMACS "$output" "thread:XXX   2009-11-18 [4/4] Jjgod Jiang, Alexander Botero-Lowry; [notmuch] Mac OS X/Darwin compatibility issues (inbox unread)"
 
 test_begin_subtest "Message with .. in Message-Id:"
 add_message [id]=123..456@example '[subject]="Message with .. in Message-Id"'
@@ -116,10 +121,10 @@ test_emacs '(notmuch-search "id:\"123..456@example\"")
 	    (notmuch-show-add-tag "show-remove")
 	    (notmuch-show-remove-tag "show-remove")'
 output=$(notmuch search 'id:"123..456@example"' | notmuch_search_sanitize)
-test_expect_equal "$output" "thread:XXX   2001-01-05 [1/1] Notmuch Test Suite; Message with .. in Message-Id (inbox search-add show-add)"
+test_expect_equal EMACS "$output" "thread:XXX   2001-01-05 [1/1] Notmuch Test Suite; Message with .. in Message-Id (inbox search-add show-add)"
 
 test_begin_subtest "Sending a message via (fake) SMTP"
-emacs_deliver_message \
+test_have_prereq EMACS && emacs_deliver_message \
     'Testing message sent via SMTP' \
     'This is a test that messages are sent via SMTP' \
     '(message-goto-to)
@@ -140,12 +145,12 @@ Content-Type: text/plain; charset=us-ascii
 
 This is a test that messages are sent via SMTP
 EOF
-test_expect_equal_file OUTPUT EXPECTED
+test_expect_equal_file EMACS OUTPUT EXPECTED
 
 test_begin_subtest "Verify that sent messages are saved/searchable (via FCC)"
 notmuch new > /dev/null
 output=$(notmuch search 'subject:"testing message sent via SMTP"' | notmuch_search_sanitize)
-test_expect_equal "$output" "thread:XXX   2000-01-01 [1/1] Notmuch Test Suite; Testing message sent via SMTP (inbox)"
+test_expect_equal EMACS "$output" "thread:XXX   2000-01-01 [1/1] Notmuch Test Suite; Testing message sent via SMTP (inbox)"
 
 test_begin_subtest "notmuch-fcc-dirs set to nil"
 test_emacs "(let ((notmuch-fcc-dirs nil))
@@ -157,7 +162,7 @@ To:
 Subject: 
 --text follows this line--
 EOF
-test_expect_equal_file OUTPUT EXPECTED
+test_expect_equal_file EMACS OUTPUT EXPECTED
 
 # Make another FCC maildir specific for the next test
 mkdir -p mail/sent-string/cur
@@ -175,7 +180,7 @@ Subject:
 Fcc: ${MAIL_DIR}/sent-string
 --text follows this line--
 EOF
-test_expect_equal_file OUTPUT EXPECTED
+test_expect_equal_file EMACS OUTPUT EXPECTED
 
 # Make more FCC maildirs specific for the next test
 mkdir -p mail/sent-list-match/cur
@@ -198,7 +203,7 @@ Subject:
 Fcc: ${MAIL_DIR}/sent-list-match
 --text follows this line--
 EOF
-test_expect_equal_file OUTPUT EXPECTED
+test_expect_equal_file EMACS OUTPUT EXPECTED
 
 # Make another FCC maildir specific for the next test
 mkdir -p mail/sent-list-catch-all/cur
@@ -218,7 +223,7 @@ Subject:
 Fcc: ${MAIL_DIR}/sent-list-catch-all
 --text follows this line--
 EOF
-test_expect_equal_file OUTPUT EXPECTED
+test_expect_equal_file EMACS OUTPUT EXPECTED
 
 test_begin_subtest "notmuch-fcc-dirs set to a list (no match)"
 test_emacs "(let ((notmuch-fcc-dirs
@@ -232,14 +237,14 @@ To:
 Subject: 
 --text follows this line--
 EOF
-test_expect_equal_file OUTPUT EXPECTED
+test_expect_equal_file EMACS OUTPUT EXPECTED
 
 test_begin_subtest "Reply within emacs"
 test_emacs '(notmuch-search "subject:\"testing message sent via SMTP\"")
 	    (notmuch-test-wait)
 	    (notmuch-search-reply-to-thread)
 	    (test-output)'
-sed -i -e 's/^In-Reply-To: <.*>$/In-Reply-To: <XXX>/' OUTPUT
+sed -i -e 's/^In-Reply-To: <.*>$/In-Reply-To: <XXX>/' EMACS OUTPUT
 cat <<EOF >EXPECTED
 From: Notmuch Test Suite <test_suite@notmuchmail.org>
 To: user@example.com
@@ -250,26 +255,26 @@ Fcc: $(pwd)/mail/sent
 On 01 Jan 2000 12:00:00 -0000, Notmuch Test Suite <test_suite@notmuchmail.org> wrote:
 > This is a test that messages are sent via SMTP
 EOF
-test_expect_equal_file OUTPUT EXPECTED
+test_expect_equal_file EMACS OUTPUT EXPECTED
 
 test_begin_subtest "Save attachment from within emacs using notmuch-show-save-attachments"
 # save as archive to test that Emacs does not re-compress .gz
 test_emacs '(let ((standard-input "\"attachment1.gz\""))
 	      (notmuch-show "id:cf0c4d610911171136h1713aa59w9cf9aa31f052ad0a@mail.gmail.com")
 	      (notmuch-show-save-attachments))'
-test_expect_equal_file attachment1.gz "$EXPECTED/attachment"
+test_expect_equal_file EMACS attachment1.gz "$EXPECTED/attachment"
 
 test_begin_subtest "Save attachment from within emacs using notmuch-show-save-part"
 # save as archive to test that Emacs does not re-compress .gz
 test_emacs '(let ((standard-input "\"attachment2.gz\""))
 	      (notmuch-show-save-part "id:cf0c4d610911171136h1713aa59w9cf9aa31f052ad0a@mail.gmail.com" 5))'
-test_expect_equal_file attachment2.gz "$EXPECTED/attachment"
+test_expect_equal_file EMACS attachment2.gz "$EXPECTED/attachment"
 
 test_begin_subtest "View raw message within emacs"
 test_emacs '(notmuch-show "id:cf0c4d610911171136h1713aa59w9cf9aa31f052ad0a@mail.gmail.com")
 	    (notmuch-show-view-raw-message)
 	    (test-output)'
-test_expect_equal_file OUTPUT $EXPECTED/raw-message-cf0c4d-52ad0a
+test_expect_equal_file EMACS OUTPUT $EXPECTED/raw-message-cf0c4d-52ad0a
 
 test_begin_subtest "Hiding/showing signature in notmuch-show view"
 maildir_storage_thread=$(notmuch search --output=threads id:20091117190054.GU3165@dottiness.seas.harvard.edu)
@@ -279,7 +284,7 @@ test_emacs "(notmuch-show \"$maildir_storage_thread\")
 	    (search-backward \"Click/Enter to hide.\")
 	    (button-activate (button-at (point)))
 	    (test-output)"
-test_expect_equal_file OUTPUT $EXPECTED/notmuch-show-thread-maildir-storage
+test_expect_equal_file EMACS OUTPUT $EXPECTED/notmuch-show-thread-maildir-storage
 
 test_begin_subtest "Detection and hiding of top-post quoting of message"
 add_message '[subject]="The problem with top-posting"' \
@@ -326,13 +331,13 @@ Thanks for the advice! I will be sure to put it to good use.
 -Top Poster
 
 [ 9-line hidden original message. Click/Enter to show. ]" > EXPECTED
-test_expect_equal_file OUTPUT EXPECTED
+test_expect_equal_file EMACS OUTPUT EXPECTED
 
 test_begin_subtest "Hiding message in notmuch-show view"
 test_emacs '(notmuch-show "id:f35dbb950911171438k5df6eb56k77b6c0944e2e79ae@mail.gmail.com")
 	    (notmuch-show-toggle-message)
 	    (test-visible-output)'
-test_expect_equal_file OUTPUT $EXPECTED/notmuch-show-thread-with-hidden-messages
+test_expect_equal_file EMACS OUTPUT $EXPECTED/notmuch-show-thread-with-hidden-messages
 
 test_begin_subtest "Hiding message with visible citation in notmuch-show view"
 test_emacs '(notmuch-show "id:f35dbb950911171438k5df6eb56k77b6c0944e2e79ae@mail.gmail.com")
@@ -340,7 +345,7 @@ test_emacs '(notmuch-show "id:f35dbb950911171438k5df6eb56k77b6c0944e2e79ae@mail.
 	    (button-activate (button-at (point)))
 	    (notmuch-show-toggle-message)
 	    (test-visible-output)'
-test_expect_equal_file OUTPUT $EXPECTED/notmuch-show-thread-with-hidden-messages
+test_expect_equal_file EMACS OUTPUT $EXPECTED/notmuch-show-thread-with-hidden-messages
 
 test_begin_subtest "Stashing in notmuch-show"
 add_message '[date]="Sat, 01 Jan 2000 12:00:00 -0000"' \
@@ -370,7 +375,7 @@ test_emacs '(notmuch-show "id:\"bought\"")
         (reverse-region (point-min) (point-max))
 	    (test-output)'
 sed -i -e 's/^.*tmp.emacs\/mail.*$/FILENAME/' OUTPUT
-test_expect_equal_file OUTPUT $EXPECTED/emacs-stashing
+test_expect_equal_file EMACS OUTPUT $EXPECTED/emacs-stashing
 
 test_begin_subtest "Stashing in notmuch-search"
 test_emacs '(notmuch-search "id:\"bought\"")
@@ -381,12 +386,13 @@ test_emacs '(notmuch-search "id:\"bought\"")
         (yank)
 	    (test-output)'
 sed -i -e 's/^thread:.*$/thread:XXX/' OUTPUT
-test_expect_equal $(cat OUTPUT) "thread:XXX"
+output=$(cat OUTPUT)
+test_expect_equal EMACS "$output" "thread:XXX"
 
 test_begin_subtest 'Hiding message following HTML part'
 test_subtest_known_broken
 id='html-message@notmuchmail.org'
-emacs_deliver_message \
+test_have_prereq EMACS && emacs_deliver_message \
     'HTML message' \
     '' \
     "(message-goto-eoh)
@@ -394,7 +400,7 @@ emacs_deliver_message \
      (message-goto-body)
      (mml-insert-part \"text/html\")
      (insert \"<body>This is a test HTML message</body>\")"
-emacs_deliver_message \
+test_have_prereq EMACS && emacs_deliver_message \
     'Reply to HTML message' \
     'This is a reply to the test HTML message' \
     "(message-goto-eoh)
@@ -407,7 +413,7 @@ test_emacs "(notmuch-show \"id:$id\") \
             (notmuch-show-next-message) \
             (notmuch-show-toggle-message) \
             (test-visible-output \"EXPECTED\")"
-test_expect_equal_file OUTPUT EXPECTED
+test_expect_equal_file EMACS OUTPUT EXPECTED
 
 test_begin_subtest 'notmuch-show-advance-and-archive with invisible signature'
 message1='id:20091118010116.GC25380@dottiness.seas.harvard.edu'
@@ -421,14 +427,14 @@ test_emacs "(notmuch-search \"$message1 or $message2\")
 	    (test-output)"
 test_emacs "(notmuch-show \"$message2\")
 	    (test-output \"EXPECTED\")"
-test_expect_equal_file OUTPUT EXPECTED
+test_expect_equal_file EMACS OUTPUT EXPECTED
 
 test_begin_subtest "Refresh show buffer"
 test_emacs '(notmuch-show "id:f35dbb950911171438k5df6eb56k77b6c0944e2e79ae@mail.gmail.com")
 	    (test-visible-output "EXPECTED")
 	    (notmuch-show-refresh-view)
 	    (test-visible-output)'
-test_expect_equal_file OUTPUT EXPECTED
+test_expect_equal_file EMACS OUTPUT EXPECTED
 
 test_begin_subtest "Refresh modified show buffer"
 test_subtest_known_broken
@@ -439,6 +445,6 @@ test_emacs '(notmuch-show "id:f35dbb950911171438k5df6eb56k77b6c0944e2e79ae@mail.
 	    (test-visible-output "EXPECTED")
 	    (notmuch-show-refresh-view)
 	    (test-visible-output)'
-test_expect_equal_file OUTPUT EXPECTED
+test_expect_equal_file EMACS OUTPUT EXPECTED
 
 test_done
-- 
1.7.7.3

[PATCH 6/6] test: add 'Emacs' prereq to dependent 'emacs-large-search-buffer' tests

[Thomas Jost]

From: Pieter Praet <pieter@praet.org>

Adds a new test that checks for the presence of 'emacs',
and adds that test as a prereq to all subsequent tests
that rely on Emacs.

This causes tests with unmet dependencies to be skipped.

Signed-off-by: Pieter Praet <pieter@praet.org>
---
 test/emacs-large-search-buffer |    9 +++++++--
 1 files changed, 7 insertions(+), 2 deletions(-)

diff --git a/test/emacs-large-search-buffer b/test/emacs-large-search-buffer
index 6095e9d..63eb87c 100755
--- a/test/emacs-large-search-buffer
+++ b/test/emacs-large-search-buffer
@@ -2,6 +2,11 @@
 test_description="Emacs with large search results buffer"
 . test-lib.sh
 
+# Emacs is a prereq.
+test_set_bin_prereq dtach "dtach" DTACH
+test_have_prereq DTACH && test_set_bin_prereq emacs "Emacs" EMACS
+
+
 x=xxxxxxxxxx # 10
 x=$x$x$x$x$x$x$x$x$x$x # 100
 x=$x$x$x$x$x$x$x$x$x # 900
@@ -27,7 +32,7 @@ test_emacs '(notmuch-search "*")
 	    (notmuch-test-wait)
 	    (test-output)'
 sed -i -e s',  *, ,g' -e 's/xxx*/[BLOB]/g' OUTPUT
-test_expect_equal_file OUTPUT EXPEXTED
+test_expect_equal_file EMACS OUTPUT EXPEXTED
 
 test_begin_subtest "Ensure that emacs doesn't drop error messages"
 test_emacs '(notmuch-search "--this-option-does-not-exist")
@@ -38,6 +43,6 @@ Error: Unexpected output from notmuch search:
 Unrecognized option: --this-option-does-not-exist
 End of search results. (process returned 1)
 EOF
-test_expect_equal_file OUTPUT EXPEXTED
+test_expect_equal_file EMACS OUTPUT EXPEXTED
 
 test_done
-- 
1.7.7.3

Re: [PATCH 0/6] Rebase of Pieter's "set test prereqs"

[Jameson Graef Rollins]

[-- Attachment #1: Type: text/plain, Size: 1286 bytes --]

On Wed, 16 Nov 2011 15:33:49 +0100, Thomas Jost <schnouki@schnouki.net> wrote:
> Hello list,
> 
> This is another rebased version of Pieter's series to add GPG and Emacs as test
> prereqs, plus some additions on my own. (Rebased and posted as requested by
> Pieter [1].)
> 
> Changes as compared to Pieter's patches (including parts from [2]):
> - prereqs are not tested using test_expect_success as they were in Pieter's
>   original patches, but using a new function called test_set_bin_prereq. I wrote
>   this before the gdb prereq was added, hence the different way to set it.

Hey, Thomas.  Thanks so much for this work.  This sounds like a better
solution.

However, in the patches you send I see a lot of changes of the form

  -test_expect_success 'emacs delivery of encrypted message with attachment' \
  +test_expect_success GPG 'emacs delivery of encrypted message with attachment' \

and

  -test_expect_equal \
  +test_expect_equal GPG \

which seems to contradict what you've said above.  Not to mention that I
don't see anything that modifies calls to the test_expect_ functions.
Basically I see a lot more in the diffs than I would have expected in a
cursory look.  Is this just a rebase flub, or is there something I'm
missing?

jamie.

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

Re: [PATCH 0/6] Rebase of Pieter's "set test prereqs"

[Thomas Jost]

[-- Attachment #1: Type: text/plain, Size: 2849 bytes --]

On Wed, 16 Nov 2011 10:53:42 -0800, Jameson Graef Rollins <jrollins@finestructure.net> wrote:
> On Wed, 16 Nov 2011 15:33:49 +0100, Thomas Jost <schnouki@schnouki.net> wrote:
> > Hello list,
> > 
> > This is another rebased version of Pieter's series to add GPG and Emacs as test
> > prereqs, plus some additions on my own. (Rebased and posted as requested by
> > Pieter [1].)
> > 
> > Changes as compared to Pieter's patches (including parts from [2]):
> > - prereqs are not tested using test_expect_success as they were in Pieter's
> >   original patches, but using a new function called test_set_bin_prereq. I wrote
> >   this before the gdb prereq was added, hence the different way to set it.
> 
> Hey, Thomas.  Thanks so much for this work.  This sounds like a better
> solution.
> 
> However, in the patches you send I see a lot of changes of the form
> 
>   -test_expect_success 'emacs delivery of encrypted message with attachment' \
>   +test_expect_success GPG 'emacs delivery of encrypted message with attachment' \
> 
> and
> 
>   -test_expect_equal \
>   +test_expect_equal GPG \
> 
> which seems to contradict what you've said above.  Not to mention that I
> don't see anything that modifies calls to the test_expect_ functions.
> Basically I see a lot more in the diffs than I would have expected in a
> cursory look.  Is this just a rebase flub, or is there something I'm
> missing?
> 
> jamie.

Hi Jamie,

I guess I wasn't clear in my explanations :)

Pieter's patches use this to detect the presence of GPG/Emacs and set
the prereq:

    +# GnuPG is a prereq.
    +test_expect_success "prereq: GnuPG is present" "which gpg" \
    +    && test_set_prereq GPG

There are 2 problems with this approach: 

- test_expect_success returns 0 regardless of the actual result of the
  command it runs. So even if gpg is not installed, 
    text_expect_success "..." "which gpg"
  will succeed, and "test_set_prereq GPG" will be run.
  This, however, has been fixed in commit 003e7180 -- which had not been
  pushed when I wrote this in the first place :)

- using test_expect_* to set a prereq does not make sense. If emacs is
  absent, the test suite would report a failed test. But a missing
  prereq is *not* a notmuch issue, so this should *not* be reported as a
  failed test.

Hence my first patch, which defines test_set_bin_prereq, a new helper
function to set a prereq without using any test_expect_*.

After that we can use the normal prereq syntax from the test suite:
- test_expect_success COMMAND --> run COMMAND, expecting it to succeed
- test_expect_success PREREQ COMMAND --> skip if PREREQ is not set, else
  run the test as before
(and same thing with the other test_expect_* functions)

Does it make more sense now?

Regards,

-- 
Thomas/Schnouki

[-- Attachment #2: Type: application/pgp-signature, Size: 489 bytes --]

Re: [PATCH 0/6] Rebase of Pieter's "set test prereqs"

[Pieter Praet]

[-- Attachment #1: Type: text/plain, Size: 2915 bytes --]

On Wed, 16 Nov 2011 15:33:49 +0100, Thomas Jost <schnouki@schnouki.net> wrote:
> Hello list,
> 
> This is another rebased version of Pieter's series to add GPG and Emacs as test
> prereqs, plus some additions on my own. (Rebased and posted as requested by
> Pieter [1].)
> 

Thanks Thomas!

Although... you may have misread (or maybe I mistyped :), but what I
actually intended [1] was for you to rebase *only* your fixes on top of
my rebased series (e.g. see "tjost-fixes.patch" in att), so you could
receive proper credit for cleaning up my mess.

Also, while my apprehension [2,3] re the inclusion of the SCREEN/DTACH
prereq in patches #4,5,6 didn't have much merit (it's an all-or-nothing
affair anyways), the issue [3] in patch #5 @ "Reply within emacs" still
stands: `sed' will run unconditionally, and treat "EMACS" as an input
file.  (see "sed-prereq-fix.patch" in att).


Thanks again!

> Changes as compared to Pieter's patches (including parts from [2]):
> - prereqs are not tested using test_expect_success as they were in Pieter's
>   original patches, but using a new function called test_set_bin_prereq. I wrote
>   this before the gdb prereq was added, hence the different way to set it.
> 
> - some fixes in Pieter's patches so that it actually works when gpg is not
>   installed. Can't exactly remember what (...but you can just check his original
>   patches), but in the end it was working fine in a chroot without gpg.
> 
> - since Emacs is now run using dtach, the emacs prereq also depends on dtach.
>   The presence of emacs and dtach is also checked in the test_emacs() function
>   of the test suite.
> 
> - testing for prereqs is now done using the "hash" built-in instead of "which",
>   as suggested in [3].
> 
> Tested with and without dtach. A previous version of this series was also
> without emacs/gpg in a chroot, but not this one :)
> 
> [1] id:"874ny4fcdz.fsf@praet.org"
> [2] id:"1317660447-27520-1-git-send-email-schnouki@schnouki.net"
> [3] id:"87zkgemodd.fsf@praet.org"
> 
> Pieter Praet (4):
>   test: add 'GnuPG' prereq to dependent 'crypto' tests
>   test: add 'Emacs' prereq to dependent 'crypto' tests
>   test: add 'Emacs' prereq to dependent 'emacs' tests
>   test: add 'Emacs' prereq to dependent 'emacs-large-search-buffer'
>     tests
> 
> Thomas Jost (2):
>   test: define a helper function for defining prereqs on executables
>   test: check if emacs and dtach are available in test_emacs()
> 
>  test/crypto                    |   46 +++++++++++++++-------
>  test/emacs                     |   82 +++++++++++++++++++++------------------
>  test/emacs-large-search-buffer |    9 +++-
>  test/test-lib.sh               |   17 ++++++++
>  4 files changed, 99 insertions(+), 55 deletions(-)
> 
> -- 
> 1.7.7.3
> 
> _______________________________________________
> notmuch mailing list
> notmuch@notmuchmail.org
> http://notmuchmail.org/mailman/listinfo/notmuch


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: sed-prereq-fix.patch --]
[-- Type: text/x-patch, Size: 541 bytes --]

diff --git a/test/emacs b/test/emacs
index ea641af..79edf01 100755
--- a/test/emacs
+++ b/test/emacs
@@ -244,7 +244,8 @@ test_emacs '(notmuch-search "subject:\"testing message sent via SMTP\"")
 	    (notmuch-test-wait)
 	    (notmuch-search-reply-to-thread)
 	    (test-output)'
-sed -i -e 's/^In-Reply-To: <.*>$/In-Reply-To: <XXX>/' EMACS OUTPUT
+test_have_prereq EMACS \
+    && sed -i -e 's/^In-Reply-To: <.*>$/In-Reply-To: <XXX>/' OUTPUT
 cat <<EOF >EXPECTED
 From: Notmuch Test Suite <test_suite@notmuchmail.org>
 To: user@example.com

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #3: tjost-fixes.patch --]
[-- Type: text/x-patch, Size: 9112 bytes --]

diff --git a/test/crypto b/test/crypto
index 4a00c00..085070b 100755
--- a/test/crypto
+++ b/test/crypto
@@ -8,12 +8,11 @@ test_description='PGP/MIME signature verification and decryption'
 . ./test-lib.sh
 
 # Emacs is a prereq.
-test_expect_success "prereq: Emacs is present" "which emacs" \
-    && test_set_prereq EMACS
+test_set_bin_prereq dtach "dtach" DTACH
+test_have_prereq DTACH && test_set_bin_prereq emacs "Emacs" EMACS
 
 # GnuPG is a prereq.
-test_expect_success "prereq: GnuPG is present" "which gpg" \
-    && test_set_prereq GPG
+test_set_bin_prereq gpg "GnuPG" GPG
 
 # Some tests have multiple prereqs, but the test_expect_* functions
 # accept only a single argument as prereq tag, and using test_have_prereq
@@ -28,6 +27,7 @@ add_gnupg_home ()
     local output
     [ -d ${GNUPGHOME} ] && return
     mkdir -m 0700 "$GNUPGHOME"
+    test_have_prereq GPG || return
     gpg --no-tty --import <$TEST_DIRECTORY/gnupg-secret-key.asc >"$GNUPGHOME"/import.log 2>&1
     test_debug "cat $GNUPGHOME/import.log"
     if (gpg --quick-random --version >/dev/null 2>&1) ; then
@@ -41,7 +41,7 @@ add_gnupg_home ()
 
 add_gnupg_home
 # get key fingerprint
-FINGERPRINT=$(gpg --no-tty --list-secret-keys --with-colons --fingerprint | grep '^fpr:' | cut -d: -f10)
+test_have_prereq GPG && FINGERPRINT=$(gpg --no-tty --list-secret-keys --with-colons --fingerprint | grep '^fpr:' | cut -d: -f10)
 
 # for some reason this is needed for emacs_deliver_message to work,
 # although I can't figure out why
@@ -80,7 +80,7 @@ expected='[[[{"id": "XXXXX",
  {"id": 3,
  "content-type": "application/pgp-signature"}]}]},
  []]]]'
-test_expect_equal GPG \
+test_expect_equal EMACS+GPG \
     "$output" \
     "$expected"
 
@@ -115,7 +115,7 @@ expected='[[[{"id": "XXXXX",
  {"id": 3,
  "content-type": "application/pgp-signature"}]}]},
  []]]]'
-test_expect_equal GPG \
+test_expect_equal EMACS+GPG \
     "$output" \
     "$expected"
 
@@ -148,7 +148,7 @@ expected='[[[{"id": "XXXXX",
  {"id": 3,
  "content-type": "application/pgp-signature"}]}]},
  []]]]'
-test_expect_equal GPG \
+test_expect_equal EMACS+GPG \
     "$output" \
     "$expected"
 mv "${GNUPGHOME}"{.bak,}
@@ -191,7 +191,7 @@ Non-text part: application/octet-stream
 \fpart}
 \fbody}
 \fmessage}'
-test_expect_equal GPG \
+test_expect_equal EMACS+GPG \
     "$output" \
     "$expected"
 
@@ -226,7 +226,7 @@ expected='[[[{"id": "XXXXX",
  "content-type": "application/octet-stream",
  "filename": "TESTATTACHMENT"}]}]}]},
  []]]]'
-test_expect_equal GPG \
+test_expect_equal EMACS+GPG \
     "$output" \
     "$expected"
 
@@ -237,7 +237,7 @@ output=$(notmuch show --format=json --part=4 --decrypt subject:"test encrypted m
 expected='{"id": 4,
  "content-type": "text/plain",
  "content": "This is a test encrypted message.\n"}'
-test_expect_equal GPG \
+test_expect_equal EMACS+GPG \
     "$output" \
     "$expected"
 
@@ -247,7 +247,7 @@ notmuch show \
     --part=5 \
     --decrypt \
     subject:"test encrypted message 001" >OUTPUT
-test_expect_equal_file GPG OUTPUT TESTATTACHMENT
+test_expect_equal_file EMACS+GPG OUTPUT TESTATTACHMENT
 
 test_begin_subtest "decryption failure with missing key"
 mv "${GNUPGHOME}"{,.bak}
@@ -274,7 +274,7 @@ expected='[[[{"id": "XXXXX",
  {"id": 3,
  "content-type": "application/octet-stream"}]}]},
  []]]]'
-test_expect_equal GPG \
+test_expect_equal EMACS+GPG \
     "$output" \
     "$expected"
 mv "${GNUPGHOME}"{.bak,}
@@ -314,7 +314,7 @@ expected='[[[{"id": "XXXXX",
  "content-type": "text/plain",
  "content": "This is another test encrypted message.\n"}]}]},
  []]]]'
-test_expect_equal GPG \
+test_expect_equal EMACS+GPG \
     "$output" \
     "$expected"
 
@@ -326,7 +326,7 @@ Subject: Re: test encrypted message 002
 
 On 01 Jan 2000 12:00:00 -0000, Notmuch Test Suite <test_suite@notmuchmail.org> wrote:
 > This is another test encrypted message.'
-test_expect_equal GPG \
+test_expect_equal EMACS+GPG \
     "$output" \
     "$expected"
 
@@ -367,7 +367,7 @@ expected='[[[{"id": "XXXXX",
  {"id": 3,
  "content-type": "application/pgp-signature"}]}]},
  []]]]'
-test_expect_equal GPG \
+test_expect_equal EMACS+GPG \
     "$output" \
     "$expected"
 
diff --git a/test/emacs b/test/emacs
index 41b6ee9..ea641af 100755
--- a/test/emacs
+++ b/test/emacs
@@ -4,8 +4,8 @@ test_description="emacs interface"
 . test-lib.sh
 
 # Emacs is a prereq.
-test_expect_success "prereq: Emacs is present" "which emacs" \
-    && test_set_prereq EMACS
+test_set_bin_prereq dtach "dtach" DTACH
+test_have_prereq DTACH && test_set_bin_prereq emacs "Emacs" EMACS
 
 
 EXPECTED=$TEST_DIRECTORY/emacs.expected-output
@@ -124,7 +124,7 @@ output=$(notmuch search 'id:"123..456@example"' | notmuch_search_sanitize)
 test_expect_equal EMACS "$output" "thread:XXX   2001-01-05 [1/1] Notmuch Test Suite; Message with .. in Message-Id (inbox search-add show-add)"
 
 test_begin_subtest "Sending a message via (fake) SMTP"
-emacs_deliver_message \
+test_have_prereq EMACS && emacs_deliver_message \
     'Testing message sent via SMTP' \
     'This is a test that messages are sent via SMTP' \
     '(message-goto-to)
@@ -150,7 +150,7 @@ test_expect_equal_file EMACS OUTPUT EXPECTED
 test_begin_subtest "Verify that sent messages are saved/searchable (via FCC)"
 notmuch new > /dev/null
 output=$(notmuch search 'subject:"testing message sent via SMTP"' | notmuch_search_sanitize)
-test_expect_equal "$output" "thread:XXX   2000-01-01 [1/1] Notmuch Test Suite; Testing message sent via SMTP (inbox)"
+test_expect_equal EMACS "$output" "thread:XXX   2000-01-01 [1/1] Notmuch Test Suite; Testing message sent via SMTP (inbox)"
 
 test_begin_subtest "notmuch-fcc-dirs set to nil"
 test_emacs "(let ((notmuch-fcc-dirs nil))
@@ -244,7 +244,7 @@ test_emacs '(notmuch-search "subject:\"testing message sent via SMTP\"")
 	    (notmuch-test-wait)
 	    (notmuch-search-reply-to-thread)
 	    (test-output)'
-sed -i -e 's/^In-Reply-To: <.*>$/In-Reply-To: <XXX>/' OUTPUT
+sed -i -e 's/^In-Reply-To: <.*>$/In-Reply-To: <XXX>/' EMACS OUTPUT
 cat <<EOF >EXPECTED
 From: Notmuch Test Suite <test_suite@notmuchmail.org>
 To: user@example.com
@@ -375,7 +375,7 @@ test_emacs '(notmuch-show "id:\"bought\"")
         (reverse-region (point-min) (point-max))
 	    (test-output)'
 sed -i -e 's/^.*tmp.emacs\/mail.*$/FILENAME/' OUTPUT
-test_expect_equal_file OUTPUT $EXPECTED/emacs-stashing
+test_expect_equal_file EMACS OUTPUT $EXPECTED/emacs-stashing
 
 test_begin_subtest "Stashing in notmuch-search"
 test_emacs '(notmuch-search "id:\"bought\"")
@@ -386,12 +386,13 @@ test_emacs '(notmuch-search "id:\"bought\"")
         (yank)
 	    (test-output)'
 sed -i -e 's/^thread:.*$/thread:XXX/' OUTPUT
-test_expect_equal $(cat OUTPUT) "thread:XXX"
+output=$(cat OUTPUT)
+test_expect_equal EMACS "$output" "thread:XXX"
 
 test_begin_subtest 'Hiding message following HTML part'
 test_subtest_known_broken
 id='html-message@notmuchmail.org'
-emacs_deliver_message \
+test_have_prereq EMACS && emacs_deliver_message \
     'HTML message' \
     '' \
     "(message-goto-eoh)
@@ -399,7 +400,7 @@ emacs_deliver_message \
      (message-goto-body)
      (mml-insert-part \"text/html\")
      (insert \"<body>This is a test HTML message</body>\")"
-emacs_deliver_message \
+test_have_prereq EMACS && emacs_deliver_message \
     'Reply to HTML message' \
     'This is a reply to the test HTML message' \
     "(message-goto-eoh)
@@ -426,7 +427,7 @@ test_emacs "(notmuch-search \"$message1 or $message2\")
 	    (test-output)"
 test_emacs "(notmuch-show \"$message2\")
 	    (test-output \"EXPECTED\")"
-test_expect_equal_file OUTPUT EXPECTED
+test_expect_equal_file EMACS OUTPUT EXPECTED
 
 test_begin_subtest "Refresh show buffer"
 test_emacs '(notmuch-show "id:f35dbb950911171438k5df6eb56k77b6c0944e2e79ae@mail.gmail.com")
diff --git a/test/emacs-large-search-buffer b/test/emacs-large-search-buffer
index 5757380..63eb87c 100755
--- a/test/emacs-large-search-buffer
+++ b/test/emacs-large-search-buffer
@@ -3,8 +3,8 @@ test_description="Emacs with large search results buffer"
 . test-lib.sh
 
 # Emacs is a prereq.
-test_expect_success "prereq: Emacs is present" "which emacs" \
-    && test_set_prereq EMACS
+test_set_bin_prereq dtach "dtach" DTACH
+test_have_prereq DTACH && test_set_bin_prereq emacs "Emacs" EMACS
 
 
 x=xxxxxxxxxx # 10
diff --git a/test/test-lib.sh b/test/test-lib.sh
index 1ea7fa9..ca1f412 100755
--- a/test/test-lib.sh
+++ b/test/test-lib.sh
@@ -543,6 +543,19 @@ test_have_prereq () {
 	esac
 }
 
+test_set_bin_prereq () {
+	bin=$1
+	name=$2
+	prereq=$3
+	if hash $bin &>/dev/null
+	then
+		test_set_prereq $prereq
+	else
+		say_color info "%-6s" "INFO"
+		echo " Missing test prerequisite: $name"
+	fi
+}
+
 # You are not expected to call test_ok_ and test_failure_ directly, use
 # the text_expect_* functions instead.
 
@@ -839,6 +852,10 @@ EOF
 }
 
 test_emacs () {
+	# First check that emacs and dtach are available
+	hash emacs &>/dev/null || return
+	hash dtach &>/dev/null || return
+
 	if [ -z "$EMACS_SERVER" ]; then
 		EMACS_SERVER="notmuch-test-suite-$$"
 		# start a detached session with an emacs server

[-- Attachment #4: Type: text/plain, Size: 123 bytes --]



Peace

-- 
Pieter

[1] id:"874ny4fcdz.fsf@praet.org"
[2] id:"87ipn3obvi.fsf@praet.org"
[3] id:"87fwi7obso.fsf@praet.org"

Re: [PATCH 0/6] Rebase of Pieter's "set test prereqs"

[Thomas Jost]

[-- Attachment #1: Type: text/plain, Size: 1549 bytes --]

On Wed, 16 Nov 2011 21:50:17 +0100, Pieter Praet <pieter@praet.org> wrote:
> On Wed, 16 Nov 2011 15:33:49 +0100, Thomas Jost <schnouki@schnouki.net> wrote:
> > Hello list,
> > 
> > This is another rebased version of Pieter's series to add GPG and Emacs as test
> > prereqs, plus some additions on my own. (Rebased and posted as requested by
> > Pieter [1].)
> > 
> 
> Thanks Thomas!
> 
> Although... you may have misread (or maybe I mistyped :), but what I
> actually intended [1] was for you to rebase *only* your fixes on top of
> my rebased series (e.g. see "tjost-fixes.patch" in att), so you could
> receive proper credit for cleaning up my mess.

Oh, ok, I must have misread that :) 

Right now your patches don't apply cleanly on master (conflict in patch
3 due to commit 5964a7), and I think that Dmitry's patches [1] may be a
better way to handle prereqs. So I probably won't send those patches
until we decide which approach is the way to go.

[1] id:"1321494986-18998-1-git-send-email-dmitry.kurochkin@gmail.com"

> Also, while my apprehension [2,3] re the inclusion of the SCREEN/DTACH
> prereq in patches #4,5,6 didn't have much merit (it's an all-or-nothing
> affair anyways), the issue [3] in patch #5 @ "Reply within emacs" still
> stands: `sed' will run unconditionally, and treat "EMACS" as an input
> file.  (see "sed-prereq-fix.patch" in att).

Nice catch with this sed issue. Looks like I need to be more careful
when replacing "OUTPUT" with "EMACS OUTPUT"...

Thanks,

-- 
Thomas/Schnouki

[-- Attachment #2: Type: application/pgp-signature, Size: 489 bytes --]

Re: [PATCH 0/6] Rebase of Pieter's "set test prereqs"

[Pieter Praet]

On Thu, 17 Nov 2011 11:14:11 +0100, Thomas Jost <schnouki@schnouki.net> wrote:
> On Wed, 16 Nov 2011 21:50:17 +0100, Pieter Praet <pieter@praet.org> wrote:
> > On Wed, 16 Nov 2011 15:33:49 +0100, Thomas Jost <schnouki@schnouki.net> wrote:
> > > Hello list,
> > > 
> > > This is another rebased version of Pieter's series to add GPG and Emacs as test
> > > prereqs, plus some additions on my own. (Rebased and posted as requested by
> > > Pieter [1].)
> > > 
> > 
> > Thanks Thomas!
> > 
> > Although... you may have misread (or maybe I mistyped :), but what I
> > actually intended [1] was for you to rebase *only* your fixes on top of
> > my rebased series (e.g. see "tjost-fixes.patch" in att), so you could
> > receive proper credit for cleaning up my mess.
> 
> Oh, ok, I must have misread that :) 
> 
> Right now your patches don't apply cleanly on master (conflict in patch
> 3 due to commit 5964a7), and I think that Dmitry's patches [1] may be a
> better way to handle prereqs. So I probably won't send those patches
> until we decide which approach is the way to go.
> 
> [1] id:"1321494986-18998-1-git-send-email-dmitry.kurochkin@gmail.com"
> 

Yup, Dmitry's solution is much more elegant!

> > Also, while my apprehension [2,3] re the inclusion of the SCREEN/DTACH
> > prereq in patches #4,5,6 didn't have much merit (it's an all-or-nothing
> > affair anyways), the issue [3] in patch #5 @ "Reply within emacs" still
> > stands: `sed' will run unconditionally, and treat "EMACS" as an input
> > file.  (see "sed-prereq-fix.patch" in att).
> 
> Nice catch with this sed issue. Looks like I need to be more careful
> when replacing "OUTPUT" with "EMACS OUTPUT"...
> 
> Thanks,
> 
> -- 
> Thomas/Schnouki


Peace

-- 
Pieter