Mark Emacs as prerequisite for tests which require it

Mark Emacs as prerequisite for tests which require it

[Milton Vandersloot]

Dear notmuch developers

The notmuch test suite depends on some external dependencies to finish successfully (i.e. no failures but maybe some skips), one of them being Emacs.
As I had no Emacs installed, I discovered that some tests fail and do not state properly that Emacs is needed for them.
Below are small sed commands (was easier this way) which put the "test_require_external_prereq emacs" to the missing places.

For some tests (those which test Emacs capabilities) it is clear that Emacs is a necessity, some tests in particular the crypto tests (T350, T355, T357) do require Emacs only for generating a initial mail to index/test on. Here it would in my opinion be better to find an alternative way of producing that e-mail so these tests can be run without Emacs (That would reduce the long list of test names below tremendously). A promising approach would be using the same idea as the last tests in T357-index-decryption.sh, they use "add_email_corpus crypto" for (what seems to me) populating the mail archive.

Regards
Milton

# Mark tests properly which require Emacs
# (some of them rely on Emacs to generate an test e-mail)
sed -i -e '\:^\..*/test-lib.sh: a test_require_external_prereq emacs' \
	test/T310-emacs.sh \
	test/T350-crypto.sh \
	test/T355-smime.sh \
	test/T450-emacs-show.sh \
	test/T460-emacs-tree.sh \
	test/T730-emacs-forwarding.sh

# Most tests in test/T357-index-decryption.sh require a working Emacs.
# However, a couple of tests at the end do not.
# Mark the tests needing Emacs properly.
for testname in \
	'emacs delivery of encrypted message' \
	"search for unindexed cleartext" \
	'emacs delivery of encrypted message' \
	"emacs delivery of encrypted message, indexed cleartext" \
	"emacs search by property for one message" \
	"show the message body of the encrypted message" \
	"message should go away after deletion" \
	"message cleartext not present after insert" \
	"stash decryption during show" \
	"search should now find the contents" \
	"message cleartext is present after reinserting with --decrypt=true" \
	"delete all copies of the message" \
	"message cleartext is present with insert --decrypt=true" \
	'tagging all messages' \
	"verify that tags have not changed" \
	'reindex old messages' \
	"reindexed encrypted message, including cleartext" \
	"emacs search by property for both messages" \
	'reindex in auto mode' \
	"reindexed encrypted messages, should not have changed" \
	'reindex without cleartext' \
	"reindexed encrypted messages, without cleartext" \
	'reindex using only session keys' \
	"reindexed encrypted messages, decrypting only with session keys" \
	"emacs search by property with both messages unindexed" \
	"verify that tags remain without cleartext" \
	"index cleartext without keeping session keys" \
	"Ensure that the indexed terms are present" \
	"show one of the messages with --decrypt=true" \
	"Ensure that we cannot show the message with --decrypt=auto" \
	; do
		sed -e "/test_begin_subtest [\"']${testname}[\"']/ a test_require_external_prereq emacs" \
			-i test/T357-index-decryption.sh \
			|| echo "Failed: Marking test ${testname} for skipping."
done

Re: Mark Emacs as prerequisite for tests which require it

[David Bremner]

Milton Vandersloot <miltonrobertvandersloot1412@protonmail.com> writes:

> Dear notmuch developers
>
> The notmuch test suite depends on some external dependencies to finish successfully (i.e. no failures but maybe some skips), one of them being Emacs.
> As I had no Emacs installed, I discovered that some tests fail and do not state properly that Emacs is needed for them.
> Below are small sed commands (was easier this way) which put the "test_require_external_prereq emacs" to the missing places.
>
> For some tests (those which test Emacs capabilities) it is clear that Emacs is a necessity, some tests in particular the crypto tests (T350, T355, T357) do require Emacs only for generating a initial mail to index/test on. Here it would in my opinion be better to find an alternative way of producing that e-mail so these tests can be run without Emacs (That would reduce the long list of test names below tremendously). A promising approach would be using the same idea as the last tests in T357-index-decryption.sh, they use "add_email_corpus crypto" for (what seems to me) populating the mail archive.

I think this is now handled by the commits following cd9ec20a49, which
will be in the next release of notmuch.