unofficial mirror of notmuch@notmuchmail.org
 help / color / mirror / code / Atom feed
* [PATCH] Modularize test suite
@ 2010-04-30 18:36 Jesse Rosenthal
  2010-04-30 18:52 ` Jesse Rosenthal
                   ` (2 more replies)
  0 siblings, 3 replies; 8+ messages in thread
From: Jesse Rosenthal @ 2010-04-30 18:36 UTC (permalink / raw)
  To: Notmuch developer list

Responding to this email is a patch to modularize the test suite. It
follows a relatively simple model, but it does seem like it will offer
some degree of flexibility. Plus, no relicensing is necessary.

In short, tests are put in subdirs of test/tests, e.g.:

     test/tests/notmuch-new/
     test/tests/notmuch-search/
     test/tests/json/
     ...

"notmuch-test" (which is the same as "notmuch-test all") should work as
usual. But if you call "notmuch-test json", it will only the json tests.

It will do this by running all of the files in the json subdir that have
an extension "*.test" or "*.setup" (all other files will be ignored). It
runs them in numerical order, which you can set by a numerical
prefix. So, for example, in test/tests/notmuch-reply:

         00001-notmuch-reply.setup
         00100-basic-reply.test          
         00200-multiple-recipients.test  
         00300-reply-with-cc.test        
         00400-reply-from-alternate-address.test
         00500-support-for-reply-to.test
         00600-unmunging-reply-to.test
         99999-notmuch-reply.setup

The basic numbering system is 00100, 00200, etc. to both allow a lot of
space to fit things in between, and to let the collection of tests
grow. 

00001-$(basename $(pwd)).setup and 99999-$(basename $(pwd)).setup are
reserved for setting up and cleaning up. This allows us to call
individual sets of tests without calling the entire suite in order to
get the environment that the tests expect.

Note that if you add a new subdir collection of tests, you have to add
it to the $TEST_COLLECTIONS variable in notmuch-test. It doesn't run all
subdirs automatically, in case you want to test something out, or have a
subdir that only works in certain cases (imagine future tests of remote
setups).

Sorry the patch is so large: there was a lot of back-and-forth, and it
was hard to break the patches down into coherent steps. Carl, if you'd
prefer, I could send you a more granular, but confusing, series of
patches.

Best,
Jesse

^ permalink raw reply	[flat|nested] 8+ messages in thread
end of thread, other threads:[~2010-05-05 17:33 UTC | newest]

Thread overview: 8+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2010-04-30 18:36 [PATCH] Modularize test suite Jesse Rosenthal
2010-04-30 18:52 ` Jesse Rosenthal
2010-04-30 19:07 ` [PATCH 1/2] test: add modular test collections Jesse Rosenthal
2010-04-30 19:09   ` [PATCH 2/2] test: Have notmuch-test use the " Jesse Rosenthal
2010-05-05  4:46     ` Jesse Rosenthal
2010-05-05  7:23 ` [PATCH] Modularize test suite Michal Sojka
2010-05-05  8:04   ` Jesse Rosenthal
2010-05-05 17:33   ` Carl Worth

Code repositories for project(s) associated with this public inbox

	https://yhetil.org/notmuch.git/

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).