Re: testing framework and package.el

["Eric Schulte" <schulte.eric@gmail.com>]

On the relation between tests and core code, how should regular Emacs
code standards be applied to tests.  For example
- need my tests compile cleanly w/o warnings
- may I require 'cl-seq in my testing files
- must I ensure that every test file name is unique in the first X many
  characters

I'd think one driving factor should be how we anticipate the tests being
used, e.g. will they mainly be run for one project at a time to as part
of the development process, or will the entire Emacs tests suite
(assuming one exists at some point in the future) often be run en mass?

Best -- Eric

"Stephen J. Turnbull" <stephen@xemacs.org> writes:

> Christian Ohler writes:
>
>  >   On 5/10/10 2:18, Chong Yidong wrote:
>  > > Christian Ohler<ohler+emacs@fastmail.net>  writes:
>  > >> Is there a deep reason
>  > >> why Emacs currently has test/ as a separate top-level directory?
>  > > Because the test files are not to be distributed with the tarball.
>
>  > Why not?
>
> Conceptually, tests should be developed separately from the core, just
> as the Lisp Reference is.  Of course, in practice it's not that easy,
> but I think it's a good idea to structure the tests and the reference
> at the same level as the core sources.
>
> XEmacs experience with its tests has shown that having the tests
> subsidiary to the source tree leads to regressions.  Specifically,
> changing the source code leads to changes in behavior, and these
> changes are "sanctified" by writing tests after the fact.  But these
> tests are *not* run against older versions, leading to unplanned
> divergence in behavior.  Occasionally these divergences are annoying
> to users or third-party libraries.
>
> I think as a matter of convenience both Reference and tests should be
> distributed with (ie, in) the core tarball.