so-long tests

so-long tests

[Eli Zaretskii]

Thanks for merging the branch.

I think the tests for the new package should be moved to be under
test/lisp/, because we generally mirror under test/ the source tree
structure.

Re: so-long tests

[Phil Sainty]

Hi Eli,

On 13/07/19 7:25 PM, Eli Zaretskii wrote:
> I think the tests for the new package should be moved to be under
> test/lisp/, because we generally mirror under test/ the source tree
> structure.

I've done it this way based on the documentation in
test/file-organization.org:

> Tests should normally reside in a file with ~-tests.el~ added to the
> base-name of the tested source file; hence ~ert.el~ is tested in
> ~ert-tests.el~, and ~pcase.el~ is tested in ~pcase-tests.el~.  As an
> exception, tests for a single feature may be placed into multiple
> files of any name which are themselves placed in a directory named
> after the feature with ~-tests~ appended, such as
> ~/test/lisp/emacs-lisp/eieio-tests~

Hence test/lisp/so-long-tests/*

So I *think* that's correct as-is?


> Thanks for merging the branch.

You're welcome.


-Phil

Re: so-long tests

[Eli Zaretskii]

> From: Phil Sainty <psainty@orcon.net.nz>
> Date: Sat, 13 Jul 2019 20:08:35 +1200
> Cc: emacs-devel@gnu.org
> 
> > I think the tests for the new package should be moved to be under
> > test/lisp/, because we generally mirror under test/ the source tree
> > structure.
> 
> I've done it this way based on the documentation in
> test/file-organization.org:
> 
> > Tests should normally reside in a file with ~-tests.el~ added to the
> > base-name of the tested source file; hence ~ert.el~ is tested in
> > ~ert-tests.el~, and ~pcase.el~ is tested in ~pcase-tests.el~.  As an
> > exception, tests for a single feature may be placed into multiple
> > files of any name which are themselves placed in a directory named
> > after the feature with ~-tests~ appended, such as
> > ~/test/lisp/emacs-lisp/eieio-tests~
> 
> Hence test/lisp/so-long-tests/*
> 
> So I *think* that's correct as-is?

Why do we need the exception in this case?  Exceptions should be
reserved for exceptional cases.

Re: so-long tests

[Phil Sainty]

On 13/07/19 8:19 PM, Eli Zaretskii wrote:
>> Hence test/lisp/so-long-tests/*
> 
> Why do we need the exception in this case?  Exceptions should
> be reserved for exceptional cases.

The reason I used that directory structure is that I ended up
using multiple test files.  My initial tests were indeed written
as a test/lisp/so-long-tests.el file, but I switched to the
alternative structure when I added additional files.

Going by the docs, it hadn't occurred to me that doing so would
be in any way controversial.  My reading was that either approach
was fine.


-Phil

Re: so-long tests

[Eli Zaretskii]

> Cc: emacs-devel@gnu.org
> From: Phil Sainty <psainty@orcon.net.nz>
> Date: Sat, 13 Jul 2019 21:41:10 +1200
> 
> > Why do we need the exception in this case?  Exceptions should
> > be reserved for exceptional cases.
> 
> The reason I used that directory structure is that I ended up
> using multiple test files.  My initial tests were indeed written
> as a test/lisp/so-long-tests.el file, but I switched to the
> alternative structure when I added additional files.

If you'd like to keep the tests on separate files, it's fine with me,
but please add a comment somewhere for why these tests are in a
subdirectory.

The inconvenient part of this arrangement is that I at least am in the
habit of typing "make lisp/foo-tests TAB" in the test/ directory after
making any changes to lisp/foo.el, and I normally take the lack of
completion to mean there are no tests for foo.el.  Having to remember
to look in subdirectories is a burden.  It is up to you to decide
whether your motivation for keeping separate test files is more
important than that.

Re: so-long tests

[Phil Sainty]

On 13/07/19 10:01 PM, Eli Zaretskii wrote:
> The inconvenient part of this arrangement is that I at least am in the
> habit of typing "make lisp/foo-tests TAB" in the test/ directory after
> making any changes to lisp/foo.el, and I normally take the lack of
> completion to mean there are no tests for foo.el.  Having to remember
> to look in subdirectories is a burden.

Your shell doesn't complete to the directory so-long-tests/ when you
type TAB?  (Bash does so for me.)

I wonder whether the Makefile could be made to do-the-right-thing if
"make lisp/foo-tests/" is used?  It would make sense to me if running
that command caused all the tests in that directory to run.

That would then resolve this (in)convenience issue for *any* library,
which would seem like the ideal solution to this problem.

I don't really know my way around Makefiles, but perhaps someone who
does will be able to comment?


-Phil

Re: so-long tests

[phillip.lord]

On 2019-07-13 11:01, Eli Zaretskii wrote:
>> Cc: emacs-devel@gnu.org
>> From: Phil Sainty <psainty@orcon.net.nz>
>> Date: Sat, 13 Jul 2019 21:41:10 +1200
>> 
>> > Why do we need the exception in this case?  Exceptions should
>> > be reserved for exceptional cases.
>> 
>> The reason I used that directory structure is that I ended up
>> using multiple test files.  My initial tests were indeed written
>> as a test/lisp/so-long-tests.el file, but I switched to the
>> alternative structure when I added additional files.
> 
> If you'd like to keep the tests on separate files, it's fine with me,
> but please add a comment somewhere for why these tests are in a
> subdirectory.
> 
> The inconvenient part of this arrangement is that I at least am in the
> habit of typing "make lisp/foo-tests TAB" in the test/ directory after
> making any changes to lisp/foo.el, and I normally take the lack of
> completion to mean there are no tests for foo.el.  Having to remember
> to look in subdirectories is a burden.  It is up to you to decide
> whether your motivation for keeping separate test files is more
> important than that.

There is logic in the Makefile so that "make lisp/foo-tests" or "make 
foo-tests" should
all work, but it fails with the directory option. So

make lisp/emacs-lisp/eieio ("nothing to be done")
make lisp/emacs-lisp/eieio/eieio-tests (runs test)
make lisp/emacs-lisp/eieio/eieio-tests.el (runs test)


Improving this logic so that the first runs the test would solve the 
problem. Could also
add logic so that:

make lisp/emacs-lisp

runs all the tests under emacs-lisp.

I think I put that exception in original because of eieio, and because 
some tests have resource
files associated with them. Probably having the make file support it 
better would make sense.

Phil