Re: call for more ert tests

[Eli Zaretskii <eliz@gnu.org>]

> From: Lars Magne Ingebrigtsen <larsi@gnus.org>
> Date: Tue, 25 Jun 2013 13:06:18 +0200
> 
> > Eli Zaretskii <eliz@gnu.org> writes:
> >>
> >> I believe it's good, obviously.  The problem is with introducing it
> >> without losing too many contributors.
> 
> I think the bar to contributing to Emacs is high enough as it is without
> adding further requirements.

I don't know why you are saying this.  Please elaborate by comparing
with other projects whose bar is significantly lower.

For me, the single most important issue for contributing is how easy
or hard is it to find the reason for the problem(s) that cause my
itch, and how easy or hard is it to make a change that I can convince
myself is right.  I can tell you that in my experience, Emacs is not
much harder in this regard than other projects, like GDB or Guile (to
pick just 2 random examples).

In any case, I didn't say that we actually should require tests as
part of any contribution.  All I said was that if we don't, the
chances of us getting anywhere with the test suite are extremely low.
In fact, I believe that only an appearance of a very dedicated person
who would do the job is the only alternative to a strict policy of
requiring tests with each changeset.  How probable is that such an
alternative materializes any time soon, in your opinion?

> ert is fine, but, I think, somewhat misguided.  It allows us to test the
> functions we have Lisp interfaces for, but not the deep internal C
> bits.  And that's kinda backward.
> 
> When I write Lisp code, I'm testing it interactively all the time.  What
> should this function return?  Does it return what I'm expecting?  No?
> *hack hack*  Now?  Yes.  Done.

I agree that testing C-level internals is a must.  But most of Emacs
is Lisp code, and if you examine the bug reports, I think you will
find that most of them are about Lisp code.  So testing Lisp is also
important.  It's not just the API that is being tested; it's the
behavior under different inputs and user options; in short, how well
the API keeps its side of the contract.