[PATCH] CONTRIBUTE - writing tests for understanding internals

[PATCH] CONTRIBUTE - writing tests for understanding internals

[Przemysław Wojnowski]

[-- Attachment #1: Type: text/plain, Size: 269 bytes --]

Hello everybody,

Here's a small patch that adds to CONTRIBUTE that writing tests is a
great way to learn about Emacs Internals, takes a new contributor
through committing process and is valuable to the project.

Please, check if that has been written in English. :-)


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: writing tests to understand internals --]
[-- Type: text/x-diff, Size: 1554 bytes --]

From 6d59909ba6166b0d19a2d690caec52387280e2e7 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Przemys=C5=82aw=20Wojnowski?= <esperanto@cumego.com>
Date: Tue, 10 Nov 2015 23:34:01 +0100
Subject: [PATCH] ; * CONTRIBUTE: Add testing to understanding Emacs Internals.

* CONTRIBUTE: Point that writing tests teaches about Emacs Internals and
  brings value to the project itself.
---
 CONTRIBUTE | 15 ++++++++++-----
 1 file changed, 10 insertions(+), 5 deletions(-)

diff --git a/CONTRIBUTE b/CONTRIBUTE
index 2aae251..807e7a3 100644
--- a/CONTRIBUTE
+++ b/CONTRIBUTE
@@ -264,11 +264,16 @@ top-level directory.  Most tests are in the directory
 
 ** Understanding Emacs Internals.
 
-The best way to understand Emacs Internals is to read the code,
-but the nodes "Tips" and "GNU Emacs Internals" in the Appendix
-of the Emacs Lisp Reference Manual may also help.  Some source files,
-such as xdisp.c, have large commentaries describing the design and
-implementation in more detail.
+There are at least two good ways to understand Emacs Internal.
+
+One of them is to read the code, but the nodes "Tips" and "GNU Emacs
+Internals" in the Appendix of the Emacs Lisp Reference Manual may also
+help.  Some source files, such as xdisp.c, have large commentaries
+describing the design and implementation in more detail.
+
+Another way into Emacs Internal is to write tests. This one is more
+fun, takes you through Emacs contribution process and increases
+Emacs test coverage! Don't think twice!
 
 The file etc/DEBUG describes how to debug Emacs bugs.
 
-- 
2.1.4

Re: [PATCH] CONTRIBUTE - writing tests for understanding internals

[John Wiegley]

>>>>> Przemysław Wojnowski <esperanto@cumego.com> writes:

+There are at least two good ways to understand Emacs Internal.
+
+One of them is to read the code, but the nodes "Tips" and "GNU Emacs
+Internals" in the Appendix of the Emacs Lisp Reference Manual may also
+help.  Some source files, such as xdisp.c, have large commentaries
+describing the design and implementation in more detail.
+
+Another way into Emacs Internal is to write tests. This one is more
+fun, takes you through Emacs contribution process and increases
+Emacs test coverage! Don't think twice!

Just some word smithing:

    There are at least two good ways to start getting familiar with the
    internals of Emacs:
    
    One is to read the code. Some source files, such as xdisp.c, have large
    commentaries describing the design and implementation in more detail.
    Another is to visit the nodes "Tips" and "GNU Emacs Internals" in the
    Appendix of the Emacs Lisp Reference Manual.
    
    Yet Another way is to write tests. This can be a lot of fun, introduces
    you to the Emacs contribution process, and increases Emacs test coverage!
    Don't think twice! If you want to jump in, this is a great way to start.

John

Re: [PATCH] CONTRIBUTE - writing tests for understanding internals

[Eli Zaretskii]

> From: Przemysław Wojnowski <esperanto@cumego.com>
> Date: Tue, 10 Nov 2015 23:48:41 +0100
> 
> Here's a small patch that adds to CONTRIBUTE that writing tests is a
> great way to learn about Emacs Internals, takes a new contributor
> through committing process and is valuable to the project.

Thanks, but I don't think CONTRIBUTE is the right place for
motivation-related text.  We should find some other place for that.

Re: [PATCH] CONTRIBUTE - writing tests for understanding internals

[Przemysław Wojnowski]

Eli Zaretskii <eliz@gnu.org> writes:
>> From: Przemysław Wojnowski <esperanto@cumego.com>
>> Date: Tue, 10 Nov 2015 23:48:41 +0100
>> 
>> Here's a small patch that adds to CONTRIBUTE that writing tests is a
>> great way to learn about Emacs Internals, takes a new contributor
>> through committing process and is valuable to the project.
>
> Thanks, but I don't think CONTRIBUTE is the right place for
> motivation-related text.  We should find some other place for that.

This was not for motivational purposes, but to tell how to learn about
Emacs Internals - like the old part.

Re: [PATCH] CONTRIBUTE - writing tests for understanding internals

[Andreas Röhler]

  On 11.11.2015 08:45, Przemysław Wojnowski wrote:
> Eli Zaretskii<eliz@gnu.org>  writes:
>>> From: Przemysław Wojnowski<esperanto@cumego.com>
>>> Date: Tue, 10 Nov 2015 23:48:41 +0100
>>>
>>> Here's a small patch that adds to CONTRIBUTE that writing tests is a
>>> great way to learn about Emacs Internals, takes a new contributor
>>> through committing process and is valuable to the project.
>> Thanks, but I don't think CONTRIBUTE is the right place for
>> motivation-related text.  We should find some other place for that.
> This was not for motivational purposes, but to tell how to learn about
> Emacs Internals - like the old part.
>

Hi Eli,

IMO that's a useful remark, nice idea, which may attract more testers 
and contributors alike.

Re: [PATCH] CONTRIBUTE - writing tests for understanding internals

[Eli Zaretskii]

> Date: Wed, 11 Nov 2015 09:26:44 +0100
> From: Andreas Röhler <andreas.roehler@online.de>
> CC: Przemysław Wojnowski <esperanto@cumego.com>, 
>  Eli Zaretskii <eliz@gnu.org>
> 
> IMO that's a useful remark, nice idea, which may attract more testers and contributors alike. 

I'm all for attracting new blood, I just don't think CONTRIBUTE is the
place for that.

People who are not sure whether they would like contributing to Emacs
shouldn't have to read 250 lines of boring technical information to
arrive at this text.  And people who are already contributing should
have a clear, concise, and to-the-point document which describes what
they want to know and nothing else.

Let's start a MOTIVATION file if we want to have this in our
repository.

Re: [PATCH] CONTRIBUTE - writing tests for understanding internals

[Eli Zaretskii]

> From: Przemysław Wojnowski <esperanto@cumego.com>
> Cc: emacs-devel@gnu.org
> Date: Wed, 11 Nov 2015 08:45:23 +0100
> 
> > Thanks, but I don't think CONTRIBUTE is the right place for
> > motivation-related text.  We should find some other place for that.
> 
> This was not for motivational purposes, but to tell how to learn about
> Emacs Internals - like the old part.

The style is quite different: where CONTRIBUTE is about information
for contributors, the suggested change, with the exception of its
first sentence, is a motivational text that has no informational value
for contributors.

More importantly, I disagree with your posit that writing tests is a
good way of learning about Emacs internals.  Best tests are written by
looking at the spec alone, because looking at the implementation will
bias you when you write the test.  That's why tests should ideally be
written by someone who is not the implementor.  I'm sure you are
familiar with the TDD methodology, whose strong point is precisely
that you write tests before implementing anything.

Re: [PATCH] CONTRIBUTE - writing tests for understanding internals

[Przemysław Wojnowski]

W dniu 11.11.2015 o 16:40, Eli Zaretskii pisze:
 >> From: Przemysław Wojnowski <esperanto@cumego.com>
 >> Cc: emacs-devel@gnu.org
 >> Date: Wed, 11 Nov 2015 08:45:23 +0100
 >>
 >>> Thanks, but I don't think CONTRIBUTE is the right place for
 >>> motivation-related text.  We should find some other place for that.
 >>
 >> This was not for motivational purposes, but to tell how to learn about
 >> Emacs Internals - like the old part.
 >
 > The style is quite different: where CONTRIBUTE is about information
 > for contributors, the suggested change, with the exception of its
 > first sentence, is a motivational text that has no informational value
 > for contributors.
I tend to agree that my style of writing is different than the rest of
the document and is a place to improve. That's why I've sent it here.

 > More importantly, I disagree with your posit that writing tests is a
 > good way of learning about Emacs internals.
Did you write this based on your experience? I did.

It *is* very good way to learn about a project and, in many companies,
is used to introduce new developers. (I do that every time and after
sometime devs tell me that it is a good introduction, because project
at the beginning were to big and overwhelming to them.)

It's very easy to verify this by writing a few tests to a project you
don't know.

 > Best tests are written by looking at the spec alone,
 > because looking at the implementation will bias you when
 > you write the test.  That's why tests should ideally be
 > written by someone who is not the implementor.
Ideally yes. And new contributors are very close to this ideal, because
they didn't implement the code and have docs as specs.

I submitted a couple of tests to Emacs packages and was writing them
based on available docs (even CLTL2, in case of cl-lib) and I've
learned a lot about those parts.

 >  I'm sure you are
 > familiar with the TDD methodology, whose strong point is precisely
 > that you write tests before implementing anything.
"Reality check:" we already have code base. ;-)

Cheers,
Przemysław

Re: [PATCH] CONTRIBUTE - writing tests for understanding internals

[Eli Zaretskii]

> From: Przemysław Wojnowski <esperanto@cumego.com>
> Date: Wed, 11 Nov 2015 22:08:49 +0100
> Cc: emacs-devel@gnu.org
> 
>  > The style is quite different: where CONTRIBUTE is about information
>  > for contributors, the suggested change, with the exception of its
>  > first sentence, is a motivational text that has no informational value
>  > for contributors.
> I tend to agree that my style of writing is different than the rest of
> the document and is a place to improve. That's why I've sent it here.

I didn't object to your posting it here.  You asked for a review, and
I tried to provide it.

>  > More importantly, I disagree with your posit that writing tests is a
>  > good way of learning about Emacs internals.
> Did you write this based on your experience?

Yes, of course.  Why do you ask?

> It *is* very good way to learn about a project and, in many companies,
> is used to introduce new developers. (I do that every time and after
> sometime devs tell me that it is a good introduction, because project
> at the beginning were to big and overwhelming to them.)

Then I guess we will have to agree to disagree about this.

>  > Best tests are written by looking at the spec alone,
>  > because looking at the implementation will bias you when
>  > you write the test.  That's why tests should ideally be
>  > written by someone who is not the implementor.
> Ideally yes. And new contributors are very close to this ideal, because
> they didn't implement the code and have docs as specs.

If they go by the spec and the docs, their tests will be very good, I
agree.  But that's not an efficient way of learning the internals: for
that, you need to study the implementation, not the docs and the
requirements.

>  >  I'm sure you are
>  > familiar with the TDD methodology, whose strong point is precisely
>  > that you write tests before implementing anything.
> "Reality check:" we already have code base. ;-)

How does this argument help in advancing this discussion?

Re: [PATCH] CONTRIBUTE - writing tests for understanding internals

[John Wiegley]

>>>>> Eli Zaretskii <eliz@gnu.org> writes:

>> It *is* very good way to learn about a project and, in many companies, is
>> used to introduce new developers. (I do that every time and after sometime
>> devs tell me that it is a good introduction, because project at the
>> beginning were to big and overwhelming to them.)

> Then I guess we will have to agree to disagree about this.

I think what works for us individually may or may not work for others. I know
some people find working on tests to be an excellent way to introduce
themselves to a code base; others don't. Sometimes writing tests while reading
about internals leads to tests that have little value, because they simply
repeat what the implementation says. Other times, the act of trying to
understand enough about the internals to write that test leads to valuable
insights.

Rather than agreeing or disagreeing, let's say that there is value here, we
just don't who it will attract and why. I'm OK with encouraging people to
write tests. Anything that invites them to participate more cannot really
hurt, can it?

John

Re: [PATCH] CONTRIBUTE - writing tests for understanding internals

[Eli Zaretskii]

> From: John Wiegley <jwiegley@gmail.com>
> Cc: Przemysław Wojnowski <esperanto@cumego.com>,
>   emacs-devel@gnu.org
> Date: Wed, 11 Nov 2015 13:36:19 -0800
> 
> Rather than agreeing or disagreeing, let's say that there is value here, we
> just don't who it will attract and why. I'm OK with encouraging people to
> write tests. Anything that invites them to participate more cannot really
> hurt, can it?

Of course.  But we already have that, thanks to a recent change by
Karl:

  ** Test your changes.

  Please test your changes before committing them or sending them to the
  list.  If possible, add a new test along with any bug fix or new
  functionality you commit (of course, some changes cannot be easily
  tested).

  Emacs uses ERT, Emacs Lisp Regression Testing, for testing.  See (info
  "(ert)") or https://www.gnu.org/software/emacs/manual/html_node/ert/
  for more information on writing and running tests.

Re: [PATCH] CONTRIBUTE - writing tests for understanding internals

[Przemysław Wojnowski]

W dniu 11.11.2015 o 22:26, Eli Zaretskii pisze:
>>   >  I'm sure you are
>>   > familiar with the TDD methodology, whose strong point is precisely
>>   > that you write tests before implementing anything.
>> "Reality check:" we already have code base. ;-)
>
> How does this argument help in advancing this discussion?

This was impolite, sorry.

My point here was that we already have existing code, so we cannot apply 
strict Red-Green-Refactor. But "Green-Refactor" we can.

Re: [PATCH] CONTRIBUTE - writing tests for understanding internals

[Andreas Röhler]

  On 11.11.2015 16:40, Eli Zaretskii wrote:
> [ ... ]
> written by someone who is not the implementor.  I'm sure you are
> familiar with the TDD methodology, whose strong point is precisely
> that you write tests before implementing anything.
>
>


Hmm, looks for me like TDD established a kind of religion too.
How to write tests without knowing the types?
Isn't there the danger of writing a lot of dead wood?

Re: [PATCH] CONTRIBUTE - writing tests for understanding internals

[Eli Zaretskii]

> Date: Thu, 12 Nov 2015 08:16:47 +0100
> From: Andreas Röhler <andreas.roehler@online.de>
> CC: Eli Zaretskii <eliz@gnu.org>
> 
>   On 11.11.2015 16:40, Eli Zaretskii wrote:
> > [ ... ]
> > written by someone who is not the implementor.  I'm sure you are
> > familiar with the TDD methodology, whose strong point is precisely
> > that you write tests before implementing anything.
> 
> Hmm, looks for me like TDD established a kind of religion too.
> How to write tests without knowing the types?

The API should be defined when you write the tests, but the
implementation doesn't yet exist.

Re: [PATCH] CONTRIBUTE - writing tests for understanding internals

[Stephen Leake]

Eli Zaretskii <eliz@gnu.org> writes:

>> Date: Thu, 12 Nov 2015 08:16:47 +0100
>> From: Andreas Röhler <andreas.roehler@online.de>
>> CC: Eli Zaretskii <eliz@gnu.org>
>> 
>>   On 11.11.2015 16:40, Eli Zaretskii wrote:
>> > [ ... ]
>> > written by someone who is not the implementor.  I'm sure you are
>> > familiar with the TDD methodology, whose strong point is precisely
>> > that you write tests before implementing anything.
>> 
>> Hmm, looks for me like TDD established a kind of religion too.
>> How to write tests without knowing the types?
>
> The API should be defined when you write the tests, but the
> implementation doesn't yet exist.

I use this style of development myself.

In reality, after you get started, the tests evolve along with the code.
But for every change, you change the tests first.

Often, after a change is implemented, it affects other tests that you
did not change. Then you have to decide whether that is ok, and fix the
tests to match, or if the code needs to be changed more to preserve the
previous behavior.

I have lots of commits of tests in my NASA code that say "match code
change".

-- 
-- Stephe

Re: [PATCH] CONTRIBUTE - writing tests for understanding internals

[Andreas Röhler]

  On 12.11.2015 21:03, Stephen Leake wrote:
> Eli Zaretskii<eliz@gnu.org>  writes:
>
>>> Date: Thu, 12 Nov 2015 08:16:47 +0100
>>> From: Andreas Röhler<andreas.roehler@online.de>
>>> CC: Eli Zaretskii<eliz@gnu.org>
>>>
>>>    On 11.11.2015 16:40, Eli Zaretskii wrote:
>>>> [ ... ]
>>>> written by someone who is not the implementor.  I'm sure you are
>>>> familiar with the TDD methodology, whose strong point is precisely
>>>> that you write tests before implementing anything.
>>> Hmm, looks for me like TDD established a kind of religion too.
>>> How to write tests without knowing the types?
>> The API should be defined when you write the tests, but the
>> implementation doesn't yet exist.
> I use this style of development myself.
>
> In reality, after you get started, the tests evolve along with the code.
> But for every change, you change the tests first.
>
> Often, after a change is implemented, it affects other tests that you
> did not change. Then you have to decide whether that is ok, and fix the
> tests to match, or if the code needs to be changed more to preserve the
> previous behavior.
>
> I have lots of commits of tests in my NASA code that say "match code
> change".
>

Interesting.

After all, might it be more effective to do everything with tests in 
mind, but be carefully, kind of lazy, when investing into?

Well, there is still another thing with TDD: it's irrational by its 
definition. Tests drive nothing, whilst their results come from behind.
Tests luck direction and decision, which must precede them.

Re: [PATCH] CONTRIBUTE - writing tests for understanding internals

[Eli Zaretskii]

> Date: Thu, 12 Nov 2015 21:40:49 +0100
> From: Andreas Röhler <andreas.roehler@online.de>
> CC: Eli Zaretskii <eliz@gnu.org>, emacs-devel@gnu.org
> 
> Well, there is still another thing with TDD: it's irrational by its 
> definition. Tests drive nothing

In TDD, they teach you to write a test for a spec that isn't
implemented yet.  The test fails, of course (which is a Good Thing:
now you know that your test indeed will catch a non-compliant
implementation), and then you implement the spec to see that it now
succeeds.  If you work that way, tests _do_ drive the development.

Re: [PATCH] CONTRIBUTE - writing tests for understanding internals

[Przemysław Wojnowski]

W dniu 12.11.2015 o 21:55, Eli Zaretskii pisze:
> In TDD, they teach you to write a test for a spec that isn't
> implemented yet.  The test fails, of course (which is a Good Thing:
> now you know that your test indeed will catch a non-compliant
> implementation), and then you implement the spec to see that it now
> succeeds.  If you work that way, tests _do_ drive the development.

What do you mean by "spec" here?

I've never seen nor used TDD (and I use it very often) with "spec".
What I've seen, though, was using TDD to implement User Stories (Use
Cases), which are descriptions of functionality, but far from formal
specifications - if this is what you mean.

Anyway, TDD drives design (you implement only as much as is needed to
make the tests pass) and, by side effect, gives strong regression test
suite, which _enables_ refactoring.
Many TDD practitioners say that its result is good API.

Re: [PATCH] CONTRIBUTE - writing tests for understanding internals

[Eli Zaretskii]

> Cc: emacs-devel@gnu.org
> From: Przemysław Wojnowski <esperanto@cumego.com>
> Date: Thu, 12 Nov 2015 22:42:43 +0100
> 
> W dniu 12.11.2015 o 21:55, Eli Zaretskii pisze:
> > In TDD, they teach you to write a test for a spec that isn't
> > implemented yet.  The test fails, of course (which is a Good Thing:
> > now you know that your test indeed will catch a non-compliant
> > implementation), and then you implement the spec to see that it now
> > succeeds.  If you work that way, tests _do_ drive the development.
> 
> What do you mean by "spec" here?

Sorry for using jargon here.  By "spec" I meant the specification of
the module, i.e. the APIs of its methods, and the set of requirements
that define what should be the result(s) of invoking the various
methods with the various possible combinations of arguments and given
the relevant environmental conditions.

A simple example of a "spec" for a single function is what you see
here:

  http://pubs.opengroup.org/onlinepubs/009695399/functions/stat.html

> I've never seen nor used TDD (and I use it very often) with "spec".
> What I've seen, though, was using TDD to implement User Stories (Use
> Cases), which are descriptions of functionality, but far from formal
> specifications - if this is what you mean.

A spec in the terminology I used is the set of formal requirements
produced by analyzing those use cases.

For the purposes of this discussion, I think it's immaterial whether
the "spec" is formal or not.  What's important is that you can
unequivocally determine what the code should do, and write the tests
which check that.

> Anyway, TDD drives design (you implement only as much as is needed to
> make the tests pass) and, by side effect, gives strong regression test
> suite, which _enables_ refactoring.

Indeed.

Re: [PATCH] CONTRIBUTE - writing tests for understanding internals

[Andreas Röhler]

  On 13.11.2015 08:33, Eli Zaretskii wrote:
>> Cc: emacs-devel@gnu.org
>> From: Przemysław Wojnowski<esperanto@cumego.com>
>> Date: Thu, 12 Nov 2015 22:42:43 +0100
>>
>> W dniu 12.11.2015 o 21:55, Eli Zaretskii pisze:
>>> In TDD, they teach you to write a test for a spec that isn't
>>> implemented yet.  The test fails, of course (which is a Good Thing:
>>> now you know that your test indeed will catch a non-compliant
>>> implementation), and then you implement the spec to see that it now
>>> succeeds.  If you work that way, tests _do_ drive the development.
>> What do you mean by "spec" here?
> Sorry for using jargon here.  By "spec" I meant the specification of
> the module, i.e. the APIs of its methods, and the set of requirements
> that define what should be the result(s) of invoking the various
> methods with the various possible combinations of arguments and given
> the relevant environmental conditions.
>
> A simple example of a "spec" for a single function is what you see
> here:
>
>    http://pubs.opengroup.org/onlinepubs/009695399/functions/stat.html
>
>> I've never seen nor used TDD (and I use it very often) with "spec".
>> What I've seen, though, was using TDD to implement User Stories (Use
>> Cases), which are descriptions of functionality, but far from formal
>> specifications - if this is what you mean.
> A spec in the terminology I used is the set of formal requirements
> produced by analyzing those use cases.
>
> For the purposes of this discussion, I think it's immaterial whether
> the "spec" is formal or not.  What's important is that you can
> unequivocally determine what the code should do, and write the tests
> which check that.
>
>> Anyway, TDD drives design (you implement only as much as is needed to
>> make the tests pass) and, by side effect, gives strong regression test
>> suite, which _enables_ refactoring.
> Indeed.
>
>

Agreed WRT mentioned side effect - and its importance.

Cheers,

Andreas