Re: Perl and Emacs: Developing tests for progmodes

[haj@posteo.de (Harald Jörg)]

Stefan Kangas <stefankangas@gmail.com> writes:

> Stefan Kangas <stefankangas@gmail.com> writes:
>
>> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>>
>>> I think for now we can tag the tests individually with
>>>
>>>     (ert-deftest (...)
>>>       (skip-unless (eq cperl-test-mode #'cperl-mode))
>>>       ...)
>>
>> How about something like the attached to make the tests run
>> automatically also for perl-mode?  It's a bit of a hack, admittedly.
>
> This never got a reply.  Is the idea reasonable, or should it just be
> scrapped?  I personally don't have a strong opinion, but I didn't want to
> throw it away without at least asking first.

Stefan (Monnier) already committed this as a patch (7a7847d7ad5;
Fri Aug 14) :)

> (It is sometimes hard for me to understand if no reply means "sure, go
> ahead" or "this is too dumb to even comment on"...  ;-)
>
> Of course, it will introduce some potential friction in developing new
> cperl-mode tests, but on the other hand perl-mode.el will see some
> automated testing.

Yes, there is some friction, but as the test suite is still very small.
It can easily be split into two files whenever this seems appropriate.
I don't have a strong opinion either.

In any case, it makes sense to share resources (Perl sources), providing
a nice edge case for the new ert-resource-file function. :)

Here's the experience so far:

Together with his suggestion above, Stefan (M.) rewrote one of the tests
so that it would work with both modes.  It turned out that perl-mode
showed the same bug, which he fixed.  So this was one of the cases where
a common test absolutely makes sense.

Tests which are specific for a function which only exists in cperl-mode
make no sense with perl-mode.  We have a few of those tests, and right
now I found one where I forgot to skip it when testing perl-mode.

Things get murky for tests where cperl-mode intentionally behaves
differently than perl-mode: A test makes sense, but the
"should"-condition needs to be different.  There's more friction in
those tests (Example: the modes populate the "imenu"-Index quite
differently).

My recent addition to the tests (fb26dc13: cperl-mode: Delete a
misleading comment, add tests for verification; Mon Oct 19) shows
another issue: The test (fontification of \$") makes sense for both
cperl-mode and perl-mode.  In cperl-mode, the fontification was fixed in
1997 (I think), in perl-mode it is still wrong.  I'm not deep enough in
either of the modes (yet) to figure out what to do to fix this in
perl-mode, so I decided to only run the test for cperl-mode.

Finally, perl-mode comes with its own list of ancient open bugs, many of
those don't occur in cperl-mode.  I wonder whether the authors of these
bugs would accept "use cperl-mode instead" as a workaround?

At some time, it might make sense to merge those two modes into one.
Perl continues to evolve, and upgrading two modes to support that
doesn't seem to be an economic use of time.
--
Cheers,
haj