Re: master 12ca463 1/4: ; * test/lisp/textmodes/css-mode-tests.el: Add TODO.

[Dmitry Gutov <dgutov@yandex.ru>]

On 21.09.2020 21:23, Stefan Kangas wrote:

> I was working under the assumption that these tests existed in their
> current form only for historical reasons.  So here I am, wreaking
> general havoc.

As was I, at a certain time in the past.

>> I'm asking as a developer from started with the latter and then switched
>> mostly to the former, due to the ease of debugging.
>>
>> The manual/indent tests also have an automated runner, which uses the
>> 'diff' program.
> 
> I think that the manual tests fail on at least two accounts:
> 
>    1. There is no automated verification of success.

They can even work semi-automatically, try:

$ cd test/manual/indent

$ make ruby.rb.test
../../../src/emacs --batch ruby.rb \
     --eval '(indent-region (point-min) (point-max) nil)' \
     --eval '(write-region (point-min) (point-max) "ruby.rb.new")'
Indenting region...
Indenting region...done
diff -u -B ruby.rb ruby.rb.new

Unfortunately, 'make all' doesn't work as expected there, only running 
that test for octave.m (which currently fails, to boot).

>    2. They don't run with "make check".

I think it can be made to run the above tests.

Wish someone with more Makefile-related experience than myself, as well 
as free time, could look into it.

> That said, please feel free to revert any commits you disagree with or
> that gets in the way of your work.  I think it goes without saying that
> I don't want to make debugging harder for you or anyone else.

I guess unless I do the above, I don't have a moral standing to revert them?

Having an ERT test per each file in there might seem a tad redundant 
given the above option, but maybe there are good reasons why nobody has 
fixed the situations yet. As long as said manual test files are present, 
in one form or another, the debugging workflows can be retained.

> But if there are limitations in debugging with ert, perhaps we should
> work on improving its capabilities?

It's capable enough in that regard (e.g. one can press 'd' and drop into 
the debugger, in the target buffer, exactly at the point of failure), 
yet still it requires more ceremony than the "manual" approach.