bug#15991: 24.3.50; Wishlist: Make test suite user-friendlier

[Michael Albinus <michael.albinus@gmx.de>]

Glenn Morris <rgm@gnu.org> writes:

>> I believe it is desirable to run the tests unconditionally when invoked
>> via make. People call "make check" for a reason.
>
> I suppose you are right, and this is consistent with how it used to behave.

Nope. When you cal "make check" a second time, no test runs again,
'cause the log files exist already.

>> Maybe you could even offer the test package name as target. Then one
>> could call "make -C test/automated tramp-tests",
>
> You can already do
>
>    make -C test/automated tramp-tests.log

Again, it works only when there is no log file. The second time you
call it, nothing happens:

~/src/emacs> make -C test/automated tramp-tests.log
make: Entering directory `/usr/local/src/emacs/test/automated'
make: `tramp-tests.log' is up to date.
make: Leaving directory `/usr/local/src/emacs/test/automated'

> I don't think removing the ".log" is worth it.

If you want to rerun a test, you must remove the log file
manually. Inconvenient.

>> There is another issue. By redirecting the output to a file, the
>> password prompt for tramp-tests isn't visible anymore. This is a real
>> regression, because these tests are intended to run for different remote
>> file names. Could you pipe the test output through tee or something like
>> this? I don't know how portable this would be, 'tho.
>
> Not sure I understand. Do you want to interact, ie type a password?
> I think tests that require interaction are a Bad Thing.
> Are these optional tests that do not normally run?

Nope. See the commentary section of tramp-test.el. Try

~/src/emacs> env REMOTE_TEMPORARY_FILE_DIRECTORY=/sudo::/tmp make -C test/automated tramp-tests.log

It will ask you for the sudo password, but you don't see the prompt.

Best regards, Michael.