bug#53313: [ERT] EMACS_TEST_VERBOSE can not be explicitly disabled

bug#53313: [ERT] EMACS_TEST_VERBOSE can not be explicitly disabled

[Max Nikulin]

At least empty value of EMACS_TEST_VERBOSE environment variable
should be considered as false value that disables verbose ERT summary.

Currently ERT implementation does not distinguish non-empty and empty
values of the variable and generates verbose summary when such
variable is just present in the process environment.

To improve user experience some strings widely used in configuration 
files as false values should be recognized by ERT as well. Examples 
(case insensitive): "0", "false", "N", "no", "off", "none", "nil".

Recently Ihor Radchenko proposed to enable verbose summary mode by 
default for Org Mode tests when it is built as a standalone project 
outside of Emacs source tree:
https://lists.gnu.org/archive/html/emacs-orgmode/2022-01/msg00022.html
I like the idea, but I can not say the same concerning implementation. 
Ihor introduced new make variable while I believe that it is not 
necessary and Org makefiles should be transparent in relation to 
EMACS_TEST_VERBOSE.

I expect that it should be enough to add
     EMACS_TEST_VERBOSE ?= yes
     export EMACS_TEST_VERBOSE
to the default.mk file
https://git.savannah.gnu.org/cgit/emacs/org-mode.git/tree/mk/default.mk

Actually with current implementation developer has no chance to disable
verbose summary mode from command line since empty string is considered
as true value by ERT implementation. The following does not work:

     EMACS_TEST_VERBOSE= make test
     make test EMACS_TEST_VERBOSE=

A workaround is "unexport EMACS_TEST_VERBOSE" either in the local.mk
file or wrapped with ifeq-else-endif in default.mk.

Another case is that a developer may have
"export EMACS_TEST_VERBOSE=yes" in their shell init file
but sometimes wants to disable verbose summary for a couple of commands
without changing of environment by "unset EMACS_TEST_VERBOSE"
that will affect following commands. The following commands
should cause different states of verbose summary setting
independently of current environment:

     make test EMACS_TEST_VERBOSE=
     make test EMACS_TEST_VERBOSE=yes

Standalone example: ert-sample.el
---- >8 ----
(require 'ert)
(ert-deftest ert-sample ()
   (should (equal 5 (* 2 2))))
(ert-run-tests-batch-and-exit)
---- 8< ----

     export EMACS_TEST_VERBOSE=yes  # in ~/.profile, ~/.bashrc, etc.
     EMACS_TEST_VERBOSE= emacs --batch -Q ert-sample.el

I have not found any discussion of design choices related to this variable.

To wrap up, I suppose that at least empty value of EMACS_TEST_VERBOSE
should be considered as a request to generate brief summary.
It would be great to have opportunity to explicitly say "no":
EMACS_TEST_VERBOSE=off (or "false", "0", etc.).
It will make it easier for various build tools to respect
developer's choice.

bug#53313: [ERT] EMACS_TEST_VERBOSE can not be explicitly disabled

[Lars Ingebrigtsen]

Max Nikulin <manikulin@gmail.com> writes:

> At least empty value of EMACS_TEST_VERBOSE environment variable
> should be considered as false value that disables verbose ERT summary.

I've now fixed this in Emacs 29.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#53313: [ERT] EMACS_TEST_VERBOSE can not be explicitly disabled

[Max Nikulin]

On 21/01/2022 18:32, Lars Ingebrigtsen wrote:
> Max Nikulin writes:
> 
>> At least empty value of EMACS_TEST_VERBOSE environment variable
>> should be considered as false value that disables verbose ERT summary.
> 
> I've now fixed this in Emacs 29.

Thank you, Lars. It works accordingly may expectation in its minimal 
variant.

Just to have it clearly stated in the bug tracker: strings like "0", 
"no", etc. are considered as true value as any other non-empty strings. 
Only unset variable or empty value disables verbose summary.

Commit 4d866fc0f5