bug#71988: 31.0.50; ert-test-run-tests-batch-expensive runs out of memory if previous tests failed

[Eli Zaretskii <eliz@gnu.org>]

> Date: Mon, 08 Jul 2024 04:49:54 +0000
> From:  Pip Cet via "Bug reports for GNU Emacs,
>  the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>
> 
> This bug report is mostly to get a bug number to put in the patch :-).
> 
> The test `ert-test-run-tests-batch-expensive' overrides print settings
> and prints a full backtrace to a temporary buffer repeatedly. If a
> previous test, such as `ert-test-run-tests-batch', failed, this
> eventually runs out of memory even on a 64 GB system.
> 
> For example, one can add "(should nil)" to ert-test-run-tests-batch and
> run:
> 
> $ make -C test lisp/emacs-lisp/ert-tests
> 
> The problem is the ert--stats structure's tests vector is being printed,
> which recursively prints all previous tests' data.
> 
> A possible fix is to temporarily override cl-print-object for either
> vectors or the ert--stats object to print nothing.
> 
> Unfortunately, we don't appear to have an official mechanism for
> temporarily calling cl-defmethod, such as a generalized variable one can
> cl-letf to. The patch does that by running cl-defmethod inside a
> (cl-letf* (((symbol-function 'cl-print-object) (symbol-function
> 'cl-print-object))) ...) form. As future changes to cl-generic might
> break that, an additional test is added to make sure our binding is
> undone and die loudly if it hasn't been.
> 
> This also speeds up the test significantly and it might be possible to
> remove the :unstable tag.
> 
> 
> diff --git a/test/lisp/emacs-lisp/ert-tests.el b/test/lisp/emacs-lisp/ert-tests.el
> index 1aff73d66f6..2f8e7382bea 100644
> --- a/test/lisp/emacs-lisp/ert-tests.el
> +++ b/test/lisp/emacs-lisp/ert-tests.el
> @@ -583,6 +583,7 @@ ert-test-run-tests-batch
>  		   (setq found-long (string-match long-text msg)))
>  		 (unless found-complex
>  		   (setq found-complex (string-match complex-text msg))))
> +        (should nil)
>  	(should found-long)
>  	(should found-complex)))))

Stefan, any comments or suggestions?