bug#71988: 31.0.50; ert-test-run-tests-batch-expensive runs out of memory if previous tests failed

[Pip Cet via "Bug reports for GNU Emacs, the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>]

[-- Attachment #1: Type: text/plain, Size: 1264 bytes --]

This bug report is mostly to get a bug number to put in the patch :-).

The test `ert-test-run-tests-batch-expensive' overrides print settings
and prints a full backtrace to a temporary buffer repeatedly. If a
previous test, such as `ert-test-run-tests-batch', failed, this
eventually runs out of memory even on a 64 GB system.

For example, one can add "(should nil)" to ert-test-run-tests-batch and
run:

$ make -C test lisp/emacs-lisp/ert-tests

The problem is the ert--stats structure's tests vector is being printed,
which recursively prints all previous tests' data.

A possible fix is to temporarily override cl-print-object for either
vectors or the ert--stats object to print nothing.

Unfortunately, we don't appear to have an official mechanism for
temporarily calling cl-defmethod, such as a generalized variable one can
cl-letf to. The patch does that by running cl-defmethod inside a
(cl-letf* (((symbol-function 'cl-print-object) (symbol-function
'cl-print-object))) ...) form. As future changes to cl-generic might
break that, an additional test is added to make sure our binding is
undone and die loudly if it hasn't been.

This also speeds up the test significantly and it might be possible to
remove the :unstable tag.

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-ert-tests-reproducer.patch --]
[-- Type: text/x-patch; name=0001-ert-tests-reproducer.patch, Size: 462 bytes --]

diff --git a/test/lisp/emacs-lisp/ert-tests.el b/test/lisp/emacs-lisp/ert-tests.el
index 1aff73d66f6..2f8e7382bea 100644
--- a/test/lisp/emacs-lisp/ert-tests.el
+++ b/test/lisp/emacs-lisp/ert-tests.el
@@ -583,6 +583,7 @@ ert-test-run-tests-batch
 		   (setq found-long (string-match long-text msg)))
 		 (unless found-complex
 		   (setq found-complex (string-match complex-text msg))))
+        (should nil)
 	(should found-long)
 	(should found-complex)))))
 

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #3: 0001-ert-tests.patch --]
[-- Type: text/x-patch; name=0001-ert-tests.patch, Size: 559 bytes --]

diff --git a/test/lisp/emacs-lisp/ert-tests.el b/test/lisp/emacs-lisp/ert-tests.el
index 1aff73d66f6..4cb523d2ff1 100644
--- a/test/lisp/emacs-lisp/ert-tests.el
+++ b/test/lisp/emacs-lisp/ert-tests.el
@@ -569,7 +569,7 @@ ert-test-run-tests-batch
         (save-window-excursion
           (let ((case-fold-search nil)
                 (ert-batch-backtrace-right-margin nil)
-		(ert-batch-print-level 10)
+		(ert-batch-print-level 1)
 		(ert-batch-print-length 11))
             (ert-run-tests-batch
              `(member ,failing-test-1 ,failing-test-2)))))