From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Michael Newsgroups: gmane.emacs.bugs Subject: bug#51037: [PATCH] Make `print-level` & `print-length` customizable in ERT batch tests Date: Sun, 14 Nov 2021 07:42:27 -0800 Message-ID: <86tugen3q4.fsf@runbox.com> References: <86tuhvmtpl.fsf@runbox.com> <877deqec10.fsf@gnus.org> <83fste2u4i.fsf@gnu.org> <87ily94662.fsf@gnus.org> <83fstd1bth.fsf@gnu.org> <87czogslm4.fsf@gnus.org> <86ily2mjtq.fsf@runbox.com> <87r1cp89o4.fsf@gnus.org> <86fst5m48v.fsf@runbox.com> <87a6jd5891.fsf@gnus.org> <86mtmykxro.fsf@runbox.com> <87y26hz24c.fsf@gnus.org> <86bl3blcgm.fsf@runbox.com> <87wnlya9sj.fsf@gnus.org> <86wnlbmooj.fsf@runbox.com> <83bl2nw723.fsf@gnu.org> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="24327"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: mu4e 1.4.15; emacs 28.0.50 Cc: gazally@runbox.com, larsi@gnus.org, 51037@debbugs.gnu.org To: Eli Zaretskii Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Sun Nov 14 16:43:13 2021 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1mmHex-00065c-HU for geb-bug-gnu-emacs@m.gmane-mx.org; Sun, 14 Nov 2021 16:43:12 +0100 Original-Received: from localhost ([::1]:52986 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mmHew-0003PN-2s for geb-bug-gnu-emacs@m.gmane-mx.org; Sun, 14 Nov 2021 10:43:10 -0500 Original-Received: from eggs.gnu.org ([209.51.188.92]:59396) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mmHeo-0003NA-UZ for bug-gnu-emacs@gnu.org; Sun, 14 Nov 2021 10:43:02 -0500 Original-Received: from debbugs.gnu.org ([209.51.188.43]:39833) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mmHeo-0005bk-L1 for bug-gnu-emacs@gnu.org; Sun, 14 Nov 2021 10:43:02 -0500 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1mmHeo-0003Q2-DN for bug-gnu-emacs@gnu.org; Sun, 14 Nov 2021 10:43:02 -0500 X-Loop: help-debbugs@gnu.org Resent-From: Michael Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sun, 14 Nov 2021 15:43:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 51037 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: patch Original-Received: via spool by 51037-submit@debbugs.gnu.org id=B51037.163690457813132 (code B ref 51037); Sun, 14 Nov 2021 15:43:02 +0000 Original-Received: (at 51037) by debbugs.gnu.org; 14 Nov 2021 15:42:58 +0000 Original-Received: from localhost ([127.0.0.1]:51379 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mmHej-0003Pj-Jp for submit@debbugs.gnu.org; Sun, 14 Nov 2021 10:42:58 -0500 Original-Received: from transit01.runbox.com ([91.220.196.211]:54248) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mmHeg-0003PS-G5 for 51037@debbugs.gnu.org; Sun, 14 Nov 2021 10:42:56 -0500 Original-Received: from aibo.runbox.com ([185.226.149.25]) by transit01.runbox.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.93) (envelope-from ) id 1mmHeZ-0071cF-1j; Sun, 14 Nov 2021 16:42:47 +0100 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=runbox.com; s=selector1; h=Content-Type:MIME-Version:Message-ID:Date:In-reply-to:Subject :Cc:To:From:References; bh=KVHLty3RYt2UUAAF59pXY/rAekLICPkWnAMKI/O0Ypo=; b=gi JLdd2hZFOoTag4ttp1sAFTYDH2XO1BxYKzM/quEHcv7oNl+y96FkCrlVbTq28oQEvrDoUP55t2jTc N0X96lUdESfQQH5Vnkw2PqYdaMYBrS2PB6DT4TNjOffbQEcn7gfRP2AoISdXmthXyQBaUiWreqldk i/bgB2e5UJ998FSTQTtlu7nEknVaSOYRVuvX24PWKM2t9hhU/K+LO4S5zzUPDhHa5dVQwb2sO5SQB VfaWGxI5c+/Y+wiXS/7CVZJz+uDqgyUSwr7KTG+lBRnQpPVlyUUTI5SjTxe6XOEgvkSO/dXCXATHQ QsetZ/syJSBI/5k85691YmQn/azMYyEQ==; Original-Received: from [10.9.9.72] (helo=submission01.runbox) by mailtransmit02.runbox with esmtp (Exim 4.86_2) (envelope-from ) id 1mmHeX-0004nG-9r; Sun, 14 Nov 2021 16:42:45 +0100 Original-Received: by submission01.runbox with esmtpsa [Authenticated ID (942723)] (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) id 1mmHeJ-0001Mt-HW; Sun, 14 Nov 2021 16:42:32 +0100 In-reply-to: <83bl2nw723.fsf@gnu.org> X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.io gmane.emacs.bugs:219959 Archived-At: --=-=-= Content-Type: text/plain; format=flowed >> From: Michael >> Date: Sat, 13 Nov 2021 18:55:08 -0800 >> Cc: gazally@runbox.com, 51037@debbugs.gnu.org >> >> OK-- here's the revised patch. > > Thanks. > > The patch includes some remnant of a merge conflict: > >> ++++ >> +** New ERT batch variables 'ert-batch-print-length' & >> 'ert-batch-print-level' >> +These variables will override 'print-length' & 'print-level' >> when >> +printing Lisp values in ERT batch test results. >> + >> +** Emacs now supports Unicode Standard version 14.0. >> +>>>>>>> 129b2425c79 (Make print-length & print-level in >> ert-run-tests-batch configurable) > > Also, please try to keep the lines in the log message shorter > than 65 > characters. That's embarassing. Both fixed, and commit message & NEWS entry tidied up, altogether. --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=0001-Make-results-details-in-ert-run-tests-batch-configur.patch Content-Description: Make `print-level` & `print-length` configurable in ERT batch tests >From 71ba49d5c8822430025e2921d0d4376d9962ae8a Mon Sep 17 00:00:00 2001 From: Michael Herstine Date: Tue, 5 Oct 2021 06:53:02 -0700 Subject: [PATCH] Make results details in ert-run-tests-batch configurable. This commit introduces three new ert variables: ert-batch-print-length, ert-batch-print-level & ert-batch-backtrace-length. The first two control print-length and print-level, resp., when printing test results. Since even modest values of print-level & print-length can produce extremely long lines in backtraces along with correspondingly long processing times to format them, this commit also introduces the variable ert-backtrace-line-length which can, optionally, override backtrace-line-length during ert batch test stack traces. Finally, it also removes the optional message-fn & output-buffer arguments to ert-run-test-interactively (since they were only used for testing purposes) and re-implements its tests in terms of cl-letf. * lisp/emacs-lisp/ert.el (ert-batch-print-length, ert-batch-print-level,.ert-batch-backtrace-line-length, ert-batch-test, ert-run-tests-interactively): Added the three variables, bound them to these settings when formatting batch test results including backtraces. Removed the optional parameters output-buffer & message-fn from ert-run-tests-interactively. * test/lisp/emacs-lisp/ert-tests.el (ert-test-run-tests-interactively, ert-test-run-tests-batch): use cl-letf to capture output, new tests resp. * test/lisp/ert-x-tests.el (ert-test-run-tests-interactively-2): Changed to use cl-letf to capture output instead of using message-fn. * lisp/emacs-lisp/backtrace.el (backtrace--line-length-or-nil, backtrace--print-func-and-args): Fixed a bug when setting backtrace-line-length to nil by adding a new function to check for that case & having backtrace--print-func-and-args use it. * doc/misc/ert.texi: document the new variables & their usage --- doc/misc/ert.texi | 25 +++++++++ etc/NEWS | 11 ++++ lisp/emacs-lisp/backtrace.el | 21 +++++--- lisp/emacs-lisp/ert.el | 82 +++++++++++++++++++---------- test/lisp/emacs-lisp/ert-tests.el | 81 ++++++++++++++++++++++++---- test/lisp/emacs-lisp/ert-x-tests.el | 41 +++++++-------- 6 files changed, 196 insertions(+), 65 deletions(-) diff --git a/doc/misc/ert.texi b/doc/misc/ert.texi index 440c61add8e..522b09dd617 100644 --- a/doc/misc/ert.texi +++ b/doc/misc/ert.texi @@ -390,6 +390,31 @@ Running Tests in Batch Mode emacs -batch -l ert -f ert-summarize-tests-batch-and-exit output.log @end example +@vindex ert-batch-print-level +@vindex ert-batch-print-length +ERT attempts to limit the output size for failed tests by choosing +conservative values for @code{print-level} & @code{print-length} +when printing Lisp values. This can in some cases make it difficult +to see which portions of those values are incorrect. Use +@code{ert-batch-print-level} and @code{ert-batch-print-length} +to customize that: + +@example +emacs -batch -l ert -l my-tests.el \ + --eval "(let ((ert-batch-print-level 10) \ + (ert-batch-print-length 120)) \ + (ert-run-tests-batch-and-exit))" +@end example + +@vindex ert-batch-backtrace-line-length +Even modest settings for @code{print-level} & @code{print-length} can +produce extremely long lines in backtraces, however, with attendant +pauses in execution progress. Set +@code{ert-batch-backtrace-line-length} to t to use the value of +@code{backtrace-line-length}, nil to stop any limitations on backtrace +line lengths (i.e. to get full backtraces), or a positive integer to +limit backtrace line length to that number. + @vindex ert-quiet By default, ERT in batch mode is quite verbose, printing a line with result after each test. This gives you progress information: how many diff --git a/etc/NEWS b/etc/NEWS index 3cad0995ac5..286f01f6864 100644 --- a/etc/NEWS +++ b/etc/NEWS @@ -46,6 +46,17 @@ time. * Changes in Emacs 29.1 ++++ +** New ERT batch variables + +Add three variables 'ert-batch-print-length', 'ert-batch-print-level' +& 'ert-batch-backtrace-line-length'. The first two will override +'print-length' & 'print-level' when printing Lisp values in ERT +batch test results. The third controls line length when +printing stack traces. + +** Emacs now supports Unicode Standard version 14.0. + ** Emoji +++ diff --git a/lisp/emacs-lisp/backtrace.el b/lisp/emacs-lisp/backtrace.el index a5721aa3193..90201442f08 100644 --- a/lisp/emacs-lisp/backtrace.el +++ b/lisp/emacs-lisp/backtrace.el @@ -55,9 +55,9 @@ backtrace-fontify (defcustom backtrace-line-length 5000 "Target length for lines in Backtrace buffers. Backtrace mode will attempt to abbreviate printing of backtrace -frames to make them shorter than this, but success is not -guaranteed. If set to nil or zero, Backtrace mode will not -abbreviate the forms it prints." +frames by setting `print-level' & `print-length' to make them +shorter than this, but success is not guaranteed. If set to nil +or zero, Backtrace mode will not abbreviate the forms it prints." :type 'integer :group 'backtrace :version "27.1") @@ -751,6 +751,13 @@ backtrace--print-flags (insert (make-string (- backtrace--flags-width (- (point) beg)) ?\s)) (put-text-property beg (point) 'backtrace-section 'func))) +(defun backtrace--line-length-or-nil () + "Return `backtrace-line-length' if valid, nil else." + ;; mirror the logic in `cl-print-to-string-with-limits' + (and (natnump backtrace-line-length) + (not (zerop backtrace-line-length)) + backtrace-line-length)) + (defun backtrace--print-func-and-args (frame _view) "Print the function, arguments and buffer position of a backtrace FRAME. Format it according to VIEW." @@ -769,11 +776,13 @@ backtrace--print-func-and-args (if (atom fun) (funcall backtrace-print-function fun) (insert - (backtrace--print-to-string fun (when args (/ backtrace-line-length 2))))) + (backtrace--print-to-string fun (when (and args (backtrace--line-length-or-nil)) (/ backtrace-line-length 2))))) (if args (insert (backtrace--print-to-string - args (max (truncate (/ backtrace-line-length 5)) - (- backtrace-line-length (- (point) beg))))) + args + (if (backtrace--line-length-or-nil) + (max (truncate (/ backtrace-line-length 5)) + (- backtrace-line-length (- (point) beg)))))) ;; The backtrace-form property is so that backtrace-multi-line ;; will find it. backtrace-multi-line doesn't do anything ;; useful with it, just being consistent. diff --git a/lisp/emacs-lisp/ert.el b/lisp/emacs-lisp/ert.el index 8ebc81fd418..9659c55cb64 100644 --- a/lisp/emacs-lisp/ert.el +++ b/lisp/emacs-lisp/ert.el @@ -77,6 +77,37 @@ ert-batch-backtrace-right-margin Use nil for no limit (caution: backtrace lines can be very long)." :type '(choice (const :tag "No truncation" nil) integer)) +(defvar ert-batch-print-length 10 + "`print-length' setting used in `ert-run-tests-batch'. + +When formatting lists in test conditions, `print-length' will be +temporarily set to this value. See also +`ert-batch-backtrace-line-length' for its effect on stack +traces.") + +(defvar ert-batch-print-level 5 + "`print-level' setting used in `ert-run-tests-batch'. + +When formatting lists in test conditions, `print-level' will be +temporarily set to this value. See also +`ert-batch-backtrace-line-length' for its effect on stack +traces.") + +(defvar ert-batch-backtrace-line-length t + "Target length for lines in ERT batch backtraces. + +Even modest settings for `print-length' & `print-level' can +produce extremely long lines in backtraces & lengthy delays in +forming them. This variable governs the target maximum line +length by manipulating these two variables while printing stack +traces. Setting this variable to t will re-use the value of +`backtrace-line-length' while print stack traces in ERT batch +mode. A value of nil will short-circuit this mechanism; line +lengths will be completely determined by `ert-batch-line-length' +& `ert-batch-line-level'. Any other value will be temporarily +bound to `backtrace-line-length' when producing stack traces +in batch mode.") + (defface ert-test-result-expected '((((class color) (background light)) :background "green1") (((class color) (background dark)) @@ -1402,8 +1433,7 @@ ert-run-tests-batch (ert-reason-for-test-result result) "")))) (message "%s" ""))))) - (test-started - ) + (test-started) (test-ended (cl-destructuring-bind (stats test result) event-args (unless (ert-test-result-expected-p test result) @@ -1413,8 +1443,15 @@ ert-run-tests-batch (ert-test-result-with-condition (message "Test %S backtrace:" (ert-test-name test)) (with-temp-buffer - (insert (backtrace-to-string - (ert-test-result-with-condition-backtrace result))) + (let ((backtrace-line-length + (cond + ((eq ert-batch-backtrace-line-length t) backtrace-line-length) + ((eq ert-batch-backtrace-line-length nil) nil) + (t ert-batch-backtrace-line-length))) + (print-level ert-batch-print-level) + (print-length ert-batch-print-length)) + (insert (backtrace-to-string + (ert-test-result-with-condition-backtrace result)))) (if (not ert-batch-backtrace-right-margin) (message "%s" (buffer-substring-no-properties (point-min) @@ -1433,8 +1470,8 @@ ert-run-tests-batch (ert--insert-infos result) (insert " ") (let ((print-escape-newlines t) - (print-level 5) - (print-length 10)) + (print-level ert-batch-print-level) + (print-length ert-batch-print-length)) (ert--pp-with-indentation-and-newline (ert-test-result-with-condition-condition result))) (goto-char (1- (point-max))) @@ -1962,13 +1999,13 @@ ert--results-font-lock-function (ewoc-refresh ert--results-ewoc) (font-lock-default-function enabledp)) -(defun ert--setup-results-buffer (stats listener buffer-name) +(defvar ert--output-buffer-name "*ert*") + +(defun ert--setup-results-buffer (stats listener) "Set up a test results buffer. -STATS is the stats object; LISTENER is the results listener; -BUFFER-NAME, if non-nil, is the buffer name to use." - (unless buffer-name (setq buffer-name "*ert*")) - (let ((buffer (get-buffer-create buffer-name))) +STATS is the stats object; LISTENER is the results listener." + (let ((buffer (get-buffer-create ert--output-buffer-name))) (with-current-buffer buffer (let ((inhibit-read-only t)) (buffer-disable-undo) @@ -2000,18 +2037,11 @@ ert--setup-results-buffer (defvar ert--selector-history nil "List of recent test selectors read from terminal.") -;; Should OUTPUT-BUFFER-NAME and MESSAGE-FN really be arguments here? -;; They are needed only for our automated self-tests at the moment. -;; Or should there be some other mechanism? ;;;###autoload -(defun ert-run-tests-interactively (selector - &optional output-buffer-name message-fn) +(defun ert-run-tests-interactively (selector) "Run the tests specified by SELECTOR and display the results in a buffer. -SELECTOR works as described in `ert-select-tests'. -OUTPUT-BUFFER-NAME and MESSAGE-FN should normally be nil; they -are used for automated self-tests and specify which buffer to use -and how to display message." +SELECTOR works as described in `ert-select-tests'." (interactive (list (let ((default (if ert--selector-history ;; Can't use `first' here as this form is @@ -2024,23 +2054,17 @@ ert-run-tests-interactively obarray #'ert-test-boundp nil nil 'ert--selector-history default nil))) nil)) - (unless message-fn (setq message-fn 'message)) - (let ((output-buffer-name output-buffer-name) - buffer - listener - (message-fn message-fn)) + (let (buffer listener) (setq listener (lambda (event-type &rest event-args) (cl-ecase event-type (run-started (cl-destructuring-bind (stats) event-args - (setq buffer (ert--setup-results-buffer stats - listener - output-buffer-name)) + (setq buffer (ert--setup-results-buffer stats listener)) (pop-to-buffer buffer))) (run-ended (cl-destructuring-bind (stats abortedp) event-args - (funcall message-fn + (message "%sRan %s tests, %s results were as expected%s%s" (if (not abortedp) "" diff --git a/test/lisp/emacs-lisp/ert-tests.el b/test/lisp/emacs-lisp/ert-tests.el index 79576d24032..f783532aaff 100644 --- a/test/lisp/emacs-lisp/ert-tests.el +++ b/test/lisp/emacs-lisp/ert-tests.el @@ -39,10 +39,11 @@ ert-test-body-runs (defun ert-self-test () "Run ERT's self-tests and make sure they actually ran." (let ((window-configuration (current-window-configuration))) - (let ((ert--test-body-was-run nil)) + (let ((ert--test-body-was-run nil) + (ert--output-buffer-name " *ert self-tests*")) ;; The buffer name chosen here should not compete with the default ;; results buffer name for completion in `switch-to-buffer'. - (let ((stats (ert-run-tests-interactively "^ert-" " *ert self-tests*"))) + (let ((stats (ert-run-tests-interactively "^ert-"))) (cl-assert ert--test-body-was-run) (if (zerop (ert-stats-completed-unexpected stats)) ;; Hide results window only when everything went well. @@ -519,17 +520,17 @@ ert-test-run-tests-interactively :body (lambda () (ert-skip "skip message"))))) (let ((ert-debug-on-error nil)) - (let* ((buffer-name (generate-new-buffer-name " *ert-test-run-tests*")) - (messages nil) - (mock-message-fn - (lambda (format-string &rest args) - (push (apply #'format format-string args) messages)))) + (cl-letf* ((buffer-name (generate-new-buffer-name " *ert-test-run-tests*")) + (ert--output-buffer-name buffer-name) + (messages nil) + ((symbol-function 'message) + (lambda (format-string &rest args) + (push (apply #'format format-string args) messages)))) (save-window-excursion (unwind-protect (let ((case-fold-search nil)) (ert-run-tests-interactively - `(member ,passing-test ,failing-test, skipped-test) buffer-name - mock-message-fn) + `(member ,passing-test ,failing-test, skipped-test)) (should (equal messages `(,(concat "Ran 3 tests, 1 results were " "as expected, 1 unexpected, " @@ -551,6 +552,68 @@ ert-test-run-tests-interactively (when (get-buffer buffer-name) (kill-buffer buffer-name)))))))) +(ert-deftest ert-test-run-tests-batch () + (let* ((complex-list '((:1 (:2 (:3 (:4 (:5 (:6 "abc")))))))) + (long-list (make-list 11 1)) + (failing-test-1 + (make-ert-test :name 'failing-test-1 + :body (lambda () (should (equal complex-list 1))))) + (failing-test-2 + (make-ert-test :name 'failing-test-2 + :body (lambda () (should (equal long-list 1)))))) + (let ((ert-debug-on-error nil) + messages) + (cl-letf* (((symbol-function 'message) + (lambda (format-string &rest args) + (push (apply #'format format-string args) messages)))) + (save-window-excursion + (unwind-protect + (let ((case-fold-search nil) + (ert-batch-backtrace-right-margin nil) + (ert-batch-print-level 10) + (ert-batch-print-length 11)) + (ert-run-tests-batch + `(member ,failing-test-1 ,failing-test-2)))))) + (let ((long-text "(different-types[ \t\n]+(1 1 1 1 1 1 1 1 1 1 1)[ \t\n]+1)))[ \t\n]*$") + (complex-text "(different-types[ \t\n]+((:1[ \t\n]+(:2[ \t\n]+(:3[ \t\n]+(:4[ \t\n]+(:5[ \t\n]+(:6[ \t\n]+\"abc\")))))))[ \t\n]+1)))[ \t\n]*$") + found-long + found-complex) + (cl-loop for msg in (reverse messages) + do + (unless found-long + (setq found-long (string-match long-text msg))) + (unless found-complex + (setq found-complex (string-match complex-text msg)))) + (should found-long) + (should found-complex))))) + +(ert-deftest ert-test-run-tests-batch-expensive () + (let* ((complex-list '((:1 (:2 (:3 (:4 (:5 (:6 "abc")))))))) + (failing-test-1 + (make-ert-test :name 'failing-test-1 + :body (lambda () (should (equal complex-list 1)))))) + (let ((ert-debug-on-error nil) + messages) + (cl-letf* (((symbol-function 'message) + (lambda (format-string &rest args) + (push (apply #'format format-string args) messages)))) + (save-window-excursion + (unwind-protect + (let ((case-fold-search nil) + (ert-batch-backtrace-right-margin nil) + (ert-batch-backtrace-line-length nil) + (ert-batch-print-level 6) + (ert-batch-print-length 11)) + (ert-run-tests-batch + `(member ,failing-test-1)))))) + (let ((frame "ert-fail(((should (equal complex-list 1)) :form (equal ((:1 (:2 (:3 (:4 (:5 (:6 \"abc\"))))))) 1) :value nil :explanation (different-types ((:1 (:2 (:3 (:4 (:5 (:6 \"abc\"))))))) 1)))") + found-frame) + (cl-loop for msg in (reverse messages) + do + (unless found-frame + (setq found-frame (cl-search frame msg :test 'equal)))) + (should found-frame))))) + (ert-deftest ert-test-special-operator-p () (should (ert--special-operator-p 'if)) (should-not (ert--special-operator-p 'car)) diff --git a/test/lisp/emacs-lisp/ert-x-tests.el b/test/lisp/emacs-lisp/ert-x-tests.el index 9baa9941586..1001525dee9 100644 --- a/test/lisp/emacs-lisp/ert-x-tests.el +++ b/test/lisp/emacs-lisp/ert-x-tests.el @@ -103,23 +103,24 @@ ert-propertized-string (ert-deftest ert-test-run-tests-interactively-2 () :tags '(:causes-redisplay) - (let* ((passing-test (make-ert-test :name 'passing-test - :body (lambda () (ert-pass)))) - (failing-test (make-ert-test :name 'failing-test - :body (lambda () - (ert-info ((propertize "foo\nbar" - 'a 'b)) - (ert-fail - "failure message"))))) - (skipped-test (make-ert-test :name 'skipped-test - :body (lambda () (ert-skip - "skip message")))) - (ert-debug-on-error nil) - (buffer-name (generate-new-buffer-name "*ert-test-run-tests*")) - (messages nil) - (mock-message-fn - (lambda (format-string &rest args) - (push (apply #'format format-string args) messages)))) + (cl-letf* ((passing-test (make-ert-test :name 'passing-test + :body (lambda () (ert-pass)))) + (failing-test (make-ert-test :name 'failing-test + :body (lambda () + (ert-info ((propertize "foo\nbar" + 'a 'b)) + (ert-fail + "failure message"))))) + (skipped-test (make-ert-test :name 'skipped-test + :body (lambda () (ert-skip + "skip message")))) + (ert-debug-on-error nil) + (messages nil) + (buffer-name (generate-new-buffer-name "*ert-test-run-tests*")) + ((symbol-function 'message) + (lambda (format-string &rest args) + (push (apply #'format format-string args) messages))) + (ert--output-buffer-name buffer-name)) (cl-flet ((expected-string (with-font-lock-p) (ert-propertized-string "Selector: (member " @@ -152,14 +153,12 @@ ert-test-run-tests-interactively-2 "failing-test" nil "\n Info: " '(a b) "foo\n" nil " " '(a b) "bar" - nil "\n (ert-test-failed \"failure message\")\n\n\n" - ))) + nil "\n (ert-test-failed \"failure message\")\n\n\n"))) (save-window-excursion (unwind-protect (let ((case-fold-search nil)) (ert-run-tests-interactively - `(member ,passing-test ,failing-test ,skipped-test) buffer-name - mock-message-fn) + `(member ,passing-test ,failing-test ,skipped-test)) (should (equal messages `(,(concat "Ran 3 tests, 1 results were " "as expected, 1 unexpected, " -- 2.33.1 --=-=-= Content-Type: text/plain; format=flowed -- Michael --=-=-=--