From: Stefan Kangas <stefan@marxist.se>
To: Lars Ingebrigtsen <larsi@gnus.org>
Cc: 22471@debbugs.gnu.org, Phillip Lord <phillip.lord@russet.org.uk>
Subject: bug#22471: ert batch should print compile parsable error messages
Date: Fri, 22 Oct 2021 17:15:35 -0700 [thread overview]
Message-ID: <CADwFkmnChMB0b3S3i2Oqvr8K+vZpnYxnJE2CFUv9cWFYN+yOEg@mail.gmail.com> (raw)
In-Reply-To: <87wo1tp8z2.fsf@gnus.org> (Lars Ingebrigtsen's message of "Thu, 20 Aug 2020 18:39:45 +0200")
[-- Attachment #1: Type: text/plain, Size: 1795 bytes --]
Lars Ingebrigtsen <larsi@gnus.org> writes:
> phillip.lord@russet.org.uk (Phillip Lord) writes:
>
>> Currently, when running in batch, ert prints out messages like so:
>>
>> Running 24 tests (2016-01-27 09:05:17+0000)
>> passed 1/24 buffer-string=
>> passed 2/24 buffer=
>> ...
>> passed 22/24 sisyphus-test-with-find-file
>> passed 23/24 to-string
>> passed 24/24 with-temp-buffers
>>
>> Ran 24 tests, 23 results as expected, 1 unexpected (2016-01-27 09:05:18+0000)
>>
>> 1 unexpected results:
>> FAILED crash-out
>>
>> It would be nice to add some compilation-mode parsable data to this. So,
>> something like
>>
>> passed 1/24 buffer-string= (in ./test.sisyphus-test.el:22:)
>
> I think that would look rather cluttered for the "passed" lines, but it
> would indeed be helpful on the FAILED lines.
This would in my opinion be very useful and a great timesaver. But yes,
for FAILED results only, much like how the compiler only gives us line
numbers if there is a problem. (AFAIR, that's how the test frameworks
I've used do it.)
So I updated the patch from the feature/parsable-ert-output against
current master (patch attached), made sure that one of our tests failed,
and then tested it with "make check".
I got the following backtrace, but gave up after looking at it for a
while:
Debugger entered--Lisp error: (error "Don’t know where
‘fns-tests-string-bytes’ is defin...")
signal(error ("Don’t know where ‘fns-tests-string-bytes’ is defin..."))
error("Don't know where `%s' is defined" fns-tests-string-bytes)
find-function-search-for-symbol(fns-tests-string-bytes ert-deftest nil)
find-definition-noselect(fns-tests-string-bytes ert-deftest)
ert-test-location([...])
[...]
[-- Attachment #2: 0001-Add-source-information-to-ERT-batch-mode.patch --]
[-- Type: text/x-diff, Size: 2569 bytes --]
From e6c185f280e2c330825beebad7d2393c21199564 Mon Sep 17 00:00:00 2001
From: Phillip Lord <phillip.lord@russet.org.uk>
Date: Wed, 27 Jan 2016 20:59:50 +0000
Subject: [PATCH] Add source information to ERT batch mode
* lisp/emacs-lisp/ert.el (ert-test-location): New function.
(ert-run-tests-batch): Add test location to failed tests. (Bug#22471)
---
lisp/emacs-lisp/ert.el | 24 ++++++++++++++++++++++--
1 file changed, 22 insertions(+), 2 deletions(-)
diff --git a/lisp/emacs-lisp/ert.el b/lisp/emacs-lisp/ert.el
index 57655403c2..36f5062e83 100644
--- a/lisp/emacs-lisp/ert.el
+++ b/lisp/emacs-lisp/ert.el
@@ -1353,6 +1353,23 @@ ert--insert-infos
(defvar ert-quiet nil
"Non-nil makes ERT only print important information in batch mode.")
+(defun ert-test-location (test)
+ "Return a string description the source location of TEST."
+ (let* ((loc
+ (find-definition-noselect (ert-test-name test) 'ert-deftest))
+ (buffer
+ (car loc))
+ (point (cdr loc))
+ (file
+ (file-relative-name
+ (buffer-file-name buffer)))
+ (line (with-current-buffer buffer
+ (line-number-at-pos point))))
+ (format "at %s line %s." file line)))
+
+(defvar ert-batch-backtrace-right-margin 70
+ "The maximum line length for printing backtraces in `ert-run-tests-batch'.")
+
;;;###autoload
(defun ert-run-tests-batch (&optional selector)
"Run the tests specified by SELECTOR, printing results to the terminal.
@@ -1471,14 +1488,17 @@ ert-run-tests-batch
(let* ((max (prin1-to-string (length (ert--stats-tests stats))))
(format-string (concat "%9s %"
(prin1-to-string (length max))
- "s/" max " %S (%f sec)")))
+ "s/" max " %S (%f sec)%s")))
(message format-string
(ert-string-for-test-result result
(ert-test-result-expected-p
test result))
(1+ (ert--stats-test-pos stats test))
(ert-test-name test)
- (ert-test-result-duration result))))))))
+ (ert-test-result-duration result)
+ (if (ert-test-result-expected-p test result)
+ ""
+ (concat " " (ert-test-location test))))))))))
nil))
;;;###autoload
--
2.30.2
next prev parent reply other threads:[~2021-10-23 0:15 UTC|newest]
Thread overview: 12+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-01-27 9:15 bug#22471: ert batch should print compile parsable error messages Phillip Lord
2016-01-27 16:06 ` Eli Zaretskii
2016-01-27 21:53 ` Phillip Lord
2016-01-28 1:28 ` Glenn Morris
2016-01-28 17:14 ` Phillip Lord
2020-08-20 16:39 ` Lars Ingebrigtsen
2020-08-20 16:51 ` Philipp Stephani
2021-10-23 0:15 ` Stefan Kangas [this message]
2021-10-24 12:57 ` Lars Ingebrigtsen
2021-10-24 14:27 ` Stefan Kangas
2021-10-24 20:01 ` Lars Ingebrigtsen
2022-01-24 12:54 ` Lars Ingebrigtsen
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CADwFkmnChMB0b3S3i2Oqvr8K+vZpnYxnJE2CFUv9cWFYN+yOEg@mail.gmail.com \
--to=stefan@marxist.se \
--cc=22471@debbugs.gnu.org \
--cc=larsi@gnus.org \
--cc=phillip.lord@russet.org.uk \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this external index
https://git.savannah.gnu.org/cgit/emacs.git
https://git.savannah.gnu.org/cgit/emacs/org-mode.git
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.