From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Alan Mackenzie Newsgroups: gmane.emacs.bugs Subject: bug#47320: Improve failure reporting in test/lisp/electrict-tests.el Date: Mon, 22 Mar 2021 14:24:37 +0000 Message-ID: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="6869"; mail-complaints-to="usenet@ciao.gmane.io" Cc: =?UTF-8?Q?Jo=C3=A3o_?= =?UTF-8?Q?T=C3=A1vora?= To: 47320@debbugs.gnu.org Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Mon Mar 22 16:01:37 2021 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lOM3k-0001hD-IA for geb-bug-gnu-emacs@m.gmane-mx.org; Mon, 22 Mar 2021 16:01:36 +0100 Original-Received: from localhost ([::1]:57172 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lOM3j-0002Ad-Dg for geb-bug-gnu-emacs@m.gmane-mx.org; Mon, 22 Mar 2021 11:01:35 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:41614) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lOLUN-00056C-0W for bug-gnu-emacs@gnu.org; Mon, 22 Mar 2021 10:25:03 -0400 Original-Received: from debbugs.gnu.org ([209.51.188.43]:46667) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lOLUM-00010q-Mg for bug-gnu-emacs@gnu.org; Mon, 22 Mar 2021 10:25:02 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1lOLUM-0001ev-FF for bug-gnu-emacs@gnu.org; Mon, 22 Mar 2021 10:25:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Alan Mackenzie Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Mon, 22 Mar 2021 14:25:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 47320 X-GNU-PR-Package: emacs X-Debbugs-Original-To: bug-gnu-emacs@gnu.org Original-Received: via spool by submit@debbugs.gnu.org id=B.16164230886352 (code B ref -1); Mon, 22 Mar 2021 14:25:02 +0000 Original-Received: (at submit) by debbugs.gnu.org; 22 Mar 2021 14:24:48 +0000 Original-Received: from localhost ([127.0.0.1]:58213 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lOLU8-0001eO-7O for submit@debbugs.gnu.org; Mon, 22 Mar 2021 10:24:48 -0400 Original-Received: from lists.gnu.org ([209.51.188.17]:58932) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lOLU6-0001eE-N9 for submit@debbugs.gnu.org; Mon, 22 Mar 2021 10:24:47 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:41464) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lOLU6-0004Sx-Dx for bug-gnu-emacs@gnu.org; Mon, 22 Mar 2021 10:24:46 -0400 Original-Received: from colin.muc.de ([193.149.48.1]:19058 helo=mail.muc.de) by eggs.gnu.org with smtp (Exim 4.90_1) (envelope-from ) id 1lOLU3-0000mQ-Th for bug-gnu-emacs@gnu.org; Mon, 22 Mar 2021 10:24:46 -0400 Original-Received: (qmail 55796 invoked by uid 3782); 22 Mar 2021 14:24:38 -0000 Original-Received: from acm.muc.de (p4fe15b2f.dip0.t-ipconnect.de [79.225.91.47]) (using STARTTLS) by colin.muc.de (tmda-ofmipd) with ESMTP; Mon, 22 Mar 2021 15:24:38 +0100 Original-Received: (qmail 6312 invoked by uid 1000); 22 Mar 2021 14:24:37 -0000 Content-Disposition: inline X-Submission-Agent: TMDA/1.3.x (Ph3nix) X-Primary-Address: acm@muc.de Received-SPF: pass client-ip=193.149.48.1; envelope-from=acm@muc.de; helo=mail.muc.de X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.io gmane.emacs.bugs:202834 Archived-At: Hello, Emacs. When running make check, and a failure occurs in electric-tests.elc, the resulting output in electric-tests.log is frustratingly cryptic. Although one can fairly easily get back to the source code for the test in electric-tests.el, it is difficult to reconstruct the snippet of text on which the test was performed. This is because the tests are (necessarily) generated by fairly inscrutable macros. This is a shame, since each generated test has its own exceptionally clear generated doc-string. An example of such a doc string is: ######################################################################### Electricity test in a `c-mode' buffer. Start with point at 7 in a 7-char-long buffer like this one: |"foo \"| (buffer start and end are denoted by `|') Now call this: #'electric-quote-local-mode Ensure the following bindings: '((electric-quote-replace-double . t) (electric-quote-comment . t) (electric-quote-string . t)) Now press the key for: " The buffer's contents should become: |"foo \""| , and point should be at 7. ######################################################################### My proposal is that on a test failure, this generated doc-string should be output along with the other failure stuff. It doesn't actually add all that much bulk to the .log file. Here is patch which does this. If there are no objections, I will commit it in a day or two. diff --git a/test/lisp/electric-tests.el b/test/lisp/electric-tests.el index 62a42b7fe4..44b3d8b672 100644 --- a/test/lisp/electric-tests.el +++ b/test/lisp/electric-tests.el @@ -50,7 +50,8 @@ save-electric-modes `(call-with-saved-electric-modes #'(lambda () ,@body))) (defun electric-pair-test-for (fixture where char expected-string - expected-point mode bindings fixture-fn) + expected-point mode bindings + fixture-fn &optional doc-string) (with-temp-buffer (funcall mode) (insert fixture) @@ -63,6 +64,14 @@ electric-pair-test-for (mapcar #'car bindings) (mapcar #'cdr bindings) (call-interactively (key-binding `[,last-command-event]))))) + (when + (and doc-string + (not + (and + (equal (buffer-substring-no-properties (point-min) (point-max)) + expected-string) + (equal (point) expected-point)))) + (message "\n%s\n" doc-string)) (should (equal (buffer-substring-no-properties (point-min) (point-max)) expected-string)) (should (equal (point) @@ -109,14 +118,9 @@ electric-pair-test-for (fixture (format "%s%s%s" prefix fixture suffix)) (expected-string (format "%s%s%s" prefix expected-string suffix)) (expected-point (+ (length prefix) expected-point)) - (pos (+ (length prefix) pos))) - `(ert-deftest ,(intern (format "electric-pair-%s-at-point-%s-in-%s%s" - name - (1+ pos) - mode - extra-desc)) - () - ,(format "Electricity test in a `%s' buffer.\n + (pos (+ (length prefix) pos)) + (doc-string + (format "Electricity test in a `%s' buffer.\n Start with point at %d in a %d-char-long buffer like this one: @@ -143,7 +147,14 @@ electric-pair-test-for char (if (string= fixture expected-string) "stay" "become") (replace-regexp-in-string "\n" "\\\\n" expected-string) - expected-point) + expected-point))) + `(ert-deftest ,(intern (format "electric-pair-%s-at-point-%s-in-%s%s" + name + (1+ pos) + mode + extra-desc)) + () + ,doc-string (electric-pair-test-for ,fixture ,(1+ pos) ,char @@ -151,7 +162,8 @@ electric-pair-test-for ,expected-point ',mode ,bindings - ,fixture-fn))))) + ,fixture-fn + ,doc-string))))) (cl-defmacro define-electric-pair-test (name fixture -- Alan Mackenzie (Nuremberg, Germany).