From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Michael Albinus Newsgroups: gmane.emacs.bugs Subject: bug#47320: Improve failure reporting in test/lisp/electrict-tests.el Date: Tue, 23 Mar 2021 16:24:43 +0100 Message-ID: <877dlx3nb8.fsf@gmx.de> References: <87v99i5k0f.fsf@gmx.de> Mime-Version: 1.0 Content-Type: text/plain Content-Transfer-Encoding: quoted-printable Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="23210"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.0.50 (gnu/linux) Cc: 47320@debbugs.gnu.org, =?UTF-8?Q?Jo=C3=A3o_?= =?UTF-8?Q?T=C3=A1vora?= To: Alan Mackenzie Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Tue Mar 23 16:29:10 2021 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lOixy-0005uA-0c for geb-bug-gnu-emacs@m.gmane-mx.org; Tue, 23 Mar 2021 16:29:10 +0100 Original-Received: from localhost ([::1]:40702 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lOixx-0006Ct-0B for geb-bug-gnu-emacs@m.gmane-mx.org; Tue, 23 Mar 2021 11:29:09 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:60594) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lOity-0000qR-Ue for bug-gnu-emacs@gnu.org; Tue, 23 Mar 2021 11:25:02 -0400 Original-Received: from debbugs.gnu.org ([209.51.188.43]:49457) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lOity-0005XK-Ll for bug-gnu-emacs@gnu.org; Tue, 23 Mar 2021 11:25:02 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1lOity-0001kl-I9 for bug-gnu-emacs@gnu.org; Tue, 23 Mar 2021 11:25:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Michael Albinus Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Tue, 23 Mar 2021 15:25:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 47320 X-GNU-PR-Package: emacs Original-Received: via spool by 47320-submit@debbugs.gnu.org id=B47320.16165131006720 (code B ref 47320); Tue, 23 Mar 2021 15:25:02 +0000 Original-Received: (at 47320) by debbugs.gnu.org; 23 Mar 2021 15:25:00 +0000 Original-Received: from localhost ([127.0.0.1]:32769 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lOitw-0001kK-BK for submit@debbugs.gnu.org; Tue, 23 Mar 2021 11:25:00 -0400 Original-Received: from mout.gmx.net ([212.227.15.15]:58601) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lOitt-0001k3-Bl for 47320@debbugs.gnu.org; Tue, 23 Mar 2021 11:24:58 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=badeba3b8450; t=1616513085; bh=OEum+GuWZY2xG64Q/pzD/uWRI0jA3jji7tazw8m3O5Y=; h=X-UI-Sender-Class:From:To:Cc:Subject:References:Date:In-Reply-To; b=ZVbi6cotb24jr+8HvXF2RVxMC3JoU8pzGoL9CYMC0xSoZ9+einaZvXCYrjO7VTE0t nO8DrLrgyuOsSOjQ/7F6YMX2UEBn2ZvSZu9+PxPCX8NI6cgmQJGcbRaDRKN4pbpGPQ uSZnF99wfCmM+1EbsxWoORLgwd+Ev08oi7kLIUSw= X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c Original-Received: from gandalf.gmx.de ([212.91.243.117]) by mail.gmx.net (mrgmx004 [212.227.17.190]) with ESMTPSA (Nemesis) id 1N95iH-1lloRW1I2x-016B3M; Tue, 23 Mar 2021 16:24:45 +0100 In-Reply-To: (Alan Mackenzie's message of "Tue, 23 Mar 2021 14:47:41 +0000") X-Provags-ID: V03:K1:AjiWZEgprz+TXwuUPtBUewcDgbiZUMzve6dHnIqMp8ekei2a+vb 1Z0kk3zVGSCJvaacLQne80sHaEqoS9nQakipdVr0a81PsMpqr4ekscQlMonIVMiZ1OO5F4v a5/IK+z2m7uzImFSnz6zRLRYMOVUct987TCyx0E/3nGcIIdZTK4Oq4luc7OUie5AYbJ4z6T 9tL8GwFHb4Ia5xgCXZA5Q== X-UI-Out-Filterresults: notjunk:1;V03:K0:UHJi93NVDl0=:q26PadmavSwhTbDStlU0ui SbxNj/a+tgi7EyJKA6Vc3JGU9HXMzm30Z8fK3h6YBpVUf4X0zVhre4JLkAyQHpzEi52zFB9/B QfKNz57KlbZNApZwuzUMAlTpg3tTrJcfAz29bjHONQNbTMt/1k61OuRAWiS+X4ZrhaxlYl2FL OsQ7VFtJJDxK+8xwB9KlwAsHtdtoB77UZF5rK6mbKMKes+wJGbvKuAmgiVbkOV7GxRHdXdpmt I11po0IaxTjyfJYqzcOCRkFb0niAa5dpW3a5tM4RTplQuq9/VkXNN4WcHwV5PfbeYmHfu+Qfx POlduQ5cDcuPZUrUcP/S0fExU/viwerUZE5x2+K8KOsivKqo1yj5GfzRl3trDVwB8wiQREi+i tb/b08BcNUFu6vt8Mj64MYTWnKQio78Ff3oCRVX7hJSytctcXWjG7PsRY3odQjgH2dUKpyA2h 9+g6KIoJr8O5CW8x6Zo9XizXkk1JD2tGK53WbVvDdyQaZGQXL1hqqAuosnvtTPyBalfih9JAp DVm0omnQfcI4dOD2klFwaM8g6p9EgaxKHcYvwHf0qXZ2/SD2s0/4B1J2mvOj8bAxpFtAPIHsp 5WrIBzaRMmhtFXdNXU5IISyCBQHDs46A8ld3sJyAxPAssysuXqIYczrRGXLuGVTfSagCHn8Rn INQ853IQ36yWVh8pAg1GDNSDwyDxfLa2MVZlLfXJAq2exv0r4E9HB51ebJpu9f866SUfBASCV WHuMkYIKbTY7orCbkUI0rj7N8X9dhHsOpquCcQqt1NFd4KXJukqPgOE1rVNV2hELckKEUNZV X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.io gmane.emacs.bugs:202907 Archived-At: Alan Mackenzie writes: > Hello, Michael. Hi Alan, >> There is the explainer functionality in ert, see > >> (info "(ert)Defining Explanation Functions") > > That description is rather terse. In fact it is incomplete - it does > not say when the explanation function gets called, nor does it say what > is done with any resulting explanation. 100% d'accord. The ert manual isn't as helpful as it should ... > It looks like a explanation function needs to duplicate the test code of > the test being explained - it seems the function has no access to the > internal state of the test. No, it has full access to the test via the ert machinery. (ert-get-test SYMBOL) returns the test object which has been created via (ert-deftest SYMBOL-NAME ...) (ert-running-test) returns the test object of the test just running. With that test object, you have different access functions, like ert-test-name, ert-test-documentation, ert-test-most-recent-result; see cl-defstruct ert-test. And there are also some other defstructs, which give you convenience functions like ert-test-failed-p. > I only wish to print the extra information when there is a test failure. > Otherwise the information in electric-tests.log would be swamped by > pointless voluminous doc strings. The ert explainer writes only something for failed functions. > Or am I missing something? I've spent hours and hours to understand these ert functions. You might have a look on filenotify-tests.el or tramp-tests.el, for example. >> I recommend to use it, for better readability of the test code. Example= s >> are in our test files. > > Thanks! If needed, I could help you in setting upt these explainers. But I must confess, I have no idea about electrict-tests.el. For the beginning, you might look on this: =2D-8<---------------cut here---------------start------------->8--- (defun test-function () "The result." t) (defun test-function-explainer () "The explainer, a string." (format "%s: %s" (ert-test-name (ert-running-test)) (ert-test-documentat= ion (ert-running-test)))) (ert-deftest first-test () "Just the first test" (should (test-function))) (ert-deftest second-test () "Just the second test" (should-not (test-function))) (put 'test-function 'ert-explainer 'test-function-explainer) =2D-8<---------------cut here---------------end--------------->8--- If you run both tests, you get =2D-8<---------------cut here---------------start------------->8--- .F . first-test Just the first test passed F second-test Just the second test (ert-test-failed ((should-not (test-function)) :form (test-function) :value t :explanation "second-test: Just the second test")) =2D-8<---------------cut here---------------end--------------->8--- Best regards, Michael.