From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Eli Zaretskii Newsgroups: gmane.emacs.bugs Subject: bug#57150: 29.0.50; [PATCH] Add test coverage for overlay modification hooks Date: Fri, 12 Aug 2022 21:53:23 +0300 Message-ID: <83v8qxmgyk.fsf@gnu.org> References: <87ilmygjyv.fsf@rfc20.org> <87edxmgjke.fsf@rfc20.org> <83o7wqnfu8.fsf@gnu.org> <874jyhgxak.fsf@rfc20.org> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="35952"; mail-complaints-to="usenet@ciao.gmane.io" Cc: 57150@debbugs.gnu.org To: Matt Armstrong Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Fri Aug 12 20:54:35 2022 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1oMZnm-0009AT-2U for geb-bug-gnu-emacs@m.gmane-mx.org; Fri, 12 Aug 2022 20:54:34 +0200 Original-Received: from localhost ([::1]:49684 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oMZnl-0005nU-34 for geb-bug-gnu-emacs@m.gmane-mx.org; Fri, 12 Aug 2022 14:54:33 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:32790) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oMZnH-0005lk-2n for bug-gnu-emacs@gnu.org; Fri, 12 Aug 2022 14:54:03 -0400 Original-Received: from debbugs.gnu.org ([209.51.188.43]:40940) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1oMZnG-0004cK-PC for bug-gnu-emacs@gnu.org; Fri, 12 Aug 2022 14:54:02 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1oMZnG-0003N0-CX for bug-gnu-emacs@gnu.org; Fri, 12 Aug 2022 14:54:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Eli Zaretskii Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Fri, 12 Aug 2022 18:54:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 57150 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: patch Original-Received: via spool by 57150-submit@debbugs.gnu.org id=B57150.166033041312919 (code B ref 57150); Fri, 12 Aug 2022 18:54:02 +0000 Original-Received: (at 57150) by debbugs.gnu.org; 12 Aug 2022 18:53:33 +0000 Original-Received: from localhost ([127.0.0.1]:58922 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1oMZmm-0003MJ-VP for submit@debbugs.gnu.org; Fri, 12 Aug 2022 14:53:33 -0400 Original-Received: from eggs.gnu.org ([209.51.188.92]:47982) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1oMZml-0003M5-9y for 57150@debbugs.gnu.org; Fri, 12 Aug 2022 14:53:31 -0400 Original-Received: from fencepost.gnu.org ([2001:470:142:3::e]:53570) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oMZmf-0004Zs-Ji; Fri, 12 Aug 2022 14:53:25 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-version:References:Subject:In-Reply-To:To:From: Date; bh=kqOrEoixyLHEZSCE6pwB46H8okEbOoj0KdeeAHToN7c=; b=GXVPujxxbHtjJ8Rnt2Ki tkxz9OAP3b/v8MaTa/lxc8BVBkl28zYl3XhL2RP6xsrOJPq6BGY8SOCMgvbZItN7uU+lQ1yY0R+Ij TFyeN32e01krDUVXXML+Eq+ro5ClX/XWeuKsbqrulD/ERHuZQrbz1rQeA4n2UqnrTHKs61zMzTXix 4SKznPWn1iZ35E9lu9eKv6L/FNc7uMTwVeFtDEGFRwrpgMW2KK4COEIOOBhuwYvZUd6LlcYVP7Ai0 OiwJDgMDHLS/EMAEHDygBwbuBjUyZQoPYELeERjs8m7bui7K2EJSDwjx2iyZZH1GJGT8kqdW5mEAS P6PoiR0FxoXjhA==; Original-Received: from [87.69.77.57] (port=4465 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oMZmf-0007HC-27; Fri, 12 Aug 2022 14:53:25 -0400 In-Reply-To: <874jyhgxak.fsf@rfc20.org> (message from Matt Armstrong on Fri, 12 Aug 2022 10:57:07 -0700) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.io gmane.emacs.bugs:239458 Archived-At: > From: Matt Armstrong > Cc: 57150@debbugs.gnu.org > Date: Fri, 12 Aug 2022 10:57:07 -0700 > > Do you have a preference between the two general approaches to > exercising many similar test cases: data driven vs. macro driven? I tend to like the data-driven approach better, but I have nothing against macros, assuming they solve the issues that I described. > If data driven tests are preferred, what do you think of using 'message' > to log the individual test case details, as a way of knowing which part > failed? I have no problems with that, assuming that the text emitted by 'message' is visible when running the tests in batch mode. > Other test frameworks have scoped based context messages. In > pseudocode, something like: > > (ert-deftest example () > (dolist arg '(a b c d) > (context (arg) > (should (equal arg (identity-function arg)))))) > > Printed failures would include both the normal 'should' output, but also > the values passed to 'context'. > > I notice that ERT does have two interactive features that partially > address this. These commands are available in its UI: > > ‘l’ > Show the list of ‘should’ forms executed in the test > (‘ert-results-pop-to-should-forms-for-test-at-point’). > > ‘m’ > Show any messages that were generated (with the Lisp function > ‘message’) in a test or any of the code that it invoked > (‘ert-results-pop-to-messages-for-test-at-point’). > > In simpler tests I find that 'l' is enough, since I can count the > 'should' calls to work out the iteration of whatever loop is used by the > test. In more complex cases, perhaps using 'message' to display the > context is enough? > > If you don't think 'l' and 'm' are *not* good enough, I might agree with > you. I'm not sure I understand how these commands are relevant. I'm talking about running the tests in batch mode. How do I make use of those commands in that case? > If you think adding something like 'context' to ERT is worthwhile > I can look at doing that. > > For this patch, perhaps using 'message' is best? Anything is fine with me, if it shows enough information to identify the particular "should" test that failed in the code. Thanks.