From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: =?utf-8?Q?Przemys=C5=82aw_Wojnowski?= Newsgroups: gmane.emacs.devel Subject: Re: UI tests Date: Sun, 15 Nov 2015 13:14:28 +0100 Message-ID: <878u5zjw7v.fsf@cumego.com> References: <87d1vpoees.fsf@russet.org.uk> <87r3k3igy1.fsf@russet.org.uk> <86io5ewbhg.fsf@stephe-leake.org> <8737wiyt75.fsf_-_@russet.org.uk> <87si4dsif4.fsf@cumego.com> <87oaf1wpkv.fsf@russet.org.uk> <876119sfa1.fsf@cumego.com> <87ziyk969x.fsf@russet.org.uk> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain X-Trace: ger.gmane.org 1447657407 17126 80.91.229.3 (16 Nov 2015 07:03:27 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Mon, 16 Nov 2015 07:03:27 +0000 (UTC) Cc: emacs-devel@gnu.org, stephen_leake@stephe-leake.org, Phillip Lord To: Richard Stallman Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Mon Nov 16 08:03:17 2015 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1ZyDoq-0005W4-2R for ged-emacs-devel@m.gmane.org; Mon, 16 Nov 2015 08:03:16 +0100 Original-Received: from localhost ([::1]:46216 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZyDop-00039v-JL for ged-emacs-devel@m.gmane.org; Mon, 16 Nov 2015 02:03:15 -0500 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:51521) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZxwCh-0006dJ-Au for emacs-devel@gnu.org; Sun, 15 Nov 2015 07:14:44 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ZxwCd-0006QA-8h for emacs-devel@gnu.org; Sun, 15 Nov 2015 07:14:43 -0500 Original-Received: from smtp23.iq.pl ([86.111.242.228]:56053) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZxwCc-0006PP-Ut for emacs-devel@gnu.org; Sun, 15 Nov 2015 07:14:39 -0500 Original-Received: (qmail 21487 invoked from network); 15 Nov 2015 12:14:29 -0000 Original-Received: from unknown (HELO namek.cumego.com) (esperanto@cumego.com@[159.205.193.173]) (envelope-sender ) by smtp22.iq.pl with AES128-GCM-SHA256 encrypted SMTP for ; 15 Nov 2015 12:14:29 -0000 In-Reply-To: (Richard Stallman's message of "Wed, 11 Nov 2015 12:02:58 -0500") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.1.50 (gnu/linux) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 86.111.242.228 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.devel:194520 Archived-At: Richard Stallman writes: > Unlike the typical graphical applications, Emacs's UIs are mostly > controlled by events that have a very shallow translation to Lisp > data. Even mouse clicks can be faked, and a convenient Lisp function > to make that totally easy would not be hard to write. > > Perhaps the input side of testing UI features will be fairly easy. > > The output side is not so straightforward, but if we set up something > to accumulate a sort of dribble record of X11 output operations, in > the form of Lisp-visible data, maybe tests could check that to see > if they have changed from before. I'm not sure what do you mean by "output side" here? My understanding is that inputs and outputs are on system's boundaries and isolate system's functionality using stable interfaces (common in different architectural pattern - e.g. [1], [2]). In Emacs, as you wrote, user actions can be faked and that can be used in tests. The other side - the outputs - is not that clear to me. Usually the outputs are hidden under stable interfaces. The interfaces allow to exchange implementations without affecting the system and the boundary is used to test the output of the system. In case of display output, this won't allow to test every single pixel (so no X11 output), but at least it will allow to test that the system produces correct output for display, no matter what that display (its underlying library) is. Two things to note when introducing automated tests: 1. Tests pyramid [3] - indicates which tests are most important and ratio between them. TL;DR Emacs should focus on unit tests. 2. Ice Cream Cone anti-pattern [3,4] is a common trap for existing projects that start to add automated tests. [1] Layers in "Pattern Oriented Software Architecture vol 1" [2] http://alistair.cockburn.us/Hexagonal+architecture [3] http://googletesting.blogspot.co.uk/2015/04/just-say-no-to-more-end-to-end-tests.html (from "The True Value of Tests") [4] http://watirmelon.com/2012/01/31/introducing-the-software-testing-ice-cream-cone/