From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: phillip.lord@russet.org.uk (Phillip Lord) Newsgroups: gmane.emacs.devel Subject: testing macros and fixtures Date: Mon, 12 Oct 2015 22:13:30 +0100 Message-ID: <87h9lvkcxx.fsf@russet.org.uk> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain X-Trace: ger.gmane.org 1444684449 16845 80.91.229.3 (12 Oct 2015 21:14:09 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Mon, 12 Oct 2015 21:14:09 +0000 (UTC) To: emacs-devel@gnu.org Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Mon Oct 12 23:14:00 2015 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1ZlkPq-0008H5-4J for ged-emacs-devel@m.gmane.org; Mon, 12 Oct 2015 23:13:54 +0200 Original-Received: from localhost ([::1]:58815 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZlkPp-0002K7-Ei for ged-emacs-devel@m.gmane.org; Mon, 12 Oct 2015 17:13:53 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:46453) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZlkPk-0002HJ-Oa for emacs-devel@gnu.org; Mon, 12 Oct 2015 17:13:49 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ZlkPh-00077s-Ht for emacs-devel@gnu.org; Mon, 12 Oct 2015 17:13:48 -0400 Original-Received: from cheviot12.ncl.ac.uk ([128.240.234.12]:56076) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZlkPh-00077E-Bo for emacs-devel@gnu.org; Mon, 12 Oct 2015 17:13:45 -0400 Original-Received: from smtpauth-vm.ncl.ac.uk ([10.8.233.129] helo=smtpauth.ncl.ac.uk) by cheviot12.ncl.ac.uk with esmtp (Exim 4.63) (envelope-from ) id 1ZlkPg-0001pI-Ap for emacs-devel@gnu.org; Mon, 12 Oct 2015 22:13:44 +0100 Original-Received: from cpc6-benw10-2-0-cust45.gate.cable.virginm.net ([92.238.179.46] helo=localhost) by smtpauth.ncl.ac.uk with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.63) (envelope-from ) id 1ZlkPg-0007xR-7K for emacs-devel@gnu.org; Mon, 12 Oct 2015 22:13:44 +0100 User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.5 (gnu/linux) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 128.240.234.12 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.devel:191401 Archived-At: I have been thinking about testing since it was raised in one of the last gargantuan threads. ERT is quite nice, but one of the things that I have found lacking is a nice set of predicates, for use within should. So, when I wrote test for my "lentic" package I needed some functions like, so that I could do things like: (should (test-eq-after-this "blah-before.txt" "blah-after.txt" (insert "hello"))) which opens "blah-before.txt" runs (insert "hello") then compares the result with "blah-after.txt". My version of this also does a diff of the results if the two are not equal. I've noticed that "puppet-mode" has some thing similar. For instance: (puppet-test-with-temp-buffer "# class bar" (should (eq (puppet-test-face-at 1) 'font-lock-comment-delimiter-face)) (should (eq (puppet-test-face-at 3) 'font-lock-comment-face)) (should (eq (puppet-test-face-at 7) 'font-lock-comment-face)) (should (eq (puppet-test-face-at 8) 'font-lock-comment-face)) (should-not (puppet-test-face-at 9)))) And julia-mode has indentation checking tests like so: (julia--should-indent " if foo bar else baz end" " if foo bar else baz end")) My own experience is that these are actually quite hard to right. The ones in lentic have never worked quite right -- that is, when it all works they are fine, but restoring state after a crash doesn't always work. Similarly, checking that, for example, test files are not already open before a test is run interactively. So, the point of my question is this; are there any good libraries providing this kind of fixture logic? A lot of this should surely be re-usable between different packages. Phil