From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: phillip.lord@russet.org.uk (Phillip Lord) Newsgroups: gmane.emacs.devel Subject: Re: testing macros and fixtures Date: Tue, 13 Oct 2015 10:23:50 +0100 Message-ID: <87h9lvt93t.fsf@russet.org.uk> References: <87h9lvkcxx.fsf@russet.org.uk> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain X-Trace: ger.gmane.org 1444728267 7859 80.91.229.3 (13 Oct 2015 09:24:27 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Tue, 13 Oct 2015 09:24:27 +0000 (UTC) Cc: Emacs developers To: Kaushal Modi Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Tue Oct 13 11:24:22 2015 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1Zlvoj-0007M7-0x for ged-emacs-devel@m.gmane.org; Tue, 13 Oct 2015 11:24:21 +0200 Original-Received: from localhost ([::1]:33267 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Zlvoh-000692-S4 for ged-emacs-devel@m.gmane.org; Tue, 13 Oct 2015 05:24:19 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:33071) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZlvoK-00068c-W0 for emacs-devel@gnu.org; Tue, 13 Oct 2015 05:23:57 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ZlvoF-0002zN-QW for emacs-devel@gnu.org; Tue, 13 Oct 2015 05:23:56 -0400 Original-Received: from cheviot12.ncl.ac.uk ([128.240.234.12]:50377) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZlvoF-0002zA-K1 for emacs-devel@gnu.org; Tue, 13 Oct 2015 05:23:51 -0400 Original-Received: from smtpauth-vm.ncl.ac.uk ([10.8.233.129] helo=smtpauth.ncl.ac.uk) by cheviot12.ncl.ac.uk with esmtp (Exim 4.63) (envelope-from ) id 1ZlvoE-0000HV-C5; Tue, 13 Oct 2015 10:23:50 +0100 Original-Received: from jangai.ncl.ac.uk ([10.66.67.223] helo=localhost) by smtpauth.ncl.ac.uk with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.63) (envelope-from ) id 1ZlvoE-00084B-Kj; Tue, 13 Oct 2015 10:23:50 +0100 In-Reply-To: (Kaushal Modi's message of "Mon, 12 Oct 2015 22:54:54 -0400") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.5 (gnu/linux) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 128.240.234.12 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.devel:191437 Archived-At: Okay. This is definately something worth looking at, then. I think writing tests is probably too hard at the moment! Phil Kaushal Modi writes: > verilog-mode runs similar tests from outside emacs. > >>>From the terminal, the verilog-mode functions are run in emacs batch mode > on files in the tests/ dir and and the results are compared against the > files in tests_ok/ dir (if I understand correctly). > > Repo on github which is then commited to emacs core: > https://github.com/veripool/verilog-mode > > Perl script to run batch tests: > https://github.com/veripool/verilog-mode/blob/master/batch_test.pl > > > -- > Kaushal Modi > > On Mon, Oct 12, 2015 at 5:13 PM, Phillip Lord > wrote: > >> >> I have been thinking about testing since it was raised in one of the >> last gargantuan threads. >> >> ERT is quite nice, but one of the things that I have found lacking is a >> nice set of predicates, for use within should. >> >> So, when I wrote test for my "lentic" package I needed some functions >> like, so that I could do things like: >> >> (should >> (test-eq-after-this >> "blah-before.txt" >> "blah-after.txt" >> (insert "hello"))) >> >> which opens "blah-before.txt" runs (insert "hello") then compares the >> result with "blah-after.txt". My version of this also does a diff of the >> results if the two are not equal. >> >> I've noticed that "puppet-mode" has some thing similar. For instance: >> >> (puppet-test-with-temp-buffer "# class >> bar" >> (should (eq (puppet-test-face-at 1) 'font-lock-comment-delimiter-face)) >> (should (eq (puppet-test-face-at 3) 'font-lock-comment-face)) >> (should (eq (puppet-test-face-at 7) 'font-lock-comment-face)) >> (should (eq (puppet-test-face-at 8) 'font-lock-comment-face)) >> (should-not (puppet-test-face-at 9)))) >> >> And julia-mode has indentation checking tests like so: >> >> (julia--should-indent >> " >> if foo >> bar >> else >> baz >> end" >> " >> if foo >> bar >> else >> baz >> end")) >> >> >> My own experience is that these are actually quite hard to right. The >> ones in lentic have never worked quite right -- that is, when it all >> works they are fine, but restoring state after a crash doesn't always >> work. Similarly, checking that, for example, test files are not already >> open before a test is run interactively. >> >> So, the point of my question is this; are there any good libraries >> providing this kind of fixture logic? A lot of this should surely be >> re-usable between different packages. >> >> Phil >> >> >> >> >>