From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Kaushal Modi Newsgroups: gmane.emacs.devel Subject: Re: testing macros and fixtures Date: Mon, 12 Oct 2015 22:54:54 -0400 Message-ID: References: <87h9lvkcxx.fsf@russet.org.uk> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: multipart/alternative; boundary=001a113ce6c467849a0521f394c0 X-Trace: ger.gmane.org 1444704957 656 80.91.229.3 (13 Oct 2015 02:55:57 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Tue, 13 Oct 2015 02:55:57 +0000 (UTC) Cc: Emacs developers To: Phillip Lord Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Tue Oct 13 04:55:53 2015 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1Zlpkl-0007UR-EF for ged-emacs-devel@m.gmane.org; Tue, 13 Oct 2015 04:55:51 +0200 Original-Received: from localhost ([::1]:60055 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Zlpkk-0006kd-FX for ged-emacs-devel@m.gmane.org; Mon, 12 Oct 2015 22:55:50 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:47302) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZlpkW-0006kS-EO for emacs-devel@gnu.org; Mon, 12 Oct 2015 22:55:37 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ZlpkU-0005BW-U2 for emacs-devel@gnu.org; Mon, 12 Oct 2015 22:55:36 -0400 Original-Received: from mail-oi0-x22a.google.com ([2607:f8b0:4003:c06::22a]:34740) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZlpkU-0005BP-Lt for emacs-devel@gnu.org; Mon, 12 Oct 2015 22:55:34 -0400 Original-Received: by oiak8 with SMTP id k8so2488875oia.1 for ; Mon, 12 Oct 2015 19:55:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=FZD+202Yckym/o0cliC5Og4oLj5t0VByWekiDkGwwbM=; b=AQOHAot45aVu0qnOqJFvyZApDfXgLwnqfmBdHt4nO3X4umVYswyiR1lSmXveX35/h+ +Xcg9gFPsrV1NVV0Vrpm2HJBw2U5GvU/g7WrM78TKDSvb4TlnofYDiRZ03evd6dWQK8I PtuqgqD4YWU9BEsKOgiZmGKfySQgPJyCxb1VvrWXX7exzFY6/z51fV4P/2TfcWqno9Z4 JF4wWS1Ox/f7IAmR+VcRw7DVz0eT8dYhCCChgbEplAIpYvPWznohlN4GCOU9eBYhlcld zkguoqSGoAuBxS7iLq4p6TEnHK+v8yPLZT+mkLXANMFIWAvbtVzL9qKYDOFx8JgcS3SL d5uQ== X-Received: by 10.202.173.77 with SMTP id w74mr17176729oie.128.1444704934127; Mon, 12 Oct 2015 19:55:34 -0700 (PDT) Original-Received: by 10.202.172.205 with HTTP; Mon, 12 Oct 2015 19:54:54 -0700 (PDT) In-Reply-To: <87h9lvkcxx.fsf@russet.org.uk> X-detected-operating-system: by eggs.gnu.org: Error: Malformed IPv6 address (bad octet value). X-Received-From: 2607:f8b0:4003:c06::22a X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.devel:191424 Archived-At: --001a113ce6c467849a0521f394c0 Content-Type: text/plain; charset=UTF-8 verilog-mode runs similar tests from outside emacs. >From the terminal, the verilog-mode functions are run in emacs batch mode on files in the tests/ dir and and the results are compared against the files in tests_ok/ dir (if I understand correctly). Repo on github which is then commited to emacs core: https://github.com/veripool/verilog-mode Perl script to run batch tests: https://github.com/veripool/verilog-mode/blob/master/batch_test.pl -- Kaushal Modi On Mon, Oct 12, 2015 at 5:13 PM, Phillip Lord wrote: > > I have been thinking about testing since it was raised in one of the > last gargantuan threads. > > ERT is quite nice, but one of the things that I have found lacking is a > nice set of predicates, for use within should. > > So, when I wrote test for my "lentic" package I needed some functions > like, so that I could do things like: > > (should > (test-eq-after-this > "blah-before.txt" > "blah-after.txt" > (insert "hello"))) > > which opens "blah-before.txt" runs (insert "hello") then compares the > result with "blah-after.txt". My version of this also does a diff of the > results if the two are not equal. > > I've noticed that "puppet-mode" has some thing similar. For instance: > > (puppet-test-with-temp-buffer "# class > bar" > (should (eq (puppet-test-face-at 1) 'font-lock-comment-delimiter-face)) > (should (eq (puppet-test-face-at 3) 'font-lock-comment-face)) > (should (eq (puppet-test-face-at 7) 'font-lock-comment-face)) > (should (eq (puppet-test-face-at 8) 'font-lock-comment-face)) > (should-not (puppet-test-face-at 9)))) > > And julia-mode has indentation checking tests like so: > > (julia--should-indent > " > if foo > bar > else > baz > end" > " > if foo > bar > else > baz > end")) > > > My own experience is that these are actually quite hard to right. The > ones in lentic have never worked quite right -- that is, when it all > works they are fine, but restoring state after a crash doesn't always > work. Similarly, checking that, for example, test files are not already > open before a test is run interactively. > > So, the point of my question is this; are there any good libraries > providing this kind of fixture logic? A lot of this should surely be > re-usable between different packages. > > Phil > > > > > --001a113ce6c467849a0521f394c0 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
verilog-mode runs similar tests = from outside emacs.

From the terminal, the verilog-mode functions are run in emacs = batch mode on files in the tests/ dir and and the results are compared agai= nst the files in tests_ok/ dir (if I understand correctly).

Repo on github which is= then commited to emacs core:=C2=A0https://github.com/veripool/verilog-mode

Perl script to run batc= h tests:=C2=A0https://github.com/veripool/verilog-mode/blob/master/batc= h_test.pl

=

--
Kaushal Modi

On Mon, Oct 12, 2015 at 5:13 PM, Phillip Lor= d <phillip.lord@russet.org.uk> wrote:

I have been thinking about testing since it was raised in one of the
last gargantuan threads.

ERT is quite nice, but one of the things that I have found lacking is a
nice set of predicates, for use within should.

So, when I wrote test for my "lentic" package I needed some funct= ions
like, so that I could do things like:

(should
=C2=A0 (test-eq-after-this
=C2=A0 =C2=A0 =C2=A0"blah-before.txt"
=C2=A0 =C2=A0 =C2=A0"blah-after.txt"
=C2=A0 =C2=A0 =C2=A0(insert "hello")))

which opens "blah-before.txt" runs (insert "hello") the= n compares the
result with "blah-after.txt". My version of this also does a diff= of the
results if the two are not equal.

I've noticed that "puppet-mode" has some thing similar. For i= nstance:

(puppet-test-with-temp-buffer "# class
bar"
=C2=A0 =C2=A0 (should (eq (puppet-test-face-at 1) 'font-lock-comment-de= limiter-face))
=C2=A0 =C2=A0 (should (eq (puppet-test-face-at 3) 'font-lock-comment-fa= ce))
=C2=A0 =C2=A0 (should (eq (puppet-test-face-at 7) 'font-lock-comment-fa= ce))
=C2=A0 =C2=A0 (should (eq (puppet-test-face-at 8) 'font-lock-comment-fa= ce))
=C2=A0 =C2=A0 (should-not (puppet-test-face-at 9))))

And julia-mode has indentation checking tests like so:

=C2=A0 =C2=A0(julia--should-indent
=C2=A0 =C2=A0 =C2=A0"
if foo
=C2=A0 =C2=A0 bar
else
baz
end"
=C2=A0 =C2=A0 =C2=A0"
if foo
=C2=A0 =C2=A0 bar
else
=C2=A0 =C2=A0 baz
end"))


My own experience is that these are actually quite hard to right. The
ones in lentic have never worked quite right -- that is, when it all
works they are fine, but restoring state after a crash doesn't always work. Similarly, checking that, for example, test files are not already
open before a test is run interactively.

So, the point of my question is this; are there any good libraries
providing this kind of fixture logic? A lot of this should surely be
re-usable between different packages.

Phil





--001a113ce6c467849a0521f394c0--