From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Artur Malabarba Newsgroups: gmane.emacs.devel Subject: Re: testing macros and fixtures Date: Tue, 13 Oct 2015 10:16:56 +0100 Message-ID: References: <87h9lvkcxx.fsf@russet.org.uk> Reply-To: bruce.connor.am@gmail.com NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: multipart/alternative; boundary=001a11c3f5c45315820521f8e894 X-Trace: ger.gmane.org 1444727842 32412 80.91.229.3 (13 Oct 2015 09:17:22 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Tue, 13 Oct 2015 09:17:22 +0000 (UTC) Cc: emacs-devel To: Phillip Lord Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Tue Oct 13 11:17:22 2015 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1Zlvhv-0000yV-NF for ged-emacs-devel@m.gmane.org; Tue, 13 Oct 2015 11:17:20 +0200 Original-Received: from localhost ([::1]:33236 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Zlvhv-0004b4-2k for ged-emacs-devel@m.gmane.org; Tue, 13 Oct 2015 05:17:19 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:59468) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Zlvhb-0004aN-Cw for emacs-devel@gnu.org; Tue, 13 Oct 2015 05:17:00 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Zlvha-0008Ls-9C for emacs-devel@gnu.org; Tue, 13 Oct 2015 05:16:59 -0400 Original-Received: from mail-lb0-x22d.google.com ([2a00:1450:4010:c04::22d]:36476) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZlvhZ-0008Lk-Ty for emacs-devel@gnu.org; Tue, 13 Oct 2015 05:16:58 -0400 Original-Received: by lbcao8 with SMTP id ao8so12263475lbc.3 for ; Tue, 13 Oct 2015 02:16:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:reply-to:sender:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=5TIv3k5nf6i5J6hNmS+9sAXBb2+krkwRHql6A4P8k3E=; b=iT4uAlW4H6tR/yB3FajdY5yQ80f2NbUgq9ko8UJ4LKIm3GC7HEvJMMEFmsEmaLVXW6 OSQZLaAmbRnMA603Ba62c5asejEChGVydDNUoI3WomeP7xiKKP8qLTzrtaJpevQEKVyv 4cNPyzxKY52gBGkkt8QB4kNW2KEp+99Yt9cGEcNsIMbLC/q4YHaf0ffL0YEC60colSYT CE0buBG3UnJ+c2m7eBziCk4gk6XkJspVvF/+uCznmxNSLHS5F9diUKJTsdhtDd+CTui9 LvVkrnUmIBB50KuA+NP1ukaj511mbxZN+EMhKw7NmEhqJ0FfBkLzDG1qdb/+em05ee79 iu6A== X-Received: by 10.112.54.169 with SMTP id k9mr14168998lbp.95.1444727816915; Tue, 13 Oct 2015 02:16:56 -0700 (PDT) Original-Received: by 10.25.27.78 with HTTP; Tue, 13 Oct 2015 02:16:56 -0700 (PDT) In-Reply-To: <87h9lvkcxx.fsf@russet.org.uk> X-Google-Sender-Auth: Yje0ZF3vAXZbdZTyPzH4MCDsvoA X-detected-operating-system: by eggs.gnu.org: Error: Malformed IPv6 address (bad octet value). X-Received-From: 2a00:1450:4010:c04::22d X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.devel:191436 Archived-At: --001a11c3f5c45315820521f8e894 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable 2015-10-12 22:13 GMT+01:00 Phillip Lord > ERT is quite nice, but one of the things that I have found lacking is a > nice set of predicates, for use within should. > > So, when I wrote test for my "lentic" package I needed some functions > like, so that I could do things like: > > (should > (test-eq-after-this > "blah-before.txt" > "blah-after.txt" > (insert "hello"))) > > which opens "blah-before.txt" runs (insert "hello") then compares the > result with "blah-after.txt". I think that's a pretty specific use-case. So it's best to let people write their own macros for this. (defmacro should-eq-after-body (file-before file-after &rest body) `(let ((bef (with-temp-buffer (insert-file-contents file-before) ,@body (buffer-string)))) (should (string=3D bef (with-temp-buffer (insert-file-contents file-after) (buffer-string)))))) > My version of this also does a diff of the > results if the two are not equal. This certainly seems very useful. Maybe ert should do this on *all* multi-line string comparisons instead of doing its default =E2=80=9Cdifferent-strings=E2=80=9D report (which, most of the time, just s= ays =E2=80=9Cstrings are of different length=E2=80=9D). > I've noticed that "puppet-mode" has some thing similar. For instance: > ... > > And julia-mode has indentation checking tests like so: > > ... > > My own experience is that these are actually quite hard to right. The > ones in lentic have never worked quite right -- that is, when it all > works they are fine, but restoring state after a crash doesn't always > work. Similarly, checking that, for example, test files are not already > open before a test is run interactively. Yes, there are many packages that use custom-deisgned temp-buffers for testing. In fact, most non-trivial packages do. I'd really like to see ert offer a common interface for this. The difficulty for that is that each package has very different needs when it comes to testing in buffers, so I have no idea what this common interface could be. > So, the point of my question is this; are there any good libraries > providing this kind of fixture logic? Don't know. --001a11c3f5c45315820521f8e894 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable

2015-10-12 22:13 GMT+01:00 Phillip Lord

> ERT= is quite nice, but one of the things that I have found lacking is a=C2=A0 =
> nice set of predicates, for use within should.=C2=A0
>=C2= =A0=C2=A0
> So, when I wrote test for my "lentic" package = I needed some functions=C2=A0
> like, so that I could do things like= :=C2=A0
>=C2=A0=C2=A0
> (should=C2=A0
> =C2=A0 (test-e= q-after-this=C2=A0
> =C2=A0 =C2=A0 =C2=A0"blah-before.txt"= =C2=A0
> =C2=A0 =C2=A0 =C2=A0"blah-after.txt"=C2=A0
&g= t; =C2=A0 =C2=A0 =C2=A0(insert "hello")))=C2=A0
>=C2=A0=C2= =A0
> which opens "blah-before.txt" runs (insert "hel= lo") then compares the=C2=A0
> result with "blah-after.txt= ".

I think that's a pretty specific use-case. So it's = best to let people write their own macros for this.

(defmacro should= -eq-after-body (file-before file-after &rest body)
=C2=A0 `(let ((be= f
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 (with-temp-buff= er
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 (i= nsert-file-contents file-before)
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0 ,@body
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 (buffer-string))))
=C2=A0=C2=A0=C2=A0=C2= =A0 (should (string=3D bef (with-temp-buffer
=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 (insert-file-c= ontents file-after)
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 (buffer-string))))))

> My ve= rsion of this also does a diff of the=C2=A0
> results if the two are= not equal.=C2=A0

This certainly seems very useful. Maybe ert shoul= d do this on *all* multi-line string comparisons instead of doing its defau= lt =E2=80=9Cdifferent-strings=E2=80=9D report (which, most of the time, jus= t says =E2=80=9Cstrings are of different length=E2=80=9D).

> I= 9;ve noticed that "puppet-mode" has some thing similar. For insta= nce:=C2=A0
> ...=C2=A0
>=C2=A0=C2=A0
> And julia-mode = has indentation checking tests like so:=C2=A0
>=C2=A0=C2=A0
>= ...
>=C2=A0=C2=A0
> My own experience is that these are actua= lly quite hard to right. The=C2=A0
> ones in lentic have never worke= d quite right -- that is, when it all=C2=A0
> works they are fine, b= ut restoring state after a crash doesn't always=C2=A0
> work. Si= milarly, checking that, for example, test files are not already=C2=A0
&= gt; open before a test is run interactively.=C2=A0

Yes, there are m= any packages that use custom-deisgned temp-buffers for testing. In fact, mo= st non-trivial packages do. I'd really like to see ert offer a common i= nterface for this. The difficulty for that is that each package has very di= fferent needs when it comes to testing in buffers, so I have no idea what t= his common interface could be.

> So, the point of my question is = this; are there any good libraries=C2=A0
> providing this kind of fi= xture logic?

Don't know.

--001a11c3f5c45315820521f8e894--