From mboxrd@z Thu Jan 1 00:00:00 1970 From: Nathan Neff Subject: Re: [babel] Babel as a test harness? Date: Sat, 20 Mar 2010 14:38:05 -0500 Message-ID: <211769421003201238t4ddd21b0mc2b6af9610f78d45@mail.gmail.com> References: <211769421003191643l2d00b8cai63028b369408552f@mail.gmail.com> <874okblt2p.fsf@stats.ox.ac.uk> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============1319632634==" Return-path: Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1Nt4Ur-0005Jo-Vz for emacs-orgmode@gnu.org; Sat, 20 Mar 2010 15:38:10 -0400 Received: from [140.186.70.92] (port=40507 helo=eggs.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1Nt4Uq-0005Ih-GY for emacs-orgmode@gnu.org; Sat, 20 Mar 2010 15:38:09 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.69) (envelope-from ) id 1Nt4Uo-0007R5-DK for emacs-orgmode@gnu.org; Sat, 20 Mar 2010 15:38:08 -0400 Received: from mail-gw0-f41.google.com ([74.125.83.41]:57026) by eggs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1Nt4Uo-0007Qv-9b for emacs-orgmode@gnu.org; Sat, 20 Mar 2010 15:38:06 -0400 Received: by gwj21 with SMTP id 21so2219033gwj.0 for ; Sat, 20 Mar 2010 12:38:05 -0700 (PDT) In-Reply-To: <874okblt2p.fsf@stats.ox.ac.uk> List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org Errors-To: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org To: Dan Davison Cc: emacs-orgmode@gnu.org --===============1319632634== Content-Type: multipart/alternative; boundary=000e0cd704084f2d410482409a72 --000e0cd704084f2d410482409a72 Content-Type: text/plain; charset=ISO-8859-1 On Fri, Mar 19, 2010 at 7:45 PM, Dan Davison wrote: > Nathan Neff writes: > > > I think it would be cool to use Org-babel as a simple test harness. > > > > I'd like to have an org file, with various sections that demonstrate how > to do > > something in Groovy. I'd like to be able to run all the code in the org > file > > and make sure > > they all run successfully (return code 0). > > > > In Groovy, the "assert" function will exit with a non-zero code if it > fails. > > How > > would I use org-babel to generate a "summary" table with the name of > > each patch of code and whether or not it succeeded? > > Like below? This is all Eric's doing. For quite a while now org-babel > has used a table like this to validate itself. It took me a while to > understand it, but basically Eric designed a special function (actually, > a macro) called sbe (source block evaluate) to be used in table formulas > to call org-babel source blocks. Use C-u C-c C-c to update the table. > > * Tests > | functionality | block | arg | expected | results | > pass | > > |----------------+--------------+-----+----------+----------------------+--------------------------------------------| > | simple regexp | simple_regex | | | | > pass | > | regexp with or | regex_or | | | A pretend problem... | > expected "" but was "A pretend problem..." | > #+TBLFM: $5='(if (= (length $3) 1) (sbe $2 (n $3)) (sbe $2)) :: $6='(if > (string= $4 $5) "pass" (format "expected %S but was %S" $4 $5)) > > * Code blocks for tests > ** Here's how to match "foo" > #+source: simple_regex > #+begin_src groovy > assert "foo" =~ /foo/ > #+end_src > > ** Here's how to match "bar" or "baz" > #+source: regex_or > #+begin_src groovy > assert "bar" =~ "ba(z|r)" > println "A pretend problem" > #+end_src > > You can put any arguments to your source blocks in the arg column. See > > http://orgmode.org/worg/org-contrib/babel/intro.php#spreadsheet > > (It often makes most sense to clone Worg and view the org files > themseleves) > > And for a more complex example, including how to pass arguments to > source blocks to sbe, see our full test suite in the file > development.org in the devel repo > > http://github.com/eschulte/babel-dev/ > > Dan > > This is really cool Dan! I simplified the test table -- it runs the test in column 1, then prints the output to the second column, and puts "pass" or "fail" in the 3rd column if the 2nd column is blank. * Tests, Simpler Output from each source block should be nothing. | block | output | result | |--------------+---------------------------------------------------------------+--------| | simple_regex | | pass | | regex_or | | pass | | regex_fail | Caught: java.lang.AssertionError: Expression: (bar =~ foo)... | fail | #+TBLFM: $2='(sbe $1) :: $3='(if (string= $2 "") "pass" "fail") * Code blocks for tests ** Here's how to match "foo" #+source: simple_regex #+begin_src groovy results: output assert "foo" =~ /foo/ #+end_src ** Here's how to match "bar" or "baz" #+source: regex_or #+begin_src groovy assert "bar" =~ "ba(z|r)" #+end_src ** A failing test #+source: regex_fail #+begin_src groovy assert "bar" =~ "foo" #+end_src --000e0cd704084f2d410482409a72 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable

On Fri, Mar 19, 2010 at 7:45 PM, Dan Dav= ison <daviso= n@stats.ox.ac.uk> wrote:
Nathan Neff <= nathan.neff@gmail.com> writes:

> I think it would be cool to use Org-babel as a simple test harness. >
> I'd like to have an org file, with various sections that demonstra= te how to do
> something in Groovy.=A0 I'd like to be able to run all the code in= the org file
> and make sure
> they all run successfully (return code 0).
>
> In Groovy, the "assert" function will exit with a non-zero c= ode if it fails.=A0
> How
> would I use org-babel to generate a "summary" table with the= name of
> each patch of code and whether or not it succeeded?

Like below? This is all Eric's doing. For quite a while now org-b= abel
has used a table like this to validate itself. It took me a while to
understand it, but basically Eric designed a special function (actually, a macro) called sbe (source block evaluate) to be used in table formulas to call org-babel source blocks. Use C-u C-c C-c to update the table.

* Tests
| functionality =A0| block =A0 =A0 =A0 =A0| arg | expected | results =A0 = =A0 =A0 =A0 =A0 =A0 =A0| pass =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 |
|----------------+--------------+-----+----------+----------------------+--= ------------------------------------------|
| simple regexp =A0| simple_regex | =A0 =A0 | =A0 =A0 =A0 =A0 =A0| =A0 =A0 = =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0| pass =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 |
| regexp with or | regex_or =A0 =A0 | =A0 =A0 | =A0 =A0 =A0 =A0 =A0| A pret= end problem... | expected "" but was "A pretend problem...&q= uot; |
#+TBLFM: $5=3D'(if (=3D (length $3) 1) (sbe $2 (n $3)) (sbe $2)) :: $6= =3D'(if (string=3D $4 $5) "pass" (format "expected %S bu= t was %S" $4 $5))

* Code blocks for tests
** Here's how to match "foo"
#+source: simple_regex
#+begin_src groovy
assert "foo" =3D~ /foo/
#+end_src

** Here's how to match "bar" or "baz"
#+source: regex_or
#+begin_src groovy
assert "bar" =3D~ "ba(z|r)"
println "A pretend problem"
#+end_src

You can put any arguments to your source blocks in the arg column. See

http://orgmode.org/worg/org-contrib/babel/intro.php#spre= adsheet

(It often makes most sense to clone Worg and view the org files
themseleves)

And for a more complex example, including how to pass arguments to
source blocks to sbe, see our full test suite in the file
development.org in= the devel repo

http://= github.com/eschulte/babel-dev/

Dan



This is really cool Dan!=A0 I simplifi= ed the test table -- it runs the test in column 1, then
prints the outpu= t to the second column, and puts "pass" or "fail" in th= e 3rd column
if the 2nd column is blank.

* Tests, Simpler

Output from each source block should be nothing.
| block=A0=A0=A0=A0=A0=A0=A0 |=20 output=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0 | result |
|--------------+-----------------------------------------------------------= ----+--------|
| simple_regex=20 |=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 | pass=A0=A0 |
| regex_or=A0=A0=A0=A0=20 |=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 | pass=A0=A0 |
| regex_fail=A0=A0 | Caught: java.lang.AssertionError: Expression: (bar =3D= ~=20 foo)... | fail=A0=A0 |
#+TBLFM: $2=3D'(sbe $1) :: $3=3D'(if (string=3D $2 "") &q= uot;pass" "fail")


* Code blocks for tests
** Here's how to match "foo&quo= t;
#+source: simple_regex
#+begin_src groovy results: output
asser= t "foo" =3D~ /foo/
#+end_src

** Here's how to match= "bar" or "baz"
#+source: regex_or
#+begin_src groovy
assert "bar" =3D~ &qu= ot;ba(z|r)"
#+end_src

** A failing test
#+source: regex_f= ail
#+begin_src groovy
assert "bar" =3D~ "foo" #+end_src


--000e0cd704084f2d410482409a72-- --===============1319632634== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Emacs-orgmode mailing list Please use `Reply All' to send replies to the list. Emacs-orgmode@gnu.org http://lists.gnu.org/mailman/listinfo/emacs-orgmode --===============1319632634==--