[patch][test] Avoid writes to non-temp test-example files

[patch][test] Avoid writes to non-temp test-example files

[Martyn Jago]

[-- Attachment #1: Type: text/plain, Size: 262 bytes --]


Attached is a patch to modify all tests that currently write to non-temp
test-example files such as `testing/babel.org'. Instead, the tests now
write to temp files, which are thrown away at the conclusion of the
test.

All tests still pass.

HTH

Best, Martyn


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: avoid-writes-to-non-temp-test-example-files --]
[-- Type: text/x-patch, Size: 11470 bytes --]

From c56d5101de63bda4a0df7bf0800761f51624194b Mon Sep 17 00:00:00 2001
From: Martyn Jago <martyn.jago@btinternet.com>
Date: Wed, 4 Jan 2012 17:44:09 +0000
Subject: [PATCH] Modify tests to avoid writing to non-temp test-example files.

* testing/examples/babel.org:
* testing/examples/table.org:
* testing/lisp/test-ob.el:
* testing/lisp/test-org-table.el:

Tests that used to write to a test-example file (ie testing/babel.org)
have been modified to write to temp files instead.

This avoids leaving open and /modified/ tests in Emacs when tests are
run by `org-test-run-all-tests'.
---
 testing/examples/babel.org     |   76 -------------------
 testing/examples/table.org     |   19 -----
 testing/lisp/test-ob.el        |  162 ++++++++++++++++++++++++++++++----------
 testing/lisp/test-org-table.el |   28 ++++++--
 4 files changed, 144 insertions(+), 141 deletions(-)
 delete mode 100644 testing/examples/table.org

diff --git a/testing/examples/babel.org b/testing/examples/babel.org
index bb9cab9..ba7c3e3 100644
--- a/testing/examples/babel.org
+++ b/testing/examples/babel.org
@@ -29,19 +29,6 @@
   ;; noweb-tangle-end
 #+end_src
 
-* elisp forms in header arguments
-  :PROPERTIES:
-  :ID:       22d67284-bf14-4cdc-8319-f4bd876829d7
-  :var:      prop=(+ 2 2)
-  :END:
-
-#+begin_src emacs-lisp
-  prop
-#+end_src
-
-#+name:
-: 4
-
 * excessive id links on tangling
   :PROPERTIES:
   :ID:       ef06fd7f-012b-4fde-87a2-2ae91504ea7e
@@ -59,42 +46,6 @@
 #+begin_src emacs-lisp :tangle no
   (message "for tangling")
 #+end_src
-* simple variable resolution
-  :PROPERTIES:
-  :ID:       f68821bc-7f49-4389-85b5-914791ee3718
-  :END:
-
-#+name: four
-#+begin_src emacs-lisp
-  (list 1 2 3 4)
-#+end_src
-
-#+begin_src emacs-lisp :var four=four
-  (length four)
-#+end_src
-
-#+name:
-: 4
-
-* multi-line header arguments
-  :PROPERTIES:
-  :ID:       b77c8857-6c76-4ea9-8a61-ddc2648d96c4
-  :END:
-
-#+headers: :var letters='(a b c d e f g)
-#+begin_src emacs-lisp :var numbers='(1 2 3 4 5 6 7)
-  (map 'list #'list numbers letters)
-#+end_src
-
-#+name:
-| 1 | a |
-| 2 | b |
-| 3 | c |
-| 4 | d |
-| 5 | e |
-| 6 | f |
-| 7 | g |
-
 * simple named code block
   :PROPERTIES:
   :ID:       0d82b52d-1bb9-4916-816b-2c67c8108dbb
@@ -202,14 +153,6 @@ Here is one in the middle src_sh{echo 1} of a line.
 Here is one at the end of a line. src_sh{echo 2}
 src_sh{echo 3} Here is one at the beginning of a line.
 
-* parsing header arguments
-  :PROPERTIES:
-  :ID:       7eb0dc6e-1c53-4275-88b3-b22f3113b9c3
-  :END:
-
-#+begin_src example-lang :session     :results output :var num=9
-  the body
-#+end_src
 * conflicting blocks on export
   :PROPERTIES:
   :ID:       5daa4d03-e3ea-46b7-b093-62c1b7632df3
@@ -289,22 +232,3 @@ src_sh{echo "One"} block at start of line
  One spaced block in  src_sh{ echo "middle" } of line 
 src_sh{echo 2} blocks on the src_emacs-lisp{"same"} line
  Inline block with src_sh[:results silent]{ echo "parameters" }.
-* returning file names -- interpreted as lists
-  :PROPERTIES:
-  :ID:       a73a2ab6-b8b2-4c0e-ae7f-23ad14eab7bc
-  :END:
-
-#+begin_src sh :results scalar
-  echo "[[file:./cv.cls]]"
-#+end_src
-
-#+name:
-: [[file:./cv.cls]]
-
-#+begin_src sh :results raw scalar
-   echo "[[file:./cv.cls]]"
-#+end_src
-
-#+name:
-[[file:./cv.cls]]
-
diff --git a/testing/examples/table.org b/testing/examples/table.org
deleted file mode 100644
index 3c6caed..0000000
--- a/testing/examples/table.org
+++ /dev/null
@@ -1,19 +0,0 @@
-#+TITLE: example file with tables
-#+OPTIONS: num:nil ^:nil
-
-This is an example file for use by the Org-mode tests defined in
-file:../lisp/test-org-table.el.
-
-* simple formula
-  :PROPERTIES:
-  :ID:       563523f7-3f3e-49c9-9622-9216cc9a5d95
-  :END:
-
-#+tblname: simple-formula
-|  1 |
-|  2 |
-|  3 |
-|  4 |
-|----|
-| 10 |
-  #+TBLFM: $1=vsum(@1..@-1)
diff --git a/testing/lisp/test-ob.el b/testing/lisp/test-ob.el
index ee4f298..4b57f21 100644
--- a/testing/lisp/test-ob.el
+++ b/testing/lisp/test-ob.el
@@ -114,51 +114,121 @@
 
 (ert-deftest test-org-babel/elisp-in-header-arguments ()
   "Test execution of elisp forms in header arguments."
-  ;; at the babel.org:elisp-forms-in-header-arguments header
-  (org-test-at-id "22d67284-bf14-4cdc-8319-f4bd876829d7"
-    (org-babel-next-src-block)
-    (let ((info (org-babel-get-src-block-info)))
-      (should (= 4 (org-babel-execute-src-block))))))
+  (org-test-with-temp-text-in-file "
+
+* elisp forms in header arguments
+  :PROPERTIES:
+  :var:      prop = (* 7 6)
+  :END:
+#+begin_src emacs-lisp
+  prop
+#+end_src"
+    
+    (progn
+      (goto-char (point-min))
+      (org-babel-next-src-block)
+      (let ((info (org-babel-get-src-block-info)))
+	(should (= 42 (org-babel-execute-src-block)))))))
 
 (ert-deftest test-org-babel/simple-named-code-block ()
   "Test that simple named code blocks can be evaluated."
-  (org-test-at-id "0d82b52d-1bb9-4916-816b-2c67c8108dbb"
-    (org-babel-next-src-block 1)
-    (should (= 42 (org-babel-execute-src-block)))))
+  (org-test-with-temp-text-in-file "
+
+#+name: i-have-a-name
+#+begin_src emacs-lisp
+  42
+#+end_src
+
+#+name:
+: 42
+
+#+name: i-have-a-name
+: 42"
+
+    (progn
+      (org-babel-next-src-block 1)
+      (should (= 42 (org-babel-execute-src-block))))))
 
 (ert-deftest test-org-babel/simple-variable-resolution ()
   "Test that simple variable resolution is working."
-  (org-test-at-id "f68821bc-7f49-4389-85b5-914791ee3718"
-    (org-babel-next-src-block 2)
-    (should (= 4 (org-babel-execute-src-block)))))
+  (org-test-with-temp-text-in-file "
+
+#+name: four
+#+begin_src emacs-lisp
+  (list 1 2 3 4)
+#+end_src
+
+#+begin_src emacs-lisp :var four=four
+  (length four)
+#+end_src"
+    
+    (progn
+      (org-babel-next-src-block 2)
+      (should (= 4 (org-babel-execute-src-block)))
+      (forward-line 5)
+      (should (string= ": 4" (buffer-substring
+			      (point-at-bol)
+			      (point-at-eol)))))))
 
 (ert-deftest test-org-babel/multi-line-header-arguments ()
   "Test that multi-line header arguments and can be read."
-  (org-test-at-id "b77c8857-6c76-4ea9-8a61-ddc2648d96c4"
-    (org-babel-next-src-block)
-    (let ((results (org-babel-execute-src-block)))
-      (should(equal 'a (cadr (assoc 1 results))))
-      (should(equal 'd (cadr (assoc 4 results)))))))
+  (org-test-with-temp-text-in-file "
+
+#+headers: :var letters='(a b c d e f g)
+#+begin_src emacs-lisp :var numbers='(1 2 3 4 5 6 7)
+  (map 'list #'list numbers letters)
+#+end_src"
+
+    (progn
+      (org-babel-next-src-block)
+      (let ((results (org-babel-execute-src-block)))
+	(should(equal 'a (cadr (assoc 1 results))))
+	(should(equal 'd (cadr (assoc 4 results))))))))
 
 (ert-deftest test-org-babel/parse-header-args ()
-  (org-test-at-id "7eb0dc6e-1c53-4275-88b3-b22f3113b9c3"
-    (org-babel-next-src-block)
-    (let* ((info (org-babel-get-src-block-info))
-	   (params (nth 2 info)))
-      (message "%S" params)
-      (should(equal "example-lang" (nth 0 info)))
-      (should(string= "the body" (org-babel-trim (nth 1 info))))
-      (should-not (member '(:session\ \ \ \ ) params))
-      (should(equal '(:session) (assoc :session params)))
-      (should(equal '(:result-type . output) (assoc :result-type params)))
-      (should(equal '(num . 9) (cdr (assoc :var params)))))))
+  (org-test-with-temp-text-in-file "
+
+#+begin_src example-lang :session     :results output :var num=9
+  the body
+#+end_src"
+
+    (progn
+      (org-babel-next-src-block)
+      (let* ((info (org-babel-get-src-block-info))
+	     (params (nth 2 info)))
+	(message "%S" params)
+	(should(equal "example-lang" (nth 0 info)))
+	(should(string= "the body" (org-babel-trim (nth 1 info))))
+	(should-not (member '(:session\ \ \ \ ) params))
+	(should(equal '(:session) (assoc :session params)))
+	(should(equal '(:result-type . output) (assoc :result-type params)))
+	(should(equal '(num . 9) (cdr (assoc :var params))))))))
 
 (ert-deftest test-org-babel/parse-header-args2 ()
-  (org-test-at-id "2409e8ba-7b5f-4678-8888-e48aa02d8cb4"
-    (should (string-match (regexp-quote "this is simple")
-			  (org-babel-ref-resolve "simple-subtree")))
-    (org-babel-next-src-block)
-    (should (= 14 (org-babel-execute-src-block)))))
+  (org-test-with-temp-text-in-file "
+
+* resolving sub-trees as references
+ 
+#+begin_src emacs-lisp :var text=d4faa7b3-072b-4dcf-813c-dd7141c633f3
+  (length text)
+#+end_src
+
+#+begin_src org :noweb yes
+  <<simple-subtree>>
+  <<d4faa7b3-072b-4dcf-813c-dd7141c633f3>>
+#+end_src
+
+** simple subtree with custom ID
+   :PROPERTIES:
+   :CUSTOM_ID: simple-subtree
+   :END:
+this is simple"
+    
+    (progn
+      (should (string-match (regexp-quote "this is simple")
+			    (org-babel-ref-resolve "simple-subtree")))
+      (org-babel-next-src-block)
+      (should (= 14 (org-babel-execute-src-block))))))
 
 (ert-deftest test-org-babel/inline-src-blocks ()
   (org-test-at-id "54cb8dc3-298c-4883-a933-029b3c9d4b18"
@@ -193,8 +263,7 @@
       (should (re-search-forward ":results" nil t)) ;; 4
       (should (org-babel-get-inline-src-block-matches))
       (end-of-line)
-      (should-not (org-babel-get-inline-src-block-matches))
-    )))
+      (should-not (org-babel-get-inline-src-block-matches)))))
 
 (ert-deftest test-org-babel/inline-src_blk-default-results-replace-line-1 ()
   (let ((test-line "src_sh{echo 1}"))
@@ -369,12 +438,25 @@
 			(point-min) (point-max)))))))
 
 (ert-deftest test-org-babel/combining-scalar-and-raw-result-types ()
-  (flet ((next-result ()
-		      (org-babel-next-src-block)
-		      (org-babel-execute-src-block)
-		      (goto-char (org-babel-where-is-src-block-result))
-		      (forward-line 1)))
-    (org-test-at-id "a73a2ab6-b8b2-4c0e-ae7f-23ad14eab7bc"
+  (org-test-with-temp-text-in-file "
+
+#+begin_src sh :results scalar
+echo \"[[file:./cv.cls]]\"
+#+end_src
+
+#+name:
+: [[file:./cv.cls]]
+
+#+begin_src sh :results raw scalar
+   echo \"[[file:./cv.cls]]\"
+#+end_src
+"
+    (flet ((next-result ()
+			(org-babel-next-src-block)
+			(org-babel-execute-src-block)
+			(goto-char (org-babel-where-is-src-block-result))
+			(forward-line 1)))
+      (goto-char (point-min))
       (next-result)
       (should (org-babel-in-example-or-verbatim))
       (next-result)
diff --git a/testing/lisp/test-org-table.el b/testing/lisp/test-org-table.el
index 3ecc384..26eb47b 100644
--- a/testing/lisp/test-org-table.el
+++ b/testing/lisp/test-org-table.el
@@ -54,12 +54,28 @@
 ;;    (string= "$3 = remote(FOO, @@#$2)" (org-table-convert-refs-to-rc "C& = remote(FOO, @@#B&)"))))
 
 (ert-deftest test-org-table/simple-formula ()
-  (org-test-at-id "563523f7-3f3e-49c9-9622-9216cc9a5d95"
-    (re-search-forward (regexp-quote "#+tblname: simple-formula") nil t)
-    (forward-line 1)
-    (should (org-at-table-p))
-    (should (org-table-recalculate 'all))
-    (should (string= "10" (first (nth 5 (org-table-to-lisp)))))))
+  (org-test-with-temp-text-in-file "
+
+* simple formula
+  :PROPERTIES:
+  :ID:       563523f7-3f3e-49c9-9622-9216cc9a5d95
+  :END:
+
+#+tblname: simple-formula
+|  1 |
+|  2 |
+|  3 |
+|  4 |
+|----|
+|    |
+  #+TBLFM: $1=vsum(@1..@-1)
+"
+    (progn    
+      (re-search-forward (regexp-quote "#+tblname: simple-formula") nil t)
+      (forward-line 1)
+      (should (org-at-table-p))
+      (should (org-table-recalculate 'all))
+      (should (string= "10" (first (nth 5 (org-table-to-lisp))))))))
 
 (provide 'test-org-table)
 
-- 
1.7.3.4

Re: [patch][test] Avoid writes to non-temp test-example files

[Nick Dokos]

Martyn Jago <martyn.jago@btinternet.com> wrote:

> 
> Attached is a patch to modify all tests that currently write to non-temp
> test-example files such as `testing/babel.org'. Instead, the tests now
> write to temp files, which are thrown away at the conclusion of the
> test.
> 

Are the temp files kept if the test fails? IME, that's a useful debugging aid.

Nick

> All tests still pass.
> 
> HTH
> 
> Best, Martyn
> 

Re: [patch][test] Avoid writes to non-temp test-example files

[Bastien]

Martyn Jago <martyn.jago@btinternet.com> writes:

> Attached is a patch to modify all tests that currently write to non-temp
> test-example files such as `testing/babel.org'. Instead, the tests now
> write to temp files, which are thrown away at the conclusion of the
> test.

Applied, thanks!

-- 
 Bastien

Re: [patch][test] Avoid writes to non-temp test-example files

[Martyn Jago]

Nick Dokos <nicholas.dokos@hp.com> writes:

> Martyn Jago <martyn.jago@btinternet.com> wrote:
>
>> 
>> Attached is a patch to modify all tests that currently write to non-temp
>> test-example files such as `testing/babel.org'. Instead, the tests now
>> write to temp files, which are thrown away at the conclusion of the
>> test.
>> 
>
> Are the temp files kept if the test fails? IME, that's a useful debugging aid.
>
> Nick
>
>> All tests still pass.
>> 
>> HTH
>> 
>> Best, Martyn
>> 

If a test fails (or breaks in some way) and the temp-file has been
created, it remains in the `temporary-file-directory' (the test system
uses the standard Emacs make-temp-file mechanism). The file is
referenced in the ert failure backtrace, and may be viewed in Emacs (press `b' on
the failure in ert), although there is usually enough information in the
backtrace / test-case to point to the failure.

Emacs takes care of removing the old tmp files over time.

Best, Martyn

Re: [patch][test] Avoid writes to non-temp test-example files

[Nick Dokos]

Martyn Jago <martyn.jago@btinternet.com> wrote:

> Nick Dokos <nicholas.dokos@hp.com> writes:
> 
> > Martyn Jago <martyn.jago@btinternet.com> wrote:
> >
> >> 
> >> Attached is a patch to modify all tests that currently write to non-temp
> >> test-example files such as `testing/babel.org'. Instead, the tests now
> >> write to temp files, which are thrown away at the conclusion of the
> >> test.
> >> 
> >
> > Are the temp files kept if the test fails? IME, that's a useful debugging aid.
> >
> > Nick
> >
> >> All tests still pass.
> >> 
> >> HTH
> >> 
> >> Best, Martyn
> >> 
> 
> If a test fails (or breaks in some way) and the temp-file has been
> created, it remains in the `temporary-file-directory' (the test system
> uses the standard Emacs make-temp-file mechanism). The file is
> referenced in the ert failure backtrace, and may be viewed in Emacs (press `b' on
> the failure in ert), although there is usually enough information in the
> backtrace / test-case to point to the failure.
> 
> Emacs takes care of removing the old tmp files over time.
> 

Great - thanks!

Nick

Re: [patch][test] Avoid writes to non-temp test-example files

[Achim Gratz]

Martyn Jago <martyn.jago@btinternet.com> writes:
> Attached is a patch to modify all tests that currently write to non-temp
> test-example files such as `testing/babel.org'. Instead, the tests now
> write to temp files, which are thrown away at the conclusion of the
> test.

With NTemacs24 on Win7, if neither TMP nor TEMP environment variables
are set, it choses C:\ as the temp directory, which isn't writable for
normal users.  This makes all tests fail that write to temporary files.
I'm not sure how exactly it ends up with C:\, but this should either be
fixed or we need to make sure that TEMP or TMP is set from Makefile
before running tests.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptations for Waldorf Q V3.00R3 and Q+ V3.54R2:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada

Re: [patch][test] Avoid writes to non-temp test-example files

[Martyn Jago]

Achim Gratz <Stromeko@nexgo.de> writes:

> Martyn Jago <martyn.jago@btinternet.com> writes:
>> Attached is a patch to modify all tests that currently write to non-temp
>> test-example files such as `testing/babel.org'. Instead, the tests now
>> write to temp files, which are thrown away at the conclusion of the
>> test.
>
> With NTemacs24 on Win7, if neither TMP nor TEMP environment variables
> are set, it choses C:\ as the temp directory, which isn't writable for
> normal users.  This makes all tests fail that write to temporary files.
> I'm not sure how exactly it ends up with C:\, but this should either be
> fixed or we need to make sure that TEMP or TMP is set from Makefile
> before running tests.
>
>
> Regards,
> Achim.

Wow! Strictly speaking I guess that's an NTemacs24 bug, but we do have
the option of over-riding the default Emacs tmp location, which may
incur some extra house-keeping by the test-runners. I will leave that
decision to a maintainer.

I guess the makefile could issue a message on windows if an env var is
not set?

Best, Martyn

Re: [patch][test] Avoid writes to non-temp test-example files

[Achim Gratz]

Martyn Jago <martyn.jago@btinternet.com> writes:
> Wow! Strictly speaking I guess that's an NTemacs24 bug, but we do have
> the option of over-riding the default Emacs tmp location, which may
> incur some extra house-keeping by the test-runners. I will leave that
> decision to a maintainer.

As I said, I have not yet checked why it ends up where it does (it may
be some time before I get to the bottom of this).  It may still have to
do with the environment — I start NTemacs from Cygwin to be able to use
Cygwin's make and this combination is somewhat fragile.  Thanks for
confirming that it isn't a setting that the test framework alters.

> I guess the makefile could issue a message on windows if an env var is
> not set?

I'll have to think about the ramifications of doing this, but it is
certainly possible.  In fact, I might want to do that anyway to have
texi2pdf put the auxiliary files into a place within the build
directory, but not under doc.


Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf rackAttack:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds

Re: [patch][test] Avoid writes to non-temp test-example files

[Achim Gratz]

Achim Gratz <Stromeko@nexgo.de> writes:
> It may still have to
> do with the environment — I start NTemacs from Cygwin to be able to use
> Cygwin's make and this combination is somewhat fragile.

... and that seems indeed to be the culprit, running NTemacs from the
Cygwin shell somehow causes Emacs to assume C:\ for the temporary
directory, while it would correctly pick up the default location when
started from the shortcut.  Again, setting TEMP to some sane value fixes
the problem and since it is environment related there's nothing to fix
in org mode.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptation for Waldorf Blofeld V1.15B11:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada

Re: [patch][test] Avoid writes to non-temp test-example files

[Martyn Jago]

Achim Gratz <Stromeko@nexgo.de> writes:

> Achim Gratz <Stromeko@nexgo.de> writes:
>> It may still have to
>> do with the environment — I start NTemacs from Cygwin to be able to use
>> Cygwin's make and this combination is somewhat fragile.
>
> ... and that seems indeed to be the culprit, running NTemacs from the
> Cygwin shell somehow causes Emacs to assume C:\ for the temporary
> directory, while it would correctly pick up the default location when
> started from the shortcut.  Again, setting TEMP to some sane value fixes
> the problem and since it is environment related there's nothing to fix
> in org mode.
>

Thanks for the head's up - I invariably end up on windows machines when
doing embedded work so such information could be personally useful.

Best, Martyn

>
> Regards,
> Achim.