using previous =#+results= when =:eval never=

using previous =#+results= when =:eval never=

[Greg Minshall]

hi.

i am trying to simplify adding regression test cases to a program.

to generate the base, "compared-to" results, i want to write some code
in a source block, then evaluate it, producing the "true" value.

then, later during development, i want to check if the code that ran in
that block gives the same results.  to do this, i preface the test check
block with, e.g., =:var fu=bar=.  obviously, i do *not* want to
re-create the base results; so, after producing the base case results, i
tried marking the source block that produces the results =:eval never=.

but, doing that, using `:var fu=bar` on a test check source block, fu's
value is nil. (*)

is there a way to convince org-mode to, in the face of =:eval never=, go
ahead and pass the *previous* results?  or, some other idea of how to do
this?  there will be a large number of these test cases.

cheers, Greg

(*) this is sort of confusing, so here's an example:

#+begin_src org
  ,#+name: testcountsdecompose
  ,#+begin_src R :eval never
    mtcars[1:3,]
  ,#+end_src
  
  ,#+RESULTS: testcountsdecompose
  |   21 | 6 | 160 | 110 |  3.9 |  2.62 | 16.46 | 0 | 1 | 4 | 4 |
  |   21 | 6 | 160 | 110 |  3.9 | 2.875 | 17.02 | 0 | 1 | 4 | 4 |
  | 22.8 | 4 | 108 |  93 | 3.85 |  2.32 | 18.61 | 1 | 1 | 4 | 1 |
  
  ,#+name: testcounts
  ,#+header: :var testcountsdecompose=testcountsdecompose :results output
  ,#+begin_src R
    str(testcountsdecompose)
  ,#+end_src
  
  ,#+RESULTS: testcounts
  :  chr "nil"
#+end_src

whereas, if the =testcountsdecompose= source block does *not* have
=:eval never=, my =testcountsdecompse= variable has all the rows and
columns i was hoping for.

Re: using previous =#+results= when =:eval never=

[Berry, Charles via General discussions about Org-mode.]

> On Jul 3, 2021, at 9:35 AM, Greg Minshall <minshall@umich.edu> wrote:
> 
> hi.
> 
> i am trying to simplify adding regression test cases to a program.
> 
> to generate the base, "compared-to" results, i want to write some code
> in a source block, then evaluate it, producing the "true" value.
> 
> then, later during development, i want to check if the code that ran in
> that block gives the same results.  to do this, i preface the test check
> block with, e.g., =:var fu=bar=.  obviously, i do *not* want to
> re-create the base results; so, after producing the base case results, i
> tried marking the source block that produces the results =:eval never=.
> 
> but, doing that, using `:var fu=bar` on a test check source block, fu's
> value is nil. (*)
> 
> is there a way to convince org-mode to, in the face of =:eval never=, go
> ahead and pass the *previous* results?  or, some other idea of how to do
> this?  there will be a large number of these test cases.
> 
> cheers, Greg


Greg,

I think it would be easier to leave :eval alone and instead evaluate the src block using `org-babel-ref-resolve' and compare to the current value. Something like this is a start:


#+begin_src emacs-lisp
  (defun org-babel-read-named-result (blkname)
    (save-excursion
      (org-babel-goto-named-result blkname)
      (org-babel-read-result)))
#+end_src


#+begin_src emacs-lisp
    (defun compare-old-to-new (refname)
      (let ((new (org-babel-ref-resolve refname))
	    (old (org-babel-read-named-result refname)))
	(or (equal old new)
	    (format "Comparison failed for block %s" refname))))
#+end_src

Then eval'ing `(compare-old-to-new "testcountsdecompose")` for the ECM you gave will give `t` if the result is the same and return a string saying which block failed the test.

I suppose you could loop thru `(org-babel-src-block-names)` if you want to check all the named blocks in a file.

HTH,

Chuck


> 
> (*) this is sort of confusing, so here's an example:
> 
> #+begin_src org
>  ,#+name: testcountsdecompose
>  ,#+begin_src R :eval never
>    mtcars[1:3,]
>  ,#+end_src
> 
>  ,#+RESULTS: testcountsdecompose
>  |   21 | 6 | 160 | 110 |  3.9 |  2.62 | 16.46 | 0 | 1 | 4 | 4 |
>  |   21 | 6 | 160 | 110 |  3.9 | 2.875 | 17.02 | 0 | 1 | 4 | 4 |
>  | 22.8 | 4 | 108 |  93 | 3.85 |  2.32 | 18.61 | 1 | 1 | 4 | 1 |
> 
>  ,#+name: testcounts
>  ,#+header: :var testcountsdecompose=testcountsdecompose :results output
>  ,#+begin_src R
>    str(testcountsdecompose)
>  ,#+end_src
> 
>  ,#+RESULTS: testcounts
>  :  chr "nil"
> #+end_src
> 
> whereas, if the =testcountsdecompose= source block does *not* have
> =:eval never=, my =testcountsdecompse= variable has all the rows and
> columns i was hoping for.
> 
> 

Re: using previous =#+results= when =:eval never=

[Greg Minshall]

Chuck,

thanks.  (i'm not surprised at an e-lisp suggestion from you! :)

i worry about accidental modification of the base case results during
the chaos of development.  it occurs to me (reading through
(org-babel-ref-resolve)) to keep my base case source blocks marked with
[:results silent], which should prevent accidental modification. (*)

then, i would change [:results silent] to, e.g., [:results silentx], in
the base case source blocks whenever i wanted to re-create those
results.

for, e.g., inspecting the results when things differ, it would be nice
to stay in the language of the rest of the code.  but i suspect i'll be
able to do some e-lisp magic on the RHS of a :var, e.g., crudely

: #+begin_src R :var basecaseresults=(org-babel-read-named-result "testcountsdecompose")

i'll play around with it.  (i suspect i'll be motivated to use an e-lisp
macro, to eliminate the quotes, for example.)

again, thanks.

cheers, Greg

(*) this relies on current org-mode behavior, where
(org-babel-read-result) will return results from a result block from a
source block marked [:results silent].  i have no idea how likely this
behavior is to change in the long run.

Re: using previous =#+results= when =:eval never=

[Berry, Charles via General discussions about Org-mode.]

> On Jul 3, 2021, at 10:19 PM, Greg Minshall <minshall@umich.edu> wrote:
> 
> Chuck,
> 
> thanks.  (i'm not surprised at an e-lisp suggestion from you! :)
> 
> i worry about accidental modification of the base case results during
> the chaos of development.  it occurs to me (reading through
> (org-babel-ref-resolve)) to keep my base case source blocks marked with
> [:results silent], which should prevent accidental modification. (*)
> 

This seems unnecessary. 

Be warned that the behavior of `org-babel-update-intermediate` is not intuitive - at least to me.  My reading of the doctstring is that the result of a named src block can be changed when it is non-nil. AFAICS, this never happens.  Instead, the return value from `org-babel-ref-resolve` is copied from the named result.    

For your ECM, after deleting the `:eval never`, if I append `+ 1` to the `mtcars[1:3,]` and execute then next src block, I get 

#+begin_src emacs-lisp
(compare-old-to-new "testcountsdecompose")
#+end_src

#+RESULTS:
: Comparison failed for block testcountsdecompose

and the original in-buffer value did not change.

I can remove the `+ 1` and rerun the above block and the result is `t`

HTH,

Chuck

> then, i would change [:results silent] to, e.g., [:results silentx], in
> the base case source blocks whenever i wanted to re-create those
> results.
> 
> for, e.g., inspecting the results when things differ, it would be nice
> to stay in the language of the rest of the code.  but i suspect i'll be
> able to do some e-lisp magic on the RHS of a :var, e.g., crudely
> 
> : #+begin_src R :var basecaseresults=(org-babel-read-named-result "testcountsdecompose")

> 
> i'll play around with it.  (i suspect i'll be motivated to use an e-lisp
> macro, to eliminate the quotes, for example.)
> 
> again, thanks.
> 
> cheers, Greg
> 
> (*) this relies on current org-mode behavior, where
> (org-babel-read-result) will return results from a result block from a
> source block marked [:results silent].  i have no idea how likely this
> behavior is to change in the long run.