Efficiently reuse code in org babel with R

Efficiently reuse code in org babel with R

[Naresh Gurbuxani]

I want to source some data from databases, then use this data in various blocks.  I am looking for the most efficient way to achieve this.  In below blocks, is the code in block ``connection'' executed thrice (economy of code but no saving of time) or is it only executed once then used twice?  If the code is run thrice, is there another way to create code blocks so that the code is executed only once?

Thanks,
Naresh

#+name: connection
#+begin_src R :exports results :results output
  ## my code here
#+end_src

#+begin_src R :exports results :results output graphics file :file figures/fig1.png
<<connection>>
## my code here
#+end_src

#+begin_src R :exports results output graphics file :file figures/fig2.png
<<connection>>
## my code here
#+end_src

Re: Efficiently reuse code in org babel with R

[Vikas Rawal]

[-- Attachment #1: Type: text/plain, Size: 1002 bytes --]

Why not specify a :session so that data created by one block would remain
available to subsequent blocks?

V.

On Thu, 7 Jul 2022 at 09:10, Naresh Gurbuxani <naresh_gurbuxani@hotmail.com>
wrote:

> I want to source some data from databases, then use this data in various
> blocks.  I am looking for the most efficient way to achieve this.  In below
> blocks, is the code in block ``connection'' executed thrice (economy of
> code but no saving of time) or is it only executed once then used twice?
> If the code is run thrice, is there another way to create code blocks so
> that the code is executed only once?
>
> Thanks,
> Naresh
>
> #+name: connection
> #+begin_src R :exports results :results output
>   ## my code here
> #+end_src
>
> #+begin_src R :exports results :results output graphics file :file
> figures/fig1.png
> <<connection>>
> ## my code here
> #+end_src
>
> #+begin_src R :exports results output graphics file :file figures/fig2.png
> <<connection>>
> ## my code here
> #+end_src
>

[-- Attachment #2: Type: text/html, Size: 1398 bytes --]

Re: Efficiently reuse code in org babel with R

[Greg Minshall]

Naresh,

> I want to source some data from databases, then use this data in
> various blocks.  I am looking for the most efficient way to achieve
> this.  In below blocks, is the code in block ``connection'' executed
> thrice (economy of code but no saving of time) or is it only executed
> once then used twice?  If the code is run thrice, is there another way
> to create code blocks so that the code is executed only once?

if you export this "file", the "connection" block will execute three
times.

Vikas suggested a session, which is something i often do for interactive
work.

or, the R code in "connection" could also use R variables to protect
itself against multiple executions.

or, i suppose you could invoke "connection" via a :var variable (rather
than via =<<connection>>), and use the :cache keyword to restrict
connection's evaluation to "every now and then" (for a very
precisely-defined, if not always intuitively definition of "every now
and then").  you'd have to be careful to not get bitten by the code
*not* being evaluated, and by it being evaluated too often.  (probably
too clever by half.)

hth, cheers, Greg

Re: Efficiently reuse code in org babel with R

[Jeremie Juste]

Hello Naresh,

Many thanks for sharing.
If I'm looking for efficiency, I tend not to use noweb and use an R
session instead.

One reason is that caching does not work with noweb

#+name: test
#+begin_src R  :cache yes :session *R*
Sys.sleep(10)
  a <- 1
#+end_src

#+RESULTS[36c41617bf9aa447ecc28fca8207eab38340d418]: test
: 1


#+name: add1
#+begin_src R :noweb yes :cache no :session *R*
<<test>>
 a + 1
#+end_src

#+RESULTS: add1
: 2


I would proceed in the following way

    #+name: connection
    #+begin_src R :exports results :results output :session *R* 
     ## my code here
    #+end_src

    #+begin_src R :exports results :results output graphics file
    :session *R* :file figures/fig1.png
    ## my code here
    #+end_src

    #+begin_src R :exports results output graphics file :file figures/fig2.png :session *R*
    ## my code here
    #+end_src

this will require running the first code chunk once only to run the rest

On Thursday,  7 Jul 2022 at 09:22, Greg Minshall wrote:
>  you'd have to be careful to not get bitten by the code
> *not* being evaluated, and by it being evaluated too often.  (probably
> too clever by half.)

The comment of Greg is very relevant here as you'll have to manage
updates yourself, and it can get complex very quickly. A potential
pitfall is when the tables in database you are referring to, change.
So in this case I would avoid using :cache yes in the first code chunk


I'm also tempted to adopt the following organization

    * setup table
    #+name: connection
    #+begin_src R :exports results :results output :session *R* 
     ## my code here
    #+end_src
    
    * output graphs
     :PROPERTIES:
     :header-args:R: :exports results output graphics file  :session *R*
     :END:
     
    #+begin_src R  :file figures/fig2.png
    ## my code here
    #+end_src

    #+begin_src R  :file figures/fig1.png 
    ## my code here
    #+end_src

then I would use (org-babel-execute-subtree) which is bound to C-c C-v
s on the subheading output graphs to regenerate the graphs.


HTH,
Jeremie

Re: Efficiently reuse code in org babel with R

[Naresh Gurbuxani]

Thanks for providing a detailed solution.  

I also learned from other responses.

Sent from my iPhone

> On Jul 7, 2022, at 4:42 AM, Jeremie Juste <jeremiejuste@gmail.com> wrote:
> 
> Hello Naresh,
> 
> Many thanks for sharing.
> If I'm looking for efficiency, I tend not to use noweb and use an R
> session instead.
> 
> One reason is that caching does not work with noweb
> 
> #+name: test
> #+begin_src R  :cache yes :session *R*
> Sys.sleep(10)
>  a <- 1
> #+end_src
> 
> #+RESULTS[36c41617bf9aa447ecc28fca8207eab38340d418]: test
> : 1
> 
> 
> #+name: add1
> #+begin_src R :noweb yes :cache no :session *R*
> <<test>>
> a + 1
> #+end_src
> 
> #+RESULTS: add1
> : 2
> 
> 
> I would proceed in the following way
> 
>    #+name: connection
>    #+begin_src R :exports results :results output :session *R* 
>     ## my code here
>    #+end_src
> 
>    #+begin_src R :exports results :results output graphics file
>    :session *R* :file figures/fig1.png
>    ## my code here
>    #+end_src
> 
>    #+begin_src R :exports results output graphics file :file figures/fig2.png :session *R*
>    ## my code here
>    #+end_src
> 
> this will require running the first code chunk once only to run the rest
> 
>> On Thursday,  7 Jul 2022 at 09:22, Greg Minshall wrote:
>> you'd have to be careful to not get bitten by the code
>> *not* being evaluated, and by it being evaluated too often.  (probably
>> too clever by half.)
> 
> The comment of Greg is very relevant here as you'll have to manage
> updates yourself, and it can get complex very quickly. A potential
> pitfall is when the tables in database you are referring to, change.
> So in this case I would avoid using :cache yes in the first code chunk
> 
> 
> I'm also tempted to adopt the following organization
> 
>    * setup table
>    #+name: connection
>    #+begin_src R :exports results :results output :session *R* 
>     ## my code here
>    #+end_src
> 
>    * output graphs
>     :PROPERTIES:
>     :header-args:R: :exports results output graphics file  :session *R*
>     :END:
> 
>    #+begin_src R  :file figures/fig2.png
>    ## my code here
>    #+end_src
> 
>    #+begin_src R  :file figures/fig1.png 
>    ## my code here
>    #+end_src
> 
> then I would use (org-babel-execute-subtree) which is bound to C-c C-v
> s on the subheading output graphs to regenerate the graphs.
> 
> 
> HTH,
> Jeremie