Refresher on including R/ggplot2 output via latex/pdf?

Refresher on including R/ggplot2 output via latex/pdf?

[John Hendy]

Greetings,

I used to use orgmode + R/ggplot2 for some time, but for various
reasons I've been using python almost exclusively for a couple years.
I just went to use what I think is how I used to include ggplot2
results in PDF experts, but it's not working.

In referring to the documentation, this is listed as an example:
- https://orgmode.org/worg/org-contrib/babel/languages/ob-doc-R.html#org046a0ff

* does produce a file, by using :results output
#+begin_src R :file 3.png :results output graphics
library(lattice)
xyplot(1:10 ~ 1:10)
#+end_src

My test file:
-----
* test

#+begin_src R :results output graphics :exports results :file foo.png
library(ggplot2)

df <- data.frame(x=c(1, 2), y=c(2, 4))
p <- ggplot(df, aes(x = x, y = y)) + geom_point()
print(p)
#+end_src
-----

I'm using emacs -Q, followed by M-x load-file RET path/to/min-config:
-----
;; set load paths
;; set load dirs and global config options
(add-to-list 'load-path "~/.elisp/org/lisp/")
(add-to-list 'load-path "~/.elisp/org/contrib/lisp/")
(add-to-list 'load-path "~/.elisp/ess/lisp/")

(require 'ess-site)

; setup babel languages
(org-babel-do-load-languages
 'org-babel-load-languages
 '((R . t)))
-----

M-x org-version: Org mode version 9.2.4 (release_9.2.4-381-g226363 @
/home/jwhendy/.elisp/org/lisp/)

When I run the code, the #+RESULTS: output is just blank. foo.png *is*
generated, but the file link ([[foo.png]]) I'd expect is not getting
inserted. It also, unsurprisingly, does not show in the file.

Hmmm. One last hail Mary google incantation before I sent this off
suggests that perhaps it's :results file now? Or :results graphics
file?
- https://stackoverflow.com/questions/50054520/python-vs-r-in-org-mode-babel-output-graphics

Thanks for any pointers. I'd be happy to assist with updated
documentation if the earlier link is now outdated/incorrect.


Best regards,
John

Re: Refresher on including R/ggplot2 output via latex/pdf?

[Jack Kamm]

Your code block looks fine to me. The typical R code block headers I use is as follows:

:session :results output graphics :file path/to/file.png :exports results :eval never-export

One thing I've noticed, if I have an existing graphics device open, it
can mess up plots when executing an org-babel cell. Calling dev.off() a
few times fixes this for me.

Note I pretty much always use R blocks with the :session argument,
things might be pretty different without it.

Re: Refresher on including R/ggplot2 output via latex/pdf?

[John Hendy]

On Sun, Oct 27, 2019 at 11:31 PM Jack Kamm <jackkamm@gmail.com> wrote:
>
> Your code block looks fine to me. The typical R code block headers I use is as follows:

Did you try to reproduce my example, by chance?

> :session :results output graphics :file path/to/file.png :exports results :eval never-export

Using this:
-----
#+begin_src R :session :results output graphics :file foo.png :exports
results :eval never-export
library(ggplot2)

df <- data.frame(x=c(1, 2), y=c(2, 4))
p <- ggplot(df, aes(x = x, y = y)) + geom_point()
print(p)
#+end_src
-----

I get an empty #+RESULTS: header, but no inserted file link. The plot
is not in the exported PDF either.

If I make it this:
-----
#+begin_src R :session :results file graphics :file foo.png :exports
results :eval never-export
-----

I get the expected results, and the plot is in the output PDF.
-----
#+RESULTS:
[[file:foo.png]]
-----

> One thing I've noticed, if I have an existing graphics device open, it
> can mess up plots when executing an org-babel cell. Calling dev.off() a
> few times fixes this for me.
>
> Note I pretty much always use R blocks with the :session argument,
> things might be pretty different without it.

I usually use :session as well (and have it in my actual document, but
left it off for this test case). Still, using your header (minus
changing the file path/type) didn't work for me.

Thanks for taking a look,
John

Re: Refresher on including R/ggplot2 output via latex/pdf?

[Jack Kamm]

John Hendy <jw.hendy@gmail.com> writes:

> On Sun, Oct 27, 2019 at 11:31 PM Jack Kamm <jackkamm@gmail.com> wrote:
>>
>> Your code block looks fine to me. The typical R code block headers I use is as follows:
>
> Did you try to reproduce my example, by chance?

I've just now tried executing/exporting your example code blocks, and
they worked fine for me.

When I execute them I get a link to a result png inserted into the
buffer, and when I export to latex/pdf I get the plot image inserted
correctly into the PDF.

This is for the example code blocks in both the original and reply
emails, with and without the :session header.

Re: Refresher on including R/ggplot2 output via latex/pdf?

[John Hendy]

On Mon, Oct 28, 2019 at 12:02 AM Jack Kamm <jackkamm@gmail.com> wrote:
>
> John Hendy <jw.hendy@gmail.com> writes:
>
> > On Sun, Oct 27, 2019 at 11:31 PM Jack Kamm <jackkamm@gmail.com> wrote:
> >>
> >> Your code block looks fine to me. The typical R code block headers I use is as follows:
> >
> > Did you try to reproduce my example, by chance?
>
> I've just now tried executing/exporting your example code blocks, and
> they worked fine for me.
>
> When I execute them I get a link to a result png inserted into the
> buffer, and when I export to latex/pdf I get the plot image inserted
> correctly into the PDF.
>
> This is for the example code blocks in both the original and reply
> emails, with and without the :session header.

Wow, I'm stumped then. I just did a git pull and make clean/make and
still get the same results.

Org mode version 9.2.6 (release_9.2.6-559-ga01a8f @
/home/jwhendy/.elisp/org/lisp/)

This is my full file after C-c C-c on the block:
-----
* test

#+begin_src R :session :results output graphics :file foo.png :exports
results :eval never-export
library(ggplot2)

df <- data.frame(x=c(1, 2), y=c(2, 4))
p <- ggplot(df, aes(x = x, y = y)) + geom_point()
print(p)

#+end_src

#+RESULTS:
-----

This is with emacs -Q and loading the minimal config from the initial
email. Any ideas on where I might look next?

John

Re: Refresher on including R/ggplot2 output via latex/pdf?

[Jack Kamm]

> This is with emacs -Q and loading the minimal config from the initial
> email. Any ideas on where I might look next?

Sorry, I don't have many ideas here. Have you checked that ggplot works fine in a regular R session?

I tried again with emacs -Q and the minimal config below and it still worked fine for me:

----
;; set load paths
;; set load dirs and global config options
(add-to-list 'load-path "~/.emacs.d/elpa/org-plus-contrib-20190729/")
(add-to-list 'load-path "~/.emacs.d/elpa/ess-20190627.1806/")

(require 'ess)
(require 'ess-r-mode)

; setup babel languages
(org-babel-do-load-languages
 'org-babel-load-languages
 '((R . t)))
----

Re: Refresher on including R/ggplot2 output via latex/pdf?

[John Hendy]

On Mon, Oct 28, 2019 at 1:14 AM Jack Kamm <jackkamm@gmail.com> wrote:
>
> > This is with emacs -Q and loading the minimal config from the initial
> > email. Any ideas on where I might look next?
>
> Sorry, I don't have many ideas here. Have you checked that ggplot works fine in a regular R session?

Indeed, it does. And as mentioned, if I used :results file graphics,
everything works. It's able to plot and write to file... the magic
just isn't happening with :results output graphics for some reason.

> I tried again with emacs -Q and the minimal config below and it still worked fine for me:
>
> ----
> ;; set load paths
> ;; set load dirs and global config options
> (add-to-list 'load-path "~/.emacs.d/elpa/org-plus-contrib-20190729/")
> (add-to-list 'load-path "~/.emacs.d/elpa/ess-20190627.1806/")
>
> (require 'ess)
> (require 'ess-r-mode)
>
> ; setup babel languages
> (org-babel-do-load-languages
>  'org-babel-load-languages
>  '((R . t)))
> ----

Re: Refresher on including R/ggplot2 output via latex/pdf?

[Dominique Dumont]

On Monday, 28 October 2019 05:16:00 CET John Hendy wrote:
> I just went to use what I think is how I used to include ggplot2
> results in PDF experts, but it's not working.
[...]
> My test file:
> -----
> * test
> 
> #+begin_src R :results output graphics :exports results :file foo.png
> library(ggplot2)
> 
> df <- data.frame(x=c(1, 2), y=c(2, 4))
> p <- ggplot(df, aes(x = x, y = y)) + geom_point()
> print(p)
> #+end_src

I had a similar issue to get plots in a revealjs export. Long story short, I 
solved the issue by adding these instructions at the end of the R script:

 grid::grid.newpage()
 grid::grid.draw(p)

Hope this helps

Re: Refresher on including R/ggplot2 output via latex/pdf?

[Berry, Charles]

> On Oct 28, 2019, at 12:43 AM, John Hendy <jw.hendy@gmail.com> wrote:
> 
> On Mon, Oct 28, 2019 at 1:14 AM Jack Kamm <jackkamm@gmail.com> wrote:
>> 
>>> This is with emacs -Q and loading the minimal config from the initial
>>> email. Any ideas on where I might look next?
>> 
>> Sorry, I don't have many ideas here. Have you checked that ggplot works fine in a regular R session?
> 
> Indeed, it does. And as mentioned, if I used :results file graphics,
> everything works. It's able to plot and write to file... the magic
> just isn't happening with :results output graphics for some reason.
> 
> 


To wit:

commit 5c55d3a53c982563c0409e342b8940009e1409f2
Author: Nicolas Goaziou <mail@nicolasgoaziou.fr>
Date:   Sat Aug 24 00:04:06 2019 +0200

    manual: Remove erroneous footnote about :file header argument
    
    * doc/org-manual.org (Type): :file header argument no longer
    implies :results file.

commit 26ed66b23335eb389f1f2859e409f46f66279e15
Author: Nicolas Goaziou <mail@nicolasgoaziou.fr>
Date:   Sat Oct 6 08:56:05 2018 +0200

    ob: :file and :file-ext no longer imply :results file
    
    * lisp/ob-core.el (org-babel-execute-src-block): ":results file" must
      be specified in order to return a file.
    (org-babel-merge-params): :file and :file-ext no longer imply :results
    file.
    * testing/lisp/test-ob.el (test-ob/indented-cached-org-bracket-link):
    (test-ob/result-file-link-type-header-argument):
    (test-ob/result-graphics-link-type-header-argument): Update tests.
    
    Deducing the results from some other arguments is not obvious.
    Moreover, it prevents users from setting, e.g., :file-ext, in a node
    property, as every block would then create a file.
    
    Reported-by: Alex Fenton <alex@pressure.to>
    <http://lists.gnu.org/r/emacs-orgmode/2018-05/msg00469.html>

HTH,

Chuck

Re: Refresher on including R/ggplot2 output via latex/pdf?

[John Hendy]

On Mon, Oct 28, 2019 at 11:22 AM Berry, Charles <ccberry@ucsd.edu> wrote:
>
>
>
> > On Oct 28, 2019, at 12:43 AM, John Hendy <jw.hendy@gmail.com> wrote:
> >
> > On Mon, Oct 28, 2019 at 1:14 AM Jack Kamm <jackkamm@gmail.com> wrote:
> >>
> >>> This is with emacs -Q and loading the minimal config from the initial
> >>> email. Any ideas on where I might look next?
> >>
> >> Sorry, I don't have many ideas here. Have you checked that ggplot works fine in a regular R session?
> >
> > Indeed, it does. And as mentioned, if I used :results file graphics,
> > everything works. It's able to plot and write to file... the magic
> > just isn't happening with :results output graphics for some reason.
> >
> >
>
>
> To wit:
>
> commit 5c55d3a53c982563c0409e342b8940009e1409f2
> Author: Nicolas Goaziou <mail@nicolasgoaziou.fr>
> Date:   Sat Aug 24 00:04:06 2019 +0200
>
>     manual: Remove erroneous footnote about :file header argument
>
>     * doc/org-manual.org (Type): :file header argument no longer
>     implies :results file.
>
> commit 26ed66b23335eb389f1f2859e409f46f66279e15
> Author: Nicolas Goaziou <mail@nicolasgoaziou.fr>
> Date:   Sat Oct 6 08:56:05 2018 +0200
>
>     ob: :file and :file-ext no longer imply :results file
>
>     * lisp/ob-core.el (org-babel-execute-src-block): ":results file" must
>       be specified in order to return a file.
>     (org-babel-merge-params): :file and :file-ext no longer imply :results
>     file.
>     * testing/lisp/test-ob.el (test-ob/indented-cached-org-bracket-link):
>     (test-ob/result-file-link-type-header-argument):
>     (test-ob/result-graphics-link-type-header-argument): Update tests.
>
>     Deducing the results from some other arguments is not obvious.
>     Moreover, it prevents users from setting, e.g., :file-ext, in a node
>     property, as every block would then create a file.
>
>     Reported-by: Alex Fenton <alex@pressure.to>
>     <http://lists.gnu.org/r/emacs-orgmode/2018-05/msg00469.html>

Ok, this is promising. How do we explain the results, though?

I had this issue in the original post on:

Org mode version 9.2.4 (release_9.2.4-381-g226363 @
/home/jwhendy/.elisp/org/lisp/)

commit 226363482305d596bfb91e50246eb82d78689795
Author: Nicolas Goaziou <mail@nicolasgoaziou.fr>
Date:   Sat Jun 22 09:33:33 2019 +0200

Then I updated and still got the same behavior on:

Org mode version 9.2.6 (release_9.2.6-559-ga01a8f @
/home/jwhendy/.elisp/org/lisp/)

commit a01a8f55d8bf259ebecc050c162c48e29da13bee (HEAD -> master,
origin/master, origin/HEAD)
Merge: 58ceff522 9c611fd8a
Author: Kyle Meyer <kyle@kyleam.com>
Date:   Sun Oct 27 15:55:30 2019 -0400

My questions:

1) why didn't my example work for me on the commit from Jun 22 (< Aug or Oct)
2) why does this [still] work for Jack? (Jack, what's M-x org-version for you?)
3) optionally: can you reproduce the issue?

Thanks,
John

> HTH,
>
> Chuck
>
>
>

Re: Refresher on including R/ggplot2 output via latex/pdf?

[Jack Kamm]

> 2) why does this [still] work for Jack? (Jack, what's M-x org-version for you?)

I tested on my laptop and desktop, both work for me, they are running the following 2 versions of org:

Org mode version 9.2.4 (9.2.4-13-g9a543b-elpaplus @ /home/jack/.emacs.d/elpa/org-plus-contrib-20190729/)
Org mode version 9.2.6 (9.2.6-4-ge30905-elpaplus @ /home/jack/.emacs.d/elpa/org-plus-contrib-20190930/)

Re: Refresher on including R/ggplot2 output via latex/pdf?

[John Hendy]

On Mon, Oct 28, 2019 at 12:46 PM Jack Kamm <jackkamm@gmail.com> wrote:
>
> > 2) why does this [still] work for Jack? (Jack, what's M-x org-version for you?)
>
> I tested on my laptop and desktop, both work for me, they are running the following 2 versions of org:
>
> Org mode version 9.2.4 (9.2.4-13-g9a543b-elpaplus @ /home/jack/.emacs.d/elpa/org-plus-contrib-20190729/)
> Org mode version 9.2.6 (9.2.6-4-ge30905-elpaplus @ /home/jack/.emacs.d/elpa/org-plus-contrib-20190930/)

Closing the loop. I can confirm that my example works on this commit
(one before the relevant change) (thanks, Chuck!).

commit ed9bdfd220b75233e5bae2ef39164d14624060fa (HEAD)
Merge: 0954d4c25 0ae2e656d
Author: Marco Wahl <marcowahlsoft@gmail.com>
Date:   Fri Oct 5 00:54:19 2018 +0200

Completely stumped on how that works for you.

My final question; is the documentation accurate, or more accurately,
is it unambiguous?

14.8.2.2 :results
file Interpret as path to a file. Inserts a link to the file. Usage
example: :results value file.

14.8.2.3 :file
An external :file that saves the results of execution of the code
block... A link to the file is inserted.

As written, using :results file and :file both claim to insert a link
to the file. Should this be clarified?

Thanks all,
John

Re: Refresher on including R/ggplot2 output via latex/pdf?

[Jack Kamm]

> Closing the loop. I can confirm that my example works on this commit
> (one before the relevant change) (thanks, Chuck!).
>
> commit ed9bdfd220b75233e5bae2ef39164d14624060fa (HEAD)
> Merge: 0954d4c25 0ae2e656d
> Author: Marco Wahl <marcowahlsoft@gmail.com>
> Date:   Fri Oct 5 00:54:19 2018 +0200
>
> Completely stumped on how that works for you.

My version of org-mode is from September 30, shortly before this commit,
I guess that's why it still works for me. Though I'm confused why the
example didn't work for you on your older version of Org from June.

I guess I'll have to prepare for some breakage with ob-R next time I
update -- thanks for the heads up on this. I'll test it out and report
my findings when I get a chance.

Re: Refresher on including R/ggplot2 output via latex/pdf?

[John Hendy]

On Tue, Oct 29, 2019 at 9:51 AM Jack Kamm <jackkamm@gmail.com> wrote:
>
> > Closing the loop. I can confirm that my example works on this commit
> > (one before the relevant change) (thanks, Chuck!).
> >
> > commit ed9bdfd220b75233e5bae2ef39164d14624060fa (HEAD)
> > Merge: 0954d4c25 0ae2e656d
> > Author: Marco Wahl <marcowahlsoft@gmail.com>
> > Date:   Fri Oct 5 00:54:19 2018 +0200
> >
> > Completely stumped on how that works for you.
>
> My version of org-mode is from September 30, shortly before this commit,
> I guess that's why it still works for me. Though I'm confused why the
> example didn't work for you on your older version of Org from June.

Chuck clarified offline that the commit is from Oct *2018*, not 2019.
Both 9.2.4 and 9.2.6 are much more recent.

At least I interpret your response as thinking this is a recent change
(which is exactly what I thought at first!)?

John

> I guess I'll have to prepare for some breakage with ob-R next time I
> update -- thanks for the heads up on this. I'll test it out and report
> my findings when I get a chance.

Re: Refresher on including R/ggplot2 output via latex/pdf?

[Jack Kamm]

> 2) why does this [still] work for Jack? (Jack, what's M-x org-version for you?)

This thread is old, but I finally realized why we were having discrepant
results. It's because I was installing org-plus-contrib from org-mode
ELPA via package.el, which pulls from "maint" branch.

Since I've switched to pulling from git "master" branch, I've found that
I need to specify ":results file" in order to get the link to the plot
inserted.