Problem with R source blocks

Problem with R source blocks

[Thomas S. Dye]

Aloha all,

Recently, some previously working R code (below) stopped working correctly.

It is not possible to produce a graphic without :session.  I get a red
message in the pdf file "arguments imply differing numbers of rows: 86,
79".

With :session, I first get one of two error messages in red in the pdf
file, either "Results must be all atomic, or all data frames" or
"subscript out of bounds".

With the "subscript out of bounds" error, if I run the source code block
again, I get the expected graphic.

With the "Results must be all atomic ..." message, if I re-run the
source code block several times, I see first an error "arguments imply
differing numbers of rows: 67, 60", then on the next run "arguments imply
differing numbers of rows: 86, 79" (like running outside a :session),
then the expected graphic.  Subsequent runs all produce the expected
graphic.

If I jump to the session, then print(g) works as expected, without any
error messages.

I haven't the faintest idea what might be happening, or how to debug.

#+begin_src emacs-lisp
(org-version)
#+end_src

#+results:
: 8.3beta

#+begin_src emacs-lisp
(emacs-version)
#+end_src

#+results:
: GNU Emacs 24.5.1 (x86_64-apple-darwin13.4.0, NS apple-appkit-1265.21)
:  of 2015-04-18 on tennine-slave.macports.org

All the best,
Tom

,-------------------------------------------------------------------------
| #+name: dated-interval-plot                                             
| #+header: :file ~/temp/test-plot.pdf                                    
| #+header: :var bcal = heiau-dates                                       
| #+header: :var base.font = 11                                           
| #+header: :var theme = "bw"                                             
| #+header: :var xlabel = "Years After Polynesian Discovery"              
| #+header: :var ylabel = "Probability"                                   
| #+header: :var rows=4                                                   
| #+header: :var cols=4                                                   
| #+header: :var xmin=0                                                   
| #+header: :var xmax=1000                                                
| #+header: :width 7 :height 5                                            
| #+header: :results graphics                                             
| #+BEGIN_SRC R :session                                                  
| library(ggplot2)                                                        
| library(ggthemes)                                                       
| bcal.df <- data.frame(cal=numeric(0),prob=numeric(0),label=character(0))
| for (i in seq(dim(bcal)[1]))                                            
| {                                                                       
|  t <- read.csv(file=bcal[i,1])                                          
|  names(t) <- c("cal","prob")                                            
|  t <- cbind(t,label=rep(bcal[i,2], dim(t)[1]))                          
|  bcal.df <- rbind(bcal.df,t)                                            
| }                                                                       
| switch(theme,                                                           
|     bw =, BW = theme_set(theme_bw(base_size=base.font)),                
|     grey =, gray = theme_set(theme_grey(base_size=base.font)),          
|     tufte = theme_set(theme_tufte(base_size=base.font)),                
|     calc = theme_set(theme_calc(base_size=base.font)),                  
|     econ =, economist = theme_set(theme_economist(base_size=base.font)),
|     excel = theme_set(theme_excel(base_size=base.font)),                
|     few = theme_set(theme_few(base_size=base.font)),                    
|     five = theme_set(theme_fivethirtyeight(base_size=base.font)),       
|     google =, gdocs = theme_set(theme_gdocs(base_size=base.font)),      
|     ## highchart =, hc = theme_set(theme_hc(base_size=base.font)),      
|     inverted = theme_set(theme_igray(base_size=base.font)),             
|     solarized = theme_set(theme_solarized(base_size=base.font)),        
|     solarized2 = theme_set(theme_solarized_2(base_size=base.font)),     
|     ## solid = theme_set(theme_solid(base_size=base.font)),             
|     stata = theme_set(theme_stata(base_size=base.font,scheme="s2mono")),
|     wsj = theme_set(theme_wsj(base_size=base.font)),                    
|     theme_set(theme_bw(base_size=base.font)))                           
| g <-  ggplot(bcal.df, aes(x=cal, y=prob))                               
| g <- g + geom_bar(stat='identity') + xlab(xlabel) + xlim(xmin,xmax)     
| g <- g + ylab(ylabel) + facet_wrap(~ label, nrow=rows, ncol=cols)       
| print(g)                                                                
| #+END_SRC                                                               
`-------------------------------------------------------------------------

-- 
T.S. Dye & Colleagues, Archaeologists
735 Bishop St, Suite 315, Honolulu, HI 96813
Tel: 808-529-0866, Fax: 808-529-0884
http://www.tsdye.com

Re: Problem with R source blocks

[Charles C. Berry]

On Thu, 11 Jun 2015, Thomas S. Dye wrote:

> Aloha all,
>
> Recently, some previously working R code (below) stopped working correctly.
>
> It is not possible to produce a graphic without :session.  I get a red
> message in the pdf file "arguments imply differing numbers of rows: 86,
> 79".
>
> With :session, I first get one of two error messages in red in the pdf
> file, either "Results must be all atomic, or all data frames" or
> "subscript out of bounds".
>
> With the "subscript out of bounds" error, if I run the source code block
> again, I get the expected graphic.
>
> With the "Results must be all atomic ..." message, if I re-run the
> source code block several times, I see first an error "arguments imply
> differing numbers of rows: 67, 60", then on the next run "arguments imply
> differing numbers of rows: 86, 79" (like running outside a :session),
> then the expected graphic.  Subsequent runs all produce the expected
> graphic.
>
> If I jump to the session, then print(g) works as expected, without any
> error messages.
>
> I haven't the faintest idea what might be happening, or how to debug.
>

The error messages are coming from R. cbind will say:

: Error in data.frame(..., check.names = FALSE) :
:   arguments imply differing number of rows: 2, 3

if you try cbind( data.frame(diag(2)), 1:3 )

Also, you are feeding lots of vars to the R src block, so that is a good 
place to look for trouble.

You could try to instrument `org-babel-R-evaluate-external-process' and 
see if anything jumps out at you.

But if not, ...

To debug, I'd try to replace your block with one that is utterly self 
contained - no need for external vars begin passed in from Org mode - and 
that invokes ggplot2 to produce a result. Once that works, define :var 
args and print them to a temp file from inside that R src block using

 	capture.output(print(your.var1),...,print(your.varK),
 		file = "myTemp.Rout")

Once that works, I'd add the first lines of code and print out 
intermediate values from inside your for() loop, but before the cbind() 
again using capture.output().

I expect by then the issue should be revealed.


HTH,

Chuck

Re: Problem with R source blocks

[Thomas S. Dye]

Hi Chuck,

"Charles C. Berry" <ccberry@ucsd.edu> writes:

> The error messages are coming from R. cbind will say:
>
> : Error in data.frame(..., check.names = FALSE) :
> :   arguments imply differing number of rows: 2, 3

I think the problem is in ggplot2, rather than my for loop:

,---------------------------------------------------------------------------
| [1] "ERROR : arguments imply differing number of rows: 76, 69"            
| Warning messages:                                                         
| 1: In loop_apply(n, do.ply) :                                             
|   Removed 3 rows containing missing values (position_stack).              
| 2: In loop_apply(n, do.ply) :                                             
|   Removed 1 rows containing missing values (position_stack).              
| 3: In loop_apply(n, do.ply) :                                             
|   Removed 3 rows containing missing values (position_stack).              
| null device                                                               
|           1                                                               
| > [1] "org_babel_R_eoe"                                                   
| > summary(bcal.df$label)                                                  
| 50-50-17-01088     Hapaiali`i          KAL-1        Makolea          KH2-2
|            100             60             82             69             72
|         KAL-26         KAL-27          KHO-1          KOL-2        MKI-122
|             79             72             65             63             54
|        MKI-123        MKI-124        MKI-125          KH1-3         KAL-24
|             62             69             65             65             57
|           Cook                                                            
|             41                                                            
`---------------------------------------------------------------------------

Note that the warnings removed 7 rows, which is the difference pointed
out by the error.

I don't know what position_stack might be, but it doesn't map directly
to the data frame I've constructed, which lacks an entry with 76 rows.

I'm guessing this brouhaha has to do with facet_wrap.

In any case, Org Babel seems to be behaving correctly.

I've had this code around for years and have run it without problem many
hundreds of times.

Is ess a potential source of the problem?  I upgraded that recently.

Thanks for your help,
Tom
-- 
Thomas S. Dye
http://www.tsdye.com

Re: Problem with R source blocks

[Charles C. Berry]

On Fri, 12 Jun 2015, Thomas S. Dye wrote:

> Hi Chuck,
>
> "Charles C. Berry" <ccberry@ucsd.edu> writes:
>
>> The error messages are coming from R. cbind will say:
>>
>> : Error in data.frame(..., check.names = FALSE) :
>> :   arguments imply differing number of rows: 2, 3
>
> I think the problem is in ggplot2, rather than my for loop:
>
> ,---------------------------------------------------------------------------
> | [1] "ERROR : arguments imply differing number of rows: 76, 69"
> | Warning messages:
> | 1: In loop_apply(n, do.ply) :
> |   Removed 3 rows containing missing values (position_stack).
> | 2: In loop_apply(n, do.ply) :
> |   Removed 1 rows containing missing values (position_stack).
> | 3: In loop_apply(n, do.ply) :
> |   Removed 3 rows containing missing values (position_stack).
> | null device


Those messages come from plyr, which ggplot2 uses.

[snip]
>
> In any case, Org Babel seems to be behaving correctly.
>
> I've had this code around for years and have run it without problem many
> hundreds of times.
>
> Is ess a potential source of the problem?  I upgraded that recently.
>

I don't think so.

I'd rewrite the src block with

 	save.image(file="setup-for-ggplot.RData")

before the graphics commands.

Restart emacs and execute the src block.

Start a fresh R session, then type

  load("setup-for-ggplot.RData")

and type/paste in all the graphics commands and see what happens. If the 
error repeats and you can verify that each object is as you expected it to 
be, then this is not an Org mode problem.

HTH,

Chuck

Re: Problem with R source blocks

[Thomas S. Dye]

"Charles C. Berry" <ccberry@ucsd.edu> writes:

> I'd rewrite the src block with
>
> 	save.image(file="setup-for-ggplot.RData")
>
> before the graphics commands.
>
> Restart emacs and execute the src block.
>
> Start a fresh R session, then type
>
>  load("setup-for-ggplot.RData")
>
> and type/paste in all the graphics commands and see what happens. If
> the error repeats and you can verify that each object is as you
> expected it to be, then this is not an Org mode problem.

Done.  I can verify this is not an Org mode problem.

I'll report it to the ggplot2 folks.

Thanks for your help.

All the best,
Tom

-- 
Thomas S. Dye
http://www.tsdye.com

Re: Problem with R source blocks

[Thomas S. Dye]

tsd@tsdye.com (Thomas S. Dye) writes:

> I'll report it to the ggplot2 folks.

Too late.  The bug in plyr was corrected a week ago.

> library(devtools)
> devtools::install_github("hadley/plyr")
> library(plyr)
> library(ggplot2)
> ...

Thanks again for your help.

All the best,
Tom

-- 
Thomas S. Dye
http://www.tsdye.com