org-babel strange html print in R

org-babel strange html print in R

[Jeremie Juste]

Hello,

I've found that some strange results when outputing html from R.
Do you have some insights on a potential solution?

Best regards,
Jeremie

Org mode version 9.3.4 (9.3.4-elpa @ /home/djj/.emacs.d/elpa/org-20200206/)

#+PROPERTY: header-args:R  :eval yes  :exports results :colnames yes :results output :output-dir images/    :cache yes  :session *R* 

#+begin_src R :results output html
library(xtable)
x <- rnorm(100)
y <- x + rnorm(100)
print(xtable(summary(lm(y ~ x))),type="html")
#+end_src


#+RESULTS:
#+begin_export html

<!-- html table generated in R 3.6.1 by xtable 1.8-4 package -->
<!-- Thu Feb  6 19:13:34 2020 -->
<table border=1>
<
<
 </
<
Estimate </
<
Std. Error </
<
t value </
<
Pr(&gt;|t|) </
 </tr>
  <
<td align="right"
(Intercept) </
<td align="right"
-0.0130 </
<td align="right"
0.1023 </
<td align="right"
-0.13 </
<td align="right"
0.8991 </
</tr>
  <
<td align="right"
x </
<td align="right"
1.0011 </
<td align="right"
0.0987 </
<td align="right"
10.14 </
<td align="right"
0.0000 </
</tr>
   </table>
#+end_export


** Expected results
#+begin_export html
<table border=1>
<tr> <th>  </th> <th> Estimate </th> <th> Std. Error </th> <th> t value </th> <th> Pr(&gt;|t|) </th>  </tr>
  <tr> <td align="right"> (Intercept) </td> <td align="right"> -0.0130 </td> <td align="right"> 0.1023 </td> <td align="right"> -0.13 </td> <td align="right"> 0.8991 </td> </tr>
  <tr> <td align="right"> x </td> <td align="right"> 1.0011 </td> <td align="right"> 0.0987 </td> <td align="right"> 10.14 </td> <td align="right"> 0.0000 </td> </tr>
   </table>
#+end_export

Re: org-babel strange html print in R

[Dominique Dumont]

On Thursday, 6 February 2020 19:23:36 CET Jeremie Juste wrote:
> I've found that some strange results when outputing html from R.
> Do you have some insights on a potential solution?

On my system, org-mode 3.6.1 provides the expected table. 

All the best

Re: org-babel strange html print in R

[Bastien]

Hi Dominique,

Dominique Dumont <domi.dumont@free.fr> writes:

> On Thursday, 6 February 2020 19:23:36 CET Jeremie Juste wrote:
>> I've found that some strange results when outputing html from R.
>> Do you have some insights on a potential solution?
>
> On my system, org-mode 3.6.1

This seems to be a *very* version of org-mode ;)

-- 
 Bastien

Re: org-babel strange html print in R

[Jack Kamm]

Dominique Dumont <domi.dumont@free.fr> writes:

> On my system, org-mode 3.6.1 provides the expected table. 

My system produces the expected table as well (org 9.3.2, R 3.6.2).

--Jack

Re: org-babel strange html print in R

[Dominique Dumont]

On Friday, 7 February 2020 14:38:53 CET Bastien wrote:
> This seems to be a *very* version of org-mode ;)

Bummer... I remember that I've used org-version to get the version number. I 
guess I have cut'n'pasted an unrelated number :-/

For the record, I've tested R and html output on Org mode version 9.3.1

Sorry for the confusion.

All the best

Dod 

Re: org-babel strange html print in R

[Jeremie Juste]

Hello,


Jack Kamm <jackkamm@gmail.com> writes:

> Dominique Dumont <domi.dumont@free.fr> writes:
>
>> On my system, org-mode 3.6.1 provides the expected table. 
>
> My system produces the expected table as well (org 9.3.2, R 3.6.2).
>
> --Jack
Thanks for the info. 
Unfortunately I couldn't identify the issue
I updated R and org (org 9.3.2, R 3.6.2).


launched
> emacs -Q

- executed the following commands

(add-to-list 'load-path  "/home/djj/.emacs.d/elpa/ess-20200115.225")
(add-to-list 'load-path  "/home/djj/.emacs.d/elpa/julia-mode-20180816.2117")
(require 'ess-site)
 (org-babel-do-load-languages
   'org-babel-load-languages
   '( (R . t)
     ))

but I'm still getting 

#+begin_src R :results output  :cache no :session "R2"
  library(xtable)
  x <- rnorm(100)
  y <- x + rnorm(100)
  a <- summary(lm(y ~ x))
  print(xtable(a),type="html")
#+end_src

#+RESULTS:
#+begin_example

<!-- html table generated in R 3.6.2 by xtable 1.8-4 package -->
<!-- Sat Feb  8 00:49:03 2020 -->
<table border=1>
<
<
 </
<
Estimate </
<
Std. Error </
<
t value </
<
Pr(&gt;|t|) </
 </tr>
  <
<td align="right"
(Intercept) </
<td align="right"
0.0353 </
<td align="right"
0.1019 </
<td align="right"
0.35 </
<td align="right"
0.7294 </
</tr>
  <
<td align="right"
x </
<td align="right"
0.9426 </
<td align="right"
0.0882 </
<td align="right"
10.68 </
<td align="right"
0.0000 </
</tr>
   </table>
#+end_example

Best regards,

Jeremie

Re: org-babel strange html print in R

[Jack Kamm]

Hi Jeremie,

> Thanks for the info. 
> Unfortunately I couldn't identify the issue
> I updated R and org (org 9.3.2, R 3.6.2).
> but I'm still getting [the error]

Sorry, I missed the initial property line where you set the ":session"
header argument.

When I set the ":session" header argument, I get the original buggy
behavior that you report.

It looks like the reason for this, is that the substrings like "tr>"
match `comint-prompt-regexp', and so get stripped out by
`org-babel-comint-with-output'.

I think a more reliable implementation would be to avoid
`org-babel-comint-with-output', and take the approach used in the
":results value" case, which reads the output from a temp file instead
of directly from the shell.

Actually, that's the approach I use in my ob-session-async package [0],
which re-implements ob-R sessions to allow async evaluation. It doesn't
suffer from the bug you are reporting. I'll plan to extract that
implementation and submit it as a patch here.

Long term, I am intending to contribute the full functionality in
ob-session-async here, but that's still a ways off.

[0] https://github.com/jackkamm/ob-session-async

Jack

Re: org-babel strange html print in R

[Jeremie Juste]

Hello Jack,

Thanks for taking the time to reply.

> I think a more reliable implementation would be to avoid
> `org-babel-comint-with-output', and take the approach used in the
> ":results value" case, which reads the output from a temp file instead
> of directly from the shell.
I understand better now. Thanks
>
> Actually, that's the approach I use in my ob-session-async package [0],
> which re-implements ob-R sessions to allow async evaluation. It doesn't
> suffer from the bug you are reporting. I'll plan to extract that
> implementation and submit it as a patch here.

Your solution works just fine. I came across ob-session-async a recently
and find it very interesting thanks again.

#+begin_src R :results output html :cache no :async session
  library(xtable)
  x <- rnorm(100)
  y <- x + rnorm(100)
  a <- summary(lm(y ~ x))
  print(xtable(a),type="html",comment=FALSE)
#+end_src

#+RESULTS:
#+begin_export html
<table border=1>
<tr> <th>  </th> <th> Estimate </th> <th> Std. Error </th> <th> t value </th> <th> Pr(&gt;|t|) </th>  </tr>
  <tr> <td align="right"> (Intercept) </td> <td align="right"> 0.1249 </td> <td align="right"> 0.0961 </td> <td align="right"> 1.30 </td> <td align="right"> 0.1969 </td> </tr>
  <tr> <td align="right"> x </td> <td align="right"> 0.9263 </td> <td align="right"> 0.0879 </td> <td align="right"> 10.53 </td> <td align="right"> 0.0000 </td> </tr>
   </table>
#+end_export

Best regards,
Jeremie

Re: org-babel strange html print in R

[stardiviner]

You should try ~:results value html~. Here is what I got correctly:

#+begin_src R :results value html
library(xtable)
x <- rnorm(100)
y <- x + rnorm(100)
print(xtable(summary(lm(y ~ x))),type="html")
#+end_src

#+RESULTS[<2020-02-11 13:14:59> 91ff0df2529284897e3585e3ec2e640290ee8308]:
#+begin_export html
<!-- html table generated in R 3.6.2 by xtable 1.8-3 package -->
<!-- Tue Feb 11 13:14:58 2020 -->
<table border=1>
<tr> <th>  </th> <th> Estimate </th> <th> Std. Error </th> <th> t value </th> <th> Pr(&gt;|t|) </th>  </tr>
  <tr> <td align="right"> (Intercept) </td> <td align="right"> -0.0064 </td> <td align="right"> 0.1069 </td> <td align="right"> -0.06 </td> <td align="right"> 0.9520 </td> </tr>
  <tr> <td align="right"> x </td> <td align="right"> 1.1066 </td> <td align="right"> 0.1022 </td> <td align="right"> 10.83 </td> <td align="right"> 0.0000 </td> </tr>
   </table>
#+end_export

Jeremie Juste <jeremiejuste@gmail.com> writes:

> Hello,
>
> I've found that some strange results when outputing html from R.
> Do you have some insights on a potential solution?
>
> Best regards,
> Jeremie
>
> Org mode version 9.3.4 (9.3.4-elpa @ /home/djj/.emacs.d/elpa/org-20200206/)
>
> #+PROPERTY: header-args:R  :eval yes  :exports results :colnames yes :results output :output-dir images/    :cache yes  :session *R* 
>
> #+begin_src R :results output html
> library(xtable)
> x <- rnorm(100)
> y <- x + rnorm(100)
> print(xtable(summary(lm(y ~ x))),type="html")
> #+end_src
>
>
> #+RESULTS:
> #+begin_export html
>
> <!-- html table generated in R 3.6.1 by xtable 1.8-4 package -->
> <!-- Thu Feb  6 19:13:34 2020 -->
> <table border=1>
> <
> <
>  </
> <
> Estimate </
> <
> Std. Error </
> <
> t value </
> <
> Pr(&gt;|t|) </
>  </tr>
>   <
> <td align="right"
> (Intercept) </
> <td align="right"
> -0.0130 </
> <td align="right"
> 0.1023 </
> <td align="right"
> -0.13 </
> <td align="right"
> 0.8991 </
> </tr>
>   <
> <td align="right"
> x </
> <td align="right"
> 1.0011 </
> <td align="right"
> 0.0987 </
> <td align="right"
> 10.14 </
> <td align="right"
> 0.0000 </
> </tr>
>    </table>
> #+end_export
>
>
> ** Expected results
> #+begin_export html
> <table border=1>
> <tr> <th>  </th> <th> Estimate </th> <th> Std. Error </th> <th> t value </th> <th> Pr(&gt;|t|) </th>  </tr>
>   <tr> <td align="right"> (Intercept) </td> <td align="right"> -0.0130 </td> <td align="right"> 0.1023 </td> <td align="right"> -0.13 </td> <td align="right"> 0.8991 </td> </tr>
>   <tr> <td align="right"> x </td> <td align="right"> 1.0011 </td> <td align="right"> 0.0987 </td> <td align="right"> 10.14 </td> <td align="right"> 0.0000 </td> </tr>
>    </table>
> #+end_export


-- 
[ stardiviner ]
       I try to make every word tell the meaning what I want to express.

       Blog: https://stardiviner.github.io/
       IRC(freenode): stardiviner, Matrix: stardiviner
       GPG: F09F650D7D674819892591401B5DF1C95AE89AC3
      

Re: org-babel strange html print in R

[Jack Kamm]

Hi,

stardiviner <numbchild@gmail.com> writes:

> You should try ~:results value html~. Here is what I got correctly:

Yes, this seems like the best solution for this case.

On the other hand, it would be desirable if ":results output" didn't
mangle the output, due to trying to clean up prompts. There is probably
some corner case out there where this is a more serious issue -- but for
this case, it seems best to just use ":results value html".

I was planning to submit my implementation of ":session :results output"
that avoids the prompt mangling [0], but now I'm leaning towards putting
this off, until a more compelling case arises to justify it.

[0] https://github.com/jackkamm/ob-session-async/blob/master/lisp/ob-session-async-R.el

Re: org-babel strange html print in R

[Jeremie Juste]

Hello,

Thanks for the input.
I didn't see the full mail of stardiviner for some reason.

:results value html  in the following code generates an error for me.
(Org mode version 9.2.3,R version 3.6.2).
but :results value output the "html" in an org-table format

#+begin_src R :results value html
library(xtable)
x <- rnorm(100)
y <- x + rnorm(100)
print(xtable(summary(lm(y ~ x))),type="html")
#+end_src

Best regards,
Jeremie

Jack Kamm <jackkamm@gmail.com> writes:

> Hi,
>
> stardiviner <numbchild@gmail.com> writes:
>
>> You should try ~:results value html~. Here is what I got correctly:
>
> Yes, this seems like the best solution for this case.
>
> On the other hand, it would be desirable if ":results output" didn't
> mangle the output, due to trying to clean up prompts. There is probably
> some corner case out there where this is a more serious issue -- but for
> this case, it seems best to just use ":results value html".
>
> I was planning to submit my implementation of ":session :results output"
> that avoids the prompt mangling [0], but now I'm leaning towards putting
> this off, until a more compelling case arises to justify it.
>
> [0] https://github.com/jackkamm/ob-session-async/blob/master/lisp/ob-session-async-R.el

Re: org-babel strange html print in R

[Jack Kamm]

Hi,

Jeremie Juste <jeremiejuste@gmail.com> writes:

> :results value html  in the following code generates an error for me.
> (Org mode version 9.2.3,R version 3.6.2).
> but :results value output the "html" in an org-table format

Hmm, I can confirm that ":session :results value html" is working for
me, same as for stardiviner. Could you give more info about the error
you're seeing?

I'm using R 3.6.2 and Org 9.4. But I'd be surprised if it doesn't work
for Org 9.2 -- git blame suggests that html output has been supported in
ob-R since at least 2015. But it wouldn't hurt to upgrade to the latest
released version of Org (9.3).

Re: org-babel strange html print in R

[stardiviner]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256


Jeremie Juste <jeremiejuste@gmail.com> writes:

> Hello,
>
> Thanks for the input.
> I didn't see the full mail of stardiviner for some reason.

About this, I found Gmail put my emails into Spams folder. I can't figure out
what reasons. I also found similar situations in some people in Emacs-help guys.
Anyway, If you can, please report my emails "Not Spam". Thanks :)

>
> :results value html  in the following code generates an error for me.
> (Org mode version 9.2.3,R version 3.6.2).
> but :results value output the "html" in an org-table format
>
> #+begin_src R :results value html
> library(xtable)
> x <- rnorm(100)
> y <- x + rnorm(100)
> print(xtable(summary(lm(y ~ x))),type="html")
> #+end_src
>
> Best regards,
> Jeremie
>
> Jack Kamm <jackkamm@gmail.com> writes:
>
>> Hi,
>>
>> stardiviner <numbchild@gmail.com> writes:
>>
>>> You should try ~:results value html~. Here is what I got correctly:
>>
>> Yes, this seems like the best solution for this case.
>>
>> On the other hand, it would be desirable if ":results output" didn't
>> mangle the output, due to trying to clean up prompts. There is probably
>> some corner case out there where this is a more serious issue -- but for
>> this case, it seems best to just use ":results value html".
>>
>> I was planning to submit my implementation of ":session :results output"
>> that avoids the prompt mangling [0], but now I'm leaning towards putting
>> this off, until a more compelling case arises to justify it.
>>
>> [0] https://github.com/jackkamm/ob-session-async/blob/master/lisp/ob-session-async-R.el


- -- 
[ stardiviner ]
       I try to make every word tell the meaning what I want to express.

       Blog: https://stardiviner.github.io/
       IRC(freenode): stardiviner, Matrix: stardiviner
       GPG: F09F650D7D674819892591401B5DF1C95AE89AC3
      
-----BEGIN PGP SIGNATURE-----

iQFIBAEBCAAyFiEE8J9lDX1nSBmJJZFAG13xyVromsMFAl5Do/gUHG51bWJjaGls
ZEBnbWFpbC5jb20ACgkQG13xyVromsN8ewf/YkjUlLOAAAtpBRgOdIZg2N0JKxVt
vwXxH5G+5VOJNgH0avDRrGnI+2r5Kooqs0lpfIjoy5yi9tkG9Z4GShKa2iYP2wLA
hbsN0OJxZ3ChdCjbyW2ABNdzCVL6T3oremxy8DzdZGrM2sSmFFSKK+N8s/PI7IJP
2wE7jjUCoNFcB6XGodcZiRXE+Ms/k1gPfzyJxvbeOdX1GQriQE8ecofEpomkz3F0
VQZ/sgec9mAsKZMC9054H0br5DT1c3Wo/DnaKnVpw24wrwWyMJSj02XSau+GIFLq
3hAx01mrP4ud7aUs/TnwCfaVrvH0i/4iO+KhJr6KyZB2gPMTXTs4nuf5Zw==
=dAR8
-----END PGP SIGNATURE-----