Re: org-mode and python pandas

[Eric Schulte <schulte.eric@gmail.com>]

Dov Grobgeld <dov.grobgeld@gmail.com> writes:

> Has anyone used org-mode with the python pandas package? Pandas is in
> a certain way an alternative to R, but with the (for me) familiar
> syntax of python. See: http://pandas.pydata.org/
>
> Pandas is very much built to be used interactively, and it outputs its
> data in space separated tabular format. E.g. in ipython:
>
> In [1]: import pandas as pd
> In [2]: import numpy as np
>
> In [3]: pd.DataFrame(np.random.random((4,3)), columns=['A','B','C'])
> Out[3]:
>           A         B         C
> 0  0.628365  0.424279  0.619791
> 1  0.799666  0.527572  0.132928
> 2  0.837255  0.138906  0.408233
> 3  0.388080  0.146212  0.575346
>
> Unfortunately this doesn't output as nicely when used from org-mode:
>
> #+BEGIN_SRC python
> import pandas as pd
> import numpy as np
>
> return pd.DataFrame(np.random.random((4,3)), columns=list('ABC'))
> #+END_SRC
>
> #+RESULTS:
> : A         B         C
> : 0  0.827817  0.664009  0.089161
> : 1  0.170031  0.729214  0.110918
> : 2  0.575918  0.863924  0.757536
> : 3  0.682722  0.774445  0.992041
>
> while I would like to have:
>
> |   |        A |        B |        C |
> |---+----------+----------+----------|
> | 0 | 0.827817 | 0.664009 | 0.089161 |
> | 1 | 0.170031 | 0.729214 | 0.110918 |
> | 2 | 0.575918 | 0.863924 | 0.757536 |
> | 3 | 0.682722 | 0.774445 | 0.992041 |
>

What happens if you add ":results table" to your code block?  Would that
be sufficient?

>
> The question is how to get this? Here are a few ideas:
>
> 1. Write a general filter in the org-mode elisp than uses heuristics
> to recognize ascii aligned tables and change these to org-tables.

The default value should be to convert multi-line output to tables, the
":results table" option above will force this conversion in case it is
currently not taking place due to the default header arguments in use.

> 
> 2. Add to pandas the option of globally influencing the text
> formatting so that it outputs something more parsable by org-mode.

This sounds promising, if pandas support csv output that will be
correctly parsed by Org-mode.

> 
> 3. Create a special language "pandas" that recognize the ascii aligned
> tables and saves the need to import pandas and np?  4. And the obvious
> approach of writing a python function that writes a org-mode parsable
> table and always call it as part of the return.
>
> Which is the preferable approach? Any other ideas?
>

I think a header-argument-based approach would be ideal, I'd look at the
value of org-babel-default-header-args:python, and read the portion of
the manual related to the "results" header arguments.

I don't understand multi-line strings in python, but I get the following
behavior from simple shell script blocks.

#+begin_src sh
  cat <<EOF
            A         B         C
  0  0.628365  0.424279  0.619791
  1  0.799666  0.527572  0.132928
  2  0.837255  0.138906  0.408233
  3  0.388080  0.146212  0.575346
  EOF
#+end_src

#+RESULTS:
| A |        B |        C |          |
| 0 | 0.628365 | 0.424279 | 0.619791 |
| 1 | 0.799666 | 0.527572 | 0.132928 |
| 2 | 0.837255 | 0.138906 | 0.408233 |
| 3 |  0.38808 | 0.146212 | 0.575346 |

#+begin_src sh
  cat <<EOF
  ,A,B,C
  0,0.628365,0.424279,0.619791
  1,0.799666,0.527572,0.132928
  2,0.837255,0.138906,0.408233
  3,0.388080,0.146212,0.575346
  EOF
#+end_src

#+RESULTS:
|   |        A |        B |        C |
| 0 | 0.628365 | 0.424279 | 0.619791 |
| 1 | 0.799666 | 0.527572 | 0.132928 |
| 2 | 0.837255 | 0.138906 | 0.408233 |
| 3 |  0.38808 | 0.146212 | 0.575346 |

Hope this helps,

>
> Regards,
> Dov
>

-- 
Eric Schulte
http://cs.unm.edu/~eschulte