org-mode, python and unicode

org-mode, python and unicode

[Daniele Pizzolli]

Hello,

I found some problems and workarounds in the use of python (version
2) and org-mode.  This applies to:

#+BEGIN_EXAMPLE
Org-mode version 8.3.4 (8.3.4-60-g19cf68-elpa @ /home/vagrant/.emacs.d/elpa/org-20160516/)
#+END_EXAMPLE

Some workaround are not complete, and suggestions are welcome!

Best,
Daniele

* python

** load python support for babel

#+BEGIN_SRC emacs-lisp
(org-babel-do-load-languages
 'org-babel-load-languages
 '((python . t)))
#+END_SRC

#+RESULTS:
: ((python . t))

** passing utf-8 strings to python

#+NAME: unicode_str
#+BEGIN_EXAMPLE
“this string is not ascii!”
#+END_EXAMPLE

#+NAME: error-in-passing-var
#+BEGIN_SRC python :var data=unicode_str
return data
#+END_SRC

#+RESULTS: error-in-passing-var

Will result in the following message in the buffer =*Org-Babel Error Output*=:

#+BEGIN_EXAMPLE
  File "<stdin>", line 3
SyntaxError: Non-ASCII character '\xe2' in file <stdin> on line 3, but no encoding declared; see http://www.python.org/peps/pep-0263.html for details
#+END_EXAMPLE

** passing utf-8 strings to python with workaround

A workaround is to use =:preamble= with wthe value =# -*- coding:utf-8 -*-=

#+NAME: ok-in-passing-var
#+BEGIN_SRC python :preamble "# -*- coding: utf-8 -*-" :var data=unicode_str
return data
#+END_SRC

#+RESULTS: ok-in-passing-var
: “this string is not ascii!”

** passing utf-8 tables to python

#+NAME: unicode_table
| key | value                       |
|-----+-----------------------------|
|   1 | “this string is not ascii!” |

#+NAME: error-in-passing-table
#+BEGIN_SRC python :var data=unicode_table
return data
#+END_SRC

Will result in the following message in the buffer =*Org-Babel Error Output*=:

#+BEGIN_EXAMPLE
  File "<stdin>", line 3
SyntaxError: Non-ASCII character '\xe2' in file <stdin> on line 3, but no encoding declared; see http://www.python.org/peps/pep-0263.html for details
#+END_EXAMPLE

** returning utf-8 tables from python

#+NAME: error-in-returnig-table
#+BEGIN_SRC python :preamble "# -*- coding: utf-8 -*-" :var data=unicode_table :colnames yes
return data
#+END_SRC

Will output the ascii text with escape sequence for unicode
characters.  I still do not know the best workaround

#+RESULTS: error-in-returnig-table
| key | value                                             |
|-----+---------------------------------------------------|
|   1 | \xe2\x80\x9cthis string is not ascii!\xe2\x80\x9d |

Please note that the single cell is correctly returned:

#+NAME: returnig-table-cell
#+BEGIN_SRC python :preamble "# -*- coding: utf-8 -*-" :var data=unicode_table
return data[0][1]
#+END_SRC

#+RESULTS: returnig-table-cell
: “this string is not ascii!”

** Undocumented parts

I did not found out:

- how to get the table headers in python

- in general, the magic that org-mode does to pass and retrieve data
  structures to various languages is lacking documentation

Re: org-mode, python and unicode

[William Henney]

[-- Attachment #1: Type: text/plain, Size: 661 bytes --]

Hi Daniel

On Wed, May 18, 2016 at 3:10 AM, Daniele Pizzolli <dan@toel.it> wrote:

>
> I did not found out:
>
> - how to get the table headers in python
>
>
I can answer this small part.  Use the (counter-intuitively named)
":colnames no" header argument, see [[info:org#colnames]].  The default
behavior is ":colnames nil", which supposedly strips off the table headers
on input and reattaches them on output.  But that doesn't work for me with
python - they are stripped off, but not reattached.

Will


-- 

  Dr William Henney, Instituto de Radioastronomía y Astrofísica,
  Universidad Nacional Autónoma de México, Campus Morelia

[-- Attachment #2: Type: text/html, Size: 1184 bytes --]