Unicode and Latex export

Unicode and Latex export

[suvayu ali]

Hi,

Since I am a science student, I end up using lots of unicode
characters for Greek and mathematical symbols. I usually read my notes
in Emacs itself, unicode makes this a much nicer experience. However
sometimes there is a need to export to html or pdf. Exporting to html
works great with this, but I run into problems with missing characters
when I export to latex.

So my question is, is there a convenient way to translate the unicode
characters into their corresponding latex commands for latex export
and keep the unicode characters as is in the org file or for html
export?

-- 
Suvayu

Open source is the future. It sets us free.

Re: Unicode and Latex export

[Christopher Witte]

On Sun 07 Aug 2011 01:25:32 AM CEST, suvayu ali wrote:
> So my question is, is there a convenient way to translate the unicode
> characters into their corresponding latex commands for latex export
> and keep the unicode characters as is in the org file or for html
> export?

You could try using XeLaTeX, which supports unicode.  Instructions for 
setting it up with org-mode are here
http://orgmode.org/worg/org-faq.html#using-xelatex-for-pdf-export

Chris.

-- 
Dr Christopher Witte
Postdoctoral Research Fellow
Molecular Imaging Group
Leibniz-Institut für Molekulare Pharmakologie (FMP)
Campus Berlin-Buch
Robert-Roessle-Str. 10
13125 Berlin, Germany
Phone: 00493094793-279

Re: Unicode and Latex export

[suvayu ali]

Hello Christopher,

On Sun, Aug 7, 2011 at 12:01 PM, Christopher Witte <chris@witte.net.au> wrote:
> You could try using XeLaTeX, which supports unicode.  Instructions for
> setting it up with org-mode are here
> http://orgmode.org/worg/org-faq.html#using-xelatex-for-pdf-export
>

My lisp is not very good, but what is the need for such an extensive
setup? Isn't setting org-latex-to-pdf-process to xelatex enough to
switch packends?

In any case, I tried evaluating the provided code in a scratch buffer
and processing to pdf (C-c C-e p). But it keeps failing. From the error
it seems org is still trying latex instead of xelatex and no pdf is
produced. Do you have any ideas about what could I doing wrong?

Error:

Latexmk: Run number 1 of rule 'latex'
Latexmk: Running 'latex   -interaction=batchmode
"/home/jallad/org/analysis.tex"'
This is pdfTeX, Version 3.1415926-1.40.11 (TeX Live 2010)
 restricted \write18 enabled.
entering extended mode
Latexmk: References changed.
Latexmk: References changed.
Latexmk: References changed.
Collected error summary (may duplicate other messages):
  latex: Command for 'latex' gave return code 256
Latexmk: Use the -f option to force complete processing.


> Chris.

Thanks a lot for your response.

-- 
Suvayu

Open source is the future. It sets us free.

Re: Unicode and Latex export

[Florian Beck]

suvayu ali <fatkasuvayu+linux@gmail.com> writes:

> Hello Christopher,
>
> On Sun, Aug 7, 2011 at 12:01 PM, Christopher Witte <chris@witte.net.au> wrote:
>> You could try using XeLaTeX, which supports unicode.  Instructions for
>> setting it up with org-mode are here
>> http://orgmode.org/worg/org-faq.html#using-xelatex-for-pdf-export
>>
>
> My lisp is not very good, but what is the need for such an extensive
> setup? Isn't setting org-latex-to-pdf-process to xelatex enough to
> switch packends?

It is, more or less. XeLaTeX needs a different header, that is what the
instructions set up. However, I would recommend keeping it simple and
put your packages etc in your own style file.

In `org-export-latex-classes', replace

	 "\\documentclass[11pt]{article}"

with

	 "[NO-DEFAULT-PACKAGES]
	 \\documentclass[11pt]{article}\n\\usepackage{myxe}"

and put everything you need in myxe.sty.

BTW, this only works if you use a font that provides all the characters
you need (and looks nice enough for your taste). Alternatively, you could
use `org-export-latex-final-hook' and write a function that translates
unicode characters to latex code.

-- 
Florian Beck

Re: Unicode and Latex export

[suvayu ali]

Hi Florian,

On Sun, Aug 7, 2011 at 6:33 PM, Florian Beck <abstraktion@t-online.de> wrote:
>> My lisp is not very good, but what is the need for such an extensive
>> setup? Isn't setting org-latex-to-pdf-process to xelatex enough to
>> switch packends?
>
> It is, more or less. XeLaTeX needs a different header, that is what the
> instructions set up. However, I would recommend keeping it simple and
> put your packages etc in your own style file.
>
> In `org-export-latex-classes', replace
>
>         "\\documentclass[11pt]{article}"
>
> with
>
>         "[NO-DEFAULT-PACKAGES]
>         \\documentclass[11pt]{article}\n\\usepackage{myxe}"
>
> and put everything you need in myxe.sty.
>

That is a good solution. I think I will consider this.

> BTW, this only works if you use a font that provides all the characters
> you need (and looks nice enough for your taste).

I don't usually customise my fonts and go with the default latex
fonts. Do you have any suggestions what fonts that would be
appropriate here? I tried using XITS, but that didn't help.

> Alternatively, you could
> use `org-export-latex-final-hook' and write a function that translates
> unicode characters to latex code.
>

I have considered this, but as I mentioned I am not well versed in
lisp. Going this route might be more fun, but not good for work. ;)

Thanks,

-- 
Suvayu

Open source is the future. It sets us free.

Re: Unicode and Latex export

[suvayu ali]

On Sun, Aug 7, 2011 at 6:50 PM, suvayu ali <fatkasuvayu+linux@gmail.com> wrote:
>> BTW, this only works if you use a font that provides all the characters
>> you need (and looks nice enough for your taste).
>
> I don't usually customise my fonts and go with the default latex
> fonts. Do you have any suggestions what fonts that would be
> appropriate here? I tried using XITS, but that didn't help.

I should clarify, when I say didn't work I mean xelatex couldn't find
the fonts although I have them installed.

-- 
Suvayu

Open source is the future. It sets us free.

Re: Unicode and Latex export

[Florian Beck]

suvayu ali <fatkasuvayu+linux@gmail.com> writes:

> On Sun, Aug 7, 2011 at 6:50 PM, suvayu ali <fatkasuvayu+linux@gmail.com> wrote:
>>> BTW, this only works if you use a font that provides all the characters
>>> you need (and looks nice enough for your taste).
>>
>> I don't usually customise my fonts and go with the default latex
>> fonts. Do you have any suggestions what fonts that would be
>> appropriate here? I tried using XITS, but that didn't help.
>
> I should clarify, when I say didn't work I mean xelatex couldn't find
> the fonts although I have them installed.

Not sure. XeTeX uses the fonts you have installed on your system, not
those for TeX. Everything in M-: (font-family-list) should work.

Maybe one of the DejaVu fonts? Generally, though, universal fonts aren't the
best choice.

Here is what I do. Create a custom link style for export with XeLaTeX:

(org-add-link-type "uc" 'ignore 'org-unicode-export)

(defun org-unicode-export (path desc format)
  "Export unicode characters."
  (cond
   ((eql format 'html) (format "%s" desc))
   ((eql format 'latex)
    (format "{\\fontspec[Scale=MatchLowercase]{%s}%s}"
	    path desc))
   (t desc)))

And write [[uc:Code2000][⫳]] to use Code2000 as a font.

I use it mostly for greek words, e.g.
[[uc:Alexander][φύσις κρύπτεσθαι φιλεῖ]]

But if you mostly have single characters it might be too much of a hassle.

-- 
Florian Beck

Re: Unicode and Latex export

[suvayu ali]

On Sun, Aug 7, 2011 at 8:07 PM, Florian Beck <abstraktion@t-online.de> wrote:
> But if you mostly have single characters it might be too much of a hassle.

It does indeed sound like too much hassle for my use case. I only use it
for scientific note taking. For example I would write something like
this:


CP channel: Bs⁰ -> Ds⁻ K⁺ / Ds⁺ K⁻ (interference b/w decay modes of
Bs⁰ or anti-Bs⁰)


I think I'll try to find an alternate solution. Maybe I'll end up
writing that unicode to latex translation function in elisp.

Thanks a lot for the pointers though.

-- 
Suvayu

Open source is the future. It sets us free.

Re: Unicode and Latex export

[Florian Beck]

suvayu ali <fatkasuvayu+linux@gmail.com> writes:

> On Sun, Aug 7, 2011 at 8:07 PM, Florian Beck <abstraktion@t-online.de> wrote:
>> But if you mostly have single characters it might be too much of a hassle.
>
> It does indeed sound like too much hassle for my use case. I only use it
> for scientific note taking. For example I would write something like
> this:
>
>
> CP channel: Bs⁰ -> Ds⁻ K⁺ / Ds⁺ K⁻ (interference b/w decay modes of
> Bs⁰ or anti-Bs⁰)

In this case, XeLaTeX with

\setmainfont{DejaVu Serif}

in the preamble seems to give reasonable results.

>
> I think I'll try to find an alternate solution. Maybe I'll end up
> writing that unicode to latex translation function in elisp.

I did something like that for muse-mode. IIRC the main problem was that
the translation has to be context sensitive (ie math mode vs. text
mode). Let me know if you are interested, maybe I can dig something out.

>
> Thanks a lot for the pointers though.

-- 
Florian Beck

Re: Unicode and Latex export

[suvayu ali]

Hi Florian,

On Sun, Aug 7, 2011 at 8:41 PM, Florian Beck <abstraktion@t-online.de> wrote:
>>
>> CP channel: Bs⁰ -> Ds⁻ K⁺ / Ds⁺ K⁻ (interference b/w decay modes of
>> Bs⁰ or anti-Bs⁰)
>
> In this case, XeLaTeX with
>
> \setmainfont{DejaVu Serif}
>
> in the preamble seems to give reasonable results.
>

Indeed! The output is exactly what I was hoping to achieve.

>>
>> I think I'll try to find an alternate solution. Maybe I'll end up
>> writing that unicode to latex translation function in elisp.
>
> I did something like that for muse-mode. IIRC the main problem was that
> the translation has to be context sensitive (ie math mode vs. text
> mode). Let me know if you are interested, maybe I can dig something out.

Since the above resolves my issue very well, that will not be necessary.
Probably I'll take it up when I have more free time. Thanks a lot to
both you and Chris for your help, appreciate it a lot. :)

>
> --
> Florian Beck

Cheers,

-- 
Suvayu

Open source is the future. It sets us free.

Re: Unicode and Latex export

[Aditya Mandayam]

> Do you have any suggestions what fonts that would be
> appropriate here?

http://www.thessalonica.org.ru/en/fonts-download.html

http://scholarsfonts.net/

- a

Re: Unicode and Latex export

[Aditya Mandayam]

apostrophic laboratories has a large number of handmade fonts as well,
although most of them probably do not suit scholarly needs:

http://apostrophiclab.pedroreina.net/

On Wed, Aug 31, 2011 at 9:30 AM, Aditya Mandayam <adityams@gmail.com> wrote:
>> Do you have any suggestions what fonts that would be
>> appropriate here?
>
> http://www.thessalonica.org.ru/en/fonts-download.html
>
> http://scholarsfonts.net/
>
> - a
>

Re: Unicode and Latex export

[Christopher Witte]

On 08/07/2011 02:15 PM, suvayu ali wrote:
> Hello Christopher,
>
> On Sun, Aug 7, 2011 at 12:01 PM, Christopher Witte<chris@witte.net.au>  wrote:
>> You could try using XeLaTeX, which supports unicode.  Instructions for
>> setting it up with org-mode are here
>> http://orgmode.org/worg/org-faq.html#using-xelatex-for-pdf-export
>>
> My lisp is not very good, but what is the need for such an extensive
> setup? Isn't setting org-latex-to-pdf-process to xelatex enough to
> switch packends?
>
> In any case, I tried evaluating the provided code in a scratch buffer
> and processing to pdf (C-c C-e p). But it keeps failing. From the error
> it seems org is still trying latex instead of xelatex and no pdf is
> produced. Do you have any ideas about what could I doing wrong?
>
> Error:
>
> Latexmk: Run number 1 of rule 'latex'
> Latexmk: Running 'latex   -interaction=batchmode
> "/home/jallad/org/analysis.tex"'
> This is pdfTeX, Version 3.1415926-1.40.11 (TeX Live 2010)
>   restricted \write18 enabled.
> entering extended mode
> Latexmk: References changed.
> Latexmk: References changed.
> Latexmk: References changed.
> Collected error summary (may duplicate other messages):
>    latex: Command for 'latex' gave return code 256
> Latexmk: Use the -f option to force complete processing.
>
>
>> Chris.
> Thanks a lot for your response.
>
Did you remember to set the variable

#+LATEX_CMD: xelatex

at the top of the org file?


Chris.

Re: Unicode and Latex export

[suvayu ali]

Hello Chris,

On Sun, Aug 7, 2011 at 6:36 PM, Christopher Witte <chris@witte.net.au> wrote:
> Did you remember to set the variable
>
> #+LATEX_CMD: xelatex
>
> at the top of the org file?
>

I forgot to do that. However correcting my error doesn't help either.
I get a message like this in the output buffer:

Latexmk: Run number 1 of rule 'pdflatex'
Latexmk: Running 'xelatex   -interaction=batchmode
"/home/jallad/tmp/analysis.tex"'
This is XeTeX, Version 3.1415926-2.2-0.9997.4 (TeX Live 2010)
 restricted \write18 enabled.
entering extended mode
Latexmk: References changed.
Latexmk: References changed.
Collected error summary (may duplicate other messages):
  pdflatex: Command for 'pdflatex' gave return code 256
Latexmk: Use the -f option to force complete processing.


However this time a PDF is produced but with the unicode fonts missing
(as was happening with pdflatex).

>
> Chris.



-- 
Suvayu

Open source is the future. It sets us free.

Re: Unicode and Latex export

[Sebastien Vauban]

Hi Suvayu,

suvayu ali wrote:
> Since I am a science student, I end up using lots of unicode
> characters for Greek and mathematical symbols. I usually read my notes
> in Emacs itself, unicode makes this a much nicer experience. However
> sometimes there is a need to export to html or pdf. Exporting to html
> works great with this, but I run into problems with missing characters
> when I export to latex.
>
> So my question is, is there a convenient way to translate the unicode
> characters into their corresponding latex commands for latex export
> and keep the unicode characters as is in the org file or for html
> export?

For the sake of completeness, please know you can use PDFLaTeX and UTF-8 -- I
do it for all my documents -- by having "\usepackage[utf8x]{inputenc}"
inserted at the right place(TM).

Normally, if your Org files are UTF-8, this should even be automatic, thanks
to the line:

    \\usepackage[AUTO]{inputenc}

in `org-export-latex-classes'.

Best regards,
  Seb

-- 
Sebastien Vauban

Re: Unicode and Latex export

[suvayu ali]

Hi Seb,

On Mon, Aug 8, 2011 at 8:13 AM, Sebastien Vauban
<wxhgmqzgwmuf@spammotel.com> wrote:
> For the sake of completeness, please know you can use PDFLaTeX and UTF-8 -- I
> do it for all my documents -- by having "\usepackage[utf8x]{inputenc}"
> inserted at the right place(TM).
>
> Normally, if your Org files are UTF-8, this should even be automatic, thanks
> to the line:
>
>    \\usepackage[AUTO]{inputenc}
>
> in `org-export-latex-classes'.
>

I believe you mean `org-export-latex-default-packages-alist'?
Everything seems fine with my settings "(("AUTO" "inputenc" t)..)",
but the exported latex code has this:

\usepackage[utf8]{inputenc}

I edited the tex file to utf8x and everything works wonderfully. Any
ideas how I could get this working?

> Best regards,
>  Seb

-- 
Suvayu

Open source is the future. It sets us free.

Re: Unicode and Latex export

[Nick Dokos]

suvayu ali <fatkasuvayu+linux@gmail.com> wrote:

> Hi Seb,
> 
> On Mon, Aug 8, 2011 at 8:13 AM, Sebastien Vauban
> <wxhgmqzgwmuf@spammotel.com> wrote:
> > For the sake of completeness, please know you can use PDFLaTeX and UTF-8 -- I
> > do it for all my documents -- by having "\usepackage[utf8x]{inputenc}"
> > inserted at the right place(TM).
> >
> > Normally, if your Org files are UTF-8, this should even be automatic, thanks
> > to the line:
> >
> >    \\usepackage[AUTO]{inputenc}
> >
> > in `org-export-latex-classes'.
> >
> 
> I believe you mean `org-export-latex-default-packages-alist'?
> Everything seems fine with my settings "(("AUTO" "inputenc" t)..)",
> but the exported latex code has this:
> 
> \usepackage[utf8]{inputenc}
> 
> I edited the tex file to utf8x and everything works wonderfully. Any
> ideas how I could get this working?
> 

C-h v org-export-latex-inputenc-alist RET says:

,----
| org-export-latex-inputenc-alist is a variable defined in `org-latex.el'.
| Its value is nil
| 
| Documentation:
| Alist of inputenc coding system names, and what should really be used.
| For example, adding an entry
| 
|       ("utf8" . "utf8x")
| 
| will cause \usepackage[utf8x]{inputenc} to be used for buffers that
| are written as utf8 files.
| 
| You can customize this variable.
`----

Nick

Re: Unicode and Latex export

[suvayu ali]

On Mon, Aug 8, 2011 at 9:09 AM, Nick Dokos <nicholas.dokos@hp.com> wrote:
> C-h v org-export-latex-inputenc-alist RET says:
>
>

Thank you Nick, works great now. :)

-- 
Suvayu

Open source is the future. It sets us free.

Re: Unicode and Latex export

[Stefan Nobis]

[-- Attachment #1: Type: text/plain, Size: 527 bytes --]

"Sebastien Vauban" <wxhgmqzgwmuf@spammotel.com> writes:

> having "\usepackage[utf8x]{inputenc}" inserted

Please beware that utf8x is part of the obsolete and unsupported ucs
package. As ucs deeply affects the LaTeX kernel, more and more modern
packages are incompatible with utf8x and ucs (csquotes,
hyperref,...). For proper Unicode support its preferable to use LuaTeX
or XeTeX rather than using ucs (which is one of the first attemtps at
better Unicode support for TeX).

-- 
Until the next mail...,
Stefan.

[-- Attachment #2: Type: application/pgp-signature, Size: 454 bytes --]

Re: Unicode and Latex export

[suvayu ali]

Hi Stefan,

On Mon, Aug 8, 2011 at 9:53 AM, Stefan Nobis <stefan-ml@snobis.de> wrote:
> For proper Unicode support its preferable to use LuaTeX
> or XeTeX rather than using ucs

Thanks for the warning. :)

-- 
Suvayu

Open source is the future. It sets us free.