LaTeX export: when is it more useful to use LuaTeX instead of pdfTeX?

LaTeX export: when is it more useful to use LuaTeX instead of pdfTeX?

[Juan Manuel Macías]

TL;DR: A list of use cases where using LuaTeX is more advantageous than
using pdfTeX

------------

Many times Org users who frequently need to export their documents to
LaTeX, but who do not have much LaTeX experience (or their knowledge of
the TeX ecosystem is somewhat out of date), find themselves confused by
the different versions of the TeX engine: pdfTeX, XeTeX, LuaTeX… In Org
pdfTeX is the default engine, which in 2022 has a few limitations and is
really old, but still perfectly meets the needs of a significant group
of users. However, there may be a number of cases where it is more
advantageous to compile with LuaTeX, so here I will leave a short list
of those cases where LuaTeX may be a happy choice over pdfTeX.

But first it is worth clarifying some things about LuaTeX along with
some misunderstandings:

• LuaTeX is the evolution of pdfTeX, therefore LuaTeX can be considered
  as the current de facto TeX engine. It is intended to replace pdfTeX,
  and in fact many of us already consider pdfTeX obsolete and
  deprecated.

• To use LuaTeX it is not necessary to learn anything new or to know how
  to program in Lua. LuaTeX includes a Lua interpreter and the ability
  to bypass TeX primitives through Lua scripting (hence called LuaTeX).
  But all of that is more on the side of developers and packagers. For
  example, I am currently writing two LaTeX packages (one in
  collaboration with a colleague) where 80% of the code is Lua and 20%
  is (La)TeX. Of course, any user who knows Lua can take advantage of
  the \directlua primitive or the luacode environment in their
  documents.

• A standard LaTeX document is always a LaTeX document, regardless of
  the flavor of TeX used to compile that document. There will be some
  minor differences. For example, in LuaLaTeX it is unnecessary to add
  fontenc and inputenc commands in the preamble.

And now we go with the non-exhaustive list of cases where compiling with
LuaTeX can be more advantageous for the user:

1. When you need to work in a *real* Unicode environment and not in
   pdfTeX’s 'fake Unicode'. And, especially, when it is required to work
   with languages that use a non-Latin writing system: Greek, Arabic,
   Hebrew, all the languages that use Cyrillic, oriental languages, etc.
   An extreme example you can see in this small code that I wrote for
   LuaTeX in order to be able to use the syllabograms of the ancient
   Mycenaean script:

   <https://gitlab.com/maciaschain/linealb-in-luatex>

2. When using truetype or opentype fonts is required. The pdfTeX user is
   limited to using only the included type1 fonts, the number of which
   is very limited. Besides, type1 is a deprecated and pre-unicode
   format. In fact, it almost always ends up leaving the default
   Computer Modern font. In LuaTeX we can use not only all the fonts
   installed on our system but also any font (just indicate the path),
   which is an important advantage over XeTeX. A basic command could be
   something like (loading the fontspec package):

   ┌────
   │ \setmainfont{Palatino Linotype}
   └────

   And with the latest versions of Babel we can associate fonts for
   different writing systems, without the need to change languages
   explicitly with a \selectlanguage.

   We can define all the font families we want or redefine the default
    families. For example, with this command I define the default
    monospaced font and request that it be scaled according to the
    height of the lowercase of the main font:

   ┌────
   │ \setmonofont{Iosevka Fixed Curly}[Scale=MatchLowercase]
   └────

3. When you need to take advantage, to a greater or lesser extent, of
   the opentype features of a font. For example, here we define the main
   font to use old style numbers:

   ┌────
   │ \setmainfont{Palatino Linotype}[Numbers=Lowercase]
   └────

   We can also load the otf tags directly:

   ┌────
   │ \setmainfont{Palatino Linotype}[RawFeature=+onum]
   └────

   The fontspec package for managing fonts in LuaTeX and XeTeX is very
    versatile and powerful. We can also associate a different font as
    italic for an already defined font family, use different optical
    resolutions of a font, etc. If what you are looking for is precise
    and absolute fine-tuning of the fonts used, of course LuaTeX is the
    ideal choice.

4. In general, when professional-level typographic fine-tuning is needed
   (and far superior to that offered by dtp programs like InDesign or
   QuarkXpress). For example, we can define on the fly new position
   opentype properties for a specific font, without having to edit the
   font. It is a non-destructive method that uses the
   fonts.handlers.otf.addfeature lua function. For example, we can
   define a new kerning value for the letters A and V. We’ll call it
   ’mykern’

┌────
│ \directlua{
│ fonts.handlers.otf.addfeature
│ {
│    name ="mykern",
│    type ="kern",
│    data =
│       {
│ 	 ["A"] = { ["V"] =  270 },
│ }}
│ }
│ 
│ \setmainfont{Linux Libertine O}[RawFeature=+mykern]
└────

Here you can see a more bizarre and vandalistic example, where I have
replaced the accent of the accented letters in Spanish with the image of
a hammer :-)

https://i.imgur.com/iixxJmx.png

Here I have placed the image of a dancer on the tilde of the spanish
letter ñ:

https://i.imgur.com/oIZzpbJ.png

(The process is simply to decompose the complex characters from the
precombined form to their canonical decomposition: NFC > NFD, and then
use a png image as a diacritic :-):

\newunicodechar{^^^^0301}{\raisebox{1.2ex}{\includegraphics[width=.28em]{martillo.png}}}

1. Lastly, a lot of new (increasingly) LaTeX packages are written on top
   of the advanced features of LuaTeX and require LuaTeX. A very useful
   package, for example, is impnattypo, for post-production fine-tuning
   (<https://www.ctan.org/pkg/impnattypo>). Among the many features of
   impnattypo we have the ability to prevent lines from ending in
   single-letter words. It also highlights in draft mode, homeoarchy
   cases, which are typographically incorrect. An example in one of my
   recent works:

<https://i.imgur.com/Kf8Oot0.png>

Best regards,

Juan Manuel 

Re: LaTeX export: when is it more useful to use LuaTeX instead of pdfTeX?

[Uwe Brauer]

[-- Attachment #1: Type: text/plain, Size: 1471 bytes --]

>>> "JMM" == Juan Manuel Macías <maciaschain@posteo.net> writes:

Hi Juan
> TL;DR: A list of use cases where using LuaTeX is more advantageous than
> using pdfTeX

> ------------

> Many times Org users who frequently need to export their documents to
> LaTeX, but who do not have much LaTeX experience (or their knowledge of
> the TeX ecosystem is somewhat out of date), find themselves confused by
> the different versions of the TeX engine: pdfTeX, XeTeX, LuaTeX… In Org
> pdfTeX is the default engine, which in 2022 has a few limitations and is
> really old, but still perfectly meets the needs of a significant group
> of users. However, there may be a number of cases where it is more
> advantageous to compile with LuaTeX, so here I will leave a short list
> of those cases where LuaTeX may be a happy choice over pdfTeX.

> But first it is worth clarifying some things about LuaTeX along with
> some misunderstandings:

> • LuaTeX is the evolution of pdfTeX, therefore LuaTeX can be considered
>   as the current de facto TeX engine. It is intended to replace pdfTeX,
>   and in fact many of us already consider pdfTeX obsolete and
>   deprecated.

Thanks for that list.

Well I have felt in the past the same about pdftex, but I have partially
switched to xetex precisely on the reasons you list.

I have not have the time, to really try out Luatex. Did you have the
time to compare it with XeTeX?

Regards

Uwe Brauer 

[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 5673 bytes --]

Re: LaTeX export: when is it more useful to use LuaTeX instead of pdfTeX?

[Greg Minshall]

Juan Manuel, et al.,

> TL;DR: A list of use cases where using LuaTeX is more advantageous than
> using pdfTeX

i'm guessing this would be a nice addtion to worg (if it's not already
there).

cheers, Greg

Re: LaTeX export: when is it more useful to use LuaTeX instead of pdfTeX?

[Thomas S. Dye]

Juan Manuel Macías <maciaschain@posteo.net> writes:

> TL;DR: A list of use cases where using LuaTeX is more 
> advantageous than
> using pdfTeX
>
Interesting.  Thanks!

All the best,
Tom

-- 
Thomas S. Dye
https://tsdye.online/tsdye

Re: LaTeX export: when is it more useful to use LuaTeX instead of pdfTeX?

[Juan Manuel Macías]

Hi Uwe,

Uwe Brauer writes:

> Thanks for that list.
>
> Well I have felt in the past the same about pdftex, but I have partially
> switched to xetex precisely on the reasons you list.
>
> I have not have the time, to really try out Luatex. Did you have the
> time to compare it with XeTeX?

First of all, if you feel comfortable with xetex, my advice is to use
XeTeX. I've been using XeTeX for a long time, before fully migrating to
LuaTeX, and I think it's great. In fact, XeTeX was a great revolution in
the TeX ecosystem, for being able to use system fonts in a simple way.
LuaTeX came later.

The first TeX engine I ever used was Omega, an experimental engine
(later forked as "Aleph") that was the first TeX engine with full Unicode
support. But it still produced dvi files, not pdf. LuaTeX has evolved
from pdfTeX and has incorporated the more advanced and sophisticated
features of Omega/Aleph.

Any LaTeX document that you compile in XeTeX can be easily compiled in
LuaTeX. There may be some small incompatibilities, for example if you
include some native xetex primitives. You can do the test. The reverse
path is also possible, as long as the document does not contain Lua
code or luatex primitives.

In general, for an average user, moving from xetex to luatex does not
make any visible changes. The advanced features of LuaTeX, as I said,
are more on the developer side. Of course, if the user programs in Lua,
he/she can switch LaTeX code with lua code, use callbacks and pre/post
process lua filters, which offer enormous control over the document. But
it is not necessary. But keep in mind that you can get a lot of control
over the low-level gears of TeX by the Lua scripting, which is much
simpler and lighter than pure TeX.

The essential differences between luatex and xetex are in the last two
points on my list. The last point is important to keep in mind, as
more and more packages (some tremendously useful) are coming out that
only work in LuaTeX.

On the other hand, LuaTeX is an evolutionary step from pdfTeX (xetex
would be a separate branch), so luatex is meant to be the default or
"official" engine and replace pdfTeX, just as pdfTeX replaced TeX in its
day.

Best regards,

Juan Manuel 

Re: LaTeX export: when is it more useful to use LuaTeX instead of pdfTeX?

[Bruce D'Arcus]

Today, I think the only advantage pdftex has is speed; it's a lot
faster to compile documents than luatex.

And maybe some advanced microtypography features, though I haven't tracked that.

Bruce

Re: LaTeX export: when is it more useful to use LuaTeX instead of pdfTeX?

[Thomas S. Dye]

Juan Manuel Macías <maciaschain@posteo.net> writes:

> TL;DR: A list of use cases where using LuaTeX is more 
> advantageous than
> using pdfTeX

I forgot to ask earlier.  Is Lua support in Babel potentially 
useful?  The current implementation doesn't work too well.

All the best,
Tom
-- 
Thomas S. Dye
https://tsdye.online/tsdye

Re: LaTeX export: when is it more useful to use LuaTeX instead of pdfTeX?

[Juan Manuel Macías]

Bruce D'Arcus writes:

> Today, I think the only advantage pdftex has is speed; it's a lot
> faster to compile documents than luatex.

That's true, but it seems to be a LaTeX and fontspec issue. I think
ConTeXt, which uses LuaTeX, is faster, but I don't have the hard data.
In general TeX is slow and single-threaded and cannot take advantage of
the latest hardware. The ideal would be to rewrite TeX from scratch.
There is this project (among others), very interesting:

https://sile-typesetter.org/what-is-sile/

(It is written entirely in Lua and implements the TeX algorithms. The
problem is that it lacks the LaTeX package ecosystem, is a niche.)

> And maybe some advanced microtypography features, though I haven't tracked that.

This is one of the most interesting parts of pdfTeX, based on Hermann
Zapf's theories on the Gutenberg Bible. I believe that Zapf himself
advised the author of pdfTeX for his famous thesis, from which pdfTeX
arose.

LuaTeX has inherited the microtype properties of pdfTeX, so they can be
perfectly used and applied in luaLaTeX with the microtype package
(generally speaking, LuaTeX is still a kind of evolved pdfTeX...).

BTW, This article explains very well these microtypographic theories and
their origin. InDesign's poor implementation compared to pdfTeX is also
commented on:

http://www.typografi.org/justering/gut_hz/gutenberg_hz_english.html

In general, font expansion greatly improves the visual quality of the
paragraph and the use of space. Character protrusion works well for
producing a more consistent justification as well, but relies on ad hoc
values being used for each font. There are already some defaults for
various fonts in the microtype package. Indesign, on the other hand,
just uses generic values, which doesn't make much sense.

Best regards,

Juan Manuel 

Re: LaTeX export: when is it more useful to use LuaTeX instead of pdfTeX?

[Matt Huszagh]

Juan Manuel Macías <maciaschain@posteo.net> writes:

> TL;DR: A list of use cases where using LuaTeX is more advantageous than
> using pdfTeX
>
> ------------
>
> Many times Org users who frequently need to export their documents to
> LaTeX, but who do not have much LaTeX experience (or their knowledge of
> the TeX ecosystem is somewhat out of date), find themselves confused by
> the different versions of the TeX engine: pdfTeX, XeTeX, LuaTeX… In Org
> pdfTeX is the default engine, which in 2022 has a few limitations and is
> really old, but still perfectly meets the needs of a significant group
> of users. However, there may be a number of cases where it is more
> advantageous to compile with LuaTeX, so here I will leave a short list
> of those cases where LuaTeX may be a happy choice over pdfTeX.
>
> But first it is worth clarifying some things about LuaTeX along with
> some misunderstandings:
>
> • LuaTeX is the evolution of pdfTeX, therefore LuaTeX can be considered
>   as the current de facto TeX engine. It is intended to replace pdfTeX,
>   and in fact many of us already consider pdfTeX obsolete and
>   deprecated.
>
> • To use LuaTeX it is not necessary to learn anything new or to know how
>   to program in Lua. LuaTeX includes a Lua interpreter and the ability
>   to bypass TeX primitives through Lua scripting (hence called LuaTeX).
>   But all of that is more on the side of developers and packagers. For
>   example, I am currently writing two LaTeX packages (one in
>   collaboration with a colleague) where 80% of the code is Lua and 20%
>   is (La)TeX. Of course, any user who knows Lua can take advantage of
>   the \directlua primitive or the luacode environment in their
>   documents.
>
> • A standard LaTeX document is always a LaTeX document, regardless of
>   the flavor of TeX used to compile that document. There will be some
>   minor differences. For example, in LuaLaTeX it is unnecessary to add
>   fontenc and inputenc commands in the preamble.
>
> And now we go with the non-exhaustive list of cases where compiling with
> LuaTeX can be more advantageous for the user:
>
> 1. When you need to work in a *real* Unicode environment and not in
>    pdfTeX’s 'fake Unicode'. And, especially, when it is required to work
>    with languages that use a non-Latin writing system: Greek, Arabic,
>    Hebrew, all the languages that use Cyrillic, oriental languages, etc.
>    An extreme example you can see in this small code that I wrote for
>    LuaTeX in order to be able to use the syllabograms of the ancient
>    Mycenaean script:
>
>    <https://gitlab.com/maciaschain/linealb-in-luatex>
>
> 2. When using truetype or opentype fonts is required. The pdfTeX user is
>    limited to using only the included type1 fonts, the number of which
>    is very limited. Besides, type1 is a deprecated and pre-unicode
>    format. In fact, it almost always ends up leaving the default
>    Computer Modern font. In LuaTeX we can use not only all the fonts
>    installed on our system but also any font (just indicate the path),
>    which is an important advantage over XeTeX. A basic command could be
>    something like (loading the fontspec package):
>
>    ┌────
>    │ \setmainfont{Palatino Linotype}
>    └────
>
>    And with the latest versions of Babel we can associate fonts for
>    different writing systems, without the need to change languages
>    explicitly with a \selectlanguage.
>
>    We can define all the font families we want or redefine the default
>     families. For example, with this command I define the default
>     monospaced font and request that it be scaled according to the
>     height of the lowercase of the main font:
>
>    ┌────
>    │ \setmonofont{Iosevka Fixed Curly}[Scale=MatchLowercase]
>    └────
>
> 3. When you need to take advantage, to a greater or lesser extent, of
>    the opentype features of a font. For example, here we define the main
>    font to use old style numbers:
>
>    ┌────
>    │ \setmainfont{Palatino Linotype}[Numbers=Lowercase]
>    └────
>
>    We can also load the otf tags directly:
>
>    ┌────
>    │ \setmainfont{Palatino Linotype}[RawFeature=+onum]
>    └────
>
>    The fontspec package for managing fonts in LuaTeX and XeTeX is very
>     versatile and powerful. We can also associate a different font as
>     italic for an already defined font family, use different optical
>     resolutions of a font, etc. If what you are looking for is precise
>     and absolute fine-tuning of the fonts used, of course LuaTeX is the
>     ideal choice.
>
> 4. In general, when professional-level typographic fine-tuning is needed
>    (and far superior to that offered by dtp programs like InDesign or
>    QuarkXpress). For example, we can define on the fly new position
>    opentype properties for a specific font, without having to edit the
>    font. It is a non-destructive method that uses the
>    fonts.handlers.otf.addfeature lua function. For example, we can
>    define a new kerning value for the letters A and V. We’ll call it
>    ’mykern’
>
> ┌────
> │ \directlua{
> │ fonts.handlers.otf.addfeature
> │ {
> │    name ="mykern",
> │    type ="kern",
> │    data =
> │       {
> │ 	 ["A"] = { ["V"] =  270 },
> │ }}
> │ }
> │
> │ \setmainfont{Linux Libertine O}[RawFeature=+mykern]
> └────
>
> Here you can see a more bizarre and vandalistic example, where I have
> replaced the accent of the accented letters in Spanish with the image of
> a hammer :-)
>
> https://i.imgur.com/iixxJmx.png
>
> Here I have placed the image of a dancer on the tilde of the spanish
> letter ñ:
>
> https://i.imgur.com/oIZzpbJ.png
>
> (The process is simply to decompose the complex characters from the
> precombined form to their canonical decomposition: NFC > NFD, and then
> use a png image as a diacritic :-):
>
> \newunicodechar{^^^^0301}{\raisebox{1.2ex}{\includegraphics[width=.28em]{martillo.png}}}
>
> 1. Lastly, a lot of new (increasingly) LaTeX packages are written on top
>    of the advanced features of LuaTeX and require LuaTeX. A very useful
>    package, for example, is impnattypo, for post-production fine-tuning
>    (<https://www.ctan.org/pkg/impnattypo>). Among the many features of
>    impnattypo we have the ability to prevent lines from ending in
>    single-letter words. It also highlights in draft mode, homeoarchy
>    cases, which are typographically incorrect. An example in one of my
>    recent works:
>
> <https://i.imgur.com/Kf8Oot0.png>
>
> Best regards,
>
> Juan Manuel

I typically use luatex instead of pdftex and the sole reason is
performance for pgfplots. The performance gain is night and day when
generating plots with many points. I forget exactly why this is.

When I'm generating very simple documents I stick with pdftex, which is
faster in those cases.

As for lua scripting: I made some brief forays into this but found it
not to be especially useful for me: the reason for that may just have
been lack of persistent effort, though. When I want more modern
programming features I typically use pylatex.

Matt

Re: LaTeX export: when is it more useful to use LuaTeX instead of pdfTeX?

[Max Nikulin]

On 09/07/2022 01:49, Thomas S. Dye wrote:
> Juan Manuel Macías writes:
> 
>> TL;DR: A list of use cases where using LuaTeX is more advantageous than
>> using pdfTeX
> 
> I forgot to ask earlier.  Is Lua support in Babel potentially useful?  
> The current implementation doesn't work too well.

In the context of LaTeX your question is rather ambiguous.

- I have never tried lua interpreter as the handler of source blocks in 
Org (org-babel).
- If you mean LuaLaTeX as the engine for "#+begin_src: latex" blocks 
then some incompatibilities may arise due to font selection. PdfLaTeX 
and LuaLaTeX needs different code in preamble to specify encoding and 
fonts. With LuaTeX you get more convenient OTF and TTF font selection, 
but you you have to pay for the feature. It is necessary to explicitly 
specify all families: normal, typewriter, italics, etc if you need Unicode.
- There is babel LaTeX package. A decade ago polyglossia was a 
replacement of babel for XeTeX and LuaTex, but currently babel is 
recommended for these LaTeX engines as well. Sorry, if this statement is 
not precise enough.

Re: LaTeX export: when is it more useful to use LuaTeX instead of pdfTeX?

[Thomas S. Dye]

Max Nikulin <manikulin@gmail.com> writes:

> On 09/07/2022 01:49, Thomas S. Dye wrote:
>> Juan Manuel Macías writes:
>> 
>>> TL;DR: A list of use cases where using LuaTeX is more 
>>> advantageous than
>>> using pdfTeX
>> I forgot to ask earlier.  Is Lua support in Babel potentially 
>> useful?  
>> The current implementation doesn't work too well.
>
> In the context of LaTeX your question is rather ambiguous.
>
> - I have never tried lua interpreter as the handler of source 
> blocks in Org
>  (org-babel).

Yes, what I called Babel you call org-babel.  I don't know if the 
Lua handler of source blocks in Org might be useful for someone 
interested to write Lua extensions to LaTeX.

All the best,
Tom

-- 
Thomas S. Dye
https://tsdye.online/tsdye

Re: LaTeX export: when is it more useful to use LuaTeX instead of pdfTeX?

[Tim Cross]

Max Nikulin <manikulin@gmail.com> writes:

> On 09/07/2022 01:49, Thomas S. Dye wrote:
>> Juan Manuel Macías writes:
>> 
>>> TL;DR: A list of use cases where using LuaTeX is more advantageous than
>>> using pdfTeX
>> I forgot to ask earlier.  Is Lua support in Babel potentially useful?  The current
>> implementation doesn't work too well.
>
> In the context of LaTeX your question is rather ambiguous.
>
> - I have never tried lua interpreter as the handler of source blocks in Org (org-babel).
> - If you mean LuaLaTeX as the engine for "#+begin_src: latex" blocks then some
>   incompatibilities may arise due to font selection. PdfLaTeX and LuaLaTeX needs different
>   code in preamble to specify encoding and fonts. With LuaTeX you get more convenient OTF
>   and TTF font selection, but you you have to pay for the feature. It is necessary to
>   explicitly specify all families: normal, typewriter, italics, etc if you need Unicode.
> - There is babel LaTeX package. A decade ago polyglossia was a replacement of babel for
>   XeTeX and LuaTex, but currently babel is recommended for these LaTeX engines as
>   well. Sorry, if this statement is not precise enough.

Actually, that is a good point.

Juan, I think it would be great to add your post to worg. I'm happy to
do this, but I think it wold also be good if we could include a basic
'setup' i.e. what changes people might need to (or should do to maximise
benefit) in order to try out luatex. For example, what settings to put
in org-latex-pdf-process (I'm guessing something like "lualatex
-interaction nonstopmode -output-directory %o %f") and what (if any)
packages to add/remove from the org-latex-packages-alist etc (I'm
guessing that perhaps some font related packages may need tweaking?).

Ideally, what would be good is a very simple recipe for what someone
should do in order to try out luatex and get the most out of it (or at
least see potential). 

Re: LaTeX export: when is it more useful to use LuaTeX instead of pdfTeX?

[Ihor Radchenko]

Tim Cross <theophilusx@gmail.com> writes:

> Juan, I think it would be great to add your post to worg. I'm happy to
> do this, but I think it wold also be good if we could include a basic
> 'setup' i.e. what changes people might need to (or should do to maximise
> benefit) in order to try out luatex. For example, what settings to put
> in org-latex-pdf-process (I'm guessing something like "lualatex
> -interaction nonstopmode -output-directory %o %f") and what (if any)
> packages to add/remove from the org-latex-packages-alist etc (I'm
> guessing that perhaps some font related packages may need tweaking?).
>
> Ideally, what would be good is a very simple recipe for what someone
> should do in order to try out luatex and get the most out of it (or at
> least see potential). 

If only a minor tweaking is required to make luatex work with generic
Org export, I'd add direct support of luatex to ox-latex.el depending on
the value of org-latex-compiler.

Or we may go even further and make org-latex-compiler default to luatex.
This will benefit all the non-latin language users.

Best,
Ihor

Re: LaTeX export: when is it more useful to use LuaTeX instead of pdfTeX?

[Tim Cross]

Ihor Radchenko <yantar92@gmail.com> writes:

> Tim Cross <theophilusx@gmail.com> writes:
>
>> Juan, I think it would be great to add your post to worg. I'm happy to
>> do this, but I think it wold also be good if we could include a basic
>> 'setup' i.e. what changes people might need to (or should do to maximise
>> benefit) in order to try out luatex. For example, what settings to put
>> in org-latex-pdf-process (I'm guessing something like "lualatex
>> -interaction nonstopmode -output-directory %o %f") and what (if any)
>> packages to add/remove from the org-latex-packages-alist etc (I'm
>> guessing that perhaps some font related packages may need tweaking?).
>>
>> Ideally, what would be good is a very simple recipe for what someone
>> should do in order to try out luatex and get the most out of it (or at
>> least see potential). 
>
> If only a minor tweaking is required to make luatex work with generic
> Org export, I'd add direct support of luatex to ox-latex.el depending on
> the value of org-latex-compiler.
>
> Or we may go even further and make org-latex-compiler default to luatex.
> This will benefit all the non-latin language users.
>

Perhaps easy choice/switching would be useful. Not sure about changing
the default - from what I can see, it would make processing slower for
all those using latin languages. (however, I'm not good on judging slow
- from my perspective, little is slow these days. I still recall
waiting overnight just to compile my linux kernel, which was much much
smaller back then. My thesis, written in Latex, use to take
enough time to generate a postscript file I could go and make a coffee!
Everything has gotten faster as I've gotten slower!)

Re: LaTeX export: when is it more useful to use LuaTeX instead of pdfTeX?

[Dominik Schrempf]

[-- Attachment #1: Type: text/plain, Size: 311 bytes --]

FWIW: I have switched to lualatex years ago. In my opinion the additional
compilation time is worth much more than my time debugging problems with
weird symbols. Even when I only use Latin in the main text, most of the
time, there are some special symbols that would need special attention in
the bibliography.

[-- Attachment #2: Type: text/html, Size: 405 bytes --]

Re: LaTeX export: when is it more useful to use LuaTeX instead of pdfTeX?

[Max Nikulin]

On 09/07/2022 10:50, Ihor Radchenko wrote:
> 
> Or we may go even further and make org-latex-compiler default to luatex.
> This will benefit all the non-latin language users.

1. It is necessary to detect if LuaLaTeX is installed to fallback to 
PdfLaTeX otherwise. On Ubuntu presence of lualatex binary does not mean 
that all necessary files are installed. Perhaps it is possible to ask 
kpsewhich whether lualatex.fmt is available (or some other file specific 
to lualatex).

2. It is necessary to found suitable fonts installed in the user system. 
It becomes more complicated that different fonts are required for 
different scripts (Cyrillic, Greek, Thai combined in the single 
document), it is not CSS where list of fonts may be provided and a 
rendering engine pics first one having a necessary glyph. See the 
earlier discussion:
https://list.orgmode.org/orgmode/m18s299yxm.fsf@nobis-it.eu/T/#u

Re: LaTeX export: when is it more useful to use LuaTeX instead of pdfTeX?

[Juan Manuel Macías]

Hi, Tim, thank you for your comments,

Tim Cross writes:

> Juan, I think it would be great to add your post to worg. I'm happy to
> do this, but I think it wold also be good if we could include a basic
> 'setup' i.e. what changes people might need to (or should do to maximise
> benefit) in order to try out luatex. For example, what settings to put
> in org-latex-pdf-process (I'm guessing something like "lualatex
> -interaction nonstopmode -output-directory %o %f") and what (if any)
> packages to add/remove from the org-latex-packages-alist etc (I'm
> guessing that perhaps some font related packages may need tweaking?).
>
> Ideally, what would be good is a very simple recipe for what someone
> should do in order to try out luatex and get the most out of it (or at
> least see potential).

I have no problem with my post being added to worg, but I don't have
much experience in working with worg... Of course, I can prepare
everything you need, if you think it might be useful.

The *only* difference between a minimal document for lualatex and a
minimal document for pdfLaTeX is that for LuaLaTeX it is not necessary
to load the fontenc and inputenc packages. The following mwe
compiles perfectly in LuaLaTeX:

\documentclass{article}
\begin{document}
Hello world!
á é í ó ú ñ à è ì ò ù
\end{document}

LuaTeX defaults to an otf version of the Computer Modern font, so any
user who isn't interested in fonts and writing in non-Latin languages,
but wants to work in a real Unicode environment, won't need to fine-tune
fonts, nor load any special package. The rest is exactly the same as any
document for pdfLaTeX.

If in the Org document is added:

#+LATEX_COMPILER: lualatex 

the fontenc and inputenc packages are not loaded in the output, which is
correct and it is the minimum requirement for LuaLaTeX. I think Org is
already doing a good job here.

If the user wants to use other fonts, the fontspec package must be
loaded. Depending on the user's needs, you can go from the simplest to
the most complex configurations (the different options and possibilities
are explained in detail in the fontspec manual). The simplest: if a user
just wants to use the Times New Roman font as the main font in his
document, this lines would suffice:

\usepackage{fontspec}
\setmainfont{Times New Roman}

That is, by indicating the name of the family (Times New Roman), luatex
would use this family for normal text, italics, bold, etc. Of course,
it's a good idea to load a family that has italic, bold, bold italic,
and other subtypes. Fontspec has tons more options, but this would be
the basics. But I think this aspect is more on the LaTeX side than in
the Org side.

LuaTeX can use the fonts installed on the system, without the need to
add more (that is, simply by putting the name of the family, LuaTeX
accesses them); and you can also use any font in any directory, just by
giving the path. I wrote BTW this little package to preview any font in
Emacs, and test the opentype features. it uses org-latex-preview in the
background and compiles with LuaTeX:

https://gitlab.com/maciaschain/org-font-spec-preview

Best regards,

Juan Manuel 

Re: LaTeX export: when is it more useful to use LuaTeX instead of pdfTeX?

[Juan Manuel Macías]

Hi, Maxim,

Max Nikulin writes:

> [...] With LuaTeX you get more convenient OTF and TTF font selection, but
> you you have to pay for the feature. It is necessary to explicitly
> specify all families: normal, typewriter, italics, etc if you need
> Unicode. -

Not necessarily. You can go from the simplest or basic to the most
complex, depending on the user's needs. If the user does not need to
write in non-Latin scripts, and is not particularly interested in fonts
and otf esoteric features, then the otf version of the Computer Modern
font (which LuaTeX uses by default) will suffice, as this font has
coverage for latin, latin-1, latin-extended, etc.

If you need to fine-tune fonts or work with non latin scripts, then it
is necessary to load fontspec. But equally here you can go from the most
basic to the most complex fontspec options and features, depending on
the needs of the user.

For example, if you want to write an article in both Russian (main
language) and English, and want to use the Old Standard font (which has
excellent coverage for Cyrillic), the basics might be:

\usepackage{fontspec}
\usepackage[english,russian]{babel}
\setmainfont{Old Standard}

or, using the Babel style (which requires fontspec in the background):

\babelfont{rm}{Old Standard}

That would be the classic setup. Another example, here with modern Babel
notation: an article in Spanish with the secondary language in ancient
Greek. We want to use Linux Libertine as the main font, and for
Greek Old Standard:

\usepackage[spanish]{babel}
\babelprovide[onchar=ids fonts,hyphenrules=ancientgreek]{greek}
\babelfont{rm}{Linux Libertine O}
\babelfont[greek]{rm}{Old Standard}

That would be the most basic. And, from there, everything can be made
more complex, according to the needs.

> There is babel LaTeX package. A decade ago polyglossia was
> a replacement of babel for XeTeX and LuaTex, but currently babel is
> recommended for these LaTeX engines as well.

That's correct. Babel is strongly recommended, and development of Babel is
very active.

A few months ago I proposed this patch here to adapt Org to the new
scenario regarding Babel and Polyglossia. It is a first approximation
and I have to review it. But the idea is to unify in a single list
(named `org-latex-language-alist' `org-latex-polyglossia-language-alist'
and `org-latex-babel-language-alist':

https://list.orgmode.org/87sfxiw2jp.fsf@posteo.net/

Best regards,

Juan Manuel 

Re: LaTeX export: when is it more useful to use LuaTeX instead of pdfTeX?

[Juan Manuel Macías]

Hi Thomas,

Thomas S. Dye writes:

> Yes, what I called Babel you call org-babel.  I don't know if the Lua
> handler of source blocks in Org might be useful for someone interested
> to write Lua extensions to LaTeX.

I'm writing a package for LuaLaTeX in Org[1] using lua code blocks, and
everything works fine. But if you mean to evaluate these blocks from
Org, it wouldn't really make sense because LuaTeX uses its own Lua
interpreter and that's where the code should be evaluated. For example,
in LuaTeX you should use tex.print or tex.sprint to print a result in
LaTeX, instead of 'print'.

A simple example to create a counter using Lua:

\newcommand{\mydefcounter}[2]{{\directlua{#1 = #2}}}
\newcommand{\mycounter}[2]{\directlua{%
    function count ()
    #1 = #1 + #2
    tex.print (#1)
    end
    count()
}}

\mydefcounter{foo}{0}

\mycounter{foo}{1}

\mycounter{foo}{1}

\mycounter{foo}{1}

You might want to take a look at the chickenize package, which includes
loads of examples and is very instructive.

https://www.ctan.org/pkg/chickenize

[1] btw, I thought I was the only one to write a LaTeX package in Org,
instead of the 'official' LaTeX docstrip suite (doing it in Org is
infinitely more comfortable!), but I've seen that the wallcalendar
package has also taken the unorthodox route, with all source code and
documentation in Org :-):

https://github.com/profound-labs/wallcalendar/blob/master/Makefile

Best regards,

Juan Manuel 

Re: LaTeX export: when is it more useful to use LuaTeX instead of pdfTeX?

[Max Nikulin]

On 09/07/2022 17:42, Juan Manuel Macías wrote:
> Max Nikulin writes:
> 
>> [...] With LuaTeX you get more convenient OTF and TTF font selection, but
>> you you have to pay for the feature. It is necessary to explicitly
>> specify all families: normal, typewriter, italics, etc if you need
>> Unicode. -
> 
> Not necessarily. You can go from the simplest or basic to the most
> complex, depending on the user's needs. If the user does not need to
> write in non-Latin scripts, and is not particularly interested in fonts
> and otf esoteric features, then the otf version of the Computer Modern
> font (which LuaTeX uses by default) will suffice, as this font has
> coverage for latin, latin-1, latin-extended, etc.

Juan Manuel, we are going to repeat the discussion happened a year ago. 
Has something changed since that time? LuaTeX uses Latin Modern and it 
is not nearly Unicode. PdfTeX out of the box has e.g. LH fonts - 
Cyrillic extension for Computer Modern, so it is enough to specify input 
and font encodings and it is not necessary to care concerning particular 
font.

> For example, if you want to write an article in both Russian (main
> language) and English, and want to use the Old Standard font (which has
> excellent coverage for Cyrillic), the basics might be:

My point is that it is not about what I want, it is related to what I 
have to do, namely choose, maybe install and explicitly configure some 
consistent set of fonts. With the following minimal example I got blank 
space instead of non-latin characters.

\documentclass{article}
\begin{document}
Abc — Αλφάβητο — Азбука…
\end{document}

Just have tried with
     This is LuaTeX, Version 1.10.0 (TeX Live 2019/Debian)
It is from Ubuntu-20.04 system package.

So to switch default engine in Org from PdfLaTeX to LuaLaTeX it is 
necessary to write some code inspecting available font set suitable for 
specified language.

Re: LaTeX export: when is it more useful to use LuaTeX instead of pdfTeX?

[Juan Manuel Macías]

Max Nikulin writes:

> LuaTeX uses Latin Modern
> and it is not nearly Unicode

Maxim, please look at this screenshots carefully:

https://i.imgur.com/uMfheCL.png

https://i.imgur.com/WwGybBA.png

https://i.imgur.com/hpreFNQ.png

Frankly, I don't know what Latin Modern you're referring to, and what
you mean by saying that "it is not nearly unicode".

The default LM font in LuaLaTeX and XeTeX is an otf and Unicode font.
Which is not to say that it has all the glyphs to represent all possible
characters. Because I guess you know the difference between glyph and
character...

Perhaps a font with a broader coverage could have been chosen by default
for LuaLaTeX, but here the (LaTeX) historical reasons have prevailed.
It's probably not the happiest choice, but LM is still a Unicode font
nonetheless.

And I insist: what you say about it being necessary to completely
configure everything related to fonts in LuaTeX is not correct. It
depends on the use case, and you can go (as I said in my previous email)
from the simplest to the most complex configuration.

On the other hand: I think that in the case of LuaTeX and XeTeX the
choice and configuration of fonts should be on the LaTeX side and not
Org's. Implementing that in Org would lead to a bunch of variables to
cater for all possible cases. It might suffice to give some examples of
basic use (like the ones I have put in the previous mail, and that will
be enough for most users) and recommend free license fonts for different
languages[1]. Maybe starting here:

https://tug.org/FontCatalogue/opentypefonts.html

But if the user needs more fine-tuning of fonts and languages, he/she
will undoubtedly have to read the fontspec and babel manuals, depending
on his needs, which can be very different from one user to another.

[1] Although I see it as unnecessary. Fonts are not recommended when the
output is odt...

> With the following minimal example I got
> blank space instead of non-latin characters.

> \documentclass{article}
> \begin{document}
> Abc — Αλφάβητο — Азбука…
> \end{document}

\documentclass{article}
% ================
\usepackage{fontspec}
\setmainfont{FreeSerif}
% ================
\begin{document}
Abc — Αλφάβητο — Азбука…
\end{document}

\usepackage{fontspec}\setmainfont{FreeSerif} is the same as choosing the
font in the libreoffice font menu.

You have to keep in mind that LuaTeX and XeTeX are designed so that it
is the user who decides which fonts to use and so that it's the user who
has the freedom to manage those fonts as he wishes. Okay: they could
have made a series of fallback fonts load by default to cover all
glyphs, for users who don't want to mess with typography. But I think
that this basic example that I have put is quite simple, and gives the
user the freedom to choose his preferred font and not the one imposed by
the system. And, at the end of the day, TeX is essentially a
typographical tool, whether you like it or not. Of course, LaTeX can be
used on autopilot because many scientific publications ask for articles
in LaTeX (with very little room for maneuver on the part of the authors,
since in the end a LaTeXpert will do the typesetting for the
publication). But there are also users who want to create their own book
layout from scratch, and use whatever font they want, in the same way as
when exporting to html from org there are users who like to write their
own css styles. LuaTeX and XeTeX offer the user that freedom, and it can
be done very simply. I have used pdfLaTeX for a long, long time and I
know very well how immensely laborious it was to install new type1
fonts, because I installed quite a few. Now, in LuaTeX any system font
can be used in a simple and fast way, I think it is something that users
should value more.

Re: LaTeX export: when is it more useful to use LuaTeX instead of pdfTeX?

[Juan Manuel Macías]

Max Nikulin writes:

> Characters from Latin scripts, the set is wider than latin-1 but does
> not cover other languages. I do not dispute that font encoding is
> Unicode (if it can be stated so), usually support of Unicode is
> associated with smooth experience with wide range of languages.

A Unicode font is a Unicode-encoded font. It can have 2 glyphs or 2000.
But it's still a Unicode font.

> But for default settings getting blank instead of text in some routine
> notes is hardly acceptable. Unfortunately \setmainfont is not enough.
> Starting for "the simplest of basic" on the next step a user may
> notice that bold...

Please, compile this:

\documentclass{article}
\usepackage{fontspec}
\setmainfont{FreeSerif}
\begin{document}
Abc — Αλφάβητο — Азбука…
\emph{Abc — Αλφάβητο — Азбука…}
\textbf{Abc — Αλφάβητο — Азбука…}
\textbf{\emph{Abc — Αλφάβητο — Азбука…}}

With \setmainfont{FreeSerif} I'm telling LuaTeX to use the full
FreeSerif family as the main roman font, which includes bold, italic,
and bold italic. It is the duty of the user (at least the LuaTeX user)
to know that this family is present in his/her system and includes those
styles.

> or typewriter text is missing.

\setmainfont{FreeSerif}
\setsansfont{FreeSans}
\setmonofont{FreeMono}

I honestly don't understand why you find it unacceptable that the
responsibility for managing fonts and the languages associated with
those fonts falls on the user. It is to be expected. And it is something
that has finally corrected a great anomaly that TeX and LaTeX has always
been dragging along for almost its entire history, and that put it
(being more powerful and sophisticated) behind other types of
typesetting software like Indesign or quark. LuaTeX and XeTeX are
digital typesetting systems. They are not word processors. The user who
wants fine tuning in this regard will have to read the fontspec manual
and the babel or polyglossia manual thoroughly. I can agree with you
that the choice of the "default" font (Latin Modern) is not exactly happy,
due to the little coverage that this font has for non-Latin scripts. But
the demanding LuaTeX user is rarely going to use latin modern (I've
never used it in my life for real production). I think I made it clear
in the first post of this thread what kind of use cases LuaTeX is
suitable for. If someone finds that unacceptable, of course you'd better
use pdfLaTeX. By the way, in pdfLaTeX you can't write Greek out of the
box, nor Arabic, nor Japanese, etc., etc. So I don't see where the
difference is.

And even so, I insist, it is not necessary to go into typographical
depths. A configuration like the one I put before
(FreeSerif/FreeSans/FreeMono) will satisfy 90% of lazy users or those
who want to use LaTeX in autopilot mode.

Is it possible to implement all that in Org in such a way that a minimum
preamble is generated with what is necessary? How to define "what is
necessary", when there are thousands of options in fontspec and many
ways to declare and define font families and font features with
fontspec, with babel and with polyglossia? That's not counting
specialized packages for Arabic, Japanese, etc. (and I am writing a
package for greek). I think doing something like that in Org would be
highly intrusive on Org's part. Maybe something very, very, very basic
and minimal would work in order to avoid those empty spaces that seem
unacceptable to you, maybe three variables for
setmainfont/-sansfont/-monofont[1]. Going further, in my opinion, makes
no sense. I think it is much more important to unify in org the issue of
languages, polyglossia and babel, because as it is now it collects an
obsolete scenario.

[1] In my opinion, something similar to what pandoc does by default,
using the iftex package, would suffice:

\usepackage{iftex}
\ifPDFTeX
\usepackage[T1]{fontenc}
\usepackage[utf8]{inputenc}
\usepackage{textcomp} 
\else
  \usepackage{fontspec}
\usepackage{unicode-math}
\defaultfontfeatures{Scale=MatchLowercase}
\defaultfontfeatures[\rmfamily]{Ligatures=TeX}
\setmainfont{FreeSerif}
\setsansfont{FreeSans}
\setmonofont{FreeMono}
\fi

Re: LaTeX export: when is it more useful to use LuaTeX instead of pdfTeX?

[Tim Cross]

Juan Manuel Macías <maciaschain@posteo.net> writes:

> Hi, Tim, thank you for your comments,
>
> Tim Cross writes:
>
>> Juan, I think it would be great to add your post to worg. I'm happy to
>> do this, but I think it wold also be good if we could include a basic
>> 'setup' i.e. what changes people might need to (or should do to maximise
>> benefit) in order to try out luatex. For example, what settings to put
>> in org-latex-pdf-process (I'm guessing something like "lualatex
>> -interaction nonstopmode -output-directory %o %f") and what (if any)
>> packages to add/remove from the org-latex-packages-alist etc (I'm
>> guessing that perhaps some font related packages may need tweaking?).
>>
>> Ideally, what would be good is a very simple recipe for what someone
>> should do in order to try out luatex and get the most out of it (or at
>> least see potential).
>
> I have no problem with my post being added to worg, but I don't have
> much experience in working with worg... Of course, I can prepare
> everything you need, if you think it might be useful.
>
> The *only* difference between a minimal document for lualatex and a
> minimal document for pdfLaTeX is that for LuaLaTeX it is not necessary
> to load the fontenc and inputenc packages. The following mwe
> compiles perfectly in LuaLaTeX:
>
> \documentclass{article}
> \begin{document}
> Hello world!
> á é í ó ú ñ à è ì ò ù
> \end{document}
>
> LuaTeX defaults to an otf version of the Computer Modern font, so any
> user who isn't interested in fonts and writing in non-Latin languages,
> but wants to work in a real Unicode environment, won't need to fine-tune
> fonts, nor load any special package. The rest is exactly the same as any
> document for pdfLaTeX.
>
> If in the Org document is added:
>
> #+LATEX_COMPILER: lualatex 
>
> the fontenc and inputenc packages are not loaded in the output, which is
> correct and it is the minimum requirement for LuaLaTeX. I think Org is
> already doing a good job here.
>
> If the user wants to use other fonts, the fontspec package must be
> loaded. Depending on the user's needs, you can go from the simplest to
> the most complex configurations (the different options and possibilities
> are explained in detail in the fontspec manual). The simplest: if a user
> just wants to use the Times New Roman font as the main font in his
> document, this lines would suffice:
>
> \usepackage{fontspec}
> \setmainfont{Times New Roman}
>
> That is, by indicating the name of the family (Times New Roman), luatex
> would use this family for normal text, italics, bold, etc. Of course,
> it's a good idea to load a family that has italic, bold, bold italic,
> and other subtypes. Fontspec has tons more options, but this would be
> the basics. But I think this aspect is more on the LaTeX side than in
> the Org side.
>
> LuaTeX can use the fonts installed on the system, without the need to
> add more (that is, simply by putting the name of the family, LuaTeX
> accesses them); and you can also use any font in any directory, just by
> giving the path. I wrote BTW this little package to preview any font in
> Emacs, and test the opentype features. it uses org-latex-preview in the
> background and compiles with LuaTeX:
>
> https://gitlab.com/maciaschain/org-font-spec-preview
>

Thanks Juan. It will be fairly trivial to compile the information you
have provided into a basic org document which I can then add to org. If
on the other hand you would prefer to write it up, all I need is an org
document which is based on the (current) org 'worg' template, which is
very simple i.e.

#+:begin_src org

#+TITLE:      [No title for now, please update]
#+AUTHOR:     Worg people
#+OPTIONS:    H:3 num:nil toc:t \n:nil ::t |:t ^:t -:t f:t *:t tex:t d:(HIDE) tags:not-in-toc
#+STARTUP:    align fold nodlcheck hidestars oddeven lognotestate
#+SEQ_TODO:   TODO(t) INPROGRESS(i) WAITING(w@) | DONE(d) CANCELED(c@)
#+TAGS:       Write(w) Update(u) Fix(f) Check(c) 
#+LANGUAGE:   en
#+PRIORITIES: A C B
#+CATEGORY:   worg
#+HTML_LINK_UP:    index.html
#+HTML_LINK_HOME:  https://orgmode.org/worg/

# This file is released by its authors and contributors under the GNU
# Free Documentation license v1.3 or later, code examples are released
# under the GNU General Public License v3 or later.

# This file is the default header for new Org files in Worg.  Feel free
# to tailor it to your needs.

#+end_src

Re: LaTeX export: when is it more useful to use LuaTeX instead of pdfTeX?

[Max Nikulin]

I'm sorry, again, replying to the private copy of the message sent as 
Cc, I dropped mail list address at first.

Please, consider my response in the following context:

https://list.orgmode.org/orgmode/87a69j9c6s.fsf@localhost/
Ihor Radchenko, 2022-07-09:
> Or we may go even further and make org-latex-compiler default to luatex.
> This will benefit all the non-latin language users.

On 09/07/2022 21:58, Juan Manuel Macías wrote:
> Max Nikulin writes:
> 
>> LuaTeX uses Latin Modern
>> and it is not nearly Unicode
> 
> Maxim, please look at this screenshots carefully:
> 
> https://i.imgur.com/uMfheCL.png

This set of characters is covered by latin-1.

> https://i.imgur.com/WwGybBA.png

Characters from Latin scripts, the set is wider than latin-1 but does 
not cover other languages. I do not dispute that font encoding is 
Unicode (if it can be stated so), usually support of Unicode is 
associated with smooth experience with wide range of languages.

> Frankly, I don't know what Latin Modern you're referring to, and what
> you mean by saying that "it is not nearly unicode".

/usr/share/texmf/fonts/opentype/public/lm/lmroman10-regular.otf I 
noticed in the LuaLaTeX log. Do you get non-latin characters with my 
example (without modifying \setmainfont) on your machine?

> \documentclass{article}
> % ================
> \usepackage{fontspec}
> \setmainfont{FreeSerif}
> % ================
> \begin{document}
> Abc — Αλφάβητο — Азбука…
> \end{document}
> 
> \usepackage{fontspec}\setmainfont{FreeSerif} is the same as choosing the
> font in the libreoffice font menu.

I rarely use libreoffice, so settings should be close to defaults. I can 
just paste this text and I see whole snippet without additional actions. 
I have no idea why Liberation Serif is chosen, but the default font has 
much better coverage, so it is suitable for more users.

> But I think
> that this basic example that I have put is quite simple, and gives the
> user the freedom to choose his preferred font and not the one imposed by
> the system.

My point it that such freedom is not for free. If you know which font 
you would like to have in a book, you are ready to add some settings and 
LuaLaTeX has advantages in such case.

But for default settings getting blank instead of text in some routine 
notes is hardly acceptable. Unfortunately \setmainfont is not enough. 
Starting for "the simplest of basic" on the next step a user may notice 
that bold or typewriter text is missing.

So LuaLaTeX should be a conscious choice of users ready to add set of 
fonts for each language used in the document. I do not try to say that 
LuaLaTeX has no advantages. Application such as browsers or office has a 
feature suitable for routine documents: graceful degradation in respect 
to glyphs missed in the specified font. For publishers in some cases it 
may be a disaster (however I believe that ideally such issues should be 
discovered from logs even when not apparent from visual appearance of 
the document).

I am unsure if it was made by design or TeX engines with native support 
of Unicode fonts should made another step further, but currently Org is 
able to provide default preamble for PdfLaTeX, but not for LuaLaTeX and 
the latter is at least not trivial.

M-x org-create-worg-article command? (was: LaTeX export: when is it more useful to use LuaTeX instead of pdfTeX?)

[Ihor Radchenko]

Tim Cross <theophilusx@gmail.com> writes:

> Juan Manuel Macías <maciaschain@posteo.net> writes:
>
> Thanks Juan. It will be fairly trivial to compile the information you
> have provided into a basic org document which I can then add to org. If
> on the other hand you would prefer to write it up, all I need is an org
> document which is based on the (current) org 'worg' template, which is
> very simple i.e.
>
> #+:begin_src org
>
> #+TITLE:      [No title for now, please update]
> #+AUTHOR:     Worg people
> #+OPTIONS:    H:3 num:nil toc:t \n:nil ::t |:t ^:t -:t f:t *:t tex:t d:(HIDE) tags:not-in-toc
> ...

Maybe Org can provide this template simply from M-x?
Would you find such addition useful?

Best,
Ihor

Re: M-x org-create-worg-article command? (was: LaTeX export: when is it more useful to use LuaTeX instead of pdfTeX?)

[Tim Cross]

Ihor Radchenko <yantar92@gmail.com> writes:

> Tim Cross <theophilusx@gmail.com> writes:
>
>> Juan Manuel Macías <maciaschain@posteo.net> writes:
>>
>> Thanks Juan. It will be fairly trivial to compile the information you
>> have provided into a basic org document which I can then add to org. If
>> on the other hand you would prefer to write it up, all I need is an org
>> document which is based on the (current) org 'worg' template, which is
>> very simple i.e.
>>
>> #+:begin_src org
>>
>> #+TITLE:      [No title for now, please update]
>> #+AUTHOR:     Worg people
>> #+OPTIONS:    H:3 num:nil toc:t \n:nil ::t |:t ^:t -:t f:t *:t tex:t d:(HIDE) tags:not-in-toc
>> ...
>
> Maybe Org can provide this template simply from M-x?
> Would you find such addition useful?
>

That could be a good and easy addition. If we can streamline the worg
submission process, we can probably improve the worg content quite a
bit.

Only drawback I can see is that should we want to change the template,
we would have to wait until a new version is released and then you will
still have a mix of templates as lots of people will wait until next
Emacs version etc.

The question I guess comes down to how often we would need to change the
template - probably very infrequently. I do plan to change the current
template, but if anything, that will be simplifying it. 

I will add this idea to the TODO list!

Re: LaTeX export: when is it more useful to use LuaTeX instead of pdfTeX?

[Juan Manuel Macías]

Tim Cross writes:

> Thanks Juan. It will be fairly trivial to compile the information you
> have provided into a basic org document which I can then add to org. If
> on the other hand you would prefer to write it up, all I need is an org
> document which is based on the (current) org 'worg' template, which is
> very simple i.e.
>
> #+:begin_src org
>
> #+TITLE:      [No title for now, please update]
> #+AUTHOR:     Worg people
>
> #+OPTIONS:    H:3 num:nil toc:t \n:nil ::t |:t ^:t -:t f:t *:t tex:t d:(HIDE) tags:not-in-toc
> #+STARTUP:    align fold nodlcheck hidestars oddeven lognotestate
>
> #+SEQ_TODO:   TODO(t) INPROGRESS(i) WAITING(w@) | DONE(d) CANCELED(c@)
>
> #+TAGS:       Write(w) Update(u) Fix(f) Check(c)
> #+LANGUAGE:   en
>
> #+PRIORITIES: A C B
> #+CATEGORY:   worg
>
> #+HTML_LINK_UP:    index.html
> #+HTML_LINK_HOME:  https://orgmode.org/worg/
>
> # This file is released by its authors and contributors under the GNU
> # Free Documentation license v1.3 or later, code examples are released
> # under the GNU General Public License v3 or later.
>
> # This file is the default header for new Org files in Worg.  Feel free
> # to tailor it to your needs.
>
> #+end_src

Thanks so much for all the pointers, Tim.

I can collect and clean up (and possibly expand) all the information
I've put in this thread and make an Org document with this template. I
agree with Ihor that the template would be a great addition to Org-Mode.

Best regards,

Juan Manuel

[possible patch] Basic fontspec code for LuaLaTeX and XelaTeX (was "LaTeX export: when is it more useful...")

[Juan Manuel Macías]

Considering some discussions in the parent thread, I think maybe it
wouldn't hurt to ensure a minimal preamble when the output is compiled
with LuaLaTeX or XelaTeX, so that some very basic fontspec configuration
is loaded to be able to read PDFs in non-Latin scripts.

But before proposing the patch directly, I'd like to discuss its
structure. I think (IMHO) that a certain balance should be ensured
between a) users who don't want to mess with fontspec and want something
more out-of-the-box and b) users who prefer to be in control when
compiling with LuaTeX and XeTeX.

I think maybe it would be nice to let LaTeX do the work, via a
conditional from the iftex package (idea taken from pandoc).

The structure of the patch could be this:

1. There could be a defcustom, something like 'org-latex-use-fontspec'
(I would vote for nil by default).

2. There would be three variables for the default fonts: roman, sans and
mono. By default, the FreeSerif, FreeSans and FreeMono fonts could be
set as default value, since they are very ubiquitous and have a very
good coverage for non-Latin scripts.

3. A variable (something like 'org-latex-fontspec-default-configuration') would return something like this:

(format
 \\usepackage{iftex}
 \\ifpdftex
 \\relax
 \\else
 \\usepackage{fontspec}
 \\usepackage{unicode-math}
 \\defaultfontfeatures{Scale=MatchLowercase}
 \\defaultfontfeatures[\\rmfamily]{Ligatures=TeX}
 \\setmainfont{%s}
 \\setsansfont{%s}
 \\setmonofont{%s}
 \\fi
 org-latex-fontspec-mainfont
 org-latex-fontspec-sansfont
 org-latex-fontspec-monofont)

(and this string would be added at some point to org-latex-make-preamble)

4. Conclusion: I think the good thing about letting LaTeX do the
conditional work with iftex is that it saves us less invasive code on
our end. I also think that other more complex approaches, such as
searching for the fonts present in the system and adding them according
to the document scripts, would lead us to a completely slippery slope.
Of course, a list of recommended free-licensed fonts could be included
in the documentation.

WDYT?

Best regards,

Juan Manuel

Re: [possible patch] Basic fontspec code for LuaLaTeX and XelaTeX (was "LaTeX export: when is it more useful...")

[Juan Manuel Macías]

Sorry, I forgot to add quotes :-)  "\\usepackage{iftex}...\\fi"

Juan Manuel Macías writes:

> (format
>  \\usepackage{iftex}
>  \\ifpdftex
>  \\relax
>  \\else
>  \\usepackage{fontspec}
>  \\usepackage{unicode-math}
>  \\defaultfontfeatures{Scale=MatchLowercase}
>  \\defaultfontfeatures[\\rmfamily]{Ligatures=TeX}
>  \\setmainfont{%s}
>  \\setsansfont{%s}
>  \\setmonofont{%s}
>  \\fi
>  org-latex-fontspec-mainfont
>  org-latex-fontspec-sansfont
>  org-latex-fontspec-monofont)

Re: [possible patch] Basic fontspec code for LuaLaTeX and XelaTeX (was "LaTeX export: when is it more useful...")

[Tim Cross]

Juan Manuel Macías <maciaschain@posteo.net> writes:

> Considering some discussions in the parent thread, I think maybe it
> wouldn't hurt to ensure a minimal preamble when the output is compiled
> with LuaLaTeX or XelaTeX, so that some very basic fontspec configuration
> is loaded to be able to read PDFs in non-Latin scripts.
>
> But before proposing the patch directly, I'd like to discuss its
> structure. I think (IMHO) that a certain balance should be ensured
> between a) users who don't want to mess with fontspec and want something
> more out-of-the-box and b) users who prefer to be in control when
> compiling with LuaTeX and XeTeX.
>
> I think maybe it would be nice to let LaTeX do the work, via a
> conditional from the iftex package (idea taken from pandoc).
>
> The structure of the patch could be this:
>
> 1. There could be a defcustom, something like 'org-latex-use-fontspec'
> (I would vote for nil by default).
>
> 2. There would be three variables for the default fonts: roman, sans and
> mono. By default, the FreeSerif, FreeSans and FreeMono fonts could be
> set as default value, since they are very ubiquitous and have a very
> good coverage for non-Latin scripts.
>
> 3. A variable (something like 'org-latex-fontspec-default-configuration') would return something like this:
>
> (format
>  \\usepackage{iftex}
>  \\ifpdftex
>  \\relax
>  \\else
>  \\usepackage{fontspec}
>  \\usepackage{unicode-math}
>  \\defaultfontfeatures{Scale=MatchLowercase}
>  \\defaultfontfeatures[\\rmfamily]{Ligatures=TeX}
>  \\setmainfont{%s}
>  \\setsansfont{%s}
>  \\setmonofont{%s}
>  \\fi
>  org-latex-fontspec-mainfont
>  org-latex-fontspec-sansfont
>  org-latex-fontspec-monofont)
>
> (and this string would be added at some point to org-latex-make-preamble)
>
> 4. Conclusion: I think the good thing about letting LaTeX do the
> conditional work with iftex is that it saves us less invasive code on
> our end. I also think that other more complex approaches, such as
> searching for the fonts present in the system and adding them according
> to the document scripts, would lead us to a completely slippery slope.
> Of course, a list of recommended free-licensed fonts could be included
> in the documentation.
>
> WDYT?
>

I'll prefix this by being very clear that I'm so out of my depth, I know
nothing! I'm an Australian who lives on the worlds largest
island. Despite being a country where 1/4 people have at least one
parent born in a non-english speaking country, Australia is at this time
monolingual. As a consequence, I've never had to deal with fonts other
than trying to select one which 'looks nice'. This tends to be something
I leave to Latex as my aesthetic skills are only slightly better than my
language skills!

Like many, I have had to struggle with fonts at one time or another -
typically, it was with respect to type formatting of mathematics/logic
and it was what got me using Latex originally (30+ years ago). I rarely
need to do this now.

So, my perspective on this is fairly basic.

   - I think the move to luatex is important for org, especially given
     the rise of packages which use/need it

   - It seems like luatex could make org easier to use for those who do
     need support for other non-latin languages and especially for those
     who need to work in multiple languages.

   - For many simpler people like me, I just want it to work. When I
     export to a PDF document, I want to continue to have people say
     "Wow, that is a good looking document, what is your secret" and I
     can reply, "Don't use MS Office!". I don't want to mess with
     selecting fonts, defining font specs etc. I want good defaults.

  - For many people, it seems fonts are a very personal and important
    component and they want the power to manage them at a lower
    level. Therefore, support for this user is as important as my use
    case. They need to be able to adapt their document to their
    preferred fonts without having to code elisp or low level latex/tex.

Juan, if I understand your proposal correctly, I think your on the right
track. It sounds like what you are proposing would have almost no impact
on basic users like me, but would allow those with more demanding
requirements to adjust without too much effort. I originally raised the
question regarding what would need to change with the switch to uatex to
ensure that we do actually get things positioned to enable people to
exploit the benefits and not just switch out one tool for another which
only appears to be a little slower. I think what you are suggesting
addresses that concern. 

but as I said, I know nothing....

Re: M-x org-create-worg-article command? (was: LaTeX export: when is it more useful to use LuaTeX instead of pdfTeX?)

[Ihor Radchenko]

Tim Cross <theophilusx@gmail.com> writes:

> Only drawback I can see is that should we want to change the template,
> we would have to wait until a new version is released and then you will
> still have a mix of templates as lots of people will wait until next
> Emacs version etc.

The template can be automatically updated by the command, possibly after
asking user.

Best,
Ihor

Re: [possible patch] Basic fontspec code for LuaLaTeX and XelaTeX (was "LaTeX export: when is it more useful...")

[Ihor Radchenko]

Juan Manuel Macías <maciaschain@posteo.net> writes:

> Considering some discussions in the parent thread, I think maybe it
> wouldn't hurt to ensure a minimal preamble when the output is compiled
> with LuaLaTeX or XelaTeX, so that some very basic fontspec configuration
> is loaded to be able to read PDFs in non-Latin scripts.

+1

> But before proposing the patch directly, I'd like to discuss its
> structure. I think (IMHO) that a certain balance should be ensured
> between a) users who don't want to mess with fontspec and want something
> more out-of-the-box and b) users who prefer to be in control when
> compiling with LuaTeX and XeTeX.
>
> I think maybe it would be nice to let LaTeX do the work, via a
> conditional from the iftex package (idea taken from pandoc).
>
> The structure of the patch could be this:
>
> 1. There could be a defcustom, something like 'org-latex-use-fontspec'
> (I would vote for nil by default).

Does it mean that unicode text (like це or 这个) will not be exported by default?

> 2. There would be three variables for the default fonts: roman, sans and
> mono. By default, the FreeSerif, FreeSans and FreeMono fonts could be
> set as default value, since they are very ubiquitous and have a very
> good coverage for non-Latin scripts.

+1
But can someone check if Free* fonts are available on Windows and Mac by
default? If not, can we distribute these fonts with Org?

> 3. A variable (something like 'org-latex-fontspec-default-configuration') would return something like this:
>
> (format
>  \\usepackage{iftex}
>  \\ifpdftex
>  \\relax
>  \\else
>  \\usepackage{fontspec}
>  \\usepackage{unicode-math}
>  \\defaultfontfeatures{Scale=MatchLowercase}
>  \\defaultfontfeatures[\\rmfamily]{Ligatures=TeX}
>  \\setmainfont{%s}
>  \\setsansfont{%s}
>  \\setmonofont{%s}
>  \\fi
>  org-latex-fontspec-mainfont
>  org-latex-fontspec-sansfont
>  org-latex-fontspec-monofont)
>
> (and this string would be added at some point to org-latex-make-preamble)

Makes sense. Though I'd rather use format-spec instead to allow
arbitrary order of variable formatting.

> 4. Conclusion: I think the good thing about letting LaTeX do the
> conditional work with iftex is that it saves us less invasive code on
> our end. I also think that other more complex approaches, such as
> searching for the fonts present in the system and adding them according
> to the document scripts, would lead us to a completely slippery slope.
> Of course, a list of recommended free-licensed fonts could be included
> in the documentation.
>
> WDYT?

This unified preamble approach is consistent with what we do now.
However, our currently used large preambles will slow down compilation.

As I recall, Timothy has been working on simplifying preamble
generation. If we do not put unnecessary packages into preamble,
compilation will be significantly faster.

If Timothy can come up with a patch some time soon, I'd prefer to have a
more targeted preamble. Otherwise, the proposed approach is the way to
go.

Best,
Ihor

Re: M-x org-create-worg-article command? (was: LaTeX export: when is it more useful to use LuaTeX instead of pdfTeX?)

[Tim Cross]

Ihor Radchenko <yantar92@gmail.com> writes:

> Tim Cross <theophilusx@gmail.com> writes:
>
>> Only drawback I can see is that should we want to change the template,
>> we would have to wait until a new version is released and then you will
>> still have a mix of templates as lots of people will wait until next
>> Emacs version etc.
>
> The template can be automatically updated by the command, possibly after
> asking user.
>

Not sure I understand. Maybe we are imagining different things?

If org has a template to assist in creating an org file suitable for
worg and then we need to update that template for some reason, where
would that new 'update' come from in order to be automatically updated?

Re: M-x org-create-worg-article command? (was: LaTeX export: when is it more useful to use LuaTeX instead of pdfTeX?)

[Ihor Radchenko]

Tim Cross <theophilusx@gmail.com> writes:

>> The template can be automatically updated by the command, possibly after
>> asking user.
>>
>
> Not sure I understand. Maybe we are imagining different things?
>
> If org has a template to assist in creating an org file suitable for
> worg and then we need to update that template for some reason, where
> would that new 'update' come from in order to be automatically updated?

By "updated" I meant downloaded from orgmode.org

Best,
Ihor

Re: [possible patch] Basic fontspec code for LuaLaTeX and XelaTeX (was "LaTeX export: when is it more useful...")

[Greg Minshall]

Juan Manuel,

> 1. There could be a defcustom, something like 'org-latex-use-fontspec'
> (I would vote for nil by default).

i just wanted to check: the "nil" case is for those of us who just want
it to work "out of the box"?

and, in the non-nil case, it would be up to the user to use "fontspec",
or whatever?

cheers, Greg

Re: M-x org-create-worg-article command? (was: LaTeX export: when is it more useful to use LuaTeX instead of pdfTeX?)

[Tim Cross]

Ihor Radchenko <yantar92@gmail.com> writes:

> Tim Cross <theophilusx@gmail.com> writes:
>
>>> The template can be automatically updated by the command, possibly after
>>> asking user.
>>>
>>
>> Not sure I understand. Maybe we are imagining different things?
>>
>> If org has a template to assist in creating an org file suitable for
>> worg and then we need to update that template for some reason, where
>> would that new 'update' come from in order to be automatically updated?
>
> By "updated" I meant downloaded from orgmode.org
>

OK, that would probably work. We would need to have some sort of
version tracking so that the template function can know when there is a
new template available - probably doable with either a comment in the
template file or perhaps adding a custom http header using nginx's
headers module. Advantage of the header is you could do a quick query
without having to download and parse the template file to get version
info. 

Will add it to the list

Re: M-x org-create-worg-article command? (was: LaTeX export: when is it more useful to use LuaTeX instead of pdfTeX?)

[Ihor Radchenko]

Tim Cross <theophilusx@gmail.com> writes:

>> By "updated" I meant downloaded from orgmode.org
>>
>
> OK, that would probably work. We would need to have some sort of
> version tracking so that the template function can know when there is a
> new template available - probably doable with either a comment in the
> template file or perhaps adding a custom http header using nginx's
> headers module. Advantage of the header is you could do a quick query
> without having to download and parse the template file to get version
> info. 

I do not think that we need to go this far. The template will anyway be
a few kb. The ping time to query template version may take longer than
downloading.

Instead, we can simply use org-persist to download the template and then
set some reasonable value of expiry (or just use the default 30 days).
Then, the template will be refreshed monthly, which is not a big deal
IMHO. I do not expect that we need to update the template more
frequently.

Best,
Ihor

Re: [possible patch] Basic fontspec code for LuaLaTeX and XelaTeX (was "LaTeX export: when is it more useful...")

[Stefan Nobis]

Ihor Radchenko <yantar92@gmail.com> writes:

> But can someone check if Free* fonts are available on Windows and
> Mac by default?

I just checked TeXLive (on MacOS, but should be the same on all
systems): The Free* fonts are part of TeXLive as truetype and as
opentype versions (and partly in other formats).

For Windows I remember vaguely that some prefer the MikTeX
distribution (TeXLive is also available for Windows and has the same
fonts as everywhere). A short search shows that the gnu-freefont set
is also availabe for MikTeX, but I currently don't know whether it
will be installed with a default MikTeX installation.

> This unified preamble approach is consistent with what we do now.
> However, our currently used large preambles will slow down compilation.

Not that much. The time consuming packages like tikz/pgf (used by
beamer) are not part of out default preamble. There is not that much
speed to gain (all times are for a single lualatex run):

1) Only hyperref loaded, no other packages:
   0.46s user 0.10s system 99% cpu 0.568 total

2) Complete default preamble for lualatex:
   0.48s user 0.14s system 99% cpu 0.623 total

3) The same as above, but with babel and mathtools:
   0.51s user 0.15s system 99% cpu 0.673 total

4) And another variant (same as before, but package caption instead of
   capt-of):
   0.53s user 0.14s system 98% cpu 0.674 total

5) Back to our default preamble, but adding fontspec:
   0.60s user 0.14s system 99% cpu 0.748 total

6) With fontspec, unicode-math, babel, mathtools, caption:
   1.02s user 0.19s system 99% cpu 1.220 total

Therefore most of out default packages (and even the addition of
babel) does not change much for the speed of compilation. I don't
think its worth to try to further optimize this default preamble.

Adding fontspec and especially unicode-math adds quite some time, so
maybe its worth to take care to only add these if necessary (only for
lualatex/xelatex and only if e.g. if a font has been selected or math
seems to be used in the document).

And, by the way, our preamble is neither large nor complex. For my
LaTeX documents, the preamble is usually *much* larger. :)


Here is the test file for the default preamble (but with mathtools
instead of amsmath and with babel; test run 3):

--8<---------------cut here---------------start------------->8---

% Intended LaTeX compiler: lualatex
\documentclass{article}
\usepackage[english, safe=none, math=normal]{babel}
\usepackage{graphicx}
\usepackage{longtable}
\usepackage{wrapfig}
\usepackage{rotating}
\usepackage[normalem]{ulem}
\usepackage{mathtools}
\usepackage{amssymb}
\usepackage{capt-of}
\usepackage{hyperref}
\date{\today}
\title{Test Document}
\hypersetup{
 pdfcreator={Emacs 28.1 (Org mode 9.5.4)},
 pdflang={English}}
\begin{document}
This is a short test document.
\end{document}

--8<---------------cut here---------------end--------------->8---


Here is the test file for the last run with all extra packages:

--8<---------------cut here---------------start------------->8---

% Intended LaTeX compiler: lualatex
\documentclass{article}
\usepackage{fontspec}
\usepackage[english, safe=none, math=normal]{babel}
\usepackage{graphicx}
\usepackage{longtable}
\usepackage{wrapfig}
\usepackage{rotating}
\usepackage[normalem]{ulem}
\usepackage{mathtools}
\usepackage{amssymb}
\usepackage[warnings-off={mathtools-colon,mathtools-overbracket}]{unicode-math}
\usepackage{caption}
\usepackage{hyperref}
\date{\today}
\title{Test Document}
\hypersetup{
 pdfcreator={Emacs 28.1 (Org mode 9.5.4)},
 pdflang={English}}
\begin{document}
This is a short test document.
\end{document}

--8<---------------cut here---------------end--------------->8---



-- 
Until the next mail...,
Stefan.

Re: [possible patch] Basic fontspec code for LuaLaTeX and XelaTeX (was "LaTeX export: when is it more useful...")

[Stefan Nobis]

Juan Manuel Macías <maciaschain@posteo.net> writes:

> 1. There could be a defcustom, something like 'org-latex-use-fontspec'
> (I would vote for nil by default).

I would vote to activate this by default.

> (format
>  \\usepackage{iftex}
>  \\ifpdftex
>  \\relax
>  \\else
>  \\usepackage{fontspec}
>  \\usepackage{unicode-math}
>  \\defaultfontfeatures{Scale=MatchLowercase}
>  \\defaultfontfeatures[\\rmfamily]{Ligatures=TeX}
>  \\setmainfont{%s}
>  \\setsansfont{%s}
>  \\setmonofont{%s}
>  \\fi
>  org-latex-fontspec-mainfont
>  org-latex-fontspec-sansfont
>  org-latex-fontspec-monofont)

I would prefer to make it easier to stick with the default fonts. So
only add the font selection commands (including defaultfontfeatures)
when the font variables are non-nil. If no font has been explicitly
chosen, just use the default (in case of lualatex Latin Modern).

For me, it does not matter whether the 'org-latex-fontspec-*'
variables have a default of nil or set to the Free* fonts or something
else. For my configuration, I would set these variable to nil in order
to get the LaTeX default fonts and would like to go with the default
preamble of Org and then add to this on a per document basis.

This way, the whole configuration would be a little more composable, I
think.

-- 
Until the next mail...,
Stefan.

Re: [possible patch] Basic fontspec code for LuaLaTeX and XelaTeX (was "LaTeX export: when is it more useful...")

[Juan Manuel Macías]

Stefan Nobis writes:

> Juan Manuel Macías <maciaschain@posteo.net> writes:
>
>> 1. There could be a defcustom, something like 'org-latex-use-fontspec'
>> (I would vote for nil by default).
>
> I would vote to activate this by default.

I voted nil because of the available fonts issue. But I think what you
say below is a good idea, so it could be activated by default

>> (format
>>  \\usepackage{iftex}
>>  \\ifpdftex
>>  \\relax
>>  \\else
>>  \\usepackage{fontspec}
>>  \\usepackage{unicode-math}
>>  \\defaultfontfeatures{Scale=MatchLowercase}
>>  \\defaultfontfeatures[\\rmfamily]{Ligatures=TeX}
>>  \\setmainfont{%s}
>>  \\setsansfont{%s}
>>  \\setmonofont{%s}
>>  \\fi
>>  org-latex-fontspec-mainfont
>>  org-latex-fontspec-sansfont
>>  org-latex-fontspec-monofont)
>
> I would prefer to make it easier to stick with the default fonts. So
> only add the font selection commands (including defaultfontfeatures)
> when the font variables are non-nil. If no font has been explicitly
> chosen, just use the default (in case of lualatex Latin Modern).
>
> For me, it does not matter whether the 'org-latex-fontspec-*'
> variables have a default of nil or set to the Free* fonts or something
> else. For my configuration, I would set these variable to nil in order
> to get the LaTeX default fonts and would like to go with the default
> preamble of Org and then add to this on a per document basis.
>
> This way, the whole configuration would be a little more composable, I
> think.

Sounds like a good idea and I agree. If I understand correctly, if the
sans, roman, and mono font variables (or any of them) are non-nil,
enable font selection. Otherwise, leave the default Latin Modern font.

By the way, although I've already commented on it in some post in the
parent thread, i think this package I wrote might be useful for doing a
quick visual test of a font (including opentype features test), using
org-latex-preview (compiling with LuaTeX). It can be done on any font
marked in dired. There are three options: insert arbitrary characters,
insert the Unicode code of the characters, or display a specimen of the
font. The default specimen is in the file specimen.tex, which can be
edited to add examples and languages.

Some screenshots:

https://i.imgur.com/3faKSjA.png

https://i.imgur.com/OJfUcO9.png

To create font tables I often use the LaTeX package unicodefonttable. An
example of usage within Org:

#+header: :headers '("\\usepackage{unicodefonttable}")
#+begin_src latex :imagemagick yes :iminoptions -density 600 :results raw :results file :file -2256080143431736233.png
\displayfonttable*[range-start=1F00,range-end=1FFE]{Old Standard}
\displayfonttable*[range-start=0600,range-end=06FF]{FreeSerif}
#+end_src

A screenshot:

https://i.imgur.com/Fwsg7bb.png

Maybe it could also be added as an emergency fallback font GNU Unifont:

https://unifoundry.com/

Best regards,

Juan Manuel 

Re: [possible patch] Basic fontspec code for LuaLaTeX and XelaTeX (was "LaTeX export: when is it more useful...")

[Juan Manuel Macías]

Juan Manuel Macías writes:

> By the way, although I've already commented on it in some post in the
> parent thread, i think this package I wrote might be useful for doing a
> quick visual test of a font (including opentype features test), using
> org-latex-preview (compiling with LuaTeX). It can be done on any font
> marked in dired. There are three options: insert arbitrary characters,
> insert the Unicode code of the characters, or display a specimen of the
> font. The default specimen is in the file specimen.tex, which can be
> edited to add examples and languages.
>
> Some screenshots:
>
> https://i.imgur.com/3faKSjA.png
>
> https://i.imgur.com/OJfUcO9.png

Sorry, I forgot the link:

https://gitlab.com/maciaschain/org-font-spec-preview

Re: [possible patch] Basic fontspec code for LuaLaTeX and XelaTeX (was "LaTeX export: when is it more useful...")

[Max Nikulin]

On 11/07/2022 03:23, Juan Manuel Macías wrote:
> 
> 3. A variable (something like 'org-latex-fontspec-default-configuration') would return something like this:
> 
> (format
>   \\usepackage{iftex}
>   \\ifpdftex

I like the idea of unified preamble suitable for pdflatex and lualatex. 
For pdftex \usepackage[...]{fontenc} line may be added here.

>   \\relax
>   \\else

Is it the case of latex as the old engine with tex->dvi->ps workflow 
besides new XeTeX and LuaTeX? However such engine is not used by Org.

>   \\usepackage{fontspec}
>   \\usepackage{unicode-math}
>   \\defaultfontfeatures{Scale=MatchLowercase}
>   \\defaultfontfeatures[\\rmfamily]{Ligatures=TeX}
>   \\setmainfont{%s}
>   \\setsansfont{%s}
>   \\setmonofont{%s}
>   \\fi
>   org-latex-fontspec-mainfont
>   org-latex-fontspec-sansfont
>   org-latex-fontspec-monofont)

Too many variables to my taste. It can be single property list. If I 
remember correctly, changing of mainfont requires setting of a 
consistent font for mathematics, so more options may be required.

ox-latex has some code searching for e.g. \usepackage[...]{inputenc} in 
the current configuration to avoid adding of extra copy. It would be 
great to have something similar for fontspec commands. I guess, it is 
harder to detect, since per-language font configuration may be required 
as babel options.

Finally, default value may be language-dependent or alternative font set 
may be activated when non-latin characters are detected in the document.

Re: [possible patch] Basic fontspec code for LuaLaTeX and XelaTeX (was "LaTeX export: when is it more useful...")

[Juan Manuel Macías]

Tim Cross writes:

> Juan, if I understand your proposal correctly, I think your on the right
> track. It sounds like what you are proposing would have almost no impact
> on basic users like me, but would allow those with more demanding
> requirements to adjust without too much effort. I originally raised the
> question regarding what would need to change with the switch to uatex to
> ensure that we do actually get things positioned to enable people to
> exploit the benefits and not just switch out one tool for another which
> only appears to be a little slower. I think what you are suggesting
> addresses that concern. 

Tim, thanks a lot for your interesting comments.

Indeed, I think that LuaTeX is a good direction for the TeX ecosystem.
And it seems that the third edition of The LaTeX Companion makes the way
clear:

https://tex.stackexchange.com/questions/612573/the-latex-companion-3rd-edition/612586

Of course, LuaTeX is still a kind of cyborg (someone defined it that
funny way :-). TeX has not been rewritten here from scratch (that would
have been preferable), but LuaTeX has brought, in my opinion, two
revolutionary things: being able to control TeX internals from a
scripting language as light and minimalist as Lua (which drastically
influences the creation of packages every increasingly powerful and
sophisticated for all areas) and the fact that TeX is finally native
unicode. From the latter, of course, follows the fact that the user is
no longer dependent on Computer Modern and can choose whatever font he
wants, just like in any other modern textual software, from a simple
word processor to more advanced tools like InDesign-style dtp programs.

Even though pdfTeX was light years ahead of InDesign, this simple
operation of choosing the font or font family has always been horribly
difficult in LaTeX. There were a few packages that incorporated specific
font families (Times, Fourier, etc.), but if one wanted to manually
install Adobe Garamond in pdfTeX ---for example---, the process became
absurdly long and cumbersome. Now in LuaLaTeX and XelaTeX that is as
simple as doing it in libreoffice.

Of course, TeX and LaTeX have always had their historical typeface,
Computer Modern, which is almost like one of their distinguishing
features. This extreme reliance on Computer Modern has often given
people who don't use LaTeX the misconception that any document made in
LaTeX always looks the same.

Despite the fact that I feel enormous admiration for Donald Knuth, and I
believe that to a greater or lesser extent many or almost all of us are
indebted to him, I believe that the design of Computer Modern is not
good and has many legibility problems (imho), especially legibility
screen (precisely because of its Didot-style design, with such a marked
contrast between the strokes). Since there is a thread on this list
about accessibility, it's worth remembering that Computer Modern isn't
exactly an easy-to-read font. Of course, you have to put things in their
historical context. When TeX was created there was nothing similar to
what we have today in fonts, there was no truetype or opentype, there
were no free fonts either. It was all to do. And, naturally, if one
creates "a new typesetting system intended for the creation of beautiful
books" (Texbook page 5, Preface), it would be somewhat strange if this
new typesetting system were born without a typeface to show the world
the excellence of TeX. For that reason Knuth created Metafont and the
Computer Modern font.

Now with LuaTeX and XeTeX choosing the font, any font, is easy, fast and
trivial.

> but as I said, I know nothing....

I don't think so. Knowing (or not knowing) things or facts (after all, all
of this is just "data") is not the same as being wise and speaking
wisely :-)

Best regards,

Juan Manuel 

Re: [possible patch] Basic fontspec code for LuaLaTeX and XelaTeX (was "LaTeX export: when is it more useful...")

[Timothy]

[-- Attachment #1: Type: text/plain, Size: 1786 bytes --]

Hi Ihor & co.,

> As I recall, Timothy has been working on simplifying preamble
> generation. If we do not put unnecessary packages into preamble,
> compilation will be significantly faster.
>
> If Timothy can come up with a patch some time soon, I’d prefer to have a
> more targeted preamble. Otherwise, the proposed approach is the way to
> go.

Yep, I’ve got something in my config currently that intercepts LaTeX preamble
creation and generates it only the fly from a list of detected features based on
the exported document and capability providers. I use this in my config to
automatically switch to LuaLaTeX when necessary and use pdfLaTeX the rest of the
time.

This should also be able to be able clean up some of the currently kludgy
preamble modifications like in oc-csl.el.

This has been on the back-burner for a while (I want to implement this is a way
that can be generalised across all output backends), but I’ll see if I can make
some progress and hopefully have a preliminary patch set in the next few weeks.

Lastly, there’s something extra I want to note. If we talk about including a
font customisation, I’d advocate for supporting font sets, not fonts. Once
again, this is something I’m a fan of from my config, and this could potentially
be supported across multiple export formats.

As an illustrative example, if I include this in one of my documents and create
a PDF:
┌────
│ #+options: fontset:biolinum
└────

Then I’ll get text with:
⁃ libertine roman as the serif font
⁃ biolinum as the serif, and default, font
⁃ source code pro as the mono font
⁃ newtxmath as the maths font

Similarly I can do `fontset:noto' and you can guess what that does.

All the best,
Timothy

Re: [possible patch] Basic fontspec code for LuaLaTeX and XelaTeX (was "LaTeX export: when is it more useful...")

[Juan Manuel Macías]

Max Nikulin writes:

>>   \\relax
>>   \\else
>
> Is it the case of latex as the old engine with tex->dvi->ps workflow
> besides new XeTeX and LuaTeX? However such engine is not used by Org.

According to the iftex documentation (p. 2):

\ifpdftex, \ifPDFTeX
True if PDFTEX is in use (whether writing PDF or DVI), so this is true
for documents processed with both the latex and pdflatex commands.

So the code says: if pdfTeX is used, do nothing; else, add this (luatex
and xetex related) code.

>>   \\usepackage{fontspec}
>>   \\usepackage{unicode-math}
>>   \\defaultfontfeatures{Scale=MatchLowercase}
>>   \\defaultfontfeatures[\\rmfamily]{Ligatures=TeX}
>>   \\setmainfont{%s}
>>   \\setsansfont{%s}
>>   \\setmonofont{%s}
>>   \\fi
>>   org-latex-fontspec-mainfont
>>   org-latex-fontspec-sansfont
>>   org-latex-fontspec-monofont)
>
> Too many variables to my taste. It can be single property list. If I
> remember correctly, changing of mainfont requires setting of a
> consistent font for mathematics, so more options may be required.

Yes, that is true, sorry. I don't work with math. But:

\setmathrm{⟨font name⟩}
\setmathsf{⟨font name⟩}
\setmathtt{⟨font name⟩}
\setboldmathrm{⟨font name⟩}

Now, weren't we talking about ensuring a minimum readability out of the
box in case non-Latin characters are used? I assume that by default the
mathematical notation is assured, although the default mathematical font
may be typographically or aesthetically incompatible with the chosen
text fonts. For example, Computer Modern and FreeSerif are antipodes in
design. The first is a Didotian font and the second is a times style
typeface. But I think that what is sought here is that certain (non
latin) glyphs are represented in the PDF, beyond other typographical or
aesthetic considerations. My idea here is that a) the user who doesn't
want to mess with all these issues has a minimum of readability out of
the box; b) the user who wants to have full control over the fontspec
options has the possibility to do so; c) the user who does not want Org
to write the preamble under any circumstances (that is, people like me),
has the possibility of continuing doing so.

> Finally, default value may be language-dependent or alternative font
> set may be activated when non-latin characters are detected in the
> document.

If I had to choose between both options, I would prefer the second one.
But don't you think it would be much simpler to ensure the readability
of non-Latin characters (at least in a high percentage) by means of
three default fonts (roman, sans, mono), and let the user who needs
another font be able to choose it freely, simply by changing the value
of those variables? Generally, users working with a certain non-Latin
script are already used to using a certain font (I mean, they haven't
suddenly teleported into the digital world), and they know perfectly
well which fonts to use for their use case and their language. And for
those users who are a bit more lost, a list of recommended fonts can be
added to the documentation (many of which are already installed on their
system or are included with TeX live).

The other more extreme possibility is to default to GNU unifont
(https://unifoundry.com/unifont/index.html). With this font I think the
readability of almost everything is ensured (although it is a horrible
font, but it is not the case here). Or Google's Noto Fonts (but I don't
remember now what license terms those fonts are under).

Best regards,

Juan Manuel 

Re: [possible patch] Basic fontspec code for LuaLaTeX and XelaTeX (was "LaTeX export: when is it more useful...")

[Juan Manuel Macías]

Timothy writes:

> As an illustrative example, if I include this in one of my documents and create
> a PDF:
> ┌────
> │ #+options: fontset:biolinum
> └────
>
> Then I’ll get text with:
> ⁃ libertine roman as the serif font
> ⁃ biolinum as the serif, and default, font
> ⁃ source code pro as the mono font
> ⁃ newtxmath as the maths font
>
> Similarly I can do `fontset:noto' and you can guess what that does.

I think it's a very good idea to be able to add the options using
#+options:..., forgive the redundancy :-)

When you talk about fontset, I understand that you mean lists of
families with their options that you have previously defined, is that
right? 

In any case, I think it would also be nice if the user could add only
one family for roman, sans, mono or math, if he/she prefers it that way.
Something like:

#+options: rmfont:Minion Pro

Best regards,

Juan Manuel 

Re: LaTeX export: when is it more useful to use LuaTeX instead of pdfTeX?

[Max Nikulin]

On 10/07/2022 03:22, Juan Manuel Macías wrote:
> 
> LuaTeX and XeTeX are
> digital typesetting systems. They are not word processors.

I have skimmed through the discussion happened exactly a year ago
https://list.orgmode.org/orgmode/scuirf$m7o$1@ciao.gmane.io/
and I should repeat that you are too much concentrated on books and 
carefully designed camera-ready files.

LaTeX is a kind of word processor as well in the sense that it is used 
for notes that are not intended to be published. In some cases it is 
merely a tool to make readable a text heavy loaded with math. Balance of 
efforts and quality is quite different. As much as possible should be 
delegated to "word processor". Forcing users to select particular fonts 
makes documents less portable, it increases a chance that a colleague 
does not have a font installed on your machine or you get a file 
requiring a proprietary font you do not have.

For such quick notes the feature currently provided by browser, office, 
etc. is indispensable: list of implicit fallbacks to find some available 
font having glyphs missed in the primary fonts.

I do not mind that LuaLaTeX alleviated issues with configuring of fonts, 
so it is more convenient for books or decorated documents. Personally I 
was quite happy with PdfLaTeX fonts I get out of the box without 
necessity to override ≥ 6 font families. I did not use hieroglyphs or 
fancy Unicode characters, but with LuaLaTeX they anyway require setting 
of additional fonts. My current impression is that LuaLaTeX may be 
significant step forward for publishers, but for quick notes it is a 
kind of trade-off.

Re: [possible patch] Basic fontspec code for LuaLaTeX and XelaTeX (was "LaTeX export: when is it more useful...")

[Max Nikulin]

On 11/07/2022 21:23, Juan Manuel Macías wrote:
> Max Nikulin writes:
> 
> \ifpdftex, \ifPDFTeX
> True if PDFTEX is in use (whether writing PDF or DVI), so this is true
> for documents processed with both the latex and pdflatex commands.
> 
> So the code says: if pdfTeX is used, do nothing; else, add this (luatex
> and xetex related) code.

Likely it is not relevant to Org, but for the following document

\documentclass{article}
\usepackage{ifpdf}
\begin{document}
\ifpdf PDF\else Not PDF\fi
\end{document}

I get "PDF" when I use pdflatex and "Not PDF" when I run latex.

> Yes, that is true, sorry. I don't work with math. But:
> 
> \setmathrm{⟨font name⟩}
> \setmathsf{⟨font name⟩}
> \setmathtt{⟨font name⟩}
> \setboldmathrm{⟨font name⟩}
> 
> Now, weren't we talking about ensuring a minimum readability out of the
> box in case non-Latin characters are used?

Mathematical expressions may contain non-latin characters as well. 
\text{} may be a rescue (anyway such expression usually appears poorly 
formatted), but if I remember correctly, it is better to use math mode 
commands to get accurate spaces in accordance to TeX design. So math 
fonts with wider coverage is somewhere close to minimum readability.

Re: fontsets (was: [possible patch] Basic fontspec code for LuaLaTeX and XelaTeX (was "LaTeX export: when is it more useful..."))

[Timothy]

[-- Attachment #1: Type: text/plain, Size: 1432 bytes --]

Hi Juan,

> When you talk about fontset, I understand that you mean lists of
> families with their options that you have previously defined, is that
> right?

Yep, so in my config’s implementation I have an alist of fontset names and
individual fonts. For something part of org-mode itself, we’d probably want to
add a format level to this, something like:

┌────
│ ((fontset-name .
│   ((serif .
│     ((pdflatex . "\\usepackage{myserif}")
│      (lualatex . "etc.")
│      (html . "and so on")))
│    (sans ...) ... ))
│ (another-fontset ...) ...)
└────

Actually, now that I think of it maybe it would be better to seperate out the
fontsets and fots, e.g.

┌────
│ ;; Fonts
│ ((myfonta . ((pdflatex . "etc.") (lualatex ...) (html ...) ...))
│  (myfontb ...)
│  ...)
│ ;; Fontsets
│ ((myfontset .
│   ((sans . myfonta)
│    (serif . myfontb)
│    (mono . myfontc)
│    ...))
│  ...)
└────

> In any case, I think it would also be nice if the user could add only
> one family for roman, sans, mono or math, if he/she prefers it that way.
> Something like:
>
> #+options: rmfont:Minion Pro

Sure. There’s another bit of functionality in my config which I think is worth
noting, you can add a -sans/-serif/-mono suffix to the fontset name to override
the default body text font.

All the best,
Timothy

Re: fontsets

[Juan Manuel Macías]

Hi, Timothy,

Timothy writes:

> Yep, so in my config’s implementation I have an alist of fontset names and
> individual fonts. For something part of org-mode itself, we’d probably want to
> add a format level to this, something like:
>
> ┌────
> │ ((fontset-name .
> │   ((serif .
> │     ((pdflatex . "\\usepackage{myserif}")
> │      (lualatex . "etc.")
> │      (html . "and so on")))
> │    (sans ...) ... ))
> │ (another-fontset ...) ...)
> └────
>
> Actually, now that I think of it maybe it would be better to seperate out the
> fontsets and fots, e.g.
>
> ┌────
> │ ;; Fonts
> │ ((myfonta . ((pdflatex . "etc.") (lualatex ...) (html ...) ...))
> │  (myfontb ...)
> │  ...)
> │ ;; Fontsets
> │ ((myfontset .
> │   ((sans . myfonta)
> │    (serif . myfontb)
> │    (mono . myfontc)
> │    ...))
> │  ...)
> └────
>
>> In any case, I think it would also be nice if the user could add only
>> one family for roman, sans, mono or math, if he/she prefers it that way.
>> Something like:
>>
>> #+options: rmfont:Minion Pro
>
> Sure. There’s another bit of functionality in my config which I think is worth
> noting, you can add a -sans/-serif/-mono suffix to the fontset name to override
> the default body text font.

I see. I like your approach. And the idea of fontsets also seems very
productive. I suppose that a minimum configuration in fontspec
(Scale=MatchLowercase) should be ensured, in order to balance the
x-height when using fonts from different families in a single document[1].

The fact that all of this can also be "reusable" for other outputs such as
html, is a not insignificant plus! I definitely really like all of these
ideas and I don't think there is any contradiction with a balance
between those users who are content with minimal out-of-the-box font
configuration to be able to read non-latin characters, and those who
want more control over fontspec (font features, etc). And there are also
the users (me among them) who leave little or almost no space for Org to
write the preamble for us :-)

On the other hand, maybe (I think) it would be nice not to differentiate
between xelatex and lualatex, since at least at this level both engines
support the same fontspec settings.

[1] I have to add, by the way, that MatchLowercase is not always a
panacea. In many cases, and depending on the fonts, it may be better to
allow some contrast between families. Maybe it would be nice to add to
the documentation or to worg (at least for users who may be interested
in these topics) some basic recommendations for combining families (for
example, combining a bodoni with a bembo is usually a catastrophic
marriage :-).

Best regards,

Juan Manuel

Re: fontsets

[Stefan Nobis]

Juan Manuel Macías <maciaschain@posteo.net> writes:

> [1] I have to add, by the way, that MatchLowercase is not always a
> panacea.

Hmmm... maybe add optional extra config/output option to the fontset,
like so:

┌────
│ ;; Fonts
│ ((myfonta . ((pdflatex . "etc.") (lualatex ...) (html ...) ...))
│  (myfontb ...)
│  ...)
│ ;; Fontsets
│ ((myfontset .
│   ((sans . myfonta)
│    (serif . myfontb)
│    (mono . myfontc)
│    (extra . ((lualatex . "\\defaultfontfeatures{Scale=MatchLowercase}")
│              (html "some CSS...")...))
│    ...))
│  ...)
└────

This way it may be possible to build a fontset library of fonts that
blend well together, including some necessary fine-tuning.

-- 
Until the next mail...,
Stefan.

Re: fontsets

[Juan Manuel Macías]

Stefan Nobis writes:

> Hmmm... maybe add optional extra config/output option to the fontset,
> like so:
>
> ┌────
> │ ;; Fonts
> │ ((myfonta . ((pdflatex . "etc.") (lualatex ...) (html ...) ...))
> │  (myfontb ...)
> │  ...)
> │ ;; Fontsets
> │ ((myfontset .
> │   ((sans . myfonta)
> │    (serif . myfontb)
> │    (mono . myfontc)
> │    (extra . ((lualatex . "\\defaultfontfeatures{Scale=MatchLowercase}")
> │              (html "some CSS...")...))
> │    ...))
> │  ...)
> └────
>
> This way it may be possible to build a fontset library of fonts that
> blend well together, including some necessary fine-tuning.

I think it's a good idea. And I definitely like Timothy's idea of
fontsets, but I still think that LuaLaTeX and XelaTeX should be unified
in some way.

Perhaps one or two default fontsets could be added.

It was commented in some previous message about the possibility of
checking if a font is present in the system. To add some extra
information, TeX live 2022 includes a lua script, luafindfont, which
runs luaotfload in the background. It is very useful because, in
addition to system fonts, it also returns results from TeX live fonts. I
use this script via helm, and I wrote this to extract a list of
candidates:

#+begin_src emacs-lisp
(defun build-luafindfont-candidates-list (candidate)
  (interactive)
  (let* ((query (shell-command-to-string (concat "luafindfont " candidate)))
	 (str (with-temp-buffer
		(insert query)
		(goto-char (point-min))
		(let ((from
		       (save-excursion
			 (re-search-forward "1\\." nil t)
			 (beginning-of-line)
			 (point)))
		      (to
		       (save-excursion
			 (goto-char (point-max))
			 (point))))
		  (save-restriction
		    (narrow-to-region from to)
		    (buffer-string)))))
	 (str-clean (split-string
		      (with-temp-buffer
			(insert str)
			(replace "[[:digit:]]+\\.\s+" "")
			(replace "\\(.+\\)\\(\\.otf\\|\\.ttf\\)\s+" "")
			(replace "\s+" " -- ")
			(buffer-string)) "\n" t)))
    (setq luafindfont-list (mapcar 'identity
					   str-clean))))
#+end_src

On the other hand, fontspec includes the \IfFontExistsTF command.
According to the fontspec documentation:

------
\IfFontExistsTF{⟨font name⟩}{⟨true branch⟩}{⟨false branch⟩}

The conditional \IfFontExistsTF is provided to test whether the ⟨font name⟩ exists or
is loadable. If it is, the ⟨true branch⟩ code is executed; otherwise, the ⟨false branch⟩ code is.
This command can be slow since the engine may resort to scanning the filesystem for a
missing font. Nonetheless, it has been a popular request for users who wish to define ‘fallback
fonts’ for their documents for greater portability.

In this command, the syntax for the ⟨font name⟩ is a restricted/simplified version of the
font loading syntax used for \fontspec and so on. Fonts to be loaded by filename are detected
by the presence of an appropriate extension (.otf, etc.), and paths should be included inline.

E.g.:
\IfFontExistsTF{cmr10}{T}{F}
\IfFontExistsTF{Times New Roman}{T}{F}
\IfFontExistsTF{texgyrepagella-regular.otf}{T}{F}
\IfFontExistsTF{/Users/will/Library/Fonts/CODE2000.TTF}{T}{F}
-------

Best regards,

Juan Manuel 

Fallback fonts in LuaTeX via 'luaotfload.add_fallback' (was "Fontsets")

[Juan Manuel Macías]

Today I discovered that luaotfload included in v. 3.12 a new
experimental function, luaotfload.add_fallback, to be able to add a list
of fallback fonts to a LuaTeX document, at a low level.

(More info on page 18 of the luaotfload manual, with some examples).

I've been experimenting a bit with this function. For example:

#+begin_src latex
  \documentclass{article}
  \usepackage{fontspec}
  \directlua{
  luaotfload.add_fallback ("fallbacktest",
  {
  "oldstandard:mode=harf;script=grek;color=0000FF;",
  "oldstandard:mode=harf;script=cyrl;color=0000FF;",
  "freeserif:mode=harf;script=arab;color=0000FF;",
  "freeserif:mode=harf;script=dev2;color=0000FF;",
  }
  )
  }

  \setmainfont{latinmodernroman}[RawFeature={fallback=fallbacktest}]

  \begin{document}

  Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Donec
  hendrerit tempor tellus. Donec pretium posuere tellus.

  Δαρείου καὶ Παρυσάτιδος γίγνονται παῖδες δύο, πρεσβύτερος μὲν
  Ἀρταξέρξης, νεώτερος δὲ Κῦρος· ἐπεὶ δὲ ἠσθένει Δαρεῖος καὶ ὑπώπτευε
  τελευτὴν τοῦ βίου, ἐβούλετο τὼ παῖδε ἀμφοτέρω παρεῖναι. ὁ μὲν οὖν
  πρεσβύτερος παρὼν ἐτύγχανε· Κῦρον δὲ μεταπέμπεται ἀπὸ τῆς ἀρχῆς ἧς
  αὐτὸν σατράπην ἐποίησε, καὶ στρατηγὸν δὲ αὐτὸν ἀπέδειξε πάντων ὅσοι ἐς
  Καστωλοῦ πεδίον ἁθροίζονται.

  Emacs написан на двух языках: C и Emacs Lisp (Elisp, диалект Лиспа).
  При этом сам редактор является интерпретатором Elisp. По сути дела,
  большая часть Emacs написана на языке Elisp, и её можно рассматривать
  как расширение к основной программе. Пользователи могут сами создавать
  части Emacs, от отдельных функций до новых основных режимов. При этом
  можно переопределять любые Elisp-функции, в том числе и те, что
  являются частью самого редактора, и модифицировать функциональность
  Emacs, изменив соответствующим образом некоторые функции.

  Native speakers of Arabic generally do not distinguish between Modern
  Standard Arabic and Classical Arabic as separate languages; they refer
  to both as al-ʻArabīyah al-Fuṣḥá (العربية الفصحى) meaning the pure
  Arabic. They consider the two forms to be two registers of one
  language. When the distinction is made, they are referred to as فصحى
  العصر Fuṣḥá al-ʻAṣr (MSA) and فصحى التراث Fuṣḥá al-Turāth (CA)
  respectively.

  उदु ज्योतिरमृतं विश्वजन्यं विश्वानरः सविता देवो अश्रेत् ।\\
  क्रत्वा देवानामजनिष्ट चक्षुराविरकर्भुवनं विश्वमुषाः ॥ १ ॥\\
  प्र मे पन्था देवयाना अदृश्रन्नमर्धन्तो वसुभिरिष्कृतासः ।\\
  अभूदु केतुरुषसः पुरस्तात्प्रतीच्यागादधि हर्म्येभ्यः ॥ २ ॥

  \end{document}
#+end_src

The main drawback I've found to this (at least I don't know how to solve
it at the moment) is that the fallback feature cannot be added via
\defaultfontfeatures. That would avoid having to (re)define all the
main/sans/mono/math families.

Best regards,

Juan Manuel 

Re: Fallback fonts in LuaTeX via 'luaotfload.add_fallback' (was "Fontsets")

[Max Nikulin]

On 12/07/2022 22:26, Juan Manuel Macías wrote:
> Today I discovered that luaotfload included in v. 3.12 a new
> experimental function, luaotfload.add_fallback, to be able to add a list
> of fallback fonts to a LuaTeX document, at a low level.
...
>    \documentclass{article}
>    \usepackage{fontspec}
>    \directlua{
>    luaotfload.add_fallback ("fallbacktest",{
>    "oldstandard:mode=harf;script=grek;color=0000FF;",
>    "oldstandard:mode=harf;script=cyrl;color=0000FF;",
>    "freeserif:mode=harf;script=arab;color=0000FF;",
>    "freeserif:mode=harf;script=dev2;color=0000FF;",
>    })}
>    \setmainfont{latinmodernroman}[RawFeature={fallback=fallbacktest}]
...
> The main drawback I've found to this (at least I don't know how to solve
> it at the moment) is that the fallback feature cannot be added via
> \defaultfontfeatures. That would avoid having to (re)define all the
> main/sans/mono/math families.

I agree that defining fallbacks for each font family is inconvenient. 
Defining font per script resembles specifying fonts per language in 
babel configuration, however fallback should work without explicit 
switching of language. I have seen that babel may determine language 
from character code points, but I have not tried if it works reliable 
and if it affects performance (as it does for fallback fonts).

Maybe I did not read the manual with enough attention, maybe I tried it 
with too old version of LuaTeX, but I had a problem with Emoji. 
Depending on font such symbols either broke compilation or did not 
appear in PDF (accordingly to pdffonts font was embedded, text may be 
copied, but PDF viewers displayed blank space).

https://list.orgmode.org/orgmode/scuirf$m7o$1@ciao.gmane.io/
Maxim Nikulin. Re: org-mode export to (latex) PDF. Sat, 17 Jul 2021 
19:35:57 +0700

Re: [possible patch] Basic fontspec code for LuaLaTeX and XelaTeX (was "LaTeX export: when is it more useful...")

[Max Nikulin]

On 11/07/2022 21:23, Juan Manuel Macías wrote:
> Max Nikulin writes:
>>>  \\ifpdftex
>>>    \\relax
>>>    \\else
>>
>> Is it the case of latex as the old engine with tex->dvi->ps workflow
>> besides new XeTeX and LuaTeX? However such engine is not used by Org.
> 
> According to the iftex documentation (p. 2):
> 
> \ifpdftex, \ifPDFTeX
> True if PDFTEX is in use (whether writing PDF or DVI), so this is true
> for documents processed with both the latex and pdflatex commands.
> 
> So the code says: if pdfTeX is used, do nothing; else, add this (luatex
> and xetex related) code.

I have noticed the \iftutex command in the iftex.sty manual. It detects 
XeTeX, LuaTeX, LuaHBTeX, so it should be more suitable here.

At first I had intention to suggest \ifdefined\XeTeXrevision 
\ifdefined\XeTeXrevision you mentioned in
Re [PATCH] ox-latex.el: Unify in one single list Babel and Polyglossia 
languages alists. Fri, 15 Jul 2022 14:36:07 +0000. 
https://list.orgmode.org/878rou30ko.fsf@posteo.net

P.S. I do not remember if CMU Serif, etc. family (that is Computer 
Modern Unicode) has been mentioned in this thread. It is not installed 
but default, but allows to generate documents with traditional TeX fonts.