Org as a book publisher

Org as a book publisher

[Juan Manuel Macías]

Hi,

I would like to share here two samples of one of the most intense uses
that I give Org Mode: for typesetting, layout and editorial design. In
other words, I use Org (and Org-Publish) where publishers today use DTP
proprietary software like InDesign or QuarkXpress (a type of software,
on the other hand, that was never intended to compose books but rather
magazines, posters, brochures and so on). The samples are from a book
on classical philology, recently published here in Spain, and from a
fairly extensive dictionary, still work in progress:

https://imgur.com/gallery/yxAVkrY

Naturally, what acts in the background here is TeX and LaTeX
(specifically Lua(La)TeX), so what I really do is use Org and
Org-Publish as a sort of high-level interface for LaTeX. But I don't
mean to avoid LaTeX: in fact, I've been working with LaTeX for a long
time. I like LaTeX and behind these jobs there is a lot of LaTeX code.
But Org gives me a much more light and productive workflow, allowing me
to work at two levels.

The main advantages that I see for this workflow with Org/Org-Publish
are:

1. Lightness: LaTeX is too verbose.
2. Control of the composition process at various points. One of the
   qualities of LuaTeX is the possibility to control TeX primitives
   through scripts in Lua, and to act at various points in the pre- or
   post-process. But I have realized that with the happy fusion of Elisp
   and Org we can be much more precise and "surgical" ;-). Here,
   Org/LaTeX is much more powerful than LuaLaTeX.
3. Org's synaptic and org-anizational ability to control and manage the
   entire process of the creation of a book, from when the originals are
   received until everything is prepared to send to the printer.
4. An unique origin. The book can be produced on paper from a single
   source, but you can also export, from that source consistently, to
   other formats (HTML or Epub).

Best regards,

Juan Manuel 

Re: Org as a book publisher

[M. ‘quintus’ Gülker]

Am 06. März 2021 um 20:34 Uhr +0100 schrieb Juan Manuel Macías:
> I would like to share here two samples of one of the most intense
> uses that I give Org Mode: for typesetting, layout and editorial
> design. [...] The samples are from a book on classical philology,
> recently published here in Spain [...]. Naturally, what acts in the
> background here is TeX and LaTeX (specifically Lua(La)TeX), so what
> I really do is use Org and Org-Publish as a sort of high-level
> interface for LaTeX. But I don't mean to avoid LaTeX: in fact, I've
> been working with LaTeX for a long time. I like LaTeX and behind
> these jobs there is a lot of LaTeX code. But Org gives me a much
> more light and productive workflow, allowing me to work at two
> levels.

Thank you very much for sharing. This is an interesting insight. Many
people seem to use org rather than direct LaTeX because they dislike
LaTeX's syntax or find LaTeX too complex, which I never really
understood. But you make some great points for why this combination is
useful other than for that reason.

> 2. Control of the composition process at various points. One of the
>    qualities of LuaTeX is the possibility to control TeX primitives
>    through scripts in Lua, and to act at various points in the pre- or
>    post-process. But I have realized that with the happy fusion of Elisp
>    and Org we can be much more precise and "surgical" ;-). Here,
>    Org/LaTeX is much more powerful than LuaLaTeX.

For those who still use pdfLaTeX rather than LuaLaTeX (probably due to
Microtype) there is not even an equivalent available.

> 4. An unique origin. The book can be produced on paper from a single
>    source, but you can also export, from that source consistently, to
>    other formats (HTML or Epub).

This is actually a strong argument. Even though I enjoy writing LaTeX
code, this one is a tough nut to crack with pure LaTeX, where I
achieved the best results with LaTeXML, but it was still lots of work.
May I ask what tooling you use to go from org to Epub?

  -quintus

-- 
Dipl.-Jur. M. Gülker | https://mg.guelker.eu |    For security:
Passau, Germany      | kontakt@guelker.eu    | () Avoid HTML e-mail
European Union       | PGP: see homepage     | /\ http://asciiribbon.org

Re: Org as a book publisher

[Juan Manuel Macías]

Hi Quintus:

Thank you very much for your comments.

M. ‘quintus’ Gülker <post+orgmodeml@guelker.eu> writes:

> [...] Many people seem to use org rather than direct LaTeX because they
> dislike LaTeX's syntax or find LaTeX too complex, which I never really
> understood. But you make some great points for why this combination is
> useful other than for that reason.

I think Org does a good job with what it offers out of the box for
`simple' documents, when the user also does not have a special interest
in LaTeX code and typographic `refinement'. But as the document becomes
more and more complex, there is a duty to study LaTeX and learn to write
code in LaTeX (even in pure TeX, if things get too demanding); and also
to study the documentation of the packages involved, because the great
power of LaTeX lies in its rich and extensive package ecosystem (two or
three new packages always appear every week!). If Org was a complete
translation of everything that each LaTeX package is capable of, then
Org would become as (needlessly) complex as LaTeX. The good thing about
Org is that it provides us with the tools needed to work with our LaTeX
code in the background. A simple example that comes to me now are the
lists, which in LaTeX can be managed in a very clean way with packages
like enumitem. In an Org document it would then be enough to put before
a list `#+ATTR_LaTeX: :options [enumitem options]' (or to write a
replacement macro), but of course the user have to read the enumitem
documentation...

(By the way, it seems that a fellow 'co-lister' is working on an Org to
Context exporter: https://github.com/Jason-S-Ross/ox-context/)

> For those who still use pdfLaTeX rather than LuaLaTeX (probably due to
> Microtype) there is not even an equivalent available.

Well, Microtype can already be used in LuaLaTeX. Except for a few minor
limitations, protrusion and expansion properties work well. In fact I
recently wrote a custom table of protrusion values for a font, and
microtype reads them perfectly in LuaLaTeX. In general nowadays the
migration from pdfTeX to LuaTeX is quite smooth, and it is worth it, for
various reasons: for example, there are a few cool new packages that
take advantage of LuaTeX. And there is also the LuaTeX ability to use
otf fonts and manage opentypes features. XeTeX can do that too, but
with LuaTeX we can even define new otf features on the fly, in Lua, for
a document (like kerning, contextual substitution, etc.) and apply them
to a certain font, without need to edit that font with a dedicated
software (fontforge).

But yeah, for pre-/post-process control I prefer Elisp/Org a thousand
times over Lua :-) Recently I needed to modify certain combining
diacritical marks only in the italics, and with Org it's a delight to do
that (writing a filter for org-export-filter-italic-functions)

> [...] May I ask what tooling you use to go from org to Epub?

Ox-epub works reasonably well, in my short experience: I'm afraid I
haven't explored the Epub output much, partly because this is a format
that I do not like, and I have used it for editorial requirements only.
And, in fact, it is a very limited format for certain types of books.

Best regards,

Juan Manuel 

Re: Org as a book publisher

[Diego Zamboni]

[-- Attachment #1: Type: text/plain, Size: 2962 bytes --]

Hi Juan Manuel,

Thanks for sharing this - the output looks very nice.

I think with Org and a setup like you describe, we are one step closer to
separating content (what) from form (how) in a document. This was one of
the original goals of LaTeX, but of course in a LaTeX document much of the
"how" is still visible through the "what". With Org the separation becomes
clearer, by hiding the LaTeX structures (almost) completely, and by
allowing to produce multiple formats from the same source document.

I have done something similar with my books, which I publish through
Leanpub. I keep the source of each book in Org, and from there the exporter
takes care of producing the Leanpub markup and format, which in turn takes
care of converting it to PDF, ePub or other formats.

Best,
--Diego



On Sat, Mar 6, 2021 at 8:35 PM Juan Manuel Macías <maciaschain@posteo.net>
wrote:

> Hi,
>
> I would like to share here two samples of one of the most intense uses
> that I give Org Mode: for typesetting, layout and editorial design. In
> other words, I use Org (and Org-Publish) where publishers today use DTP
> proprietary software like InDesign or QuarkXpress (a type of software,
> on the other hand, that was never intended to compose books but rather
> magazines, posters, brochures and so on). The samples are from a book
> on classical philology, recently published here in Spain, and from a
> fairly extensive dictionary, still work in progress:
>
> https://imgur.com/gallery/yxAVkrY
>
> Naturally, what acts in the background here is TeX and LaTeX
> (specifically Lua(La)TeX), so what I really do is use Org and
> Org-Publish as a sort of high-level interface for LaTeX. But I don't
> mean to avoid LaTeX: in fact, I've been working with LaTeX for a long
> time. I like LaTeX and behind these jobs there is a lot of LaTeX code.
> But Org gives me a much more light and productive workflow, allowing me
> to work at two levels.
>
> The main advantages that I see for this workflow with Org/Org-Publish
> are:
>
> 1. Lightness: LaTeX is too verbose.
> 2. Control of the composition process at various points. One of the
>    qualities of LuaTeX is the possibility to control TeX primitives
>    through scripts in Lua, and to act at various points in the pre- or
>    post-process. But I have realized that with the happy fusion of Elisp
>    and Org we can be much more precise and "surgical" ;-). Here,
>    Org/LaTeX is much more powerful than LuaLaTeX.
> 3. Org's synaptic and org-anizational ability to control and manage the
>    entire process of the creation of a book, from when the originals are
>    received until everything is prepared to send to the printer.
> 4. An unique origin. The book can be produced on paper from a single
>    source, but you can also export, from that source consistently, to
>    other formats (HTML or Epub).
>
> Best regards,
>
> Juan Manuel
>
>

[-- Attachment #2: Type: text/html, Size: 3585 bytes --]

Re: Org as a book publisher

[Vikas Rawal]

A few years ago, I had produced this book entirely on orgmode: https://cup.columbia.edu/book/ending-malnutrition/9789382381648. The source files of the book are here: https://github.com/vikasrawal/endingmalnutrition.

This was some years back, and there has been some change in the org mode syntax since then and the files would need some fixing before they could be compiled with the current versions of orgmode.

Vikas


On Sun, Mar 07, 2021 at 01:08:20PM +0100, Diego Zamboni wrote:
> Hi Juan Manuel,
>
> Thanks for sharing this - the output looks very nice.
>
> I think with Org and a setup like you describe, we are one step closer to
> separating content (what) from form (how) in a document. This was one of
> the original goals of LaTeX, but of course in a LaTeX document much of the
> "how" is still visible through the "what". With Org the separation becomes
> clearer, by hiding the LaTeX structures (almost) completely, and by
> allowing to produce multiple formats from the same source document.
>
> I have done something similar with my books, which I publish through
> Leanpub. I keep the source of each book in Org, and from there the exporter
> takes care of producing the Leanpub markup and format, which in turn takes
> care of converting it to PDF, ePub or other formats.
>
> Best,
> --Diego
>
>
>
> On Sat, Mar 6, 2021 at 8:35 PM Juan Manuel Macías <maciaschain@posteo.net>
> wrote:
>
> > Hi,
> >
> > I would like to share here two samples of one of the most intense uses
> > that I give Org Mode: for typesetting, layout and editorial design. In
> > other words, I use Org (and Org-Publish) where publishers today use DTP
> > proprietary software like InDesign or QuarkXpress (a type of software,
> > on the other hand, that was never intended to compose books but rather
> > magazines, posters, brochures and so on). The samples are from a book
> > on classical philology, recently published here in Spain, and from a
> > fairly extensive dictionary, still work in progress:
> >
> > https://imgur.com/gallery/yxAVkrY
> >
> > Naturally, what acts in the background here is TeX and LaTeX
> > (specifically Lua(La)TeX), so what I really do is use Org and
> > Org-Publish as a sort of high-level interface for LaTeX. But I don't
> > mean to avoid LaTeX: in fact, I've been working with LaTeX for a long
> > time. I like LaTeX and behind these jobs there is a lot of LaTeX code.
> > But Org gives me a much more light and productive workflow, allowing me
> > to work at two levels.
> >
> > The main advantages that I see for this workflow with Org/Org-Publish
> > are:
> >
> > 1. Lightness: LaTeX is too verbose.
> > 2. Control of the composition process at various points. One of the
> >    qualities of LuaTeX is the possibility to control TeX primitives
> >    through scripts in Lua, and to act at various points in the pre- or
> >    post-process. But I have realized that with the happy fusion of Elisp
> >    and Org we can be much more precise and "surgical" ;-). Here,
> >    Org/LaTeX is much more powerful than LuaLaTeX.
> > 3. Org's synaptic and org-anizational ability to control and manage the
> >    entire process of the creation of a book, from when the originals are
> >    received until everything is prepared to send to the printer.
> > 4. An unique origin. The book can be produced on paper from a single
> >    source, but you can also export, from that source consistently, to
> >    other formats (HTML or Epub).
> >
> > Best regards,
> >
> > Juan Manuel
> >
> >

Re: Org as a book publisher

[Colin Baxter]

>>>>> Vikas Rawal <vikasrawal@gmail.com> writes:

    > A few years ago, I had produced this book entirely on orgmode:
    > https://cup.columbia.edu/book/ending-malnutrition/9789382381648. The
    > source files of the book are here:
    > https://github.com/vikasrawal/endingmalnutrition.

    > This was some years back, and there has been some change in the
    > org mode syntax since then and the files would need some fixing
    > before they could be compiled with the current versions of
    > orgmode.

That's the problem with org-mode. I have LaTeX and TeX files from 30+
years ago and they still compile.

Re: Org as a book publisher

[Juan Manuel Macías]

Hi Diego,

Thank you very much for your comments.

Diego Zamboni <diego@zzamboni.org> writes:

> I think with Org and a setup like you describe, we are one step closer
> to separating content (what) from form (how) in a document. This was
> one of the original goals of LaTeX, but of course in a LaTeX document
> much of the "how" is still visible through the "what". With Org the
> separation becomes clearer, by hiding the LaTeX structures (almost)
> completely, and by allowing to produce multiple formats from the same
> source document.

I Totally agree. Leslie Lamport originally created LaTeX (as far as I
know) as a simplified way of handling TeX for his own documents, since
the other format of TeX, plainTeX, was quite spartan. Then LaTeX has
grown incredibly thanks to its extensibility qualities: a small kernel
(unlike ConTeXt, which is more monolithic) and a ecosystem of macro
packages. If we make an analogy with the old days of mechanical
printing, I always say that TeX is the typographer, the one who gets his
hands dirty with ink, while LaTeX wears a tie and is in the editorial
design department. TeX works on the merely physical plane, and he is
only interested in how each element on the page is positioned in
relation to other elements. Here the minimum indivisible element would
be the letter, which to TeX's eyes is a box with certain dimensions.
LaTeX lives more on a semantic plane: for LaTeX there is no lines or
letters or paragraphs, but rather headings, heading levels, lists,
quotes, verses, chapters, tables, equations, tables of contents... But,
as you say, in LaTeX you can still see too many gears. With Org we can
work on a lighter and cleaner document. And with a single source for
multiple formats!

Before moving to Org, I applied this `philosophy' to the markdown/pandoc
tandem. But since I migrated to Emacs/Org a few years ago, it's almost
like having superpowers :-D

Best regards,

Juan Manuel 

Re: Org as a book publisher

[Jonathan McHugh]

To build on your analogy, would Tikz be the Graphics Designer?


Jonathan

Juan Manuel Macías <maciaschain@posteo.net> writes:

> Hi Diego,
>
> Thank you very much for your comments.
>
> Diego Zamboni <diego@zzamboni.org> writes:
>
>> I think with Org and a setup like you describe, we are one step closer
>> to separating content (what) from form (how) in a document. This was
>> one of the original goals of LaTeX, but of course in a LaTeX document
>> much of the "how" is still visible through the "what". With Org the
>> separation becomes clearer, by hiding the LaTeX structures (almost)
>> completely, and by allowing to produce multiple formats from the same
>> source document.
>
> I Totally agree. Leslie Lamport originally created LaTeX (as far as I
> know) as a simplified way of handling TeX for his own documents, since
> the other format of TeX, plainTeX, was quite spartan. Then LaTeX has
> grown incredibly thanks to its extensibility qualities: a small kernel
> (unlike ConTeXt, which is more monolithic) and a ecosystem of macro
> packages. If we make an analogy with the old days of mechanical
> printing, I always say that TeX is the typographer, the one who gets his
> hands dirty with ink, while LaTeX wears a tie and is in the editorial
> design department. TeX works on the merely physical plane, and he is
> only interested in how each element on the page is positioned in
> relation to other elements. Here the minimum indivisible element would
> be the letter, which to TeX's eyes is a box with certain dimensions.
> LaTeX lives more on a semantic plane: for LaTeX there is no lines or
> letters or paragraphs, but rather headings, heading levels, lists,
> quotes, verses, chapters, tables, equations, tables of contents... But,
> as you say, in LaTeX you can still see too many gears. With Org we can
> work on a lighter and cleaner document. And with a single source for
> multiple formats!
>
> Before moving to Org, I applied this `philosophy' to the markdown/pandoc
> tandem. But since I migrated to Emacs/Org a few years ago, it's almost
> like having superpowers :-D
>
> Best regards,
>
> Juan Manuel 


-- 
Jonathan McHugh
indieterminacy@libre.brussels

Re: Org as a book publisher

[Dr. Arne Babenhauserheide]

[-- Attachment #1: Type: text/plain, Size: 3188 bytes --]

Hi Juan,

I’ve been going that route for a few years now, and I setup an autotools
pipeline with all the little tweaks and hacks I needed to make
everything work well together.

I’m using LaTeX (pdflatex), scribus, calibre and imagemagick to publish
a roleplaying book with charactersheet, 

Maybe some of it can help you. The entrypoints are the Makefile, the
setup, and the configure.ac (for the hacks):
https://hg.sr.ht/~arnebab/ews/browse/Hauptdokument/ews30/Makefile.am
https://hg.sr.ht/~arnebab/ews/browse/Hauptdokument/ews30/basesetup.tex
https://hg.sr.ht/~arnebab/ews/browse/Hauptdokument/ews30/ews30setup.tex
https://hg.sr.ht/~arnebab/ews/browse/Hauptdokument/ews30/ews30setup.el
https://hg.sr.ht/~arnebab/ews/browse/Hauptdokument/ews30/configure.ac

The main document is
https://hg.sr.ht/~arnebab/ews/browse/Hauptdokument/ews30/ews.org

I also have some derived documents that use the included tables as data.
Most complex example:
https://hg.sr.ht/~arnebab/ews/browse/Hauptdokument/ews30/chargen.org.in


Best wishes,
Arne

> Hi,
>
> I would like to share here two samples of one of the most intense uses
> that I give Org Mode: for typesetting, layout and editorial design. In
> other words, I use Org (and Org-Publish) where publishers today use DTP
> proprietary software like InDesign or QuarkXpress (a type of software,
> on the other hand, that was never intended to compose books but rather
> magazines, posters, brochures and so on). The samples are from a book
> on classical philology, recently published here in Spain, and from a
> fairly extensive dictionary, still work in progress:
>
> https://imgur.com/gallery/yxAVkrY
>
> Naturally, what acts in the background here is TeX and LaTeX
> (specifically Lua(La)TeX), so what I really do is use Org and
> Org-Publish as a sort of high-level interface for LaTeX. But I don't
> mean to avoid LaTeX: in fact, I've been working with LaTeX for a long
> time. I like LaTeX and behind these jobs there is a lot of LaTeX code.
> But Org gives me a much more light and productive workflow, allowing me
> to work at two levels.
>
> The main advantages that I see for this workflow with Org/Org-Publish
> are:
>
> 1. Lightness: LaTeX is too verbose.
> 2. Control of the composition process at various points. One of the
>    qualities of LuaTeX is the possibility to control TeX primitives
>    through scripts in Lua, and to act at various points in the pre- or
>    post-process. But I have realized that with the happy fusion of Elisp
>    and Org we can be much more precise and "surgical" ;-). Here,
>    Org/LaTeX is much more powerful than LuaLaTeX.
> 3. Org's synaptic and org-anizational ability to control and manage the
>    entire process of the creation of a book, from when the originals are
>    received until everything is prepared to send to the printer.
> 4. An unique origin. The book can be produced on paper from a single
>    source, but you can also export, from that source consistently, to
>    other formats (HTML or Epub).
>
> Best regards,
>
> Juan Manuel 


-- 
Unpolitisch sein
heißt politisch sein
ohne es zu merken

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 1125 bytes --]

Re: Org as a book publisher

[Juan Manuel Macías]

Hi Arne,

Thank you very much for sharing the code of your book. It seems very
interesting, I have to take a closer look at it. I want to upload to
GitLab all the code of those two books of my samples, but I need to
rearrange it before, as most of that code is in Spanish :-)

But, broadly speaking, my workflow (especially in these books that are
so long) consists of using Org Publish and compiling everything at the
end with the latexmk script. Each part of the book (chapters, or letters
in the case of the dictionary) are Org documents. And then I have
another Org document which works as a master document (which is the one
I compile in the end with latexmk; the rest of the documents are
exported to * .tex using org-publish, and I automate all that process
through a function in Elisp). I have also another Org document just for
the preamble and my LaTeX code (which is tangled to a tex file), another
document only with the Elisp code involved in the process of the books
(export filters) and, finally, a .setup file. I also add a * .xdy file,
since I use xindy for the index, instead of makeindex. The xdy file is
in Common Lisp, and unfortunately I don't have much knowledge of CL,
but I manage for a few adjustments.

Of course, some Emacs packages are very useful to me too, like the
excellent Org-Ref, Magit or Projectile.

Best regards,

Juan Manuel 

"Dr. Arne Babenhauserheide" <arne_bab@web.de> writes:

> Hi Juan,
>
> I’ve been going that route for a few years now, and I setup an autotools
> pipeline with all the little tweaks and hacks I needed to make
> everything work well together.
>
> I’m using LaTeX (pdflatex), scribus, calibre and imagemagick to publish
> a roleplaying book with charactersheet, 
>
> Maybe some of it can help you. The entrypoints are the Makefile, the
> setup, and the configure.ac (for the hacks):
> https://hg.sr.ht/~arnebab/ews/browse/Hauptdokument/ews30/Makefile.am
> https://hg.sr.ht/~arnebab/ews/browse/Hauptdokument/ews30/basesetup.tex
> https://hg.sr.ht/~arnebab/ews/browse/Hauptdokument/ews30/ews30setup.tex
> https://hg.sr.ht/~arnebab/ews/browse/Hauptdokument/ews30/ews30setup.el
> https://hg.sr.ht/~arnebab/ews/browse/Hauptdokument/ews30/configure.ac
>
> The main document is
> https://hg.sr.ht/~arnebab/ews/browse/Hauptdokument/ews30/ews.org
>
> I also have some derived documents that use the included tables as data.
> Most complex example:
> https://hg.sr.ht/~arnebab/ews/browse/Hauptdokument/ews30/chargen.org.in
>
>
> Best wishes,
> Arne

Re: Org as a book publisher

[Juan Manuel Macías]

Hi Bob, thank you for your comment.

Bob Newell <bobnewell@bobnewell.net> writes:

> Aloha,
>
> Thank you for your interesting and useful post.
>
> I must really look into your examples and process. I have
> published quite a number of books with LaTeX but my process
> has been to write in org-mode, then export, and do all the
> design and typesetting directly in LaTeX. I end up with a good
> result but I would really rather work more in org-mode.
>
> Layouts require a lot of fine-tuning, particularly complex
> layouts with multiple columns. If some or most of that could
> be accomplished in org-mode, it would be a great benefit.

Multiple column layout, for example, can be achieved from Org with a
special block (and loading the multicol package). For example:

#+LaTeX_Header: \usepackage{multicol}
#+ATTR_LaTeX: :options {2}
#+begin_multicols
...
#+end_multicols

As I mentioned in a previous message, I have to organize and clean all
the code in those two books of my samples, and upload it to GitLab, in
case it could be useful to someone...

Best regards,

Juan Manuel