Re: [tip] org-publish to work with (very) large books

[Christian Moe <mail@christianmoe.com>]

Thanks for this, really interesting.

Do I understand correctly that the main advantage of this approach (over
#+INCLUDE) is the ability to continuously update preview of the whole
book with latexmk -pvc even if you only re-export one chapter from
Org-mode?

I couldn't find the :body-only publishing option in the docs ...?

Yours,
Christian

Juan Manuel Macías writes:

> Hi all,
>
> - tl; dr: I describe here my workflow with org-publish to work with long
> books.
>
> —
>
> I discovered a long time ago that `org-publish' not only works very well
> for managing websites but also for working with long and complex books
> with many parts, with output to LaTeX/PDF. I developed a workflow around
> it that has been quite productive for me, and that I briefly describe
> here in case someone finds it useful and wants to try it or modify/adapt
> it to their needs. I usually use it for my typesetting work, but I think
> it can also be useful for personal projects, such as doctoral theses.
>
> First of all, each folder of my project-books has the same structure:
> two subdirectories named `/org' and `/tex', for the source `org' files
> and for the output `.tex' documents, respectively. And, inside the `org'
> directory I include a `setup' file, an `elisp' file (for export
> filters), and another `/img' directory for image files. Each `org' file
> is a part of the book, and usually begins simply with the directives:
>
> ┌────
> │ #+SETUPFILE: xxx.setup
> │ #+INCLUDE: "elisp"
> └────
>
> `Org-publish' exports the subdocuments (body only!) as `.tex' documents
> in the `/tex' folder, but they are not compiled.
>
> What gets compiled is a master `.org' file, which is also inside the
> `org' folder. I compile this master file using an asynchronous function
> that calls `latexmk'. I put all the LaTeX configuration, the packages to
> load, the (re)defined commands and macros, the necessary Lua code, etc.
> in a `.sty' file that I load at the very beginning of the master
> document. Subdocuments are loaded into this file by the LaTeX command
> (\input), not by the org #+INCLUDE directive. So the master file usually
> looks like this:
>
> ┌────
> │ #+LaTeX_CLASS: my-custom-latex-class
> │ #+LaTeX_Header: \input{my-custom-conf.sty}
> │ #+SETUPFILE: xxxx.setup
> │ #+INCLUDE: "elisp"
> │
> │ * Part 1
> │ ** Chapter 1
> │ #+LaTeX: \input{chapter1.tex}
> └────
>
> When I eval my function, `latexmk' compiles the entire book with the
> `-pvc' option, which keeps the session open, and if it detects any
> changes to the entire document, it recompiles and refresh the pdf view.
> For example, if I edit one of the subdocuments and run
> `org-publish-current-file', everything is automatically recompiled.
>
> When I have the project folder ready, I add this to
> `org-publish-project-alist' (this is an example from one of the books I
> did recently):
>
> ┌────
> │ ("cartas-org"
> │  :base-directory "~/Git/cartas/libro/org/"
> │  :base-extension "org"
> │  ;; Directorio para los ficheros *.tex
> │  :publishing-directory "~/Git/cartas/libro/tex/"
> │  :publishing-function org-latex-publish-to-latex
> │  :body-only t ;; this is important!
> │  :exclude "cartas-master\\.org\\|bibliografia-cartas\\.org"
> │  :recursive t)
> │
> │ ("cartas-img"
> │  :base-directory "~/Git/cartas/libro/org/img/"
> │  :base-extension "jpg\\|png"
> │  :publishing-directory "~/Git/cartas/libro/tex/img/"
> │  :recursive t
> │  :publishing-function org-publish-attachment)
> │
> │ ("cartas" :components ("cartas-tex" "cartas-img"))
> └────
>
> And finally, this is the function that compiles everything (each project
> usually has some local variables, like the value of ’jobname’ or the
> status of the printing proofs).
>
> Nota Bene: The reason for using async is that in some projects,
> especially bilingual editions, I need to pre-compile some files first.
> Under normal conditions I don't think it's necessary to use async, since
> org-publish just exports everything to .tex documents (short timeout)
> and then start-process-shell-command run latexmk asynchronously.
>
> Best regards,
>
> Juan Manuel
>
> ┌────
> │ (require 'async)
> │ (require 'projectile)
> │
> │ (defun latexmk-compile-project-async ()
> │   (interactive)
> │   (let*
> │       ((project-root (projectile-project-root))
> │        (master-file (read-file-name "Compile: "
> │ 				    (concat project-root "libro/org/")))
> │        (master-file-tex (file-name-sans-extension
> │ 			 (expand-file-name master-file)))
> │        (dir-tex (file-name-directory
> │ 		 (expand-file-name
> │ 		  (replace-regexp-in-string "/org/" "/tex/" master-file)))))
> │     ;; save the master document
> │     (with-current-buffer
> │ 	(find-file-noselect master-file)
> │       (save-buffer))
> │     (async-start
> │      (lambda ()
> │        (load "~/.emacs")
> │        (with-current-buffer
> │ 	   (find-file-noselect master-file)
> │ 	 (org-show-all)
> │ 	 (save-buffer)
> │ 	 (org-latex-export-to-latex nil nil nil nil nil))
> │        ;; remove all old auxiliary files before compiling
> │        (shell-command (concat "rm -r " dir-tex (file-name-base master-file-tex) "*"))
> │        (shell-command (concat "mv " master-file-tex ".tex" " " dir-tex))
> │        "Document exported")
> │      (lambda (resultado)
> │        (message resultado)
> │        (let
> │ 	   ((default-directory dir-tex)
> │ 	    (jobname (if (and jobname-local printing-proofs-state)
> │ 			 (concat jobname-local "_" printing-proofs-state "_"
> │ 				 (format-time-string "%d-%m-%y"))
> │ 		       (concat (file-name-sans-extension
> │ 				(file-name-nondirectory master-file-tex))
> │ 			       "_"
> │ 			       (format-time-string "%d-%m-%y")))))
> │ 	 (start-process-shell-command
> │ 	  "project"
> │ 	  "*project*"
> │ 	  (concat
> │ 	   "latexmk"
> │ 	   " -jobname="
> │ 	   jobname
> │ 	   " -pvc -lualatex -e '$lualatex=q/lualatex %O -shell-escape %S/' "
> │ 	   (file-name-nondirectory master-file-tex)
> │ 	   ".tex")))))))
> └────