[PATCH] Run latex more than once for LaTeX src block evaluation

[PATCH] Run latex more than once for LaTeX src block evaluation

[Michael]

[-- Attachment #1: Type: text/plain, Size: 461 bytes --]

Hello,

I have a small patch for `org-preview-latex-process-alist' making
the default setting for LaTeX source block evaluation be running
latex three times (instead of the current one). It is sometimes
necessary to run latex more than once to produce the final
document, a phenomenom described in the docstring for
`org-latex-pdf-process' which itself runs latex three times by
default. This patch just brings the former variable into parity
with the latter.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: lisp/org.el (org-preview-latex-process-alist): Run latex multiple times --]
[-- Type: text/x-patch, Size: 2852 bytes --]

From 7b297f61ee6dfb74e64c9b2a706982638a69e7f0 Mon Sep 17 00:00:00 2001
From: Michael Herstine <sp1ff@pobox.com>
Date: Thu, 21 Mar 2024 15:50:27 -0700
Subject: [PATCH] lisp/org.el (org-preview-latex-process-alist): Run latex
 multiple times

* lisp/org.el (org-preview-latex-process-alist): Run latex three
times by default

It is sometimes necessary to run latex more than once against a
given .tex file; something ably explained in the docstring for
`org-latex-pdf-process', which exports LaTeX by running latex
three times (by default). This patch brings
`org-preview-latex-process-alist' into parity with that variable,
running latex three times by default when previewing LaTeX.
---
 lisp/org.el | 12 +++++++++---
 1 file changed, 9 insertions(+), 3 deletions(-)

diff --git a/lisp/org.el b/lisp/org.el
index caddf22a1..73b6a1e0a 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -3300,7 +3300,9 @@ All available processes and theirs documents can be found in
      :image-input-type "dvi"
      :image-output-type "png"
      :image-size-adjust (1.0 . 1.0)
-     :latex-compiler ("latex -interaction nonstopmode -output-directory %o %f")
+     :latex-compiler ("latex -interaction nonstopmode -output-directory %o %f"
+                      "latex -interaction nonstopmode -output-directory %o %f"
+                      "latex -interaction nonstopmode -output-directory %o %f")
      :image-converter ("dvipng -D %D -T tight -o %O %f")
      :transparent-image-converter
      ("dvipng -D %D -T tight -bg Transparent -o %O %f"))
@@ -3311,7 +3313,9 @@ All available processes and theirs documents can be found in
      :image-input-type "dvi"
      :image-output-type "svg"
      :image-size-adjust (1.7 . 1.5)
-     :latex-compiler ("latex -interaction nonstopmode -output-directory %o %f")
+     :latex-compiler ("latex -interaction nonstopmode -output-directory %o %f"
+                      "latex -interaction nonstopmode -output-directory %o %f"
+                      "latex -interaction nonstopmode -output-directory %o %f")
      :image-converter ("dvisvgm %f --no-fonts --exact-bbox --scale=%S --output=%O"))
     (imagemagick
      :programs ("latex" "convert")
@@ -3320,7 +3324,9 @@ All available processes and theirs documents can be found in
      :image-input-type "pdf"
      :image-output-type "png"
      :image-size-adjust (1.0 . 1.0)
-     :latex-compiler ("pdflatex -interaction nonstopmode -output-directory %o %f")
+     :latex-compiler ("pdflatex -interaction nonstopmode -output-directory %o %f"
+                      "pdflatex -interaction nonstopmode -output-directory %o %f"
+                      "pdflatex -interaction nonstopmode -output-directory %o %f")
      :image-converter
      ("convert -density %D -trim -antialias %f -quality 100 %O")))
   "Definitions of external processes for LaTeX previewing.
-- 
2.44.0


[-- Attachment #3: Type: text/plain, Size: 32 bytes --]


-- 
Michael <sp1ff@runbox.com>

Re: [PATCH] Run latex more than once for LaTeX src block evaluation

[Ihor Radchenko]

Michael <sp1ff@runbox.com> writes:

> I have a small patch for `org-preview-latex-process-alist' making
> the default setting for LaTeX source block evaluation be running
> latex three times (instead of the current one). It is sometimes
> necessary to run latex more than once to produce the final
> document, a phenomenom described in the docstring for
> `org-latex-pdf-process' which itself runs latex three times by
> default. This patch just brings the former variable into parity
> with the latter.

Thanks for the patch!
May you please describe a use case when it is necessary to run latex
multiple times for previews?

CCing Timothy and Kathik - I see that
https://code.tecosaur.net/tec/org-mode/src/branch/dev/lisp/org-latex-preview.el#L102
still uses a single compiler invocation.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [PATCH] Run latex more than once for LaTeX src block evaluation

[Michael]

>> I have a small patch for `org-preview-latex-process-alist' 
>> making
>> the default setting for LaTeX source block evaluation be 
>> running
>> latex three times (instead of the current one). It is sometimes
>> necessary to run latex more than once to produce the final
>> document, a phenomenom described in the docstring for
>> `org-latex-pdf-process' which itself runs latex three times by
>> default. This patch just brings the former variable into parity
>> with the latter.
>
> Thanks for the patch!
> May you please describe a use case when it is necessary to run 
> latex
> multiple times for previews?

Sure: it's required whenever you have LaTeX that refers to
other document elements whose positions are only known after
they've been typeset. This often comes up with bibliographies; to
quote the docstring for `org-latex-pdf-process': "The reason why
this is a list is that it usually takes several runs of
‘pdflatex’, maybe mixed with a call to ‘bibtex’."

I personally encountered it when using the tikzmark library from
the tikzpicture package to draw  annotations on a table (LaTeX
needed to be run once to typeset the table & only then did it
"know" where the arrows needed to be drawn).

Oh-- and it's not just previews (sorry if I gave that
impression)-- it's also required for "evaluating" LaTeX source
blocks.

> CCing Timothy and Kathik - I see that
> https://code.tecosaur.net/tec/org-mode/src/branch/dev/lisp/org-latex-preview.el#L102
> still uses a single compiler invocation.


-- 
Michael <mherstine@pobox.com>

Re: [PATCH] Run latex more than once for LaTeX src block evaluation

[Ihor Radchenko]

Michael <sp1ff@runbox.com> writes:

>> May you please describe a use case when it is necessary to run 
>> latex
>> multiple times for previews?
> ...
> Oh-- and it's not just previews (sorry if I gave that
> impression)-- it's also required for "evaluating" LaTeX source
> blocks.

This makes more sense. I hardly imagine previews that will require
running latex multiple times. Such previews would be broken anyway,
because they have no access to the document context.

May you please add a code comment explaining why multiple invocations
are needed, referring to ob-latex? Also, please link to this thread form
the commit message. Finally, you may need to add TINYCHANGE cookie at
the bottom of the commit message if you do not have FSF copyright
assignment (see https://orgmode.org/worg/org-contribute.html#first-patch)

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [PATCH] Run latex more than once for LaTeX src block evaluation

[Max Nikulin]

On 22/03/2024 05:55, Michael wrote:
> I have a small patch for `org-preview-latex-process-alist' making
> the default setting for LaTeX source block evaluation be running
> latex three times (instead of the current one).

I suspect it may make the LaTeX preview feature unacceptably slow for 
documents heavily loaded with math snippets. There was a proposal to use 
LuaLaTeX instead of PdfLaTeX to provide better Unicode coverage, but it 
was discarded due to significant slowdown.

If it is true then it should not be active by default. It makes sense as 
a user option, but I am unsure concerning required efforts.

Re: [PATCH] Run latex more than once for LaTeX src block evaluation

[Ihor Radchenko]

Max Nikulin <manikulin@gmail.com> writes:

> On 22/03/2024 05:55, Michael wrote:
>> I have a small patch for `org-preview-latex-process-alist' making
>> the default setting for LaTeX source block evaluation be running
>> latex three times (instead of the current one).
>
> I suspect it may make the LaTeX preview feature unacceptably slow for 
> documents heavily loaded with math snippets. There was a proposal to use 
> LuaLaTeX instead of PdfLaTeX to provide better Unicode coverage, but it 
> was discarded due to significant slowdown.
>
> If it is true then it should not be active by default. It makes sense as 
> a user option, but I am unsure concerning required efforts.

Then, we may instead use latexmk - it will run latex as many times as
necessary.

Also, performance will be less problematic after Timothy and Karthik
merge their latex preview branch. (they may want to do measurements
though).

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [PATCH] Run latex more than once for LaTeX src block evaluation

[Max Nikulin]

On 24/03/2024 15:58, Ihor Radchenko wrote:
> Max Nikulin writes:
>> On 22/03/2024 05:55, Michael wrote:
>>> I have a small patch for `org-preview-latex-process-alist' making
>>> the default setting for LaTeX source block evaluation be running
>>> latex three times (instead of the current one).
>>
>> I suspect it may make the LaTeX preview feature unacceptably slow for
>> documents heavily loaded with math snippets.
> 
> Then, we may instead use latexmk - it will run latex as many times as
> necessary.

Sorry I have no experience with latexmk. At least a fallback should be 
available if this script is not installed.

The LaTeX preview feature is close to dangerous per se. With latexmk the 
barrier may be even lower. Some users may have -shell-escape enabled in 
latexmk configuration. I hope, it is not possible to force latexmk 
execute something weird by spitting a crafted message into logs.

I have no idea how to enable preview for trusted files only without 
ruining usability.

Re: [PATCH] Run latex more than once for LaTeX src block evaluation

[Michael]

Max Nikulin <manikulin@gmail.com> writes:

> On 24/03/2024 15:58, Ihor Radchenko wrote:
>> Max Nikulin writes:
>>> On 22/03/2024 05:55, Michael wrote:
>>>> I have a small patch for `org-preview-latex-process-alist' 
>>>> making
>>>> the default setting for LaTeX source block evaluation be 
>>>> running
>>>> latex three times (instead of the current one).
>>>
>>> I suspect it may make the LaTeX preview feature unacceptably 
>>> slow for
>>> documents heavily loaded with math snippets.
>> Then, we may instead use latexmk - it will run latex as many 
>> times
>> as
>> necessary.
>
> Sorry I have no experience with latexmk. At least a fallback 
> should be
> available if this script is not installed.
>
> The LaTeX preview feature is close to dangerous per se. With 
> latexmk
> the barrier may be even lower. Some users may have -shell-escape
> enabled in latexmk configuration. I hope, it is not possible to 
> force
> latexmk execute something weird by spitting a crafted message 
> into
> logs.
>
> I have no idea how to enable preview for trusted files only 
> without
> ruining usability.

Should we perhaps have different variables for preview & Org
source block evaluation?

-- 
Michael <mherstine@pobox.com>

Re: [PATCH] Run latex more than once for LaTeX src block evaluation

[Ihor Radchenko]

Michael <sp1ff@runbox.com> writes:

> Should we perhaps have different variables for preview & Org
> source block evaluation?

Likely yes.

In fact, ob-latex is making use of `org-preview-latex-process-alist'
only in a single cond branch in `org-babel-execute:latex' - when we have
:file foo.png

However, that branch assumes that `org-preview-latex-default-process' is
'dvipng (the default value). If one changes it to, say dvisvgm, the
generated image will not be a .png image:

(setq org-preview-latex-default-process 'dvisvgm)

#+begin_src latex :results file link :file test.png
x^2
#+end_src

#+RESULTS:
[[attachment:test.png]]

^ This is actually an svg image, renamed to "test.png".

So, it makes sense for `org-babel-execute:latex' to override
`org-preview-latex-default-process' temporarily, to something actually
generating .png file.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [PATCH] Run latex more than once for LaTeX src block evaluation

[Michael]

Ihor Radchenko <yantar92@posteo.net> writes:

> Michael <sp1ff@runbox.com> writes:
>
>> Should we perhaps have different variables for preview & Org
>> source block evaluation?
>
> Likely yes.
>
> In fact, ob-latex is making use of 
> `org-preview-latex-process-alist'
> only in a single cond branch in `org-babel-execute:latex' - when 
> we have
> :file foo.png
>
> However, that branch assumes that 
> `org-preview-latex-default-process' is
> 'dvipng (the default value). If one changes it to, say dvisvgm, 
> the
> generated image will not be a .png image:
>
> (setq org-preview-latex-default-process 'dvisvgm)
>
> #+begin_src latex :results file link :file test.png
> x^2
> #+end_src
>
> #+RESULTS:
> [[attachment:test.png]]
>
> ^ This is actually an svg image, renamed to "test.png".
>
> So, it makes sense for `org-babel-execute:latex' to override
> `org-preview-latex-default-process' temporarily, to something 
> actually
> generating .png file.

Sorry-- got sidetracked by something else. I see what you
mean. OK, so the proposal is: change the ".png" branch in
`org-babel-execute:latex' to override `org-preview-process-alist'
with something that will:

  - actually produce a PNG formatted file
  - by default, run latex more than once, tho this could of
  - course be customized

-- 
Michael <mherstine@pobox.com>

Re: [PATCH] Run latex more than once for LaTeX src block evaluation

[Michael]

Oh-- and pass `processing-type' to `org-create-formula-image' 
explicitly.

-- 
Michael <mherstine@pobox.com>

Re: [PATCH] Run latex more than once for LaTeX src block evaluation

[Karthik Chikmagalur]

>> Thanks for the patch!
>> May you please describe a use case when it is necessary to run 
>> latex
>> multiple times for previews?
>
> Sure: it's required whenever you have LaTeX that refers to
> other document elements whose positions are only known after
> they've been typeset. This often comes up with bibliographies; to
> quote the docstring for `org-latex-pdf-process': "The reason why
> this is a list is that it usually takes several runs of
> ‘pdflatex’, maybe mixed with a call to ‘bibtex’."

I assume this is a general example of a situation where you have to run
LaTeX multiple times, and not something applicable to Org's LaTeX
previews.

> I personally encountered it when using the tikzmark library from
> the tikzpicture package to draw  annotations on a table (LaTeX
> needed to be run once to typeset the table & only then did it
> "know" where the arrows needed to be drawn).

Was this for in-buffer previews or a LaTeX source block?  Could you
provide the LaTeX source for a minimal example?  I'd like to try it with
the new preview system.

Karthik

Re: [PATCH] Run latex more than once for LaTeX src block evaluation

[Ihor Radchenko]

Michael <sp1ff@runbox.com> writes:

>> ...
>> So, it makes sense for `org-babel-execute:latex' to override
>> `org-preview-latex-default-process' temporarily, to something 
>> actually
>> generating .png file.
>
> Sorry-- got sidetracked by something else. I see what you
> mean. OK, so the proposal is: change the ".png" branch in
> `org-babel-execute:latex' to override `org-preview-process-alist'
> with something that will:
>
>   - actually produce a PNG formatted file
>   - by default, run latex more than once, tho this could of
>   - course be customized

May you prepare a patch?

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [PATCH] Run latex more than once for LaTeX src block evaluation

[Ihor Radchenko]

Ihor Radchenko <yantar92@posteo.net> writes:

> (setq org-preview-latex-default-process 'dvisvgm)
>
> #+begin_src latex :results file link :file test.png
> x^2
> #+end_src
>
> #+RESULTS:
> [[attachment:test.png]]
>
> ^ This is actually an svg image, renamed to "test.png".
>
> So, it makes sense for `org-babel-execute:latex' to override
> `org-preview-latex-default-process' temporarily, to something actually
> generating .png file.

Now done on main.
https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=43ba00b2b

I introduced a new custom option org-babel-latex-process-alist to
control ob-latex behaviour specifically.

For now, it is preserving the previous process settings. No repetitive
latex runs.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [PATCH] Run latex more than once for LaTeX src block evaluation

[Ihor Radchenko]

Ihor Radchenko <yantar92@posteo.net> writes:

> Michael <sp1ff@runbox.com> writes:
>
>>> ...
>>> So, it makes sense for `org-babel-execute:latex' to override
>>> `org-preview-latex-default-process' temporarily, to something 
>>> actually
>>> generating .png file.
>>
>> Sorry-- got sidetracked by something else. I see what you
>> mean. OK, so the proposal is: change the ".png" branch in
>> `org-babel-execute:latex' to override `org-preview-process-alist'
>> with something that will:
>>
>>   - actually produce a PNG formatted file
>>   - by default, run latex more than once, tho this could of
>>   - course be customized
>
> May you prepare a patch?

I now changed the defaults for ob-latex on main.
Either latexmk is used or latex is executed trice.

https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=c426f4936

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [PATCH] Run latex more than once for LaTeX src block evaluation

[Michael]

Ihor Radchenko <yantar92@posteo.net> writes:

> Michael <sp1ff@runbox.com> writes:
>
>>> ...
>>> So, it makes sense for `org-babel-execute:latex' to override
>>> `org-preview-latex-default-process' temporarily, to something 
>>> actually
>>> generating .png file.
>>
>> Sorry-- got sidetracked by something else. I see what you
>> mean. OK, so the proposal is: change the ".png" branch in
>> `org-babel-execute:latex' to override 
>> `org-preview-process-alist'
>> with something that will:
>>
>>   - actually produce a PNG formatted file
>>   - by default, run latex more than once, tho this could of
>>   - course be customized
>
> May you prepare a patch?

My apologies, Ihor... completely missed this. Yes, give me a day
or two.

-- 
Michael <mherstine@pobox.com>

Re: [PATCH] Run latex more than once for LaTeX src block evaluation

[Ihor Radchenko]

Michael <sp1ff@runbox.com> writes:

>>> Sorry-- got sidetracked by something else. I see what you
>>> mean. OK, so the proposal is: change the ".png" branch in
>>> `org-babel-execute:latex' to override 
>>> `org-preview-process-alist'
>>> with something that will:
>>>
>>>   - actually produce a PNG formatted file
>>>   - by default, run latex more than once, tho this could of
>>>   - course be customized
>>
>> May you prepare a patch?
>
> My apologies, Ihor... completely missed this. Yes, give me a day
> or two.

FYI, the patch implementing the suggestions is already in place.
https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=c426f4936

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [PATCH] Run latex more than once for LaTeX src block evaluation

[Michael]

Ihor Radchenko <yantar92@posteo.net> writes:

> Michael <sp1ff@runbox.com> writes:
>
>>>> Sorry-- got sidetracked by something else. I see what you
>>>> mean. OK, so the proposal is: change the ".png" branch in
>>>> `org-babel-execute:latex' to override 
>>>> `org-preview-process-alist'
>>>> with something that will:
>>>>
>>>>   - actually produce a PNG formatted file
>>>>   - by default, run latex more than once, tho this could of
>>>>   - course be customized
>>>
>>> May you prepare a patch?
>>
>> My apologies, Ihor... completely missed this. Yes, give me a 
>> day
>> or two.
>
> FYI, the patch implementing the suggestions is already in place.
> https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=c426f4936

My bad, Thank you.

-- 
Michael <mherstine@pobox.com>