[PATCH] Avoid ignoring LaTeX export output errors when org-latex-pdf-process is a list

[PATCH] Avoid ignoring LaTeX export output errors when org-latex-pdf-process is a list

[Ihor Radchenko]

[-- Attachment #1: Type: text/plain, Size: 545 bytes --]

Hi,

The attached patch is fixing a rather annoying problem when
org-latex-pdf-process is set to a list. Currently, only output of the
last command in the list is preserved in *Org PDF LaTeX output* buffer,
which sometimes prevents ox-latex from detecting compilation warnings.

ox-latex relies on *Org PDF LaTeX output* to contain the output of latex
compiler. However, unless the last command in the org-latex-pdf-process
is something like pdflatex, the compiler errors are erased by
`shell-command' called by `org-compile-file'.

Best,
Ihor


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Avoid-ignoring-LaTeX-export-output-errors.patch --]
[-- Type: text/x-patch, Size: 2290 bytes --]

From d4f15ed260f47195b01cb60061b57823428d2eef Mon Sep 17 00:00:00 2001
Message-Id: <d4f15ed260f47195b01cb60061b57823428d2eef.1653191200.git.yantar92@gmail.com>
From: Ihor Radchenko <yantar92@gmail.com>
Date: Sun, 22 May 2022 11:44:56 +0800
Subject: [PATCH] Avoid ignoring LaTeX export output errors

* lisp/org-macs.el (org-compile-file): When PROCESS is a list, keep
output of all the listed commands, not just the last one.
* lisp/ox-latex.el (org-latex-pdf-process): Clarify that the process
output is parsed to detect compilation errors/warnings.
---
 lisp/org-macs.el | 9 +++++++--
 lisp/ox-latex.el | 3 +++
 2 files changed, 10 insertions(+), 2 deletions(-)

diff --git a/lisp/org-macs.el b/lisp/org-macs.el
index 10eed2686..2a7131ded 100644
--- a/lisp/org-macs.el
+++ b/lisp/org-macs.el
@@ -323,8 +323,13 @@ (defun org-compile-file (source process ext &optional err-msg log-buf spec)
 			       (?F . ,(shell-quote-argument full-name))
 			       (?o . ,(shell-quote-argument out-dir))
 			       (?O . ,(shell-quote-argument output))))))
-	   (dolist (command process)
-	     (shell-command (format-spec command spec) log-buf))
+           ;; Combine output of all commands in PROCESS.
+           (with-current-buffer log-buf
+             (let (buffer-read-only)
+               (erase-buffer)))
+           (let ((shell-command-dont-erase-buffer t))
+	     (dolist (command process)
+	       (shell-command (format-spec command spec) log-buf)))
 	   (when log-buf (with-current-buffer log-buf (compilation-mode)))))
 	(_ (error "No valid command to process %S%s" source err-msg))))
     ;; Check for process failure.  Output file is expected to be
diff --git a/lisp/ox-latex.el b/lisp/ox-latex.el
index 5a2de8afb..ed16ec46f 100644
--- a/lisp/ox-latex.el
+++ b/lisp/ox-latex.el
@@ -1426,6 +1426,9 @@ (defcustom org-latex-pdf-process
       "%latex -interaction nonstopmode -output-directory %o %f"))
   "Commands to process a LaTeX file to a PDF file.
 
+The command output will be parsed to extract compilation errors and
+warnings according to `org-latex-known-warnings'.
+
 This is a list of strings, each of them will be given to the
 shell as a command.  %f in the command will be replaced by the
 relative file name, %F by the absolute file name, %b by the file
-- 
2.35.1

Re: [PATCH] Avoid ignoring LaTeX export output errors when org-latex-pdf-process is a list

[Daniel Fleischer]

Ihor Radchenko [2022-05-22 Sun 11:51] wrote:

> The attached patch is fixing a rather annoying problem when
> org-latex-pdf-process is set to a list. Currently, only output of the
> last command in the list is preserved in *Org PDF LaTeX output* buffer,
> which sometimes prevents ox-latex from detecting compilation warnings.

> ox-latex relies on *Org PDF LaTeX output* to contain the output of latex
> compiler. However, unless the last command in the org-latex-pdf-process
> is something like pdflatex, the compiler errors are erased by
> `shell-command' called by `org-compile-file'.

Looks good. So the log buffer is going to be a concatenation of all the
logs. Will ox-latex parse the longer log in a meaningful way? Is it just
for the user to find issues by herself after a failed compilation?

Daniel Fleischer

Re: [PATCH] Avoid ignoring LaTeX export output errors when org-latex-pdf-process is a list

[Eric S Fraga]

This is very useful, especially being able to see the output of BiBTeX,
say.  Thank you.
-- 
: Eric S Fraga, with org release_9.5.3-508-ge268e4 in Emacs 29.0.50

Re: [PATCH] Avoid ignoring LaTeX export output errors when org-latex-pdf-process is a list

[Max Nikulin]

On 22/05/2022 10:51, Ihor Radchenko wrote:
> 
> The attached patch is fixing a rather annoying problem when
> org-latex-pdf-process is set to a list. Currently, only output of the
> last command in the list is preserved in *Org PDF LaTeX output* buffer,
> which sometimes prevents ox-latex from detecting compilation warnings.

I may be wrong, but I believed that some code might check buffer content 
for specific messages to determine if additional pass is required to 
resolve or adjust cross-reference links. If it is done by external tools 
working with log files than there is no problem, otherwise lisp code 
analyzing whole buffer may be confusing by first pass messages when 
there was no e.g. .aux file.

Erasing messages from previous commands making debugging of problems 
harder, so having complete logs should be an improvement.

Re: [PATCH] Avoid ignoring LaTeX export output errors when org-latex-pdf-process is a list

[Ihor Radchenko]

Daniel Fleischer <danflscr@gmail.com> writes:

> Ihor Radchenko [2022-05-22 Sun 11:51] wrote:
>
>> The attached patch is fixing a rather annoying problem when
>> org-latex-pdf-process is set to a list. Currently, only output of the
>> last command in the list is preserved in *Org PDF LaTeX output* buffer,
>> which sometimes prevents ox-latex from detecting compilation warnings.
>
>> ox-latex relies on *Org PDF LaTeX output* to contain the output of latex
>> compiler. However, unless the last command in the org-latex-pdf-process
>> is something like pdflatex, the compiler errors are erased by
>> `shell-command' called by `org-compile-file'.
>
> Looks good. So the log buffer is going to be a concatenation of all the
> logs. Will ox-latex parse the longer log in a meaningful way?

ox-latex parses warnings according to org-latex--collect-warnings and
org-latex-known-warnings. It simply uses regular expressions.

By default, org-latex-pdf-process is using latexmk and will have no
issues. In theory, if the user adds some funny command outputting
warning/error-like string to org-latex-pdf-process, there might be a
confusing. But I see it as unlikely. The current behavior of keeping
only the last command output is worse, IMHO.

>  Is it just for the user to find issues by herself after a failed
> compilation?

It is orthogonal to the patch, but yes.
org-latex-compile does the following if it finds warnings/errors:

(let ((warnings (org-latex--collect-warnings log-buf)))
	(message (concat "PDF file produced"
			 (cond
			  ((eq warnings 'error) " with errors.")
			  (warnings (concat " with warnings: " warnings))
			  (t ".")))))

You just get a message indicating that there are some errors/warnings
(or even no message in some cases, when the proposed patch is not
applied).

Best,
Ihor

Re: [PATCH] Avoid ignoring LaTeX export output errors when org-latex-pdf-process is a list

[Ihor Radchenko]

Max Nikulin <manikulin@gmail.com> writes:

> On 22/05/2022 10:51, Ihor Radchenko wrote:
>> 
>> The attached patch is fixing a rather annoying problem when
>> org-latex-pdf-process is set to a list. Currently, only output of the
>> last command in the list is preserved in *Org PDF LaTeX output* buffer,
>> which sometimes prevents ox-latex from detecting compilation warnings.
>
> I may be wrong, but I believed that some code might check buffer content 
> for specific messages to determine if additional pass is required to 
> resolve or adjust cross-reference links. If it is done by external tools 
> working with log files than there is no problem, otherwise lisp code 
> analyzing whole buffer may be confusing by first pass messages when 
> there was no e.g. .aux file.

There is no such code. Otherwise, org-latex-pdf-process would be a lot
more constrained.

Best,
Ihor

Re: [PATCH] Avoid ignoring LaTeX export output errors when org-latex-pdf-process is a list

[Ihor Radchenko]

Ihor Radchenko <yantar92@gmail.com> writes:

> The attached patch is fixing a rather annoying problem when
> org-latex-pdf-process is set to a list. Currently, only output of the
> last command in the list is preserved in *Org PDF LaTeX output* buffer,
> which sometimes prevents ox-latex from detecting compilation warnings.
>
> ox-latex relies on *Org PDF LaTeX output* to contain the output of latex
> compiler. However, unless the last command in the org-latex-pdf-process
> is something like pdflatex, the compiler errors are erased by
> `shell-command' called by `org-compile-file'.

Applied onto main via 4d0295b31.
https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=4d0295b315b86d4cc1d85565b3e83ded494a1e67

Best,
Ihor