Re: [possible patch] Remove the '\\[0pt]' string from the last line of a verse block in LaTeX export

[Ihor Radchenko <yantar92@posteo.net>]

[-- Attachment #1: Type: text/plain, Size: 1386 bytes --]

Juan Manuel Macías <maciaschain@posteo.net> writes:

>> Although I am still a bit hesitant to remove
>> `org-latex-line-break-safe'.
>> What would be the benefit of removing it? For now, I mostly just see
>> that it will make the life harder for users in Scenario B.
>
> It's a complicated situation, because we now have two solutions to the
> same problem... It certainly sounds a bit abrupt to remove
> org-latex-line-break-safe (at least for now). I see no problem in both
> solutions coexisting. After all, the user can always give an "\\\\"
> value to org-latex-line-break-safe. The other possibility is that
> org-latex-line-break-safe is selectively deleted, as you mentioned in a
> previous email. In tables and verse blocks, unless I'm missing
> something, I think adding [0pt] would be unnecessary, with the new
> solution.

Upon looking closer into selective removal, it turned out to be more
tricky than I thought. I'm afraid that using \\[0pt] only in some places
may become a bit of headache to maintain - we may accidentally break
certain regexp replacements in `org-latex-verse-block'. In particular,
when verse contents is not straightforward and uses \\[0pt].

Given that `org-latex-line-break-safe' also introduced the problem with
verse blocks, I decided that it is better to remove it at the end.

See the attached tentative patchset.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: v2-0001-ox-latex-Make-sure-that-text-is-not-misinterprete.patch --]
[-- Type: text/x-patch, Size: 2518 bytes --]

From d2e74cc2734eaf8d782f5acf66b11956d6ffa47e Mon Sep 17 00:00:00 2001
Message-ID: <d2e74cc2734eaf8d782f5acf66b11956d6ffa47e.1705844367.git.yantar92@posteo.net>
From: Ihor Radchenko <yantar92@posteo.net>
Date: Thu, 18 Jan 2024 14:01:32 +0100
Subject: [PATCH v2 1/2] ox-latex: Make sure that [text] is not misinterpreted
 as LaTeX argument

* lisp/ox-latex.el (org-latex-plain-text): Protect plain text starting
from [.  It might be misinterpreted as optional command argument if
previous exported fragment ends with a command accepting such.
*
testing/lisp/test-ox-latex.el (text-ox-latex/protect-square-brackets):
Add new test.

Link: https://orgmode.org/list/87o7dju9vn.fsf@posteo.net
---
 lisp/ox-latex.el              |  9 +++++++++
 testing/lisp/test-ox-latex.el | 23 +++++++++++++++++++++++
 2 files changed, 32 insertions(+)

diff --git a/lisp/ox-latex.el b/lisp/ox-latex.el
index df20345f8..a64dd5a87 100644
--- a/lisp/ox-latex.el
+++ b/lisp/ox-latex.el
@@ -3095,6 +3095,15 @@ (defun org-latex-plain-text (text info)
 		    "\\(?:[ \t]*\\\\\\\\\\)?[ \t]*\n"
                     (concat org-latex-line-break-safe "\n")
                     output nil t)))
+    ;; Protect [foo] at the beginning of lines / beginning of the
+    ;; plain-text object.  This prevents LaTeX from unexpectedly
+    ;; interpreting @@latex:\pagebreak@@ [foo] as a command with
+    ;; optional argument.
+    (setq output (replace-regexp-in-string
+                  (rx bol (0+ space) (group "["))
+                  "{[}"
+                  output
+                  nil nil 1))
     ;; Return value.
     output))
 
diff --git a/testing/lisp/test-ox-latex.el b/testing/lisp/test-ox-latex.el
index 41df1b823..237ad97ec 100644
--- a/testing/lisp/test-ox-latex.el
+++ b/testing/lisp/test-ox-latex.el
@@ -29,6 +29,29 @@ (unless (featurep 'ox-latex)
 
 \f
 
+(ert-deftest text-ox-latex/protect-square-brackets ()
+  "Test [foo] being interpreted as plain text even after LaTeX commands."
+  (org-test-with-exported-text
+      'latex
+      "* This is test
+lorem @@latex:\\pagebreak@@ [ipsum]
+
+#+begin_figure
+[lorem] figure
+#+end_figure
+
+| [foo] | 2 |
+| [bar] | 3 |
+
+- [bax]
+- [aur]
+"
+    (goto-char (point-min))
+    (should (search-forward "lorem \\pagebreak {[}ipsum]"))
+    (should (search-forward "{[}lorem] figure"))
+    (should (search-forward "{[}foo]"))
+    (should (search-forward "\\item {[}bax]"))))
+
 (ert-deftest test-ox-latex/verse ()
   "Test verse blocks."
   (org-test-with-exported-text
-- 
2.43.0


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #3: v2-0002-ox-latex-Remove-org-latex-line-break-safe.patch --]
[-- Type: text/x-patch, Size: 11526 bytes --]

From dd4618b31dc5070d6802e484fd58cf50f5d3606d Mon Sep 17 00:00:00 2001
Message-ID: <dd4618b31dc5070d6802e484fd58cf50f5d3606d.1705844367.git.yantar92@posteo.net>
In-Reply-To: <d2e74cc2734eaf8d782f5acf66b11956d6ffa47e.1705844367.git.yantar92@posteo.net>
References: <d2e74cc2734eaf8d782f5acf66b11956d6ffa47e.1705844367.git.yantar92@posteo.net>
From: Ihor Radchenko <yantar92@posteo.net>
Date: Sun, 21 Jan 2024 14:21:33 +0100
Subject: [PATCH v2 2/2] ox-latex: Remove org-latex-line-break-safe

This reverts commit 3f60acff7717e472d06833e9cf9fff6ca3d59337 and
subsequent relevant comments.

* lisp/ox-latex.el (org-latex-line-break-safe): Remove constant.  The
\\[0pt] is actually not safe to use in some scenarios. We use a
different approach to avoid plain text [...] being interpreted as
LaTeX optional argument - we escape [ like {[}; that's what pandoc
does.
(org-latex-clock):
(org-latex-line-break):
(org-latex-plain-text):
(org-latex-planning):
(org-latex--org-table):
(org-latex--math-table):
(org-latex-table-row):
(org-latex-verse-block):
* testing/lisp/test-org-table.el (test-org-table/to-latex):
* testing/lisp/test-ox-latex.el (test-ox-latex/verse):
(test-ox-latex/longtable): Remove references to
`org-latex-line-break-safe'.
* etc/ORG-NEWS (=ox-latex=: ~org-latex-line-break-safe~ is removed):
Announce removal.
* lisp/org-compat.el (org-latex-line-break-safe): Make obsolete.

Link: https://orgmode.org/list/878r4jg37s.fsf@posteo.net
---
 etc/ORG-NEWS                   | 10 +++++++
 lisp/org-compat.el             | 18 +++++++++++++
 lisp/ox-latex.el               | 49 ++++++++++++----------------------
 testing/lisp/test-org-table.el |  6 ++---
 testing/lisp/test-ox-latex.el  | 16 +++++------
 5 files changed, 56 insertions(+), 43 deletions(-)

diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS
index 847ddf614..46f1e66a7 100644
--- a/etc/ORG-NEWS
+++ b/etc/ORG-NEWS
@@ -13,6 +13,16 @@ Please send Org bug reports to mailto:emacs-orgmode@gnu.org.
 
 * Version 9.7 (not released yet)
 ** Important announcements and breaking changes
+*** =ox-latex=: ~org-latex-line-break-safe~ is removed
+
+~org-latex-line-break-safe~ constant was previously introduced to deal
+with edge cases when LaTeX interprets [...] as LaTeX command
+argument.  However, it caused a number of other issues and proved
+itself not to be as "safe" as it supposed to be.
+
+We now use a Pandoc's approach to deal with the same problem,
+utilizing ={[}= to escape =[...]= instances where needed.
+
 *** ~org-agenda-search-headline-for-time~ now ignores all the timestamp in headings
 
 Previously, ~org-agenda-search-headline-for-time~ made Org agenda
diff --git a/lisp/org-compat.el b/lisp/org-compat.el
index 17f07db9d..27d094b07 100644
--- a/lisp/org-compat.el
+++ b/lisp/org-compat.el
@@ -629,6 +629,24 @@ (define-obsolete-variable-alias 'org-reveal-start-hook
 (define-obsolete-function-alias 'org-file-url-p 'org-url-p "9.6")
 (define-obsolete-variable-alias 'org-plantuml-executable-args 'org-plantuml-args
   "Org 9.6")
+
+(defconst org-latex-line-break-safe "\\\\[0pt]"
+  "Linebreak protecting the following [...].
+
+Without \"[0pt]\" it would be interpreted as an optional argument to
+the \\\\.
+
+This constant, for example, makes the below code not err:
+
+\\begin{tabular}{c|c}
+    [t] & s\\\\[0pt]
+    [I] & A\\\\[0pt]
+    [m] & kg
+\\end{tabular}")
+(make-obsolete 'org-latex-line-break-safe
+               "should not be used - it is not safe in all the scenarios."
+               "9.7")
+
 (defun org-in-fixed-width-region-p ()
   "Non-nil if point in a fixed-width region."
   (save-match-data
diff --git a/lisp/ox-latex.el b/lisp/ox-latex.el
index a64dd5a87..d6e74cb45 100644
--- a/lisp/ox-latex.el
+++ b/lisp/ox-latex.el
@@ -298,23 +298,10 @@ (defconst org-latex-language-alist
 - `:script-tag' the script otf tag.")
 
 
-(defconst org-latex-line-break-safe "\\\\[0pt]"
-  "Linebreak protecting the following [...].
 
-Without \"[0pt]\" it would be interpreted as an optional argument to
-the \\\\.
-
-This constant, for example, makes the below code not err:
-
-\\begin{tabular}{c|c}
-    [t] & s\\\\[0pt]
-    [I] & A\\\\[0pt]
-    [m] & kg
-\\end{tabular}")
-
-(defconst org-latex-table-matrix-macros `(("bordermatrix" . "\\cr")
+(defconst org-latex-table-matrix-macros '(("bordermatrix" . "\\cr")
 					  ("qbordermatrix" . "\\cr")
-					  ("kbordermatrix" . ,org-latex-line-break-safe))
+					  ("kbordermatrix" . "\\\\"))
   "Alist between matrix macros and their row ending.")
 
 (defconst org-latex-math-environments-re
@@ -2124,7 +2111,7 @@ (defun org-latex-clock (clock _contents info)
 	   (concat (org-timestamp-translate (org-element-property :value clock))
 		   (let ((time (org-element-property :duration clock)))
 		     (and time (format " (%s)" time)))))
-   org-latex-line-break-safe))
+   "\\\\"))
 
 
 ;;;; Code
@@ -2722,7 +2709,7 @@ ;;;; Line Break
 (defun org-latex-line-break (_line-break _contents _info)
   "Transcode a LINE-BREAK object from Org to LaTeX.
 CONTENTS is nil.  INFO is a plist holding contextual information."
-  (concat org-latex-line-break-safe "\n"))
+  "\\\\\n")
 
 
 ;;;; Link
@@ -3092,9 +3079,7 @@ (defun org-latex-plain-text (text info)
     ;; Handle break preservation if required.
     (when (plist-get info :preserve-breaks)
       (setq output (replace-regexp-in-string
-		    "\\(?:[ \t]*\\\\\\\\\\)?[ \t]*\n"
-                    (concat org-latex-line-break-safe "\n")
-                    output nil t)))
+		    "\\(?:[ \t]*\\\\\\\\\\)?[ \t]*\n" "\\\\\n" output nil t)))
     ;; Protect [foo] at the beginning of lines / beginning of the
     ;; plain-text object.  This prevents LaTeX from unexpectedly
     ;; interpreting @@latex:\pagebreak@@ [foo] as a command with
@@ -3139,7 +3124,7 @@ (defun org-latex-planning (planning _contents info)
 		(format (plist-get info :latex-active-timestamp-format)
 			(org-timestamp-translate scheduled)))))))
     " ")
-   org-latex-line-break-safe))
+   "\\\\"))
 
 
 ;;;; Property Drawer
@@ -3919,11 +3904,11 @@ (defun org-latex--org-table (table contents info)
 		(format "\\begin{%s}%s{%s}\n" table-env width alignment)
 		(and above?
 		     (org-string-nw-p caption)
-		     (concat caption org-latex-line-break-safe "\n"))
+		     (concat caption "\\\\\n"))
 		contents
 		(and (not above?)
 		     (org-string-nw-p caption)
-		     (concat caption org-latex-line-break-safe "\n"))
+		     (concat caption "\\\\\n"))
 		(format "\\end{%s}" table-env)
 		(and fontsize "}"))))
      (t
@@ -4008,7 +3993,7 @@ (defun org-latex--math-table (table info)
 		 (lambda (cell)
 		   (substring (org-element-interpret-data cell) 0 -1))
 		 (org-element-map row 'table-cell #'identity info) "&")
-		(or (cdr (assoc env org-latex-table-matrix-macros)) org-latex-line-break-safe)
+		(or (cdr (assoc env org-latex-table-matrix-macros)) "\\\\")
 		"\n")))
 	   (org-element-map table 'table-row #'identity info) "")))
     (concat
@@ -4083,7 +4068,7 @@ (defun org-latex-table-row (table-row contents info)
             (setq table-head-cache (make-hash-table :test #'eq))
             (plist-put info :org-latex-table-head-cache table-head-cache))
           (if-let ((head-contents (gethash (org-element-parent table-row) table-head-cache)))
-              (puthash (org-element-parent table-row) (concat head-contents org-latex-line-break-safe "\n" contents)
+              (puthash (org-element-parent table-row) (concat head-contents "\\\\\n" contents)
                        table-head-cache)
             (puthash (org-element-parent table-row) contents table-head-cache))))
       ;; Return LaTeX string as the transcoder.
@@ -4092,7 +4077,7 @@ (defun org-latex-table-row (table-row contents info)
        ;; hline was specifically marked.
        (and booktabsp (not (org-export-get-previous-element table-row info))
 	    "\\toprule\n")
-       contents org-latex-line-break-safe "\n"
+       contents "\\\\\n"
        (cond
 	;; Special case for long tables.  Define header and footers.
 	((and longtablep (org-export-table-row-ends-header-p table-row info))
@@ -4100,9 +4085,9 @@ (defun org-latex-table-row (table-row contents info)
 			      (org-element-lineage table-row 'table) info))))
 	   (format "%s
 \\endfirsthead
-\\multicolumn{%d}{l}{%s} \\\\[0pt]
+\\multicolumn{%d}{l}{%s} \\\\
 %s
-%s \\\\[0pt]\n
+%s \\\\\n
 %s
 \\endhead
 %s\\multicolumn{%d}{r}{%s} \\\\
@@ -4211,15 +4196,15 @@ (defun org-latex-verse-block (verse-block contents info)
 	       (replace-regexp-in-string
                 (if (not lit)
 		    (rx-to-string
-                     `(seq (group ,org-latex-line-break-safe "\n")
-		           (1+ (group line-start (0+ space) ,org-latex-line-break-safe "\n"))))
-		  (concat "^[ \t]*" (regexp-quote org-latex-line-break-safe) "$"))
+                     `(seq (group "\\\\\n")
+		           (1+ (group line-start (0+ space) "\\\\\n"))))
+		  "^[ \t]*\\\\$")
 	        (if (not lit)
 		    (if lin "\\\\!\n\n" "\n\n")
 		  "\\vspace*{\\baselineskip}")
 	        (replace-regexp-in-string
 	         "\\([ \t]*\\\\\\\\\\)?[ \t]*\n"
-                 (concat org-latex-line-break-safe "\n")
+                 "\\\\\n"
 	         (if (not lit)
 		     (concat (org-trim contents t) "\n")
 		   contents)
diff --git a/testing/lisp/test-org-table.el b/testing/lisp/test-org-table.el
index 92ccd2a05..6ee790894 100644
--- a/testing/lisp/test-org-table.el
+++ b/testing/lisp/test-org-table.el
@@ -1633,11 +1633,11 @@ (ert-deftest test-org-table/to-generic ()
 (ert-deftest test-org-table/to-latex ()
   "Test `orgtbl-to-latex' specifications."
   (should
-   (equal "\\begin{tabular}{l}\na\\\\[0pt]\n\\end{tabular}"
+   (equal "\\begin{tabular}{l}\na\\\\\n\\end{tabular}"
 	  (orgtbl-to-latex (org-table-to-lisp "| a |") nil)))
   ;; Test :environment parameter.
   (should
-   (equal "\\begin{tabularx}{l}\na\\\\[0pt]\n\\end{tabularx}"
+   (equal "\\begin{tabularx}{l}\na\\\\\n\\end{tabularx}"
 	  (orgtbl-to-latex (org-table-to-lisp "| a |")
 			   '(:environment "tabularx"))))
   ;; Test :booktabs parameter.
@@ -1646,7 +1646,7 @@ (ert-deftest test-org-table/to-latex ()
     "\\toprule" (orgtbl-to-latex (org-table-to-lisp "| a |") '(:booktabs t))))
   ;; Handle LaTeX snippets.
   (should
-   (equal "\\begin{tabular}{l}\n\\(x\\)\\\\[0pt]\n\\end{tabular}"
+   (equal "\\begin{tabular}{l}\n\\(x\\)\\\\\n\\end{tabular}"
 	  (orgtbl-to-latex (org-table-to-lisp "| $x$ |") nil)))
   ;; Test pseudo objects and :raw parameter.
   (should
diff --git a/testing/lisp/test-ox-latex.el b/testing/lisp/test-ox-latex.el
index 237ad97ec..d0be4e5a4 100644
--- a/testing/lisp/test-ox-latex.el
+++ b/testing/lisp/test-ox-latex.el
@@ -71,14 +71,14 @@ (ert-deftest test-ox-latex/verse ()
     (should
      (search-forward
       "\\begin{verse}
-lorem ipsum dolor\\\\[0pt]
+lorem ipsum dolor\\\\
 lorem ipsum dolor
 
-lorem ipsum dolor\\\\[0pt]
+lorem ipsum dolor\\\\
 lorem ipsum dolor
 
-lorem ipsum dolor\\\\[0pt]
-lorem ipsum dolor\\\\[0pt]
+lorem ipsum dolor\\\\
+lorem ipsum dolor\\\\
 \\end{verse}"))))
 
 (ert-deftest test-ox-latex/longtable ()
@@ -98,15 +98,15 @@ (ert-deftest test-ox-latex/longtable ()
     (should
      (search-forward
       "\\begin{longtable}{lr}
-First & Second\\\\[0pt]
-Column & Column\\\\[0pt]
+First & Second\\\\
+Column & Column\\\\
 \\hline
 \\endfirsthead"))
     (goto-char (point-min))
     (should
      (search-forward
-      "First & Second\\\\[0pt]
-Column & Column \\\\[0pt]
+      "First & Second\\\\
+Column & Column \\\\
 
 \\hline
 \\endhead"))
-- 
2.43.0


[-- Attachment #4: Type: text/plain, Size: 224 bytes --]


-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>