[PATCH] Unit tests for function calling MathML converters (Re: Warn about shell-expansion in the docstring of org-latex-to-html-convert-command)

[Max Nikulin <manikulin@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 692 bytes --]

On 09/03/2024 22:23, Max Nikulin wrote:
> On 08/03/2024 18:16, Ihor Radchenko wrote:
> 
>> Attaching tentative patch that fixes the problem.

I propose to add unit tests, see first attachment.

> I have tried to add some unit tests, but I faced an issue with 
> `org-create-math-formula'. It creates temporary files in 
> `default-directory' and does not remove them on failure. Moreover, it 
> does not work in a container where git is not installed:
> 
> Debugger entered--Lisp error: (file-missing "Searching for program" "No 
> such file or directory" "git")
> 
> that is called from `find-file-hook'.

Is second attachment appropriate to fix the issue or it has some 
undesired effects.

[-- Attachment #2: 0002-test-org.el-LaTeX-to-MathML-tests-of-shell-escaping.patch --]
[-- Type: text/x-patch, Size: 4128 bytes --]

From 73541dbeafff47f03d4aa2f47da70ba71d5b8253 Mon Sep 17 00:00:00 2001
From: Max Nikulin <manikulin@gmail.com>
Date: Sun, 10 Mar 2024 11:16:46 +0700
Subject: [PATCH 2/3] test-org.el: LaTeX to MathML tests of shell escaping

* testing/lisp/test-org.el (test-org/format-latex-as-html)
(test-org/create-math-formula): New tests for escaping of shell specials
in commands executed by `org-format-latex-as-html'
and `org-create-math-formula'.

These tests do not require applications for conversion of LaTeX
snippets and use simple shell commands instead.
---
 testing/lisp/test-org.el | 75 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 75 insertions(+)

diff --git a/testing/lisp/test-org.el b/testing/lisp/test-org.el
index cecf9b412..652fc0676 100644
--- a/testing/lisp/test-org.el
+++ b/testing/lisp/test-org.el
@@ -9590,6 +9590,81 @@ (ert-deftest test-org/org--open-file-format-command ()
                                (string-match-p "\\`Invalid format.*%2" err-text))
                     err)))))
 
+\f
+;;; LaTeX-related functions.
+
+(ert-deftest test-org/format-latex-as-html ()
+  "Test shell special characters escaping in `org-format-latex-as-html'."
+  (let ((org-latex-to-html-convert-command
+         "printf \"<I%%sI>\" %i"))
+    ;; No backslashes added by `shell-quote-argumet'
+    ;; are leaked to command arguments.  See e.g. dash(1) "Double Quotes":
+    ;;
+    ;;     The backslash inside double quotes is historically weird,
+    ;;     and serves to quote only the following characters:
+    ;;         $ ` " \ <newline>.
+    ;;     Otherwise it remains literal.
+    ;;
+    ;; Actually extra backslashes may appear if a user adds
+    ;; double quotes around "%i", however it is not subject
+    ;; of this test.
+    (should
+     (equal "<I(|)`[[\\]]{}#$'!I>"
+            (org-format-latex-as-html "(|)`[[\\]]{}#$'!")))
+    ;; The following tests generate shell errors if %i
+    ;; substitution is not passed throuhg `shell-quote-argument'.
+    ;; Multiple words.
+    (should
+     (equal "<Iwords ; |I>"
+                   (org-format-latex-as-html "words ; |")))
+    ;; Bypass single quote.
+    ;; The same snippet causes shell error if '%i' is wrapped
+    ;; in single quotes in user configuration.
+    (should
+     (equal "<Iapostrophe' ; |I>"
+            (org-format-latex-as-html "apostrophe' ; |")))
+    ;; Bypass double quote.
+    ;; Double quotes around "%i" in user configuration leads
+    ;; to extra backslashes (see above), however likely
+    ;; such user error can not cause execution of the argument
+    ;; as raw shell commands.
+    (should
+     (equal "<Iquote\" ; |I>"
+            (org-format-latex-as-html "quote\" ; |")))))
+
+(defun test-org/extract-mathml-math (xml)
+  "Extract body from result of `org-create-math-formula'."
+  (and (string-match "<math[^>]*>\\(\\(?:.\\|\n\\)*\\)</math>" xml)
+       (match-string 1 xml)))
+
+(ert-deftest test-org/create-math-formula ()
+  "Test shell special characters escaping in `org-create-math-formula'."
+  ;; The function requires <math>...</math> elements.
+  (let ((org-latex-to-mathml-convert-command
+         "printf \"<math xmlns=\\\"http://www.w3.org/1998/Math/MathML\\\"><I%%sI></math>\" %i >%o"))
+    ;; See comments in `test-org/format-latex-as-html'.
+    ;;
+    ;; No backslashes added by `shell-quote-argumet'
+    ;; are leaked to command arguments.
+    (should (equal "<I(|)`[[\\]]{}#$'!I>"
+            (test-org/extract-mathml-math
+             (org-create-math-formula "(|)`[[\\]]{}#$'!"))))
+    ;; Multiple words.
+    (should
+     (equal "<Iwords ; |I>"
+            (test-org/extract-mathml-math
+             (org-create-math-formula "words ; |"))))
+    ;; Bypass single quote.
+    (should
+     (equal "<Iapostrophe' ; |I>"
+            (test-org/extract-mathml-math
+             (org-create-math-formula "apostrophe' ; |"))))
+    ;; Bypass double quote.
+    (should
+     (equal "<Iquote\" ; |I>"
+            (test-org/extract-mathml-math
+             (org-create-math-formula "quote\" ; |"))))))
+
 (provide 'test-org)
 
 ;;; test-org.el ends here
-- 
2.39.2


[-- Attachment #3: 0003-org.el-Avoid-find-file-no-select-during-MathML-gener.patch --]
[-- Type: text/x-patch, Size: 1668 bytes --]

From cb5d20b54349dabea25241072feca4822ae0e77d Mon Sep 17 00:00:00 2001
From: Max Nikulin <manikulin@gmail.com>
Date: Sun, 10 Mar 2024 11:22:15 +0700
Subject: [PATCH 3/3] org.el: Avoid `find-file-no-select' during MathML
 generation

* lisp/org.el (org-create-math-formula): Bypass `find-file-hook' during
reading MathML files.

At least in Emacs-28 calling `find-file-noselect' for a file in a
directory under git control when git is not installed causes failures:

    Lisp error:
    (file-missing "Searching for program" "No such file or directory" "git")

`org-create-math-formula' uses document `default-directory' for
temporary files for conversion process input and output.
I have faced an issue with Org mode unit tests running in a minimal
container with no git installed.
---
 lisp/org.el | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/lisp/org.el b/lisp/org.el
index a00d50c51..caf1bfa67 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -16215,14 +16215,15 @@ (defun org-create-math-formula (latex-frag &optional mathml-file)
     (setq shell-command-output (shell-command-to-string cmd))
     (setq mathml
 	  (when (file-readable-p tmp-out-file)
-	    (with-current-buffer (find-file-noselect tmp-out-file t)
+	    (with-temp-buffer
+              (insert-file-contents tmp-out-file)
 	      (goto-char (point-min))
 	      (when (re-search-forward
 		     (format "<math[^>]*?%s[^>]*?>\\(.\\|\n\\)*</math>"
 			     (regexp-quote
 			      "xmlns=\"http://www.w3.org/1998/Math/MathML\""))
 		     nil t)
-		(prog1 (match-string 0) (kill-buffer))))))
+		(match-string 0)))))
     (cond
      (mathml
       (setq mathml
-- 
2.39.2