all messages for Emacs-related lists mirrored at yhetil.org
 help / color / mirror / code / Atom feed
From: Max Nikulin <manikulin@gmail.com>
To: emacs-orgmode@gnu.org
Subject: [PATCH] Unit tests for function calling MathML converters (Re: Warn about shell-expansion in the docstring of org-latex-to-html-convert-command)
Date: Sun, 10 Mar 2024 12:02:29 +0700	[thread overview]
Message-ID: <usjet6$15r7$1@ciao.gmane.io> (raw)
In-Reply-To: <ushuu6$hpn$1@ciao.gmane.io>

[-- Attachment #1: Type: text/plain, Size: 692 bytes --]

On 09/03/2024 22:23, Max Nikulin wrote:
> On 08/03/2024 18:16, Ihor Radchenko wrote:
> 
>> Attaching tentative patch that fixes the problem.

I propose to add unit tests, see first attachment.

> I have tried to add some unit tests, but I faced an issue with 
> `org-create-math-formula'. It creates temporary files in 
> `default-directory' and does not remove them on failure. Moreover, it 
> does not work in a container where git is not installed:
> 
> Debugger entered--Lisp error: (file-missing "Searching for program" "No 
> such file or directory" "git")
> 
> that is called from `find-file-hook'.

Is second attachment appropriate to fix the issue or it has some 
undesired effects.

[-- Attachment #2: 0002-test-org.el-LaTeX-to-MathML-tests-of-shell-escaping.patch --]
[-- Type: text/x-patch, Size: 4128 bytes --]

From 73541dbeafff47f03d4aa2f47da70ba71d5b8253 Mon Sep 17 00:00:00 2001
From: Max Nikulin <manikulin@gmail.com>
Date: Sun, 10 Mar 2024 11:16:46 +0700
Subject: [PATCH 2/3] test-org.el: LaTeX to MathML tests of shell escaping

* testing/lisp/test-org.el (test-org/format-latex-as-html)
(test-org/create-math-formula): New tests for escaping of shell specials
in commands executed by `org-format-latex-as-html'
and `org-create-math-formula'.

These tests do not require applications for conversion of LaTeX
snippets and use simple shell commands instead.
---
 testing/lisp/test-org.el | 75 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 75 insertions(+)

diff --git a/testing/lisp/test-org.el b/testing/lisp/test-org.el
index cecf9b412..652fc0676 100644
--- a/testing/lisp/test-org.el
+++ b/testing/lisp/test-org.el
@@ -9590,6 +9590,81 @@ (ert-deftest test-org/org--open-file-format-command ()
                                (string-match-p "\\`Invalid format.*%2" err-text))
                     err)))))
 
+\f
+;;; LaTeX-related functions.
+
+(ert-deftest test-org/format-latex-as-html ()
+  "Test shell special characters escaping in `org-format-latex-as-html'."
+  (let ((org-latex-to-html-convert-command
+         "printf \"<I%%sI>\" %i"))
+    ;; No backslashes added by `shell-quote-argumet'
+    ;; are leaked to command arguments.  See e.g. dash(1) "Double Quotes":
+    ;;
+    ;;     The backslash inside double quotes is historically weird,
+    ;;     and serves to quote only the following characters:
+    ;;         $ ` " \ <newline>.
+    ;;     Otherwise it remains literal.
+    ;;
+    ;; Actually extra backslashes may appear if a user adds
+    ;; double quotes around "%i", however it is not subject
+    ;; of this test.
+    (should
+     (equal "<I(|)`[[\\]]{}#$'!I>"
+            (org-format-latex-as-html "(|)`[[\\]]{}#$'!")))
+    ;; The following tests generate shell errors if %i
+    ;; substitution is not passed throuhg `shell-quote-argument'.
+    ;; Multiple words.
+    (should
+     (equal "<Iwords ; |I>"
+                   (org-format-latex-as-html "words ; |")))
+    ;; Bypass single quote.
+    ;; The same snippet causes shell error if '%i' is wrapped
+    ;; in single quotes in user configuration.
+    (should
+     (equal "<Iapostrophe' ; |I>"
+            (org-format-latex-as-html "apostrophe' ; |")))
+    ;; Bypass double quote.
+    ;; Double quotes around "%i" in user configuration leads
+    ;; to extra backslashes (see above), however likely
+    ;; such user error can not cause execution of the argument
+    ;; as raw shell commands.
+    (should
+     (equal "<Iquote\" ; |I>"
+            (org-format-latex-as-html "quote\" ; |")))))
+
+(defun test-org/extract-mathml-math (xml)
+  "Extract body from result of `org-create-math-formula'."
+  (and (string-match "<math[^>]*>\\(\\(?:.\\|\n\\)*\\)</math>" xml)
+       (match-string 1 xml)))
+
+(ert-deftest test-org/create-math-formula ()
+  "Test shell special characters escaping in `org-create-math-formula'."
+  ;; The function requires <math>...</math> elements.
+  (let ((org-latex-to-mathml-convert-command
+         "printf \"<math xmlns=\\\"http://www.w3.org/1998/Math/MathML\\\"><I%%sI></math>\" %i >%o"))
+    ;; See comments in `test-org/format-latex-as-html'.
+    ;;
+    ;; No backslashes added by `shell-quote-argumet'
+    ;; are leaked to command arguments.
+    (should (equal "<I(|)`[[\\]]{}#$'!I>"
+            (test-org/extract-mathml-math
+             (org-create-math-formula "(|)`[[\\]]{}#$'!"))))
+    ;; Multiple words.
+    (should
+     (equal "<Iwords ; |I>"
+            (test-org/extract-mathml-math
+             (org-create-math-formula "words ; |"))))
+    ;; Bypass single quote.
+    (should
+     (equal "<Iapostrophe' ; |I>"
+            (test-org/extract-mathml-math
+             (org-create-math-formula "apostrophe' ; |"))))
+    ;; Bypass double quote.
+    (should
+     (equal "<Iquote\" ; |I>"
+            (test-org/extract-mathml-math
+             (org-create-math-formula "quote\" ; |"))))))
+
 (provide 'test-org)
 
 ;;; test-org.el ends here
-- 
2.39.2


[-- Attachment #3: 0003-org.el-Avoid-find-file-no-select-during-MathML-gener.patch --]
[-- Type: text/x-patch, Size: 1668 bytes --]

From cb5d20b54349dabea25241072feca4822ae0e77d Mon Sep 17 00:00:00 2001
From: Max Nikulin <manikulin@gmail.com>
Date: Sun, 10 Mar 2024 11:22:15 +0700
Subject: [PATCH 3/3] org.el: Avoid `find-file-no-select' during MathML
 generation

* lisp/org.el (org-create-math-formula): Bypass `find-file-hook' during
reading MathML files.

At least in Emacs-28 calling `find-file-noselect' for a file in a
directory under git control when git is not installed causes failures:

    Lisp error:
    (file-missing "Searching for program" "No such file or directory" "git")

`org-create-math-formula' uses document `default-directory' for
temporary files for conversion process input and output.
I have faced an issue with Org mode unit tests running in a minimal
container with no git installed.
---
 lisp/org.el | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/lisp/org.el b/lisp/org.el
index a00d50c51..caf1bfa67 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -16215,14 +16215,15 @@ (defun org-create-math-formula (latex-frag &optional mathml-file)
     (setq shell-command-output (shell-command-to-string cmd))
     (setq mathml
 	  (when (file-readable-p tmp-out-file)
-	    (with-current-buffer (find-file-noselect tmp-out-file t)
+	    (with-temp-buffer
+              (insert-file-contents tmp-out-file)
 	      (goto-char (point-min))
 	      (when (re-search-forward
 		     (format "<math[^>]*?%s[^>]*?>\\(.\\|\n\\)*</math>"
 			     (regexp-quote
 			      "xmlns=\"http://www.w3.org/1998/Math/MathML\""))
 		     nil t)
-		(prog1 (match-string 0) (kill-buffer))))))
+		(match-string 0)))))
     (cond
      (mathml
       (setq mathml
-- 
2.39.2


  reply	other threads:[~2024-03-10  5:03 UTC|newest]

Thread overview: 33+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-02-16 23:10 Warn about shell-expansion in the docstring of org-latex-to-html-convert-command Martin Edström
2024-02-18 16:06 ` Ihor Radchenko
2024-02-18 18:56   ` Martin Edström
2024-02-18 19:36     ` Martin Edström
2024-02-19  8:30       ` Ihor Radchenko
2024-02-21 14:38       ` Max Nikulin
2024-02-21 14:57         ` Martin Edström
2024-02-21 15:04         ` Martin Edström
2024-02-21 15:08           ` Martin Edström
2024-02-23 12:46         ` Ihor Radchenko
2024-02-25 10:41           ` Max Nikulin
2024-02-26 10:48             ` Ihor Radchenko
2024-02-26 16:37               ` Max Nikulin
2024-03-08 11:16                 ` Ihor Radchenko
2024-03-09 15:23                   ` Max Nikulin
2024-03-10  5:02                     ` Max Nikulin [this message]
2024-03-31  8:27                       ` [PATCH] Unit tests for function calling MathML converters (Re: Warn about shell-expansion in the docstring of org-latex-to-html-convert-command) Ihor Radchenko
2024-04-01 10:39                         ` Max Nikulin
2024-04-01 11:23                           ` Ihor Radchenko
2024-03-12 13:03                     ` Warn about shell-expansion in the docstring of org-latex-to-html-convert-command Ihor Radchenko
2024-03-13 14:27                       ` Max Nikulin
2024-03-15 13:49                         ` Ihor Radchenko
2024-03-18 10:50                           ` Max Nikulin
2024-03-19 14:48                             ` Ihor Radchenko
2024-03-19 14:49                               ` Ihor Radchenko
2024-03-19 16:22                                 ` Max Nikulin
2024-03-19 16:27                                   ` Ihor Radchenko
2024-03-19 16:45                                     ` fixup! and git Max Nikulin
2024-03-19 16:50                                       ` Ihor Radchenko
2024-03-31  8:25                     ` Warn about shell-expansion in the docstring of org-latex-to-html-convert-command Ihor Radchenko
2024-04-01 10:29                       ` Max Nikulin
2024-04-01 11:15                         ` Ihor Radchenko
2024-03-05 12:01             ` Max Nikulin

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='usjet6$15r7$1@ciao.gmane.io' \
    --to=manikulin@gmail.com \
    --cc=emacs-orgmode@gnu.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this external index

	https://git.savannah.gnu.org/cgit/emacs.git
	https://git.savannah.gnu.org/cgit/emacs/org-mode.git

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.