patch: add custom latex->html conversion command

patch: add custom latex->html conversion command

[Matt Huszagh]

[-- Attachment #1: Type: text/plain, Size: 552 bytes --]

Hi all,

The patch below allow's you to provide your own shell command the
generates HTML from a latex fragment and places the output directly in
the exported HTML file.

I've added this feature to be able to use latexml for processing latex
fragments to HTML. Unfortunately, mathjax can't process macros from the
siunitx package, but latexml can. Here's how I use it

  (setq org-html-with-latex 'html)
  (setq org-latex-to-html-convert-command "latexmlc 'literal:%i' --profile=math --preload=siunitx.sty 2>/dev/null")

Any/all feedback appreciated!


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-add-custom-command-option-when-converting-latex-frag.patch --]
[-- Type: text/x-patch, Size: 4807 bytes --]

From 056d23d9e5caa6fc22907014e0128519fcc84b6e Mon Sep 17 00:00:00 2001
From: Matt Huszagh <huszaghmatt@gmail.com>
Date: Sat, 15 Feb 2020 18:42:11 -0800
Subject: [PATCH] add custom command option when converting latex fragments to
 html

This allows you to set a custom command
`org-latex-to-html-convert-command' that will take as input a latex
fragment and use it to generate html for export. This is very
open-ended in the sense that you can use any shell-command you want. I
envisioned this for use with latexml, but there's nothing preventing
you from using something else.
---
 lisp/org.el     | 30 ++++++++++++++++++++++++++++++
 lisp/ox-html.el |  9 +++++++--
 2 files changed, 37 insertions(+), 2 deletions(-)

diff --git a/lisp/org.el b/lisp/org.el
index 97ce7ec43..94557bf86 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -3203,6 +3203,22 @@ When using LaTeXML set this option to
 	  (const :tag "None" nil)
 	  (string :tag "\nShell command")))
 
+(defcustom org-latex-to-html-convert-command nil
+  "Command to convert LaTeX fragments to HTML.
+This command is very open-ended: the output of the command will
+directly replace the latex fragment in the resulting HTML.
+Replace format-specifiers in the command as noted below and use
+`shell-command' to convert LaTeX to HTML.
+%i:     The latex fragment to be converted.
+
+For example, this could be used with LaTeXML as
+\"latexmlc 'literal:%i' --profile=math --preload=siunitx.sty 2>/dev/null\"."
+  :group 'org-latex
+  :version "26.1"
+  :type '(choice
+	  (const :tag "None" nil)
+	  (string :tag "\nShell command")))
+
 (defcustom org-preview-latex-default-process 'dvipng
   "The default process to convert LaTeX fragments to image files.
 All available processes and theirs documents can be found in
@@ -15613,6 +15629,10 @@ Some of the options can be changed using the variable
 		    (if (string= (match-string 0 value) "$$")
 			(insert "\\[" (substring value 2 -2) "\\]")
 		      (insert "\\(" (substring value 1 -1) "\\)"))))
+		 ((eq processing-type 'html)
+		  (goto-char beg)
+		  (delete-region beg end)
+		  (insert (org-format-latex-as-html value)))
 		 ((assq processing-type org-preview-latex-process-alist)
 		  ;; Process to an image.
 		  (cl-incf cnt)
@@ -15778,6 +15798,16 @@ inspection."
       ;; Failed conversion.  Return the LaTeX fragment verbatim
       latex-frag)))
 
+(defun org-format-latex-as-html (latex-frag)
+  "Convert latex to html with a custom conversion command.
+`LATEX-FRAG' is the latex fragment
+Set the custom command with `org-latex-to-html-convert-command'."
+  (let ((cmd (format-spec
+	      org-latex-to-html-convert-command
+	      `((?i . ,latex-frag)))))
+    (message "Running %s" cmd)
+    (setq shell-command-output (shell-command-to-string cmd))))
+
 (defun org--get-display-dpi ()
   "Get the DPI of the display.
 The function assumes that the display has the same pixel width in
diff --git a/lisp/ox-html.el b/lisp/ox-html.el
index ea6aa63c3..b5cbf4cfc 100644
--- a/lisp/ox-html.el
+++ b/lisp/ox-html.el
@@ -776,6 +776,8 @@ e.g. \"tex:mathjax\".  Allowed values are:
   `verbatim'    Keep everything in verbatim
   `mathjax', t  Do MathJax preprocessing and arrange for MathJax.js to
                 be loaded.
+  `html'        Use `org-latex-to-html-convert-command' to convert
+                LaTeX fragments to HTML.
   SYMBOL        Any symbol defined in `org-preview-latex-process-alist',
                 e.g., `dvipng'."
   :group 'org-export-html
@@ -2776,12 +2778,13 @@ CONTENTS is nil.  INFO is a plist holding contextual information."
 (defun org-html-format-latex (latex-frag processing-type info)
   "Format a LaTeX fragment LATEX-FRAG into HTML.
 PROCESSING-TYPE designates the tool used for conversion.  It can
-be `mathjax', `verbatim', nil, t or symbols in
+be `mathjax', `verbatim', `html', nil, t or symbols in
 `org-preview-latex-process-alist', e.g., `dvipng', `dvisvgm' or
 `imagemagick'.  See `org-html-with-latex' for more information.
 INFO is a plist containing export properties."
   (let ((cache-relpath "") (cache-dir ""))
-    (unless (eq processing-type 'mathjax)
+    (unless (or (eq processing-type 'mathjax)
+                (eq processing-type 'html))
       (let ((bfn (or (buffer-file-name)
 		     (make-temp-name
 		      (expand-file-name "latex" temporary-file-directory))))
@@ -2895,6 +2898,8 @@ CONTENTS is nil.  INFO is a plist holding contextual information."
     (cond
      ((memq processing-type '(t mathjax))
       (org-html-format-latex latex-frag 'mathjax info))
+     ((memq processing-type '(t html))
+      (org-html-format-latex latex-frag 'html info))
      ((assq processing-type org-preview-latex-process-alist)
       (let ((formula-link
 	     (org-html-format-latex latex-frag processing-type info)))
-- 
2.25.0

Re: patch: add custom latex->html conversion command

[Bastien]

Hi Matt,

Matt Huszagh <huszaghmatt@gmail.com> writes:

> The patch below allow's you to provide your own shell command the
> generates HTML from a latex fragment and places the output directly in
> the exported HTML file.

Can you get the same output by configuring a new symbol in
`org-preview-latex-process-alist' then setting `org-html-with-latex'
to this new symbol?

I've not tested it, so perhaps it does not work.

But any patch about adding latexmlc support should surely look in this
direction first.

HTH,

-- 
 Bastien

Re: patch: add custom latex->html conversion command

[Matt Huszagh]

[-- Attachment #1: Type: text/plain, Size: 949 bytes --]

Bastien,

Thanks for the reply. I’ll look into it. However, I’d initially ruled that
out because the final output is meant to be an image. So, to get just text
insertion into the html output I imagine would be a bit of a hack. For
instance what to set image-converter to etc.

Matt

On Sun, Feb 16, 2020 at 4:06 PM Bastien <bzg@gnu.org> wrote:

> Hi Matt,
>
> Matt Huszagh <huszaghmatt@gmail.com> writes:
>
> > The patch below allow's you to provide your own shell command the
> > generates HTML from a latex fragment and places the output directly in
> > the exported HTML file.
>
> Can you get the same output by configuring a new symbol in
> `org-preview-latex-process-alist' then setting `org-html-with-latex'
> to this new symbol?
>
> I've not tested it, so perhaps it does not work.
>
> But any patch about adding latexmlc support should surely look in this
> direction first.
>
> HTH,
>
> --
>  Bastien
>

[-- Attachment #2: Type: text/html, Size: 1443 bytes --]

Re: patch: add custom latex->html conversion command

[Bastien]

Matt Huszagh <huszaghmatt@gmail.com> writes:

> Thanks for the reply. I’ll look into it. However, I’d initially ruled
> that out because the final output is meant to be an image. So, to get
> just text insertion into the html output I imagine would be a bit of
> a hack. For instance what to set image-converter to etc.

Oh, I misunderstood what latexml does, it looks nice.  I'll comment on
your patch.

In the meantime, can you fill this form?
https://orgmode.org/request-assign-future.txt

You'll need this to contribute this patch, which will surely make its
way after 9.4, but still.

Thanks,

-- 
 Bastien

Re: patch: add custom latex->html conversion command

[Bastien]

Hi Matt,

Matt Huszagh <huszaghmatt@gmail.com> writes:

> From 056d23d9e5caa6fc22907014e0128519fcc84b6e Mon Sep 17 00:00:00 2001
> From: Matt Huszagh <huszaghmatt@gmail.com>
> Date: Sat, 15 Feb 2020 18:42:11 -0800
> Subject: [PATCH] add custom command option when converting latex fragments to
>  html
>
> This allows you to set a custom command
> `org-latex-to-html-convert-command' that will take as input a latex
> fragment and use it to generate html for export. This is very
> open-ended in the sense that you can use any shell-command you want. I
> envisioned this for use with latexml, but there's nothing preventing
> you from using something else.

The commit message needs to be formatted using the Emacs changelog
format.  See `add-change-log-entry-other-window' on how to add a
changelog entry from a patch (or from magit's diff view).

> ---
>  lisp/org.el     | 30 ++++++++++++++++++++++++++++++
>  lisp/ox-html.el |  9 +++++++--
>  2 files changed, 37 insertions(+), 2 deletions(-)
>
> diff --git a/lisp/org.el b/lisp/org.el
> index 97ce7ec43..94557bf86 100644
> --- a/lisp/org.el
> +++ b/lisp/org.el
> @@ -3203,6 +3203,22 @@ When using LaTeXML set this option to
>  	  (const :tag "None" nil)
>  	  (string :tag "\nShell command")))
>  
> +(defcustom org-latex-to-html-convert-command nil
> +  "Command to convert LaTeX fragments to HTML.
> +This command is very open-ended: the output of the command will
> +directly replace the latex fragment in the resulting HTML.
> +Replace format-specifiers in the command as noted below and use
> +`shell-command' to convert LaTeX to HTML.
> +%i:     The latex fragment to be converted.
> +
> +For example, this could be used with LaTeXML as
> +\"latexmlc 'literal:%i' --profile=math --preload=siunitx.sty 2>/dev/null\"."
> +  :group 'org-latex
> +  :version "26.1"

No need for :version "26.1" (which is false).

Better add :package-version '(Org . "9.5") for when this will be in 9.5.

> +  :type '(choice
> +	  (const :tag "None" nil)
> +	  (string :tag "\nShell command")))
                                        ^ missing value ?

> +(defun org-format-latex-as-html (latex-frag)
> +  "Convert latex to html with a custom conversion command.
> +`LATEX-FRAG' is the latex fragment
> +Set the custom command with `org-latex-to-html-convert-command'."
> +  (let ((cmd (format-spec
> +	      org-latex-to-html-convert-command
> +	      `((?i . ,latex-frag)))))
              ^ indentation looks weird.

Otherwise it looks good.  Thanks!

-- 
 Bastien

Re: patch: add custom latex->html conversion command

[Matt Huszagh]

[-- Attachment #1: Type: text/plain, Size: 598 bytes --]

Thanks for the feedback. I've filled out the form you sent and sent it
to the email listed.

Adjusted patch below. I didn't add a value to the customization. I'm
honestly not very familiar with the customize interface since I never
use it. Is that a default value if the user selects string from the
customize interface? If it is I think it's probably best to leave it
blank since this is really meant to be open-ended. For instance, I doubt
the example I give would work on windows, so that might be more
confusing than helpful in that case. What do you think? Happy to set it
to whatever.

Matt


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-add-custom-command-option-when-converting-latex-frag.patch --]
[-- Type: text/x-patch, Size: 5454 bytes --]

From 6b2495c8aef0b67fd00ad27a0056e79f42c23c06 Mon Sep 17 00:00:00 2001
From: Matt Huszagh <huszaghmatt@gmail.com>
Date: Sun, 16 Feb 2020 16:52:02 -0800
Subject: [PATCH] add custom command option when converting latex fragments to
 html

* lisp/org.el (org-latex-to-html-convert-command): Add custom command
option to convert a latex fragment directly into HTML.
(org-format-latex): Add condition for this command to org-format-latex.
(org-format-latex-as-html): Command that ultimately does the
conversion work. It uses the user-specified command and applies to
the latex fragment. It then returns the resulting HTML.
* lisp/ox-html.el (org-html-with-latex): Document 'html symbol.
(org-html-format-latex): This custom HTML conversion, like mathjax,
doesn't require preprocessing.
(org-html-latex-fragment): Add condition to org-html-latex-fragment
for html symbol.

This allows you to set a custom command
`org-latex-to-html-convert-command' that will take as input a latex
fragment and use it to generate html for export. This is very
open-ended in the sense that you can use any shell-command you
want. I've added the ability in order to use latexml, but you could
use any other tool that generates HTML output text.
---
 lisp/org.el     | 29 +++++++++++++++++++++++++++++
 lisp/ox-html.el |  9 +++++++--
 2 files changed, 36 insertions(+), 2 deletions(-)

diff --git a/lisp/org.el b/lisp/org.el
index 97ce7ec43..7cc9e8687 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -3203,6 +3203,22 @@ When using LaTeXML set this option to
 	  (const :tag "None" nil)
 	  (string :tag "\nShell command")))
 
+(defcustom org-latex-to-html-convert-command nil
+  "Command to convert LaTeX fragments to HTML.
+This command is very open-ended: the output of the command will
+directly replace the latex fragment in the resulting HTML.
+Replace format-specifiers in the command as noted below and use
+`shell-command' to convert LaTeX to HTML.
+%i:     The latex fragment to be converted.
+
+For example, this could be used with LaTeXML as
+\"latexmlc 'literal:%i' --profile=math --preload=siunitx.sty 2>/dev/null\"."
+  :group 'org-latex
+  :package-version '(Org . "9.5")
+  :type '(choice
+	  (const :tag "None" nil)
+	  (string :tag "\nShell command")))
+
 (defcustom org-preview-latex-default-process 'dvipng
   "The default process to convert LaTeX fragments to image files.
 All available processes and theirs documents can be found in
@@ -15613,6 +15629,10 @@ Some of the options can be changed using the variable
 		    (if (string= (match-string 0 value) "$$")
 			(insert "\\[" (substring value 2 -2) "\\]")
 		      (insert "\\(" (substring value 1 -1) "\\)"))))
+		 ((eq processing-type 'html)
+		  (goto-char beg)
+		  (delete-region beg end)
+		  (insert (org-format-latex-as-html value)))
 		 ((assq processing-type org-preview-latex-process-alist)
 		  ;; Process to an image.
 		  (cl-incf cnt)
@@ -15778,6 +15798,15 @@ inspection."
       ;; Failed conversion.  Return the LaTeX fragment verbatim
       latex-frag)))
 
+(defun org-format-latex-as-html (latex-frag)
+  "Convert latex to html with a custom conversion command.
+`LATEX-FRAG' is the latex fragment
+Set the custom command with `org-latex-to-html-convert-command'."
+  (let ((cmd (format-spec org-latex-to-html-convert-command
+			  `((?i . ,latex-frag)))))
+    (message "Running %s" cmd)
+    (setq shell-command-output (shell-command-to-string cmd))))
+
 (defun org--get-display-dpi ()
   "Get the DPI of the display.
 The function assumes that the display has the same pixel width in
diff --git a/lisp/ox-html.el b/lisp/ox-html.el
index ea6aa63c3..b5cbf4cfc 100644
--- a/lisp/ox-html.el
+++ b/lisp/ox-html.el
@@ -776,6 +776,8 @@ e.g. \"tex:mathjax\".  Allowed values are:
   `verbatim'    Keep everything in verbatim
   `mathjax', t  Do MathJax preprocessing and arrange for MathJax.js to
                 be loaded.
+  `html'        Use `org-latex-to-html-convert-command' to convert
+                LaTeX fragments to HTML.
   SYMBOL        Any symbol defined in `org-preview-latex-process-alist',
                 e.g., `dvipng'."
   :group 'org-export-html
@@ -2776,12 +2778,13 @@ CONTENTS is nil.  INFO is a plist holding contextual information."
 (defun org-html-format-latex (latex-frag processing-type info)
   "Format a LaTeX fragment LATEX-FRAG into HTML.
 PROCESSING-TYPE designates the tool used for conversion.  It can
-be `mathjax', `verbatim', nil, t or symbols in
+be `mathjax', `verbatim', `html', nil, t or symbols in
 `org-preview-latex-process-alist', e.g., `dvipng', `dvisvgm' or
 `imagemagick'.  See `org-html-with-latex' for more information.
 INFO is a plist containing export properties."
   (let ((cache-relpath "") (cache-dir ""))
-    (unless (eq processing-type 'mathjax)
+    (unless (or (eq processing-type 'mathjax)
+                (eq processing-type 'html))
       (let ((bfn (or (buffer-file-name)
 		     (make-temp-name
 		      (expand-file-name "latex" temporary-file-directory))))
@@ -2895,6 +2898,8 @@ CONTENTS is nil.  INFO is a plist holding contextual information."
     (cond
      ((memq processing-type '(t mathjax))
       (org-html-format-latex latex-frag 'mathjax info))
+     ((memq processing-type '(t html))
+      (org-html-format-latex latex-frag 'html info))
      ((assq processing-type org-preview-latex-process-alist)
       (let ((formula-link
 	     (org-html-format-latex latex-frag processing-type info)))
-- 
2.25.0

Re: patch: add custom latex->html conversion command

[Bastien]

Hi Matt,

Matt Huszagh <huszaghmatt@gmail.com> writes:

> Thanks for the feedback. I've filled out the form you sent and sent it
> to the email listed.

Thanks!  It looks good.

Let me know when you get the answer from the FSF.

If you don't in four weeks, please ping them again cc'ing me.

-- 
 Bastien

Re: patch: add custom latex->html conversion command

[Matt Huszagh]

[-- Attachment #1: Type: text/plain, Size: 410 bytes --]

Will do, thanks!

On Sun, Feb 16, 2020 at 5:10 PM Bastien <bzg@gnu.org> wrote:

> Hi Matt,
>
> Matt Huszagh <huszaghmatt@gmail.com> writes:
>
> > Thanks for the feedback. I've filled out the form you sent and sent it
> > to the email listed.
>
> Thanks!  It looks good.
>
> Let me know when you get the answer from the FSF.
>
> If you don't in four weeks, please ping them again cc'ing me.
>
> --
>  Bastien
>

[-- Attachment #2: Type: text/html, Size: 798 bytes --]

Re: patch: add custom latex->html conversion command

[Bastien]

Hi Matt,

Matt Huszagh <huszaghmatt@gmail.com> writes:

> Thanks for the feedback. I've filled out the form you sent and sent it
> to the email listed.

this is now in master, thanks.

-- 
 Bastien