bug#73641: 30.0.90; Math in EWW/shr

bug#73641: 30.0.90; Math in EWW/shr

[Augusto Stoffel]

shr has a tendency to display garbled math formulas.  For example
opening https://developer.mozilla.org/en-US/docs/Web/MathML/Examples/MathML_Pythagorean_Theorem
in EWW shows this:

> if a and b are the legs and c is the hypotenuse, then a2+b2=c2a^2 +
> b^2 = c^2.

This happens because <math> HTML elements tend to contain a MathML
description of the formula and possibly some "annotations", so what we
see above is "a2+b2=c2", the naive rendering of the MathML, concatenated
with "a^2 + b^2 = c^2" which is an annotation saying what the formula is
in TeX notation.

So in as a first improvement I would like to suggest adding this
definition:

  (defun shr-tag-math (dom)
    (shr-generic (thread-first
                   dom
                   (dom-child-by-tag 'semantics)
                   (dom-child-by-tag 'annotation)
                   (or dom))))

The above page then gets rendered like this:

> Specifically, if a and b are the legs and c is the hypotenuse, then
> a^2 + b^2 = c^2.

In other words, only the TeX annotation is displayed, which is as good
as it gets when trying to show math as plain text, IMO.

It is possible for the <math> element to include a SVG rendition of the
formula.  We could consider displaying that if available (not sure how
common it is, though).  Also, we might consider including infrastructure
to render MathML, but this would require external programs (there is no
reason to reinvent MathJax).  I've done that in one of my packages, see
this commit for the relevant code:

https://github.com/astoff/devdocs.el/commit/04a8793072f1fc7c23f115e2c7ee83ce669b6fa4

(Actually, I added a version of the above to two of my packages already,
so I think it's pretty clear Emacs should have some general
math-rendering infrastructure.)

bug#73641: 30.0.90; Math in EWW/shr

[Eli Zaretskii]

> From: Augusto Stoffel <arstoffel@gmail.com>
> Date: Sat, 05 Oct 2024 16:01:36 +0200
> 
> It is possible for the <math> element to include a SVG rendition of the
> formula.  We could consider displaying that if available (not sure how
> common it is, though).  Also, we might consider including infrastructure
> to render MathML, but this would require external programs (there is no
> reason to reinvent MathJax).  I've done that in one of my packages, see
> this commit for the relevant code:
> 
> https://github.com/astoff/devdocs.el/commit/04a8793072f1fc7c23f115e2c7ee83ce669b6fa4
> 
> (Actually, I added a version of the above to two of my packages already,
> so I think it's pretty clear Emacs should have some general
> math-rendering infrastructure.)

I agree that it would be good for Emacs to support math rendering, but
we need to find the best way of doing that, and then look for a
library or a program that implements that.  Maybe the first step would
be to conduct a survey of available methods and tools.

bug#73641: 30.0.90; Math in EWW/shr

[Augusto Stoffel]

On Sat,  5 Oct 2024 at 18:24, Eli Zaretskii wrote:

>> From: Augusto Stoffel <arstoffel@gmail.com>
>> Date: Sat, 05 Oct 2024 16:01:36 +0200
>> 
>> It is possible for the <math> element to include a SVG rendition of the
>> formula.  We could consider displaying that if available (not sure how
>> common it is, though).  Also, we might consider including infrastructure
>> to render MathML, but this would require external programs (there is no
>> reason to reinvent MathJax).  I've done that in one of my packages, see
>> this commit for the relevant code:
>> 
>> https://github.com/astoff/devdocs.el/commit/04a8793072f1fc7c23f115e2c7ee83ce669b6fa4
>> 
>> (Actually, I added a version of the above to two of my packages already,
>> so I think it's pretty clear Emacs should have some general
>> math-rendering infrastructure.)
>
> I agree that it would be good for Emacs to support math rendering, but
> we need to find the best way of doing that, and then look for a
> library or a program that implements that.  Maybe the first step would
> be to conduct a survey of available methods and tools.

Sure, but my suggestion above for a "baseline" implementation of the
math tag in shr makes sense anyway IMO.  Should I prepare a patch?

Now, about your question, I think there aren't too many options:

- Good old TeX: widely available among (and only among) scientific
  people, gigantic and bloated and slow, no MathML support, excellent at
  rendering TeX, has "shell escape" to worry about.

- LuaTeX: like TeX, but probably can do MathML as well.

- MathJax: needs JavaScript, otherwise can be bundled with Emacs (takes
  just 2MB), supports MathML, supports TeX notation but not arbitrary
  TeX packages, safe by default.

bug#73641: 30.0.90; Math in EWW/shr

[Eli Zaretskii]

> From: Augusto Stoffel <arstoffel@gmail.com>
> Cc: 73641@debbugs.gnu.org
> Date: Thu, 10 Oct 2024 08:41:32 +0200
> 
> On Sat,  5 Oct 2024 at 18:24, Eli Zaretskii wrote:
> 
> >> From: Augusto Stoffel <arstoffel@gmail.com>
> >> Date: Sat, 05 Oct 2024 16:01:36 +0200
> >> 
> >> It is possible for the <math> element to include a SVG rendition of the
> >> formula.  We could consider displaying that if available (not sure how
> >> common it is, though).  Also, we might consider including infrastructure
> >> to render MathML, but this would require external programs (there is no
> >> reason to reinvent MathJax).  I've done that in one of my packages, see
> >> this commit for the relevant code:
> >> 
> >> https://github.com/astoff/devdocs.el/commit/04a8793072f1fc7c23f115e2c7ee83ce669b6fa4
> >> 
> >> (Actually, I added a version of the above to two of my packages already,
> >> so I think it's pretty clear Emacs should have some general
> >> math-rendering infrastructure.)
> >
> > I agree that it would be good for Emacs to support math rendering, but
> > we need to find the best way of doing that, and then look for a
> > library or a program that implements that.  Maybe the first step would
> > be to conduct a survey of available methods and tools.
> 
> Sure, but my suggestion above for a "baseline" implementation of the
> math tag in shr makes sense anyway IMO.  Should I prepare a patch?

I don't think I understand what you are proposing.  What is the
"baseline" implementation for math tags? what will it entail and what
will it allow/enable?

> Now, about your question, I think there aren't too many options:
> 
> - Good old TeX: widely available among (and only among) scientific
>   people, gigantic and bloated and slow, no MathML support, excellent at
>   rendering TeX, has "shell escape" to worry about.
> 
> - LuaTeX: like TeX, but probably can do MathML as well.
> 
> - MathJax: needs JavaScript, otherwise can be bundled with Emacs (takes
>   just 2MB), supports MathML, supports TeX notation but not arbitrary
>   TeX packages, safe by default.

The basic question is what these produce as output, and how can Emacs
display what they produce?  The best alternative for Emacs is to have
a tool that can generate characters, which could then be rendered
using a suitable font.  The (distant) second best is a tool that
generates an image.

Thanks.

bug#73641: 30.0.90; Math in EWW/shr

[Augusto Stoffel]

On Thu, 10 Oct 2024 at 10:43, Eli Zaretskii wrote:

> I don't think I understand what you are proposing.  What is the
> "baseline" implementation for math tags? what will it entail and what
> will it allow/enable?

I refer precisely to the code from the bug's first message, which also
has an example of the result.

>> Now, about your question, I think there aren't too many options:
>> 
>> - Good old TeX: widely available among (and only among) scientific
>>   people, gigantic and bloated and slow, no MathML support, excellent at
>>   rendering TeX, has "shell escape" to worry about.
>> 
>> - LuaTeX: like TeX, but probably can do MathML as well.
>> 
>> - MathJax: needs JavaScript, otherwise can be bundled with Emacs (takes
>>   just 2MB), supports MathML, supports TeX notation but not arbitrary
>>   TeX packages, safe by default.
>
> The basic question is what these produce as output, and how can Emacs
> display what they produce?  The best alternative for Emacs is to have
> a tool that can generate characters, which could then be rendered
> using a suitable font.

MathJax is this tool.  It can render formulas as "HTML with complicated
styling" to resize and move around characters and symbols.  I don't
think Emacs could display such things in general.

>  The (distant) second best is a tool that generates an image.

I think an image with readable text beneath it is quite alright.

bug#73641: 30.0.90; Math in EWW/shr

[Eli Zaretskii]

> From: Augusto Stoffel <arstoffel@gmail.com>
> Cc: 73641@debbugs.gnu.org
> Date: Thu, 10 Oct 2024 12:49:10 +0200
> 
> On Thu, 10 Oct 2024 at 10:43, Eli Zaretskii wrote:
> 
> > I don't think I understand what you are proposing.  What is the
> > "baseline" implementation for math tags? what will it entail and what
> > will it allow/enable?
> 
> I refer precisely to the code from the bug's first message, which also
> has an example of the result.

Ah, okay.  That's fine, but please add some comment there that we fall
back to the text variant because we don't (yet) know how to do better.

> > The basic question is what these produce as output, and how can Emacs
> > display what they produce?  The best alternative for Emacs is to have
> > a tool that can generate characters, which could then be rendered
> > using a suitable font.
> 
> MathJax is this tool.  It can render formulas as "HTML with complicated
> styling" to resize and move around characters and symbols.  I don't
> think Emacs could display such things in general.

Do you know what is missing for us to be able to display that?

> >  The (distant) second best is a tool that generates an image.
> 
> I think an image with readable text beneath it is quite alright.

Display of inline images in Emacs has quite a few limitations that
make it less desirable for this purpose, IMO.

Thanks.

bug#73641: 30.0.90; Math in EWW/shr

[Jeremy Bryant via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Augusto Stoffel <arstoffel@gmail.com>
>> Cc: 73641@debbugs.gnu.org
>> Date: Thu, 10 Oct 2024 12:49:10 +0200
>> 
>> On Thu, 10 Oct 2024 at 10:43, Eli Zaretskii wrote:
>> 
>> > I don't think I understand what you are proposing.  What is the
>> > "baseline" implementation for math tags? what will it entail and what
>> > will it allow/enable?
>> 
>> I refer precisely to the code from the bug's first message, which also
>> has an example of the result.
>
> Ah, okay.  That's fine, but please add some comment there that we fall
> back to the text variant because we don't (yet) know how to do better.
>
>> > The basic question is what these produce as output, and how can Emacs
>> > display what they produce?  The best alternative for Emacs is to have
>> > a tool that can generate characters, which could then be rendered
>> > using a suitable font.
>> 
>> MathJax is this tool.  It can render formulas as "HTML with complicated
>> styling" to resize and move around characters and symbols.  I don't
>> think Emacs could display such things in general.
>
> Do you know what is missing for us to be able to display that?
>
>> >  The (distant) second best is a tool that generates an image.
>> 
>> I think an image with readable text beneath it is quite alright.
>
> Display of inline images in Emacs has quite a few limitations that
> make it less desirable for this purpose, IMO.
>
> Thanks.

As this is related to shr and displaying svg images, this reminds me of
a patch of Sacha earlier this year.

Perhaps Sacha you have a suggestion on this?

bug#73641: 30.0.90; Math in EWW/shr

[Augusto Stoffel]

[-- Attachment #1: Type: text/plain, Size: 318 bytes --]

On Thu, 10 Oct 2024 at 21:24, Jeremy Bryant wrote:

> As this is related to shr and displaying svg images, this reminds me of
> a patch of Sacha earlier this year.
>
> Perhaps Sacha you have a suggestion on this?

Independent of that, I've attached a patch containing the "baseline"
action that can be later refined.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-shr.el-Define-rendering-rule-for-math-tag.patch --]
[-- Type: text/x-patch, Size: 989 bytes --]

From 8b772c709eb8527fd8e7ebfab65980d64dfa2f02 Mon Sep 17 00:00:00 2001
From: Augusto Stoffel <arstoffel@gmail.com>
Date: Sat, 12 Oct 2024 16:20:47 +0200
Subject: [PATCH] shr.el: Define rendering rule for math tag

* lisp/net/shr.el (shr-tag-math): New function, see bug#73641.
---
 lisp/net/shr.el | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/lisp/net/shr.el b/lisp/net/shr.el
index f1062acf7e4..fff478a69c9 100644
--- a/lisp/net/shr.el
+++ b/lisp/net/shr.el
@@ -2264,6 +2264,15 @@ shr-tag-bdi
   (shr-generic dom)
   (insert ?\N{POP DIRECTIONAL ISOLATE}))
 
+(defun shr-tag-math (dom)
+  ;; Display an annotation (typically TeX notation) if available.
+  (shr-generic
+   (thread-first
+     dom
+     (dom-child-by-tag 'semantics)
+     (dom-child-by-tag 'annotation)
+     (or dom))))
+
 ;;; Outline Support
 (defun shr-outline-search (&optional bound move backward looking-at)
   "A function that can be used as `outline-search-function' for rendered html.
-- 
2.46.2

bug#73641: 30.0.90; Math in EWW/shr

[Eli Zaretskii]

> From: Augusto Stoffel <arstoffel@gmail.com>
> Cc: Eli Zaretskii <eliz@gnu.org>,  73641@debbugs.gnu.org,  Sacha Chua
>  <sacha@sachachua.com>
> Date: Sat, 12 Oct 2024 16:24:12 +0200
> 
> On Thu, 10 Oct 2024 at 21:24, Jeremy Bryant wrote:
> 
> > As this is related to shr and displaying svg images, this reminds me of
> > a patch of Sacha earlier this year.
> >
> > Perhaps Sacha you have a suggestion on this?
> 
> Independent of that, I've attached a patch containing the "baseline"
> action that can be later refined.

Thanks, but can we please have a comment here explaining why we do
that?

bug#73641: 30.0.90; Math in EWW/shr

[Augusto Stoffel]

[-- Attachment #1: Type: text/plain, Size: 147 bytes --]

On Sat, 12 Oct 2024 at 18:07, Eli Zaretskii wrote:

> Thanks, but can we please have a comment here explaining why we do
> that?

How about this?


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-shr.el-Define-rendering-rule-for-math-tag.patch --]
[-- Type: text/x-patch, Size: 1177 bytes --]

From 0d3f0ad219e9cd0c9f79113d137cab5bbdeed2e1 Mon Sep 17 00:00:00 2001
From: Augusto Stoffel <arstoffel@gmail.com>
Date: Sat, 12 Oct 2024 16:20:47 +0200
Subject: [PATCH] shr.el: Define rendering rule for math tag

* lisp/net/shr.el (shr-tag-math): New function, see bug#73641.
---
 lisp/net/shr.el | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/lisp/net/shr.el b/lisp/net/shr.el
index f1062acf7e4..0c37ef67bbb 100644
--- a/lisp/net/shr.el
+++ b/lisp/net/shr.el
@@ -2264,6 +2264,18 @@ shr-tag-bdi
   (shr-generic dom)
   (insert ?\N{POP DIRECTIONAL ISOLATE}))
 
+(defun shr-tag-math (dom)
+  ;; Sometimes a math element contains a plain text annotation
+  ;; (typically TeX notation) in addition to MathML markup.  If we pass
+  ;; that to `dom-generic', the formula is printed twice.  So we select
+  ;; only the annotation if available.
+  (shr-generic
+   (thread-first
+     dom
+     (dom-child-by-tag 'semantics)
+     (dom-child-by-tag 'annotation)
+     (or dom))))
+
 ;;; Outline Support
 (defun shr-outline-search (&optional bound move backward looking-at)
   "A function that can be used as `outline-search-function' for rendered html.
-- 
2.46.2

bug#73641: 30.0.90; Math in EWW/shr

[Eli Zaretskii]

> From: Augusto Stoffel <arstoffel@gmail.com>
> Cc: jb@jeremybryant.net,  73641@debbugs.gnu.org,  sacha@sachachua.com
> Date: Sat, 12 Oct 2024 19:13:48 +0200
> 
> On Sat, 12 Oct 2024 at 18:07, Eli Zaretskii wrote:
> 
> > Thanks, but can we please have a comment here explaining why we do
> > that?
> 
> How about this?

LGTM, thanks.