Patch to align baseline of latex fragments and surrounding text

Patch to align baseline of latex fragments and surrounding text

[Matt Huszagh]

[-- Attachment #1: Type: text/plain, Size: 1567 bytes --]

Hello,

I've created a patch to align the baseline of latex image fragments to
the surrounding text. The patch consists of several parts. First, it
adds a customizable variable that can be set to a user supplied function
to compute the value of :ascent passed to `overlay-put'. It can also be
set to a symbol (e.g., 'center, which is the current setting and still
the default), or an integer (:ascent can take an integer
percentage). The patch also modifies `org--make-preview-overlay' to use
this new variable.

The other part of the patch is a new function that computes the correct
value for :ascent from an SVG file.

Unfortunately, this isn't a general-purpose solution to baseline text
alignment. It currently only works with SVG images and requires that
these SVG images encode the text baseline in the viewBox attribute (I've
explained in the function documentation how to achieve this).

I was not initially planning to include the function because it only
works in certain special cases (though if you setup your SVG image
generation as I've described in the documentation it should work
correctly for all of your LaTeX fragments). However, I ultimately
decided it was beneficial to include it for several reasons: (1) it
would not be obvious for many users how to write this function on their
own, (2) by including it, others can improve upon it, and (3) I tried to
minimize corner cases by leaving the default value as 'center and having
the function return 'center whenever it detects an incorrect :ascent
value.

Let me know what you think!

Best
Matt


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-org-make-preview-overlay-Add-ability-to-set-vertical.patch --]
[-- Type: text/x-patch, Size: 4968 bytes --]

From 6c33fa1875cc169616d878b59054d50980f741f3 Mon Sep 17 00:00:00 2001
From: Matt Huszagh <huszaghmatt@gmail.com>
Date: Sun, 10 Oct 2021 15:46:05 -0700
Subject: [PATCH] org--make-preview-overlay: Add ability to set vertical
 alignment of latex fragments

* lisp/org.el (org--match-text-baseline-ascent): Add function that
computes the value of :ascent to match the baseline of the fragment to
the surrounding text.
(org-latex-fragment-overlay-ascent): Add custom variable that allows a
function to be used to compute the value of :ascent.
(org--make-preview-overlay): Incorporate the custom variable.
---
 lisp/org.el | 89 +++++++++++++++++++++++++++++++++++++++++++++++------
 1 file changed, 80 insertions(+), 9 deletions(-)

diff --git a/lisp/org.el b/lisp/org.el
index 405f0f0f9..dbc288a3c 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -320,6 +320,74 @@ identifier."
   :version "24.1"
   :group 'org-id)
 
+(defun org--match-text-baseline-ascent (imagefile imagetype)
+  "Set :ascent to match the text baseline of an image to the surrounding text.
+IMAGEFILE is the path of the image file and IMAGETYPE is the
+image file type.
+
+This function currently only works for SVG images (defaulting to
+'center otherwise).  It also requires that the SVG's viewbox is
+correctly set according to the text baseline position.
+Specifically, it computes
+
+ascent = 100 * -min-y / height
+
+where min-y and height are taken from the viewbox.  If this
+computation returns a non-sensical value (below 0 or above 100),
+it will return 'center instead.
+
+It's possible to include text baseline information in the viewbox
+by using \\documentclass[preview]{standalone} in the input LaTeX
+file, compiling first to DVI and then converting this to an SVG
+image with dvisvgm.
+
+For example,
+
+(setq my-dvisvgm
+      '(my-dvisvgm :programs (\"latex\" \"dvisvgm\")
+                   :description \"dvi > svg\"
+                   :message \"you need to install latex and dvisvgm.\"
+                   :use-xcolor t
+                   :image-input-type \"dvi\"
+                   :image-output-type \"svg\"
+                   :latex-compiler (\"latex -output-directory=%o %f\")
+                   :image-converter (\"dvisvgm --no-fonts --exact-bbox -c %S -o %O %f\")))
+
+(add-to-list 'org-preview-latex-process-alist my-dvisvgm)
+(setq org-preview-latex-default-process 'my-dvisvgm)
+
+(setq org-format-latex-header
+  \"\\documentclass[preview]{standalone}
+  [PACKAGES]
+  [DEFAULT-PACKAGES]\")"
+  (if (eq imagetype 'svg)
+      (let* ((viewbox (split-string
+                       (xml-get-attribute (car (xml-parse-file imagefile)) 'viewBox)))
+             (min-y (string-to-number (nth 1 viewbox)))
+             (height (string-to-number (nth 3 viewbox)))
+             (ascent (round (* -100 (/ min-y height)))))
+        (if (or (< ascent 0) (> ascent 100))
+            'center
+          ascent))
+    'center))
+
+(defcustom org-latex-fragment-overlay-ascent 'center
+  "Value of :ascent used for LaTeX image fragments.
+Org uses `overlay-put' to overlay LaTeX image fragments on inline
+math.  `overlay-put' takes an :ascent parameter that can specify
+the vertical offset of this image fragment.  See the 'Image
+Descriptors' section of the elisp manual for more information.
+
+If this variable is a symbol or integer it will be passed
+directly to :ascent.  It can also be a function that takes the
+image file as an argument.  The function will be evaluated and
+its return value passed to :ascent.  The primary benefit of this
+option is to be able to compute the baseline of the latex
+fragment and to align it with the baseline of the surrounding
+text."
+  :type '(symbol function integer)
+  :group 'org)
+
 ;;; Version
 (org-check-version)
 
@@ -15928,15 +15996,18 @@ Argument IMAGETYPE is the extension of the displayed image,
 as a string.  It defaults to \"png\"."
   (let ((ov (make-overlay beg end))
 	(imagetype (or (intern imagetype) 'png)))
-    (overlay-put ov 'org-overlay-type 'org-latex-overlay)
-    (overlay-put ov 'evaporate t)
-    (overlay-put ov
-		 'modification-hooks
-		 (list (lambda (o _flag _beg _end &optional _l)
-			 (delete-overlay o))))
-    (overlay-put ov
-		 'display
-		 (list 'image :type imagetype :file image :ascent 'center))))
+    (let ((ascent (if (functionp org-latex-fragment-overlay-ascent)
+                      (funcall org-latex-fragment-overlay-ascent image imagetype)
+                    org-latex-fragment-overlay-ascent)))
+      (overlay-put ov 'org-overlay-type 'org-latex-overlay)
+      (overlay-put ov 'evaporate t)
+      (overlay-put ov
+		   'modification-hooks
+		   (list (lambda (o _flag _beg _end &optional _l)
+			   (delete-overlay o))))
+      (overlay-put ov
+		   'display
+		   (list 'image :type imagetype :file image :ascent ascent)))))
 
 (defun org-clear-latex-preview (&optional beg end)
   "Remove all overlays with LaTeX fragment images in current buffer.
-- 
2.31.1

Re: Patch to align baseline of latex fragments and surrounding text

[Matt Huszagh]

Matt Huszagh <huszaghmatt@gmail.com> writes:

> I've created a patch to align the baseline of latex image fragments to
> the surrounding text. The patch consists of several parts. First, it
> adds a customizable variable that can be set to a user supplied function
> to compute the value of :ascent passed to `overlay-put'. It can also be
> set to a symbol (e.g., 'center, which is the current setting and still
> the default), or an integer (:ascent can take an integer
> percentage). The patch also modifies `org--make-preview-overlay' to use
> this new variable.
>
> The other part of the patch is a new function that computes the correct
> value for :ascent from an SVG file.
>
> Unfortunately, this isn't a general-purpose solution to baseline text
> alignment. It currently only works with SVG images and requires that
> these SVG images encode the text baseline in the viewBox attribute (I've
> explained in the function documentation how to achieve this).
>
> I was not initially planning to include the function because it only
> works in certain special cases (though if you setup your SVG image
> generation as I've described in the documentation it should work
> correctly for all of your LaTeX fragments). However, I ultimately
> decided it was beneficial to include it for several reasons: (1) it
> would not be obvious for many users how to write this function on their
> own, (2) by including it, others can improve upon it, and (3) I tried to
> minimize corner cases by leaving the default value as 'center and having
> the function return 'center whenever it detects an incorrect :ascent
> value.

I should also mention that I've tested this for different image scaling
factors (`:image-size-adjust' in `org-preview-latex-process-alist') and
it works regardless of the scale factor. That is, the baseline will be
correctly aligned irrespective of the scale factor you set (though
obviously the text median
(https://en.wikipedia.org/wiki/Baseline_(typography))) will only match
the surrounding text for the correct scaling factor.

Matt

Re: Patch to align baseline of latex fragments and surrounding text

[Matt Huszagh]

[-- Attachment #1: Type: text/plain, Size: 427 bytes --]

Timothy, looping you in on this.

I feel that maybe it would be useful to attach screenshots to show the
improvement from this patch? Anyway, I've attached two images: one with
the correct baseline alignment to surrounding text and the other with
the current, incorrect, baseline alignment.

I think a lot of people would like this functionality. It looks much
better than the current behavior.

Anyway, feedback appreciated.


[-- Attachment #2: wrong --]
[-- Type: image/png, Size: 8008 bytes --]

[-- Attachment #3: right --]
[-- Type: image/png, Size: 8059 bytes --]

Re: Patch to align baseline of latex fragments and surrounding text

[Sébastien Miquel]

Hi,

Matt Huszagh writes:
> I feel that maybe it would be useful to attach screenshots to show the
> improvement from this patch? Anyway, I've attached two images: one with
> the correct baseline alignment to surrounding text and the other with
> the current, incorrect, baseline alignment.
>
> I think a lot of people would like this functionality. It looks much
> better than the current behavior.
This looks great indeed but I've failed to reproduce in my
environment.

I couldn't get ~org--match-text-baseline-ascent~ to compute the
ascent : the ~xml-get-attribute~ call returns
  : ("-16.945024" "12.153473" "16.148855" "8.064997")
which gives an ascent < -100, and the code then defaults to 'center.

The options described in your =my-dvisvgm= seem outdated, you can
check the latest default value of =dvisvgm= : =use-xcolor= is
deprecated and a =:image-size-adjust= property is provided for the
images to be sized properly. Are the arguments =--no-fonts= and
=--exact-bbox= necessary ?

If there are no drawbacks, perhaps this behaviour should be the
default. Otherwise, it should at least be easier to toggle.

Can something similar be done with =dvipng= ?

Regards,

-- 
Sébastien Miquel

Re: Patch to align baseline of latex fragments and surrounding text

[Matt Huszagh]

[-- Attachment #1: Type: text/plain, Size: 2821 bytes --]

Sébastien Miquel <sebastien.miquel@posteo.eu> writes:

> This looks great indeed but I've failed to reproduce in my
> environment.

Thanks for testing this Sébastien.

> I couldn't get ~org--match-text-baseline-ascent~ to compute the
> ascent : the ~xml-get-attribute~ call returns
>   : ("-16.945024" "12.153473" "16.148855" "8.064997")
> which gives an ascent < -100, and the code then defaults to 'center.

I'd need to know more about your setup for generating latex
fragments. Did you follow all the directions in
org--match-text-baseline-ascent? How is your org-format-latex-header
set? In particular, are you using \documentclass[preview]{standalone}?
If you can provide me with the TeX file used to generate the fragment,
as well as the SVG file you get as a result, that would be helpful too.

> The options described in your =my-dvisvgm= seem outdated, you can
> check the latest default value of =dvisvgm= : =use-xcolor= is
> deprecated and a =:image-size-adjust= property is provided for the
> images to be sized properly. Are the arguments =--no-fonts= and
> =--exact-bbox= necessary ?

Hm, I'm not actually sure why I put use-xcolor in there, but it isn't
necessary. I've removed it in the updated patch (attached).

:image-size-adjust isn't necessary, I just mentioned it to point out
that baseline alignment works regardless of the size of a latex fragment
(I have another open PR that allows setting the size of fragments in a
context dependent way, which can be used for instance, to keep fragments
size-aligned to the surrounding text). I expect using the scale
parameter in org-format-latex-options will work similarly, but I'll
investigate.

--no-fonts is the same as -n. The --exact-bbox flag is necessary to
avoid cropping in certain cases (see
https://github.com/mgieseki/dvisvgm/issues/8). You're free to use
--bbox=min, but your glyphs may be cut off in places and this may affect
the baseline location too, though I haven't tested it.

> If there are no drawbacks, perhaps this behaviour should be the
> default. Otherwise, it should at least be easier to toggle.

I didn't attempt to make this the default because it requires a specific
setup, which is also different from the current default setup in other
respects. Most importantly, it requires using the standalone document
class, though I believe article is used at the moment.

> Can something similar be done with =dvipng= ?

Unfortunately I don't think so. This code isn't doing anything fancy; it
has no way of computing the text baseline from some arbitrary image file
displaying text. Instead, it relies on some other tool providing this
information inside the image file. In this case, standalone encodes this
information in the viewbox of the SVG file.

Thanks
Matt


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-org-make-preview-overlay-Add-ability-to-set-vertical.patch --]
[-- Type: text/x-patch, Size: 4934 bytes --]

From 2640b24d2cce8e2b61101125c348db35800570ff Mon Sep 17 00:00:00 2001
From: Matt Huszagh <huszaghmatt@gmail.com>
Date: Sun, 10 Oct 2021 15:46:05 -0700
Subject: [PATCH] org--make-preview-overlay: Add ability to set vertical
 alignment of latex fragments

* lisp/org.el (org--match-text-baseline-ascent): Add function that
computes the value of :ascent to match the baseline of the fragment to
the surrounding text.
(org-latex-fragment-overlay-ascent): Add custom variable that allows a
function to be used to compute the value of :ascent.
(org--make-preview-overlay): Incorporate the custom variable.
---
 lisp/org.el | 88 +++++++++++++++++++++++++++++++++++++++++++++++------
 1 file changed, 79 insertions(+), 9 deletions(-)

diff --git a/lisp/org.el b/lisp/org.el
index 405f0f0f9..91a4d0bf5 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -320,6 +320,73 @@ identifier."
   :version "24.1"
   :group 'org-id)
 
+(defun org--match-text-baseline-ascent (imagefile imagetype)
+  "Set :ascent to match the text baseline of an image to the surrounding text.
+IMAGEFILE is the path of the image file and IMAGETYPE is the
+image file type.
+
+This function currently only works for SVG images (defaulting to
+'center otherwise).  It also requires that the SVG's viewbox is
+correctly set according to the text baseline position.
+Specifically, it computes
+
+ascent = 100 * -min-y / height
+
+where min-y and height are taken from the viewbox.  If this
+computation returns a non-sensical value (below 0 or above 100),
+it will return 'center instead.
+
+It's possible to include text baseline information in the viewbox
+by using \\documentclass[preview]{standalone} in the input LaTeX
+file, compiling first to DVI and then converting this to an SVG
+image with dvisvgm.
+
+For example,
+
+(setq my-dvisvgm
+      '(my-dvisvgm :programs (\"latex\" \"dvisvgm\")
+                   :description \"dvi > svg\"
+                   :message \"you need to install latex and dvisvgm.\"
+                   :image-input-type \"dvi\"
+                   :image-output-type \"svg\"
+                   :latex-compiler (\"latex -output-directory=%o %f\")
+                   :image-converter (\"dvisvgm --no-fonts --exact-bbox -c %S -o %O %f\")))
+
+(add-to-list 'org-preview-latex-process-alist my-dvisvgm)
+(setq org-preview-latex-default-process 'my-dvisvgm)
+
+(setq org-format-latex-header
+  \"\\documentclass[preview]{standalone}
+  [PACKAGES]
+  [DEFAULT-PACKAGES]\")"
+  (if (eq imagetype 'svg)
+      (let* ((viewbox (split-string
+                       (xml-get-attribute (car (xml-parse-file imagefile)) 'viewBox)))
+             (min-y (string-to-number (nth 1 viewbox)))
+             (height (string-to-number (nth 3 viewbox)))
+             (ascent (round (* -100 (/ min-y height)))))
+        (if (or (< ascent 0) (> ascent 100))
+            'center
+          ascent))
+    'center))
+
+(defcustom org-latex-fragment-overlay-ascent 'center
+  "Value of :ascent used for LaTeX image fragments.
+Org uses `overlay-put' to overlay LaTeX image fragments on inline
+math.  `overlay-put' takes an :ascent parameter that can specify
+the vertical offset of this image fragment.  See the 'Image
+Descriptors' section of the elisp manual for more information.
+
+If this variable is a symbol or integer it will be passed
+directly to :ascent.  It can also be a function that takes the
+image file as an argument.  The function will be evaluated and
+its return value passed to :ascent.  The primary benefit of this
+option is to be able to compute the baseline of the latex
+fragment and to align it with the baseline of the surrounding
+text."
+  :type '(symbol function integer)
+  :group 'org)
+
 ;;; Version
 (org-check-version)
 
@@ -15928,15 +15995,18 @@ Argument IMAGETYPE is the extension of the displayed image,
 as a string.  It defaults to \"png\"."
   (let ((ov (make-overlay beg end))
 	(imagetype (or (intern imagetype) 'png)))
-    (overlay-put ov 'org-overlay-type 'org-latex-overlay)
-    (overlay-put ov 'evaporate t)
-    (overlay-put ov
-		 'modification-hooks
-		 (list (lambda (o _flag _beg _end &optional _l)
-			 (delete-overlay o))))
-    (overlay-put ov
-		 'display
-		 (list 'image :type imagetype :file image :ascent 'center))))
+    (let ((ascent (if (functionp org-latex-fragment-overlay-ascent)
+                      (funcall org-latex-fragment-overlay-ascent image imagetype)
+                    org-latex-fragment-overlay-ascent)))
+      (overlay-put ov 'org-overlay-type 'org-latex-overlay)
+      (overlay-put ov 'evaporate t)
+      (overlay-put ov
+		   'modification-hooks
+		   (list (lambda (o _flag _beg _end &optional _l)
+			   (delete-overlay o))))
+      (overlay-put ov
+		   'display
+		   (list 'image :type imagetype :file image :ascent ascent)))))
 
 (defun org-clear-latex-preview (&optional beg end)
   "Remove all overlays with LaTeX fragment images in current buffer.
-- 
2.31.1

Re: Patch to align baseline of latex fragments and surrounding text

[Sébastien Miquel]

Matt Huszagh writes:
>> I couldn't get ~org--match-text-baseline-ascent~ to compute the
>> ascent : the ~xml-get-attribute~ call returns
>>    : ("-16.945024" "12.153473" "16.148855" "8.064997")
>> which gives an ascent < -100, and the code then defaults to 'center.
> I'd need to know more about your setup for generating latex
> fragments. Did you follow all the directions in
> org--match-text-baseline-ascent? How is your org-format-latex-header
> set? In particular, are you using \documentclass[preview]{standalone}?
> If you can provide me with the TeX file used to generate the fragment,
> as well as the SVG file you get as a result, that would be helpful too.
My mistake. For some reason, I had thought that
=\documentclass[preview]{standalone}= was used by default for LaTeX
previews. Setting it as described in your patch, it now works
properly, even with the default value of =dvisvgm=.

>> If there are no drawbacks, perhaps this behaviour should be the
>> default. Otherwise, it should at least be easier to toggle.
> I didn't attempt to make this the default because it requires a specific
> setup, which is also different from the current default setup in other
> respects. Most importantly, it requires using the standalone document
> class, though I believe article is used at the moment.

To make this behavior easier to toggle, you could
  1. Change the default value of =org-format-latex-header=. The
     =standalone= class makes sense, but I don't know if that might
     break things.
  2. Specify the =:latex-header= of the default =dvisvgm= option. Same
     caveat applies.
  3. Add a =dvisvgm-with-ascent= option to the default value of
     =org-preview-latex-process-alist=.

Instead of the new variable =org-latex-fragment-overlay-ascent=,
perhaps the function used to compute the ascent could be provided as
another property, such as =:ascent=, added to the relevant options in
=org-preview-latex-process-alist=. It seems to make more sense since
it only applies to svg output, and it makes it easier to have this
behavior as default. It would require =org--make-preview-overlay= to
take the ascent as an additional argument.

Please note that I am not a maintainer, these are just a few thoughts.
I do hope your work can be applied and that LaTeX fragments can be
properly aligned by default.

You should add [PATCH] to the subject of your mail, so that it gets
listed at https://updates.orgmode.org/ and not forgotten. A maintainer
will reply eventually, but it might take up to a few months.

Regards,

-- 
Sébastien Miquel

Re: [Patch] Align baseline of latex fragments and surrounding text

[Matt Huszagh]

Sébastien Miquel <sebastien.miquel@posteo.eu> writes:

> My mistake. For some reason, I had thought that
> =\documentclass[preview]{standalone}= was used by default for LaTeX
> previews. Setting it as described in your patch, it now works
> properly, even with the default value of =dvisvgm=.

No worries; glad you got it working.

> To make this behavior easier to toggle, you could
>   1. Change the default value of =org-format-latex-header=. The
>      =standalone= class makes sense, but I don't know if that might
>      break things.
>   2. Specify the =:latex-header= of the default =dvisvgm= option. Same
>      caveat applies.
>   3. Add a =dvisvgm-with-ascent= option to the default value of
>      =org-preview-latex-process-alist=.
>
> Instead of the new variable =org-latex-fragment-overlay-ascent=,
> perhaps the function used to compute the ascent could be provided as
> another property, such as =:ascent=, added to the relevant options in
> =org-preview-latex-process-alist=. It seems to make more sense since
> it only applies to svg output, and it makes it easier to have this
> behavior as default. It would require =org--make-preview-overlay= to
> take the ascent as an additional argument.

These are nice ideas, and ones I hadn't considered. Thanks for the
suggestions! I'll think about it a bit and may modify the patch
accordingly.

> Please note that I am not a maintainer, these are just a few thoughts.
> I do hope your work can be applied and that LaTeX fragments can be
> properly aligned by default.
>
> You should add [PATCH] to the subject of your mail, so that it gets
> listed at https://updates.orgmode.org/ and not forgotten. A maintainer
> will reply eventually, but it might take up to a few months.

Also a good suggestion. Fortunately, "Patch" seems to have been caught
by that tool. But [PATCH] seems more typical and I've updated the
subject for that.

Matt

Re: [Patch] Align baseline of latex fragments and surrounding text

[Ihor Radchenko]

Matt Huszagh <huszaghmatt@gmail.com> writes:

> These are nice ideas, and ones I hadn't considered. Thanks for the
> suggestions! I'll think about it a bit and may modify the patch
> accordingly.

May I ask if you further considered changing the patch?

Best,
Ihor

Re: [Patch] Align baseline of latex fragments and surrounding text

[Matt Huszagh]

Ihor Radchenko <yantar92@gmail.com> writes:

> Matt Huszagh <huszaghmatt@gmail.com> writes:
>
>> These are nice ideas, and ones I hadn't considered. Thanks for the
>> suggestions! I'll think about it a bit and may modify the patch
>> accordingly.
>
> May I ask if you further considered changing the patch?

Hey Ihor,

I'm sorry for the late reply. I'm just now looking at several patches I
submitted a few years ago and left hanging...

The answer to this question is yes I did, and IIRC got most of the way
through implementing them but ran into a largely unrelated issue that I
didn't, at the time, find a solution for. It looks like I still have the
branch with those changes and I can take a look back at it and try to
resolve it.

However, I noticed that there appears to be another (large) patch in the
works that also does this (email subject line "Overhaul of the LaTeX
preview system"). That's a long thread and I haven't followed it. Is
there still scope for my change? Is it worth getting this in?

Matt

Re: [Patch] Align baseline of latex fragments and surrounding text

[Timothy]

[-- Attachment #1: Type: text/plain, Size: 908 bytes --]

Hi Matt,

> However, I noticed that there appears to be another (large) patch in the
> works that also does this (email subject line “Overhaul of the LaTeX
> preview system”). That’s a long thread and I haven’t followed it. Is
> there still scope for my change? Is it worth getting this in?

So, in the large patch (actually ~150 patches in a branch), we actually
have had baseline alignment implemented for the last ~year 🙂.

Currently it’s just being polished up so it’s good to merge, you can see a list
of todo tasks we’d like to get sorted first here:
<https://github.com/tecosaur/org-latex-preview-todos/issues>

All the best,
Timothy

-- 
Timothy (‘tecosaur’/‘TEC’), Org mode contributor.
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/tec>.