[PATCH] Don't fill displayed equations

[PATCH] Don't fill displayed equations

[Timothy]

[-- Attachment #1: Type: text/plain, Size: 898 bytes --]

Hi All,

If a displayed equation (`\[ ... \]') starts on its own line, I don’t think it
should be filled into the rest of the text. I.e.,

┌────
│ some nice text
│ \[
│   1+1=2
│ \]
│ more text.
└────
should not become,
┌────
│ some nice text \[ 1+1=3 \] more text.
└────

While the above example may not look bad, with non-trivial equations
this can become quite messy.

As such, I have attached a patch which adds fill “cuts” around `\[' and
`\]', when `\[' occurs at the start of a line. This leaves the display equation
delimiters on their own line.

I think this motivation for this change is fairly clear, and so without feedback
I intend to push this in a few days, however I if anybody has comments on the way
I’m currently implementing this, I’d like to hear your thoughts.

All the best,
Timothy

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-org-Don-t-fill-displayed-equations-into-text.patch --]
[-- Type: text/x-patch, Size: 2168 bytes --]

From d008ff3c2218e19bb501137715b05e06a0c511e3 Mon Sep 17 00:00:00 2001
From: TEC <tec@tecosaur.com>
Date: Fri, 1 Oct 2021 01:14:21 +0800
Subject: [PATCH] org: Don't fill displayed equations into text

* list/org.el (org-fill-element): If a displayed equation (\[ ... \])
starts on its own line, it should not be filled into the rest of the
text. I.e.,

some nice text
\[
  1+1=2
\]
more text.

should not become,

some nice text \[ 1+1=3 \] more text.

While the above example may not look bad, with non-trivial equations
this can become quite messy.
---
 lisp/org.el | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)

diff --git a/lisp/org.el b/lisp/org.el
index 405f0f0f9..daf1d91b4 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -19551,10 +19551,10 @@ (defun org-fill-element (&optional justify)
 			 (org-element-property :contents-end element))))
 	   ;; Do nothing if point is at an affiliated keyword.
 	   (if (< (line-end-position) beg) t
-	     ;; Fill paragraph, taking line breaks into account.
 	     (save-excursion
 	       (goto-char beg)
 	       (let ((cuts (list beg)))
+                 ;; Cut fill on line breaks.
 		 (while (re-search-forward "\\\\\\\\[ \t]*\n" end t)
 		   (when (eq 'line-break
 			     (org-element-type
@@ -19562,6 +19562,16 @@ (defun org-fill-element (&optional justify)
 					      (org-element-context))))
 		     (push (point) cuts)))
 		 (dolist (c (delq end cuts))
+                 ;; Cut fill on displayed equations.
+                 (while (re-search-forward "^[ \t]*\\\\\\[" end t)
+                   (let ((el (org-element-context)))
+                     (when (eq 'latex-fragment (org-element-type el))
+                       (setf cuts (append
+                                   (list (org-element-property :end el)
+                                         (- (org-element-property :end el) 2)
+                                         (+ (org-element-property :begin el) 2)
+                                         (org-element-property :begin el))
+                                   cuts)))))
 		   (fill-region-as-paragraph c end justify)
 		   (setq end c))))
 	     t)))
-- 
2.33.0

Re: [PATCH] Don't fill displayed equations

[Timothy]

[-- Attachment #1: Type: text/plain, Size: 307 bytes --]


The initial patch was just slightly off. Here's a correct version.

Timothy <tecosaur@gmail.com> writes:
> As such, I have attached a patch which adds fill “cuts” around `\[' and
> `\]', when `\[' occurs at the start of a line. This leaves the display equation
> delimiters on their own line.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-org-Don-t-fill-displayed-equations-into-text.patch --]
[-- Type: text/x-patch, Size: 2129 bytes --]

From d6d2221a7a9bf5e3cf93265c13cb643bcfe46929 Mon Sep 17 00:00:00 2001
From: TEC <tec@tecosaur.com>
Date: Fri, 1 Oct 2021 01:14:21 +0800
Subject: [PATCH] org: Don't fill displayed equations into text

* list/org.el (org-fill-element): If a displayed equation (\[ ... \])
starts on its own line, it should not be filled into the rest of the
text. I.e.,

some nice text
\[
  1+1=2
\]
more text.

should not become,

some nice text \[ 1+1=3 \] more text.

While the above example may not look bad, with non-trivial equations
this can become quite messy.
---
 lisp/org.el | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)

diff --git a/lisp/org.el b/lisp/org.el
index 405f0f0f9..ff86a9dd0 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -19551,16 +19551,26 @@ (defun org-fill-element (&optional justify)
 			 (org-element-property :contents-end element))))
 	   ;; Do nothing if point is at an affiliated keyword.
 	   (if (< (line-end-position) beg) t
-	     ;; Fill paragraph, taking line breaks into account.
 	     (save-excursion
 	       (goto-char beg)
 	       (let ((cuts (list beg)))
+                 ;; Cut fill on line breaks.
 		 (while (re-search-forward "\\\\\\\\[ \t]*\n" end t)
 		   (when (eq 'line-break
 			     (org-element-type
 			      (save-excursion (backward-char)
 					      (org-element-context))))
 		     (push (point) cuts)))
+                 ;; Cut fill on displayed equations.
+                 (while (re-search-forward "^[ \t]*\\\\\\[" end t)
+                   (let ((el (org-element-context)))
+                     (when (eq 'latex-fragment (org-element-type el))
+                       (setf cuts (append
+                                   (list (org-element-property :end el)
+                                         (- (org-element-property :end el) 2)
+                                         (+ (org-element-property :begin el) 2)
+                                         (org-element-property :begin el))
+                                   cuts)))))
 		 (dolist (c (delq end cuts))
 		   (fill-region-as-paragraph c end justify)
 		   (setq end c))))
-- 
2.33.0

Re: [PATCH] Don't fill displayed equations

[Nicolas Goaziou]

Hello,

Timothy <tecosaur@gmail.com> writes:

> If a displayed equation (`\[ ... \]') starts on its own line, I don’t think it
> should be filled into the rest of the text. I.e.,
>
> ┌────
> │ some nice text
> │ \[
> │   1+1=2
> │ \]
> │ more text.
> └────
> should not become,
> ┌────
> │ some nice text \[ 1+1=3 \] more text.
> └────
>
> While the above example may not look bad, with non-trivial equations
> this can become quite messy.

I strongly disagree with this. \[...\] is an inline element, not a block
element. As such, it can be filled, and filling function should obey to
the inner structure of the document.

You can use a real block element here, e.g.,
\begin{equation*}...\end{equation*}, which will not be filled.

Regards,
-- 
Nicolas Goaziou

Re: [PATCH] Don't fill displayed equations

[Timothy]

Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

> I strongly disagree with this. \[...\] is an inline element, not a block
> element. As such, it can be filled, and filling function should obey to
> the inner structure of the document.
>
> You can use a real block element here, e.g.,
> \begin{equation*}...\end{equation*}, which will not be filled.

Given that \[ ... \] is an alias for \begin{equation*} ...
\end{equation*} I don't see why it should be treated any differently
when filling.

--
Timothy

Re: [PATCH] Don't fill displayed equations

[Nicolas Goaziou]

Timothy <tecosaur@gmail.com> writes:

> Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:
>
>> I strongly disagree with this. \[...\] is an inline element, not a block
>> element. As such, it can be filled, and filling function should obey to
>> the inner structure of the document.
>>
>> You can use a real block element here, e.g.,
>> \begin{equation*}...\end{equation*}, which will not be filled.
>
> Given that \[ ... \] is an alias for \begin{equation*} ...
> \end{equation*}

This is true in LaTeX, not in Org, obviously.

> I don't see why it should be treated any differently
> when filling.

As I wrote above, they do not belong to the same category of syntax.
There's no reason to special case \[...\]. 

Filling is a red herring, its outcome depends on the syntax of the
underlying object, as expected. So, as long as \[...\] is an inline
element, it should be filled.

Regards,

Re: [PATCH] Don't fill displayed equations

[Colin Baxter]

>>>>> Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

    > Timothy <tecosaur@gmail.com> writes:
    >> Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:
    >> 
    >>> I strongly disagree with this. \[...\] is an inline element, not
    >>> a block element. As such, it can be filled, and filling function
    >>> should obey to the inner structure of the document.
    >>> 
    >>> You can use a real block element here, e.g.,
    >>> \begin{equation*}...\end{equation*}, which will not be filled.
    >> 
    >> Given that \[ ... \] is an alias for \begin{equation*} ...
    >> \end{equation*}

    > This is true in LaTeX, not in Org, obviously.

But shouldn't org be consistent with LaTeX. In both org-mode and LaTeX I
would write an in-line equation as $1+3=3$ and not as \[1+2=3\]. I would
reserve the latter for an unlabelled block equation. As has been said,
\[ is an alias for \begin{equation*}. For me, I would rather not write
equations in org-mode differently from LaTeX.

Best wishes,

Colin Baxter.

Re: [PATCH] Don't fill displayed equations

[Timothy]

Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

>> Given that \[ ... \] is an alias for \begin{equation*} ...
>> \end{equation*}
>
> This is true in LaTeX, not in Org, obviously.

Isn't the whole point of the \[ ... \], \( ... \), $ ... $, $$ ... $$,
and \begin{env} ... \end{env} and constructs in Org to be consistent
with LaTeX?

--
Timothy

Re: [PATCH] Don't fill displayed equations

[Timothy]

[-- Attachment #1: Type: text/plain, Size: 3095 bytes --]

Hi Nicolas,

I think there are also some relevant points which I haven’t mentioned so far,
separate from my thoughts that since we’re using the LaTeX syntax we should be
consistent with how LaTeX treats this.

> As I wrote above, they do not belong to the same category of syntax.
> There’s no reason to special case ....

I think we already do special-case `\[ ... \]' somewhat. When refer to inline
elements like bold, verbatim, italic, etc. they sit in the text. Semantically,
this doesn’t hold for `\[ ... \]' either. The semantically inline maths element is
`\( ... \)'. Considering other “inline” syntax elements, like bold, verbatim,
italic, etc. if you spread the delimiters across multiple lines that doesn’t
work. So I’d argue the ship has already sailed on treating `\[ ... \]' differently
to other inline elements.

If you’re wondering why I’m so opposed to the current behaviour, that is probably
best explained by a more realistic demo that what I have in the commit message.

┌────
│ Since \(\cos\) is an even function, we can negate the numerator of the argument
│ without changing the result, giving
│ \[
│   \cos \left( \pi \frac{C_1-x}{2C_1+D} \right) \ , \quad C_1 = \frac{D}{2}.
│ \]
│ this will be positive over \(x \in (0,D)\), and so we can rewrite \(\tilde{y}\) as,
│ \[
│   \tilde{y}(x) = \frac{2D}{\pi} \log \cos \left( \pi \frac{\frac{D}{2}-x}{2D} \right) + C_2.
│ \]
│ Once again considering that \(y(0)=y(D)=0\), it is clear that
│ \[
│   C_2 = - \frac{2D}{\pi} \log \cos \left( \frac{\pi}{4} \right) = - \frac{2D}{\pi} \log 2^{-\frac{1}{2}} = \frac{D}{\pi} \log 2.
│ \]
│ The complete solution for \(\tilde{y}\) is hence,
│ \[
│   \tilde{y} = \frac{2D}{\pi} \log \cos \left( \pi \frac{D-2x}{4D} \right) + \frac{D}{\pi} \log 2.
│ \]
└────
is currently filled to
┌────
│ Since \(\cos\) is an even function, we can negate the numerator of the argument
│ without changing the result, giving \[ \cos \left( \pi \frac{C_1-x}{2C_1+D}
│ \right) \ , \quad C_1 = \frac{D}{2}. \] this will be positive over \(x \in (0,D)\),
│ and so we can rewrite \(\tilde{y}\) as, \[ \tilde{y}(x) = \frac{2D}{\pi} \log \cos \left( \pi
│ \frac{\frac{D}{2}-x}{2D} \right) + C_2. \] Once again considering that
│ \(y(0)=y(D)=0\), it is clear that \[ C_2 = - \frac{2D}{\pi} \log \cos \left(
│ \frac{\pi}{4} \right) = - \frac{2D}{\pi} \log 2^{-\frac{1}{2}} = \frac{D}{\pi} \log 2.
│ \] The complete solution for \(\tilde{y}\) is hence, \[ \tilde{y} = \frac{2D}{\pi} \log \cos
│ \left( \pi \frac{D-2x}{4D} \right) + \frac{D}{\pi} \log 2. \]
└────

Suffice to say, I find the second terrible to read compared to the first.
Furthermore, when using org-latex-preview, wrapped `\[ ... \]' blocks join lines,
resulting in lines that are displayed going off edge of the page after filling.

Basically, this leads to a worse experience when using Org in what I would think
to be a perfectly reasonably way.

All the best,
Timothy

Re: [PATCH] Don't fill displayed equations

[Nicolas Goaziou]

Hello,

Colin Baxter <m43cap@yandex.com> writes:

>>>>>> Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:
>
>     > Timothy <tecosaur@gmail.com> writes:
>     >> Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:
>     >> 
>     >>> I strongly disagree with this. \[...\] is an inline element, not
>     >>> a block element. As such, it can be filled, and filling function
>     >>> should obey to the inner structure of the document.
>     >>> 
>     >>> You can use a real block element here, e.g.,
>     >>> \begin{equation*}...\end{equation*}, which will not be filled.
>     >> 
>     >> Given that \[ ... \] is an alias for \begin{equation*} ...
>     >> \end{equation*}
>
>     > This is true in LaTeX, not in Org, obviously.
>
> But shouldn't org be consistent with LaTeX. 

Org supports, as a small part of its syntax, some limited LaTeX markup.
It doesn't imply it should strive for consistency with LaTeX. Actually,
I think it's quite the opposite. Org is not a LaTeX front-end.

Regards,
-- 
Nicolas Goaziou

Re: [PATCH] Don't fill displayed equations

[Tim Cross]

Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

> Hello,
>
> Colin Baxter <m43cap@yandex.com> writes:
>
>>>>>>> Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:
>>
>>     > Timothy <tecosaur@gmail.com> writes:
>>     >> Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:
>>     >> 
>>     >>> I strongly disagree with this. \[...\] is an inline element, not
>>     >>> a block element. As such, it can be filled, and filling function
>>     >>> should obey to the inner structure of the document.
>>     >>> 
>>     >>> You can use a real block element here, e.g.,
>>     >>> \begin{equation*}...\end{equation*}, which will not be filled.
>>     >> 
>>     >> Given that \[ ... \] is an alias for \begin{equation*} ...
>>     >> \end{equation*}
>>
>>     > This is true in LaTeX, not in Org, obviously.
>>
>> But shouldn't org be consistent with LaTeX. 
>
> Org supports, as a small part of its syntax, some limited LaTeX markup.
> It doesn't imply it should strive for consistency with LaTeX. Actually,
> I think it's quite the opposite. Org is not a LaTeX front-end.
>

I think this is an important point. Org, like other 'markdown' style
abstractions is a lot more than just a convenience abstraction for
writing Latex. Like other markdown dialects which have an 'escape hatch'
which allows you to embed raw HTML in your document, org allows
embedding 'latex' fragments, but that does not mean it is a latex
front-end. How document elements are displayed in the buffer should not
be determined just by what/how latex does it - it might provide some
guidance, but should not be the sole decider (i.e. because this is how
latex does it is not sufficient justification in itself).

With respect to this patch, I can see both sides having merit. Timothy's
points make sense from an end user's perspective and how things look in
the buffer. However, Nicolas point is also very relevant, especially if
we hope to have a markup syntax which is consistent and parsed
consistently. I'm not convinced that one inline element should be
treated differently  because in some situations, it is easier to
read/edit. Changing the visual representation of this inline block may
also have impact on user expectations for export and could lead to
further confusion. 

Re: [PATCH] Don't fill displayed equations

[Nicolas Goaziou]

Timothy <tecosaur@gmail.com> writes:

> I think there are also some relevant points which I haven’t mentioned so far,
> separate from my thoughts that since we’re using the LaTeX syntax we should be
> consistent with how LaTeX treats this.

I'm not convinced about this. I don't think it is even possible.

>> As I wrote above, they do not belong to the same category of syntax.
>> There’s no reason to special case ....
>
> I think we already do special-case `\[ ... \]' somewhat. When refer to inline
> elements like bold, verbatim, italic, etc. they sit in the text. Semantically,
> this doesn’t hold for `\[ ... \]' either. The semantically inline maths element is
> `\( ... \)'. Considering other “inline” syntax elements, like bold, verbatim,
> italic, etc. if you spread the delimiters across multiple lines that doesn’t
> work. So I’d argue the ship has already sailed on treating `\[ ... \]' differently
> to other inline elements.

I'm not sure about what you mean. \[...\] is no different than, e.g.,
verbatim. It's an inline element, with all that it implies.

Now, if you want to discuss changing syntax for \[...\] and make it
a block element, you can of course do it to your heart's content (it has
been discussed already in this ML and I don't have an opinion on the
subject), but please don't make filling do bizarre things (not all Org
users use LaTeX or even like LaTeXisms), just because LaTeX modes behave
differently.

> If you’re wondering why I’m so opposed to the current behaviour, that is probably
> best explained by a more realistic demo that what I have in the commit message.
>
> ┌────
> │ Since \(\cos\) is an even function, we can negate the numerator of the argument
> │ without changing the result, giving
> │ \[
> │   \cos \left( \pi \frac{C_1-x}{2C_1+D} \right) \ , \quad C_1 = \frac{D}{2}.
> │ \]
> │ this will be positive over \(x \in (0,D)\), and so we can rewrite \(\tilde{y}\) as,
> │ \[
> │   \tilde{y}(x) = \frac{2D}{\pi} \log \cos \left( \pi \frac{\frac{D}{2}-x}{2D} \right) + C_2.
> │ \]
> │ Once again considering that \(y(0)=y(D)=0\), it is clear that
> │ \[
> │   C_2 = - \frac{2D}{\pi} \log \cos \left( \frac{\pi}{4} \right) = - \frac{2D}{\pi} \log 2^{-\frac{1}{2}} = \frac{D}{\pi} \log 2.
> │ \]
> │ The complete solution for \(\tilde{y}\) is hence,
> │ \[
> │   \tilde{y} = \frac{2D}{\pi} \log \cos \left( \pi \frac{D-2x}{4D} \right) + \frac{D}{\pi} \log 2.
> │ \]
> └────

In every case above, you can already use
\begin{equation*}...\end{equation*}, so I don't see the point. You
already have all you need without breaking filling function for the rest
of us.

> Basically, this leads to a worse experience when using Org in what
> I would think to be a perfectly reasonably way.

I don't think it is a worse experience, unless you apply expectations
from LaTeX to Org. It just doesn't work. Notwithstanding filling
behaviour, \[...\] in Org is much more limited than \[...\] in LaTeX.
They just happen to use the same syntax for convenience in simple cases.
The same holds for, e.g., LaTeX commands. To put it differently, you
cannot just paste some LaTeX code in an Org buffer and expect Org to
properly deal with it. But that's fine. If you need to write or copy
"advanced" LaTeX code, Org provides dedicated environments.

Regards,

Re: [PATCH] Don't fill displayed equations

[Stefan Nobis]

Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

> Now, if you want to discuss changing syntax for \[...\] and make it
> a block element

I wonder, why it is not a block element. As far as I know, the only
difference (even in the context of Org) between \(...\) and \[...\]
is, that the former denotes inline math and the latter denotes a math
block. And at least exporting to HTML (with MathJax) and LaTeX results
in a block equation for \[...\].

Do you have a short summary or a pointer why \[...\] has been choosen
to be an inline element? What's the advantage or what would be the
downside of making it a block element?

-- 
Until the next mail...,
Stefan.

Re: [PATCH] Don't fill displayed equations

[Timothy]

Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

> Timothy <tecosaur@gmail.com> writes:
>
>> I think there are also some relevant points which I haven’t mentioned so far,
>> separate from my thoughts that since we’re using the LaTeX syntax we should be
>> consistent with how LaTeX treats this.
>
> I'm not convinced about this. I don't think it is even possible.

By this, I'm specifically thinking of the fact that \( ... \) is inline,
and \[ ... \] is a display equation. What's the point of Org having both
if they're treated the same in "org syntax", on top of the inconstancy
that creates with HTML, LaTeX exports etc. where it is once again
treated as a display equation? Perhaps the discussion should shift from
my specific patch to this general situation, but this behaviour feels
wrong to me.

>>> As I wrote above, they do not belong to the same category of syntax.
>>> There’s no reason to special case ....
>>
>> I think we already do special-case `\[ ... \]' somewhat. When refer to inline
>> elements like bold, verbatim, italic, etc. they sit in the text. Semantically,
>> this doesn’t hold for `\[ ... \]' either. The semantically inline maths element is
>> `\( ... \)'. Considering other “inline” syntax elements, like bold, verbatim,
>> italic, etc. if you spread the delimiters across multiple lines that doesn’t
>> work. So I’d argue the ship has already sailed on treating `\[ ... \]' differently
>> to other inline elements.
>
> I'm not sure about what you mean. \[...\] is no different than, e.g.,
> verbatim. It's an inline element, with all that it implies.

Is it? I can't use verbatim like this:

=
some
verbatim
text
=

but I can do

\[
some
display
equation
\]

It seems to me that \[ ... \] is already treated differently from other
inline markup.

> Now, if you want to discuss changing syntax for \[...\] and make it
> a block element, you can of course do it to your heart's content (it has
> been discussed already in this ML and I don't have an opinion on the
> subject), but please don't make filling do bizarre things (not all Org
> users use LaTeX or even like LaTeXisms), just because LaTeX modes behave
> differently.

If that's the only way that Org could treat \[ ... \] differently from
\( ... \), I'd be strongly in favour of this.

>> If you’re wondering why I’m so opposed to the current behaviour, that is probably
>> best explained by a more realistic demo that what I have in the commit message.
>> [*snip*]
> In every case above, you can already use
> \begin{equation*}...\end{equation*}, so I don't see the point.

I prefer \[ ... \] over \begin{equation*}...\end{equation*} as it's much
more succinct, and helps reduce the "markup noise" in my documents. I
don't think this is an insignificant concern, brevity may not be
something I'm very good at in emails 😛 but is something I look for in
syntax.

> You already have all you need without breaking filling function for
> the rest of us.

I must admit, I don't see the downside here --- how does this break the
filling function for the rest of you? This only affects \[ ... \] blocks
that have already been put on their own line.

> I don't think it is a worse experience, unless you apply expectations
> from LaTeX to Org. It just doesn't work.

Why can't we apply LaTeX expectations to LaTeX elements in Org? Applying
LaTeX expectations to Org as a whole is clearly a silly idea, but Org
copies \[ .. \] from LaTeX and it is a LaTeX construct.

> Notwithstanding filling behaviour, \[...\] in Org is much more limited
> than \[...\] in LaTeX.

I'd be curious to hear how, as I personally haven't run into any
instances where \[ ... \] has behaved differently other than when an
environment starts on a new line in of a \[ ... \] block (which can
easily be fixed by putting something like \!\ at the start of the line).

> If you need to write or copy "advanced" LaTeX code, Org provides
> dedicated environments.

I don't want "advanced" LaTeX code, I just want my display equations to
be treated as display equations consistently 😂.

Anyway, thanks for engaging with this Nicolas. Even if my patch is a bad
idea, I hope that by the end of this conversation we may arrive at an
agreement on how \[ ... \] should be treated.

--
Timothy

Re: [PATCH] Don't fill displayed equations

[Greg Minshall]

hi, Timothy, et al.,

like Tim, i can see both sides here.  but, a fundamental of org-mode is
this statement of Nicolas', "Org is not a LaTeX front-end."

i ran into this precisely with $...$, i.e., the restrictions that
org-mode requires, in order to protect from confusing a currency
indicator with an equation.

my "take away" from that issue (a long time ago) was that i needed to
drop the idea of "org mode latex syntax support" == "latex syntax
support".  but, i remember to not finding the conclusion pleasant
(though, by now, i no longer notice it).

cheers, Greg

Re: [PATCH] Don't fill displayed equations

[Nicolas Goaziou]

Hello,

Stefan Nobis <stefan-ml@snobis.de> writes:

> I wonder, why it is not a block element. As far as I know, the only
> difference (even in the context of Org) between \(...\) and \[...\]
> is, that the former denotes inline math and the latter denotes a math
> block. And at least exporting to HTML (with MathJax) and LaTeX results
> in a block equation for \[...\].

That's not true. Only some export back-ends can tell the difference
between \(...\) and \[...\], so in the context of Org, they are the
same.

> Do you have a short summary or a pointer why \[...\] has been choosen
> to be an inline element? 

Yes: habit. Also, I don't think LaTeX treats it as a block element.
E.g.,

    text
    \[1+1=3\]
    text

is a single paragraph in LaTeX.

> What's the advantage or what would be the downside of making it
> a block element?

If it's a block element, you cannot write \[...\] mid-line. Such
constructs must start at the beginning of the line, barring some initial
indentation. Also, they would end the current paragraph, so the example
above would generate three paragraphs.

The advantage, at least for some users, is that they are not subjects to
filling, and can contain empty lines. This is already provided by
\begin{equation*}...\end{equation*} LaTeX environments.

Regards,
-- 
Nicolas Goaziou

Org syntax: \[ \] as block element instead of inline object

[Max Nikulin]

I like Timothy's idea that \[ \] markup should behave like block-level 
element rather than inline object. On the other hand, Nicolas convinced 
me that the problem is not only with filling paragraphs in Emacs using 
M-q and it requires change of Org grammar. That is why I do not think 
that suggested patch should be applied.

Although Org has its own syntax, \[ \] constructs are added for LaTeX 
snippets, so unless there is a strong reason, such construct should not 
be a surprise for users familiar with LaTeX.

As it pointed out by Nicolas, the following is a compatibility issue 
with such change.

The following will be invalid Org structure, however the problem is 
hidden in the case of exporting to LaTeX:
A \[
   1 \ne 2,
\] B

It should be typed as

A
\[ 1 \ne 2, \]
B

or

A
\[
   1 \ne 2,
\]
B

On 02/10/2021 03:41, Nicolas Goaziou wrote:
> 
>> I wonder, why it is not a block element. As far as I know, the only
>> difference (even in the context of Org) between \(...\) and \[...\]
>> is, that the former denotes inline math and the latter denotes a math
>> block. And at least exporting to HTML (with MathJax) and LaTeX results
>> in a block equation for \[...\].
> 
> That's not true. Only some export back-ends can tell the difference
> between \(...\) and \[...\], so in the context of Org, they are the
> same.

Is there a valid use case for that backends, so they should not consider 
\[ \] as a block-level element? Otherwise I would tell that it is a bug 
in that backend and in Org parser.

>> Do you have a short summary or a pointer why \[...\] has been choosen
>> to be an inline element?
> 
> Yes: habit. Also, I don't think LaTeX treats it as a block element.
> E.g.,
> 
>      text
>      \[1+1=3\]
>      text
> 
> is a single paragraph in LaTeX.

It is a single LaTeX paragraphs assembled in vertical mode from parts 
created in horizontal or math mode. Splitting text into lines (in 
horizontal mode) is performed for each part of paragraph independently. 
So LaTeX paragraph is more like "greater element" than just "element". 
There is no of similar concept in HTML + CSS, there are blocks and 
inline parts. Formatting like paragraph indents and vertical spacing 
between paragraphs may be achieved with CSS rules.

Visually formatted paragraph may contain several block-level elements.

P.S. Examples of HTML vs. LaTeX paragraphs in Org export that were not 
intuitive for me:
https://list.orgmode.org/orgmode/sd9h2q$t9a$1@ciao.gmane.io/T/#u

Re: [PATCH] Don't fill displayed equations

[Stefan Nobis]

Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

> Also, I don't think LaTeX treats it as a block element.
> E.g.,
>
>     text
>     \[1+1=3\]
>     text
>
> is a single paragraph in LaTeX.

Yes and no. LaTeX is a bit more complex and does not only see inline
or block elements. It has paragraph mode (and inner paragraph mode
like \parbox, which does not allow page breaks), vertical mode, LR
mode, math mode and within math mode there is inline math style
and display math style (and some more little details I forgot).

But still, the equation in the example is in display math style and it
is typeset as a separate line. Additionally, as far as I remember,
with processing "\]" LaTeX leaves math mode, typesets the necessary
vertical space, and switches back to paragraph mode. Then, if an empty
line (or the command "\par") is encountered, the marking of a new
paragraph is typeset (e.g. using "\parindent" and "\parskip"), else
normal output resumes.

Therefore, I would argue, LaTeX modes are not really comparable to Org
element types (inline or block).

See also the following LaTeX code:

--8<---------------cut here---------------start------------->8---
\documentclass{article}
\begin{document}
text\par text\\ and more text
\end{document}
--8<---------------cut here---------------end--------------->8---

Here, with an Org point of view, one may say, this is a single block
or paragraph. But LaTeX will typeset two paragraphs in three lines
(and internally stays the whole time in paragraph mode, if I correctly
remember the small fragments I once learned about the LaTeX kernel).

> If it's a block element, you cannot write \[...\] mid-line.

Hmm... so maybe, it's really worth to have both (\[...\] as inline
element and the \begin{equation*}...\end{equation*} block). I'm
undecided. :)

But thanks for your explanations!

-- 
Until the next mail...,
Stefan.

Re: [PATCH] Don't fill displayed equations

[Eric S Fraga]

On Friday,  1 Oct 2021 at 22:41, Nicolas Goaziou wrote:
> Yes: habit. Also, I don't think LaTeX treats it as a block element.
> E.g.,
>
>     text
>     \[1+1=3\]
>     text
>
> is a single paragraph in LaTeX.

This is true and probably the most convincing argument in this debate.

I have no problem with the current parsing/interpretation in org even
though most of my writing in org treats it as a LaTeX front end...

Thank you,
eric
-- 
: Eric S Fraga via Emacs 28.0.50, Org 9.5-g9a4a24
: Latest paper written in org: https://arxiv.org/abs/2106.05096

Re: [PATCH] Don't fill displayed equations

[Timothy]

[-- Attachment #1: Type: text/plain, Size: 1220 bytes --]

Hi  Eric,

>> text
>> 1+1=3
>> text
>>
>> is a single paragraph in LaTeX.

> This is true and probably the most convincing argument in this debate.

I can’t help but think that this line of thinking is a bit of a trap, because
what LaTeX’s idea of a paragraph does not line up with Org’s. I think the more
pertinent consideration is that `\[ ... \]' changes LaTeX into vertical mode,
which does not allow for any other content on the ’line’ — which is far
closer to Org’s concept of a block element than an inline element.

Frankly, I think the issue is that the semantics of `\[ ... \]' simply don’t play
well with Org. Unfortunately, due to the Org/LaTeX difference I don’t see any
ideologically “pure” way we can have it in Org. The most idiomatic solution
would be to just remove support for `\[ ... \]' *. Otherwise, we have to treat
`\[ ... \]' specially in if we want ’sensible’ behaviour — as I mention in my
reply to Nicolas, we’re already parsing it differently to other inline markup…

All the best,
Timothy

* However, I am not advocating for this: There’s both the obvious comparability
  issue, but I also rather like using `\[ ... \]' myself.

Re: Org syntax: \[ \] as block element instead of inline object

[Stefan Nobis]

Max Nikulin <manikulin@gmail.com> writes:

> I like Timothy's idea that \[ \] markup should behave like
> block-level element rather than inline object.

On the other hand, we will loose the ability to inline small parts of
math, that should be typeset as separate equation, like:

    ...text \[ 1+2=3 \] more text...

I'm quite undecided in this regard. Personally, in almost all cases I
try to let my equations stand out even in the source (Org or LaTeX),
so I mostly use the "\begin{equation*}...\end{equation*}" variant.

At least, Nicolas convinced me, that comparing to LaTeX does not help
much (how LaTeX treats \[...\] is independent from how we want to
structure Org documents).

BTW: In LaTeX, I like to make equations stand out even further like
this:

--8<---------------cut here---------------start------------->8---
Some text
%
\begin{equation*}
...
\end{equation*}
%
more text
--8<---------------cut here---------------end--------------->8---

Is there a simple way to achieve this in Org (without using ugly
solutions like "@@latex:%@@" - something concise, at most 2-3 chars,
better only 1, but still visible in order to distinguish between
compound block belonging to a single paragraph and separate
paragraphs)?

-- 
Until the next mail...,
Stefan.

Re: [PATCH] Don't fill displayed equations

[Nicolas Goaziou]

Hello,

Timothy <tecosaur@gmail.com> writes:

> Is it? I can't use verbatim like this:
>
> =
> some
> verbatim
> text
> =
>
> but I can do
>
> \[
> some
> display
> equation
> \]
>
> It seems to me that \[ ... \] is already treated differently from other
> inline markup.

There is some misunderstanding here. 

You cannot use verbatim like the above because its definition forbids
spaces on the inner side of the markers (for obvious reasons). There is
no such restriction in \[...\] markup. For citations, you can also write

  [@cite:
  @foo
  ]

if you like.

But this is orthogonal to the type of element, i.e., inline or block.

Inline means the object is always enclosed in a paragraph (or a verse
block, or possibly a table cell). In particular, it cannot get past the
boundaries of its container. Corollary: since a blank line in Org ends
a paragraph, objects cannot contain blank lines. Both verbatim objects
and \[...\] snippets share those limitations.

> If that's the only way that Org could treat \[ ... \] differently from
> \( ... \), I'd be strongly in favour of this.

I think it is not necessarily a bad thing that \(...\) and \[...\] are
the same. Some export back-ends can tell the difference between them,
others do not care. This is the same for, e.g., verbatim and code. Not
all back-ends use a different output for them. IOW, it is not
necessarily right to treat them differently just because some back-ends
do treat them differently. Org is simply agnostic to this subtleties.

> I prefer \[ ... \] over \begin{equation*}...\end{equation*} as it's much
> more succinct, and helps reduce the "markup noise" in my documents. 

This is all about taste. But at least you have a choice. With your
patch, I may have to struggle with filling whenever some paragraph
contains \[...\], without any choice.

Also, it could be possible to overlay "\[" over \begin{equation*} thus
negating the markup noise.

> I don't think this is an insignificant concern, brevity may not be
> something I'm very good at in emails 😛 but is something I look for in
> syntax.

You probably have noticed that Org syntax is not very much into brevity.

> I must admit, I don't see the downside here --- how does this break the
> filling function for the rest of you? This only affects \[ ... \] blocks
> that have already been put on their own line.

No it doesn't. Without additional guards, filling a paragraph could
split a line and send an otherwise mid-line block at the beginning of
a line. But this is not the point. The point is much more basic,
actually. It is related to the uniformity of filling behaviour, as
already explained.

> Why can't we apply LaTeX expectations to LaTeX elements in Org? Applying
> LaTeX expectations to Org as a whole is clearly a silly idea, but Org
> copies \[ .. \] from LaTeX and it is a LaTeX construct.

Nope, it is obviously borrowed to LaTeX, but they are not the same.

I think I understand where you're coming from. Relying much on LaTeX,
you probably grew habits on how your equations should be formatted in
a LaTeX document. Applying this formatting in an Org document doesn't
work, tho, because Org has little understanding of true LaTeX syntax. It
just needs a way to quickly write maths.

  \[
  ... equation ...
  \]

should be seen as a LaTeXism.

>> Notwithstanding filling behaviour, \[...\] in Org is much more limited
>> than \[...\] in LaTeX.
>
> I'd be curious to hear how, as I personally haven't run into any
> instances where \[ ... \] has behaved differently other than when an
> environment starts on a new line in of a \[ ... \] block (which can
> easily be fixed by putting something like \!\ at the start of the
> line).

As explained above, for example, \[...\] cannot contain blank lines.
They cannot contain, e.g., "|" at the beginning of a line, too.
Full-fledged LaTeX environments do not have those limitations.

> I don't want "advanced" LaTeX code, I just want my display equations to
> be treated as display equations consistently 😂.

It is a "display equation" in LaTeX. There is no such thing as
a "display equation" in Org, even though you probably see it as such due
to your LaTeX background. There, \[...\] is just another way to write
maths within a paragraph.

Regards,
-- 
Nicolas Goaziou

Re: [PATCH] Don't fill displayed equations

[Eric S Fraga]

On Saturday,  2 Oct 2021 at 18:18, Timothy wrote:
> I can’t help but think that this line of thinking is a bit of a trap, because

I think you're right.  Nicolas's view that we should not think of org as
a front-end for LaTeX is probably more to the point.  As Stefan has
noted, how LaTeX processes \[...\] in the context of paragraphs is much
more complex (as it should be as typography is an incredibly difficult
task to do well, and TeX/LaTeX generally does it very well).  Trying to
have org mirror LaTeX in some way is doomed to fail.

I am happy with \[...\] being treated as an inline element.  That is
more sympathetic to LaTeX than the opposite.

-- 
: Eric S Fraga via Emacs 28.0.50, Org 9.5-g9a4a24
: Latest paper written in org: https://arxiv.org/abs/2106.05096

Re: [PATCH] Don't fill displayed equations

[Timothy]

[-- Attachment #1: Type: text/plain, Size: 487 bytes --]

Hi Nicolas,

> *snip lots of text*

Thanks for going through my points in detail. I think I understand your
perspective much better now. At this point though, I’m not really sure what to
make of `\[ ... \]', I now feel like it’s sitting in some sort of markup limbo
where it can’t be either fully LaTeX-y or fully Org-y. I still think it would be
good to improve this, but I no longer have such a firm idea that “modifying fill
is the way”.

All the best,
Timothy

Re: [PATCH] Don't fill displayed equations

[Max Nikulin]

On 02/10/2021 18:24, Eric S Fraga wrote:
> On Saturday,  2 Oct 2021 at 18:18, Timothy wrote:
>> I can’t help but think that this line of thinking is a bit of a trap, because
> 
> I think you're right.  Nicolas's view that we should not think of org as
> a front-end for LaTeX is probably more to the point.  As Stefan has
> noted, how LaTeX processes \[...\] in the context of paragraphs is much
> more complex (as it should be as typography is an incredibly difficult
> task to do well, and TeX/LaTeX generally does it very well).  Trying to
> have org mirror LaTeX in some way is doomed to fail.
> 
> I am happy with \[...\] being treated as an inline element.  That is
> more sympathetic to LaTeX than the opposite.

\begin{equation}
   1 != 2
\end{equation}

does not start new paragraph in LaTeX and formats expression as a 
separate block in display math mode similar to \[ \]. However in Org 
\begin{equation} starts block element, so terminates preceding 
paragraph. At the same time Org export backend is transparent enough, so 
Org paragraphs separated by equation blocks may become parts of single 
paragraphs in LaTeX.

So formatting paragraphs properly by LaTeX does not depend whether \[ \] 
is treated as inline object or as block-level element in Org. I really 
do not see a reason for idiosyncrasy that markup intended to add LaTeX 
snippet that looks like exactly as LaTeX commands for this purpose and 
even actually preserved during export to LaTeX should have different 
semantics for Org parser. It is rather a source of confusion and nothing 
more.

Re: [PATCH] Don't fill displayed equations

[Tom Gillespie]

> do not see a reason for idiosyncrasy that markup intended to add LaTeX
> snippet that looks like exactly as LaTeX commands for this purpose and
> even actually preserved during export to LaTeX should have different
> semantics for Org parser.

The answer is that \[ \] can only occur inside paragraphs. The issues
here are exactly the same as the issues for inline footnotes. Org gives
us a bit more power, but not the full power because Org is Org, not
Latex. Making \[ \] available outside of a paragraph would be a massive
breaking change.

In Timothy's original example he is narrowly skirting the syntax to
allow that all to remain a single paragraph, but stick in a newline
anywhere and boom, no more paragraph, no more equation.

I guess one thing I'm missing/not understanding is when/why people
want to use \[ \] instead of full #+begin_export latex block?

Best,
Tom

Re: [PATCH] Don't fill displayed equations

[Timothy]

[-- Attachment #1: Type: text/plain, Size: 1148 bytes --]

Hi Tom,

> The answer is that  can only occur inside paragraphs. The issues
> here are exactly the same as the issues for inline footnotes. Org gives
> us a bit more power, but not the full power because Org is Org, not
> Latex. Making  available outside of a paragraph would be a massive
> breaking change.
>
> In Timothy’s original example he is narrowly skirting the syntax to
> allow that all to remain a single paragraph, but stick in a newline
> anywhere and boom, no more paragraph, no more equation.

I don’t understand what you’re talking about here. You can already use `\[ ... \]'
outside a paragraph, e.g.

┌────
│ blah blah blah
│ 
│ \[
│   not part of a paragraph
│ \]
│ 
│ blah blah blah
└────

I also don’t see how footnotes are analogous, as footnotes are placed in the
middle of a line of text.

If you could explain your thoughts here a bit more, that would be appreciated.

> I guess one thing I’m missing/not understanding is when/why people
> want to use  instead of full #+begin_export latex block?

org-latex-preview :)

All the best,
Timothy

Re: [PATCH] Don't fill displayed equations

[Tom Gillespie]

Hi Timothy,

> │ \[
> │   not part of a paragraph
> │ \]

My point is that that parses first as a paragraph (check org-element-at-point).
\[ and \] would be meaningless if it did not first parse as a paragraph.

> I also don’t see how footnotes are analogous, as footnotes are placed in the
> middle of a line of text.

Inline footnotes [fn::
can span
multiple lines] but can't contain empty lines because the empty line ends the
paragraph that they are contained in.

> org-latex-preview :)

But surely #+begin_export latex works with org-latex-preview? If not then
that would be a feature request to org-latex-preview yes?

Best!
Tom

Re: org-latex-preview and latex export blocks

[Timothy]

[-- Attachment #1: Type: text/plain, Size: 752 bytes --]

Tom Gillespie <tgbugs@gmail.com> writes:

> But surely #+begin_export latex works with org-latex-preview? If not then
> that would be a feature request to org-latex-preview yes?

It might be possible, but I don’t think you can generally expect #+begin_export
latex blocks to be either previewable in isolation, or something one would want
to preview.

This separation is also useful for packages like org-fragtog, which auto-toggles
equations[1] previews on cursor entry/exit.

Basically, LaTeX export blocks and maths are (in my view) two different things,
even if both use LaTeX syntax.

All the best,
Timothy



Footnotes
─────────

[1] `\( ... \) =, =\[ ... \]', and `\begin{*math} ... \end{*math}' bits

Re: [PATCH] Don't fill displayed equations

[Ihor Radchenko]

Timothy <tecosaur@gmail.com> writes:

> Hi Nicolas,
>
>> *snip lots of text*
>
> Thanks for going through my points in detail. I think I understand your
> perspective much better now. At this point though, I’m not really sure what to
> make of `\[ ... \]', I now feel like it’s sitting in some sort of markup limbo
> where it can’t be either fully LaTeX-y or fully Org-y. I still think it would be
> good to improve this, but I no longer have such a firm idea that “modifying fill
> is the way”.

What about making org-fill-element modular?  We may define separate fill
functions for different elements and let the user override them
individually if the user prefer so.  It may be implemented similar to
export functionality with customisable formatters for different
elements.

Best,
Ihor

Re: [PATCH] Don't fill displayed equations

[Max Nikulin]

On 03/10/2021 00:51, Tom Gillespie wrote:
>> do not see a reason for idiosyncrasy that markup intended to add LaTeX
>> snippet that looks like exactly as LaTeX commands for this purpose and
>> even actually preserved during export to LaTeX should have different
>> semantics for Org parser.
> 
> The answer is that \[ \] can only occur inside paragraphs.

Nicolas mentioned that meaning of \[ \] might be changed and that will 
allow among other things correctly working of paragraph filling. I like 
such idea.

> Org gives
> us a bit more power, but not the full power because Org is Org, not
> Latex.

There is \( \) object for inline math. Is there use cases for 2 
(actually 2 groups of 2 elements counting $...$ and $$...$$) variants 
for inline math when primary backend for them behaves differently?

> Making \[ \] available outside of a paragraph would be a massive
> breaking change.

Is it really breaking? I can not estimate required amount of work to 
implement it. However at the user side at first glance most of files 
should remain valid and I could not imagine markup that will be broken 
during export to LaTeX.

> I guess one thing I'm missing/not understanding is when/why people
> want to use \[ \] instead of full #+begin_export latex block?

For example, because document without equations may become almost 
useless in the case of export to HTML or ODT file.

Re: [PATCH] Don't fill displayed equations

[Timothy]

[-- Attachment #1: Type: text/plain, Size: 525 bytes --]

Hi Ihor,

> What about making org-fill-element modular?  We may define separate fill
> functions for different elements and let the user override them
> individually if the user prefer so.  It may be implemented similar to
> export functionality with customisable formatters for different
> elements.

Thanks for that idea. Perhaps something along those lines may be the best
solution here. If we don’t want to make the whole function modular, perhaps we
could make a setting for this?

All the best,
Timothy

Re: [PATCH] Don't fill displayed equations

[Ihor Radchenko]

Timothy <tecosaur@gmail.com> writes:

> Hi Ihor,
>
>> What about making org-fill-element modular?  We may define separate fill
>> functions for different elements and let the user override them
>> individually if the user prefer so.  It may be implemented similar to
>> export functionality with customisable formatters for different
>> elements.
>
> Thanks for that idea. Perhaps something along those lines may be the best
> solution here. If we don’t want to make the whole function modular, perhaps we
> could make a setting for this?

I personally would prefer modular function as a whole.  For my taste,
Org code has too much of (case (variant 1) (variant 2) ...)-style
functions (i.e. org-todo, org-cycle, org-ctrl-c-ctrl-c, org-store-link,
etc) and they are a pain to debug and advice for users. 

Best,
Ihor

Re: [PATCH] Don't fill displayed equations

[Timothy]

[-- Attachment #1: Type: text/plain, Size: 568 bytes --]

Ihor Radchenko <yantar92@gmail.com> writes:

> I personally would prefer modular function as a whole.  For my taste,
> Org code has too much of (case (variant 1) (variant 2) …)-style
> functions (i.e. org-todo, org-cycle, org-ctrl-c-ctrl-c, org-store-link,
> etc) and they are a pain to debug and advice for users. 

Mmmm, I’m with you on this in general. I also think the code base has far too
many monolithic “200 line” functions which are crying out to be split up.
Now if I could just clone myself once or twice… 😛

All the best,
Timothy

Re: [PATCH] Don't fill displayed equations

[Ihor Radchenko]

Timothy <tecosaur@gmail.com> writes:

> Ihor Radchenko <yantar92@gmail.com> writes:
>
>> I personally would prefer modular function as a whole.  For my taste,
>> Org code has too much of (case (variant 1) (variant 2) …)-style
>> functions (i.e. org-todo, org-cycle, org-ctrl-c-ctrl-c, org-store-link,
>> etc) and they are a pain to debug and advice for users. 
>
> Mmmm, I’m with you on this in general. I also think the code base has far too
> many monolithic “200 line” functions which are crying out to be split up.
> Now if I could just clone myself once or twice… 😛

Then would you mind proposing a patch for org-fill-element in
particular? At least, you seem to have a motivation for this particular
function ;)

Best,
Ihor

Re: [PATCH] Don't fill displayed equations

[Timothy]

[-- Attachment #1: Type: text/plain, Size: 304 bytes --]

Hi Ihor,

> Then would you mind proposing a patch for org-fill-element in
> particular? At least, you seem to have a motivation for this particular
> function ;)

Perhaps in a few weeks, for now I’m a bit to busy for anything other than
“accidental” 😛 patches.

All the best,
Timothy

Re: [PATCH] Don't fill displayed equations

[Stefan Nobis]

Max Nikulin <manikulin@gmail.com> writes:

> On 03/10/2021 00:51, Tom Gillespie wrote:

>> I guess one thing I'm missing/not understanding is when/why people
>> want to use \[ \] instead of full #+begin_export latex block?

> For example, because document without equations may become almost
> useless in the case of export to HTML or ODT file.

Remember, as Nicolas said, the alternative to \[...\] is not a
"#+begin_export" block, but

\begin{equation*}
...
\end{equation*}

-- 
Until the next mail...,
Stefan.

Re: [PATCH] Don't fill displayed equations

[Max Nikulin]

On 03/10/2021 17:56, Stefan Nobis wrote:
> Max Nikulin writes:
>> On 03/10/2021 00:51, Tom Gillespie wrote:
> 
>>> I guess one thing I'm missing/not understanding is when/why people
>>> want to use \[ \] instead of full #+begin_export latex block?
> 
>> For example, because document without equations may become almost
>> useless in the case of export to HTML or ODT file.
> 
> Remember, as Nicolas said, the alternative to \[...\] is not a
> "#+begin_export" block, but
> 
> \begin{equation*}
> ...
> \end{equation*}

Maybe you are right and Tom was actually assuming \begin{equation*}, not 
#+begin_export latex. Since Tom and Timothy started discussion of LaTeX 
preview issues, it was better to clarify that #+begin_export is not a 
rescue here at all, it has different purpose.

Just as Timothy, I believe that \begin{equation*} is unnecessary verbose 
when \[ works *mostly* in a similar way. It shifts the border when it is 
easier to avoid Org for particular paper and to write LaTeX markup directly.

Another branch of this thread is refactoring of `org-fill-element', but 
in my opinion, if \[ \] remains an inline object, adding a special case 
for it is anyway increasing overall mess. It \[ \] were became 
block-level element, it would not be necessary.

Re: [PATCH] Don't fill displayed equations

[Ihor Radchenko]

Max Nikulin <manikulin@gmail.com> writes:

>> Making \[ \] available outside of a paragraph would be a massive
>> breaking change.
>
> Is it really breaking? I can not estimate required amount of work to 
> implement it. However at the user side at first glance most of files 
> should remain valid and I could not imagine markup that will be broken 
> during export to LaTeX.

If I understand correctly, making \[ \] available outside paragraph
would mean that it becomes a new element (currently \[\] is a
latex-fragment object). New element means that all the exporters will
need to be updated and handle this new element specially. Indeed, it
will be a breaking change.

Best,
Ihor

Re: [PATCH] Don't fill displayed equations

[Tom Gillespie]

Some thoughts.

> Maybe you are right and Tom was actually assuming \begin{equation*}, not
> #+begin_export latex.

Correct. My bad on that one.

> Just as Timothy, I believe that \begin{equation*} is unnecessary verbose
> when \[ works *mostly* in a similar way.

\begin{equation*} is absolutely required if you want to be able to include
newlines because \[ and \begin are not similar at all as far as parsing
is concerned.

From the spec: https://orgmode.org/worg/dev/org-syntax.html#LaTeX_Environments
> CONTENTS can contain anything but the “\end{NAME}” string.
The spec is not completely accurate since latex environments can't
contain a new heading, but the point is that latex environments are
elements, whereas \[ \] is an object.

> If I understand correctly, making \[ \] available outside paragraph
> would mean that it becomes a new element (currently \[\] is a
> latex-fragment object).

Correct. Promoting \[ to an element would mean every \ in an org file
becomes a stop word. Also, Since full fledged latex environments
already exist to serve this purpose I find it hard to justify, especially
given that Org tries to give clear indication of when a block structure
is starting and ending.

> Isn't the whole point of the \[ ... \], \( ... \), $ ... $, $$ ... $$,
> and \begin{env} ... \end{env} and constructs in Org to be consistent
> with LaTeX?

For \begin and \end yes. For the others no. In general it would be to
make it possible to express things using latex-like syntax that would
otherwise require Org to come up with some new and different syntax.
These are values that may be translated to latex, but they exist inside
a larger syntax that is decidedly not latex, and thus they only have
meaningful translation to latex if they exist as well formed Org.

As a side note, the $ syntax is slated to be deprecated and removed.
https://orgmode.org/worg/dev/org-syntax.html#Entities_and_LaTeX_Fragments
> It would introduce incompatibilities with previous Org versions, but
> support for $...$ (and for symmetry, $$...$$) constructs ought to be removed.

> Indeed, it will be a breaking change.

I'm actually fairly certain that such a change should never be made
due to the recent changes in org link syntax. Specifically given how
\[ is used for escapes in links. https://orgmode.org/manual/Link-Format.html
This means that the only place you could reliably use \[ is at the start of a
new line preceded only by whitespace. However, if this were to happen then
pretty much every org document that uses \[ \] is at risk for being broken
because something that was once a single paragraph will now be multiple
paragraphs.

If you need multiline use \begin \end, that is what they are there for, and they
fit better with org's general extensible approach to blocks. I would dearly love
to be able to have a single shorthand for src blocks that worked inline and
standalone, but the complexity that it would induce is just not worth it. Same
thing for \[ \]. It seems simple until you get down to account for all the edge
that it would induce in the grammar.

Consider the case where you have something like

\[ something something

more content
more content [[www.example.com/\]oops][evil link]] \]

I've seen enough cases that are similar to this in the existing implementation
that have inconsistent behavior that I can safely say that this one would too.
Not to mention that I can think of at least 3 different cases that will all have
slightly different behavior that is inexplicable to users at best and
infuriating
at worst.

\[ a

b \]

\[
a
b
\]

a \[ b

c \] d

etc. There are plenty more variants that would all be subtly different depending
on the exact way such a thing were implemented.

In short. Just not worth it.

Re: [PATCH] Don't fill displayed equations

[Timothy]

Hi All,

Given the way the conversation has evolved I think it may be worth revisiting
another viewpoint to see if anybody has any nice ideas.

As things are, regardless of the complexities of parsing and inline vs.
block elements etc., just considering the user experience when running
org-fill on an example like the one below, the behaviour isn't /nice/.

Might there be some way we could allow for people who are used to LaTeX
and/or like \[ ... \] to have fill behaviour that leaves \[ ... \]
blocks alone? My original idea was changing how org-fill works for
everyone, the discussion has gone on to changing \[ ... \] to a block
element -- both of these seem to have rather significant issues.

Ihor came up with the idea of making org-fill something more
customisable by the user, so that someone could have this behaviour
without making Org behave un-idiomatically for everyone. I proposed
maybe just having a setting could be a halfway house between my original
patch and his idea.

Does anybody have any other thoughts?

> If you’re wondering why I’m so opposed to the current behaviour, that is probably
> best explained by a more realistic demo that what I have in the commit message.
>
> ┌────
> │ Since \(\cos\) is an even function, we can negate the numerator of the argument
> │ without changing the result, giving
> │ \[
> │   \cos \left( \pi \frac{C_1-x}{2C_1+D} \right) \ , \quad C_1 = \frac{D}{2}.
> │ \]
> │ this will be positive over \(x \in (0,D)\), and so we can rewrite \(\tilde{y}\) as,
> │ \[
> │   \tilde{y}(x) = \frac{2D}{\pi} \log \cos \left( \pi \frac{\frac{D}{2}-x}{2D} \right) + C_2.
> │ \]
> │ Once again considering that \(y(0)=y(D)=0\), it is clear that
> │ \[
> │   C_2 = - \frac{2D}{\pi} \log \cos \left( \frac{\pi}{4} \right) = - \frac{2D}{\pi} \log 2^{-\frac{1}{2}} = \frac{D}{\pi} \log 2.
> │ \]
> │ The complete solution for \(\tilde{y}\) is hence,
> │ \[
> │   \tilde{y} = \frac{2D}{\pi} \log \cos \left( \pi \frac{D-2x}{4D} \right) + \frac{D}{\pi} \log 2.
> │ \]
> └────
> is currently filled to
> ┌────
> │ Since \(\cos\) is an even function, we can negate the numerator of the argument
> │ without changing the result, giving \[ \cos \left( \pi \frac{C_1-x}{2C_1+D}
> │ \right) \ , \quad C_1 = \frac{D}{2}. \] this will be positive over \(x \in (0,D)\),
> │ and so we can rewrite \(\tilde{y}\) as, \[ \tilde{y}(x) = \frac{2D}{\pi} \log \cos \left( \pi
> │ \frac{\frac{D}{2}-x}{2D} \right) + C_2. \] Once again considering that
> │ \(y(0)=y(D)=0\), it is clear that \[ C_2 = - \frac{2D}{\pi} \log \cos \left(
> │ \frac{\pi}{4} \right) = - \frac{2D}{\pi} \log 2^{-\frac{1}{2}} = \frac{D}{\pi} \log 2.
> │ \] The complete solution for \(\tilde{y}\) is hence, \[ \tilde{y} = \frac{2D}{\pi} \log \cos
> │ \left( \pi \frac{D-2x}{4D} \right) + \frac{D}{\pi} \log 2. \]
> └────

--
Timothy

Re: [PATCH] Don't fill displayed equations

[Tom Gillespie]

> Does anybody have any other thoughts?

From time to time I encounter random patterns that I don't want to be
reformatted during a fill operation. Maybe a custom variable like
org-fill-paragraph-skip-regexp or similar that could be set by the user?
For Timothy's use case he would set it to the regexp provided in the
original patch? Not sure how much of the implementation in the patch
is dependent on that particular regexp, but a general solution that
could even be set per org file might be a very useful new feature.

Best!
Tom

Re: [PATCH] Don't fill displayed equations

[Timothy]

[-- Attachment #1: Type: text/plain, Size: 764 bytes --]

Hi Tom,

> From time to time I encounter random patterns that I don’t want to be
> reformatted during a fill operation. Maybe a custom variable like
> org-fill-paragraph-skip-regexp or similar that could be set by the user?
> For Timothy’s use case he would set it to the regexp provided in the
> original patch? Not sure how much of the implementation in the patch
> is dependent on that particular regexp, but a general solution that
> could even be set per org file might be a very useful new feature.

Hmmm, that’s an interesting idea. I suppose you could have something like a list
of cons cells where the car is a regexp matching the start of the “don’t touch”
region and the cdr a regexp matching the end.

All the best,
Timothy

Re: [PATCH] Don't fill displayed equations

[Przemysław Pietrzak]

[-- Attachment #1: Type: text/plain, Size: 705 bytes --]

Hi Org-mode community!

> Does anybody have any other thoughts?

I have been following your discussion for several days
and if I understand it correctly promoting \[ to an element
is the most controversial.

Isn’t the behavior of fill-paragraph
(see https://emacs.stackexchange.com/q/29122/14127 <https://emacs.stackexchange.com/q/29122/14127>)
the biggest inconvenience?

Perhaps adding a proper fill-nobreak-predicate 
(see https://emacs.stackexchange.com/a/12419/14127 <https://emacs.stackexchange.com/a/12419/14127>)
would be enough?

This regex https://stackoverflow.com/a/14537848 <https://stackoverflow.com/a/14537848> may be a good starting point.

Best regards!
Premo




[-- Attachment #2: Type: text/html, Size: 3750 bytes --]

Re: [PATCH] Don't fill displayed equations

[Max Nikulin]

My thanks to Ihor for the argument concerning code of (maybe external) 
exproters and to Tom for parser-related considerations.

I think it is a kind of trade-off: breaking change vs. continuously 
breaking user experience. Do not worry too much, there is almost no 
chance that I will try to implement my plan, so current state will 
remain as is. In naive form my idea was:

1. Find the code where \( \) is parsed and remove \[ \] option
2. Find the place where \begin{equation} is handled and add \[ somewhere 
around.

I fill some misunderstanding since some arguments has the form "it is 
too hard because it requires ..." where "..." exactly what I was going 
to discuss.

On 04/10/2021 12:57, Tom Gillespie wrote:
> 
>> Maybe you are right and Tom was actually assuming \begin{equation*}, not
>> #+begin_export latex.
> 
> Correct. My bad on that one.

By the way, I realized that choice of "\begin{equation}" and not 
"#+begin_equation" is not clear to me. However it allows to pass 
\begin{equation} to LaTeX or HTML+MathJax transparently.

>> Just as Timothy, I believe that \begin{equation*} is unnecessary verbose
>> when \[ works *mostly* in a similar way.
> 
> \begin{equation*} is absolutely required if you want to be able to include
> newlines because \[ and \begin are not similar at all as far as parsing
> is concerned.
> 
>  From the spec: https://orgmode.org/worg/dev/org-syntax.html#LaTeX_Environments
>> CONTENTS can contain anything but the “\end{NAME}” string.
> The spec is not completely accurate since latex environments can't
> contain a new heading, but the point is that latex environments are
> elements, whereas \[ \] is an object.
                           ^^^^^^^^^^^^
I was going to discuss namely this change.

>> Isn't the whole point of the \[ ... \], \( ... \), $ ... $, $$ ... $$,
>> and \begin{env} ... \end{env} and constructs in Org to be consistent
>> with LaTeX?
> 
> For \begin and \end yes. For the others no.

Unsure, but then forbidding \[ \] at all may result in less confusion 
then current state "inline object that is transformed to block element 
during export". Making \[ \] a special case for paragraph filling when 
it is inline object would be even more confusing.

> I'm actually fairly certain that such a change should never be made
> due to the recent changes in org link syntax. Specifically given how
> \[ is used for escapes in links. https://orgmode.org/manual/Link-Format.html
> This means that the only place you could reliably use \[ is at the start of a
> new line preceded only by whitespace. However, if this were to happen then
> pretty much every org document that uses \[ \] is at risk for being broken
> because something that was once a single paragraph will now be multiple
> paragraphs.

Agree that context-dependent "\]" may cause problems.

> If you need multiline use \begin \end, that is what they are there for, and they
> fit better with org's general extensible approach to blocks. I would dearly love
> to be able to have a single shorthand for src blocks that worked inline and
> standalone,

I do not suggest using \[ \] both for inline object and block elements. 
I hope that even if behavior of \[ \] inside paragraphs were not 
specified it would not break user files since it will be transparently 
passed to LaTeX or HTML. However, LaTeX exporter may escape them to have 
it literally in the output.

> Consider the case where you have something like
> 
> \[ something something
> 
> more content
> more content [[www.example.com/\]oops][evil link]] \]

Frankly speaking, with Org I never sure in such cases whether everything 
inside \[ \] will be literally copied to LaTeX file or some some Org 
markup will be handled, so it is possible to insert something e.g. 
inside \text{}.

> \[ a
> 
> b \]
> 
> \[
> a
> b
> \]
> 
> a \[ b
> 
> c \] d

Similar variants exist for \begin{equation} \end{equation}

> In short. Just not worth it.

The value of TeX, markdown, reStructured text, Org is convenience to 
type in comparison with SGML&XML-based formats that are not really human 
friendly. In this sense \[ is noticeably better than \begin{equation*}.

Re: [PATCH] Don't fill displayed equations

[Ihor Radchenko]

Timothy <tecosaur@gmail.com> writes:

> Hi Ihor,
>
>> Then would you mind proposing a patch for org-fill-element in
>> particular? At least, you seem to have a motivation for this particular
>> function ;)
>
> Perhaps in a few weeks, for now I’m a bit to busy for anything other than
> “accidental” 😛 patches.

Quite a few weeks have passed ;)

Best,
Ihor