Bug: wrong interpretation of LaTeX [8.2.6 (8.2.6-47-ge3d2c1-elpa @ c:/Users/beffa/.emacs.d/elpa/org-20140526/)]

Bug: wrong interpretation of LaTeX [8.2.6 (8.2.6-47-ge3d2c1-elpa @ c:/Users/beffa/.emacs.d/elpa/org-20140526/)]

[Federico Beffa]

Hi,

I'm a long time LaTeX user starting to use the excellent org-mode.
I've noticed what I believe is a wrong interpretation of the LaTeX
syntax by org-mode:

According to the LaTeX manual and reference "LaTeX: A Document
Preparation System", L. Lamport, \[ ... \] is a short form for a
displaymath environment. Citing the reference:
"... Because displayed equations are used so frequently in
mathematics, LaTeX allows you to type \[ ... \] instead of
\begin{displaymath} ... \end{displaymath}. ..."

However, org-mode classify \[ ... \] as a latex-fragment, the same as
\(...\). The two are however very different in LaTeX because, while
the latter displays some mathematical expression *inline*, the former
makes its content stand out by putting it on *its own line*.

What I do not like about this is that "org-fill-paragraph" considers
the \[ ...\] environment part of a paragraph and therefore the
environment gets "lost" in the middle of a line.

Here a simple org file:
------------------------------------------------------------------
* Intro

First paragraph with equation environment:
\begin{equation}
E = mc^2  .
\end{equation}

Second paragraph with short form of displaymath environment:
\[
E = \hbar\omega  .
\]

(setq fbe-tmp (org-element-parse-buffer))
------------------------------------------------------------------

and here how it is parsed
------------------------------------------------------------------
(org-data nil
      (headline
       (:raw-value "Intro" :begin 1 :end 226 :pre-blank 1 :hiddenp nil
:contents-begin 10 :contents-end 226 :level 1 :priority nil :tags nil
:todo-keyword nil :todo-type nil :post-blank 0 :footnote-section-p nil
:archivedp nil :commentedp nil :quotedp nil :CATEGORY nil :title
               (#("Intro" 0 5
              (:parent #1)))
               :parent #0)
       (section
        (:begin 10 :end 226 :contents-begin 10 :contents-end 226
:post-blank 0 :parent #1)
        (paragraph
         (:begin 10 :end 53 :contents-begin 10 :contents-end 53
:post-blank 0 :post-affiliated 10 :parent #2)
         #("First paragraph with equation environment:\n" 0 43
           (:parent #3)))
        (latex-environment
         (:begin 53 :end 98 :value "\\begin{equation}\nE = mc^2
.\n\\end{equation}\n" :post-blank 1 :post-affiliated 53 :parent #2))
        (paragraph
         (:begin 98 :end 185 :contents-begin 98 :contents-end 184
:post-blank 1 :post-affiliated 98 :parent #2)
         #("Second paragraph with short form of displaymath environment:\n" 0 61
           (:parent #3))
         (latex-fragment
          (:value "\\[\nE = \\hbar\\omega  .\n\\]" :begin 159 :end 183
:post-blank 0 :parent #3))
         #("\n" 0 1
           (:parent #3)))
        (paragraph
         (:begin 185 :end 226 :contents-begin 185 :contents-end 226
:post-blank 0 :post-affiliated 185 :parent #2)
         #("(setq fbe-tmp (org-element-parse-buffer))" 0 41
           (:parent #3))))))

------------------------------------------------------------------

Regards,
Fede


Emacs  : GNU Emacs 24.3.1 (i386-mingw-nt6.1.7601)
 of 2013-03-17 on MARVIN
Package: Org-mode version 8.2.6 (8.2.6-47-ge3d2c1-elpa @
c:/Users/beffa/.emacs.d/elpa/org-20140526/)

Re: Bug: wrong interpretation of LaTeX [8.2.6 (8.2.6-47-ge3d2c1-elpa @ c:/Users/beffa/.emacs.d/elpa/org-20140526/)]

[Nicolas Goaziou]

Hello,

Federico Beffa <beffa@ieee.org> writes:

> According to the LaTeX manual and reference "LaTeX: A Document
> Preparation System", L. Lamport, \[ ... \] is a short form for a
> displaymath environment. Citing the reference:
> "... Because displayed equations are used so frequently in
> mathematics, LaTeX allows you to type \[ ... \] instead of
> \begin{displaymath} ... \end{displaymath}. ..."
>
> However, org-mode classify \[ ... \] as a latex-fragment, the same as
> \(...\). The two are however very different in LaTeX because, while
> the latter displays some mathematical expression *inline*, the former
> makes its content stand out by putting it on *its own line*.

AFAIK, LaTeX allows to inline "\[...\]" constructs, so something like

  Some \[1+1\] text

is perfectly valid. Thus, I think we need to support them.

The other thing to consider is that having the same syntax for an inline
and a non-inline element could introduce some bugs (e.g. when filling
a paragraph).

OTOH, allowing inline \[...\] is pretty harmless.

> What I do not like about this is that "org-fill-paragraph" considers
> the \[ ...\] environment part of a paragraph and therefore the
> environment gets "lost" in the middle of a line.

This is a minor annoyance, indeed. However, you can use the verbose form
in this case (i.e., "\begin{displaymath}").


Regards,

-- 
Nicolas Goaziou

Re: Bug: wrong interpretation of LaTeX [8.2.6 (8.2.6-47-ge3d2c1-elpa @ c:/Users/beffa/.emacs.d/elpa/org-20140526/)]

[Federico Beffa]

Of course \[ 1+1 \] is valid LaTeX syntax, just as inline
\begin{displaymath} 1+1 \end{displaymath} is valid. In LaTeX you can
also separate paragraphs with \par without using any empty line, but
that's not very readable and in practice  this method is only used
when constructing macros.
My point is that the two forms have identical meaning in LaTeX, but
they are represented differently in org-mode.

In my opinion a construct which will be displayed on a line by itself
and with some space separating it from the preceding and the following
text lines such as "\[ ... \]" would be better represented as a
latex-environment and not an inline latex-fragment. In this way the
org-mode text representation with proper fill-paragraph handling would
be much more readable and consistent with the meaning of the syntax
that it borrowed.

Regards,
Fede

On Fri, Jul 25, 2014 at 8:32 PM, Nicolas Goaziou <mail@nicolasgoaziou.fr> wrote:
> Hello,
>
> Federico Beffa <beffa@ieee.org> writes:
>
>> According to the LaTeX manual and reference "LaTeX: A Document
>> Preparation System", L. Lamport, \[ ... \] is a short form for a
>> displaymath environment. Citing the reference:
>> "... Because displayed equations are used so frequently in
>> mathematics, LaTeX allows you to type \[ ... \] instead of
>> \begin{displaymath} ... \end{displaymath}. ..."
>>
>> However, org-mode classify \[ ... \] as a latex-fragment, the same as
>> \(...\). The two are however very different in LaTeX because, while
>> the latter displays some mathematical expression *inline*, the former
>> makes its content stand out by putting it on *its own line*.
>
> AFAIK, LaTeX allows to inline "\[...\]" constructs, so something like
>
>   Some \[1+1\] text
>
> is perfectly valid. Thus, I think we need to support them.
>
> The other thing to consider is that having the same syntax for an inline
> and a non-inline element could introduce some bugs (e.g. when filling
> a paragraph).
>
> OTOH, allowing inline \[...\] is pretty harmless.
>
>> What I do not like about this is that "org-fill-paragraph" considers
>> the \[ ...\] environment part of a paragraph and therefore the
>> environment gets "lost" in the middle of a line.
>
> This is a minor annoyance, indeed. However, you can use the verbose form
> in this case (i.e., "\begin{displaymath}").
>
>
> Regards,
>
> --
> Nicolas Goaziou

Re: Bug: wrong interpretation of LaTeX [8.2.6 (8.2.6-47-ge3d2c1-elpa @ c:/Users/beffa/.emacs.d/elpa/org-20140526/)]

[Nicolas Goaziou]

Federico Beffa <beffa@ieee.org> writes:

> Of course \[ 1+1 \] is valid LaTeX syntax, just as inline
> \begin{displaymath} 1+1 \end{displaymath} is valid.

But \begin{displaymath} 1+1 \end{displaymath} isn't valid in an Org
paragraph.

> In my opinion a construct which will be displayed on a line by itself
> and with some space separating it from the preceding and the following
> text lines such as "\[ ... \]" would be better represented as a
> latex-environment and not an inline latex-fragment. In this way the
> org-mode text representation with proper fill-paragraph handling would
> be much more readable and consistent with the meaning of the syntax
> that it borrowed.

That would prevent \[...\] to be recognized within a paragraph, as in
the example above. I don't think it's worth removing it as long as there
is a latex-environment alternative for it.


Regards,

-- 
Nicolas Goaziou

Re: Bug: wrong interpretation of LaTeX [8.2.6 (8.2.6-47-ge3d2c1-elpa @ c:/Users/beffa/.emacs.d/elpa/org-20140526/)]

[Federico Beffa]

I understand that. However, using \[...\] inline within a paragraph is
not the intended use of \[...\]. The use of \[...\] is for
mathematical expressions which you want to stand out, on their own
lines.

For inline mathematical expressions there are the constructs \(...\) and $...$.

Regards,
Fede

On Sat, Jul 26, 2014 at 10:26 AM, Nicolas Goaziou
<mail@nicolasgoaziou.fr> wrote:
> Federico Beffa <beffa@ieee.org> writes:
>
>> Of course \[ 1+1 \] is valid LaTeX syntax, just as inline
>> \begin{displaymath} 1+1 \end{displaymath} is valid.
>
> But \begin{displaymath} 1+1 \end{displaymath} isn't valid in an Org
> paragraph.
>
>> In my opinion a construct which will be displayed on a line by itself
>> and with some space separating it from the preceding and the following
>> text lines such as "\[ ... \]" would be better represented as a
>> latex-environment and not an inline latex-fragment. In this way the
>> org-mode text representation with proper fill-paragraph handling would
>> be much more readable and consistent with the meaning of the syntax
>> that it borrowed.
>
> That would prevent \[...\] to be recognized within a paragraph, as in
> the example above. I don't think it's worth removing it as long as there
> is a latex-environment alternative for it.
>
>
> Regards,
>
> --
> Nicolas Goaziou

Re: Bug: wrong interpretation of LaTeX [8.2.6 (8.2.6-47-ge3d2c1-elpa @ c:/Users/beffa/.emacs.d/elpa/org-20140526/)]

[Federico Beffa]

[-- Attachment #1: Type: text/plain, Size: 2834 bytes --]

Hi Nicolas,

I understand that org-mode is not LaTeX and therefore the org-mode
internal representation of LaTeX should not matter to end users like
me. That's fair enough. In the end of the day the reason why I started
looking into this is the fact that I do not find user friendly that
org-mode, when filling a paragraph, interprets \[...\] as any inline
snippet and does not keep it on a separate line (as recommended by the
LaTeX manual, see attached document). That's it.
I've therefore started messing around with
org-element-paragraph-separate, but after a while I thought that it
would be better to look at how the \begin...\end environments are
handled and try to do the same with \[...\]. That's how I noticed the
difference between the two (and perceived it as wrong handling of
\[...\]). I just thought that correcting the filling behavior of
org-mode would be useful to all users of org-mode and not just me.

In any case, if it's just me, I would greatly appreciate an advice on
how to achieve my goal from somebody with a good understanding of the
internals of org-mode. Is there a better way than just changing
org-element-paragraph-separate?

Thank you in advance for any advice.
Best Regards,
Fede

P.S.: I've attached for reference two pages, from the book written by
LaTeX's creator (the book I've mentioned on my first email), on how to
use the various mathematical constructs.

On Sat, Jul 26, 2014 at 11:20 AM, Federico Beffa <beffa@ieee.org> wrote:
> I understand that. However, using \[...\] inline within a paragraph is
> not the intended use of \[...\]. The use of \[...\] is for
> mathematical expressions which you want to stand out, on their own
> lines.
>
> For inline mathematical expressions there are the constructs \(...\) and $...$.
>
> Regards,
> Fede
>
> On Sat, Jul 26, 2014 at 10:26 AM, Nicolas Goaziou
> <mail@nicolasgoaziou.fr> wrote:
>> Federico Beffa <beffa@ieee.org> writes:
>>
>>> Of course \[ 1+1 \] is valid LaTeX syntax, just as inline
>>> \begin{displaymath} 1+1 \end{displaymath} is valid.
>>
>> But \begin{displaymath} 1+1 \end{displaymath} isn't valid in an Org
>> paragraph.
>>
>>> In my opinion a construct which will be displayed on a line by itself
>>> and with some space separating it from the preceding and the following
>>> text lines such as "\[ ... \]" would be better represented as a
>>> latex-environment and not an inline latex-fragment. In this way the
>>> org-mode text representation with proper fill-paragraph handling would
>>> be much more readable and consistent with the meaning of the syntax
>>> that it borrowed.
>>
>> That would prevent \[...\] to be recognized within a paragraph, as in
>> the example above. I don't think it's worth removing it as long as there
>> is a latex-environment alternative for it.
>>
>>
>> Regards,
>>
>> --
>> Nicolas Goaziou

[-- Attachment #2: latex-math.pdf --]
[-- Type: application/pdf, Size: 87567 bytes --]

Re: Bug: wrong interpretation of LaTeX [8.2.6 (8.2.6-47-ge3d2c1-elpa @ c:/Users/beffa/.emacs.d/elpa/org-20140526/)]

[Nicolas Goaziou]

Hello,

Federico Beffa <beffa@ieee.org> writes:

> In the end of the day the reason why I started looking into this is
> the fact that I do not find user friendly that org-mode, when filling
> a paragraph, interprets \[...\] as any inline snippet and does not
> keep it on a separate line (as recommended by the LaTeX manual, see
> attached document). That's it.

As explained previously, I'm not against the idea, but it would break
existing documents containing "\[...\]" in the middle of a line, even
though it is valid LaTeX. In any case, you will not get total LaTeX
conformance.

Anyway, I can't see any technical limitation to your suggestion. So we
might as well ask the ML. I'll write a consultative poll about it.


Regards,

-- 
Nicolas Goaziou

Re: Bug: wrong interpretation of LaTeX [8.2.6 (8.2.6-47-ge3d2c1-elpa @ c:/Users/beffa/.emacs.d/elpa/org-20140526/)]

[Federico Beffa]

In principle, to avoid breaking existing documents, you could
introduce a variable to set org-mode in legacy mode.

Regards,
Fede

On Sun, Jul 27, 2014 at 6:46 PM, Nicolas Goaziou <mail@nicolasgoaziou.fr> wrote:
> Hello,
>
> Federico Beffa <beffa@ieee.org> writes:
>
>> In the end of the day the reason why I started looking into this is
>> the fact that I do not find user friendly that org-mode, when filling
>> a paragraph, interprets \[...\] as any inline snippet and does not
>> keep it on a separate line (as recommended by the LaTeX manual, see
>> attached document). That's it.
>
> As explained previously, I'm not against the idea, but it would break
> existing documents containing "\[...\]" in the middle of a line, even
> though it is valid LaTeX. In any case, you will not get total LaTeX
> conformance.
>
> Anyway, I can't see any technical limitation to your suggestion. So we
> might as well ask the ML. I'll write a consultative poll about it.
>
>
> Regards,
>
> --
> Nicolas Goaziou

Re: Bug: wrong interpretation of LaTeX [8.2.6 (8.2.6-47-ge3d2c1-elpa @ c:/Users/beffa/.emacs.d/elpa/org-20140526/)]

[Nicolas Goaziou]

Hello,

Federico Beffa <beffa@ieee.org> writes:

> In principle, to avoid breaking existing documents, you could
> introduce a variable to set org-mode in legacy mode.

I don't want to maintain two (or more) grammars for Org. So, no, thanks.


Regards,

-- 
Nicolas Goaziou

Re: Bug: wrong interpretation of LaTeX [8.2.6 (8.2.6-47-ge3d2c1-elpa @ c:/Users/beffa/.emacs.d/elpa/org-20140526/)]

[Bastien]

Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

> That would prevent \[...\] to be recognized within a paragraph, as in
> the example above. I don't think it's worth removing it as long as there
> is a latex-environment alternative for it.

FWIW I fully agree here.

-- 
 Bastien