Re: [PATCH] Add support for $…$ latex fragments followed by a dash

Re: [PATCH] Add support for $…$ latex fragments followed by a dash

[Goran Zuzic]

[-- Attachment #1: Type: text/plain, Size: 242 bytes --]

On a personal note, I would be very happy to see this patch merged. It has
been annoying me for years, see for instance this thread:
https://emacs.stackexchange.com/questions/37207/org-latex-fragment-succeeded-by-a-dash-is-not-showing

Goran

[-- Attachment #2: Type: text/html, Size: 406 bytes --]

Re: [PATCH] Add support for $…$ latex fragments followed by a dash

[autofrettage]

Rudolf wrote:

> Further, \(\) brings 100% more characters than $$, resulting in more noise in the sentence.

Now where did I put my APL keyboard...

Cheers
Rasmus

[PATCH] Add support for $…$ latex fragments followed by a dash

[Sébastien Miquel]

[-- Attachment #1: Type: text/plain, Size: 380 bytes --]

Hi,

The attached patch adds support for $…$ latex fragments followed by a
dash, such as $n$-th.

This pattern is quite common, and the issue was mentionned by Rudolf
Adamkovič, here: https://list.orgmode.org/m2mtjvefrf.fsf@me.com/.

This extension of the syntax doesn't conflict with the use of $'s in
table formulas, or for currency, AFAICT.

Regards,

-- 
Sébastien Miquel

[-- Attachment #2: 0001-org-element-Support-LaTeX-fragments-followed-by-a-da.patch --]
[-- Type: text/x-patch, Size: 3159 bytes --]

From b023fff129a5cc3b1f12b9f170f2e018dc34d237 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?S=C3=A9bastien=20Miquel?= <sebastien.miquel@posteo.eu>
Date: Sun, 23 Jan 2022 13:28:03 +0100
Subject: [PATCH] =?UTF-8?q?org-element:=20Support=20$=E2=80=A6$=20LaTeX=20?=
 =?UTF-8?q?fragments=20followed=20by=20a=20dash?=
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

* lisp/org-element.el (org-element-latex-fragment-parser):
* lisp/org.el (org-latex-regexps): Allow a dash right after a
$…$ fragment.
* testing/lisp/test-org-element.el (test-org-element/latex-fragment-parser):
Add test.

Allows such uses as `$n$-th' or `$n$-dimensional'.
---
 lisp/org-element.el              | 2 +-
 lisp/org.el                      | 4 ++--
 testing/lisp/test-org-element.el | 4 ++++
 3 files changed, 7 insertions(+), 3 deletions(-)

diff --git a/lisp/org-element.el b/lisp/org-element.el
index b82475a14..40a604578 100644
--- a/lisp/org-element.el
+++ b/lisp/org-element.el
@@ -3416,7 +3416,7 @@ Assume point is at the beginning of the LaTeX fragment."
 		     (not (memq (char-before (match-beginning 0))
 				'(?\s ?\t ?\n ?, ?.)))
 		     (looking-at-p
-		      "\\(\\s.\\|\\s-\\|\\s(\\|\\s)\\|\\s\"\\|'\\|$\\)")
+		      "\\(\\s.\\|\\s-\\|\\s(\\|\\s)\\|\\s\"\\|'\\|-\\|$\\)")
 		     (point)))))
 	     (post-blank
 	      (if (not after-fragment) (throw 'no-object nil)
diff --git a/lisp/org.el b/lisp/org.el
index 2d439cd05..3a164ee77 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -629,8 +629,8 @@ An entry can be toggled between COMMENT and normal with
   '(("begin" "^[ \t]*\\(\\\\begin{\\([a-zA-Z0-9\\*]+\\)[^\000]+?\\\\end{\\2}\\)" 1 t)
     ;; ("$" "\\([ \t(]\\|^\\)\\(\\(\\([$]\\)\\([^ \t\n,.$].*?\\(\n.*?\\)\\{0,5\\}[^ \t\n,.$]\\)\\4\\)\\)\\([ \t.,?;:'\")]\\|$\\)" 2 nil)
     ;; \000 in the following regex is needed for org-inside-LaTeX-fragment-p
-    ("$1" "\\([^$]\\|^\\)\\(\\$[^ \t\r\n,;.$]\\$\\)\\(\\s.\\|\\s-\\|\\s(\\|\\s)\\|\\s\"\\|\000\\|'\\|$\\)" 2 nil)
-    ("$"  "\\([^$]\\|^\\)\\(\\(\\$\\([^ \t\n,;.$][^$\n\r]*?\\(\n[^$\n\r]*?\\)\\{0,2\\}[^ \t\n,.$]\\)\\$\\)\\)\\(\\s.\\|\\s-\\|\\s(\\|\\s)\\|\\s\"\\|\000\\|'\\|$\\)" 2 nil)
+    ("$1" "\\([^$]\\|^\\)\\(\\$[^ \t\r\n,;.$]\\$\\)\\(\\s.\\|\\s-\\|\\s(\\|\\s)\\|\\s\"\\|\000\\|'\\|-\\|$\\)" 2 nil)
+    ("$"  "\\([^$]\\|^\\)\\(\\(\\$\\([^ \t\n,;.$][^$\n\r]*?\\(\n[^$\n\r]*?\\)\\{0,2\\}[^ \t\n,.$]\\)\\$\\)\\)\\(\\s.\\|\\s-\\|\\s(\\|\\s)\\|\\s\"\\|\000\\|'\\|-\\|$\\)" 2 nil)
     ("\\(" "\\\\([^\000]*?\\\\)" 0 nil)
     ("\\[" "\\\\\\[[^\000]*?\\\\\\]" 0 nil)
     ("$$" "\\$\\$[^\000]*?\\$\\$" 0 nil))
diff --git a/testing/lisp/test-org-element.el b/testing/lisp/test-org-element.el
index 6d7376a96..2055e3a7b 100644
--- a/testing/lisp/test-org-element.el
+++ b/testing/lisp/test-org-element.el
@@ -1776,6 +1776,10 @@ e^{i\\pi}+1=0
    (eq 'latex-fragment
        (org-test-with-temp-text "$a$'"
 	 (org-element-type (org-element-context)))))
+  (should
+   (eq 'latex-fragment
+       (org-test-with-temp-text "$a$-"
+	 (org-element-type (org-element-context)))))
   ;; Test forbidden characters inside $...$.
   (should-not
    (eq 'latex-fragment
-- 
2.34.1

Re: [PATCH] Add support for $…$ latex fragments followed by a dash

[Eric S Fraga]

On Monday, 24 Jan 2022 at 16:42, Sébastien Miquel wrote:
> The attached patch adds support for $…$ latex fragments followed by a
> dash, such as $n$-th.

But is not necessary (and further complicates the issue of support $...$
in org as recently discussed on this list) as you can simply type
\(n\)-th?

-- 
: Eric S Fraga, with org release_9.5.2-312-g5d05f5 in Emacs 29.0.50

Re: [PATCH] Add support for $…$ latex fragments followed by a dash

[Sébastien Miquel]

Hi,

Eric S Fraga writes:
> But is not necessary (and further complicates the issue of support $...$
> in org as recently discussed on this list) as you can simply type
> \(n\)-th?

What complication are you referring to, precisely ?

The patch is fairly trivial, and a similar extension was already
implemented by Nicolas in 2017[1]. Yes, the $…$ syntax is redundant,
but I do not think it follows that this change is unnecessary.

This patch should have no bearing on the possible deprecation of the
$…$ syntax.

[1]: 
https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=c0369a798470763f8f3c69cf2079c3a194635feb

Regards,

-- 
Sébastien Miquel

Re: [PATCH] Add support for $…$ latex fragments followed by a dash

[Rudolf Adamkovič]

Eric S Fraga <e.fraga@ucl.ac.uk> writes:

> But is not necessary (and further complicates the issue of support
> $...$ in org as recently discussed on this list) as you can simply
> type \(n\)-th?

What you call "not necessary", I call a game changer.  As of now, one
has to mix and match $$ and \(\) based on the context (such as "-th").
It makes mathematical writing in both harder to /read/ and harder to
/change/.  Consider:

Let $r_i$ denote the \(i\)-th rotation of $t$ with a suffix of $\ell|t|$
characters deleted, for […]

Me, if I could, I would pay money for this feature, for it would allow
me to use $$ consistently, focusing on mathematics instead of markup
idiosyncrasies of "rotation $i$" versus "\(i\)-th rotation".

Further, \(\) brings 100% more characters than $$, resulting in more
noise in the sentence.

Rudy
-- 
"Thinking is a momentary dismissal of irrelevancies."
-- Richard Buckminster Fuller, 1969

Rudolf Adamkovič <salutis@me.com> [he/him]
Studenohorská 25
84103 Bratislava
Slovakia

Re: [PATCH] Add support for $…$ latex fragments followed by a dash

[Ihor Radchenko]

Rudolf Adamkovič <salutis@me.com> writes:

> Let $r_i$ denote the \(i\)-th rotation of $t$ with a suffix of $\ell|t|$
> characters deleted, for […]
>
> Me, if I could, I would pay money for this feature, for it would allow
> me to use $$ consistently, focusing on mathematics instead of markup
> idiosyncrasies of "rotation $i$" versus "\(i\)-th rotation".

Would it improve things for you if we change how \(...\) _looks_ in Org
buffers?

The problem with parsing is more than just supporting $i$-th and
similar. For example, AMS style guide explicitly advises against using
$i$-th in favour of $i$th [1]:

    Do not hyphenate “th” expressions: xth, not x-th or xth .

We can theoretically make a change to support "-", but then it will be
logical to support $i$th as well. (If we don't some users will still be
confused after trying to write $i$th and then not getting the expected
results). In this question, it would make sense to implement
all-or-everything approach. Otherwise, confusion (like raised in this
thread) will be inevitable.

However, from point of view of Org mode parser, supporting $i$th is a
nightmare.  Remember that Org mode is _not_ LaTeX and we have to support
a lot more frivolous syntax (even in LaTeX, runaway $ is often a source
of cryptic compilation errors). Currently, we _must_ rely on heuristics
to determine $$-style latex fragments. I do not know any way to support
$$ syntax without creating deviations from LaTeX. Extending the
heuristics will not resolve the underlying ambiguity of $$ syntax, just
hide it within even more obscure cases.

Given the raised concerns, may we solve the issue with too verbose
\(...\) unambiguous syntax using the following approach:
1. Fontify \(...\) replacing the brackets with a single character. For
   example:

      \(...\) -> ⁅...⁆

2. Provide convenient way to input \(\) brackets through
   electric-pair-mode or trough org-cdlatex-mode.

Best,
Ihor

[1] https://www.ams.org/publications/authors/AMS-StyleGuide-online.pdf

Re: [PATCH] Add support for $…$ latex fragments followed by a dash

[Tim Cross]

Ihor Radchenko <yantar92@gmail.com> writes:

> Rudolf Adamkovič <salutis@me.com> writes:
>
>> Let $r_i$ denote the \(i\)-th rotation of $t$ with a suffix of $\ell|t|$
>> characters deleted, for […]
>>
>> Me, if I could, I would pay money for this feature, for it would allow
>> me to use $$ consistently, focusing on mathematics instead of markup
>> idiosyncrasies of "rotation $i$" versus "\(i\)-th rotation".
>
> Would it improve things for you if we change how \(...\) _looks_ in Org
> buffers?
>
> The problem with parsing is more than just supporting $i$-th and
> similar. For example, AMS style guide explicitly advises against using
> $i$-th in favour of $i$th [1]:
>
>     Do not hyphenate “th” expressions: xth, not x-th or xth .
>
> We can theoretically make a change to support "-", but then it will be
> logical to support $i$th as well. (If we don't some users will still be
> confused after trying to write $i$th and then not getting the expected
> results). In this question, it would make sense to implement
> all-or-everything approach. Otherwise, confusion (like raised in this
> thread) will be inevitable.
>
> However, from point of view of Org mode parser, supporting $i$th is a
> nightmare.  Remember that Org mode is _not_ LaTeX and we have to support
> a lot more frivolous syntax (even in LaTeX, runaway $ is often a source
> of cryptic compilation errors). Currently, we _must_ rely on heuristics
> to determine $$-style latex fragments. I do not know any way to support
> $$ syntax without creating deviations from LaTeX. Extending the
> heuristics will not resolve the underlying ambiguity of $$ syntax, just
> hide it within even more obscure cases.
>
> Given the raised concerns, may we solve the issue with too verbose
> \(...\) unambiguous syntax using the following approach:
> 1. Fontify \(...\) replacing the brackets with a single character. For
>    example:
>
>       \(...\) -> ⁅...⁆
>
> 2. Provide convenient way to input \(\) brackets through
>    electric-pair-mode or trough org-cdlatex-mode.
>
> Best,
> Ihor
>
> [1] https://www.ams.org/publications/authors/AMS-StyleGuide-online.pdf

+1.

Just my $0.02 worth -

I think this is the right approach. Retaining support for $..$ doesn't
seem feasible given all the complexities it brings with it. The main
objections to the alternative appear to centre around readability and
inconvenience of having to type additional characters or dealing with
muscle memory use to $...$. These are essentially interface issues and I
think we can largely address them using existing Emacs facilities. This
will reduce the change impact to that sub-set of org users accustomed to
$...$ while bringing the benefit of a cleaner and potentially more
efficient parser to all org users.

If we do deprecate support for $...$, it might also be a good idea to
see if we can add a utility function which would make it easier for
people to migrate existing documents to the new/alternative syntax. For
the same reason it is hard to reliably parse $...$ syntax, we probably
can't automate that transition, but we should be able to reduce the effort
required to update existing documents. 

Re: [PATCH] Add support for $…$ latex fragments followed by a dash

[Max Nikulin]

On 28/01/2022 02:15, Tim Cross wrote:
> 
> If we do deprecate support for $...$, it might also be a good idea to
> see if we can add a utility function which would make it easier for
> people to migrate existing documents to the new/alternative syntax. For
> the same reason it is hard to reliably parse $...$ syntax, we probably
> can't automate that transition, but we should be able to reduce the effort
> required to update existing documents.

There is no point to discuss deprecation of $...$ and $$...$$ while 
usage of such constructs is not discouraged by the org manual info 
"(org) LaTeX fragmets" https://orgmode.org/manual/LaTeX-fragments.html 
and by the compact guide https://orgmode.org/guide/Embedded-LaTeX.html

Even though org syntax says that support of these constructs ought to be 
removed, users expect that they should work well.

Re: [PATCH] Add support for $…$ latex fragments followed by a dash

[Timothy]

[-- Attachment #1: Type: text/plain, Size: 1240 bytes --]

Hi Max,

> There is no point to discuss deprecation of $…$ and ... while usage of
> such constructs is not discouraged by the org manual

Isn’t the point of discussing depreciation now that we’d change the manual etc.
to discourage it? 😛

FWIW, I have an extra 2c to add: I think that there’s a danger in having $…$
/almost/ behave like LaTeX, as the closer it is the more surprising the edge cases
are, and due to the nature of Org there will /always/ be a collection of edge
cases with $…$. By comparison, \(…\) has much less ’surprising’ behaviour.

>> If we do deprecate support for $…$, it might also be a good idea to
>> see if we can add a utility function which would make it easier for
>> people to migrate existing documents to the new/alternative syntax. For
>> the same reason it is hard to reliably parse $…$ syntax, we probably
>> can’t automate that transition, but we should be able to reduce the effort
>> required to update existing documents.

Tim, as mentioned before I’m strongly in favour of a ~half decade transition
including utility functions to shift existing TeX constructs to LaTeX, and
adding warnings, well before dropping support.

All the best,
Timothy

Re: [PATCH] Add support for $…$ latex fragments followed by a dash

[Max Nikulin]

On 28/01/2022 23:37, Timothy wrote:
> 
>> There is no point to discuss deprecation of $…$ and ... while usage of
>> such constructs is not discouraged by the org manual
> 
> Isn’t the point of discussing depreciation now that we’d change the manual etc.
> to discourage it? 😛

It is funny that nobody have checked what the manual actually states:

> Text within the usual LaTeX math delimiters. To avoid conflicts with
> currency specifications, single ‘$’ characters are only recognized as
> math delimiters if the enclosed text contains at most two line breaks,
> is directly attached to the ‘$’ characters with no whitespace in
> between, and if the closing ‘$’ is followed by whitespace, punctuation
> or a dash. For the other delimiters, there is no such restriction, so
      ^^^^^^
> when in doubt, use ‘\(...\)’ as inline math delimiters.

I have no idea whether it ever worked in such way. So either the manual 
must be updated or a fix must be committed.

False positive with the proposed patch:

 > Balance decreased from $10 to negative value ($-2 approximately)

certainly it is more rare than $n$-th valid case.

>>> If we do deprecate support for $…$, it might also be a good idea to
>>> see if we can add a utility function which would make it easier for
>>> people to migrate existing documents to the new/alternative syntax. For
>>> the same reason it is hard to reliably parse $…$ syntax, we probably
>>> can’t automate that transition, but we should be able to reduce the effort
>>> required to update existing documents.
> 
> Tim, as mentioned before I’m strongly in favour of a ~half decade transition
> including utility functions to shift existing TeX constructs to LaTeX, and
> adding warnings, well before dropping support.

That period should be started when manual is updated.

Functions updating old file to the current syntax is an interesting 
idea, but it assumes a copy of org-element as it was in that old org 
version.

Re: [PATCH] Add support for $…$ latex fragments followed by a dash

[Max Nikulin]

manual states:
> 
>> Text within the usual LaTeX math delimiters. To avoid conflicts with
>> currency specifications, single ‘$’ characters are only recognized as
>> math delimiters if the enclosed text contains at most two line breaks,
>> is directly attached to the ‘$’ characters with no whitespace in
>> between, and if the closing ‘$’ is followed by whitespace, punctuation
>> or a dash. For the other delimiters, there is no such restriction, so
>       ^^^^^^
>> when in doubt, use ‘\(...\)’ as inline math delimiters.

It is even more interesting. Support of dash likely was unintentionally 
lost in the following commit:
https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=6779f8f424883ffd202e24cbd8bb4e241db294b0
that generalizes handling of punctuation, unfortunately dash and 
apostrophe are not always belong to punctuation symbols in *text* modes. 
That commit even updates manual to a less precise phrase, however .texi 
file only, so the change was lost. Nicolas later restored apostrophe in 
the commit c0369a798470763f8f3c69cf2079c3a194635feb

> False positive with the proposed patch:
> 
>  > Balance decreased from $10 to negative value ($-2 approximately)
> 
> certainly it is more rare than $n$-th valid case.
> 
>> Tim, as mentioned before I’m strongly in favour of a ~half decade 
>> transition

Half of decade already passed since dash after currency symbol was 
broken so maybe it is better to fix current state by updating the manual 
(including bugfix branch) and by adding some tests.

P.S. It is deja vu, I almost certainly saw quite recent discussions 
whether punctuation may be handled in some regexps in more generic way. 
Consequences may be similar in respect to characters that are almost 
punctuation...

Re: [PATCH] Add support for $…$ latex fragments followed by a dash

[Sébastien Miquel]

Hi,

Ihor Radchenko writes:
> We can theoretically make a change to support "-", but then it will be
> logical to support $i$th as well. (If we don't some users will still be
> confused after trying to write $i$th and then not getting the expected
> results).

I disagree.
  1. The $…$- pattern is also used for other common constructions, such
     as $n$-dimensional, $K$-lipschitz, etc. As for $n$−th vs $n$th, both
     are commonly used. In french, $n$− is the correct one.
  2. It does not logically follow that we should support $i$th as well,
     since, as you point out, it'd be impossible. One argument for the
     patch is that is it very simple.
  3. The $n$th construction failing is not as confusing. One
     understands quickly what the limitation is, and several
     workarounds are available, whereas there's no good reason for the
     $n$− limitation, and it's harder to think of a workaround.

I should mention that the zero-width space character can be used to
work around both limitations.

> Given the raised concerns, may we solve the issue with too verbose
> \(...\) unambiguous syntax using the following approach:
> 1. Fontify \(...\) replacing the brackets with a single character. For
>     example:
>
>        \(...\) -> ⁅...⁆
>
> 2. Provide convenient way to input \(\) brackets through
>     electric-pair-mode or trough org-cdlatex-mode.
If the $…$ syntax were to be deprecated, this would be a nice
addition, indeed. As a user, I find it quite satisfactory (with some
added utility to easily switch between the \(…\) and \[…\] syntax).
Some possible drawbacks :
  - are the display hacks scalable in a large document, with many LaTeX
    snippets ?
  - this feature may still be hard to discover, turn on and use by
    users more concerned with mathematical content and less familiar
    with emacs features such as fontification, automatic insertion of
    complex delimiters and whatnot.
  - hiding the syntax feels a bit weird, although it is already
    done with emphasis markers.

With respect to the possible deprecation of the $…$ syntax, the
drawbacks and complexity of this alternative should be weighted
against those of the current $…$ syntax, but no one has really spelled
out what the latter are. We're trading complexity here for complexity
there, fixing the false positives (but how common are they ?) and the
false negatives (which can be an annoyance while writing a snippet).

Thank you for bringing this up,

Regards,

-- 
Sébastien Miquel

Re: [PATCH] Add support for $…$ latex fragments followed by a dash

[Rudolf Adamkovič]

Ihor Radchenko <yantar92@gmail.com> writes:

> We can theoretically make a change to support "-", but then it will be
> logical to support $i$th as well. […]

I do not know about others, but supporting the dash alone would mean a
world to me.  In fact, I never use \(\) for anything else in all my Org
documents.  That dash, though, it constantly gets into my way.

> 1. Fontify \(...\) replacing the brackets with a single character. For
>    example:
>
>       \(...\) -> ⁅...⁆

Me, I cannot use any of these "pretty" features because, simply put,
they break plain text.  For example, they cause misaligned tables and
make the text overflow the fill column, which I keep at 72 columns.

> 2. Provide convenient way to input \(\) brackets through
>    electric-pair-mode or trough org-cdlatex-mode.

This would help a bit.

Rudy
-- 
"'Contrariwise,' continued Tweedledee, 'if it was so, it might be; and
if it were so, it would be; but as it isn't, it ain't. That's logic.'"
-- Lewis Carroll, Through the Looking Glass, 1871/1872

Rudolf Adamkovič <salutis@me.com> [he/him]
Studenohorská 25
84103 Bratislava
Slovakia

Re: [PATCH] Add support for $…$ latex fragments followed by a dash

[João Pedro de Amorim Paula]

On 01 February 2022 22:00, Rudolf Adamkovič <salutis@me.com> wrote:

> Me, I cannot use any of these "pretty" features because, simply put,
> they break plain text.  For example, they cause misaligned tables and
> make the text overflow the fill column, which I keep at 72 columns.

I know that this is a bit divergent from the original subject -- and
also keep in mind that this shouldn't be considered a solution to the
problem, given that it should be considered for all users with different
fonts --, but there are fonts that enforce Unicode (all characters, from
what I understand, Unicode included) characters to be exactly 1 unit
width, i.e. Unicode characters are the same widths as other characters.
At least I can guarantee that any of the Term (all characters are the
same length, but has ligatures) or Fixed (same as Term but no ligatures)
for Iosekva [1] have this given property.

[1] https://typeof.net/Iosevka/

Best regards,

-- 
João Pedro de Amorim Paula
IT undergraduate at Universidade Federal do Rio Grande do Norte (UFRN)

Re: [PATCH] Add support for $…$ latex fragments followed by a dash

[Rudolf Adamkovič]

João Pedro de Amorim Paula <jpedrodeamorim@gmail.com> writes:

> On 01 February 2022 22:00, Rudolf Adamkovič <salutis@me.com> wrote:
>
>> Me, I cannot use any of these "pretty" features because, simply put,
>> they break plain text.  For example, they cause misaligned tables and
>> make the text overflow the fill column, which I keep at 72 columns.
>
> […] there are fonts that enforce Unicode (all characters, from what I
> understand, Unicode included) characters to be exactly 1 unit width,
> i.e. Unicode characters are the same widths as other characters.  At
> least I can guarantee that any of the Term (all characters are the
> same length, but has ligatures) or Fixed (same as Term but no
> ligatures) for Iosekva [1] have this given property.
>
> [1] https://typeof.net/Iosevka/

Thank you for sharing!  I use the amazing "Hack" typeface [1].  Then, I
fail to understand how the font changes the fact that Org breaks "the
promise of plain text" in this regard.

For example, Org can hide its emphasis markers.  Later, one opens the
file in another editor and sees the truth.  The lines go over the fill
column, the tables have misaligned columns, and so on.

Rudy

[1] https://sourcefoundry.org/hack/
-- 
"Strange as it may sound, the power of mathematics rests on its evasion
of all unnecessary thought and on its wonderful saving of mental
operations."
-- Ernst Mach, 1838-1916

Rudolf Adamkovič <salutis@me.com> [he/him]
Studenohorská 25
84103 Bratislava
Slovakia

Re: [PATCH] Add support for $…$ latex fragments followed by a dash

[Tim Cross]

Rudolf Adamkovič <salutis@me.com> writes:

> João Pedro de Amorim Paula <jpedrodeamorim@gmail.com> writes:
>
>> On 01 February 2022 22:00, Rudolf Adamkovič <salutis@me.com> wrote:
>>
>>> Me, I cannot use any of these "pretty" features because, simply put,
>>> they break plain text.  For example, they cause misaligned tables and
>>> make the text overflow the fill column, which I keep at 72 columns.
>>
>> […] there are fonts that enforce Unicode (all characters, from what I
>> understand, Unicode included) characters to be exactly 1 unit width,
>> i.e. Unicode characters are the same widths as other characters.  At
>> least I can guarantee that any of the Term (all characters are the
>> same length, but has ligatures) or Fixed (same as Term but no
>> ligatures) for Iosekva [1] have this given property.
>>
>> [1] https://typeof.net/Iosevka/
>
> Thank you for sharing!  I use the amazing "Hack" typeface [1].  Then, I
> fail to understand how the font changes the fact that Org breaks "the
> promise of plain text" in this regard.
>
> For example, Org can hide its emphasis markers.  Later, one opens the
> file in another editor and sees the truth.  The lines go over the fill
> column, the tables have misaligned columns, and so on.
>

The promise of plain text has no inherent promise regarding alignment,
line wrapping etc. The promise of plain text is simply that - a promise
that org files will be just plain text rather than some application
specific format like many other systems (MS Word, LibreOffice, etc).
Provided you can still edit the file with a plain text editor, org has
not broken its promise.

Ironically, it is this plain text basis of org files which makes
on-going support of $..$ (and any extension) so problematic. Without
this restriction, org files could have a format where elements were
'tagged' to remove ambiguity and simplify the parsing process. However,
this would of course mean that either you had to edit the file within
org mode or you would have to be intimately familiar with the internal
structure of org mode files and use an editor which made
editing/updating the internal structure possible. To be very clear, I am
NOT advocating that org should adopt some form of internal tagging or
structure. I'm only attempting to highlight that having a system which
aims to maintain the data source in plain text comes at a cost.

In LaTex et. al. $..$ works well because $ is a special character. If
you want a literal $, you have to escape it. In org, $ is not a special
character. This means we need complex regular expressions in order to
identify when $ is being used for LaTex specific markup and when it is
being used in other contexts (e.g. literal $ sign, variable name). These
regular expressions are difficult to get right, error prone and above
all, incur a significant performance hit. Extending the syntax further
to support $..$- simply makes the situation worse by adding further
complexity.

So far, all the arguments against removal of support for $..$ are based
on inconvenience and reduced readability associated with the
alternatives. Both legitimate concerns. For many, the proposed change
may seem to be just inconvenient change for no real purpose because they
don't deal with the complexities inherent in making $..$ work and for
others, change is something they would always prefer to avoid.

In this instance, I feel we really need to listen to those who put in
such a dedicated effort in maintaining org mode and accept their
position that sustaining $..$ syntax going forward is problematic and
focus on what can be done to mitigate the impact from this change, make
the alternative syntax more convenient and address the readability
issues.

Re: [PATCH] Add support for $…$ latex fragments followed by a dash

[Tom Gillespie]

> The attached patch adds support for $…$ latex fragments followed by a
> dash, such as $n$-th.

Unfortunately this falls into the realm of changes to syntax. The current
behavior is not a bug and is working as specified because hyphen minus
(U+002D) does not count as punctuation for the purposes of org syntax.
We should specify which chars count as punctuation in the syntax doc.
As noted by Eric \(\) has no such restrictions.

From https://orgmode.org/worg/dev/org-syntax.html#Entities_and_LaTeX_Fragments
> POST is any punctuation (including parentheses and quotes) or space character, or the end of line.

Best,
Tom

Re: [PATCH] Add support for $…$ latex fragments followed by a dash

[Sébastien Miquel]

[-- Attachment #1: Type: text/plain, Size: 693 bytes --]

Hi,

Tom Gillespie writes:
> Unfortunately this falls into the realm of changes to syntax. The current
> behavior is not a bug and is working as specified because hyphen minus
> (U+002D) does not count as punctuation for the purposes of org syntax.
> We should specify which chars count as punctuation in the syntax doc.
> As noted by Eric \(\) has no such restrictions.
>
> Fromhttps://orgmode.org/worg/dev/org-syntax.html#Entities_and_LaTeX_Fragments
>> POST is any punctuation (including parentheses and quotes) or space character, or the end of line.

With this patch, I also propose to change the specification
accordingly. The change is local and minor.

Regards,

-- 
Sébastien Miquel

[-- Attachment #2: Type: text/html, Size: 1536 bytes --]

Re: [PATCH] Add support for $…$ latex fragments followed by a dash

[Tom Gillespie]

> The change is local and minor.
We can't know that. Consider for example someone that has
the following line somewhere in their files.
#+begin_src org
I spent $20 on food and was paid$-10 dollars by friends so
I am down $10.
#+end_src
Yes =paid$-10= is probably a typo that should have a space
in between, but it could still be in a file and cause an issue.
The more likely case would be of someone that has $ in the
name of a variable that also uses dashes. For example if
I have a list of variable names such as
#+begin_src org
Text a $A_BASH_VAR
Text b some-$-lisp-var
#+end_src

The proposed change would break any file with a pattern like
this.

We have no way of seeing every org file that users have
written so we don't know the extent of the impact, and thus
have to assume that there would be some impact. Making
such a change with an unknown blast radius in the midst of
considering removing support for that syntax altogether is
inviting disaster.

Best,
Tom

Re: [PATCH] Add support for $…$ latex fragments followed by a dash

[Sébastien Miquel]

Hi,

Tom Gillespie writes:
>> The change is local and minor.
> We can't know that.
I meant that the change to the syntax would be minor and local, with
respect to the linked syntax document.

> #+begin_src org
> I spent $20 on food and was paid$-10 dollars by friends so
> I am down $10.
> #+end_src
> [...]
> #+begin_src org
> Text a $A_BASH_VAR
> Text b some-$-lisp-var
> #+end_src
>
> The proposed change would break any file with a pattern like
> this.
As you say, your first example is a typo. As for the second example,
it's a very uncommon pattern, and if such variable names occur in a
document, they should either be inside src blocks, or inside verbatim
or code markers. In both case, there's no breakage with respect to
parsing, export and tangling. The fontification may fail though. It
wont fail if your src blocks are fontified natively, and the
fontification of $…$ can be globally disabled.

> We have no way of seeing every org file that users have
> written so we don't know the extent of the impact, and thus
> have to assume that there would be some impact. Making
> such a change with an unknown blast radius in the midst of
> considering removing support for that syntax altogether is
> inviting disaster.
We can make an educated guess. It is quite likely that this change
will fix more documents than it breaks. Hardly a disaster.

Regards,

-- 
Sébastien Miquel