emacs-orgmode@gnu.org archives
 help / color / mirror / code / Atom feed
* c47b535bb origin/main org-element: Remove dependency on ‘org-emphasis-regexp-components’
@ 2021-11-15  0:53 Ihor Radchenko
  2021-11-15  9:56 ` Nicolas Goaziou
  0 siblings, 1 reply; 27+ messages in thread
From: Ihor Radchenko @ 2021-11-15  0:53 UTC (permalink / raw)
  To: emacs-orgmode, Nicolas Goaziou

This commit may cause random failures when
org-emphasis-regexp-components is changed by user. org-emph-re is
calculated according to org-emphasis-regexp-components. Changing
org-emphasis-regexp-components can make "(when (looking-at org-emph-re)"
in parsers return nil. The emphasised text will still be fontified, but
not available in the parsed buffer.

Maybe we need to move the logic for org-emph-re from org.el to
org-element.el?

Also, there is org-emphasis-alist. It is even defcustom, but ignored by
org-element.el.

Best,
Ihor


^ permalink raw reply	[flat|nested] 27+ messages in thread

* Re: c47b535bb origin/main org-element: Remove dependency on ‘org-emphasis-regexp-components’
  2021-11-15  0:53 c47b535bb origin/main org-element: Remove dependency on ‘org-emphasis-regexp-components’ Ihor Radchenko
@ 2021-11-15  9:56 ` Nicolas Goaziou
  2021-11-15 15:20   ` Ihor Radchenko
  0 siblings, 1 reply; 27+ messages in thread
From: Nicolas Goaziou @ 2021-11-15  9:56 UTC (permalink / raw)
  To: Ihor Radchenko; +Cc: emacs-orgmode

Hello,

Ihor Radchenko <yantar92@gmail.com> writes:

> This commit may cause random failures when
> org-emphasis-regexp-components is changed by user.

This is not supported anyway.

> org-emph-re is calculated according to org-emphasis-regexp-components.
> Changing org-emphasis-regexp-components can make "(when (looking-at
> org-emph-re)" in parsers return nil. The emphasised text will still be
> fontified, but not available in the parsed buffer.

That’s exactly my point. The syntax is not meant to be configurable.
I wrote a patch also removing ‘org-emph-re’ depedency from
"org-element.el", but I was delayed. I just applied it.

> Maybe we need to move the logic for org-emph-re from org.el to
> org-element.el?

‘org-emph-re’ has some limitations which do not belong to syntax
definition. There’s no point in adding it in "org-element.el".

The grand scheme is to remove most "org.el" dependencies from
"org-element.el", and move the others.

> Also, there is org-emphasis-alist. It is even defcustom, but ignored by
> org-element.el.

This variable is a defcustom for the faces, not the markers. I.e., it is
not meant to add, remove, or change emphasis markup, but rather alter
how they appear. IMO, this should be removed altogether: it’s up to
a theme to set such a thing.

Regards,
-- 
Nicolas Goaziou


^ permalink raw reply	[flat|nested] 27+ messages in thread

* Re: c47b535bb origin/main org-element: Remove dependency on ‘org-emphasis-regexp-components’
  2021-11-15  9:56 ` Nicolas Goaziou
@ 2021-11-15 15:20   ` Ihor Radchenko
  2021-11-15 16:25     ` Max Nikulin
  0 siblings, 1 reply; 27+ messages in thread
From: Ihor Radchenko @ 2021-11-15 15:20 UTC (permalink / raw)
  To: Nicolas Goaziou; +Cc: emacs-orgmode

Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

> Ihor Radchenko <yantar92@gmail.com> writes:
>
>> This commit may cause random failures when
>> org-emphasis-regexp-components is changed by user.
>
> This is not supported anyway.

Yeah. Though I have seen people changing this variable.
Maybe we should change defvar to defconst?

>> org-emph-re is calculated according to org-emphasis-regexp-components.
>> Changing org-emphasis-regexp-components can make "(when (looking-at
>> org-emph-re)" in parsers return nil. The emphasised text will still be
>> fontified, but not available in the parsed buffer.
>
> That’s exactly my point. The syntax is not meant to be configurable.
> I wrote a patch also removing ‘org-emph-re’ depedency from
> "org-element.el", but I was delayed. I just applied it.

Thanks!

>> Maybe we need to move the logic for org-emph-re from org.el to
>> org-element.el?
>
> ‘org-emph-re’ has some limitations which do not belong to syntax
> definition. There’s no point in adding it in "org-element.el".
>
> The grand scheme is to remove most "org.el" dependencies from
> "org-element.el", and move the others.

That would be great. I was thinking about unifying the grammar better.
Things like org-set-regexps-and-options define part of the grammar
non-transparently outside org-element. As a result, the new org-persist
library can be potentially broken if the user changes grammar between
Emacs session (e.g. org-off-levels-only, org-todo-keywords, etc). Keeping
all the variables that change the grammar in one place would be helpful.

Maybe we could save the complete grammar state right inside output of
org-element-parse-buffer. In the updated element cache code, I
introduced properties into org-data element. Maybe we can keep all the
important variables for the buffer grammar in org-data? That way, the
job of org-set-regexps-and-options will be done by
org-element-org-data-parser. Moreover, users exporting using ox-org will
be able to create self-sufficient Org files that can be shared without
relying on the same configuration in init.el.

>> Also, there is org-emphasis-alist. It is even defcustom, but ignored by
>> org-element.el.
>
> This variable is a defcustom for the faces, not the markers. I.e., it is
> not meant to add, remove, or change emphasis markup, but rather alter
> how they appear. IMO, this should be removed altogether: it’s up to
> a theme to set such a thing.

Unless I miss something, org-emphasis-alist is used in org-emphasise.
Though it just another reason to remove it.

Best,
Ihor


^ permalink raw reply	[flat|nested] 27+ messages in thread

* Re: c47b535bb origin/main org-element: Remove dependency on ‘org-emphasis-regexp-components’
  2021-11-15 15:20   ` Ihor Radchenko
@ 2021-11-15 16:25     ` Max Nikulin
  2021-11-16  7:43       ` Ihor Radchenko
  0 siblings, 1 reply; 27+ messages in thread
From: Max Nikulin @ 2021-11-15 16:25 UTC (permalink / raw)
  To: emacs-orgmode

On 15/11/2021 22:20, Ihor Radchenko wrote:
> Nicolas Goaziou writes:
>> Ihor Radchenko writes:
>>
>>> This commit may cause random failures when
>>> org-emphasis-regexp-components is changed by user.
>>
>> This is not supported anyway.
> 
> Yeah. Though I have seen people changing this variable.
> Maybe we should change defvar to defconst?

Better docs and some restriction on defcustom values were discussed earlier:
https://list.orgmode.org/87k0oyd3pw.fsf@nicolasgoaziou.fr/
Nicolas Goaziou. Re: Using backticks for the inline code delimeter? Mon, 
19 Apr 2021 11:27:07 +0200

Sorry, I have not prepared a patch. I am not confident with defcustom 
fine tuning and have not experimented with it since that time.



^ permalink raw reply	[flat|nested] 27+ messages in thread

* Re: c47b535bb origin/main org-element: Remove dependency on ‘org-emphasis-regexp-components’
  2021-11-15 16:25     ` Max Nikulin
@ 2021-11-16  7:43       ` Ihor Radchenko
  2021-11-16 21:56         ` Samuel Wales
                           ` (2 more replies)
  0 siblings, 3 replies; 27+ messages in thread
From: Ihor Radchenko @ 2021-11-16  7:43 UTC (permalink / raw)
  To: Max Nikulin; +Cc: emacs-orgmode

[-- Attachment #1: Type: text/plain, Size: 477 bytes --]

Max Nikulin <manikulin@gmail.com> writes:

> Better docs and some restriction on defcustom values were discussed earlier:
> https://list.orgmode.org/87k0oyd3pw.fsf@nicolasgoaziou.fr/
> Nicolas Goaziou. Re: Using backticks for the inline code delimeter? Mon, 
> 19 Apr 2021 11:27:07 +0200
>
> Sorry, I have not prepared a patch. I am not confident with defcustom 
> fine tuning and have not experimented with it since that time.

Maybe something like the attached?

Best,
Ihor


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-org-emphasis-alist-Update-defcustom-making-emphasis-.patch --]
[-- Type: text/x-diff, Size: 2397 bytes --]

From 8057cdb57f6600443b3605c1e7f00a30bea2a9ea Mon Sep 17 00:00:00 2001
Message-Id: <8057cdb57f6600443b3605c1e7f00a30bea2a9ea.1637048505.git.yantar92@gmail.com>
From: Ihor Radchenko <yantar92@gmail.com>
Date: Tue, 16 Nov 2021 15:40:35 +0800
Subject: [PATCH] org-emphasis-alist: Update defcustom making emphasis
 characters constant

* lisp/org.el (org-emphasis-alist): Mention that emphasis characters
should not be changed by user.  Update the defcustom type accordingly.
---
 lisp/org.el | 39 ++++++++++++++++++++++++++++++++++-----
 1 file changed, 34 insertions(+), 5 deletions(-)

diff --git a/lisp/org.el b/lisp/org.el
index 603b57279..7af5e26c6 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -3833,18 +3833,47 @@ (defcustom org-emphasis-alist
 marker characters and the face to be used by font-lock for highlighting
 in Org buffers.
 
+The characters in the alist must not be changed.  They do not affect
+the actual Org syntax, just fontification.
+
 You need to reload Org or to restart Emacs after customizing this."
   :group 'org-appearance
   :set 'org-set-emph-re
   :version "24.4"
   :package-version '(Org . "8.0")
-  :type '(repeat
+  :type '(list
 	  (list
-	   (string :tag "Marker character")
+	   (const "*")
+	   (choice
+	    (face :tag "Bold emphasis face")
+	    (plist :tag "Bold emphasis face property list")))
+          (list
+	   (const "/")
+	   (choice
+	    (face :tag "Italic emphasis face")
+	    (plist :tag "Italic emphasis face property list")))
+          (list
+	   (const "_")
+	   (choice
+	    (face :tag "Underline emphasis face")
+	    (plist :tag "Underline emphasis face property list")))
+          (list
+	   (const "=")
+	   (choice
+	    (face :tag "Verbatim emphasis face")
+	    (plist :tag "Verbatim emphasis face property list"))
+	   (const verbatim))
+          (list
+	   (const "~")
+	   (choice
+	    (face :tag "Code emphasis face")
+	    (plist :tag "Code emphasis face property list"))
+	   (const verbatim))
+          (list
+	   (const "+")
 	   (choice
-	    (face :tag "Font-lock-face")
-	    (plist :tag "Face property list"))
-	   (option (const verbatim)))))
+	    (face :tag "Strike-through emphasis face")
+	    (plist :tag "Strike-through emphasis face property list")))))
 
 (defvar org-protecting-blocks '("src" "example" "export")
   "Blocks that contain text that is quoted, i.e. not processed as Org syntax.
-- 
2.32.0


^ permalink raw reply	[flat|nested] 27+ messages in thread

* Re: c47b535bb origin/main org-element: Remove dependency on ‘org-emphasis-regexp-components’
  2021-11-16  7:43       ` Ihor Radchenko
@ 2021-11-16 21:56         ` Samuel Wales
  2021-11-16 22:16           ` Samuel Wales
  2021-11-17 16:44         ` Max Nikulin
  2021-11-20 12:02         ` Max Nikulin
  2 siblings, 1 reply; 27+ messages in thread
From: Samuel Wales @ 2021-11-16 21:56 UTC (permalink / raw)
  To: Ihor Radchenko; +Cc: Max Nikulin, emacs-orgmode

hmm i will have to try not setting o-e-r-c [ordinary user].  probably
a lot of users do.

might be useful to know whether a default regexp change could satisfy
everybody?  in my case i remove " and , from third re.  i also have a
note "org-set-emph-re is what you are supposed to use, but it is
undocumented".  for some reason.  that might be obsolete,


On 11/16/21, Ihor Radchenko <yantar92@gmail.com> wrote:
> Max Nikulin <manikulin@gmail.com> writes:
>
>> Better docs and some restriction on defcustom values were discussed
>> earlier:
>> https://list.orgmode.org/87k0oyd3pw.fsf@nicolasgoaziou.fr/
>> Nicolas Goaziou. Re: Using backticks for the inline code delimeter? Mon,
>> 19 Apr 2021 11:27:07 +0200
>>
>> Sorry, I have not prepared a patch. I am not confident with defcustom
>> fine tuning and have not experimented with it since that time.
>
> Maybe something like the attached?
>
> Best,
> Ihor
>
>


-- 
The Kafka Pandemic

Please learn what misopathy is.
https://thekafkapandemic.blogspot.com/2013/10/why-some-diseases-are-wronged.html


^ permalink raw reply	[flat|nested] 27+ messages in thread

* Re: c47b535bb origin/main org-element: Remove dependency on ‘org-emphasis-regexp-components’
  2021-11-16 21:56         ` Samuel Wales
@ 2021-11-16 22:16           ` Samuel Wales
  0 siblings, 0 replies; 27+ messages in thread
From: Samuel Wales @ 2021-11-16 22:16 UTC (permalink / raw)
  To: Ihor Radchenko; +Cc: Max Nikulin, emacs-orgmode

i should point out that my changes are old and i don't know if they
have already been done

On 11/16/21, Samuel Wales <samologist@gmail.com> wrote:
> hmm i will have to try not setting o-e-r-c [ordinary user].  probably
> a lot of users do.
>
> might be useful to know whether a default regexp change could satisfy
> everybody?  in my case i remove " and , from third re.  i also have a
> note "org-set-emph-re is what you are supposed to use, but it is
> undocumented".  for some reason.  that might be obsolete,
>
>
> On 11/16/21, Ihor Radchenko <yantar92@gmail.com> wrote:
>> Max Nikulin <manikulin@gmail.com> writes:
>>
>>> Better docs and some restriction on defcustom values were discussed
>>> earlier:
>>> https://list.orgmode.org/87k0oyd3pw.fsf@nicolasgoaziou.fr/
>>> Nicolas Goaziou. Re: Using backticks for the inline code delimeter? Mon,
>>> 19 Apr 2021 11:27:07 +0200
>>>
>>> Sorry, I have not prepared a patch. I am not confident with defcustom
>>> fine tuning and have not experimented with it since that time.
>>
>> Maybe something like the attached?
>>
>> Best,
>> Ihor
>>
>>
>
>
> --
> The Kafka Pandemic
>
> Please learn what misopathy is.
> https://thekafkapandemic.blogspot.com/2013/10/why-some-diseases-are-wronged.html
>


-- 
The Kafka Pandemic

Please learn what misopathy is.
https://thekafkapandemic.blogspot.com/2013/10/why-some-diseases-are-wronged.html


^ permalink raw reply	[flat|nested] 27+ messages in thread

* Re: c47b535bb origin/main org-element: Remove dependency on ‘org-emphasis-regexp-components’
  2021-11-16  7:43       ` Ihor Radchenko
  2021-11-16 21:56         ` Samuel Wales
@ 2021-11-17 16:44         ` Max Nikulin
  2021-11-17 22:44           ` Samuel Wales
  2021-11-18 12:25           ` Ihor Radchenko
  2021-11-20 12:02         ` Max Nikulin
  2 siblings, 2 replies; 27+ messages in thread
From: Max Nikulin @ 2021-11-17 16:44 UTC (permalink / raw)
  To: emacs-orgmode

On 17/11/2021 04:56, Samuel Wales wrote:
> might be useful to know whether a default regexp change could satisfy
> everybody?  in my case i remove " and , from third re.

Samuel, I have seen your next message, but it still may be helpful to 
have an example (for a case if you will meet the problem again).

These regexps will always fail under some conditions, since it is not 
strict markup. An example is emphasis terminated inside link target

/A link [[https://orgmode.org/?oops=true][Org Mode]]

On 16/11/2021 14:43, Ihor Radchenko wrote:
> Max Nikulin writes:
> 
>> Better docs and some restriction on defcustom values were discussed earlier:
>> https://list.orgmode.org/87k0oyd3pw.fsf@nicolasgoaziou.fr/
>> Nicolas Goaziou. Re: Using backticks for the inline code delimeter? Mon,
>> 19 Apr 2021 11:27:07 +0200
>>
>> Sorry, I have not prepared a patch. I am not confident with defcustom
>> fine tuning and have not experimented with it since that time.
> 
> Maybe something like the attached?

Thank you, Ihor. Another reason why I have not tried to do it myself is 
that it is necessary to test behavior for users who customized markers. 
The change should not be fatal for them. I have not checked it with you 
patch yet.

I was considering some way to warn users if improper customization is 
detected (unexpected marker is noticed). It should be noticeable to make 
user aware of export issues but not too annoying.

> +The characters in the alist must not be changed.  They do not affect
> +the actual Org syntax, just fontification.

Since this is known point of abuses, maybe stronger words are appropriate.

Do not change makers and do not add new ones to use custom markers for 
existing styles or to introduce new styles. Org syntax is not meant to 
be configurable and such modifications will not work with export. If you 
are interested in fontification of custom markup inside Emacs only, 
there are other ways to achieve desired effect.

In addition, I do not like the following phrase in the manual:

> To narrow down the list of available markup syntax, you can customize org-emphasis-alist.



^ permalink raw reply	[flat|nested] 27+ messages in thread

* Re: c47b535bb origin/main org-element: Remove dependency on ‘org-emphasis-regexp-components’
  2021-11-17 16:44         ` Max Nikulin
@ 2021-11-17 22:44           ` Samuel Wales
  2021-11-18 12:25           ` Ihor Radchenko
  1 sibling, 0 replies; 27+ messages in thread
From: Samuel Wales @ 2021-11-17 22:44 UTC (permalink / raw)
  To: Max Nikulin; +Cc: emacs-orgmode

i think i found it useful, long ago, when it was ok/tolerated to
change the var, and probably still, to /"do this"/ and /this,/.

although the latter seems weird to me now so i must not now understand
what i was doing.

i think changing the var was at least in a faq or on worg or so, so it
might be needed to compare user's value with standard-value.


On 11/17/21, Max Nikulin <manikulin@gmail.com> wrote:
> On 17/11/2021 04:56, Samuel Wales wrote:
>> might be useful to know whether a default regexp change could satisfy
>> everybody?  in my case i remove " and , from third re.
>
> Samuel, I have seen your next message, but it still may be helpful to
> have an example (for a case if you will meet the problem again).
>
> These regexps will always fail under some conditions, since it is not
> strict markup. An example is emphasis terminated inside link target
>
> /A link [[https://orgmode.org/?oops=true][Org Mode]]
>
> On 16/11/2021 14:43, Ihor Radchenko wrote:
>> Max Nikulin writes:
>>
>>> Better docs and some restriction on defcustom values were discussed
>>> earlier:
>>> https://list.orgmode.org/87k0oyd3pw.fsf@nicolasgoaziou.fr/
>>> Nicolas Goaziou. Re: Using backticks for the inline code delimeter? Mon,
>>> 19 Apr 2021 11:27:07 +0200
>>>
>>> Sorry, I have not prepared a patch. I am not confident with defcustom
>>> fine tuning and have not experimented with it since that time.
>>
>> Maybe something like the attached?
>
> Thank you, Ihor. Another reason why I have not tried to do it myself is
> that it is necessary to test behavior for users who customized markers.
> The change should not be fatal for them. I have not checked it with you
> patch yet.
>
> I was considering some way to warn users if improper customization is
> detected (unexpected marker is noticed). It should be noticeable to make
> user aware of export issues but not too annoying.
>
>> +The characters in the alist must not be changed.  They do not affect
>> +the actual Org syntax, just fontification.
>
> Since this is known point of abuses, maybe stronger words are appropriate.
>
> Do not change makers and do not add new ones to use custom markers for
> existing styles or to introduce new styles. Org syntax is not meant to
> be configurable and such modifications will not work with export. If you
> are interested in fontification of custom markup inside Emacs only,
> there are other ways to achieve desired effect.
>
> In addition, I do not like the following phrase in the manual:
>
>> To narrow down the list of available markup syntax, you can customize
>> org-emphasis-alist.
>
>
>


-- 
The Kafka Pandemic

Please learn what misopathy is.
https://thekafkapandemic.blogspot.com/2013/10/why-some-diseases-are-wronged.html


^ permalink raw reply	[flat|nested] 27+ messages in thread

* Re: c47b535bb origin/main org-element: Remove dependency on ‘org-emphasis-regexp-components’
  2021-11-17 16:44         ` Max Nikulin
  2021-11-17 22:44           ` Samuel Wales
@ 2021-11-18 12:25           ` Ihor Radchenko
  2021-11-18 12:35             ` Nicolas Goaziou
  2021-11-19 16:34             ` c47b535bb origin/main org-element: Remove dependency on ‘org-emphasis-regexp-components’ Max Nikulin
  1 sibling, 2 replies; 27+ messages in thread
From: Ihor Radchenko @ 2021-11-18 12:25 UTC (permalink / raw)
  To: Max Nikulin, Nicolas Goaziou; +Cc: emacs-orgmode

[-- Attachment #1: Type: text/plain, Size: 2569 bytes --]

Max Nikulin <manikulin@gmail.com> writes:

> These regexps will always fail under some conditions, since it is not 
> strict markup. An example is emphasis terminated inside link target
>
> /A link [[https://orgmode.org/?oops=true][Org Mode]]

Your example actually reveals a much bigger issue with current element
parser. Even though the link is fontified by Org, if you run
org-element-context on the link, it will return nil. The parsed sentence
will look like:

<begin italic>/A link [[https://orgmode.org/<end italic><begin ordinary
text - it is not a link!>?oops=true][Org Mode]]<end of ordinary text>

The reason behind is partially regexps used to detect emphasised text
and partially the way our parser works - no intersected objects are
allowed.

My intuition says that the current parser behaviour is not correct. It
would make more sense to prioritise link over italics. However, it would
require a major change in the parser - instead of a single pass, the
parser may parse different types of objects sequentially. The emphasis
objects should come last avoiding the markers to have different parents.

Nicolas, WDYT?

>> Maybe something like the attached?
>
> Thank you, Ihor. Another reason why I have not tried to do it myself is 
> that it is necessary to test behavior for users who customized markers. 
> The change should not be fatal for them. I have not checked it with you 
> patch yet.
>
> I was considering some way to warn users if improper customization is 
> detected (unexpected marker is noticed). It should be noticeable to make 
> user aware of export issues but not too annoying.
>
>> +The characters in the alist must not be changed.  They do not affect
>> +the actual Org syntax, just fontification.
>
> Since this is known point of abuses, maybe stronger words are appropriate.
>
> Do not change makers and do not add new ones to use custom markers for 
> existing styles or to introduce new styles. Org syntax is not meant to 
> be configurable and such modifications will not work with export. If you 
> are interested in fontification of custom markup inside Emacs only, 
> there are other ways to achieve desired effect.
> ...
> In addition, I do not like the following phrase in the manual:
>
>> To narrow down the list of available markup syntax, you can customize org-emphasis-alist.

I updated the patch dropping your last suggested sentence in the
docstring. I find it not very helpful for the user. We should either say
nothing (as in the patch) or give a concrete reference how to "achieve
the desired effect".

Best,
Ihor

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-org-emphasis-alist-Update-defcustom-making-emphasis-.patch --]
[-- Type: text/x-diff, Size: 3276 bytes --]

From c950768bc08b4cae07ba3cb451d8af0abfec7dc8 Mon Sep 17 00:00:00 2001
Message-Id: <c950768bc08b4cae07ba3cb451d8af0abfec7dc8.1637237807.git.yantar92@gmail.com>
From: Ihor Radchenko <yantar92@gmail.com>
Date: Tue, 16 Nov 2021 15:40:35 +0800
Subject: [PATCH] org-emphasis-alist: Update defcustom making emphasis
 characters constant

* lisp/org.el (org-emphasis-alist): Mention that emphasis characters
should not be changed by user.  Update the defcustom type accordingly.
* doc/org-manual.org (Emphasis and Monospace): Update the referece to
`org-emphasis-alist'.
---
 doc/org-manual.org |  5 +++--
 lisp/org.el        | 41 ++++++++++++++++++++++++++++++++++++-----
 2 files changed, 39 insertions(+), 7 deletions(-)

diff --git a/doc/org-manual.org b/doc/org-manual.org
index c4bb7ef30..9fea2c42e 100644
--- a/doc/org-manual.org
+++ b/doc/org-manual.org
@@ -10814,9 +10814,10 @@ ** Emphasis and Monospace
 exported verbatim.
 
 #+vindex: org-fontify-emphasized-text
+#+vindex: org-emphasis-alist
 To turn off fontification for marked up text, you can set
-~org-fontify-emphasized-text~ to ~nil~.  To narrow down the list of
-available markup syntax, you can customize ~org-emphasis-alist~.
+~org-fontify-emphasized-text~ to ~nil~.  For more fine-tuned
+fontification, you can customize ~org-emphasis-alist~.
 
 ** Subscripts and Superscripts
 :PROPERTIES:
diff --git a/lisp/org.el b/lisp/org.el
index 603b57279..00ac00ab3 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -3833,18 +3833,49 @@ (defcustom org-emphasis-alist
 marker characters and the face to be used by font-lock for highlighting
 in Org buffers.
 
+Do not change the characters and do not add new ones to use custom
+markers for existing styles or to introduce new styles.  Org syntax is
+not meant to be configurable and such modifications will not work with
+export.
+
 You need to reload Org or to restart Emacs after customizing this."
   :group 'org-appearance
   :set 'org-set-emph-re
   :version "24.4"
   :package-version '(Org . "8.0")
-  :type '(repeat
+  :type '(list
 	  (list
-	   (string :tag "Marker character")
+	   (const "*")
+	   (choice
+	    (face :tag "Bold emphasis face")
+	    (plist :tag "Bold emphasis face property list")))
+          (list
+	   (const "/")
+	   (choice
+	    (face :tag "Italic emphasis face")
+	    (plist :tag "Italic emphasis face property list")))
+          (list
+	   (const "_")
+	   (choice
+	    (face :tag "Underline emphasis face")
+	    (plist :tag "Underline emphasis face property list")))
+          (list
+	   (const "=")
+	   (choice
+	    (face :tag "Verbatim emphasis face")
+	    (plist :tag "Verbatim emphasis face property list"))
+	   (const verbatim))
+          (list
+	   (const "~")
+	   (choice
+	    (face :tag "Code emphasis face")
+	    (plist :tag "Code emphasis face property list"))
+	   (const verbatim))
+          (list
+	   (const "+")
 	   (choice
-	    (face :tag "Font-lock-face")
-	    (plist :tag "Face property list"))
-	   (option (const verbatim)))))
+	    (face :tag "Strike-through emphasis face")
+	    (plist :tag "Strike-through emphasis face property list")))))
 
 (defvar org-protecting-blocks '("src" "example" "export")
   "Blocks that contain text that is quoted, i.e. not processed as Org syntax.
-- 
2.32.0


^ permalink raw reply	[flat|nested] 27+ messages in thread

* Re: c47b535bb origin/main org-element: Remove dependency on ‘org-emphasis-regexp-components’
  2021-11-18 12:25           ` Ihor Radchenko
@ 2021-11-18 12:35             ` Nicolas Goaziou
  2021-11-18 12:55               ` Ihor Radchenko
  2021-11-19 16:34             ` c47b535bb origin/main org-element: Remove dependency on ‘org-emphasis-regexp-components’ Max Nikulin
  1 sibling, 1 reply; 27+ messages in thread
From: Nicolas Goaziou @ 2021-11-18 12:35 UTC (permalink / raw)
  To: Ihor Radchenko; +Cc: Max Nikulin, emacs-orgmode

Hello,

Ihor Radchenko <yantar92@gmail.com> writes:

> My intuition says that the current parser behaviour is not correct. It
> would make more sense to prioritise link over italics. However, it would
> require a major change in the parser - instead of a single pass, the
> parser may parse different types of objects sequentially. The emphasis
> objects should come last avoiding the markers to have different parents.
>
> Nicolas, WDYT?

I disagree. Priority should be given to the first object being started.
This is, IMO, the only sane way to handle syntax.

Regards,
-- 
Nicolas Goaziou


^ permalink raw reply	[flat|nested] 27+ messages in thread

* Re: c47b535bb origin/main org-element: Remove dependency on ‘org-emphasis-regexp-components’
  2021-11-18 12:35             ` Nicolas Goaziou
@ 2021-11-18 12:55               ` Ihor Radchenko
  2021-11-19  8:18                 ` Nicolas Goaziou
  0 siblings, 1 reply; 27+ messages in thread
From: Ihor Radchenko @ 2021-11-18 12:55 UTC (permalink / raw)
  To: Nicolas Goaziou; +Cc: Max Nikulin, emacs-orgmode

Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

>> My intuition says that the current parser behaviour is not correct. It
>> would make more sense to prioritise link over italics. However, it would
>> require a major change in the parser - instead of a single pass, the
>> parser may parse different types of objects sequentially. The emphasis
>> objects should come last avoiding the markers to have different parents.
>>
>> Nicolas, WDYT?
>
> I disagree. Priority should be given to the first object being started.
> This is, IMO, the only sane way to handle syntax.

Multi-pass may indeed over-complicate the syntax.
However, it is not clear then how to handle situations like

/A link [[https://orgmode.org/?oops=true][Org Mode]]
or
/A code ~user/?my-user-variable~ with slash/
or
/A verbatim =text/.= with slash/

Should we modify the closing-re for emphasis?

>> (seq (not space)
>>      (group ,mark)
>>      (or (any space ?- ?. ?, ?\; ?: ?! ?? ?' ?\" ?\) ?\} ?\\ ?\[)
>>          line-end))

Best,
Ihor


^ permalink raw reply	[flat|nested] 27+ messages in thread

* Re: c47b535bb origin/main org-element: Remove dependency on ‘org-emphasis-regexp-components’
  2021-11-18 12:55               ` Ihor Radchenko
@ 2021-11-19  8:18                 ` Nicolas Goaziou
  2021-11-19 11:38                   ` [PATCH] " Ihor Radchenko
  0 siblings, 1 reply; 27+ messages in thread
From: Nicolas Goaziou @ 2021-11-19  8:18 UTC (permalink / raw)
  To: Ihor Radchenko; +Cc: Max Nikulin, emacs-orgmode

Hello,

Ihor Radchenko <yantar92@gmail.com> writes:

> However, it is not clear then how to handle situations like
>
> /A link [[https://orgmode.org/?oops=true][Org Mode]]
> or
> /A code ~user/?my-user-variable~ with slash/
> or
> /A verbatim =text/.= with slash/

You can use a zero-width space to help Org sorting out the ambiguity,
for example (| denotes the zero-width space):

  /|A link [[https://orgmode.org/?oops=true][Org Mode]]

  /A code ~user|/?my-user-variable~ with slash/

> Should we modify the closing-re for emphasis?
>
>>> (seq (not space)
>>>      (group ,mark)
>>>      (or (any space ?- ?. ?, ?\; ?: ?! ?? ?' ?\" ?\) ?\} ?\\ ?\[)
>>>          line-end))

I don't think it is necessary. /word/? seems a valid sentence closing.

Regards,
-- 
Nicolas Goaziou


^ permalink raw reply	[flat|nested] 27+ messages in thread

* [PATCH] Re: c47b535bb origin/main org-element: Remove dependency on ‘org-emphasis-regexp-components’
  2021-11-19  8:18                 ` Nicolas Goaziou
@ 2021-11-19 11:38                   ` Ihor Radchenko
  2021-11-19 12:37                     ` Nicolas Goaziou
  0 siblings, 1 reply; 27+ messages in thread
From: Ihor Radchenko @ 2021-11-19 11:38 UTC (permalink / raw)
  To: Nicolas Goaziou; +Cc: Max Nikulin, emacs-orgmode

[-- Attachment #1: Type: text/plain, Size: 809 bytes --]

Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

> You can use a zero-width space to help Org sorting out the ambiguity,
> for example (| denotes the zero-width space):
>
>   /|A link [[https://orgmode.org/?oops=true][Org Mode]]
>
>   /A code ~user|/?my-user-variable~ with slash/

Makes sense. Maybe we should also mention it in the Markup section of
the manual? I attached a tentative patch.

Also, part of the problem with
/|A link [[https://orgmode.org/?oops=true][Org Mode]]
is that the link is emphasised despite not being parser as a link by
org-element. I attached a patch for our link/emphasis fontification that
makes sure that fontification is always consistent with the parser. The
patch might hit the performance slightly, but I do not see obvious
slowdown using my 15Mb notes file.

Best,
Ihor

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-org-manual.org-Clarify-how-to-handle-markup-ambiguit.patch --]
[-- Type: text/x-diff, Size: 1410 bytes --]

From 3b4a857582e848e9688a49c01b853ed577dd4935 Mon Sep 17 00:00:00 2001
Message-Id: <3b4a857582e848e9688a49c01b853ed577dd4935.1637321577.git.yantar92@gmail.com>
From: Ihor Radchenko <yantar92@gmail.com>
Date: Fri, 19 Nov 2021 19:27:56 +0800
Subject: [PATCH] org-manual.org: Clarify how to handle markup ambiguity

* doc/org-manual.org (Emphasis and Monospace): Advice users to insert
zero width space when Org does not parse emphasized text correctly.
---
 doc/org-manual.org | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/doc/org-manual.org b/doc/org-manual.org
index a38dbec4a..1db993d3d 100644
--- a/doc/org-manual.org
+++ b/doc/org-manual.org
@@ -10818,6 +10818,18 @@ ** Emphasis and Monospace
 ~org-fontify-emphasized-text~ to ~nil~.  To narrow down the list of
 available markup syntax, you can customize ~org-emphasis-alist~.
 
+Sometimes, Org mode has difficulties recognising markup.  It usually
+happens when markup marker symbols are present inside verbatim or code
+blocks:
+
+#+begin_example
+/The whole line is supposed to be marked italic, but the following
+~user/?variable~ contains italics =/= marker and confuses Org parser/.
+#+end_example
+
+You can use zero width space to help Org sorting out the ambiguity.
+See [[*Escape Character]] for more details.
+
 ** Subscripts and Superscripts
 :PROPERTIES:
 :DESCRIPTION: Simple syntax for raising/lowering text.
-- 
2.32.0


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #3: 0001-org.el-Make-emphasis-and-link-fontification-consiste.patch --]
[-- Type: text/x-diff, Size: 4619 bytes --]

From d5413e772fe6aedb8a8aa094f98c96fc20b82d3a Mon Sep 17 00:00:00 2001
Message-Id: <d5413e772fe6aedb8a8aa094f98c96fc20b82d3a.1637321613.git.yantar92@gmail.com>
From: Ihor Radchenko <yantar92@gmail.com>
Date: Fri, 19 Nov 2021 19:13:54 +0800
Subject: [PATCH] org.el: Make emphasis and link fontification consistent with
 parser

* lisp/org.el (org-do-emphasis-faces):
(org-activate-links): Do not fontify just based on approximate
regexps.  Make sure the current object is emphasis.
---
 lisp/org.el | 62 ++++++++++++++++++++++++++++-------------------------
 1 file changed, 33 insertions(+), 29 deletions(-)

diff --git a/lisp/org.el b/lisp/org.el
index cb1b58c51..d9f073100 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -5106,12 +5106,15 @@ (defun org-do-emphasis-faces (limit)
 			       (looking-at-p org-outline-regexp-bol))))
 		   ;; Match full emphasis markup regexp.
 		   (looking-at (if verbatim? org-verbatim-re org-emph-re))
-		   ;; Do not span over paragraph boundaries.
-		   (not (string-match-p org-element-paragraph-separate
-					(match-string 2)))
-		   ;; Do not span over cells in table rows.
-		   (not (and (save-match-data (org-match-line "[ \t]*|"))
-			     (string-match-p "|" (match-string 4))))))
+                   ;; Verify that we are at the right object.
+                   (let ((object (save-excursion
+                                   (save-match-data
+                                     (goto-char (match-beginning 2))
+                                     (org-element-context)))))
+                     (and (memq (org-element-type object)
+                                '(bold italic verbatim code strike-through))
+                          (eq (match-beginning 2)
+                              (org-element-property :begin object))))))
 	    (pcase-let ((`(,_ ,face ,_) (assoc marker org-emphasis-alist))
 			(m (if org-hide-emphasis-markers 4 2)))
 	      (font-lock-prepend-text-property
@@ -5206,7 +5209,7 @@ (defun org-activate-links (limit)
 				 (eq 'org-tag face))))))
 	  (let* ((link-object (save-excursion
 				(goto-char start)
-				(save-match-data (org-element-link-parser))))
+				(save-match-data (org-element-context))))
 		 (link (org-element-property :raw-link link-object))
 		 (type (org-element-property :type link-object))
 		 (path (org-element-property :path link-object))
@@ -5229,29 +5232,30 @@ (defun org-activate-links (limit)
 					((and (pred functionp) f) (funcall f))
 					(_ `(:uri ,link)))
 			'font-lock-multiline t)))
-	    (org-remove-flyspell-overlays-in start end)
-	    (org-rear-nonsticky-at end)
-	    (if (not (eq 'bracket style))
-		(progn
+            (when (eq (org-element-type link-object) 'link)
+              (org-remove-flyspell-overlays-in start end)
+	      (org-rear-nonsticky-at end)
+	      (if (not (eq 'bracket style))
+		  (progn
+                    (add-face-text-property start end face-property)
+		    (add-text-properties start end properties))
+	        ;; Handle invisible parts in bracket links.
+	        (remove-text-properties start end '(invisible nil))
+	        (let ((hidden
+		       (append `(invisible
+			         ,(or (org-link-get-parameter type :display)
+				      'org-link))
+			       properties)))
+		  (add-text-properties start visible-start hidden)
                   (add-face-text-property start end face-property)
-		  (add-text-properties start end properties))
-	      ;; Handle invisible parts in bracket links.
-	      (remove-text-properties start end '(invisible nil))
-	      (let ((hidden
-		     (append `(invisible
-			       ,(or (org-link-get-parameter type :display)
-				    'org-link))
-			     properties)))
-		(add-text-properties start visible-start hidden)
-                (add-face-text-property start end face-property)
-		(add-text-properties visible-start visible-end properties)
-		(add-text-properties visible-end end hidden)
-		(org-rear-nonsticky-at visible-start)
-		(org-rear-nonsticky-at visible-end)))
-	    (let ((f (org-link-get-parameter type :activate-func)))
-	      (when (functionp f)
-		(funcall f start end path (eq style 'bracket))))
-	    (throw :exit t)))))		;signal success
+		  (add-text-properties visible-start visible-end properties)
+		  (add-text-properties visible-end end hidden)
+		  (org-rear-nonsticky-at visible-start)
+		  (org-rear-nonsticky-at visible-end)))
+	      (let ((f (org-link-get-parameter type :activate-func)))
+	        (when (functionp f)
+		  (funcall f start end path (eq style 'bracket))))
+	      (throw :exit t))))))		;signal success
     nil))
 
 (defun org-activate-code (limit)
-- 
2.32.0


^ permalink raw reply	[flat|nested] 27+ messages in thread

* Re: [PATCH] Re: c47b535bb origin/main org-element: Remove dependency on ‘org-emphasis-regexp-components’
  2021-11-19 11:38                   ` [PATCH] " Ihor Radchenko
@ 2021-11-19 12:37                     ` Nicolas Goaziou
  2021-11-19 13:53                       ` Ihor Radchenko
  0 siblings, 1 reply; 27+ messages in thread
From: Nicolas Goaziou @ 2021-11-19 12:37 UTC (permalink / raw)
  To: Ihor Radchenko; +Cc: Max Nikulin, emacs-orgmode

Hello,

Ihor Radchenko <yantar92@gmail.com> writes:

> * doc/org-manual.org (Emphasis and Monospace): Advice users to insert
> zero width space when Org does not parse emphasized text correctly.
> ---

[...]

> +Sometimes, Org mode has difficulties recognising markup.  It usually
> +happens when markup marker symbols are present inside verbatim or code
> +blocks:

I disagree with the wording. Org mode has no difficulties recognizing
markup nor does it parses text incorrectly. This is similar to the well
known surprise:

  #+begin_example
  * something
  #+end_example

Here, we are simply fooled by the fontification process.

Otherwise, the patch looks good to me.

> +                   ;; Verify that we are at the right object.
> +                   (let ((object (save-excursion
> +                                   (save-match-data
> +                                     (goto-char (match-beginning 2))
> +                                     (org-element-context)))))
> +                     (and (memq (org-element-type object)
> +                                '(bold italic verbatim code strike-through))
> +                          (eq (match-beginning 2)
> +                              (org-element-property :begin object))))))

I sympathize with the idea. As long as fontification does not rely on
the parser, we will face issues like the one at stake. So, ultimately,
Org will hopefully move in that direction.

However, I'm not convinced making such local changes is going to help us
in the long run. It may be more fruitful to think this evolution
globally, as it involves a fair bit of rewriting of the fontification
process. For example, it may only be necessary to parse the part of the
Org document being fontified only once, and then apply all fontification
functions to the resulting tree rather than have each of them calling
the parser, like the above.

In any case, I think fontification deserves a dedicated thread, in
addition to some love.

Regards,
-- 
Nicolas Goaziou


^ permalink raw reply	[flat|nested] 27+ messages in thread

* Re: [PATCH] Re: c47b535bb origin/main org-element: Remove dependency on ‘org-emphasis-regexp-components’
  2021-11-19 12:37                     ` Nicolas Goaziou
@ 2021-11-19 13:53                       ` Ihor Radchenko
  2021-11-20 18:25                         ` Nicolas Goaziou
  0 siblings, 1 reply; 27+ messages in thread
From: Ihor Radchenko @ 2021-11-19 13:53 UTC (permalink / raw)
  To: Nicolas Goaziou; +Cc: Max Nikulin, emacs-orgmode

[-- Attachment #1: Type: text/plain, Size: 2046 bytes --]

Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

>> +Sometimes, Org mode has difficulties recognising markup.  It usually
>> +happens when markup marker symbols are present inside verbatim or code
>> +blocks:
>
> I disagree with the wording. Org mode has no difficulties recognizing
> markup nor does it parses text incorrectly. This is similar to the well
> known surprise:
>
>   #+begin_example
>   * something
>   #+end_example
>
> Here, we are simply fooled by the fontification process.
>
> Otherwise, the patch looks good to me.

I updated the patch. If you have no objections on the new wording, I
will push it to main.

>> +                   ;; Verify that we are at the right object.
>> +                   (let ((object (save-excursion
>> +                                   (save-match-data
>> +                                     (goto-char (match-beginning 2))
>> +                                     (org-element-context)))))
>> +                     (and (memq (org-element-type object)
>> +                                '(bold italic verbatim code strike-through))
>> +                          (eq (match-beginning 2)
>> +                              (org-element-property :begin object))))))
>
> I sympathize with the idea. As long as fontification does not rely on
> the parser, we will face issues like the one at stake. So, ultimately,
> Org will hopefully move in that direction.
>
> However, I'm not convinced making such local changes is going to help us
> in the long run. It may be more fruitful to think this evolution
> globally, as it involves a fair bit of rewriting of the fontification
> process. For example, it may only be necessary to parse the part of the
> Org document being fontified only once, and then apply all fontification
> functions to the resulting tree rather than have each of them calling
> the parser, like the above.
>
> In any case, I think fontification deserves a dedicated thread, in
> addition to some love.

Ok. I will create a new discussion thread on fontification.

Best,
Ihor


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-org-manual.org-Clarify-how-to-handle-markup-ambiguit.patch --]
[-- Type: text/x-diff, Size: 1379 bytes --]

From a1a497a80578669ef1e96700aa592aadd8d0d7ec Mon Sep 17 00:00:00 2001
Message-Id: <a1a497a80578669ef1e96700aa592aadd8d0d7ec.1637329857.git.yantar92@gmail.com>
From: Ihor Radchenko <yantar92@gmail.com>
Date: Fri, 19 Nov 2021 19:27:56 +0800
Subject: [PATCH] org-manual.org: Clarify how to handle markup ambiguity

* doc/org-manual.org (Emphasis and Monospace): Advice users to insert
zero width space when Org does not parse emphasized text correctly.
---
 doc/org-manual.org | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/doc/org-manual.org b/doc/org-manual.org
index a38dbec4a..9615209a1 100644
--- a/doc/org-manual.org
+++ b/doc/org-manual.org
@@ -10818,6 +10818,17 @@ ** Emphasis and Monospace
 ~org-fontify-emphasized-text~ to ~nil~.  To narrow down the list of
 available markup syntax, you can customize ~org-emphasis-alist~.
 
+=*=, =/=, =_=, ===, and =~= symbols inside =verbatim= or ~code~ can
+sometimes produce unexpected markup.  For example,
+
+#+begin_example
+/The whole line is supposed to be marked italic, but the following
+~user/?variable~ contains italics =/= marker and confuses Org parser/.
+#+end_example
+
+You can use zero width space to help Org sorting out the ambiguity.
+See [[*Escape Character]] for more details.
+
 ** Subscripts and Superscripts
 :PROPERTIES:
 :DESCRIPTION: Simple syntax for raising/lowering text.
-- 
2.32.0


^ permalink raw reply	[flat|nested] 27+ messages in thread

* Re: c47b535bb origin/main org-element: Remove dependency on ‘org-emphasis-regexp-components’
  2021-11-18 12:25           ` Ihor Radchenko
  2021-11-18 12:35             ` Nicolas Goaziou
@ 2021-11-19 16:34             ` Max Nikulin
  1 sibling, 0 replies; 27+ messages in thread
From: Max Nikulin @ 2021-11-19 16:34 UTC (permalink / raw)
  To: emacs-orgmode

On 18/11/2021 05:44, Samuel Wales wrote:
> i think i found it useful, long ago, when it was ok/tolerated to
> change the var, and probably still, to /"do this"/ and /this,/.

D. Knuth in "The TeXbook" put punctuation outside of emphasized text, 
however he mentioned that accordingly to old manuals semicolon should be 
typed in the same font as previous word.

On 18/11/2021 19:25, Ihor Radchenko wrote:
> I updated the patch dropping your last suggested sentence in the
> docstring. I find it not very helpful for the user. We should either say
> nothing (as in the patch) or give a concrete reference how to "achieve
> the desired effect".

I just have never tried to add custom decorations, so it is difficult 
for me to express it by a concise phrase. I even do not know if the 
following recipe could be recommended

https://emacs.stackexchange.com/questions/35626/how-to-make-my-own-org-mode-text-emphasis-work-again

 From other messages I had an impression that it is still possible to 
create custom markup with new marker but actually it was broken years 
ago: 
https://list.orgmode.org/orgmode/CAN9DXH2Xwf+9GGcPrpXFeTFUMeCrxUHtyjduDAYG0Z49FpaJ4g@mail.gmail.com/T/#u
Bug: org-emphasis-alist not fully applied [9.1.1 (9.1.1-1-g80cbf9-elpa @ 
x:/folder/user/.emacs.d/elpa/org-20170918/)] 2017-09-19  4:16

It makes details of implementation less important.

To my taste fixed order of markers (:type list) is too rigid. If a user 
reordered a couple of items then multi-value customization form becomes 
single multiline input area. :set function might sort items, but I do 
not like such solution.

Since I have not found a way to improve customization interface with 
reasonable efforts, I believe that the patch is acceptable. It should be 
an improvement, but not an outstanding one.





^ permalink raw reply	[flat|nested] 27+ messages in thread

* Re: c47b535bb origin/main org-element: Remove dependency on ‘org-emphasis-regexp-components’
  2021-11-16  7:43       ` Ihor Radchenko
  2021-11-16 21:56         ` Samuel Wales
  2021-11-17 16:44         ` Max Nikulin
@ 2021-11-20 12:02         ` Max Nikulin
  2021-11-21 10:01           ` Ihor Radchenko
  2 siblings, 1 reply; 27+ messages in thread
From: Max Nikulin @ 2021-11-20 12:02 UTC (permalink / raw)
  To: emacs-orgmode

[-- Attachment #1: Type: text/plain, Size: 864 bytes --]

On 16/11/2021 14:43, Ihor Radchenko wrote:
> Max Nikulin writes:
> 
>> Better docs and some restriction on defcustom values were discussed earlier:
>> https://list.orgmode.org/87k0oyd3pw.fsf@nicolasgoaziou.fr/
>> Nicolas Goaziou. Re: Using backticks for the inline code delimeter? Mon,
>> 19 Apr 2021 11:27:07 +0200
>>
>> Sorry, I have not prepared a patch. I am not confident with defcustom
>> fine tuning and have not experimented with it since that time.
> 
> Maybe something like the attached?

My draft version is attached. Ihor, thank you for inspiration. Feel free 
to improve it. I hope, it makes problem more apparent to user who tries 
to customize markers. Have I missed some undesired side effects?

Unfortunately with "set" type in defcustom "catch inappropriate" item is 
not hidden for valid configuration.

I do not mind if you commit any variant.

[-- Attachment #2: restrict-org-emphasis-alist.patch --]
[-- Type: text/x-patch, Size: 3680 bytes --]

diff --git a/doc/org-manual.org b/doc/org-manual.org
index a38dbec4a..b62c52e61 100644
--- a/doc/org-manual.org
+++ b/doc/org-manual.org
@@ -10814,9 +10814,11 @@ and verbatim string is not processed for Org specific syntax; it is
 exported verbatim.
 
 #+vindex: org-fontify-emphasized-text
+#+vindex: org-emphasis-alist
 To turn off fontification for marked up text, you can set
-~org-fontify-emphasized-text~ to ~nil~.  To narrow down the list of
-available markup syntax, you can customize ~org-emphasis-alist~.
+~org-fontify-emphasized-text~ to ~nil~.  For more fine-tuned
+fontification consider themes. It is possible to customize
+~org-emphasis-alist~ variable, but it will be deprecated.
 
 ** Subscripts and Superscripts
 :PROPERTIES:
diff --git a/lisp/org.el b/lisp/org.el
index cb1b58c51..ea62ae0b2 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -3771,6 +3771,25 @@ newline      The maximum number of newlines allowed in an emphasis exp.
 
 You need to reload Org or to restart Emacs after setting this.")
 
+(defun org-set-emphasis-alist (var value)
+  "Set VAR (`org-emphasis-alist') to VALUE and check it for ignored characters.
+Warn user that Org syntax can not be extended with new emphasis markers
+if such attempt is detected.  The function is intended for :set argument
+of `defcustom'."
+  (set var value)
+  (let ((unsupported
+	 (delq nil
+	       (mapcar
+                (lambda (entry)
+                  (let ((marker (car entry)))
+                    (unless (member marker '("*" "/" "_" "=" "~" "+")) marker)))
+                value))))
+    (when unsupported
+      (message "Warning! Unsupported markup characters '%s' detected in `%s'"
+	       (mapconcat #'identity unsupported " ")
+	       (symbol-name var))))
+  value)
+
 (defcustom org-emphasis-alist
   '(("*" bold)
     ("/" italic)
@@ -3779,23 +3798,41 @@ You need to reload Org or to restart Emacs after setting this.")
     ("~" org-code verbatim)
     ("+" (:strike-through t)))
   "Alist of characters and faces to emphasize text.
+Warning! This variable will be deprecated in favor of themes.
+
 Text starting and ending with a special character will be emphasized,
 for example *bold*, _underlined_ and /italic/.  This variable sets the
 marker characters and the face to be used by font-lock for highlighting
 in Org buffers.
 
+Do not change the characters and do not add new ones to use custom
+markers for existing styles or to introduce new styles.  Org syntax is
+not meant to be configurable and such modifications will not work with
+export.
+
 You need to reload Org or to restart Emacs after customizing this."
   :group 'org-appearance
   :set 'org-set-emph-re
   :version "24.4"
   :package-version '(Org . "8.0")
+  :set #'org-set-emphasis-alist
   :type '(repeat
-	  (list
-	   (string :tag "Marker character")
-	   (choice
-	    (face :tag "Font-lock-face")
-	    (plist :tag "Face property list"))
-	   (option (const verbatim)))))
+          (group
+           (choice
+            :tag "Marker"
+            (const :tag "*Bold*" "*")
+            (const :tag "/Italic/" "/")
+            (const :tag "_Underline_" "_")
+            (const :tag "+Strike-through+" "+")
+            (const :tag "=Verbatim=" "=")
+            (const :tag "~Code~" "~")
+            ;; To warn users that it does not work.
+            (string :tag "Unsupported ignored character"))
+           (choice
+            :tag "Font"
+            (face :tag "Face")
+            (plist :tag "Property list"))
+           (option (const verbatim)))))
 
 (defvar org-protecting-blocks '("src" "example" "export")
   "Blocks that contain text that is quoted, i.e. not processed as Org syntax.

^ permalink raw reply	[flat|nested] 27+ messages in thread

* Re: [PATCH] Re: c47b535bb origin/main org-element: Remove dependency on ‘org-emphasis-regexp-components’
  2021-11-19 13:53                       ` Ihor Radchenko
@ 2021-11-20 18:25                         ` Nicolas Goaziou
  2021-11-21  9:28                           ` Ihor Radchenko
  0 siblings, 1 reply; 27+ messages in thread
From: Nicolas Goaziou @ 2021-11-20 18:25 UTC (permalink / raw)
  To: Ihor Radchenko; +Cc: Max Nikulin, emacs-orgmode

Hello,

Ihor Radchenko <yantar92@gmail.com> writes:

> I updated the patch. If you have no objections on the new wording, I
> will push it to main.

Thanks for the update, and apologies in advance for being bold, as
I have some additional comments about it.

> * doc/org-manual.org (Emphasis and Monospace): Advice users to insert
> zero width space when Org does not parse emphasized text correctly.

Org _does_ parse emphasized text correctly. It may be seen as
unintuitive, but it's really a fontification problem. Anyway, this is
just a commit message…

> +=*=, =/=, =_=, ===, and =~= symbols inside =verbatim= or ~code~ can
> +sometimes produce unexpected markup.  

OK, but it's not limited to symbols within verbatim or code. What about
something like:

  Sometimes, when marked text also contains the marker character itself,
  the result may be unsettling.

  ...example follows (see below)...

> +#+begin_example
> +/The whole line is supposed to be marked italic, but the following
> +~user/?variable~ contains italics =/= marker and confuses Org parser/.
> +#+end_example

The whole line is not supposed to be marked as italic, as long as we
follow Org syntax. And the parser is not confused at all. The user may
be, however.

I suggest:

  /One may expect this whole sentence to be italicized, but the
  following ~user/?variable~ contains =/= character, which effectively
  stops emphasis there./

> +You can use zero width space to help Org sorting out the ambiguity.
> +See [[*Escape Character]] for more details.

Thinking about it a bit more, you might be right: we may slightly change
the closing part of the emphasis regexp, e.g.:

  (seq
   (not space)
   (group ,mark)
   (or (any space ?- ?')
       (and (any ?. ?, ?\; ?: ?! ?? ?\" ?\) ?\} ?\\ ?\[) (or space line-end))
       line-end))

The logic behind this is that in regular text, we assume usual
punctuation rules apply.

My concern is that the more complicated is the rule, the more difficult
it is to predict. Also, we introduce new corner case, e.g.,

  Woot! I just released Org *10*.0!

So, I'm not totally convinced it is worth the trouble.

Regards,
-- 
Nicolas Goaziou


^ permalink raw reply	[flat|nested] 27+ messages in thread

* Re: [PATCH] Re: c47b535bb origin/main org-element: Remove dependency on ‘org-emphasis-regexp-components’
  2021-11-20 18:25                         ` Nicolas Goaziou
@ 2021-11-21  9:28                           ` Ihor Radchenko
  2021-11-22 18:44                             ` Nicolas Goaziou
  2021-11-27 12:16                             ` org parser and priorities of inline elements Max Nikulin
  0 siblings, 2 replies; 27+ messages in thread
From: Ihor Radchenko @ 2021-11-21  9:28 UTC (permalink / raw)
  To: Nicolas Goaziou; +Cc: Max Nikulin, emacs-orgmode

[-- Attachment #1: Type: text/plain, Size: 2037 bytes --]

Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

> Thanks for the update, and apologies in advance for being bold, as
> I have some additional comments about it.

Constructive critics and suggestions are always welcome. And we do not
have pressing deadlines here :)

>> * doc/org-manual.org (Emphasis and Monospace): Advice users to insert
>> zero width space when Org does not parse emphasized text correctly.
>
> Org _does_ parse emphasized text correctly. It may be seen as
> unintuitive, but it's really a fontification problem. Anyway, this is
> just a commit message…

Agree. It just that the example in the patch _feels_ wrong considering
intuitive definition of verbatim borrowed from LaTeX.

Commit messages are also important, especially years later. I updated
the commit message in the attached new version of the patch.

> Thinking about it a bit more, you might be right: we may slightly change
> the closing part of the emphasis regexp, e.g.:
>
>   (seq
>    (not space)
>    (group ,mark)
>    (or (any space ?- ?')
>        (and (any ?. ?, ?\; ?: ?! ?? ?\" ?\) ?\} ?\\ ?\[) (or space line-end))
>        line-end))
>
> The logic behind this is that in regular text, we assume usual
> punctuation rules apply.

This will fail for "*Bold*?!" or "/Italics/!!!"

Also, is there any reason why we are not simply using punctuation
character class instead of listing punctuation chars explicitly (and
only for English)? What about "_你叫什么名字_?"

Maybe just

(seq
 (not space)
 (group ,mark)
 (0+ (in punctuation))
 (or space line-end))

> My concern is that the more complicated is the rule, the more difficult
> it is to predict. Also, we introduce new corner case, e.g.,
>
>   Woot! I just released Org *10*.0!
>
> So, I'm not totally convinced it is worth the trouble.

I am not sure if "Org *10*.0" is a good general example. It is probably
one of those cases when users want fine control over emphasis and must
use zero width space.

Best,
Ihor


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-org-manual.org-Clarify-how-to-handle-markup-ambiguit.patch --]
[-- Type: text/x-diff, Size: 1380 bytes --]

From 9ad522e8d1f1184ef097611fc30b326b08d5b432 Mon Sep 17 00:00:00 2001
Message-Id: <9ad522e8d1f1184ef097611fc30b326b08d5b432.1637486504.git.yantar92@gmail.com>
From: Ihor Radchenko <yantar92@gmail.com>
Date: Fri, 19 Nov 2021 19:27:56 +0800
Subject: [PATCH] org-manual.org: Clarify how to handle markup ambiguity

* doc/org-manual.org (Emphasis and Monospace): Advice users to insert
zero width space to force Org ignore emphasis markers.
---
 doc/org-manual.org | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/doc/org-manual.org b/doc/org-manual.org
index 1d0213934..19f42fc77 100644
--- a/doc/org-manual.org
+++ b/doc/org-manual.org
@@ -10818,6 +10818,18 @@ ** Emphasis and Monospace
 ~org-fontify-emphasized-text~ to ~nil~.  To narrow down the list of
 available markup syntax, you can customize ~org-emphasis-alist~.
 
+Sometimes, when marked text also contains the marker character itself,
+the result may be unsettling.  For example,
+
+#+begin_example
+/One may expect this whole sentence to be italicized, but the
+following ~user/?variable~ contains =/= character, which effectively
+stops emphasis there./
+#+end_example
+
+You can use zero width space to help Org sorting out the ambiguity.
+See [[*Escape Character]] for more details.
+
 ** Subscripts and Superscripts
 :PROPERTIES:
 :DESCRIPTION: Simple syntax for raising/lowering text.
-- 
2.32.0


^ permalink raw reply	[flat|nested] 27+ messages in thread

* Re: c47b535bb origin/main org-element: Remove dependency on ‘org-emphasis-regexp-components’
  2021-11-20 12:02         ` Max Nikulin
@ 2021-11-21 10:01           ` Ihor Radchenko
  2021-11-21 16:36             ` Max Nikulin
  2021-11-23 17:05             ` [PATCH] org.el: Warning for unsupported markers in `org-set-emphasis-alist' Max Nikulin
  0 siblings, 2 replies; 27+ messages in thread
From: Ihor Radchenko @ 2021-11-21 10:01 UTC (permalink / raw)
  To: Max Nikulin; +Cc: emacs-orgmode

Max Nikulin <manikulin@gmail.com> writes:

> My draft version is attached. Ihor, thank you for inspiration. Feel free 
> to improve it. I hope, it makes problem more apparent to user who tries 
> to customize markers. Have I missed some undesired side effects?

Your version looks a lot better. Thanks!

> Unfortunately with "set" type in defcustom "catch inappropriate" item is 
> not hidden for valid configuration.

I am not very familiar with :set customisation. Could you elaborate what
you mean? I do not understand what "catch inappropriate" refers to.

> + ... It is possible to customize
> +~org-emphasis-alist~ variable, but it will be deprecated.

We probably do not need to mention to-be-deprecated variables in manual.
It will only confuse users.

Maybe we should also take a chance and deprecate org-emphasis-alist now?
We can introduce org-bold/italics/... faces and use them in
org-emphasis-alist default value.

Best,
Ihor


^ permalink raw reply	[flat|nested] 27+ messages in thread

* Re: c47b535bb origin/main org-element: Remove dependency on ‘org-emphasis-regexp-components’
  2021-11-21 10:01           ` Ihor Radchenko
@ 2021-11-21 16:36             ` Max Nikulin
  2021-11-23 17:05             ` [PATCH] org.el: Warning for unsupported markers in `org-set-emphasis-alist' Max Nikulin
  1 sibling, 0 replies; 27+ messages in thread
From: Max Nikulin @ 2021-11-21 16:36 UTC (permalink / raw)
  To: emacs-orgmode

On 21/11/2021 17:01, Ihor Radchenko wrote:
> Max Nikulin writes:
> 
>> My draft version is attached. Ihor, thank you for inspiration. Feel free
>> to improve it. I hope, it makes problem more apparent to user who tries
>> to customize markers. Have I missed some undesired side effects?
> 
>> Unfortunately with "set" type in defcustom "catch inappropriate" item is
>> not hidden for valid configuration.
 >
> I am not very familiar with :set customisation. Could you elaborate what
> you mean? I do not understand what "catch inappropriate" refers to.

I am afraid of confusion of ":set" defcustom parameter and "set" type in 
":type" parameter.

I mean "set" instead of "repeat" (or "list" in your patch). It is 
collection of *unique* items. Other variants allow to add several 
entries for the same marker, e.g. "*". Set does not allow users to add 
duplicated entries. I started from your description of allowed entries
and changed rigid "list" type to "set" and added (approximately)

           (list :tag "Unsupported ignored"
	   (string :tag "Marker")
	   (choice :tag "Font"
	    (face :tag "Face")
	    (plist :tag "Property list"))
	   (option (const verbatim))

to notify user that some part of configuration does not work, it is 
always shown in `customize-variable' interface, even as a disabled 
option for default configuration.

With repeat, "Unsupported" options are not present in customization 
buffer, unless user has something like

(custom-set-variables
   '(org-emphasis-alist
      (quote
        (
	("*" bold)
	("_" underline)
	("=" org-verbatim verbatim)
	("/" italic)
	("~" org-code verbatim)
	("!" org-todo) ;; does not work
	("+" (:strike-through t)))))
   )

It is possible to add "seq" entry to catch list entries having 
completely invalid structure, but I decided that it is redundant.

So with "repeat" type user may notice "Unsupported ignored" option in 
customization interface. I think, it is not enough, so I added ":set" 
function that issues a warning during load time. Invalid configuration 
may still remain hidden in the case of `setq' instead of 
`custom-set-variables' in the init file or `eval-after-load' hook. I am 
afraid, nothing could be done to reliably detect the last case.

>> + ... It is possible to customize
>> +~org-emphasis-alist~ variable, but it will be deprecated.
> 
> We probably do not need to mention to-be-deprecated variables in manual.
> It will only confuse users.

I do not insist. My intention is to inform users that wide spread 
recipes should be avoided.

> Maybe we should also take a chance and deprecate org-emphasis-alist now?
> We can introduce org-bold/italics/... faces and use them in
> org-emphasis-alist default value.

I do not mind. I have never experimented with themes, so I would leave 
this part of work to someone else.



^ permalink raw reply	[flat|nested] 27+ messages in thread

* Re: [PATCH] Re: c47b535bb origin/main org-element: Remove dependency on ‘org-emphasis-regexp-components’
  2021-11-21  9:28                           ` Ihor Radchenko
@ 2021-11-22 18:44                             ` Nicolas Goaziou
  2021-11-23 14:28                               ` Ihor Radchenko
  2021-11-27 12:16                             ` org parser and priorities of inline elements Max Nikulin
  1 sibling, 1 reply; 27+ messages in thread
From: Nicolas Goaziou @ 2021-11-22 18:44 UTC (permalink / raw)
  To: Ihor Radchenko; +Cc: Max Nikulin, emacs-orgmode

Hello,

Ihor Radchenko <yantar92@gmail.com> writes:

> Commit messages are also important, especially years later. I updated
> the commit message in the attached new version of the patch.

Note I'm not saying commit messages are not important. I just won't
spend energy on the wording there.

>> Thinking about it a bit more, you might be right: we may slightly change
>> the closing part of the emphasis regexp, e.g.:
>>
>>   (seq
>>    (not space)
>>    (group ,mark)
>>    (or (any space ?- ?')
>>        (and (any ?. ?, ?\; ?: ?! ?? ?\" ?\) ?\} ?\\ ?\[) (or space line-end))
>>        line-end))
>>
>> The logic behind this is that in regular text, we assume usual
>> punctuation rules apply.
>
> This will fail for "*Bold*?!" or "/Italics/!!!"

Of course. Any regexp will fail somehow.

> Also, is there any reason why we are not simply using punctuation
> character class instead of listing punctuation chars explicitly (and
> only for English)? What about "_你叫什么名字_?"
>
> Maybe just
>
> (seq
>  (not space)
>  (group ,mark)
>  (0+ (in punctuation))
>  (or space line-end))

Historically, Org only focused on ASCII. But it makes sense to extend
the allowed punctuation characters, indeed.

This is orthogonal to OP's issue, however.

>> My concern is that the more complicated is the rule, the more difficult
>> it is to predict. Also, we introduce new corner case, e.g.,
>>
>>   Woot! I just released Org *10*.0!
>>
>> So, I'm not totally convinced it is worth the trouble.
>
> I am not sure if "Org *10*.0" is a good general example. It is probably
> one of those cases when users want fine control over emphasis and must
> use zero width space.

This is simply the first example that crossed my mind. My point is that
changing the regexp substantially may not be rewarding, ultimately.

> +Sometimes, when marked text also contains the marker character itself,
> +the result may be unsettling.  For example,
> +
> +#+begin_example
> +/One may expect this whole sentence to be italicized, but the
> +following ~user/?variable~ contains =/= character, which effectively
> +stops emphasis there./
> +#+end_example
> +
> +You can use zero width space to help Org sorting out the ambiguity.
> +See [[*Escape Character]] for more details.

LGTM!

Regards,
-- 
Nicolas Goaziou


^ permalink raw reply	[flat|nested] 27+ messages in thread

* Re: [PATCH] Re: c47b535bb origin/main org-element: Remove dependency on ‘org-emphasis-regexp-components’
  2021-11-22 18:44                             ` Nicolas Goaziou
@ 2021-11-23 14:28                               ` Ihor Radchenko
  0 siblings, 0 replies; 27+ messages in thread
From: Ihor Radchenko @ 2021-11-23 14:28 UTC (permalink / raw)
  To: Nicolas Goaziou; +Cc: Max Nikulin, emacs-orgmode


>> I am not sure if "Org *10*.0" is a good general example. It is probably
>> one of those cases when users want fine control over emphasis and must
>> use zero width space.
>
> This is simply the first example that crossed my mind. My point is that
> changing the regexp substantially may not be rewarding, ultimately.

Agree. At least not until we find a clearly better variant.

Yet, I do not like the current state of things. Especially with regards
to mixing emphasis and hidden parts of links. Situations like in
/A link [[https://orgmode.org/?oops=true][Org Mode]]
or like reported in
https://list.orgmode.org/orgmode/87pmtqp79s.fsf@web.de/ can be very hard
to notice and cause "mysterious" Org failures.

>> +Sometimes, when marked text also contains the marker character itself,
>> +the result may be unsettling.  For example,
[ 8 more citation lines. Click/Enter to show. ]
>> +
>> +#+begin_example
>> +/One may expect this whole sentence to be italicized, but the
>> +following ~user/?variable~ contains =/= character, which effectively
>> +stops emphasis there./
>> +#+end_example
>> +
>> +You can use zero width space to help Org sorting out the ambiguity.
>> +See [[*Escape Character]] for more details.
>
> LGTM!

Applied.

Best,
Ihor


^ permalink raw reply	[flat|nested] 27+ messages in thread

* [PATCH] org.el: Warning for unsupported markers in `org-set-emphasis-alist'
  2021-11-21 10:01           ` Ihor Radchenko
  2021-11-21 16:36             ` Max Nikulin
@ 2021-11-23 17:05             ` Max Nikulin
  1 sibling, 0 replies; 27+ messages in thread
From: Max Nikulin @ 2021-11-23 17:05 UTC (permalink / raw)
  To: emacs-orgmode

[-- Attachment #1: Type: text/plain, Size: 1989 bytes --]

On 21/11/2021 17:01, Ihor Radchenko wrote:
> Max Nikulin writes:
> 
>> My draft version is attached. Ihor, thank you for inspiration. Feel free
>> to improve it. I hope, it makes problem more apparent to user who tries
>> to customize markers.

A patch is attached. I have dropped changes in documentation since I am 
not the author of them.

For init file (entries may be in arbitrary order)

(custom-set-variables
   '(org-emphasis-alist
      (quote
        (
	("*" bold)
	("_" underline)
	("=" org-verbatim verbatim)
	("/" italic)
	("~" org-code verbatim)
	("!" org-todo) ;; does not work
	("+" (:strike-through t)))))
   )

emacs --eval "(customize-variable 'org-emphasis-alist)"

looks as the following

Hide Org Emphasis Alist:
[INS] [DEL] :
             Marker: [Value Menu] *Bold*
             Font: [Value Menu] Face: (sample) bold 
 
 

             [ ] verbatim 
 
 

[INS] [DEL] :
             Marker: [Value Menu] _Underline_
             Font: [Value Menu] Face: (sample) underline 
 
 

             [ ] verbatim 
 
 

[INS] [DEL] :
             Marker: [Value Menu] =Verbatim=
             Font: [Value Menu] Face: (sample) org-verbatim 
 
 

             [X] verbatim
[INS] [DEL] :
             Marker: [Value Menu] /Italic/
             Font: [Value Menu] Face: (sample) italic 
 
 

             [ ] verbatim 
 
 

[INS] [DEL] :
             Marker: [Value Menu] ~Code~
             Font: [Value Menu] Face: (sample) org-code 
 
 

             [X] verbatim
[INS] [DEL] :
             Marker: [Value Menu] Unsupported ignored character: ! 
 
 

             Font: [Value Menu] Face: (sample) org-todo 
 
 

             [ ] verbatim 
 
 

[INS] [DEL] :
             Marker: [Value Menu] +Strike-through+
             Font: [Value Menu] Property list:
                         [INS] [DEL] :
                         Key: :strike-through 
 
 

                         Value: t 
 
 

             [INS]
             [ ] verbatim 
 
 

[INS]
    [ State ]: SAVED and set.

[-- Attachment #2: 0001-org.el-Warning-for-unsupported-markers-in-org-set-em.patch --]
[-- Type: text/x-patch, Size: 3361 bytes --]

From 1ee52f7ffc1039fd442775e9267403f1dca86b88 Mon Sep 17 00:00:00 2001
From: Max Nikulin <manikulin@gmail.com>
Date: Mon, 22 Nov 2021 23:56:15 +0700
Subject: [PATCH] org.el: Warning for unsupported markers in
 `org-set-emphasis-alist'

* lisp/org.el (org-emphasis-alist, org-set-emphasis-alist): Change
custom variable type definition and add :set parameter to warn users
that non-standard marker characters are ignored.

Attempts to introduce new markers have been discussed enough times to
add some code that should prevent wasting of time.

Unfortunately there is no way to issue warning for e.g. `setq'.
---
 lisp/org.el | 47 +++++++++++++++++++++++++++++++++++++++++------
 1 file changed, 41 insertions(+), 6 deletions(-)

diff --git a/lisp/org.el b/lisp/org.el
index df3d139c7..1a65b6db8 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -3771,6 +3771,25 @@ newline      The maximum number of newlines allowed in an emphasis exp.
 
 You need to reload Org or to restart Emacs after setting this.")
 
+(defun org-set-emphasis-alist (var value)
+  "Set VAR (`org-emphasis-alist') to VALUE and check it for ignored characters.
+Warn user that Org syntax can not be extended with new emphasis markers
+if such attempt is detected.  The function is intended for :set argument
+of `defcustom'."
+  (set var value)
+  (let ((unsupported
+	 (delq nil
+	       (mapcar
+                (lambda (entry)
+                  (let ((marker (car entry)))
+                    (unless (member marker '("*" "/" "_" "=" "~" "+")) marker)))
+                value))))
+    (when unsupported
+      (message "Warning! Unsupported markup characters '%s' detected in `%s'"
+	       (mapconcat #'identity unsupported " ")
+	       (symbol-name var))))
+  value)
+
 (defcustom org-emphasis-alist
   '(("*" bold)
     ("/" italic)
@@ -3784,18 +3803,34 @@ for example *bold*, _underlined_ and /italic/.  This variable sets the
 marker characters and the face to be used by font-lock for highlighting
 in Org buffers.
 
+Do not change the characters and do not add new ones to use custom
+markers for existing styles or to introduce new styles.  Org syntax is
+not meant to be configurable and such modifications will not work with
+export.
+
 You need to reload Org or to restart Emacs after customizing this."
   :group 'org-appearance
   :set 'org-set-emph-re
   :version "24.4"
   :package-version '(Org . "8.0")
+  :set #'org-set-emphasis-alist
   :type '(repeat
-	  (list
-	   (string :tag "Marker character")
-	   (choice
-	    (face :tag "Font-lock-face")
-	    (plist :tag "Face property list"))
-	   (option (const verbatim)))))
+          (group
+           (choice
+            :tag "Marker"
+            (const :tag "*Bold*" "*")
+            (const :tag "/Italic/" "/")
+            (const :tag "_Underline_" "_")
+            (const :tag "+Strike-through+" "+")
+            (const :tag "=Verbatim=" "=")
+            (const :tag "~Code~" "~")
+            ;; To warn users that it does not work.
+            (string :tag "Unsupported ignored character"))
+           (choice
+            :tag "Font"
+            (face :tag "Face")
+            (plist :tag "Property list"))
+           (option (const verbatim)))))
 
 (defvar org-protecting-blocks '("src" "example" "export")
   "Blocks that contain text that is quoted, i.e. not processed as Org syntax.
-- 
2.25.1


^ permalink raw reply	[flat|nested] 27+ messages in thread

* org parser and priorities of inline elements
  2021-11-21  9:28                           ` Ihor Radchenko
  2021-11-22 18:44                             ` Nicolas Goaziou
@ 2021-11-27 12:16                             ` Max Nikulin
  2021-11-27 19:02                               ` Nicolas Goaziou
  1 sibling, 1 reply; 27+ messages in thread
From: Max Nikulin @ 2021-11-27 12:16 UTC (permalink / raw)
  To: emacs-orgmode

On 21/11/2021 16:28, Ihor Radchenko wrote:
> 
> Also, is there any reason why we are not simply using punctuation
> character class instead of listing punctuation chars explicitly (and
> only for English)? What about "_你叫什么名字_?"

It seems punctuation character class is too broad. E.g.
     ¿ INVERTED QUESTION MARK
normally appears before words, while "?" is usually after them. I do not 
see anything special in
     (category-set-mnemonics (char-category-set ?¿))
that may help to discriminate such cases.

An example that confuses fontification but not parser:
     : false [[http://te.st/dir?b-=&a=-][verbatim]] fontification
It is a simplified example, original one:
Chris Hunt. Bug: Tildes in URL impact visible link text
Sun, 27 Dec 2020 11:44:07 -0500.
https://list.orgmode.org/CAH+Wm4-_XHUZKFTf=ZtbfnCPvQWkbEoeGs8EpYm+8SPmu8LHFg@mail.gmail.com/

Nicolas Goaziou. Thu, 18 Nov 2021 13:35:19 +0100.
https://list.orgmode.org/87y25l8wvs.fsf@nicolasgoaziou.fr
> Ihor Radchenko writes:
> 
>> My intuition says that the current parser behaviour is not correct. It
>> would make more sense to prioritise link over italics. However, it would
>> require a major change in the parser - instead of a single pass, the
>> parser may parse different types of objects sequentially. The emphasis
>> objects should come last avoiding the markers to have different parents.
> 
> I disagree. Priority should be given to the first object being started.
> This is, IMO, the only sane way to handle syntax.

Origin of such expectation is not only TeX that changes category of 
characters for argument of verbatim commands. In markdown links and code 
have higher priorities than emphasis as well:

     echo 'A _b `c_ d` e_ f' | pandoc -f markdown -t html -
     <p>A <em>b <code>c_ d</code> e</em> f</p>

Org:
     A _b =c_ d= e_ f
export result (it is more concise and easier to read than output of 
`org-element-parse-secondary-string'):
     <p>
     A <span class="underline">b =c</span> d= e_ f
     </p>

Link in markdown:

     echo 'A _b c <https://orgmode.org/index.htm_?k=v> d e_ f' \
          | pandoc -f markdown -t html -
     <p>A <em>b c <a href="https://orgmode.org/index.htm_?k=v" 
class="uri">https://orgmode.org/index.htm_?k=v</a> d e</em> f</p>

Org:
     <p>
     A <span class="underline">b /c &lt;<a 
href="https://orgmode.org/index.htm">https://orgmode.org/index.htm</a></span>?k=v&gt; 
d/ e_ f
     </p>

I can not estimate efforts necessary to implement priorities of objects 
(verbatim - link - emphasis) in org-elements parser since I have not 
looked into its code. Comparing the following snippets, I might naively 
expect some kind of backtracking:

- A /b *c +d e+ f*    g/ h
- A /b *c +d    f* e+    h

I admit that I can be wrong and "first wins" approach handles buffer of 
incomplete parsed entities in a different way.

P.S. In reStructured text simple nesting is not allowed, maybe it is 
possible to use replacements.



^ permalink raw reply	[flat|nested] 27+ messages in thread

* Re: org parser and priorities of inline elements
  2021-11-27 12:16                             ` org parser and priorities of inline elements Max Nikulin
@ 2021-11-27 19:02                               ` Nicolas Goaziou
  0 siblings, 0 replies; 27+ messages in thread
From: Nicolas Goaziou @ 2021-11-27 19:02 UTC (permalink / raw)
  To: Max Nikulin; +Cc: emacs-orgmode

Hello,

Max Nikulin <manikulin@gmail.com> writes:

> I can not estimate efforts necessary to implement priorities of
> objects (verbatim - link - emphasis) in org-elements parser since
> I have not looked into its code. Comparing the following snippets,
> I might naively expect some kind of backtracking:
>
> - A /b *c +d e+ f*    g/ h
> - A /b *c +d    f* e+    h
>
> I admit that I can be wrong and "first wins" approach handles buffer
> of incomplete parsed entities in a different way.

I don't see any incentive to change the order objects are parsed, once
you know how Org does it. This is just a red herring. What is useful,
however, is to fontify them the way Org sees them.

Regards,
-- 
Nicolas Goaziou


^ permalink raw reply	[flat|nested] 27+ messages in thread

end of thread, other threads:[~2021-11-27 19:03 UTC | newest]

Thread overview: 27+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-11-15  0:53 c47b535bb origin/main org-element: Remove dependency on ‘org-emphasis-regexp-components’ Ihor Radchenko
2021-11-15  9:56 ` Nicolas Goaziou
2021-11-15 15:20   ` Ihor Radchenko
2021-11-15 16:25     ` Max Nikulin
2021-11-16  7:43       ` Ihor Radchenko
2021-11-16 21:56         ` Samuel Wales
2021-11-16 22:16           ` Samuel Wales
2021-11-17 16:44         ` Max Nikulin
2021-11-17 22:44           ` Samuel Wales
2021-11-18 12:25           ` Ihor Radchenko
2021-11-18 12:35             ` Nicolas Goaziou
2021-11-18 12:55               ` Ihor Radchenko
2021-11-19  8:18                 ` Nicolas Goaziou
2021-11-19 11:38                   ` [PATCH] " Ihor Radchenko
2021-11-19 12:37                     ` Nicolas Goaziou
2021-11-19 13:53                       ` Ihor Radchenko
2021-11-20 18:25                         ` Nicolas Goaziou
2021-11-21  9:28                           ` Ihor Radchenko
2021-11-22 18:44                             ` Nicolas Goaziou
2021-11-23 14:28                               ` Ihor Radchenko
2021-11-27 12:16                             ` org parser and priorities of inline elements Max Nikulin
2021-11-27 19:02                               ` Nicolas Goaziou
2021-11-19 16:34             ` c47b535bb origin/main org-element: Remove dependency on ‘org-emphasis-regexp-components’ Max Nikulin
2021-11-20 12:02         ` Max Nikulin
2021-11-21 10:01           ` Ihor Radchenko
2021-11-21 16:36             ` Max Nikulin
2021-11-23 17:05             ` [PATCH] org.el: Warning for unsupported markers in `org-set-emphasis-alist' Max Nikulin

Code repositories for project(s) associated with this inbox:

	https://git.savannah.gnu.org/cgit/emacs/org-mode.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).