Bug? Highlighting inconsistent with export

Bug? Highlighting inconsistent with export

[Clément Pit--Claudel]

[-- Attachment #1.1: Type: text/plain, Size: 1077 bytes --]

Hi all,

In the following snippets, I have marked which bits are syntax-highlighted as code, and which bits are exported as code.

                 Some “=code=” [fn:: ~more~ ~of it~]. ~even more code~— ~this~ —.
   Highlighted?        No            Yes    No       No                No
   Exported?           No            Yes    Yes      No                No

Note the inconsistency in the footnote case.  ~this~ isn't highlighted because it's surrounded by non-breaking space.  ~of it~ isn't highlighted because of the ‘]’, but on export it does get rendered as code.

Some suggestions: Paired delimiters and quotes, such as “” ‘’ <> [], could be allowed around code (by changing the default for ‘org-emphasis-regexp-components’).  Spaces (including zero-width spaces) could also be allowed: this would provide a universal mechanism for ensuring that a bit of code is highlighted. For example, “There are many ~let~s in this code” wouldn't work, but “There are many ~let~​s in this code” would work fine.

Cheers,
Clément.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: Bug? Highlighting inconsistent with export

[Nicolas Goaziou]

Hello,

Clément Pit--Claudel <clement.pit@gmail.com> writes:

> In the following snippets, I have marked which bits are syntax-highlighted as code, and which bits are exported as code.
>
>                  Some “=code=” [fn:: ~more~ ~of it~]. ~even more code~— ~this~ —.
>    Highlighted?        No            Yes    No       No                No
>    Exported?           No            Yes    Yes      No                No
>
> Note the inconsistency in the footnote case. ~this~ isn't highlighted
> because it's surrounded by non-breaking space. ~of it~ isn't
> highlighted because of the ‘]’, but on export it does get rendered as
> code.
>
> Some suggestions: Paired delimiters and quotes, such as “” ‘’ <> [],
> could be allowed around code

It could be hard to catch them all.

> Spaces (including zero-width spaces) could also be allowed

I have no objection to this.

However, another option is to get rid of
`org-emphasis-regexp-components', make every paired "=" character
trigger verbatim mode and every paired "~" characters trigger code mode,
but provide a way to escape "=" and "~". It should probably be extended
to any emphasis markup and special characters like "|", "#"...

There's more work involved, though.

Regards,

-- 
Nicolas Goaziou

Re: Bug? Highlighting inconsistent with export

[Clément Pit--Claudel]

[-- Attachment #1.1.1: Type: text/plain, Size: 1996 bytes --]

On 2016-08-30 06:08, Nicolas Goaziou wrote:
> I have no objection to this.

Ok; patch attached :) Here's a test case:

    ~this~~won't~~work~
    ~but~​~this~​~will~
    ~and~⁠~so~⁠~will~⁠~this~

    ⇒ <code>this~~won't~~work</code>
      <code>but</code>​<code>this</code>​<code>will</code>
      <code>and</code>⁠<code>so</code>⁠<code>will</code>⁠<code>this</code>

You can see the difference in behaviour by resizing the browser window: the second line will break at code boundaries (it uses zero-width spaces) while the second line will stay together (it uses word joiners, aka zero-width no-break spaces)

I detailed the change in the commit message.

> However, another option is to get rid of
> `org-emphasis-regexp-components', make every paired "=" character
> trigger verbatim mode and every paired "~" characters trigger code mode,
> but provide a way to escape "=" and "~". It should probably be extended
> to any emphasis markup and special characters like "|", "#"...

I agree, though this seems non-trivial.  Here's another idea (I don't know how it compares to other options :): src_<lang>[]{} could be extended to allow different delimiters; this would make it more widely usable. For example, the following forms would all be equivalent:

    src_elisp[…options…]<…code…>
    src_elisp[…options…]“…code…”
    src_elisp[…options…]⟨…code…⟩
    src_elisp[…options…]«…code…»

This is inspired by LaTeX's \verb syntax, but using paired delimiters.  The nice part of this is that it would solve the current issue that makes src_<lang> not always usable, namely unmatched curly braces:

    #+PROPERTY: header-args :exports code
    src_elisp{(defun foo () ?})}
    ⇒ <code class="src src-lisp">(defun foo () ?</code>)}

    #+PROPERTY: header-args :exports code
    src_elisp{(defun foo () ?\})} is an ELisp function
    ⇒ <code class="src src-lisp">(defun foo () ?\})</code>

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.1.2: 0001-Add-zero-width-spaces-to-org-emphasis-regexp-compone.patch --]
[-- Type: text/x-diff; name="0001-Add-zero-width-spaces-to-org-emphasis-regexp-compone.patch", Size: 1902 bytes --]

From 8ac4260f312c5ddfc0c350ba30285dd0efde471d Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Cl=C3=A9ment=20Pit--Claudel?= <clement.pitclaudel@live.com>
Date: Tue, 30 Aug 2016 12:22:29 -0400
Subject: [PATCH] Add zero-width spaces to org-emphasis-regexp-components

* lisp/org.el: Add U+8203 and U+8288 to pre and post regexps of
  org-emphasis-regexp-components; this makes it easy to fix cases in
  which a marker is not properly detected, by inserting a zero-width
  space (breaking) or a word joiner (non breaking) before or after the
  marker.
---
 lisp/org.el | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/lisp/org.el b/lisp/org.el
index bec8a99..80f7b06 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -4496,7 +4496,7 @@ After a match, the match groups contain these elements:
 ;; set this option proved cumbersome.  See this message/thread:
 ;; http://article.gmane.org/gmane.emacs.orgmode/68681
 (defvar org-emphasis-regexp-components
-  '(" \t('\"{" "- \t.,:!?;'\")}\\[" " \t\r\n" "." 1)
+  '(" \t('\"{​⁠" "- \t.,:!?;'\")}\\[​⁠" " \t\r\n" "." 1)
   "Components used to build the regular expression for emphasis.
 This is a list with five entries.  Terminology:  In an emphasis string
 like \" *strong word* \", we call the initial space PREMATCH, the final
@@ -4511,7 +4511,9 @@ body-regexp  A regexp like \".\" to match a body character.  Don't use
              non-shy groups here, and don't allow newline here.
 newline      The maximum number of newlines allowed in an emphasis exp.
 
-You need to reload Org or to restart Emacs after customizing this.")
+The default for both pre and post includes zero-width spaces (​)
+and word joiners (⁠, aka zero-width no-break space).  You need to
+reload Org or to restart Emacs after customizing this variable.")
 
 (defcustom org-emphasis-alist
   '(("*" bold)
-- 
2.7.4


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]