Bug: org-mode interprets * as a headline in text between #+BEGIN_.. and #+END_...

Bug: org-mode interprets * as a headline in text between #+BEGIN_.. and #+END_...

[David Talmage]

[-- Attachment #1: Type: text/plain, Size: 1885 bytes --]

I often paste verbatim text into the lab notebook I keep with org-mode.
Org-mode always interprets any line that begins with an asterisk as a
headline, even when the line is surrounded by #+BEGIN_... and #+END_...
patterns. This breaks org-special-edit, making it complain, "No special
environment to edit here", unless I manually insert another character at
the beginning of every line in the block that begins with an asterisk.

The behavior surprised me.  I found two ways to work around it.  First, I
can edit the would-be verbatim text as described above. It will always look
like the original text in org-special-edit. That's marginally acceptable
because it alters my original text and makes me take one more step before I
can copy and paste it elsewhere.  Second, I can put such text in a drawer.
I discovered that org-mode does not mis-interpret my text in a drawer.


Here is an example.  It's markdown text.  I use #+{BEGIN,END}_EXAMPLE but
this behavior occurs in all of the #+BEGIN_.. and #+END_... patterns.

#+BEGIN_EXAMPLE
This is the README.md for rfc-tools, a collection of programs for
processing IETF RFCs.

* fetch-rfcs-by-title.sh downloads into the current directory the RFCs
   whose titles contain the string given on the command line.  Uses an
  rfc-index file in the current directory.  Prefers the PDF version of
  RFCs but will obtain the text version if the PDF is not available.

* fetch-sip-rfcs.sh downloads RFCs that contain "Session Initiation"
  in their titles into the current directory.

* search-rfc-index.sh searches an rfc-index file in the current
  directory for the string given on the command line.  The string can
  contain spaces.

* join-titles.awk turns the contents of an rfc-index file into a
  series of long lines.  Each line begins with the RFC number, then a
  space, then the rest of the entry from the rfc-index.
#+END_EXAMPLE

[-- Attachment #2: Type: text/html, Size: 2278 bytes --]

Re: Bug: org-mode interprets * as a headline in text between #+BEGIN_.. and #+END_...

[John Hendy]

On Thu, Oct 6, 2016 at 3:28 PM, David Talmage
<david.talmage@shoutpoint.com> wrote:
> I often paste verbatim text into the lab notebook I keep with org-mode.
> Org-mode always interprets any line that begins with an asterisk as a
> headline, even when the line is surrounded by #+BEGIN_... and #+END_...
> patterns. This breaks org-special-edit, making it complain, "No special
> environment to edit here", unless I manually insert another character at the
> beginning of every line in the block that begins with an asterisk.
>
> The behavior surprised me.  I found two ways to work around it.  First, I
> can edit the would-be verbatim text as described above. It will always look
> like the original text in org-special-edit. That's marginally acceptable
> because it alters my original text and makes me take one more step before I
> can copy and paste it elsewhere.  Second, I can put such text in a drawer.
> I discovered that org-mode does not mis-interpret my text in a drawer.
>
>

I don't have a good explanation of why, but I know you need a comma to
escape org syntax, even within src blocks:
- mention of it in the manual (footnote 4):
http://orgmode.org/manual/Literal-examples.html
- SO answer: http://stackoverflow.com/questions/7431167/escaping-org-mode-example-block-inside-of-an-example-block

> Here is an example.  It's markdown text.  I use #+{BEGIN,END}_EXAMPLE but
> this behavior occurs in all of the #+BEGIN_.. and #+END_... patterns.
>
> #+BEGIN_EXAMPLE
> This is the README.md for rfc-tools, a collection of programs for
> processing IETF RFCs.
>
> * fetch-rfcs-by-title.sh downloads into the current directory the RFCs
>    whose titles contain the string given on the command line.  Uses an
>   rfc-index file in the current directory.  Prefers the PDF version of
>   RFCs but will obtain the text version if the PDF is not available.
>
> * fetch-sip-rfcs.sh downloads RFCs that contain "Session Initiation"
>   in their titles into the current directory.
>
> * search-rfc-index.sh searches an rfc-index file in the current
>   directory for the string given on the command line.  The string can
>   contain spaces.
>
> * join-titles.awk turns the contents of an rfc-index file into a
>   series of long lines.  Each line begins with the RFC number, then a
>   space, then the rest of the entry from the rfc-index.
> #+END_EXAMPLE
>

Re: Bug: org-mode interprets * as a headline in text between #+BEGIN_.. and #+END_...

[Nick Dokos]

John Hendy <jw.hendy@gmail.com> writes:

> On Thu, Oct 6, 2016 at 3:28 PM, David Talmage
> <david.talmage@shoutpoint.com> wrote:
>> I often paste verbatim text into the lab notebook I keep with org-mode.
>> Org-mode always interprets any line that begins with an asterisk as a
>> headline, even when the line is surrounded by #+BEGIN_... and #+END_...
>> patterns. This breaks org-special-edit, making it complain, "No special
>> environment to edit here", unless I manually insert another character at the
>> beginning of every line in the block that begins with an asterisk.
>>
>> The behavior surprised me.  I found two ways to work around it.  First, I
>> can edit the would-be verbatim text as described above. It will always look
>> like the original text in org-special-edit. That's marginally acceptable
>> because it alters my original text and makes me take one more step before I
>> can copy and paste it elsewhere.  Second, I can put such text in a drawer.
>> I discovered that org-mode does not mis-interpret my text in a drawer.
>>
>>
>
> I don't have a good explanation of why, but I know you need a comma to
> escape org syntax, even within src blocks:
> - mention of it in the manual (footnote 4):
> http://orgmode.org/manual/Literal-examples.html
> - SO answer:
> http://stackoverflow.com/questions/7431167/escaping-org-mode-example-block-inside-of-an-example-block
>

C-c ' to edit the block will rewrite it with a , in front - but only
if the line does not start with a * - if it starts with a *, then C-c
' will complain as the OP posted. And if you have speed keys enabled,
then comma or space will not self-insert, and you got to do something
like C-q , to enter the comma or C-q SPC for a space, making the whole
thing even more confusing. I don't know how it would interact with
org-indent turned off: I didn't try that.

Is the "* on column 0 is a headline" convention a consequence of the
dependence on outline.el? Or is it just historical baggage? I cannot
remember the grammar rules, but I'm wondering if the parser could be
made to say "not a headline" in this situation, and if so, what the
consequences might be (particularly, the bad consequences).

I'm not sure whether Nicolas has actually explained this situation in
the past, but, with gmane non compos mentis, I did not attempt to find
an earlier discussion.

-- 
Nick

Re: Bug: org-mode interprets * as a headline in text between #+BEGIN_.. and #+END_...

[Nicolas Goaziou]

Hello,

Nick Dokos <ndokos@gmail.com> writes:

> Is the "* on column 0 is a headline" convention a consequence of the
> dependence on outline.el? Or is it just historical baggage? I cannot
> remember the grammar rules, but I'm wondering if the parser could be
> made to say "not a headline" in this situation, and if so, what the
> consequences might be (particularly, the bad consequences).
>
> I'm not sure whether Nicolas has actually explained this situation in
> the past, but, with gmane non compos mentis, I did not attempt to find
> an earlier discussion.

This is inherent on how the syntax is defined, and how the parser is
implemented. As an outliner, the main syntactic element in an Org
document is the headline. Everything else has a lower priority.
Therefore, a block will not be parsed before a headline.

The good side of headlines being top level elements is that parsing
(without cache) only needs to start from the closest headline above. If
blocks could be parsed before headlines, parsing should start from the
beginning of the document every time.

Regards,

-- 
Nicolas Goaziou

Re: Bug: org-mode interprets * as a headline in text between #+BEGIN_.. and #+END_...

[David Talmage]

[-- Attachment #1: Type: text/plain, Size: 1058 bytes --]

On Fri, Oct 7, 2016 at 2:09 AM, Nicolas Goaziou <mail@nicolasgoaziou.fr>
wrote:

> Hello,
>
> Nick Dokos <ndokos@gmail.com> writes:
>
> > Is the "* on column 0 is a headline" convention a consequence of the
> > dependence on outline.el? Or is it just historical baggage? I cannot
> > remember the grammar rules, but I'm wondering if the parser could be
> > made to say "not a headline" in this situation, and if so, what the
> > consequences might be (particularly, the bad consequences).
> > ...
>
> This is inherent on how the syntax is defined, and how the parser is
> implemented. As an outliner, the main syntactic element in an Org
> document is the headline. Everything else has a lower priority.
> Therefore, a block will not be parsed before a headline.
>
> ...


That's interesting.  Thanks for the explanation.

What does the parser do with a drawer?  How is that different from a
block?  It seems that one can put text that looks like a headline in a
drawer and it won't be interpreted as a headline.  Why couldn't blocks be
parsed the same way?

[-- Attachment #2: Type: text/html, Size: 1679 bytes --]

Re: Bug: org-mode interprets * as a headline in text between #+BEGIN_.. and #+END_...

[Nicolas Goaziou]

Hello,

David Talmage <david.talmage@shoutpoint.com> writes:

> What does the parser do with a drawer?  How is that different from a
> block?  It seems that one can put text that looks like a headline in a
> drawer and it won't be interpreted as a headline.  Why couldn't blocks be
> parsed the same way?

Drawers behave in exactly the same way as blocks wrt headlines. Nothing
can contain a headline.

Regards,

-- 
Nicolas Goaziou

Re: Bug: org-mode interprets * as a headline in text between #+BEGIN_.. and #+END_...

[Nick Dokos]

Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

> Hello,
>
> Nick Dokos <ndokos@gmail.com> writes:
>
>> Is the "* on column 0 is a headline" convention a consequence of the
>> dependence on outline.el? Or is it just historical baggage? I cannot
>> remember the grammar rules, but I'm wondering if the parser could be
>> made to say "not a headline" in this situation, and if so, what the
>> consequences might be (particularly, the bad consequences).
>>
>> I'm not sure whether Nicolas has actually explained this situation in
>> the past, but, with gmane non compos mentis, I did not attempt to find
>> an earlier discussion.
>
> This is inherent on how the syntax is defined, and how the parser is
> implemented. As an outliner, the main syntactic element in an Org
> document is the headline. Everything else has a lower priority.
> Therefore, a block will not be parsed before a headline.
>
> The good side of headlines being top level elements is that parsing
> (without cache) only needs to start from the closest headline above. If
> blocks could be parsed before headlines, parsing should start from the
> beginning of the document every time.
>

Thank you. I appreciate both your patience and your explanations.

-- 
Nick