[RFC] Org syntax (draft)

[RFC] Org syntax (draft)

[Nicolas Goaziou]

Hello,

As discussed a few days ago, here is a document describing the complete
Org syntax as read by the parser. I also added some comments. I am going
to put the Org file on Worg, so anyone can update it and fix mistakes.

                          ━━━━━━━━━━━━━━━━━━━━
                           ORG SYNTAX (DRAFT)
                          ━━━━━━━━━━━━━━━━━━━━


Table of Contents
─────────────────

1 Headlines and Sections
2 Affiliated Keywords
3 Greater Elements
.. 3.1 Greater Blocks
.. 3.2 Drawers and Property Drawers
.. 3.3 Dynamic Blocks
.. 3.4 Footnote Definitions
.. 3.5 Inlinetasks
.. 3.6 Plain Lists and Items
.. 3.7 Tables
4 Elements
.. 4.1 Babel Call
.. 4.2 Blocks
.. 4.3 Clock, Diary Sexp and Planning
.. 4.4 Comments
.. 4.5 Fixed Width Areas
.. 4.6 Horizontal Rules
.. 4.7 Keywords
.. 4.8 LaTeX Environments
.. 4.9 Node Properties
.. 4.10 Paragraphs
.. 4.11 Table Rows
5 Objects
.. 5.1 Entities and LaTeX Fragments
.. 5.2 Export Snippets
.. 5.3 Footnote References
.. 5.4 Inline Babel Calls and Source Blocks
.. 5.5 Line Breaks
.. 5.6 Links
.. 5.7 Macros
.. 5.8 Targets and Radio Targets
.. 5.9 Statistics Cookies
.. 5.10 Subscript and Superscript
.. 5.11 Table Cells
.. 5.12 Timestamps
.. 5.13 Text Markup


This document describes and comments Org syntax as it is currently read
by its parser (Org Elements) and, therefore, by the export framework.
It also includes a few comments on that syntax.

A core concept in this syntax is that only headlines and sections are
context-free[1][2].  Every other syntactical part only exists within
specific environments.

Three categories are used to classify these environments: “Greater
elements”, “elements”, and “objects”, from the broadest scope to the
narrowest.

The paragraph is the unit of measurement.  An element defines
syntactical parts that are at the same level as a paragraph, i.e. which
cannot contain or be included in a paragraph.  An object is a part that
could be included in an element.  Greater elements are all parts that
can contain an element.

Empty lines belong to the largest element ending before them.  For
example, in a list, empty lines between items belong are part of the
item before them, but empty lines at the end of a list belong to the
plain list element.

Unless specified otherwise, case is not significant.


1 Headlines and Sections
════════════════════════

  A headline is defined as:

  ╭────
  │ STARS KEYWORD PRIORITY TITLE TAGS
  ╰────

  STARS is a string starting at column 0 and containing at least one
  asterisk (and up to `org-inlinetask-min-level' if `org-inlinetask'
  library is loaded).  It’s the sole compulsory part of a headline.

  KEYWORD is a TODO keyword, which have to belong to the list defined in
  `org-todo-keywords'.  Case is significant.

  PRIORITY is a priority cookie, i.e. a single letter preceded by a hash
  sign # and enclosed within square brackets.  Case is significant.

  TITLE can be made of any character but a new line.  Though, it will
  match after every other part have been matched.

  TAGS is made of words containing any alpha-numeric character,
  underscore, at sign, hash sign or percent sign, and separated with
  colons.

  Examples of valid headlines include:

  ╭────
  │ *
  │ 
  │ ** DONE
  │ 
  │ *** Some e-mail
  │ 
  │ **** TODO [#A] COMMENT Title :tag:a2%:
  ╰────

  If the first word appearing in the title is `org-comment-keyword', the
  headline will be considered as “commented”.  If that first word is
  `org-quote-string', it will be considered as “quoted”.  In both
  situations, case is significant.

  If its title is `org-footnote-section', it will be considered as
  a “footnote section”.  Case is significant.

  If `org-archive-tag' is one of its tags, it will be considered as
  “archived”.  Case is significant.

  A headline contains directly at most one section, followed by any
  number of headlines.  Only a section can contain another section.

  A section contains directly any greater element or element.  Only
  a headline can contain a section.  As an exception, text before the
  first headline in the document also belongs to a section.

  In a quoted headline contains a section, the latter will be considered
  as a “quote section”.

  As an example, consider the following document:

  ╭────
  │ An introduction.
  │ 
  │ * A Headline 
  │ 
  │   Some text.
  │ 
  │ ** Sub-Topic 1
  │ 
  │ ** Sub-Topic 2
  │ 
  │ *** Additional entry 
  │ 
  │ ** QUOTE Another Sub-Topic
  │ 
  │    Some other text.
  ╰────

  Its internal structure could be summarized as:

  ╭────
  │ (document
  │  (section)
  │  (headline
  │   (section)
  │   (headline)
  │   (headline
  │    (headline))
  │   (headline
  │    (quote-section))))
  ╰────


2 Affiliated Keywords
═════════════════════

  With the exception of [inlinetasks], [items], [planning], [clocks],
  [node properties] and [table rows], every other element type can be
  assigned attributes.

  This is done by adding specific keywords, named “affiliated keywords”,
  just above the element considered, no blank line allowed.

  Affiliated keywords are built upon one of the following patterns:
  “#+KEY: VALUE”, “#+KEY[OPTIONAL]: VALUE” or “#+ATTR_BACKEND: VALUE”.

  KEY is either “CAPTION”, “HEADER”, “NAME”, “PLOT” or “RESULTS” string.

  BACKEND is a string constituted of alpha-numeric characters, hyphens
  or underscores.

  OPTIONAL and VALUE can contain any character but a new line.  Only
  keywords in `org-element-dual-keywords' can have an optional value.

  An affiliated keyword can appear on multiple lines if KEY belongs to
  `org-element-multiple-keywords' or if its pattern is “#+ATTR_BACKEND:
  VALUE”.

  Affiliated keywords whose KEY belong to `org-element-parsed-keywords'
  can contain objects in their value and their optional value, if
  applicable.


  [inlinetasks] See section 3.5

  [items] See section 3.6

  [planning] See section 4.3

  [clocks] See section 4.3

  [node properties] See section 4.9

  [table rows] See section 4.11


3 Greater Elements
══════════════════

  Unless specified otherwise, greater elements can contain directly any
  other element or greater element excepted:

  • elements of their own type,
  • [node properties], which can only be found in [property drawers],
  • [items], which can only be found in [plain lists].


  [node properties] See section 4.9

  [property drawers] See section 3.2

  [items] See section 3.6

  [plain lists] See section 3.6


3.1 Greater Blocks
──────────────────

  Greater blocks consist in the following pattern:

  ╭────
  │ #+BEGIN_NAME PARAMETERS
  │ CONTENTS
  │ #+END_NAME
  ╰────

  NAME can contain any non-whitespace character.

  PARAMETERS can contain any character, and can be omitted.

  If NAME is “CENTER”, it will be a “center block”.  If it is “QUOTE”,
  it will be a “quote block”.

  If the block is neither a center block, a quote block or a [block
  element], it will be a “special block”.

  CONTENTS can contain any element, but another greater block of the
  same type.


  [block element] See section 4.2


3.2 Drawers and Property Drawers
────────────────────────────────

  Pattern for drawers is:

  ╭────
  │ :NAME:
  │ CONTENTS
  │ :END:
  ╰────

  NAME has to either be “PROPERTIES” or belong to `org-drawers' list.

  If NAME is “PROPERTIES”, the drawer will become a “property drawer”.

  In a property drawers, CONTENTS can only contain [node property]
  elements.  Otherwise it can contain any element but another drawer or
  property drawer.

                                  ―――――

  It would be nice if users hadn’t to register drawers names before
  using them in `org-drawers' (or through the `#+DRAWERS:' keyword).
  Anything starting with `^[ \t]*:\w+:[ \t]$' and ending with
  `^[ \t]*:END:[ \t]$' could be considered as a drawer.  — ngz


  [node property] See section 4.9


3.3 Dynamic Blocks
──────────────────

  Pattern for dynamic blocks is:

  ╭────
  │ #+BEGIN: NAME PARAMETERS
  │ CONTENTS
  │ #+END:
  ╰────

  NAME cannot contain any whitespace character.

  PARAMETERS can contain any character and can be omitted.


3.4 Footnote Definitions
────────────────────────

  Pattern for footnote definitions is:

  ╭────
  │ [LABEL] CONTENTS
  ╰────

  It must start at column 0.

  LABEL is either a number or follows the pattern “fn:WORD”, where word
  can contain any word-constituent character, hyphens and underscore
  characters.

  CONTENTS can contain any element excepted another footnote definition.
  It ends at the next footnote definition, the next headline, two
  consecutive empty lines or the end of buffer.


3.5 Inlinetasks
───────────────

  Inlinetasks are defined by `org-inlinetask-min-level' contiguous
  asterisk characters starting at column 0, followed by a whitespace
  character.

  Optionally, inlinetasks can be ended with a string constituted of
  `org-inlinetask-min-level' contiguous characters starting at column 0,
  followed by a space and the “END” string.

  Inlinetasks are recognized only after `org-inlinetask' library is
  loaded.


3.6 Plain Lists and Items
─────────────────────────

  Items are defined by a line starting with the following pattern:
  “BULLET COUNTER-SET CHECK-BOX TAG”, in which only BULLET is mandatory.

  BULLET is either an asterisk, a hyphen, a plus sign character or
  follows either the pattern “COUNTER.” or “COUNTER)".  In any case,
  BULLET is follwed by a whitespace character or line ending.

  COUNTER can be a number or a single letter.

  COUNTER-SET follows the pattern [@COUNTER].

  CHECK-BOX is either a single whitespace character, a “X” character or
  a hyphen, enclosed within square brackets.

  TAG follows “TAG-TEXT ::” pattern, where TAG-TEXT can contain any
  character but a new line.

  An item ends before the next item, the first line less or equally
  indented than its starting line, or two consecutive empty lines.
  Indentation of lines within other greater elements do not count,
  neither do inlinetasks boundaries.

  A plain list is a set of consecutive items of the same indentation.
  It can only directly contain items.

  If first item in a plain list has a counter in its bullet, the plain
  list will be an “ordered plain-list”.  If it contains a tag, it will
  be a “descriptive list”.  Otherwise, it will be an “unordered list”.
  List types are mutually exclusive.

  For example, consider the following excerpt of an Org document:

  ╭────
  │ 1. item 1
  │ 2. [X] item 2
  │    - some tag :: item 2.1
  ╰────

  Its internal structure is as follows:

  ╭────
  │ (ordered-plain-list
  │  (item)
  │  (item
  │   (descriptive-plain-list
  │    (item))))
  ╰────


3.7 Tables
──────────

  Tables start at lines beginning with either a vertical bar or the “+-”
  string followed by plus or minus signs only, assuming they are not
  preceded with lines of the same type.  These lines can be indented.

  A table starting with a vertical bar has “org” type.  Otherwise it has
  “table.el” type.

  Org tables end at the first line not starting with a vertical bar.
  Table.el tables end at the first line not starting with either
  a vertical line or a plus sign.  Such lines can be indented.

  An org table can only contain table rows.  A table.el table does not
  contain anything.


4 Elements
══════════

  Elements cannot contain any other element.

  Only [keywords] whose name belongs to
  `org-element-document-properties', [verse blocks] , [paragraphs] and
  [table rows] can contain objects.


  [keywords] See section 4.7

  [verse blocks] See section 4.2

  [paragraphs] See section 4.10

  [table rows] See section 4.11


4.1 Babel Call
──────────────

  Pattern for babel calls is:

  ╭────
  │ #+CALL: VALUE
  ╰────

  VALUE is optional.  It can contain any character but a new line.


4.2 Blocks
──────────

  Like [greater blocks], pattern for blocks is:

  ╭────
  │ #+BEGIN_NAME DATA
  │ CONTENTS
  │ #+END_NAME
  ╰────

  NAME cannot contain any whitespace character.

  If NAME is “COMMENT”, it will be a “comment block”.  If it is
  “EXAMPLE”, it will be an “example block”.  If it is “SRC”, it will be
  a “source block”.  If it is “VERSE”, it will be a “verse block”.

  If NAME is a string matching the name of any export back-end loaded,
  the block will be an “export block”.

  DATA can contain any character but a new line.  It can be ommitted,
  unless the block is a “source block”.  In this case, it must follow
  the pattern “LANGUAGE SWITCHES ARGUMENTS”, where SWITCHES and
  ARGUMENTS are optional.

  LANGUAGE cannot contain any whitespace character.

  SWITCHES is made of any number of “SWITCH” patterns, separated by
  blank lines.

  A SWITCH pattern is either “-l “FORMAT"", where FORMAT can contain any
  character but a double quote and a new line, “-S” or “+S”, where
  S stands for a single letter.

  ARGUMENTS can contain any character but a new line.

  CONTENTS can contain any character, including new lines.  Though it
  will only contain Org objects if the block is a verse block.
  Otherwise, contents will not be parsed.


  [greater blocks] See section 3.1


4.3 Clock, Diary Sexp and Planning
──────────────────────────────────

  A clock follows the pattern:

  ╭────
  │ CLOCK: TIMESTAMP DURATION
  ╰────

  Both TIMESTAMP and DURATION are optional.

  TIMESTAMP is a [timestamp] object.

  DURATION follows the pattern:

  ╭────
  │ => HH:MM
  ╰────

  HH is a number containing any number of digits.  MM is a two digit
  numbers.

  A diary sexp is a line starting at column 0 with “%%(" string.  It can
  then contain any character besides a new line.

  A planning is a line filled with more at most three INFO parts, where
  each INFO part follows the pattern:

  ╭────
  │ KEYWORD: TIMESTAMP
  ╰────

  KEYWORD is a string among `org-deadline-string',
  `org-scheduled-string' and `org-closed-string'.  TIMESTAMP is is
  a [timestamp] object.

  Even though a planning element can exist anywhere in a section or
  a greater element, it will only affect the headline containing the
  section if it is put on the line following that headline.


  [timestamp] See section 5.12


4.4 Comments
────────────

  A “comment line” starts with a hash signe and a whitespace character
  or an end of line.

  Comments can contain any number of consecutive comment lines.


4.5 Fixed Width Areas
─────────────────────

  A “fixed-width line” start with a colon character and a whitespace or
  an end of line.

  Fixed width areas can contain any number of consecutive fixed-width
  lines.


4.6 Horizontal Rules
────────────────────

  A horizontal rule is a line made of at least 5 consecutive hyphens.
  It can be indented.


4.7 Keywords
────────────

  Keywords follow the syntax:

  ╭────
  │ #+KEY: VALUE
  ╰────

  KEY can contain any non-whitespace character, but it cannot be equal
  to “CALL” or any affiliated keyword.

  VALUE can contain any character excepted a new line.

  If KEY belongs to `org-element-document-properties', VALUE can contain
  objects.


4.8 LaTeX Environments
──────────────────────

  Pattern for LaTeX environments is:

  ╭────
  │ \begin{NAME}
  │ CONTENTS
  │ \end{NAME}
  ╰────

  NAME is constituted of alpha-numeric characters and may end with an
  asterisk.

  CONTENTS can contain anything but the “\end{NAME}” string.


4.9 Node Properties
───────────────────

  Patter for node properties is:

  ╭────
  │ :PROPERTY: VALUE
  ╰────

  PROPERTY can contain any non-whitespace character.  VALUE can contain
  any character but a new line.

  Node properties can only exist in a [property drawers].


  [property drawers] See section 3.2


4.10 Paragraphs
───────────────

  Paragraphs are the default element, which means that any unrecognized
  context is a paragraph.

  Empty lines and other elements end paragraphs.

  Paragraphs can contain every type of object.


4.11 Table Rows
───────────────

  A table rows is either constituted of a vertical bar and any number of
  [table cells] or a vertical bar followed by a hyphen.

  In the first case the table row has the “standard” type.  In the
  second case, it has the “rule” type.

  Table rows can only exist in [tables].


  [table cells] See section 5.11

  [tables] See section 3.7


5 Objects
═════════

  Objects can only be found in the following locations:

  • [affiliated keywords] defined in `org-element-parsed-keywords',
  • [document properties],
  • [headline] titles,
  • [inlinetask] titles,
  • [item] tags,
  • [paragraphs],
  • [table cells],
  • [table rows], which can only contain table cell objects,
  • [verse blocks].

  Most objects cannot contain objects.  Those which can will be
  specified.


  [affiliated keywords] See section 2

  [document properties] See section 4.7

  [headline] See section 1

  [inlinetask] See section 3.5

  [item] See section 3.6

  [paragraphs] See section 4.10

  [table cells] See section 5.11

  [table rows] See section 4.11

  [verse blocks] See section 4.2


5.1 Entities and LaTeX Fragments
────────────────────────────────

  An entity follows the pattern:

  ╭────
  │ \NAME POST
  ╰────

  where NAME has a valid association in either `org-entities' or
  `org-entities-user'.

  POST is the end of line, "{}" string, or a non-alphabetical character.
  It isn’t separated from NAME by a whitespace character.

  A LaTeX fragment can follow multiple patterns:

  ╭────
  │ \NAME POST
  │ \(CONTENTS\)
  │ \[CONTENTS\]
  │ $$CONTENTS$$
  │ PRE$CHAR$POST
  │ PRE$BORDER1 BODY BORDER2$
  ╰────

  NAME contains alphabetical characters only and must not have an
  association in either `org-entities' or `org-entities-user'.

  POST is the same as for entities.

  CONTENTS can contain any character but cannot contain “\)" in the
  second template or “\]" in the third one.

  PRE is either the beginning of line or a character different from `$'.

  CHAR is a non-whitespace character different from `.', ~,~, `?', `;',
  ~’~ or a double quote.

  POST is any of `-', `.', ~,~, `?', `;', `:', ~’~, a double quote,
  a whitespace character and the end of line.

  BORDER1 is a non-whitespace character different from `.', `;', `.'
  and `$'.

  BODY can contain any character excepted `$', and may not span over
  more than 3 lines.

  BORDER2 is any non-whitespace character different from ~,~, `.' and
  `$'.

                                  ―――――

        It would introduce incompatibilities with previous Org
        versions, but support for “$…$” (and for symmetry,
        `$$...$$') constructs ought to be removed.

        They are slow to parse, fragile, redundant, imply false
        positives and do not look good in LaTeX output anyway.
        Even the LaTeX community suggests to use `\(...\)' over
        `$...$'.  — ngz


5.2 Export Snippets
───────────────────

  Patter for export snippets is:

  ╭────
  │ @@NAME:VALUE@@
  ╰────

  NAME can contain any alpha-numeric character and hyphens.

  VALUE can contain anything but “@@” string.


5.3 Footnote References
───────────────────────

  There are four patterns for footnote references:

  ╭────
  │ [MARK]
  │ [fn:LABEL]
  │ [fn:LABEL:DEFINITION]
  │ [fn::DEFINITION]
  ╰────

  MARK is a number.

  LABEL can contain any word constituent character, hyphens and
  underscores.

  DEFINITION can contain any character.  Though opening and closing
  square brackets must be balanced in it.  It can contain any object
  encountered in a paragraph, even other footnote references.

  If the reference follows the third pattern, it is called an “inline
  footnote”.  If it follows the fourth one, i.e. if LABEL is omitted, it
  is an “anonymous footnote”.


5.4 Inline Babel Calls and Source Blocks
────────────────────────────────────────

  Inline Babel calls follow any of the following patterns:

  ╭────
  │ call_NAME(ARGUMENTS)
  │ call_NAME[HEADER](ARGUMENTS)[HEADER]
  ╰────

  NAME can contain any character besides `(', `)' and “\n”.

  HEADER can contain any character besides `]' and “\n”.

  ARGUMENTS can contain any character besides `)' and “\n”.

  Inline source blocks follow any of the following patterns:

  ╭────
  │ src_LANG{BODY}
  │ src_LANG[OPTIONS]{BODY}
  ╰────

  LANG can contain any non-whitespace character.

  OPTIONS and BODY can contain any character but “\n”.


5.5 Line Breaks
───────────────

  A line break consists in “\\SPACE” pattern at the end of an otherwise
  non-empty line.

  SPACE can contain any number of tabs and spaces, including 0.


5.6 Links
─────────

  There are 4 major types of links:

  ╭────
  │ RADIO                     ("radio" link)
  │ <PROTOCOL:PATH>           ("angle" link)
  │ PRE PROTOCOL:PATH2 POST   ("plain" link)
  │ [[PATH3]DESCRIPTION]      ("regular" link)
  ╰────

  RADIO is a string matched by some [radio target].  It can contain
  [entities], [latex fragments], [subscript] and [superscript] only.

  PROTOCOL is a string among `org-link-types'.

  PATH can contain any character but `]', `<', `>' and `\n'.

  PRE and POST are non word constituent.  They can be, respectively, the
  beginning or the end of a line.

  PATH2 can contain any non-whitespace character excepted `(', `)', `<'
  and `>'.  It must end with a word-constituent character, or any
  non-whitespace non-punctuation character followed by `/'.

  DESCRIPTION must be enclosed within square brackets.  It can contain
  any character but square brackets.  Object-wise, it can contain any
  object found in a paragraph excepted a [footnote reference], a [radio
  target] and a [line break].  It cannot contain another link either,
  unless it is a plain link.

  DESCRIPTION is optional.

  PATH3 is built according to the following patterns:

  ╭────
  │ FILENAME           ("file" type)
  │ PROTOCOL:PATH4     ("PROTOCOL" type)
  │ id:ID              ("id" type)
  │ #CUSTOM-ID         ("custom-id" type)
  │ (CODEREF)          ("coderef" type)
  │ FUZZY              ("fuzzy" type)
  ╰────

  FILENAME is a file name, either absolute or relative.

  PATH4 can contain any character besides square brackets.

  ID is constituted of hexadecimal numbers separated with hyphens.

  PATH4, CUSTOM-ID, CODEREF and FUZZY can contain any character besides
  square brackets.

                                  ―――――

        I suggest to remove angle links.  If one needs spaces in
        PATH, she can use standard link syntax instead.

        I also suggest to remove `org-link-types' dependency in
        PROTOCOL and match `[a-zA-Z]' instead, for portability.  —
        ngz


  [radio target] See section 5.8

  [entities] See section 5.1

  [latex fragments] See section 5.1

  [subscript] See section 5.10

  [superscript] See section 5.10

  [footnote reference] See section 5.3

  [line break] See section 5.5


5.7 Macros
──────────

  Macros follow the pattern:

  ╭────
  │ {{{NAME(ARGUMENTS)}}}
  ╰────

  NAME must start with a letter and can be followed by any number of
  alpha-numeric characters, hyphens and underscores.

  ARGUMENTS can contain anything but "}}}" string.  Values within
  ARGUMENTS are separated by commas.  Non-separating commas have to be
  escaped with a backslash character.


5.8 Targets and Radio Targets
─────────────────────────────

  Radio targets follow the pattern:

  ╭────
  │ <<<CONTENTS>>>
  ╰────

  CONTENTS can be any character besides `<', `>' and “\n”.  As far as
  objects go, it can contain [entities], [latex fragments], [subscript]
  and [superscript] only.

  Targets follow the pattern:

  ╭────
  │ <<TARGET>>
  ╰────

  TARGET can contain any character besides `<', `>' and “\n”.  It cannot
  contain any object.


  [entities] See section 5.1

  [latex fragments] See section 5.1

  [subscript] See section 5.10

  [superscript] See section 5.10


5.9 Statistics Cookies
──────────────────────

  Statistics cookies follow either pattern:

  ╭────
  │ [PERCENT%]
  │ [NUM1/NUM2]
  ╰────

  PERCENT, NUM1 and NUM2 are numbers or the empty string.


5.10 Subscript and Superscript
──────────────────────────────

  Pattern for subscript is:

  ╭────
  │ CHAR_SCRIPT
  ╰────

  Pattern for superscript is:

  ╭────
  │ CHAR^SCRIPT
  ╰────

  CHAR is any non-whitespace character.

  SCRIPT can be `*', a string made of word-constituent characters maybe
  preceded by a plus or a minus sign, an expression enclosed in
  parenthesis (resp. curly brackets) containing balanced parenthesis
  (resp. curly brackets).


5.11 Table Cells
────────────────

  Table cells follow the pattern:

  ╭────
  │ CONTENTS|
  ╰────

  CONTENTS can contain any character excepted a vertical bar.


5.12 Timestamps
───────────────

  There are seven possible patterns for timestamps:

  ╭────
  │ <%%(SEXP)>                                     (diary)
  │ <DATE TIME REPEATER>                         (active)
  │ [DATE TIME REPEATER]                         (inactive)
  │ <DATE TIME REPEATER>--<DATE TIME REPEATER>   (active range)
  │ <DATE TIME-TIME REPEATER>                    (active range)
  │ [DATE TIME REPEATER]--[DATE TIME REPEATER]   (inactive range)
  │ [DATE TIME-TIME REPEATER]                    (inactive range)
  ╰────

  SEXP can contain any character excepted `>' and `\n'.

  DATE follows the pattern:

  ╭────
  │ YYYY-MM-DD DAYNAME
  ╰────

  Y, M and D are digits.  DAYNAME can contain any non
  whitespace-character besides `+', `-', `]', `>', a digit or `\n'.

  TIME follows the pattern =H:MM~.  H can be one or two digit long and
  can start with 0.

  REPEATER follows the patter:

  ╭────
  │ MARK VALUE UNIT
  ╰────

  MARK is `+' (cumulate type), `++' (catch-up type) or `.+' (restart
  type).

  VALUE is a number.

  UNIT is a character among `h' (hour), `d' (day), `w' (week), `m'
  (month), `y' (year).

  MARK, VALUE and UNIT are not separated by whitespace characters.


5.13 Text Markup
────────────────

  Text markup follows the pattern:

  ╭────
  │ PRE MARKER CONTENTS MARKER POST
  ╰────

  PRE is a whitespace character, `(', `{' ~’~ or a double quote.  It can
  also be a beginning of line.

  MARKER is a character among `*' (bold), `=' (verbatim), `/' (italic),
  `+' (strike-through), `_' (underline), `~' (code).

  CONTENTS is a string following the pattern:

  ╭────
  │ BORDER BODY BORDER
  ╰────

  BORDER can be any non-whitespace character excepted ~,~, ~’~ or
  a double quote.

  BODY can contain contain any character but may not span over more than
  3 lines.

  BORDER and BODY are not separated by whitespaces.

  CONTENTS can contain any object encountered in a paragraph when markup
  is “bold”, “italic”, “strike-through” or “underline”.

  POST is a whitespace character, `-', `.', ~,~, `:', `!', `?', ~’~,
  `)', `}' or a double quote.  It can also be an end of line.

  PRE, MARKER, CONTENTS, MARKER and POST are not separated by whitespace
  characters.

                                  ―――――

        All of this is wrong if `org-emphasis-regexp-components'
        or `org-emphasis-alist' are modified.

        This should really be simplified and made persistent
        (i.e. no defcustom allowed).  Otherwise, portability and
        parsing are jokes.

        Also, CONTENTS should be anything within code and verbatim
        emphasis, by definition.  — ngz



Footnotes
─────────

[1] In particular, the parser requires stars at column 0 to be quoted
by a comma when they do not define a headline.

[2] It also means that only headlines and sections can be recognized
just by looking at the beginning of the line.

As a consequence, using `org-element-at-point' or
`org-element-context' will move up to the parent headline, and parse
top-down from there until context around is found.



Regards,

-- 
Nicolas Goaziou

Re: [RFC] Org syntax (draft)

[Carsten Dominik]

woooooow, this is awesome Nicolas, thank you!

- Carsten

On 7.3.2013, at 21:37, Nicolas Goaziou <n.goaziou@gmail.com> wrote:

> Hello,
> 
> As discussed a few days ago, here is a document describing the complete
> Org syntax as read by the parser. I also added some comments. I am going
> to put the Org file on Worg, so anyone can update it and fix mistakes.
> 
>                          ━━━━━━━━━━━━━━━━━━━━
>                           ORG SYNTAX (DRAFT)
>                          ━━━━━━━━━━━━━━━━━━━━
> 
> 
> Table of Contents
> ─────────────────
> 
> 1 Headlines and Sections
> 2 Affiliated Keywords
> 3 Greater Elements
> .. 3.1 Greater Blocks
> .. 3.2 Drawers and Property Drawers
> .. 3.3 Dynamic Blocks
> .. 3.4 Footnote Definitions
> .. 3.5 Inlinetasks
> .. 3.6 Plain Lists and Items
> .. 3.7 Tables
> 4 Elements
> .. 4.1 Babel Call
> .. 4.2 Blocks
> .. 4.3 Clock, Diary Sexp and Planning
> .. 4.4 Comments
> .. 4.5 Fixed Width Areas
> .. 4.6 Horizontal Rules
> .. 4.7 Keywords
> .. 4.8 LaTeX Environments
> .. 4.9 Node Properties
> .. 4.10 Paragraphs
> .. 4.11 Table Rows
> 5 Objects
> .. 5.1 Entities and LaTeX Fragments
> .. 5.2 Export Snippets
> .. 5.3 Footnote References
> .. 5.4 Inline Babel Calls and Source Blocks
> .. 5.5 Line Breaks
> .. 5.6 Links
> .. 5.7 Macros
> .. 5.8 Targets and Radio Targets
> .. 5.9 Statistics Cookies
> .. 5.10 Subscript and Superscript
> .. 5.11 Table Cells
> .. 5.12 Timestamps
> .. 5.13 Text Markup
> 
> 
> This document describes and comments Org syntax as it is currently read
> by its parser (Org Elements) and, therefore, by the export framework.
> It also includes a few comments on that syntax.
> 
> A core concept in this syntax is that only headlines and sections are
> context-free[1][2].  Every other syntactical part only exists within
> specific environments.
> 
> Three categories are used to classify these environments: “Greater
> elements”, “elements”, and “objects”, from the broadest scope to the
> narrowest.
> 
> The paragraph is the unit of measurement.  An element defines
> syntactical parts that are at the same level as a paragraph, i.e. which
> cannot contain or be included in a paragraph.  An object is a part that
> could be included in an element.  Greater elements are all parts that
> can contain an element.
> 
> Empty lines belong to the largest element ending before them.  For
> example, in a list, empty lines between items belong are part of the
> item before them, but empty lines at the end of a list belong to the
> plain list element.
> 
> Unless specified otherwise, case is not significant.
> 
> 
> 1 Headlines and Sections
> ════════════════════════
> 
>  A headline is defined as:
> 
>  ╭────
>  │ STARS KEYWORD PRIORITY TITLE TAGS
>  ╰────
> 
>  STARS is a string starting at column 0 and containing at least one
>  asterisk (and up to `org-inlinetask-min-level' if `org-inlinetask'
>  library is loaded).  It’s the sole compulsory part of a headline.
> 
>  KEYWORD is a TODO keyword, which have to belong to the list defined in
>  `org-todo-keywords'.  Case is significant.
> 
>  PRIORITY is a priority cookie, i.e. a single letter preceded by a hash
>  sign # and enclosed within square brackets.  Case is significant.
> 
>  TITLE can be made of any character but a new line.  Though, it will
>  match after every other part have been matched.
> 
>  TAGS is made of words containing any alpha-numeric character,
>  underscore, at sign, hash sign or percent sign, and separated with
>  colons.
> 
>  Examples of valid headlines include:
> 
>  ╭────
>  │ *
>  │ 
>  │ ** DONE
>  │ 
>  │ *** Some e-mail
>  │ 
>  │ **** TODO [#A] COMMENT Title :tag:a2%:
>  ╰────
> 
>  If the first word appearing in the title is `org-comment-keyword', the
>  headline will be considered as “commented”.  If that first word is
>  `org-quote-string', it will be considered as “quoted”.  In both
>  situations, case is significant.
> 
>  If its title is `org-footnote-section', it will be considered as
>  a “footnote section”.  Case is significant.
> 
>  If `org-archive-tag' is one of its tags, it will be considered as
>  “archived”.  Case is significant.
> 
>  A headline contains directly at most one section, followed by any
>  number of headlines.  Only a section can contain another section.
> 
>  A section contains directly any greater element or element.  Only
>  a headline can contain a section.  As an exception, text before the
>  first headline in the document also belongs to a section.
> 
>  In a quoted headline contains a section, the latter will be considered
>  as a “quote section”.
> 
>  As an example, consider the following document:
> 
>  ╭────
>  │ An introduction.
>  │ 
>  │ * A Headline 
>  │ 
>  │   Some text.
>  │ 
>  │ ** Sub-Topic 1
>  │ 
>  │ ** Sub-Topic 2
>  │ 
>  │ *** Additional entry 
>  │ 
>  │ ** QUOTE Another Sub-Topic
>  │ 
>  │    Some other text.
>  ╰────
> 
>  Its internal structure could be summarized as:
> 
>  ╭────
>  │ (document
>  │  (section)
>  │  (headline
>  │   (section)
>  │   (headline)
>  │   (headline
>  │    (headline))
>  │   (headline
>  │    (quote-section))))
>  ╰────
> 
> 
> 2 Affiliated Keywords
> ═════════════════════
> 
>  With the exception of [inlinetasks], [items], [planning], [clocks],
>  [node properties] and [table rows], every other element type can be
>  assigned attributes.
> 
>  This is done by adding specific keywords, named “affiliated keywords”,
>  just above the element considered, no blank line allowed.
> 
>  Affiliated keywords are built upon one of the following patterns:
>  “#+KEY: VALUE”, “#+KEY[OPTIONAL]: VALUE” or “#+ATTR_BACKEND: VALUE”.
> 
>  KEY is either “CAPTION”, “HEADER”, “NAME”, “PLOT” or “RESULTS” string.
> 
>  BACKEND is a string constituted of alpha-numeric characters, hyphens
>  or underscores.
> 
>  OPTIONAL and VALUE can contain any character but a new line.  Only
>  keywords in `org-element-dual-keywords' can have an optional value.
> 
>  An affiliated keyword can appear on multiple lines if KEY belongs to
>  `org-element-multiple-keywords' or if its pattern is “#+ATTR_BACKEND:
>  VALUE”.
> 
>  Affiliated keywords whose KEY belong to `org-element-parsed-keywords'
>  can contain objects in their value and their optional value, if
>  applicable.
> 
> 
>  [inlinetasks] See section 3.5
> 
>  [items] See section 3.6
> 
>  [planning] See section 4.3
> 
>  [clocks] See section 4.3
> 
>  [node properties] See section 4.9
> 
>  [table rows] See section 4.11
> 
> 
> 3 Greater Elements
> ══════════════════
> 
>  Unless specified otherwise, greater elements can contain directly any
>  other element or greater element excepted:
> 
>  • elements of their own type,
>  • [node properties], which can only be found in [property drawers],
>  • [items], which can only be found in [plain lists].
> 
> 
>  [node properties] See section 4.9
> 
>  [property drawers] See section 3.2
> 
>  [items] See section 3.6
> 
>  [plain lists] See section 3.6
> 
> 
> 3.1 Greater Blocks
> ──────────────────
> 
>  Greater blocks consist in the following pattern:
> 
>  ╭────
>  │ #+BEGIN_NAME PARAMETERS
>  │ CONTENTS
>  │ #+END_NAME
>  ╰────
> 
>  NAME can contain any non-whitespace character.
> 
>  PARAMETERS can contain any character, and can be omitted.
> 
>  If NAME is “CENTER”, it will be a “center block”.  If it is “QUOTE”,
>  it will be a “quote block”.
> 
>  If the block is neither a center block, a quote block or a [block
>  element], it will be a “special block”.
> 
>  CONTENTS can contain any element, but another greater block of the
>  same type.
> 
> 
>  [block element] See section 4.2
> 
> 
> 3.2 Drawers and Property Drawers
> ────────────────────────────────
> 
>  Pattern for drawers is:
> 
>  ╭────
>  │ :NAME:
>  │ CONTENTS
>  │ :END:
>  ╰────
> 
>  NAME has to either be “PROPERTIES” or belong to `org-drawers' list.
> 
>  If NAME is “PROPERTIES”, the drawer will become a “property drawer”.
> 
>  In a property drawers, CONTENTS can only contain [node property]
>  elements.  Otherwise it can contain any element but another drawer or
>  property drawer.
> 
>                                  ―――――
> 
>  It would be nice if users hadn’t to register drawers names before
>  using them in `org-drawers' (or through the `#+DRAWERS:' keyword).
>  Anything starting with `^[ \t]*:\w+:[ \t]$' and ending with
>  `^[ \t]*:END:[ \t]$' could be considered as a drawer.  — ngz
> 
> 
>  [node property] See section 4.9
> 
> 
> 3.3 Dynamic Blocks
> ──────────────────
> 
>  Pattern for dynamic blocks is:
> 
>  ╭────
>  │ #+BEGIN: NAME PARAMETERS
>  │ CONTENTS
>  │ #+END:
>  ╰────
> 
>  NAME cannot contain any whitespace character.
> 
>  PARAMETERS can contain any character and can be omitted.
> 
> 
> 3.4 Footnote Definitions
> ────────────────────────
> 
>  Pattern for footnote definitions is:
> 
>  ╭────
>  │ [LABEL] CONTENTS
>  ╰────
> 
>  It must start at column 0.
> 
>  LABEL is either a number or follows the pattern “fn:WORD”, where word
>  can contain any word-constituent character, hyphens and underscore
>  characters.
> 
>  CONTENTS can contain any element excepted another footnote definition.
>  It ends at the next footnote definition, the next headline, two
>  consecutive empty lines or the end of buffer.
> 
> 
> 3.5 Inlinetasks
> ───────────────
> 
>  Inlinetasks are defined by `org-inlinetask-min-level' contiguous
>  asterisk characters starting at column 0, followed by a whitespace
>  character.
> 
>  Optionally, inlinetasks can be ended with a string constituted of
>  `org-inlinetask-min-level' contiguous characters starting at column 0,
>  followed by a space and the “END” string.
> 
>  Inlinetasks are recognized only after `org-inlinetask' library is
>  loaded.
> 
> 
> 3.6 Plain Lists and Items
> ─────────────────────────
> 
>  Items are defined by a line starting with the following pattern:
>  “BULLET COUNTER-SET CHECK-BOX TAG”, in which only BULLET is mandatory.
> 
>  BULLET is either an asterisk, a hyphen, a plus sign character or
>  follows either the pattern “COUNTER.” or “COUNTER)".  In any case,
>  BULLET is follwed by a whitespace character or line ending.
> 
>  COUNTER can be a number or a single letter.
> 
>  COUNTER-SET follows the pattern [@COUNTER].
> 
>  CHECK-BOX is either a single whitespace character, a “X” character or
>  a hyphen, enclosed within square brackets.
> 
>  TAG follows “TAG-TEXT ::” pattern, where TAG-TEXT can contain any
>  character but a new line.
> 
>  An item ends before the next item, the first line less or equally
>  indented than its starting line, or two consecutive empty lines.
>  Indentation of lines within other greater elements do not count,
>  neither do inlinetasks boundaries.
> 
>  A plain list is a set of consecutive items of the same indentation.
>  It can only directly contain items.
> 
>  If first item in a plain list has a counter in its bullet, the plain
>  list will be an “ordered plain-list”.  If it contains a tag, it will
>  be a “descriptive list”.  Otherwise, it will be an “unordered list”.
>  List types are mutually exclusive.
> 
>  For example, consider the following excerpt of an Org document:
> 
>  ╭────
>  │ 1. item 1
>  │ 2. [X] item 2
>  │    - some tag :: item 2.1
>  ╰────
> 
>  Its internal structure is as follows:
> 
>  ╭────
>  │ (ordered-plain-list
>  │  (item)
>  │  (item
>  │   (descriptive-plain-list
>  │    (item))))
>  ╰────
> 
> 
> 3.7 Tables
> ──────────
> 
>  Tables start at lines beginning with either a vertical bar or the “+-”
>  string followed by plus or minus signs only, assuming they are not
>  preceded with lines of the same type.  These lines can be indented.
> 
>  A table starting with a vertical bar has “org” type.  Otherwise it has
>  “table.el” type.
> 
>  Org tables end at the first line not starting with a vertical bar.
>  Table.el tables end at the first line not starting with either
>  a vertical line or a plus sign.  Such lines can be indented.
> 
>  An org table can only contain table rows.  A table.el table does not
>  contain anything.
> 
> 
> 4 Elements
> ══════════
> 
>  Elements cannot contain any other element.
> 
>  Only [keywords] whose name belongs to
>  `org-element-document-properties', [verse blocks] , [paragraphs] and
>  [table rows] can contain objects.
> 
> 
>  [keywords] See section 4.7
> 
>  [verse blocks] See section 4.2
> 
>  [paragraphs] See section 4.10
> 
>  [table rows] See section 4.11
> 
> 
> 4.1 Babel Call
> ──────────────
> 
>  Pattern for babel calls is:
> 
>  ╭────
>  │ #+CALL: VALUE
>  ╰────
> 
>  VALUE is optional.  It can contain any character but a new line.
> 
> 
> 4.2 Blocks
> ──────────
> 
>  Like [greater blocks], pattern for blocks is:
> 
>  ╭────
>  │ #+BEGIN_NAME DATA
>  │ CONTENTS
>  │ #+END_NAME
>  ╰────
> 
>  NAME cannot contain any whitespace character.
> 
>  If NAME is “COMMENT”, it will be a “comment block”.  If it is
>  “EXAMPLE”, it will be an “example block”.  If it is “SRC”, it will be
>  a “source block”.  If it is “VERSE”, it will be a “verse block”.
> 
>  If NAME is a string matching the name of any export back-end loaded,
>  the block will be an “export block”.
> 
>  DATA can contain any character but a new line.  It can be ommitted,
>  unless the block is a “source block”.  In this case, it must follow
>  the pattern “LANGUAGE SWITCHES ARGUMENTS”, where SWITCHES and
>  ARGUMENTS are optional.
> 
>  LANGUAGE cannot contain any whitespace character.
> 
>  SWITCHES is made of any number of “SWITCH” patterns, separated by
>  blank lines.
> 
>  A SWITCH pattern is either “-l “FORMAT"", where FORMAT can contain any
>  character but a double quote and a new line, “-S” or “+S”, where
>  S stands for a single letter.
> 
>  ARGUMENTS can contain any character but a new line.
> 
>  CONTENTS can contain any character, including new lines.  Though it
>  will only contain Org objects if the block is a verse block.
>  Otherwise, contents will not be parsed.
> 
> 
>  [greater blocks] See section 3.1
> 
> 
> 4.3 Clock, Diary Sexp and Planning
> ──────────────────────────────────
> 
>  A clock follows the pattern:
> 
>  ╭────
>  │ CLOCK: TIMESTAMP DURATION
>  ╰────
> 
>  Both TIMESTAMP and DURATION are optional.
> 
>  TIMESTAMP is a [timestamp] object.
> 
>  DURATION follows the pattern:
> 
>  ╭────
>  │ => HH:MM
>  ╰────
> 
>  HH is a number containing any number of digits.  MM is a two digit
>  numbers.
> 
>  A diary sexp is a line starting at column 0 with “%%(" string.  It can
>  then contain any character besides a new line.
> 
>  A planning is a line filled with more at most three INFO parts, where
>  each INFO part follows the pattern:
> 
>  ╭────
>  │ KEYWORD: TIMESTAMP
>  ╰────
> 
>  KEYWORD is a string among `org-deadline-string',
>  `org-scheduled-string' and `org-closed-string'.  TIMESTAMP is is
>  a [timestamp] object.
> 
>  Even though a planning element can exist anywhere in a section or
>  a greater element, it will only affect the headline containing the
>  section if it is put on the line following that headline.
> 
> 
>  [timestamp] See section 5.12
> 
> 
> 4.4 Comments
> ────────────
> 
>  A “comment line” starts with a hash signe and a whitespace character
>  or an end of line.
> 
>  Comments can contain any number of consecutive comment lines.
> 
> 
> 4.5 Fixed Width Areas
> ─────────────────────
> 
>  A “fixed-width line” start with a colon character and a whitespace or
>  an end of line.
> 
>  Fixed width areas can contain any number of consecutive fixed-width
>  lines.
> 
> 
> 4.6 Horizontal Rules
> ────────────────────
> 
>  A horizontal rule is a line made of at least 5 consecutive hyphens.
>  It can be indented.
> 
> 
> 4.7 Keywords
> ────────────
> 
>  Keywords follow the syntax:
> 
>  ╭────
>  │ #+KEY: VALUE
>  ╰────
> 
>  KEY can contain any non-whitespace character, but it cannot be equal
>  to “CALL” or any affiliated keyword.
> 
>  VALUE can contain any character excepted a new line.
> 
>  If KEY belongs to `org-element-document-properties', VALUE can contain
>  objects.
> 
> 
> 4.8 LaTeX Environments
> ──────────────────────
> 
>  Pattern for LaTeX environments is:
> 
>  ╭────
>  │ \begin{NAME}
>  │ CONTENTS
>  │ \end{NAME}
>  ╰────
> 
>  NAME is constituted of alpha-numeric characters and may end with an
>  asterisk.
> 
>  CONTENTS can contain anything but the “\end{NAME}” string.
> 
> 
> 4.9 Node Properties
> ───────────────────
> 
>  Patter for node properties is:
> 
>  ╭────
>  │ :PROPERTY: VALUE
>  ╰────
> 
>  PROPERTY can contain any non-whitespace character.  VALUE can contain
>  any character but a new line.
> 
>  Node properties can only exist in a [property drawers].
> 
> 
>  [property drawers] See section 3.2
> 
> 
> 4.10 Paragraphs
> ───────────────
> 
>  Paragraphs are the default element, which means that any unrecognized
>  context is a paragraph.
> 
>  Empty lines and other elements end paragraphs.
> 
>  Paragraphs can contain every type of object.
> 
> 
> 4.11 Table Rows
> ───────────────
> 
>  A table rows is either constituted of a vertical bar and any number of
>  [table cells] or a vertical bar followed by a hyphen.
> 
>  In the first case the table row has the “standard” type.  In the
>  second case, it has the “rule” type.
> 
>  Table rows can only exist in [tables].
> 
> 
>  [table cells] See section 5.11
> 
>  [tables] See section 3.7
> 
> 
> 5 Objects
> ═════════
> 
>  Objects can only be found in the following locations:
> 
>  • [affiliated keywords] defined in `org-element-parsed-keywords',
>  • [document properties],
>  • [headline] titles,
>  • [inlinetask] titles,
>  • [item] tags,
>  • [paragraphs],
>  • [table cells],
>  • [table rows], which can only contain table cell objects,
>  • [verse blocks].
> 
>  Most objects cannot contain objects.  Those which can will be
>  specified.
> 
> 
>  [affiliated keywords] See section 2
> 
>  [document properties] See section 4.7
> 
>  [headline] See section 1
> 
>  [inlinetask] See section 3.5
> 
>  [item] See section 3.6
> 
>  [paragraphs] See section 4.10
> 
>  [table cells] See section 5.11
> 
>  [table rows] See section 4.11
> 
>  [verse blocks] See section 4.2
> 
> 
> 5.1 Entities and LaTeX Fragments
> ────────────────────────────────
> 
>  An entity follows the pattern:
> 
>  ╭────
>  │ \NAME POST
>  ╰────
> 
>  where NAME has a valid association in either `org-entities' or
>  `org-entities-user'.
> 
>  POST is the end of line, "{}" string, or a non-alphabetical character.
>  It isn’t separated from NAME by a whitespace character.
> 
>  A LaTeX fragment can follow multiple patterns:
> 
>  ╭────
>  │ \NAME POST
>  │ \(CONTENTS\)
>  │ \[CONTENTS\]
>  │ $$CONTENTS$$
>  │ PRE$CHAR$POST
>  │ PRE$BORDER1 BODY BORDER2$
>  ╰────
> 
>  NAME contains alphabetical characters only and must not have an
>  association in either `org-entities' or `org-entities-user'.
> 
>  POST is the same as for entities.
> 
>  CONTENTS can contain any character but cannot contain “\)" in the
>  second template or “\]" in the third one.
> 
>  PRE is either the beginning of line or a character different from `$'.
> 
>  CHAR is a non-whitespace character different from `.', ~,~, `?', `;',
>  ~’~ or a double quote.
> 
>  POST is any of `-', `.', ~,~, `?', `;', `:', ~’~, a double quote,
>  a whitespace character and the end of line.
> 
>  BORDER1 is a non-whitespace character different from `.', `;', `.'
>  and `$'.
> 
>  BODY can contain any character excepted `$', and may not span over
>  more than 3 lines.
> 
>  BORDER2 is any non-whitespace character different from ~,~, `.' and
>  `$'.
> 
>                                  ―――――
> 
>        It would introduce incompatibilities with previous Org
>        versions, but support for “$…$” (and for symmetry,
>        `$$...$$') constructs ought to be removed.
> 
>        They are slow to parse, fragile, redundant, imply false
>        positives and do not look good in LaTeX output anyway.
>        Even the LaTeX community suggests to use `\(...\)' over
>        `$...$'.  — ngz
> 
> 
> 5.2 Export Snippets
> ───────────────────
> 
>  Patter for export snippets is:
> 
>  ╭────
>  │ @@NAME:VALUE@@
>  ╰────
> 
>  NAME can contain any alpha-numeric character and hyphens.
> 
>  VALUE can contain anything but “@@” string.
> 
> 
> 5.3 Footnote References
> ───────────────────────
> 
>  There are four patterns for footnote references:
> 
>  ╭────
>  │ [MARK]
>  │ [fn:LABEL]
>  │ [fn:LABEL:DEFINITION]
>  │ [fn::DEFINITION]
>  ╰────
> 
>  MARK is a number.
> 
>  LABEL can contain any word constituent character, hyphens and
>  underscores.
> 
>  DEFINITION can contain any character.  Though opening and closing
>  square brackets must be balanced in it.  It can contain any object
>  encountered in a paragraph, even other footnote references.
> 
>  If the reference follows the third pattern, it is called an “inline
>  footnote”.  If it follows the fourth one, i.e. if LABEL is omitted, it
>  is an “anonymous footnote”.
> 
> 
> 5.4 Inline Babel Calls and Source Blocks
> ────────────────────────────────────────
> 
>  Inline Babel calls follow any of the following patterns:
> 
>  ╭────
>  │ call_NAME(ARGUMENTS)
>  │ call_NAME[HEADER](ARGUMENTS)[HEADER]
>  ╰────
> 
>  NAME can contain any character besides `(', `)' and “\n”.
> 
>  HEADER can contain any character besides `]' and “\n”.
> 
>  ARGUMENTS can contain any character besides `)' and “\n”.
> 
>  Inline source blocks follow any of the following patterns:
> 
>  ╭────
>  │ src_LANG{BODY}
>  │ src_LANG[OPTIONS]{BODY}
>  ╰────
> 
>  LANG can contain any non-whitespace character.
> 
>  OPTIONS and BODY can contain any character but “\n”.
> 
> 
> 5.5 Line Breaks
> ───────────────
> 
>  A line break consists in “\\SPACE” pattern at the end of an otherwise
>  non-empty line.
> 
>  SPACE can contain any number of tabs and spaces, including 0.
> 
> 
> 5.6 Links
> ─────────
> 
>  There are 4 major types of links:
> 
>  ╭────
>  │ RADIO                     ("radio" link)
>  │ <PROTOCOL:PATH>           ("angle" link)
>  │ PRE PROTOCOL:PATH2 POST   ("plain" link)
>  │ [[PATH3]DESCRIPTION]      ("regular" link)
>  ╰────
> 
>  RADIO is a string matched by some [radio target].  It can contain
>  [entities], [latex fragments], [subscript] and [superscript] only.
> 
>  PROTOCOL is a string among `org-link-types'.
> 
>  PATH can contain any character but `]', `<', `>' and `\n'.
> 
>  PRE and POST are non word constituent.  They can be, respectively, the
>  beginning or the end of a line.
> 
>  PATH2 can contain any non-whitespace character excepted `(', `)', `<'
>  and `>'.  It must end with a word-constituent character, or any
>  non-whitespace non-punctuation character followed by `/'.
> 
>  DESCRIPTION must be enclosed within square brackets.  It can contain
>  any character but square brackets.  Object-wise, it can contain any
>  object found in a paragraph excepted a [footnote reference], a [radio
>  target] and a [line break].  It cannot contain another link either,
>  unless it is a plain link.
> 
>  DESCRIPTION is optional.
> 
>  PATH3 is built according to the following patterns:
> 
>  ╭────
>  │ FILENAME           ("file" type)
>  │ PROTOCOL:PATH4     ("PROTOCOL" type)
>  │ id:ID              ("id" type)
>  │ #CUSTOM-ID         ("custom-id" type)
>  │ (CODEREF)          ("coderef" type)
>  │ FUZZY              ("fuzzy" type)
>  ╰────
> 
>  FILENAME is a file name, either absolute or relative.
> 
>  PATH4 can contain any character besides square brackets.
> 
>  ID is constituted of hexadecimal numbers separated with hyphens.
> 
>  PATH4, CUSTOM-ID, CODEREF and FUZZY can contain any character besides
>  square brackets.
> 
>                                  ―――――
> 
>        I suggest to remove angle links.  If one needs spaces in
>        PATH, she can use standard link syntax instead.
> 
>        I also suggest to remove `org-link-types' dependency in
>        PROTOCOL and match `[a-zA-Z]' instead, for portability.  —
>        ngz
> 
> 
>  [radio target] See section 5.8
> 
>  [entities] See section 5.1
> 
>  [latex fragments] See section 5.1
> 
>  [subscript] See section 5.10
> 
>  [superscript] See section 5.10
> 
>  [footnote reference] See section 5.3
> 
>  [line break] See section 5.5
> 
> 
> 5.7 Macros
> ──────────
> 
>  Macros follow the pattern:
> 
>  ╭────
>  │ {{{NAME(ARGUMENTS)}}}
>  ╰────
> 
>  NAME must start with a letter and can be followed by any number of
>  alpha-numeric characters, hyphens and underscores.
> 
>  ARGUMENTS can contain anything but "}}}" string.  Values within
>  ARGUMENTS are separated by commas.  Non-separating commas have to be
>  escaped with a backslash character.
> 
> 
> 5.8 Targets and Radio Targets
> ─────────────────────────────
> 
>  Radio targets follow the pattern:
> 
>  ╭────
>  │ <<<CONTENTS>>>
>  ╰────
> 
>  CONTENTS can be any character besides `<', `>' and “\n”.  As far as
>  objects go, it can contain [entities], [latex fragments], [subscript]
>  and [superscript] only.
> 
>  Targets follow the pattern:
> 
>  ╭────
>  │ <<TARGET>>
>  ╰────
> 
>  TARGET can contain any character besides `<', `>' and “\n”.  It cannot
>  contain any object.
> 
> 
>  [entities] See section 5.1
> 
>  [latex fragments] See section 5.1
> 
>  [subscript] See section 5.10
> 
>  [superscript] See section 5.10
> 
> 
> 5.9 Statistics Cookies
> ──────────────────────
> 
>  Statistics cookies follow either pattern:
> 
>  ╭────
>  │ [PERCENT%]
>  │ [NUM1/NUM2]
>  ╰────
> 
>  PERCENT, NUM1 and NUM2 are numbers or the empty string.
> 
> 
> 5.10 Subscript and Superscript
> ──────────────────────────────
> 
>  Pattern for subscript is:
> 
>  ╭────
>  │ CHAR_SCRIPT
>  ╰────
> 
>  Pattern for superscript is:
> 
>  ╭────
>  │ CHAR^SCRIPT
>  ╰────
> 
>  CHAR is any non-whitespace character.
> 
>  SCRIPT can be `*', a string made of word-constituent characters maybe
>  preceded by a plus or a minus sign, an expression enclosed in
>  parenthesis (resp. curly brackets) containing balanced parenthesis
>  (resp. curly brackets).
> 
> 
> 5.11 Table Cells
> ────────────────
> 
>  Table cells follow the pattern:
> 
>  ╭────
>  │ CONTENTS|
>  ╰────
> 
>  CONTENTS can contain any character excepted a vertical bar.
> 
> 
> 5.12 Timestamps
> ───────────────
> 
>  There are seven possible patterns for timestamps:
> 
>  ╭────
>  │ <%%(SEXP)>                                     (diary)
>  │ <DATE TIME REPEATER>                         (active)
>  │ [DATE TIME REPEATER]                         (inactive)
>  │ <DATE TIME REPEATER>--<DATE TIME REPEATER>   (active range)
>  │ <DATE TIME-TIME REPEATER>                    (active range)
>  │ [DATE TIME REPEATER]--[DATE TIME REPEATER]   (inactive range)
>  │ [DATE TIME-TIME REPEATER]                    (inactive range)
>  ╰────
> 
>  SEXP can contain any character excepted `>' and `\n'.
> 
>  DATE follows the pattern:
> 
>  ╭────
>  │ YYYY-MM-DD DAYNAME
>  ╰────
> 
>  Y, M and D are digits.  DAYNAME can contain any non
>  whitespace-character besides `+', `-', `]', `>', a digit or `\n'.
> 
>  TIME follows the pattern =H:MM~.  H can be one or two digit long and
>  can start with 0.
> 
>  REPEATER follows the patter:
> 
>  ╭────
>  │ MARK VALUE UNIT
>  ╰────
> 
>  MARK is `+' (cumulate type), `++' (catch-up type) or `.+' (restart
>  type).
> 
>  VALUE is a number.
> 
>  UNIT is a character among `h' (hour), `d' (day), `w' (week), `m'
>  (month), `y' (year).
> 
>  MARK, VALUE and UNIT are not separated by whitespace characters.
> 
> 
> 5.13 Text Markup
> ────────────────
> 
>  Text markup follows the pattern:
> 
>  ╭────
>  │ PRE MARKER CONTENTS MARKER POST
>  ╰────
> 
>  PRE is a whitespace character, `(', `{' ~’~ or a double quote.  It can
>  also be a beginning of line.
> 
>  MARKER is a character among `*' (bold), `=' (verbatim), `/' (italic),
>  `+' (strike-through), `_' (underline), `~' (code).
> 
>  CONTENTS is a string following the pattern:
> 
>  ╭────
>  │ BORDER BODY BORDER
>  ╰────
> 
>  BORDER can be any non-whitespace character excepted ~,~, ~’~ or
>  a double quote.
> 
>  BODY can contain contain any character but may not span over more than
>  3 lines.
> 
>  BORDER and BODY are not separated by whitespaces.
> 
>  CONTENTS can contain any object encountered in a paragraph when markup
>  is “bold”, “italic”, “strike-through” or “underline”.
> 
>  POST is a whitespace character, `-', `.', ~,~, `:', `!', `?', ~’~,
>  `)', `}' or a double quote.  It can also be an end of line.
> 
>  PRE, MARKER, CONTENTS, MARKER and POST are not separated by whitespace
>  characters.
> 
>                                  ―――――
> 
>        All of this is wrong if `org-emphasis-regexp-components'
>        or `org-emphasis-alist' are modified.
> 
>        This should really be simplified and made persistent
>        (i.e. no defcustom allowed).  Otherwise, portability and
>        parsing are jokes.
> 
>        Also, CONTENTS should be anything within code and verbatim
>        emphasis, by definition.  — ngz
> 
> 
> 
> Footnotes
> ─────────
> 
> [1] In particular, the parser requires stars at column 0 to be quoted
> by a comma when they do not define a headline.
> 
> [2] It also means that only headlines and sections can be recognized
> just by looking at the beginning of the line.
> 
> As a consequence, using `org-element-at-point' or
> `org-element-context' will move up to the parent headline, and parse
> top-down from there until context around is found.
> 
> 
> 
> Regards,
> 
> -- 
> Nicolas Goaziou
> 

Re: [RFC] Org syntax (draft)

[Achim Gratz]

Nicolas Goaziou writes:
> As discussed a few days ago, here is a document describing the complete
> Org syntax as read by the parser. I also added some comments. I am going
> to put the Org file on Worg, so anyone can update it and fix mistakes.

Wonderful.  This will be really useful!


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Waldorf MIDI Implementation & additional documentation:
http://Synth.Stromeko.net/Downloads.html#WaldorfDocs

Re: [RFC] Org syntax (draft)

[Bastien]

Nicolas Goaziou <n.goaziou@gmail.com> writes:

> As discussed a few days ago, here is a document describing the complete
> Org syntax as read by the parser. I also added some comments. I am going
> to put the Org file on Worg, so anyone can update it and fix mistakes.

Thanks Nicolas -- yep, that's really *great*!

-- 
 Bastien

was: [RFC] Org syntax (draft)

[Andreas Röhler]

[ ... ]
>
>    ╭────
>    │ STARS KEYWORD PRIORITY TITLE TAGS
>    ╰────
>

Hi,

my thanks too.

BTW does that mean, stars are hard-coded in a way, they can't be replaced by another car?
If yes, what about to keep the core more abstract?

Best,

Andreas

Re: (no subject)

[Bastien]

Hi Andreas,

Andreas Röhler <andreas.roehler@easy-emacs.de> writes:

> BTW does that mean, stars are hard-coded in a way, they can't be
> replaced by another car?

Yes, this is the case.

> If yes, what about to keep the core more abstract?

This is a hard problem and I personally don't think it's worth
tackling it.  But I don't want to decourage anyone, of course :)

-- 
 Bastien

Re: (no subject)

[Andreas Röhler]

Am 08.03.2013 11:46, schrieb Bastien:
> Hi Andreas,
>
> Andreas Röhler <andreas.roehler@easy-emacs.de> writes:
>
>> BTW does that mean, stars are hard-coded in a way, they can't be
>> replaced by another car?
>
> Yes, this is the case.
>
>> If yes, what about to keep the core more abstract?
>
> This is a hard problem

Hi Bastien,

can it be more difficult than related use of comment-start?

[ ... ]

Cheers,

Andreas

Re: (no subject)

[Bastien]

Hi Andreas,

Andreas Röhler <andreas.roehler@easy-emacs.de> writes:

> can it be more difficult than related use of comment-start?

Yes -- check this FAQ: 
http://orgmode.org/worg/org-faq.html#sec-8-12

HTH,

-- 
 Bastien

Re: (no subject)

[Andreas Röhler]

Am 08.03.2013 12:05, schrieb Bastien:
> Hi Andreas,
>
> Andreas Röhler <andreas.roehler@easy-emacs.de> writes:
>
>> can it be more difficult than related use of comment-start?
>
> Yes -- check this FAQ:
> http://orgmode.org/worg/org-faq.html#sec-8-12
>
> HTH,
>

Thanks, still non-believing :)

(defcustom outline-regexp "[*\^L]+"
   "Regular expression to match the beginning of a heading.
Any line whose beginning matches this regexp is considered to start a heading.
Note that Outline mode only checks this regexp at the start of a line,
so the regexp need not (and usually does not) start with `^'.
The recommended way to set this is with a Local Variables: list
in the file it applies to.  See also `outline-heading-end-regexp'."
   :type 'regexp
   :group 'outlines)

IMHO remains to make that buffer-local and use outline-regexp from inside org-mode.

Andreas

Re: (no subject)

[Bastien]

Andreas Röhler <andreas.roehler@easy-emacs.de> writes:

> IMHO remains to make that buffer-local and use outline-regexp from
> inside org-mode.

Have a go and let us know :)

-- 
 Bastien

Re: (no subject)

[Andreas Röhler]

[-- Attachment #1: Type: text/plain, Size: 265 bytes --]

Am 08.03.2013 12:23, schrieb Bastien:
> Andreas Röhler <andreas.roehler@easy-emacs.de> writes:
>
>> IMHO remains to make that buffer-local and use outline-regexp from
>> inside org-mode.
>
> Have a go and let us know :)
>

That should work already - to start with.

[-- Attachment #2: org-heading-char.patch --]
[-- Type: text/x-patch, Size: 1194 bytes --]

commit 8cf6bc6faeb2a3b3fec0780e56a04ef0e13c3c62
Author: Andreas Roehler <andreas.roehler@online.de>
Date:   Fri Mar 8 13:58:13 2013 +0100

    Enable different heading chars, not just `*'
    
    Use variable `org-heading-char', customizable later on,
    instead of hard-coded `*'
    
    TINYCHANGE

diff --git a/lisp/org.el b/lisp/org.el
index 811506a..2db63ae 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -90,13 +90,17 @@
 ;; `org-outline-regexp'.  The only function still directly relying on
 ;; `outline-regexp' is `org-overview' so that `org-cycle' can do its
 ;; job when `orgstruct-mode' is active.
+(defvar org-heading-char "*")
+
 (defvar org-outline-regexp "\\*+ "
   "Regexp to match Org headlines.")
+(setq org-outline-regexp (concat (regexp-quote org-heading-char) "+ "))
 
 (defvar org-outline-regexp-bol "^\\*+ "
   "Regexp to match Org headlines.
 This is similar to `org-outline-regexp' but additionally makes
 sure that we are at the beginning of the line.")
+(setq org-outline-regexp-bol (concat "^" (regexp-quote org-heading-char) "+ "))
 
 (defvar org-heading-regexp "^\\(\\*+\\)\\(?: +\\(.*?\\)\\)?[ \t]*$"
   "Matches a headline, putting stars and text into groups.

Re: (no subject)

[Bastien]

Hi Andreas,

Andreas Röhler <andreas.roehler@easy-emacs.de> writes:

> Am 08.03.2013 12:23, schrieb Bastien:
>> Andreas Röhler <andreas.roehler@easy-emacs.de> writes:
>>
>>> IMHO remains to make that buffer-local and use outline-regexp from
>>> inside org-mode.
>>
>> Have a go and let us know :)
>
> That should work already - to start with.

Sorry, I should not let you go into that direction without a warning
that I won't commit anything related to this myself.

Btw, did you read the thread as the FAQ suggests?

One reason why the stars are hardcoded (like the comment characters
and other syntactic elements) is that Org files are used outside Emacs
and allowing users to use something else here is calling for trouble.

So -- just to let you know that it's fine playing with the idea and
providing proof-of-concept patches, but there are many reasons why
this cannot be considered.

Also, we are trying to focus on 8.0 right now... help welcome!

-- 
 Bastien

Re: (no subject)

[Andreas Röhler]

Am 08.03.2013 14:12, schrieb Bastien:
> Hi Andreas,
>
> Andreas Röhler <andreas.roehler@easy-emacs.de> writes:
>
>> Am 08.03.2013 12:23, schrieb Bastien:
>>> Andreas Röhler <andreas.roehler@easy-emacs.de> writes:
>>>
>>>> IMHO remains to make that buffer-local and use outline-regexp from
>>>> inside org-mode.
>>>
>>> Have a go and let us know :)
>>
>> That should work already - to start with.
>
> Sorry, I should not let you go into that direction without a warning
> that I won't commit anything related to this myself.
>

Hi Bastien,

don't worry, didn't expect that for now or soon.


> Btw, did you read the thread as the FAQ suggests?

Nothing different resp. new so far. Let's stress that solution in mind also would be quite simple.
See the example patch.
No-one would be compelled to change the default BTW, it would just open the possibility.

>
> One reason why the stars are hardcoded (like the comment characters
> and other syntactic elements) is that Org files are used outside Emacs
> and allowing users to use something else here is calling for trouble.
>

Hmm, AFAIS trouble might occur only if someone tries to load a non-default --i.e. not-starred--
org-file while the default is active.

But even then it's quite easy to write a guess, which might start if org-mode didn't encounter the
stars where expected.

[ ... ]

Re: (no subject)

[Bastien]

Hi Andreas,

Andreas Röhler <andreas.roehler@easy-emacs.de> writes:

> Hmm, AFAIS trouble might occur only if someone tries to load a
> non-default --i.e. not-starred-- org-file while the default is
> active.

... or if someone shares a file online using non-star character
as the prefix for headlines: this file won't be processed by
Org tools like org-ruby and the like.

> But even then it's quite easy to write a guess, which might start if
> org-mode didn't encounter the stars where expected.

Org files are not just for Emacs, that's were the problem lies...

-- 
 Bastien

Re: (no subject)

[T.F. Torrey]

Hello,

Bastien <bzg@altern.org> writes:

> Hi Andreas,
>
> Andreas Röhler <andreas.roehler@easy-emacs.de> writes:
>
>> Hmm, AFAIS trouble might occur only if someone tries to load a
>> non-default --i.e. not-starred-- org-file while the default is
>> active.
>
> ... or if someone shares a file online using non-star character
> as the prefix for headlines: this file won't be processed by
> Org tools like org-ruby and the like.
>
>> But even then it's quite easy to write a guess, which might start if
>> org-mode didn't encounter the stars where expected.
>
> Org files are not just for Emacs, that's were the problem lies...

I don't understand this heavy-handed approach.

Plain text is great because I can do whatever I want.  What I come up
with might not work correctly in other tools (or anything at all), but I
have the freedom to do interesting things, and to have my files look
just the way I want them to.

Emacs is great because it allows me the freedom of near-infinite
customization.  It has sensible defaults, but it allows me to break
things however I want.

Org, on the other hand, seems to be moving away from that in many ways.
Headlines must start with stars because I might someday post something
on the web and it wouldn't work for someone else?  Other tools might not
recognize my file correctly?  A developer of some other tool might
someday have a problem?  These are not good reasons for limiting what I
can do with my own Org files.

I don't need or want supervision in how I create my files.  I want
freedom.  If I wanted supervision, I wouldn't be using Emacs.  Have you
seen the lisp posted to the web?  Somehow, Emacs and I survive that.

Org started as a great tool that let me do cool things with my text
files.  I don't want to see it change to a rigid format for me to force
my files into, where my only options are conform or leave.

Org should err on the side of user freedom.

IMHO,
Terry
-- 
T.F. Torrey

Re: (no subject)

[Nicolas Goaziou]

Hello,

tftorrey@tftorrey.com (T.F. Torrey) writes:

> Hello,
>
> Bastien <bzg@altern.org> writes:
>
>> Hi Andreas,
>>
>> Andreas Röhler <andreas.roehler@easy-emacs.de> writes:
>>
>>> Hmm, AFAIS trouble might occur only if someone tries to load a
>>> non-default --i.e. not-starred-- org-file while the default is
>>> active.
>>
>> ... or if someone shares a file online using non-star character
>> as the prefix for headlines: this file won't be processed by
>> Org tools like org-ruby and the like.
>>
>>> But even then it's quite easy to write a guess, which might start if
>>> org-mode didn't encounter the stars where expected.
>>
>> Org files are not just for Emacs, that's were the problem lies...
>
> I don't understand this heavy-handed approach.
>
> Plain text is great because I can do whatever I want.  What I come up
> with might not work correctly in other tools (or anything at all), but I
> have the freedom to do interesting things, and to have my files look
> just the way I want them to.
>
> Emacs is great because it allows me the freedom of near-infinite
> customization.  It has sensible defaults, but it allows me to break
> things however I want.
>
> Org, on the other hand, seems to be moving away from that in many ways.
> Headlines must start with stars because I might someday post something
> on the web and it wouldn't work for someone else?  Other tools might not
> recognize my file correctly?  A developer of some other tool might
> someday have a problem?  These are not good reasons for limiting what I
> can do with my own Org files.
>
> I don't need or want supervision in how I create my files.  I want
> freedom.  If I wanted supervision, I wouldn't be using Emacs.  Have you
> seen the lisp posted to the web?  Somehow, Emacs and I survive that.
>
> Org started as a great tool that let me do cool things with my text
> files.  I don't want to see it change to a rigid format for me to force
> my files into, where my only options are conform or leave.
>

I disagree.

Org is a plain text format. Like any format, plain-text or not, it needs
a proper definition. At least, it helps users and developers to agree on
what they are talking about. As for myself, I cannot play any game if
I don't know its rules.

My point of view is the following: Org (as a format) definition
shouldn't depend on Emacs. It should be totally parseable by any
language (which is not the case actually, since syntax relies on
variables defined in Emacs). IOW, we should work to make it a real
plain-text markup format.

> Org should err on the side of user freedom.

You still have the freedom to choose what you write down in Org format.
You have the freedom use, or to not use Org. You have the freedom to
modify Org code to bend it to your will. IMO, freedom is totally
unrelated to this subject.


Regards,

-- 
Nicolas Goaziou

Re: (no subject)

[Suvayu Ali]

Hi,

On Fri, Mar 08, 2013 at 10:19:56PM +0100, Nicolas Goaziou wrote:
> tftorrey@tftorrey.com (T.F. Torrey) writes:
> >
> > Org, on the other hand, seems to be moving away from that in many ways.
> > Headlines must start with stars because I might someday post something
> > on the web and it wouldn't work for someone else?  Other tools might not
> > recognize my file correctly?  A developer of some other tool might
> > someday have a problem?  These are not good reasons for limiting what I
> > can do with my own Org files.

[...]

> My point of view is the following: Org (as a format) definition
> shouldn't depend on Emacs. It should be totally parseable by any
> language (which is not the case actually, since syntax relies on
> variables defined in Emacs). IOW, we should work to make it a real
> plain-text markup format.

This discussion comes up again and again over the years.  Every time the
conclusion has been, there are many who want to use other characters
instead of a star but the rational is always cosmetic.  And in the end
people end up agreeing it is not worth the pain.

To give you an example of breakage, after long whining github started
supporting org file formatting for READMEs via org-ruby (even source
blocks I think!).  Do you expect we can have similar support if
something as fundamental as the headline is not well defined?  Other
possible breakages could be in org-merge-driver, VimOrganizer, org-sync,
some of the collaborative editing projects that have been spawning, and
the list goes on ...

FWIW, assuming that the rational behind this feature request is
cosmetic, there was a proposal to implement this as overlays.  So the
file still uses *-s but a user can customise how they are displayed.  I
think this addresses almost all requests in this category (majority
being "I want really cool UTF characters instead of boring old
asterisk").  If someone is interested in implementing this, I'm pretty
sure the maintainers would give that a serious consideration for
inclusion.  Changing the file format on the other hand is a no go
amongst many of the users (myself included) and developers alike.

Hope my 2¢ shed some light on the implications of a change like this.

Cheers,

-- 
Suvayu

Open source is the future. It sets us free.

interoperability matters Re: (no subject)

[Gregor Zattler]

Hi T.F.,
* T.F. Torrey <tftorrey@tftorrey.com> [08. Mar. 2013]:
> Plain text is great because I can do whatever I want.  What I come up
> with might not work correctly in other tools (or anything at all), but I
> have the freedom to do interesting things, and to have my files look
> just the way I want them to.

One argument against changing the headlne marker was
interoperability: Share org files with others, use other tools to
manipulate org files.  Interoperability is a product of
rules/formats/protocols.

Email is a plain text protocol too, but some headers are required
for interoperability.  It wouldn't make sense for instance for me
to use my freedom to create/change headers arbitrary, "and to
have my emails look just the way I want them to".  E.g. to use
german words Betreff, Datum, An, Von instead of Subject, Date,
To, From, only because I can --> no mail transport agent would
handle this plain text file.

Regards, Gregor

Re: (no subject)

[Bastien]

Hi Terry,

I hear you.  I completely agree that Org should not be less flexible
than it has been so far.  At least not for very good reasons, shared
by both the developers and the users.  IOW: ease of maintainance and
code consistency should not let us introduce rigidity for the users.

Let's focus on the regressions and let's try to fix the ones that we
can fix.

As discussions have shown so far, Nicolas holds the keys when it comes
to honoring Org's consistency regarding its syntax -- the document he
wrote will help us all to speak about the same syntax and rules.  But
as you may have felt, I'm more on the "user conveniency" side, even if
we need to sacrifice some consistency.  There is a balance here, and I
hope we keep a good one.

So as I said: let's focus on what you perceive as regressions wrt what
Org allows.

The subject of this thread does not fall in this category: headlines
have always been starting with stars, there is no regression here.  On
the contrary: a few years ago, we had no answer to this FAQ, now we
can help users with several solution when the problem is aesthetic.

Finally, I agree with Suvayu that the problem *is* mostly aesthetic,
so the solutions we provide are enough (i.e., the FAQ, org-bullets.el
in contrib/.)  The question is rather whether we should have an Org
option in core to allow users to tweak the appearance of the stars:
my answer here is "no", because I don't see why users would stop here.
Once we offer such an option for headlines, why not for comments and
other characters with a syntactic role?  (I replied a question on
stackoverflow on how to use "%" instead of "#" for comments...)

Anyway -- Org still stands on the side of users' freedom, let's
fix the real regressions.

Thanks!

-- 
 Bastien

Re: (no subject)

[T.F. Torrey]

Thank you for your thoughtful reply. -- T.

Bastien <bzg@altern.org> writes:

> Hi Terry,
>
> I hear you.  I completely agree that Org should not be less flexible
> than it has been so far.  At least not for very good reasons, shared
> by both the developers and the users.  IOW: ease of maintainance and
> code consistency should not let us introduce rigidity for the users.
>
> Let's focus on the regressions and let's try to fix the ones that we
> can fix.
>
> As discussions have shown so far, Nicolas holds the keys when it comes
> to honoring Org's consistency regarding its syntax -- the document he
> wrote will help us all to speak about the same syntax and rules.  But
> as you may have felt, I'm more on the "user conveniency" side, even if
> we need to sacrifice some consistency.  There is a balance here, and I
> hope we keep a good one.
>
> So as I said: let's focus on what you perceive as regressions wrt what
> Org allows.
>
> The subject of this thread does not fall in this category: headlines
> have always been starting with stars, there is no regression here.  On
> the contrary: a few years ago, we had no answer to this FAQ, now we
> can help users with several solution when the problem is aesthetic.
>
> Finally, I agree with Suvayu that the problem *is* mostly aesthetic,
> so the solutions we provide are enough (i.e., the FAQ, org-bullets.el
> in contrib/.)  The question is rather whether we should have an Org
> option in core to allow users to tweak the appearance of the stars:
> my answer here is "no", because I don't see why users would stop here.
> Once we offer such an option for headlines, why not for comments and
> other characters with a syntactic role?  (I replied a question on
> stackoverflow on how to use "%" instead of "#" for comments...)
>
> Anyway -- Org still stands on the side of users' freedom, let's
> fix the real regressions.
>
> Thanks!

Re: [RFC] Org syntax (draft)

[François Pinard]

Nicolas Goaziou <n.goaziou@gmail.com> writes:

> As discussed a few days ago, here is a document describing the complete
> Org syntax as read by the parser.

Fantastique! :-)  I'm preciously saving this!

François

Re: [RFC] Org syntax (draft)

[Nicolas Richard]

Nicolas Goaziou <n.goaziou@gmail.com> writes:
> As discussed a few days ago, here is a document describing the complete
> Org syntax as read by the parser. I also added some comments. I am going
> to put the Org file on Worg, so anyone can update it and fix mistakes.

[for the record, the org file mentionned by Nicolas is currently at
<http://orgmode.org/worg/dev/org-syntax.org>]

This looks truly awesome. I give some (naïve) comments below, from my
non-expert point of view.

> The paragraph is the unit of measurement.  An element defines
> syntactical parts that are at the same level as a paragraph, i.e. which
> cannot contain or be included in a paragraph.  An object is a part that
> could be included in an element.  Greater elements are all parts that
> can contain an element.

This is very clear but I'm slightly worried about confusion that might come
from "Greater element" not being an "element", and the word "element"
being a common word :

> Empty lines belong to the largest element ending before them.  For
> example, in a list, empty lines between items belong are part of the
> item before them, but empty lines at the end of a list belong to the
> plain list element.

Is the word "element" (in /largest element ending.../) to be understood
as an "element" from the above definition ? I guess not (this would
require both list items and plain lists to be on the level 'element',
from your example)

> 1 Headlines and Sections
> ════════════════════════
>
>   A headline is defined as:
>
>   ╭────
>   │ STARS KEYWORD PRIORITY TITLE TAGS
>   ╰────
>
>   STARS is a string starting at column 0 and containing at least one
>   asterisk (and up to `org-inlinetask-min-level' if `org-inlinetask'
>   library is loaded).  It’s the sole compulsory part of a headline.

Perhaps it should be mentionned that STARS has to end by a space (see
below). I suggest adding : The number of stars defines the level of the
headline.

>   KEYWORD is a TODO keyword, which have to belong to the list defined in
>   `org-todo-keywords'.  Case is significant.

The option #+TODO: is used also.

>   PRIORITY is a priority cookie, i.e. a single letter preceded by a hash
>   sign # and enclosed within square brackets.  Case is significant.

I suggest dropping "Case is significant" (or maybe give the whole story :
IIRC, it is the ascii code of the given letter that is used as priority)

>   ╭────
>   │ *

I don't see a space character after that one in your email and it
doesn't seem to be recognized as a headline by the exporter (hence my
above suggestion)

>   If the first word appearing in the title is `org-comment-keyword',
>   the

That should be `org-comment-string' I guess.

>   A headline contains directly at most one section, followed by any
>   number of headlines.  Only a section can contain another section.

From what I understand, "A section is delimited by two headlines (and
buffer limits)." [I initially thought it was "by two headlines of the
same level", which it is not from the structure example you give later.]

>   A section contains directly any greater element or element.  Only
>   a headline can contain a section.  As an exception, text before the
>   first headline in the document also belongs to a section.


>   In a quoted headline contains a section, the latter will be considered
>   as a “quote section”.

s/In/If/
unsure: s/quote section/quoted section/ ?

>   As an example, consider the following document:

<snip, useful example>

>   BACKEND is a string constituted of alpha-numeric characters, hyphens
>   or underscores.

I suggest: BACKEND is a string which is an element of (mapcar 'car
org-export-registered-backends).

>   OPTIONAL and VALUE can contain any character but a new line.  Only
>   keywords in `org-element-dual-keywords' can have an optional value.

I guess OPTIONAL cannot contain a closing square bracket ]

>   An affiliated keyword can appear on multiple lines if KEY belongs to
>   `org-element-multiple-keywords' or if its pattern is “#+ATTR_BACKEND:
>   VALUE”.

I suggest s/on multiple lines/more than once/

>   PARAMETERS can contain any character, and can be omitted.

any other than new line, I guess.

>   CONTENTS can contain any element, but another greater block of the
>   same type.

What is the type of a greater block ? the /name/ ?

I did have a quick look at the rest of your mail, and it is very nice to
have all of it written down explicitly, so again a big thanks for all of
this (and the rest of your) work. Unfortunately I don't have much time
right now to read it thoroughtfully, so just one single comment :

>        Even the LaTeX community suggests to use `\(...\)' over
>        `$...$'.  — ngz

AFAIK that's not for technical reasons and also I would be curious to
know who does that in real documents : '$' is so much more convenient.
But one might think of rebinding $ to a command which would insert \( and
\) appropriately within org-mode (see below). (OTOH, there are technical
reasons for avoiding $$ and $$.)

Here some elisp for the above behaviour :
(defun yf/org-electric-dollar nil
"When called once, insert \\(\\) and leave point in between.
When called twice, replace the previously inserted \\(\\) by one $."
  (interactive)
  (if (and (looking-at "\\\\)") (looking-back "\\\\("))
      (progn (delete-char 2)
             (delete-char -2)
             (insert "$"))
    (insert "\\(\\)")
    (backward-char 2)))
(define-key org-mode-map (kbd "$") 'yf/org-electric-dollar)

-- 
N.

Re: [RFC] Org syntax (draft)

[Nicolas Goaziou]

Hello,

"Nicolas Richard" <theonewiththeevillook@yahoo.fr> writes:

> Nicolas Goaziou <n.goaziou@gmail.com> writes:
>> As discussed a few days ago, here is a document describing the complete
>> Org syntax as read by the parser. I also added some comments. I am going
>> to put the Org file on Worg, so anyone can update it and fix mistakes.
>
> [for the record, the org file mentionned by Nicolas is currently at
> <http://orgmode.org/worg/dev/org-syntax.org>]
>
> This looks truly awesome. I give some (naïve) comments below, from my
> non-expert point of view.

Thank you for your comments.

>> The paragraph is the unit of measurement.  An element defines
>> syntactical parts that are at the same level as a paragraph, i.e. which
>> cannot contain or be included in a paragraph.  An object is a part that
>> could be included in an element.  Greater elements are all parts that
>> can contain an element.
>
> This is very clear but I'm slightly worried about confusion that might come
> from "Greater element" not being an "element", and the word "element"
> being a common word :

element means "Element + Greater Element". It is to be understood as the
opposite of object. I think there shouldn't be much ambiguity according
to context.

>> Empty lines belong to the largest element ending before them.  For
>> example, in a list, empty lines between items belong are part of the
>> item before them, but empty lines at the end of a list belong to the
>> plain list element.
>
> Is the word "element" (in /largest element ending.../) to be understood
> as an "element" from the above definition ? I guess not (this would
> require both list items and plain lists to be on the level 'element',
> from your example)

Again, it's a shortcut for "in the largest element or greater element
ending before them".

>> 1 Headlines and Sections
>> ════════════════════════
>>
>>   A headline is defined as:
>>
>>   ╭────
>>   │ STARS KEYWORD PRIORITY TITLE TAGS
>>   ╰────
>>
>>   STARS is a string starting at column 0 and containing at least one
>>   asterisk (and up to `org-inlinetask-min-level' if `org-inlinetask'
>>   library is loaded).  It’s the sole compulsory part of a headline.
>
> Perhaps it should be mentionned that STARS has to end by a space (see
> below).

I agree.

> I suggest adding : The number of stars defines the level of the
> headline.

Does it belong to the syntax definition? Level is how Org uses syntax
internally. Also the sentence, although right, is misleading, because
level definition also depends on `org-odd-levels-only'.

>>   KEYWORD is a TODO keyword, which have to belong to the list defined in
>>   `org-todo-keywords'.  Case is significant.
>
> The option #+TODO: is used also.

Then it should be ~org-todo-keywords-1~, which is where all TODO
keywords are added eventually.

>>   PRIORITY is a priority cookie, i.e. a single letter preceded by a hash
>>   sign # and enclosed within square brackets.  Case is significant.
>
> I suggest dropping "Case is significant" (or maybe give the whole story :
> IIRC, it is the ascii code of the given letter that is used as
> priority)

I'm not sure that the purpose of this document should be to explain how
syntax will be used.

>>   ╭────
>>   │ *
>
> I don't see a space character after that one in your email and it
> doesn't seem to be recognized as a headline by the exporter (hence my
> above suggestion)
>
>>   If the first word appearing in the title is `org-comment-keyword',
>>   the
>
> That should be `org-comment-string' I guess.

Indeed. Btw, I think this variable should be a defconst, not
a defcustom. It just makes things harder for little benefit.

>>   A headline contains directly at most one section, followed by any
>>   number of headlines.  Only a section can contain another section.
>
> From what I understand, "A section is delimited by two headlines (and
> buffer limits)." [I initially thought it was "by two headlines of the
> same level", which it is not from the structure example you give
> later.]

"Only a section can contain another section" is wrong. It should be
removed.

>>   A section contains directly any greater element or element.  Only
>>   a headline can contain a section.  As an exception, text before the
>>   first headline in the document also belongs to a section.
>
>
>>   In a quoted headline contains a section, the latter will be considered
>>   as a “quote section”.
>
> s/In/If/

Yes.

> unsure: s/quote section/quoted section/ ?

No, it is "quote section".

>>   BACKEND is a string constituted of alpha-numeric characters, hyphens
>>   or underscores.
>
> I suggest: BACKEND is a string which is an element of (mapcar 'car
> org-export-registered-backends).

Not really. Parser can understand #+attr_foo even if foo is not
registered as a valid back-end.

>>   OPTIONAL and VALUE can contain any character but a new line.  Only
>>   keywords in `org-element-dual-keywords' can have an optional value.
>
> I guess OPTIONAL cannot contain a closing square bracket ]

It can.

>>   An affiliated keyword can appear on multiple lines if KEY belongs to
>>   `org-element-multiple-keywords' or if its pattern is “#+ATTR_BACKEND:
>>   VALUE”.
>
> I suggest s/on multiple lines/more than once/

Ok.

>>   PARAMETERS can contain any character, and can be omitted.
>
> any other than new line, I guess.

Correct.

>>   CONTENTS can contain any element, but another greater block of the
>>   same type.
>
> What is the type of a greater block ? the /name/ ?

Yes. I think it should be better to say something like: CONTENTS cannot
contain the string "#+END_NAME" on a line on its own.

> I did have a quick look at the rest of your mail, and it is very nice to
> have all of it written down explicitly, so again a big thanks for all of
> this (and the rest of your) work. Unfortunately I don't have much time
> right now to read it thoroughtfully, so just one single comment :
>
>>        Even the LaTeX community suggests to use `\(...\)' over
>>        `$...$'.  — ngz
>
> AFAIK that's not for technical reasons and also I would be curious to
> know who does that in real documents : '$' is so much more convenient.

Yes, I mixed $$...$$ and $...$. This sentence could be removed. Though
I still maintain my POV about $...$. It may be convenient in a latex
file, but in a free-form text format like Org, it's error prone.

I also forgot to write about optional #+tblfm: line below Org tables.

Would you (or Someone) mind updating the org-syntax.org file on Worg?

Thank you again.


Regards,

-- 
Nicolas Goaziou

Re: [RFC] Org syntax (draft)

[Waldemar Quevedo]

Hey Nicolas, this looks very detailed and I think it could be useful
for people trying to write other parsers implementations for org-mode.
Thanks for sharing!

By the way, does it exist somewhere a set of examples of Emacs
org-mode -> html conversion for all org-mode features?
(How are changes from org-mode -> html converstion from Emacs tested
during development?)

I am mantaining the org-ruby gem which is used to render org-mode texts to html,
and currently there is no "roadmap" of features to implement for it.
As a result, features and tweaks are added to the library
as long as someone submits a ticket requesting the feature in Github.
(Here is a list of the export features supported in case someone wants
to take a look:
https://github.com/bdewey/org-ruby/tree/master/spec/html_examples )
Having a set of examples features from org-mode would be very useful
to see how much coverage other implementations of org-mode exporting
features have.

Cheers everyone, keep org-mode being an awesome tool :)

- Waldemar

On Sat, Mar 9, 2013 at 7:06 AM, Nicolas Goaziou <n.goaziou@gmail.com> wrote:
> Hello,
>
> "Nicolas Richard" <theonewiththeevillook@yahoo.fr> writes:
>
>> Nicolas Goaziou <n.goaziou@gmail.com> writes:
>>> As discussed a few days ago, here is a document describing the complete
>>> Org syntax as read by the parser. I also added some comments. I am going
>>> to put the Org file on Worg, so anyone can update it and fix mistakes.
>>
>> [for the record, the org file mentionned by Nicolas is currently at
>> <http://orgmode.org/worg/dev/org-syntax.org>]
>>
>> This looks truly awesome. I give some (naïve) comments below, from my
>> non-expert point of view.
>
> Thank you for your comments.
>
>>> The paragraph is the unit of measurement.  An element defines
>>> syntactical parts that are at the same level as a paragraph, i.e. which
>>> cannot contain or be included in a paragraph.  An object is a part that
>>> could be included in an element.  Greater elements are all parts that
>>> can contain an element.
>>
>> This is very clear but I'm slightly worried about confusion that might come
>> from "Greater element" not being an "element", and the word "element"
>> being a common word :
>
> element means "Element + Greater Element". It is to be understood as the
> opposite of object. I think there shouldn't be much ambiguity according
> to context.
>
>>> Empty lines belong to the largest element ending before them.  For
>>> example, in a list, empty lines between items belong are part of the
>>> item before them, but empty lines at the end of a list belong to the
>>> plain list element.
>>
>> Is the word "element" (in /largest element ending.../) to be understood
>> as an "element" from the above definition ? I guess not (this would
>> require both list items and plain lists to be on the level 'element',
>> from your example)
>
> Again, it's a shortcut for "in the largest element or greater element
> ending before them".
>
>>> 1 Headlines and Sections
>>> ════════════════════════
>>>
>>>   A headline is defined as:
>>>
>>>   ╭────
>>>   │ STARS KEYWORD PRIORITY TITLE TAGS
>>>   ╰────
>>>
>>>   STARS is a string starting at column 0 and containing at least one
>>>   asterisk (and up to `org-inlinetask-min-level' if `org-inlinetask'
>>>   library is loaded).  It’s the sole compulsory part of a headline.
>>
>> Perhaps it should be mentionned that STARS has to end by a space (see
>> below).
>
> I agree.
>
>> I suggest adding : The number of stars defines the level of the
>> headline.
>
> Does it belong to the syntax definition? Level is how Org uses syntax
> internally. Also the sentence, although right, is misleading, because
> level definition also depends on `org-odd-levels-only'.
>
>>>   KEYWORD is a TODO keyword, which have to belong to the list defined in
>>>   `org-todo-keywords'.  Case is significant.
>>
>> The option #+TODO: is used also.
>
> Then it should be ~org-todo-keywords-1~, which is where all TODO
> keywords are added eventually.
>
>>>   PRIORITY is a priority cookie, i.e. a single letter preceded by a hash
>>>   sign # and enclosed within square brackets.  Case is significant.
>>
>> I suggest dropping "Case is significant" (or maybe give the whole story :
>> IIRC, it is the ascii code of the given letter that is used as
>> priority)
>
> I'm not sure that the purpose of this document should be to explain how
> syntax will be used.
>
>>>   ╭────
>>>   │ *
>>
>> I don't see a space character after that one in your email and it
>> doesn't seem to be recognized as a headline by the exporter (hence my
>> above suggestion)
>>
>>>   If the first word appearing in the title is `org-comment-keyword',
>>>   the
>>
>> That should be `org-comment-string' I guess.
>
> Indeed. Btw, I think this variable should be a defconst, not
> a defcustom. It just makes things harder for little benefit.
>
>>>   A headline contains directly at most one section, followed by any
>>>   number of headlines.  Only a section can contain another section.
>>
>> From what I understand, "A section is delimited by two headlines (and
>> buffer limits)." [I initially thought it was "by two headlines of the
>> same level", which it is not from the structure example you give
>> later.]
>
> "Only a section can contain another section" is wrong. It should be
> removed.
>
>>>   A section contains directly any greater element or element.  Only
>>>   a headline can contain a section.  As an exception, text before the
>>>   first headline in the document also belongs to a section.
>>
>>
>>>   In a quoted headline contains a section, the latter will be considered
>>>   as a “quote section”.
>>
>> s/In/If/
>
> Yes.
>
>> unsure: s/quote section/quoted section/ ?
>
> No, it is "quote section".
>
>>>   BACKEND is a string constituted of alpha-numeric characters, hyphens
>>>   or underscores.
>>
>> I suggest: BACKEND is a string which is an element of (mapcar 'car
>> org-export-registered-backends).
>
> Not really. Parser can understand #+attr_foo even if foo is not
> registered as a valid back-end.
>
>>>   OPTIONAL and VALUE can contain any character but a new line.  Only
>>>   keywords in `org-element-dual-keywords' can have an optional value.
>>
>> I guess OPTIONAL cannot contain a closing square bracket ]
>
> It can.
>
>>>   An affiliated keyword can appear on multiple lines if KEY belongs to
>>>   `org-element-multiple-keywords' or if its pattern is “#+ATTR_BACKEND:
>>>   VALUE”.
>>
>> I suggest s/on multiple lines/more than once/
>
> Ok.
>
>>>   PARAMETERS can contain any character, and can be omitted.
>>
>> any other than new line, I guess.
>
> Correct.
>
>>>   CONTENTS can contain any element, but another greater block of the
>>>   same type.
>>
>> What is the type of a greater block ? the /name/ ?
>
> Yes. I think it should be better to say something like: CONTENTS cannot
> contain the string "#+END_NAME" on a line on its own.
>
>> I did have a quick look at the rest of your mail, and it is very nice to
>> have all of it written down explicitly, so again a big thanks for all of
>> this (and the rest of your) work. Unfortunately I don't have much time
>> right now to read it thoroughtfully, so just one single comment :
>>
>>>        Even the LaTeX community suggests to use `\(...\)' over
>>>        `$...$'.  — ngz
>>
>> AFAIK that's not for technical reasons and also I would be curious to
>> know who does that in real documents : '$' is so much more convenient.
>
> Yes, I mixed $$...$$ and $...$. This sentence could be removed. Though
> I still maintain my POV about $...$. It may be convenient in a latex
> file, but in a free-form text format like Org, it's error prone.
>
> I also forgot to write about optional #+tblfm: line below Org tables.
>
> Would you (or Someone) mind updating the org-syntax.org file on Worg?
>
> Thank you again.
>
>
> Regards,
>
> --
> Nicolas Goaziou
>

Re: [RFC] Org syntax (draft)

[Carsten Dominik]

On 9.3.2013, at 11:52, Waldemar Quevedo <waldemar.quevedo@gmail.com> wrote:

> Hey Nicolas, this looks very detailed and I think it could be useful
> for people trying to write other parsers implementations for org-mode.
> Thanks for sharing!

Maybe someone knowledgeable can turn Nicola's description into a formal parser description that can then be used by something like yacc to produce code for arbitrary languages?  I am not sure if I am making sense though.

- Carsten

> 
> By the way, does it exist somewhere a set of examples of Emacs
> org-mode -> html conversion for all org-mode features?
> (How are changes from org-mode -> html converstion from Emacs tested
> during development?)
> 
> I am mantaining the org-ruby gem which is used to render org-mode texts to html,
> and currently there is no "roadmap" of features to implement for it.
> As a result, features and tweaks are added to the library
> as long as someone submits a ticket requesting the feature in Github.
> (Here is a list of the export features supported in case someone wants
> to take a look:
> https://github.com/bdewey/org-ruby/tree/master/spec/html_examples )
> Having a set of examples features from org-mode would be very useful
> to see how much coverage other implementations of org-mode exporting
> features have.
> 
> Cheers everyone, keep org-mode being an awesome tool :)
> 
> - Waldemar
> 
> On Sat, Mar 9, 2013 at 7:06 AM, Nicolas Goaziou <n.goaziou@gmail.com> wrote:
>> Hello,
>> 
>> "Nicolas Richard" <theonewiththeevillook@yahoo.fr> writes:
>> 
>>> Nicolas Goaziou <n.goaziou@gmail.com> writes:
>>>> As discussed a few days ago, here is a document describing the complete
>>>> Org syntax as read by the parser. I also added some comments. I am going
>>>> to put the Org file on Worg, so anyone can update it and fix mistakes.
>>> 
>>> [for the record, the org file mentionned by Nicolas is currently at
>>> <http://orgmode.org/worg/dev/org-syntax.org>]
>>> 
>>> This looks truly awesome. I give some (naïve) comments below, from my
>>> non-expert point of view.
>> 
>> Thank you for your comments.
>> 
>>>> The paragraph is the unit of measurement.  An element defines
>>>> syntactical parts that are at the same level as a paragraph, i.e. which
>>>> cannot contain or be included in a paragraph.  An object is a part that
>>>> could be included in an element.  Greater elements are all parts that
>>>> can contain an element.
>>> 
>>> This is very clear but I'm slightly worried about confusion that might come
>>> from "Greater element" not being an "element", and the word "element"
>>> being a common word :
>> 
>> element means "Element + Greater Element". It is to be understood as the
>> opposite of object. I think there shouldn't be much ambiguity according
>> to context.
>> 
>>>> Empty lines belong to the largest element ending before them.  For
>>>> example, in a list, empty lines between items belong are part of the
>>>> item before them, but empty lines at the end of a list belong to the
>>>> plain list element.
>>> 
>>> Is the word "element" (in /largest element ending.../) to be understood
>>> as an "element" from the above definition ? I guess not (this would
>>> require both list items and plain lists to be on the level 'element',
>>> from your example)
>> 
>> Again, it's a shortcut for "in the largest element or greater element
>> ending before them".
>> 
>>>> 1 Headlines and Sections
>>>> ════════════════════════
>>>> 
>>>>  A headline is defined as:
>>>> 
>>>>  ╭────
>>>>  │ STARS KEYWORD PRIORITY TITLE TAGS
>>>>  ╰────
>>>> 
>>>>  STARS is a string starting at column 0 and containing at least one
>>>>  asterisk (and up to `org-inlinetask-min-level' if `org-inlinetask'
>>>>  library is loaded).  It’s the sole compulsory part of a headline.
>>> 
>>> Perhaps it should be mentionned that STARS has to end by a space (see
>>> below).
>> 
>> I agree.
>> 
>>> I suggest adding : The number of stars defines the level of the
>>> headline.
>> 
>> Does it belong to the syntax definition? Level is how Org uses syntax
>> internally. Also the sentence, although right, is misleading, because
>> level definition also depends on `org-odd-levels-only'.
>> 
>>>>  KEYWORD is a TODO keyword, which have to belong to the list defined in
>>>>  `org-todo-keywords'.  Case is significant.
>>> 
>>> The option #+TODO: is used also.
>> 
>> Then it should be ~org-todo-keywords-1~, which is where all TODO
>> keywords are added eventually.
>> 
>>>>  PRIORITY is a priority cookie, i.e. a single letter preceded by a hash
>>>>  sign # and enclosed within square brackets.  Case is significant.
>>> 
>>> I suggest dropping "Case is significant" (or maybe give the whole story :
>>> IIRC, it is the ascii code of the given letter that is used as
>>> priority)
>> 
>> I'm not sure that the purpose of this document should be to explain how
>> syntax will be used.
>> 
>>>>  ╭────
>>>>  │ *
>>> 
>>> I don't see a space character after that one in your email and it
>>> doesn't seem to be recognized as a headline by the exporter (hence my
>>> above suggestion)
>>> 
>>>>  If the first word appearing in the title is `org-comment-keyword',
>>>>  the
>>> 
>>> That should be `org-comment-string' I guess.
>> 
>> Indeed. Btw, I think this variable should be a defconst, not
>> a defcustom. It just makes things harder for little benefit.
>> 
>>>>  A headline contains directly at most one section, followed by any
>>>>  number of headlines.  Only a section can contain another section.
>>> 
>>> From what I understand, "A section is delimited by two headlines (and
>>> buffer limits)." [I initially thought it was "by two headlines of the
>>> same level", which it is not from the structure example you give
>>> later.]
>> 
>> "Only a section can contain another section" is wrong. It should be
>> removed.
>> 
>>>>  A section contains directly any greater element or element.  Only
>>>>  a headline can contain a section.  As an exception, text before the
>>>>  first headline in the document also belongs to a section.
>>> 
>>> 
>>>>  In a quoted headline contains a section, the latter will be considered
>>>>  as a “quote section”.
>>> 
>>> s/In/If/
>> 
>> Yes.
>> 
>>> unsure: s/quote section/quoted section/ ?
>> 
>> No, it is "quote section".
>> 
>>>>  BACKEND is a string constituted of alpha-numeric characters, hyphens
>>>>  or underscores.
>>> 
>>> I suggest: BACKEND is a string which is an element of (mapcar 'car
>>> org-export-registered-backends).
>> 
>> Not really. Parser can understand #+attr_foo even if foo is not
>> registered as a valid back-end.
>> 
>>>>  OPTIONAL and VALUE can contain any character but a new line.  Only
>>>>  keywords in `org-element-dual-keywords' can have an optional value.
>>> 
>>> I guess OPTIONAL cannot contain a closing square bracket ]
>> 
>> It can.
>> 
>>>>  An affiliated keyword can appear on multiple lines if KEY belongs to
>>>>  `org-element-multiple-keywords' or if its pattern is “#+ATTR_BACKEND:
>>>>  VALUE”.
>>> 
>>> I suggest s/on multiple lines/more than once/
>> 
>> Ok.
>> 
>>>>  PARAMETERS can contain any character, and can be omitted.
>>> 
>>> any other than new line, I guess.
>> 
>> Correct.
>> 
>>>>  CONTENTS can contain any element, but another greater block of the
>>>>  same type.
>>> 
>>> What is the type of a greater block ? the /name/ ?
>> 
>> Yes. I think it should be better to say something like: CONTENTS cannot
>> contain the string "#+END_NAME" on a line on its own.
>> 
>>> I did have a quick look at the rest of your mail, and it is very nice to
>>> have all of it written down explicitly, so again a big thanks for all of
>>> this (and the rest of your) work. Unfortunately I don't have much time
>>> right now to read it thoroughtfully, so just one single comment :
>>> 
>>>>       Even the LaTeX community suggests to use `\(...\)' over
>>>>       `$...$'.  — ngz
>>> 
>>> AFAIK that's not for technical reasons and also I would be curious to
>>> know who does that in real documents : '$' is so much more convenient.
>> 
>> Yes, I mixed $$...$$ and $...$. This sentence could be removed. Though
>> I still maintain my POV about $...$. It may be convenient in a latex
>> file, but in a free-form text format like Org, it's error prone.
>> 
>> I also forgot to write about optional #+tblfm: line below Org tables.
>> 
>> Would you (or Someone) mind updating the org-syntax.org file on Worg?
>> 
>> Thank you again.
>> 
>> 
>> Regards,
>> 
>> --
>> Nicolas Goaziou
>> 
> 

Re: [RFC] Org syntax (draft)

[Nicolas Goaziou]

Hello,

Carsten Dominik <carsten.dominik@gmail.com> writes:

> On 9.3.2013, at 11:52, Waldemar Quevedo <waldemar.quevedo@gmail.com> wrote:
>
>> Hey Nicolas, this looks very detailed and I think it could be useful
>> for people trying to write other parsers implementations for org-mode.
>> Thanks for sharing!
>
> Maybe someone knowledgeable can turn Nicola's description into
> a formal parser description that can then be used by something like
> yacc to produce code for arbitrary languages? I am not sure if I am
> making sense though.

*cough* you mean GNU Bison or, perhaps better, Wisent (from Semantic).
I don't know how well they handle context sensitive grammars, though..


Regards,

-- 
Nicolas Goaziou

Re: [RFC] Org syntax (draft)

[Carsten Dominik]

On 9.3.2013, at 15:42, Nicolas Goaziou <n.goaziou@gmail.com> wrote:

> Hello,
> 
> Carsten Dominik <carsten.dominik@gmail.com> writes:
> 
>> On 9.3.2013, at 11:52, Waldemar Quevedo <waldemar.quevedo@gmail.com> wrote:
>> 
>>> Hey Nicolas, this looks very detailed and I think it could be useful
>>> for people trying to write other parsers implementations for org-mode.
>>> Thanks for sharing!
>> 
>> Maybe someone knowledgeable can turn Nicola's description into
>> a formal parser description that can then be used by something like
>> yacc to produce code for arbitrary languages? I am not sure if I am
>> making sense though.
> 
> *cough* you mean GNU Bison

Told you I am not sure if I am making sense.

Anyway, a general parser would be useful for extensions like org-ruby...

- Carsten

Re: [RFC] Org syntax (draft)

[Nicolas Goaziou]

Hello,

Waldemar Quevedo <waldemar.quevedo@gmail.com> writes:

> By the way, does it exist somewhere a set of examples of Emacs
> org-mode -> html conversion for all org-mode features?
> (How are changes from org-mode -> html converstion from Emacs tested
> during development?)

I don't think something like this exists. Though, html back-end supports
all Org syntax described in the document.


Regards,

-- 
Nicolas Goaziou

Re: [RFC] Org syntax (draft)

[Samuel Wales]

On 3/9/13, Waldemar Quevedo <waldemar.quevedo@gmail.com> wrote:
> By the way, does it exist somewhere a set of examples of Emacs
> org-mode -> html conversion for all org-mode features?
> (How are changes from org-mode -> html converstion from Emacs tested
> during development?)

+1

That would be great.  I'd definitely donate any of my blog posts, but
perhaps others have more comprehensive tests.

Samuel

-- 
The Kafka Pandemic: http://thekafkapandemic.blogspot.com

The disease DOES progress.  MANY people have died from it.  Just like
AIDS, it attacks MANY body systems.  ANYBODY can get it.  There is NO
hope without activist action.  This means YOU.

Re: [RFC] Org syntax (draft)

[Bastien]

Hi Waldemar,

Waldemar Quevedo <waldemar.quevedo@gmail.com> writes:

> By the way, does it exist somewhere a set of examples of Emacs
> org-mode -> html conversion for all org-mode features?

Not really -- and it would be nice to have one, especially for
developers like you who are in charge of an external exporter!

> I am mantaining the org-ruby gem which is used to render org-mode texts to html,
> and currently there is no "roadmap" of features to implement for it.
> As a result, features and tweaks are added to the library
> as long as someone submits a ticket requesting the feature in Github.
> (Here is a list of the export features supported in case someone wants
> to take a look:
> https://github.com/bdewey/org-ruby/tree/master/spec/html_examples )
> Having a set of examples features from org-mode would be very useful
> to see how much coverage other implementations of org-mode exporting
> features have.
>
> Cheers everyone, keep org-mode being an awesome tool :)

Thank *you* for maintaining the org-ruby gem -- truly a gem to the
github community!  Hopefully you'll be able to update the gem wrt
the latest syntactic changes.  There are not too many of them, and
not every will use Org 8.0 so soon, but still.

Best,

-- 
 Bastien

Re: [RFC] Org syntax (draft)

[Nicolas Richard]

Hi,

I obviously did not send and actually lost a message I prepared two days
ago. I'll try again.

>> I suggest adding : The number of stars defines the level of the
>> headline.
>
> Does it belong to the syntax definition? Level is how Org uses syntax
> internally. Also the sentence, although right, is misleading, because
> level definition also depends on `org-odd-levels-only'.

I think it's partly in the syntax, since it defines "parentness" for
headlines (the numeric level is of no importance, but the relative level
is used).

>
>> I suggest dropping "Case is significant" (or maybe give the whole story :
>> IIRC, it is the ascii code of the given letter that is used as
>> priority)
>
> I'm not sure that the purpose of this document should be to explain how
> syntax will be used.

That is why I suggested dropping the mention : case is not significant
for the syntax. Very minor though, obviously.

>> That should be `org-comment-string' I guess.
>
> Indeed. Btw, I think this variable should be a defconst, not
> a defcustom. It just makes things harder for little benefit.

As you know, "Comment" is also a french word meaning "how", and that
could very well appear uppercased as the first word of a title. (I'd
personally recommend against uppercasing titles, but I'd understand if
someone wanted to customize the word for such reasons)

> Would you (or Someone) mind updating the org-syntax.org file on Worg?

Please review the attached patch and apply parts as you wish (even if I
wanted to do it myself, I don't have worg access.)

Last word about #+TBLFM: I'm not sure if that should go into the "affiliated
keywords" section (thus rewriting parts of it, because that one goes
below the table, unlike other affiliated keywords) or a special section
on its own. Thus I'm not changing anything wrt that.

From f97c00bfbd8a14d0b2953ee0e8b6817a2b9f0306 Mon Sep 17 00:00:00 2001
From: Nicolas Richard <theonewiththeevillook@yahoo.fr>
Date: Mon, 11 Mar 2013 16:25:21 +0100
Subject: [PATCH] dev/org-syntax.org: minor

---
 dev/org-syntax.org | 34 +++++++++++++++++++---------------
 1 file changed, 19 insertions(+), 15 deletions(-)

diff --git a/dev/org-syntax.org b/dev/org-syntax.org
index 9b2a843..a918a75 100644
--- a/dev/org-syntax.org
+++ b/dev/org-syntax.org
@@ -15,7 +15,8 @@ within specific environments.
 
 Three categories are used to classify these environments: "Greater
 elements", "elements", and "objects", from the broadest scope to the
-narrowest.
+narrowest.  The word "element" is used for both Greater and non-Greater
+elements, the context should make that clear.
 
 The paragraph is the unit of measurement.  An element defines
 syntactical parts that are at the same level as a paragraph,
@@ -41,16 +42,17 @@ Unless specified otherwise, case is not significant.
   STARS KEYWORD PRIORITY TITLE TAGS
   #+END_EXAMPLE
 
-  STARS is a string starting at column 0 and containing at least one
+  STARS is a string starting at column 0, containing at least one
   asterisk (and up to ~org-inlinetask-min-level~ if =org-inlinetask=
-  library is loaded).  It's the sole compulsory part of a headline.
+  library is loaded) and ended by a space character.  The number of
+  asterisks is used to define the level of the headline.  It's the
+  sole compulsory part of a headline.
 
   KEYWORD is a TODO keyword, which has to belong to the list defined
-  in ~org-todo-keywords~.  Case is significant.
+  in ~org-todo-keywords-1~.  Case is significant.
 
   PRIORITY is a priority cookie, i.e. a single letter preceded by
-  a hash sign # and enclosed within square brackets.  Case is
-  significant.
+  a hash sign # and enclosed within square brackets.
 
   TITLE can be made of any character but a new line.  Though, it will
   match after every other part have been matched.
@@ -71,7 +73,7 @@ Unless specified otherwise, case is not significant.
   ,**** TODO [#A] COMMENT Title :tag:a2%:
   #+END_EXAMPLE
     
-  If the first word appearing in the title is ~org-comment-keyword~,
+  If the first word appearing in the title is ~org-comment-string~,
   the headline will be considered as "commented".  If that first word
   is ~org-quote-string~, it will be considered as "quoted".  In both
   situations, case is significant.
@@ -82,14 +84,14 @@ Unless specified otherwise, case is not significant.
   If ~org-archive-tag~ is one of its tags, it will be considered as
   "archived".  Case is significant.
 
-  A headline contains directly at most one section, followed by any
-  number of headlines.  Only a section can contain another section.
+  A headline contains directly one section, followed by any
+  number of deeper level headlines.
 
   A section contains directly any greater element or element.  Only
   a headline can contain a section.  As an exception, text before the
   first headline in the document also belongs to a section.
 
-  In a quoted headline contains a section, the latter will be
+  If a quoted headline contains a section, the latter will be
   considered as a "quote section".
 
   As an example, consider the following document:
@@ -136,7 +138,8 @@ Unless specified otherwise, case is not significant.
   attributes.
 
   This is done by adding specific keywords, named "affiliated
-  keywords", just above the element considered, no blank line allowed.
+  keywords", just above the element considered, no blank line
+  allowed.
 
   Affiliated keywords are built upon one of the following patterns:
   "#+KEY: VALUE", "#+KEY[OPTIONAL]: VALUE" or "#+ATTR_BACKEND: VALUE".
@@ -150,7 +153,7 @@ Unless specified otherwise, case is not significant.
   OPTIONAL and VALUE can contain any character but a new line.  Only
   "CAPTION" and "RESULTS" keywords can have an optional value.
 
-  An affiliated keyword can appear on multiple lines if KEY is either
+  An affiliated keyword can appear more than once if KEY is either
   "CAPTION" or "HEADER" or if its pattern is "#+ATTR_BACKEND: VALUE".
 
   "CAPTION", "AUTHOR", "DATE" and "TITLE" keywords can contain objects
@@ -183,7 +186,8 @@ Unless specified otherwise, case is not significant.
 
    NAME can contain any non-whitespace character.
 
-   PARAMETERS can contain any character, and can be omitted.
+   PARAMETERS can contain any character other than new line, and can
+   be omitted.
 
    If NAME is "CENTER", it will be a "center block".  If it is
    "QUOTE", it will be a "quote block".
@@ -191,8 +195,8 @@ Unless specified otherwise, case is not significant.
    If the block is neither a center block, a quote block or a [[#Blocks][block
    element]], it will be a "special block".
 
-   CONTENTS can contain any element, but another greater block of the
-   same type.
+   CONTENTS can contain any element, except : a line =#+END_NAME= on
+   its own and lines beginning with STARS must be quoted by a comma.
 
 ** Drawers and Property Drawers
    :PROPERTIES:
-- 
1.8.1.4

Re: [RFC] Org syntax (draft)

[Nicolas Goaziou]

Hello,

"Nicolas Richard" <theonewiththeevillook@yahoo.fr> writes:

> As you know, "Comment" is also a french word meaning "how", and that
> could very well appear uppercased as the first word of a title. (I'd
> personally recommend against uppercasing titles, but I'd understand if
> someone wanted to customize the word for such reasons)

"Comment" is not "COMMENT", as you say. Also, it is a minor annoyance.

On the other hand, crippling portability of Org format because it
depends on an external variable is a major problem.

Therefore, I stand on my ground: I suggest to turn `org-comment-string'
and al. into defconst.

>> Would you (or Someone) mind updating the org-syntax.org file on Worg?
>
> Please review the attached patch and apply parts as you wish (even if I
> wanted to do it myself, I don't have worg access.)

I applied the patch. Thank you.

> Last word about #+TBLFM: I'm not sure if that should go into the "affiliated
> keywords" section (thus rewriting parts of it, because that one goes
> below the table, unlike other affiliated keywords) or a special section
> on its own. Thus I'm not changing anything wrt that.

TBLFM is not an affiliated keyword. It's a keyword specific to Org
tables. It can exist anywhere, but it only makes sense right after
a table.


Regards,

-- 
Nicolas Goaziou

Re: [RFC] Org syntax (draft)

[Achim Gratz]

Hi Nicolas,

here are my first comments. I'm still trying to wrap my head around some
things, so if I'm off the map on something, please be patient.

Do you mind if I fix some obvious typos directly on Worg or do you'd
rather want patches?


Nicolas Goaziou writes:
> A core concept in this syntax is that only headlines and sections are
> context-free[1][2].  Every other syntactical part only exists within
> specific environments.

Blank lines or empty lines are also context-free syntactical elements,
I'd think.

> Three categories are used to classify these environments: “Greater
> elements”, “elements”, and “objects”, from the broadest scope to the
> narrowest.

It might be easier to talk about those things if "Greater Element" was
called "Collection" to perhaps keep with the thingies theme of naming
the syntax.

> The paragraph is the unit of measurement.  An element defines
> syntactical parts that are at the same level as a paragraph, i.e. which
> cannot contain or be included in a paragraph.  An object is a part that
> could be included in an element.  Greater elements are all parts that
> can contain an element.

Here's my main contention with that model: I think there should be an
greater element, maybe named "paragraph block" that translates into a
paragraph at the backend level.  Most backends will have a paragraph
model that is much less limited than what the current definition of an
Org paragraph is.  This could be optionally be an implicit greater
block that is defined by the presence or absence of blank lines between
elements, I'd think.

> 3.1 Greater Blocks
> ──────────────────

The same naming confusion as with the various "elements", for now I'd
link to think of these as "Box".

>   Greater blocks consist in the following pattern:
>
>   ╭────
>   │ #+BEGIN_NAME PARAMETERS
>   │ CONTENTS
>   │ #+END_NAME
>   ╰────

I'm beginning to wonder if these should have the same syntax as blocks.
Maybe that's a too fine a distinction visually, but adding a colon would
disambiguate the greater blocks from the normal ones.  In other words

#+BEGIN_CENTER: humdum
…
#+END_CENTER:

would be a center block, while

#+BEGIN_CENTER humdum
…
#+END_CENTER

would be an export block for the center backend.

> 4.2 Blocks
> ──────────
>
>   Like [greater blocks], pattern for blocks is:
>
>   ╭────
>   │ #+BEGIN_NAME DATA
>   │ CONTENTS
>   │ #+END_NAME
>   ╰────
[…]
>   DATA can contain any character but a new line.

I'd keep with PARAMETERS here.

>   If NAME is a string matching the name of any export back-end loaded,
>   the block will be an “export block”.

Conversely, blocks that are not having a recognizable name will simply
insert their content as if the block markers were not there, e.g. it
seems to treat these as parsed blocks.  I don't think this should
happen, instead Org should parse this as an unknown export backend and
drop the content with a warning, not unlike a comment.

This will be a major sticking point with external parsers: they'd
otherwise need to know about the Org export backends to when to use the
content of the block and when not.  A portable Org document should be
able to specify which export backends it expects to be available (and
maybe what standard backend it is derived from) to elicit the correct
behaviour.

>   CONTENTS can contain any character, including new lines.  Though it
>   will only contain Org objects if the block is a verse block.
>   Otherwise, contents will not be parsed.

Would it make sense to make a general distinction between parsed and
non-parsed blocks based on some configuration, even though this would
produce the same issue as with export backends?

>         I suggest to remove angle links.  If one needs spaces in
>         PATH, she can use standard link syntax instead.

They are very ubiquitous on certain platforms, so copy&paste would be
made frustrating there if you'd need to re-format them each time around.



Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptation for Waldorf Blofeld V1.15B11:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada

Re: [RFC] Org syntax (draft)

[Nicolas Goaziou]

Hello,

Achim Gratz <Stromeko@nexgo.de> writes:

> Do you mind if I fix some obvious typos directly on Worg or do you'd
> rather want patches?

Please go ahead. This is on Worg so anyone can improve it.

> Nicolas Goaziou writes:
>> A core concept in this syntax is that only headlines and sections are
>> context-free[1][2].  Every other syntactical part only exists within
>> specific environments.
>
> Blank lines or empty lines are also context-free syntactical elements,
> I'd think.

No, the aren't, as they belong to the broadest element ending before
them. So you need to know what this element is.

>> Three categories are used to classify these environments: “Greater
>> elements”, “elements”, and “objects”, from the broadest scope to the
>> narrowest.
>
> It might be easier to talk about those things if "Greater Element" was
> called "Collection" to perhaps keep with the thingies theme of naming
> the syntax.

Collection could also be ambiguous as a paragraph may be seen as
a collection of objects.

>> The paragraph is the unit of measurement.  An element defines
>> syntactical parts that are at the same level as a paragraph, i.e. which
>> cannot contain or be included in a paragraph.  An object is a part that
>> could be included in an element.  Greater elements are all parts that
>> can contain an element.
>
> Here's my main contention with that model: I think there should be an
> greater element, maybe named "paragraph block" that translates into a
> paragraph at the backend level.  Most backends will have a paragraph
> model that is much less limited than what the current definition of an
> Org paragraph is.  This could be optionally be an implicit greater
> block that is defined by the presence or absence of blank lines between
> elements, I'd think.

I don't get it. What would be the exact definition of a "paragraph
block"? What limitations are you talking about?

>> 3.1 Greater Blocks
>> ──────────────────
>
> The same naming confusion as with the various "elements", for now I'd
> link to think of these as "Box".

This naming was for the org-syntax file only. "Greater blocks" means
nothing for org-element.el, but "center block", "quote block", "special
block" do.

>>   Greater blocks consist in the following pattern:
>>
>>   ╭────
>>   │ #+BEGIN_NAME PARAMETERS
>>   │ CONTENTS
>>   │ #+END_NAME
>>   ╰────
>
> I'm beginning to wonder if these should have the same syntax as blocks.
> Maybe that's a too fine a distinction visually, but adding a colon would
> disambiguate the greater blocks from the normal ones.  In other words
>
> #+BEGIN_CENTER: humdum
> &
> #+END_CENTER:
>
> would be a center block, while
>
> #+BEGIN_CENTER humdum
> &
> #+END_CENTER
>
> would be an export block for the center backend.

I agree. More on that below.

>> 4.2 Blocks
>> ──────────
>>
>>   Like [greater blocks], pattern for blocks is:
>>
>>   ╭────
>>   │ #+BEGIN_NAME DATA
>>   │ CONTENTS
>>   │ #+END_NAME
>>   ╰────
> […]
>>   DATA can contain any character but a new line.
>
> I'd keep with PARAMETERS here.

Ok. Just fix it.

>>   If NAME is a string matching the name of any export back-end loaded,
>>   the block will be an “export block”.
>
> Conversely, blocks that are not having a recognizable name will simply
> insert their content as if the block markers were not there, e.g. it
> seems to treat these as parsed blocks.

You are talking about "special blocks", right? They have a special
purpose. In latex back-end,

  #+begin_special
  ...
  #+end_special

becomes

  \begin{special}
  ...
  \end{special}

IOW this is an Org feature.

> I don't think this should happen, instead Org should parse this as an
> unknown export backend and drop the content with a warning, not unlike
> a comment.

This would remove special blocks.

> This will be a major sticking point with external parsers: they'd
> otherwise need to know about the Org export backends to when to use the
> content of the block and when not.  A portable Org document should be
> able to specify which export backends it expects to be available (and
> maybe what standard backend it is derived from) to elicit the correct
> behaviour.

I agree, as notified above. If we want to separate Org /format/ from
Emacs, we need to separate special blocks from export blocks. The former
cannot be the fallback type when the latter isn't recognized.

In that case, a different syntax for export blocks would be needed.
Maybe the colons you suggested above. I think that something more
visible would be better, though.

>>   CONTENTS can contain any character, including new lines.  Though it
>>   will only contain Org objects if the block is a verse block.
>>   Otherwise, contents will not be parsed.
>
> Would it make sense to make a general distinction between parsed and
> non-parsed blocks based on some configuration, even though this would
> produce the same issue as with export backends?

This is inherent from the block type. This mustn't be configurable.
There is no point in parsing a src-block, for example. On the other
hand, if you don't parse (partially) contents of a verse-block, you get
an example-block, and one of them becomes useless.

Then, there are special blocks. It was suggested, a few days ago, that
a parameter could be set in order to tell the parser what to do with
their contents. That's an interesting idea. But it only makes sense if
there is also a way to specify a transformation function on these
contents (otherwise, an export block would be sufficient). Also the same
could be achieved with Babel, the non-parsed data being an example
block, and the transformation function a src-block.


Regards,

-- 
Nicolas Goaziou

Re: [RFC] Org syntax (draft)

[Jambunathan K]

Nicolas

>> Do you mind if I fix some obvious typos directly on Worg or do you'd
>> rather want patches?
>
> Please go ahead. This is on Worg so anyone can improve it.

Please consider adding the Org spec (and also the exporter reference)
document to the Org manual.

This will be a good excuse for exercising the TexInfo exporter and see
where it leads.

Committing to Org or Worg has same load cycle.  I feel there is more
value if it is right within the Org manual (i.e, part of Emacs).

Only reason you may want to have it on Worg is possibly because it is
likely to be read by wider audience.  People seem to like browsers.

(You can consider building a standalone PDF/HTML document out of .texi
sources and have people read it)

-- 

Re: [RFC] Org syntax (draft)

[Nicolas Goaziou]

Hello,

Jambunathan K <kjambunathan@gmail.com> writes:

> Please consider adding the Org spec (and also the exporter reference)
> document to the Org manual.
>
> This will be a good excuse for exercising the TexInfo exporter and see
> where it leads.
>
> Committing to Org or Worg has same load cycle.  I feel there is more
> value if it is right within the Org manual (i.e, part of Emacs).
>
> Only reason you may want to have it on Worg is possibly because it is
> likely to be read by wider audience.  People seem to like browsers.

It is not ready to go into the manual in its current state. As specified
in its title, it's nothing more than a draft. Some parts have to be
rewritten, some information is missing, my notes have to be removed,
etc. Once it becomes clear enough, Bastien may consider adding it to the
manual.

The same holds for the exporter document.


Regards,

-- 
Nicolas Goaziou

Re: [RFC] Org syntax (draft)

[Bastien]

Hi Nicolas,

the manual would enjoy a subsection in "Hacking" on how to create
a new exporter, either from scratch or as a derived exporter.
(Such a subsection can be short enough, thanks to derived backend.)

From this section, we can throw links to the exporter reference
document and the Org syntax document published on Worgs.

We may also add footnotes referring to the Org syntax relevant
sections, when needed.

But both reference documents don't fit into the manual IMO.  They
are great resources for developers, not for users.  The footnotes
are enough for advanced users who want to go beyond the manual.

Thanks!

-- 
 Bastien

Re: [RFC] Org syntax (draft)

[Bastien]

Bastien <bzg@altern.org> writes:

> But both reference documents don't fit into the manual IMO.  They
> are great resources for developers, not for users.  The footnotes
> are enough for advanced users who want to go beyond the manual.

That said, we can also bundle both documents into Org's distribution,
as .org files in the doc/ directory.  And have a make rule to convert
them to .pdf and info docs.

-- 
 Bastien

Re: [RFC] Org syntax (draft)

[Achim Gratz]

Bastien writes:
> That said, we can also bundle both documents into Org's distribution,
> as .org files in the doc/ directory.  And have a make rule to convert
> them to .pdf and info docs.

I don't want to be the party pooper, but if these documents should go
into the distribution, then we must insist that all modifications be FSF
copyrighted, otherwise we'd have to remove and/or rewrite the ones that
aren't when they are incorporated into the distribution.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptations for Waldorf Q V3.00R3 and Q+ V3.54R2:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada

Re: [RFC] Org syntax (draft)

[Bastien]

Achim Gratz <Stromeko@nexgo.de> writes:

> Bastien writes:
>> That said, we can also bundle both documents into Org's distribution,
>> as .org files in the doc/ directory.  And have a make rule to convert
>> them to .pdf and info docs.
>
> I don't want to be the party pooper, but if these documents should go
> into the distribution, then we must insist that all modifications be FSF
> copyrighted, otherwise we'd have to remove and/or rewrite the ones that
> aren't when they are incorporated into the distribution.

No, the documents can go into the distribution with contributions from
anyone, because they won't be in Emacs.  FSF assignment is needed only
for things that go into Emacs.

Best,

-- 
 Bastien

Re: [RFC] Org syntax (draft)

[Achim Gratz]

Bastien writes:
> No, the documents can go into the distribution with contributions from
> anyone, because they won't be in Emacs.  FSF assignment is needed only
> for things that go into Emacs.

I understood that these or substantial parts of it will end up in the
Org manual, which is in Emacs.  If that's not the case, then disregard
my comment.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptation for Waldorf rackAttack V1.04R1:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada

Re: [RFC] Org syntax (draft)

[Jambunathan K]

Achim Gratz <Stromeko@nexgo.de> writes:

> Bastien writes:
>> No, the documents can go into the distribution with contributions from
>> anyone, because they won't be in Emacs.  FSF assignment is needed only
>> for things that go into Emacs.
>
> I understood that these or substantial parts of it will end up in the
> Org manual, which is in Emacs.  If that's not the case, then disregard
> my comment.

Emacs lisp has a manual of it's own.  I don't see how Org export
reference *cannot* end in Emacs.  

Bastien is doing what(ever) suits his whims and you are approving of it.
I disapprove of what you are doing, Achim.  Export syntax deserves to be
part of Org/Emacs.  Let the maintainer go to hell.  He is talking
irreverently/hand-wavingly about some work which has stretched to good
part of around 3 years.  The Orgmode maintainership is in the hands of
the wrong person, he calls shots based on his owh whims and I regret it.

Jambunathan K.

>
>
> Regards,
> Achim.

-- 

Re: [RFC] Org syntax (draft)

[Achim Gratz]

Jambunathan K writes:
> Emacs lisp has a manual of it's own.  I don't see how Org export
> reference *cannot* end in Emacs.  

I said that I'm expecting these references to become part of the
manual(s).  I still expect that and will try to help it along, but it
doesn't necessarily need to take the exact sequence of events that I
envisioned.

> Bastien is doing what(ever) suits his whims and you are approving of
> it.

I haven't approved or disapproved anything.  I have only stated the
plain fact that if my understanding of the future course of events is
incorrect, then my comment does not apply (and conversely, if it does,
then the issue I've stated needs to be dealt with).

> I disapprove of what you are doing, Achim.

You're welcome.  (Sun Tzu, III/2)


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptations for Waldorf Q V3.00R3 and Q+ V3.54R2:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada

Re: [RFC] Org syntax (draft)

[Jonathan Leech-Pepin]

[-- Attachment #1: Type: text/plain, Size: 1379 bytes --]

Hello

On 10 March 2013 13:12, Achim Gratz <Stromeko@nexgo.de> wrote:

> Jambunathan K writes:
> > Emacs lisp has a manual of it's own.  I don't see how Org export
> > reference *cannot* end in Emacs.
>
> I said that I'm expecting these references to become part of the
> manual(s).  I still expect that and will try to help it along, but it
> doesn't necessarily need to take the exact sequence of events that I
> envisioned.
>

I have to agree with Bastien that they do not really fit into the main
Org manual.

Providing them with Emacs (so that they are immediately available) is
a good thing in my mind, however I would put them as a separate
document similarly to how the Elisp manual is separate.

Regards,

Jon


> > Bastien is doing what(ever) suits his whims and you are approving of
> > it.
>
> I haven't approved or disapproved anything.  I have only stated the
> plain fact that if my understanding of the future course of events is
> incorrect, then my comment does not apply (and conversely, if it does,
> then the issue I've stated needs to be dealt with).
>
> > I disapprove of what you are doing, Achim.
>
> You're welcome.  (Sun Tzu, III/2)
>
>
> Regards,
> Achim.
> --
> +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+
>
> SD adaptations for Waldorf Q V3.00R3 and Q+ V3.54R2:
> http://Synth.Stromeko.net/Downloads.html#WaldorfSDada
>
>
>

[-- Attachment #2: Type: text/html, Size: 2174 bytes --]

Re: [RFC] Org syntax (draft)

[Jambunathan K]

Bastien

> But both reference documents don't fit into the manual IMO.  

You are a jerk, a BIG JERK. 

Jambunathan K.
-- 

Re: [RFC] Org syntax (draft)

[Eric S Fraga]

Jambunathan K <kjambunathan@gmail.com> writes:

> You are a jerk, a BIG JERK. 

This is completely uncalled.  What satisfaction do you gain from this?
This is a brilliant, informative and polite mailing list except when it
comes to your contributions.

Don't bother answering because I've added you to my spam database.  I've
not had to do this for an individual for some years now.  I'll never see
any of your emails again.  Bye.

-- 
: Eric S Fraga, GnuPG: 0xC89193D8FFFCF67D
: in Emacs 24.3.50.1 and Org release_7.9.3f-1199-g3a0e55

Re: [RFC] Org syntax (draft)

[Jambunathan K]

Eric

Eric S Fraga <e.fraga@ucl.ac.uk> writes:

> Jambunathan K <kjambunathan@gmail.com> writes:
>
>> You are a jerk, a BIG JERK. 
>
> This is completely uncalled.  What satisfaction do you gain from this?
> This is a brilliant, informative and polite mailing list except when it
> comes to your contributions.
>
> Don't bother answering because I've added you to my spam database.  I've
> not had to do this for an individual for some years now.  I'll never see
> any of your emails again.  Bye.

Still you haven't answered my "Fudging the mail reply headers" question
to my satisfaction.  I just let you off the hook (at the last moment)
before charging ahead.

Jambunathan K.
-- 

Re: [RFC] Org syntax (draft)

[David Engster]

Jambunathan K. writes:
> Still you haven't answered my "Fudging the mail reply headers" question
> to my satisfaction.

http://www.gnu.org/software/emacs/manual/html_node/message/Mailing-Lists.html

"A mailing list poster can use MFT to express that responses should be
sent to just the list, and not the poster as well. This will happen if
the poster is already subscribed to the list."

-David

[Out-of-Thread] Re: [RFC] Org syntax (draft)

[Jambunathan K]

David Engster <deng@randomsample.de> writes:

> Jambunathan K. writes:
>> Still you haven't answered my "Fudging the mail reply headers" question
>> to my satisfaction.
>
> http://www.gnu.org/software/emacs/manual/html_node/message/Mailing-Lists.html
>
> "A mailing list poster can use MFT to express that responses should be
> sent to just the list, and not the poster as well. This will happen if
> the poster is already subscribed to the list."

(Nicolas, I am sorry.  I have added OT to this post)

I know that.  

But that doesn't answer the question why Carsten will appear in the To
header of a mail that I reply to a mail I receive from Eric S Fraga.

See my question here:

        http://lists.gnu.org/archive/html/emacs-orgmode/2013-02/msg00525.html

If I "wide reply" to your post, I see that the TO header points to the
ML.  Why it doesn't happen with Eric S Fraga's post?  The said behaviour
will change based on the phase of moon.

I have noticed such fudging of the To headers by Eric for a long time.
Unless there is a satisfactory explanataion, I will assume Eric is
faking innocence.

> -David

Re: [Out-of-Thread] Re: [RFC] Org syntax (draft)

[David Engster]

Jambunathan K. writes:
> I know that.  
>
> But that doesn't answer the question why Carsten will appear in the To
> header of a mail that I reply to a mail I receive from Eric S Fraga.

Because Carsten started the thread and did not set MFT.

-David

Re: [Out-of-Thread] Re: [RFC] Org syntax (draft)

[Jambunathan K]

David Engster <deng@randomsample.de> writes:

> Jambunathan K. writes:
>> I know that.  
>>
>> But that doesn't answer the question why Carsten will appear in the To
>> header of a mail that I reply to a mail I receive from Eric S Fraga.
>
> Because Carsten started the thread and did not set MFT.

In this very specific thread - the one I am typing right now, Nicolas
Goaziou doesn't appear in To - why?  Normally, when I wide reply to a
Ngz's post it gets sent to him as well as the mailing list.

If your argument is right, Ngz should appear in To or CC headers of the
post.  I see only ML address.  So what am I missing?

> -David

Re: [Out-of-Thread] Re: [RFC] Org syntax (draft)

[David Engster]

Jambunathan K. writes:
> David Engster <deng@randomsample.de> writes:
>
>> Jambunathan K. writes:
>>> I know that.  
>>>
>>> But that doesn't answer the question why Carsten will appear in the To
>>> header of a mail that I reply to a mail I receive from Eric S Fraga.
>>
>> Because Carsten started the thread and did not set MFT.
>
> In this very specific thread - the one I am typing right now, Nicolas
> Goaziou doesn't appear in To - why?  Normally, when I wide reply to a
> Ngz's post it gets sent to him as well as the mailing list.
>
> If your argument is right, Ngz should appear in To or CC headers of the
> post.  I see only ML address.  So what am I missing?

You didn't do a wide reply in <871ubntg32.fsf@gmail.com>.

-David

Re: [Out-of-Thread] Re: [RFC] Org syntax (draft)

[Jambunathan K]

David Engster <deng@randomsample.de> writes:

> Jambunathan K. writes:
>> David Engster <deng@randomsample.de> writes:
>>
>>> Jambunathan K. writes:
>>>> I know that.  
>>>>
>>>> But that doesn't answer the question why Carsten will appear in the To
>>>> header of a mail that I reply to a mail I receive from Eric S Fraga.
>>>
>>> Because Carsten started the thread and did not set MFT.
>>
>> In this very specific thread - the one I am typing right now, Nicolas
>> Goaziou doesn't appear in To - why?  Normally, when I wide reply to a
>> Ngz's post it gets sent to him as well as the mailing list.
>>
>> If your argument is right, Ngz should appear in To or CC headers of the
>> post.  I see only ML address.  So what am I missing?
>
> You didn't do a wide reply in <871ubntg32.fsf@gmail.com>.

I don't what that mail is.  My Gnus-foo is just OKish.

Anyways, S W is what I do on all my mails - those that I reply to on the
ML and on those that I reply to from my Inbox.

If I reply to ESF's posts, I might have edited the headers myself.
Normally - 99% of the cases - I don't edit headers by hand.

ps: I don't want to continue this thread further.  For your
satisfaction, I am willing to concede that I am wrong.

> -David
>
>

-- 

Re: [RFC] Org syntax (draft)

[Bastien]

Hi all,

Bastien <bzg@altern.org> writes:

> the manual would enjoy a subsection in "Hacking" on how to create
> a new exporter, either from scratch or as a derived exporter.
> (Such a subsection can be short enough, thanks to derived backend.)

FWIW, I started a rudimentary one.  

This is "Adding export back-ends" in the current manual from master.

-- 
 Bastien

Re: [RFC] Org syntax (draft)

[Achim Gratz]

Nicolas Goaziou writes:
> As discussed a few days ago, here is a document describing the complete
> Org syntax as read by the parser. I also added some comments. I am going
> to put the Org file on Worg, so anyone can update it and fix mistakes.

after some playing with the Org manual in Org that Tom has been working
on I am starting to think that there should be a way to define the same
macro differently for different export backends.  That would be mainly
so that you could have a macro expansion use export snippets tailored to
that backend where (after stripping the export snippets) the expansion
makes little or no sense in other backends.  What do you think?


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

DIY Stuff:
http://Synth.Stromeko.net/DIY.html

Re: [RFC] Org syntax (draft)

[Sebastien Vauban]

Hi Achim,

Achim Gratz wrote:
> Nicolas Goaziou writes:
>> As discussed a few days ago, here is a document describing the complete
>> Org syntax as read by the parser. I also added some comments. I am going
>> to put the Org file on Worg, so anyone can update it and fix mistakes.
>
> after some playing with the Org manual in Org that Tom has been working
> on I am starting to think that there should be a way to define the same
> macro differently for different export backends.  That would be mainly
> so that you could have a macro expansion use export snippets tailored to
> that backend where (after stripping the export snippets) the expansion
> makes little or no sense in other backends.  What do you think?

This is already possible, as once explained by Nicolas on this ML:

  ╭────
  │ You can also have macros generating raw code geared towards LaTeX or HTML
  │ back-ends (through export-snippets). For example:
  │ 
  │ #+MACRO: my-mod @@e-latex:\something{$1}@@@@e-html:<div class="something">$1</div>@@
  │ 
  │ This is an example: {{{my-mod(text)}}}.
  ╰────

Best regards,
  Seb

-- 
Sebastien Vauban

Re: [Out-of-Thread] Re: [RFC] Org syntax (draft)

[zeltak]

[-- Attachment #1: Type: text/plain, Size: 528 bytes --]

Hi all

i just finished a great conversation on #org-mode with some great people.
they told me about this thread and the planned changes that may or may not
occur to the syntax and id like to just raise the newbee perspective.

I find the ability to add custom emphasise with custom faces invaluable. i
find it very easy to add and very useful for me since i use ALOT of color
highlights in my academic work.

i really hope that there is some way to leave the current mphasise with
custom faces options as is


thx alot guys

Z

[-- Attachment #2: Type: text/html, Size: 997 bytes --]

Re: [Out-of-Thread] Re: [RFC] Org syntax (draft)

[Carsten Dominik]

Hi Z,

can you show an example on how you use it?  Maybe we can find a better way.  Nicolas is right that portability is compromised by customizable emphasis.

- Carsten



On 18.3.2013, at 00:02, zeltak <zeltak@gmail.com> wrote:

> Hi all
> 
> i just finished a great conversation on #org-mode with some great people. they told me about this thread and the planned changes that may or may not occur to the syntax and id like to just raise the newbee perspective.
> 
> I find the ability to add custom emphasise with custom faces invaluable. i find it very easy to add and very useful for me since i use ALOT of color highlights in my academic work.
> 
> i really hope that there is some way to leave the current mphasise with custom faces options as is
> 
> 
> thx alot guys
> 
> Z

Re: [Out-of-Thread] Re: [RFC] Org syntax (draft)

[Thorsten Jolitz]

Carsten Dominik <carsten.dominik@gmail.com> writes:

> can you show an example on how you use it? Maybe we can find a better way.
> Nicolas is right that portability is compromised by customizable emphasis.

> On 18.3.2013, at 00:02, zeltak <zeltak@gmail.com> wrote:

>> I find the ability to add custom emphasise with custom faces invaluable. i
>> find it very easy to add and very useful for me since i use ALOT of color
>> highlights in my academic work.

Here is an excerpt from my .emacs, actually stolen from Fabrice Nielson
(http://www.mygooglest.com/fni/), that might give some hints how to do
interactive color highlighting without any changes to Org-mode:

,------------------------------------------------------------------------------
| ;; *** 14.13 (info "(emacs)Highlight Interactively")
| 
| ;; You can use `hi-lock-mode' to highlight words:
| ;;     `M-x hi-lock-mode RET'
| ;;     `C-x w h <match> RET hi-blue RET'
| ;; You can also write your settings to the buffer you're using with
| ;; `C-x w b', and read them back in again next time with `C-x w i'.
| 
| ;; TODO Have a look at http://www.emacswiki.org/emacs/color-moccur.el for
| ;; searching regexp in buffer
| 
| (GNUEmacs
|     ;; "identical token highlighting" commands
|     (when (try-require 'highlight)
| 
|         (defface hlt-1 '((t (:background "#FFFFA0"))) nil)
|         (defface hlt-2 '((t (:background "#A0FFA0"))) nil)
|         (defface hlt-3 '((t (:background "#A0FFFF"))) nil)
|         (defface hlt-4 '((t (:background "#FFA0FF"))) nil)
|         (defface hlt-5 '((t (:background "#FFA0A0"))) nil)
|         (defface hlt-6 '((t (:background "#FFFFA0"))) nil)
|         (defface hlt-7 '((t (:background "#A0FFA0"))) nil)
|         (defface hlt-8 '((t (:background "#A0FFFF"))) nil)
|         (defface hlt-9 '((t (:background "#FFA0FF"))) nil)
|         (defface hlt-10 '((t (:background "#FFA0A0"))) nil)
| 
|         (global-set-key (kbd "C-S-p") 'hlt-previous-highlight)
|         (global-set-key (kbd "C-S-n") 'hlt-next-highlight)
| 
|         (defun hlt-highlight-current-word ()
|           (interactive)
|           (let ((var_name (current-word t)))
|             (when var_name
|               (save-excursion
|                 (hlt-highlight-regexp-region
|                  (point-min)
|                  (point-max)
|                  (regexp-quote var_name))))))
| 
|         ;; emulation of Vim's `*' search
|         (global-set-key (kbd "C-*") 'hlt-highlight-current-word)
|         ))
| 
| ;; ;; bind the hi-lock commands to more finger-friendly sequences
| ;; (define-key hi-lock-map (kbd "C-z i") 'hi-lock-find-patterns)
| ;; (define-key hi-lock-map (kbd "C-z p") 'highlight-phrase)
| ;; (define-key hi-lock-map (kbd "C-z r") 'unhighlight-regexp)
| 
| ;; (define-key hi-lock-map (kbd "C-z h") 'highlight-regexp)
| ;; (define-key hi-lock-map (kbd "C-z C-h") 'highlight-lines-matching-regexp)
| ;; (define-key hi-lock-map (kbd "C-z b") 'hi-lock-write-interactive-patterns)
| 
| ;; Highlight based on regexps
| (global-set-key [M-f1] 'highlight-regexp)
| (global-set-key [M-f2] 'highlight-lines-matching-regexp)
| (global-set-key [M-f3] 'hi-lock-mode)
| (global-set-key [M-f4] 'hi-lock-write-interactive-patterns)
| 
| ;; highlight current symbol
| (when (try-require 'light-symbol)
|   (light-symbol-mode))
| 
| ;; highlight current symbol
| (setq highlight-symbol-idle-delay 0.5)
| (when (try-require 'highlight-symbol)
|   (highlight-symbol-mode))
`------------------------------------------------------------------------------

-- 
cheers,
Thorsten

Re: [Out-of-Thread] Re: [RFC] Org syntax (draft)

[zeltak]

[-- Attachment #1: Type: text/plain, Size: 3436 bytes --]

Dear Carsten,

Thank you for your quick reply. Let me start by first thanking you for your
great work on orgmode, I only recently discovered it (someone referred me
to your great talk on youtube) and it made me have the courage to start
learning emacs and use orgmode.

I  (actually me and several colleagues here at the school of public health
at Harvard) have been using for the past 4 years a note taking app called
notecase pro (http://www.notecasepro.com/) which is nice but not FLOSS and
lacking in other areas. I am a post doc who takes alot of notes (30-40)
daily which include images and color markings.

colors are especially important to us since we use them to
mark different commands, research areas, paths, comments and warning so
that we have a clear easy to remember color visual clue. We use 15-20 color
fg/bg commands. An example note could look like this:

http://i.imgur.com/Ncq6ozs.png

i am a complete new orgmode user but i am so impressed and overblown with
the features and definitely want to transition to use it full time and also
convince my colleagues to do so as well. with the help of the #irc channel
i was able to use the customizing features for the emphasize symbols to
achieve color support . the config now looks like this:

 '(org-emphasis-alist (quote (("@" (:foreground "#B40000" :background
"#FFDDDD" :weight bold) "" "") ("$" (:foreground "#FF0000") "" "") ("*"
bold "<b>" "</b>") ("/" italic "<i>" "</i>") ("_" underline "<span
style=\"text-decoration:underline;\">" "</span>") ("=" org-code "<code>"
"</code>" verbatim) ("~" org-verbatim "<code>" "</code>" verbatim) ("+"
(:strike-through t) "<del>" "</del>"))))

That would have worked for me but i understood that there are plans to
actually disable these customization's in the next version to allow
better portability.

If its not to hard It would be great to have a method similar to the
customizable
emphasis that lets a user define custom colors of FG/BG for inline viewing.
I am less concerned about exporting since at least for me i plan to do all
the editing/viewing inline inside emacs (though it would be nice of course
to be able to use it with mobile org and/or other mobile solutions for when
we do field work).

I will hold on with starting with the mammoth task of converting my
Notecase notes into orgmode until the issues is resolved, i assume i should
follow the mailing list to check on this?

Sorry for the long email and thank you so much again for all your work,
its truly fantastic

Z.












On Mon, Mar 18, 2013 at 2:08 AM, Carsten Dominik
<carsten.dominik@gmail.com>wrote:

> Hi Z,
>
> can you show an example on how you use it?  Maybe we can find a better
> way.  Nicolas is right that portability is compromised by customizable
> emphasis.
>
> - Carsten
>
>
>
> On 18.3.2013, at 00:02, zeltak <zeltak@gmail.com> wrote:
>
> > Hi all
> >
> > i just finished a great conversation on #org-mode with some great
> people. they told me about this thread and the planned changes that may or
> may not occur to the syntax and id like to just raise the newbee
> perspective.
> >
> > I find the ability to add custom emphasise with custom faces invaluable.
> i find it very easy to add and very useful for me since i use ALOT of color
> highlights in my academic work.
> >
> > i really hope that there is some way to leave the current mphasise with
> custom faces options as is
> >
> >
> > thx alot guys
> >
> > Z
>
>

[-- Attachment #2: Type: text/html, Size: 5533 bytes --]

Re: [Out-of-Thread] Re: [RFC] Org syntax (draft)

[W. Greenhouse]

zeltak <zeltak-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> writes:

> Dear Carsten,
>
> Thank you for your quick reply. Let me start by first thanking you
> for your great work on orgmode, I only recently discovered it
> (someone referred me to your great talk on youtube) and it made me
> have the courage to start learning emacs and use orgmode.

[snip]

>  '(org-emphasis-alist (quote (("@" (:foreground "#B40000" :background
> "#FFDDDD" :weight bold) "" "") ("$" (:foreground "#FF0000") "" "")
> ("*" bold "<b>" "</b>") ("/" italic "<i>" "</i>") ("_" underline "
> <span style=\"text-decoration:underline;\">" "</span>") ("=" org-code
> "<code>" "</code>" verbatim) ("~" org-verbatim "<code>" "</code>"
> verbatim) ("+" (:strike-through t) "<del>" "</del>")))) 
>
> That would have worked for me but i understood that there are plans
> to actually disable these customization's in the next version to
> allow better portability. 
>
> If its not to hard It would be great to have a method similar to the 
> customizable emphasis that lets a user define custom colors of FG/BG
> for inline viewing. I am less concerned about exporting since at
> least for me i plan to do all the editing/viewing inline inside emacs
> (though it would be nice of course to be able to use it with mobile
> org and/or other mobile solutions for when we do field work).
>
> I will hold on with starting with the mammoth task of converting my
> Notecase notes into orgmode until the issues is resolved, i assume i
> should follow the mailing list to check on this?
>
> Sorry for the long email and thank you so much again for all your
> work, its truly fantastic
>
> Z.

I want to add, as one of the people that helped Z with this on IRC--and
as another person that made the leap into Emacs largely because of Org,
about five years ago--that I think there are a lot of users like this:
people who value Org as a tool that is tightly integrated with the power
and flexibility of Emacs and Emacs Lisp, who aren't necessarily closely
following Org's upstream development or this list (I didn't follow it
closely until recently, either), and who are more concerned with keeping
Org flexible and customizable enough to exactly fit their needs within
Emacs than they are about making it available as yet another plain-text
markup language outside of Emacs.  Much as my Gnus is heavily customized
to my needs at this point, with Elisp-based features such as adaptive
scoring and virtual groups that other news and mail readers simply don't
have, I would never really dream of reproducing Org outside of Org.  And
there are plenty of things that I would never expect to work in an
external application or parser that speaks the Org format (dynamic
blocks that run Elisp, for example), which everyone nonetheless wants to
have.

Perhaps a compromise could be reached on variables such as
`org-emphasis-alist' and others possibly slated for the defconst
treatment: instead of doing that, let's consider keeping them
customizable but include the default values in the Org format
specification.  Org users who are never using Org outside of Emacs will
never see a problem using custom emphasis marks inside Emacs, unless Org
drops the feature.  For those who know that they want to use their Org
files with some external parser, we could have an `org-rfc-check'
function that warns about non-standard values of things like
org-emphasis-alist and offers to revert them to the defaults (which
would be the same as the values in the spec).

What do you think?  Is this a crazy scheme?

-- 
Regards,
WGG

Re: [Out-of-Thread] Re: [RFC] Org syntax (draft)

[Marcin Borkowski]

Dnia 2013-03-18, o godz. 15:21:54
wgreenhouse@riseup.net (W. Greenhouse) napisał(a):

> Perhaps a compromise could be reached on variables such as
> `org-emphasis-alist' and others possibly slated for the defconst
> treatment: instead of doing that, let's consider keeping them
> customizable but include the default values in the Org format
> specification.  Org users who are never using Org outside of Emacs
> will never see a problem using custom emphasis marks inside Emacs,
> unless Org drops the feature.  For those who know that they want to
> use their Org files with some external parser, we could have an
> `org-rfc-check' function that warns about non-standard values of
> things like org-emphasis-alist and offers to revert them to the
> defaults (which would be the same as the values in the spec).
> 
> What do you think?  Is this a crazy scheme?

Just Another N00b's 2cents: what about moving such things from core to
contrib?

Best,

-- 
Marcin Borkowski
http://octd.wmi.amu.edu.pl/en/Marcin_Borkowski
Adam Mickiewicz University

Re: [Out-of-Thread] Re: [RFC] Org syntax (draft)

[Carsten Dominik]

Hi everyone,

first a disclaimer:  Nicolas has thought about all things parser a lot more than I have, so he might disagree.  But here is my take on the issue.

First of all, we should not see Org as just another plain text markup language (no offense meant, I am sure, and none taken).  Because of its unique treatment of source code inclusion, source code markup, and executability, it is very much unique, I think.  Therefore it deserved a clean definition and a well defined parser, with a goal to make files function correctly in different peoples setup.  This is what Nicolas is trying to give to us - aside from turning an incredible parser/exporter mess into a sane system.  We used to have as many parsers as exporters, all slightly different, bugs having to be fixed in different ways in each parser.  That is gone, with a very limited amount of pain, and I am entirely amazed that this was even possible, let alone that someone had the stamina to do it and the patience it takes to put this, with the small limitations it brings, onto the community.

Now, I also agree that it is desirable that Org remains to be a hackable system with as much flexibility as we can allow, and we need to carefully consider where the line is.

For example, every users setup has some dependency of parser behavior on external variables: todo keywords.  The way this is fixed in the sense that we can guarantee a stable parsing of such a system is to tell people that including the TODO keyword definitions with a #+TODO line into the file.  So you can have a global setting in a global customization variable to make things easy for yourself and get good behavior in every new file you open, but you can stabilize a file for the parser with in-buffer settings when you need compatibility for other users.

There are a few exceptions that Nicolas has pointed out in this thread.  The COMMENT keyword is one.  We could define an in-buffer setting for it, or we could just fix it, the pain here would be minimal.

Another example is the emphasis stuff.  There are no in-buffer settings for it, and they would be pretty hard to make.

The reason why the emphasis regexp components were made configurable in the first place is because when the feature was introduced, I had no idea what would work, and I redesigned this part several times over.  Emphasis is a very heuristic system, the character that are allowed before and after the markers are necessarily a compromise, and we will always find people for who the chosen selection will not work.  That is why I would like to argue for keeping this part hackable, even if I agree that the official definition should be fixed.  Keeping this variable a customize variable invites changes also by people who do not really know what they are doing.  Turning it into a defvar or defconst and somewhere document how to hack around the restriction if you really need to sounds like a good solution for me.

Now to the discussion with Z about additional emphasis definitions which he/she uses for custom highlighting of stuff.  Right now this relies on modifying the emph-alist variable.  However, for the purpose of in-buffer only highlighting, it is not necessary to go through parser-sensitive code.  You can do this simply with additions to font-lock, for example using font-lock-add-keywords or something like this, see also Thorsten's post.  If someone wants, I can provide an example for Z's case, and we could encapsulate such behavior into a little library in contrib, to make it easy to configure such behavior.  Compromising the parser for this application is not necessary.

Nicolas, I would assume that your wish to disallow customization is focused on the regexp components, and the marker characters for emphasis, but not on the possibility to change the in-buffer face that is being used to highlight the stuff in the buffer?

- Carsten


On 18.3.2013, at 16:21, W. Greenhouse <wgreenhouse@riseup.net> wrote:

> zeltak <zeltak@gmail.com> writes:
> 
>> Dear Carsten,
>> 
>> Thank you for your quick reply. Let me start by first thanking you
>> for your great work on orgmode, I only recently discovered it
>> (someone referred me to your great talk on youtube) and it made me
>> have the courage to start learning emacs and use orgmode.
> 
> [snip]
> 
>>  '(org-emphasis-alist (quote (("@" (:foreground "#B40000" :background
>> "#FFDDDD" :weight bold) "" "") ("$" (:foreground "#FF0000") "" "")
>> ("*" bold "<b>" "</b>") ("/" italic "<i>" "</i>") ("_" underline "
>> <span style=\"text-decoration:underline;\">" "</span>") ("=" org-code
>> "<code>" "</code>" verbatim) ("~" org-verbatim "<code>" "</code>"
>> verbatim) ("+" (:strike-through t) "<del>" "</del>")))) 
>> 
>> That would have worked for me but i understood that there are plans
>> to actually disable these customization's in the next version to
>> allow better portability. 
>> 
>> If its not to hard It would be great to have a method similar to the 
>> customizable emphasis that lets a user define custom colors of FG/BG
>> for inline viewing. I am less concerned about exporting since at
>> least for me i plan to do all the editing/viewing inline inside emacs
>> (though it would be nice of course to be able to use it with mobile
>> org and/or other mobile solutions for when we do field work).
>> 
>> I will hold on with starting with the mammoth task of converting my
>> Notecase notes into orgmode until the issues is resolved, i assume i
>> should follow the mailing list to check on this?
>> 
>> Sorry for the long email and thank you so much again for all your
>> work, its truly fantastic
>> 
>> Z.
> 
> I want to add, as one of the people that helped Z with this on IRC--and
> as another person that made the leap into Emacs largely because of Org,
> about five years ago--that I think there are a lot of users like this:
> people who value Org as a tool that is tightly integrated with the power
> and flexibility of Emacs and Emacs Lisp, who aren't necessarily closely
> following Org's upstream development or this list (I didn't follow it
> closely until recently, either), and who are more concerned with keeping
> Org flexible and customizable enough to exactly fit their needs within
> Emacs than they are about making it available as yet another plain-text
> markup language outside of Emacs.  Much as my Gnus is heavily customized
> to my needs at this point, with Elisp-based features such as adaptive
> scoring and virtual groups that other news and mail readers simply don't
> have, I would never really dream of reproducing Org outside of Org.  And
> there are plenty of things that I would never expect to work in an
> external application or parser that speaks the Org format (dynamic
> blocks that run Elisp, for example), which everyone nonetheless wants to
> have.
> 
> Perhaps a compromise could be reached on variables such as
> `org-emphasis-alist' and others possibly slated for the defconst
> treatment: instead of doing that, let's consider keeping them
> customizable but include the default values in the Org format
> specification.  Org users who are never using Org outside of Emacs will
> never see a problem using custom emphasis marks inside Emacs, unless Org
> drops the feature.  For those who know that they want to use their Org
> files with some external parser, we could have an `org-rfc-check'
> function that warns about non-standard values of things like
> org-emphasis-alist and offers to revert them to the defaults (which
> would be the same as the values in the spec).
> 
> What do you think?  Is this a crazy scheme?
> 
> -- 
> Regards,
> WGG
> 
> 

Re: [Out-of-Thread] Re: [RFC] Org syntax (draft)

[Rasmus]

Carsten Dominik <carsten.dominik@gmail.com> writes:

> The reason why the emphasis regexp components were made configurable
> in the first place is because when the feature was introduced, I had
> no idea what would work, and I redesigned this part several times
> over.  Emphasis is a very heuristic system, the character that are
> allowed before and after the markers are necessarily a compromise, and
> we will always find people for who the chosen selection will not work.
> That is why I would like to argue for keeping this part hackable, even
> if I agree that the official definition should be fixed.  Keeping this
> variable a customize variable invites changes also by people who do
> not really know what they are doing.  Turning it into a defvar or
> defconst and somewhere document how to hack around the restriction if
> you really need to sounds like a good solution for me.

To some extend I disagree, I think.  Well, a contrib library is of
course OK, but I think it's not the right way to go about it. . .

Would it be possible to make it easier to make 'custom' highlights?
In a previous thread a [cite:key] syntax was suggested.  Perhaps, a
better way for custom emphasis would be [type:value] allowing for
custom functions for each type.  E.g. [TYPE:value] would run function
a function org-type-keys-TYPE which returns value formatted with a
special face.

Perhaps this is more cumbersome and perhaps it is no more 'structured'
than using customized emphases.

–Rasmus

-- 
El Rey ha muerto. ¡Larga vida al Rey!

Re: [Out-of-Thread] Re: [RFC] Org syntax (draft)

[Aaron Ecay]

Hi Carsten,

Thank you for your very insightful thoughts.  I would like to make one note.

2013ko martxoak 18an, Carsten Dominik-ek idatzi zuen:

> Now to the discussion with Z about additional emphasis definitions
> which he/she uses for custom highlighting of stuff.  Right now this
> relies on modifying the emph-alist variable.  However, for the purpose
> of in-buffer only highlighting, it is not necessary to go through
> parser-sensitive code.  You can do this simply with additions to
> font-lock, for example using font-lock-add-keywords or something like
> this, see also Thorsten's post.  If someone wants, I can provide an
> example for Z's case, and we could encapsulate such behavior into a
> little library in contrib, to make it easy to configure such behavior.
> Compromising the parser for this application is not necessary.

I use org to write articles which discuss words in foreign languages.
These need to be distinctively typeset (in italics), and sometimes need
to be set in a special font with coverage for exotic characters.  I
would find it very useful to be able to define special emphasis
environments for these words.  Using macros feels too much like writing
LaTeX (which I use org to avoid having to write directly, as much as
possible...)

I see the goal of the syntactic standardization as making it easier to
operate with non-emacs tools; as Nicolas said:

> My point of view is the following: Org (as a format) definition
> shouldn't depend on Emacs. It should be totally parseable by any
> language (which is not the case actually, since syntax relies on
> variables defined in Emacs). IOW, we should work to make it a real
> plain-text markup format.

Personally, I am OK with articles I have written for export never being
able to be read by non-emacs tools (as opposed to other uses of org as a
database/schedule/agenda, where the ability to access the information in
other programs/programming languages would be useful).  I sympathize
with the goal of making the format accessible to other tools, but I also
think the ability to have within emacs additional flexibility
wrt. formatting (for both display and export) is worth preserving.

-- 
Aaron Ecay

Re: [Out-of-Thread] Re: [RFC] Org syntax (draft)

[Nicolas Richard]

Carsten Dominik <carsten.dominik@gmail.com> writes:
> Another example is the emphasis stuff.  There are no in-buffer
> settings for it, and they would be pretty hard to make.

An in-buffer way of doing elisp is File Local Variables ; or is that not
appropriate ? Maybe the question I'm askign is : why were "#+KEYWORD"
lines favored over emacs' own mechanism of file local variables ? (I did
google on this a bit, but no success.)

-- 
Nico.

Re: [Out-of-Thread] Re: [RFC] Org syntax (draft)

[Nicolas Goaziou]

Hello,

Carsten Dominik <carsten.dominik@gmail.com> writes:

> First of all, we should not see Org as just another plain text markup
> language (no offense meant, I am sure, and none taken). Because of its
> unique treatment of source code inclusion, source code markup, and
> executability, it is very much unique, I think.

Indeed. Or, to put it differently, an external tool wishing to export an
/any/ Org document has to implement Babel in addition to the parser.

> For example, every users setup has some dependency of parser behavior
> on external variables: todo keywords. The way this is fixed in the
> sense that we can guarantee a stable parsing of such a system is to
> tell people that including the TODO keyword definitions with a #+TODO
> line into the file. So you can have a global setting in a global
> customization variable to make things easy for yourself and get good
> behavior in every new file you open, but you can stabilize a file for
> the parser with in-buffer settings when you need compatibility for
> other users.
>
> There are a few exceptions that Nicolas has pointed out in this
> thread. The COMMENT keyword is one. We could define an in-buffer
> setting for it, or we could just fix it, the pain here would be
> minimal.

I think we should only add in-buffer settings for important parts of Org
syntax (e.g. TODO keywords). A hard-coded value for small details like
COMMENT keyword or EFFORT property is good enough, IMO.

> The reason why the emphasis regexp components were made configurable
> in the first place is because when the feature was introduced, I had
> no idea what would work, and I redesigned this part several times
> over. Emphasis is a very heuristic system, the character that are
> allowed before and after the markers are necessarily a compromise, and
> we will always find people for who the chosen selection will not work.
>
> That is why I would like to argue for keeping this part hackable, even
> if I agree that the official definition should be fixed. Keeping this
> variable a customize variable invites changes also by people who do
> not really know what they are doing. Turning it into a defvar or
> defconst and somewhere document how to hack around the restriction if
> you really need to sounds like a good solution for me.

We can also use a very simple and tolerant regexp (e.g. =[^\000]+=), and
introduce a syntax to escape markers for fine-grained control.

> Nicolas, I would assume that your wish to disallow customization is
> focused on the regexp components, and the marker characters for
> emphasis, but not on the possibility to change the in-buffer face that
> is being used to highlight the stuff in the buffer?

That's correct.


Regards,

-- 
Nicolas Goaziou

Re: [Out-of-Thread] Re: [RFC] Org syntax (draft)

[Bastien]

Hi Nicolas,

Nicolas Goaziou <n.goaziou@gmail.com> writes:

> We can also use a very simple and tolerant regexp (e.g. =[^\000]+=), and
> introduce a syntax to escape markers for fine-grained control.

FWIW this looks like the correct approach to me.

-- 
 Bastien

Re: [Out-of-Thread] Re: [RFC] Org syntax (draft)

[Bastien]

Hi Carsten,

Carsten Dominik <carsten.dominik@gmail.com> writes:

> Keeping this variable a customize variable invites changes also by
> people who do not really know what they are doing.  Turning it into
> a defvar or defconst and somewhere document how to hack around the
> restriction if you really need to sounds like a good solution for
> me.

Agreed.  `org-emphasis-regexp-components' is now a defvar.

Also, I added `org-emphasis-alist' to the manual.

-- 
 Bastien

Re: [Out-of-Thread] Re: [RFC] Org syntax (draft)

[Samuel Wales]

On 3/18/13, W. Greenhouse <wgreenhouse@riseup.net> wrote:
> Perhaps a compromise could be reached on variables such as
> `org-emphasis-alist' and others possibly slated for the defconst
> treatment: instead of doing that, let's consider keeping them
> customizable but include the default values in the Org format
> specification.  Org users who are never using Org outside of Emacs will

I think this is a good idea.

===

Here is a brainstorm involving an old idea:

If the problem is external parsers vs. flexibility, I wonder if it
might be possible to have zeltak's cake and eat it too.  This applies
to more than just coloring in the buffer transiently.

In the case of color, $[face :foreground "red"]we can actually color
something in a way that external parsers can understand and export
backends can support$[end].  To some degree, at least.

We can do so $[face :foreground "yellow" :underline t]using a syntax
that will work for other features also and can be upgraded$[end].  Not
only for faces.  Just change "face" to the name of your feature.

Every feature that uses the $[] syntax starts out as experimental or
in the user's .emacs.  Then after we experiment with a feature and
decide it's worth being a default, we add it to the RFC and no longer
call it Emacs-only.

One big win is that Emacs and external parsers need only understand
the $[] notation, and do not need to come up with new parsing
techniques for syntax, because it is a Lisp lambda list.  Even the
user can do this.

All sorts of things then come for free.  For example, quoting,
escaping, nesting, pretty-printing, exporting, hiding, making the $[]
parts look human-friendly by making them invisible or collapsing them,
and so on.  They are all available for new features that need syntax,
including user-defined ones.

A possibility for what it's worth.

Samuel

-- 
The Kafka Pandemic: http://thekafkapandemic.blogspot.com

The disease DOES progress.  MANY people have died from it.  ANYBODY
can get it.  There is NO hope without action.  This means YOU.

Re: [Out-of-Thread] Re: [RFC] Org syntax (draft)

[zeltak]

[-- Attachment #1: Type: text/plain, Size: 549 bytes --]

Hi again

Thank you all for the responses. So as a neewb again, I dont really
understand fully all the technical specifications from the above posts,
what do you guys recommended i do if i want to start moving and using org
now full time in terms of color support? should i use the current emp.
method, use another method suggested above which will be 100% supported,
wait with my move to org (what is the rough time frame for the new
specification)?

I would greatly appreciate any help as i am very keen to move over to
orgmode full time

best

Z

[-- Attachment #2: Type: text/html, Size: 860 bytes --]