org-mode linter for BNF? (was Re: Concerns about community contributor support)

org-mode linter for BNF? (was Re: Concerns about community contributor support)

[trx2358-gitter]

[-- Attachment #1: Type: text/plain, Size: 4354 bytes --]

------------------------------------------------------------------------

Hi there,

I'm new to this mailing list and I'm sure I'm not caught up on discussions on using a BNF for org-mode.
I have seen Gustav Wilkstroem's warning about the difficulties involved.  I wonder if I might propose
(what seems to me) a novel approach.  Maybe instead of using the BNF inside org-mode, the BNF can be used
for a companion piece of software which runs alongside org-mode.  As a sort of "linter" for org-mode.

Taking code editors as an example.  During editing the code need not conform to the spec, but once saved
and handed to a compiler, it must or else the code will break.  Tools such as [[https://emacs-lsp.github.io/lsp-mode/][lsp]]
run a companion process alongside the editor which continuously check for conformity to the spec (and more!)
and report back any issues they've found.

If we had an lsp server for org-mode, then third party tools could be written to what *that* accepts rather than
what ~org-element.el~ accepts.  Tools could specify that only files which pass "linting" can be expected to work
100%.  Of course other editing tools could also incorporate the lsp server to have linting available to them.

In this way the specification of what's "pure org-mode grammar" is separate from "what's possible in emacs with
org-mode".  Obviously we wouldn't want the two to stray too far apart, but the added flexibility seems like it might
be a boon to the community.  It would be good though if such a tool were "part of the org-mode family" rather than
developed independently.

What do people think of the idea?

-Doug

FWIW, on a somewhat related note, I've started a github repository of org-mode snippets with the thought that these
could be used as examples to discuss what should and should not be part of "pure org-mode grammar".
([[https://github.com/gitonthescene/org-mode-samples][org-mode-samples]])

As such I'm actively seeking input since discussions with myself seem to mostly go in circles.  :-)

Also, I've laid out this same argument with slightly different wording in that project
[[https://github.com/gitonthescene/org-mode-samples/discussions/2][here]].

Tom Gillespie <tgbugs@gmail.com> writes:

>/Hi Tim, David, and Gustav,/

Hi

>/I am fairly certain that with only a few exceptions it is possible/
>/to specify a context free grammar for org syntax, followed by a second/
>/pass that deals specifically with markup and a few other forms,/
>/notably the reassembly of things like plain lists. The fact that this/
>/is possible because most org constructs are line oriented./

What do you think about having multiple Org grammars that parse specific
parts of an Org file and treat the rest as text blobs?  The idea is that
small tools (on smartphones) could concentrate on (say) gathering and
using the info related to one of:
+ task management
+ tangled code
+ Org file options
+ etc.

>/Just a note that the linked parser.rkt [0] is indeed a BNF describing org/
>/syntax in the same style as a bison/yacc grammar. One of the reasons/
>/why I set out to work on this was precisely so that there could be a/
>/reference that could be consulted by the community when questions/
>/about extended org come up./

How complete do you feel this grammar is?

>/In all my work on the grammar I have found maybe 2 or 3 places where/
>/the grammar could be "extended" but it isn't so much extended as it is/
>/regularized, where some parts of org already parse a more complex/
>/grammar while other very similar parts choose not to. Overall the cost/
>/of not parsing certain forms in certain situations adds complexity/
>/rather than reducing it./

Hmm.

>/Overcoming this is why I started working on the grammar, because/
>/in the absence of a formal spec for what org should do, it is very hard/
>/to make changes to what it is currently doing without having nasty/
>/side effects./

Thanks for the effort! This could lead to Org developments on non-Enacs
platforms (smartphones & tablets)

>/0. https://github.com/tgbugs/laundry/blob/next/laundry/parser.rkt
<https://github.com/tgbugs/laundry/blob/next/laundry/parser.rkt> note/
>/the upcoming path change (which I will note in the original thread when/
>/it happens)./

I'll see if I can look at this -- it's been decades since I played with
grammars. 

-- 
David Masterson



[-- Attachment #2: Type: text/html, Size: 5040 bytes --]

Re: org-mode linter for BNF? (was Re: Concerns about community contributor support)

[trx2358-gitter]

[-- Attachment #1: Type: text/plain, Size: 4811 bytes --]

I can even imagine some productive interplay between a linter and
org-mode itself.  For instance, the linter could be used for deprecating
features in org-mode which are no longer maintainable.  I'm sure we
could come up with other examples.

On 7/11/21 2:40 PM, trx2358-gitter@yahoo.com wrote:
> ------------------------------------------------------------------------
> Hi there,
>
> I'm new to this mailing list and I'm sure I'm not caught up on discussions on using a BNF for org-mode.
> I have seen Gustav Wilkstroem's warning about the difficulties involved.  I wonder if I might propose
> (what seems to me) a novel approach.  Maybe instead of using the BNF inside org-mode, the BNF can be used
> for a companion piece of software which runs alongside org-mode.  As a sort of "linter" for org-mode.
>
> Taking code editors as an example.  During editing the code need not conform to the spec, but once saved
> and handed to a compiler, it must or else the code will break.  Tools such as [[https://emacs-lsp.github.io/lsp-mode/][lsp]]
> run a companion process alongside the editor which continuously check for conformity to the spec (and more!)
> and report back any issues they've found.
>
> If we had an lsp server for org-mode, then third party tools could be written to what *that* accepts rather than
> what ~org-element.el~ accepts.  Tools could specify that only files which pass "linting" can be expected to work
> 100%.  Of course other editing tools could also incorporate the lsp server to have linting available to them.
>
> In this way the specification of what's "pure org-mode grammar" is separate from "what's possible in emacs with
> org-mode".  Obviously we wouldn't want the two to stray too far apart, but the added flexibility seems like it might
> be a boon to the community.  It would be good though if such a tool were "part of the org-mode family" rather than
> developed independently.
>
> What do people think of the idea?
>
> -Doug
>
> FWIW, on a somewhat related note, I've started a github repository of org-mode snippets with the thought that these
> could be used as examples to discuss what should and should not be part of "pure org-mode grammar".
> ([[https://github.com/gitonthescene/org-mode-samples][org-mode-samples]])
>
> As such I'm actively seeking input since discussions with myself seem to mostly go in circles.  :-)
>
> Also, I've laid out this same argument with slightly different wording in that project
> [[https://github.com/gitonthescene/org-mode-samples/discussions/2][here]].
>
> Tom Gillespie <tgbugs@gmail.com> writes:
>
> >/Hi Tim, David, and Gustav,/
>
> Hi
>
> >/I am fairly certain that with only a few exceptions it is possible/
> >/to specify a context free grammar for org syntax, followed by a second/
> >/pass that deals specifically with markup and a few other forms,/
> >/notably the reassembly of things like plain lists. The fact that this/
> >/is possible because most org constructs are line oriented./
>
> What do you think about having multiple Org grammars that parse specific
> parts of an Org file and treat the rest as text blobs?  The idea is that
> small tools (on smartphones) could concentrate on (say) gathering and
> using the info related to one of:
> + task management
> + tangled code
> + Org file options
> + etc.
>
> >/Just a note that the linked parser.rkt [0] is indeed a BNF describing org/
> >/syntax in the same style as a bison/yacc grammar. One of the reasons/
> >/why I set out to work on this was precisely so that there could be a/
> >/reference that could be consulted by the community when questions/
> >/about extended org come up./
>
> How complete do you feel this grammar is?
>
> >/In all my work on the grammar I have found maybe 2 or 3 places where/
> >/the grammar could be "extended" but it isn't so much extended as it is/
> >/regularized, where some parts of org already parse a more complex/
> >/grammar while other very similar parts choose not to. Overall the cost/
> >/of not parsing certain forms in certain situations adds complexity/
> >/rather than reducing it./
>
> Hmm.
>
> >/Overcoming this is why I started working on the grammar, because/
> >/in the absence of a formal spec for what org should do, it is very hard/
> >/to make changes to what it is currently doing without having nasty/
> >/side effects./
>
> Thanks for the effort! This could lead to Org developments on non-Enacs
> platforms (smartphones & tablets)
>
> >/0. https://github.com/tgbugs/laundry/blob/next/laundry/parser.rkt
> <https://github.com/tgbugs/laundry/blob/next/laundry/parser.rkt> note/
> >/the upcoming path change (which I will note in the original thread when/
> >/it happens)./
>
> I'll see if I can look at this -- it's been decades since I played with
> grammars. 
>
> -- 
> David Masterson
>
>

[-- Attachment #2: Type: text/html, Size: 5810 bytes --]