[ELPA] New package: bnf-mode

[ELPA] New package: bnf-mode

[Serghei]

[-- Attachment #1: Type: text/plain, Size: 647 bytes --]

Hello,

I created a major mode for editing BNF grammars [1] and would like to
add this
package to the ELPA [2].  Currently it provides basic syntax and
font-locking
for BNF files.  BNF notation is supported exactly form as it was first
announced
in the ALGOL 60 report.  EBNF and ABNF are not supported but in my plans
for the
near future.

I have no push access to ELPA repository so I heed a help.  Could you please
assist me with the following steps.

N.B. I've attached the package as a tar archive.


[1]: https://github.com/sergeyklay/bnf-mode
[2]: https://git.savannah.gnu.org/git/emacs/elpa.git


Have a nice day,

--
Serghei Iakovlev

[-- Attachment #2: bnf-mode-0.4.1.tar --]
[-- Type: application/x-tar, Size: 68096 bytes --]

Re: [ELPA] New package: bnf-mode

[Stefan Monnier]

> I created a major mode for editing BNF grammars [1] and would like to
> add this package to the ELPA [2].  Currently it provides basic syntax
> and font-locking for BNF files.  BNF notation is supported exactly
> form as it was first announced in the ALGOL 60 report.  EBNF and ABNF
> are not supported but in my plans for the near future.

In what kind of circumstances would this be used?
I've seen BNF-like syntax used by various tools (like bison and
friends, for example), but never with this particular syntax.

    ;; Basically `syntax-propertize-function' is a construct which belongs
    ;; to `font-lock'.

No.  It's used by font-lock (as well as various other things), but it
does not belong to it.

    ;; But correct indentation depends on
    ;; syntax properties of the text, and that should ideally be
    ;; independent of font-lock being activated or not.

Which is why syntax-propertize was created (as opposed to the previous
font-lock-syntactic-keywords, which *did* belong to font-lock and
suffered from the problem you describe).

    ;; For `bnf-mode', this means that with `font-lock' disabled, we wont
    ;; have our syntax properties set correctly, and indentation will
    ;; suffer.

That's not true.

    ;; To patch our way around this, we issue a `syntax-propertize' call
    ;; manually, `font-lock' enabled or not.
    (with-silent-modifications
      (when bnf-mode-algol-comments-style
        (funcall syntax-propertize-function (point-min) (point-max))))
        
Don't do that: syntax-propertize-function will be called when needed
(e.g. by indent-according-to-mode).


        Stefan

Fwd: [ELPA] New package: bnf-mode

[Serghei Iakovlev]

[-- Attachment #1: Type: text/plain, Size: 2908 bytes --]

---------- Forwarded message ---------
From: Serghei Iakovlev <sadhooklay@gmail.com>
Date: Sun, May 5, 2019 at 5:44 PM
Subject: Re: [ELPA] New package: bnf-mode
To: Stefan Monnier <monnier@iro.umontreal.ca>


Stefan Monnier <monnier@iro.umontreal.ca> writes:

Hello Stefan,

Thank you for the review.  I've removed wrongly used (and documented)
`syntax-propertize-function' funcall.  Thank you for clarifying it
out.  As for the question: BNF is a notation technique for context
free grammars, often used to describe the syntax of languages used
in computing, such as programming languages, document formats,
instruction sets and communication protocols.

And I have to say than Bison is not BNF.  Probable you're meant yacc,
which is from 197x.  However BNF is designed (in 1959) to create
formal grammars in a formal language, what is not a subject area for
yacc or bison.  Yes, yacc, antlr, bison, META II, etc.  All they use
a dialect slightly reminiscent of BNF, but the do not use pure BNF
grammar.  In addition, they don't use a formal language.

I've updated the code in the repository but still not sure what
exactly should I attach on this reply.

Best regards,
Serghey

>> I created a major mode for editing BNF grammars [1] and would like to
>> add this package to the ELPA [2].  Currently it provides basic syntax
>> and font-locking for BNF files.  BNF notation is supported exactly
>> form as it was first announced in the ALGOL 60 report.  EBNF and ABNF
>> are not supported but in my plans for the near future.
>
> In what kind of circumstances would this be used?
> I've seen BNF-like syntax used by various tools (like bison and
> friends, for example), but never with this particular syntax.
>
>     ;; Basically `syntax-propertize-function' is a construct which belongs
>     ;; to `font-lock'.
>
> No.  It's used by font-lock (as well as various other things), but it
> does not belong to it.
>
>     ;; But correct indentation depends on
>     ;; syntax properties of the text, and that should ideally be
>     ;; independent of font-lock being activated or not.
>
> Which is why syntax-propertize was created (as opposed to the previous
> font-lock-syntactic-keywords, which *did* belong to font-lock and
> suffered from the problem you describe).
>
>     ;; For `bnf-mode', this means that with `font-lock' disabled, we wont
>     ;; have our syntax properties set correctly, and indentation will
>     ;; suffer.
>
> That's not true.
>
>     ;; To patch our way around this, we issue a `syntax-propertize' call
>     ;; manually, `font-lock' enabled or not.
>     (with-silent-modifications
>       (when bnf-mode-algol-comments-style
>         (funcall syntax-propertize-function (point-min) (point-max))))
>
> Don't do that: syntax-propertize-function will be called when needed
> (e.g. by indent-according-to-mode).
>
>
>         Stefan
>
>
<#secure method=pgpmime mode=encrypt>

[-- Attachment #2: Type: text/html, Size: 3888 bytes --]

Re: [ELPA] New package: bnf-mode

[Serghei Iakovlev]

Hello Stefan,

Thank you for the review.  I've removed wrongly used (and documented)
`syntax-propertize-function' funcall.  Thank you for clarifying it
out.  As for the question: BNF is a notation technique for context
free grammars, often used to describe the syntax of languages used
in computing, such as programming languages, document formats,
instruction sets and communication protocols.

And I have to say than Bison is not BNF.  Probable you're meant yacc,
which is from 197x.  However BNF is designed (in 1959) to create
formal grammars in a formal language, what is not a subject area for
yacc or bison.  Yes, yacc, antlr, bison, META II, etc.  All they use
a dialect slightly reminiscent of BNF, but the do not use pure BNF
grammar.  In addition, they don't use a formal language.

I've updated the code in the repository but still not sure what
exactly should I attach on this reply.

Best regards,
Serghey

Stefan Monnier <monnier@iro.umontreal.ca> writes:
>> I created a major mode for editing BNF grammars [1] and would like to
>> add this package to the ELPA [2].  Currently it provides basic syntax
>> and font-locking for BNF files.  BNF notation is supported exactly
>> form as it was first announced in the ALGOL 60 report.  EBNF and ABNF
>> are not supported but in my plans for the near future.
>
> In what kind of circumstances would this be used?
> I've seen BNF-like syntax used by various tools (like bison and
> friends, for example), but never with this particular syntax.
>
>     ;; Basically `syntax-propertize-function' is a construct which belongs
>     ;; to `font-lock'.
>
> No.  It's used by font-lock (as well as various other things), but it
> does not belong to it.
>
>     ;; But correct indentation depends on
>     ;; syntax properties of the text, and that should ideally be
>     ;; independent of font-lock being activated or not.
>
> Which is why syntax-propertize was created (as opposed to the previous
> font-lock-syntactic-keywords, which *did* belong to font-lock and
> suffered from the problem you describe).
>
>     ;; For `bnf-mode', this means that with `font-lock' disabled, we wont
>     ;; have our syntax properties set correctly, and indentation will
>     ;; suffer.
>
> That's not true.
>
>     ;; To patch our way around this, we issue a `syntax-propertize' call
>     ;; manually, `font-lock' enabled or not.
>     (with-silent-modifications
>       (when bnf-mode-algol-comments-style
>         (funcall syntax-propertize-function (point-min) (point-max))))
>         
> Don't do that: syntax-propertize-function will be called when needed
> (e.g. by indent-according-to-mode).
>
>
>         Stefan
>
>
<#secure method=pgpmime mode=encrypt>

Re: [ELPA] New package: bnf-mode

[Serghei Iakovlev]

I apologize for the possible duplication of this reply.  I sent it by
mistake to my personal Stefan address.

Hello Stefan,

Thank you for the review.  I've removed wrongly used (and documented)
`syntax-propertize-function' funcall.  Thank you for clarifying it
out.  As for the question: BNF is a notation technique for context
free grammars, often used to describe the syntax of languages used
in computing, such as programming languages, document formats,
instruction sets and communication protocols.

And I have to say than Bison is not BNF.  Probable you're meant yacc,
which is from 197x.  However BNF is designed (in 1959) to create
formal grammars in a formal language, what is not a subject area for
yacc or bison.  Yes, yacc, antlr, bison, META II, etc.  All they use
a dialect slightly reminiscent of BNF, but the do not use pure BNF
grammar.  In addition, they don't use a formal language.

I've updated the code in the repository but still not sure what
exactly should I attach on this reply.

Best regards,
Serghei

Stefan Monnier <monnier@iro.umontreal.ca> writes:
>> I created a major mode for editing BNF grammars [1] and would like to
>> add this package to the ELPA [2].  Currently it provides basic syntax
>> and font-locking for BNF files.  BNF notation is supported exactly
>> form as it was first announced in the ALGOL 60 report.  EBNF and ABNF
>> are not supported but in my plans for the near future.
>
> In what kind of circumstances would this be used?
> I've seen BNF-like syntax used by various tools (like bison and
> friends, for example), but never with this particular syntax.
>
>     ;; Basically `syntax-propertize-function' is a construct which belongs
>     ;; to `font-lock'.
>
> No.  It's used by font-lock (as well as various other things), but it
> does not belong to it.
>
>     ;; But correct indentation depends on
>     ;; syntax properties of the text, and that should ideally be
>     ;; independent of font-lock being activated or not.
>
> Which is why syntax-propertize was created (as opposed to the previous
> font-lock-syntactic-keywords, which *did* belong to font-lock and
> suffered from the problem you describe).
>
>     ;; For `bnf-mode', this means that with `font-lock' disabled, we wont
>     ;; have our syntax properties set correctly, and indentation will
>     ;; suffer.
>
> That's not true.
>
>     ;; To patch our way around this, we issue a `syntax-propertize' call
>     ;; manually, `font-lock' enabled or not.
>     (with-silent-modifications
>       (when bnf-mode-algol-comments-style
>         (funcall syntax-propertize-function (point-min) (point-max))))
>         
> Don't do that: syntax-propertize-function will be called when needed
> (e.g. by indent-according-to-mode).
>
>
>         Stefan
>
>

Re: [ELPA] New package: bnf-mode

[Stefan Monnier]

> And I have to say than Bison is not BNF.

I understand that, but BNF is a general term referring to an
*abstract* (meta)syntax and every textbook presents it with slightly
different syntactic conventions.

I use BNF regularly, but never using the specific syntax your
mode supports.  Hence my question:

>> In what kind of circumstances would this be used?

E.g. what have you used it for?


        Stefan

Re: [ELPA] New package: bnf-mode

[John Yates]

[-- Attachment #1: Type: text/plain, Size: 715 bytes --]

Emacs comes with ebnf2ps.el.  (I remember using it about 10 years ago.)
How does the syntax supported by this proposed mode relate?

/john

On Sun, May 5, 2019 at 12:13 PM Stefan Monnier <monnier@iro.umontreal.ca>
wrote:

> > And I have to say than Bison is not BNF.
>
> I understand that, but BNF is a general term referring to an
> *abstract* (meta)syntax and every textbook presents it with slightly
> different syntactic conventions.
>
> I use BNF regularly, but never using the specific syntax your
> mode supports.  Hence my question:
>
> >> In what kind of circumstances would this be used?
>
> E.g. what have you used it for?
>
>
>         Stefan
>
>
>

-- 
John Yates
505 Tremont St, #803
Boston, MA 02116

[-- Attachment #2: Type: text/html, Size: 1557 bytes --]

RE: [ELPA] New package: bnf-mode

[Drew Adams]

> Emacs comes with ebnf2ps.el.  (I remember using it
> about 10 years ago.)  How does the syntax supported
> by this proposed mode relate?

And ebnf2ps.el allows for several different BNF syntaxes.
You can even define your own syntax variety, IIRC.  It's
quite flexible.

No idea how it relates to the proposed `bnf-mode'.

ebnf2ps.el was written by Vinicius Jose Latorre, who also
wrote the psprint.el and printing.el packages.

https://www.emacswiki.org/emacs/EbnfToPsPackage

https://www.emacswiki.org/emacs/ViniciusJoseLatorre

https://www.emacswiki.org/emacs/PsPrintPackage

https://www.emacswiki.org/emacs/PrintingPackage

RE: [ELPA] New package: bnf-mode

[Drew Adams]

Oh, and there's also this simple bnf-mode:

https://www.emacswiki.org/emacs/BNFMode

RE: [ELPA] New package: bnf-mode

[Serghei Iakovlev]

[-- Attachment #1: Type: text/plain, Size: 540 bytes --]

No, Emacs Wiki has a wrong mode. For more see my comments here:
https://www.reddit.com/r/emacs/comments/b1lyyf/gnu_emacs_major_mode_for_editing_bnf_grammars/
<https://www.reddit.com/r/emacs/comments/b1lyyf/gnu_emacs_major_mode_for_editing_bnf_grammars/?utm_source=share&utm_medium=ios_app>

And no guys, ebnf2ps does not provide syntax locking for BNF grammar

-- 
Serghei

On May 6, 2019 at 3:48:43 AM, Drew Adams (drew.adams@oracle.com) wrote:

> Oh, and there's also this simple bnf-mode:
>
> https://www.emacswiki.org/emacs/BNFMode
>
>

[-- Attachment #2: Type: text/html, Size: 957 bytes --]

Re: [ELPA] New package: bnf-mode

[Stefan Monnier]

> No, Emacs Wiki has a wrong mode. For more see my comments here:
> https://www.reddit.com/r/emacs/comments/b1lyyf/gnu_emacs_major_mode_for_editing_bnf_grammars/
> <https://www.reddit.com/r/emacs/comments/b1lyyf/gnu_emacs_major_mode_for_editing_bnf_grammars/?utm_source=share&utm_medium=ios_app>
>
> And no guys, ebnf2ps does not provide syntax locking for BNF grammar

ebnf2ps doesn't provide a mode at all.  The mention of ebnf wasn't meant
as "an alternative" to your mode, but rather I think as a suggestion
for why your mode might be useful (i.e. as a complement to ebnf2ps).

While looking at adding it to GNU ELPA I saw that you apparently haven't
yet signed the corresponding paperwork.  IOW the next step is to fill
the form below and email it to the FSF as instructed so they can send
you the relevant paperwork to sign.


        Stefan "still wondering what you use your mode for"


Please email the following information to assign@gnu.org, and we
will send you the assignment form for your past and future changes.

Please use your full legal name (in ASCII characters) as the subject
line of the message.
----------------------------------------------------------------------
REQUEST: SEND FORM FOR PAST AND FUTURE CHANGES

[What is the name of the program or package you're contributing to?]
Emacs

[Did you copy any files or text written by someone else in these changes?
Even if that material is free software, we need to know about it.]


[Do you have an employer who might have a basis to claim to own
your changes?  Do you attend a school which might make such a claim?]


[For the copyright registration, what country are you a citizen of?]


[What year were you born?]


[Please write your email address here.]


[Please write your postal address here.]





[Which files have you changed so far, and which new files have you written
so far?]

Re: [ELPA] New package: bnf-mode

[Serghei Iakovlev]

Stefan Monnier <monnier@IRO.UMontreal.CA> writes:

>> No, Emacs Wiki has a wrong mode. For more see my comments here:
>> https://www.reddit.com/r/emacs/comments/b1lyyf/gnu_emacs_major_mode_for_editing_bnf_grammars/
>> <https://www.reddit.com/r/emacs/comments/b1lyyf/gnu_emacs_major_mode_for_editing_bnf_grammars/?utm_source=share&utm_medium=ios_app>
>>
>> And no guys, ebnf2ps does not provide syntax locking for BNF grammar
>
> ebnf2ps doesn't provide a mode at all.  The mention of ebnf wasn't meant
> as "an alternative" to your mode, but rather I think as a suggestion
> for why your mode might be useful (i.e. as a complement to ebnf2ps).
>
> While looking at adding it to GNU ELPA I saw that you apparently haven't
> yet signed the corresponding paperwork.  IOW the next step is to fill
> the form below and email it to the FSF as instructed so they can send
> you the relevant paperwork to sign.
>
>
>         Stefan "still wondering what you use your mode for"

I use this mode for various experiments with languages.  In particular,
I do research in the field of creating a programming language called
Zephir.  I needed to get a formal description of the language and
this is exactly where I used the BNF.  This description (I prefer
"specification" word) used during systems analysis and requirements
analysis.

>
> Please email the following information to assign@gnu.org, and we
> will send you the assignment form for your past and future changes.
>
> Please use your full legal name (in ASCII characters) as the subject
> line of the message.
> ----------------------------------------------------------------------
> REQUEST: SEND FORM FOR PAST AND FUTURE CHANGES
>
> [What is the name of the program or package you're contributing to?]
> Emacs
>
> [Did you copy any files or text written by someone else in these changes?
> Even if that material is free software, we need to know about it.]
>
>
> [Do you have an employer who might have a basis to claim to own
> your changes?  Do you attend a school which might make such a claim?]
>
>
> [For the copyright registration, what country are you a citizen of?]
>
>
> [What year were you born?]
>
>
> [Please write your email address here.]
>
>
> [Please write your postal address here.]
>
>
>
>
>
> [Which files have you changed so far, and which new files have you written
> so far?]

I'll do this ASAP.

Thank you,
Serghei

Re: [ELPA] New package: bnf-mode

[Stefan Monnier]

> I use this mode for various experiments with languages.  In particular,
> I do research in the field of creating a programming language called
> Zephir.  I needed to get a formal description of the language and
> this is exactly where I used the BNF.  This description (I prefer
> "specification" word) used during systems analysis and requirements
> analysis.

So you don't use it together with a tool to process it?
IOW you opted to use that particular syntax but could have used "any other"?

Have you used the "; comment ... ;" commenting style?

Any plan to add support for the syntax supported by ebnf2ps, or the
syntax used in RFCs in the short term?  That would make it a lot more
useful, I think.

Also, if you need help adding SMIE support to it, feel free to ask.
It should be pretty easy to get something going.

>> Please email the following information to assign@gnu.org, and we
>> will send you the assignment form for your past and future changes.
> I'll do this ASAP.

Great, thanks.  Let us known when you get confirmation that the FSF
received your signed paperwork,


        Stefan

Re: [ELPA] New package: bnf-mode

[Serghei Iakovlev]

I apologize for the duplicate, the previous mail was sent without CC
to mailing list.  I'm in Gnus studying stage.  I'm sorry.

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> I use this mode for various experiments with languages.  In particular,
>> I do research in the field of creating a programming language called
>> Zephir.  I needed to get a formal description of the language and
>> this is exactly where I used the BNF.  This description (I prefer
>> "specification" word) used during systems analysis and requirements
>> analysis.
>
> So you don't use it together with a tool to process it?
> IOW you opted to use that particular syntax but could have used "any other"?

No, right now I don't use any tool with BNF together to process it.
Some things are not clear to me yet and I am in the way of
researching some convenient tools.  As for grammar - I use one more
step to convert final BNF grammar to yacc/bison syntax.  So as I said
before, I use BNF grammars mostly in system analysis and
analysis of requirements.

> Have you used the "; comment ... ;" commenting style?

As a far as I understood, this syntax is convenient for declarations
ending with a semicolon.  An ALGOL example:

--
procedure euler (fct,sum,eps,tim); value eps,tim; integer tim;
real procedure fct; real sum,eps;
comment euler computes the sum of fct(i) for i from zero up to
infinity by means of a suitably refined euler transformation;
--

Here "euler computes the sum of fct(i) for i from zero up to
infinity by means of a suitably refined euler transformation" is
a comment.

IOW you can't make a comment anywhere, you need a starting
semicolon.  However, if you choose a different commenting style
(just a semicolon), then you'll be forced to use quotes for
terminals:

--
<function-label> <identifier> "(" ")" "->" <return-type> ";"
--

Thus, my personal choice is to use "orthodox" (ALGOL) comments style
to keep grammar formal and short:

--
<function-label> <identifier> ( ) -> <return-type> ;
comment ... ;
--

The original BNF didn't even have a restriction in use of space
symbols, so that I can even use something like this:

--
<any sequence of symbols not containing ` or ' >
--

> Any plan to add support for the syntax supported by ebnf2ps, or the
> syntax used in RFCs in the short term?  That would make it a lot more
> useful, I think.

My next planned steps are to provide ABNF used in RFCs and EBNF
described in ISO/IEC 14977.  Honestly, I am new to ebnf2ps and so far
I have no clear understanding of possible integrations. Could you
elaborate on that?

> Also, if you need help adding SMIE support to it, feel free to ask.
> It should be pretty easy to get something going.

Yes, I definitely will need help.  Thank you.  Could you explain
any possible use case for SMIE?

>>> Please email the following information to assign@gnu.org, and we
>>> will send you the assignment form for your past and future changes.
>> I'll do this ASAP.
>
> Great, thanks.  Let us known when you get confirmation that the FSF
> received your signed paperwork,
>

There is no answer yet so I suspect that I could fill in something
wrong in the form.  But let's wait.

-- 
Serghei

Re: [ELPA] New package: bnf-mode

[Stefan Monnier]

> --
> <function-label> <identifier> ( ) -> <return-type> ;
> comment ... ;
> --

Makes sense (as long as your language doesn't have a construction where
";" and "comment" follow each other as terminals ;-).

> Yes, I definitely will need help.  Thank you.  Could you explain
> any possible use case for SMIE?

It's a package for (navigation and) indentation.


        Stefan

Re: [ELPA] New package: bnf-mode

[Serghei Iakovlev]

Stefan Monnier <monnier@IRO.UMontreal.CA> writes:

>> --
>> <function-label> <identifier> ( ) -> <return-type> ;
>> comment ... ;
>> --
>
> Makes sense (as long as your language doesn't have a construction where
> ";" and "comment" follow each other as terminals ;-).

Yes, it is inconvenient and probably was developed mainly for
ALGOL.  That's why I rarely use comments.  However, in my opinion,
this is better than quoting terminals.

>> Yes, I definitely will need help.  Thank you.  Could you explain
>> any possible use case for SMIE?
>
> It's a package for (navigation and) indentation.

I know that.  However at the moment I don't see possible indentation,
except for the following two use cases:

--
<nonterminal> ::= ...
        <foo> ::= ...
    <foo-bar> ::= ...
--
Figure #1: Align by "::="

and

--
<nonterminal> := A | B | C 
               | D | E | F
               | G | ...
--
Figure #2: Align alternatives

So, honestly I don't know how SMIE can help here.

-- 
Serghei

Re: [ELPA] New package: bnf-mode

[Stefan Monnier]

> --
> <nonterminal> ::= ...
>         <foo> ::= ...
>     <foo-bar> ::= ...
> --
> Figure #1: Align by "::="
>
> and
>
> --
> <nonterminal> := A | B | C 
>                | D | E | F
>                | G | ...
> --
> Figure #2: Align alternatives

I guess for pure BNF there's no that much to do, indeed.
Tho there's still also

    <nonterminal> := A | B | C 
                   | D1 D2 D3
                     D4 D5 D6
                   | G | ...

once you go to EBNF there can be some more interesting cases.

Anyway, you can all return to your regular Elisping,


        Stefan

Re: [ELPA] New package: bnf-mode

[Serghei Iakovlev]

Stefan Monnier <monnier@IRO.UMontreal.CA> writes:

>> No, Emacs Wiki has a wrong mode. For more see my comments here:
>> https://www.reddit.com/r/emacs/comments/b1lyyf/gnu_emacs_major_mode_for_editing_bnf_grammars/
>> <https://www.reddit.com/r/emacs/comments/b1lyyf/gnu_emacs_major_mode_for_editing_bnf_grammars/?utm_source=share&utm_medium=ios_app>
>>
>> And no guys, ebnf2ps does not provide syntax locking for BNF grammar
>
> ebnf2ps doesn't provide a mode at all.  The mention of ebnf wasn't meant
> as "an alternative" to your mode, but rather I think as a suggestion
> for why your mode might be useful (i.e. as a complement to ebnf2ps).
>
> While looking at adding it to GNU ELPA I saw that you apparently haven't
> yet signed the corresponding paperwork.  IOW the next step is to fill
> the form below and email it to the FSF as instructed so they can send
> you the relevant paperwork to sign.

Done.  I've signed the form and received signed form from the FSF.
What are next steps?

-- 
Serghei