bug#39663: 27.1; ebnf2ps bugs

bug#39663: 27.1; ebnf2ps bugs

[Mattias Engdegård]

A few bugs and other things noticed in ebnf2ps:

* The code resides in lisp/progmodes but isn't actually a mode at all: it converts BNF into railway diagrams in Postscript. It looks a bit out of place. What about moving it to ELPA?

* Prose values in ABNF don't seem to work. Guilty line:

	    (or (looking-at ebnf-abn-non-terminal-letter-chars)

where 'ebnf-abn-non-terminal-letter-chars' is not a regexp but a skip-set used in skip-chars-{forward,backward}. This was found by a relint scan.

* Despite a lot of effort been spent on making it work with Latin-1, I get UTF-8 mojibake in the PostScript output when using chars in the U+00A0..U+00FF range in the rules, eg in nonterminals. Unibyte/multibyte confusion, perhaps?

* The 'ebnf-range-regexp' function is misnamed -- it doesn't return a regexp -- and triggers relint's bad-regexp-o-meter. Moreover, it is only there to compensate for a bug in Emacs 20, long ago fixed. Calls to the function could be replaced by string literals.

bug#39663: 27.1; ebnf2ps bugs

[Stefan Monnier]

> * The code resides in lisp/progmodes but isn't actually a mode at all: it
> converts BNF into railway diagrams in Postscript. It looks a bit out of place.

Agreed, tho it's harmless.

> What about moving it to ELPA?

No opinion about that,


        Stefan

bug#39663: 27.1; ebnf2ps bugs

[Mattias Engdegård]

A refactoring that removes ebnf-range-regexp has now been pushed to master. The bugs (Latin-1 mojibake, ABNF prose values) remain. Maybe Vinicius would have a look?

bug#39663: 27.1; ebnf2ps bugs

[Stefan Monnier]

> A refactoring that removes ebnf-range-regexp has now been pushed to
> master.  The bugs (Latin-1 mojibake, ABNF prose values) remain.  Maybe
> Vinicius would have a look?

Not sure if Vinicius is still paying attention.
I can probably help with the mojibake (I suspect it's a result
of the code being written for Emacs<23), but I'd need a recipe to
reproduce the problem.


        Stefan

bug#39663: 27.1; ebnf2ps bugs

[Mattias Engdegård]

24 feb. 2020 kl. 17.31 skrev Stefan Monnier <monnier@iro.umontreal.ca>:

> Not sure if Vinicius is still paying attention.
> I can probably help with the mojibake (I suspect it's a result
> of the code being written for Emacs<23), but I'd need a recipe to
> reproduce the problem.

Thanks -- I'm not sure it's worth the effort, but thought it was worth reporting since the author obviously went to some trouble to make Latin-1 work, and that it in fact does not.

Write this line in a buffer:

naïve = façade.

and then run (ebnf-print-buffer "out.ps"). Look at the generated file.

'ebnf-eps-string' looks like an attempt to do it right, but it's not used for productions, only page headers and footer comments. Other strings just end up in... ps-output-string-prim?

'ps-print-buffer' does seem to work, at least up to U+00FF. Anything beyond that becomes '?'.

bug#39663: 27.1; ebnf2ps bugs

[Lars Ingebrigtsen]

Mattias Engdegård <mattiase@acm.org> writes:

> * Prose values in ABNF don't seem to work. Guilty line:
>
> 	    (or (looking-at ebnf-abn-non-terminal-letter-chars)
>
> where 'ebnf-abn-non-terminal-letter-chars' is not a regexp but a
> skip-set used in skip-chars-{forward,backward}. This was found by a
> relint scan.

Do you have a test case for this?

> * Despite a lot of effort been spent on making it work with Latin-1, I
> get UTF-8 mojibake in the PostScript output when using chars in the
> U+00A0..U+00FF range in the rules, eg in
> nonterminals. Unibyte/multibyte confusion, perhaps?

This now works for the non-ASCII test case.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#39663: 27.1; ebnf2ps bugs

[Mattias Engdegård]

30 aug. 2021 kl. 03.15 skrev Lars Ingebrigtsen <larsi@gnus.org>:

> Do you have a test case for this?

Write this line in a buffer:

nisse = <totte>

and then run

 (let ((ebnf-syntax 'abnf)) (ebnf-print-buffer "out.ps"))

The result is a syntax error. (It works without the <>.)

> This now works for the non-ASCII test case.

Thank you! Doesn't seem to work with ABNF though (set `ebnf-syntax` to `abnf`, and remove the full stop).

Although it could still be useful, the ebnf2ps package does show its age in many respects: non-Unicode, output in EPS rather than PDF or SVG, no interactive preview, etc. It's also misplaced (in 'progmodes' despite not being a programming mode) and, in effect, abandoned. Does it really belong in Emacs core?

(And does anyone use railway diagrams these days?)

bug#39663: 27.1; ebnf2ps bugs

[Lars Ingebrigtsen]

Mattias Engdegård <mattiase@acm.org> writes:

> Thank you! Doesn't seem to work with ABNF though (set `ebnf-syntax` to
> `abnf`, and remove the full stop).

I tried

naïve = façade

(setq ebnf-syntax 'abnf)
(ebnf-print-buffer "/tmp/out.ps")

and it seemed to work?

(Is it apparent that I've never even heard of this package before?  :-))

> Although it could still be useful, the ebnf2ps package does show its
> age in many respects: non-Unicode, output in EPS rather than PDF or
> SVG, no interactive preview, etc. It's also misplaced (in 'progmodes'
> despite not being a programming mode) and, in effect, abandoned. Does
> it really belong in Emacs core?
>
> (And does anyone use railway diagrams these days?)

Yeah, it does seem a bit long in the tooth...

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#39663: [External] : bug#39663: 27.1; ebnf2ps bugs

[Drew Adams]

> > Although it could still be useful, the ebnf2ps package does show its
> > age in many respects: non-Unicode, output in EPS rather than PDF or
> > SVG, no interactive preview, etc. It's also misplaced (in 'progmodes'
> > despite not being a programming mode) and, in effect, abandoned. Does
> > it really belong in Emacs core?
> >
> > (And does anyone use railway diagrams these days?)

Yes, docs still use railroad diagrams.
And BNF (lots of flavors).

And EPS figures are still used in
technical doc production (of PDFs).

bug#39663: [External] : bug#39663: 27.1; ebnf2ps bugs

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

>> > (And does anyone use railway diagrams these days?)
> Yes, docs still use railroad diagrams.

I'd be surprised if anything else ever used railroad diagrams, so this
is not very helpful.  Which kinds of "docs" would use railroad diagrams?

> And BNF (lots of flavors).

That, yes.

> And EPS figures are still used in
> technical doc production (of PDFs).

Embedded PDF seems to have completely replaced it around here.


        Stefan

bug#39663: [External] : bug#39663: 27.1; ebnf2ps bugs

[Mattias Engdegård]

31 aug. 2021 kl. 05.28 skrev Stefan Monnier <monnier@iro.umontreal.ca>:

> Embedded PDF seems to have completely replaced it around here.

Yes, with SVG seen now and then. EPS are occasionally spotted in LaTeX document builds but there is at least one native *TeX package for railway diagrams for those who really want them.

I hereby propose that ebnf2ps be moved to GNU ELPA. This is not a change of status or maintainership, just packaging. It is also not a comment on the quality of the package: it now seems to work as originally intended. ELPA didn't exist in the current form when ebnf2ps was originally added to Emacs.

Would such a move be acceptable?

bug#39663: [External] : bug#39663: 27.1; ebnf2ps bugs

[Eli Zaretskii]

> From: Mattias Engdegård <mattiase@acm.org>
> Date: Tue, 31 Aug 2021 11:14:24 +0200
> Cc: Drew Adams <drew.adams@oracle.com>, Lars Ingebrigtsen <larsi@gnus.org>,
>         Eli Zaretskii <eliz@gnu.org>,
>         "39663@debbugs.gnu.org" <39663@debbugs.gnu.org>,
>         Vinicius Jose Latorre <viniciusjl.gnu@gmail.com>
> 
> I hereby propose that ebnf2ps be moved to GNU ELPA. This is not a change of status or maintainership, just packaging. It is also not a comment on the quality of the package: it now seems to work as originally intended. ELPA didn't exist in the current form when ebnf2ps was originally added to Emacs.
> 
> Would such a move be acceptable?

Do we want it to be part of an Emacs release tarball, or don't we?

If the latter, what would we tell users who may still be using it?

(I don't use this package, so I personally have nothing against
removing it from Emacs.  But maybe others do use it.)

bug#39663: [External] : bug#39663: 27.1; ebnf2ps bugs

[Drew Adams]

> I hereby propose that ebnf2ps be moved to GNU ELPA. This is not a change of
> status or maintainership, just packaging. It is also not a comment on the
> quality of the package: it now seems to work as originally intended. ELPA
> didn't exist in the current form when ebnf2ps was originally added to Emacs.
> 
> Would such a move be acceptable?

FWIW, that makes sense to me.

bug#39663: [External] : bug#39663: 27.1; ebnf2ps bugs

[Lars Ingebrigtsen]

Mattias Engdegård <mattiase@acm.org> writes:

> I hereby propose that ebnf2ps be moved to GNU ELPA. This is not a
> change of status or maintainership, just packaging. It is also not a
> comment on the quality of the package: it now seems to work as
> originally intended. ELPA didn't exist in the current form when
> ebnf2ps was originally added to Emacs.

We've only moved one package from Emacs to ELPA so far, and that was not
well received, so I don't think we should repeat that experiment.  (At
least not until we've got the ELPA-bunding stuff actually implemented.)

(And in general, I think moving things to ELPA generally only makes
sense if it's a package that somebody wants to work on a lot outside of
the Emacs release schedule -- not because it's an outdated package.)

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#39663: [External] : bug#39663: 27.1; ebnf2ps bugs

[Mattias Engdegård]

1 sep. 2021 kl. 09.53 skrev Lars Ingebrigtsen <larsi@gnus.org>:

> We've only moved one package from Emacs to ELPA so far, and that was not
> well received, so I don't think we should repeat that experiment.  (At
> least not until we've got the ELPA-bunding stuff actually implemented.)

Very well, I will not pursue the matter further. (What I write below is not an attempt to contradict that claim.)

I still think that the likely small user base of ebnf2ps would make negative reactions unlikely -- we have been talking about moving packages to ELPA for along time, and if we can't do it with this package then when?

> (And in general, I think moving things to ELPA generally only makes
> sense if it's a package that somebody wants to work on a lot outside of
> the Emacs release schedule -- not because it's an outdated package.)

Not sure I agree with that principle -- we'd have to be awfully careful with what we allow in Emacs then, because once in, it can never be removed, even if it didn't really belong in the tree in the first place. Shouldn't we be more forgiving of the past mistakes of others (and of our own)?

In any case this bug can be closed as far as I'm concerned -- once again, thanks for the help.

bug#39663: [External] : bug#39663: 27.1; ebnf2ps bugs

[Lars Ingebrigtsen]

Mattias Engdegård <mattiase@acm.org> writes:

> Not sure I agree with that principle -- we'd have to be awfully
> careful with what we allow in Emacs then, because once in, it can
> never be removed, even if it didn't really belong in the tree in the
> first place. Shouldn't we be more forgiving of the past mistakes of
> others (and of our own)?

We do deprecate (and remove) stuff from Emacs if we don't think it's
useful.  I don't think it's useful to think of ELPA as a cemetery for
dead code.  :-)

But I understand that there's many different opinions on this, and I'm
not opposing moving stuff to ELPA on principle -- when we finally get
ELPA bundling, then it's time to re-start the discussions about specific
packages.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#39663: [External] : bug#39663: 27.1; ebnf2ps bugs

[Eli Zaretskii]

> From: Mattias Engdegård <mattiase@acm.org>
> Date: Wed, 1 Sep 2021 10:23:28 +0200
> Cc: Stefan Monnier <monnier@iro.umontreal.ca>,
>         Drew Adams <drew.adams@oracle.com>, Eli Zaretskii <eliz@gnu.org>,
>         "39663@debbugs.gnu.org" <39663@debbugs.gnu.org>,
>         Vinicius Jose Latorre <viniciusjl.gnu@gmail.com>
> 
> > (And in general, I think moving things to ELPA generally only makes
> > sense if it's a package that somebody wants to work on a lot outside of
> > the Emacs release schedule -- not because it's an outdated package.)
> 
> Not sure I agree with that principle -- we'd have to be awfully careful with what we allow in Emacs then, because once in, it can never be removed, even if it didn't really belong in the tree in the first place.

No, we can move such packages to lisp/obsolete/, and after some time
there delete them.  It happened in the past.

bug#39663: [External] : bug#39663: 27.1; ebnf2ps bugs

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: Stefan Monnier <monnier@iro.umontreal.ca>,  Drew Adams
>  <drew.adams@oracle.com>,  Eli Zaretskii <eliz@gnu.org>,
>   "39663@debbugs.gnu.org" <39663@debbugs.gnu.org>,  Vinicius Jose Latorre
>  <viniciusjl.gnu@gmail.com>
> Date: Wed, 01 Sep 2021 09:53:12 +0200
> 
> We've only moved one package from Emacs to ELPA so far, and that was not
> well received, so I don't think we should repeat that experiment.  (At
> least not until we've got the ELPA-bunding stuff actually implemented.)
> 
> (And in general, I think moving things to ELPA generally only makes
> sense if it's a package that somebody wants to work on a lot outside of
> the Emacs release schedule -- not because it's an outdated package.)

Agreed.

bug#39663: [External] : bug#39663: 27.1; ebnf2ps bugs

[Drew Adams]

> >> > (And does anyone use railway diagrams these days?)
> > Yes, docs still use railroad diagrams.
> 
> I'd be surprised if anything else ever used railroad diagrams, so this
> is not very helpful.  Which kinds of "docs" would use railroad diagrams?

1. There are websites (such as bottlecaps.de/rr/ui)
that let anyone create railroad diagrams, and they
let you do so with many different forms of BNF.
Why do they exist?

2. Here's one set of doc that uses both railroad
diagrams and BNF (which is both more accessible
and usable by code).  This page is just for the
syntax of a single (giant) command: CREATE TABLE.

https://docs.oracle.com/en/database/oracle/oracle-database/21/sqlrf/CREATE-TABLE.html#GUID-F9CE0CC3-13AE-4744-A43C-EAC7A71AAAB6

> > And EPS figures are still used in
> > technical doc production (of PDFs).
> 
> Embedded PDF seems to have completely replaced it around here.

OK! A sample of one.  What docs does "around here"
produce?  Is the syntax they document complex?
Are some of their users (doc readers) disinclined
to read other syntax spec presentations?

bug#39663: 27.1; ebnf2ps bugs

[Lars Ingebrigtsen]

Mattias Engdegård <mattiase@acm.org> writes:

>> Do you have a test case for this?
>
> Write this line in a buffer:
>
> nisse = <totte>
>
> and then run
>
>  (let ((ebnf-syntax 'abnf)) (ebnf-print-buffer "out.ps"))
>
> The result is a syntax error. (It works without the <>.)

Thanks; this now works in Emacs 28.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#39663: 27.1; ebnf2ps bugs

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Date: Mon, 30 Aug 2021 03:15:00 +0200
> Cc: 39663@debbugs.gnu.org, Vinicius Jose Latorre <viniciusjl.gnu@gmail.com>,
>  Stefan Monnier <monnier@iro.umontreal.ca>
> 
> Mattias Engdegård <mattiase@acm.org> writes:
> 
> > * Despite a lot of effort been spent on making it work with Latin-1, I
> > get UTF-8 mojibake in the PostScript output when using chars in the
> > U+00A0..U+00FF range in the rules, eg in
> > nonterminals. Unibyte/multibyte confusion, perhaps?
> 
> This now works for the non-ASCII test case.

Can you explain the change you installed?  I don't understand why you
need to decode the input -- isn't that text from an Emacs Lisp string
or from a buffer?

bug#39663: 27.1; ebnf2ps bugs

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> Can you explain the change you installed?  I don't understand why you
> need to decode the input -- isn't that text from an Emacs Lisp string
> or from a buffer?

The decoding is unnecessary -- but it gives marginally less wrong
results if you have raw bytes in the buffer, I think?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#39663: 27.1; ebnf2ps bugs

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: mattiase@acm.org,  39663@debbugs.gnu.org,  viniciusjl.gnu@gmail.com,
>   monnier@iro.umontreal.ca
> Date: Tue, 31 Aug 2021 03:19:09 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Can you explain the change you installed?  I don't understand why you
> > need to decode the input -- isn't that text from an Emacs Lisp string
> > or from a buffer?
> 
> The decoding is unnecessary -- but it gives marginally less wrong
> results if you have raw bytes in the buffer, I think?

The raw bytes in the buffer, if they are present, are already in their
"decoded" multibyte form.  So I don't think I see how the decoding
could help there.  Do you have an example?

bug#39663: 27.1; ebnf2ps bugs

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> The raw bytes in the buffer, if they are present, are already in their
> "decoded" multibyte form.  So I don't think I see how the decoding
> could help there.  Do you have an example?

Uhm...  no, I think that was a thinko -- the decode is a NOOP here,
isn't it?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#39663: 27.1; ebnf2ps bugs

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: mattiase@acm.org,  39663@debbugs.gnu.org,  viniciusjl.gnu@gmail.com,
>   monnier@iro.umontreal.ca
> Date: Wed, 01 Sep 2021 10:05:00 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > The raw bytes in the buffer, if they are present, are already in their
> > "decoded" multibyte form.  So I don't think I see how the decoding
> > could help there.  Do you have an example?
> 
> Uhm...  no, I think that was a thinko -- the decode is a NOOP here,
> isn't it?

I think so.  And quite an expensive one, because I don't see any test
anywhere that would return without doing anything.

bug#39663: 27.1; ebnf2ps bugs

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> I think so.  And quite an expensive one, because I don't see any test
> anywhere that would return without doing anything.

Yup.  Now fixed on the trunk.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no