Hack for JSON sequences with trailing commas?

Hack for JSON sequences with trailing commas?

[Skip Montanaro]

I've been using programming languages where sequences can contain
trailing commas since my earliest days with C back in the early 80s.
Of course, Python (the language in which I write most things these
days) supports that. As does JavaScript, though for some crazy reason,
not if that JavaScript is just JSON. This is a valid JSON input
string:

"[1,2,3]"

but attempts to decode this string using Python's json.loads()
function provoke a JSONDecodeError:

"[1,2,3,]"

*sigh*

I suppose if I was to dig enough I'd find a reason JSON's author
thought that was a good idea.

I'm using JSON as a config file format for some code (not as an
interchange format, so I could care less what Chrome or IE think about
it), so I'm actually writing it by hand. Emacs tells me "*.json" files
are JavaScript, so it's quite happy with trailing commas.I installed
json-mode, but it's also happy with trailing commas.

Is there some clever hack to coax Emacs into stripping those trailing
commas I love so much? Maybe a JSON write-file hook to strip them and
a read-file-hook to add them back?

For the moment, for sequences which extend across multiple lines, I've
sort of adopted preceding the second through n-th elements with commas
instead of appending commas to the first through n-1st elements.

Thanks,

Skip, missing his commas in Illinois...

Re: Hack for JSON sequences with trailing commas?

[Barry Margolin]

In article <mailman.4482.1533056121.1292.help-gnu-emacs@gnu.org>,
 Skip Montanaro <skip.montanaro@gmail.com> wrote:

> I've been using programming languages where sequences can contain
> trailing commas since my earliest days with C back in the early 80s.
> Of course, Python (the language in which I write most things these
> days) supports that. As does JavaScript, though for some crazy reason,
> not if that JavaScript is just JSON. This is a valid JSON input
> string:
> 
> "[1,2,3]"
> 
> but attempts to decode this string using Python's json.loads()
> function provoke a JSONDecodeError:
> 
> "[1,2,3,]"
> 
> *sigh*
> 
> I suppose if I was to dig enough I'd find a reason JSON's author
> thought that was a good idea.

JSON is a very limited subset of Javascript notation for literals. Many 
things that are allowed in Javascript source code are not allowed in 
JSON. The reason is presumably to simplify the design of JSON parsers -- 
they don't have to deal with all possible input formats.

For instance, Javascript allows both single and double quotes to be used 
as string delimiters, but JSON only allows double quotes. Javascript 
allows object keys to be written without quoting them if they're valid 
JS names, JSON requires all keys to be quoted. And JS allows trailing 
commas, JSON doesn't.

-- 
Barry Margolin, barmar@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***

Re: Hack for JSON sequences with trailing commas?

[Yuri Khan]

On Wed, Aug 1, 2018 at 1:55 PM Barry Margolin <barmar@alum.mit.edu> wrote:

> JSON is a very limited subset of Javascript notation for literals. […]
>
> For instance, Javascript allows both single and double quotes to be used
> as string delimiters, but JSON only allows double quotes. Javascript
> allows object keys to be written without quoting them if they're valid
> JS names, JSON requires all keys to be quoted. And JS allows trailing
> commas, JSON doesn't.

More importantly, Javascript allows comments and string concatenation
expressions, while JSON doesn’t. Which makes it a really bad choice
for hand-written configuration files.

Re: Hack for JSON sequences with trailing commas?

[Skip Montanaro]

On Wed, Aug 1, 2018, 2:23 AM Yuri Khan <yurivkhan@gmail.com> wrote:

> On Wed, Aug 1, 2018 at 1:55 PM Barry Margolin <barmar@alum.mit.edu> wrote:
>
> > JSON is a very limited subset of Javascript notation for literals. […]
>
> More importantly, Javascript allows comments and string concatenation
> expressions, while JSON doesn’t. Which makes it a really bad choice
> for hand-written configuration files.
>

Thanks, folks. I know all this. I can (and do) handle the comment
situation. I was just asking if there was some reasonable way to support
trailing commas in sequences. If not, it won't kill me.

Skip

>

RE: Hack for JSON sequences with trailing commas?

[Drew Adams]

> JSON is a very limited subset of Javascript notation for literals. Many
> things that are allowed in Javascript source code are not allowed in
> JSON. The reason is presumably to simplify the design of JSON parsers --
> they don't have to deal with all possible input formats.

Yes, but there is a difference between JSON as defined by its standard
and JSON as it is used in practice in many situations. The former is more
strict than the latter.

Here is one description of possible differences between the strict syntax
of the standard and a lax syntax (one that does allow an extra, final comma):

https://docs.oracle.com/en/database/oracle/oracle-database/18/adjsn/conditions-is-json-and-is-not-json.html#GUID-1B6CFFBE-85FE-41DD-BA14-DD1DE73EAB20

Re: Hack for JSON sequences with trailing commas?

[Barry Margolin]

In article <mailman.4559.1533134429.1292.help-gnu-emacs@gnu.org>,
 Drew Adams <drew.adams@oracle.com> wrote:

> > JSON is a very limited subset of Javascript notation for literals. Many
> > things that are allowed in Javascript source code are not allowed in
> > JSON. The reason is presumably to simplify the design of JSON parsers --
> > they don't have to deal with all possible input formats.
> 
> Yes, but there is a difference between JSON as defined by its standard
> and JSON as it is used in practice in many situations. The former is more
> strict than the latter.
> 
> Here is one description of possible differences between the strict syntax
> of the standard and a lax syntax (one that does allow an extra, final comma):
> 
> https://docs.oracle.com/en/database/oracle/oracle-database/18/adjsn/conditions
> -is-json-and-is-not-json.html#GUID-1B6CFFBE-85FE-41DD-BA14-DD1DE73EAB20

I think Oracle is unusual in being so permissive. I don't think the JSON 
parsers in PHP, Python, or Javascript allow as much as Oracle does.

I think the only common extension to JSON that was commonly allowed was 
allowing the top-level value to be something other than an object or 
array. The JSON spec was recently updated to legitimize this.

-- 
Barry Margolin, barmar@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***

RE: Hack for JSON sequences with trailing commas?

[Drew Adams]

> > > JSON is a very limited subset of Javascript notation for literals. Many
> > > things that are allowed in Javascript source code are not allowed in
> > > JSON. The reason is presumably to simplify the design of JSON parsers --
> > > they don't have to deal with all possible input formats.
> >
> > Yes, but there is a difference between JSON as defined by its standard
> > and JSON as it is used in practice in many situations. The former is more
> > strict than the latter.
> >
> > Here is one description of possible differences between the strict syntax
> > of the standard and a lax syntax (one that does allow an extra, final
> > comma):
> >
> > https://docs.oracle.com/en/database/oracle/oracle-
> database/18/adjsn/conditions
> > -is-json-and-is-not-json.html#GUID-1B6CFFBE-85FE-41DD-BA14-
> DD1DE73EAB20
> 
> I think Oracle is unusual in being so permissive. I don't think the JSON
> parsers in PHP, Python, or Javascript allow as much as Oracle does.

It's only permissive if/when you want it to be. You can easily get strict behavior (i.e., per the JSON standard). IOW, you have a choice.

And what you want can depend on what you're doing. You might want to be able to store documents that are not strictly well-formed but are well-formed according to the lax syntax. But you might also want to process such stored documents in various ways, some of which accept only documents that are strictly well-formed. You can specify the behavior you want for storing and for various operations.

> I think the only common extension to JSON that was commonly allowed was
> allowing the top-level value to be something other than an object or
> array. The JSON spec was recently updated to legitimize this.

Yes, that's something different. The JSON spec is one thing. Uses of JSON data in practice are another thing. Only a subset of such uses require documents that adhere to the standard. Sometimes you can't control the kinds of documents you receive, but you want to be able to process them whether they strictly conform to the standard or not.

Re: Hack for JSON sequences with trailing commas?

[Barry Margolin]

In article <mailman.4627.1533257510.1292.help-gnu-emacs@gnu.org>,
 Drew Adams <drew.adams@oracle.com> wrote:

> > > > JSON is a very limited subset of Javascript notation for literals. Many
> > > > things that are allowed in Javascript source code are not allowed in
> > > > JSON. The reason is presumably to simplify the design of JSON parsers 
> > > > --
> > > > they don't have to deal with all possible input formats.
> > >
> > > Yes, but there is a difference between JSON as defined by its standard
> > > and JSON as it is used in practice in many situations. The former is more
> > > strict than the latter.
> > >
> > > Here is one description of possible differences between the strict syntax
> > > of the standard and a lax syntax (one that does allow an extra, final
> > > comma):
> > >
> > > https://docs.oracle.com/en/database/oracle/oracle-
> > database/18/adjsn/conditions
> > > -is-json-and-is-not-json.html#GUID-1B6CFFBE-85FE-41DD-BA14-
> > DD1DE73EAB20
> > 
> > I think Oracle is unusual in being so permissive. I don't think the JSON
> > parsers in PHP, Python, or Javascript allow as much as Oracle does.
> 
> It's only permissive if/when you want it to be. You can easily get strict 
> behavior (i.e., per the JSON standard). IOW, you have a choice.
> 
> And what you want can depend on what you're doing. You might want to be able 
> to store documents that are not strictly well-formed but are well-formed 
> according to the lax syntax. But you might also want to process such stored 
> documents in various ways, some of which accept only documents that are 
> strictly well-formed. You can specify the behavior you want for storing and 
> for various operations.
> 
> > I think the only common extension to JSON that was commonly allowed was
> > allowing the top-level value to be something other than an object or
> > array. The JSON spec was recently updated to legitimize this.
> 
> Yes, that's something different. The JSON spec is one thing. Uses of JSON 
> data in practice are another thing. Only a subset of such uses require 
> documents that adhere to the standard. Sometimes you can't control the kinds 
> of documents you receive, but you want to be able to process them whether 
> they strictly conform to the standard or not.

The primary use of JSON is for communication between unrelated code, 
e.g. Javascript clients using PHP APIs. If you're creating JSON, you 
can't usually depend on it being parsed by a particular application, 
which might use a more permissive parser.

If you're designing a self-contained system where you control both the 
creator and parser, then you can do anything you want. The purpose of 
standards is to ensure compatibility between code from different 
creators.

-- 
Barry Margolin, barmar@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***