bug#48274: 27.2; libjanson: \u0000 is not allowed without JSON_ALLOW_NUL

bug#48274: 27.2; libjanson: \u0000 is not allowed without JSON_ALLOW_NUL

[Peter]

Hello ;)

It seems that Emacs' `json-parse-buffer' cannot correctly parse strings
that contain an escaped ascii 0 value.

To reproduce:

- Create a new buffer, insert into it:
  { "foo": "this is a string including a literal \u0000" }
- Move point to the beginning of the buffer [ M-< ]
- Execute `json-parse-buffer' [ M-: (json-parse-buffer) ]
- See the error message.

According to all other systems I've tried, \u0000 is valid in json
strings.

The error message seems to imply that something (probably libjanson?)
should be compiled with -DJSON_ALLOW_NUL.

As this seems to be non-conformant behaviour, is there a way to change
this in the default Emacs build?

Thanks, Greetings,
Peter

In GNU Emacs 27.2 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.27, cairo version 1.17.4)
 of 2021-03-26 built on juergen
Windowing system distributor 'The X.Org Foundation', version 11.0.12011000
System Description: Arch Linux

Recent messages:
Configured using:
 'configure --prefix=/usr --sysconfdir=/etc --libexecdir=/usr/lib
 --localstatedir=/var --with-x-toolkit=gtk3 --with-xft --with-wide-int
 --with-modules --with-cairo --with-harfbuzz 'CFLAGS=-march=x86-64
 -mtune=generic -O2 -pipe -fno-plt' CPPFLAGS=-D_FORTIFY_SOURCE=2
 LDFLAGS=-Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now'

Configured features:
XPM JPEG TIFF GIF PNG RSVG CAIRO SOUND GPM DBUS GSETTINGS GLIB NOTIFY
INOTIFY ACL GNUTLS LIBXML2 FREETYPE HARFBUZZ M17N_FLT LIBOTF ZLIB
TOOLKIT_SCROLL_BARS GTK3 X11 XDBE XIM MODULES THREADS LIBSYSTEMD JSON
PDUMPER LCMS2 GMP

Important settings:
  value of $LC_COLLATE: C
  value of $LC_MESSAGES:
  value of $LANG: en_GB.UTF-8
  locale-coding-system: utf-8-unix

Major mode: JSON

Memory information:
((conses 16 1891646 539122)
 (symbols 48 122145 126)
 (strings 32 464687 51556)
 (string-bytes 1 14083739)
 (vectors 16 192214)
 (vector-slots 8 3676318 291674)
 (floats 8 1942 24899)
 (intervals 56 37686 13084)
 (buffers 1000 91))

bug#48274: 27.2; libjanson: \u0000 is not allowed without JSON_ALLOW_NUL

[Philipp]

> Am 07.05.2021 um 09:16 schrieb Peter <craven@gmx.net>:
> 
> Hello ;)
> 
> It seems that Emacs' `json-parse-buffer' cannot correctly parse strings
> that contain an escaped ascii 0 value.
> 
> To reproduce:
> 
> - Create a new buffer, insert into it:
>  { "foo": "this is a string including a literal \u0000" }
> - Move point to the beginning of the buffer [ M-< ]
> - Execute `json-parse-buffer' [ M-: (json-parse-buffer) ]
> - See the error message.
> 
> According to all other systems I've tried, \u0000 is valid in json
> strings.
> 
> The error message seems to imply that something (probably libjanson?)
> should be compiled with -DJSON_ALLOW_NUL.
> 
> As this seems to be non-conformant behaviour, is there a way to change
> this in the default Emacs build?

We use the default behavior of Jansson here, see the remark about JSON_ALLOW_NUL in https://jansson.readthedocs.io/en/2.13/apiref.html#decoding.
I wouldn't mind allowing escaped NUL characters though; Emacs should be able to handle them just fine.
Also see the FIXME in the unit test `json-parse-string/null' in tests/src/json-tests.el.

bug#48274: 27.2; libjanson: \u0000 is not allowed without JSON_ALLOW_NUL

[Lars Ingebrigtsen]

Philipp <p.stephani2@gmail.com> writes:

> We use the default behavior of Jansson here, see the remark about
> JSON_ALLOW_NUL in
> https://jansson.readthedocs.io/en/2.13/apiref.html#decoding.
> I wouldn't mind allowing escaped NUL characters though; Emacs should
> be able to handle them just fine.

I think it would make sense to allow nul characters here, too.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#48274: 27.2; libjanson: \u0000 is not allowed without JSON_ALLOW_NUL

[Lars Ingebrigtsen]

Philipp <p.stephani2@gmail.com> writes:

> We use the default behavior of Jansson here, see the remark about
> JSON_ALLOW_NUL in
> https://jansson.readthedocs.io/en/2.13/apiref.html#decoding.
> I wouldn't mind allowing escaped NUL characters though; Emacs should
> be able to handle them just fine.
> Also see the FIXME in the unit test `json-parse-string/null' in
> tests/src/json-tests.el.

So I've now made this change in Emacs 29.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no