bug#46486: 28.0.50; json-parse-string missing support for key-less json strings

bug#46486: 28.0.50; json-parse-string missing support for key-less json strings

[Thomas Hisch]

The `json-read-from-string' function doesn't support json strings that
only contain a single value, which is neither a list nor a dict, e.g.

"123"

or

"\"abc\""

However, `json-read-from-string' doesn't have this issue:

(require 'json)
(json-read-from-string "\"abc\"")
-> "abc"

(json-parse-string "\"abc\"")
-> progn: could not parse JSON stream: "'[' or '{' expected near '\"abc\"'", "<string>", 1, 5, 5

Note that strings that neither start with [ nor { are valid JSON objects
according to the JSON schema.

Since the docstring of `json-parse-string' mentions `json-serialize' as
essentially being the reverse operation of `json-parse-string', I've
tried to serialize a lisp string:

(json-serialize "abc")
-> progn: Wrong type argument: json-value-p, "abc"

I would also expect this `json-serialize' call to work.



In GNU Emacs 28.0.50 (build 1, x86_64-pc-linux-gnu, X toolkit, cairo version 1.16.0, Xaw scroll bars)
 of 2021-02-12 built on thomas-XPS-13-9360
Repository revision: db237850abc240e2c3e765e9cc7e15ee5681dcaf
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.12009000
System Description: Ubuntu 20.10

Configured using:
 'configure 'CFLAGS=-march=native -O3' --prefix=/opt/emacs
 --with-x-toolkit=lucid --with-xpm --without-jpeg --without-gif
 --without-tiff --with-modules --with-json'

Configured features:
CAIRO DBUS FREETYPE GLIB GMP GNUTLS GSETTINGS HARFBUZZ JSON LIBSELINUX
LIBXML2 MODULES NOTIFY INOTIFY PDUMPER PNG SOUND THREADS
TOOLKIT_SCROLL_BARS X11 XDBE XIM XPM LUCID ZLIB

Important settings:
  value of $LC_MONETARY: en_US.UTF-8
  value of $LC_NUMERIC: en_US.UTF-8
  value of $LC_TIME: en_US.UTF-8
  value of $LANG: en_US.UTF-8
  value of $XMODIFIERS: @im=ibus
  locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
  counsel-projectile-mode: t
  counsel-mode: t
  ivy-rich-mode: t
  ivy-mode: t
  global-ethan-wspace-mode: t
  ethan-wspace-mode: t
  ethan-wspace-clean-many-nls-eof-mode: t
  ethan-wspace-clean-no-nl-eof-mode: t
  ethan-wspace-clean-eol-mode: t
  ethan-wspace-clean-tabs-mode: t
  save-place-mode: t
  show-paren-mode: t
  delete-selection-mode: t
  savehist-mode: t
  global-hi-lock-mode: t
  hi-lock-mode: t
  winner-mode: t
  window-numbering-mode: t
  global-undo-tree-mode: t
  undo-tree-mode: t
  thi::spell-fu-mode: t
  spell-fu-mode: t
  rainbow-delimiters-mode: t
  yas-minor-mode: t
  idle-highlight-mode: t
  async-bytecomp-package-mode: t
  highlight-function-calls-mode: t
  recentf-mode: t
  global-git-commit-mode: t
  magit-auto-revert-mode: t
  global-flycheck-mode: t
  flycheck-mode: t
  flx-ido-mode: t
  shell-dirtrack-mode: t
  evil-mode: t
  evil-local-mode: t
  envrc-global-mode: t
  envrc-mode: t
  projectile-mode: t
  outline-minor-mode: t
  override-global-mode: t
  bug-reference-prog-mode: t
  url-handler-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  global-prettify-symbols-mode: t
  prettify-symbols-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  size-indication-mode: t
  column-number-mode: t
  line-number-mode: t
  transient-mark-mode: t
  hs-minor-mode: t

Load-path shadows:
/home/thomas/.emacs.d/elpa/magit-20210209.1110/magit-section hides /home/thomas/.emacs.d/elpa/magit-section-20210124.1824/magit-section
/home/thomas/.emacs.d/user-lisp/custom hides /opt/emacs/share/emacs/28.0.50/lisp/custom

Features:
(shadow sort mail-extr emacsbug sendmail pulse vc-git cl-print debug
backtrace hippie-exp mule-util magit-bookmark bookmark url-cache
counsel-projectile counsel swiper ivy-rich ivy-hydra ivy ivy-faces
ivy-overlay colir ethan-wspace hideshow url-tramp ims-jira saveplace
ffap paren delsel savehist hi-lock thi-projects winner window-numbering
which-key wgrep-ag vterm-toggle tramp-sh vterm term disp-table ehelp
vterm-module term/xterm xterm visual-fill-column undo-tree
typescript-mode stickyfunc-enhance semantic/util-modes semantic/util
semantic semantic/tag semantic/lex semantic/fw mode-local cedet
ssh-config-mode sphinx-doc spell-fu smex
smart-mode-line-respectful-theme smart-mode-line-dark-theme
sanityinc-tomorrow-night-theme smart-mode-line rich-minority rg
rg-info-hack rg-menu rg-ibuffer rg-result wgrep-rg wgrep rg-history
rg-header cus-edit pp cus-start cus-load rainbow-delimiters
python-docstring py-isort pyimport shut-up protobuf-mode cc-langs
paredit magit-gerrit magit-gerrit-section magit-gerrit-comment-ui
magit-gerrit-requests magit-gerrit-comment lsp-ui lsp-ui-flycheck
lsp-ui-doc goto-addr lsp-ui-imenu lsp-ui-peek lsp-ui-sideline face-remap
lsp-mode yasnippet-snippets yasnippet spinner network-stream inline ewoc
bindat lsp-ui-util lsp-protocol ht idle-highlight-mode helm-projectile
helm-files docker-tramp tramp-cache tramp tramp-loaddefs trampver
tramp-integration files-x tramp-compat ls-lisp helm-tags helm-buffers
helm-occur helm-grep helm-regexp helm-utils helm-locate helm-help
helm-types helm async-bytecomp helm-global-bindings helm-easymenu
helm-source eieio-compat helm-multi-match helm-lib async
highlight-indentation highlight-function-calls gnus nnheader gerrit
gerrit-rest recentf tree-widget wid-edit forge-list forge-commands
forge-semi forge-bitbucket buck forge-gogs gogs forge-gitea gtea
forge-gitlab glab forge-github ghub-graphql treepy gsexp ghub gnutls
forge-notify forge-revnote forge-pullreq forge-issue forge-topic
forge-post forge-repo forge forge-core forge-db closql emacsql-sqlite
emacsql emacsql-compiler url-http url-auth url-gw nsm magit-submodule
magit-obsolete magit-blame magit-stash magit-reflog magit-bisect
magit-push magit-pull magit-fetch magit-clone magit-remote magit-commit
magit-sequence magit-notes magit-worktree magit-tag magit-merge
magit-branch magit-reset magit-files magit-refs magit-status magit
magit-repos magit-apply magit-wip magit-log which-func magit-diff
smerge-mode diff diff-mode git-commit log-edit message rmc puny dired
dired-loaddefs rfc822 mml mml-sec epa derived epg epg-config gnus-util
rmail rmail-loaddefs mailabbrev mail-utils gmm-utils mailheader
pcvs-util add-log magit-core magit-libgit libgit libegit2
magit-autorevert autorevert filenotify magit-margin magit-transient
magit-process with-editor server magit-mode transient format-spec
magit-git magit-utils crm format-all language-id reformatter
flycheck-package package-lint let-alist imenu finder flycheck find-func
flx-ido ido flx evil evil-keybindings evil-integration evil-maps
evil-commands reveal flyspell ispell evil-jumps evil-command-window
evil-types evil-search evil-ex shell pcomplete evil-macros evil-repeat
evil-states evil-core advice evil-common windmove rect evil-digraphs
evil-vars equake envrc inheritenv ediff ediff-merg ediff-mult ediff-wind
ediff-diff ediff-help ediff-init ediff-util cc-mode cc-fonts cc-guess
cc-menus cc-cmds cc-styles cc-align cc-engine cc-vars cc-defs dumb-jump
popup dracula-theme diminish doom-themes doom-themes-base
color-theme-sanityinc-tomorrow cargo cargo-process markdown-mode
edit-indirect color rust-mode bufler projectile grep compile
text-property-search comint ansi-color ibuf-ext ibuffer ibuffer-loaddefs
thingatpt pretty-hydra hydra lv bufler-group-tree magit-section f s
dash-functional dash noutline outline vc vc-dispatcher parse-time
iso8601 time-date quelpa-use-package quelpa mm-decode mm-bodies
mm-encode mail-parse rfc2231 rfc2047 rfc2045 mm-util ietf-drums
mail-prsvr lisp-mnt help-fns radix-tree cl-extra help-mode use-package
use-package-ensure use-package-delight use-package-diminish
use-package-bind-key bind-key use-package-core finder-inf xref project
ring bug-reference xdg tab-line pcase easy-mmode edmacro kmacro rx info
package easymenu browse-url url url-proxy url-privacy url-expand
url-methods url-history url-cookie url-domsuf url-util mailcap
url-handlers url-parse auth-source cl-seq eieio eieio-core cl-macs
eieio-loaddefs password-cache json subr-x map url-vars seq byte-opt gv
bytecomp byte-compile cconv cl-loaddefs cl-lib iso-transl tooltip eldoc
electric uniquify ediff-hook vc-hooks lisp-float-type mwheel term/x-win
x-win term/common-win x-dnd tool-bar dnd fontset image regexp-opt fringe
tabulated-list replace newcomment text-mode elisp-mode lisp-mode
prog-mode register page tab-bar menu-bar rfn-eshadow isearch timer
select scroll-bar mouse jit-lock font-lock syntax facemenu font-core
term/tty-colors frame minibuffer cl-generic cham georgian utf-8-lang
misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms
cp51932 hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese composite charscript charprop case-table epa-hook
jka-cmpr-hook help simple abbrev obarray cl-preloaded nadvice button
loaddefs faces cus-face macroexp files window text-properties overlay
sha1 md5 base64 format env code-pages mule custom widget
hashtable-print-readable backquote threads dbusbind inotify
dynamic-setting system-font-setting font-render-setting cairo x-toolkit
x multi-tty make-network-process emacs)

Memory information:
((conses 16 860745 303250)
 (symbols 48 75095 48)
 (strings 32 356592 50903)
 (string-bytes 1 8345016)
 (vectors 16 98242)
 (vector-slots 8 2608232 546550)
 (floats 8 587 674)
 (intervals 56 18840 12630)
 (buffers 984 29))
Diese E-Mail enthält vertrauliche und/oder rechtlich geschützte Informationen. Wenn Sie nicht der richtige Adressat sind oder diese E-Mail irrtümlich erhalten haben, informieren Sie bitte sofort den Absender und vernichten Sie diese Mail. Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser Mail ist nicht gestattet.

This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly prohibited.

IMS Nanofabrication GmbH FN: 208021 p, FB-Gericht: Wien; Sitz: Schreygasse 3, 1020, Vienna (Austria),Tel. +43 1 214 48 94; E-Mail: imsoffice@ims.co.at; DVR-Nr: 0374369; www.ims.co.at

bug#46486: (no subject)

[Thomas Hisch]

 > The `json-read-from-string' function doesn't support json strings that
 > only contain a single value, which is neither a list nor a dict, e.g.

I'm sry, but I meant `json-parse-string' here.
Diese E-Mail enthält vertrauliche und/oder rechtlich geschützte Informationen. Wenn Sie nicht der richtige Adressat sind oder diese E-Mail irrtümlich erhalten haben, informieren Sie bitte sofort den Absender und vernichten Sie diese Mail. Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser Mail ist nicht gestattet.

This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly prohibited.

IMS Nanofabrication GmbH FN: 208021 p, FB-Gericht: Wien; Sitz: Schreygasse 3, 1020, Vienna (Austria),Tel. +43 1 214 48 94; E-Mail: imsoffice@ims.co.at; DVR-Nr: 0374369; www.ims.co.at

bug#46486: 28.0.50; json-parse-string missing support for key-less json strings

[Philipp]

> Am 13.02.2021 um 13:04 schrieb Thomas Hisch <thomas.hisch@ims.co.at>:
> 
> 
> (json-parse-string "\"abc\"")
> -> progn: could not parse JSON stream: "'[' or '{' expected near '\"abc\"'", "<string>", 1, 5, 5
> 
> Note that strings that neither start with [ nor { are valid JSON objects
> according to the JSON schema.
> 

That contradicts RFC 4627, which states: "A JSON text is a serialized object or array."

bug#46486: 28.0.50; json-parse-string missing support for key-less json strings

[Basil L. Contovounesios]

forcemerge 42994 46486
quit

Thomas Hisch <thomas.hisch@ims.co.at> writes:

> The [json-parse-string] function doesn't support json strings that
> only contain a single value, which is neither a list nor a dict, e.g.
>
> "123"

Thanks.  This is a duplicate of https://bugs.gnu.org/42994.

[...]

> Note that strings that neither start with [ nor { are valid JSON objects
> according to the JSON schema.

Yes, since ECMA-404 from 2013, but unfortunately json.c currently
conforms only with RFC4627 from 2006 in this respect.

-- 
Basil

bug#46486: 28.0.50; json-parse-string missing support for key-less json strings

[Philipp]

> Am 13.02.2021 um 13:24 schrieb Philipp <p.stephani2@gmail.com>:
> 
> 
> 
>> Am 13.02.2021 um 13:04 schrieb Thomas Hisch <thomas.hisch@ims.co.at>:
>> 
>> 
>> (json-parse-string "\"abc\"")
>> -> progn: could not parse JSON stream: "'[' or '{' expected near '\"abc\"'", "<string>", 1, 5, 5
>> 
>> Note that strings that neither start with [ nor { are valid JSON objects
>> according to the JSON schema.
>> 
> 
> That contradicts RFC 4627, which states: "A JSON text is a serialized object or array.“
> 

OTOH, RFC 8259 does seem to allow any value type at the toplevel now, so we might as well support this.  I don’t have a strong opinion here.  Using RFC 4627 semantics seems a bit safer and more conservative, but given that it’s considered obsoleted by RFC 8259, maybe we should switch to the semantics of the newer standard.

bug#46486: 28.0.50; json-parse-string missing support for key-less json strings

[Basil L. Contovounesios]

Philipp <p.stephani2@gmail.com> writes:

>> Am 13.02.2021 um 13:04 schrieb Thomas Hisch <thomas.hisch@ims.co.at>:
>> 
>> 
>> (json-parse-string "\"abc\"")
>> -> progn: could not parse JSON stream: "'[' or '{' expected near '\"abc\"'", "<string>", 1, 5, 5
>> 
>> Note that strings that neither start with [ nor { are valid JSON objects
>> according to the JSON schema.
>> 
>
> That contradicts RFC 4627, which states: "A JSON text is a serialized object or array."

RFC 4627 is from 2006 and is obsoleted by RFCs 7158, 7159, and 8259, as
well as both versions of ECMA-404, since at least as far back as 2013,
all of which lift the strict object/array restriction.

The current json.c produces maximally compatible JSON text in that all
implementations must accept object/array text, but it would be nice to
support scalar values too.

Thanks,

-- 
Basil

bug#46486: [External] : bug#46486: 28.0.50; json-parse-string missing support for key-less json strings

[Drew Adams]

> > (json-parse-string "\"abc\"")
> > -> progn: could not parse JSON stream: "'[' or '{' expected near
> '\"abc\"'", "<string>", 1, 5, 5
> >
> > Note that strings that neither start with [ nor { are valid JSON
> objects
> > according to the JSON schema.
> >
> 
> That contradicts RFC 4627, which states: "A JSON text is a serialized
> object or array."

Apologies for not following this thread.

RFC 4627 is extended by RFC 8259.

https://tools.ietf.org/html/rfc8259

With that, JSON scalars are included - not just
JSON objects an arrays.

(Again, dunno what the bug context is, or whether
my comment is relevant.  Please ignore, if not.) 

bug#46486: [External] : bug#46486: 28.0.50; json-parse-string missing support for key-less json strings

[Drew Adams]

> OTOH, RFC 8259 does seem to allow any value type at the toplevel now,
> so we might as well support this.  I don’t have a strong opinion here.
> Using RFC 4627 semantics seems a bit safer and more conservative, but
> given that it’s considered obsoleted by RFC 8259, maybe we should
> switch to the semantics of the newer standard.

Sorry, my previous comment duplicates what you just said.

IMO, nowadays applications should generally
consider all possible JSON values as JSON
texts: objects, arrays, and scalars.