[PATCH] Support custom 'null' and 'false' objects when parsing JSON

["João Távora" <joaotavora@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 1269 bytes --]

Hi,

I've been working on integrating my jsonrpc.el library with very new
json.c library in Emacs 27.

After adding plist support to json.c, I got jsonrpc.el mostly working
(and eglot.el working on top of that). There is, however, a rough edge
in json.c that is breaking core functionality in eglot.el: given that
LSP servers may sometimes send :null as a key's value and sometimes omit
the key completely, and given that JSON null is unavoidably parsed as
the keyword :null, I have to explicitly check for non-`nil'ness *and*
non-`:null'ness, whereas before only one check was necessary.  Other
applications may care about the subtle difference, but eglot.el (and, I
think, LSP in general) doesn't.

This patch adds two new keyword arguments to json-parse-string and
json-parse-buffer to control this behaviour.  This is analogous to
json.el's `json-false' and `json-null'.

The defaults I have chosen ensure the change is backward compatible, but
maybe they needn't be, and other defaults compatible to json.el's could
be chosen. This is because I don't know of any json.c users: AFAIK
jsonrpc.el will be the first one.

Patch attached, or find the code in the
scratch/allow-custom-null-and-false-objects-in-jsonc branch

Thanks,
João


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Support-custom-null-and-false-objects-when-parsing-J.patch --]
[-- Type: text/x-diff, Size: 14881 bytes --]

From 97975bec6126be5a99377a0e73921cd1e757c97f Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Jo=C3=A3o=20T=C3=A1vora?= <joaotavora@gmail.com>
Date: Thu, 7 Jun 2018 17:41:19 +0100
Subject: [PATCH] Support custom 'null' and 'false' objects when parsing JSON

* doc/lispref/text.texi (Parsing JSON): Describe new :null-object
and :false-object kwargs to json-parse-string and
json-parse-buffer.

* src/json.c (json_to_list): Take null_object and false_object
params.
(json_parse_args): Rename from json_parse_object_type.
(Fjson_parse_string): Rework docstring.
(Fjson_parse_string, Fjson_parse_buffer): Update call to
json_to_lisp.
(syms_of_json): Two new syms, QCnull_object and QCfalse_object.

* test/src/json-tests.el
(json-parse-with-custom-null-and-false-objects): New test.
---
 doc/lispref/text.texi  |  45 ++++++++++-----
 src/json.c             | 121 ++++++++++++++++++++++++++++-------------
 test/src/json-tests.el |  31 +++++++++++
 3 files changed, 145 insertions(+), 52 deletions(-)

diff --git a/doc/lispref/text.texi b/doc/lispref/text.texi
index 2c5b5a1b42..c5b75e6357 100644
--- a/doc/lispref/text.texi
+++ b/doc/lispref/text.texi
@@ -5008,9 +5008,10 @@ Parsing JSON
 @itemize
 
 @item
-JSON has a couple of keywords: @code{null}, @code{false}, and
-@code{true}.  These are represented in Lisp using the keywords
-@code{:null}, @code{:false}, and @code{t}, respectively.
+JSON uses three keywords: @code{true}, @code{null}, @code{false}.
+@code{true} is represented by the symbol @code{t}.  By default, the
+remaining two are represented, respectively, by the symbols
+@code{:false} and @code{:null}.
 
 @item
 JSON only has floating-point numbers.  They can represent both Lisp
@@ -5062,14 +5063,6 @@ Parsing JSON
 type.  Likewise, the parsing functions will only return vectors,
 hashtables, alists, and plists.
 
-  The parsing functions accept keyword arguments.  Currently only one
-keyword argument, @code{:object-type}, is recognized; its value
-decides which Lisp object to use for representing the key-value
-mappings of a JSON object.  It can be either @code{hash-table}, the
-default, to make hashtables with strings as keys, @code{alist} to use
-alists with symbols as keys or @code{plist} to use plists with keyword
-symbols as keys.
-
 @defun json-serialize object
 This function returns a new Lisp string which contains the JSON
 representation of @var{object}.
@@ -5080,16 +5073,38 @@ Parsing JSON
 current buffer before point.
 @end defun
 
-@defun json-parse-string string &key (object-type @code{hash-table})
+@defun json-parse-string string &rest args
 This function parses the JSON value in @var{string}, which must be a
-Lisp string.
+Lisp string.  The arguments @var{args} are a list of keyword/argument
+pairs. The following keywords are accepted:
+
+@itemize
+
+@item @code{:object-type}
+The value decides which Lisp object to use for representing the key-value
+mappings of a JSON object.  It can be either @code{hash-table}, the
+default, to make hashtables with strings as keys, @code{alist} to use
+alists with symbols as keys or @code{plist} to use plists with keyword
+symbols as keys.
+
+@item @code{:null-object}
+The value decides which Lisp object use to represent the JSON keyword
+@code{null}.  It defaults to the lisp symbol @code{:null}.
+
+@item @code{:false-object}
+The value decides which Lisp object use to represent the JSON keyword
+@code{false}.  It defaults to the lisp symbol @code{:false}.
+
+@end itemize
+
 @end defun
 
-@defun json-parse-buffer &key (object-type @code{hash-table})
+@defun json-parse-buffer &rest args
 This function reads the next JSON value from the current buffer,
 starting at point.  It moves point to the position immediately after
 the value if a value could be read and converted to Lisp; otherwise it
-doesn't move point.
+doesn't move point.  @var{args} is interpreted as in
+@code{json-parse-string}.
 @end defun
 
 
diff --git a/src/json.c b/src/json.c
index afb81587a4..db6a042d3a 100644
--- a/src/json.c
+++ b/src/json.c
@@ -633,14 +633,17 @@ enum json_object_type {
 /* Convert a JSON object to a Lisp object.  */
 
 static _GL_ARG_NONNULL ((1)) Lisp_Object
-json_to_lisp (json_t *json, enum json_object_type object_type)
+json_to_lisp (json_t *json,
+              enum json_object_type object_type,
+              Lisp_Object null_object,
+              Lisp_Object false_object)
 {
   switch (json_typeof (json))
     {
     case JSON_NULL:
-      return QCnull;
+      return null_object;
     case JSON_FALSE:
-      return QCfalse;
+      return false_object;
     case JSON_TRUE:
       return Qt;
     case JSON_INTEGER:
@@ -667,7 +670,10 @@ json_to_lisp (json_t *json, enum json_object_type object_type)
         Lisp_Object result = Fmake_vector (make_natnum (size), Qunbound);
         for (ptrdiff_t i = 0; i < size; ++i)
           ASET (result, i,
-                json_to_lisp (json_array_get (json, i), object_type));
+                json_to_lisp (json_array_get (json, i),
+                              object_type,
+                              null_object,
+                              false_object));
         --lisp_eval_depth;
         return result;
       }
@@ -696,7 +702,11 @@ json_to_lisp (json_t *json, enum json_object_type object_type)
                   /* Keys in JSON objects are unique, so the key can't
                      be present yet.  */
                   eassert (i < 0);
-                  hash_put (h, key, json_to_lisp (value, object_type), hash);
+                  hash_put (h, key, json_to_lisp (value,
+                                                  object_type,
+                                                  null_object,
+                                                  false_object),
+                            hash);
                 }
               break;
             }
@@ -709,7 +719,10 @@ json_to_lisp (json_t *json, enum json_object_type object_type)
                 {
                   Lisp_Object key = Fintern (json_build_string (key_str), Qnil);
                   result
-                    = Fcons (Fcons (key, json_to_lisp (value, object_type)),
+                    = Fcons (Fcons (key, json_to_lisp (value,
+                                                       object_type,
+                                                       null_object,
+                                                       false_object)),
                              result);
                 }
               result = Fnreverse (result);
@@ -731,7 +744,11 @@ json_to_lisp (json_t *json, enum json_object_type object_type)
                   /* Build the plist as value-key since we're going to
                      reverse it in the end.*/
                   result = Fcons (key, result);
-                  result = Fcons (json_to_lisp (value, object_type), result);
+                  result = Fcons (json_to_lisp (value,
+                                                object_type,
+                                                null_object,
+                                                false_object),
+                                  result);
                   SAFE_FREE ();
                 }
               result = Fnreverse (result);
@@ -749,31 +766,41 @@ json_to_lisp (json_t *json, enum json_object_type object_type)
   emacs_abort ();
 }
 
-static enum json_object_type
-json_parse_object_type (ptrdiff_t nargs, Lisp_Object *args)
-{
-  switch (nargs)
-    {
-    case 0:
-      return json_object_hashtable;
-    case 2:
+static void
+json_parse_args (ptrdiff_t nargs,
+                 Lisp_Object *args,
+                 enum json_object_type* object_type,
+                 Lisp_Object **null_object,
+                 Lisp_Object **false_object)
+{
+  if ((nargs % 2) != 0)
+    wrong_type_argument (Qplistp, Flist (nargs, args));
+
+  /* Start from the back so first value is honoured. */
+  for (ptrdiff_t i = nargs; i > 0; i -= 2) {
+    Lisp_Object key = args[i - 2];
+    Lisp_Object* value = &args[i - 1];
+    if (EQ (key, QCobject_type))
       {
-        Lisp_Object key = args[0];
-        Lisp_Object value = args[1];
-        if (!EQ (key, QCobject_type))
-          wrong_choice (list1 (QCobject_type), key);
-        if (EQ (value, Qhash_table))
-          return json_object_hashtable;
-        else if (EQ (value, Qalist))
-          return json_object_alist;
-        else if (EQ (value, Qplist))
-          return json_object_plist;
+        if (EQ (*value, Qhash_table))
+          *object_type = json_object_hashtable;
+        else if (EQ (*value, Qalist))
+          *object_type = json_object_alist;
+        else if (EQ (*value, Qplist))
+          *object_type = json_object_plist;
         else
-          wrong_choice (list3 (Qhash_table, Qalist, Qplist), value);
+          wrong_choice (list3 (Qhash_table, Qalist, Qplist), *value);
       }
-    default:
-      wrong_type_argument (Qplistp, Flist (nargs, args));
-    }
+    else if (EQ (key, QCnull_object))
+      *null_object = value;
+    else if (EQ (key, QCfalse_object))
+      *false_object = value;
+    else
+      wrong_choice (list3 (QCobject_type,
+                           QCnull_object,
+                           QCfalse_object),
+                    *value);
+  }
 }
 
 DEFUN ("json-parse-string", Fjson_parse_string, Sjson_parse_string, 1, MANY,
@@ -785,11 +812,20 @@ plist.  Its elements will be `:null', `:false', t, numbers, strings,
 or further vectors, hashtables, alists, or plists.  If there are
 duplicate keys in an object, all but the last one are ignored.  If
 STRING doesn't contain a valid JSON object, an error of type
-`json-parse-error' is signaled.  The keyword argument `:object-type'
+`json-parse-error' is signaled.
+
+The keyword argument `:object-type'
 specifies which Lisp type is used to represent objects; it can be
 `hash-table', `alist' or `plist'.
-usage: (json-parse-string STRING &key (OBJECT-TYPE \\='hash-table)) */)
-  (ptrdiff_t nargs, Lisp_Object *args)
+
+The keyword argument `:null-object' specifies which object to use
+to represent a JSON null value.  It defaults to `:null'.
+
+The keyword argument `:false-object' specifies which object to use to
+represent a JSON false value.  It defaults to `:false'.
+
+usage: (json-parse-string STRING &key (OBJECT-TYPE \\='hash-table) (NULL-OBJECT \\=':null) (FALSE-OBJECT \\=':false)) */)
+     (ptrdiff_t nargs, Lisp_Object *args)
 {
   ptrdiff_t count = SPECPDL_INDEX ();
 
@@ -811,8 +847,9 @@ usage: (json-parse-string STRING &key (OBJECT-TYPE \\='hash-table)) */)
   Lisp_Object string = args[0];
   Lisp_Object encoded = json_encode (string);
   check_string_without_embedded_nulls (encoded);
-  enum json_object_type object_type
-    = json_parse_object_type (nargs - 1, args + 1);
+  enum json_object_type object_type = json_object_hashtable;
+  Lisp_Object *null_object = NULL, *false_object = NULL;
+  json_parse_args (nargs - 1, args + 1, &object_type, &null_object, &false_object);
 
   json_error_t error;
   json_t *object = json_loads (SSDATA (encoded), 0, &error);
@@ -823,7 +860,10 @@ usage: (json-parse-string STRING &key (OBJECT-TYPE \\='hash-table)) */)
   if (object != NULL)
     record_unwind_protect_ptr (json_release_object, object);
 
-  return unbind_to (count, json_to_lisp (object, object_type));
+  return unbind_to (count, json_to_lisp (object,
+                                         object_type,
+                                         null_object? *null_object : QCnull,
+                                         false_object? *false_object : QCfalse));
 }
 
 struct json_read_buffer_data
@@ -881,7 +921,9 @@ usage: (json-parse-buffer &key (OBJECT-TYPE \\='hash-table))  */)
     }
 #endif
 
-  enum json_object_type object_type = json_parse_object_type (nargs, args);
+  enum json_object_type object_type = json_object_hashtable;
+  Lisp_Object *null_object = NULL, *false_object = NULL;
+  json_parse_args (nargs, args, &object_type, &null_object, &false_object);
 
   ptrdiff_t point = PT_BYTE;
   struct json_read_buffer_data data = {.point = point};
@@ -896,7 +938,10 @@ usage: (json-parse-buffer &key (OBJECT-TYPE \\='hash-table))  */)
   record_unwind_protect_ptr (json_release_object, object);
 
   /* Convert and then move point only if everything succeeded.  */
-  Lisp_Object lisp = json_to_lisp (object, object_type);
+  Lisp_Object lisp = json_to_lisp (object,
+                                   object_type,
+                                   null_object? *null_object : QCnull,
+                                   false_object? *false_object : QCfalse);
 
   /* Adjust point by how much we just read.  */
   point += error.position;
@@ -959,6 +1004,8 @@ syms_of_json (void)
   Fput (Qjson_parse_string, Qside_effect_free, Qt);
 
   DEFSYM (QCobject_type, ":object-type");
+  DEFSYM (QCnull_object, ":null-object");
+  DEFSYM (QCfalse_object, ":false-object");
   DEFSYM (Qalist, "alist");
   DEFSYM (Qplist, "plist");
 
diff --git a/test/src/json-tests.el b/test/src/json-tests.el
index 7a193545b1..5e7a350372 100644
--- a/test/src/json-tests.el
+++ b/test/src/json-tests.el
@@ -209,6 +209,37 @@ 'json-tests--error
     (should-not (bobp))
     (should (looking-at-p (rx " [456]" eos)))))
 
+
+
+(ert-deftest json-parse-with-custom-null-and-false-objects ()
+  (let ((input
+         "{ \"abc\" : [1, 2, true], \"def\" : null, \"abc\" : [9, false] }\n"))
+    (should (equal (json-parse-string input
+                                      :object-type 'plist
+                                      :null-object :json-null
+                                      :false-object :json-false)
+                   '(:abc [9 :json-false] :def :json-null)))
+    (should (equal (json-parse-string input
+                                      :object-type 'plist
+                                      :false-object :json-false)
+                   '(:abc [9 :json-false] :def :null)))
+    (should (equal (json-parse-string input
+                                      :object-type 'alist
+                                      :null-object :zilch)
+                   '((abc . [9 :false]) (def . :zilch))))
+    (should (equal (json-parse-string input
+                                      :object-type 'alist
+                                      :false-object nil
+                                      :null-object nil)
+                   '((abc . [9 nil]) (def))))
+    (let* ((thingy '(1 2 3))
+           (retval (json-parse-string input
+                                      :object-type 'alist
+                                      :false-object thingy
+                                      :null-object nil)))
+      (should (equal retval `((abc . [9 ,thingy]) (def))))
+      (should (eq (elt (cdr (car retval)) 1) thingy)))))
+
 (ert-deftest json-insert/signal ()
   (skip-unless (fboundp 'json-insert))
   (with-temp-buffer
-- 
2.17.0