Parse a field in JSON given a path to the field

Parse a field in JSON given a path to the field

[Husain Alshehhi]

Does emacs provide a function that can return a path from a JSON object? I find something like this useful if I want a field from a nested, large JSON. Here is an example of a JSON string

,----
| {
| "field" : {
|   "field1" : {
|     "field2" : {
|       "field3" : "value"
|     }
|   }
| }
`----

I do something like:

,----
| ;; assume that the value is in json-string
| (let ((json (json-parse json-string))
|       (field (gethash "field" json))
|       (field1 (gethash "field1" field))
|       (field2 (gethash "field2" field1))
|       (field3 (gethash "field3" field2)))
|   ;; process field3
|   )
`----

I wonder if there is something like

,----
| (let ((field3 (json-parse-path "field/field1/field2/field3" json-string)))
|   ;; process field3
|   )
`----

Re: Parse a field in JSON given a path to the field

[Andreas Röhler]

Am 11.11.21 um 19:42 schrieb Husain Alshehhi:
> Does emacs provide a function that can return a path from a JSON object? I find something like this useful if I want a field from a nested, large JSON. Here is an example of a JSON string
>
> ,----
> | {
> | "field" : {
> |   "field1" : {
> |     "field2" : {
> |       "field3" : "value"
> |     }
> |   }
> | }
> `----
>
> I do something like:
>
> ,----
> | ;; assume that the value is in json-string
> | (let ((json (json-parse json-string))
> |       (field (gethash "field" json))
> |       (field1 (gethash "field1" field))
> |       (field2 (gethash "field2" field1))
> |       (field3 (gethash "field3" field2)))
> |   ;; process field3
> |   )
> `----
>
> I wonder if there is something like
>
> ,----
> | (let ((field3 (json-parse-path "field/field1/field2/field3" json-string)))
> |   ;; process field3
> |   )
> `----
>
>

Maybe go on with this, which puts the value into messaging for now.

Idea is to specify the nesting in order to get the value of interest:


(defun getjsonfield (nesting)
   "Get the json-value of the provided nesting."
   (interactive "p")
   (while (< (nth 0 (parse-partial-sexp (point-min) (point))) nesting)
     (down-list))
   (when (looking-at "\\([[:blank:]]*\\)\\([[:print:]]+\\)")
     (goto-char (match-end 2))
     (skip-chars-forward " \t\r\n\f")
     (when (looking-at "\\([[:blank:]]*\\)\\([^: 
]+\\)[[:blank:]]*:[[:blank:]]*\\([[:print:]]+\\)")
       (message "%s" (match-string-no-properties 2))
       (message "%s" (match-string-no-properties 3)))))

Re: Parse a field in JSON given a path to the field

[Yuri Khan]

On Fri, 12 Nov 2021 at 01:59, Husain Alshehhi <husain@alshehhi.io> wrote:
>
> Does emacs provide a function that can return a path from a JSON object? I find something like this useful if I want a field from a nested, large JSON.

I happen to have a limited implementation of JSON Pointer (RFC 6901).
The limitation is that it assumes a JSON representation produced by:

    (let ((json-object-type 'alist)
          (json-array-type 'vector)
          (json-key-type 'string)
          (json-null :json-null))
      (json-read-from-string "…"))

because at the time of writing I found that the most unambiguous
representation offered:

* distinguishes arrays from objects
* distinguishes null, false, empty array, and empty object
* preserves object property order


(If someone wants to turn this into a proper package, please be my
guest. A useful addition would be for ‘jsonpointer-eval’ to support
hash representation for JSON objects; this has better performance at
the expense of losing property order and value readability.)

```
(require 'ert)
(require 'seq)
(eval-when-compile
  (require 'pcase)
  (require 'rx)
  (require 'subr-x))

(defun jsonpointer--unescape-tilde (token)
  "Unescape ‘~0’ to ‘~’ and ‘~1’ to ‘/’ in TOKEN."
  (thread-last token
    (replace-regexp-in-string "~1" "/")
    (replace-regexp-in-string "~0" "~")))

(ert-deftest jsonpointer--unescape-tilde/test-slash ()
  (should (equal (jsonpointer--unescape-tilde "foo~10") "foo/0")))
(ert-deftest jsonpointer--unescape-tilde/test-tilde ()
  (should (equal (jsonpointer--unescape-tilde "foo~01") "foo~1")))
(ert-deftest jsonpointer--unescape-tilde/test-multiple ()
  (should (equal (jsonpointer--unescape-tilde "foo~01~10~00~11")
"foo~1/0~0/1")))

(defun jsonpointer-parse (string)
  "Parse a JSON Pointer from string representation STRING.
Return a list of reference tokens with ‘~0’ and ‘~1’ sequences unescaped.
See RFC 6901 §§ 3, 5."
  (when (not (string-empty-p string))
    (assert (string-prefix-p "/" string))
    (seq-map #'jsonpointer--unescape-tilde
             (cdr (split-string string "/")))))

(ert-deftest jsonpointer-parse/test-empty-pointer ()
  (should (equal (jsonpointer-parse "") '())))
(ert-deftest jsonpointer-parse/test-empty-token ()
  (should (equal (jsonpointer-parse "/") '(""))))
(ert-deftest jsonpointer-parse/test-unescaped ()
  (should (equal (jsonpointer-parse "/unescaped") '("unescaped"))))
(ert-deftest jsonpointer-parse/test-tilde ()
  (should (equal (jsonpointer-parse "/escaped~01") '("escaped~1"))))
(ert-deftest jsonpointer-parse/test-slash ()
  (should (equal (jsonpointer-parse "/escaped~10") '("escaped/0"))))
(ert-deftest jsonpointer-parse/test-multiple ()
  (should (equal (jsonpointer-parse "/mix~01/match~10/unescaped")
                 '("mix~1" "match/0" "unescaped"))))

(defun jsonpointer-eval (pointer json)
  "Evaluate a JSON Pointer against a JSON document.
POINTER should be a list of reference tokens
as parsed by ‘jsonpointer-parse’.
See RFC 6901 § 4.
If at any point the token cannot be followed,
signal an error."
  (seq-reduce
   (lambda (json token)
     (pcase `(,json ,token)
       (`(,(and (pred listp) (app (assoc token) `(,_ . ,item)))
          ,_)
        item)
       (`(,(pred vectorp)
          ,(and (rx bos (or "0" (seq (any "1-9") (* (any "0-9")))) eos)
                (app string-to-number
                     (and (pred (<= 0)) (pred (> (length json))) index))))
        (aref json index))
       (_ (error "Cannot follow JSON pointer: %s  token: %s  json: %s"
                 pointer token json))))
   pointer json))

(defconst jsonpointer--eval-test-json
  '(("foo" . ["bar" "baz"])
    ("" . 0)
    ("a/b" . 1)
    ("c%d" . 2)
    ("e^f" . 3)
    ("g|h" . 4)
    ("i\\j" . 5)
    ("k\"l" . 6)
    (" " . 7)
    ("m~n" . 8)))

(ert-deftest jsonpointer-eval/test-empty-pointer ()
  (should (equal (jsonpointer-eval '() jsonpointer--eval-test-json)
                 jsonpointer--eval-test-json)))
(ert-deftest jsonpointer-eval/test-unescaped ()
  (should (equal (jsonpointer-eval '("foo") jsonpointer--eval-test-json)
                 ["bar" "baz"])))
(ert-deftest jsonpointer-eval/test-multi-token ()
  (should (equal (jsonpointer-eval '("foo" "0")
                                   jsonpointer--eval-test-json)
                 "bar")))
(ert-deftest jsonpointer-eval/test-empty-token ()
  (should (equal (jsonpointer-eval '("") jsonpointer--eval-test-json)
                 0)))
(ert-deftest jsonpointer-eval/test-slash ()
  (should (equal (jsonpointer-eval '("a/b") jsonpointer--eval-test-json)
                 1)))
(ert-deftest jsonpointer-eval/test-percent ()
  (should (equal (jsonpointer-eval '("c%d") jsonpointer--eval-test-json)
                 2)))
(ert-deftest jsonpointer-eval/test-caret ()
  (should (equal (jsonpointer-eval '("e^f") jsonpointer--eval-test-json)
                 3)))
(ert-deftest jsonpointer-eval/test-pipe ()
  (should (equal (jsonpointer-eval '("g|h") jsonpointer--eval-test-json)
                 4)))
(ert-deftest jsonpointer-eval/test-backslash ()
  (should (equal (jsonpointer-eval '("i\\j") jsonpointer--eval-test-json)
                 5)))
(ert-deftest jsonpointer-eval/test-quote ()
  (should (equal (jsonpointer-eval '("k\"l") jsonpointer--eval-test-json)
                 6)))
(ert-deftest jsonpointer-eval/test-space ()
  (should (equal (jsonpointer-eval '(" ") jsonpointer--eval-test-json)
                 7)))
(ert-deftest jsonpointer-eval/test-tilde ()
  (should (equal (jsonpointer-eval '("m~n") jsonpointer--eval-test-json)
                 8)))

(provide 'jsonpointer)
```

Re: Parse a field in JSON given a path to the field

[Tim Landscheidt]

Husain Alshehhi <husain@alshehhi.io> wrote:

> Does emacs provide a function that can return a path from a JSON object? I find something like this useful if I want a field from a nested, large JSON. Here is an example of a JSON string

> ,----
> | {
> | "field" : {
> |   "field1" : {
> |     "field2" : {
> |       "field3" : "value"
> |     }
> |   }
> | }
> `----

> I do something like:

> ,----
> | ;; assume that the value is in json-string
> | (let ((json (json-parse json-string))
> |       (field (gethash "field" json))
> |       (field1 (gethash "field1" field))
> |       (field2 (gethash "field2" field1))
> |       (field3 (gethash "field3" field2)))
> |   ;; process field3
> |   )
> `----

> I wonder if there is something like

> ,----
> | (let ((field3 (json-parse-path "field/field1/field2/field3" json-string)))
> |   ;; process field3
> |   )
> `----

If the path is fixed, you could also use let-alist:

| ELISP> (let ((json (json-parse-string "{\n  \"field\": {\n    \"field1\": {\n      \"field2\": {\n        \"field3\": \"value\"\n      }\n    }\n  }\n}\n"
|                                       :object-type 'alist)))
|          (let-alist json
|            (message "field/field1/field2/field3 = %S" .field.field1.field2.field3)))
| "field/field1/field2/field3 = \"value\""
| ELISP>

Tim

Re: Parse a field in JSON given a path to the field

[2QdxY4RzWzUUiLuE]

On 2022-01-26 at 22:05:49 +0000,
Tim Landscheidt <tim@tim-landscheidt.de> wrote:

> Husain Alshehhi <husain@alshehhi.io> wrote:
> 
> > Does emacs provide a function that can return a path from a JSON object? I find something like this useful if I want a field from a nested, large JSON. Here is an example of a JSON string
> 
> > ,----
> > | {
> > | "field" : {
> > |   "field1" : {
> > |     "field2" : {
> > |       "field3" : "value"
> > |     }
> > |   }
> > | }
> > `----
> 
> > I do something like:
> 
> > ,----
> > | ;; assume that the value is in json-string
> > | (let ((json (json-parse json-string))
> > |       (field (gethash "field" json))
> > |       (field1 (gethash "field1" field))
> > |       (field2 (gethash "field2" field1))
> > |       (field3 (gethash "field3" field2)))
> > |   ;; process field3
> > |   )
> > `----
> 
> > I wonder if there is something like
> 
> > ,----
> > | (let ((field3 (json-parse-path "field/field1/field2/field3" json-string)))
> > |   ;; process field3
> > |   )
> > `----
> 
> If the path is fixed, you could also use let-alist:
> 
> | ELISP> (let ((json (json-parse-string "{\n  \"field\": {\n    \"field1\": {\n      \"field2\": {\n        \"field3\": \"value\"\n      }\n    }\n  }\n}\n"
> |                                       :object-type 'alist)))
> |          (let-alist json
> |            (message "field/field1/field2/field3 = %S" .field.field1.field2.field3)))
> | "field/field1/field2/field3 = \"value\""
> | ELISP>

I wrote this in Common Lisp; it should work (but I haven't tested it) in
Elisp:

    (defun json-drill (json keys)
      (let ((value json))
        (dolist (key keys value)
          (setf value (gethash key value)))))

The keys argument is a list of keys rather than a single string with the
keys separated by slashes.  Your example would look like this:

    (let ((json-data (json-parse json-string)))
      (let ((field3 (json-drill json '("field" "field1" "field2" "field3"))))
        ;; process field3
        ))

Obviously, whatever you pass to json-drill as keys doesn't have to be a
constant.