* bug#21616: [PATCH] Enable sorting of JSON object keys when encoding @ 2015-10-04 19:15 Simen Heggestøyl 2015-11-09 0:01 ` Dmitry Gutov 0 siblings, 1 reply; 5+ messages in thread From: Simen Heggestøyl @ 2015-10-04 19:15 UTC (permalink / raw) To: 21616 When working with JSON data, it is often convenient to be able to prettify the data, and having object keys sorted in a fixed order. This patch adds the variable `json-encoding-object-sort-key' which enables defining a sort key to be used when encoding JSON objects. Additionally, the commands `json-pretty-print-ordered' and `json-pretty-print-buffer-ordered' are added for convenience, providing prettification with alphabetical ordering of object keys. This gets rid of a lot of redundant code from `json-encode-hash-table' and `json-encode-plist' by going via alists, and using the logic of `json-encode-alist' commonly for all the structures. I was in doubt whether to make `json-pretty-print-ordered' and `json-pretty-print-buffer-ordered' their own commands, or if it would be better to provide this functionality by having `json-pretty-print' and `json-pretty-print-buffer' accept prefix arguments. I decided on the former, because I think it makes the commands easier to discover. A proposed patch follows! -- Simen From bf32702520b17e9bb975c68adb79c0d84e2cbf41 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Simen=20Heggest=C3=B8yl?= <simenheg@gmail.com> Date: Sun, 4 Oct 2015 12:21:57 +0200 Subject: [PATCH] Enable sorting of JSON object keys when encoding * lisp/json.el (json-encoding-object-sort-key): New variable for specifying a sort key for JSON objects during encoding. (json--plist-to-alist): New utility function. (json-encode-hash-table): Re-use `json-encode-alist'. (json-encode-alist): Sort output by `json-encoding-object-sort-key, when set. (json-encode-plist): Re-use `json-encode-alist'. (json-pretty-print-buffer-ordered): New command to pretty print the buffer with object keys sorted alphabetically. (json-pretty-print-ordered): New command to pretty print the region with object keys sorted alphabetically. * test/automated/json-tests.el (test-json-plist-to-alist) (test-json-encode-plist, test-json-encode-hash-table) (test-json-encode-with-sort-key): New tests. * etc/NEWS: Add an entry for the new commands. --- etc/NEWS | 4 +++ lisp/json.el | 77 +++++++++++++++++++++----------------------- test/automated/json-tests.el | 22 +++++++++++++ 3 files changed, 63 insertions(+), 40 deletions(-) diff --git a/etc/NEWS b/etc/NEWS index dbe0de3..062f17a 100644 --- a/etc/NEWS +++ b/etc/NEWS @@ -318,6 +318,10 @@ standards. --- *** `json-pretty-print' and `json-pretty-print-buffer' now maintain the ordering of object keys by default. +--- +*** New commands `json-pretty-print-ordered' and +`json-pretty-print-buffer-ordered' pretty prints JSON objects with +object keys sorted alphabetically. ** You can recompute the VC state of a file buffer with `M-x vc-refresh-state' ** Prog mode has some support for multi-mode indentation. diff --git a/lisp/json.el b/lisp/json.el index e2c7cc7..33247bf 100644 --- a/lisp/json.el +++ b/lisp/json.el @@ -52,6 +52,8 @@ ;;; Code: +(require 'map) + ;; Parameters (defvar json-object-type 'alist @@ -100,6 +102,13 @@ this around your call to `json-read' instead of `setq'ing it.") "The default indentation level for encoding. Used only when `json-encoding-pretty-print' is non-nil.") +(defvar json-encoding-object-sort-key nil + "Sort key for JSON object keys during encoding. +If nil, no sorting is performed. Else, JSON object keys are +ordered by the specified sort key during encoding. For instance, +setting this to `string<' will have JSON object keys ordered +alphabetically.") + (defvar json--encoding-current-indentation "\n" "Internally used to keep track of the current indentation level of encoding. Used only when `json-encoding-pretty-print' is non-nil.") @@ -148,6 +157,15 @@ Unlike `reverse', this keeps the property-value pairs intact." (push prop res))) res)) +(defun json--plist-to-alist (plist) + "Return an alist of the property-value pairs in PLIST." + (let (res) + (while plist + (let ((prop (pop plist)) + (val (pop plist))) + (push (cons prop val) res))) + (nreverse res))) + (defmacro json--with-indentation (body) `(let ((json--encoding-current-indentation (if json-encoding-pretty-print @@ -421,32 +439,17 @@ Please see the documentation of `json-object-type' and `json-key-type'." (defun json-encode-hash-table (hash-table) "Return a JSON representation of HASH-TABLE." - (format "{%s%s}" - (json-join - (let (r) - (json--with-indentation - (maphash - (lambda (k v) - (push (format - (if json-encoding-pretty-print - "%s%s: %s" - "%s%s:%s") - json--encoding-current-indentation - (json-encode-key k) - (json-encode v)) - r)) - hash-table)) - r) - json-encoding-separator) - (if (or (not json-encoding-pretty-print) - json-encoding-lisp-style-closings) - "" - json--encoding-current-indentation))) + (json-encode-alist (map-into hash-table 'list))) ;; List encoding (including alists and plists) (defun json-encode-alist (alist) "Return a JSON representation of ALIST." + (when json-encoding-object-sort-key + (setq alist + (sort alist (lambda (a b) + (funcall json-encoding-object-sort-key + (car a) (car b)))))) (format "{%s%s}" (json-join (json--with-indentation @@ -466,25 +469,7 @@ Please see the documentation of `json-object-type' and `json-key-type'." (defun json-encode-plist (plist) "Return a JSON representation of PLIST." - (let (result) - (json--with-indentation - (while plist - (push (concat - json--encoding-current-indentation - (json-encode-key (car plist)) - (if json-encoding-pretty-print - ": " - ":") - (json-encode (cadr plist))) - result) - (setq plist (cddr plist)))) - (concat "{" - (json-join (nreverse result) json-encoding-separator) - (if (and json-encoding-pretty-print - (not json-encoding-lisp-style-closings)) - json--encoding-current-indentation - "") - "}"))) + (json-encode-alist (json--plist-to-alist plist))) (defun json-encode-list (list) "Return a JSON representation of LIST. @@ -622,6 +607,18 @@ Advances point just past JSON object." (txt (delete-and-extract-region begin end))) (insert (json-encode (json-read-from-string txt)))))) +(defun json-pretty-print-buffer-ordered () + "Pretty-print current buffer with object keys ordered." + (interactive) + (let ((json-encoding-object-sort-key 'string<)) + (json-pretty-print-buffer))) + +(defun json-pretty-print-ordered (begin end) + "Pretty-print the region with object keys ordered." + (interactive "r") + (let ((json-encoding-object-sort-key 'string<)) + (json-pretty-print begin end))) + (provide 'json) ;;; json.el ends here diff --git a/test/automated/json-tests.el b/test/automated/json-tests.el index d1b7a2f..cdf85cc 100644 --- a/test/automated/json-tests.el +++ b/test/automated/json-tests.el @@ -28,11 +28,33 @@ (should (equal (json--plist-reverse '(:a 1 :b 2 :c 3)) '(:c 3 :b 2 :a 1)))) +(ert-deftest test-json-plist-to-alist () + (should (equal (json--plist-to-alist '()) '())) + (should (equal (json--plist-to-alist '(:a 1)) '((:a . 1)))) + (should (equal (json--plist-to-alist '(:a 1 :b 2 :c 3)) + '((:a . 1) (:b . 2) (:c . 3))))) + +(ert-deftest test-json-encode-plist () + (let ((plist '(:a 1 :b 2))) + (should (equal (json-encode plist) "{\"a\":1,\"b\":2}")))) + (ert-deftest json-encode-simple-alist () (should (equal (json-encode '((a . 1) (b . 2))) "{\"a\":1,\"b\":2}"))) +(ert-deftest test-json-encode-hash-table () + (let ((hash-table (make-hash-table)) + (json-encoding-object-sort-key 'string<)) + (puthash :a 1 hash-table) + (puthash :b 2 hash-table) + (should (equal (json-encode hash-table) "{\"a\":1,\"b\":2}")))) + +(ert-deftest test-json-encode-with-sort-key () + (let ((alist '((:a . 1) (:b . 2) (:c . 3))) + (json-encoding-object-sort-key 'string>)) + (should (equal (json-encode alist) "{\"c\":3,\"b\":2,\"a\":1}")))) + (ert-deftest json-read-simple-alist () (let ((json-object-type 'alist)) (should (equal (json-read-from-string "{\"a\": 1, \"b\": 2}") -- 2.5.3 ^ permalink raw reply related [flat|nested] 5+ messages in thread
* bug#21616: [PATCH] Enable sorting of JSON object keys when encoding 2015-10-04 19:15 bug#21616: [PATCH] Enable sorting of JSON object keys when encoding Simen Heggestøyl @ 2015-11-09 0:01 ` Dmitry Gutov 2015-11-11 18:59 ` Simen Heggestøyl 0 siblings, 1 reply; 5+ messages in thread From: Dmitry Gutov @ 2015-11-09 0:01 UTC (permalink / raw) To: Simen Heggestøyl, 21616 Hi Simen, On 10/04/2015 10:15 PM, Simen Heggestøyl wrote: > When working with JSON data, it is often convenient to be able to > prettify the data, and having object keys sorted in a fixed order. > This patch adds the variable `json-encoding-object-sort-key' which I think it's a "predicate", not a "key". See the argument names in `sort' and `cl-sort': the latter has a :key keyword argument, but it has different purpose. Call it json-[encode-]key-sort-predicate, maybe? > This gets rid of a lot of redundant code from `json-encode-hash-table' > and `json-encode-plist' by going via alists, and using the logic of > `json-encode-alist' commonly for all the structures. The unification part makes me concerned, again, from the performance standpoint. Did you do any measuring here? > I was in doubt whether to make `json-pretty-print-ordered' and > `json-pretty-print-buffer-ordered' their own commands, or if it would > be better to provide this functionality by having `json-pretty-print' > and `json-pretty-print-buffer' accept prefix arguments. I decided on > the former, because I think it makes the commands easier to discover. Yes, making them separate seems to be more in line with other Emacs commands. ^ permalink raw reply [flat|nested] 5+ messages in thread
* bug#21616: [PATCH] Enable sorting of JSON object keys when encoding 2015-11-09 0:01 ` Dmitry Gutov @ 2015-11-11 18:59 ` Simen Heggestøyl 2015-11-12 2:39 ` Dmitry Gutov 0 siblings, 1 reply; 5+ messages in thread From: Simen Heggestøyl @ 2015-11-11 18:59 UTC (permalink / raw) To: Dmitry Gutov; +Cc: 21616 [-- Attachment #1.1: Type: text/plain, Size: 1799 bytes --] Hello, Dmitry. On Mon, Nov 9, 2015 at 1:01 AM, Dmitry Gutov <dgutov@yandex.ru> wrote: > Call it json-[encode-]key-sort-predicate, maybe? Indeed, "predicate" seems more appropriate. I'll change the name. > The unification part makes me concerned, again, from the performance > standpoint. Did you do any measuring here? I didn't before now, but as you feared, the performance became measurably worse when encoding plists and hash-tables. I performed some benchmarks on the same 560K JSON file as before (http://folk.uio.no/simenheg/huge.json), with the following results: With json-object-type set to hash-table: Before the patch: (benchmark-run 100 (json-encode huge)) ⇒ (17.396825399 800 7.954568797999999) After the patch: (benchmark-run 100 (json-encode huge)) ⇒ (19.338006244000002 700 9.664285849000013) With json-object-type set to plist: Before the patch: (benchmark-run 100 (json-encode huge)) ⇒ (15.152158676 1101 7.109026906) After the patch: (benchmark-run 100 (json-encode huge)) ⇒ (18.122579887 777 8.46547749400001) How about keeping the old encoding code as the default, and only do the {hash-table, plist} → alist transform when the output is to be sorted? That keeps new code to a minimum, and the they would need to be transformed to an intermediate structure to be sorted anyway. A patch implementing this suggestion is attached. Here are the same benchmarks with the new patch applied: With json-object-type set to hash-table: (benchmark-run 100 (json-encode huge)) ⇒ (17.229601604 750 8.015517397999995) With json-object-type set to plist: (benchmark-run 100 (json-encode huge)) ⇒ (14.363185863 1101 6.992225689000007) -- Simen [-- Attachment #1.2: Type: text/html, Size: 2735 bytes --] [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #2: 0001-Enable-sorting-of-JSON-object-keys-when-encoding.patch --] [-- Type: text/x-patch, Size: 9574 bytes --] From 07ae7e0e3aa2db77a6bd2a9c97ebd34367c76972 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Simen=20Heggest=C3=B8yl?= <simenheg@gmail.com> Date: Sun, 4 Oct 2015 12:21:57 +0200 Subject: [PATCH] Enable sorting of JSON object keys when encoding * lisp/json.el (json-encoding-object-sort-predicate): New variable for specifying a sorting predicate for JSON objects during encoding. (json--plist-to-alist): New utility function. (json-encode-hash-table): Re-use `json-encode-alist' when object keys are to be sorted. (json-encode-alist): Sort output by `json-encoding-object-sort-predicate, when set. (json-encode-plist): Re-use `json-encode-alist' when object keys are to be sorted. (json-pretty-print-buffer-ordered): New command to pretty print the buffer with object keys sorted alphabetically. (json-pretty-print-ordered): New command to pretty print the region with object keys sorted alphabetically. * test/automated/json-tests.el (test-json-plist-to-alist) (test-json-encode-plist, test-json-encode-hash-table) (test-json-encode-alist-with-sort-predicate) (test-json-encode-plist-with-sort-predicate): New tests. * etc/NEWS: Add an entry for the new commands. --- etc/NEWS | 4 ++ lisp/json.el | 117 ++++++++++++++++++++++++++++--------------- test/automated/json-tests.el | 29 +++++++++++ 3 files changed, 111 insertions(+), 39 deletions(-) diff --git a/etc/NEWS b/etc/NEWS index f3df92e..46910b0 100644 --- a/etc/NEWS +++ b/etc/NEWS @@ -332,6 +332,10 @@ unlike `bookmark-set' which silently updates an existing bookmark. --- *** `json-pretty-print' and `json-pretty-print-buffer' now maintain the ordering of object keys by default. +--- +*** New commands `json-pretty-print-ordered' and +`json-pretty-print-buffer-ordered' pretty prints JSON objects with +object keys sorted alphabetically. ** You can recompute the VC state of a file buffer with `M-x vc-refresh-state' ** Prog mode has some support for multi-mode indentation. diff --git a/lisp/json.el b/lisp/json.el index 97cf993..0214a3e 100644 --- a/lisp/json.el +++ b/lisp/json.el @@ -52,6 +52,8 @@ ;;; Code: +(require 'map) + ;; Parameters (defvar json-object-type 'alist @@ -111,6 +113,13 @@ json-encoding-lisp-style-closings "If non-nil, ] and } closings will be formatted lisp-style, without indentation.") +(defvar json-encoding-object-sort-predicate nil + "Sorting predicate for JSON object keys during encoding. +If nil, no sorting is performed. Else, JSON object keys are +ordered by the specified sort predicate during encoding. For +instance, setting this to `string<' will have JSON object keys +ordered alphabetically.") + (defvar json-pre-element-read-function nil "Function called (if non-nil) by `json-read-array' and `json-read-object' right before reading a JSON array or object, @@ -159,6 +168,15 @@ json--plist-reverse (push prop res))) res)) +(defun json--plist-to-alist (plist) + "Return an alist of the property-value pairs in PLIST." + (let (res) + (while plist + (let ((prop (pop plist)) + (val (pop plist))) + (push (cons prop val) res))) + (nreverse res))) + (defmacro json--with-indentation (body) `(let ((json--encoding-current-indentation (if json-encoding-pretty-print @@ -492,32 +510,39 @@ json-read-object (defun json-encode-hash-table (hash-table) "Return a JSON representation of HASH-TABLE." - (format "{%s%s}" - (json-join - (let (r) - (json--with-indentation - (maphash - (lambda (k v) - (push (format - (if json-encoding-pretty-print - "%s%s: %s" - "%s%s:%s") - json--encoding-current-indentation - (json-encode-key k) - (json-encode v)) - r)) - hash-table)) - r) - json-encoding-separator) - (if (or (not json-encoding-pretty-print) - json-encoding-lisp-style-closings) - "" - json--encoding-current-indentation))) + (if json-encoding-object-sort-predicate + (json-encode-alist (map-into hash-table 'list)) + (format "{%s%s}" + (json-join + (let (r) + (json--with-indentation + (maphash + (lambda (k v) + (push (format + (if json-encoding-pretty-print + "%s%s: %s" + "%s%s:%s") + json--encoding-current-indentation + (json-encode-key k) + (json-encode v)) + r)) + hash-table)) + r) + json-encoding-separator) + (if (or (not json-encoding-pretty-print) + json-encoding-lisp-style-closings) + "" + json--encoding-current-indentation)))) ;; List encoding (including alists and plists) (defun json-encode-alist (alist) "Return a JSON representation of ALIST." + (when json-encoding-object-sort-predicate + (setq alist + (sort alist (lambda (a b) + (funcall json-encoding-object-sort-predicate + (car a) (car b)))))) (format "{%s%s}" (json-join (json--with-indentation @@ -537,25 +562,27 @@ json-encode-alist (defun json-encode-plist (plist) "Return a JSON representation of PLIST." - (let (result) - (json--with-indentation - (while plist - (push (concat - json--encoding-current-indentation - (json-encode-key (car plist)) - (if json-encoding-pretty-print - ": " - ":") - (json-encode (cadr plist))) - result) - (setq plist (cddr plist)))) - (concat "{" - (json-join (nreverse result) json-encoding-separator) - (if (and json-encoding-pretty-print - (not json-encoding-lisp-style-closings)) + (if json-encoding-object-sort-predicate + (json-encode-alist (json--plist-to-alist plist)) + (let (result) + (json--with-indentation + (while plist + (push (concat json--encoding-current-indentation - "") - "}"))) + (json-encode-key (car plist)) + (if json-encoding-pretty-print + ": " + ":") + (json-encode (cadr plist))) + result) + (setq plist (cddr plist)))) + (concat "{" + (json-join (nreverse result) json-encoding-separator) + (if (and json-encoding-pretty-print + (not json-encoding-lisp-style-closings)) + json--encoding-current-indentation + "") + "}")))) (defun json-encode-list (list) "Return a JSON representation of LIST. @@ -698,6 +725,18 @@ json-pretty-print (txt (delete-and-extract-region begin end))) (insert (json-encode (json-read-from-string txt)))))) +(defun json-pretty-print-buffer-ordered () + "Pretty-print current buffer with object keys ordered." + (interactive) + (let ((json-encoding-object-sort-predicate 'string<)) + (json-pretty-print-buffer))) + +(defun json-pretty-print-ordered (begin end) + "Pretty-print the region with object keys ordered." + (interactive "r") + (let ((json-encoding-object-sort-predicate 'string<)) + (json-pretty-print begin end))) + (provide 'json) ;;; json.el ends here diff --git a/test/automated/json-tests.el b/test/automated/json-tests.el index fa1f548..8f0cd6f 100644 --- a/test/automated/json-tests.el +++ b/test/automated/json-tests.el @@ -28,11 +28,40 @@ (should (equal (json--plist-reverse '(:a 1 :b 2 :c 3)) '(:c 3 :b 2 :a 1)))) +(ert-deftest test-json-plist-to-alist () + (should (equal (json--plist-to-alist '()) '())) + (should (equal (json--plist-to-alist '(:a 1)) '((:a . 1)))) + (should (equal (json--plist-to-alist '(:a 1 :b 2 :c 3)) + '((:a . 1) (:b . 2) (:c . 3))))) + +(ert-deftest test-json-encode-plist () + (let ((plist '(:a 1 :b 2))) + (should (equal (json-encode plist) "{\"a\":1,\"b\":2}")))) + (ert-deftest json-encode-simple-alist () (should (equal (json-encode '((a . 1) (b . 2))) "{\"a\":1,\"b\":2}"))) +(ert-deftest test-json-encode-hash-table () + (let ((hash-table (make-hash-table)) + (json-encoding-object-sort-predicate 'string<)) + (puthash :a 1 hash-table) + (puthash :b 2 hash-table) + (puthash :c 3 hash-table) + (should (equal (json-encode hash-table) + "{\"a\":1,\"b\":2,\"c\":3}")))) + +(ert-deftest test-json-encode-alist-with-sort-predicate () + (let ((alist '((:c . 3) (:a . 1) (:b . 2))) + (json-encoding-object-sort-predicate 'string<)) + (should (equal (json-encode alist) "{\"a\":1,\"b\":2,\"c\":3}")))) + +(ert-deftest test-json-encode-plist-with-sort-predicate () + (let ((plist '(:c 3 :a 1 :b 2)) + (json-encoding-object-sort-predicate 'string<)) + (should (equal (json-encode plist) "{\"a\":1,\"b\":2,\"c\":3}")))) + (ert-deftest json-read-simple-alist () (let ((json-object-type 'alist)) (should (equal (json-read-from-string "{\"a\": 1, \"b\": 2}") -- 2.6.2 ^ permalink raw reply related [flat|nested] 5+ messages in thread
* bug#21616: [PATCH] Enable sorting of JSON object keys when encoding 2015-11-11 18:59 ` Simen Heggestøyl @ 2015-11-12 2:39 ` Dmitry Gutov 2015-11-12 17:37 ` Simen Heggestøyl 0 siblings, 1 reply; 5+ messages in thread From: Dmitry Gutov @ 2015-11-12 2:39 UTC (permalink / raw) To: Simen Heggestøyl; +Cc: 21616 Hi Simen, On 11/11/2015 08:59 PM, Simen Heggestøyl wrote: > How about keeping the old encoding code as the default, and only do the > {hash-table, plist} → alist transform when the output is to be sorted? > That keeps new code to a minimum, and the they would need to be > transformed to an intermediate structure to be sorted anyway. > > A patch implementing this suggestion is attached. Here are the same > benchmarks with the new patch applied: LGTM, please install. Thanks. ^ permalink raw reply [flat|nested] 5+ messages in thread
* bug#21616: [PATCH] Enable sorting of JSON object keys when encoding 2015-11-12 2:39 ` Dmitry Gutov @ 2015-11-12 17:37 ` Simen Heggestøyl 0 siblings, 0 replies; 5+ messages in thread From: Simen Heggestøyl @ 2015-11-12 17:37 UTC (permalink / raw) To: Dmitry Gutov; +Cc: 21616-done [-- Attachment #1: Type: text/plain, Size: 644 bytes --] Thanks for your time, Dmitry. Installed! On Thu, Nov 12, 2015 at 3:39 AM, Dmitry Gutov <dgutov@yandex.ru> wrote: > Hi Simen, > > On 11/11/2015 08:59 PM, Simen Heggestøyl wrote: > >> How about keeping the old encoding code as the default, and only do >> the >> {hash-table, plist} → alist transform when the output is to be >> sorted? >> That keeps new code to a minimum, and the they would need to be >> transformed to an intermediate structure to be sorted anyway. >> >> A patch implementing this suggestion is attached. Here are the same >> benchmarks with the new patch applied: > > LGTM, please install. Thanks. [-- Attachment #2: Type: text/html, Size: 742 bytes --] ^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2015-11-12 17:37 UTC | newest] Thread overview: 5+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2015-10-04 19:15 bug#21616: [PATCH] Enable sorting of JSON object keys when encoding Simen Heggestøyl 2015-11-09 0:01 ` Dmitry Gutov 2015-11-11 18:59 ` Simen Heggestøyl 2015-11-12 2:39 ` Dmitry Gutov 2015-11-12 17:37 ` Simen Heggestøyl
Code repositories for project(s) associated with this external index https://git.savannah.gnu.org/cgit/emacs.git https://git.savannah.gnu.org/cgit/emacs/org-mode.git This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.