From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Simen =?UTF-8?Q?Heggest=C3=B8yl?= Newsgroups: gmane.emacs.bugs Subject: bug#21798: 25.0.50; [PATCH] Add support for retrieving paths to JSON elements Date: Sat, 07 Nov 2015 19:50:15 +0100 Message-ID: <1446922215.4732.0@smtp.gmail.com> References: <1446281162.2607.0@smtp.gmail.com> <5634CEE7.3070200@yandex.ru> <1446407553.4906.0@smtp.gmail.com> <1446420466.13180.0@smtp.gmail.com> <56381558.7050607@yandex.ru> <1446827488.11140.2@smtp.gmail.com> <563CE043.9060404@yandex.ru> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="=-mXzLxhu9TJt1K29PGzeR" X-Trace: ger.gmane.org 1446922284 8165 80.91.229.3 (7 Nov 2015 18:51:24 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Sat, 7 Nov 2015 18:51:24 +0000 (UTC) Cc: 21798@debbugs.gnu.org To: Dmitry Gutov Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Sat Nov 07 19:51:13 2015 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1Zv8Zz-0001sh-U5 for geb-bug-gnu-emacs@m.gmane.org; Sat, 07 Nov 2015 19:51:12 +0100 Original-Received: from localhost ([::1]:44949 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Zv8Zz-00039Z-Cz for geb-bug-gnu-emacs@m.gmane.org; Sat, 07 Nov 2015 13:51:11 -0500 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:53103) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Zv8Zu-00039M-IE for bug-gnu-emacs@gnu.org; Sat, 07 Nov 2015 13:51:08 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Zv8Zq-0001bt-Em for bug-gnu-emacs@gnu.org; Sat, 07 Nov 2015 13:51:06 -0500 Original-Received: from debbugs.gnu.org ([208.118.235.43]:38163) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Zv8Zq-0001bp-8s for bug-gnu-emacs@gnu.org; Sat, 07 Nov 2015 13:51:02 -0500 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.80) (envelope-from ) id 1Zv8Zp-0002cE-WB for bug-gnu-emacs@gnu.org; Sat, 07 Nov 2015 13:51:02 -0500 X-Loop: help-debbugs@gnu.org Resent-From: Simen =?UTF-8?Q?Heggest=C3=B8yl?= Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sat, 07 Nov 2015 18:51:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 21798 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: patch Original-Received: via spool by 21798-submit@debbugs.gnu.org id=B21798.144692222310000 (code B ref 21798); Sat, 07 Nov 2015 18:51:01 +0000 Original-Received: (at 21798) by debbugs.gnu.org; 7 Nov 2015 18:50:23 +0000 Original-Received: from localhost ([127.0.0.1]:57104 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Zv8ZB-0002bC-Op for submit@debbugs.gnu.org; Sat, 07 Nov 2015 13:50:23 -0500 Original-Received: from mail-lb0-f178.google.com ([209.85.217.178]:34333) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Zv8Z8-0002b2-TK for 21798@debbugs.gnu.org; Sat, 07 Nov 2015 13:50:20 -0500 Original-Received: by lbbwb3 with SMTP id wb3so76727315lbb.1 for <21798@debbugs.gnu.org>; Sat, 07 Nov 2015 10:50:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:subject:to:cc:message-id:in-reply-to:references :mime-version:content-type; bh=UYSR6gOn8Y1/5GvzTc0jYVKsnX1YPJ6aVlXa/XtCq64=; b=yYwMpCJb46QOYX3pgA5YZtQuPD3iwcMIL4GzC85QfVk7O2ubG38ZYluhPBUCTznNT8 82wzCKuANBdy4t51LrDbksllNQslRlEZE0osNcN3tNYgjGMuo/LdcDN2yt7QzXRnn/ks 0N/1IFK81r8pb9tI7ERogvdcCBYVDj2c+qEYUs/X+yzFddp5ue+NufYbs0o+EaCPBhuz H2zRU67W1uZeFRi2t6dADGsKq8vp15ALp46Y01j8WlHbQ7XypCBSwoZDoq5qWxYJpPLw 9wBxodlSW2TREme6780EFiojwbrtfCElzOfiClWvDPfAQy8mi/J9CSfGmErD2Yps2+x5 O5PA== X-Received: by 10.112.64.72 with SMTP id m8mr9856923lbs.41.1446922218041; Sat, 07 Nov 2015 10:50:18 -0800 (PST) Original-Received: from [192.168.100.7] (cm-84.210.143.4.getinternet.no. [84.210.143.4]) by smtp.gmail.com with ESMTPSA id q197sm1014622lfd.36.2015.11.07.10.50.16 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 07 Nov 2015 10:50:17 -0800 (PST) In-Reply-To: <563CE043.9060404@yandex.ru> X-Mailer: geary/0.10.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 208.118.235.43 X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.bugs:108548 Archived-At: --=-mXzLxhu9TJt1K29PGzeR Content-Type: multipart/alternative; boundary="=-DqbhF2blkE3mXuhW8IMG" --=-DqbhF2blkE3mXuhW8IMG Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable On Fri, Nov 6, 2015 at 6:15 PM, Dmitry Gutov wrote: > That sounds fine to me in terms of design, but it might add some=20 > performance overhead. So some testing is needed. Good! A revised patch is attached. Benchmarks follow below, with the same setup as last time. Before the patch: (benchmark-run 100 (json-read-from-string huge-json)) =E2=87=92 (16.84457266 1007 4.886441912999999) After the patch: (benchmark-run 100 (json-read-from-string huge-json)) =E2=87=92 (16.905379125000003 1007 4.722544520000007) -- Simen = --=-DqbhF2blkE3mXuhW8IMG Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable On Fri, Nov 6, 2015 at 6:15 PM, Dmitry Gutov <dgutov@yandex.ru> wrote= :
That sounds fine to me in terms of design, but it might add s= ome performance overhead. So some testing is needed.

=
Good! A revised patch is attached.

Bench= marks follow below, with the same setup as last time.

<= div>Before the patch:

(benchmark-run 100 (json-rea= d-from-string huge-json))
     =E2=87=92 (16.84457= 266 1007 4.886441912999999)

After the patch:
=

(benchmark-run 100 (json-read-from-string huge-json))
     =E2=87=92 (16.905379125000003 1007 4.722544520= 000007)

-- Simen
= --=-DqbhF2blkE3mXuhW8IMG-- --=-mXzLxhu9TJt1K29PGzeR Content-Type: text/x-patch Content-Disposition: attachment; filename=0001-Add-support-for-retrieving-paths-to-JSON-elements.patch Content-Transfer-Encoding: quoted-printable >From b6c10884b48770143468d93c6a816564834c77be Mon Sep 17 00:00:00 2001 From: =3D?UTF-8?q?Simen=3D20Heggest=3DC3=3DB8yl?=3D Date: Sun, 25 Oct 2015 14:44:59 +0100 Subject: [PATCH] Add support for retrieving paths to JSON elements Add support for retrieving the path to a JSON element. This can for instance be useful to retrieve paths in deeply nested JSON structures. * lisp/json.el (json-pre-read-function, json-post-read-function): New variables to hold pre- and post read callback functions for `json-read-array' and `json-read-object'. (json--path): New variable used internally by `json-path-to-position'. (json--record-path, json--check-position): New functions used internally by `json-path-to-position'. (json-path-to-position): New function for retrieving the path to a JSON element at a given position. (json-read-object, json-read-array): Call `json-pre-read-function' and `json-post-read-function' when set. * test/automated/json-tests.el (test-json-path-to-position-with-objects) (test-json-path-to-position-with-arrays): New tests for `json-path-to-position'. --- lisp/json.el | 75 ++++++++++++++++++++++++++++++++++++++++= ++++ test/automated/json-tests.el | 14 +++++++++ 2 files changed, 89 insertions(+) diff --git a/lisp/json.el b/lisp/json.el index b23d12a..4cc4f97 100644 --- a/lisp/json.el +++ b/lisp/json.el @@ -111,6 +111,16 @@ json-encoding-lisp-style-closings "If non-nil, ] and } closings will be formatted lisp-style, without indentation.") =20 +(defvar json-pre-read-function nil + "Function called (if non-nil) by `json-read-array' and +`json-read-object' right before reading a JSON array or object, +respectively.") + +(defvar json-post-read-function nil + "Function called (if non-nil) by `json-read-array' and +`json-read-object' right after reading a JSON array or object, +respectively.") + =0C =20 ;;; Utilities @@ -196,6 +206,61 @@ 'json-end-of-file =20 =0C =20 +;;; Paths + +(defvar json--path '() + "Used internally by `json-path-to-position' to keep track of +the path during recursive calls to `json-read'.") + +(defun json--record-path (key) + "Record the KEY to the current JSON path. Used internally by +`json-path-to-position'." + (push (cons (point) key) json--path)) + +(defun json--check-position (position) + "Check if the last parsed JSON structure passed POSITION. Used +internally by `json-path-to-position'." + (let ((start (caar json--path))) + (when (< start position (+ (point) 1)) + (throw :json-path (list :path (nreverse (mapcar #'cdr json--path)) + :match-start start + :match-end (point))))) + (pop json--path)) + +(defun json-path-to-position (position &optional string) + "Return the path to the JSON element at POSITION. + +When STRING is provided, return the path to the position in the +string, else to the position in the current buffer. + +The return value is a property list with the following +properties: + +:path -- A list of strings and numbers forming the path to + the JSON element at the given position. Strings + denote object names, while numbers denote array + indexes. + +:match-start -- Position where the matched JSON element begins. + +:match-end -- Position where the matched JSON element ends. + +This can for instance be useful to determine the path to a JSON +element in a deeply nested structure." + (save-excursion + (unless string + (goto-char (point-min))) + (let* ((json--path '()) + (json-pre-read-function #'json--record-path) + (json-post-read-function + (apply-partially #'json--check-position position)) + (path (catch :json-path + (if string + (json-read-from-string string) + (json-read))))) + (when (plist-get path :path) + path)))) + ;;; Keywords =20 (defvar json-keywords '("true" "false" "null") @@ -403,7 +468,12 @@ json-read-object (if (char-equal (json-peek) ?:) (json-advance) (signal 'json-object-format (list ":" (json-peek)))) + (json-skip-whitespace) + (when json-pre-read-function + (funcall json-pre-read-function key)) (setq value (json-read)) + (when json-post-read-function + (funcall json-post-read-function)) (setq elements (json-add-to-object elements key value)) (json-skip-whitespace) (unless (char-equal (json-peek) ?}) @@ -509,7 +579,12 @@ json-read-array ;; read values until "]" (let (elements) (while (not (char-equal (json-peek) ?\])) + (json-skip-whitespace) + (when json-pre-read-function + (funcall json-pre-read-function (length elements))) (push (json-read) elements) + (when json-post-read-function + (funcall json-post-read-function)) (json-skip-whitespace) (unless (char-equal (json-peek) ?\]) (if (char-equal (json-peek) ?,) diff --git a/test/automated/json-tests.el b/test/automated/json-tests.el index d1b7a2f..e0672dd 100644 --- a/test/automated/json-tests.el +++ b/test/automated/json-tests.el @@ -49,5 +49,19 @@ (should (equal (json-read-from-string "\"\\nasd\\u0444\\u044b\\u0432fgh\= \t\"") "\nasd=D1=84=D1=8B=D0=B2fgh\t"))) =20 +(ert-deftest test-json-path-to-position-with-objects () + (let* ((json-string "{\"foo\": {\"bar\": {\"baz\": \"value\"}}}") + (matched-path (json-path-to-position 32 json-string))) + (should (equal (plist-get matched-path :path) '("foo" "bar" "baz"))) + (should (equal (plist-get matched-path :match-start) 25)) + (should (equal (plist-get matched-path :match-end) 32)))) + +(ert-deftest test-json-path-to-position-with-arrays () + (let* ((json-string "{\"foo\": [\"bar\", [\"baz\"]]}") + (matched-path (json-path-to-position 20 json-string))) + (should (equal (plist-get matched-path :path) '("foo" 1 0))) + (should (equal (plist-get matched-path :match-start) 18)) + (should (equal (plist-get matched-path :match-end) 23)))) + (provide 'json-tests) ;;; json-tests.el ends here --=20 2.6.2 = --=-mXzLxhu9TJt1K29PGzeR--