From: Eli Zaretskii <eliz@gnu.org>
To: "Géza Herman" <geza.herman@gmail.com>
Cc: emacs-devel@gnu.org
Subject: Re: [PATCH] Implement fast verisons of json-parse functions
Date: Sat, 30 Mar 2024 10:43:59 +0300 [thread overview]
Message-ID: <865xx4dv0g.fsf@gnu.org> (raw)
In-Reply-To: <87edc1rzig.fsf@gmail.com> (message from Herman, Géza on Sat, 23 Mar 2024 11:40:41 +0100)
> From: Herman, Géza <geza.herman@gmail.com>
> Cc: Géza Herman <geza.herman@gmail.com>,
> emacs-devel@gnu.org
> Date: Sat, 23 Mar 2024 11:40:41 +0100
>
> I attached the latest version of the parser. I reverted the Lisp
> vector based workspace to use a malloc based one, and added a note
> regarding that the GC most not run during parsing. I also
> implemented some additional optimizations, this version runs ~10x
> faster on my LSP dataset. I also run the parser on all the JSON
> files I found on my computer (~60,000 files, 1.5 GiB), no problems
> were found. The parser runs ~6.5x faster (3.5 sec vs 23.4 sec) on
> this dataset.
Thanks. I installed this on the master branch, after adding the
required commit log messages and some cleanup of unused functions.
However:
. 3 tests in test/src/json-tests.el are now failing, where they
succeeded before; see the log at the end
. the times of the relevant tests don't seem to be faster than
the libjansson version, perhaps because this is an unoptimized
build
Here are the failed tests; please see how to fix them:
Test json-parse-string/invalid-unicode backtrace:
signal(ert-test-failed (((should-error (json-parse-string "[\"\200\"
ert-fail(((should-error (json-parse-string "[\"\200\"]") :type 'json
ert--should-error-handle-error((closure ((form-description-404 (shou
(condition-case -condition- (unwind-protect (setq value-402 (apply f
(let ((errorp405 nil) (form-description-fn-406 #'(lambda nil form-de
(let (form-description-404) (let ((errorp405 nil) (form-description-
(let ((value-402 'ert-form-evaluation-aborted-403)) (let (form-descr
(let* ((fn-400 #'json-parse-string) (args-401 (condition-case err (l
(closure (t) nil (let* ((fn-395 #'fboundp) (args-396 (condition-case
#f(compiled-function () #<bytecode 0x1581ad7c20e41085>)()
handler-bind-1(#f(compiled-function () #<bytecode 0x1581ad7c20e41085
ert--run-test-internal(#s(ert--test-execution-info :test #s(ert-test
ert-run-test(#s(ert-test :name json-parse-string/invalid-unicode :do
ert-run-or-rerun-test(#s(ert--stats :selector ... :tests ... :test-m
ert-run-tests((not (or (tag :unstable) (tag :nativecomp))) #f(compil
ert-run-tests-batch((not (or (tag :unstable) (tag :nativecomp))))
ert-run-tests-batch-and-exit((not (or (tag :unstable) (tag :nativeco
eval((ert-run-tests-batch-and-exit '(not (or (tag :unstable) (tag :n
command-line-1(("-L" ";." "-l" "ert" "--eval" "(setq treesit-extra-l
command-line()
normal-top-level()
Test json-parse-string/invalid-unicode condition:
(ert-test-failed
((should-error (json-parse-string "[\"\200\"]") :type
'json-parse-error)
:form (json-parse-string "[\"\200\"]") :condition
(json-utf8-decode-error 1 3 3) :fail-reason
"the error signaled did not have the expected type"))
FAILED 7/21 json-parse-string/invalid-unicode (0.000198 sec) at src/json-tests.el:196
Test json-parse-string/null backtrace:
signal(ert-test-failed (((should-error (json-parse-string "\0") :typ
ert-fail(((should-error (json-parse-string "\0") :type 'wrong-type-a
ert--should-error-handle-error((closure ((form-description-377 (shou
(condition-case -condition- (unwind-protect (setq value-375 (apply f
(let ((errorp378 nil) (form-description-fn-379 #'(lambda nil form-de
(let (form-description-377) (let ((errorp378 nil) (form-description-
(let ((value-375 'ert-form-evaluation-aborted-376)) (let (form-descr
(let* ((fn-373 #'json-parse-string) (args-374 (condition-case err (l
(closure (t) nil (let* ((fn-368 #'fboundp) (args-369 (condition-case
#f(compiled-function () #<bytecode 0x1581ad7c20e41085>)()
handler-bind-1(#f(compiled-function () #<bytecode 0x1581ad7c20e41085
ert--run-test-internal(#s(ert--test-execution-info :test #s(ert-test
ert-run-test(#s(ert-test :name json-parse-string/null :documentation
ert-run-or-rerun-test(#s(ert--stats :selector ... :tests ... :test-m
ert-run-tests((not (or (tag :unstable) (tag :nativecomp))) #f(compil
ert-run-tests-batch((not (or (tag :unstable) (tag :nativecomp))))
ert-run-tests-batch-and-exit((not (or (tag :unstable) (tag :nativeco
eval((ert-run-tests-batch-and-exit '(not (or (tag :unstable) (tag :n
command-line-1(("-L" ";." "-l" "ert" "--eval" "(setq treesit-extra-l
command-line()
normal-top-level()
Test json-parse-string/null condition:
(ert-test-failed
((should-error (json-parse-string "\0") :type 'wrong-type-argument)
:form (json-parse-string "\0") :condition (json-end-of-file 1 1 1)
:fail-reason "the error signaled did not have the expected type"))
FAILED 8/21 json-parse-string/null (0.000281 sec) at src/json-tests.el:187
Test json-parse-string/object backtrace:
signal(ert-test-failed (((should (equal (json-parse-string input :ob
ert-fail(((should (equal (json-parse-string input :object-type 'alis
(if (unwind-protect (setq value-231 (apply fn-229 args-230)) (setq f
(let (form-description-233) (if (unwind-protect (setq value-231 (app
(let ((value-231 'ert-form-evaluation-aborted-232)) (let (form-descr
(let* ((fn-229 #'equal) (args-230 (condition-case err (list (json-pa
(let ((input "{ \"abc\" : [1, 2, true], \"def\" : null, \"abc\" : [9
(closure (t) nil (let* ((fn-209 #'fboundp) (args-210 (condition-case
#f(compiled-function () #<bytecode 0x1581ad7c20e41085>)()
handler-bind-1(#f(compiled-function () #<bytecode 0x1581ad7c20e41085
ert--run-test-internal(#s(ert--test-execution-info :test #s(ert-test
ert-run-test(#s(ert-test :name json-parse-string/object :documentati
ert-run-or-rerun-test(#s(ert--stats :selector ... :tests ... :test-m
ert-run-tests((not (or (tag :unstable) (tag :nativecomp))) #f(compil
ert-run-tests-batch((not (or (tag :unstable) (tag :nativecomp))))
ert-run-tests-batch-and-exit((not (or (tag :unstable) (tag :nativeco
eval((ert-run-tests-batch-and-exit '(not (or (tag :unstable) (tag :n
command-line-1(("-L" ";." "-l" "ert" "--eval" "(setq treesit-extra-l
command-line()
normal-top-level()
Test json-parse-string/object condition:
(ert-test-failed
((should
(equal (json-parse-string input :object-type ...) '(... ...)))
:form
(equal ((abc . [1 2 t]) (def . :null) (abc . [9 :false]))
((abc . [9 :false]) (def . :null)))
:value nil :explanation
(proper-lists-of-different-length 3 2
((abc . [1 2 t]) (def . :null)
(abc . [9 :false]))
((abc . [9 :false]) (def . :null))
first-mismatch-at 0)))
FAILED 9/21 json-parse-string/object (0.000276 sec) at src/json-tests.el:135
next prev parent reply other threads:[~2024-03-30 7:43 UTC|newest]
Thread overview: 21+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-03-19 18:23 [PATCH] Implement fast verisons of json-parse functions Herman, Géza
2024-03-19 18:47 ` Eli Zaretskii
2024-03-19 18:50 ` Herman, Géza
2024-03-19 19:28 ` Eli Zaretskii
2024-03-19 19:50 ` Herman, Géza
2024-03-20 3:37 ` Eli Zaretskii
2024-03-23 10:40 ` Herman, Géza
2024-03-23 11:11 ` Eli Zaretskii
2024-03-30 7:43 ` Eli Zaretskii [this message]
2024-03-30 10:50 ` Herman, Géza
2024-03-30 13:00 ` Eli Zaretskii
2024-03-30 18:36 ` Herman, Géza
2024-03-30 20:18 ` [External] : " Drew Adams
2024-03-30 20:51 ` Herman, Géza
2024-03-30 23:28 ` Drew Adams
2024-03-31 5:08 ` Eli Zaretskii
2024-03-31 14:51 ` Mattias Engdegård
2024-03-31 15:44 ` Eli Zaretskii
2024-04-01 8:48 ` Mattias Engdegård
2024-04-01 16:54 ` Herman, Géza
2024-03-31 17:48 ` Adam Porter
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://www.gnu.org/software/emacs/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=865xx4dv0g.fsf@gnu.org \
--to=eliz@gnu.org \
--cc=emacs-devel@gnu.org \
--cc=geza.herman@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/emacs.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).