JSON encoding and decoding performance

JSON encoding and decoding performance

[Clément Pit--Claudel]

[-- Attachment #1: Type: text/plain, Size: 774 bytes --]

Hi emacs-devel,

I'm running into performance issues with JSON encoding and decoding.  I have a language mode whose eldoc function makes a call to a language server running as a subprocess, and small lags tend to be visible when moving quickly around the buffer.  Profiling suggests that roughly 25 percent of the time is spent encoding and decoding JSON strings.

Would there be strong objections to supporting a C JSON library in addition to the current ELisp implementation?  This was suggested at one point in https://lists.gnu.org/archive/html/bug-gnu-emacs/2015-03/msg00770.html .  It would be similar in spirit to supporting libxml-parse-region along xml-parse-region, and it could provide nice speed boosts to json-heavy ELisp programs.

Cheers,
Clément.


[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 1979 bytes --]

Re: JSON encoding and decoding performance

[Eli Zaretskii]

> From: Clément Pit--Claudel <cpitcla@mit.edu>
> Date: Sat, 15 Apr 2017 01:35:45 -0400
> 
> I'm running into performance issues with JSON encoding and decoding.  I have a language mode whose eldoc function makes a call to a language server running as a subprocess, and small lags tend to be visible when moving quickly around the buffer.  Profiling suggests that roughly 25 percent of the time is spent encoding and decoding JSON strings.

Can you show the profiles?  IME, the devil is almost always in the
details.

> Would there be strong objections to supporting a C JSON library in addition to the current ELisp implementation?  This was suggested at one point in https://lists.gnu.org/archive/html/bug-gnu-emacs/2015-03/msg00770.html .  It would be similar in spirit to supporting libxml-parse-region along xml-parse-region, and it could provide nice speed boosts to json-heavy ELisp programs.

Since we have modules now, something that wasn't available back then,
you could roll your own library, right?

Anyway, I think more details about the bottlenecks would be necessary
to make the decision on what would be the best course of action.

Thanks.

Re: JSON encoding and decoding performance

[Clément Pit--Claudel]

[-- Attachment #1.1: Type: text/plain, Size: 1211 bytes --]

On 2017-04-15 03:59, Eli Zaretskii wrote:
>> From: Clément Pit--Claudel <cpitcla@mit.edu> Date: Sat, 15 Apr 2017
>> 01:35:45 -0400
>> 
>> I'm running into performance issues with JSON encoding and
>> decoding.  I have a language mode whose eldoc function makes a call
>> to a language server running as a subprocess, and small lags tend
>> to be visible when moving quickly around the buffer.  Profiling
>> suggests that roughly 25 percent of the time is spent encoding and
>> decoding JSON strings.
> 
> Can you show the profiles?  IME, the devil is almost always in the 
> details.

Sure; I've attached one.

>> Would there be strong objections to supporting a C JSON library in
>> addition to the current ELisp implementation?  This was suggested
>> at one point in
>> https://lists.gnu.org/archive/html/bug-gnu-emacs/2015-03/msg00770.html
>> .  It would be similar in spirit to supporting libxml-parse-region
>> along xml-parse-region, and it could provide nice speed boosts to
>> json-heavy ELisp programs.
> 
> Since we have modules now, something that wasn't available back
> then, you could roll your own library, right?

That's right! I forgot about these. Let me try :)


[-- Attachment #1.2: slow-json --]
[-- Type: text/plain, Size: 35487 bytes --]


[profiler-profile "24.3" cpu #s(hash-table size 217 test equal rehash-size 1.5 rehash-threshold 0.8125 data ([redisplay sit-for execute-extended-command smex-read-and-run smex funcall-interactively "#<subr call-interactively>" ad-Advice-call-interactively apply call-interactively command-execute nil nil nil nil nil] 3 [read-event sit-for execute-extended-command smex-read-and-run smex funcall-interactively "#<subr call-interactively>" ad-Advice-call-interactively apply call-interactively command-execute nil nil nil nil nil] 6 [line-move-visual line-move previous-line funcall-interactively "#<subr call-interactively>" ad-Advice-call-interactively apply call-interactively command-execute nil nil nil nil nil nil nil] 98 [json-encode-alist json-encode-list json-encode let fstar-subp--serialize-query setq progn if let fstar-subp--query if fstar-subp--query-and-wait-1 let* fstar-subp--query-and-wait let* progn] 8 [nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil] 3278 [derived-mode-p save-place-to-alist kill-buffer "#<compiled 0x160a185>" json-read-from-string json-encode-key "#<compiled 0x391c33>" mapcar json-encode-alist json-encode-list json-encode let fstar-subp--serialize-query setq progn if] 7 [company-pre-command nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil] 2 ["#<compiled 0x391c33>" mapcar json-encode-alist json-encode-list json-encode "#<compiled 0x391c33>" mapcar json-encode-alist json-encode-list json-encode "#<compiled 0x391c33>" mapcar json-encode-alist json-encode-list json-encode let] 3 [funcall eval redisplay_internal\ \(C\ function\) nil nil nil nil nil nil nil nil nil nil nil nil nil] 28 [kill-buffer "#<compiled 0x1a07089>" json-read-from-string json-encode-key "#<compiled 0x391c33>" mapcar json-encode-alist json-encode-list json-encode let fstar-subp--serialize-query setq progn if let fstar-subp--query] 25 [eldoc-display-message-p eldoc-print-current-symbol-info "#<compiled 0x28ea7d>" apply timer-event-handler nil nil nil nil nil nil nil nil nil nil nil] 5 [let* save-current-buffer fstar--log progn if if let if save-current-buffer fstar-subp-process-queue apply timer-event-handler nil nil nil nil] 4 [ispell-decode-string ispell-get-decoded-string ispell-get-otherchars flyspell-check-pre-word-p flyspell-post-command-hook nil nil nil nil nil nil nil nil nil nil nil] 11 [apply let* save-current-buffer fstar--log progn if cond let* fstar-subp--query-and-wait let* progn if if let fstar--eldoc-function eldoc-print-current-symbol-info] 6 [json-encode-key "#<compiled 0x391c33>" mapcar json-encode-alist json-encode-list json-encode "#<compiled 0x391c33>" mapcar json-encode-alist json-encode-list json-encode let fstar-subp--serialize-query setq progn if] 18 [if let progn if fstar-subp-filter accept-process-output while if fstar-subp--query-and-wait-1 let* fstar-subp--query-and-wait let* progn if if let] 3 [forward-symbol forward-thing "#<compiled 0x18fd05d>" bounds-of-thing-at-point thing-at-point symbol-at-point let save-excursion progn unwind-protect let fstar--fqn-at-point or cons list make-fstar-subp-query] 39 [sml/generate-modified-status eval format-mode-line sml/fill-width-available sml/generate-minor-modes eval redisplay_internal\ \(C\ function\) nil nil nil nil nil nil nil nil nil] 16 [eq fstar-subp-status-eq or if progn if while let* sort fstar-subp-tracking-overlays car-safe let if save-current-buffer fstar-subp-process-queue apply] 4 [and if fstar-subp-start let fstar-subp--query if fstar-subp--query-and-wait-1 let* fstar-subp--query-and-wait let* progn if if let fstar--eldoc-function eldoc-print-current-symbol-info] 18 [line-move-partial line-move next-line funcall-interactively "#<subr call-interactively>" ad-Advice-call-interactively apply call-interactively command-execute nil nil nil nil nil nil nil] 34 ["#<compiled 0xe9f7f5>" mapconcat rm--mode-list-as-string-list sml/generate-minor-modes eval redisplay_internal\ \(C\ function\) nil nil nil nil nil nil nil nil nil nil] 72 [apply json-read-string json-read-object apply json-read condition-case progn if let* fstar-subp-json--read-response and let* while fstar-subp-json--find-response if fstar-subp-find-response] 3 [json-encode "#<compiled 0x391c33>" mapcar json-encode-alist json-encode-list json-encode "#<compiled 0x391c33>" mapcar json-encode-alist json-encode-list json-encode let fstar-subp--serialize-query setq progn if] 5 [generate-new-buffer json-read-from-string json-encode-key "#<compiled 0x391c33>" mapcar json-encode-alist json-encode-list json-encode "#<compiled 0x391c33>" mapcar json-encode-alist json-encode-list json-encode "#<compiled 0x391c33>" mapcar json-encode-alist] 6 [apply "#<compiled 0x46efc7>" redisplay_internal\ \(C\ function\) nil nil nil nil nil nil nil nil nil nil nil nil nil] 12 [set-buffer save-current-buffer fstar--log progn if if let if save-current-buffer fstar-subp-process-queue apply timer-event-handler nil nil nil nil] 4 [apply let* save-current-buffer fstar--log progn if cond let* fstar-subp-json--read-response and let* while fstar-subp-json--find-response if fstar-subp-find-response save-current-buffer] 13 [bounds-of-thing-at-point thing-at-point symbol-at-point let save-excursion progn unwind-protect let fstar--fqn-at-point or cons list make-fstar-subp-query if fstar-subp--positional-lookup-query let*] 6 [ispell-decode-string ispell-get-decoded-string ispell-get-otherchars flyspell-get-word flyspell-word flyspell-post-command-hook nil nil nil nil nil nil nil nil nil nil] 3 [json-encode-key "#<compiled 0x391c33>" mapcar json-encode-alist json-encode-list json-encode let fstar-subp--serialize-query setq progn if let fstar-subp--query if fstar-subp--query-and-wait-1 let*] 33 [rm--mode-list-as-string-list sml/generate-minor-modes eval redisplay_internal\ \(C\ function\) nil nil nil nil nil nil nil nil nil nil nil nil] 15 ["#<compiled 0x391c33>" mapcar json-encode-alist json-encode-list json-encode "#<compiled 0x391c33>" mapcar json-encode-alist json-encode-list json-encode let fstar-subp--serialize-query setq progn if let] 16 [or if progn if while let* sort fstar-subp-tracking-overlays car-safe let if save-current-buffer fstar-subp-process-queue apply timer-event-handler nil] 6 [replace-regexp-in-string if let fstar--indent-str insert let* save-current-buffer fstar--log progn if progn if fstar-subp-filter accept-process-output while if] 6 [list list make-fstar-subp-query if fstar-subp--positional-lookup-query let* progn if if let fstar--eldoc-function eldoc-print-current-symbol-info "#<compiled 0x28ea7d>" apply timer-event-handler nil] 4 [sml/generate-modified-status eval redisplay_internal\ \(C\ function\) nil nil nil nil nil nil nil nil nil nil nil nil nil] 57 [json-encode json-encode-key "#<compiled 0x391c33>" mapcar json-encode-alist json-encode-list json-encode "#<compiled 0x391c33>" mapcar json-encode-alist json-encode-list json-encode let fstar-subp--serialize-query setq progn] 13 [eval redisplay_internal\ \(C\ function\) nil nil nil nil nil nil nil nil nil nil nil nil nil nil] 7 [font-lock-extend-jit-lock-region-after-change run-hook-with-args jit-lock-after-change insert let* save-current-buffer fstar--log progn if if let if save-current-buffer fstar-subp-process-queue apply timer-event-handler] 3 [eldoc-print-current-symbol-info "#<compiled 0x28ea7d>" apply timer-event-handler nil nil nil nil nil nil nil nil nil nil nil nil] 6 [timer-inc-time timer-event-handler nil nil nil nil nil nil nil nil nil nil nil nil nil nil] 6 [kill-buffer "#<compiled 0x15823e5>" json-read-from-string json-encode-key "#<compiled 0x391c33>" mapcar json-encode-alist json-encode-list json-encode "#<compiled 0x391c33>" mapcar json-encode-alist json-encode-list json-encode let fstar-subp--serialize-query] 5 [timer-event-handler nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil] 3 [file-remote-p format-mode-line sml/fill-width-available sml/generate-minor-modes eval redisplay_internal\ \(C\ function\) nil nil nil nil nil nil nil nil nil nil] 13 [insert let* save-current-buffer fstar--log progn if cond let* fstar-subp-json--read-response and let* while fstar-subp-json--find-response if fstar-subp-find-response save-current-buffer] 12 [json-encode-list json-encode "#<compiled 0x391c33>" mapcar json-encode-alist json-encode-list json-encode "#<compiled 0x391c33>" mapcar json-encode-alist json-encode-list json-encode let fstar-subp--serialize-query setq progn] 6 [redisplay--update-region-highlight run-hook-with-args redisplay--pre-redisplay-functions apply "#<compiled 0x46efc7>" redisplay_internal\ \(C\ function\) nil nil nil nil nil nil nil nil nil nil] 1 [or fstar--log-buffer set-buffer save-current-buffer fstar--log progn if cond let* fstar-subp-json--read-response and let* while fstar-subp-json--find-response if fstar-subp-find-response] 4 [fstar--log progn if if let if save-current-buffer fstar-subp-process-queue apply timer-event-handler nil nil nil nil nil nil] 11 [sml/fill-width-available sml/generate-minor-modes eval redisplay_internal\ \(C\ function\) nil nil nil nil nil nil nil nil nil nil nil nil] 31 [replace-buffer-in-windows kill-buffer "#<compiled 0x12f2b35>" json-read-from-string json-encode-key "#<compiled 0x391c33>" mapcar json-encode-alist json-encode-list json-encode let fstar-subp--serialize-query setq progn if let] 4 [line-move-visual line-move next-line funcall-interactively "#<subr call-interactively>" ad-Advice-call-interactively apply call-interactively command-execute nil nil nil nil nil nil nil] 35 ["#<compiled 0x46ef11>" redisplay--update-region-highlight run-hook-with-args redisplay--pre-redisplay-functions apply "#<compiled 0x46efc7>" redisplay_internal\ \(C\ function\) nil nil nil nil nil nil nil nil nil] 8 [replace-regexp-in-string if let fstar--indent-str insert let* save-current-buffer fstar--log progn if cond let* fstar-subp-json--read-response and let* while] 4 [ruler-mode-ruler funcall eval redisplay_internal\ \(C\ function\) nil nil nil nil nil nil nil nil nil nil nil nil] 50 [progn if let if let if let let let progn if let* while fstar-subp-json--find-response if fstar-subp-find-response] 4 [ispell-decode-string ispell-get-decoded-string ispell-get-otherchars flyspell-check-word-p flyspell-post-command-hook nil nil nil nil nil nil nil nil nil nil nil] 3 [save-current-buffer progn if let if let if let let let progn if let* while fstar-subp-json--find-response if] 3 [json-read-from-string json-encode-key "#<compiled 0x391c33>" mapcar json-encode-alist json-encode-list json-encode "#<compiled 0x391c33>" mapcar json-encode-alist json-encode-list json-encode "#<compiled 0x391c33>" mapcar json-encode-alist json-encode-list] 4 [winner-remember winner-save-old-configurations nil nil nil nil nil nil nil nil nil nil nil nil nil nil] 11 [ido-ubiquitous-get-command-override ad-Advice-call-interactively apply call-interactively command-execute nil nil nil nil nil nil nil nil nil nil nil] 3 ["#<compiled 0x193d4ed>" json-read-from-string json-encode-key "#<compiled 0x391c33>" mapcar json-encode-alist json-encode-list json-encode "#<compiled 0x391c33>" mapcar json-encode-alist json-encode-list json-encode "#<compiled 0x391c33>" mapcar json-encode-alist] 3 [condition-case progn if let* fstar-subp-json--read-response and let* while fstar-subp-json--find-response if fstar-subp-find-response save-current-buffer progn if let if] 4 [line-move previous-line funcall-interactively "#<subr call-interactively>" ad-Advice-call-interactively apply call-interactively command-execute nil nil nil nil nil nil nil nil] 8 [or fstar--has-feature if fstar-subp--positional-lookup-query let* progn if if let fstar--eldoc-function eldoc-print-current-symbol-info "#<compiled 0x28ea7d>" apply timer-event-handler nil nil] 6 [timerp internal-timer-start-idle nil nil nil nil nil nil nil nil nil nil nil nil nil nil] 3 [json-encode-key "#<compiled 0x391c33>" mapcar json-encode-alist json-encode-list json-encode "#<compiled 0x391c33>" mapcar json-encode-alist json-encode-list json-encode "#<compiled 0x391c33>" mapcar json-encode-alist json-encode-list json-encode] 3 [ispell-get-otherchars flyspell-check-pre-word-p flyspell-post-command-hook nil nil nil nil nil nil nil nil nil nil nil nil nil] 8 [file-remote-p sml/generate-modified-status eval redisplay_internal\ \(C\ function\) nil nil nil nil nil nil nil nil nil nil nil nil] 11 [generate-new-buffer json-read-from-string json-encode-key "#<compiled 0x391c33>" mapcar json-encode-alist json-encode-list json-encode let fstar-subp--serialize-query setq progn if let fstar-subp--query if] 15 [while fstar-subp-json--find-response if fstar-subp-find-response save-current-buffer progn if let if let if let if let progn if] 4 [fstar--log progn if progn if fstar-subp-filter accept-process-output while if fstar-subp--query-and-wait-1 let* fstar-subp--query-and-wait let* progn if if] 4 [and if fstar-subp-start save-current-buffer fstar-subp-process-queue apply timer-event-handler nil nil nil nil nil nil nil nil nil] 13 ["#<compiled 0x13c92dd>" funcall if let "#<lambda 0x17714d019fc50c4>" funcall if "#<lambda 0x15dd3274c8bc504e>" funcall let if let* if let* if let*] 3 [or fstar-subp--in-issue-p not and if if let fstar--eldoc-function eldoc-print-current-symbol-info "#<compiled 0x28ea7d>" apply timer-event-handler nil nil nil nil] 5 [if let if let let let progn if let* while fstar-subp-json--find-response if fstar-subp-find-response save-current-buffer progn if] 4 [fstar--eldoc-function eldoc-print-current-symbol-info "#<compiled 0x28ea7d>" apply timer-event-handler nil nil nil nil nil nil nil nil nil nil nil] 5 [fstar-subp-find-response save-current-buffer progn if let if let if let if let progn if fstar-subp-filter accept-process-output while] 3 [insert let* save-current-buffer fstar--log progn if let fstar-subp--query if fstar-subp--query-and-wait-1 let* fstar-subp--query-and-wait let* progn if if] 6 [json-read-string apply json-read json-read-from-string json-encode-key "#<compiled 0x391c33>" mapcar json-encode-alist json-encode-list json-encode let fstar-subp--serialize-query setq progn if let] 6 [fstar-subp-tracking-overlays car-safe let if save-current-buffer fstar-subp-process-queue apply timer-event-handler nil nil nil nil nil nil nil nil] 3 [while if fstar-subp--query-and-wait-1 let* fstar-subp--query-and-wait let* progn if if let fstar--eldoc-function eldoc-print-current-symbol-info "#<compiled 0x28ea7d>" apply timer-event-handler nil] 16 [line-move next-line funcall-interactively "#<subr call-interactively>" ad-Advice-call-interactively apply call-interactively command-execute nil nil nil nil nil nil nil nil] 10 [rm-format-mode-line-entry mapcar rm--mode-list-as-string-list sml/generate-minor-modes eval redisplay_internal\ \(C\ function\) nil nil nil nil nil nil nil nil nil nil] 26 [ad-Advice-call-interactively apply call-interactively command-execute nil nil nil nil nil nil nil nil nil nil nil nil] 7 [if let fstar-subp--query if fstar-subp--query-and-wait-1 let* fstar-subp--query-and-wait let* progn if if let fstar--eldoc-function eldoc-print-current-symbol-info "#<compiled 0x28ea7d>" apply] 5 [let* fstar-subp--query-and-wait let* progn if if let fstar--eldoc-function eldoc-print-current-symbol-info "#<compiled 0x28ea7d>" apply timer-event-handler nil nil nil nil] 3 [if if let fstar--eldoc-function eldoc-print-current-symbol-info "#<compiled 0x28ea7d>" apply timer-event-handler nil nil nil nil nil nil nil nil] 6 [font-lock-fontify-keywords-region font-lock-default-fontify-region font-lock-fontify-region "#<compiled 0x15dd2b1>" run-hook-wrapped jit-lock--run-functions jit-lock-fontify-now jit-lock-function redisplay_internal\ \(C\ function\) nil nil nil nil nil nil nil] 8 [save-current-buffer fstar--log progn if cond let* fstar-subp--query-and-wait let* progn if if let fstar--eldoc-function eldoc-print-current-symbol-info "#<compiled 0x28ea7d>" apply] 7 [sml/generate-minor-modes eval redisplay_internal\ \(C\ function\) nil nil nil nil nil nil nil nil nil nil nil nil nil] 12 [forward-thing "#<compiled 0x19068c5>" bounds-of-thing-at-point thing-at-point symbol-at-point let save-excursion progn unwind-protect let fstar--fqn-at-point or cons list make-fstar-subp-query if] 2 [default-font-height default-line-height line-move-partial line-move next-line funcall-interactively "#<subr call-interactively>" ad-Advice-call-interactively apply call-interactively command-execute nil nil nil nil nil] 5 [fstar-subp-tracking-overlay-p if while let* sort fstar-subp-tracking-overlays let* or fstar-subp-unprocessed-beginning let fstar-subp-enqueue-until while let* let save-excursion fstar-subp-advance-until] 6 [progn if let* fstar-subp-highlight-issue "#<lambda 0x3f42c47ffa6b2b>" mapc fstar-subp-highlight-issues progn if let* fstar-subp-parse-and-highlight-issues if fstar-subp--overlay-continuation apply "#<compiled 0x19f74f7>" funcall] 3 [jit-lock-refontify font-lock-flush elisp--font-lock-flush-elisp-buffers run-hook-with-args do-after-load-evaluation pulse-momentary-highlight-region progn if let* fstar-subp-highlight-issue "#<lambda 0x3f42c47ffa6b2b>" mapc fstar-subp-highlight-issues progn if let*] 4 [syntax-ppss nameless--compose-as eval font-lock-fontify-keywords-region font-lock-default-fontify-region font-lock-fontify-region "#<compiled 0x18827dd>" run-hook-wrapped jit-lock--run-functions jit-lock-fontify-now jit-lock-function redisplay_internal\ \(C\ function\) nil nil nil nil] 4 [rm--propertize mapcar rm--mode-list-as-string-list sml/generate-minor-modes eval redisplay_internal\ \(C\ function\) nil nil nil nil nil nil nil nil nil nil] 4 [insert let* save-current-buffer fstar--log progn if if let if save-current-buffer fstar-subp-process-queue apply timer-event-handler nil nil nil] 1 [apply json-read condition-case progn if let* fstar-subp-json--read-response and let* while fstar-subp-json--find-response if fstar-subp-find-response save-current-buffer progn if] 4 [let* save-current-buffer fstar--log progn if cond let* fstar-subp-json--read-response and let* while fstar-subp-json--find-response if fstar-subp-find-response save-current-buffer progn] 3 [previous-line funcall-interactively "#<subr call-interactively>" ad-Advice-call-interactively apply call-interactively command-execute nil nil nil nil nil nil nil nil nil] 1 [save-current-buffer fstar--log progn if if let if save-current-buffer fstar-subp-process-queue apply timer-event-handler nil nil nil nil nil] 3 [flyspell-post-command-hook nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil] 8 [font-lock-extend-jit-lock-region-after-change run-hook-with-args jit-lock-after-change insert let* save-current-buffer fstar--log progn if let fstar-subp--query if fstar-subp--query-and-wait-1 let* fstar-subp--query-and-wait let*] 5 [timer--time-less-p timer--activate timer-activate-when-idle timer-event-handler nil nil nil nil nil nil nil nil nil nil nil nil] 3 [jit-lock-after-change insert let* save-current-buffer fstar--log progn if cond let* fstar-subp-json--read-response and let* while fstar-subp-json--find-response if fstar-subp-find-response] 3 [string-prefix-p ido-ubiquitous-spec-match ido-ubiquitous-get-command-override ad-Advice-call-interactively apply call-interactively command-execute nil nil nil nil nil nil nil nil nil] 7 [prettify-symbols--compose-symbol eval font-lock-fontify-keywords-region font-lock-default-fontify-region font-lock-fontify-region font-lock-default-fontify-buffer "#<compiled 0x4f4bc5>" font-lock-ensure if save-current-buffer fstar-highlight-string concat fstar-lookup-result-sig funcall progn if] 3 ["#<lambda 0x14848f852fc50c4>" funcall if "#<lambda 0x15dd32db8cbc504e>" funcall let if let* if let* if let* cond let* if let*] 3 [tramp-flush-file-function kill-buffer "#<compiled 0xcd46c5>" json-read-from-string json-encode-key "#<compiled 0x391c33>" mapcar json-encode-alist json-encode-list json-encode "#<compiled 0x391c33>" mapcar json-encode-alist json-encode-list json-encode let] 4 [generate-new-buffer json-read-from-string json-encode-key "#<compiled 0x391c33>" mapcar json-encode-alist json-encode-list json-encode "#<compiled 0x391c33>" mapcar json-encode-alist json-encode-list json-encode let fstar-subp--serialize-query setq] 7 [json-encode-number json-encode "#<compiled 0x391c33>" mapcar json-encode-alist json-encode-list json-encode "#<compiled 0x391c33>" mapcar json-encode-alist json-encode-list json-encode "#<compiled 0x391c33>" mapcar json-encode-alist json-encode-list] 23 [save-current-buffer fstar--log progn if cond let* fstar-subp-json--read-response and let* while fstar-subp-json--find-response if fstar-subp-find-response save-current-buffer progn if] 3 [if let if let progn if fstar-subp-filter accept-process-output while if fstar-subp--query-and-wait-1 let* fstar-subp--query-and-wait let* progn if] 3 [apply eldoc-minibuffer-message "#<compiled 0x28eb17>" apply let if fstar--eldoc-truncate-message apply eldoc-message eldoc-print-current-symbol-info "#<compiled 0x28ea7d>" apply timer-event-handler nil nil nil] 18 [font-lock-default-fontify-region font-lock-fontify-region font-lock-default-fontify-buffer "#<compiled 0x4f4bc5>" font-lock-ensure if save-current-buffer fstar-highlight-string concat fstar-lookup-result-sig funcall progn if fstar--eldoc-continuation apply "#<compiled 0x198024d>"] 9 [thing-at-point symbol-at-point let save-excursion progn unwind-protect let fstar--fqn-at-point or cons list make-fstar-subp-query if fstar-subp--positional-lookup-query let* progn] 3 [and fstar-subp-available-p and if if let fstar--eldoc-function eldoc-print-current-symbol-info "#<compiled 0x28ea7d>" apply timer-event-handler nil nil nil nil nil] 4 [progn if if let if save-current-buffer fstar-subp-process-queue apply timer-event-handler nil nil nil nil nil nil nil] 3 [window-current-scroll-bars ruler-mode-ruler funcall eval pos-visible-in-window-p line-move-partial line-move next-line funcall-interactively "#<subr call-interactively>" ad-Advice-call-interactively apply call-interactively command-execute nil nil] 3 [fstar-subp-tracking-overlay-p if while let* sort fstar-subp-tracking-overlays car-safe let if save-current-buffer fstar-subp-process-queue apply timer-event-handler nil nil nil] 4 [mapcar rm--mode-list-as-string-list sml/generate-minor-modes eval pos-visible-in-window-p line-move-partial line-move next-line funcall-interactively "#<subr call-interactively>" ad-Advice-call-interactively apply call-interactively command-execute nil nil] 5 ["#<compiled 0xe9f7f5>" mapconcat rm--mode-list-as-string-list sml/generate-minor-modes eval pos-visible-in-window-p line-move-partial line-move next-line funcall-interactively "#<subr call-interactively>" ad-Advice-call-interactively apply call-interactively command-execute nil] 3 ["#<compiled 0x14a452d>" json-read-from-string json-encode-key "#<compiled 0x391c33>" mapcar json-encode-alist json-encode-list json-encode let fstar-subp--serialize-query setq progn if let fstar-subp--query if] 12 [let progn if fstar-subp-filter accept-process-output while if fstar-subp--query-and-wait-1 let* fstar-subp--query-and-wait let* progn if if let fstar--eldoc-function] 3 [save-current-buffer fstar--log progn if progn if fstar-subp-filter accept-process-output while if fstar-subp--query-and-wait-1 let* fstar-subp--query-and-wait let* progn if] 5 [funcall flycheck-pos-tip-hide-messages nil nil nil nil nil nil nil nil nil nil nil nil nil nil] 2 [json-read-from-string json-encode-key "#<compiled 0x391c33>" mapcar json-encode-alist json-encode-list json-encode "#<compiled 0x391c33>" mapcar json-encode-alist json-encode-list json-encode let fstar-subp--serialize-query setq progn] 7 [ido-ubiquitous-spec-match ido-ubiquitous-get-command-override ad-Advice-call-interactively apply call-interactively command-execute nil nil nil nil nil nil nil nil nil nil] 5 [line-number-at-pos cons list list make-fstar-subp-query if fstar-subp--positional-lookup-query let* progn if if let fstar--eldoc-function eldoc-print-current-symbol-info "#<compiled 0x28ea7d>" apply] 4 [progn if let if let if let if let progn if fstar-subp-filter accept-process-output while if fstar-subp--query-and-wait-1] 3 [mapconcat json-join json-encode-alist json-encode-list json-encode "#<compiled 0x391c33>" mapcar json-encode-alist json-encode-list json-encode let fstar-subp--serialize-query setq progn if let] 4 [while let* sort fstar-subp-tracking-overlays length fstar--log progn if if let if save-current-buffer fstar-subp-process-queue apply timer-event-handler nil] 2 [apply eldoc-minibuffer-message "#<compiled 0x28eb17>" apply let if fstar--eldoc-truncate-message apply eldoc-message eldoc-pre-command-refresh-echo-area nil nil nil nil nil nil] 4 ["#<compiled 0x19fba8b>" json-read-from-string json-encode-key "#<compiled 0x391c33>" mapcar json-encode-alist json-encode-list json-encode "#<compiled 0x391c33>" mapcar json-encode-alist json-encode-list json-encode let fstar-subp--serialize-query setq] 3 [fstar--log progn if let fstar-subp--query if fstar-subp--query-and-wait-1 let* fstar-subp--query-and-wait let* progn if if let fstar--eldoc-function eldoc-print-current-symbol-info] 3 [eldoc-pre-command-refresh-echo-area nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil] 2 [fstar-subp--query-and-wait-1 let* fstar-subp--query-and-wait let* progn if if let fstar--eldoc-function eldoc-print-current-symbol-info "#<compiled 0x28ea7d>" apply timer-event-handler nil nil nil] 4 [flyspell-check-word-p flyspell-post-command-hook nil nil nil nil nil nil nil nil nil nil nil nil nil nil] 3 [json-encode-string json-encode json-encode-key "#<compiled 0x391c33>" mapcar json-encode-alist json-encode-list json-encode let fstar-subp--serialize-query setq progn if let fstar-subp--query if] 3 [help-at-pt-maybe-display apply timer-event-handler nil nil nil nil nil nil nil nil nil nil nil nil nil] 4 [fstar-subp-json--parse-info if and let "#<lambda 0x1471d85c4fc50c4>" funcall if "#<lambda 0x15dd33e940bc504e>" funcall let if let* if let* if let*] 3 [syntax-propertize font-lock-fontify-syntactically-region font-lock-default-fontify-region font-lock-fontify-region font-lock-default-fontify-buffer "#<compiled 0x4f4bc5>" font-lock-ensure if save-current-buffer fstar-highlight-string concat fstar-lookup-result-sig funcall progn if fstar--eldoc-continuation] 3 [apply json-read json-read-object apply json-read condition-case progn if let* fstar-subp-json--read-response and let* while fstar-subp-json--find-response if fstar-subp-find-response] 2 [fstar--log progn if cond let* fstar-subp--query-and-wait let* progn if if let fstar--eldoc-function eldoc-print-current-symbol-info "#<compiled 0x28ea7d>" apply timer-event-handler] 3 [replace-regexp-in-string font-lock-fontify-syntactically-region font-lock-default-fontify-region font-lock-fontify-region font-lock-default-fontify-buffer "#<compiled 0x4f4bc5>" font-lock-ensure if save-current-buffer fstar-highlight-string concat fstar-lookup-result-sig funcall progn if fstar--eldoc-continuation] 2 [run-hook-with-args jit-lock-after-change insert let* save-current-buffer fstar--log progn if cond let* fstar-subp--query-and-wait let* progn if if let] 3 [ispell-send-string flyspell-word flyspell-post-command-hook nil nil nil nil nil nil nil nil nil nil nil nil nil] 2 [or progn or cons list json-encode let fstar-subp--serialize-query setq progn if let fstar-subp--query if fstar-subp--query-and-wait-1 let*] 3 [save-current-buffer fstar-highlight-string concat fstar-lookup-result-sig funcall progn if fstar--eldoc-continuation apply "#<compiled 0x15225ed>" funcall if let "#<lambda 0x14ff3a435fc50c4>" funcall if] 5 [eldoc-message eldoc-print-current-symbol-info "#<compiled 0x28ea7d>" apply timer-event-handler nil nil nil nil nil nil nil nil nil nil nil] 5 [progn cons list json-encode let fstar-subp--serialize-query setq progn if let fstar-subp--query if fstar-subp--query-and-wait-1 let* fstar-subp--query-and-wait let*] 3 [backtrace-frame--internal backtrace-frame "#<compiled 0x222b91>" called-interactively-p previous-line funcall-interactively "#<subr call-interactively>" ad-Advice-call-interactively apply call-interactively command-execute nil nil nil nil nil] 2 [unrecord-window-buffer replace-buffer-in-windows kill-buffer "#<compiled 0x1085353>" json-read-from-string json-encode-key "#<compiled 0x391c33>" mapcar json-encode-alist json-encode-list json-encode let fstar-subp--serialize-query setq progn if] 3 [compose-string setq let save-current-buffer if fstar--spin-tick "#<lambda 0xb42f6e168514528>" apply timer-event-handler nil nil nil nil nil nil nil] 3 [fstar-subp--query if fstar-subp--query-and-wait-1 let* fstar-subp--query-and-wait let* progn if if let fstar--eldoc-function eldoc-print-current-symbol-info "#<compiled 0x28ea7d>" apply timer-event-handler nil] 3 [default-font-height default-line-height line-move-partial line-move previous-line funcall-interactively "#<subr call-interactively>" ad-Advice-call-interactively apply call-interactively command-execute nil nil nil nil nil] 5 [fstar--log progn if cond let* fstar-subp-json--read-response and let* while fstar-subp-json--find-response if fstar-subp-find-response save-current-buffer progn if let] 3 [timer-activate timer-event-handler nil nil nil nil nil nil nil nil nil nil nil nil nil nil] 4 [if fstar-subp-find-response save-current-buffer progn if let if let if let if let progn if fstar-subp-filter accept-process-output] 3 [replace-regexp-in-string if let fstar--indent-str insert let* save-current-buffer fstar--log progn if cond let* fstar-subp--query-and-wait let* progn if] 4 [progn if if let fstar--eldoc-function eldoc-print-current-symbol-info "#<compiled 0x28ea7d>" apply timer-event-handler nil nil nil nil nil nil nil] 4 [run-hook-with-args jit-lock-after-change insert let* save-current-buffer fstar--log progn if if let if save-current-buffer fstar-subp-process-queue apply timer-event-handler nil] 3 [json-encode mapconcat json-encode-array json-encode-list json-encode "#<compiled 0x391c33>" mapcar json-encode-alist json-encode-list json-encode "#<compiled 0x391c33>" mapcar json-encode-alist json-encode-list json-encode let] 4 [window-edges window-inside-pixel-edges window-screen-lines line-move-partial line-move next-line funcall-interactively "#<subr call-interactively>" ad-Advice-call-interactively apply call-interactively command-execute nil nil nil nil] 3 [file-remote-p redisplay_internal\ \(C\ function\) nil nil nil nil nil nil nil nil nil nil nil nil nil nil] 6 [apply let* save-current-buffer fstar--log progn if if let if save-current-buffer fstar-subp-process-queue apply timer-event-handler nil nil nil] 3 [if let fstar--indent-str insert let* save-current-buffer fstar--log progn if progn if fstar-subp-filter accept-process-output while if fstar-subp--query-and-wait-1] 3 [save-excursion progn unwind-protect let fstar--fqn-at-point or cons list make-fstar-subp-query if fstar-subp--positional-lookup-query let* progn if if let] 10 [if fstar-subp-start let fstar-subp--query if fstar-subp--query-and-wait-1 let* fstar-subp--query-and-wait let* progn if if let fstar--eldoc-function eldoc-print-current-symbol-info "#<compiled 0x28ea7d>"] 4 ["#<compiled 0x153d3b5>" mapatoms smex-detect-new-commands smex funcall-interactively "#<subr call-interactively>" ad-Advice-call-interactively apply call-interactively command-execute nil nil nil nil nil nil] 6 [mapatoms smex-detect-new-commands smex funcall-interactively "#<subr call-interactively>" ad-Advice-call-interactively apply call-interactively command-execute nil nil nil nil nil nil nil] 4 ["#<compiled 0x153c7a9>" mapatoms smex-rebuild-cache smex-update smex funcall-interactively "#<subr call-interactively>" ad-Advice-call-interactively apply call-interactively command-execute nil nil nil nil nil] 4 [mapatoms smex-rebuild-cache smex-update smex funcall-interactively "#<subr call-interactively>" ad-Advice-call-interactively apply call-interactively command-execute nil nil nil nil nil nil] 12 [smex-rebuild-cache smex-update smex funcall-interactively "#<subr call-interactively>" ad-Advice-call-interactively apply call-interactively command-execute nil nil nil nil nil nil nil] 4 [sort smex-rebuild-cache smex-update smex funcall-interactively "#<subr call-interactively>" ad-Advice-call-interactively apply call-interactively command-execute nil nil nil nil nil nil] 8 ["#<compiled 0xbd4bb5>" mapc ido-set-matches-1 ido-set-matches "#<compiled 0xbcb8d1>" ad-Advice-ido-read-internal apply ido-read-internal "#<compiled 0xbd9d95>" ad-Advice-ido-completing-read apply ido-completing-read smex-completing-read smex-read-and-run smex funcall-interactively] 4 ["#<compiled 0xbd4bb5>" mapc ido-set-matches-1 ido-set-matches ido-exhibit read-from-minibuffer "#<compiled 0xbcb8d1>" ad-Advice-ido-read-internal apply ido-read-internal "#<compiled 0xbd9d95>" ad-Advice-ido-completing-read apply ido-completing-read smex-completing-read smex-read-and-run] 41 [mapcar ido-completions ido-exhibit read-from-minibuffer "#<compiled 0xbcb8d1>" ad-Advice-ido-read-internal apply ido-read-internal "#<compiled 0xbd9d95>" ad-Advice-ido-completing-read apply ido-completing-read smex-completing-read smex-read-and-run smex funcall-interactively] 4 ["#<compiled 0xe9f7f5>" mapconcat rm--mode-list-as-string-list sml/generate-minor-modes eval redisplay_internal\ \(C\ function\) read-from-minibuffer "#<compiled 0xbcb8d1>" ad-Advice-ido-read-internal apply ido-read-internal "#<compiled 0xbd9d95>" ad-Advice-ido-completing-read apply ido-completing-read smex-completing-read] 3 [read-from-minibuffer "#<compiled 0xbcb8d1>" ad-Advice-ido-read-internal apply ido-read-internal "#<compiled 0xbd9d95>" ad-Advice-ido-completing-read apply ido-completing-read smex-completing-read smex-read-and-run smex funcall-interactively "#<subr call-interactively>" ad-Advice-call-interactively apply] 35 [mapcar ido-find-common-substring ido-set-common-completion ido-exhibit read-from-minibuffer "#<compiled 0xbcb8d1>" ad-Advice-ido-read-internal apply ido-read-internal "#<compiled 0xbd9d95>" ad-Advice-ido-completing-read apply ido-completing-read smex-completing-read smex-read-and-run smex] 40 [ido-set-matches ido-exhibit read-from-minibuffer "#<compiled 0xbcb8d1>" ad-Advice-ido-read-internal apply ido-read-internal "#<compiled 0xbd9d95>" ad-Advice-ido-completing-read apply ido-completing-read smex-completing-read smex-read-and-run smex funcall-interactively "#<subr call-interactively>"] 4 [ido-set-matches-1 ido-set-matches ido-exhibit read-from-minibuffer "#<compiled 0xbcb8d1>" ad-Advice-ido-read-internal apply ido-read-internal "#<compiled 0xbd9d95>" ad-Advice-ido-completing-read apply ido-completing-read smex-completing-read smex-read-and-run smex funcall-interactively] 4 ["#<compiled 0xbcb8d1>" ad-Advice-ido-read-internal apply ido-read-internal "#<compiled 0xbd9d95>" ad-Advice-ido-completing-read apply ido-completing-read smex-completing-read smex-read-and-run smex funcall-interactively "#<subr call-interactively>" ad-Advice-call-interactively apply call-interactively] 6 [window-size-fixed-p window--min-size-1 window-min-size "#<compiled 0x156fedd>" walk-window-tree-1 walk-window-tree-1 walk-window-tree window--sanitize-window-sizes read-from-minibuffer "#<compiled 0xbcb8d1>" ad-Advice-ido-read-internal apply ido-read-internal "#<compiled 0xbd9d95>" ad-Advice-ido-completing-read apply] 4 [Automatic\ GC] 90)) (22769 44135 170432 985000) nil]

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 1979 bytes --]

Re: JSON encoding and decoding performance

[Mark Oteiza]

Clément Pit--Claudel <cpitcla@mit.edu> writes:
>>> Would there be strong objections to supporting a C JSON library in
>>> addition to the current ELisp implementation?  This was suggested
>>> at one point in
>>> https://lists.gnu.org/archive/html/bug-gnu-emacs/2015-03/msg00770.html
>>> .  It would be similar in spirit to supporting libxml-parse-region
>>> along xml-parse-region, and it could provide nice speed boosts to
>>> json-heavy ELisp programs.
>> 
>> Since we have modules now, something that wasn't available back
>> then, you could roll your own library, right?
>
> That's right! I forgot about these. Let me try :)

Here's one: https://github.com/syohex/emacs-parson

Re: JSON encoding and decoding performance

[Eli Zaretskii]

> Cc: emacs-devel@gnu.org
> From: Clément Pit--Claudel <cpitcla@mit.edu>
> Date: Sun, 16 Apr 2017 11:43:13 -0400
> 
> > Can you show the profiles?  IME, the devil is almost always in the 
> > details.
> 
> Sure; I've attached one.

Looks like json-encode-alist takes most of the time, so the problem
might be in that function, not in the encoding itself.

Re: JSON encoding and decoding performance

[Philipp Stephani]

[-- Attachment #1: Type: text/plain, Size: 1271 bytes --]

Clément Pit--Claudel <cpitcla@mit.edu> schrieb am Sa., 15. Apr. 2017 um
08:59 Uhr:

> Hi emacs-devel,
>
> I'm running into performance issues with JSON encoding and decoding.  I
> have a language mode whose eldoc function makes a call to a language server
> running as a subprocess, and small lags tend to be visible when moving
> quickly around the buffer.  Profiling suggests that roughly 25 percent of
> the time is spent encoding and decoding JSON strings.
>
> Would there be strong objections to supporting a C JSON library in
> addition to the current ELisp implementation?  This was suggested at one
> point in
> https://lists.gnu.org/archive/html/bug-gnu-emacs/2015-03/msg00770.html .
> It would be similar in spirit to supporting libxml-parse-region along
> xml-parse-region, and it could provide nice speed boosts to json-heavy
> ELisp programs.


I think that's a great idea. I've heard complaints that JSON encoding is
too slow for low-latency applications (e.g.
https://github.com/abingham/emacs-ycmd/issues/163). This has apparently
improved a bit in recent Emacs versions, but given the importance of JSON I
think it definitely makes sense to have the fastest implementation
available, even if that means reimplementing it in C.

[-- Attachment #2: Type: text/html, Size: 1703 bytes --]

Re: JSON encoding and decoding performance

[Philipp Stephani]

[-- Attachment #1: Type: text/plain, Size: 888 bytes --]

Mark Oteiza <mvoteiza@udel.edu> schrieb am So., 16. Apr. 2017 um 19:06 Uhr:

>
> Clément Pit--Claudel <cpitcla@mit.edu> writes:
> >>> Would there be strong objections to supporting a C JSON library in
> >>> addition to the current ELisp implementation?  This was suggested
> >>> at one point in
> >>> https://lists.gnu.org/archive/html/bug-gnu-emacs/2015-03/msg00770.html
> >>> .  It would be similar in spirit to supporting libxml-parse-region
> >>> along xml-parse-region, and it could provide nice speed boosts to
> >>> json-heavy ELisp programs.
> >>
> >> Since we have modules now, something that wasn't available back
> >> then, you could roll your own library, right?
> >
> > That's right! I forgot about these. Let me try :)
>
> Here's one: https://github.com/syohex/emacs-parson


Thanks. I think the performance gain measured for that is quite convincing.

[-- Attachment #2: Type: text/html, Size: 1544 bytes --]