From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED.blaine.gmane.org!not-for-mail From: yyoncho Newsgroups: gmane.emacs.bugs Subject: bug#31138: Native json slower than json.el Date: Mon, 25 Mar 2019 23:34:38 +0200 Message-ID: References: <87sh806xwa.fsf@chapu.is> <834lkf7ely.fsf@gnu.org> <878t9own1p.fsf@chapu.is> <838t9o4hvl.fsf@gnu.org> <83r2ayovkx.fsf@gnu.org> <83pnqiormy.fsf@gnu.org> <83lg15pvzr.fsf@gnu.org> <83k1gppu73.fsf@gnu.org> <83ftrdprmj.fsf@gnu.org> <83d0mhpn99.fsf@gnu.org> <83zhplo25s.fsf@gnu.org> <83va09nwg3.fsf@gnu.org> <83tvftne0j.fsf@gnu.org> <40DA9396-044E-4D00-946E-42B776B51BFA@gnu.org> <83r2awnw0w.fsf@gnu.org> <83d0mgnn31.fsf@gnu.org> <835zs7och6.fsf@gnu.org> <83tvfqnbxc.fsf@gnu.org> <83lg12n75s.fsf@gnu.org> <83h8bqn2ik.fsf@gnu.org> Mime-Version: 1.0 Content-Type: multipart/alternative; boundary="0000000000006e9f7f0584f1f9bd" Injection-Info: blaine.gmane.org; posting-host="blaine.gmane.org:195.159.176.226"; logging-data="252321"; mail-complaints-to="usenet@blaine.gmane.org" Cc: =?UTF-8?Q?S=C3=A9bastien?= Chapuis , 31138@debbugs.gnu.org To: Eli Zaretskii Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Mon Mar 25 22:50:19 2019 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([209.51.188.17]) by blaine.gmane.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1h8XU0-0013P2-Pj for geb-bug-gnu-emacs@m.gmane.org; Mon, 25 Mar 2019 22:50:17 +0100 Original-Received: from localhost ([127.0.0.1]:49000 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1h8XTz-0005Zm-NB for geb-bug-gnu-emacs@m.gmane.org; Mon, 25 Mar 2019 17:50:15 -0400 Original-Received: from eggs.gnu.org ([209.51.188.92]:44359) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1h8XTi-0004ov-Ee for bug-gnu-emacs@gnu.org; Mon, 25 Mar 2019 17:50:00 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1h8XFG-0005DN-AD for bug-gnu-emacs@gnu.org; Mon, 25 Mar 2019 17:35:03 -0400 Original-Received: from debbugs.gnu.org ([209.51.188.43]:45200) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1h8XFG-0005CY-5t for bug-gnu-emacs@gnu.org; Mon, 25 Mar 2019 17:35:02 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1h8XFF-0002lK-Nt for bug-gnu-emacs@gnu.org; Mon, 25 Mar 2019 17:35:01 -0400 X-Loop: help-debbugs@gnu.org Resent-From: yyoncho Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Mon, 25 Mar 2019 21:35:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 31138 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: moreinfo Original-Received: via spool by 31138-submit@debbugs.gnu.org id=B31138.155354969810605 (code B ref 31138); Mon, 25 Mar 2019 21:35:01 +0000 Original-Received: (at 31138) by debbugs.gnu.org; 25 Mar 2019 21:34:58 +0000 Original-Received: from localhost ([127.0.0.1]:58744 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1h8XFB-0002kz-Fn for submit@debbugs.gnu.org; Mon, 25 Mar 2019 17:34:57 -0400 Original-Received: from mail-qk1-f170.google.com ([209.85.222.170]:42996) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1h8XF9-0002ki-RN for 31138@debbugs.gnu.org; Mon, 25 Mar 2019 17:34:56 -0400 Original-Received: by mail-qk1-f170.google.com with SMTP id b74so6332961qkg.9 for <31138@debbugs.gnu.org>; Mon, 25 Mar 2019 14:34:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=2fydiyiApVfamTiqwdxNRvRdBDIYQgwlw+0nPKNIv/I=; b=VFU3vkD/e1KjMUVzASXzxkrZ7JRVhcroSjT/M2LXBnM8UjDYXlNjd1rtCpYhVr+nWo vww5WZVjLs+hwJjzBt4jBXxJcwgEsP9bSoSF1QnHGDd3NqRWv4wEJYIulYLJVIGR7bFf mhjudcxKhNSKSp25+4Cf8uX1lwWju4Ei+Cp5GSErqnOnUQr7Z8ct+F8gSI64BVPGh3zL xV7wp5ejQ83ANZvAe5Hzff1Xqf/8qWrk4hp69GrtQay45tU7qy21qKsAgK+Y+n3iJQ5w E7blsH2V3GIeUzBSPOjyMmFln7NBPKQJ0Gr4+d8+6bwaxm1A07MRpgg+IqZNye2PtYyL 6N0A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=2fydiyiApVfamTiqwdxNRvRdBDIYQgwlw+0nPKNIv/I=; b=h1P6cEMBOVFvfrupRZOghsY2NbHJt98VHXjjXVIgxW9LjzTGa27YD+Y0NRGIPJ9oAT yNbrOyjiNFCO+73iBaHJD+DpbSMrlwwPILMygQLk0BzH6jE4o+beW0vJx4dIM+gyXpBb 2Ein+AQ9GUKgZyIGrvuKoYmF7QcIL3/9tF284OHN1shuGemfQA6i/oKk9mAXTymwh5l6 06M+OahMEZjbivBqV3OoBb8Dv86IteIZs/+ZIYmOOyx+2ZFzpAfIMtWmjz2S5wNOEb+R CJwWXrJ8Ve6ZkhASgZZE6iShyrwof4YnRh2iMrlAj3AyIuGFeBn6nBpPV68S9hZNIiOl PTMw== X-Gm-Message-State: APjAAAV+zaD2v/kdf8+b6w89jJyf1/SmWERFlz3Jr7VppcoXrRW1pQGK P074Q349M+JZSIAwV0PBMp5tCUK+mM7gcGTU7Nk= X-Google-Smtp-Source: APXvYqzvX0fsq4gl7hjOg2z72kdZydT8xJY37B+Wp50qZq0zW7n4wCh758HWlQCjrxYU5iAYxHiggnEzbwigz0t6eZ0= X-Received: by 2002:a37:624e:: with SMTP id w75mr20514474qkb.11.1553549690312; Mon, 25 Mar 2019 14:34:50 -0700 (PDT) In-Reply-To: <83h8bqn2ik.fsf@gnu.org> X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.51.188.43 X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.org gmane.emacs.bugs:156792 Archived-At: --0000000000006e9f7f0584f1f9bd Content-Type: text/plain; charset="UTF-8" Hi Eli, You mean, in your setup it's twice slower than in "emacs -Q"? > Yes. > And you are saying that the changes I made have no effect on the > performance? Then what about the 100-fold slowdown you were talking > about, allegedly caused by the hooks? > The slowdown caused by the hooks was caused by the C-g and the issue does not exist and it is fixed by your patch. Otherwise, it is ~0.6 vs ~1.2secs. > > I believe that it is caused by code_convert_string . > > This needs some more specific explanation, because code_convert_string > is called in both your setup and in "emacs -Q". So it isn't > code_convert_string itself, it's something else, perhaps triggered by > code_convert_string, like those hooks I disabled. > Yes, I think that it is related to switching to different buffers. > > > I compiled emacs without that call and > > there is no difference in performance in both setups and the parsing is > 2 times faster than emacs -q with > > code_convert_string. > > It's small wonder that removing code makes functions which called that > code work faster. What does removing code_convert_string achieve > except showing this truism in action? > It is actually very helpful for me and I guess for you: 1. Now we know where to look for if we want to optimize further emacs -q parsing performance. 2. Now we know what the difference between emacs -q performance and my setup is caused by something in that function. > > > I want to discuss the native json performance in the context of lsp-mode > needs. Is it ok to do it in this thread? > > It depends on what you want to discuss, exactly. > > And I'm still confused regarding the performance that bothers you. Is > the problem the two-fold slowdown relative to "emacs -Q", or is the > problem much worse slowdown in some other situation? Is the patch I > sent useful and should be pushed, or do you no longer care about it > because it doesn't help you? > IMO the fix should go in, I summarized what are lsp-mode problems. > I feel that I no longer understand what problem we are trying to > solve, and that no matter what improvements I propose, the discussion > always ends up insisting that code_convert_string is the culprit. > ...and it actually *was* the culprit, right? Furthermore, it the performance test that I mentioned proves it is *still* the culprit... Here it is the summary of the issues that I see in native json parsing from lsp-mode (beware that I will list issues that are not relevant to that bug bug#31138). 1. Hooks might be triggered when performing json parsing(fixed by you). I hoped that this will fix 2) but apparently these are two separate issues. 2. There is a difference in performance between emacs -q and non-"emacs -q" . I would expect json parsing to be side effect free. (My guess is that it is related to switching buffers because with-temp-buffer has some positive effect). 3. Even in the best case scenario native parsing is not fast enough. In lsp-mode, you might receive 3.5mb json as a server-side response while you are typing. E. g. I might type "abc" and get 10.5 mb json. I compared the performance of nodejs parsing and it is ~10 times faster than emacs -q. 4. JSON parsing can be performed only on the UI thread. Alternatively, we (lsp-mode team) will be able to solve all these in a dynamic module if emacs module mechanism is extended to allow creating emacs lisp structures efficiently. PS: I really enjoy the ironical passive-aggressive style of communication which I guess is common for emacs-devel but I will still be trying to be constructive. I just want you to know that I won't bother answering and focus only on the stuff that will help to solve the issue... Thanks, Ivan --0000000000006e9f7f0584f1f9bd Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable

Hi Eli,

You mean, in your setup it's twice slower than in = "emacs -Q"?
=C2=A0
Yes.
= =C2=A0
And you are s= aying that the changes I made have no effect on the
performance?=C2=A0 T= hen what about the 100-fold slowdown you were talking
about, allegedly c= aused by the hooks?

The slowdown caused= by the hooks was caused by the C-g and the issue does not exist and it is = fixed by your patch. Otherwise, it is ~0.6 vs ~1.2secs.=C2=A0
=C2= =A0
> I believe t= hat it is caused by code_convert_string .

This needs some more specific explanation, because code_convert_string<= br>is called in both your setup and in "emacs -Q".=C2=A0 So it is= n't
code_convert_string itself, it's something else, perhaps tri= ggered by
code_convert_string, like those hooks I disabled.

Yes, I think that it is related to switching to dif= ferent buffers.
=C2=A0

> I compiled emacs without that call and
> there is no differe= nce in performance in both setups and the parsing is 2 times faster than em= acs -q with
> code_convert_string.

It's small wonder that removing code makes functions which called t= hat
code work faster.=C2=A0 What does removing code_convert_string achie= ve
except showing this truism in action?

=
It is actually very helpful for me and I guess for you:

=
1. Now we know where to look for if we want to optimize further = emacs -q parsing performance.
2. Now we know what the difference = between emacs -q performance and my setup is caused by something in that fu= nction.
=C2=A0

> I want to discuss the native json performance in the context of ls= p-mode needs. Is it ok to do it in this thread?

It depends on what you want to discuss, exactly.

And I'm still confused regarding the performance that bothers you.= =C2=A0 Is
the problem the two-fold slowdown relative to "emacs -Q&q= uot;, or is the
problem much worse slowdown in some other situation?=C2= =A0 Is the patch I
sent useful and should be pushed, or do you no longer= care about it
because it doesn't help you?
IMO the fix should go in, I summarized what are lsp-mode proble= ms.=C2=A0


I feel that I no longer understand what problem we are trying tosolve, and that no matter what improvements I propose, the discussion
= always ends up insisting that code_convert_string is the culprit.

...and it actually *was* the culprit, right?=C2= =A0 Furthermore, it the performance test that I mentioned proves it is *sti= ll* the culprit...

Here it is the summary of the i= ssues that I see in native json=C2=A0parsing from lsp-mode (beware that I w= ill list issues that are not relevant to that bug bug#31138).
1. Hooks might be triggered when performing json parsing(fixed = by you). I hoped that this will fix 2) but apparently these are two separat= e issues.=C2=A0
2. There is a difference in performance between e= macs -q and non-"emacs -q" .=C2=A0I would expect json parsing to = be side effect free. (My guess is that it is related to switching buffers b= ecause with-temp-buffer has some positive effect).
3. Even in the= best case scenario native parsing is not fast enough. In lsp-mode, you mig= ht receive 3.5mb=C2=A0json as a server-side response while you are typing. = E. g. I might type "abc" and get 10.5 mb=C2=A0json. I compared th= e performance of nodejs parsing and it is ~10 times faster than emacs -q.= =C2=A0
4. JSON parsing can be performed only on the UI thread.=C2= =A0

Alternatively, we (lsp-mode team) will be able= to solve all these in a dynamic module if emacs module mechanism is extend= ed to allow creating emacs lisp structures efficiently.=C2=A0

PS: I really enjoy the ironical passive-aggressive style of= communication which I guess is common for emacs-devel but I will still be = trying to be constructive. I just want you to know that I won't bother = answering and focus only on the stuff that will help to solve the issue...<= /div>

Thanks,
Ivan=C2=A0
--0000000000006e9f7f0584f1f9bd--