From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: Stefan Monnier Newsgroups: gmane.emacs.devel Subject: Re: Running process filters in another thread Date: Sat, 29 Sep 2018 16:30:29 -0400 Message-ID: References: NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain X-Trace: blaine.gmane.org 1538252924 11291 195.159.176.226 (29 Sep 2018 20:28:44 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Sat, 29 Sep 2018 20:28:44 +0000 (UTC) User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.0.50 (gnu/linux) Cc: emacs-devel@gnu.org To: yyoncho Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Sat Sep 29 22:28:40 2018 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1g6Lqy-0002rN-0h for ged-emacs-devel@m.gmane.org; Sat, 29 Sep 2018 22:28:40 +0200 Original-Received: from localhost ([::1]:52529 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g6Lt4-0001kI-LL for ged-emacs-devel@m.gmane.org; Sat, 29 Sep 2018 16:30:50 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:60548) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g6Lst-0001kC-0N for emacs-devel@gnu.org; Sat, 29 Sep 2018 16:30:39 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g6Lsp-0002Zi-Q3 for emacs-devel@gnu.org; Sat, 29 Sep 2018 16:30:38 -0400 Original-Received: from chene.dit.umontreal.ca ([132.204.246.20]:41695) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g6Lsp-0002YU-KG for emacs-devel@gnu.org; Sat, 29 Sep 2018 16:30:35 -0400 Original-Received: from fmsmemgm.homelinux.net (lechon.iro.umontreal.ca [132.204.27.242]) by chene.dit.umontreal.ca (8.14.7/8.14.1) with ESMTP id w8TKUUZP007801; Sat, 29 Sep 2018 16:30:31 -0400 Original-Received: by fmsmemgm.homelinux.net (Postfix, from userid 20848) id B5F98AE099; Sat, 29 Sep 2018 16:30:29 -0400 (EDT) In-Reply-To: (yyoncho@gmail.com's message of "Sat, 29 Sep 2018 22:23:33 +0300") X-NAI-Spam-Flag: NO X-NAI-Spam-Threshold: 5 X-NAI-Spam-Score: 0 X-NAI-Spam-Rules: 2 Rules triggered EDT_SA_DN_PASS=0, RV6384=0 X-NAI-Spam-Version: 2.3.0.9418 : core <6384> : inlines <6908> : streams <1799886> : uri <2720714> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 132.204.246.20 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.org gmane.emacs.devel:230150 Archived-At: > Some of the responses/notifications might be big e. g. > 1. Diagnostics > 2. Semantic highlight data(proportional to file size) So, IIUC the scenario is that the LSP sends us large JSON data structures, but we only ever use a fairly small portion of it, so presumably a significant part of the total processing time is spent gobbling data and parsing it into an internal data structure. Beside limiting the total amount of time spent doing this (e.g. by processing the data more efficiently (e.g. in C rather in Elisp) or in parallel), the other issue is the fact that it shouldn't cause pauses in the user interaction. This can be obtained via true concurrency, or by slicing the processing so it can be interleaved finely with user interaction without any noticeable pauses. Maybe an "easy" way to solve this particular problem is to define the json parser as a "yield point". This would even allow the implementation of the json parser to spawn a separate thread and do the parsing in a truly concurrent thread (tho the GC would need to be extended to be able to deal with such truly concurrent non-Elisp threads allocating Elisp data structures, maybe this can be done by making those json threads use their own Elisp heap which gets joined into the main heap at the end). > a filter to that pipeline won't be a problem. One we (lsp-mode > contributors) move it to native json I will resume the discussion if > with some concrete testing data. IIUC eglot does use the native json parser. BTW, I'd be happy to hear about the use of jsonrpc.el in lsp-mode (either failures or successes) and more generally consolidation of effort between eglot and lsp-mode. Stefan