From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: yyoncho Newsgroups: gmane.emacs.devel Subject: Re: Running process filters in another thread Date: Sat, 29 Sep 2018 22:23:33 +0300 Message-ID: References: NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: multipart/alternative; boundary="000000000000ab24b0057707826c" X-Trace: blaine.gmane.org 1538248964 25552 195.159.176.226 (29 Sep 2018 19:22:44 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Sat, 29 Sep 2018 19:22:44 +0000 (UTC) Cc: emacs-devel@gnu.org To: monnier@iro.umontreal.ca Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Sat Sep 29 21:22:40 2018 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1g6Kp6-0006Xh-9w for ged-emacs-devel@m.gmane.org; Sat, 29 Sep 2018 21:22:40 +0200 Original-Received: from localhost ([::1]:52159 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g6KrC-000859-QL for ged-emacs-devel@m.gmane.org; Sat, 29 Sep 2018 15:24:50 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:48922) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g6KqB-00083q-LN for emacs-devel@gnu.org; Sat, 29 Sep 2018 15:23:48 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g6KqA-0006CN-R7 for emacs-devel@gnu.org; Sat, 29 Sep 2018 15:23:47 -0400 Original-Received: from mail-ot1-x335.google.com ([2607:f8b0:4864:20::335]:46800) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1g6KqA-0006B8-K5 for emacs-devel@gnu.org; Sat, 29 Sep 2018 15:23:46 -0400 Original-Received: by mail-ot1-x335.google.com with SMTP id q4-v6so9192302otf.13 for ; Sat, 29 Sep 2018 12:23:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=zuwjZosrQOnH0mgTv0B2nBPc09aqCGBE6mslaLlmnaw=; b=dLt29hg7XWwkF3yN1eWFYpJjQA5S4fNk7+cfiDTXNHdo5QpV0hgZCcVERi2ziqbnnq q5mzno7tQOUStT8G57zKk3nvUsqr0itt9S9mfxpOZ9XSS8TnNG/V9XSCzYVZYHvfPMBO lfzk6HRq6q98qnGnA16Fd+FkKZnDihn7uKGxS1fBBhs5B+lJ1NWg2HTRHVtJ2D8BDeEx Usa8pgRDeiX4qLFzoVhOPWxowv0ShvVmAQW/zJ+c6VXJI8ZU+1+Zan589H9SEleEM4so YAlztPpeW7q57eWMzdzgK+MqLt3a6bUpaqWgdkfmIexZOM02sAp/RuQCM3OG3a8rus27 v5eA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=zuwjZosrQOnH0mgTv0B2nBPc09aqCGBE6mslaLlmnaw=; b=a8m1eqnUb/0KPhtcyk+ENKHGfjC+W5wrIs9/+Vv4atVzIVn/m+UsS9cu8QSQ0fLK3W TvQPGLOMOqG4HKqGmUGDCXxYr9SYIM5fI/txziZooq016CmYbIv5UnoD9t8hJWy8HLa8 AgcmmHXO2/UaYOjxEfWWYJHvVbfXijJH0lkuJk/AeUT3m6gf7W+3gExTQ4ghLJSbO9pD +OH3j2nhhXvSyZWBa+Qx+rdHzMM88qiZpJC8ZspsN9aTppWnOtSKg19F+d46EEmkgLvn 0Huu4vI5mgI9jmAGntEtNmjBP4ZKDxvZhKlwxZscXIs3fXvWUA/USba9NkFgNWiiqZrr Nweg== X-Gm-Message-State: ABuFfojBk8RBfXD/HsQlkpSNrfz8k8olt45MnK4CtcMCbgyhOadaWJp5 jinLgB2MZjqEhUdwe5sRUR+322j+kg5TFq1hvNX7ply/zVM= X-Google-Smtp-Source: ACcGV63j1FWrFMQoVIcQl+G14mdBACgUxo8kCu3LSA5MjiIxygXa1ze1/yNDJ50LBWmLFEcE9dqfhYYP0Lg6JhlA2VY= X-Received: by 2002:a9d:7698:: with SMTP id j24-v6mr2307064otl.167.1538249024282; Sat, 29 Sep 2018 12:23:44 -0700 (PDT) In-Reply-To: X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::335 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.org gmane.emacs.devel:230148 Archived-At: --000000000000ab24b0057707826c Content-Type: text/plain; charset="UTF-8" Hi Stefan, > E.g. why would the LSP servers send us > megabytes of data? Some of the responses/notifications might be big e. g. 1. Diagnostics 2. Semantic highlight data(proportional to file size) and these combined with several concurrent LSP/DAP servers sending async notifications. But as Eli mentioned I will have to prepare some test results and also with emacs 27 native json support to see how it behaves. I have started the discussion assuming that the reading of the process output is not performed in the emacs UI thread but in OS thread and I was thinking that adding a filter to that pipeline won't be a problem. One we (lsp-mode contributors) move it to native json I will resume the discussion if with some concrete testing data. Thank you for your time. Thanks, Ivan On Sat, Sep 29, 2018 at 9:11 PM Stefan Monnier wrote: > >> It shouldn't be too hard to use a separate (OS-level) Emacs *process* > >> if you want to do the parsing without blocking the normal UI thread. > >> But it comes with other performance tradeoffs. > > > > Please correct me if I am wrong but I believe that I will still have > > to parse the input on UI thread but from a different > > format (e. g. using built-in print1/read) which in the end will have > > the same result. > > The idea is that the other process would do the bulk of the "digestion" > so the amount of data it would send to the main Emacs (the "UI thread") > would hopefully be much smaller. > > But I must admit not being sufficiently familiar with the details to be > able to discuss it seriously. E.g. why would the LSP servers send us > megabytes of data? Is it because it sends us back the whole file with > lots of detailed annotations everywhere (e.g. the kind of info we'd use > for font-lock-like purposes rather than for flymake-like purposes)? > > > Stefan > > > --000000000000ab24b0057707826c Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi Stefan,

= > E.g. why would the LSP servers send us
> megabytes of dat= a?

Some of the responses/notifications might be bi= g e. g.
1. Diagnostics
2. Semantic highlight data(propo= rtional to file size)

and these combined with seve= ral concurrent LSP/DAP servers sending async
notifications. But a= s Eli mentioned I will have to prepare some test results and
also= with emacs 27 native json support to see how it behaves. I have
= started the discussion assuming that the reading of the process output is n= ot
performed in the emacs UI thread but in OS thread and I was th= inking that adding
a filter to that pipeline won't be a probl= em. One we (lsp-mode contributors) move
it to native json I will = resume the discussion if with some concrete testing data.

Thank you for your time.

Thanks,
Ivan





On Sat, Sep 29, 2018 at 9:= 11 PM Stefan Monnier <monnier@iro.umontreal.ca> wrote:
>> It shouldn't be too hard to use a separate (O= S-level) Emacs *process*
>> if you want to do the parsing without blocking the normal UI threa= d.
>> But it comes with other performance tradeoffs.
>
> Please correct me if I am wrong but I believe that I will still have > to parse the input on UI thread but from a different
> format (e. g. using built-in print1/read) which in the end will have > the same result.

The idea is that the other process would do the bulk of the "digestion= "
so the amount of data it would send to the main Emacs (the "UI thread&= quot;)
would hopefully be much smaller.

But I must admit not being sufficiently familiar with the details to be
able to discuss it seriously.=C2=A0 E.g. why would the LSP servers send us<= br> megabytes of data?=C2=A0 Is it because it sends us back the whole file with=
lots of detailed annotations everywhere (e.g. the kind of info we'd use=
for font-lock-like purposes rather than for flymake-like purposes)?


=C2=A0 =C2=A0 =C2=A0 =C2=A0 Stefan


--000000000000ab24b0057707826c--