From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED.blaine.gmane.org!not-for-mail From: yyoncho Newsgroups: gmane.emacs.devel Subject: Re: Using incremental parsing in Emacs Date: Sun, 5 Jan 2020 23:12:56 +0200 Message-ID: References: <83blrkj1o1.fsf@gnu.org> <20200105141900.GA71296@breton.holly.idiocy.org> <83eewdg3vy.fsf@gnu.org> <835zhpg0fz.fsf@gnu.org> Mime-Version: 1.0 Content-Type: multipart/alternative; boundary="00000000000073a72a059b6b0225" Injection-Info: blaine.gmane.org; posting-host="blaine.gmane.org:195.159.176.226"; logging-data="165503"; mail-complaints-to="usenet@blaine.gmane.org" Cc: Eli Zaretskii , alan@idiocy.org, emacs-devel To: Stefan Monnier Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Sun Jan 05 22:13:44 2020 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([209.51.188.17]) by blaine.gmane.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.89) (envelope-from ) id 1ioDDU-000gto-6C for ged-emacs-devel@m.gmane.org; Sun, 05 Jan 2020 22:13:44 +0100 Original-Received: from localhost ([::1]:45334 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ioDDT-0006jI-4q for ged-emacs-devel@m.gmane.org; Sun, 05 Jan 2020 16:13:43 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:60274) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ioDCz-0006JR-M0 for emacs-devel@gnu.org; Sun, 05 Jan 2020 16:13:14 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ioDCy-0003I1-Eg for emacs-devel@gnu.org; Sun, 05 Jan 2020 16:13:13 -0500 Original-Received: from mail-lj1-x22d.google.com ([2a00:1450:4864:20::22d]:36070) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1ioDCw-0003GA-EI; Sun, 05 Jan 2020 16:13:10 -0500 Original-Received: by mail-lj1-x22d.google.com with SMTP id r19so48953401ljg.3; Sun, 05 Jan 2020 13:13:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=lkbOxJA7/fYkelqltGjKFbwc4Kx1tbzhA7RbGQAKsSo=; b=b5pRerreS8+w/KzEfC9uh2IL5Va+dfl46cJWJyaSVdaU9serOC934zZX1CbbwWrYoG bfyvao5X2MZLnkeUYRkfJnOHcaikfTGX7XqmsXpgTfFRgBa9WIbuuNof54vhI3CEgTKN YK0EHrvkieYmrqhPBFKV8mvCUWAMIGRZqDVhJkkw6e8CbJFfVARQYxBLhcPAnNu0+n8G 9n/s4YWMAnrmK8M9so2AWQE1PRqeYCxov7hNtFTunlioee25rgba9CqjTLqur9GRnDf1 f7Jsvlw9ixjEkZpa6mUZ+YePBV+oxhzAPVrDWaGd2hDuMv+GJ3RpJBqeSee/NwN3eBAD rkqA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=lkbOxJA7/fYkelqltGjKFbwc4Kx1tbzhA7RbGQAKsSo=; b=sDVFp/gjtSC+PE5jWyBa/Kaz7hkDB/uejRyjeTNVcI8RJjKv8x9KYwOIxF5dLTjID5 99nL6XyQgmAcetMUDHOS9VUfOU4DB80LCKcZWLhWDEvwKYeNm1jjEOuqpc+q3ikPjHhp tk/rZ7cYPePAYhtYkcakS6ZlPp8cVISkd+wV1lMP+MjwcuNbfiyP9GIObubsO/BcYJlL r2TJrShf/zS1nmq4thRoYLiPSlqVV88L2Q9g1IYWYsmzg4gqg+JjmIze86GMrDpaJOgB Tx4cxNuzsf5Zn3lyiixJIsZRPGL27+rcM6SvqjkvxYzY7oxjV2wXgRoZ7n2SoHCZPe21 JmSQ== X-Gm-Message-State: APjAAAWb4WpZ1IpNzAp4AtnYGwcedGWz9XMKzB1eIQUjyMONNFQN/8eA SRY9DbTsDwWT8FCDoap0+ct3PObNBZ/y87CyWKo= X-Google-Smtp-Source: APXvYqwaM6pRjM6NFfheF7RhBeUmpBP6sbm1fctyFjqkKs5h42RQ/pJNtt8AuM3kl77jqY3a1pvDRwc+LMHA9pxs8wA= X-Received: by 2002:a2e:2c04:: with SMTP id s4mr60257380ljs.35.1578258788491; Sun, 05 Jan 2020 13:13:08 -0800 (PST) In-Reply-To: X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:4864:20::22d X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.org gmane.emacs.devel:244004 Archived-At: --00000000000073a72a059b6b0225 Content-Type: text/plain; charset="UTF-8" tree-sitter seems to designed to handle that case OOTB As per tree-sitter docs: 1. You start parsing via ts_parser_parse(in a separate thread). 2. Document is changed 3. Call ts_parser_set_cancellation_flag 4. Call ts_tree_edit with the edits from 2 5. You call ts_parser_parse with the same params and tree-sitter is smart enough to reuse the stuff that has already been parsed. On Sun, Jan 5, 2020 at 10:28 PM Stefan Monnier wrote: > > That's exactly the issue: if we need to make a copy just to run the > > parser asynchronously, then there's no advantage significant in having > > such asynchronous processing inside the Emacs process, > > There can still be advantages depending on many other details. > > Another option is to give them direct access to the buffer, but only > allow read-only access and impose some synchronization between the > threads, e.g.: prepare_before_change could signal the concurrent > threads and wait for them to acknowledge that they can't look at the > buffer positions after START and then re-allow access past START when we > finish the buffer modification or when we return to the command loop). > > Similarly, when buffer relocation takes place, we'd first signal to > concurrent threads and wait for them to acknowledge that they've stopped > accessing the buffer's content, and later re-signal them to let them > know they can access it again. > > > Stefan > > > --00000000000073a72a059b6b0225 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
tree-sitter seems to designed to handle that case OOT= B

As per tree-sitter docs:

1. You start par= sing via=C2=A0ts_parser_parse(in a separate thread).
2. Document is chan= ged
3. Call=C2=A0ts_parser_set_cancellation_flag=C2=A0
4. Call=C2=A0t= s_tree_edit with the edits from 2
5. You call ts_parser_parse with the s= ame params and tree-sitter is
smart enough to reuse the stuff tha= t has already been parsed.


On Sun, Jan 5, 2020 at 1= 0:28 PM Stefan Monnier <monn= ier@iro.umontreal.ca> wrote:
> That's exactly the issue: if we need to make a= copy just to run the
> parser asynchronously, then there's no advantage significant in ha= ving
> such asynchronous processing inside the Emacs process,

There can still be advantages depending on many other details.

Another option is to give them direct access to the buffer, but only
allow read-only access and impose some synchronization between the
threads, e.g.: prepare_before_change could signal the concurrent
threads and wait for them to acknowledge that they can't look at the buffer positions after START and then re-allow access past START when we finish the buffer modification or when we return to the command loop).

Similarly, when buffer relocation takes place, we'd first signal to
concurrent threads and wait for them to acknowledge that they've stoppe= d
accessing the buffer's content, and later re-signal them to let them know they can access it again.


=C2=A0 =C2=A0 =C2=A0 =C2=A0 Stefan


--00000000000073a72a059b6b0225--