From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED.blaine.gmane.org!not-for-mail From: Eli Zaretskii Newsgroups: gmane.emacs.devel Subject: Re: Using incremental parsing in Emacs Date: Sat, 04 Jan 2020 10:55:17 +0200 Message-ID: <83lfqnha8a.fsf@gnu.org> References: <83blrkj1o1.fsf@gnu.org> <41b3e9a0-2866-4692-a35c-6d9541bc3aaa@Spark> Injection-Info: blaine.gmane.org; posting-host="blaine.gmane.org:195.159.176.226"; logging-data="259878"; mail-complaints-to="usenet@blaine.gmane.org" Cc: emacs-devel@gnu.org To: HaiJun Zhang Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Sat Jan 04 09:55:53 2020 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([209.51.188.17]) by blaine.gmane.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.89) (envelope-from ) id 1infDs-0015V8-Os for ged-emacs-devel@m.gmane.org; Sat, 04 Jan 2020 09:55:52 +0100 Original-Received: from localhost ([::1]:60706 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1infDr-0003Ed-6K for ged-emacs-devel@m.gmane.org; Sat, 04 Jan 2020 03:55:51 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:46847) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1infDO-0002km-Hg for emacs-devel@gnu.org; Sat, 04 Jan 2020 03:55:23 -0500 Original-Received: from fencepost.gnu.org ([2001:470:142:3::e]:49601) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1infDO-000659-6A; Sat, 04 Jan 2020 03:55:22 -0500 Original-Received: from [176.228.60.248] (port=2170 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1infDJ-0004l8-IP; Sat, 04 Jan 2020 03:55:21 -0500 In-reply-to: (message from HaiJun Zhang on Sat, 4 Jan 2020 12:57:24 +0800) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.org gmane.emacs.devel:243911 Archived-At: > Date: Sat, 4 Jan 2020 12:57:24 +0800 > From: HaiJun Zhang > > For font-lock, I think it may work with tree-sitter like this: > 1. After openning a file, parse the whole buffer with tree-sitter. We get a syntax tree from tree-sitter. > 2. Get the syntax nodes with ts_node_descendant_for_point_range and fontify the buffer text in the visible > region or whole buffer. > 3. After each modification of buffer text, make a copy of the syntax tree as the new one. Update the new one > with the modification. > 4. Get the changed range and changed nodes list by comparing the old and new syntax trees. Then free the > old syntax tree. > 5. Update the text properties in the changed range. > 6. Goto 3 I encourage you to study how JIT font lock works in Emacs, and in particular how it plugs itself into the display engine. Because using any incremental parsing technology for font-lock needs a good understanding of how font-lock is typically used in Emacs, and any practical suggestions for integration and interfaces must take that into consideration. E.g., step 1 is anathema to JIT font-lock: it would produce a long delay in displaying a file's buffer when the file is first visited. For example, think about visiting a large and complex source file such as xdisp.c: even if it takes tens of milliseconds to parse all of it, as some tree-sitter presentation claims, waiting for that long before we even start displaying the first window-full would be an annoyance. And that's even before we consider the time to compute all the face text properties from the syntax tree, something that will also take time. Thanks.