From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Eli Zaretskii Newsgroups: gmane.emacs.devel Subject: Re: How to add pseudo vector types Date: Fri, 16 Jul 2021 10:30:51 +0300 Message-ID: <83czri67h0.fsf@gnu.org> References: <83h7gw6pyj.fsf@gnu.org> <45EBF16A-C953-42C7-97D1-3A2BFEF7DD01@gmail.com> <83y2a764oy.fsf@gnu.org> <83v95b60fn.fsf@gnu.org> <00DD5BFE-D14E-449A-9319-E7B725DEBFB3@gmail.com> <83r1fz5xr9.fsf@gnu.org> <1AAB1BCC-362B-4249-B785-4E0530E15C60@gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="22849"; mail-complaints-to="usenet@ciao.gmane.io" Cc: monnier@iro.umontreal.ca, emacs-devel@gnu.org To: Yuan Fu Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Fri Jul 16 09:32:31 2021 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1m4IKl-0005lM-2Q for ged-emacs-devel@m.gmane-mx.org; Fri, 16 Jul 2021 09:32:31 +0200 Original-Received: from localhost ([::1]:55294 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1m4IKk-00043l-3R for ged-emacs-devel@m.gmane-mx.org; Fri, 16 Jul 2021 03:32:30 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:51420) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m4IJP-0002lV-UF for emacs-devel@gnu.org; Fri, 16 Jul 2021 03:31:07 -0400 Original-Received: from fencepost.gnu.org ([2001:470:142:3::e]:51090) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1m4IJO-00007X-KO; Fri, 16 Jul 2021 03:31:06 -0400 Original-Received: from 84.94.185.95.cable.012.net.il ([84.94.185.95]:3996 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m4IJO-0007d6-8b; Fri, 16 Jul 2021 03:31:06 -0400 In-Reply-To: <1AAB1BCC-362B-4249-B785-4E0530E15C60@gmail.com> (message from Yuan Fu on Thu, 15 Jul 2021 14:23:02 -0400) X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.io gmane.emacs.devel:271289 Archived-At: > From: Yuan Fu > Date: Thu, 15 Jul 2021 14:23:02 -0400 > Cc: Stefan Monnier , > emacs-devel@gnu.org > > >> Say you have a buffer with some content and scrolled through it, so tree-sitter has parsed the whole buffer. Then some elisp edited some text outside the visible portion. Redisplay doesn’t happen, we don’t tell this edit to tree-sitter. Then I scroll to the place that has been edited. What now? > > > > Now you call tree-sitter passing it the part of the buffer that needs > > to be parsed (e.g., the chunk that is about to be displayed). If > > tree-sitter needs to look back, it will. > > > >> I’ve lost the change information, and tree-sitter’s tree is out-dated. > > > > No information is lost because the updated buffer text is available. > > > >> We can fontify on-demand, but we can’t parse on-demand. > > > > Sorry, I don't believe this is true. tree-sitter _must_ be able to > > deal with these situations, because it must be able to deal with > > incomplete text that cannot be parsed without parse errors. > > > I think my assertion was too strong. By “can’t parse on-demand” I mean we can’t easily pass tree-sitter a random chunk of text and not letting it to parse from BOB. You must start from BOB only in languages that require that; not every language does. And even with languages that require starting from BOB, you could do that only once, the first time a buffer needs parsing; thereafter, you can only pass to tree-sitter the parts that were changed since the last time. Emacs records that information for the display engine, see BEG_UNCHANGED and END_UNCHANGED. If that is not enough, we could record more information about changes to buffer text. The main issue here is to pass the buffer text to tree-sitter lazily, only when and as much as needed. > I understand. I want to point out that parsing is separated from fontification, and syntax-pass flushes its cache in before-change-hook. I was hoping to use the parse tree for more than fontification, e.g., motion commands like sexp-forward/backward or structural editing commands like expand-region. Another scenario: some elisp edited some text before the visible portion, the tree is not updated, now I want to select the node at point (like expand-region), I look for the leave node that contains the byte position of point. However, because the tree is out-dated, the byte position of point will not correspond to the node I want. Each command/feature that needs an updated TS tree will take care of updating TS with the relevant information. We should record whatever we need for that as side effect of primitives that change buffer text (in insdel.c), and use the recorded info to update TS. But the actual passing of text to TS should happen lazily, when we actually need its re-parsing, not when the changes to buffer text are done.