From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Eli Zaretskii Newsgroups: gmane.emacs.bugs Subject: bug#65451: 30.0.50; `after-change-functions' are not triggered in the same order the changes are made Date: Tue, 22 Aug 2023 19:02:31 +0300 Message-ID: <83ttsrrroo.fsf@gnu.org> References: <871qfv2zlk.fsf@localhost> <83a5ujtgfo.fsf@gnu.org> <87jztn1c5x.fsf@localhost> <834jkrters.fsf@gnu.org> <87v8d7i48y.fsf@localhost> Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="35431"; mail-complaints-to="usenet@ciao.gmane.io" Cc: casouri@gmail.com, 65451@debbugs.gnu.org To: Ihor Radchenko Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Tue Aug 22 18:03:54 2023 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1qYTrF-0008yJ-PJ for geb-bug-gnu-emacs@m.gmane-mx.org; Tue, 22 Aug 2023 18:03:54 +0200 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qYTqO-0002sW-Qt; Tue, 22 Aug 2023 12:03:00 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qYTqN-0002pS-R0 for bug-gnu-emacs@gnu.org; Tue, 22 Aug 2023 12:02:59 -0400 Original-Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qYTqN-0005n7-H8 for bug-gnu-emacs@gnu.org; Tue, 22 Aug 2023 12:02:59 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1qYTqQ-0003eT-AG for bug-gnu-emacs@gnu.org; Tue, 22 Aug 2023 12:03:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Eli Zaretskii Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Tue, 22 Aug 2023 16:03:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 65451 X-GNU-PR-Package: emacs Original-Received: via spool by 65451-submit@debbugs.gnu.org id=B65451.169272014213987 (code B ref 65451); Tue, 22 Aug 2023 16:03:02 +0000 Original-Received: (at 65451) by debbugs.gnu.org; 22 Aug 2023 16:02:22 +0000 Original-Received: from localhost ([127.0.0.1]:60238 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qYTpm-0003dX-Cc for submit@debbugs.gnu.org; Tue, 22 Aug 2023 12:02:22 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:48366) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qYTpj-0003dK-Pj for 65451@debbugs.gnu.org; Tue, 22 Aug 2023 12:02:21 -0400 Original-Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qYTpb-0005he-KP; Tue, 22 Aug 2023 12:02:11 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=2K/2TiaMWvdYUyvbioH9vx7EnJTgkxDBOaZR8PWhHLw=; b=Bd4yINZ5t9tf /DUO7L2Zoz9s/t0JwDMtWHQ3qmZE7K21JaxtJfc2NvJRRQnP/HsraRXadokrQJm2GVH7uzaJZ4A9K Z8gIc3tXueJQLTStVDJTsZVcbH1qKZ50YwFjFwyaUHDDtD3dnwuvBnAR2kCo6ZFlkN/eKWYn5vZWR 3O4xf/cauHMobgMfzjuJDJxHN2qaVFuZto17irKZJ5Ge8yCOqkqL0CKUomldZeViqm+T7VAXUX/m4 C1vasxZyAwal10XhWaGyn1J4YLHsZOM/mEEzbjzSi4cGV/NJm9gIbVuJKUHvC4u6llxacy4CEnxQT 7XVPk7LFgaH22of28M+jyg==; In-Reply-To: <87v8d7i48y.fsf@localhost> (message from Ihor Radchenko on Tue, 22 Aug 2023 13:41:17 +0000) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.bugs:268183 Archived-At: > From: Ihor Radchenko > Cc: Yuan Fu , 65451@debbugs.gnu.org > Date: Tue, 22 Aug 2023 13:41:17 +0000 > > Eli Zaretskii writes: > > >> Then, I'd like to point back to the previous discussion where I asked to > >> expose to Elisp information about buffer changes available to > >> tree-sitter. > >> https://yhetil.org/emacs-devel/83tu8jq2vl.fsf@gnu.org/ > > > > I don't want to do that, sorry. Not without a good understanding of > > what exactly do you need from that and in what way. If we will expose > > anything, it will have to be the minimum possible exposure, not the > > maximum, so I would like to understand this very well before I agree > > to any change in this direction. > > Org wants to do the same thing tree-sitter does - keep parsed AST in > sync with buffer modifications without having to re-parse the whole > buffer. So, we basically need the same information tree-sitter needs - > the sequence of buffer text changes, in their order. We don't expose the data you want to tree-sitter in Lisp. What is exposed to Lisp are the parser and parse-tree objects that we build (in C) based on tree-sitter parsing results. When the buffer is modified, the information about the modifications is used internally by Emacs, in C code, to find and update the relevant parsers, and for that we call the tree-sitter functions involved in this process. See the function treesit_record_change which does that, and which is called from C when buffer text changes in a way relevant to treesit.el functionalities. (Note that some changes of buffer text are not visible even to tree-sitter, because we decided they are not relevant, for now.) > Note that the markers discussed in the thread I linked are not > sufficient. When editing near AST node boundaries, even if the > boundaries are represented by markers, we have to re-parse the AST > around to account for the possible structural changes. So, information > about buffer edits is still required. If tracking markers is not enough, then I wonder how the information from the lower levels, which is basically the same but noisier, will be able to help you. > >> In fact, I am not sure if tree-sitter will behave correctly if it is > >> signaled changes in incorrect order. > > > > I will defer to Yuan, but tree-sitter doesn't use these hooks, we call > > its functions directly from insdel.c where needed. This makes sense > > for a library to which we link and whose interface code we control, > > but giving such access to Lisp (and Org on top of that) is out of the > > question. We don't even give such access to modules. > > I hope that we can solve this issue one way or another. This currently > breaks the very core functionality of Org. Every part of Org relies on > it to obtain reasonable performance. Prior to using cache, we had orders > of magnitude slowdowns. If you can arrange your design such that Lisp sees only AST-specific objects affected by the modifications in buffer text, then I believe we will have a good chance of finding a satisfactory solution. If that requires to have some of your code in C (preferably, generalized to some extent), then so be it. You see, I think the buffer-change hooks we have are already too much: Lisp programs abuse them all the time (you can see a good example in the bug which I mentioned up-thread, and which led to the change you are now complaining about). Doing more of that is not very wise, to say the least. Moreover, I think the solution you think you want you actually _don't_ want, because it will overwhelm you with changes that are not relevant to your purposes. You can see a clear evidence to that in the fact that treesit_record_change is called only in several strategical places, not everywhere where we change buffer text, and not at the lowest level of such changes. There's a reason to that.