From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Dmitry Gutov Newsgroups: gmane.emacs.bugs Subject: bug#66732: tree-sitter fontification doesn't update multi-line syntax reliably Date: Sat, 16 Dec 2023 19:23:49 +0200 Message-ID: References: <878r7s5cdf.fsf@honnef.co> <83fs1tbou1.fsf@gnu.org> <835y1zo3rw.fsf@gnu.org> <2ce274aa-6d01-4d0a-b10c-07f821343fed@gmail.com> <50920549-006c-0153-2471-02e41a3dada7@gutov.dev> <8c7cd429-bdc3-4fac-ad1c-fbad793bf1a0@gmail.com> <231ebcd1-ec30-0432-82e7-d63e11cd65f7@gutov.dev> <765D713E-9923-4F66-9044-9D69C104C9B0@gmail.com> <33fe5d61-5022-67c5-6a65-babde4fb7f91@gutov.dev> <92CACD38-9534-4A07-8DE3-CE8408272FB6@gmail.com> <59CC46F7-867E-4C74-83EC-49B41DF0FAB8@gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="9047"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.13.0 Cc: Eli Zaretskii , 66732@debbugs.gnu.org, dominik@honnef.co To: Stefan Monnier , Yuan Fu Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Sat Dec 16 18:25:21 2023 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1rEYPg-00027m-5C for geb-bug-gnu-emacs@m.gmane-mx.org; Sat, 16 Dec 2023 18:25:21 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rEYPQ-00025L-BT; Sat, 16 Dec 2023 12:25:04 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rEYPO-000252-2r for bug-gnu-emacs@gnu.org; Sat, 16 Dec 2023 12:25:02 -0500 Original-Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rEYPN-0005ZE-QP for bug-gnu-emacs@gnu.org; Sat, 16 Dec 2023 12:25:01 -0500 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1rEYPO-00036a-FA for bug-gnu-emacs@gnu.org; Sat, 16 Dec 2023 12:25:02 -0500 X-Loop: help-debbugs@gnu.org Resent-From: Dmitry Gutov Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sat, 16 Dec 2023 17:25:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 66732 X-GNU-PR-Package: emacs Original-Received: via spool by 66732-submit@debbugs.gnu.org id=B66732.170274744511809 (code B ref 66732); Sat, 16 Dec 2023 17:25:02 +0000 Original-Received: (at 66732) by debbugs.gnu.org; 16 Dec 2023 17:24:05 +0000 Original-Received: from localhost ([127.0.0.1]:55898 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rEYOS-00034N-RK for submit@debbugs.gnu.org; Sat, 16 Dec 2023 12:24:05 -0500 Original-Received: from out3-smtp.messagingengine.com ([66.111.4.27]:60739) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rEYOO-00033f-Pz for 66732@debbugs.gnu.org; Sat, 16 Dec 2023 12:24:03 -0500 Original-Received: from compute2.internal (compute2.nyi.internal [10.202.2.46]) by mailout.nyi.internal (Postfix) with ESMTP id ECD215C00AD; Sat, 16 Dec 2023 12:23:54 -0500 (EST) Original-Received: from mailfrontend1 ([10.202.2.162]) by compute2.internal (MEProxy); Sat, 16 Dec 2023 12:23:54 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gutov.dev; h=cc :cc:content-transfer-encoding:content-type:content-type:date :date:from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to; s=fm3; t=1702747434; x=1702833834; bh=p/PrxtoCRxO6n2QDc//dx6p/IBU5kZbGxHe2uckS9Jk=; b= hB5s6aa5S9UY718w+QpwC4SWdAy8C+KjMtre0CJAFWs6DgZhZV4rvdywQmA6CTVx D/pAN3LZO/QLnrP7y+8MSGEQufi3sEOQxYKbPW5/GZokn4ik3qrL+Gw6etsmRS5Q s/c+SaQqjHJMltLJCrM6Rp/GfNscKyDaHbJByNNLnmezQyn+UCq+KYxeriS/Ac88 X5tRD6DZ/5FS9KC+AwkT0CEP4Pbzyqn748yqVYLldOzmOWc8/NF4ePVXRa6TD49s cway/+Vv7qlDh8d1tNPQDb81XKrtHZb/xrWykbk46K4fYCNwh4O35SPWuk7yTFvT 8oX+oq+slj7toYV0jBfW8g== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm2; t=1702747434; x= 1702833834; bh=p/PrxtoCRxO6n2QDc//dx6p/IBU5kZbGxHe2uckS9Jk=; b=9 X32ogmgqnFIhIgUzF8Q+xf4ToUhU+4mEkJ2R6yPn2QVlXYnZjJAhpLYRLDQY4JMK FC2wPdob1THqywSa+cXXVdJYV0F7ZVZxHdMQc+kpzrEQ0RAgjwcaabNtYDP4cMR1 z8OGpALR3i7tI8plWhjLo2EwvDBY33U/bXoZDbOLy7+ViGquWiQ7QCPjyNt1PYra sCRLepL43q/7kIgGl69vEru0PkEDl0ojImndpL+eXPAtmP0oBF599eXtpOaTFv+i t/rd6VKYIRm8hGFgYcaEz1LSBMpw0aF/1dS97vlF6ybRAw8kXKCQnyZN+0GoEwzw dnBwL4ubbckFB1HT9yxeg== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvkedrvddtgedguddtudcutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd enucfjughrpefkffggfgfuvfevfhfhjggtgfesthejredttdefjeenucfhrhhomhepffhm ihhtrhihucfiuhhtohhvuceoughmihhtrhihsehguhhtohhvrdguvghvqeenucggtffrrg htthgvrhhnpeeigfetveehveevffehledtueekieeikeeufeegudfgfeeghfdulefgfeev ledvveenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpe gumhhithhrhiesghhuthhovhdruggvvh X-ME-Proxy: Feedback-ID: i0e71465a:Fastmail Original-Received: by mail.messagingengine.com (Postfix) with ESMTPA; Sat, 16 Dec 2023 12:23:53 -0500 (EST) Content-Language: en-US In-Reply-To: X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.bugs:276367 Archived-At: On 16/12/2023 19:11, Stefan Monnier wrote: >> Hmm... This `put-text-property` can't be right. >> Can you trace `font-lock-fontify-region` and >> `font-lock-default-fontify-region` to check that they correctly return >> the proper (extended) bounds of the region they (re)fontified? > > BTW, using the extend-region-function isn't ideal: this is meant for > things which can't be refontified separately, so if a change in the > buffer causes notification to make changes over a large portion of the > buffer we'll end up (re)fontified *right away* that whole large > portion even if only a small part (or even no part at all) is displayed. I think most of the time the parser notifier will send ranges of limited size (as one or other piece of text gets recognized as e.g. a comment, or a raw string -- the bounds of the respective node that changed). > IOW, it'd be better to mark the changed chunks with this > `put-text-property`, but to do it before jit&font-lock get triggered. > > Basically, we'd like to call it from `after-change-functions`, but this > can be called *many* times within a single command, so we want to delay > it. So we can probably just set a flag that says "there are unprocessed > changes" and then sprinkle calls to a "lazy update" function which > checks this flag before calling `treesit--font-lock-notifier` (or > something similar). I thought doing this lazily (inside font-lock-default-fontify-region), only when the region is set to be redisplayed, would be more economical than doing more work inside after-change-functions. > One of the places where we'd need the sprinkle would be > `pre-redisplay-functions`. Another one might be `syntax-ppss`? syntax-ppss has syntax-propertize-extend-region-functions. I haven't found any better suitable hooks than that.