From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Dmitry Gutov Newsgroups: gmane.emacs.bugs Subject: bug#66732: tree-sitter fontification doesn't update multi-line syntax reliably Date: Fri, 15 Dec 2023 03:01:02 +0200 Message-ID: <33fe5d61-5022-67c5-6a65-babde4fb7f91@gutov.dev> References: <878r7s5cdf.fsf@honnef.co> <83fs1tbou1.fsf@gnu.org> <835y1zo3rw.fsf@gnu.org> <2ce274aa-6d01-4d0a-b10c-07f821343fed@gmail.com> <50920549-006c-0153-2471-02e41a3dada7@gutov.dev> <8c7cd429-bdc3-4fac-ad1c-fbad793bf1a0@gmail.com> <231ebcd1-ec30-0432-82e7-d63e11cd65f7@gutov.dev> <765D713E-9923-4F66-9044-9D69C104C9B0@gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="37465"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.13.0 Cc: Eli Zaretskii , 66732@debbugs.gnu.org, Stefan Monnier , dominik@honnef.co To: Yuan Fu Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Fri Dec 15 02:02:46 2023 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1rDwbE-0009Um-84 for geb-bug-gnu-emacs@m.gmane-mx.org; Fri, 15 Dec 2023 02:02:45 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rDwas-0008Ix-4B; Thu, 14 Dec 2023 20:02:22 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rDwab-0008GC-1b for bug-gnu-emacs@gnu.org; Thu, 14 Dec 2023 20:02:05 -0500 Original-Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rDwaY-0002RU-J0 for bug-gnu-emacs@gnu.org; Thu, 14 Dec 2023 20:02:04 -0500 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1rDwaY-0001sB-2g for bug-gnu-emacs@gnu.org; Thu, 14 Dec 2023 20:02:02 -0500 X-Loop: help-debbugs@gnu.org Resent-From: Dmitry Gutov Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Fri, 15 Dec 2023 01:02:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 66732 X-GNU-PR-Package: emacs Original-Received: via spool by 66732-submit@debbugs.gnu.org id=B66732.17026020737118 (code B ref 66732); Fri, 15 Dec 2023 01:02:02 +0000 Original-Received: (at 66732) by debbugs.gnu.org; 15 Dec 2023 01:01:13 +0000 Original-Received: from localhost ([127.0.0.1]:51284 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rDwZk-0001qf-RI for submit@debbugs.gnu.org; Thu, 14 Dec 2023 20:01:13 -0500 Original-Received: from wout1-smtp.messagingengine.com ([64.147.123.24]:58581) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rDwZj-0001nY-Mj for 66732@debbugs.gnu.org; Thu, 14 Dec 2023 20:01:12 -0500 Original-Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.west.internal (Postfix) with ESMTP id E8CBB3200A26; Thu, 14 Dec 2023 20:01:05 -0500 (EST) Original-Received: from mailfrontend1 ([10.202.2.162]) by compute4.internal (MEProxy); Thu, 14 Dec 2023 20:01:06 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gutov.dev; h=cc :cc:content-transfer-encoding:content-type:content-type:date :date:from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to; s=fm3; t=1702602065; x=1702688465; bh=ispmfNQVNCntCAUliyx14FjB4lZzoPpPHe5W+tVl4Tg=; b= qrNR92QClJB6x79gsE/9C5Iy/2yDLNji9OvhL06+sAvcw6uMvvuxsGF/je3J9rxY i8e8CylLC18Pont3hdEqOEhuaLsteIVeIR/s5oE9z/NoP6pGo4ocpg0c5EUQzZNJ wNTKMx64K91+KUBy7sUlhK10Yi//0+ObclKhgQfk661sPxcm5QUORZI52NJn7jhD K90ZOpDZThJNQjm6De7MYaDaq64i9QaR+tcX4Q3NUMfmf1N5iJrNy07MpyzB3ozt 3F+y/ODl3OTcILfSqSfpqtQUpNJXOI3PdmzPMlc+Aaqp8/GkfCP0mdZeon9h62yG Is3wD+GKctnyTfNAP0WmGw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=1702602065; x= 1702688465; bh=ispmfNQVNCntCAUliyx14FjB4lZzoPpPHe5W+tVl4Tg=; b=J 2xg8N492ta3EN3WFaWFHOfYL0XUJcba6B2k+Grm4ROm3qfvloESO7fsawfqP6mHC 0vksUzpZd+Wkt9F2LLRFOCvXlt9a0ExNsIn5XoFUeyvvN5E/W7b/s/BEDt/WuhwQ YRY83jPSSm3QT53ngvJwhLjt408Knqj49F//eF1q+uM5+8ny31jsTEiBwxifg0k5 14u9S2KomPPOHmCbQpvlOwyIQdKc4xEHmx7jvwlGC5lEXk3txJU7WBecGWsbCSX2 f6F17dQl4/rNfMy/+l85Feq9qpfmuodpEARpcJayUvO8W0C/+SXQg6mtCa7MzeqP WnfMmxHUN4INW1KhJRh8w== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvkedrvddttddgvdelucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepkfffgggfuffvvehfhfgjtgfgsehtkeertddtfeejnecuhfhrohhmpeffmhhi thhrhicuifhuthhovhcuoegumhhithhrhiesghhuthhovhdruggvvheqnecuggftrfgrth htvghrnhephfffheeljeffgeffueeghfekkedtfffgheejvdegjeettdduheeufffggfef jeehnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepug hmihhtrhihsehguhhtohhvrdguvghv X-ME-Proxy: Feedback-ID: i0e71465a:Fastmail Original-Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 14 Dec 2023 20:01:03 -0500 (EST) Content-Language: en-US In-Reply-To: <765D713E-9923-4F66-9044-9D69C104C9B0@gmail.com> X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.bugs:276238 Archived-At: On 14/12/2023 10:29, Yuan Fu wrote: > > >> On Dec 13, 2023, at 5:43 PM, Dmitry Gutov wrote: >> >> On 13/12/2023 05:28, Yuan Fu wrote: >>>>> c-ts-mode--emacs-set-ranges is registered as a range rule, so many tree-sitter function calls it before doing anything to make sure range is up-to-date. treesit-font-lock-fontify-region calls treesit-update-ranges at the beginning of its body, and treesit-update-ranges calls c-ts-mode--emacs-set-ranges. >>>> >>>> That seems to mean that any feature accessing the parse tree should call treesit-update-ranges first. Including syntax-propertize-functions and *-extend-region-functions, which we currently don't do. >>>> >>>> But which boundaries is it supposed to use? Should treesit--syntax-extend-region call treesit-update-ranges, when wait for the parser updates, then possibly call treesit-update-ranges again on the extended boundaries, and so on, until the parser stops sending notifications? Will it stop? >>> It'll stop. If the ranges don't change, no reparse will happen. And only buffer content change can cause range change. Reparse itself can't. So at most you'll see reparse -> update ranges -> reparse. >>>> Anyway, could you try my patch? Like I said, I'm not sure if the insufficient fontification I'm observing in c-ts-mode is due to the problem with the solution, or due to the other redisplay-related problems on my system. >>> Yeah, it doesn't solve the problem in c-ts-mode regarding block comments. >>> We might need to run (progn (force-parse) (update-ranges) (force-parse)) before jit-lock-fontify-now and sytax-ppss. >> >> Well... I've replaced >> >> (treesit-buffer-root-node (treesit-language-at (point))) >> >> in treesit--syntax-extend-region with >> >> (treesit-buffer-root-node (treesit-language-at (point))) >> (treesit-update-ranges beg end) >> (treesit-buffer-root-node (treesit-language-at (point))) >> >> and even tried commenting out the call to 'treesit-update-ranges' inside treesit-font-lock-fontify-region, but the result looks unchanged. And the region does get extended, according to my print-debugging inside font-lock-default-fontify-region. >> >> Could you check if you're seeing the same? > > It should be > > (treesit-buffer-root-node 'emacs-c) > (treesit-update-ranges) > (treesit-buffer-root-node ‘c) This might be difficult for treesit--syntax-extend-region to do, since it's supposed to work across modes. But I suppose it could iterate through (mapcar #'treesit-parser-language (treesit-parser-list)). > I tried, and it doesn’t make a difference. But I must admit I’m not very clear on what your patch tries to do. Does it try to extend the to-be-fontified region before jit-lock runs? Yes, it extends the region to-be-fontified. If you set treesit--font-lock-verbose to t, you should see the appropriate messages "Fontifying region: x-y" in the message log where x is the position before the start of the comment (printed by treesit-font-lock-fontify-region).