From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Dmitry Gutov Newsgroups: gmane.emacs.bugs Subject: bug#66732: tree-sitter fontification doesn't update multi-line syntax reliably Date: Sun, 17 Dec 2023 00:56:16 +0200 Message-ID: <8fa0e506-6efc-57d4-6034-e938f97b1fb0@gutov.dev> References: <878r7s5cdf.fsf@honnef.co> <83fs1tbou1.fsf@gnu.org> <835y1zo3rw.fsf@gnu.org> <2ce274aa-6d01-4d0a-b10c-07f821343fed@gmail.com> <50920549-006c-0153-2471-02e41a3dada7@gutov.dev> <8c7cd429-bdc3-4fac-ad1c-fbad793bf1a0@gmail.com> <231ebcd1-ec30-0432-82e7-d63e11cd65f7@gutov.dev> <765D713E-9923-4F66-9044-9D69C104C9B0@gmail.com> <33fe5d61-5022-67c5-6a65-babde4fb7f91@gutov.dev> <92CACD38-9534-4A07-8DE3-CE8408272FB6@gmail.com> <59CC46F7-867E-4C74-83EC-49B41DF0FAB8@gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="18653"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.13.0 Cc: Eli Zaretskii , 66732@debbugs.gnu.org, Stefan Monnier , dominik@honnef.co To: Yuan Fu Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Sat Dec 16 23:57:30 2023 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1rEdb6-0004de-Vf for geb-bug-gnu-emacs@m.gmane-mx.org; Sat, 16 Dec 2023 23:57:30 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rEdah-0002GW-Sp; Sat, 16 Dec 2023 17:57:03 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rEdag-0002GM-B8 for bug-gnu-emacs@gnu.org; Sat, 16 Dec 2023 17:57:02 -0500 Original-Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rEdag-0000O5-32 for bug-gnu-emacs@gnu.org; Sat, 16 Dec 2023 17:57:02 -0500 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1rEdag-0002HN-Gv for bug-gnu-emacs@gnu.org; Sat, 16 Dec 2023 17:57:02 -0500 X-Loop: help-debbugs@gnu.org Resent-From: Dmitry Gutov Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sat, 16 Dec 2023 22:57:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 66732 X-GNU-PR-Package: emacs Original-Received: via spool by 66732-submit@debbugs.gnu.org id=B66732.17027673908722 (code B ref 66732); Sat, 16 Dec 2023 22:57:02 +0000 Original-Received: (at 66732) by debbugs.gnu.org; 16 Dec 2023 22:56:30 +0000 Original-Received: from localhost ([127.0.0.1]:56293 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rEdaA-0002Gc-5D for submit@debbugs.gnu.org; Sat, 16 Dec 2023 17:56:30 -0500 Original-Received: from out1-smtp.messagingengine.com ([66.111.4.25]:47689) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rEda6-0002GN-9B for 66732@debbugs.gnu.org; Sat, 16 Dec 2023 17:56:29 -0500 Original-Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.nyi.internal (Postfix) with ESMTP id 2F9A05C00D2; Sat, 16 Dec 2023 17:56:20 -0500 (EST) Original-Received: from mailfrontend1 ([10.202.2.162]) by compute4.internal (MEProxy); Sat, 16 Dec 2023 17:56:20 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gutov.dev; h=cc :cc:content-transfer-encoding:content-type:content-type:date :date:from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to; s=fm3; t=1702767380; x=1702853780; bh=NpBF5jBpOzsp2RAF06dmbDrSVOgDedh3zjHCIch82HA=; b= Xb0MHOgINhrKpq8j6m8J1bCBGx8+nomjQHwUiEo90IZ/0UdY2BLnT+VcPpZmWbR0 XoaQaJjEFK1OdhHjI5i1FARzcXYX9nuh1lnz/58lOIrJH/8C3qVzMQ3/4zWvfk9I AC1xtvwOxZCUMtm2QI2AaWA4Nr6SyWRVuZphYmCPvYiHmv0kY1cbqE9rwh8pkb9+ eN2kf77BE0lTTijcfeuII14AJ+P4YArgyD2i6RoOFYbaWI/RRYkuZUS0DYerMfpV je3U1ONgowGbjSQVZ9PFYBLlhA/w/9SQK1vjq3GNa5/pwoQraIGsqJIKuLWfcDbh 29vID4CmFtiwduQ02aolNg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm2; t=1702767380; x= 1702853780; bh=NpBF5jBpOzsp2RAF06dmbDrSVOgDedh3zjHCIch82HA=; b=i WFdZMzVYjsy7rj/8b1VWjcbAAuthog8keYlurbrMxRkbCrCJETtJoyh0vYzK9Bmh parnBPqFfqkiXxvpIoFw7twWDMVNgwXGnjIhsuamqdfEg9Rq67mr+GeMGyd38k5/ vGShluvMAyRzmmCXgpjE2Uscpdy/GA7goGRR6CJSqFR0IfLWdcVXpz9oT0+8p9/q w0BxQnHq5TRgtmeMcAtZM/i7cA/UZYkr9rEvgSmJpte5hKe1EQHgJ2s3a4NP0aFN qBbiKVSI8SUoUUgy9ihvWyu2Svri3jNvxHo1UP/qqMCJd/vYcE39yrg+VsfQVYt0 Wrjdz1iqorvZeO64KN03A== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvkedrvddthedgtdehucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepkfffgggfuffvvehfhfgjtgfgsehtkeertddtfeejnecuhfhrohhmpeffmhhi thhrhicuifhuthhovhcuoegumhhithhrhiesghhuthhovhdruggvvheqnecuggftrfgrth htvghrnhephfffheeljeffgeffueeghfekkedtfffgheejvdegjeettdduheeufffggfef jeehnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepug hmihhtrhihsehguhhtohhvrdguvghv X-ME-Proxy: Feedback-ID: i0e71465a:Fastmail Original-Received: by mail.messagingengine.com (Postfix) with ESMTPA; Sat, 16 Dec 2023 17:56:18 -0500 (EST) Content-Language: en-US In-Reply-To: <59CC46F7-867E-4C74-83EC-49B41DF0FAB8@gmail.com> X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.bugs:276387 Archived-At: On 16/12/2023 07:56, Yuan Fu wrote: >> Hmm, let me have a closer look tomorrow. It can’t be that difficult, we must have missed something. > > Please see this patch, which basically does what you did in your patch. I just started from scratch when experimenting. I didn’t do the context thing, that’s a bit confusing for me. Yeah, it's more complex than I'd have liked but basically the "context" thing was to dispatch to the right invalidator: for font-lock and/or for syntax. Whether to return the range using the corresponding extend-function, or to invalidate push-style (when the notifier is called outside of any extend-function's). I've tried your patch, and it seemed to have similar problems to mine. In particular, that the comment is not unfontified right away when the closing slash is removed. Also, I think we're targeting Emacs 29 for this bug (the patch needed adjustments). > I save updated ranges in a local hash table and access it in extend-region functions to extend the region. Ideally treesit should have a function that returns the updated ranges for a parser, we don’t have that yet, so I’m using the hash table to simulate it. Okay. I wonder how other users (which care about specific parsers) would look. > I found that if you don’t set text prop fontified to nil for the whole extended region, redisplay doesn’t seem to, well, redisplay the full region, even thought the new face has been correctly applied to them. That sounds like a bug in font-lock? At the end of jit-lock-fontify-now, there is a call creating a timer with jit-lock-force-redisplay. And that function ends with this: ;; Don't cause refontification (it's already been done), but just do ;; some random buffer change, so as to force redisplay. (put-text-property start end 'fontified t))))) If I just change t to nil there (or to some other value, like 42), either of our patches starts behaving well. Perhaps Stefan could comment.