From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Stefan Monnier via "Bug reports for GNU Emacs, the Swiss army knife of text editors" Newsgroups: gmane.emacs.bugs Subject: bug#66732: tree-sitter fontification doesn't update multi-line syntax reliably Date: Mon, 18 Dec 2023 14:12:02 -0500 Message-ID: References: <878r7s5cdf.fsf@honnef.co> <83fs1tbou1.fsf@gnu.org> <835y1zo3rw.fsf@gnu.org> <2ce274aa-6d01-4d0a-b10c-07f821343fed@gmail.com> <50920549-006c-0153-2471-02e41a3dada7@gutov.dev> <8c7cd429-bdc3-4fac-ad1c-fbad793bf1a0@gmail.com> <231ebcd1-ec30-0432-82e7-d63e11cd65f7@gutov.dev> <765D713E-9923-4F66-9044-9D69C104C9B0@gmail.com> <33fe5d61-5022-67c5-6a65-babde4fb7f91@gutov.dev> <92CACD38-9534-4A07-8DE3-CE8408272FB6@gmail.com> <59CC46F7-867E-4C74-83EC-49B41DF0FAB8@gmail.com> <8fa0e506-6efc-57d4-6034-e938f97b1fb0@gutov.dev> Reply-To: Stefan Monnier Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="40201"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Cc: Eli Zaretskii , 66732@debbugs.gnu.org, Yuan Fu , dominik@honnef.co To: Dmitry Gutov Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Mon Dec 18 20:13:16 2023 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1rFJ3B-000A7p-U5 for geb-bug-gnu-emacs@m.gmane-mx.org; Mon, 18 Dec 2023 20:13:14 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rFJ32-0004DL-BH; Mon, 18 Dec 2023 14:13:04 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rFJ2y-00049o-Ax for bug-gnu-emacs@gnu.org; Mon, 18 Dec 2023 14:13:00 -0500 Original-Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rFJ2x-0003fI-WE for bug-gnu-emacs@gnu.org; Mon, 18 Dec 2023 14:13:00 -0500 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1rFJ30-0006JB-0N for bug-gnu-emacs@gnu.org; Mon, 18 Dec 2023 14:13:02 -0500 X-Loop: help-debbugs@gnu.org Resent-From: Stefan Monnier Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Mon, 18 Dec 2023 19:13:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 66732 X-GNU-PR-Package: emacs Original-Received: via spool by 66732-submit@debbugs.gnu.org id=B66732.170292673424090 (code B ref 66732); Mon, 18 Dec 2023 19:13:01 +0000 Original-Received: (at 66732) by debbugs.gnu.org; 18 Dec 2023 19:12:14 +0000 Original-Received: from localhost ([127.0.0.1]:33635 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rFJ2E-0006GU-E2 for submit@debbugs.gnu.org; Mon, 18 Dec 2023 14:12:14 -0500 Original-Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:29196) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rFJ2C-0006G2-CH for 66732@debbugs.gnu.org; Mon, 18 Dec 2023 14:12:12 -0500 Original-Received: from pmg3.iro.umontreal.ca (localhost [127.0.0.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id E3793444E53; Mon, 18 Dec 2023 14:12:04 -0500 (EST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1702926723; bh=H7Pb04TG5I95p92hZR+dzeS3NqyyiGL2jKXDhbqBqiY=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=j7S8saiAjwDpcWynK4eGPPugBTvcOKGZME95J+9vMvFPqFnztHK8D3j++s2dGhPSb iCWorP7nTF0JLfC1IRjM64M3EapQ3Yhk9wDD6mE6tjzZ/+73oOyiWiQv9WnLXDHznb 3q7WfKmiTcB2upfRgzJQQmfZDkAA3/FUx0MoGMwRMdc98pFLeM9E6K6KFQiFyVjeWh XNLwHJuPuixcdb1EKh5gEy8B2nOiLcnucJlDn2FzrTiP0jF5FdRVOIwY2ESYD8ZPum CEutrtKaRg764c2fDrSgDLcCzevgvfaTuB8w66KcPpNm8ism6K32QeFSuX0FBSEOl1 JIjUqNeHOIigQ== Original-Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id 7D9ED444E50; Mon, 18 Dec 2023 14:12:03 -0500 (EST) Original-Received: from pastel (65-110-221-238.cpe.pppoe.ca [65.110.221.238]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 49DD5120EC3; Mon, 18 Dec 2023 14:12:03 -0500 (EST) In-Reply-To: (Dmitry Gutov's message of "Mon, 18 Dec 2023 20:27:59 +0200") X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.bugs:276480 Archived-At: >>> I found that if you don=E2=80=99t set text prop fontified to nil for th= e whole >>> extended region, redisplay doesn=E2=80=99t seem to, well, redisplay the= full >>> region, even thought the new face has been correctly applied to them. >> That sounds like a bug in font-lock? At the end of jit-lock-fontify-now, >> there is a call creating a timer with jit-lock-force-redisplay. >> And that function ends with this: >> =C2=A0=C2=A0=C2=A0 ;; Don't cause refontification (it's already been do= ne), but just do >> =C2=A0=C2=A0=C2=A0 ;; some random buffer change, so as to force redispl= ay. >> =C2=A0=C2=A0=C2=A0 (put-text-property start end 'fontified t))))) >> If I just change t to nil there (or to some other value, like 42), either >> of our patches starts behaving well. Perhaps Stefan could comment. This chunk of code is present so as to cause a re-render (i.e. recompute the glyph matrices) of the affected region, but not a re-fontification. So a nil value would be wrong. I'm surprised that 42 behaves differently from t. What we really need here is to force the redisplay to consider that this part of the buffer needs to be re-rendered. In practice changing any text-property on that chunk of text should do the trick, in my experience, but from what you describe it seems that some optimisation is sufficiently clever to notice that the old value was t and the new value is identical so the region is not marked as modified? Indeed, in `add_text_properties_1` I see we skip over intervals which already have the right property values, so that might be what's happening. I suggest we introduce a separate function with a name indicating what we intend it to do (like `force-region-update`) so the code will be clearer. And its implementation could consist in adding and then removing some dummy text property (tho a better implementation would go and modify the underlying C-level variables in the buffer like BUF_*_UNCHANGED). Stefan