From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Dmitry Gutov Newsgroups: gmane.emacs.bugs Subject: bug#66732: tree-sitter fontification doesn't update multi-line syntax reliably Date: Tue, 19 Dec 2023 01:08:19 +0200 Message-ID: <94a85548-5b9f-0698-98aa-f6495babbd1a@gutov.dev> References: <878r7s5cdf.fsf@honnef.co> <83fs1tbou1.fsf@gnu.org> <835y1zo3rw.fsf@gnu.org> <2ce274aa-6d01-4d0a-b10c-07f821343fed@gmail.com> <50920549-006c-0153-2471-02e41a3dada7@gutov.dev> <8c7cd429-bdc3-4fac-ad1c-fbad793bf1a0@gmail.com> <231ebcd1-ec30-0432-82e7-d63e11cd65f7@gutov.dev> <765D713E-9923-4F66-9044-9D69C104C9B0@gmail.com> <33fe5d61-5022-67c5-6a65-babde4fb7f91@gutov.dev> <92CACD38-9534-4A07-8DE3-CE8408272FB6@gmail.com> <59CC46F7-867E-4C74-83EC-49B41DF0FAB8@gmail.com> <8fa0e506-6efc-57d4-6034-e938f97b1fb0@gutov.dev> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="2364"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.13.0 Cc: Eli Zaretskii , 66732@debbugs.gnu.org, Yuan Fu , dominik@honnef.co To: Stefan Monnier Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Tue Dec 19 00:09:19 2023 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1rFMjb-0000Kr-PR for geb-bug-gnu-emacs@m.gmane-mx.org; Tue, 19 Dec 2023 00:09:17 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rFMjO-0001nC-PG; Mon, 18 Dec 2023 18:09:02 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rFMjN-0001n3-Rc for bug-gnu-emacs@gnu.org; Mon, 18 Dec 2023 18:09:01 -0500 Original-Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rFMjM-0007OQ-Is for bug-gnu-emacs@gnu.org; Mon, 18 Dec 2023 18:09:00 -0500 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1rFMjO-0007Py-BG for bug-gnu-emacs@gnu.org; Mon, 18 Dec 2023 18:09:02 -0500 X-Loop: help-debbugs@gnu.org Resent-From: Dmitry Gutov Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Mon, 18 Dec 2023 23:09:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 66732 X-GNU-PR-Package: emacs Original-Received: via spool by 66732-submit@debbugs.gnu.org id=B66732.170294091628455 (code B ref 66732); Mon, 18 Dec 2023 23:09:02 +0000 Original-Received: (at 66732) by debbugs.gnu.org; 18 Dec 2023 23:08:36 +0000 Original-Received: from localhost ([127.0.0.1]:33781 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rFMix-0007Op-O6 for submit@debbugs.gnu.org; Mon, 18 Dec 2023 18:08:36 -0500 Original-Received: from out4-smtp.messagingengine.com ([66.111.4.28]:52875) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rFMiv-0007OT-ND for 66732@debbugs.gnu.org; Mon, 18 Dec 2023 18:08:34 -0500 Original-Received: from compute7.internal (compute7.nyi.internal [10.202.2.48]) by mailout.nyi.internal (Postfix) with ESMTP id 5DE4B5C029B; Mon, 18 Dec 2023 18:08:25 -0500 (EST) Original-Received: from mailfrontend1 ([10.202.2.162]) by compute7.internal (MEProxy); Mon, 18 Dec 2023 18:08:25 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gutov.dev; h=cc :cc:content-transfer-encoding:content-type:content-type:date :date:from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to; s=fm3; t=1702940905; x=1703027305; bh=3q8Tg3b8+F9WwxkX5DtlG1du5sqmJ4nrPbpsLw7wY/8=; b= QBDqNJFnhdENh8UDS+zY70E738J+p+cW0Yd9r7X/+ZSOfsRFNI5Qs59L4YBBeXGp Y1k9WzsnExXgXttH24JVTuMLmI5tGzjlado+a1lXbT0/BjCPzwcSGya+bySE547V CCNKUB7jQwzMxv2Q/qCHfTaEwhxnpgrozzUjVfQKZlpNHk2vn2DraQb9+UkPdFi5 /Rfa85FGQQ9X7ucWTokuRgffYe8MadFKLtnhYeIl/AKCGCoXqjd47KK+0zKi5py3 IoNUwL3pQgaJ8pGXN4/MfOTI3X8YySgelJwK3ES5CG7YdM2VrEHkzCWiO6GipBhu UYCIPwBkqrMbFk/5//A8hQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm2; t=1702940905; x= 1703027305; bh=3q8Tg3b8+F9WwxkX5DtlG1du5sqmJ4nrPbpsLw7wY/8=; b=u 3ZmjnfEZPk61saq0v7qF3nELv/UhvyeN2kYKn7KUsibYQVvpwxYbRdF81QF8nBlw MYqwALY4yNuxfW9QksbZk6gzlHAJ3o2QgcIvvA2s/RC5qYyWunwOoxtGNHkPrwMM ql83KmdgiBdx6L96boY4Xr4R/nstnGiMCo6bSEafmmzLzT4WYauPYAi6e7LuG/cz GVubGbTv3FNTwsZtsXmEdsLuCbWaWfKy/DTDV6oWaqjEw/MKda0Q874TZkOGci6B 0fU2caJL8ZforK9kUTPSLj0P1DHhm4mPzpV99j6syCCrCBdl07DTBzugQNzU6S9q fxqcJXTwwLqlbueBMRcEw== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvkedrvddtledgtdefucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepkfffgggfuffvvehfhfgjtgfgsehtkeertddtfeejnecuhfhrohhmpeffmhhi thhrhicuifhuthhovhcuoegumhhithhrhiesghhuthhovhdruggvvheqnecuggftrfgrth htvghrnhephfffheeljeffgeffueeghfekkedtfffgheejvdegjeettdduheeufffggfef jeehnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepug hmihhtrhihsehguhhtohhvrdguvghv X-ME-Proxy: Feedback-ID: i0e71465a:Fastmail Original-Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon, 18 Dec 2023 18:08:23 -0500 (EST) Content-Language: en-US In-Reply-To: X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.bugs:276486 Archived-At: On 18/12/2023 21:12, Stefan Monnier wrote: >>>> I found that if you don’t set text prop fontified to nil for the whole >>>> extended region, redisplay doesn’t seem to, well, redisplay the full >>>> region, even thought the new face has been correctly applied to them. >>> That sounds like a bug in font-lock? At the end of jit-lock-fontify-now, >>> there is a call creating a timer with jit-lock-force-redisplay. >>> And that function ends with this: >>>     ;; Don't cause refontification (it's already been done), but just do >>>     ;; some random buffer change, so as to force redisplay. >>>     (put-text-property start end 'fontified t))))) >>> If I just change t to nil there (or to some other value, like 42), either >>> of our patches starts behaving well. Perhaps Stefan could comment. > > This chunk of code is present so as to cause a re-render (i.e. recompute > the glyph matrices) of the affected region, but not a re-fontification. > So a nil value would be wrong. > I'm surprised that 42 behaves differently from t. This patch also changes the visible behavior, fixing the problem that I'm seeing: diff --git a/lisp/jit-lock.el b/lisp/jit-lock.el index 452cbd1ca51..43db2d31856 100644 --- a/lisp/jit-lock.el +++ b/lisp/jit-lock.el @@ -499,6 +499,7 @@ jit-lock-force-redisplay (setq start (point-min) end (max start end))) ;; Don't cause refontification (it's already been done), but just do ;; some random buffer change, so as to force redisplay. + (put-text-property start end 'fontified nil) (put-text-property start end 'fontified t))))) ;;; Stealth fontification. So it must be not about the eventual value of the property, but about triggering some counter, like (buffer-modified-tick). Which does get incremented after the above sequence (by 2). > What we really need here is to force the redisplay to consider that this > part of the buffer needs to be re-rendered. In practice changing any > text-property on that chunk of text should do the trick, in my > experience, but from what you describe it seems that some optimisation > is sufficiently clever to notice that the old value was t and the new > value is identical so the region is not marked as modified? > > Indeed, in `add_text_properties_1` I see we skip over intervals which > already have the right property values, so that might be what's happening. > I suggest we introduce a separate function with a name indicating what > we intend it to do (like `force-region-update`) so the code will > be clearer. > And its implementation could consist in adding and then removing some > dummy text property (tho a better implementation would go and modify > the underlying C-level variables in the buffer like BUF_*_UNCHANGED). Adding an extra 'put-text-property' call to jit-lock-force-redisplay seems cheap enough, but something even faster could be good too.