From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Dmitry Gutov Newsgroups: gmane.emacs.bugs Subject: bug#66732: tree-sitter fontification doesn't update multi-line syntax reliably Date: Wed, 20 Dec 2023 03:52:11 +0200 Message-ID: References: <878r7s5cdf.fsf@honnef.co> <83fs1tbou1.fsf@gnu.org> <835y1zo3rw.fsf@gnu.org> <2ce274aa-6d01-4d0a-b10c-07f821343fed@gmail.com> <50920549-006c-0153-2471-02e41a3dada7@gutov.dev> <8c7cd429-bdc3-4fac-ad1c-fbad793bf1a0@gmail.com> <231ebcd1-ec30-0432-82e7-d63e11cd65f7@gutov.dev> <765D713E-9923-4F66-9044-9D69C104C9B0@gmail.com> <33fe5d61-5022-67c5-6a65-babde4fb7f91@gutov.dev> <92CACD38-9534-4A07-8DE3-CE8408272FB6@gmail.com> <59CC46F7-867E-4C74-83EC-49B41DF0FAB8@gmail.com> <2C418114-3773-4D62-90CB-E4C7F1B7D467@gmail.com> <029f6911-c4f5-1217-73bf-f57b7f941ae2@gutov.dev> <3beef57a-12c5-0ea0-d6d7-752d0aa4fb8d@gutov.dev> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="1744"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.13.0 Cc: Eli Zaretskii , 66732@debbugs.gnu.org, Stefan Monnier , dominik@honnef.co To: Yuan Fu Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Wed Dec 20 02:53:24 2023 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1rFllz-0000Eq-7u for geb-bug-gnu-emacs@m.gmane-mx.org; Wed, 20 Dec 2023 02:53:24 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rFllg-0003VG-Fy; Tue, 19 Dec 2023 20:53:05 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rFllb-0003V2-M5 for bug-gnu-emacs@gnu.org; Tue, 19 Dec 2023 20:52:59 -0500 Original-Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rFllb-0007sz-AA for bug-gnu-emacs@gnu.org; Tue, 19 Dec 2023 20:52:59 -0500 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1rFlle-00036V-6F for bug-gnu-emacs@gnu.org; Tue, 19 Dec 2023 20:53:02 -0500 X-Loop: help-debbugs@gnu.org Resent-From: Dmitry Gutov Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Wed, 20 Dec 2023 01:53:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 66732 X-GNU-PR-Package: emacs Original-Received: via spool by 66732-submit@debbugs.gnu.org id=B66732.170303714511886 (code B ref 66732); Wed, 20 Dec 2023 01:53:02 +0000 Original-Received: (at 66732) by debbugs.gnu.org; 20 Dec 2023 01:52:25 +0000 Original-Received: from localhost ([127.0.0.1]:38012 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rFll3-00035d-1E for submit@debbugs.gnu.org; Tue, 19 Dec 2023 20:52:25 -0500 Original-Received: from out1-smtp.messagingengine.com ([66.111.4.25]:53601) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rFll2-00035R-4a for 66732@debbugs.gnu.org; Tue, 19 Dec 2023 20:52:24 -0500 Original-Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailout.nyi.internal (Postfix) with ESMTP id AAE565C02BA; Tue, 19 Dec 2023 20:52:15 -0500 (EST) Original-Received: from mailfrontend1 ([10.202.2.162]) by compute3.internal (MEProxy); Tue, 19 Dec 2023 20:52:15 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gutov.dev; h=cc :cc:content-transfer-encoding:content-type:content-type:date :date:from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to; s=fm3; t=1703037135; x=1703123535; bh=JKVnUre9U1+xnRn2uWAeBklOdqaHHuOmpJFbIVPp0cs=; b= 00BIW4gQ/TGN9DTdsMMJTTxorbgnD9hPgidyvZs+pBXREoPueYOT8/5L97GDXXk8 nBuu1NaTBPHo9NLqKdH+kaUekISZxMxtc+RpwIhVsJ6a8oMxqrR4S6mRFzDypRbF jbDwdmJfgs96WgSoj50EvNMGUplmxmnpbTdWMLFYy/M+Re/YGrC+c+j14CM1C0ou x9Kv53rdtuXeda5kIJw6sEO6U4dEnEmY4424EhZpJNxVC104FWbotucAtpw+zl9e Jk5a1IEGG0bdB0zKo8UJmqhK8Oya0SmsYeP2rKkvxIa4mM1M9cWwY8zhqLbZ1F/u k0KwpqdzIOd5qaSohShB+g== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm2; t=1703037135; x= 1703123535; bh=JKVnUre9U1+xnRn2uWAeBklOdqaHHuOmpJFbIVPp0cs=; b=w otYdjsVul3LCCK7X93bWK76WtU5sxSYO4IH0Jjsg5v1T/dcdIJeuFID6GlQ5j8d7 KuiS87dvEO0VIpv5NQeC8wZdb6mSW04OrKZv8LIQYDyzAk7u8+05cYwqP4BZz1Gs OeziauKy3K2f3aIIPDxO0il4DfRFMpPb3BbBkkFr+/+7Nyo2i3rniTW+hW6ArGvT IRjM4AXEcCzaRna6ZIM2ihuvEz9LOTC4F8NUNzk4ewNAzeZqNxZdU1HL46IUZ18w jh57up0gf30sRUIq9B/q8v7Rn2EyzomskvyZ+pU8+d9jkLBGokZKCUHIlt7XIfiV /SYYnknyJVSGjK+vVFu+Q== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvkedrvdduuddgfeelucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepkfffgggfuffvvehfhfgjtgfgsehtjeertddtfeejnecuhfhrohhmpeffmhhi thhrhicuifhuthhovhcuoegumhhithhrhiesghhuthhovhdruggvvheqnecuggftrfgrth htvghrnhepiefgteevheevveffheeltdeukeeiieekueefgedugfefgefhudelgfefveel vdevnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepug hmihhtrhihsehguhhtohhvrdguvghv X-ME-Proxy: Feedback-ID: i0e71465a:Fastmail Original-Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue, 19 Dec 2023 20:52:13 -0500 (EST) Content-Language: en-US In-Reply-To: X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.bugs:276546 Archived-At: On 19/12/2023 05:12, Yuan Fu wrote: > >> On Dec 17, 2023, at 10:32 AM, Dmitry Gutov wrote: >> >> On 17/12/2023 03:16, Yuan Fu wrote: >>> +(defun treesit--pre-syntax-ppss (&rest _) >>> + "Force reparse and consequently run all notifiers. >>> + >>> +Similar to font-lock, we want to update the `syntax' text >>> +property before `syntax-ppss' starts working on the text." >>> + (treesit--pre-redisplay) >>> + nil) >> This callback might be the best place to inform syntax-propertize about the extended bounds which need to be re-parsed, so I'm not sure about Stefan's suggestion to return nil here. >> >> Even calling syntax-ppss-flush-cache wouldn't be a full solution, because that wouldn't update the value of START in syntax-propertize. > Makes sense. Then I think a treeist function that returns the latest affected regions is in order? But latest compared to what? To the previous invocation of the parser? What if the current caller uses this function and expects to see changes since time X, but some other feature already instantiated the parser sometime between X and NOW? The callbacks which would run now would send notifications compared to that other previous (unknown) invocation. There could be some system which keeps checkpoints that the callers could reference, but I'm not sure what's the best shape. And how the older history, not needed by any callers anymore, would be evicted (some weak hash map could work, but any caller that's not careful could create leaks...). And without the above, we'll need a separate callback which would call syntax-ppss-flush-cache when the parser sends notifcations while invoked outside of syntax-propertize-extend-region-functions. And then its code will have to care about context anyway (inside s-p-e-r-f or not).