From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Stefan Monnier via "Bug reports for GNU Emacs, the Swiss army knife of text editors" Newsgroups: gmane.emacs.bugs Subject: bug#66732: tree-sitter fontification doesn't update multi-line syntax reliably Date: Sat, 16 Dec 2023 12:43:20 -0500 Message-ID: References: <878r7s5cdf.fsf@honnef.co> <83fs1tbou1.fsf@gnu.org> <835y1zo3rw.fsf@gnu.org> <2ce274aa-6d01-4d0a-b10c-07f821343fed@gmail.com> <50920549-006c-0153-2471-02e41a3dada7@gutov.dev> <8c7cd429-bdc3-4fac-ad1c-fbad793bf1a0@gmail.com> <231ebcd1-ec30-0432-82e7-d63e11cd65f7@gutov.dev> <765D713E-9923-4F66-9044-9D69C104C9B0@gmail.com> <33fe5d61-5022-67c5-6a65-babde4fb7f91@gutov.dev> <92CACD38-9534-4A07-8DE3-CE8408272FB6@gmail.com> <59CC46F7-867E-4C74-83EC-49B41DF0FAB8@gmail.com> Reply-To: Stefan Monnier Mime-Version: 1.0 Content-Type: text/plain Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="20116"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Cc: Yuan Fu , 66732@debbugs.gnu.org, Eli Zaretskii , dominik@honnef.co To: Dmitry Gutov Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Sat Dec 16 18:44:27 2023 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1rEYi9-00054S-Ow for geb-bug-gnu-emacs@m.gmane-mx.org; Sat, 16 Dec 2023 18:44:26 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rEYhn-0007Cm-Sg; Sat, 16 Dec 2023 12:44:03 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rEYhm-00077v-KY for bug-gnu-emacs@gnu.org; Sat, 16 Dec 2023 12:44:02 -0500 Original-Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rEYhm-0004mw-Bm for bug-gnu-emacs@gnu.org; Sat, 16 Dec 2023 12:44:02 -0500 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1rEYhm-0006Nd-FK for bug-gnu-emacs@gnu.org; Sat, 16 Dec 2023 12:44:02 -0500 X-Loop: help-debbugs@gnu.org Resent-From: Stefan Monnier Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sat, 16 Dec 2023 17:44:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 66732 X-GNU-PR-Package: emacs Original-Received: via spool by 66732-submit@debbugs.gnu.org id=B66732.170274861524440 (code B ref 66732); Sat, 16 Dec 2023 17:44:02 +0000 Original-Received: (at 66732) by debbugs.gnu.org; 16 Dec 2023 17:43:35 +0000 Original-Received: from localhost ([127.0.0.1]:55994 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rEYhK-0006M7-Oq for submit@debbugs.gnu.org; Sat, 16 Dec 2023 12:43:35 -0500 Original-Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:11442) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rEYhF-0006LZ-D3 for 66732@debbugs.gnu.org; Sat, 16 Dec 2023 12:43:32 -0500 Original-Received: from pmg3.iro.umontreal.ca (localhost [127.0.0.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id F3F44441D87; Sat, 16 Dec 2023 12:43:22 -0500 (EST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1702748601; bh=f6VJ4DWWFfX7uQDwtHvE86W9BFl6RJbkBQ+L+BkqCiI=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=EcrOc/NjqY3dUJQt9ta+ronx0HfUlWWGzuyW1dZpKV5cGzwiNKYaLPvnpi+ali4Q7 nwnEoo8MKIFdjKAn6S8BdTrsJEQCsYlwgPcCwNM7if+Pybj0XscAccHQj0fYxqnJWg /JcxpN6AJ885l5oy1ZXAG36BtKLzK6+j6a5dKLMTHxEFUMy460FYZ2nCQm198DFREs lFD5cVmonjlZtAh5+3+BEHGEPe2/cIATV3CcKZU4LfgW87K07JF8F2RW9VB+1q9VVw yTXVAtAqd4fEAJ/uhXAaR8Oy0AUFIvpez4aj7OZEwdJTzZuz9VwXtWt/iEiFpBJVPW FKqHhoPULPWdw== Original-Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg3.iro.umontreal.ca (Proxmox) with ESMTP id 6DC33440B05; Sat, 16 Dec 2023 12:43:21 -0500 (EST) Original-Received: from pastel (65-110-221-238.cpe.pppoe.ca [65.110.221.238]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 35760120E7F; Sat, 16 Dec 2023 12:43:21 -0500 (EST) In-Reply-To: (Dmitry Gutov's message of "Sat, 16 Dec 2023 19:23:49 +0200") X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.bugs:276369 Archived-At: > I think most of the time the parser notifier will send ranges of limited > size (as one or other piece of text gets recognized as e.g. a comment, or > a raw string -- the bounds of the respective node that changed). Yes, of course, in practice it will usually work OK. I'm talking about what should happen ideally so that we can make an informed decision about the code we decide to use in the end, knowing its downsides. >> IOW, it'd be better to mark the changed chunks with this >> `put-text-property`, but to do it before jit&font-lock get triggered. >> Basically, we'd like to call it from `after-change-functions`, but this >> can be called *many* times within a single command, so we want to delay >> it. So we can probably just set a flag that says "there are unprocessed >> changes" and then sprinkle calls to a "lazy update" function which >> checks this flag before calling `treesit--font-lock-notifier` (or >> something similar). > > I thought doing this lazily (inside font-lock-default-fontify-region), only > when the region is set to be redisplayed, would be more economical than > doing more work inside after-change-functions. Indeed we don't want to do it directly in `after-change-functions`, but doing it in `font-lock-default-fontify-region` is too late because the redisplay and jit-lock have already (to some extent) decided what should be refontified at that point. The infrastructure does offer ways to make it work (to fix previous incorrect assumptions about what needed to be redisplayed/refontified), but it's best if we can avoid it. >> One of the places where we'd need the sprinkle would be >> `pre-redisplay-functions`. Another one might be `syntax-ppss`? > syntax-ppss has syntax-propertize-extend-region-functions. I haven't found > any better suitable hooks than that. Indeed, I don't think there's a good hook for that yet, tho maybe `syntax-propertize-extend-region-functions` is good enough (basically make it call `treesit--font-lock-notifier` but return nil), assuming we have set a `syntax-propertize-function`. Stefan