From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Yuan Fu Newsgroups: gmane.emacs.bugs Subject: bug#66732: tree-sitter fontification doesn't update multi-line syntax reliably Date: Thu, 14 Dec 2023 23:12:33 -0800 Message-ID: <92CACD38-9534-4A07-8DE3-CE8408272FB6@gmail.com> References: <878r7s5cdf.fsf@honnef.co> <83fs1tbou1.fsf@gnu.org> <835y1zo3rw.fsf@gnu.org> <2ce274aa-6d01-4d0a-b10c-07f821343fed@gmail.com> <50920549-006c-0153-2471-02e41a3dada7@gutov.dev> <8c7cd429-bdc3-4fac-ad1c-fbad793bf1a0@gmail.com> <231ebcd1-ec30-0432-82e7-d63e11cd65f7@gutov.dev> <765D713E-9923-4F66-9044-9D69C104C9B0@gmail.com> <33fe5d61-5022-67c5-6a65-babde4fb7f91@gutov.dev> Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.700.6\)) Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="15881"; mail-complaints-to="usenet@ciao.gmane.io" Cc: Eli Zaretskii , 66732@debbugs.gnu.org, Stefan Monnier , dominik@honnef.co To: Dmitry Gutov Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Fri Dec 15 08:13:27 2023 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1rE2Nz-0003xe-Mh for geb-bug-gnu-emacs@m.gmane-mx.org; Fri, 15 Dec 2023 08:13:27 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rE2Ne-0007LM-7J; Fri, 15 Dec 2023 02:13:06 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rE2Na-0007Jd-UO for bug-gnu-emacs@gnu.org; Fri, 15 Dec 2023 02:13:04 -0500 Original-Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rE2Na-0005AX-AA for bug-gnu-emacs@gnu.org; Fri, 15 Dec 2023 02:13:02 -0500 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1rE2NZ-0005by-Qg for bug-gnu-emacs@gnu.org; Fri, 15 Dec 2023 02:13:01 -0500 X-Loop: help-debbugs@gnu.org Resent-From: Yuan Fu Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Fri, 15 Dec 2023 07:13:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 66732 X-GNU-PR-Package: emacs Original-Received: via spool by 66732-submit@debbugs.gnu.org id=B66732.170262437421556 (code B ref 66732); Fri, 15 Dec 2023 07:13:01 +0000 Original-Received: (at 66732) by debbugs.gnu.org; 15 Dec 2023 07:12:54 +0000 Original-Received: from localhost ([127.0.0.1]:51558 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rE2NR-0005bc-Cf for submit@debbugs.gnu.org; Fri, 15 Dec 2023 02:12:53 -0500 Original-Received: from mail-pl1-x62d.google.com ([2607:f8b0:4864:20::62d]:55523) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rE2NP-0005bO-CE for 66732@debbugs.gnu.org; Fri, 15 Dec 2023 02:12:52 -0500 Original-Received: by mail-pl1-x62d.google.com with SMTP id d9443c01a7336-1d08a924fcfso2539315ad.2 for <66732@debbugs.gnu.org>; Thu, 14 Dec 2023 23:12:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1702624365; x=1703229165; darn=debbugs.gnu.org; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=DBxz1Z7mX9b3W/dEk86scIGWTfaojGmUTwSq72AHuqs=; b=DJeppjAU2sYrvHMRvf6KJo1LfE3k240+qk9fQbILV2D4ZBsyAiMhBWLU6vAA1cpi4w AoW9VSZ4T2pn3FjSZWITGv5XYoKTXUVGws+lY9ZkWA1v1rhXTjetqCr9mn77iKj11/K9 5D3+doeJLZKDYXNCUgdnHfEKU+WS8ZnUDTkHslwd9WeLPGR3bd8PRksYZ9K+oQPtSp7z gSH7qYpgDpdB8g6b2auydiyzJHbR7z0i82P73PLyYrjKZG7wXW/veTYJ7zfyirWJvJxM fBwTrfit0Dn86PCjcAnDXorLYAE+UmikWwY4/lRXxCB4PzdlitbCXBdMEFy1U8r6+x7K XaZw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1702624365; x=1703229165; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=DBxz1Z7mX9b3W/dEk86scIGWTfaojGmUTwSq72AHuqs=; b=wblwTuTA4W6y/NYz40TT3N1Vh7h3dffR/zYXTkqMcKV3+tnqxijQcr9TTZ1f5X7r5E Vt/3z6O8lxdUNvWzzy7DotRyzFq2bcTx72emn+pK4NMdB79wMg1uvVc9dHAQLEzRVZVg tuEECZW9KmhfKUJqS8bAcsOTqqtQA1dko7kKPewrFIgTgUOXBsoNXC0TY7U+pQhSY2vN zLrzAMSGeflDhwKtEovrgA+chQ/mkYMuRYcEEMDhBNUbikwJSd2AimEijwX75lFPG6ug K4oR/MpL8tE+LDsyMJF994ehezT+mP1tLdoMX0p4Flnehf/wWfL4i92B7fQxm3oWnfPU w/tA== X-Gm-Message-State: AOJu0YxypKla9flZNZf05vdyfCj6V4VDTY9tAAbz2zjVs3DxgEXBszvn 7ALrY/H2aWTeE/pIJwyrWj4= X-Google-Smtp-Source: AGHT+IEdl8ZmpaObTf1XwXobf/LsIBRErknxuxFrIh0i9EF75EG9lv5yKoIV/yMbd+vPcTT7iexPiw== X-Received: by 2002:a17:902:6b0c:b0:1d0:8c63:cfe4 with SMTP id o12-20020a1709026b0c00b001d08c63cfe4mr9173541plk.77.1702624365283; Thu, 14 Dec 2023 23:12:45 -0800 (PST) Original-Received: from smtpclient.apple (172-117-161-177.res.spectrum.com. [172.117.161.177]) by smtp.gmail.com with ESMTPSA id 13-20020a170902c24d00b001d35d62b066sm3366414plg.283.2023.12.14.23.12.44 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 14 Dec 2023 23:12:44 -0800 (PST) In-Reply-To: <33fe5d61-5022-67c5-6a65-babde4fb7f91@gutov.dev> X-Mailer: Apple Mail (2.3731.700.6) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.bugs:276252 Archived-At: > On Dec 14, 2023, at 5:01 PM, Dmitry Gutov wrote: >=20 > On 14/12/2023 10:29, Yuan Fu wrote: >>> On Dec 13, 2023, at 5:43 PM, Dmitry Gutov wrote: >>>=20 >>> On 13/12/2023 05:28, Yuan Fu wrote: >>>>>> c-ts-mode--emacs-set-ranges is registered as a range rule, so = many tree-sitter function calls it before doing anything to make sure = range is up-to-date. treesit-font-lock-fontify-region calls = treesit-update-ranges at the beginning of its body, and = treesit-update-ranges calls c-ts-mode--emacs-set-ranges. >>>>>=20 >>>>> That seems to mean that any feature accessing the parse tree = should call treesit-update-ranges first. Including = syntax-propertize-functions and *-extend-region-functions, which we = currently don't do. >>>>>=20 >>>>> But which boundaries is it supposed to use? Should = treesit--syntax-extend-region call treesit-update-ranges, when wait for = the parser updates, then possibly call treesit-update-ranges again on = the extended boundaries, and so on, until the parser stops sending = notifications? Will it stop? >>>> It'll stop. If the ranges don't change, no reparse will happen. And = only buffer content change can cause range change. Reparse itself can't. = So at most you'll see reparse -> update ranges -> reparse. >>>>> Anyway, could you try my patch? Like I said, I'm not sure if the = insufficient fontification I'm observing in c-ts-mode is due to the = problem with the solution, or due to the other redisplay-related = problems on my system. >>>> Yeah, it doesn't solve the problem in c-ts-mode regarding block = comments. >>>> We might need to run (progn (force-parse) (update-ranges) = (force-parse)) before jit-lock-fontify-now and sytax-ppss. >>>=20 >>> Well... I've replaced >>>=20 >>> (treesit-buffer-root-node (treesit-language-at (point))) >>>=20 >>> in treesit--syntax-extend-region with >>>=20 >>> (treesit-buffer-root-node (treesit-language-at (point))) >>> (treesit-update-ranges beg end) >>> (treesit-buffer-root-node (treesit-language-at (point))) >>>=20 >>> and even tried commenting out the call to 'treesit-update-ranges' = inside treesit-font-lock-fontify-region, but the result looks unchanged. = And the region does get extended, according to my print-debugging inside = font-lock-default-fontify-region. >>>=20 >>> Could you check if you're seeing the same? >> It should be >> (treesit-buffer-root-node 'emacs-c) >> (treesit-update-ranges) >> (treesit-buffer-root-node =E2=80=98c) >=20 > This might be difficult for treesit--syntax-extend-region to do, since = it's supposed to work across modes. But I suppose it could iterate = through (mapcar #'treesit-parser-language (treesit-parser-list)). Actually, only calling (treesit-update-ranges) should be enough (and is = generic), the range update function surely would access the parser, = which would force the reparse, and the update function will access the = parser in the correct order: host language first, then embedded = languages. >=20 >> I tried, and it doesn=E2=80=99t make a difference. But I must admit = I=E2=80=99m not very clear on what your patch tries to do. Does it try = to extend the to-be-fontified region before jit-lock runs? >=20 > Yes, it extends the region to-be-fontified. If you set = treesit--font-lock-verbose to t, you should see the appropriate messages = "Fontifying region: x-y" in the message log where x is the position = before the start of the comment (printed by = treesit-font-lock-fontify-region). Hmm, let me have a closer look tomorrow. It can=E2=80=99t be that = difficult, we must have missed something. Yuan