From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Yuan Fu Newsgroups: gmane.emacs.bugs Subject: bug#60691: 29.0.60; Slow tree-sitter font-lock in ruby-ts-mode Date: Sun, 29 Jan 2023 15:23:34 -0800 Message-ID: References: <867cxv3dnn.fsf@mail.linkov.net> <90D8581D-C543-43EE-8BEB-51B5AFBCBAEE@gmail.com> <0e552ada-081f-ad90-19c2-645a64ef50ac@yandex.ru> Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.300.101.1.3\)) Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="10084"; mail-complaints-to="usenet@ciao.gmane.io" Cc: 60691@debbugs.gnu.org, juri@linkov.net To: Dmitry Gutov Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Mon Jan 30 00:25:23 2023 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1pMH34-0002Q8-LO for geb-bug-gnu-emacs@m.gmane-mx.org; Mon, 30 Jan 2023 00:25:22 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pMH2m-00025E-Su; Sun, 29 Jan 2023 18:25:04 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pMH2l-00024r-F5 for bug-gnu-emacs@gnu.org; Sun, 29 Jan 2023 18:25:03 -0500 Original-Received: from debbugs.gnu.org ([209.51.188.43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pMH2k-0006z4-P1 for bug-gnu-emacs@gnu.org; Sun, 29 Jan 2023 18:25:02 -0500 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1pMH2k-0005mA-LJ for bug-gnu-emacs@gnu.org; Sun, 29 Jan 2023 18:25:02 -0500 X-Loop: help-debbugs@gnu.org Resent-From: Yuan Fu Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sun, 29 Jan 2023 23:25:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 60691 X-GNU-PR-Package: emacs Original-Received: via spool by 60691-submit@debbugs.gnu.org id=B60691.167503467922170 (code B ref 60691); Sun, 29 Jan 2023 23:25:02 +0000 Original-Received: (at 60691) by debbugs.gnu.org; 29 Jan 2023 23:24:39 +0000 Original-Received: from localhost ([127.0.0.1]:45572 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pMH2M-0005lV-W9 for submit@debbugs.gnu.org; Sun, 29 Jan 2023 18:24:39 -0500 Original-Received: from mail-pj1-f41.google.com ([209.85.216.41]:41529) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pMH2L-0005lI-LN for 60691@debbugs.gnu.org; Sun, 29 Jan 2023 18:24:38 -0500 Original-Received: by mail-pj1-f41.google.com with SMTP id z1-20020a17090a66c100b00226f05b9595so9531317pjl.0 for <60691@debbugs.gnu.org>; Sun, 29 Jan 2023 15:24:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=S+O5xk0WxjqbnURzVnqCS0FuG59/bZrjU85yOrksg7U=; b=XwOam6OkjOYpn9ErqoVS1Z9gBHzFUSmaSHfsDlDzj/kqVdp+PRTg5YvkT3xnt7N2ui 8+aguUQL97LthjfMNC2R1Ms+oE4mVk1QDQq2TgJmMvaYbXq3vKWRrS3pwlAHDjisisZb C4aMQnbXNGPCwxrbYZrUdJ+tuoYLbQTmPT4NzkBAIqBVSQJP9wOZIydywdcO0AkLedDd 5Igs+DDbk5OvMutJxdQmjzo+eodsFcrfaYHbN49gF+yle3N0HLu9mEd3A72IlgSERpjP lWoNJHRSLK3QXdz8/IikHS25STGCX0DygDV7duoG4y6MqZ/gDOtV2IJBc/zZf6DXHbdI cbxg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=S+O5xk0WxjqbnURzVnqCS0FuG59/bZrjU85yOrksg7U=; b=y34SjkNluVwAAC0FmOkxKcTi1VRSENEgwQpqB3R0Bz+SUpS8pFNSzqLIjls60BLars jenR3Hz9+0b4yx+0890gRu46Eo5wNJpu6tZ1QAsyTe3ztBArYFKsDgikrQM1RW6BxWD/ dL+97rAhkRCmwX2vFiRDADBljwWyVlXNlqtKm9/NuqaOtPLSV/q9KRcVatwlXBobU16V SLvpyl1f4Q/4AVlGqpm7QRNyv0QMYax7LgF+qxjl5ATLjjNb9digMAOnvjfL7OTggbQi IDTs+b2P7BNoW1+Rb8zvyPtyq9KqLkPqzhDlWUnn3BBlRDor0LpFSR1ph7JbB5JU1AN+ eIRA== X-Gm-Message-State: AO0yUKVL4KFLBEZi8/fj4mxAsPbGZzH6p/JPecICnXgGWpNlOpXzZY27 ZVV4ntNc+Ld2nex/lEHyYGI= X-Google-Smtp-Source: AK7set899EM4dbxz6E86gsdg392jUvvGnccGM1P1hroVYKyzmJqIa5OaKda1Aq26H7vAWaCOsnui3w== X-Received: by 2002:a05:6a20:8f06:b0:bc:d738:592e with SMTP id b6-20020a056a208f0600b000bcd738592emr4860024pzk.14.1675034671668; Sun, 29 Jan 2023 15:24:31 -0800 (PST) Original-Received: from smtpclient.apple (cpe-172-117-161-177.socal.res.rr.com. [172.117.161.177]) by smtp.gmail.com with ESMTPSA id v25-20020a63ac19000000b0047685ed724dsm5510278pge.40.2023.01.29.15.23.46 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 29 Jan 2023 15:23:59 -0800 (PST) In-Reply-To: <0e552ada-081f-ad90-19c2-645a64ef50ac@yandex.ru> X-Mailer: Apple Mail (2.3731.300.101.1.3) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.bugs:254385 Archived-At: > On Jan 29, 2023, at 3:07 PM, Dmitry Gutov wrote: >=20 > Hi Yuan, >=20 > On 29/01/2023 10:25, Yuan Fu wrote: >=20 >>>>> So if previously it happened once somehow during a certain = scenario, now I have to repeat the same scenario 4 times, and the = condition is met. >>>> I was hoping that the scenario only happen once, oh well :-) I=E2=80=99= ll >>>> change the decision based on analyzing the tree=E2=80=99s = dimension: too >>>> deep or too wide activates the fast mode. Let=E2=80=99s see how it = works. >>>=20 >>> Thank you, let me know when it's time to test again. >> Sorry for the delay. Now treesit-font-lock-fontify-region uses >> treesit-subtree-stat to determine whether to enable the "fast mode". = Now >> it should be impossible to activate the fast mode on moderately sized >> buffers. >=20 > Thank you, it seems to work just fine in my scenario. And = treesit-subtree-stat makes sense. >=20 > I have a few more questions about the current strategy, though. >=20 > IIUC, we only do the treesit--font-lock-fast-mode test once in = treesit-font-lock-fontify-region, and then use the detected value for = the whole later life of the buffer. Is that right? >=20 > What if the buffer didn't originally have the problematic error nodes = we are guarding from, and then later the user wrote enough code to have = at least one of them? If they didn't close Emacs, or revert the buffer, = our logic still wouldn't use the "fast node", would it? >=20 > Or vice versa: if the buffer started out with error nodes, and = consequently, "fast mode", but then the user has edited it so that those = error nodes disappeared, shouldn't the buffer stop using the "fast = mode"? >=20 > =46rom my measurements, in ruby-mode, at least treesit-subtree-stat is = 20-40x faster than refontifying the whole buffer. So one possible = strategy would be to repeat the test every time. I'm not sure it's fast = enough in the "problem" buffers, though, and I don't have any to test. >=20 > In those I did test, though, it takes ~1 ms. >=20 > But we could repeat the test only once every couple of seconds and/or = after the buffer has changed again. That would hopefully make it a = non-bottleneck in all cases. I should mention this in the comments, but the fast mode is only for = very rare cases, where the file is mechanically generated and has some = peculiarities that causes tree-sitter to work poorly. If the file is = hand-written and =E2=80=9Cnormal=E2=80=9D, even huge files like xdisp.c = is well below the bar. Therefore I don=E2=80=99t think =E2=80=9Ccrossing = the line=E2=80=9D will realistically happen when editing source files. Here is the stats of two =E2=80=9Cproblematic files=E2=80=9D, named = packet and dec_mask, comparing to xdisp.c: ;; max-depth max-width count ;; cut-off 100 4000 ;; packet (98159 46581 1895137) ;; dec mask (3 64301 283995) ;; xdisp.c (29 985 218971) I=E2=80=99d say that any regular source file, even mechanically = generated, wouldn=E2=80=99t go beyond ~50 levels in depth, and = hand-written files should never has a node that has 4000+ direct = children in the parse tree. Yuan=